Built LazyMoE — run 120B LLMs on 8GB RAM with no GPU using lazy expert loading + TurboQuant

Totally valid concern and it's a common misconception worth clarifying. 1-bit here refers to BitNet b1.58 specifically — weights are ternary {-1, 0, +1} rather than just binary {0, 1}. The key difference is that BitNet models are TRAINED from scratch at 1-bit, not post-quantized from fp16. That's what makes the quality surprisingly competitive. You're absolutely right that aggressively quantizing a normal fp16 model down to 2-3 bit is lobotomy territory. GPTQ/GGUF quantization below Q4 degrades fast. But BitNet trained natively at 1-bit is a different story — Microsoft's research shows near parity with fp16 at 70B+ scale because the model learns to work within the constraint from day one rather than having precision crushed out of it afterward. The practical catch is that true BitNet 70B+ weights don't publicly exist yet. So right now the project uses standard Q4\_K\_M for actual inference and BitNet is on the roadmap for when those weights drop. Your instinct on Q4 being the sweet spot for standard models is spot on though — that's exactly what I'm running locally.