The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits

BitNet trained Transformers with weights of +1 or −1 from scratch and got close to FP16 quality. BitNet b1.58 adds one more value, zero, so each weight is −1, 0 or +1. Three values need log₂3 ≈ 1.58 bits, hence the name. The zero lets the model switch individual inputs off, which the authors call feature filtering, and it makes the method much stronger.

The quantizer is simple. Divide each weight matrix by its mean absolute value, round, and clip to the range −1 to 1; about a third of the weights land on zero. Activations are quantized to 8 bits per token by their absolute maximum. The model is otherwise a LLaMA-style Transformer: RMSNorm, SwiGLU, rotary embeddings, no biases. As in BitNet, training keeps full-precision latent weights and quantizes on the fly.

The authors trained their own FP16 LLaMA baselines and BitNet b1.58 models at 700M, 1.3B and 3B parameters on 100 billion tokens of RedPajama. The ternary model trails at 700M and 1.3B and draws level at 3B: perplexity 9.91 against 10.04, and a seven-task zero-shot average of 50.2 against 49.7. Measured with a 2-bit kernel in FasterTransformer, the 3B model used 3.55 times less GPU memory and decoded 2.71 times faster. At 70B (costs only, no quality reported), it decoded 4.1 times faster, and two A100s served 11 times the batch size and 8.9 times the throughput. An energy model puts the matrix-multiply arithmetic at 71.4 times less energy, because ternary weights turn multiplications into 8-bit additions.

The paper argues this defines a new scaling law and calls for hardware built for 1-bit models. It is a short report, labeled work in progress, and every cost comparison is against FP16, not against the 4-bit quantized models people actually deploy.

BitNet b1.58 is the most cited argument that future models could be trained low-bit from the start. Knowing exactly what it shows keeps you from overselling or dismissing it.

It only helps models trained this way. Like BitNet, the gains require pretraining with ternary weights. A customer's existing model cannot be converted. The value depends on someone releasing a strong ternary model in the size they need.

The baseline is FP16, not 4-bit. Memory and speed ratios are against 16-bit models. The realistic alternative is a 4-bit post-training quantized model, which already cuts weight memory about fourfold. The paper does not measure that comparison, so do not quote "3.55x less memory" as the gain over what a customer would otherwise deploy.

Batch size and throughput are where it shines. At 70B on two A100s, the small weights left room for 11 times the batch. For high-volume serving, freed memory turns into throughput, the same mechanism that makes 4-bit attractive, pushed further.