Intel packs ternary LLM weights into 1.485 bits per weight, and decoding runs up to 27 percent faster on GPUs

Intel researchers have compressed a ternary language model below the 1.58-bit theoretical floor by rethinking how the weights are stored, while leaving the model itself untouched. Their new format, called BITCOS, brought one checkpoint down to 1.485 bits per weight and lifted decoding throughput by as much as 18 percent on CPUs and 27 percent on GPUs. The paper went up on arXiv on 14 September 2026, and The New Stack reported the numbers on 17 September.

Where the 1.58-bit figure comes from

A ternary model uses three weight values: minus one, zero, and plus one. If those values were equally likely, 1.58 bits would be the smallest number needed to code each one. In practice, storage sits higher than that. The common approach fits five ternary values into a byte, which is 1.6 bits per weight, and once you store weights in blocks of 128 the actual rate climbs to 1.625 bits per weight because the last byte in a block is only part used.

Intel measured 29 checkpoints across seven ternary model families and found that between 29.7 percent and 51.5 percent of the weights were zero. An equal split would give one third. Once more than 37.5 percent of a model's weights are zero, storing them differently starts to save space, and 26 of the 29 checkpoints crossed that line.

BITCOS stores zeros almost for free

BITCOS stands for "BITmap and COmpacted Signs". The format splits weights into two streams. The first assigns one bit to every weight to record whether it is zero or not. The second records a sign for the nonzero weights only. A positive or negative weight takes two bits between the two streams, and a zero takes only the presence bit, because there is no sign to record. If z is the fraction of zeros, the cost is two minus z bits per weight. At 40 percent zeros that is 1.6 bits, at 51.5 percent it is 1.485 bits.

The sparsest checkpoint in Intel's set was a ternary Qwen3-1.7B produced with CAT-Q post-training quantization, at 51.48 percent zero. Because BITCOS changes only the storage format, unpacking gives back the same minus one, zero and plus one values. Accuracy is unchanged.

The speed numbers, and where they do not hold

Intel wrote unpacking kernels for AVX-512 and AVX2 CPUs and for Xe2 GPUs. Compared against the two-bit kernels used as a baseline, BITCOS ran 10 to 18 percent faster on a 64-core Xeon server and 2 to 15 percent faster on a 24-core Core Ultra 9. On graphics, it was 9 to 22 percent faster on the integrated Arc 140V and 2 to 27 percent faster on the discrete Arc Pro B70. All numbers are decoding throughput at batch size one, measured after the model has loaded.

One system beat BITCOS: on an eight-core Lunar Lake CPU, the fixed two-bit kernel won on every model because memory bandwidth was not the bottleneck there. The paper has not been peer-reviewed, all five test systems used Intel silicon, and the end-to-end benchmarks covered seven models at batch size one. Nothing has been measured on Nvidia, AMD or Arm.

Ternary formats are chosen because token generation is bandwidth-bound. The arithmetic is cheap and the weights have to be fetched anyway, so trimming another sixth of a bit off each weight moves the bottleneck on the hardware where fetching them was still the slow part.

Source

Evangelos Georganas, Alexander Heinecke and Pradeep Dubey, "Breaking the 1.58-bit Barrier for Ternary LLMs", arXiv, 14 September 2026. Reported by The New Stack.