The AI Runtime

If your inference bill is climbing or you are running out of GPU memory, you have two ways to make a model smaller. Quantization cuts the most bytes but changes the model’s outputs, which is a problem for anything regulated or already validated. Lossless compression cuts about 30% of the bytes by re-packing the wasted space in BF16 weights, and the outputs come back bit-for-bit identical. The DFloat11 research confirms the 30% with zero accuracy change, and ZipNN reports similar. The 30% is a fixed ceiling, not a knob, so treat it as a free one-time discount for BF16 workloads that are memory-bound and cannot tolerate changed output. ISIRO Runtime is one commercial product built on this technique, with vendor-reported numbers worth testing rather than trusting. Before you quantize anything, run a bit-exact diff on a compiled model and measure whether your decode path is actually memory-bound. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit theairuntime.com