Quantization, Grafting, and Self-Optimizing AI

We explore how aggressive quantization can keep model quality flat while boosting throughput, thanks to error cancellation across transformer layers. The episode also dives into modular model grafting, disaggregated prefill and decode systems, and LLMs that can profile and optimize their own serving kernels.