QLoRA: Efficient Finetuning of Quantized LLMs

The paper that put large-model fine-tuning on a single GPU. Maya and Leo open QLoRA's central rule — read the compressed thing, write somewhere else — and follow gradients through a frozen four-bit base into full-precision LoRA adapters. Along the way: NF4's bell-curve-shaped buckets, double quantization's compress-the-labels trick, paged optimizers as the relief valve that saves hour-nine runs, and the thousand-model study behind Guanaco that both topped the Vicuna benchmark and warned against trusting chatbot benchmarks. The staged debate takes on the choice QLoRA created: a bigger model at four bits or a smaller one at full precision — settled, as ever, by testing the exact configuration you ship on the tails you fear. The hospital discharge-note summarizer returns to make it concrete. Sources: • QLoRA: Efficient Finetuning of Quantized LLMs: https://arxiv.org/pdf/2305.14314 • LoRA: Low-Rank Adaptation of Large Language Models: https://arxiv.org/pdf/2106.09685 • LowRA: Accurate and Efficient LoRA Fine-Tuning of LLMs under 2 Bits: https://arxiv.org/abs/2502.08141