EP061: Fine-Tuning LLaMA 65B on One GPU

QLoRA: Efficient Finetuning of Quantized LLMs presents a highly efficient approach that drastically reduces the memory required to finetune large language models. By backpropagating gradients through a frozen, 4-bit quantized pretrained model into Low Rank Adapters (LoRA), QLoRA makes it possible to finetune a 65-billion parameter model on a single 48GB GPU while preserving full 16-bit task performance.The paper introduces three key innovations to achieve this memory efficiency without sacrificing performance:4-bit NormalFloat (NF4): A new data type that is theoretically optimal for normally distributed weights.Double Quantization: A technique that quantizes the quantization constants to further reduce the average memory footprint.Paged Optimizers: A memory management strategy used to handle memory spikes during processing.Using this method, the researchers developed a model family called Guanaco, which achieved 99.3% of ChatGPT's performance on the Vicuna benchmark after just 24 hours of finetuning on a single GPU. Ultimately, the authors' analysis of over 1,000 finetuned models demonstrates that using QLoRA on small, high-quality datasets can yield state-of-the-art results, even when utilizing smaller base models.