LoRA: Low-Rank Adaptation of Large Language Models

The paper that made fine-tuning feel modular. Maya and Leo open up LoRA's central trick — freeze the pretrained weights and learn the update as the product of two thin matrices, around sixty-five thousand trainable numbers standing in for sixteen million — then follow it through the merge fork (flatten for zero-overhead serving, or keep adapters swappable on one frozen base), the rank and alpha dials, and why low intrinsic dimension makes the whole bet plausible. Two real practitioner arguments get staged on air: attention-only versus broad target modules, and whether LoRA's cheapness has made fine-tuning the default move when it shouldn't be. The hospital discharge-note summarizer returns to show why frozen storage is not frozen behavior. Sources: • LoRA: Low-Rank Adaptation of Large Language Models: https://arxiv.org/pdf/2106.09685