Series Overview — Mastering Language Models: From Architecture to Optimization

Maya and Leo open the series with the map: seven stops from the Transformer blueprint to the machinery under massive models, anchored by a three-person startup building an insurance-claims assistant on eight GPUs. They lay out the mental models every LLM expert shares — trust curves, find the bottleneck, separate capability from behavior — then stage the field's cleanest fight on air: bigger models versus more data, from OpenAI's 2020 scaling curves to Chinchilla's flip to the serving-cost era that ran past both camps. Plus trailers for the live attention debate and the alignment fight to come. Sources: • Attention Is All You Need: https://arxiv.org/pdf/1706.03762 • Kimi Linear: An Expressive, Efficient Attention Architecture: https://arxiv.org/pdf/2510.26692 • Scaling Laws for Neural Language Models: https://arxiv.org/pdf/2001.08361 • Training Compute-Optimal Large Language Models: https://arxiv.org/pdf/2203.15556 • FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness: https://arxiv.org/pdf/2205.14135 • LoRA: Low-Rank Adaptation of Large Language Models: https://arxiv.org/pdf/2106.09685 • Direct Preference Optimization: Your Language Model is Secretly a Reward Model: https://arxiv.org/pdf/2305.18290 • Constitutional AI: Harmlessness from AI Feedback: https://arxiv.org/pdf/2212.08073 • Llama 2: Open Foundation and Fine-Tuned Chat Models: https://arxiv.org/pdf/2307.09288