Foundations of Sequence Modeling: The Transformer Revolution

Topic 1 opens the series at the foundation: how 'Attention Is All You Need' replaced step-by-step recurrence with self-attention — every token seeing every other token in one parallel hop — and why that single move made large-scale pre-training possible. Maya and Leo build the topic's shared mental models (attention as content-based lookup, the two bills of training versus serving, architectures as hardware bets), then stage the field's live fight on air: exact full attention versus Kimi Linear's expressive hybrid, with the KV cache and million-token contexts as the battleground. A running law-firm contracts assistant grounds every turn. Sources: • Attention Is All You Need: https://arxiv.org/pdf/1706.03762 • Kimi Linear: An Expressive, Efficient Attention Architecture: https://arxiv.org/pdf/2510.26692 • FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness: https://arxiv.org/pdf/2205.14135