Attention Is All You Need

The first deep dive of the series opens the 2017 Transformer paper itself. Maya walks the machine in three stations — the Matchmaker (query-key-value lookup), the Committee (eight specialist attention heads), and the Chord (wave-stamped word order) — while Leo brings the receipts: a two-point BLEU jump over every published system, base-model training in about twelve hours on eight GPUs, and an ablation table that proves the committee earns its seat. Then the skeptic's pass: what the paper never claimed, why 'foundation of modern AI' was the field's later inference, and how the authors wrote the quadratic-cost limitation — the seed of the linear-attention debate — into their own final page. The running law-firm contracts assistant grounds the mechanism throughout. Sources: • Attention Is All You Need: https://arxiv.org/pdf/1706.03762 • Kimi Linear: An Expressive, Efficient Attention Architecture: https://arxiv.org/pdf/2510.26692 • BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding: https://arxiv.org/pdf/1810.04805 • Improving Language Understanding by Generative Pre-Training (GPT): https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf