EP 51 | CS224N: AI Reasoning (Part 2)
from AI Bites: The Academic Series Hosted by Jack Lakkapragada
Today, we are pushing the absolute limits of how Language Models generate text. We move beyond basic architecture to explore how engineers are making AI insanely fast, teaching models to recover from their own mistakes, expanding context windows so AI can read entire books, and proving that a small model can beat an industry giant just by "thinking" longer. Key Topics: Speculative Decoding: How pairing a massive "Senior Architect" model with a tiny "Junior Coder" draft model speeds up AI text generation by up to 3x with zero loss in quality. On-Policy Distillation: Why teaching a student model to generate text and recover from its own mistakes (Reverse KL Divergence) is superior to blindly copying a teacher model. Extending the Context Window: The brilliant math behind Rotary Position Embedding (RoPE) and how manipulating the rotation of word vectors allows models to extrapolate and process massive documents. Inference-Time Scaling: The paradigm shift of test-time compute. We explain why letting models self-correct via Process-supervised Reward Models (PRMs) is the new frontier of AI efficiency. Note: This is an AI-generated discussion created using Google's NotebookLM, based on publicly available Stanford University course material (specifically CS224N) and personal study notes from my learning journey.
Embed this episode
No transcript yet.