EP 53 | CS224N: Model Interpretability & Editing
from AI Bites: The Academic Series Hosted by Jack Lakkapragada
How do we communicate with an AI that thinks in alien, superhuman concepts? In this episode, featuring insights from Google Brain’s Dr. Been Kim, we explore the massive gap between what we think machines know and what they actually know. We expose the fatal flaws in our current interpretability tools and look at how researchers are extracting brand-new strategies from AI to teach the World Chess Champion. Key Topics: The Illusion of Saliency Maps: The shocking mathematical proof that popular interpretability tools (like SHAP and Integrated Gradients) perform no better than random guessing when trying to explain a model's behavior. The ROME Paradox: Why locating where a fact lives inside a model's brain has absolutely zero correlation with successfully editing or fixing that fact. Observing AI in the Wild: How researchers are using unsupervised clustering to map emergent AI behaviors—like discovering how multi-agent systems learn to build forts and cheat physics engines. Teaching Magnus Carlsen: The fascinating AlphaZero project where researchers force an AI to forget human chess strategies so it can isolate entirely new, superhuman concepts to teach the World Chess Champion. Note: This is an AI-generated discussion created using Google's NotebookLM, based on publicly available Stanford University course material (specifically CS224N) and personal study notes from my learning journey.
Embed this episode
No transcript yet.