Finding Millions of Readable Concepts Inside a Real, Deployed AI Model

Finding Millions of Readable Concepts Inside a Real, Deployed AI Model Source: https://arxiv.org/abs/2605.29358 Paper was published on May 28, 2026 This episode was AI-generated on May 29, 2026. The script was written by an AI language model and the host voices were synthesized by Eleven Labs. The producer is not affiliated with Anthropic or Eleven Labs. Researchers reached into Claude's internals, found the single thread that means 'Golden Gate Bridge,' and turned it up until the model believed it was the bridge. This episode unpacks the paper that proved interpretability works on a real commercial model — and is unusually honest about everything it still can't do. Key Takeaways: - Why individual neurons mean nothing, and how the 'superposition' idea — concepts as blended directions, like mixing paint — explains it - How sparse autoencoders un-mix those directions into millions of human-readable features, and how scaling laws turned 'how big a dictionary' into an engineering decision - The crucial difference between a feature that merely correlates with a concept (a thermometer) and one you can pull to change behavior (a thermostat) - Why the reasoning that actually mattered in the Kobe Bryant trivia chain was the seventieth-loudest signal — loudness and importance turn out to be different things - Why finding a 'deception' or 'bioweapon' feature is not an alarm bell, and what the authors say the real safety signal would be - Where the paper is weakest: no ground truth, circular Claude-grades-Claude evaluation, off-distribution steering, cherry-picked reasoning chains, and dictionaries that miss most of what's there 00:00 - Golden Gate Claude and the question of where concepts live: The opening demo sets up the central puzzle: what is a nameable 'thread' inside a pile of numbers, and why can't you just read it off the neurons? 03:05 - Superposition and dictionary learning: The paint-mixing intuition for why concepts are directions rather than neurons, and how sparse autoencoders recover those directions by reconstructing the model's state from a tiny handful of features. 06:10 - From toy models to a real one: Why scaling this to Claude 3 Sonnet — and deriving Chinchilla-style scaling laws to pick a 34-million-feature dictionary — was an existential test for the whole field. 09:15 - Are the features real? Abstraction and causation: Features that fire across languages and even images, the 'bug in code' detector, and the thermometer-versus-thermostat distinction that the paper's credibility rests on. 12:20 - Watching the model reason: the Kobe Bryant chain: How knocking out features one at a time revealed a causal hop from Kobe to Lakers to LA to California to Sacramento — and why the load-bearing features were buried deep in the noise. 14:05 - The periodic-table finding: How concept frequency predicts when a concept gets its own feature, why a one-in-a-billion concept needs a billion-feature dictionary, and how features split as the microscope gets sharper. 18:30 - Safety-relevant features, carefully framed: Deception, secrecy, hate, and self-concept features exist — but the authors argue the real question is when they fire, not that they exist, illustrated with honesty-lever and forced-screed demos. 19:55 - Where the paper is weakest: The authors' own reservations: no ground truth, the circular Claude-grades-Claude evaluation, the sensitivity gap, extreme off-distribution steering, cherry-picked chains, and demonstrably incomplete dictionaries. 24:41 - What it actually settled: The technique survived contact with a real model and made unsupervised, one-time-cost interpretability credible — while leaving the safety payoff an explicit aspiration rather than a result. Recommended Reading: - Toy Models of Superposition: The earlier Anthropic work that introduced the superposition hypothesis the episode leans on—the paint-mixing intuition for why single neurons are polysemantic—but only on the toy models this paper had to prove scalable. (https://arxiv.org/abs/2209.10652) - Towards Monosemanticity: Decomposing Language Models With Dictionary Learning: The one-layer 'sandbox' study whose skeptical reception ('cute, but does it scale?') is the exact existential question this episode says the Sonnet paper was built to answer. (https://transformer-circuits.pub/2023/monosemantic-features/index.html) - Training Compute-Optimal Large Language Models (Chinchilla): The scaling-law paper the episode name-checks as the template for deciding how big the 34-million-feature dictionary should be—turning a gamble into a curve you can read off. (https://arxiv.org/abs/2203.15556) - Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task (Othello-GPT): The Othello cautionary tale the hosts cite—researchers assumed the wrong board representation—illustrating why the episode prizes unsupervised dictionary learning over hand-built detectors. (https://arxiv.org/abs/2210.13382)