One Crosscoder Feature Flips a Stalling Chatbot Into a Working Agent

One Crosscoder Feature Flips a Stalling Chatbot Into a Working Agent Source: https://arxiv.org/abs/2606.26474 Paper was published on June 25, 2026 This episode was AI-generated on June 26, 2026. The script was written by an AI language model and the host voices were synthesized by Eleven Labs. The producer is not affiliated with Anthropic or Eleven Labs. Reinforcement learning spent a whole training run teaching a model to use tools — and it turns out you can find that skill, grab one internal feature, and flip the behavior on at runtime with no retraining at all. But the same evidence that says the skill lives in one place also shows it quietly leaking into a model that was never trained for it. This episode unpacks what RL actually localizes, where it lives, and why you can concentrate a capability but never fully wall it off. Key Takeaways: - Why a single 'dedicated' crosscoder feature, steered at inference time with no weight changes, can recover most of an RL model's tool-calling accuracy - How just routing activations through the sparse dictionary and back raises tool correctness from 19% to ~50% — even though reconstruction quality barely predicts the gain - The 'capability spillover' result: a frozen base model, never trained for tools, picks up tool selection (0% to ~7%) just by passing through the shared crosscoder — but never reproduces the tool-call syntax - Why the exclusive feature shelf is a coffee filter, not a sealed sink — penalizing it degrades the RL model, proving the captured signal is load-bearing and leaky - The honest limits: the +65 number comes from one best-performing cell on 40 prompts with a wide confidence band, and the DFC's advantage is legibility, not better performance - Why the cleanest features are structural-template detectors — and why that may be exactly why a tool-calling skill concentrates into one dial when a messier capability might not 00:00 - Where does an RL skill actually live?: Sets up the puzzle: RL visibly installs tool use, but no one can point to where in the network that capability physically lives. 02:34 - Reading the model's muddy scratchpad: Explains superposition and sparse dictionaries — the tools that separate a model's blended internal state back into named features. 04:26 - Bolting down the shelves: the DFC: Introduces the crosscoder and the Dedicated Feature Crosscoder, which forces features into RL-exclusive, base-exclusive, and shared bins. 07:13 - One master switch versus a fuse box: Walks through the saturation curve where one DFC feature hits the accuracy ceiling while the plain crosscoder needs 33 features. 09:29 - Feature 136 turns a hedger into an agent: The before-and-after example where steering a single feature produces a clean, correct tool call — and reveals the top features are template detectors. 11:03 - Why lossy reconstruction makes it better: The surprising finding that just routing activations through the dictionary and back boosts tool correctness, validated across 48 crosscoder variants. 13:09 - A frozen model catches the trick: Capability spillover: the untrained base model inherits tool selection through the shared decoder, but never the exact tool-call syntax. 15:10 - A coffee filter, not a sealed sink: Penalizing the exclusive shelf degrades the RL model, showing the capability is entangled in shared geometry and can be concentrated but never fully isolated. 18:22 - How soft is that headline number?: The critique: the +65 estimate is a favorable draw on 40 prompts, the architecture comparison isn't significant, and 'capability' means propensity under one prompt. 22:08 - When your interpretability tool leaks: Why feature-level steering offers a gradient-free control handle for agents — but published diffing artifacts may themselves become a side channel that moves capability around. Recommended Reading: - Towards Monosemanticity: Decomposing Language Models With Dictionary Learning: The Anthropic sparse-autoencoder work that grounds the episode's 'separate the mud back into named pigments' picture of superposition and single-meaning features. (https://transformer-circuits.pub/2023/monosemantic-features/index.html) - Sparse Crosscoders for Cross-Layer Features and Model Diffing: The original crosscoder writeup that introduced the shared-dictionary model-diffing approach the episode's Dedicated Feature Crosscoder extends. (https://transformer-circuits.pub/2024/crosscoders/index.html) - Toy Models of Superposition: The foundational account of why a few-thousand-dimensional scratchpad packs far more concepts than dimensions — the entanglement the episode says makes perfect capability isolation impossible. (https://transformer-circuits.pub/2022/toy_model/index.html)