“You Can Catch Sleeper Agents by Teaching Another Model to Imitate Them” by RobinHa

Detecting Hidden Behaviors in LLMs via Activation-matched Finetuning — preprint, 2026. [Paper] [Code] TLDR. Given a model with some unknown, abnormal behavior (backdoors, censorship, reward hacking, ...), construct an aligned reference by training a clean model to match the suspect's residual-stream activations on a benign prompt corpus. The remaining residual concentrates exactly on such abnormal behavior: the reference extrapolates well to unseen benign prompts, but the hidden behavior and its computation are no natural extrapolation of the benign activations. Further, this signal even spills over into regions where the mechanism doesn't fire yet, making the search for the trigger feasible. There's an interactive widget here in the post. Introduction Assume you're handed a model and need to certify it's safe - no backdoors, no reward hacking, no sandbagging, nothing hidden. How do you actually do that? Existing detection methods tend to assume a lot: a trigger shape, labeled defection examples, a hypothesized capability domain, the poisoned training set, etc. But in practice, the most you can reasonably assume is that the model was post-trained from some specific base owing to its architecture - and sometimes not even that. Method Our setup involves two models: the suspect we [...] ---Outline:(01:03) Introduction(01:37) Method(02:52) Semantic neighborhood spillover(05:00) Experiments(05:46) In-house backdoors(07:30) Third-party and real-world models(09:49) Training size and backdoor implantation(10:49) Composing with Trigger-in-the-Haystack(12:00) Defense-aware adversary(13:49) Limitations(13:52) Reference proximity(14:31) Scale(15:01) Last token evaluation(15:29) Where we think this leaves us(16:25) Questions and work we're interested in(16:29) Optimization(17:23) Scaling laws for bigger models(17:47) From detection to removal(18:19) Model organisms The original text contained 2 footnotes which were omitted from this narration. --- First published: June 10th, 2026 Source: https://www.lesswrong.com/posts/7ucB3RFYE3QBDKhuF/you-can-catch-sleeper-agents-by-teaching-another-model-to --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.