“Research Sabotage in ML Codebases” by egan

One of the main hopes for AI safety is using AIs to automate AI safety research. However, if models are misaligned, then they may sabotage the safety research. For example, misaligned AIs may try to: Perform sloppy research in order to slow down the rate of research progressMake AI systems appear safer than they areTrain a successor model to be misaligned Whether we should worry about those things depends substantially on how hard it is to sabotage research in ways that are hard for reviewers to detect. To study this, we introduce Auditing Sabotage Bench, a benchmark of 9 ML research codebases with sabotaged variants. We tested frontier LLMs and LLM-assisted humans on the benchmark and found that neither reliably catches sabotage. Our best auditor, Gemini 3.1 Pro, achieved an AUROC of 0.77 and a top-1 fix rate of 42%. LLM-assisted humans performed comparably but slightly worse. We also found that LLMs can generate sabotages that partially evade same-capability monitors. 📄 Paper 💻 Code Methodology To make the benchmark, I (Eric) found 9 existing ML codebases,[1] and for each one, produced an honest version that reproduces a key finding and one or two sabotaged versions that [...] ---Outline:(01:23) Methodology(03:30) Audits(04:07) Results(05:49) Takeaways(06:05) 1. Predicting experiment outcomes is a key red team skill(07:22) 2. Omission beats design beats logic(08:45) 3. Benign distractors make auditors miscalibrated(09:37) 4. LLMs can also produce sabotages(11:05) 5. More categorizations(12:40) Conclusion The original text contained 4 footnotes which were omitted from this narration. --- First published: April 29th, 2026 Source: https://www.lesswrong.com/posts/LByP4qsF8a4g7Pz3p/research-sabotage-in-ml-codebases --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.