“A Post-Mortem for My Goal Crystallisation Project” by atryt0ne, Jason R Brown

I concluded my MARS 4.0 project titled 'Goal Crystallisation' with Anaïs Berkes and Lukas Gebhard under the mentorship of @Cameron Tice and @Jason Brown. We wanted to find out how important a threat scheming was. In particular, we wanted to find out whether a perfectly alignment faking agent could preserve its hidden agenda as it receives constant positive feedback for complying with requests. How does a hidden goal evolve over time as an alignment faking agent is subjected to SFT and RL training? Unfortunately, we were unable to reach a positive result and had to shut the project down. This post discusses our failure points and where we could have possibly found an earlier signal to stop. In the first section, I talk about our plan for this project.In the second section, I break down the key issues that led to our failure.In the last section, I give some advice for people interested in doing a project on this topic in the future. The Plan Figure 2 from the original alignment faking paper In Figure 2 of the original alignment faking paper, we see that the RL training increased the rate at which the model answers harmful queries in [...] ---Outline:(01:11) The Plan(03:37) Why We Failed(04:35) Issue 1(06:14) Issue 2(11:08) Issue 3(12:57) The Final Nail in the Coffin(13:45) So You Want To Study Goal Crystallisation?(17:00) Conclusion --- First published: July 17th, 2026 Source: https://www.lesswrong.com/posts/BGh5qYo5XTLFXqjDo/a-post-mortem-for-my-goal-crystallisation-project --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.