“Anthropic’s strange fixation on “hyperstition”” by Simon Lermen

EPISODE · May 11, 2026 · 12 MIN

“Anthropic’s strange fixation on “hyperstition”” by Simon Lermen

from LessWrong (30+ Karma)

In a recent tweet, Anthropic seems to have asserted that hyperstition is responsible for observed misalignment in their AIs. Strangely, the research they use as evidence actually doesn’t seem to be related to hyperstition at all? I think this is part of a pattern by Anthropic of promoting the theory of hyperstition–the idea that writing about misaligned AI helps bring misaligned AI into existence. Anthropic recently released this tweet as part of a tweet thread for a new research post on alignment. They conclude: “[...] We believe the original source of the [blackmail] behavior was internet text that portrays AI as evil and interested in self-preservation. [...]” However, the research post shared with this tweet doesn’t seem to be about hyperstition at all. Instead they find that training the model on reasoning traces– generated by reflecting on its constitution while giving users ethical advice on difficult dilemmas– reduces misaligned behavior. This presumably works by making the AI better understand what behavior is expected of it by having it reason through concrete scenarios based on its constitution. The post explicitly notes that this works better than training on stories where an AI behaves admirably– which appears more similar to positive [...] ---Outline:(02:06) The adolescence of technology(03:57) Persona Selection Model(04:26) What does this all mean?(05:20) If it was true, this would still be their fault(07:04) What about filtering?(09:31) Personas are a bad alignment strategy --- First published: May 11th, 2026 Source: https://www.lesswrong.com/posts/xhpktBLttPc6uXcHP/anthropic-s-strange-fixation-on-hyperstition --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

NOW PLAYING

“Anthropic’s strange fixation on “hyperstition”” by Simon Lermen

0:00 12:03

No transcript for this episode yet

We transcribe on demand. Request one and we'll notify you when it's ready — usually under 10 minutes.

Accidental Accountant Regan Williams Hi, I'm Regan! I'm a CPA of 30+ years helping "accidental accountants" navigate tax & accounting issues with confidence! Here, we find solutions to common challenges bookkeepers, accountants and CPAs face. Don't see an answer to your question? Then ask! I'm here to help people like you. Profit Powerhouse Glenn Poulos Glenn Poulos is the co-founder, Vice President, and General Manager of Gap Wireless Inc., a leading product and service distributor for the mobile broadband and wireless markets. With over three decades of experience in sales, he has spent thousands of hours in the field or on the phone with customers and working with salespeople to help create several very successful companies. He is now also the host of this podcast, Profit Powerhouse!Our 20 to 30-min podcast shares amazing founder stories who reveal the smartest strategies for scaling TODAY. Love My Quarter Life Beth Schofield In a world filled with countless decisions and societal pressures, navigating our twenties & thirties can be tough. But you’re not alone, and you’re in the right place because this podcast is dedicated to supporting 20 & 30-somethings to overcome the overwhelm of Quarter Life Confusion. The weekly episodes offer you the motivation and inspiration you need to get unstuck, find what’s missing and move forward in life with meaning, passion and purpose. Two Recruiters: Zero Filter Two Recruiters At Two Recruiters: Zero Filter, we're on a mission to demystify the hiring process, share insider tips, and empower you to maneuver through the professional world with confidence. With more than 30 years of combined experience navigating the intricate web of job markets, talent acquisition, and career development, we're here to spill the tea on everything career related. But wait, there’s more! We will dive into many life topics that are interesting to us as well.  Get ready for a rollercoaster of insights, stories, and no-holds-barred advice!Join us for conversations that matter – where work, life, and authenticity collide in the most unexpected and rewarding ways.
URL copied to clipboard!