“Sympathy for both sides of the egregious misalignment debate” by Steven Byrnes

On one side of this debate is Yudkowsky & Soares, who think that (if AI progress continues) we’re on a direct path to egregiously-misaligned, scheming, out-of-control, rogue superintelligence (ASI), not even slightly nice, in the absence of yet-to-be-invented breakthrough technical alignment ideas. On the other side of this debate is almost everyone who works on or studies LLMs. Some of them are very concerned about egregious scheming, others much less so, and as a group they’re equally or more concerned about lots of other potential AI problems—AI-assisted bioterrorism, AI-assisted dictatorships, etc. And if they’re concerned about egregious misalignment and scheming, they’ll probably say that it would come about through race dynamics, careless programmers, bad actors, etc., as opposed to the simpler Yudkowsky & Soares story of “we get egregious misalignment and scheming because nobody has the faintest clue how to avoid that”. Here's my brief idiosyncratic take on this debate. I think BOTH of the following are true: (1) If you really think carefully about the properties of ASI, you really do find good reasons to strongly expect it to be egregiously misaligned, scheming, and ruthless, in the absence of yet-to-be-invented breakthrough technical alignment ideas.(2) If you [...] ---Outline:(01:58) Yudkowsky & Soares's position \[caricatured\]:(03:18) LLM people's position \[caricatured\]:(04:09) Conclusion(04:19) Bonus section: Further commentary(04:28) My "true objection" to Yudkowsky & Soares:(05:04) My within-frame complaint at Yudkowsky & Soares:(06:42) My "true objection" to LLM people:(07:11) My within-frame complaint at LLM people: --- First published: June 12th, 2026 Source: https://www.lesswrong.com/posts/DZaZ3fqHnvfLCftPu/sympathy-for-both-sides-of-the-egregious-misalignment-debate --- Narrated by TYPE III AUDIO.