Alignment Anxieties & Persuasion Problems

Dónal and Ciarán continue the 2025 season with a second quarterly update that looks at some recent themes in AI development. They're pondering doom again, as we increasingly grapple with the evidence that AI systems are powerfully persuasive and full of flattery at the same time as our ability to meaningfully supervise them seems to be diminishing.Topics in this episodeCan we see how reasoning models reason? If AI is thinking, or sharing information and it's not in human language, how can we check that it's aligned with our values. This interpretability issue is tied to the concept of neuralese - inscrutable machine thoughts!We discuss the predictions and prophetic doom visions of the AI-2027 documentIncreasing ubiquity and sometimes invisibility of AI, as it's inserted into other products. Is this more enshittification? AI is becoming a persuasion machine - we look at the recent issues on Reddit's r/ChangeMyView, where researchers skipped good ethics practice but ended up with worrying resultsWe talk about flattery, manipulation, and Eli Yudkowsky's AI-Box thought experimentResources & LinksThe AI-2027 piece, from Daniel Kokotajlo et al. is a must-read!Dario Amodei's latest essay, The Urgency of InterpretabilityT.O.P.I.C. - A detailed referencing model for indicating the use of GenAI Tools in academic assignments. Yudkowsky's AI-box Experiment, described on his site."The Worst Internet-Research Ethics Violation I Have Ever Seen" - coverage of the University of Zurich / Reddit study, by Tom Barlett for The AtlanticChatGPT wants us to buy things via our AI conversations (reported by Reece Rogers, for Wired)You can get in touch with us - [email protected] - where we'd love to hear your questions, comments or suggestions!