Nate Soares — If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All - with Jon Wolfsthal

In 2023, hundreds of AI luminaries signed an open letter warning that artificial intelligence poses a serious risk of human extinction. Since then, the AI race has only intensified. Companies and countries are rushing to build machines that will be smarter than any person. And the world is devastatingly unprepared for what would come next.For decades, two signatories of that letter--Eliezer Yudkowsky and Nate Soares--have studied how smarter-than-human intelligences will think, behave, and pursue their objectives. Their research says that sufficiently smart AIs will develop goals of their own that put them in conflict with us--and that if it comes to conflict, an artificial superintelligence would crush us. The contest wouldn't even be close.How could a machine superintelligence wipe out our entire species? Why would it want to? Would it want anything at all? In this urgent book, Yudkowsky and Soares walk through the theory and the evidence, present one possible extinction scenario, and explain what it would take for humanity to survive.The world is racing to build something truly new under the sun. And if anyone builds it, everyone dies.Nate Soares is the President of MIRI. He has been working in the field for over a decade, after previous experience at Microsoft and Google. Soares is the author of a large body of technical and semi-technical writing on AI alignment, has been interviewed in Vanity Fair and the Financial Times, and has spoken on conference panels alongside many of the AI field's leaders.Soares is in conversation with Jon Wolfsthal, the Director of Global Risk at FAS. Jon B. Wolfsthal is also a senior adjunct fellow at the Center for a New American Security and member of the Science and Security Board of the Bulletin of the Atomic Scientists. He was appointed to the US Department of State’s International Security Advisory Board in 2022. He served previously as senior advisor to Global Zero in Washington, DC. Before 2017, Mr. Wolfsthal served as Special Assistant to President of the United States Barack Obama for National Security Affairs and is a former senior director at the National Security Council for arms control and nonproliferation. He also served from 2009-2012 as Special Advisor to Vice President Joseph R. Biden for nuclear security and nonproliferation and as a director for nonproliferation on the National Security Council.https://politics-prose.com/book/9780316595643?ic_referral=j16l5IPPCHC-rdT-DrLqrBP7EVpsbMLgZWV9kLMYJBIwM039knrOSPLQqIVe-u89dRgkSpcDODPK_1w8h93fDYdQ5Gjah8FxrEuGxAhV3TqBD6n6nQZURkmBqF4XzOs7t9PRa28