David McKay: AI and the end of the world as we know it

This week on Unsupervised Learning, Razib and his guest, David McKay, of the Standing on the Shoulders of Giants podcast (Razib was an early guest), discuss the rise of artificial intelligence (AI) and the prospects for artificial general intelligence (AGI). This discussion arose after Razib heard McKay's explainer, Zen and the Art of ChatGPT, a 30-minute layman's intro to the topic, where he breaks down the technical elements that come together to allow for AI. In this episode, McKay, a Cambridge University-trained computer scientist who has worked at Hotmail and Google, digs deeper into the nature of Large Language Models (LLMs) and how they give rise to probabilistic generative AI like ChatGPT and whether we should be worried. Razib's conversation with McKay follows another recent episode on AI. I  the earlier podcast, Nikolai Yakovenko: GPT-3 and the rise of the thinking machines, the interviewee, a computer scientist, was relatively sanguine about the world-ending possibilities of AGI. McKay generally takes the same position, highlighting the reality that most computer scientists and AI researchers are less worried about science-fictional apocalyptic scenarios than the general public or AI-skeptics like Eliezer Yudkowsky and Nick Bostrom (the author of Superintelligence: Paths, Dangers, Strategies) are. And yet the reason that AI is so topical is it seems that the development of the technology is proceeding along an exponential path; ChatGPT 4 was released months after ChatGPT 3. McKay and Razib also discuss the release of Bard, Google's chatbot, and the offering from Microsoft's Bing, and how they are similar and different from ChatGPT.  While McKay is optimistic about the possibilities of AI as a tool, ultimately, he is in the camp that believes it really isn't intelligent in the same way as a human. Because it relies on the corpus from the internet, ChatGPT cannot really do math. It lacks true conceptual understanding that would allow it to grasp truth beyond what the internet might tell it. Razib and McKay also talk about the energetic resources that LLMs consume (Microsoft had to reallocate compute resources after the release of Bing's chatbot), and how that might be a limitation on their scalability.