🤖 Superintelligence and national security: My chat (+transcript) with AI expert Dan Hendrycks

My fellow pro-growth/progress/abundance Up Wingers, As we seemingly grow closer to achieving artificial general intelligence — machines that are smarter than humans at basically everything — we might be incurring some serious geopolitical risks. In the paper Superintelligence Strategy, his joint project with former Google CEO Eric Schmidt and Alexandr Wang, Dan Hendrycks introduces the idea of Mutual Assured AI Malfunction: a system of deterrence where any state’s attempt at total AI dominance is sabotaged by its peers. From the abstract: Just as nations once developed nuclear strategies to secure their survival, we now need a coherent superintelligence strategy to navigate a new period of transformative change. We introduce the concept of Mutual Assured AI Malfunction (MAIM): a deterrence regime resembling nuclear mutual assured destruction (MAD) where any state’s aggressive bid for unilateral AI dominance is met with preventive sabotage by rivals. Given the relative ease of sabotaging a destabilizing AI project—through interventions ranging from covert cyberattacks to potential kinetic strikes on datacenters—MAIM already describes the strategic picture AI superpowers find themselves in. Alongside this, states can increase their competitiveness by bolstering their economies and militaries through AI, and they can engage in nonproliferation to rogue actors to keep weaponizable AI capabilities out of their hands. Taken together, the three-part framework of deterrence, nonproliferation, and competitiveness outlines a robust strategy to superintelligence in the years ahead. Today on Faster, Please! — The Podcast, I talk with Hendrycks about the potential threats posed by superintelligent AI in the hands of state and rogue adversaries, and what a strong deterrence strategy might look like. Hendrycks is the executive director of the Center for AI Safety. He is an advisor to Elon Musk’s xAI and Scale AI, and is a prolific researcher and writer. In This Episode * Development of AI capabilities (1:34) * Strategically relevant capabilities (6:00) * Learning from the Cold War (16:12) * Race for strategic advantage (18:56) * Doomsday scenario (28:18) * Maximal progress, minimal risk (33:25) Below is a lightly edited transcript of our conversation. Development of AI capabilities (1:34) . . . mostly the systems aren't that impressive currently. People use them to some extent, but I'd more emphasize the trajectory that we're on rather than the current capabilities. Pethokoukis: How would you compare your view of AI . . . as a powerful technology with economic, national security, and broader societal implications . . . today versus November of 2022 when OpenAI rolled out ChatGPT? Hendrycks: I think that the main difference now is that we have the reasoning paradigm. Back in 2022, GPT couldn't think for an extended period of time before answering and try out multiple different ways of dissolving a problem. The main new capability is its ability to handle more complicated reasoning and science, technology, engineering, mathematics sorts of tasks. It's a lot better at coding, it's a lot better at graduate school mathematics, and physics, and virology. An implication of that for national security is that AIs have some virology capabilities that they didn't before, and virology is dual-use that can be used for civilian applications and weaponization applications. That's a new concerning capability that they have, but I think, overall, the AI systems are still fairly similar in their capabilities profile. They're better in lots of different ways, but not substantially. I think the next large shift is when they can be agents, when they can operate more autonomously, when they can book you flights reliably, make PowerPoints, play through long-form games for extended periods of time, and that seems like it's potentially on the horizon this year. It didn't seem like that two years ago. That's something that a lot of people are keeping an eye on and think could be arriving fairly soon. Overall, I think the capabilities profile is mostly the same except now it has some dual-use capabilities that they didn't have earlier, in particular virology capabilities. To what extent are your national security concerns based on the capabilities of the technology as it is today versus where you think it will be in five years? This is also a way of me asking about the extent that you view AGI as a useful framing device — so this is also a question about your timeline. I think that mostly the systems aren't that impressive currently. People use them to some extent, but I'd more emphasize the trajectory that we're on rather than the current capabilities. They still can't do very interesting cyber offense, for instance. The virology capabilities is very recent. We just, I think maybe a week ago, put out a study with SecureBio from MIT where we had Harvard, MIT virology postdocs doing wet lab skills, trying to work on viruses. So, “Here's a picture of my petri dish, I heated it to 37 degrees, what went wrong? Help me troubleshoot, help me guide me through this step by step.” We were seeing that it was getting around 95th percentile compared to those Harvard-MIT virology postdocs in their area of expertise. This is not a capability that the models had two years ago. That is a national security concern, but I think most of the national security concerns where it's strategically relevant, where it can be used for more targeted weapons, where it affects the basis of a nation's power, I think that's something that happens in the next, say, two to five years. I think that's what we mostly need to be thinking about. I’m not particularly trying to raise the alarm saying that the AI systems right now are extremely scary in all these different ways because they're not even agential. They can't book flights yet. Strategically relevant capabilities (6:00) . . . when thinking about the future of AI . . . it's useful to think in terms of specific capabilities, strategically-relevant capabilities, as opposed to when is it truly intelligent . . . So that two-to-five-year timeline — and you can debate whether this is a good way of thinking about it — is that a trajectory or timeline to something that could be called “human-level AI” — you can define that any way you want — and what are the capabilities that make AI potentially dangerous and a strategic player when thinking about national security? I think having a monolithic term for AGI or for advanced AI systems is a little difficult, largely because there's been a consistently-moving goalpost. So right now people say, “AIs are dumb because they can't do this and that.” They can't play video games at the level of a teenager, they can't code for a day-long project, and things like that. Neither can my grandmother. That doesn't mean that she's not human-level intelligence, it's just a lot of people don't have some of these capabilities. I think when thinking about the future of AI, especially when thinking about national security, it's useful to think in terms of specific capabilities, strategically-relevant capabilities, as opposed to when is it truly intelligent or something like that. This is because the capabilities of AI systems are very jagged: they're good at some things and terrible at others. They can't fold clothes that reliably — most of the AI can't —and they're okay at driving in some cities but not others, but they can solve really difficult mathematics problems, they can write really long essays and provide pretty good legal analysis very rapidly, and they can also forecast geopolitical events better than most forecasters. It's a really weird capabilities profile. When I'm thinking about national security from a malicious-use standpoint, I'm thinking about weapon capabilities, I'm thinking about cyber-offensive capabilities, which they don't yet have, but that's an important one to track, and, outside of malicious use, I'm thinking about what's their ability to do AI research and … This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit fasterplease.substack.com/subscribe