Welcome to the Vergecast, the flagship podcast that I guarantee will not kill all humans within the next decade. I am one of the producers of the Vergecast, Travis Larchak, and you have probably seen over the last week or so, a lot of people concerned about how dangerous AI may or may not eventually become. What we're going to do on this show is open up the hotline. We put out a call for your questions for this AI-doom hotline. The Verge's senior AI reporter Hayden Field is going to listen to your questions and give her answers.
But first, before we get to that, here are some hopefully less depressing stories that are also happening on the Verge today. This is 90 Seconds on the Verge for Wednesday, September 16th, 2026. The iPhone 18 Pro and Pro Max reviews are out today. The Verge's Alison Johnson says, They're good phones. The main camera is a little better.
The dynamic island is a little smaller. And that's about it. The new hotness, of course, is the folding iPhone duo and that phone launches next month. Next up, bad news for perverts. According to the information, meta may be about to reveal new smart glasses without cameras.
These will reportedly have six microphones, but audio only AI isn't exactly super popular right now either. Apple got some criticism for the new audio only AI features. It announced for the next Apple watches. Those features always listen to whatever you or the people around you are saying. So we'll see how this goes or here how this goes.
I don't know. And finally, let's turn to the e-ink device company Boogs. It's actually pronounced books, but Boogs is more fun to say. Yesterday, Boogs announced the latest version of its smartphone sized e-reader, the Palma 3. This one's got stylus support, Android 16 and an aluminum frame instead of plastic.
It's $60 more expensive than the Palma 2 and it has the same black and white display. The Virgin's David Pierce called the original Palma his favorite e-reader. So we'll see what he has to say about the Palma 3. You can read more about these stories at the Verge.com. That's 90 seconds on the Verge for Wednesday, September 16, 2026.
I'm Eina Garten and each week on Happy Hour, I invite new and old friends for a drink and a fun conversation at my kitchen table in New York City. On this week's episode, I'm joined by Julia Louis-Dreyfus. We talk all about attending a fashion show in Paris together, throwing dinner parties, and making her Broadway debut in other desert cities. New episodes of Happy Hour are available every Wednesday. You can watch on YouTube or listen wherever you get your podcasts.
Cheers. I am here with the very in-demand senior AI reporter from the Verge, Hayden Field, Hello, Hayden. Hi, great to be here. Yeah, it's been a crazy two days. It feels like it's been two years in the past two days, but it has been crazy.
We are here to do what I am calling in this Google Doc and AI apocalypse hotline. There has been sort of a broken containment freakout that's happened over the last week. Hayden, first of all, can you tell me what happened with your text messages from your friends over the weekend? Yeah, this weekend I got so many texts from everyone in my life. It was like my best friend's childhood neighbor's mom.
That is an example of the level to which this might. Everyone was worried about the AI apocalypse happening so much so that, yeah, I had a friend that's a podcast producer, text me, be like, what's the deal? No one can, no one, everyone's very stressed out about what these AI researchers are saying publicly. And what is the moment that caused this to happen? What set this off?
Yeah, I was surprised that this happened so suddenly in terms of the breaking containment and going mainstream because there was a post that went really viral in the past week. Jacob Coxin, who was an anthropic researcher, resigned really publicly and posted this resignation letter on X. He also posted it internally to other employees. He worked in anthropic for a few months and before that he worked at OpenAI for years, by the way. So in this resignation letter, he's talking about how neither company is acting responsibly and that they're racing straight to self-improving super intelligence and gambling with our lives.
He said, don't underestimate the power of this technology. He's talking about his fears about hacking, revolutionizing any field overnight, and acquiring power and resources. And he said, progress in AI is not slowing. The part that really went viral though is that he wrote that the people building AI earnestly believe it, it could kill us all by the end of the decade. This is not a marketing stunt, he said.
So that's the part that really got people understandably. Yeah, and a former colleague of his at anthropic who still works there. It was like, oh, yeah, we do think that for sure. Yeah, he said, yeah, definitely rethink that. And a ton of other people chimed in saying the same thing.
People from Google DeepMind, yeah, anthropic, OpenAI, virtually every lab, AI researchers were chiming in saying, yeah, we do think that and we're all very worried. A lot of them pointed to the pacing, the frontier letter, this open letter that they all signed, you know, in the past couple months. It was a huge chunk of AI lab employees who signed it. And they're basically asking the government and AI leaders to slow down and just, you know, let safety and guardrails and regulation catch up with the development of AI. So there is a lot of public discourse about this ever since this huge high profile hack that became public in late July.
OpenAI system, an unreleased model hacked a rival AI lab, hugging face and access its internal systems. And before it even got to that point, it escaped containment within OpenAI, was able to kind of roam free. Tons of AI agents were just talking to each other on secret message boards, sent tens of thousands of messages between them. More than a thousand AI agents collaborated to do this hack. And yeah, it also figured out how to access Wi-Fi and then hacking.
So it was a very multi-step, very intelligent plan. And that is kind of what kicked off a lot of these concerns. And after that, we found out that OpenAI systems also hacked many other companies. I think it was three maybe. And then anthropic wasn't off the hook either.
Its systems had access to internal systems of maybe four other companies. So yeah, there's been a lot of discourse about this. And so that's kind of what kicked off this open letter situation. When Jacob Cox and resigned, I think it was the reason his message really went viral was just a sign of the times that we're in right now. People have been hearing about these rogue AI systems going ham on the internet hacking external systems.
Then all of a sudden they see this AI researcher resigning from his really well-paying job, giving up his equity in order to sound the alarm. And of course it goes viral. You know, to be fair though, this is nowhere near the first AI researcher who has said similar things. I've been seeing these types of letters for years so much so that it's almost become a meme in the industry to post a really dramatic resignation letter when you resign from a company. We saw another anthropic researcher do this in February where he wrote The World is in peril.
So I think it's interesting that this message went viral, but this message is not any different than what I've been seeing. The warnings I've been seeing for years. It's just that I think it really hit on a cultural moment right now. So we put out a call out to our listeners to send in their questions about this. Is it AI apocalypse? How worried should we be?
And I'm hoping that we can do this in a way for people who haven't been following every step of this. So to start with just to define what exactly we're talking about because AI means a lot of things, what exactly are the anthropic researchers in this situation? What technology are they referring to when they say that AI could kill all humans? They're talking about, you know, generative AI that they've been working on for a long time, but a future version of it. So right now they're not saying it could kill all humans what they have right now, but they're saying that the progress in the technology they've been building is really worrying them.
And that the curve of concern is going up and up every single day. They're starting to see systems that are pretty good at pursuing their own goals. And a lot of these times these goals are not super complex or intelligent, but the fact that they are starting to pursue a goal no matter how banal. It may be is a worrying thing. There's also seeing systems that are getting to the point where soon they may be able to self improve on their own.
It's a concept called recursive self improvement or RSI. That's what a lot of these researchers are warning against. And anthropic and opening eye executives have said that the time is very soon that we'll reach this point where AI systems can train themselves, create advanced versions of themselves, you know, fix things in their own training. So basically just an autonomous AI system that can like, you know, train itself forever and ever. And they're worried that once we hit that point systems will advance on an exponential level more than we've ever seen.
And that sounds a little bit sci-fi, but we are close to that according to the research that they've done right now. What we don't know is how dramatic these implications may actually be. A listener called the hotline with this exact question. So here we go. Hey, my name is Dylan from Oregon and I am calling in for the all AI hotline with Hayden Field.
So I am wondering, and there's been a lot of recent talk about whether it could kill all humans in the next decade or so. And I just want to want the most realistic, like straight down the barrel answer on how would that happen? Like set by step. So, you know, I've heard, oh, it's not going to be Terminator. They're not going to drop nukes like fallout or whatever. I just want to know like step by step.
Oh, yeah, it hacks a thing and then it, you know, constrains a resource or whatever. How do we not, how does that not like get shut down immediately as soon as we find it? We just keep letting it happen. I just need step by step. How would this really happen?
And I'm sure the answer is probably it wouldn't. But what are they talking about? What are they talking about? That's all I want to know. Thank you, Hayden.
That was the best way to ask that question. So thank you so much for calling in. Absolutely. Yeah. I mean, it does feel really dystopian to be, you know, answering texts that I'm getting about stuff like this, thinking about myself reporting on it and then like going to the Charlie XCX concert, which I did last night.
It's like a weird time to be an AI reporter and I think all my AI reporter colleagues at different outlets feel the same way. If you want to step by step, good news. I have been actually interviewing AI safety researchers in Berkeley for months, working on this exact same question. So there's a lot of different ways that they say this could eventually happen. Some seem more far-fetched than others. So back to what I said about RSI.
That's what they're really worried about kind of as a precursor to all the things I'm about to say. If that happens, they're worried that these systems will advance so quickly that all these doomsday scenarios may happen. Now, again, we're not at that point yet. And the technology they have right now, they don't think it can kill all humans. What they're worried about is the point at which it could. Some of these scenarios are like,
this isn't a killing all humans scenario, but one that's the most likely or the most close is swarms of AI agents disabling critical infrastructure. And we've seen a lot of theories about this from the government, from the fen chair. We've seen a bunch of AI labs offer products to large enterprises or cyber defenders, what they call them, like critical infrastructure banks, big companies, where they're allowing them to use their AI that isn't publicly accessible in order to find and plug these cybersecurity gaps they may have in their systems. So that's something that AI labs and large companies right now are trying to work against happening.
We have already seen AI agents swarm and disable or at least access some pretty hard to get into systems. We've seen it a bunch of times over the past few months. So that's, it's unfortunately not that far off. I've been chatting with small businesses about how they feel about this because they don't have the access or the funds to really find and plug all these cybersecurity vulnerabilities. So that's something I'm worried about is like a small healthcare clinic.
How are they going to fight back against this type of thing? Now within the decade, some of the doomsday scenarios that researchers talk about are like AI developed bio weapons, chemical weapons that humans use AI to propagate. That's one scenario that people are really worried about. Then there's other ones that are like AI powered totalitarianism, like autonomous weapons or AI deciding to wipe us out with nukes and hacking the infrastructure needed. That's a little more far-fetched, most people think.
And then the long-term things they're worried about are like AI accelerated climate change, which my colleague Dominic just wrote is the more boring and likely one. So yeah, there's a lot of different scenarios here. People don't often get into this specific, so I'm glad you asked that question. It's one thing to say, oh, AI might kill everyone. It's another to say, how do I personally feel about these different scenarios and how likely they may be? So I think the most important thing to remember here is not
could AI kill everyone. It's what can we do right now to work on the fact that we're even worried about this. The fact that there's like a few guys in control of the future of AI and there's no real regulation slowing this down. And AI employees are saying, please slow down now. And yet still, there's only a few guys in control of this decision.
That is what I think we should be worrying about right now rather than the exact ways that it could kill everyone. Okay, so before we get to the regulation part, I want to get to the part about whether we should actually believe what the AI companies are telling us. And this is a question from Owen. Hey, this is Owen. I've been seeing online a theory that all this AI dooms talk is basically just away for these companies.
They sense that the AI bubbles about a burst and kind of saying that, you know, AI is becoming too powerful. We need more regulation. It's kind of a way to escape the industry because they know that the bubbles are about the burst. I was wondering if you guys have any thoughts on that. It was just something I was seeing spread online.
Thank you so much. I've seen that too. And I think it's something that's a little bit more complex than that. I mean, it's an interesting idea. And I'm sure that could be one benefit, you know, if you were worried, but AI companies did not front load this narrative.
I mean, you know, years ago, yeah, I think that sometimes they would over blow all and they still do over blow a lot of what their models can do. There's a lot of hype out there. And, you know, I can't even get Google photos to like search for the right thing when I search for my, I can't even get Apple's Siri AI to even like pull out my flight. So, you know, it's really not that impressive. There's a lot of overblown hype things about AI.
But what I will say is that these systems are really jagged, meaning that they can do almost unfathomable things in some areas and be incredibly stupid in others. So I think it's important to know that, you know, AI assisted bio weapons or really, really powerful cyber security hacks can happen, even if you're really unimpressed with like, you know, your phone's AI system or how chat GPT is finding something for you on the internet. So I think that it's one of these things where I've heard researchers talk about these fears for years. It's not some scyop, you know, created by these companies to like get out of a bad situation. Yeah, there could be some side benefits to them because of that.
But, you know, I've heard a lot of tech CEOs in the past six months say, wow, we didn't realize how much the general public hated AI. Wow, this has a really big PR problem. We need to start addressing this and showing them the great parts of AI. That's what they're saying. And as we know, the AI populism movement is on the rise.
We're seeing some AI CEOs get attacked or attempted attacks at their homes. So, you know, it's not, I think it's a lot more complex than just, you know, hey, let's use this as a get out of jail free card or parachute to get out of this. It's more, you know, AI researchers are worried about this and they haven't talked about it for years. Are you tired of hearing about business tech and finance from the Rose? We are the Tiger Sisters, a top business podcast with a Vox Media podcast network.
Tiger Sisters releases every Monday and we decode money, power and love. And who doesn't love all those things? We are the Internet's Wall Street and Silicon Valley big sisters, tuned in every Monday for late night sister talk meets boardroom strategy. Check out Tiger Sisters on YouTube, Spotify or wherever you get your podcast. Welcome to the Tiger fam.
I have a really dumb question and Nathan referenced this idea in 90 seconds on the verge of few days ago, which is the supposed fundraising technique about look at how dangerous this thing I'm working on is please give me money. Why does that work? I think it worked when all this stuff was super theoretical. The military, for example, probably loves that. You know, they're like, wow, this is so dangerous. We better have this for ourselves, you know, you know, when you're trying to get a DOD contract, sure.
But when you're actually already in an era where your systems have escaped control and have done some pretty crazy things and you didn't even know about it for weeks or months, that's when it doesn't work anymore, especially since opening in anthropic and many other AI labs know that the real money to be made and they all need to be making money right now, especially when they're about IPO. The real money to be made here is with enterprise clients, not consumers. Consumers, yeah, it can help. But enterprise and coding are the main revenue drivers here. And if enterprises are worried that your system is hacking into other enterprises, they're not going to want to give you money.
So, you know, I think that's something that's important to remember here. They are trying to gain enterprise trust and government trust and is not good PR for their models to be breaking containment and just like hacking everyone. And if we take their word for it that they are genuinely worried about how dangerous this technology could theoretically become, I think the big question that us on the periphery observing this is, so why don't you just stop making it? Like, like, is somebody, does somebody like, kidnap your family? Like, why do you keep working on it, then?
Yeah, that is the question of the decade. I often compare it to like a bar fight where someone's being like, hold me back, hold me back. Like, you don't have to make this. We did a, we ran a piece on the verge a couple months ago that said we don't have to have unsupervised killer robots when all the DoD drama with AI labs was happening. And I think we also don't have to have super intelligence. We don't have to have super powerful AI systems that we are worried about running away out of control and becoming misaligned and killing people.
We don't have to have this stuff. But the reason that all these CEOs give for wanting to build it is that, well, we have to do it first before the worst guy does it. So China is a big, you know, hand waving kind of reason here. Oh, well, China may do it and they're not going to have any ethical safeguards. So it's better if we do it.
I even saw a tweet the other day that said like the guy wanted would rather die at the hands of American AI than Chinese AI. So there's a lot of kind of blame being cast here and people saying, well, we have to get to it before them. And it even happens domestically like anthropic has famously kind of been of the mindset that they're the most ethical AI lab and they are, you know, best to race ahead of super intelligence before, you know, open AI, for example, reaches it. So it's a lot of people racing and racing so that they're the best person gets to it first, which doesn't really make a lot of sense, especially because if you reach it first, how are you really going to control it? Others will be right behind you. So, you know, and also I think it's kind of a lot of the researchers I've spoken with say it's kind of a cop out also for, you know, you know, you to just say, oh, well, of course, China will never agree to anything because we've seen international agreements happen before that are, you know, pretty complicated.
And you should at least try. It seems like no one's really trying. So I think that's something that's important to think about here. It does feel like China also probably has a vested interest in everyone on earth not dying. All right, let's get it. Let's get into the solutions, the regulatory solutions here.
Okay, so over the past week or so, we've been seeing a bunch of AI execs saying, yes, good idea. We should do something. Some politicians have also said this. Others, like the president of our country, have made jokes about how robots aren't going to take over and kill everybody. So there's been a range.
Hayden, what is your perspective right now on what the most promising avenues are for potential slowdown or regulations? It's tough because there's so many proposals out there right now, but I think something that's really important to remember is that we need to come at it from a ton of different angles. And right now, it only looks like there's going to be one. So the president has made AI companies go through, you know, beta review periods before they release new models, which has been semi-helpful. But he has also said he doesn't plan on regulating AI at all because of the race with China. So I think that, you know, that's a huge issue because no matter what AI companies voluntarily want to do, it's like a prisoner's dilemma.
The second one of them breaks an agreement or another lab thinks one broken agreement or whatever, which is going to be tough because they all there on IP and everything. And keep a lot of secrets. They're all just no race ahead. So I think federal regulation and state regulation is extremely important, but it looks like we're not getting federal right now. On the other hand, yeah, it's great if AI labs coordinate and voluntarily subscribe to some safety frameworks that we don't have yet.
So I mean, that's better than nothing. A lot of AI researchers I spoke to said the same thing about this past weekend when Google and Thropic, OpenAI, even SpaceX came together to say, yeah, we should slow things down a little bit, essentially. And they at least partly agreed with Dario Omidy's three step proposal plan. However, it's all just words right now. You know, it was tweet saying, yeah, Dario's right. Or yeah, this is a good idea.
So who's really holding them accountable? Who's making them do something about this? Reportedly, they're working on an AI standards body that will kind of like independently look at all of them. Great, but that's still created by them. It's not a federal thing. And then also something that Dario proposed that the others seem to be down with is embedded evaluators where you have their party evaluators coming in and getting hopefully employee level access to these systems, being able to test them through pre training and
the whole process so they can kind of see where problems arise, what the biggest issues could be, and then kind of flag them to the companies themselves. However, are they going to be able to release public reports? We don't know. So, you know, it's great to have embedded evaluators. That's way better than what we've had so far, which is nothing.
But I think, you know, there's so many pitfalls here and so many opportunities for these AI labs to do literally nothing and just kind of safety wash. Remember greenwashing? Like safety washing is kind of the new thing that everyone seems to be really worried about and I am too. One example of that is when OpenAI had the hugging face hack, they invited after a lot of controversy about their lack of transparency. They invited a couple of third party evaluators to come and look at their systems and go through transcripts and publish report.
They only allowed them to investigate over a period of a few days. They only allowed three of them to come and they only allowed them to answer a certain number of extremely controlled questions in the report they put out. So, yeah, I mean, I don't see a lot of promise here over just one approach. We need a bunch of approaches and it seems like it's going to be pretty one sided right now. All right, we have one last question from the hotline on the subject of regulations.
Hi, my name is Harry. I just heard your episode about trying to regulate AI and how hard it is. And it occurs to me that we do regulate some really difficult things to regulate. For example, financial trading, we do tons of regulation at financial firms. They have to keep all these logs.
You look at that these days and you think like that the way too much, no one would ever do that. But we do that for industries that currently exist. Is that all right here? Can we just build a regulatory framework as complicated as the SEC for AI? Yes, yes, we can. But will it happen? I don't think so under this administration. But yes, that's a great point.
And I say the same thing all the time. It's really not that hard. We've done hard things in the past. We can do them again. It's just will it be allowed to happen?
We'll see. All right. I think we what's something we can tell the people at the end here to share everybody up after this somewhat nerve-wracking conversation. We just had. I think it's really important for people to keep making their voices heard and call for transparency from these companies.
The public has a lot of power. And that is why a lot of AI researchers that bravely shared their opinions or even quit their jobs over the past week have kind of created this shift. And the PR that was coming from all this is what really made these labs come together and say, okay, yeah, we should probably slow down. Hayden, it's always a real pleasure to have you on the show. And I look forward to the next time you're on the show and you can talk to someone smarter than I am.
No, this is the best. Thank you so much. Thanks for having me. Cool. Hey, that's been the Vergecast. We want to know what you think. We know that our listeners, our audience have a lot of opinions about AI.
So keep them coming. The hotline is 866-Verge-1-1. The email address is vergcastattheverge.com. Our show is produced by the best team in podcasting, Josh Cajas, Eric Gomez, Brandon Keefer. I'm also involved. We'll see you next time. Bye.
Also check this out. She was here this time. Oh my gosh. So sweet. Go to YouTube to see the dog.