AI Is Going to Kill Us. Please Regulate Us.

[MARKER: THE REAL DANGER] You’ve probably heard claims that there’s a 10% chance AI could kill all humans within the next decade. That’s a researcher’s subjective risk estimate — not a measured scientific probability. There’s no little meteorologist standing beside the green screen saying: “Tonight, expect scattered superintelligence with a 30% chance of extinction.” But AI systems are increasingly capable of coding, research, planning and interacting with external systems. The machine is now capable of helping build the machine. That’s the interesting part. Not necessarily killer robots marching down Main Street. Acceleration. Faster research.Faster coding.Faster experimentation.Faster weapons development.Faster cyberattacks.Faster biological research.Faster propaganda. Faster everything. And human beings are already having enough trouble keeping up with email. [MARKER: THE QUESTION] So here’s the question nobody wants on the bumper sticker: What happens when technological capability starts increasing faster than human institutions can adapt? That’s the real AI argument. Not: “Will the robot have red eyes?” But: “Can civilization maintain control over a technology that improves faster than civilization can regulate it?” And here’s another: Who gets to decide when the machine has become too powerful? Because if the answer is the companies building it, congratulations. You’ve just invented the world’s largest self-licking ice-cream cone. The people who control the technology get to determine how dangerous it is, what safety standards apply, who gets access, and which regulations the government adopts. [MARKER: THE OTHER SIDE] But suppose they’re right. Suppose the danger really is enormous. Then the regulatory-capture argument doesn’t make the danger disappear. It means you’ve got a second problem: The people warning you about the fire may also be standing next to the only fire-extinguisher factory. AI doesn’t need to hate you. It doesn’t need to conquer America. It doesn’t need to develop a little digital erection and decide it wants to rule the planet. It merely needs a goal. A very powerful system pursuing a badly specified goal can produce consequences nobody intended. And that’s a much more boring apocalypse. Which means it’s probably the one worth worrying about. [MARKER: THE QUESTION THAT MAKES EVERYBODY SQUIRM] So when somebody tells you: “AI is going to kill us.” Don’t immediately laugh. And don’t immediately believe them. Ask: “What happens to your business if I believe you?” Because if the answer is: Government regulation becomes mandatory. Smaller competitors disappear. Open-source development gets squeezed. The companies already at the top become even more indispensable. Well... We’ve finally built a machine capable of doing exactly what humans have always done: Make a fortune, scare the hell out of everybody, and then ask the government to protect us from the competition. That’s capitalism. [DING.] That’s regulation. [DING.] That’s artificial intelligence. [DING.] And somewhere in Silicon Valley, a billion-dollar computer is staring into the future. Probably wondering why Mildred is still eating the dog kibble. Text or leave a voice message: 310-737-TALK This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit caryharrison.substack.com/subscribe