Don't Overregulate AI—Supra-Regulate the Frontier

Hi and welcome to Brandon’s newsletter We are on episode 10 Continuing our conversation On AI data centers and now we have veered off into a main highway That would be the controversy concerning AI itself Last time we were together we were discussing The we examine the growing tension between accelerating AI development And slowing it down for safety The dilemma is more complicated than simply choosing speed or caution If the United States slows while strategic competitors continue advancing It could surrender important economic scientific cyber and military advantages Yet accelerating without adequate safeguards That that could produce capabilities we are unprepared to control At the same time Poorly designed regulation could unintentionally protect today’s largest AI companies By imposing compliance costs that smaller more efficient competitors cannot afford A classic problem of regulatory capture it’s that Moat we were talking about last time The central question then is not simply whether we should accelerate or decelerate AI But how can we restrain genuinely dangerous capabilities while preserving innovation competition and international security And without concentrating control over this transformative technology in only a few hands Or a few companies in this case so this time I’d I’d like to go into something that’s scares people on a certain wing of politics : regulation Sometimes you need rules We need rules if we play sports I mean it would be Bedlam if we didn’t And it would be unexciting I’m sure we’ve played against people who have bent the rules or have changed rules in the middle of the game and we don’t appreciate those people very much And we don’t stay friends for very long because of that So what I’d like to go into now is what what I’ve termed supra regulate So the title of the episode is going to be,”Don’t Overregulate AI, Supra Regulate the Frontier” Maybe we’re asking we were asking the wrong question about artificial intelligence We’ve been arguing about whether AI should be regulated more or regulated less but perhaps neither is quite right Perhaps we don’t need to overregulate artificial intelligence Perhaps we need to supra regulate the frontier And by that I mean something very specific Most AI should remain open to experimentation, competition and innovation Let the startups compete with a trillion dollar company Let researchers experiment Let somebody discover a model that produces better intelligence using one tenth the compute Let somebody slash the price of a token Let open research challenge proprietary systems Let the race continue But when an AI system crosses certain capability thresholds as we’ve uh seen lately Um when we’re no longer simply talking about a better chatbot But about capabilities that could create extraordinary consequences for cybersecurity Biological research autonomous weapons Critical infrastructure Or increasingly autonomous AI itself Then perhaps that system enters a different category altogether That’s what I mean by supra regulation Not super regulation supra not more regulation everywhere A higher layer of governance that activates when an AI system crosses uh clearly defined thresholds Of extraordinary capability and potential consequence And here’s the important part Not because of who built it But but because of what it can do That distinction matters Because um regulating companies creates an enormous danger of regulatory capture Regulating capabilities gives us another possibility A 20-person startup producing an efficient but ordinary AI system shouldn’t need the compliance department of a trillion dollar company but if that same 20-person company somehow produces a system capable of autonomously discovering catastrophic cyber vulnerabilities the fact that they’re small shouldn’t exempt that capability from scrutiny either now we we’ve seen that type of situation unfold if you’ve read into the COVID controversy and the origins of COVID you will discover through uh receipts you will discover that the United States government actually was funding gain of gain of function research so there was a back door in our government through some I don’t know if I should say overzealous you know people I’m being very kind I don’t think that they were overzealous I think they were over evil so you know we compartmentalize a lot of secret programs in a government are compartmentalized And I I believe that AI is most likely we don’t know it’s a mystery but I’m pretty sure there’s sections of the government that are a little compartments that have discovered things that would blow our minds and that are dangerous So that that distinction I was talking about is very important It doesn’t matter how small the group is what I’m trying to say it could be a lab of of a couple dozen people but what they’re working on is mad scientist stuff So we don’t need more Frankensteins So what I what I’m proposing is that we have a threshold the the threshold follows the danger Not the size of the corporation And something President Xi Jinping said this week during his visit with President Trump provides an interesting way of thinking about this Speaking about competition between China and the US he described healthy competition as a race of catching up with one another rather than a wrestling match where one side has to defeat the other Now I don’t know what China’s ultimate intentions are I don’t know what America’s intentions will be and diplomatic language is not the same thing as a verifiable behavior we’ve talked about that before trust but verify Again trust but verify But I think that metaphor itself is is useful in this way Imagine uh two runners one pulls ahead The other discovers something better and catches him Then the first innovates again Back and forth Each competitor pushing the other Competition doesn’t have to disappear in fact competition may be precisely what keeps AI from becoming permanently controlled by handful of enormous institutions So perhaps the objective shouldn’t be to stop the race Maybe the objective should be to put guardrails around the uh racetrack You can run, you can innovate, you can overtake me And you can build more efficient model You can dramatically reduce inference costs You can invent an architecture nobody has considered before But there are certain things neither runner gets to do Neither gets to leave the track and endanger the spectators Neither gets to secretly remove the guardrails And neither gets to write the rules in a way that prevents another runner from ever catching up That’s the distinction I’m interested in competition inside boundaries Extraordinary scrutiny at the frontier And perhaps international coordination around the handful of capabilities whose consequences could extend beyond national borders This is what I mean by supra regulation Not a giant regulatory blanket thrown across artificial intelligence A higher layer of governance reserved for extraordinary capabilities And importantly that higher layer cannot simply be written by the companies currently leading the race Otherwise We’ve solved one safety problem by creating another problem Regulatory capture Nor can this simply depend upon promises between governments Because we’ve already discussed the international dilemma What happens if America slows down and another country doesn’t Any meaningful international framework therefore eventually needs something more than good intentions Verification independent testing Clearly defined thresholds Incident notification I think “red phone” Some degree of visibility into enormous concentrations of compute associated with the most advanced training runs and consequences when agreed boundaries are deliberately crossed Interestingly one small piece of something that architecture of that resembling it actually is already being discussed The US has proposed an AI incident notification mechanism with China for serious artificial intelligence intelligence events involving national security concerns Now let’s be careful here This is not an AI arms control treaty It’s not even an in an established international AI regulatory system It is a proposal being discussed between the world’s two leading AI powers But perhaps that’s how something larger begins Not everyone stop And not everyone races as fast as possible Instead keep competing But begin building mechanisms that prevent competition from becoming catastrophe Become Or because perhaps the choice we’ve been given between between acceleration and deceleration is it itself too simplistic We may need to accelerate some things I can think of some efficiency, medical discovery, scientific research, energy technology, education, smaller models, cheaper inference, better reasoning, and simultaneously we need to place much stronger scrutiny around around other capabilities, autonomous cyber operations, biological design capabilities, critical infrastructure access, think shutting down power plants, nuke plants, electrical electrical grids it it this one is actually pretty scary autonomous weapons who you know all I can think about is Terminator with that one systems capable of substantially advancing AI research without meaningful human supervision In other words Don’t regulate intelligence simply because it is becoming more intelligent Regulate the capabilities whose failure or misuse could impose extraordinary consequences on people who never consented to the risk That creates a very different AI future Not monopoly, not anarchy, not a global pause and not a blind race toward whatever comes next Instead, imagine a competitive ecosystem surrounded by increasingly strong guard rails as capabilities become increasingly consequential Ordinary innovation remains broadly open Frontier capabilities receive extraordinary scrutiny and capabilities capable of producing consequences across national borders become the subject of international coordination and verification Perhaps Chairman Xi’s racing metaphor is useful after all Let America run let China run let the startups run let today’s leaders be overtaken if somebody builds something better Competition can make everyone better but the more powerful the runners become the more important the rules of the race become, too Don’t stop the race build a safer track And that is what we need to think about we should have thought about this yesterday so I’m trying to stay ahead of the game. It’s very difficult everything’s moving uh it’s like trying to, you know, drink water from a fire hydrant there’s so much coming at you so what I want to discuss next time I believe that we should go into obviously we’re going to go into AI So what I’d what I’d like to discuss is the actual like warp and woof of AI infrastructure, the physical infrastructure that is. So I really appreciate everyone who’s listening I am not trying to be boring. It’s just all of this is so very interesting and there’s so much there’s so much to learn and I would like to learn together. Thank you for listening to Brendan’s newsletter. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit brandonfreywald.substack.com/subscribe