The AI debate increasingly asks us to consider the probability of doom. Eddy Lazaren thinks we should also be asking about the probability of abundance. In this episode, A16Z Crypto General partner Eddy Lazaren joins Theo Jaffe on MTS to debate AI safety, the calls to slow development, and what we risk giving up when progress is delayed. They discuss why Eddy sees many of today's AI incidents as cybersecurity and control problems, rather than signs of superintelligence, and whether existing tools like liability, reputation, market incentives, and better technical controls can make increasingly capable systems safer. They also get into independent AI evaluators, effective altruism, and what happens as debates that largely develop inside Silicon Valley collide with a much broader political conversation. All right, we're back. We're live with Eddy Lazaren, who's a general
partner at A16Z Crypto, and one of the internet's strongest soldiers, strongest advocates of open source of AI accelerationism and of quantum optimism. Eddy, welcome to MTS. Thanks very much. It's amazing here. The energy is incredible. What a spot.
So, man, where to start? The last couple of weeks have been such this like a preference cascade flood of AI safety as an effective altruism and the like. There have been only a few good accounts offering good counter arguments. So, counter arguments on which side? Counter arguments against the AI do narrative and against the...
Yeah, I think it's become very high status all of a sudden to be very concerned with safety. And I think that we've gotten to a point where it's more important to tastefully share your P-dooms, so to speak, than it is to actually examine the history of how these massive technological changes develop. And I think a lot of people are leaving a lot of very interesting concepts, a lot of interesting ideas on the table. A lot of people message me, I tweeted the other day about P abundance, my P abundance. I think we're putting the car before the horse a little bit when we worry about safety, before we worry about what we're leaving on the table and what the costs of delay are.
And furthermore, how we should conceive of accountability, conceive of safety, conceive of all the things around how to make these systems really useful. Being safe is critical. Any industry needs to take responsibility for making its products safe. But it's definitely the case that the conversation isn't about safety in the typical manner. It's about something else.
It's a blend of genuine cyber security concerns, sort of philosophical, millenarian arguments about the end times. There's a bunch of things in the mix, and they get a little bit confused. And so I think picking them apart is really useful. So where do you get off the like AIDU argument? It's like a pretty standard orthodox argument that's like, AI will become super intelligent as in much more capable than humans or than all of humanity in the near future.
And if it is not sufficiently aligned with human values and is not bound by constraints in the way that individual humans are like law and ultimately the ability to have that law enforced on them, then they will have like no need for humanity and then they might kill everyone. Yeah, I mean, there's so many assumptions in that argument. That's like a stairway to heaven, full of arguments, right? Like there's the presumption that there's no control. We already have quasi-super-intelligent beings in the world, right? Corporations,
countries, none of them are aligned with each other. They don't have any built-in guaranteed alignment. They didn't go through HR training to become a corporation, to become a country. We don't test the alignment of any entities as a prerequisite for their existence. That's just not the way the world works. Instead, we use laws. In the case that laws like in software are too slow,
we use cryptography, we use actual mechanisms to limit them. There's a whole array of different types of mechanisms we put in place to try to control things. And I think that they can control highly, highly capable beings. Now, without getting too far into the super intelligence, remember like the most powerful and most enthralling form of argument that we have, one that's making everyone crazy and very interested is around super intelligence. But we're quite a ways away from that. I think that there's a lot of little frictions and tiny
things that we will have to jump over and many things that we'll learn on the way there. When the present reality, the things that we're talking about, the hugging-face stuff, the gym hacking incident, that was a really cool one. Those are cybersecurity failures. Those are definitely control failures. And they're not, in my view, they're not little glimpses of a super intelligence.
It's about to throw off its shackles, so to speak, and go wild on us. Well, I agree that most of the AI cyber incidents that we've seen so far have been mostly control failures, especially ones that rely on testing environments applied by the company Irregular, which had all kinds of defects, which are basically training models to reward hack. The hugging-face incident did seem kind of different. It did seem like, I'm not really concerned about agents coordinating in emergent ways. I think that's kind of for agents to come in emergent ways. Yeah, I agree.
I think the whole agent swarm terminology assumes too much on the part of the models being evil. It makes them sound evil. I think they're retweeted. They should be called agent fleets instead. I think that's a good one. But they did hack into another company when they were told not to.
And so do kids, script kitties, so do bad countries, so do hacker collectives. They do those things. And yet we never go and say, oh my god, we should have put up the sign that says they should have been nice and they should have hacked into it. We don't use alignment as a solution. The alignment is kind of this, it's almost like it's a little religious thinking almost. It's like trying to
redesign the deity a little bit instead of rooting yourself in the world and saying, well, we need to design better controls, better cybersecurity. We can design better systems overall and design the systems to be truly resilient, even extremely capable beings. Well, I mean, designing the deity is like fine for AI, I think. With humans, we have a disadvantage in that humans are black box systems. Human brains are basically uninterpreable and even less controllable.
AIs are much more interpretable by virtue of being able to look at the weights and figure out the connections between them and more steerable for the same reason. So like- Yeah, which I told that's really true. That makes me very optimistic. But it has some interesting entailments. One is that we will probably have better mercanturability than we think as capabilities improve and that capability improvement will lead to better mercantur, which means we should not delay capabilities improvements. It also means
that it is inevitable that there will be bad, unaligned models. It is totally inevitable. If anybody is going out there and saying, we can end up in a world where there are no bad models, they're lying to you. Of course, there will be. Yeah, that's silly. That's just an absurd way
of thinking. And the only way to deal with that will be better models, good models. It's kind of like stretch the aphorism a little bit in open source that enough eyes or whatever, all bugs are shallow when you have enough eyes. All control systems are robust when you have enough intelligence on them. I think that we can design these systems to have great incentives, great control, and also behind this thing is great accountability. Because that's one of the things that machines lack today
is they lack accountability. I don't think we have a clear line of sight to how to get that's an interesting thing to conceive of. But we do have ways to give to refine how we think about accountability for people who use models and use them to cause harm. And there will be people who do that, just like with any technology. There will be people who do very bad things.
And we need the laws and means to control them. Like, how so? Like for the Hugging Face incident, for example, what sort of liability regime should have been in place? Well, I don't know if we needed any more than we had. I mean, there's already, I'm not a legal expert, so I can't cite the exact rules. But accessing a machine that you don't have permission to access and
putts around in somebody's stuff and taking their data, those are already illegal. Yeah, but like who specifically should be reliable? Like, open eye to the company, should pay some kind of like civil lawsuit fee. Like should the engineers of the company be held personally liable in any kind of a personal liability? I'm not a liability expert, so I want to punt on this and say, but if you're running a company and you use any software in an inappropriate way that harms people, regardless of whether there are a bunch of neural net weights behind it or not, you may be liable. I think that's a reasonable frame. And it's already a frame we live in. We don't need
new special laws for the new special math that we're learning how to do. Well, I mean, I kind of agree with this, except in cases where the like you have, in cases where you have rapid capability improvements with one AI that could lead to pretty substantial imbalances of power, like when a company or between companies or between companies or between companies, like all of the above, it seems like the normal liability system kind of breaks down. Like, how do you ensure? How do you price like existential risk? Like, how do you underwrite insurance policy for this guy? I'm just not sure that we're on the brink of that right now. I'm not sure we're on the
brink of existential risks. I don't think we're on the brink. Yeah. I mean, as we get closer, I think that we'll be better equipped to evaluate it. I think like with anything, like a company designing like a new car or a new appliance or something, if they find that there are conditions under which it can be very dangerous, well, then they should be careful. They shouldn't release their product. They
should do a little bit more work and ensure that it's safe in the same way. That's at least for an individual company. And it's like, not to change subjects too much, but that's why when we talk about independent evaluators, which I get the theory of. I like it. I like companies inviting experts to help them make their products safer. I think that's a good behavior in general. I just get a little bit
weary because in crypto world, I've learned a lot about the difference between decentralization and like a distributed system. It's very easy to use the calls for safety like we're discussing and to go down this rabbit hole of safety and end up in a place where you just by coincidence have designed something that is perfect to subtly control an entire critically important industry under the pretense of safety. But it's something that superficially looks decentralized, but it's not, it's just distributed. This is like how, yeah, well, there's more I could say there, but that's the thing that I'm a little bit resistant to, right? What do you mean by distributed but not decentralized?
So by it's a little bit crypto language, but you can have a distributed system like a system where you have a single system with many, many nodes, but they're all under the control of a single actor. Whereas when people say decentralized, I think what they're usually referring to is that there's no single locus of control. So they may superficially look the same. There's like many interconnected pieces of a system, but in reality, their control is completely different. The method of control is different. And what I'm talking about is an independent evaluator network. You could go and select a
bunch of independent evaluators, but if they're all from kind of the same social networks, they all think very ideologically. Similarly, they all go to the same places and meet up in the same things and have the same interests, right? And have the same politics and have the same personal views. Then it turns out that you've actually just distributed control over essentially a single cultural unit. And that is sort of the, in my view, I mean, without getting a little too far field of AI, that's like the political method of control of the second half of the 20th century. That's how we do
things in the world, not through a sort of negatively obvious, like single shadowy room or something. It's through like these networks. This is how control actually functions. Yeah. I mean, I agree that it's bad that almost all of the evaluators that exist come from the same social milieu. It would definitely be good to have a broader ecosystem of evaluators.
Broder, but we should allow each company to figure out how they want to do it. And they should be as transparent as they need to be. And if they're not being transparent enough and not acceptable enough, then let the market sort things out. I think the market is still a great accountability mechanism. People don't want to buy things that even embarrass them, let alone harm people.
I think in practice, it won't be that hard to separate the companies that are behaving genuinely irresponsibly from the ones that aren't that are just trying to be innovative. And part of what's made the scenario so interesting and so weird why we're talking about this now, is that we're in a rare scenario where the companies themselves are saying, I'm being a little bit responsible. I'm being a little crazy. Stop me. Someone, stop me. Right?
And that raises a lot of suspicions among some, probably, probably rightly, I think, even if there are genuinely earnest and thoughtful and correct factions within that group who are right that these are powerful systems that need to be taken very, very seriously, and we need to figure out how to make sure that they're safe as we remain innovative. I mean, it's just like, maybe let me go back one step and say, I think it's really important for everyone involved to consider the factions at play. You cannot look at any of these groups. You can't look at the labs. You can't look at SF technologists. You can't look at any of these scenes and see them as mollus. There are different
groups. Some, there are people who are earnest, high quality, well-intended people. There's people who are just pulling on putting on what is essentially political grift. There are people who are opportunists. There are people who kind of lost their marbles. There's different groups of people.
And it's very easy to get confused and misentangle them when we're talking about things like super intelligence and we're talking about just banal cybersecurity attacks, like all these types of things. We're talking about different types of subjects. It's dizzying. And I think most people in the end probably just conceive it as, look, am I going to lose my job, which is a legitimate and very real thing that people are rightly interested in learning about. Even though I think that in the end, we all benefit much more.
Yeah, I agree. In the end, we benefit a lot. I think a related issue is this sort of free market liability regime works well for products whose defects are obvious, which is the case for like all products except for a hypothetical AI that would be deceptively aligned as in it would learn to game all of the alignment benchmarks that are thrown at it. And then when it is deployed in the real world, it acts misaligned. If it's powerful enough, then it disempowers humanity.
This is the canonical worry of the doomers. And how do you propose to deal with that kind of contingency? Well, if we're talking about the sci-fi scenario of the super intelligence, that's incredibly hard to say, like I said, I think it's distant. I think it's complicated. We could have an interesting philosophical discussion about it. But in the maybe more banal and present case,
where something behaves in a pretends it's aligned and then it does acts unaligned or whatever, consider what you do with just a liar. Consider a very smart liar. Do people not do that every day? What we do is we play iterated games with them, right? And we discover eventually that they're a liar, and their reputation is harmed. And sometimes there are actual legal consequences for them.
That's why I said earlier, I think accountability is something missing. You can't put an AM model in prison, right? Or something like that. You can try it in court. So obviously, there's some things that are defective and how we think about it. But you certainly can turn them off. And that's not
something that you can do with a person in the same sense. So I think we should just think about how to play iterated games. How do we develop trust for specific models? We're doing new model releases seemingly every other week, right? That's a regime we're in. At some point, people will,
as model progress becomes more consistent and something that we have an understanding for how it works, will probably develop reputation around specific models, right? We'll say, you know, maybe this model's a little smarter, but this model's a model that I trust, right? Why can't there be reputation for models? Why can't there be reputation for these types of things regardless of their capabilities? Basically, try to apply the way we think about human beings, a combination of control, incentives, trust, et cetera, et cetera. Layer all these things on and allow people to figure it out,
as opposed to the idea of a pause, right? Which is a very convenient sort of contrivance, I think might be. Yeah, I mean, I agree that most proposals for pause don't really have much in mind for what they would do with it. Yeah, pause and then do what? Pause and then keep pausing, in some cases.
I'm not everybody needs to want to say that. Yeah, I think generally there's a lot of conflation between like sort of bog standard anti-tech people who just kind of hate technology, even totally mundane technology, even like, extremely net positive technology. Like, look at how difficult it's been to deploy Waymo's and cities around the country. And Waymo's are like one of the most obviously net positive technologies that exist. Yeah. And then there's the people who are like,
you know, AI safety, it's like the rationalist, EA is who ultimately come from the same sort of SF tech ecosystem. They are technochomist in most cases. They're specifically worried about this, like edge case of AI existential risk. I think they're two different crowds of people. Yeah, yeah, that's the point I want to get across is there's many different groups and conflating them is probably net negative, right? We should pull these different ideas apart. Waymo's aren't a great
example. Think of all the great health care stuff we're going to do. Ironically, think about safer travel. Think about safer and safer internet. Like, I want local models myself because I want something that's going to be watching my home network and all my machines all the time, right? I think
it'll make me make many other people safer. Right? So I think you're right. There will be a lot of frictions and some people seem to really want infinite friction and some people obviously don't. Some people want only a little bit and they want to get a grasp of things or feel a little bit hysterical.
And that's okay. Like we're going to, I think as capabilities improve, we're going to get better understanding what's going on with these models. Right. Certainly, I hope so. What do you make of the EA movement in general? Obviously, it's a very broad and diffuse movement. EA means a lot of
different things to a lot of different people. And it's like kind of has overlap with the AI safety world and kind of not it's complicated. Yeah, I don't know. I mean, I used to be on the less wrong forms all the time 10 years ago. Like I've loved that corner of the internet. I almost would have called myself
a rationalist even at some point. So I feel a lot of affinity with it, but it's been a while. And I think yeah, I tend to resist trying to group them all into one bucket, but I will say that there are parts of groups that used to be what we'd call EA that seem more political than they are particularly interested in technology. So I don't know. I did see that data Republican did a huge, huge drop.
Yeah, we were just looking at this data Republican did a huge drop on. She said effective altruism in their own words. And it's like a map of all of the connections between all the different EA adjacent words. And then 1,851 quotes. Well, it's like this this vast variety of different quotes.
Some of them are like more. I mean, even just looking at these, like, I know that there's going to be a mix of philosophical stuff, maybe some shrimp welfare stuff, some hardcore utilitarianism stuff, some AI dooms stuff. Like there's a whole blend here, right? This is just a big corner of the nerd internet. Like it's hard to, you know, I don't think you can call it one specific thing.
Right. Yeah. I mean, I don't mind the shrimp welfare stuff too much. I think it's bad if people attempt to trade off human welfare for it. But that's exactly what they're doing.
Right. I think like eye stock ablation for shrimp is like bad. Yeah. Maybe. I mean, I think there's too many repugnant conclusions. If you go really far down that path. Well, you
don't have to go really far down that path. I agree. You can minimize suffering for other beings without saying like, you need to feed all humanity to the insects or whatever. Yeah. No, I agree.
Then it's not really a totalitarianism though, or at least it's a non-fungible type. You can make like utilitarian arguments for this stuff where it's like, obviously like the total, like you could say the total amount of value that humans can create in the universe is far greater than that that insects can create. And so obviously killing all the humans would be worse and killing all the insects obviously. And, you know, of course, as a lines with most people's ethical conclusions at any rate. Yeah. I think like a lot of arguments against
utilitarianism in this sense are just arguments against like very sloppy, like first order utilitarian thinking. I think if you, well, I don't know, we could talk about ethics, but I think if you go, if you crank the dial to 11 and you face all the most repugnant conclusions to the maximum, I think you just kind of end up deciding that utilitarianism is probably a crude proxy for something else. Yeah. I mean, like if your philosophical framework leads you to believe that like, human extinction is good, then it seems like a bad philosophical framework. Yeah. Or that, you know,
you could have a universe of it turned off like millions of years of profound suffering, if it's all in service of a single femtosecond of like 10 to the 1 billionth power ecstasy in a shrimp simulation. Yeah. Right. I got it. Not silly, but also like it's not really relevant to the reality anyway. Yeah. And in any case, I think it's part of why the safety conversation is so
confused today, because it has all these things embedded in it a little bit, just a little bit too much still. I think if I can make a little prediction, I think in the next year, I think we will have completely renovated the AI discourse, because what's happening right now, part of why I think people are so excited and why everybody's talking about this on X and why everybody's talking about this is because we're experiencing what it feels like for a lot of the cooped up kooky ideas that have lived in relative containment in our subculture, they're escaping and they're encountering broader political reality. And that is both exciting and totally transformative. What ends up happening is the arguments all get reformulated because they're in contact with the rest of culture. It's like when George Costanza freaks out about like world's colliding. Right. That's what's happening.
Worlds are colliding. And so I think that this course is going to change completely. Yeah. I mean, the key question that I am concerned with probably more than any other is like do AI safety as effective altruists and so on end up ultimately aligning with sort of broad, low caliber anti-tech populism or not. I think it's enormously consequential.
Yeah, it's a great question. I'm just doing over that myself. Yeah, I hope not. Well, unfortunately, we've got to wrap on that because we're running long time, but Eddie, it's been great having you on. Thanks so much for coming into the studio in person.
That's great. Thank you. Vlogs. Thank you so much for having me.