Welcome to Concerning AI, this is Ted. And this is Brandon, and today we're talking about existential risks and why we're doing the show. Yeah. Whoo, what is an existential risk?
Well, I think it maybe helps to start out by considering what is an existential threat for an individual human organism. You mean like somebody shoots me, can I die? Yes, so that would be a terminal incident. There's no future after that, so that would count as an existential threat.
Yeah, something that might be endurable is, you know, you run your motorcycle and you crash and you damage your leg and it gets taken off. You know, you've got your legs gone and you've got chronic pain for the rest of your life. That's still endurable. It'd be horrible, but let's go on.
That, if it itself is not a terminal event, although writing in my motorcycle is fraught with existential risk for the individual human. OK, good. Well, what about more than just the individual human? What about a larger group?
Well, so that sort of defines one axis, right? The axis of how survivable is an terminal or is it endurable? We're getting this from Nick Bostrom's paper. He's got a two-dimensional figure here that has two axes.
One is that first one that we're talking about, endurable to terminal. OK, and then you're bringing up the second axis. Yeah. So if I go bigger than just me into my local group, whatever size local is, it could be a family, it could be a nation.
But it's not everybody. It's just part of humanity. Yeah, it's hard to think about the intermediate. I mean, I guess some cultures go extinct.
You know, we can see that happening to, well, yeah, I mean, if the genocide were complete, that would be a terminal threat for that group of people. Yes, for that culture. Whether that could be language or certainly languages have gone extinct, yeah, or it could be an ethnicity. However you define that.
Yeah, hard to think about. But species is something that we're dealing with all the time. Species are going extinct. Yes.
Not our species, thank goodness. Yeah. So a species going extinct is indorable for, if we think about all life, if we think about DNA-based life forms, losing one species is not the end of the world. Right.
It's terrible to think about that. I hate to even say that. Well, it's a little bit like saying losing an individual human life. Right.
It's not the end of humanity or anything, right? That's like it's your life. It's sort of the end. That's right.
And if it's your culture that's lost or your species that's lost, that's kind of the end. It is the end for you. So something that's indorable though on that local level would be something like a recession in a particular country or a depression, something that you'll think about how bad it could be economically mass poverty for a particular country or, yeah, or an earthquake or some sort of natural disaster that temporarily shuts down, yeah, shuts everything down. You could even think about things as bad as a nuclear war.
Could be in this category. Possibly, yes. Of indorable but global. That would be a or you are you saying that that's a yeah.
Yeah. So it could be either way, right? Like so a major city gets nuked, even a few, but it doesn't end up turning everything to glowing radiation and nuclear. So we'll get into how bad a nuclear incident would have to be in order for it to be this third level, which is global, global and thermal.
So what's global and durable first? I think I would put climate change into that category. It may be excruciating, but I think it's in the indorable category in most scenarios. Unless we turn into Venus or something like that, it's kind of on the fringe of what people think it's going.
And how do we get to Venus? We get to Venus by the greenhouse effect going. By a runaway greenhouse. Runaway.
We can't stop it. We can't stop it. It takes off. And it's self-reinforcing.
It just gets worse and worse. It's a positive feedback loop of, yeah. OK. So other than that scenario, that scenario does appear to be terminal four, humanity.
Right. What we're most likely looking at is a lot of misery and suffering, especially around coastal communities and quite a few species extinctions and loss of diversity. So it's bad. It's bad in the same way that the major extinctions in the past, the major five or six extinctions in all of life's history are bad, bad, bad, but not terminal.
Right. OK. So what are we talking about if we're talking about global and terminal? Well, I think it's useful to think about two different cases when we talk about global.
And that is the extinction of humanity and the extinction of DNA-based life. There's two different things that would take kind of a different scale of event. So something that is terminal for the entire human species, but not terminal for DNA-based life form. Yeah.
There's not very many of the nuclear scenarios, I think, that would get rid of all the microbes and bacteria and stuff that live deep in the earth and down in the oceans and stuff. Right. So if life goes on, then presumably, evolution would continue to do its thing. And some examples are like the early snowball earth scenarios where all life on the surface of the earth was dead.
But the life that started on the surface and then moved into the soil carried things on for thousands of years. So that type of thing could happen again. Yeah. That sounds terrible.
This happens. No, it happened like any scenario in which humanity goes extinct. Really. But an example of this, it's harder to come up with examples of DNA-based life form, all DNA-based life gone.
An example that I think about more than I wish I did is the Dark Forest. OK. What's the Dark Forest? The Dark Forest is an answer to Jeremy's paradox of why aren't there more people, why aren't there more noise out there from other intelligent civilizations?
OK. Because it's the kind of where here and then there should be more out there. And so the answer is one I got from a Chinese, I can't remember his name, science fiction author. And his answer is that there's not enough to go around.
So there's the threat of somebody coming and taking your stuff in the global universe or the galaxy. So there's at least the risk and the perceived scarcity. And it's too far to communicate. So anytime you become aware of another intelligent species through their unwisely broadcast TV or whatever it is, anytime you become aware of that, you have to make a decision.
You have to make a decision because it could be 50 or 100 years or even 1,000 years that that signal takes to get to you. So your window of opportunity to strike decisively is very, very short. And they're going to be thinking the same thing about you. So this is like this back and forth chain of suspicion is what he calls it, where you don't know what they're thinking and they don't know what you're thinking.
And so what it always ends up being is you better strike first. And so everybody who survives in the Dark Forest idea is quiet. So you're creeping around quietly through the forest. And I worry that we've been a little noisy.
And that hasn't propagated very far through our galaxy yet. What it is propagating. And it's a square law how far out our noise is reached. So as soon as that noise reaches one of these other intelligence species, then they're going to come out first.
And not just any intelligent species. It just has to be one of the ones that has very much come down the side of, you know, strike first. Well, that could be term of the goal of our son or something. Right, sort of crazy thing.
And the problem of not striking first is if you're wrong about the intention of the other. Yeah, so that is an existential threat for both sides. So to know about is to have an existential threat you have to deal with. Because of the intelligence explosion, you look at what's happened with humanity just in the last 100 years.
Well, I call it 120 years. We've gone from being ground-based to having a presence in space in 120 years. You know, you think about 500 years from now. That's got to be very threatening to anybody.
Right. For sure. And we encounter a similar situation. So the dark forest is, I think, a potential DNA-based life form could end threat.
That I, for some reason, is very palpably real to me after reading those books. Yeah. Well, I read a lot about this idea that we are living in a simulation. Or the probability that it seemed, but some of the stuff I read seemed likely that we are living in a simulation.
And give us the skinny on that argument. OK, so the idea is that intelligence develops. Yeah. We believe that intelligence develops.
We've seen that happen here at least once. And if intelligence develops, eventually, intelligence is going to be, because of the intelligence and the explosion, eventually we're going to be able to simulate other intelligence. Maybe ourselves. Let's say that future humanity, let's say we're not doing a simulation for an easy part.
I'm talking about this. So future humanity, our descendants, have the ability to simulate the past. And they want to simulate maybe going all the way back to the big bang. They want to simulate physics.
They want to simulate the universe. And so there are going to be individuals that are living in those simulations. They're going to simulate individual minds. And those individual minds, depending on what the fidelity is of your simulation, they could be conscious.
Right. Or at least appear to be conscious. We talked about consciousness. And we like all of that as a definition.
For the purpose of this podcast. So the idea is, if we believe that intelligence does explode, then why would we think that those intelligencees are not going to simulate their own past? Or simulate alternative intelligencees? Right.
And then you've got this turtle all the way down. Because once those intelligences get sophisticated enough, then they'll want to simulate. That's right. And so the math gets big very fast.
And if there are an unimaginable huge number of simulations out there, why wouldn't we be living in one? It seems likely that we would. So the only way that we are not in a simulation is if we're first. Or if intelligent life forms cannot, for some fundamental reason, or some inclination, absolutely never do simulate.
Right. And that seems unlikely. Yeah. I mean, the part that seems realistic to me, possibly, is that you might need the universe to simulate the universe.
And there's only one. Yeah. And that might be the way we cut through this gordionon. And this one's not a big deal.
And yet, no. Yeah. I think about this one a fair amount too. Yeah.
And so the existential threat is not that we're living in a simulation. The existential threat is that they turn off. Right. But when I think about that one, I cannot think of any action that I could possibly take.
At all, that would make any difference, whether they're going to turn it off or not. Yeah. I kind of feel like the dark forest and another example of that. I think the cat is out of the bag.
I guess the one thing we could do is militant against active SETI, where we send out really high energy. We could try to contain the damage that we've already done. Well, there are those who propose not just listening, but actually sending out. OK.
So we limit the damage to what's already been done and not doing more damage by intentionally looking for it, by not intentionally yelling. Yeah. By stopping it, let's keep from yelling. Yeah.
We're already having a cocktail party. Right. All right. So we should be listening off.
Sure. Yeah. Let's go through a few of those. So one of them is nanotechnology, which would be that some nanotech agent is created that is then used by a hostile force.
So like a nanotechnology weapon. Yeah. So in that case, it's somebody who's actually deliberately trying to do us in. Yeah.
They may not be trying to do everybody in, but they might make a mistake. And maybe they're only trying to do in people with a certain color skin or is that the same as the Gregu or is the Gregu? No, I don't expect a case of that. I think the Gregu is similar because it's also dealing with nanotechnology.
But it's more of this idea of the self-replication. The self-replication of nanotechnology seems to be the determining fact of the Gregu. Right. And that's any nanotechnology.
I see. So in the first case, it's just the ability to make incredibly powerful weapons that aren't necessarily turning the world into Gregu. They could just be liquefying the insides of every living creature. Right.
Right. Which is not the same as Gregu. That's right. Although it's pretty bad.
Yeah. It's very bad. It's very bad. It's very powerful.
And then the Gregu is you let loose some replicators and they turn everything into them. Yeah. Yeah. Everything is wrong material for them.
And so yes, they just take over. And it's palpable for, I said that word twice. Sorry. I said that word twice.
Sorry. One thing that I think is maybe more realistic at the moment and that people think about is deliberately engineered viruses or other biological agents. So they're taking HIV, for example, and making it much more contagious, so making it much more deadly. Sure.
Any of these viruses that we know about, most of them are not very, very, very, you know, they don't transmit very easily. So you could imagine that we could genetically engineer one that would be much worse. Right. I mean, what you want to do is you want to create something that doesn't have effects for a long time, but it spreads very rapidly.
Right. And then it's hopeless because everybody's got it before you realize it. Right. So yeah.
There are some other ones, like an asteroid or comet. Right. That one seems relatively easy to understand. Yeah.
The dinosaurs got wiped out. Yeah. As far as we know. Yeah.
And then there's the one that we're talking about. Well, yeah. Right. The badly programmed super intelligent AI.
Yeah. And that's why we're having the show. Or not even necessarily badly programmed, but just, I guess if you're not thinking about it and you're just careless with that kind of badly. I think careless is also badly.
I think so. Fair enough. Fair enough. I think so.
And I think that we're having the show because we believe we can actually make a difference in that one. And then we could be able to think about the first thing that was being said. But once you're done, it's difficult. Well, we both think about the first thing that was done throughout the year.
Yeah. Yeah, yeah. So yeah. And now let's talk about whether that's the possibility for the first level of the set of the show.
Well, we can talk about the first level of the show. And we can talk about the potential for which gamers then for a moment and talk about the actual second level of the show. We need to talk about the first level of the show. I'm not a bad player at all.
I'm not a bad player at all. But I'm glad. I'm glad to talk about this show because I think that's something very important for us to be able to talk about unexpected academic movements. they've had.
And they need to think about scale, tractability, impact and fit in order to decide which areas they want to focus on. Yeah, because you've got about 80,000 hours in your career. So, you know, start now to go in a direction that you're going to be happy about. So I think those same issues apply to us.
So scale here is, I think, on the continuum of indurable to terminal. And it's actually a quadrant of that graph where you've got the terminal and global. Yup. It's anything that's up in there has the expect, well, I'm bringing the expected value of the future lives.
Yeah, which is a hard thing to think about. It's hard to think about our descendants for thousands and thousands of generations. Yeah. Well, so let's just just hit it quickly.
Like if you were able to give a nudge of, you know, 0.01% towards less existential risk from super intelligent ais, you know, that 0.01% in future lives could be billions of lives. The equivalent of saving billions of lives today. Yes. Okay.
So, except saving billions of lives today would probably affect it more than 0.01%. Yeah. Well, we'll have to get into that. That's more complicated.
That's weird, isn't it? Yeah. So the idea is to look for something with a large scale. Right.
And if you were choosing between two things, one that had a smaller scale, one that had a larger scale, it gave me to go big. What about tractability? What does tractability mean in this case? You know, is there, are there promising things that could be done?
Like, is it going to be running our head into a concrete wall where there's really no hope? I don't have a good example, sorry, of this. But, I mean, since that climate change has gotten in that category for me a little bit. Like, I mean, that's what we said in an earlier episode, when we talked about why climate change is not the issue that's taking our attention right now.
It's because it appears like it will take the concerted effort of potentially billions of humans in concert to make a difference there. So it is tractable, though, in that we can imagine solutions to it. We could, yeah. Right.
And there, but that takes us to the third category. There are scotias. There are many of them. There's lots of people working on that.
Yeah. So the marginal gain from adding the Ted and the brand Ted, you know, the to Rand Ted Rand and adding our brains to it. The incremental marginal gain of that is not very high because lots of people are already working on it thinking about it doing the best they can. There's a lot of good focus energy pointed in that direction.
And yeah. So where you can find a big problem that has the potential to make it, you know, that you can make a difference on both in terms of there's a possible solution and adding your brain is significant. And then the fourth one is a fit. Like, do you have aptitude and sort of appreciation for that?
This one to me seems so obvious. Like, when I came to this thing, you know, 10 years ago thinking about immortality, we had a whole show about are we were running towards something or running away from something. And to me, the reason why concerning AI is not scary, of course, it's scary, but it's not scary because I see the Nirvana. I see something that feels like a personal fit for me because I want what's possible.
You want the positive end of it. That's right. We're going to tap one direction or another. It appears to me like we're going to tap.
Yes. And I want I want the one. It's it may not if I didn't want the one, maybe this wouldn't be as much of a thing. It's funny because you know, I don't really care about that as much.
I'm really afraid of the other one. Yeah. So where's the fit there too? Yeah.
Is it because of technology that this one is a fit for you? I mean, that too. I mean, my my background is a good is a decent fit. I mean, I have a PhD in AI.
Okay. So there may be a little bit of a fit there. So there's a decent fit there. Yeah.
All right. So these forcing like pretty pretty useful. So let's hold up concerning AI against those. Okay.
So scale this existential threat, love the, what is it called the future of humanity Institute? Is that the? Yeah. Yeah.
What they put at a 5% chance of human extinction via super intelligent AI by the end of 20, 100. That's right. In the next 35 years. So it's big.
Of course, they're, you know, they're there people like us who think that it's going to happen. And in my mind, even if you're way conservative and you say it's only 10% as likely as they think it is, that's still 0.5%. That's just like it's worth working on. Yeah.
So it's big enough. And we, and we believe it. Let's forget about what they say. Let's just talk about what we feel in our heart.
Yeah. And I don't want to put a number on it because I don't feel like the numbers based on the kind of reality, but it's too much. It's too much. It's too much.
Like it's too much. Yeah. All right. So so this challenge, scalar fits, tractability could could are their solutions possible solutions to this.
I don't want to, I actually have questions about I do even sure that it would be possible to keep the genie in the bottle or yeah, I do let it out in a controlled way. I agree. And as I've framed what we're doing here, I've asked the question, should we be concerned about the future of AI? And if we are, if we should be, then what should we be doing about it?
Right. I think those are the two questions that we're trying to answer on the show. And I don't know the answer to what we should be doing about it. Right.
I think we just answered the fact that we should be doing something. We should be concerned. You know, what we should be doing about it depends partly on the trajectory that we take together. Well, we're going to have to learn more about that.
I don't think we should do another show. Let's do that next episode. All right. Let's do that next show.
All right. But so, so tractability is a question mark, let's say. Yeah. Okay.
For sure. Impact. Is this an area that's oversaturated with human minds and energy? I don't think so.
No, I don't think it's anywhere near. So oversaturated. Although it just got a pretty decent boost recently, I still think it's nowhere near oversaturated. I feel that adding our modest, though they may be energies, well, you know, could have an impact.
I think so. I think so. And then we'll fit. So check, check, check question mark.
All right. Three out of three and a half or four. Yeah. So let's let's start to answer that tractability question mark.
I think that seems like a good place to go. And next time, look at paths from here to artificial general intelligence. All right. Sounds good.
Let's call that show. All right. This is Ted Brandon signing off. Bye.
Bye. The public perception of artificial intelligence is that a robot is going to come and break into my house and take all my things or become my evil overlord. And I think that this has become more popular when we see sort of scientific luminaries, like Stephen Hawking and Elon Musk saying things that are kind of terrifying.