Welcome to Concerning AI. This is Ted. And this is Brandon. This is the show where we ask two questions.
Two questions, yes. Should we worry about superhuman AI? Is there an existential risk to humanity? And I think the answer for that is absolutely.
Yeah. And if so, then what do we do about it? Yes, the harder thing. Yeah.
All right. What's the format for today's show? Well, today we have a very special interview that I did with Peter Scott. Brandon wasn't able to make it.
So we'll play the interview now. And then Brandon and I will come back and discuss it afterwards, so stick around. All right. So here we go with the interview.
I am here with Peter Scott. Welcome, Peter. Hi, Ted. Thanks for having me on.
Yeah. Really glad to have you. Could you start by introducing yourself? Tell us about yourself?
Sure. Well, I have been in information technology for over 30 years now, working in one capacity or another for NASA's Jet Propulsion Laboratory, for the most part. But I am also independent. And I've also started a career as a coach for personal and business purposes.
And I have recently been talking and writing about the existential risk of artificial intelligence, which is why I'm excited to be on the show. I have a book out earlier this year, 2017, called Crisis of Control, How Artificial Superintelligence May Destroy or Save the Human Race. And it's something that I take very personally. And I feel that this is the mission that has called me to it.
Fantastic. For the purpose of tonight's discussion, I'd really like to zoom in. I'm sure that we could have wide ranging discussion about your book. And there's lots of great stuff there.
We'll have to do that another time. I'd really love to talk to you about your recent timeline for artificial intelligence risks. I believe you published that on December 3, 2017. And we'll put the link in the show so people can actually see what we're talking about.
Is this the first time you had published it? Have you published it before? Is this revision? What's the, tell us the state of this graphic and where to come from and tell us a little bit about it?
Sure. That's the first time I published it. And I came up with this because I thought that it would be useful for us to have something to throw dots at that would give us an idea of what risks we were facing on what scale and when. And there's no formal work behind this.
And it's pretty obvious when you look at this, that any formal derivation of this that would support this numerically would require a fantastic amount of magic fixie dust it. No one has. It's based on educated guesses that we could get into more. Well, it's a lot more precise than an Amazon graph.
When Bezos puts up his numbers for all things going well at Amazon, they never even have numbers on the axes. So this is pretty good. Well, and I don't have error bars on it. I wanted to be excessive.
Right. OK, so can you describe it? So people will need to go look at it in order to really follow this discussion. But in case somebody's driving or whatever, they're going to look it up later.
Could you just describe how the graph is laid out and what we're looking at here? You bet. It's a timescale. X axis is years from now going up to 50 years from now at the right-hand timescale.
And on the y-axis, we have a logarithmic number of humans affected 10 to the power n. So we start out at 100 or 10 to the 2 and go up to over a billion at 10 to the ninth. And then there are various curves on this, showing the effects of various types of AI risks or deployments. And that would correspond to how many people do they affect when?
Effect doesn't mean necessarily exterminate. It just means effect. And there's five different risks that you've identified on here. Yes.
How did you choose those five? Autonomous weapons is one that I'm sure many of your listeners have already seen the viral video that Stuart Russell and the Future of Life Institute made recently about what they call slaughterbots, which is a great tag for finding it. And they postulated a scenario. I think that's where I probably got the idea for this.
Because they're going out a little bit into the future and saying, here's the risk of drones that have artificial intelligence and weapons in them being subverted to destroy dissidents or anyone that disagrees with you. And they had in there a fake news broadcast, some 8,000 students, I think being assassinated by these things, which is where I started thinking, ah, well, we could put some numbers on this. And if we have autonomous weapons being created, then how many people might be affected by those? That's one peg on that scale.
That's one point on the scale that they are suggesting there is a possibility. And it's plausible. It wouldn't, I think, get much higher than that without there being enough of a pushback, a reaction, to limit that in some way. Because the ease of constructing autonomous weapons, particularly shaped explosive charges, is not something that's accessible to the average person in the basement.
Right. OK. So autonomous weapons is one that potentially begins today, as zero years from now, according to your graph here. And the other one is employment automation, if I'm reading the graph correctly, that employment automation could be affecting, is that tens of tens of thousands, hundreds of thousands, hundreds of thousands, in the very short term.
Correct. And that's based on several studies. The Brookfield Institute and the Oxford Martin program in England have done some very comprehensive studies using real science and numbers to show how many people would be affected by automation. And they agreed to, within a percentage point of each other, roughly 47%.
What they didn't say was, over what time scale, but when I talked to them, they said, we're thinking 10 to 20 years from now. And so that's half the world's population in one way or another. And what they did say was, it's important to understand that figures is not meaning that half of people are going to be laid off, and the other half are going to be fine. What it means is that 47% or 50% of job functions can be automated.
And so few people would have their jobs completely eliminated. But they would have the amount of work that they do in those jobs drastically reduced and replaced by automation, which on large scales translates to the same number of people being made redundant. Sure. So on average, it could be that the average 40-hour work week goes down to 10, and everybody still has their same job.
Or it could be that three quarters of people lose their jobs in one quarter are still working the same 40 hours. But that seems unlikely. Right. I have a talk coming up to a group of auditors.
And if you look at the charts and locate them on there, it says they have a 93% likelihood of automation, or 93% of their function is likely to be automated. And so I will tell them this will either mean that the people that pay your salaries will choose to keep you all and do 10 times as much work, assuming that's available, or they will keep one tenth of you and do the same amount of work. OK. Let's move on to your next risk, your control failures.
And that one doesn't start today, according to your analysis here. It starts somewhere in the 10 to 15 year range. Tell us about that one. This is the risk of artificial intelligence not doing what we wanted to, the value alignment problem, but not necessarily for artificial intelligence that has any motivation, any consciousness, just sufficiently complexity, just being sufficiently complex, that a bug in it and who trusts the version 1.0 is going to have wide-ranging consequences.
These days you can write sophisticated machine learning programs in a handful of lines of code, because of the libraries that are available underneath them to perform all the complex functions. And that means that a bug is something that could happen at a very high level of abstraction, and so cause it to pursue the wrong goal, for instance. And this is where the paperclip problem, have you introduced a paperclip problem on the show? I imagine.
This is where the paperclip problem can come in. You don't have to have motivation or consciousness or free will or awareness for this to happen. You just have to have something that can indulge in creative thinking. And creative thinking doesn't mean, again, consciousness.
It just means good random number generators. And start making independent decisions, which is something that we are already playing with, already doing in certain respects. We on the show have been using a distinction between sort of narrow AI and artificial intelligence, artificial superintelligence. And I'm hearing you draw maybe a different distinction.
Is that right? Am I hearing you right? We potentially have the control problem long before superintelligence. You could certainly have the control problem with artificial narrow intelligence, and then it would show up in certain narrow domains, like self-driving cars or medical equipment or other applications, or the same, such as production line automation.
Yeah. But your graph, though, goes all the way into the hundreds and millions, maybe even more than that, eventually. In control failures, do we need? Tell me about that.
Well, control failure doesn't have to be a bug. It could be malicious. This could be the consequence of hacking. That you fail to control something, because someone else has taken control of it or decided to throw a monkey wrench in the work so that it doesn't do what anyone wants anymore.
A virus of some kind. Imagine the effect of something like Stuxnet on artificial intelligence that could control power grids or transportation or supply chains. It's not very far reach to get yourself to hundreds of millions people affected by something like that. Right.
Particularly as we increase our dependence on automation and AI far more. I like to tell people that it's hard to draw an analogy with the effect of a catastrophic virus in terms of how many years it would put us back, because you could say, well, we would be back in the Victorian era in terms of what we had lost in terms of control systems. But the Victorians had had two millenio to get used to the level that they were at. Us being thrown back there in a matter of weeks would be far more catastrophic.
We wouldn't be prepared to deal with that level of technology. Billions would die. Okay. Well, that was pretty scary.
Awesome. Right up our alley. Let's move on to the next one. So the next one, you label conscious AI.
And what does that mean to you? There isn't a useful definition of consciousness that I have found yet. So I'm going to put one on here, which is this comes to the point, this arrives when you have artificial intelligence that you interact with as though it were human. If you program something by arguing with it or bribing it, then I suggest that you might as well think of it as being conscious.
Okay. So this, I think is what we think of as AGI on the show. This is a system that may tell us that it's human. You know, it's Chris Weill's famous for saying, when the machines tell us they're human, we'll believe them.
That's very interesting that you should say that because I've come to believe that the Turing test is actually a lie. And the reason for that is quite simple, that it requires that you can't tell that the thing you're talking to is not human. But if the thing you're talking to is conscious, then it will know that it's a machine. It should know that it's not a machine.
Otherwise you've managed to fool it, which is not a pleasant thing to do to a conscious entity. So we are asking that this conscious entity presents itself as being human when it never grew up as one. And so this is a very human centric view of things. There's so many problems with this conversation in all different directions.
I love it. Yeah. So conscious AI is really the control problem, but at a different level of granularity, because now you have what might be termed motivation, or something that you can't tell isn't a motivation. You might have to justify yourself to these machines.
You have the hard take off in terms of intelligence. It looks intelligence. You may or may not be able to have a conversation with it, but you can tell that it's intelligent by what it's doing. And now you have to perhaps justify yourself to it.
Now you have artificial super intelligence, because as soon as we have something that equals human intelligence, you know, it's not going to be long before it's far beyond us. And now we have to look at philosophy and morals and ethics for these machines and look at those for ourselves, which is why we need decades to get ready. Yeah, absolutely. Okay, so I'm curious about the way you have had each of these curves, they rise, they peak, and then they fall.
What does it mean when they fall? Does that mean they're not important anymore? Does that mean they've been superseded? What is the falling edge of the curve mean?
Well, I think that control failures in this respect is superseded by conscious AI. It's a kind of control failure that I was distinguishing between ones that weren't necessarily arising from something we would think I was being conscious, and something we would. I think autonomous weapons as a class is something that we will eventually get a handle on. Maybe maybe maybe maybe soon, but maybe not.
We already have autonomous weapons. The gun emplacements in the Korean Demilitarized Zone can recognize surrender gestures from humans and take surrenders. So those are already autonomous weapons. Fortunately, the people that go into the Demilitarized Zone have a pretty good idea that their life is on the line to begin with.
But I think that this is something that is likely to get away from us in the near future, but it could be controlled like any other technology through arms limitation, treaties of one kind or another. OK. And with the other, so control failures superseded by conscious AI. Autonomous weapons is something that we possibly solve.
So the idea is that as the curve is falling, these are no longer the most important thing that we need to think about. They're no longer the biggest threat that the other threats that have come along. Right. As we get increasingly powerful, AI, some more of it might be considered autonomous weapons, but I would want to think of the problems with those as being control failures or conscious AI, as opposed to just autonomous weapons.
So it's a labeling issue here. It's a taxonomy that I'm attempting to convey here. And of course, the last curve is for self-replicating machines, which could again be another kind of autonomous weapon, but doesn't have to be. But once we get machines that are self-replicating, and at this time scale, we'll also have artificial intelligence.
And now we're really in paperclip territory, right? And each of these curves is really a best-case scenario in a way. We won't get to the future threats if we don't solve the early ones well. Exactly.
The big unknown in this is that the singularity is somewhere around 27 years from now. And so I have a version of this graphic where I put that line on there, and then I blank out the right-hand side of the question mark. Because by definition, we have no idea what happens on the other side of the singularity. Right.
Right. Well, let me ask you a question that we've been asking ourselves. And we've been asking, and guess recently, we would ask our community in the Facebook group. And your singularity being 27 years out may guide your answer here.
But so three different questions, all just with different levels of certainty. When will superintelligence arrive? When do you have a 5% shot? When do we, as humans, your estimate of a 5% shot at superintelligence having arrived?
What year? And then 50% chance? And when are you 95% certain that we will have superintelligence arrived? Could you give us a three numbers?
Well, that's great. And I like the statistical approach there, the quality control limits there. 5% chance of artificial superintelligence, which would, in my estimate, arrive probably days after artificial general intelligence. Sure.
I think the 5% chance of that is 15 years. OK. So 20, 32, 20, 32. And the next one was what?
50%. I think the 50% shot of that is around 27 to 30 years from now. And 95%. I put around 40 to 45 years.
Yeah. OK. Excellent. I think you're a little bit shorter than most of our numbers that we've, that we've, I can't remember.
I can't remember exactly. I shouldn't try to do this for memory. That's it. Memory's a bad choice for sure.
Trying to come up with numbers. There have been some surveys done of AI professionals of when we will have conscious or human level artificial intelligence. And Nick Bostrom took all the surveys and averaged them out and showed that the median figure was around 20, 42. Uh-huh.
Yeah. That seems in the ballpark. I find it fascinating how everybody's point of view, even when people say, oh, yeah, I mean, that's 30 years from now. We don't need to worry about that now.
And other people wait, that's only 30 years from now. We definitely have to be working on it right now. Yes. Like if I were to say, we just got contacted by a superior alien race, and they're on their way.
But it's going, they're 15 light years away. So it's going to take them 16 years to get here. Right. How many people would say, well, OK, then let's just forget about it for 15 years because we waste our energy.
Yeah. It's a good point. All right. Very good.
This was, this was really fun to walk through this kind of quantified thinking, you know, whether we agree on the quantities or not. It's really not the point of exercise in my view. I really love the idea that we can think through, think through some more numbers-based scenario planning and thinking through this. So I really value this work that you're done.
Well, thank you. I think that as fanciful as it is, and Aragon is to put numbers on this, the fact that we can't know numbers like this exactly doesn't mean that we shouldn't start thinking about things like this, so that we have an idea of what we should be trying to find out. Right. And let me throw another numerical thing in here.
There's a quirk of statistics that means that we're actually more likely to die individually in an asteroid impact than a plane crash, which sounds bizarre, but that's one of those things about statistics because the asteroid impact happens, yes, about once every hundred million years. But when it does, it wipes out everyone. So you've got 10 billion people being killed at that point. So you do the math and the expectation, this is a statistical term, right, is that that's 100 people a year.
Well, more people that's more than the number of people being killed in plane crashes. So it would be appropriate to expend at least as much energy and money on preventing asteroid impacts as preventing plane crashes. And we are at least doing something in that respect. So if you look at that, the math there is based on the complete extinction of the human race every hundred million years.
What are the odds of AI causing an extinction level event in much less than that time frame? If you had to put a number on, well, how long do we have before that would be the cause of the end of the human race? Would it not be much less than a hundred million years? And would it therefore make it worth expending considerably more effort on than we are doing to mitigate plane disasters?
Yeah, I think that's a point that will be obvious to our listeners. And I think that it's the old saw about looking for your keys under the lamppost because the light is better there. Even when you lost them in the dark alley, we don't really feel like we know what we're doing with AI. So solving the asteroid problems, like, well, we know what asteroids are.
We know they're out there. It's almost it feels like it's an easier problem. So it's easy to put effort there and not on this AI thing, which is so vague. And what are we even talking about?
Exactly. But then we haven't had that long to think about this. In fact, probably most of the thinking about what we're talking about has been done in science fiction, which is what makes these conversations so difficult to have with people because you sound like you're talking about science fiction and they can't get out of their head that you're talking about something that is either very unlikely or very far off or not really that dangerous. Right.
Absolutely. Well, thank you very much for coming on the show and talking about your timeline with this is really great. I'm sure that we'll be talking about it for a long time, maybe not in specific of your timeline, to kick off a conversation. And we'd love to stay in touch and talk more about your book.
Do you have any last thoughts for listeners as we close it up for tonight? I would like everyone to think that they are part of the solution of this. This is the message that I put out in my talks and my books that this is not something where you leave it up to the politicians or the generals or the top Google and Facebook developers to solve this problem. At one point in the future, we may be judged by an artificial super intelligence and it will look at the human race as a whole.
And so it's up to all of us to make sure that we come out looking good at that point. And so I believe that this calls for us to develop a much better understanding of ethics and much better alignment with ethics for the human race. Very good. Thank you, Peter.
How do people reach you online? I have a website humancusp.com at hu man cusp.com and I am emailable at Peter at humancusp.com. Okay, very good. I'll put those in the show.
Thank you much for joining us and have a good night. Thank you, Ted. Love the show. Okay, so Brandon, what do you think?
I love it. I love it and I'm sad that I missed the conversation, but you did a fantastic job as Peter. Super interesting. Love it.
Cool. Well, I look forward to reading his book and having an opportunity for the three of us to have a conversation at some point in the future. Yeah. That was fun.
Yeah. There are a couple things in particular that I kind of wanted to just touch on again briefly, not that we're going to cover them in depth at this point, but three things really jumped out at me. And I'll start with the banger, the beautiful one. Okay.
Paul said, you knew this was coming and you didn't do anything? Wow. It's a little chilling. It's easy to get complacent.
I mean, people are, the world and history is just littered with stories where, you know, the civilization or the city or the, you know, whatever was heading towards a just a predictable, very predictable and predicted like disaster. And even those doing the predicting didn't really do much, you know? Yeah. So yeah, so I don't know.
I found that was a real call to action for me. You know, I feel that I've kind of slipped into some complacencies or doing our little do our little podcast, you know, which is fun and is now kind of routine and relatively easy. So, you know, it makes me wonder whether, you know, there's something more to be done to go outside of my comfort zone. Yeah.
Yeah. Yeah. And then, you know, the other thing goes, one of the other things goes hand in hand with this. And that is when he made his call to action, which was, you know, hey, this is really something that every single one of us needs to take action on.
And his framing of that was interesting because he says that he believes that humanity will be judged by a superhuman AI sometime soon. And we need to be found worth preserving. I mean, he didn't say it exactly that way, but that was, you know, that's the call to action that each one of us needs to help humanity be our best selves and improve. Yeah.
That's a... So it's an interesting point that we will be judged or could be judged at some point, you know, by a super human AI. And as you and I have discussed so many times on the show, it's really hard to even write a scenario where it's a good outcome for humanity. You know, but one of the good outcomes is to be pets, you know, is to be pets to the super human AI.
You know, I don't know how likely the pet scenario is, but the idea that somehow the super human AI might make a choice to either wipe us out or keep us around. And then if it is going to make a choice, boy, wouldn't it be great to have it make the choice to keep us around because we're judged worthy, you know? Sure. That's the outcome I want.
If, you know, if that's the scenario and that's the decision point, I want to be kept around. Wouldn't it be amazing to be able, you know, to witness the sort of actually the sort of attend that future as that super intelligence, it does whatever it does. Yeah. And just be a witness to that.
I think that would be an amazing outcome. And yeah. So I like that outcome. I mean, I think it's indifference is probably feels fairly likely to me.
I think that's why I, you know, haven't really had that be a live, particularly live outcome, the idea that we'd be judged. I don't really judge hyenas, particularly. I don't really judge whales or orcas or crocodiles or ants or any of those things. You know, they are what they are.
Yeah. And I appreciate they are part of the great chain of life and evolution. So, you know, perhaps depending on what point in the space of consciousness this super AI comes in, perhaps that same sort of appreciation for the ancestors and the appreciation for life itself will be a thing. You know?
Yeah. And I would rather not have humanity judged the way that sort of mosquitoes might be judged by most of, you know, in the super human AI was judging humanity and it was seeing us the way that we see mosquitoes. I would say a huge percentage of humans on earth would be fine if all mosquitoes were obliterated. We wouldn't say that at smallpox.
Yeah. There's another sort of... That's even further in that direction. There probably are few people that say mosquitoes are part of the ecosystem and we need to preserve them.
But nobody's going to say that at smallpox. Yeah. Some snakes, some other things are less direction, but still, you know, there's just not a lot that's attractive to humanity in those creatures. Yeah.
Yeah. Yeah, for sure. All right. Yeah.
So, that's another point that I enjoyed a lot. And then finally, you know, I appreciated his touching on expected value again and the conversation about the expected value that humans today are more likely to die from the expected value of me dying from an asteroid collision is higher than the expected value of me dying from an air crash, an air traffic crash. And it's true. He's correct that that is a calculation that we as humans handle a little less than sort of rigorously in our minds.
Yeah. I can feel like lately, you know, I don't know how lately is lately, last few months. But I've been feeling like we probably need to have a math show. I mean, an episode, you know, we need an episode of this show where we dig into some of the math that's underneath some of these processes of prediction about different, you know, different things.
And I feel like this is an important place, even just the math of exponential, you know, the idea that 30 steps, if I take 30 steps away from my computer, we all know where I'm going to be, right? We know I'm going to be about 30 meters away roughly, right? I'll be out on the street. And if you double your distance to your computer 30 times.
Yeah. I still don't know the answer, even though I've gone through this exercise several times, but I don't intuitively know the answer. I can't remember. Is it like, so the moon and back or is it we for the, I can't remember?
Well, 32 times is like two billion something times. So if you fold a piece of paper, I don't remember how many times it is that we go to the moon and back. But anyway, I feel like we need to dig in on the math a little bit more. So this expected value math is not necessarily intuitive.
And I'm quite sure that everybody listening to the show could do that math if they, if they so chose, right? We got smart people listening to the show. But yeah, even so, how often do we really do that math and how often do we train ourselves to be intuitive about things that are not necessarily intuitive? Yeah.
And we're living in an interesting time where there are some existential threats that are that have a much higher expected value than an asteroid collision. You know, I mean, this what happens when super, super intelligence develops. And you know, I'll admit that there's a chance that it won't for sure. Maybe maybe a substantial chance.
I don't know. But even if it's, it certainly feels like a lot more than one in a hundred to me, to be honest. And even if it's one in a hundred, the expected value for the impact of that is just mind blowing. Yeah.
Mind blowing. It's just so much bigger than any other thing in life. You know, there may be some other things like synthetic biology and existential threat from super viruses and things like that, you know, weaponized and created in somebody's kitchen. You know, that there are other things that I don't know about that all of these technologies enable.
So, you know, it just feels like there are so many of these things right now. And those are two that I can conceive of very easily. Yeah. I also have come back to this question in my mind about have we chosen the right topic for the show, right?
Have we prematurely limited the possibility space for ourselves by choosing AI as the topic? Have we chosen the wrong existential risk? Yeah. That's what I get.
That's my question. Yeah. Well, you know, I think that that's possible that we chose the wrong existential risk. And I also think that it's it's okay.
Because we don't know. And this is enough of an existential risk that it's worth it. Yeah. And now we have a show where we try to try to sort of be analytical about that.
But you know, I just feel like another episode where we go back and look at that again would be good. Because you know, the more I talk, you know, we've been doing the show almost three years. And the more I talk to people in my day to day life about the fact that we have the show and we're talking about these questions, the more I get different answers from people, different answers that I haven't really considered or, you know, I talked to a guy this weekend who was convinced that climate change was going to wipe out humanity in the next 15 years. Now, I don't think that's true.
Yeah. I'll be good. Sure. But you know, I didn't really have much to say or he was convinced that capitalism was the thing that was going to wipe out humanity.
And I just don't, you know, I don't think that's true either. But I don't think that's true either. So how much of my conclusion on this is based on the fact that the name of the show that we published is called Concerning AI rather than Concerning Existential Threats. You know, would we think about things differently if we were sort of looking at a broader case?
Yeah. So are we in some sort of confirmation bias of the chamber? Yes. Exactly.
We gather an audience of people that is, you know, is inclined to believe that there is an existential threat. And so all the feedback we get from our lovely listeners, thank you, all of you, you know, is that we're on the right track. Yeah, but we don't get just feedback from our listeners. We also get Rodney Brooks and Marvin Minsky and other AI luminaries.
And IBM and, you know, all of these people who are poo-pooing the risk of saying these people who are scaremongering should just stop, just stop already. Right. There's no big deal. You're just don't understand it.
Yeah. Yeah. I worry the confirmation bias also allows us to disregard those. Just dismiss them.
Sure. Well, I mean, I think that it's pretty well established that we, the way we made decisions about what to do with new facts is, you know, if we like the facts, we say, can I accept these? You know, and if we don't like them, we say, must I accept these and we find a way of incorporating those up. So, confirmation bias is a big deal here.
And so, you know, I mean, a question that I have for us on this, okay, you're right. You should be sure to let it go. Because I'm about to dive into the topic. Okay.
Yeah. But anyway, I just, I want to say thanks to Peter and to you, I guess the last thing I wanted to say about Peter's work on that chart that you went over was that I agree that I think it's really useful to try to put quantitative numbers on things. Yeah. And if you don't know what you're, even if you don't know what you're doing, because nobody knows what we're doing on those numbers.
You know, they're all kind of wild guesses. Right. But even the wild guesses, uh, advance the conversation, I think a little bit and create different frameworks for thinking about it. I agree.
And I didn't really bring this up. I don't think when I was talking to Peter, but most of these, all these threats, the five that he's got on here and, you know, I don't know if there are others that we would want to add if we were to do an exercise like that, but they're all very amorphous and we don't really know how to talk about them. You know, we say, Hey, we're doing a show called concerning AI and our friends say, Oh, yeah, I've heard we're gonna have lots of job losses, you know, and let's, you know, that's one of the five that Peter's got on there. It's not the one that we normally think about this show is the one that matters the most.
It doesn't feel at all existential. It's a big deal. Yeah. The expected value of that impact is tiny compared to, you know, the existential threat.
Exactly. But if you put them then on a graph together and you can, you know, you can look at them and then you can debate whether the graph is right, you know, you could trust, but it gives the class give you something, right? It gives you a framework. Yeah.
But the graph, I'm going to take issue with the graph because the graph does not take into account future humans. Right. So, you know, if you actually put that existential termination of DNA based life on or none of this stuff that's shown on your shows up on the graph anymore, it's just out of the scale, even though it's a log scale, it's just out of the scale. Right.
Because there's, there's all the future generations of the descendants of humanity. That's right. And presumably, for example, we solve employment automation. We either solve it because we find other ways for people to have useful work or we redefine what it means to be useful as a human.
It's not based on our work. We have some kind of universal base of income or whatever we solve scarcity, right? We live in a world of abundance. Employment automation then drops to zero because it doesn't matter for any future humans.
Whereas, I have to say, I, as we've talked about many times, if we do wipe out DNA based life, the math is staggering at that point. Yeah. Yeah. And, you know, it is unclear whether there are limiting processes in place or if there are dynamics that can make such a thing stable.
You know, is it that we're coming up against a limit? Like some of the trigonomic functions, you know, they just go to infinity. Right. The closer to get to dividing by zero, you know, it just goes to infinity.
There's no go going to sort of pass that point. You know, is it one of these processes that is, that just runs away? Or is there some set of feedback forces or resource exhaustion that limits the growth? Or that the problem space grows double exponential while our capabilities to solve those problems are only growing at a single exponential.
Maybe, you know, there might be something in that realm that that allows us to not, you know, that doesn't allow us to solve this intelligence problem. Yeah. So, wrap up our kind of comments about Peter's work on this very fascinating timeline for artificial intelligence risks and just a wonderful contribution to the conversation. Anything else you want to say on this?
No, I just, I look forward to that conversation. The three of us can have it some point once we've read Peter's work. And as always, I look forward to hearing comments from our listeners. If you'd like to have your voice on the show, you can send it off in a voice memo format, keep it to a minute or less and email it to concerningai at gmail.com.
And if you'd like to participate in conversations like these in between shows, then join us at the Concerning AI Facebook group. That's where people have a great conversation over there. Any last words from you Brad? Yeah, be your best self.
All right. Until next time, keep your Wuffy up. That was my attempt to replace the E to Wuffy up, be your best self. Oh yeah, I walked right over it.
You just, it was like I dropped it. Yeah, I didn't, I didn't catch it, didn't pick it up, didn't, you know, I just kind of stopped on it. Yeah, I looked free on it then, okay, because I still like it. Okay, say again.
Be your best self. All right, be your best self. Okay.