Welcome to Concerning AI, this is Ted. And this is Brandon. This is the show where we ask two questions. Should we be worried about an existential threat to humanity from the development of superhuman artificial intelligence?
We always say yes, but today we're going to look a little different. We're really going to look at that question on this episode, yeah. And that question felt longer tonight when I said it. I wonder if there's a shorter version of it.
Maybe should we be worried about AI? Maybe that's the bigger, the short version. Yeah, well, maybe we can, maybe we can, is AI concerning? Yeah.
There you go. There you go. It's like the name is just made for this. All right.
So the first question is, is AI concerning? And the second question is, if so, what do we do about it? And we're not going to address that question tonight. True.
All right. So should we be concerned? That's really the question on our minds tonight. Yeah.
Yeah. And we, oh, sorry. Go ahead. That's OK.
I was just going to say Brandon and I had lunch with a new friend recently where he heard our concerns about AI. And he had another point of view that this stuff is not going to be a problem because there's something coming sooner than that. Right. This point was not that he knew exactly what that thing was.
But he's, if you've ever heard of Joseph Tainter and the collapse of complex civilizations, you might want to check that out. We haven't really checked it out. I guess it's the collapse of complex societies by Joseph Tainter. And anyway, he studies how societies get more and more complex and eventually collapse under the weight of that complexity.
So the question is, is collapse coming before general AI? And a second question that I think is highly pertinent is, does AI destabilize society and add additional complexities that increase the likelihood of collapse as well? Yeah. So I think we need to be clear on what we mean by collapse.
In the United States, we had a housing crisis in 2008, which resulted in an economic downturn. And some people call it the Great Recession. It wasn't as bad as the Great Depression. We're in the world by Great Depression in the 30s, but it was pretty bad.
There have been in the history of the exponential that we track on this show. There have been global wars. There have been genocides. There have been, if you go back far enough, you would have the Dark Ages.
You could say there have been intermediate, relatively short-term interruptions in the Forward March of Progress. And I think we're talking about with collapse something unlike those, something far worse. Is that accurate? Yeah.
Well, let's talk about it. I think there's a lot of different threads to pull. I think one of those threads is resource consumption. And kind of like the general wealth of the planet, in terms of natural resources and stuff.
There are a lot of different resources that are being exhausted over time, even making plastics maybe come challenging if we run out of the fossil fuels that we make them from. Yeah. Lithium. There's only a certain amount of lithium.
So all of our lithium ion batteries. I think there's probably a lot of lithium. I mean, it seems like, given that it's the third-lightest element. And I think there might be a lot of lithium.
All right. So we're not running out of carbon, right? That's not going to be a problem. That's true.
But there are elements that are more rare that. Ah, the rare earth elements. Yes. They go into what?
They go into semiconductors. Well, they go into semiconductors in particular, the substrate that we use. And we're throwing a lot of those into landfills in the form of disposable consumer merchandise. Which, who knows?
Maybe landfills will be mining landfills will become a problem. I've been saying that for years. I've been saying that for at least 20 years. Eventually.
We all open arms about recycling. And I think that's funny, because eventually we'll figure it out. We'll figure out how to get resources from landfills that there are far bigger issues to deal with than recycling. Interesting.
Yeah. I've heard you rant on this topic before. And as always, fascinating. Yeah.
I just think we make ourselves feel good by doing some small ineffectual act, like being really uptight about recycling, where there are just bigger issues. And I think the bigger issues we're going to talk about tonight. So assume that we do use the elements, some rare elements. We use them up.
Not just one. Because there's always some solution that we can find technologically for shifting from one element to another or whatever. But if we extract enough. Well, so let's lay a few threads out here.
So this is one thread, right? Resource exhaustion. Another thread is diminishing returns. And an example of that would be that the next doubling of transistor, like the next doubling according of sort of compute operations per second that we're going to get requires just so much more capital.
So much more sophisticated factory setups. A longer, more fragile supply chain of both talent, people who know the special things to do. And the materials and machines themselves are at higher and higher tolerances. It gets, it's now to the point where there are only a few semiconductor fabs in the world that can, or firms that can even put together these semiconductor fabs for the new denser and denser chips.
So speaking of fragile, so what's the author Telab, the anti-fragile and Black Swan author? That's a great question. I've heard the name many times. I haven't read it.
Yeah, I've read anti-fragile. I haven't read the Black Swan. But those are good works to think about this issue of systems. That seems Telab, right?
That seems Telab, yeah. This issue of systems that are fragile versus systems that are anti-fragile. Yes, yes. So that's another thread, right?
We'll leave that as an exercise for the listener. Well, no, I think that's another thread. I think we've got to lay out a few threads. So we'll lay out some threads.
OK, so resource exhaustion is one. Fragility complexity is another. Dimituring returns. Those are two different things.
Fragility and complexity are two different things? I think diminishing returns. Oh, diminishing returns. And the fragility from complexity are two different, many important threads.
OK, great. Yes. So I think the final thing that's really kind of like the driver of all of this is this highly global system that has a lot of single points of failure. An example of that has happened recently with Puerto Rico.
There's a whole bunch of medical supplies that we count on that we are short on now, because the supply chain for them has been disrupted. Since the hurricane. From the hurricane. So there's this strong impetus for specialization.
And specialization ends up creating these clusters of capabilities. One firm then spawns several firms. Before long all the people who know that special thing. And they're all clustered together, because that's where the people who know how to do that are.
And there's this probabilistic feedback loop, where if you have a tiny little advantage that gets compound together time. So I think that ends up focusing and specializing us in a way that leads to these multiple single points of failure in the supply chain. Yeah. What about the idea that the global economy today is itself a kind of a paperclip maximizer?
That it's sort of beyond our control. We set it up and we let it go. And it's just turning the earth into the money. That's funny.
Yeah. Yeah. I mean, the consumer, I mean, another sort of cake on that is a consumer culture. Right.
Is turning the earth into stuff. Depression. In the land. It's turning the earth into landfills and people into depressed people.
Yeah. Isolated to depressed people. OK. Yeah.
I don't know. I'll totally buy that though. I mean, I think I'm kind of with Steven Pinker and the. Yeah, that's certainly by any of these individually.
But I mean, I think they start to hold more weight when we talk about them all together, I think. Yeah. So the last thing we should add in here, the last thread we should add in here that, again, points towards collapse, potentially is the destabilizing influence of artificial intelligence. Yeah.
Yeah. So it may be that the seeds for its own sort of terminal winter. OK. Let's go further than that one.
I mean, are we talking about self-driving technology, putting millions of truck drivers out of work and a huge percentage of the economy being uncertain? Because we have all these workers with nothing to do anymore. I mean, is that the kind of, I'm not saying that's the only one or that's the short path. I'm just saying, for example, AI could lead to massive societal changes, which would themselves be destabilizing enough to potentially disrupt the supply chains or the economy or whatever, that they would prevent a super-human AI from being developed.
Right. So there's a lot of different ways that AI could contribute as you're pointing out. Right. And inequitable distribution of the windfall that comes from additional automation.
Yeah. So that's one thing. Like when you have a shrinking middle class, and you're getting more and more kind of polarization of the halves and the half knots, that is a destabilizing force. One.
Two that's also related to this is that the jobs that get taken may not be proportionally distributed as well. So that might, again, point us towards this sort of two-class system, those who have the AI and those who don't, or who are kind of the beneficiaries of it and those who aren't. I mean, I guess one possible counter-argument to that, that I think the utopians, 0.2, is that this idea of abundance, and that there's just enough for everybody, and that you don't have to work, you can pursue creative pursuits and stuff because there's just so much abundance. Problem is, we just aren't that way.
The human psyche is not great at distributing abundance. So the news that I heard this week, I went to Pittsburgh last fall, and I was moderating a panel on AI. The moderator of the other panel on AI has thrown his hat into the ring for US President 2020, and his platform is based on the universal basic income. So looking to call on our better nature, right?
When you say we are not that way, well, yeah, absolutely. And what would it take to get there? What would it take to change us? What would it take to choose it?
What would it take to on purpose go beyond the race toward haves and have-nots that will ultimately potentially destroy us all? I've never thought of that as destroying us all, by the way. I've always thought that it's a bad thing, sort of, on its face, regardless of what results from it, if we have just a very small cadre of super-rich and the rest of the world falls into poverty. And I sort of have always naively thought we'd just kind of work it out.
But I never really thought about the idea that that could, logically, even result in an actual devastation or collapse into the loss of our current level of technology and knocking us back 10,000 years or more. But that's what this conversation has got me thinking about. Yeah, so the 10,000 years feels challenging. Like how do we get knocked back 10,000 years?
It's hard to imagine losing all of the knowledge that we have accumulated. And that knowledge is just a huge enabler. Well, okay, what's realistic about being knocked back? Let's not go too many shock levels.
I mean, what if it were 500 years? Or what if it were 200 years in terms of, I don't know, the ability to have an electric grid? Or 200 years ago? Wow, the world was very different.
Yeah, in 1818. Yeah. Wow, yeah. I don't think, let's see, when did we get the telegraph?
I don't think we had the telegraph even, did we? I don't know. I mean, it was on that back, that order back to go. I don't know, I don't know exactly what it was.
Yeah, I mean, we certainly were nowhere near automobiles. Right. And certainly nowhere near ubiquitous, you know, electricity and plumbing. How did plumbing work?
Anyway, I think this maybe is digression. But 200 years ago does feel pretty, I don't want to live at that level. No. And wouldn't it be much harder for us today who haven't grown up living 200 years ago to all of a sudden start living 200 years ago?
It would be even harder than those that just lived in that time. Yeah, yeah, that'd be true. Because we've grown accustomed to our comfort. So is it possible that we could, that a collapse could lead to an 1818 kind of a world?
Yeah, I question that a little bit. I mean, are we to the point where, I guess it would have to be kind of, I don't know, it feels like it's an all or nothing kind of thing to me, a little bit. If we're going to collapse and not be able to like recover from it, it feels like we're probably going to get wiped out completely to me. Doesn't it?
I mean, think about that. Take the medical supply that you said. It's like IV bags or something like that. That's being made only in Puerto Rico or mostly in Puerto Rico.
And therefore, after the hurricane, we're short on it all across the United States. Take something like that, but worse. Something that's more ubiquitous and more supply constrained today that has some kind of a, you know, like what would be a lynchkin? Any lynchkin.
I mean, it doesn't have to be right. But let's make one up that allows us to hang our hat on a supply chain being disrupted that would then lead to devastation or the inability for us to get food delivered, maybe. What would be a lynchkin? I don't know.
I don't know. I don't know. I don't really have a candidate. It sounds like we need a little more sophistication in our understanding of the topic.
We do. We do. Let's take a different thing. Okay.
So we can't come up with that one. We don't want to stop quite yet. So I think I just feel like the world is in terms of nuclear danger. I feel like the world is as dangerous as it's been in my lifetime.
And I grew up during the Cold War when the Cold War was still on. And it just feels like even just as risky, maybe more even now to me. One of the things I heard on Sam Harris's podcast was this idea, which he was just repeating, he and his guests were just repeating something that I think maybe came out in the New Yorker or anyway, it came out in an essay, somebody wrote about how there's no safeguard in place. The president can order launch of our nuclear weapons.
And there is nobody to counterman that. And I have to think that the same thing is true in Russia. And we have a couple of, just not the paragons of wisdom running those two countries. And what happens when an accident happens?
Just one blows up in a silo or that false alarm that happened in Hawaii. What happens when the false alarm gets to the president? It's that sort of a mistake. So this is why I say it feels a little bit all or nothing to me.
Well, you're getting back into some older conversations that we've had around the nature of existential threats and have we picked the right one. And that feels a little bit different than what we're trying to do tonight to me of finding a threat or set of threats that are non-existential. But in fact, I think I flippantly said on episode four or five that some disaster that does not expect 10,000 years might be the best answer we have to the AI threat. And I didn't really say that in a serious manner.
But now that we're talking about collapse and we're talking about collapse in a serious way, we haven't done the full analysis. It's hard to unferrest to cover it. Other people have done a lot better job here. But it's making us think, well, what if that were true?
What if we don't reach superhuman AI in the short term and when I say short term, I mean the next hundred years, what if we don't reach superhuman AI because of some catastrophe that's not that it'll collapse that is not existential, but that puts a whole bunch of other barriers in front of us? Yeah. And we keep talking about collapse like it's an acute thing, like it happens within the space of months or years. But maybe it's just a long, slow grind, kind of like how the Roman Empire fell.
The Roman Empire, I believe, the Roman Empire just sort of slowly ground down. They just got overextended and just were unable to fund the armies that they had to fund to kind of hold on to their huge far-flung empire and keep it going. And it just slowly ground them down. I wonder if that is a better, a more apt analogy.
So the breakdowns that we talk about end up being kind of cumulative over time. Non-supply chain breaks down and then suddenly something is harder in life and people are spending more time dealing with that thing. And then another one kind of breaks down is maybe part of that same kind of a cascade of failures, but one that takes years or many years and decades to play out. So that it's not that it would be impossible for us to build the next generation of semiconductors.
It's just there's no appetite for it because everybody is focused on the problem of keeping waterborne illness down in the United States. Suddenly we're having trouble with our sewer systems and that's the actual problem. Yeah, the electrical grid maybe gets fragile and unreliable and rebuilding that. Transportation system, maybe something in that work.
It seems like it would, and that's kind of where I was looking for the lynchpan to try to understand what this could be. Maybe we don't need to understand it in an example fashion. But I wonder globally, what are the markers we might see of a global collapse? Is the Brexit vote written in leaving Europe?
Is that an exemplar? What could we see the United States faltering such that one or more states attempted to break away in some way, like the Roman Empire overextended? What if the United States actually is overextended or something? Here's another example in the United States.
I feel that our university is sort of a higher education system is in danger, but from a lot of different things, including people just doing the math and saying to themselves, it's just so expensive and it isn't worth it. That when I come out with that and all the different things, it's just not worth it to go to school anymore for one thing. I feel that there's a lot of different things. I don't want to get into some of the politically charged ones here.
But I worry about the assault on education as well and being educated and being an expert at things that has sort of seeming to happen with the ability to Google things. Suddenly, you're an expert at things. In a way, you are, but in another way, you're not. But what if that type of thing happens where we just lose the United States, at least, loses the ability to have a well-educated generation, to keep everything running and to keep things keep the economy growing and keep.
Yeah. So somebody like Elon Musk relies on literally millions of people all around the world to do what they do in order to provide the entire supply chain of everything that he does. And if everything is just declining, it doesn't matter how many individual geniuses we might have, Elon Musk can't build a rocket by himself. Nobody can, right?
That thing about nobody even knows how to build a pencil. It's that whole story. That now starts to... Well, I mean, you even think about computers and how many people use computers and how few people actually understand what goes into making those things work.
There's not that many people as a percentage of people who understand the semiconductor fabs, how operating systems work. Just all the different bits and pieces. Is that true? I feel like maybe I'm out on a limb here.
I don't know if it's true. And I wonder if what would happen if the next generation fab was somehow disrupted and we never got it and we had to keep living with the current technology. That doesn't seem that bad. Keep running the factories.
I mean, eventually things break down, but you've got people working there who know how to fix things. I don't know. It's the people who are adamant about this collab scenario are way more learned about it than we are, which makes me feel both inadequate to effectively give them their due on this show. We didn't even need one of them.
We didn't want them on the show probably, right, to really get into this. But I don't even know if I know enough to show where they have holes in their theory. And that also makes it hard to evaluate whether we ought to be putting our attention elsewhere rather than on the AI thread. That's why we're having this conversation right now, right?
Isn't that what we're looking at? Yeah. Let's come back to the impetus for the show in the first place, which is our kids, right? And thinking about where can we, you know, where is the highest leverage point for a good future for them that we can talk about and think about and maybe do something that matters.
And our answer has always been AI. But what if, and I think Sean, who's our friend that we, that Ted alluded to earlier, I think Sean was making the case that we're just never going to get there. So it is such a vanishingly small, it's just not going to happen. We're just not going to get there.
This is his belief, right? We're just not going to get there. So spending any time there is a waste and that we, where you should be spending your time is on creating resilient, robust communities and seeds almost like that can recede civilization, a new flowering of civilization that has the understanding of what collapse means, but that also has a lot of the benefits of all of that knowledge and information that we currently have. So how do we recede civilization and how do we live well as well as we can for the most people in a world where things have collapsed or run down.
And I find myself very aware of confirmation bias when we're having this conversation because you and I have invested a lot of time and a lot of energy into saying yes to our first question of should we be concerned about AI and confirmation bias allows me to find a way to say no to other threats while I'm only looking for evidence that supports my mind already being made up. So I'm trying really hard on this episode not to poke holes in the collapse theory but to really immerse myself in it and look for a way to find it to be true rather than a way to find it to be false. I don't know if that's helpful but it feels important to do that rather than simply dismiss it out of hand because I haven't done the research there. And I don't necessarily find the story compelling yet.
I'm not convinced but I don't think that is necessarily a valid reason to dismiss. Yeah well I think that intuition is actually a significant factor here and for many I would say Sean included the intuition is that artificial general intelligence is a long ways way and that there is it's just a long long way from here and there and I think that for you and I the intuition runs a little different I think our intuition runs towards we don't see the path from here to there but it feels like we're not that far away from there. And the timelines seem like they really matter here don't they? If AGI is 200 years away the collapse story starts looking more likely than if AGI is 20 years away.
Yeah yeah. There's no story that I can tell that there's no doomsday that I can tell about the collapse like that there will be some individual event that precipitates the disaster scenarios. That's the same reason I kind of don't put energy into climate change not because I don't think climate change is important but just that it feels like it's far enough the disaster from climate change the global disaster from climate change feels so far off that the existential risk of AI will overwhelm it and it won't matter the suffering of coastal peoples around the world in the next 40 years that is they might be won't matter if they're all dead from AI 50 years from now. You know it's funny that you bring up climate change because my family went on a ski vacation this weekend just to really bring home the yeah I'm very aware of several things around that ski vacation.
It's super expensive. Yeah. So you know there's kind of this resource inequity thing that we're playing out my family is playing out and I see that you know and skeebel which is on Mount Hood here was closed it's completely closed because it's just like mud and three out of the last five years have been like that last year and the year before tons of snow but three out of the last five years skeebel has really had a hard time having days open and I think back to when I was a kid because I grew up around here as well you know it just it just wasn't like that. So so this this happens to be one sat I was feeling like you know like it feels like skiing is going away on Mount Hood where I grew up because of climate change and for my kids you know they'll probably get it keeps skiing at the ones that are higher up the mountain but even those are probably eventually going away and then that just that recognition that skiing going away is such a pathetic thing to lament when whole societies and ways of life and cultures are at risk because their island is going to get completely covered over or the shore where everybody lives is going to get inundated or you know the collapse of food chains because of things just not being adapted to the climate that the climate region that they now find themselves in diseases in forests and other things like that and yet you know the thing that really brings it home to me is that my recreational hobby might be affected.
Yeah so that's another thing where the timeline really matters if we if we thought that global collapse from climate change was happening in the next 50 years but the AGI was 60 years from now that would that would change the equation. Yes so the timeline is important and I'm just I don't know I'm doing a terrible job but I'm trying to bring up how bad I am at thinking about these things and how bad my intuition is and how bad I am at actually taking action you know like the sort of things that end up motivating me are ridiculous and the sort of things that don't motivate me and that I can just basically turn a blind eye to it just makes me wonder how much of humanity is turning a blind eye to the things that matter. Well I think that's absolutely true and that's that's potentially what might lead to this kind of collapse. It's it's a different calculus than the AI thread.
I mean I think if we believe the timelines are such that AI makes the other problems moot then we're going to try to influence the AI community which is a relatively small community in the world and these other problems require much more influence and much more coordination and potentially make us better humans. So you know I'm not sure that there's an either or here I'm pretty sure that I understand the climate science enough that I'm not going to become a climate change activist because I see AI as a much greater short term threat. But this collapse one you know is I'm wondering whether I need to know more about it or what you know before design. Do you think they saw it coming or knew that they were in the throws of it?
Oh these different civilizations. So many civilizations I mean it's just like most of the civilizations have collapsed. I mean it's just they all collapsed. It may take a while or it may be quick.
I've been reading about the indigenous people of the Americas pre-contact in a book called 1491 and some of the stories they tell in the book are about how the European diseases showed up and within a matter of months or years huge percentage of population was just decimated. And that story played out among lots of different kinds of people. All kinds of different you know you call it civilization or settlement or whatever like different groups of people were just decimated by something completely out of their realm. And of course nobody at the time understood the germ theory even so nobody knew what was happening.
What are those kinds of things that seemingly come out of left field? Some of some things like astronomers is identifying some of these things and it's not that they're unknown completely but it's complicated. Well you know just a couple days ago that was the big stock market drop and one of the cartoons that I loved was it wasn't a cartoon sorry it was a picture and it said stock traders react to the record setting drop and it shows this great big server farm. You know I mean just a financial like collapse driven by all of the high frequency trading and the you know the different algorithms that we have running that seems like a possibility for sure you know like we hit a condition where everybody all of the different programs think that it makes sense to sell.
Well we're gonna have to cover that another time I feel like that's we're too far into open up a whole new avenue but I mean your point is that there are many ways that this collapse scenario could happen and we're not necessarily equipped to judge them at this stage. Not equipped to judge them and the point I was going to make with this stock thing is that's a place where machine learning and AI I think creates the greater fragility and greater instability and greater risk of a sudden crash of the economy because so much of it is being driven by these algorithms. Okay so there may be an argument that the time lines are intertwined that the closer we get to AGI or maybe it has to do with AGI but the more advanced our narrow AI's get the more fragile we might be. Yes yes yes so that's the one that really is kind of like to me.
So it basically is saying that we should be concerned about AI. Yeah but in a different way than we're concerned about existential threats it's a different focus. Yeah yeah yeah. But you know it would allow us at least not to you know have to give up the show.
Yeah. Whoo. Disaster collapse of the show narrowly avoided. Yeah which is really what matters you know it's my ego.
So all right shall we wrap? Yeah. All right let's wrap. All right so are you gonna are you gonna do the rhythm part and I'll do the lyrics in our wrap here.
Oh boy oh boy. Ah. This may be the worst ending of the show ever. It's pretty bad.
It's pretty bad. It's pretty bad. It's almost unsalvageable. Maybe we could salvage it by inviting our listeners to take part in conversations of this nature on our Facebook page.
Do you think if we had an algorithm running our podcast for us that it would make these sort of endings more or less likely. See I feel like you just sort of pull out the anti-fragile let's go to the listener feedback thing. Yeah. Could a could an AGI or even a narrow AI if it were running this podcast I pulled that out of the back pocket like you did.
Yeah I don't know but you know you're making it harder on me here by interrupting. Well take it. Yeah well. Take it home.
Take it home. Alright let's go to Facebook. Send us a voice memo concerning AI at gmail.com just recording your phone a minute or less. Eventually we'll have another listener feedback episode.
You know conversations really matter. This conversation we have with Sean created this episode so feel free to reach out and get in touch. Share links. Great stuff going on in the Facebook group.
So meet us there and until next time try not to collapse. I like that. I like that. I like that.
It's good. It's catchy. Yeah. It's my memo.