0064: AI Go Foom

Welcome to Concerning AI with your host Ted and Brandon where we explore existential concerns about AI and what to do about them This is Ted and this is Brandon and I just can't let that lighting longer It's the first moment that I've heard the intro although I know that it was added to the last episode the magic of editing and time shifting and you know recording out of order So wow, it's it's cool Ted. I can't believe you can make your voice sound like that. That's neat. Thanks.

Thanks All right, what do we got today? Well, I think that we are talking about rapid self-improvement and this idea of an intelligence Exclusions AI go boom. Yeah, what is boom? Yeah, you know, it's something that I've kind of heard about a debate between Eliezer and Robin Hanson I guess the name fella for a while and I haven't really dug into it very much But I'm kind of digging into it now and yeah, it's uh, it's worth digging into it's a fascinating topic All right, cool full disclosure then Brandon has done a little bit of homework And I have not done my homework here or any homework And so we're gonna work a little bit on what he's read and a little bit on first principles and a little bit on my Asking you know, somewhat naive questions and we may or may not be able to answer that But at least we'll get ourselves some good thinking and then you know, listen You can chime in and help us along the way.

Does that work? Yeah, that works for me. It sounds like it sounds like I get a talk a lot Which always works for me. Yeah, all right.

So yeah, you want to you want to give us an overview? Like what is an Intelligence explosion? What is food? What is what is an agrofone?

Sure? So there's there's a notion of an intelligence explosion where a Process that's smart makes itself smarter and it just gets smarter and smarter and smarter and pretty soon It's like a runaway chain reaction like a feedback loop that goes critical. Uh-huh and suddenly it's so it's way smarter than it started out So it's out around human level intelligence and for long. It's we're chimps compared to it or ants compared to it sounds like you're describing Methodically you're describing exponential curve, you know, that goes to infinity sure so it's like well Yeah, I mean, it's a it's rapid capability gain that that yeah It has a a co-efficient greater than one, you know, maybe it takes an hour to get to the 1.001 Time to smart as it was an hour before but if it can do that consistently Every hour doesn't take long for all that take to add up to a very significant difference in capability Okay Yeah, so then the question it so there's this notion of a soft takeoff versus a hard takeoff So let's say you posit an intelligence explosion that is you can get to Self-improvement over unity that it can improve itself more than just one doesn't reach diminishing returns Then how long does it take like what is the coefficient?

Is it such that we have years to deal with it? Is it such that it takes? Seconds is it such that it takes you know somewhere between it's probably somewhere between seconds and decades, right? Yeah, so does it if it happens at all what is it?

Is it useful to talk about the implications of soft versus hard in terms of impact or is it better to now begin on what yeah? Let's just talk about hard like let's just talk about the AI go-thooms scenario and what makes this seem plausible and what makes it seem unlikely, you know, kind of some arguments for it again And we won't get through all that much probably in one show but okay So we're talking about from the time that we have an artificial general intelligence at the level of a human being Within a matter of seconds or hours or days. We're going to have More or less infinite intelligence or or oh, I so that we don't understand at all. Is that the kind of idea?

Yeah, I think the infinite intelligence isn't correct. I mean it, you know, but this knows it's just a very Just significantly beyond human intelligence. Just just to say that so I guess that raises the first question Which is how much headroom is there right a book human how much more intelligence is possible in a being more than humans? Yes, right so the intuition there varies widely among humans.

I think yeah Yeah, we talked about this on the on the last show a little bit in fact I think I titled it something like there's no room at the top or something like that kind of ironically titled Yeah, yeah, you know I think that there are some of us and Eliezer is one of those and I'm one of those and I think you're one of those who think That humans aren't very good at being smart Right and that there's actually a lot of room at the top and part of the you know one of the arguments that Eliezer makes is that there's Several different things that we could see that would make us a lot more of a general intelligence So one would be the ability to expand across more hardware. Hmm. We can't do that right right? Yeah, another would be able to reconfigure our internal representations to optimize our algorithms to actually improve our own Thinking apparatus now the algorithms that we run the operating system that we run this would be like noticing that confirmation bias is a thing And therefore changing our thinking so the confirmation bias was no longer a thing that would you mean sure?

Yeah, so there's an example of if we could actually change How our thinking works then that would be that would be an improvement. Yeah another one is this idea of being able to Do cause and effect reasoning but kind of perfectly on really long chains? So to me that's not the right term. It's not cause and effect reasoning as much as it is well Okay, so here's an example So deep blue when it when it was playing chess at a world class level was doing so because it could follow causal chains Cause and effect things perfectly to a great depth in a way that a human cannot do Yeah, so we cannot we can even in a perfect information scenario like that humans can't do that So being having an algorithm that you can follow to practically arbitrary, you know billions of steps deep Perfectly without any mistakes seems like a capability that might be useful.

Yeah And it seems like having a better memory than we currently have would be one way to get in that direction You know we forget a lot we forget a lot of what we experienced we forget a lot of which if we study a chessboard And we think about three or four or five moves in in one particular line Then we come back and try to think it in another line, you know unless we're that unless we're a world-class expert We're gonna forget most of the analysis that we just did exactly Yep So so you can remember those billions of steps and the conclusion and pick up at any point and explore a different branch or a different Direction in that series of steps anytime you want to that's not that's not a capability We have no that seems clearly useful and more broad broad that would make you a special case of just having better memory Yes, right and then access to access to knowledge Communication efficiency like that network and provide between two sort of parts of an algorithm boy human language is so inefficient It is there's so much nuance and how do I know that the words? I'm saying really mean the same thing to you even when we're saying things that are fairly straightforward We've misunderstandings all the time seems like that would be an obvious place we can get better Let's just take a simple one like the ability to multiply two three digit numbers in your head Oh, man, there's a lot of people who can't do that and could never it would never be able to do that right then that's and that ain't nothing I mean that's that's nothing right for so many different kinds of modeling and yeah Just think about the efficiencies of being able to directly control a model Hmm and to have an intuitive connection with what's going on in the model rather than you know think about punch cards and computers versus the keyboard Mm-hmm and time slicing and the interactive ability to to have a tight loop like in software development One of the things that makes finding a bug or tracking something down easier the first thing I always do and that I try to teach other people who are new to software development is to shorten your development cycle as Make it as tight as possible when you're iterating around a particular problem Mm-hmm, you know if it takes you we have this this build system where it can take you know It can take a half an hour to get through to the point where you pushed up your source code and you get a build out If you're waiting for the test to run in a build system To see whether the change you made is fixed the failing test or the change you made You know getting it to deploy then to a system after the build has succeeded all the testing and you know You never gonna figure out that bug or that problem But if you can just change a line of code in seconds later see the result of that and have the system tell you what's wrong You know that is a huge win and so imagine having like we can visually scan and interpret a visual field like Instantaneously imagine having that with an abstract model Instantaneously being able to see it and so yeah, yeah, there feels like there's a lot of room at the top Yeah, just just from things that we can see right now with our limited intelligence, right? Right there's got to be things that we don't even know about Capabilities and ways of doing that. We don't even know that we don't know Yeah, it seems like we could continue this conversation of ways that our intelligence could be better by looking at the current Technologies we have whether you call them AI or not where are they better than humans and of course every technology is better than humans That's something because that's why we have it.

That's why we created it and every single one of those areas It's gonna lead to another another area of the of where we could be smarter, right? Yeah, so there's so you know the question is are we are we you just barely smart enough to Like do amazing things or are we as smart as you know? Like are all the amazing things done because we're so smart and it just seems so Unlikely to me that just even just this little argument that we have that we said right here Where these these all these things that we can see that would make us so much more effective if we have those it seems Almost ridiculous to think that we are up against some kind of heart limit. It feels like we're barely across the threshold Okay, let's take it from the other direction if we were up against the heart limit It seems like all of our advances that we've made so far we would we would see no future to go beyond them Like what if what if we were as smart as you could get with space travel today and that a moonshot is all we get?

There is no Mars. There is no anything else, but we're not seeing that right? We're seeing SpaceX land their boosters We're seeing you know, we're seeing reusable craft now We're seeing new advances all time and it seems like just that one tiny example is a pretty strong and we can do that in any field We're not running out of breakthroughs, which is there's more smarts for us to accomplish even with where we are today Yeah, so there's these all of these one-time thresholds that we're crossing the ability to land your boosters Yep, we've crossed that threshold now other people are you know other space companies that want to be competitive We're gonna do that we kind of know how to do it. We have you know, we've kind of got that one So you know, there'll be other the transistors that was one you like to bring up Moore's Law and they both all the curves that Add on top of each other right different Different serial breakthroughs kind of one-time thresholds that we passed over and then optimized So there's no reason to posit that we have come to the end of those and that the only ones that are you know All of those thresholds significant thresholds are behind us and that there aren't some in front of us that we don't even see that something smarter could see All right So can we simply like based on this conversation that there is room above us on the intelligence ladder?

Yeah, I my my intuition has been shaped by our conversations on this and and just think about it And it has is changed to believe that there's just a ton of room. It's intuitively obvious to me now Yeah, there's a lot of room a lot headroom. Okay, so let me ask our listeners if there is something we're missing here and there's some Critical piece of information or argument that would tell us that there is no room or that the headroom is very small Please let us know please get in touch all right and tell us what that is because we really want to know otherwise We're gonna move on from this as a solved question and move on to say so let's check that one check the box Yep, okay, we've now taken one step of the billions of steps that we're gonna be able to take with our perfect serial Algorithm here. Yeah, no, okay, great.

So there's a lot of room at the top The the next idea that I think was really useful. Did we call out the paper by the way? I if you haven't if you haven't read intelligence explosion microeconomics by Eliezer it has a lot of information in it that is Counterintuitive and unfortunately it's a little hard to read because Eliezer kind of uses a lot of technical terms because they're more accurate I guess so it isn't like a sort of Carl Sagan presentation where it just uses analogies and all those kind of things So I guess that's a little bit of what we're trying to do is boil some of what Eliezer wrote down into sort of normal speak Yeah, yeah Yeah, so it's 90 some pages long and I'm finding it really worth digging into and I find it to be a little bit of a tough road Oh, so you know check it out if that's if that's at all interesting I think it's worth worth checking out a lot of the things that we're talking about here He unpacks a great detail in that paper a lot of the ideas come from that paper not original with us great All right disclaimer done. What's the next what's the next piece of the puzzle later?

Can I get a check check? Check so that's two checks now. Okay. This is a very successful podcast Yeah, okay.

So the next the next one is what's the difference between chimp chimp and people where does the big effective difference come from and The first thing is you know, what does brain size do like what's the role of brain size in that difference? So let's talk about the difference in size of brains So if you count the number of neurons or you count the mass between a chimp brain and a human brain By the way, you do it. It's just somewhere between three and four times as big a human brain is three and four times Hey, by the way, I held a human brain in my hands this weekend for the first time This isn't gonna get me in trouble with a lot. It was so cool.

It was OHSU our local research hospital in Oregon here most most well-known They had a brain I don't know supposing or something open to the public where they had lots and lots of different kinds of brain Science all in one room and one of the tables had a human brain that they had the gloves there And you put up, you know, like put your own disposable gloves and you could hold the brain and it was so cool I was heavy and it was dense and it was it was really impactful So I'd love to hear more about human brains and chimp brains now. I have now. I have jealousy You brain envy. Yeah, that sounds so cool.

Yeah. How neat is that? Yeah, it was great cool Yeah, so four three to four times the size that's amazing. Yeah, yeah, so I ain't nothing right right that's substantial But it also isn't like a hundred times.

It's not you know, it's not a thousand times It's kind of surprising to me given you know given how similar our DNA is we share what 99.5% of our DNA with them something I don't know what it is, but it's on the order of 99% or something like that. Yeah, sure. Yeah, yeah, so so thinking about how natural selection works and You know why brains would get bigger There's an interesting thing that that Elie Ezra talks about that I've heard him say many times and I never understood it until I started reading this paper It was that the marginal returns on brain size changed like the calculus for the marginal returns on brain size change and what that means is that when The algorithms in the brain got better suddenly it was worth it More worth it to invest sort of units of energy consumption into brain So let's let's let's there's something that that is a little bit hard for me to articulate right now But that is real important to this so I'm gonna try an analogy here. I'm gonna give you a thousand dollars Yeah, that's great.

That's not and you are playing the improve the TED body and TED mind game It's like you know some some some game on the computer and you need you need your mind in your brain to be good Oftentimes it's spaceships or cars or whatever it is that you actually improve in those games But it's the same kind of thing you got a thousand dollars and you are you have a lot of choices to make you could grow your brain by one ATP unit Okay, you could Increase you could you could or you could invest that in making your Your quadricep one ATP unit bigger. Yeah, jump jump more and run faster. That'd be awesome. Mm-hmm.

Yeah Yeah, you could invest it one ATP into your immune system. Oh, yeah, yeah, okay So the question is which one are you gonna do? Well? Yeah, how how would I make such a decision?

Well the answer is you're gonna put it where it you get the most bang for the buck. Yeah So at any given point in time For an organism that's sort of an evolutionary equilibrium One unit of ATP invested in the immune system is gonna buy the same amount that one ATP unit in ATP is a Dancing triphosphate the sugar fuel that we use to run things. So so one ATP unit one unit of energy invested buys the same Amount of return on investment everywhere. Otherwise, you would have already bought the cheaper thing you'd have you've spent less here and put more over there Okay, okay, does that make sense?

So that's what evolutionary equilibrium is is there's there's no real significant Disparity between spinning one unit here versus one unit there Uh-huh because if you if there were then the selection pressure would quickly point would quickly drive you to that equilibrium point. Yeah, okay Yeah, so during the time that human intelligence sort of rose to the floor human brains grew by a factor of three to four times what they were before so what that says is that at some point the There was some kind of a feedback loop where Improve the algorithm make it worth more To spend money on hardware to run that algorithm. You mean that there was an equilibrium point just like you said Where there is an equilibrium point with the algorithms that chimps out? Yeah, there's no reason to make the brain better I mean there's no reason to invest the energy there.

Everything's okay. Yeah, stick it in the testes. Yeah. Boom done, right?

Okay, so what what just did they equilibrium point that what made what made that increase? You said you said improvement of the algorithm. What does that mean better algorithms? I well it just means that that the architecture of the brain the way things were organized the way things are connected got better And it's like um alpha zero Right, I was zero was an improvement in algorithms And it made all the available hardware that we had more effective.

We're able to use it more effectively Yeah, so that hard work more right same same kind of thing And and yeah, so so that's the point he's making is that the algorithm clearly algorithms changed enough to make it worth it From a natural selection perspective to spin more on larger brain sizes. Okay, all right So this is this is the feedback loop that he is talking about about for AI boom Because here is the third point He is a boom proponent because he says that he does not believe that the AI will need to invent Any new computing infrastructure? Oh, this is a big point to go boom. This is a really big important point This is a big important point This is one of the key areas that we get criticized for all the time that you can't have a hard take off because the humans still control the Means of production and the computers aren't gonna be able to do hard science all by themselves They're gonna need some way, you know They're gonna need to invent new technologies in order to have enough headroom, you know for better, you know for better hardware They're not gonna be able to do that quickly.

It's gonna take them years or decades But you're saying the opposite. Yeah, so it's the notion that there already exists a hardware overhang if you just know how to use the resources better All right, so what why would we believe that why would we believe that the current hardware is enough? Since we haven't gotten there yet What makes us think it's an algorithmic problem rather than a hardware problem? So one one thing is that the hardware that we're running kind of our shape the world algorithms on is Special-purpose hardware that is you know, that's that's built to distinguish tigers from On the Savannah to shoot bows and arrows to you know see whether that person is receptive to mating or not it's not the Hardware that is really great at modeling how economic markets will respond to a particular impulse It's not hardware that is great at understanding the protein folding problem or working on that or even playing chess or go Or these other things that we feel pretty proud of ourselves for being able to do so There may be a lot more modest hardware demands than what the human brain possesses if you just have a lot better algorithms ones that weren't you know kind of in the an after effect of the Run down the gazelle through an endurance hunt kind of algorithms.

Are you still there still there? Are you? Yeah, I'm trying to I'm trying to get my mind around this idea that the human brain hardware Could be used way better if we had better software running on it and how that connects to the idea that a super intelligence explosion You know an intelligence explosion can come from existing silicone hardware. Yeah, so the important thing is whether or not it's possible to make a An algorithm where the intelligence researcher or to make an intelligent to make something that can actually improve its ability To improve its ability.

Yeah, so here's an example of a way in which a better algorithm can take advantage of existing Resources a lot better. Okay, so when humans write algorithms we tend to write Serial algorithms the right things that have one threat of execution So they are their cereal rather than parallel. Yeah, if you take all the computational resources in the world and you stick it into one big bucket It is almost entirely parallel every PC Everything you know that if you want to take advantage of all of that you have to figure out how to map serial algorithms on to to be parallel Mm-hmm. So that is a you know, that's a an algorithmic Modification and so people smart people will take serial algorithms and and with more or less successful Efforts will turn them into parallel versions of those algorithms that can achieve speed up by running on multiple processors multiple computers communicating Network, you know different things like that.

So can I try an example to see what I'm talking about here So like we want a brute force crack a password We could just try one and then the next and then the next and the next you know in cereal one at a time And if you take that and have you know, however many processors We've got however many computers around the world if we could run those in parallel and everyone try one And we had enough computers to try all the combinations the brute force would happen more or less instantly is that Yeah, there's mostly a coordination problem, you know, like which which part which one do you try? At each of the different ones, you know, that's that's a difficult problem to solve but that's something you can get better at solving as well Yeah, okay. All right. So parallelization is one one area where our algorithms could be way better Yes, yes And so if you get so imagine getting something that has the ability to recursively improve itself so that it's it's Architecture it's sort of intelligence software architecture is getting better with each effort that it makes that would be exciting I'd be an exciting result.

Yeah to get to so one of the sort of AI fear scenarios that That Elie as are six out there is that some well-heeled research group is super excited about how promising their their results are and They decide that they're gonna try out it out on like a hundred thousand times the resources Because that's one of the things that you know the Google Compute Cloud and Amazon Web Services and these things provide is huge amounts of Computational resources that are available Like almost effectively instantaneously by effectively instantaneously. I mean within minutes or hours You can have hundreds or thousands or hundreds of thousands of compute nodes up in these huge data centers all over the place So if if these excited researchers decide to stick this this thing that's showing a lot of promise on a thousand nodes to take it to 100,000 nodes and see what happens, you know, let's get this thing out to nature as soon as nature or science or get it One of the procedures magazine as soon as possible. Let's let's see what this let's see what happens. Yeah, that could be bad Sure indeed from a fume perspective.

Yeah, so then there's two other things that I think came out of this paper I thought were interested to consider and one is the idea that Cognition itself like the ability to be smart provides some sort of fitness returns in the landscape of the world, right? General intelligence or just the ability to predict things and model and stuff it provides some sort of a fitness return We would think so. I mean we exist and that would seem to be a proof of that idea when it We seem to run the world we see the top of the food chain every food chain and at this point And that would seem to be that there is some return some benefit to evolution into you know Or to our being to our organism for having this intelligence. Yeah, so it seems like like that maybe is true Right the second thing that's separable from that is the idea that the cognitive returns based on more Resources like bigger brain sizes You know, do you get better thinking if you have a bigger brain and it's I think it's pretty clear This is again Elie other saying this but it's pretty clear that if you take and add 25% more brain You don't get a super intelligence automatically All right, because that's within the normal range of variation.

Yeah, so there is some correlation between brain size and intelligence Okay, but it is in tell us. I think I've gotten into a cul-de-sac here. I don't know Okay, if yeah, I'm not sure. I'm not sure what that means is that if the variation in brain size in humans were correlated strongly to intelligence Well, but it doesn't strongly strongly coordinate, right?

What does that tell us? I don't maybe doesn't matter I'm like well, there's something that I haven't really figured out here in the article that is related to this Okay, and so I guess I guess we're treating our listeners to some serious bumbling here But but let's just go there anyway since we like spiders sure don't we like spiders? I got a little afraid of spiders, you know, we had a mascot though We'd be like spiders probably because it'd be a big spider with a small human yeah They provide a window into what different intelligence could be like yeah, okay Yeah, but he says if hominids were limited to spider size brains It'd be much harder to develop human level intelligence because the incremental fitness return on improved algorithms would be lower Because each algorithm runs on less hardware Hmm, so there's a feedback loop kind of a positive feedback loop here that You may have to be above a certain threshold to get to sort of get any benefit To really adding bigger brains. So so let's let's let's think about that in terms of the spiders So why aren't why aren't spider brains really big?

Why aren't spider brains? Why aren't they getting bigger right now? There must be an equilibrium. They're at equilibrium because their algorithm sucks.

Oh bad. Hmm one and Two because they just don't have the ability to have a great algorithm with a limited amount of brain that they have and Yes, so and to improving those algorithms doesn't buy you all hardly anything like it doesn't buy you anything because it's running on such a Little hardware. It's like this chicken and a thing. This is the same reason why my clients have to be a certain size before it makes sense For either of us to work together right because it's too expensive for them to Work with me if they're smaller and the benefit that I would provide them is too small Whereas when they get big enough then they can invest in me and they get tremendous benefit from that This is sounds like saying it sounds like I was saying the same thing.

Yeah Yeah, yeah, the amount you get from spending a dollar on Ted is too small when you're too small Yeah, but when you multiply that time something bigger. Yeah, it's worth it. It's been a dollar on Ted. Yeah, okay cool Yeah, cool.

All right, so last thing last thing seems that most opponents to the hard take off to the AI food violence That's the most let's say the Robin Hanson camp the argument the main argument is that human effectiveness this big leap in your general intelligence was a one-time thing We crossed a one-time threshold that we can never cross again Therefore, there are no equivalently large gains to be had in the future and the threshold that I think he posits is the Accumulation of knowledge the ability to accumulate knowledge and communicate that and so what we see I guess maybe to put a different way that the reason Why we see so many amazing things is not because people are getting smarter It's because one we have the ability to accumulate knowledge and two there are a lot of us You know, yeah, so there's no reason to think that Anything that's coming is gonna be significantly smarter Just me significantly more general significantly more effective. So yeah, and we've talked about the headroom already It doesn't just doesn't seem doesn't seem Reasonable to me given that we can't expand on the new hardware. We can't rewrite our own source code Right can't we can't run these error-free cognitive processes that are you know super super deep many many steps? Yeah, perfectly unwind those or remember those so I just I'm still struggling I think I'm gonna have to go read his his papers next yeah Well, I think we've done some important work here.

Obviously, we're still novices in this area, but the Thinking thus far is that hard takeoff is possible whether it's more or less likely than a soft takeoff I don't think we have you know, sort of proven anything but We've been talking for more than two years about doing a show about soft versus hard take off So this is you know, we're getting there. I think this is a good beginning. Yeah. Yeah, are we crossing some sort of one-time threshold?

Oh, you mean we won't be able to learn a lot more about soft versus hard to go at another show about it No, I don't think so. I don't think We have cross the threshold we've crossed the threshold of having a substantive conversation about boom Where we've been meaning to for quite some time. So that's great Yeah, yeah, and now after after I read the second half of the paper They'll be other things to glean from that all right, so hopefully listeners do listeners you will come along for that Right and some of you will take out the challenge of reading this paper I'll put a link in the show notes you can just google elisa ydkowski Intelligence explosion microeconomics intelligence explosion microeconomics awesome great. See see I should just call it a iPhone iPhone AI go boom.

Yep. I can understand that. Yeah, he's not right for understanding though. He's right for like precision precision Yeah, he's a super smart guy.

He's hard to read I find it is I totally appreciate he's I heard him in the Sam The podcast with Sam here. He's a lot easier to understand talking. Yeah. Yeah, just which is cool.

Yeah I think if you're your work elia's really appreciate your work So not nothing that I say here about it being hard to understand you as a criticism you're doing good stuff Yeah, absolutely when you write in the way that elia's are right see invites people like us to try to translate it So presumably some of our listeners are better translators. So we are so please get in touch Let us know how we can do better or you know what else we could cover in this in this universe of the intelligence explosion Boom. Thank you for listening to concerning AI to get your voice heard on the show We're recording one minute voice memo on your phone and send it to concerning AI at gmail.com