Hey, this is Brandon. Welcome to Concerning AI. Yeah, and we're like we're right back in the saddle here Here we are. Yeah, better than only been two weeks.
Yeah, we're you know trying to get the weekly, but you know soon I think is is I've been he's been nothing been thinking about nothing else. Yeah for a couple of things a couple of things So let's jump right in we got some follow-up from two shows ago. That's kind of how the follow-up goes sometimes Yeah, but it's all right. What do we got?
So the show two shows ago. Maybe it wasn't two shows ago Maybe it was my last show. I don't know. Let me just read it.
Oh, this sounds good I haven't actually read it. Okay. Yeah, this is in the Facebook concerning a group from Lyle Cantor He says regarding the open eye open AI strategy. This was from us.
This is from us show. This is feedback in my show Okay, sorry, Lyle good here. We go regarding the open AI strategy of making sure we have more than one superhuman AI This is not as good an idea as it might seem for whatever reason humans find analogies to nature highly appealing and have positive associations to terms like Ecosystem and bio-mimetic. I actually don't know if that means loud, but you know bio bio means life Medic open AI seems to think something like this the solution to friendly AI is to have an ecosystem of competing ais So no AI gets too much power as we have temporarily escaped the Malthusian condition first world humans have a poor understanding of how hellish an ecosystem can be Natural selection the engine of any ecosystem is a search process that is literally powered by death mother nature is about as Lovecraftian as it is possible to be Yeah, people think the selection pressures acting on an ecosystem of ais will lead to a human compatible outcome.
Why? As ais are able to alter their own Yeah, he goes on This is so awesome Thank you, while as ais are also as ais are able to alter their own source code It seems to me you would get an odd form of natural selection where each agent is aware of selection effects and is able to alter its own Capabilities to make it more competitive one simplifying assumption We can make is that all ais are equally capable or able to quickly modify themselves to be equally capable save for modifications of their utility function This may be false, but it's necessary to maintain the scenario in these conditions It seems to me the most fit agent in such an ecosystem would be an agent that desires only to replicate itself Why is this say you have an agent X who values paperclips an agent? Why who values only to replicate itself agent? Why is able to make more copies of itself as it doesn't have to spend resources making paperclips?
Mm-hmm pretty good. That's true Yeah, because this is an odd sort of meta natural selection agent X would realize this and have incentive to approximate Why as much as necessary to compete with agent? Why? While maximizing paperclip production, but it could not do so completely without abandoning paperclips entirely because of this we should expect a Universe full of agent wise or agents that are able to approximate agent?
Why very effectively humans don't seem to meet either criteria I don't see anything immediately wrong with that. So yeah, wow. Yeah, and I have not understood at all Why open AI believes diversity is somehow a solution that gets us out of the AI safety problem like that They seem to be very magic wand wavy diversity have a lot and everything will be fine. So I okay, so I got a response Okay, all right So my response is that complicated systems are difficult to predict their behavior, especially when they're perturbed by kind of Unpredictable processes which you get our environment So if you just imagine making a thousand copies of me, you know and none of them can occupy exactly the same space So there's slightly different sensory input coming in each one.
They immediately start to diverge Yep, they're not the same agent anymore, right? And I'm simple so go more complicated with an AI why in this case as soon as you make a copy you no longer have two Wise you have why and why prime so okay, but that's assuming we want to make copies at all right? So okay go on with your copy copy copy argument I'm not sure that copying I'm not sure that there needs to be generations well, but the other the other piece of this Okay, so there's so there's these copies, you know that diverge immediately sure So they and they because they're complex systems, you know a butterfly flapping its wing in one part of the mind leads to a like completely different outcome in a different part of the mind so they immediately Can diverge wildly even though they start out the same, okay? So I think that that feels reasonable to me and also then combine that with that why itself if it's at least as complicated as we are Actually is complicated enough to house multiple intelligence simultaneously like the society of mine in a human mind There are multiple minds Operating and sometimes one is in control.
Sometimes another one is in control Yeah, where you've got you know our inner mind that we get from the reptiles and you know Is this a simplification of course but the inner mind that we get from the reptiles and we are a million mind that we get from You know evolving as mammals and then our exit cortex It's not like one of those is in control and even within those there are different kind of other ways of dividing up our mind work So I still feel like the diversity of agents Yeah, I guess I feel like maybe there's some possibility there because of the diversity that's present even within the very close Physical confines of a human brain. Well, yeah, a couple I mean a couple thoughts on that Which you know we're not examined yet because I'm just just thinking these thoughts for the first time But just because we have the biological history and therefore have the mammalian brain or the reptilian brain the mammalian brain Why do we think that there will be multiple intelligences in an AI system? Maybe that maybe that's not true. Maybe there's only one and what would that even mean?
I guess we don't know I've read you know We've read some fiction where this idea holds and so I think it's an interesting idea But even if it's true, how does that help us if the singleton I guess we need to come back to the idea of the singleton and this the singleton This is an idea that I examined when I was reading super intelligence What's up? What's your name? Bostrom Nick Bostrom super intelligence where he really got into this idea of singleton and he didn't find a way around it the idea that the first Super intelligent AI all had to be is 15 minutes ahead or 10 minutes ahead or one minute ahead of the next one becoming super intelligent It would be smart enough to know that those other ones coming behind would be a danger to it and it would subvert them somehow So the idea of a diversity in theory if multiple super intelligence is emerged all at once Then maybe a diversity would keep them all at stasis or you know keep them all in harmony somehow or you know We'll get the upper hand, but otherwise if one is ahead even by five minutes It seems like the diversity doesn't necessarily help us and I'm not sure how the multiple parts of the AI Yeah, it does not feel like a four-bound conclusion to me that the singleton does not feel like a four-bound conclusion again because of The locality effect I need a better name for this We need a better name for this but the sense that you know the lag of the speed of light between here and like the west coast or the east coast like across the United States is I don't know maybe a hundred milliseconds or on the order of it's on the order of hundreds of a hundred milliseconds Okay, all right, maybe not quite that maybe it's 15 milliseconds, but it's it's not it's huge compared to the speed of light across Like a centimeter or something like that, right? It's it's it's one centimeter.
It's however What is that 25? I don't know how far this is I've tried to figure out what the orders of magnitude here are but it's on the order of Four orders of magnitude five five six five or six orders of magnitude difference So imagine that AI using all the earth is its computational substrate all the matter in the earth is its computational substrate There is like human level and much greater level Computation happening in human brain-sized chunks of this thing that are separated by what feels like decades in difference Compared to a human mind right now. How can those possibly maintain a unity? I don't think that's possible I think that you'll end up with coherence in like localized coherence that acts as kind of its own thing I don't know am I talking about here?
I think you've expressed this similar idea on this podcast several times I know I can't stop it I need to come up with it with a pithy little name for it because I'll make it seem more true Probably that there's a locality the spatio low-cal No the locality principle. Yeah, there we go. I still don't understand it's just 20% true or since I can't I don't understand Why even if it's true it helps us that's the part that I have not yet the grass well, it's okay I'm gonna make what I think what might be your assumption and you correct me, okay? So I think your assumption might be alright So I'm smart because I'm this AI that's getting smarter than we are yeah And it realizes that you know It's not gonna be able to control what happens if there's lots of these chunks of smartness around so it because it has Fast infrastructure it creates a bunch of replicators that just turn everything else into great goo except for where it is Oh, I thought of that that's the only thing I can imagine good That's a good idea Some sort of simple simple active suppression Yeah, I mean yeah sure because there's no way if you've got like if you've got wide spread computation going on It cannot stay coherent and still operate its maximum frequency Well, I don't know if it has to operate its maximum frequency I guess I guess one argument would be if you have well then that has to be you know there has to be slowing thing There has to be deliberate damping well, maybe or maybe it works out That's because there's internal communications protocols that allow for that delay no no no no Why why wouldn't it it's smart right it's smarter than we are but how would that possibly work?
I don't know like just I mean think about like you're talking about order of a hundred milliseconds I mean I'm saying if their first super intelligent AI is let's say it is located located the first one is located somewhere Yeah, just for the second part of it. Okay. It could be that it needs to be distributed to be actual super-tolerant But let's just say it's physically located somewhere. It's located in Portland, Oregon All right, that's right Okay, and so its optimal strategy is to kill off any newly emerging super-intelligences So it's looking everywhere.
It's connected to sensors every other thing right? So it tries to figure out where is the thing where is the thing I need to suppress and if the thing I need to suppress is somehow somewhere else Then it needs to act somewhere else right? Uh-huh and so is it does it have more trouble acting somewhere that's far away? Yeah, I'm like no question then it does here because it can't as soon as it makes a copy of itself clear over there It now is a competitor.
It's there is now there are now two super intelligent AI What is it? I'm gonna make a diver So maybe it does it maybe it does simple Maybe it's I mean that the the limit of the simple separation case is Gregu everywhere except for where I am so when we send a probe to Mars Uh-huh and there's a 32 minute round trip delay or wherever it is right? It's really hard to control. We don't we can't control it.
We can't control it right? It's got to be very autonomous and we do we do we do send instructions at some point But it's yeah if it gets stuck on a rock and it's autonomous stuff can't get it off It's gonna take us weeks to get it off right whereas if it were right there with us It would be off you know like in minutes short So if the new super intelligence was gonna was going to emerge on Mars then it would be impossible to suppress it from here So is this is this is exactly it? This is exactly and is when you've got a thousand times human intelligence in the space of like a refrigerator size here Or even like the size of human human you know human brain here Yeah, then something that's happening across the country is going to be just effectively completely a mystery and unknowable and and confusing and worrisome And I think it's you know one of the okay So here's a here's a conjecture that I find axiomatic Alright, it's part of my system of axioms and that is that in order to predict what's happening in a sufficiently complicated system This is not my own idea. I don't know what the idea is But in order to predict what's going to happen in a sufficiently complicated system The model itself that does the prediction has to be as complicated as the complicated system So you can only maybe know the answer to what happens here on earth in in a hundred years or even in a couple of days If you have earth an earth equivalent with all of the complexities built into that Does that make sense?
So I can't know what's going on over there without simulate Oh wait simulator I can't know what's gonna go on in you know Washington DC from here without Simulating the entire earth at this starting condition faster than it can actually happen in real life if I'm that thing So I just can't effectively control I cannot predict what's gonna happen So the hundred millisecond delay or whatever it is is enough that emerging super intelligence in Washington DC Will cannot be forwarded from here. Is that is that your no you're working conjecture here? No, no, no, no I mean because you could other than the gregu the gregu which is the limit of simple suppression simple suppression I mean you prefer a regular to great just even a nuclear a conventional nuclear weapon or something like that might be enough to You know destroy a something that doesn't have time or there's not another space that it can like move itself in time If it's big and it just can't move itself quick enough, you know, you can just blow up That's not a gregu but that's simple suppression. Yeah, that's not like sending in something as complicated as you are We are assuming I think in this conversation that the super intelligence and its rivals need to be embodied You need to be in a place physically.
Oh for sure. Of course there has to be some sort of computational substrate Yeah, absolutely. Well sure but couldn't that substrate be diversified couldn't that be in the cloud or we were in the same problem? It's got the same exact problem of it takes forever for us, you know a computation to occur that is distributed across They see I mean you could do some kind of forms of distributed computation, but then you're slow slow to react right?
You're not responsive you're slow. So you by the time you finish your distributed computation Already grego. Have you heard about there's a certain building in New York near the New York Stock Exchange that all of the algorithmic trading companies have offices in because The physical distance than the speed of light matters in terms of speed of training. Oh, yeah, absolutely There's an example right there's a there's a there's a if you're across the street even you're too slow You won't get the trades you want and you're out of business.
Yeah. All right. So this is sufficiently complicated I'm ready to move on. Is that is that right?
Yeah? So I hope I hope Lyle will listen and make some more comments. Yeah, tell us how much we get us get us back on track Eric Yeah, you certainly provocative. Thanks.
Okay, so then Eric our good friend Eric responded to Lyle in the Facebook group But I'd like to read that too. All right. I'm excited natural selection works the same way replication is all that matters But you still end up with peacocks that spend huge amounts of energy on tails and fast swordfish and slow snails and 30 million other variations Why because success at replication in an ecosystem isn't all about reproduction rate It is also about long-term survival and not becoming too easy a food source for something else long-term survival is about diversity and that was why sex was invented sexuality Replicating sexually replicating speeches species are more diverse than a sexually reproducing species And so they don't end up being a big buffet for something else some of them always have the right defense against the predator against the predator or disease Does diversity of super AI protecting manatee? Probably not change is dangerous fast change particularly so the fast changes We have unleashed are all existential dangers, but a diversity of super ais may at least protect the ais not the individual ones But the population of them as a whole not reliably But it gives them a fighting chance the worst possible scenario is that humanity is killed off by an insane soup singleton super AI that also destroys itself because it is a singleton and insane a Diversity of AI gives the same AI is a fighting chance at surviving long enough to establish a stable ecosystem Humanity may not be part of the ecosystem But if we accept that those super ais are effectively our children then we can at least see some good in the scenario Oh, Eric you are a cosmos and you know I find some comfort in your cosmos Cosmis ideas here.
Yeah, well doesn't know no no live comes back. That also makes sense. You ever see a further on the roof I want to know yeah Yeah, well, I have to just invoke to be here. Okay, he's there's like this big hub up happening where you know this guy One guy's making his point very vociferously and heavy and rub his beard says you are right You are right and then another guy makes his point really like also vociferously.
Yeah, you are also right and the third one says he is right And he is right they cannot both be right you two are all right. I feel that way hearing Lyle and Eric. Yeah, you are right Yeah, that's good. All right.
So Lyle comes back and says sex was invented. Well, okay, so we'll let that slide I think Sex was invented because it provided a better source of variation than pure random mutation Sex but sex makes no sense when agents can edit their own source code as an intelligent agent can design its own variation It will have no future in an AI ecosystem And so neither will love as most our values like art love dance music etc. We're formed through sexual selection It seems unlikely such drives will remain adaptive once sex is no longer adaptive Humanity may not be part of that ecosystem But if we accept that those super ais are effectively our children he's quoting a scoring Eric there Yeah, and he answers nope ais are not our children our children are children a universe that likes humans and the things we value may as well be empty Interesting, so that's a that's a tear that's definitely a tear in perspective That's a tear in perspective which we we had an episode a long time ago that we called out Terrence and cosmos I don't think we ever really got much clarity we didn't get into it too much Introduce the concepts to maybe we should review you guys out there know about these let's let's review if you don't this is what you go to Garris I think yeah, and then and Taren the Taren point of view is that the the life we have on earth today the biological life That is what's valuable. That's the most important thing to you know Taren meaning earth violence from here, right?
That is the most important thing and if technology somehow wipes it out then that would be bad That's right the alternative is the cosmos perspective Which says the thing that matters is complexity in diversity and that the next step even if it doesn't include carbon based like well I mean I car to base like DNA based life in it even if it doesn't include DNA based life It's our moral obligation in fact. It's our almost sacred trust to bring that to fruition because it's it's next It's all Eric Eric's greater complexity here. I think is really a beautiful, you know a beautiful example of how to think about it in a way I think he might call himself a guy in though where you know that the danger is Large enough right like if you truly do value complexity in diversity Then you know that the chance that we end up with nothing which is that insane scenario he put out there right is this is an upsetting Possibility right yeah, I think about nothing. I think I think the cause miss or the Taren's window would like that No, certainly not so the the insane singleton has to be avoided all costs Which remember the premise of this show right?
We call the show concerning AI because we want to try to figure out should we be concerned about the future of humanity? You know is there an existential risk here and if there is what should we do about it? And I'm not sure that we're any clearer on either point than we were when we started well I feel pretty confident that there's an existential risk I was I was worried about before but I feel pretty confident about it now It's maybe you know, maybe it's it's not for gone conclusion No, it never was but if there's if there's a 1% chance of a legitimate 1% chance and actual not like a not like a theoretical Just made up 1% chance but like if you know like why are we qualifying what 1% means? I mean if there's a 1% chance the humanity is gonna be wiped out so Eric might be suggesting that I'd be okay So this is a great opportunity to bring up Bayesian versus frequent it frequentist interpretation.
I wish you would yeah Yeah, I'm glad that you I'm glad that you didn't roll your eyes and And show any sort of exasperation with that I wish you a cause I'm still gonna go on about it So frequencies interpretations of probability are if you run the experiment and times if you run the experiment a hundred times or like a hundred Thousand times or whatever the percentage of times you get a particular outcome is the probability of that outcome Okay, so that's the frequencies approach and in this case it would be let's say we start the world at this condition that we're out Right now today a hundred thousand times Yeah, how many times out of the hundred thousand times do we end up in a crazy an insane a I singleton that wipes everything out And maybe that's not as interesting for most people as the superset of that which is the case in which humanity gets wiped out Or all DNA based life gets wiped out how many of those hundred thousand runs there is some number there Right? So how many of those hundred thousand runs end up in humanity and wiped out? So if there's a one person if the one percent of those humanity gets wiped out that sucks that really sucks, right? Really bad even if it's even if it's you know even if it's one out of one of those hundred thousand things It's still be maybe the most important problem in the world today because it because it affects all future generations Not just like today I mean if 99.99% of the human population got wiped out by some terrible disease But the rest were immune it would go on and and all of the people who have ever lived that number is small compared to all the future generations Kind of from the perspective of the chance of a super intelligence wiping out all humanity that disease scenario might be the best possible outcome Yeah, that might prolong humanity's run quite a bit.
Yeah, yeah for sure But so yeah, I mean if you take the expected utility yep of this then any you know one in a hundred thousand chance Right and we covered this before we are trying to figure out whether any of the other You know threats that are out there seem like worthy of our time right now Like for example climate change feels like it's something that biological life on earth may be able to adapt to yeah No, it seems I think there's hardly anybody who's credible hardly anybody's credible believes that all biological life is getting wiped out by climate And let's be super clear. We're not we're not unfeeling. We're not we're not saying it's okay to you know Have a couple billion humans wiped out. That's that's not a premise.
Absolutely not right however a couple billion humans wiped out Would pale in comparison that would not even be culpable at all We're not even in the same conversation to all future human All living and all all future life that we have right now all the generations and complexity that would come from that That is hard to actually empathize with though. So that's why I'm totally that's what that's why we're having the show and we know that That's not a specific so so I would say that it's incredibly clear to me and you know It's not provable But it's incredibly clear to me that there is a significant chance that we end with no more DNA based life Yeah, and there's not much point in trying to mathematically define what significant means because it's just non-zero Yeah, is it is it more than one in a hundred thousand and I for me the answer is absolutely yes And what is the threshold where I care about it and worry about it? I don't know I'm not sure what it is But it's it's it's it feels like it's above that threshold by a long ways. Yeah, okay So I am very concerned.
There was another way to do the math another statistical percentage. What was the other one? Oh, yeah, so that's the Bayesian interpretation so Do we want to talk about how that works? You tell me it's just a method of inference where you turn things around so you say I want to predict given something given some X Y so I'm gonna turn it around and I'm gonna say in the past I've seen that when we have Y that X follows so often so it's kind of a you it's it's a way It's not you're not actually running the freak.
It's not a frequentist experiment anymore. You know what? I'm off I'm off. I'm not sure I'm actually talking about this right so I'm gonna stop all right We'll bring it up another time after I've got my We'll look into it and if you want to look into it and let us know and give us something to read We'll read it if we totally screw that up.
We apologize. Yeah, I normally love my segues But I'm really disappointed with this one. Yeah, well so much for the king of the segway. Yeah, yeah, maybe the the bigger This other one beggar in the segway.
Do you want to talk about Elon? I think we should just talk about it like for briefly Just really briefly something came up this week a little article It wasn't even that great article, but it was it was Elon talking about us provocative It was provocative in you know the news Elon so it's often click-baby and whatever But this I mean Elon is actually talking about the human human computer interface. He's calling it the neural was he calling it a neural lace or something? He's got some idea for how we could connect our exocortex to our neocortex basically right?
I think how we could be the a eyes how we could be the eyes right? Yeah, one of the very first episodes of the show We were talking about how I heard while that I thought you know I had kind of a naive ring of course well says that you know augmentation is one of the paths and you know We will evolve ourselves away from needing biology at all that will be that Augmentation will happen first and then eventually we will get nanobot foglets that will be able to create a body whenever we need one But we'll live mostly in the bio in the non-biological substrate and then eventually we won't need bodies all anymore Because we'll live completely in virtual reality That was sort of the path that he laid out and you were pretty strong I think thinking that the development of the human computer interface has got to be too slow and that the super intelligence the development of the Super intelligence just is going to be faster than that. Yeah, there's got to be there's got to be a slope So if you take a log plot right like a semi log plot where you've got the growth in one line one way an exponential becomes a straight line And exponential with different growth rates become straight lines with different slopes Yeah, so right now people are smarter general purpose intelligence than augmented people because we have all our exocortex And we're smarter than just the augmentation itself So just to be super clear I as a human being because I have all of Google and all the Wikipedia in my pocket You're smarter than I used to be you're smarter than yeah, and Wikipedia by itself not very smart It needs a human being exactly there's not a lot of smarts there you rest on so so our our computer programs and stuff are getting smarter Yep, and they are on some slope. Yep on this blog plot.
Where do those two lines cross? Well after that the human wetware part does nothing begin away, right? Yeah, no question I mean, it's not like yeah, I don't buy it. I don't know why he's excited about it doesn't make any sense I mean there may be quite a period of time where like we may be able to so even if the super intelligence explosion Happens sooner rather than later, you know, we may be able to that line still may not cross over until after that Explosion has already started and we're you know hundreds or thousands of times smarter than we are now before it crosses over That's a possibility Yeah, so if if somehow the it doesn't seem likely to me But that's if the augmentation problem is a lot easier than we think it is and the AI the super intelligent AI problem is a lot harder Than we think it is then that would be how the blinds don't cross they just don't cross until further out But they cross they absolutely clock cross and as we augment ourselves that I guess hurts of course Well, I was thinking would be that I guess if we become non-biological life soon enough then maybe yeah, he was not I don't remember exactly the timeline But the argument he was he was saying that we would not need our biological bodies by the end of the century Yeah, that seems like that would be true if the lines hadn't already crossed right But we are projecting it's kind of like those busters crossing the stream don't cross the streams are about to cross I mean, we've been thinking in the next, you know, 20 to 40 years is when super intelligence, you know singularity happens It seems like a reasonable at that.
I guess borrowing from Eliezer did Kousie I saw something he wrote that said when do I actually believe it's gonna happen? Well, if you look at how I'm acting I must actually believe based on how I'm acting that it's gonna happen within an Gave a window. Yeah, so I guess the way I'm acting I believe it's gonna happen within 20 to 40 years, right? So the augmentation I guess it's such a poserist thing Po-poser kind of thing to say because I'm not really doing anything other than this podcast hardly so I just am trying to be like Eliezer well, you know, why would I mean you remember why we're doing podcasts because before we do the podcast we didn't have any Actions to take yeah, so we so true.
Let's get together in our podcast because out of the podcast That's something some actions will come out. It's not as if we're not looking for actions We're not necessarily like daily pursuing action other actions than having the podcast But we're hoping that by talking about it there will be some actions I've actually started I've started working on it, but it's small. It's like I'm doing 15 minutes a day Well, how many of you out there are doing something? You know that you're thinking about your and you're doing something that you think's gonna make a difference in AI safety Please let us know we'd love to know if anybody's actively doing anything that you think has some bearing on AI safety That would be super interesting Yeah, and maybe and maybe we could you know sort of get the bootstrapping your AI safety career program together Yeah, some kind of play boot camp if there's some way other listeners could help you or what do you know?
What do you need next or whatever? You know if you don't want to you don't want to tell us all the details and it's seek two secrets and don't tell us that too But to tell us what you can go to the concerning AI group in Facebook and let us know what you're doing Yeah, and maybe share your ideas on what you think a recipe would be that would make sense for almost anybody Like a general recipe to become an AI weight lifter awesome. Yeah, sure that was yeah. Oh, that's cool I can't wait to see hopefully something will come up with something that I can use.
Yeah, that's my AI Awesome, all right. Well, I think I'll show Show Okay show show. All right, this is Ted. He's a brand it's not enough for considering.
I will talk to you It'll be fun. Bye. Bye I'm gonna get more sleep next time before We do this