Welcome to Concerning AI, this is Ted. And Brandon. So you may not know this listeners, but it's been a very long time since we've recorded. So.
Yeah, it's interesting. We live in a world where you can shift things in time and space so that they appear to be in other locations. It may actually work. So some of the recording's recently were made much earlier.
Yeah. So for you and there may not be a gap, but for us, it's a little bit like, no, what are we doing here and how do we do this? And that's how it feels for me anyway. Yeah, and as I look back at some of the recent we published episodes, there's one in particular I want to touch back on, which is the one with Julia.
OK. All right. Well, before we do that, maybe we should just remind people what we're doing here. Oh, yeah.
That makes sense. Yeah. Concerning AI is a show where we ask two questions. One is, is there an existential threat to humanity from Superhuman AI, from the development of Superhuman AI?
And that's one of the things I want to talk about. Yeah. OK. And if there is a threat, then what do we do about it?
And then what do we do about it? Yeah. The idea possible question. Yeah.
So we published a show on December 19, 2017. This is episode 56, where we interviewed Julia who of Lark, which is an AI health coach. And that's the one you wanted to go back and revisit, right? Yeah.
Yeah. And she says something we were talking to her about Marvin Minsky and said there is no AI. And this, Rod Brooks was another one. These elder statesmen of the AI field, poo poo.
Many of them, not all of them, but many of them, poo poo the idea that there's any risk or that there's any chance that we're going to actually create a strong AI or a general intelligence anytime soon. And by any time soon, they're actually talking about long time periods of hundreds of years, not anytime soon as in, not within the next two decades. It really is a long time frame. So I think Julia was talking about her team too, kind of having that same feeling.
Yeah, we asked her our prediction questions. We asked her, what year do we think there's a 5% chance that we'll have developed superhuman AI? And she said 2087. So 70 years from now.
A 5% chance. A 5% chance. A 15% chance, she thought 2137. So 120 years from now.
And then we didn't even bother to get a 95%. Because at that point, we just have no idea. Right. And I don't think it's an outlier.
I mean, I think that there are, it's like a bimodal distribution or something where there's the people who are the, we're nowhere near. And that means hundreds of years. And there are the people that say, we got a while yet. And that means decades.
And in our community of listeners to the show, and people that contribute to the Facebook group, it looks like an outlier. She looks like an outlier to us because we only attract and keep the people. It's so biased, right? Yeah, there's a really strong collection bias for our group of listeners.
Yeah. And so I know I've touched on this before, but I want to revisit it again. I think that it's really easy when you've worked really hard and failed to come to the conclusion that no one can succeed. And these elder statesmen, and in some ways even, you know, Julius company, I mean, you work really hard and you have these bright visions of what you're going to accomplish.
And you accomplish something wonderful, but it's still nowhere near what you've visioned a lot of times, you know? And you know what it's like to be inside the sausage factory. And it's a little disturbing. And then people make up these hyperbolic claims about the things that you created that you know are not true.
And so then you start to question every other claim out there. Yeah. So I think there's a number of factors that go into this. How many is that that we decided to find there?
How many factors? Yeah. There's the I tried and I'm smart and I tried really hard and it didn't work for me. So nobody's just going to work for anybody.
Right. And then we covered that when we talked about the Rodney Brooks article. Yeah. A few episodes ago, episode 51.
All right. So that's one factor. It'd be nice to have a nice little name for that factor. I tried.
So it can't be done. There's that one. There is the I did something. And then people made a lot more of it than it really was.
Therefore, I no longer trust any of the claims. Yeah. You know, it all feels like sensation and hoopla. That's a clickbait one, right?
It's kind of like a clickbait one. Yeah. Well, it's not just, I guess maybe it's maybe it's a clickbait. It's also kind of a people don't understand.
Like they just don't understand the limits. What was that bias? We really need to look up that bias again, where we see performance in one dimension and then generalized dimension. That's dimension.
So that bias, I think, strikes and then people start to, you know, they take that instance, their own instance, and then they do the same sort of generalization. If in my instance, this wild, overblown thing is going on, then that's probably what's happening everywhere. Yeah. And I wonder how similar this is to like heavier than air flight, for example.
So at the time when heavier than air flight had not yet occurred, I wonder if there were elder statesmen in the engineering field who had tried and just come to the conclusion that it was in place. Yeah. I don't know enough about the history, but I imagine that there was. Yeah, I think that it was probably a bit divided as well.
So probably even up until the first demonstration, I'm guessing that there was this polarization happening about that. Well, right. And was the same true for rockets, for humans leaving our atmosphere and orbiting and going to the moon? Are there a same kind of dynamic playing out then?
That's funny, because that's not one of the ones that gets trotted out. I think the human flight one is an example that gets trotted out. But I feel like the rocket one, people, I think anybody who knew about it didn't doubt the possibility. That's weird.
That's an interesting difference. It's an interesting difference. And I think probably human flight is what paved the way for that optimism, because those two are very optically similar. So I think this relates to alpha zero and virtual reality, developments in those areas.
Because I'm starting to see glimmers of a way forward. All right, so let's review. Before we start talking about what the implications are of alpha zero. So sure, most of you who are listening know about alpha go, and it was a big deal when DeepMind was able to beat Lisa Dole four games to one.
Lisa Dole was one of the best, if not the best, go players over the previous decade. This was in the spring of 2016 when nobody predicted that this was going to happen for at least a decade. This was just unbelievable. Just for background, if anybody wants to learn more about this, Netflix just made available the AlphaGo documentary just last just this month, just January of 2018, at least for the US.
I don't know what it is like the rest of the world. But I watched the documentary and it was fascinating to see Lisa Dole interviewed just before the match has started. They played a five game match over the course of, I can't remember, seven days or something or 10 days. And before the first match, he was super loose and super relaxed.
And he just said, it's going to be five to zero and I'm going to win. And then he lost the first game. And at that point, he said, well, I think it's probably 50-50 if I'm going to be able to win the second game. His prediction changed dramatically.
And then he lost the second game. And in the second game, there was a move played that people in the go world didn't understand that AlphaGo made this move that they said the commentators in all the languages in the documentary. They had an English language commentator. They had a Korean commentary that I think of maybe Chinese language commentator.
And all of them were just flummoxed by this move and they couldn't figure out AlphaGo did it. And that's the move that won the game for AlphaGo. And so it's just fascinating that here we have a machine that can play moves that humans don't understand. At least I ended up losing the third game and then winning the fourth game by playing a move that most humans wouldn't have found and that AlphaGo didn't think anybody was going to play.
And the final score was four games to one, AlphaGo won four games and at least it only won one. So this was almost two years ago now. But there have been some more developments. And Brandon, do you want to talk about AlphaZero?
Sure. You know, AlphaZero has been applied to several other challenging strategy games, most notably recently chess. And in the process has absolutely demolished the best computer chess program in the world. And the thing about it that is both exciting and frightening to me is that it is very close to a human-style game in a way that the computer games are not.
And example, let me just explain what I mean by that. So AlphaZero playing chess searches through like three orders of magnitude fewer moves per second than then Stockfish, which was the state of the best player before that time. So it is evaluating positions a lot more effectively than Stockfish is. It is less about exhaustive search and more about aesthetic and judgment.
And that is part of what has set up chess and Go and some of the other things as human paragons, like something that we look to, people who are good at this. And we definitely apply terms that we also apply to art and other aesthetic works. So let's back up though, just a second. I think this is really important.
20 years ago when Gary Kasparov lost to Deep Blue, everybody thought that it would be a big deal when chess was defeated by a computer. But then the commentary afterwards, and ever since it has been, well, you got enough computing power and you just do an exhaustive search and it is not about art anymore. It is not about actual understanding. It is not about learning.
It is just throw a bunch of processing power. Although the Blue was programmed specifically by chess experts. There were a lot of chess experts on the team. AlphaZero did not have any chess experts on the team and it was not taught anything.
It was just given the rules of the game and it figured itself out for first principles. The same with AlphaZero that played Go was massively better than the AlphaGo that beat Lisa Dole a year later. And it learned it in something like three weeks. Yes, and AlphaZero had exactly four hours to learn the chess and that before it played.
Four hours? Oh, it was. It was played a stockish. Wow.
And so what is it about AlphaZero that is exciting to me? So the deep learning is modestly exciting. The idea that we have neural networks with many layers and you are learning some generalization and other things. That is modestly exciting.
But that combines so well with reinforcement learning. That the reinforcement learning is the piece that has always seemed like if we get that to work has the greatest potential to lead to general intelligence. Because it is about these sequences of these causal chains where you are learning good things to do to bring about a desired state in the world. And yes, chess is fully visible.
The state is fully visible. So is Go. So these are simple worlds. But the worlds that are simple are getting more and more complex.
And by that I am thinking about virtual reality and the physics that are involved. You know, everything about it, it seems like we are getting closer and closer to the point where learning that actually maps over to the physical world that we have all been is getting closer and closer to being possible at a very rapid weight. Okay. But wait, but wait Brandon.
I hear some listeners asking. It is not as if AlphaZero took something it learned from chess and applied that to Go. There is no generalizing here at all. It is still just narrow AI pointed at a specific problem.
And then even though that engine was used again, it is still narrow AI pointed at a different problem. And nothing is generalizing. What do you mean when you say there is a possibility of using this kind of technique in order to build the general intelligence? Yeah.
Well, so I don't know that that is actually true. You just said there. Because the meta parameters, like how many levels, what sort of how many nodes per level, what is the function that you use to combine the inputs at each level, like all of those meta parameters have been and are being learned. So those are turning out to generalize really well across these domains that are fairly similar.
So I don't think it is accurate to say that this isn't learning. We are starting from a much better place than we have ever started from before because of that learning. But a human being who has played Go but never played chess is likely going to approach chess differently than somebody that has never played either. Don't you think?
Yeah. Yeah. I think there is probably some. That is the point that I was pointing to that I don't.
They are still not going to be very good for a while until they put in the human equivalent. I mean, they will learn it faster. But they still have to put in there a thousand hours or ten thousand hours or whatever to get up to master. They still have to put in a huge amount of time learning.
Right. But as a general intelligence, the fact that we know something about games, I don't know. I guess that is a question. Maybe that is maybe a premature making a statement that we are generalizing from playing one kind of game to playing another kind of game.
So I think that is a valid criticism. I think that is true of AI and machine learning in general. It is still narrow. No doubt.
But what does it need to become broad in general? And I really believe that for it to be recognizably generally intelligent and have a conversation with us where we feel like we are talking to something that understands the world and the way that we understand it, it has to be able to interact with and learn the way that we have learned. So we have millions, maybe if you go back far in a few years of evolution optimizing us to interact with this world. And we have our infancy and our toddlerhood.
And this time where we, so imagine that as starting with great meta parameters. So alpha zero is starting with great meta parameters. And then it learns the actual values of all of the links in the network. So the meta parameters are the thing that has had this really long optimization process.
But there is still this time when we are infants. There is still this time when we are toddlers. I think that we are getting to the point where we are going to be able to learn those meta parameters in a simulated world a lot quicker than we did with evolution. Like really quick.
And as we learn those, those things are going to build up. They are going to build on top of each other. So that is the part that Julia who kind of rejected. We didn't directly ask for this question.
We didn't ask for it to delve into it. But when she considered our question about super intelligence, she answered with something like, well, our work, it took us three years just to get enough training examples to get the thing to work at all. And so I feel like she is thinking linearly in that moment or maybe less than linearly. The improvement that they are making is not exponential.
And you are talking about the idea that we could build on the wins of the past and that they are going to add to each other. That is an exponential statement. And that is the disconnect. And I don't know.
Maybe this goes back to what you were trying to cite as the types of problems that gets in her way. I tried it so it can't be done. Maybe she is thinking linearly just because the marketplace is really, really hard. Getting to a place where somebody will actually pay you for the output of your thing is ridiculously hard.
And that is another reason why you don't extrapolate your work into a super intelligent future. Yeah. Well, and there are two very different approaches to engineering these kind of things. And one is a very engineered, like a human engineered decision tree sort of approach.
And for anything in the medical field, you are going to end up taking something like that because you have to be able to demonstrate why this thing came up with the conclusion that it came up with and that it is based on some sort of expert analysis. So that is the sort of approach that they took at large decision tree sort of approach, which is again, a very manual human process. Because of the medical realities of regulation and finances and everything else, it is the way you have to do it. That is the way you have to do it unless you are maybe IBM and maybe you have some deep pockets and you show some really good results that come from the surprising direction.
But the other way to get at that. And I think almost all of robotic has been in that camp. So almost all of our embodied AI and learning efforts have been in the very engineered. And that deep blue came that way too, didn't it?
Sort of approach. So deep blue is, I don't want to put deep blue in that category because it is not like there was a lot of manual engineering. It was just a brute force. It was sort of a brute force search that got to the point where there was enough computation.
Right. But they took a lot of openings for example. Fair enough. Right.
Exactly. That is a good point. So deep blue may be also in that category. So that is the one approach.
The other approach is having a sort of a generalized learning framework and finding better and better starting places where you and broader and broader applications you can apply that. And when you can beat the best computer in the world at chess after four hours of training, you've arrived at a new plateau. You're at a new point that we, the human species was not at before. And that's significant.
And it's those approaches. Those are the approaches that are going to get it to. Not the ones that Rodney Brooks took. Not the ones that Marvin Minsky took.
Not the ones even the Julia has taken. Those approaches are the ones that will get it there. Not these manual human compilation and feedback and control mechanisms that again are devised by a human mind trying to come up with some functions that work for a particular project. So here's something that I found fantastic and really amazing when thinking about Alpha Zero playing chess and doing what you just said.
It's no longer can we say, well chess isn't as, you know chess playing humans are not as intelligent as we thought before. This isn't, you know, that was one of the criticisms that came after deep blue. Oh, it's just, it's just computation power. But what this is showing is not that computers can be humans at chess.
It's that Alpha Zero can jump ahead of 20 years of computer researchers in AI. Right. That's the breakthrough. It's not about chess.
It's about AI. Yeah. And the breakthroughs we need stacked up to get the general intelligence app this moment. I don't think we do.
No. But we had a big one just recently that I didn't expect and that's the Alpha Zero. Yeah. And you know, it doesn't take too many more of those and it doesn't take too many more enabling technologies like really high fidelity virtual reality for me to be able to see a pathway forward that has that sort of well-foundedness that can grow and that can build on itself rather than the asymptotically tailing off that you get with human engineering.
Yeah. I mean, I think that's a big one right there. One group of people, one group of experts sees us asymptotically getting diminishing returns for our effort. And they're right.
But the other group sees us getting exponentially growing returns for our efforts and it's because the two approaches are different. Yeah. Sorry. I interrupted you.
No, that's okay. No, I think what you just made was an important point. I just wanted to alert people to a book that I am in the middle of now called Otherworld. It's a very fast read.
It's a 2017 publication novel. I don't want to go too far into the story because I'd like people to have a chance to read including New Brandon before we spoil it. The fact is we spoil everything on the show. So you just have to read everything now.
And so I'm going to hold off on spoiling it on the show. But one of the interesting things about Otherworld, Otherworld is in the story, a commercially available virtual reality world that you can go into. And one of the things that happens in there is that people who have played other types of games have a little bit of an advantage because they know kind of where the risks might be and they know kind of how games work. They know how worlds work.
Not only do we have the same physics in Otherworld that we do in our world. And so our human brains, our human brains are generalizable into working in the virtual world. But having some knowledge of how puzzle games and, you know, MMOs work, that I thought that was really it was an interesting, you know, tiny little piece of the book that you just kind of know what to look out for. And of course, that's not always true.
A game could be designed in different ways. But if a game is designed by people who also have the same frameworks or what a game it looks like, then- And the story telling bias. Yes. It's the game designed by us.
The people that designed a virtual reality game only have experience with certain kinds of games in their own life. And so they built the virtual reality in a way to conform to those. Now, it doesn't have to always conform to those. But imagine if it only diverges 5%, well, then the game player only has to learn where the divergence is and doesn't have to start from scratch.
Right. So they already start with a useful body of knowledge. Exactly. Exactly.
So what else are you thinking about in terms of virtual reality and how that can influence where we are in terms of heading toward general intelligence? Well, you know, as you start to, you know, I come back to, you know, see what is Facebook. I don't remember. I remember seeing some, I guess maybe it was actually DeepMind possibly where it was learning to run through a virtual world, the sort of person-weight looking thing.
And it was really goofy the way that it ended up running. I think though, as you know, that's not a very high fidelity world that learning is happening in. And you could increase the fidelity of the computer graphics and that would take a lot of computing resources. But you could also add some higher fidelity in other dimensions that wouldn't take very many computational resources that would have a tremendous influence, I think, on the quality algorithms.
For example, you know, energy usage. You know, that's a huge one. Like evolution has clearly optimized features for energy conservation. So so many of the things that we do, so many of the ways that we move are highly efficient from an energy conservation standpoint.
So that's an example of adding to the fidelity of a simulation without taking tremendously more computational resources. I think there are probably other ways that are interesting as well. And some of those ways I think have to do with the motivation landscape. Like what is the thing trying to accomplish?
And you know, like you think about what a baby is trying to accomplish. I don't know how hard it is to create a value function that is similar to what a baby is doing. But I don't think it's that hard. Basically, you know, picking things up, waving them around, putting them in your mouth, right?
And if you do those things, that's successful. And if you hit yourself in the head, that's not successful. And if you fail to pick it up, that's not, you know, I mean, it's kind of, I think there's a fairly straightforward value space that has us starting to generate this kind of affordance landscape of that's something I can pick up. That's something I can sit on.
That so many of the meanings that we use day to day are based on that are sort of bootstrapped up from. So back to the deep mind, the people that did AlphaGo, you remember that they started with Atari games, but they used like breakout and they used all those other like, I don't know, space invaders or those other kinds of games where they just gave it the raw pixels and they gave it the score, right? And they didn't give it anything else. They didn't tell it.
It doesn't know what a paddle is. It doesn't even know that it has a paddle or that it has controls that can move the paddle. But within a matter of hours, it figured out through random, through random operations that it could increase its score by making the paddle line up with where the ball was going to end up. And within less, I don't know how long it took.
Less than a day, I think it was better than the best human players. And so this idea of the value function is super, super powerful. Yeah. So the value function is you have to have an algorithm that is fast enough at convergence general enough to find non-obvious maxima.
So those two things are difficult to come up with. And that fitness function that is sufficiently rich to guide you toward a meaningful conclusion. And so we don't have all of those things yet. It doesn't really does the three, but it really did.
That's enough. And the baby is so complicated in so many ways. If you think about the value function of DNA, if you back up and go on a different level instead of the level of the baby, think of the level of the DNA, what the DNA is trying to do is just get passed on. Yeah.
And yeah. That's a very simple fitness function. It's a very simple fitness function. Yeah.
And it could be, well, it absolutely is now with birth control in our modern age, in the last, you know, whatever, 100 years or however long we've been practicing family planning at all, we on the level of a human being are acting in ways that are contrary to the fitness function of our own DNA. Right. So we've in a very real way are choosing something else. We're choosing to act on different value function, whatever that may be.
And then you can't put a number on it necessarily. Not the way we think of it today. It's interesting. Yeah.
What is the, yeah, it's kind of like we are thwarting what has been in charge for a long time, which was the DNA. I guess it's still probably as long as death exists, I think it's still probably the DNA that's in charge. Really. That's that fitness function of evolution and passing on genes is probably still going to be in control as long as...
Well, it's true, but there's a very strong cultural layer on top of it, which seems to be acting in a different way. I mean, for men, don't need to as sperm bank would be the way the DNA would, uh, seems like that would be the best strategy for DNA. Yeah, and it probably is. And that's not maybe that's not one that we at the memetic level of culture talk about.
It's maybe not something where we even admire people or anything like that, but it's probably where the most effective selection is occurring at this time that where the most future copies of genes are coming from, that type of thing. Like just because we think about something doesn't mean that it actually has, can phase any sort of evolutionary survival, you know, survival or passing on of genes advantage. Like that's a hubris, that's a vanity. For sure, but there's passing on of genes, which seems to have always been in charge, and you know, you're making the case maybe it still is in charge, but there's also the passing on of memes.
And we've talked on the show quite a bit about the aesthetic preferences that human beings seem to have and we're moving towards including more people in our concept of we, and the idea that there's more and more justice or opportunity for people in the world and that doesn't, I mean, that's, that's on a level of memes, I would imagine, not on the level of genes. I don't know. Yeah, I think that there's a fair amount of debate around that topic. And on any given day, I can argue fairly ignorantly and passionately about one side or the other.
All right, well maybe we'll leave that to another day. Hmm. All right, let's see if we can recap this rambling sort of. Is there a thread that ties us all together, Ted?
Well, I think that there is there is reason to understand why some people, some experts in the field of AI are not at all concerned the way we're concerned. There's nothing wrong with them. There's nothing defective about them that has them thinking that way. And yet we are seeing some breakthroughs that also leave our conclusions about superhuman AI and its possibilities also very well supported and that these breakthroughs are not necessarily what those AI experts are dealing with on a day to day basis.
Right. They're not necessarily the diminishing returns sort of, you know, iterative evolution of a human engineered solution. Yeah, I saw a picture which tried to explain what a different difference of point of view, how different points of view make a big difference and there was two people sitting at a table and there was a glyph in between them and one of them was saying, six, and the other one was saying, nine. Right.
Well, that's a difference of point of view. That's that seeing the same object but interpreting it differently based on your point of view. I think we're talking about something on a different level. And it's similar.
It's like Julia who from Lark is simply looking at different parts of the world. It's more like the blind people touching the elephant, I think, that some of them feel the leg and say, oh, it's like a tree and somebody's touching the tail. It's like, no, it's like a rope. It feels a little bit like that.
Yeah, there's definitely an element of that I think that plays into it as well. And I think, you know, I guess I want to say that they could, that they have a very high chance of being right as well, that it is of diminishing returns that we are hundreds of years out, even if no catastrophe set us back a long way. I do think that there is a high chance of that. I mean, but there's still this non-trivial chance that they're wrong.
Yeah. I think we're probably going to need to do a future show coming back to the math again and thinking about this idea, you know, well, what if there's a non, you know, like, this is a fundamentals of science. Is there a non-zero chance that, you know, I can walk outside and fly up in the air? Well, maybe, you know, like quantum, whatever, I mean, a non-zero, but it's actually not going to happen.
That's different than the non-zero chance we're talking about, super human AI coming along in our lifetimes. Yeah, it's not a, I mean, it's not even a one in a hundred. It's on that order. It's on the order of one in a hundred, ten in a hundred, thirty in a hundred, you know, it's in that range, I really think.
And that's terrifying. That's huge. Yeah. All right.
Well, let's come back to that. If folks want to be involved in this conversation in between episodes, well, yeah, in between episodes and join us at Facebook, go to the concerning AI group, get yourself signed up, generally approve those requests within a day, you'll get a chance to give us your AI prediction, which I'll do about Julia's prediction on the show. So you can give us your predictions. Join us in the group.
That's a lot of fun. If you're not interested in this, please send us your feedback, record an audio memo in your phone or your computer or everyone to do that and email it to concerningai at gmail.com. And periodically we have listener feedback shows and we'd love to have your voice. Keep it to under a minute or so and you're much more likely to actually have your voice heard on the show.
Any other last words? This is fun. We should do this more often. We should.
Yeah. Thanks for, thanks for spending time with me on this Ted. This was really fun. I like spending time with YouTube, Brandon.
And I really enjoy having this conversation, this wider conversation beyond just the two of us. And the fun of it was just the two of us. But the fact that we get to share with the world is also a great aspect of this. I appreciate that about you as well.
Let's have our robots be more likely to do it. I'd like to. Alright everybody. Until next time, keep your coffee up.
Keep your coffee up. Bye. Bye. We can't say bye at the end.
No, I always cut that out.