Welcome to Concerning AI, this is Ted. And this is Brandon. This is the show where we ask two questions. I forgot what they are.
It feels like I'm supposed to jump in there. When you say that. Should we be worried about AI as an existential threat to all life that we care about currently? Oh yeah, that question, right.
Yeah. And then what's the second question? What do we do about it? Oh yeah, boy, that one.
Yeah. Well, I think we're going to have a little bit of a short show today. Just life happens sometimes. And so today's going to be a little bit of a short show.
I think we should check in with that first question. Because it seems like we kind of implicitly frame that. We implicitly answer it. And every now and then we explicitly answer it as well.
But it seems like we should check in with that every now and then to me, like explicitly. And it's been a while. Yeah. And there's an article that I came across recently.
It turns out it's three years old. So it's actually older than the show. No, it's two years old. It was about the time we started the show, actually, in 2015.
But it's an article from Romesh Nam, who wrote the Nexus, the Nexus trilogy, which we have read. And we've talked about on the show before. Which was really quite interesting. Yeah.
And we won't be talking about probably not spoiling that too much tonight. But as we said on the last show or two shows ago, really, you should just read everything. Because we're going to spoil everything eventually. Yeah, it's true.
So if you haven't read that, go ahead and read it. Us or our AI as well. One of the tips. Exactly.
Exactly. So anyway, the article is called The Singularity is Further than it appears. And this is from a guy who wrote a series about an AI that woke up, about a super intelligent AI. So kind of interesting to think about, we've talked about timing before.
But what do we think now? Yes. And who knows whether anything actually in the world has changed to make the change of thinking. But maybe.
So let's talk about it. Yeah. So it's interesting. I read a book this weekend.
And it was a short book. It was kind of like an Amazon thriller. You might be called Natureist or Naturalist or something like that. It is a serial killer kind of thing.
Anyway, it was a little bit forgettable. But there was AI. And the protagonist, it was OK. It was funny.
It was short. The protagonist uses an AI. And as a, you know, someone who is somewhat connected, admittedly a bit in the past, but somewhat connected with machine learning and AI research, I found it just wildly implausible that his AI would work in a way that it did. He's given it's really under constrained problems, coding these very abstract and difficult things up very easily to express them to the computers.
It's a lot like the, it's a lot like what you would see back in the 50s and 60s where robots would be real smart. And yet there was, you know, your basically flipping switches or your, yeah. I remember seeing this one where this basketball team is, the owner is a real techie. You know, he wants his team to win.
So he goes out into the computer lab. And that guy is flipping switch. I'm saying now that we need somebody and a computer saying back, you know, this like sort of oscilloscope looking thing coming out of the computer and that it shows these like running bodies. It's just absolutely ludicrous that that would be how computers work.
And I felt that way reading this. And that kind of made me question again, like how much of the hype around this particular AI bubble is hype and how much of it is new due to substantial progress? And at the moment, I'm feeling skeptical a little bit. Today in October of 2017 is when you're feeling skeptical.
I'm feeling a little skeptical. I just know there's a lot of hype right now, a ton of hype. And you know, there've been a couple of modest breakthroughs, many of which are driven by the platform, you know, getting faster, some of which are actual, you know, genuine progress in the research. I don't know why I feel like we're closer now than when I was back in grad school.
Back then, I felt like 75 years was very minimum before we would be anywhere near a dangerous spot, given the shoddy state of research and how little we know. But now for some reason, I don't feel that way. I haven't been feeling that way. And I just wonder if, you know, Rames and these other folks who are talking about how overhyped and over sensationalized this stuff is, you know, if they aren't more right than wrong.
Yeah. Well, why don't we look a little bit at Rames's argument? I think that might be interesting. OK.
Yeah, I haven't seen this. So in the article, he talks about how do you say Werner, is it Vengue? It's Vinje. Vengue.
Werner Vengue's intelligence explosion, how Vengue wrote in 1993 that within 30 years, we would have super intelligence, super human intelligence, you know, super human intelligence. So 30 years from 1993, if we do some quick math, that's six years from now, got to say, doesn't look like it's happening. I'd be surprised for sure. It would be pretty surprising.
Yeah. He talks about the hard takeoff. And early on, maybe two years ago, we talked about maybe that should be a topic for us, fast takeoff or slow takeoff. He talks about hard takeoff.
We never did come back to that. So I don't actually know much about that. But there's some graphs and they seem kind of compelling about size of molecules versus computation time in chemistry and biology and all that stuff, which makes it look really, really hard. So even though computing is growing exponentially, the problems are growing looks like double exponentially.
And the idea of self improving AI has got some more graphs in there about how if an AI was designing intelligence that is two times as intelligent as the entire team that built it, then that AI would only be able to design a new AI that's 40% more intelligent than it's old stuff. And then the next one would only be 17% more intelligent than that. And so it would basically top off. So he's saying it's a sigmoid.
They're not a sigmoid. It's like a asymptotically increasing rather than exponentially increase. Exactly. So that's interesting.
So why does he say that? What makes him say that? I don't understand the math. I used to understand math, but I've been long away from math.
So I don't really understand. It looks like it's scaling it n squared, which is far worse than linear. That intelligence looks like an n squared problem. But I don't understand why.
Far worse than linear. That wouldn't be the case. I mean, n squared is what is he saying here? Oh, it's four times as hard to develop two units of intelligence as it is to develop one unit of intelligence.
So the difficulty is the difficulty of developing a new unit of intelligence goes up as you get more and more intelligence. Interesting. That is a very interesting idea and seems plausible. Seems very plausible.
Yeah. It's interesting. Then he goes on to say we already have super human AIs. Or we have super intelligence.
We basically have corporations. These are entities with vastly greater than human intelligence working on the problem of augmenting their own intelligence. We call it a corporation, which is really fascinating to think of a corporation by that definition. Sure.
I thought of this very familiar and makes a lot of sense to me. You have these superorganisms that are made up of other smaller individual organisms. Exactly. Exactly.
And we know today that a corporation or really any grouping of people can do more than just individuals. The Intel corporation, the Google corporation, the Apple corporation, they do more than any individual in them. They're way smarter than one human. And they're always trying to get smarter.
But we don't see a runaway. We don't see them producing super AI right now. We don't see a takeoff at all. Yeah.
So he makes the point that there's a huge lack of incentive to make anything that's resembling sentience at all. And yeah, I think that's true for the most part. It's not right where the money is ever. So I guess it has to be an accident.
Yeah. I don't know. I think that there is always an economic incentive. If you could make an AI that could be smarter than the stock market, smarter than everybody that's playing the stock market and therefore make money that way or smarter.
It doesn't have to be sentient to do that. No, it does. But I don't think that helps us. If super intelligence, I don't know.
We're going to have to talk about sentience again at some point. We talked about that early, early on in our fundamental series way at the beginning. I feel like we need to go back to the beginning and talk about sentience and consciousness and what is intelligence even. I feel like we need to go back to the basics on all this stuff.
My colleague is always saying, time is a flat circle. I never know what he means about that. But somehow I feel like it applies. He's making a joke.
I think I wrote a TV show reference. But I snuck it in. I have to tell him to work tomorrow. I snuck it into the show here.
So yeah, time is a flat circle. We're coming back around to the beginning again. Right. Yeah.
Well, I mean, can we get on the record with kind of what we feel intuitively before we go over the basics again? Yeah. All right. Just sort of just sort of like gut check.
I don't remember what I said last time. All right. Well, just what is the framework? We talked about three numbers.
And in fact, anybody that joins the Facebook group has asked for their three numbers. And the three numbers are, where do you think there's a 5% chance? What year in what year do you think there's a 5% chance that we will have super intelligent AI? And then what year do you think it's likely that we'll have super intelligent AI?
Sort of the 50% mark of your curve if you're going to graph out where you think it's likely to happen. So give us your far left, your 5% mark, your 50% mark. And then what year do you think it's 95% likely that we will finally see super intelligent AI? So we're kind of trying to map out what the whole curve looks like with those three numbers.
OK. Right. Yeah. All right.
So who's going to go first? I'll go first. All right. I'll give you my three numbers.
You're going to influence me here. Yeah. Well, maybe you should write yours down. Maybe we should do it.
It's good scientific about it. You know. Pause the AI that's doing the recording for just a moment. Yeah.
Yeah. Let's do do do do do do do do. See, now there's a spot where they beat us. Oh, we've already lost that one.
We've lost that one to a non-sentient intelligence. Yeah. So should we do this years from today? Or should we do it in years?
Should we do it in years? In clock calendar years, right? So it doesn't move. Yeah.
We should put a date on it. You are genius. Yeah. You should do a show.
Yeah. We should do a show. That's a good idea. We would make a sure.
We should totally do a show. We're better together than we are individually. Yeah, because we're super organism. Yeah.
Yeah. All right. I'm ready. You ready?
Yeah. I want to round my numbers just a little bit, though. Give me one second more here. OK.
I'm going to round my numbers. OK. I'll round it off. You've run a lot so they're no longer time of day on a particular day?
Mm-hmm. OK. All right. All right.
I was down to microseconds. I'm taking chemistry so I know about significant figures. Excellent. Excellent.
All right. Well, I'm going to go with 2027, 2042, and 2099. Those are fascinating. Yeah.
You're seeing more scarieness than I am today. All right. What are you seeing? 2070 and 2110.
OK. Interesting. So I'm giving it almost 100 years to feel confident. Yeah.
Well, we're only basically saying I have no idea, right? I mean, that's just been saying a long time. Yeah. And that's really 11 years apart on the longer side.
Yeah, because both of us are saying we don't know. It's a long time. Right. That number is only interesting if it's close, I think.
The 95% threshold. Yeah. Any of them are there, only interesting if they're somewhat close, because then we feel like they're kind of imminent. Yeah.
But we're disagreeing with Rema as well. Right. We just started off with that article. He didn't say directly in the article, but I tweeted at him afterwards to ask.
And he responded that he says century or more. Maybe centuries. Yeah. Greater than century, maybe multiple centuries.
Right. And I'm guessing that that's the 50% number for him. Although maybe century would be the 5% number for him. So all right.
So what's your 5% number again, 2027? Yeah. 10 years from now. 10 years from now.
Yeah, 2007 to 2017 doesn't feel like that big of a difference. But yeah. Wait a minute. Think about it.
Think about it. That was when the first iPhone came out. Like the idea of the smartphone, it was very rare. Not everybody has one.
And then everybody. Andy, and Uber, and Lyft, and all of these, yeah. These things that just overnight overturn how an entire industry works. Yeah.
Not just any industry, but a very entrenched industry. Yeah. Yeah. Those are some, those are some, some point you get some massive technology shifts.
Yeah. How many billion dollar companies have been created just in that last 10 years? Like Snapchat. And I don't know when Instagram started, but it might be less than 10 years old at this point.
So, yeah. I mean, you know, I was, I put that as my 5%. I don't. I don't think that's terribly likely.
But given what we saw with AlphaGo, and then what we saw with AlphaGo Zero, where DeepMind started over without giving any go games to study, just let it play itself from, literally from first principles. And it was able to, within a matter of weeks, be better than the original AlphaGo. That was pretty sounding. Yeah, are we going to be able to constrain the problem space in such a way that a computer can do that kind of thing?
I mean, go as a perfect knowledge game. I agree. I agree. And very well.
But there's still more possibilities in that game than there are atoms in the universe. Yeah, no doubt. And it made us think that we were special, that it was our sort of special aesthetic sense that made it possible for us to play that at such a high level. Our general intelligence was required.
Wrong. But we were wrong about that. Yeah. Yeah.
Well, see, I ended up thinking about it in terms of computational cycles. Like, when do I feel like the platform will be coming up close to human level of computational cycles available at a decent price point? And I kind of feel like that's around 2040 maybe. Yeah.
I mean, who knows what will happen between now and then. There are some massive breakthroughs on the horizon that could happen never or that could happen relatively soon in particular in quantum computing. That's the one that I know of. So wait, so you're thinking that super intelligence will just come along for the ride once we've got enough hardware?
I think that certain approaches become easy that we're not even easy to think about. And I don't know. I don't know. I don't know.
I always come back to the analogy of like a Z-buffer. Yeah. How computer graphics got just hugely easier once we had a Z-buffer. Enough memory to have a Z-buffer for this way.
Okay. Suddenly, algorithms and everything got simple to do the things. It's really hard to keep track of all the state and doing all these complex calculations. Really easy to get things wrong.
It's just this hard. Yeah. Really hard. And then suddenly, you know, anybody can do it.
And it's easy. So I just wonder if there won't be something like that. Okay. So if this were the case that at some point we'll have another Z-buffer moment, that would be a reason why the neural nets V1 and V2 didn't really work out for humanity.
We're now we might be in neural nets V3 and we've moved forward with hardware. So maybe we have more of a chance this time. But there might still be a Z-buffer moment in our future. Is that what you're kind of thinking about?
Yeah, that's what I'm thinking about. And I would expect it to be kind of at the earliest, I would expect it to be around the, you know, when the cycles available to our computing systems are within an order of magnitude of what we think humans possess. Yeah. Hey, by the way, quick a sign about the exponential.
I clicked on a Facebook ad the other day, which I vow never to do it right now. I always tell people I don't click on ads. Well, I did. I clicked on one the other day.
And then I filled out the form and they're sending me a kit where they're going to take my gut bacteria DNA and they're going to analyze it for me in order to, I don't know what they're going to tell me. They're going to give me some diagnosis or some kind of plan for feeling better based on the microbiome inside my individualized gut. And holy crap, we're sequencing everybody now for free on the internet. So what you don't realize is that now forever after they're going to have you and all the insurance companies and everybody's going to know whether to fund you or not.
I know I signed a document. I know I gave it right to them. They're going to sequence my gut file. That's our bottom of my broken ladder friend.
Yeah, it's crazy. How much was the first genome to sequence? It was a billion dollars? No, billions.
It was billions. And now I can click a link on the internet and give away my DNA for free to the internet. It's amazing. Yeah.
Well, see, that may be one of the reasons why AI never happens. Because of synthetic biology destroying us in a simple way before we have AI. Yeah, it's so hard to isolate any one of these trends because they all are going to influence each other. And the exponential applies to them all, it seems.
Yeah. I'm just going to pass it to you today. I'm going to say the sigmoid. Yeah.
That's a big difference, right? The sigmoid versus the exponential. Like, well, it's true. I mean, it's true.
But the technological, if it is a sigmoid, we've never, why would we think that we're living today in the time when it's going to flatten out where the rest of human history it hasn't yet? Things are getting pretty small. I mean, we're getting down to quantum effects in our transistors dominating, right? I mean, we're getting down to, we're getting down to spots where transistors are really hard to model because they're so, they're no longer kind of classical devices.
They're these dealing with individual electrons matters. What size are transistors compared to neurons? Ah, that seems like something that should just spring right off the tip of our tongues, doesn't it? Hey, we're the co-host of Concerning AI.
This is stuff we should know, don't you think? Yeah. So can you edit this, right? Do you ever have time out and give us a chance to go look something up so we seem smart?
I never, I never do. No. Yeah, that would be a lot of work, right? Well, we'd have to take a time out for like a year sometimes.
All right. They say that neurons vary in size from four microns, so 0.004 millimeters to 100 microns, 0.1 millimeters. So they're way, way bigger than transistors. So what's the size of transistors?
They're on the nanometer size, right? I think we're down to 14, 14 nanometers, 10 nanometers. Yeah, I mean, there, I think it's more like five nanometers, maybe, maybe, maybe, maybe it is 14 nanometers. All right.
Well, it seems like that, that, that difference in size doesn't help us in this industry. Shipping devices. 16 nanometers, 14 nanometers. It's like kind of right around there.
All right. So, so what is that? That's, so let's take the 100 microns. I mean, it's just so huge.
Yeah, that's hard to understand. How is it that our brains can have units of computation that are so much bigger than we've gotten in silicon? Yeah, they're analog though too, right? They're not digital in the same way.
There's, each one is a, is a pretty complex function. Yeah. Yeah, that is interesting. Yeah.
So anyway, I mean, it does feel like we're getting down to the spot where, where, you know, you just can't make the transistor smaller very easily. The world goes down as you try to fit more transistors on, you get, get, get, introduce, you know, errors mean that you scrap more of your chips, everything. Yeah. So, I mean, you're, you're answering, you're answering the silicon answer to my, to my question.
You know, why would we think now would be the one time when it's, when we're ready to flatten out? But you could ask the same question and say, well, why would we think that the runaway, why would we think this transition would be different than all the other ones in human history where we've solved it. We've figured it out. You know, why should AI take off?
Why, why would we have to be at this place where people are literally talking about becoming immortal now? And it's at this moment in history. Why would that be true where other moments in history has been beyond, beyond the pale of current thought? And maybe the answer to both is the same.
Maybe the answer is that this isn't unprecedented time by definition, the singularity, unprecedented that we can't see beyond it. So maybe it is flattening out. Maybe it is the, the flat part of the signal weight coming up. Well, you know, that is the thing that I just keep coming back to you is that no one knows, like we're already kind of in that period where people have pretty strong aesthetic reactions to the prediction that super intelligence will arise, but it feels like, you know, and then we come back and justify it with arguments.
That's what it feels like to me. Yeah. We're probably doing the same, right? I mean, oh, no doubt.
I mean, I'm, I have no doubt that I'm doing the same. We're trying to really think things through from all sides, but we're human. We're not, we don't probably not better than anybody else at falling into the trap of our own arises. There's this bias thing where you're like, if you want to believe something, you're like, can I believe this?
Is there a way I can make this for anything? You don't want to believe it. You're like, hmm, must I believe this? Do I have to believe this?
Is there a way I can get out of this? Yeah. So, you know, I think we're probably on one side of that. It just seems like a lot more plausible than all the other doomsday scenarios.
Well, that's not true. That's not true. The synthetic biology is pretty. If humanity gets wiped out, whether from an AI mistake or synthetic biology or whatever, that's the worst mistake any ever made by not just human beings, but made by life on earth, because that's the end.
And that feels like we, it just doesn't seem certain enough to like not care, right? Yeah. Yeah. It's only, I feel like it's only got a two and a hundred chance of happening.
Or one and a hundred or one. Or one and a thousand. It's like, it's important to know ourselves with this. It's only one and a thousand or whatever.
Yeah. Yeah. If there were a lottery ticket where your odds were one and a thousand and the prize was a trillion dollars, we'd take that bet. We'd buy lots of tickets, right?
Oh, no doubt. We'd put our life savings into tickets. Oh, for sure. Yeah.
And that feels like that's the kind of, that's the kind of honest we're talking about. Yeah. I mean, I think it would be irresponsible to claim otherwise that you, that you understand what's happening well enough. I mean, one of the, one of the reasons why people say it's going to be a long time is because they say we don't understand what's happening.
We don't know how to do it. Like, there's too much we don't know. But that, that is also a reason why to say that it's not going to happen is ridiculous and irresponsible. Right.
If we don't know enough about how to do it, then we don't know, that doesn't mean it's because it's intractable. Right. That just means that we don't know. Right.
Yeah. So here's, here's my takeaway. You want my takeaway? Yeah.
My takeaway is we should keep doing the show. Okay. Very good. I have a second takeaway.
Let's hear your takeaway. We will publish these numbers, your numbers and my numbers, 20, 27, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, I we'll publish those in our Facebook group and invite listeners to come and do the same. Put their numbers in there. We did this one before, but our methodology is different.
We should find what the equivalent of ours was last time so we can see how it's evolved. Yeah. So we'll link up the previous version, but we'll have a new version starting now with this show, which is number 54, I think. Everybody can play and we'll see where we are.
Yeah. And if we, if we stick our AI on it, that will really improve the answer to, like, we can actually turn that into concrete for-show predictions. Awesome. I love it.
Yeah. You can look us up there. If you like to have your voice on the show, you can record a quick voice memo on your phone, whatever voice memo app you've got, and send it off to us concerningai.com or concerning.ai at gmail.com, whichever you prefer. It all works.
And send it off to us, and we'll get you on a future listener feedback episode. Sweet. Alright, so next time, keep it up. Keep it up.
And with augmented reality just coming along here, Wuffy things like, like, timely. Alright, that's cool. Bye. Bye.