Here we are again after a long holiday break. Not that long. Thought long. Yeah, I missed it so much.
I did too. This is Ted and Brandon. And this is the Concerning AI Podcast. All right, let's concern ourselves with AI.
That's right. So we're going to start with some follow up from the last episode. Episode 8 was about Exponential's. And we got a great comment from Eric, a friend of the show, and friend of the real life.
Yeah. Thanks, Eric. Let me just read it. Go for it.
Worrying about Dangerous AI now is exactly like worrying about overpopulation on Mars. Is he distant? No. No.
He's not surprisingly. He's not distant. On the last episode, we talked about there's AI researcher Andrew Hootning. I don't think we mentioned it by name, but that's what it is.
Very well known. And he is very dismissive about AI safety. And he says it's like thinking about overpopulation on Mars today. And so Eric is saying, no, that's exactly the right analogy.
And it's concerning. So let me read the rest of Eric's. OK, let's hear. Whoops.
I lost the message here. Hold on. OK, here we go. Consider the fact that the exponential period of human population growth has ended.
For the last 25 years or so, it has been hovering around 75 million more people per year. That is a linear rate. Remember, we're talking about exponential. OK.
Flat amount per year, not percentage. And in fact, the trend, even in the flat amount, has been declining. So when it comes to human population, the first bottle is full. He's talking about it right here.
OK, good. Mars would be a second bottle. If Elon et al succeed in making travel to Mars cheap, freeing us to reproduce as fast as we want again, how long does it take to fill the second bottle? If we went back up to 2.2% population growth per year, the answer is 32 years.
In 32 years, we would have covered the surface of Mars in greenhouses and cities and garbage dumps. And it would be as crowded there as heater. I think we imagine that if we found a way to send people to Mars and live there, then we would have centuries or millennia of growth before we filled it up. But with a full bottle next door, we could live to see it fill up.
That is the power of exponentials. Oh my goodness, Eric. Really? Thank you.
Really? Wow. I don't know if I can even believe it, even it supports what we were saying before. I know.
It's so not intuitive. I know. I think this is another example where you and I had a whole show talking about how humans don't think exponentials. And we were trying to train ourselves to think that way.
And yet we missed this completely. Eric, man, pulled it out. And yeah, I mean, I don't forget it. Yeah, and he used the bottle.
You know, like when you've got your current bottle is half full, you think, ah, we better get a couple more bottles. And that only buys you like four minutes or three of us. Yeah. So thank you, Eric, for taking our point and making it that much stronger.
And mind blowing. Mind blowing. Yeah. All right.
So thanks, Eric. We're going to move on. Deep learning. Deep learning.
We've been seeing a lot of press about deep learning being one of the ways that we're getting AI success right now. You've described this right now. If you're talking about other people describing this as an AI spring, you know, there seems to be a cycle, a boom bus cycle of AI spring followed by the hype not delivering AI winter. And then yet again, 10, 15 years later, whenever it is, AI spring again.
Yeah. So deep learning in AI spring right now. And deep learning seems to be the term that's coming up over and over again. In this AI spring, we're getting lots of successes.
Yeah. One of the big successes that a lot of people will be familiar with is our speech recognition. I mean, like Siri or Cortana or there's a whole bunch of them now. And Amazon's got one.
Yeah. Google's got one. Deep learning networks have been producing state-of-the-art results in syllable recognition and other important tasks that are part of speech recognition and the front end of natural language understanding, which we still can't do. Right.
And as we've been hearing about deep learning, I realized very quickly that I didn't know what that was. And so we went to find it. And I still found it very difficult to understand. And so I didn't know what it was either, because my AI training is 10 years in the past.
Yeah. So what did you find as you started to dig into this? Well, what do we know about deep learning? It's the connectionist model, again, of, I think, neural networks.
Neural networks is a good buzzword. So it's the third coming of neural networks, the second coming, I'm not sure which of neural networks re-imaged and repackaged up. OK. And so can we give a, is there an easy explanation of what is a neural network for people that don't know or just to refresh our memory?
I wish I could do it just as well. What I can say is it's good for classification tasks, where you want to say, this is a particular type of syllable. This is another particular type of syllable. An example, well, let's talk about single layer networks, because deep neural networks are deep learning networks, or whatever you want to call them, are multiple layer.
But what can a single layer perceptron network do for you? And an example of that is if you take a red marker and a green marker and make a bunch of dots on a sheet of paper, that is a two-dimensional space with two different classes, red dots and green dots. And if you can take and draw a line, a straight line, with a ruler that separates the green dots from the red dots, those dots are linearly separable. And a perceptron network, single layer, is going to be able to solve that problem.
Going to be able to tell, this dot is green, this dot is red. It'd be right. Yeah, you're going to be able to train it up to tell that apart. And this generalizes to higher dimensions than just the two dimensions.
If, on the other hand, you know, red is the one, my three-year-old is the one who makes all the dots. And they're not going to be linear or separable. They're not going to be able to draw straight line. But maybe you could draw a line that kind of curves around and separates them.
That is no longer linearly separable. It's separable. But it's no longer linearly separable. So you are going to need multiple layers of your neural network.
Or you're going to need a lot more complicated units. OK. So you've got a neural network. Do we understand them?
I mean, when we're a researcher and you've got a neural network and you sort of set it loose or you give it the task to learn, you have to train data, do you know how it's going to work? Well, that's one of the really, I think, salient points about neural networks in general. It also applies to evolutionary algorithms and stuff like that, is that they're kind of inscrutable. You know, you train it up, given a bunch of training data, supervised training data for the most part.
You train it up, and you think, ah, this is going to work or not. You try it, and it kind of works or it kind of doesn't work. But to tell why it's working or how it's working is not straightforward at all. Yeah.
OK. So I would say I would use the word inscrutable. Yeah. And so in some ways, this feels like we're pretty much in the weeds because we just have no idea what we're talking about.
And yet, it feels important to me because, you know, we've had quite a few shows where we've been kind of theoretical and very forward thinking, like, you know, thinking 30 years from now or 50 years from now or whenever the AI, whenever we get to the artificial general intelligence, we've been thinking way out of this future. We stepped back to Exponentialist to try to understand what's happening today. Today, right now, where are we? Why do we think that this is not just purely science fiction?
Because it sounds crazy to everybody we talk to about everybody. And most people are just dismissive because they don't want to talk to us about the holes in our own theories here. So we're trying to step back to say, all right, what are we missing if we're missing anything? And for every piece of data we put into our own brains that we find, you know, that we're not disproving our own crazy theory about the future AI.
It feels like it's stronger. I've mostly found dismissive shallow thinkers on the negative side. I'm sure that they are not the only ones. But so far, like when I read comments and stuff from your average Joe, and even when I read people who are, you know, a little bit more, than I respect in other domains, for example, they tend to have very shallow dismissive.
Right. So that tells me that we're going to need to do our own thinking on the con side. Yeah. I think we're going to have to do it.
And that would be fun. I want to touch on one thing before we get into that, though. That as I was looking at the deep learning networks and realizing that, ah, this is the thing I dismissed a lot when I was doing AI, which is neural networks. And it's just a sort of a second coming of those.
The question that I don't know the answer to, but that I really want to know the answer to, is how much of the recent success of neural networks is due to our computational resources, the exponential growth of our computational resources? Because they're computationally intensive to train. Right. So how much of the success comes from that that we have, like GPUs now, like NVIDIA makes and stuff, graphics processing units that have lots of units and a great matrix and vector calculations.
How much of it comes from those and how much of it comes from our genius researchers, coming up with new, smarter ways to use the stuff we already have. Yeah. And Mahanj is that it's mostly hardware driven. And this is, again, directly related to what we talked about, the exponential's episode.
Because if Moore's law is at an end or is tailing off, then we were positing that maybe computing growth slows down unless we jump to the next thing, right? We don't know if we're going to jump to the next thing. Humanity has always jumped to the next thing so far, but that doesn't mean we necessarily are going to jump to the next thing now. Right.
And so if we're going to sort of be stuck with the computing power that we're going to have five or 10 years from now is going to be the end of the line, then will the future of AI happen? Like, can it be driven just on the algorithmic side if we stop the growth in hardware? Yeah. That's a good question.
Because some people would say that unbounded exponential growth has never been possible for anything. Yeah. Yeah. I think there is still quite a bit of momentum.
I mean, I think there is too. In the processing power growth. Right. But if we let's say that we get to the place where $1,000 of computing power gets to the same number of computations as in a human brain, that doesn't necessarily mean we've got AI at that point.
Right. No. And if the growth, if the hardware growth is not continuing, then the rest of the growth, even though we've got momentum now, the rest of the growth is going to need to come in software to get us to the place of. Right.
And until the machine is going to figure out the next information. Yeah. And the positive feedback loops. I mean, the positive feedback loops, it could be that 10 times more computing power than we have now will help us create better algorithms because maybe the researchers don't get any smarter, but they get better tools, tools of creating algorithms.
And we just don't know. We just don't know what that will look like. Yeah. So I think it would be fun to answer that question of, you know, where is where the recent wins come from?
Smarty Pants researchers coming up with new things or processing power, finally catching up to the needs of the artificial known networks. It's fun. We're coming up with so many questions now that are actually harder to answer. You know, and it's harder.
Like this question about deep networks, I found it much more difficult to do the reading. And so I'd like to try to figure out anything useful to talk about today. Yeah. And if we make errors, which is totally possible, please do feel warmly welcome to correct us on Facebook.
Absolutely concerning AI is where we've mostly hang out. Yeah. Yeah. Please do.
Yeah. So get in touch. And no promises on what next week will be none at all. Yeah, we're not quite sure.
But we're fine. I hope you are too. And I hope we're not overly concerning. Yeah.
AI. Yeah. See you next time. All right.
Bye. Bye.