43 – AI – 1 of 2

The truth about Artificial Intelligence and Generative AI. This is the first of two episodes on AI. Navigation: Intro (01:33) What is AI and AGI? Why now? (02:07) Setting the Record Straight (08:55) Verticals (20:30) Other AIs (28:48) The Big Guys (31:20) Conclusion (38:38) Our co-hosts: Bertrand Schmitt, Entrepreneur in Residence at Red River West, co-founder of App Annie / Data.ai, business angel, advisor to startups and VC funds, @bschmitt Nuno Goncalves Pedro, Investor, Managing Partner, Founder at Chamaeleon, @ngpedro Our show: Tech DECIPHERED brings you the Entrepreneur and Investor views on Big Tech, VC and Start-up news, opinion pieces and research. We decipher their meaning, and add inside knowledge and context. Being nerds, we also discuss the latest gadgets and pop culture news Subscribe To Our Podcast Intro (01:34) Bertrand Schmitt Welcome to Tech DECIPHERED Episode 43. This would be the first of a series of two episodes on AI, AGI, generative AI. A lot has been happening in the past six months and we felt it was a great time where not everything is clear yet. The fog of war is still intense. There is probably a little bit more visibility into where things are going. It would be with pleasure that we'll talk about this deeply fascinating topic and for sure one of the topics most discussed today in tech. What is AI? What is AGI? Why now? (02:07) Bertrand Schmitt Nuno, maybe we should start with trying to define what is AI, what is AGI, what is generative AI? Nuno G. Pedro Easy task. AI is what is in the name. It's artificial intelligence. It's typically seen as a branch of computer science that is looking at creating mechanisms within machines that, in some ways, are similar to human intelligence or practically speaking, would refer to human intellect. Nuno G. Pedro Now, as we know, machines can't think. That's still true today. So they do this through very complex mathematical models that get implemented normally through software and hardware combinations. Then within artificial intelligence there are different fields of artificial intelligence. Nuno G. Pedro In the good old days, people used to talk about weak AI versus strong AI, which is more general intelligence, where weak AI is normally more focused within a specific field of solution set. General AI and strong AI will eventually become our overlord and think better than us. Nowadays you will hear a lot of different things around artificial intelligence. You'll hear machine learning, you'll hear deep learning, you'll hear about natural language processing, computer vision, et cetera. Nuno G. Pedro All of these fields are fields of artificial intelligence that intend to emulate what we as human beings do. So computer vision basically would look at the automatic analysis of things that get processed through vision. Could be video, could be pictures. Nuno G. Pedro Natural language processing is looking at the interaction of machines and computers with natural languages and human languages, the language that we have. Deep learning, I would allege, is a subfield of machine learning. There's still a huge argument on that or whether deep learning is a different field or not. Nuno G. Pedro I normally see it as a subfield of machine learning where deep learning normally uses things like neural networks—we'll talk about neural networks later on—which are trying to emulate how our brain structures thinking effectively. In a nutshell, AI is a field of computer science. It's an evolution of computer science. Machines can't think for themselves, so they do this through very complex algorithms and techniques that normally use a lot of mathematics and quite a bit of data. Nuno G. Pedro Although we'll also have a discussion today on how much data do you really need. Are we past the times where you need massive amounts of data or not? In some cases, these techniques and algorithms also need to be trained. There needs to be some sort of training mechanism, potentially even human in the loop, basically telling the machine, whether it's classifying things appropriately or not. Nuno G. Pedro For example, in computer vision, is this really a monkey or not really a monkey, if you're trying to classify a monkey would be an example of that. But again, that's in generic terms what artificial intelligence is. Bertrand Schmitt Yes, and to build on what you just said, interestingly enough, the field of AI started probably at the same time as computer science per se started so in the 1940s. It's a space that's been alive, I can't say well all the time, but definitely alive and ticking for decades. Interestingly enough, it has probably been a field that started, I don't want to say too early, but definitely more early than we had the computing power to achieve what we were dreaming. Bertrand Schmitt That has probably created a lot of AI winters. If you talk to people experience in that field for the past decades, they have known some boom and some incredibly long period of bust, 10 years, 15 years where no one would want to invest in anything called remotely AI given some past experience of promising a lot and under delivering. Bertrand Schmitt I think things changed around 10 years ago with the advent of the latest GPUs from Nvidia, with the advent of new coding paradigms like CUDA from Nvidia as well, that let harness the power of GPU for this type of task much more easily, and obviously some new techniques in deep learning that let you train better and at more scale. Bertrand Schmitt Of course the availability and advantage of digital data at scale because since the 2010s we have the internet, we have books, we have content, we have audio, we have video, we have photos, we have everything online. Suddenly accessing data that you can use to train at scale model became finally much easier than it used to be, so a big dramatic change. What we are probably mostly going to talk about today as routes 50, 70 years ago, but really was enabled in the past 10 years. Nuno G. Pedro To be clear and just picking up on what you said, these have mathematical roots in things that have been around for many decades. Neural networks are not new. We're now talking about convolutional neural network CNNs, recursive RNNs, adversarial. Nuno G. Pedro We're going to talk about transformers. Transformers is probably something more recent. It's an adaptation which actually is credited with Google, which is funny because it's deeply used by OpenAI, but Google were the guys who came up with it. But in general, if we look at the field, it's been around for a long time. The mathematical basis of the algorithms and techniques that we use in AI today have been around for decades. Nuno G. Pedro To your point, what has fundamentally changed? If I had to synthesise it and summarise it computational power, obviously with the advent of GPUs now there's even ASICs so there are specific semiconductors that are very, very focused on the process saving of certain techniques of AI. Computational power has definitely changed. Availability of data at scale and the ability to process that data at scale and access data pipes has obviously changed a lot. Nuno G. Pedro I would say networks have changed as well. Latencies have come down, so if you want to process stuff in the cloud or even in your own processing power, in your own device, that has obviously simplified the whole story of it. Nuno G. Pedro In some ways, it's brute force. If we think about it, it's like a lot of data, a lot of compute, and it's brute force. Now we get AI. I think this is an important point because this will come back to why our AI agents or our AI overlords will not kill us immediately because it's still brute force. They're not really intelligent, they're just doing stuff. We'll come back to AGI, to general intelligence later on, but let's leave that positive note for now. They're hopefully not going to kill us anytime soon. Bertrand Schmitt Brute force is a good point because ultimately a lot of researchers argue that we are still very, very early. In many ways, if you look at the way human baby animals are able to do stuff that AI still cannot do today, they always point to the fact that we can learn much faster with much less data, in some ways. At least, that's one way to look at it about the world around us, the current way we train this machine. Bertrand Schmitt In a way, it is definitely a different type of intelligence we are building today with what we call AI. It's not human level intelligence and it's not trained the way you would train a human being. It's very different based. That's something to always keep in mind when you talk about AI. Setting the Record Straight (08:55) Nuno G. Pedro What about Generative AI? We've all been listening to it. I'll open the hostilities and then you can tell us the real truth about generative AI. I'll open the hostilities by saying that in my opinion, generative AI is not generative at all. It's a very poor choice of words that someone at some point made on what generative means. I think it means generative within the context of the neural networks that it's running on. Nuno G. Pedro But it's not really generative. It's more of an aggregation of things. It's something that comes after something else that makes sense for that thing to comes after something else. That is true of images, it's true of text, it's true of a variety of other things. Sadly, that's what even GPT stands for, Generative Pretrained Transformers. That's a cool name. Nuno G. Pedro I think that's the first thing I would like to debunk. Generative is not generative at all. These things are not creating things. We'll come back to that later on, regulation and a bunch of other IP issues. But what is Generative AI, Bertrand, as we see it today? Bertrand Schmitt Yeah, that's a good point. I think it always go back to how do you train these models....