Simblifying AI-generated art Part 1 [the tech behind it]

I, V, M. to speak of hence it can not be artificialized that's right we have eaten the machines at their own game this might remind you of the time that your smarty pans friend claim that he would never lose a cricket match because he never played a game in the first place fair enough joining me on the show today are two paragons of intelligence and by that I mean there should be a lot of footwear headed their way especially of the third sleeper variety and that's an inside joke for those of you who know first up it's the man who puts the eye eye in AI it's Tony's ab eye stay I in hi hi if you haven't spoken of my cricket exploits Satin Tendulkar and me together have hundred centuries so wow that's That's Was 99.94 divided by two I'm just trying to figure out my connection with John Bradman Then then there is a man who because he's an engineer puts the machine in machine learning But he also puts the new in neural network the al gore and rhythm in algorithm He puts the PC would house into the doing test and if you didn't get that you have failed the doing test and when got them does Something naughty he puts the what son in what son and when he found out what a pain it would be to do a podcast with his first Co-host it was a moment of reckoning for him Hence he also puts the deep learning in remind me never to do a podcast with deep again that has been my learning Well played. Thank you And missing of course is the 8th who is feeding on some other data set at the moment And I hope I have crammed enough buzzwords in there to have achieved a IP L.O. Or AI professionals listening optimization Okay, before we go ahead folks are profundity and since only the three of us are over here This is actually something that I have taken from LinkedIn one of those rare good posts that come on LinkedIn And this is from a friend this is from gentleman called Rahul Rajput and he says the law and I don't know if this is because he's put it in I don't know if it's his own and whether he's got it from somewhere but either way kudos to him for bringing it to my attention The law of conversion says that an engineer can neither be created nor destroyed rather it transforms from one field to the other So gentlemen happy world engineers day.

Ah, yes. Yeah, two happy world engineers I don't know if the law of conversion said you could always convert something from word to PDF but not the other way around That's interesting and that's actually a great starting point for what you're going to be talking about it is a gentleman How have you been? Yeah, so I've been in and out of so many Pottold roads that I'm seriously considering suggesting to the powers that be that you know people name Roads after prominent people right why don't we name potholes also right so like there's a you have really live maybe you can rename the potholes after prominent official than the city admonition I don't know potholes stations has it yeah potholes stations that's my bright suggestion for the week yeah this is absolutely nothing to do what we're talking about today but there's a solution to observation how have you been? Oh, good man.

I mean speaking of potholes if you haven't seen this My other movie called so me or not that case ago on hot stuff go watch it It also mentions potholes not too many plot holes on it Which is probably what happens when AI creates a story or no That's also why we are having trouble scoring me these days because in the supply chain of marijuana. There are potholes The part thickens okay, actually doesn't it actually I should not display too much intelligence about this particular topic Let's move on and talk about what we actually wanted to so this is something that we have wanted to discuss for a very long time This is on AI generated art a gentleman on your Twitter timelines Instagram timeline So the last few months you must have seen weird pieces of art being generated and people would have done hashtags like daddy to And all that so I just thought today we would talk about that a little bit Just thought I'd start by asking you guys what you know about this so far Have you seen any interesting things you know anyone who's actually doing these kind of things? Yeah, sounds fascinating. I mean, I think I'm not sure if it's the same thing basically There was one place where you could put in prompts and it would create something like if you said I don't know check Planker get I mean a little bit more famous person in check maybe Raphael not I'll Planker get they won't create something that we Looked like Raphael not playing cricket for instance.

Yes, it's pretty cool. Actually like you can input words and it comes out as pictures Yeah, that's pretty much what AI generated artists So I thought that's a fascinating topic for simplification because there's a tech aspect to it And of course there is a more cultural aspect to it as well And we've spoken earlier on the show about whatever the creator economy the changing forms of nature and all that I wonder if you can get these things to generate blank picture like if you say politician telling the truth So your question is a larger one like and these tools be text attire Oh, yeah, it's like that that Indian Express did that editorial in 1975 not doing the emergency there was a blanket tutorial I don't think yeah, this I don't think Dali would be able to do that just yet I don't think it has reached that level of comprehension But anyway, let's I thought we'd look at that a little bit I just might end up being a two-part episode because there is so much to discuss just a tech aspect of one could cover one episode So it's like what we will do it's kind of fascinating because all this development has happened in an extremely short period of time back in 2015 And this is just what like seven years ago There was the technology of the time was just automatic image captioning Which is suppose you had a picture of say a file not all playing cricket then AI at the time could read that and say here Is a file not all playing tennis or some equivalent of that? So that was a technology at the time so researchers wondered whether they could so kind of like do the reverse suppose We give AI model suppose we give it text can it actually generate images and the interesting thing over here is that it generates new images Rather than retrieving images like you can go to Google images And you can search for like whatever you can search for cat playing heavy metal guitar and chances are you might know actually you might not find that at all So if you search for just cat you might find lots of pictures of cats that are already taken by somebody or created by somebody But what these tools basically searches from an index. Yeah, yeah, so it's just from indexes Google images But what Dali do and all the other guys all its other contemporaries do is they actually create new art?

And I think that's the fascinating there's a site somewhere which has this person does not exist dot com or something Which actually was a gallery of cases very realistic For people who don't actually exist kind of fascinating as that's what AI has actually done this time I've used about fascinating five times all day and I think that's a quote I'll need another word that's fascinating Yeah, that's fascinating. Yeah, and in 2016 So researchers tried to do the reverse and they succeeded but the images were pretty small like they were very blurry and they were quite small But then it started off somewhere like the first email or the first message that was ever sent over the internet was just H E from H E L And today look three intelligent folks like us can do we hide their video communication sending our thoughts and wisdom across the universe So tech has from H E to H D Thank goodness you didn't say from H E to H E L and that's basically what I was thank you went there Oh, so the tech was the queen H E to late H E Or actually that is not H E sorry, that's okay that awkward moment of silence. We weren't quite sure if we offended anyone or something Or it's okay. Yeah, it's also applicable because helium is a noble gas.

No, oh Well done. Well done. How long have you been saving that? Okay, so AI has actually been used to create art for a while now There's a German painter called Mario Klingman who created and sold some portraits that he made using AI Now the interesting thing about this is that he had trained his AI model We did those two episodes on AI right with the people who actually know what AI is any AI model trains itself on a set of data And these models are pretty much the same as well They train on a set of images and then they do all their magic of that So Mario's models they trained themselves on certain types of faces And then they were able to create new faces after that and he actually sold those for six figure dollar some amounts Which was pretty big this went up a notch I think the bios could use a little bit of AI there.

No, sure probably Cultural implications will discuss in a while so in 2021 there was a company called Open AI Which is again one of Elon Musk's many companies or at least one of the co-founders They announced something called dale which is a tribute to Salvador Dale and a play on wall e the character And this was capable of producing reasonably good images out of a text prompt Like Tony said you go to the dale website you enter a prompt like Rafael Narl playing cricket And you have a reasonably decent image that came out of it This was open only to a few people artists and tech reviewers and all that and slowly slowly started expanding and earlier this year I've just about a few months back the same company announced a far better version called dale 2 Which is way more powerful now. This is more powerful in many ways It can create an image out of almost anything and when I say anything It can be any object in any setting in but what is more interesting is that it can be in any particular situation era in the style of Somebody like Rafael Narl playing cricket in the style of a Picasso painting for instance Is something that something that dale could possibly use could possibly do or you know You can use it to create very very specific situations Like this in fact the longer the list that you give it the more specific you make it the more impressive dale ends up being So it's a model that is very good at synthesizing a long list of concepts and coming out with something unique The way dale works is that you can sign up on the dale site now It's open for purchase as well just enter a text prompt and a few minutes later You get an image I think there's something like 20 free images something you can create and then after that you need to pay for it The same model or the same concept has been used by other companies like mid journey is one of the most popular ones right now And it's probably the easiest to get started They just need to go to a discord server and type slash imagine and absolutely and things so in research for this episode I signed up for mid journey and I created a whole bunch of things myself including cats playing heavy metal guitar And I also tried simplified hosts sitting on a table having beer and it generates four images and a couple of them I mean there was nothing really to simplify I'm guessing there wasn't too much of a data set of images to draw that from but one of them Was a very happy looking beer blast which kind of made me happy so I think this is going into our illustration for the episode Yeah, I think we should I think we can use that as the illustration just that beer glass So when people look at the episode I think it might just be that it's a I must mention that it's a beer glass with a very a joker kind of the Batman enemy joker kind of smile on it Read into that what you will but it was quite interesting actually see this is basically this is Dali and it's also very powerful in a couple of other ways It has a feature called in painting where it can take an existing image say a room and it can super impose something on top of that So suppose you want to say okay, I want like sofa in the restroom whatever So it takes a picture of the restroom and it will then take a photo of a sofa And super impose it on top of that not just that it also account for things like the lighting shadows the specific setting the size and all that So in many days, it's quite a powerful model and Marcus Brownlee MKDHD he did a bunch of experiments He did a bunch of prompts and he said something quite interesting like whatever I do Dali it always gives me more like one of his Proms was a tiger in the velocity of Atlantis So he was just expecting a tiger in front of some illustration of Atlantis But he also got things like trees in the background birds some other things that he said okay This makes sense over here, but I didn't ask for it So the tool understands the context and it gives me more so yeah, that's pretty much what Dali is and what it does What do you guys think? Yeah, this is a you know, first of all, I have no clue how any of this is done because we will tell you Nareem That is what the show is for and the other thing is if it's indistinguishable from something a human might do or produce Then we are one step closer to the ideal of having an indistinguishable Sort of singularity. Yeah, yeah singularity.

I mean sure I think it's still a way away But yeah, I think this is definitely one of the most impressive and scary I suppose displays of what AI is capable of because I mean Reading captions from an image. I guess is like impressive by itself But sure okay, but creating something I think there's something that in the whole visual nature of it And now it's very easy to imagine that from here It could very easily be videos. It could very easily be you know, we're already living in the time of deep fakes and things like that So, you know, when you put all of these things together, it's kind of scary fascinating at the same time Yeah, I had a basic question like is this machine learning or is it AI and is there a difference between the two? I think for that we need to go back and listen to our own So on AI I think I've lost the AI guys who I won in the beginning with with all those buzzwords that I put An actual language processing, please don't leave So how Dali works is like all AI it rains itself on a lot of data in this case It's actually millions of images and the alternative text that is used This is controversial by itself because it could mean that it's reading open Instagram images and all that But basically open images on the web and the context in which is used and the captions that are used along with it This is the training data on which Dali is used by the side note I read this fascinating article somewhere fascinating again seven I think now where a lot of AI grunt work is actually done by translators and Data entry operators in India and other you know mid-income countries by people who actually type in these things I think it's on rest of world or I think that's a site on which I read this where there are actually people who are paid lots of money not lots of money But I guess lots for the cities and villages that they come from and in fact one city in Kerala I forget which one one religion Kerala is thriving with this where they hire a lot of people who have basic computer literacy skills And they just look at images and just caption them And these act as the training sets for AI by huge companies I'm guessing including open AI and I thought that entire story itself was kind of interesting I'll be so interesting if the actual images now all these Kerala guys are coming up to the end And some Stanford and Berkeley graduates will be sitting there actually doing it in the land which I'll enter in the background So we'll take a small break over here because after that we'll jump right into kind of like a little detail explanation of how Dali exactly works It's very interesting in terms of how it works even if you're a non-technical person Before that I just wanted to tell you about so back in the late 1800s turn of the century Someone came up with this chess playing machine There was a machine with levers and things like that which actually make moves And which was strong enough to beat most people And it used to go on tour and it was like it was a huge hit When was this?

It was around 1890 or something like that Really 1890? Yeah, before computer. And what? Yeah, way before computer then they figured out that there was a little compartment inside Where a chess playing midget who was like a master level chess player Who'd actually see it by system of you know, bearers would see the moves being played And then actually pull some string that make moves So as long as the first thing was to find a chess playing midget who was master level string Who'd actually agree to go inside?

Exactly, yeah, anyway we will take a little break now and we will see you on the other side You know that feeling when you finish a movie and you just need to talk about it That desperate need to discuss why Kallhuna who hits different Enter IBM's brand new podcast Cine5 It's streaming and current just yapping over Hindi cinema every Friday romcom's director will debut his career defining moments Whatever they're vibing with, they're breaking it down Join the podcast on IBM Pub's YouTube or any audio streaming platform Caution, you want to rewatch everything after listening to us Come join us on Cine5 And I invite you for the conversation about your personal events For each Monday on the IBM podcast app or the website Or not any podcast streaming platforms See you folks Okay, welcome back to Simplified we are taking a deep dive into how Dal E2 works Which is the AI model that OpenAI has created which takes text prompts and converts it into images Give it a try, it's quite interesting I'm surprised that they named the second version Dalit 2 and not new Dalit Which would have been Oh, yeah, that would have been sponsored Okay, so OpenAI used to have a model called clip Or rather they still have a model called clip Whose job it is to do the reverse Which is to generate text from images Kind of like a first thing that we started off with Now to cut a very long complicated story short This helped OpenAI refine the attributes of a certain image So what happens now when you enter a text prompt say I'm going to keep using the same example that you did only Rafael Nadal playing cricket Is that the AI takes the certain words out of this Rafael Nadal playing antennas It converts these into what are called text attributes or text embedding So you have some embedding for Rafael Nadal, something for playing and something for tennis Basically the system has broken down this text prompt Into sort of like machine language Into numbers that represent Rafael Nadal, numbers that represent the act of playing and tennis Then it converts each of these three things into images So it's understood that okay, there is these are the set of images that represent Rafael Nadal These are set of images that represent playing These are set of images that represent cricket And also it has enough intelligence to sort of figure out what's an aesthetically pleasing image When you put all of these things together So it's not just like randomly plopping one photoshop layer on top of each other So these text embeds are converted into image embeds Which are finely decoded and the image is created at the end of it This is slightly clunky to explain why I bought this I'll leave a couple of YouTube descriptions in the notes of this episode It's actually quite much easier to understand when you see it as a process happening But to me what's interesting in this whole thing is how Text and images both have attributes that are attached to them And it's then just a matter of the model matching them You know, so it's kind of like the example that a box video showed was If you're comparing a balloon and a banana Right, I mean to any human that's it's quite obvious What the difference is But if you were to use a attribute or if you were to use a dimension like yellowness Right, so banana is going to be on one end and the balloon is going to be on the other end But what if there is a yellow balloon also Then the machine will realize then the model will realize Okay, you probably need a second attribute like say roundness Right, so then the balloon is different from the banana on that dimension Then you can think of another situation where the banana could actually be round as well banana Shambaloon Yeah, or something like that or a non-round balloon So which now you need another dimension Which could be like shyness or popability or whatever the case may be So what ends up happening is the machine and nobody really knows how these models really work Is the AI just keeps creating hundreds and hundreds of dimensions For each image or each item, right So let me try and try and simplify things by taking Dali to it Obvious logical conclusion which is a machine that would make human beings based on prompts So suppose I wanted my own Tony's harassment But one who is also a progressive metal fan Right, something that I think is the only imperfection that you have the fact that you are not one So it would take my machine would then What is he not a fan? Is he not a Yeah, all attributes need to be different Or is he not progressive? Oh, wow, okay, you've found out, okay I'll just send that across to the researchers I'm working with I'm the only progressive meta fan No, progressive number Yeah Incredible Anyway, so this machine would ostensibly take the existing attributes of Tony Or the existing dimensions of Tony Sebastian Which is he has a well-called weird he sets crosswords He has a property for a file adult and he's a thought leader and matches this With the attributes of your average progressive metal fan Which is he wears black teeth he won't listen to anything under 10 minutes long And constantly wonders what time signature the mix he is operating in So once you mix these two things together it kind of perfects and creates, you know My ideal version of Tony Sebastian Hopefully this simplifies things for you as well But basically that's how Dali works It trains itself on a whole lot of data And it is very good at matching images to text via a series of attributes And it is very good at combining and sort of like putting two things together Oh, that made sense for everything else there is the YouTube video Yeah, but it's fascinating right like at one point you were using Same AI to generate alt text from images And now you've used those that same alt text to create alternate images That pretty much is it So they have done a couple of interesting things within this as well So there is something called up sampling that Dali to uses Which is it starts off with the same small image And then it becomes bigger and bigger So basically get a high-res image that is better Oh yeah, that's very interesting Yeah, it's quite interesting and some TVs also use that if I'm not mistaken It trains itself on something quite hardcore called a diffusion model Like I know there might be some AI experts who are listening and laughing at this But diffusion is absolutely fascinating It takes a good image and then it deliberately adds noise to it And then it reaches an image that is super grainy And then it tries to go backwards and reconstruct the original image It does this hundreds of times to the model Kind of can pretty much generate a really good image Out of something that's super grainy And when you do this across multiple attributes multiple images I'm guessing that's one damn hardened AI model So yeah, Dali to is full of things like this I have to tell you a similar thing about an engine, a chess playing engine called AlphaGo So AlphaGo was made by Google And so the engines that came before AlphaGo They were programmed Wait, AlphaGo is supposed to play Go, right? Yeah, so AlphaGo version 2, the first one they play one plays Go The other one plays chess, right?

Oh, okay I thought Go was harder to do than chess or MLK Yeah, of course I know chess and I know the effect that I think it's just called Alpha It's not called AlphaGo, it's just AlphaGo, AlphaGo Yeah And this is what happened So earlier chess engines used to have rules called heuristics So in chess, for example, peace development is a good thing So you develop your pieces, they're out on key squares Oh, you're a PIE I'm not a chess, alright? It's a game of more where peace development is good And central control And if there's an open file, you line up your books on an open file You try to control an open file If there are squares which pawns do not control You try to put things like this And they program or they make an opening database And literally 1000 and 1000 are millions of opening standard opening moves Are programmed loaded into the database So that the computer knows exactly which is the best move in a question And then there is also the proof fold calculation So it calculates and after a few calculations It can evaluate which position is better This is what engineers used to do And they used to get incrementally better What Alpha did was It just played, they wrote the rules They told the algorithm what the rules were And the machine played both the sides of the game with itself For millions and millions of games For space of about 10 hours or something like that At the speed of like really fast And it made up its own conclusions So no heuristics, no nothing And nothing was programmed It figured it out for itself And today I mean literally after the 10 hours It could kick the pants out of any Humans of course didn't stand a turn Any other chess engine Any other chess engine in the world was So in 1990s, yeah, yeah The widget was the widget Yeah, the widget was the widget So this is the power of AI Because the internet has done that It has given access to Enormous, really, really enormous amounts of data So if anybody's guess how powerful that program is now That was sort of my original question Which I don't think is the correct definition But machine learning to me suggests that You have to feed it inputs And then it uses all of the knowledge that it has learned And then tries to do something Whereas artificial intelligence to my I mean again this is the wrong definition But as a rule of thumb I assume that it will play on its own And figure out the rules for itself That's basically our key difference So a lot of AI would learn Using machine learning But there would be instances where It wouldn't even need machine learning To so extrapolate forward to But I'd imagine that there is some aspect of Machine learning and everything Because a bare machine needs to know What to connect and ask Chuck was saying Attributes and things like that You can only get And then of course the intelligence of the machine And it makes its own conclusion As to what attribute is what? You might put 100 pictures saying that this is happy But you also put 100 pictures saying This is sad And the machine might reclassify the happy and the sad Internally because it's looking for some other attributes Yeah, in fact one of the interesting use cases For Dali too Could be making Dali too itself better By generating synthetic images Like you might just create like on Marcus Brownlee's video It was something like where else are you like to find Like a picture of an avocado shaped chair for instance So people might just come out with random images Like this for the sole purpose of feeding back into The data set out of which AI models like this actually learn So that might be an interesting thing by itself Which is to use Dali on the output of Dali to make it And its own models Kind of better There are still some things that AI generated art isn't very good at One it's not very good at generating images If you give it coherent text strangely and weirdly enough It's very good when you have It operates like an engineer pretty much Like you give five or six different things Say I want this with this with this in this style It does a great job But if you a simple sentence it may be difficult to make In fact one of the common places where Dali puts up Is actually making signs SI, GNS If you just say put a sign saying Simplified is the greatest part It will screw that up Maybe for maybe that's what they're doing Maybe it's just extra But there are Twitter threads that show Okay here I asked Dali to make a sign of this And pretty much all of them are wrong So it's quite interesting You know sometimes you get that wrong Blue cube sitting on top of a red cube For some inexplicable reason Sometimes you get the other way around as well So sometimes it has problems with You know space Yeah parsing space and all that And also generation of very complicated images Like say Times Square or something That doesn't do a very good job That being said This is something that generates images In about 10-15 seconds You know and pretty much in any situation that you want Which is kind of interesting And it's also novelty Must mention that it's right now It's just something that people who You know sign up on Discord for mid-journey Or sign up for Dali to itself have access to But I'm pretty sure some part of this technology Will work its way into something That's more consumer facing So the Google Pixel has something called the Magic Edit button or Magic Eraser button or something That's so suppose you take a photo of a landscape And there's a person passing by If you use the Magic Edit tool You can remove that person out And anyone who's used Photoshop also Will be familiar with features like this You can now imagine that this is taken to another level With a layer of creativity With tools like this Suppose you actually take a nice image And say hey I want the I1 Metallica to be playing in this When you can actually then plop them in part So I'm guessing Playing in Gurgaon Oh wow wow wow wow wow Wait wait wait That is a burn that I did not expect from you of all people But anyway we will move on Wow There will be three people who are listening to this show And in case you don't know Metallica was to complain No no no no no no You got buckets everyone got buckets No I didn't beat buckets Because I anyway couldn't attend either show Because I broke my foot at that time So I was weeping buckets before then And slight if they were sleeping It was very cocker at it That day would have been Metallica's 30th birthday If they were when they're created that would go Exactly 30 days Yes sorry Anyway Leaving that aside Since all AI trains itself on existing data set Including DALLY 2 The creators of DALLY open AI I mean they're basically forcing that This could be used in a very nefarious way So they have tried their level best to restrict things By saying that they are not going to I mean the data set does not include Other activity or illegal content of any sort In fact Hilarious one was Martin's Brown D He had asked for a prompt of a Teddy bear conducting a surgery on a grape In the style of a 90s cartoon And it did a great job of generating that image But instead of a knife which a surgeon would use He was using a pair of scissors The reason being DALLY does not understand the knife Because that is illegal content Or whatever the case may be So it's kind of interesting So but actually all of these guys are very upfront about See this is at the end of the day Images that are being created out of a particular data set We'll discuss more of this I guess The next episode But a lot of Like if you search for a nurse You are going to get women If you search for CEO You're going to get a white male guy So these biases are still there Because the data set is also kind of biased So it's also I suppose a bid to try and get more diversity In terms of the data itself And it also can't do specific names So that's another limitation of I mean that's a deliberate limitation So the only unfortunately while you've been using the example of Rafael Lark Link There actually won't be an image that can be generated In terms of what open AI's mission is for this Well they say the usual things that they want to democratize creativity And all that And many people also think that it's just a way to Access the proof of concept for AI itself Kind of like what Watson was back in the day Yeah, there's this person who recently I think last week One an actual art competition is in mid-journ Yes, by mid-journ Yeah, if I have the headline opened right over here An AI generator, artwork, state fair, victory If you will arguments over what artists And I think that's very interesting right? And we'll talk more about this I suppose in the next episode Because we've already been yammering on about this for 40 minutes Yeah, what the implications of this and all that I think we will do next time But in a nutshell I hope we've been able to convey What Dali is and importantly how it works Is just basically a whole bunch of attributes A giant multi-dimensional excel file And lots of matching So when you put into Dali saying Like an example that I saw in another video was a teddy bear On a skateboard in Times Square It's taking multiple attributes of teddy bear Skateboard, Times Square What a person riding a skateboard looks like All of these things What a nice image itself looks like And passing it all together In the matter of 10 seconds to create 4 or 5 images Another very interesting feature of Dali to which I haven't spoken about Is once you generate these images You can actually create variations on them So if you don't like the way the teddy bear looks on the skateboard You can actually ask the engine to change the angle up a little bit That's also nice to speak into how algorithms What algorithms deem important And what are sort of like the non-important or non-essential things In the background of the image So all in all very interesting technologies Turd up a lot of debate We've spoken a lot about technology on the show over the last whatever Seven odd years and we've covered so much I haven't seen this much public chatter happening about And curiosity and fear or whatever it is about our technology Like this, you know, maybe since the iPhone came out So yeah, that's kind of interesting look at this But thankfully it will not replace simplified posts A.I.

can't really learn how to generate There will be AI that can generate conversation However, it can replicate the immeasurable banter that we have And the un-punctuality, the bad jokes AI has a long way to go Right, so stay safe, stay intelligent and Bye listening