So the headline here is that DeepMind's Chinchilla is a 70B model and it has crushed traditional compression algorithms and open it's, I think, really kind of opening new avenues for AI utility. So data compression has long been a kind of cornerstone of computing, enabling the efficient storage and transfer of huge sets of data. I mean, if you've watched the TV show Silicon Valley, you'll know it's all about, you know, their fake company was all about like this company that could do this really impressive compression stuff. But pretty much getting files to be as small as possible makes a big difference.
Right. Like if you're able to watch a YouTube video and let's say the YouTube video is, you know, at 10th of the size because it was super compressed, but the quality was just as good. Everything would load faster. There'd be no buffering.
There'd be, it just solves a whole lot of the problems on the internet today, if we were able to compress things more. So apparently AI is quite good at this. Alarmingly good. Some people say, but this new study from DeepMind, which is, of course, the subsidiary that Google purchased, the AI studio, Google owns now, the study by DeepMind suggests that machine learning models previously renowned for, of course, tech space tasks may actually outperform traditional algorithms in compressing data.
And in some cases, significantly so. So the study published on ARXIV. This is kind of like, how do I explain? If you don't know ARXIV is like where everyone's publishing studies now, they don't have to be super peer reviewed.
And you just kind of get them out quick and dirty there. And so some people kind of question the validity of things that are published there. So for example, if you remember a while back when we had like the whole, the whole like, what was it like that, magnets? Oh, yeah, it was room temperature, superconductor material or whatever.
That was published. The paper on that was published on ARXIV. I think now it's been disproven. I don't think it was actually truly what they said it was, but that's where it came out.
So just so, you know, in any case, the study they published is titled language modeling in compression and it asserts that DeepMind's large language model known as Chinchilla 70B can perform lossless compression on various data types more effectively than established algorithms. So specifically when tested on image patches from the image net image database, Chinchilla compressed the data to 43% of its original size. And this was outperforming the PNG algorithm. So PNG, as you may know, is kind of a file type that a lot of images are made in.
I think like maybe the most standard and common that you'll see on images, JPEG, right? And then PNG is kind of famous because it can do transparent backgrounds, but PNG is like more is a more compressed algorithm for these images. But even more than PNG, this Chinchilla is compressing it like 43% more than its original more than images original size and it's beating the PNG algorithm right now, which achieved, I think, 58%, only achieved 58%. So even more startlingly for audio samples from the lever speech data set, Chinchilla managed to compress them to a mere 16% of the raw size, which was outshining flax 30%.
So Chinchilla, and using AI to create an algorithm to compress is literally beating some of these famous algorithms that we've had. So in this, this is what they said, quote, in this case, lower numbers is the results in the result means more compression is taking place and lossless compression means that no data is lost during the compression process. So that's what the paper is kind of clarifying. I think this finding stands in contrast to lossy techniques like JPEG, which trade, which, you know, trade data fidelity for file size reduction.
So you can get essentially smaller file size in JPEG, but as it gets smaller, you know, of course, losing fidelity, the image is getting worse. So what's really interesting here is they're able to compress this smaller algorithms. They use compressed images smaller, but there is no loss in data, meaning the image is just as high quality as kind of the raw original file. So I think what makes these findings really fascinating is that Chinchilla's 70B was primarily trained for text-based tasks.
However, its capability to outperform algorithms, you know, specifically designed for image and audio compression, underscore its versatility, and opens up some new conversations about machine learning models being more than just text prediction tools, right? So when we talk about chat GPT or GPT four, we're like, yeah, this thing's great. It's just, you know, it's, it's not that like intelligent. It's just kind of predicting, um, it's just predicting the next word in a sequence.
Okay. Well, you know, how come these things are able to create really complex algorithms that are smarter than the human created algorithms we have so far, right? So it's definitely bringing up this conversation of, you know, what more can it do? Why is it capable of writing these algorithms?
I think his revelation kind of aligns with broader perspectives within computer science that associate data compression efficacy with general intelligence. So the thesis is that compressing data effectively requires identifying patterns and making sense of complexity, much like how a kind of form of understanding or representation of the world operates. So let's talk a little bit about the hutter prize and the intelligence compression nexus. You're like, what the heck is that?
So the concept of linking compression ability to general intelligence isn't new. So this is actually received some scholarly attention through initiatives like the hutter prize. Um, this was named after Marcus Hutter, who is a researcher in the field of AI. And one of the authors of the D mind paper, um, this prize aims to incentivize the most efficient compression of a fixed set of English texts.
So the rationale is that efficient text compression would necessitate an understanding of semantic and synthetic, uh, or yeah, patterns in language. So this is not dissimilar from kind of human comprehension, right? Like people really do say, like this is complex stuff. And some people say, like humans really have to understand what's going on.
This isn't just like text prediction. This model would really have to understand. So, um, I think that the paper doesn't stop at compression. Deep mind researchers argue that the relationship between data prediction and compression is bidirectional.
Um, so that is to say, you know, a robust comp compression algorithm, like Gzip could theoretically be reverse engineered to generate new original data based on what it has essentially learned. So in one experiment, both Gzip and Chinchilla were tested to predict what comes next in a sequence of data and not surprisingly, while Gzip generated, you know, completely nonsensical output, Chinchilla displayed far better generative capabilities. So though Deep Mind's research has not yet undergone peer review, it has raised, I think some really intriguing questions and opens the door to possible new applications for machine learning models in fields beyond text. I think given the ongoing debate and studies examining the relationship between compression and intelligence, it's likely that this won't be the last week here of this topic.
Um, and I think that as data continues to grow exponentially in both volume and complexity, efficient compression is going to become increasingly essential, right? You think every time the new iPhone comes out and it gets higher and higher quality, we get, you know, more megapixel cameras, like all of our data and everything we create, all of our content, our image photos, everything is just increasing in size. And I think with that, um, increasing in compression is really important. So if Deep Mind's Chinchilla's 70B is any indicator, machine learning models, just might be a game changer that we have been waiting for in the speed field.
So definitely something we'll continue to follow in the future. Thank you for listening to today's podcast episode, breaking down how AI is impacting your industry. Today's episode is sponsored by AI box, a no code AI app builder and marketplace, which just launched a crowdfunding campaign. If you are interested in investing in a new AI startup, you can go to republic.com slash AI dash box.
I'll leave a link in the show notes as well to learn more. The minimum investment is $150 and the maximum investment is $100,000. Until next time, have a fantastic day.