Skip to content
Code Crafting for AI: Modular's $100M Funding Boost and In-House Programming Language episode artwork

Feb 12, 2024 6 min

Code Crafting for AI: Modular's $100M Funding Boost and In-House Programming Language

from The Daily AI

In this episode, we discuss how Modular is crafting the code for the future of AI with a $100 million funding boost and their proprietary in-house programming language, offering a glimpse into the world of optimized AI models. Invest in AI Box: https://Republic.com/ai-box Get on the AI Box Waitlist: ⁠⁠https://AIBox.ai/⁠⁠ AI Facebook Community Learn more about AI in Music Learn more about AI Models

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Code Crafting for AI: Modular's $100M Funding Boost and In-House Programming Language

0:00 6:43
of matches

Transcript Auto-generated, may contain errors

Don't you wish everything was more rewarding? With Rakuten, almost everything is. You can earn cash back on those new shoes you've been wanting. You can save on the next Trippy book.

You can cash in on groceries. Just join, shop your favorite brands, and save. Sephora, Old Navy, Timu, Adidas, PetSmart, Expedia. The list is long.

Save online at over 750 stores. And when it's time to redeem those rewards, get your money exactly how you want it. Choose Interact, PayPal, or a check. So go ahead, take a trip, fill a card, order dessert.

Rakuten is a world of rewards. Join today for free. Go to Rakuten.ca or get the app. This is our AKUTEN.

So Modular is a Palo Alto based startup and they are aimed at optimizing and simplifying large scale AI systems. And so with this new $100 million of funding that they just secured, all this was essentially led by General Catalyst, which is a massive VC fund. And it also stopped participation from Google Ventures, SV Angel, Graylock, and Factory, bringing Modular's total funding now to $130 million because they've previously done around $30 million. So according to their CEO, Chris Lattner, the funds are going to be used for product expansion, hardware support, and growing Mojo, which is a company's proprietary programming language.

I've seen a bunch of tweets about Mojo. Some people are saying that this is going to be killing a bunch of other languages. So it's going to be very interesting to see where this actually goes. So the company was co-founded back in 2022 by Lattner and Tim Davis, who are both former Google colleagues.

So Modular's mission is to remove the friction and complexity, which is often associated with building and maintaining AI systems. So this is what they said, quote, the funding will not be primarily spent on AI compute, but rather improving our core products and scaling to meet our incredible customer demand. So that was Lattner. Both founders believe that the fragmented technical infrastructure in the AI industry is really kind of stemming progress.

And they see Modular as a solution to this challenge. So Modular's engine currently in closed preview aims to improve the inference performance of AI models on CPUs and a layer this year on GPUs, which is going to be really interesting. So the enhancement not only promises to be cost effective, but also boasts a speed of up to 7.5 times faster for trained models running on native frameworks like Google's TensorFlow or Metas PyTorch. And this has compatibility across existing cloud environments and even other AI accelerator engines.

So this offers an inviting, I think, really kind of a solid proposition for developers to use. So alongside its performance boosting engine, right? Because that's a big component of this. Modular's also developing Mojo, which I talked about a little bit in intro, but essentially this is a programming language currently in preview that combines the ease of Python with features like caching, adaptive compilation techniques, and metaprogramming.

So according to Lattner, the CEO, the programming language is aimed to help defragment their AI technology stack and speed up innovation. So I think the demand for high performance AI is really just kind of straining existing infrastructure. We're seeing a ton of different companies come out with all sorts of solutions. So I mean, even the fact that NVIDIA hit a, you know, a trillion dollar market cap at one point because the demand for the chips is so high, I mean, you can look at the stock price of NVIDIA today is absolutely skyrocketing all time highs.

And you see really in this whole space, the reason for that is because there is just so much demand for this AI infrastructure. And like they're saying, it kind of is straining the existing infrastructure, right? Because if you want the latest NVIDIA chips, there is a line around the door, you have to fight other AI companies. And I've even seen, you know, it even feels like companies that are able to get their hands on some of these train ships or have a major competitive advantage over others that may, that might have, you know, perhaps great talent, great data.

They just can't get like the infrastructure. They can't get like the chip sometimes. And so I think when you kind of see this situation, you really start to look at, you know, what are the other solutions we can find to solve some of these problems? I think this is really where they're trying to try to plug it.

So I think the demand right now is super high and Microsoft has actually already flagged a hardware shortage that could cause service disruptions. And GPU provider NVIDIA's best performing AI chips are completely sold out until 2024. That's absolutely insane, right? We're in this big AI boom and NVIDIA's chips are sold out.

So Lander acknowledges the growing crisis and he says, quote, the compute power needed for today's AI programs is massive and unsustainable under the current model. We're already seeing instances where there is not enough compute capacity to meet demand. So given this landscape, startups like Deki and OctoML are also really focusing on making training AI models more efficient, but still the adoption of new programming languages in a Python dominated landscape raises a lot of questions. So as of 2020, 87% of data scientists were regular Python users.

However, Lander remains really optimistic about Mojo noting its ability to bring together various components of AI applications without sacrificing performance or scalability. So four months after its product, Keynote, Modular's community has searched over 120,000 developers and leading tech companies are already really using the startups infrastructure. I think they have, you know, 30,000 developers on their way less and a company has made a really strong impression in a short period. So the most important enemy of Modular, I think, is complexity.

That's according to Lander, their CEO. He said, quote, the Modular Engine and Mojo together level the playing field and this is just the start in a sector. I think where demand is fast reaching its sustainability limits, right? We're seeing this in all sorts of areas.

I think Modular is kind of inflex and capital could be a real game changer for them and allow the company to kind of expand its suite of solutions to a much broader market, making AI ultimately more accessible, affordable and really sustainable for enterprises of all sizes. So I think from a funding standpoint, the startup is off to a really good start. And now I think we're just going to be keeping our eyes open and watching where they grow and what they do and how their products are received as they actually start launching and getting more massive option. Thank you for listening to today's podcast episode, breaking down how AI is impacting your industry.

Today's episode is sponsored by AI Box, a no code AI app builder and marketplace, which just launch a crowdfunding campaign. If you are interested in investing in a new AI startup, you can go to republic.com slash the i-box. I'll leave a link in the show notes as well to learn more. The minimum investment is $150 and the maximum investment is $100,000.

Until next time, have a fantastic day.

Frequently asked questions

How long is this episode of The Daily AI?

This episode is 6 minutes long.

When was this The Daily AI episode published?

This episode was published on February 12, 2024.

Is there a transcript available for this episode?

Yes, a full transcript is available for this episode. You can read the complete transcript on the episode page.

Can I download this The Daily AI episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.