Building Large AI Models

Dr. Ronen Dar (Co-Founder/ CTO of @runailabs) talks about the challenges of running compute infrastructure for AI, the GPU ecosystem, sizing LLMs and more. SHOW: 739CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotwNEW TO CLOUD? CHECK OUT - "CLOUDCAST BASICS"SHOW SPONSORS:GCore - Global Hosting, CDN, Edge and Cloud ServicesUse promocode “CLOUDCAST” to receive a €100 credit on Gcore servicesEquinix Global Data Centers and Networking Learn more and signup at https://deploy.equinix.com/. Use the coupon code CLOUDCAST to get $500 in credits to get started.Datadog Kubernetes Solution: Maximum Visibility into Container EnvironmentsStart monitoring the health and performance of your container environment with a free 14 day Datadog trial. Listeners of The Cloudcast will also receive a free Datadog T-shirt.SHOW NOTES:Run:ai - Build Your Next Large Model (homepage)Ronen Dar, Run:AI's CTO, on managing computation resources in ML pipelinesTopic 1 - Welcome to the show. Tell us a little bit about your background, and what you focus on at Run:ai.Topic 2 - Let’s begin by talking about the challenges of running AI applications. What unique characteristics and requirements do AI applications have?Topic 3 - Most AI applications run on GPUs. How do things change when using GPUs vs. CPUs to power AI applications? What is needed to get the most out of GPUs?Topic 4 - As environments grow larger, what is needed to scale-up environments, both in terms of scheduling applications and managing the underlying GPU infrastructure?Topic 5 - GPUs are not only expensive resources, but also in high-demand. How are companies doing capacity planning with GPUs? What struggles are you seeing companies have as they manage planning for AI projects?Topic 6 - Are the new Large Language Models (LLMs) much different in size than AI models of the past? Topic 7 - How well is the industry prepared to deal with the new interest in AI from across the industry? FEEDBACK?Email: show at the cloudcast dot netTwitter: @thecloudcastnetFEEDBACK?Email: show @ the enterprise ai show dot comeBluesky: @EntAIShow.bsky.socialTwitter/X: @TheEntAIShowInstagram: @TheEntAIShow