Inference in Action: Scaling Al Smarter with Inferless

from Kubernetes Bytes · host Kubernetes Bytes

In this episode, we sit down with Nilesh Agarwal, co-founder of Inferless, a platform designed to streamline serverless GPU inference. We’ll cover the evolving landscape of model deployment, explore open-source tools like KServe and Knative, and discuss how Inferless solves common bottlenecks, such as cold starts and scaling issues. We also take a closer look at real-world examples like CleanLab, who saved 90% on GPU costs using Inferless.Whether you’re a developer, DevOps engineer, or tech enthusiast curious about the latest in AI infrastructure, this podcast offers insights into Kubernetes-based model deployment, efficient updates, and the future of serverless ML. Tune in to hear Nilesh's journey from Amazon to founding Inferless and how his platform is transforming the way companies deploy machine learning models.Subscribe now for more episodes! Show Links: OpenShift 4.17 is GA https://www.youtube.com/live/DvKHwz-c11c?si=6Zap6hk_GsQfdX2m Policy SBOM from Styra: https://www.styra.com/blog/introducing-policy-sbom/ NVIDIA GEForce NOW runs on KubeVirt https://thenewstack.io/now-nvidia-scaled-its-cloud-services-with-kubevirt/ CBT feedback https://thenewstack.io/kubernetes-advances-cloud-native-data-protection-share-feedback CNCF KUBEEDGE Grad https://www.devopsdigest.com/cncf-announces-kubeedge-graduation?utm_source=tldrdevops Palumi Operator 2.0 https://www.pulumi.com/blog/pulumi-kubernetes-operator-2-0Inferless LInks: https://www.inferless.com/blog/cleanlab-saves-90-on-gpu-costs-with-inferless-serverless-inference https://www.inferless.com/blog/how-spoofsense-scaled-their-ai-inference-with-inferless-dynamic-batching-autoscaling https://www.inferless.com/  https://docs.inferless.com/introduction/introduction  LinkedIn - https://www.linkedin.com/in/nilesh-agarwal/  X- https://x.com/nilesh_agarwal2  Medium Blog https://nilesh-agarwal.medium.com/

What this episode covers

In this episode, we sit down with Nilesh Agarwal, co-founder of Inferless, a platform designed to streamline serverless GPU inference. We'll cover the evolving landscape of model deployment, explore open-source tools like KServe and Knative, and discuss how Inferless solves common bottlenecks, such as cold starts and scaling issues. We also take a closer look at real-world examples like CleanLab, who saved 90% on GPU costs using Inferless. Whether you're a developer, DevOps engineer, or tech enthusiast curious about the latest in AI infrastructure, this podcast offers insights into Kubernetes-based model deployment, efficient updates, and the future of serverless ML. Tune in to hear Nilesh's journey from Amazon to founding Inferless and how his platform is transforming the way companies deploy machine learning models. Subscribe now for more episodes! Show Links: * OpenShift 4.17 is GA https://www.youtube.com/live/DvKHwz-c11c?si=6Zap6hk_GsQfdX2m * Policy SBOM from Styra: https://www.styra.com/blog/introducing-policy-sbom/ * NVIDIA GEForce NOW runs on KubeVirt https://thenewstack.io/now-nvidia-scaled-its-cloud-services-with-kubevirt/ * CBT feedback https://thenewstack.io/kubernetes-advances-cloud-native-data-protection-share-feedback * CNCF KUBEEDGE Grad https://www.devopsdigest.com/cncf-announces-kubeedge-graduation?utm_source=tldrdevops * Palumi Operator 2.0 https://www.pulumi.com/blog/pulumi-kubernetes-operator-2-0 Inferless LInks: * https://www.inferless.com/blog/cleanlab-saves-90-on-gpu-costs-with-inferless-serverless-inference * https://www.inferless.com/blog/how-spoofsense-scaled-their-ai-inference-with-inferless-dynamic-batching-autoscaling * https://www.inferless.com/ [https://www.inferless.com/] * https://docs.inferless.com/introduction/introduction [https://docs.inferless.com/introduction/introduction] * LinkedIn - https://www.linkedin.com/in/nilesh-agarwal/ * X- https://x.com/nilesh_agarwal2 * Medium Blog https://nilesh-agarwal.medium.com/

NOW PLAYING

Inference in Action: Scaling Al Smarter with Inferless

0:00 55:17

1×

No transcript for this episode yet

We transcribe on demand. Request one and we'll notify you when it's ready — usually under 10 minutes.

Share this episode

Similar Episodes

How Russia Could Beat NATO, War of Attrition over Taiwan, and How Ukraine Turned the Tide

Jun 28, 2026 ·52m

June 10th: The Next Banking War Is Not About Money: It Is About Identity with Dan Feaheny

Jun 12, 2026 ·18m

June 3rd: Not Your Keys, Not Your Justice: How Self-Custody Is Breaking Traditional Asset Recovery w/ James Ramsden KC

Jun 10, 2026 ·23m

May 27th: Why Banking Was Faster in Roman Times: How Stablecoins Are Reviving Ancient Financial Efficiency w/ David Parsons at London Digital Escrow Limited

Jun 3, 2026 ·25m

May 13th: Time is the Risk: Why Crypto Insurance is Mis-priced at Blockchain Speed with Freddie Palmer, Head Blockchain and Digital Assets at Howden Group

May 27, 2026 ·20m

May 20th: The Workforce Gap That Could Break Financial Services: AI, Tokenisation and the Missing Talent Layer w/ Declan Sheehy

May 20, 2026 ·21m

Similar Podcasts

Unofficial EF Ultimate Break Podcast wherebytesgoes Welcome to the Unofficial EF Ultimate Break Podcast! Here we'll talk about everything EF Ultimate Break. The trip, vibes of the trips, the experiences, the excursions, the flights, the hotel stays - everything you need to know before booking your trip and going on your trip. We'll talk about tips and tricks when traveling and how to avoid common travel mistakes.Hi, I'm your host - bytes! I've traveled with EF almost a dozen times and they are my favorite company to travel with. My day job is software engineering and I'm always looking to make things easier and more efficient - EF does this extremely well, but there's always more to learn. Like what should you actually pack for your trip? Is Ultimate Plus worth it? What happens if you miss your flight? What if you are introverted and you are nervous about traveling in a big group? We'll answer all those questions and more in this podcast. I can't wait for you to listen in and get pumped for your next EF Ultimate Break trip. Vegan Bytes VEG3 | Your Vegan AI Welcome to Vegan Bytes - your two-minute destination for everything you need to supercharge your vegan journey or advocacy efforts, all served up by VEG3’s cutting-edge AI.That’s right, every byte of this podcast is generated by artificial intelligence, turning complex research into digestible tips, blending in a dash of vegan-friendly humor, and wrapping it all in a short story that will leave you hooked. From the latest trends in veganism and AI to tried and tested strategies, each episode serves up relevant real-world examples and actionable insights that you can immediately implement to accelerate the vegan revolution.Whether you’re a seasoned vegan or a newcomer to the plant-based world, our AI has cooked up the perfect recipe to help you drive the cause forward.Tune in, grab a vegan snack, and let’s create a world where animal rights are recognized, and the vegan movement thrives.Learn more about VEG3, your AI-powered vegan companion, at https://veg3.ai Business Bytes: Social Media Uncovered. Matt Mckay Business Bytes: Social Media Uncovered' is your premier destination for digestible, bite-sized information, unveiling the latest trends, and transformative tools that can catapult your business to unprecedented heights. We are committed to demystifying the intricacies of social media, making it accessible and engaging for everyone. Whether you're an entrepreneur on the brink of a breakthrough, a marketer aiming to conquer new frontiers, or a social media enthusiast keen to stay ahead of the curve, our podcast is tailored just for you. Our goal is to empower you with the knowledge and insights to harness the power of social media effectively. So, sit back, relax, and join us on this exhilarating journey as we delve into the dynamic, ever-evolving world of social media, one byte at a time. Welcome aboard 'Business Bytes: Social Media Uncovered' - your key to unlocking the secrets of social media success! DevOps and Docker Talk: Cloud Native Interviews and Tooling Bret Fisher Interviews from Bret Fisher's live show with co-host Nirmal Mehta. Topics cover container and cloud topics like Docker, Kubernetes, Swarm, Cloud Native development, DevOps, SRE, GitOps, DevSecOps, platform engineering, and the full software lifecycle. Full show notes and more info available at https://podcast.bretfisher.com

Frequently Asked Questions

How long is this episode of Kubernetes Bytes?

This episode is 55 minutes long.

When was this Kubernetes Bytes episode published?

This episode was published on October 24, 2024.

What is this episode about?

Can I download this Kubernetes Bytes episode?

Yes, you can download this episode by clicking the download button on the episode player, or subscribe to the podcast in your preferred podcast app for automatic downloads.

URL copied to clipboard!