Deep Dive into Inference Optimization for LLMs with Phili...

What this episode covers

Today we have Philip Kiely from Baseten on the show. Baseten is a Series B startup focused on providing infrastructure for AI workloads. We go deep on Inference Optimization. We cover choosing a model, discuss the hype around Compound AI, choosing an Inference Engine, Optimization Techniques like Quantization and Speculative Decoding all the way down to your GPU choice.

Share this episode

Similar Episodes

Episode 62: Cautionary Tales From A Linux Beginner (with Edward Crisler)

Mar 10, 2026 ·83m

Games For Everyone #2: Linux Gaming Is Winning

Feb 17, 2026 ·94m

Episode 61: Down The NAS Rabbit Hole (with Joe Ressington)

Jan 19, 2026 ·90m

Games For Everyone #1: Valve's Third Act

Jan 5, 2026 ·98m

Episode 60: The Computer Upcycle Project (with Mike Kelly)

Dec 22, 2025 ·85m

Episode 59: Valve’s ARM Gamble + The Future of Linux VR (with Graham Morrison)

Dec 8, 2025 ·56m

Similar Podcasts

Linux For Everyone Jason Evangelho An upbeat, conversational show about the exciting world of desktop Linux, open source software, and the community creating it. What The Tech Podcast guysfromqueens Being on the front-line of new media and technology themselves, Andrew and Paul discuss the many new technologies being introduced to the world everyday. From the very controversial, to the unboxings to software reviews, "What the Tech?!" allows tech-junkies and novices alike stay updated on the latest technology news. All Things Techie Podcast Xtreme Media / Xtreme Technology Solutions All Things TechIE is a technology-focused podcast that covers a wide range of topics related to the latest developments in the world of tech. Hosted by experts in the field, the podcast offers insights and analysis on various aspects of technology, including software, hardware, gadgets, and trends in the industry.The podcast is designed to provide listeners with a comprehensive understanding of the latest advancements in technology, as well as tips and tricks to help them make the most of their tech devices. Each episode features interviews with leading experts and innovators in the field, as well as reviews and recommendations of the latest products and services.Whether you are a seasoned tech enthusiast or just starting to explore the world of technology, All Things TechIE has something to offer. The podcast is engaging, informative, and provides an in-depth look at the latest trends and developments in the tech world. Tune in to stay up-to-date with the latest advancements in te Tech Of The Future With AppSumo (Hosted by Chris Cownden) Christopher Paul Cownden Visionaries like you and I see the world differently—and deserve the right tech to match. Listen to the people behind our favorite AppSumo tools on the Tech Of The Future podcast including BIGVU, VistaSocial, Sessions, VBOUT, Castmagic and lots more. Listen out for exclusive deals and best practices for using these tools to your advantage as a podcaster, entrepreneur or content creator. Topics include ai, software, digital marketing, productivity, email marketing, content repurposing, podcast hosting, search engine optimization and podcast editing. Guests include: Jacob Bozarth, Richard Fallah, Blaine Bolus, Rob Winters, Melinda Wittstock, Kareem Mostafa, Dante Healy, Lisa Khera, Tonya Gossage, Jonathan Reid, JaMarr John Johnson and Austin Armstrong, SA Grant, Phil Better, Mike Cavaggioni, JaMarr John Johnson and Noah Kagan. Hosted by Chris Cownden in partnership with AppSumo.

Frequently Asked Questions

How long is this episode of Software Huddle?

This episode is 1 hour and 4 minutes long.

When was this Software Huddle episode published?

This episode was published on November 5, 2024.

What is this episode about?

Today we have Philip Kiely from Baseten on the show. Baseten is a Series B startup focused on providing infrastructure for AI workloads. We go deep on Inference Optimization. We cover choosing a model, discuss the hype around Compound AI, choosing...

Can I download this Software Huddle episode?

Yes, you can download this episode by clicking the download button on the episode player, or subscribe to the podcast in your preferred podcast app for automatic downloads.