Skip to content
Goodput, Not Just Throughput: Prefill, Decode, and Rethinking AI Inference episode artwork

Jan 5, 2026 5 min

Goodput, Not Just Throughput: Prefill, Decode, and Rethinking AI Inference

from Intellectually Curious Hosted by Mike Breault

We unpack the core bottleneck in streaming AI: the split between heavy pre-fill computations and fast, memory-light decoding. From chunked prefill to physical separation (DissServe) and logical isolation (DuetServe), we explore how phase isolation eliminates interference, delivering 2x–4.5x better goodput and transforming cost efficiency. Join us as we translate GPU architecture ideas into scalable, user-friendly AI services, with practical takeaways for builders, operators, and decision-makers. Note: This podcast was AI-generated, and sometimes AI can make mistakes. Please double-check any critical information. Sponsored by Embersilk LLC

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Goodput, Not Just Throughput: Prefill, Decode, and Rethinking AI Inference

0:00 5:23

No transcript yet.

Requests help us decide what to transcribe next.

Share this episode

Frequently asked questions

How long is this episode of Intellectually Curious?

This episode is 5 minutes long.

When was this Intellectually Curious episode published?

This episode was published on January 5, 2026.

Can I download this Intellectually Curious episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.