Skip to content
Rethinking LLMs: Why Diffusion Could Change How We Build AI | Dr. Subham Sahoo episode artwork

Oct 5, 2026 49 min

Rethinking LLMs: Why Diffusion Could Change How We Build AI | Dr. Subham Sahoo

from Jay Shah Podcast Hosted by Jay Shah

Subham Sahoo is a Research Scientist and Team Lead at the Institute of Foundation Models at MBZUAI, where he leads the Diffusion LLM team. His research focuses on discrete diffusion language models and the question of whether diffusion can offer a fundamentally different, and potentially more efficient way of generating language compared with traditional autoregressive models.Prior to that, Subham completed his Ph.D. at Cornell Tech. And his recent work includes studies of how diffusion models could be scaled and evaluated against autoregressive models. His work has helped push diffusion LMs from an interesting alternative toward a much more serious research direction for language generation.00:00:00 Highlight & Introduction00:02:10 Journey into AI & First Interest in Diffusion Models00:11:37 Learning the Mathematics Behind Diffusion Models00:16:58 Research, Publishing Pressure & Finding the Right Validation00:20:31 How to Filter Noise from AI Research00:23:32 Academia vs. Industry After a PhD00:26:32 Diffusion Models vs. Autoregressive LLMs00:29:01 How Blockwise Diffusion Solves the Latency Problem00:30:57 Why Autoregressive LLMs Still Have Room for Improvement00:38:00 How Hybrid Diffusion + Autoregressive Models Work00:41:50 Can Blockwise Prediction Preserve Model Quality?00:44:03 Will Diffusion Replace Autoregressive LLMs?00:47:52 Does Diffusion Training Make LLMs Learn Gibberish?00:50:45 Comparing Training Costs & Scaling00:55:00 How Hybrid Models Scale with More Data00:59:02 Lossy vs. Lossless Speedups01:02:09 Are Hybrid Diffusion Models Production Ready?01:04:54 How to Choose a Good AI Research Direction01:08:06 Advice for Early-Career AI Researchers01:11:39 Closing ThoughtsAbout the Host:Jay is a Machine Learning Engineer III at PathAI working on improving AI for medical diagnosis and prognosis. Linkedin: https://www.linkedin.com/in/shahjay22/Twitter: https://twitter.com/jaygshah22Homepage: https://jaygshah.github.io/ for any queries.Stay tuned for upcoming webinars!***Disclaimer: The information in this video represents the views and opinions of the speaker and does not necessarily represent the views or opinions of any institution. It does not constitute an endorsement by any Institution or its affiliates of such video content.***

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Rethinking LLMs: Why Diffusion Could Change How We Build AI | Dr. Subham Sahoo

0:00 49:20

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of Jay Shah Podcast?

This episode is 49 minutes long.

When was this Jay Shah Podcast episode published?

This episode was published on October 5, 2026.

Can I download this Jay Shah Podcast episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.