Skip to content
The Next Step Beyond LLMs: Foundation Models for Inference episode artwork

Jul 22, 2026 5 min

The Next Step Beyond LLMs: Foundation Models for Inference

from Learning Bayesian Statistics Hosted by Alexandre Andorra

Today's clip is from episode 161, featuring Luigi Acerbi. In this conversation, Luigi explains one of the biggest engineering bottlenecks facing transformer-based probabilistic models—and how his group found a way around it. The core challenge is that many inference models treat data as an unordered set, making them naturally permutation invariant. That's statistically elegant, but computationally painful: every time a new data point arrives, the model has to recompute attention over the entire dataset from scratch, preventing the kind of KV caching that makes modern language models so efficient. Luigi walks through his team's solution: a hybrid architecture that keeps the original context fully set-based while introducing a causal-attention buffer for newly arriving data. The result is dramatically faster inference- up to 100× faster in some settings - opening the door to applications like reinforcement learning, active data acquisition, and, ultimately, Luigi's long-term vision of a foundation model for Bayesian inference. Get the full discussion here Support & Resources → Support the show on Patreon → Bayesian Modeling Course (first 2 lessons free) Our theme music is « Good Bayesian », by Baba Brinkman (feat MC Lars and Mega Ran). Check out his awesome work

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

The Next Step Beyond LLMs: Foundation Models for Inference

0:00 5:35

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of Learning Bayesian Statistics?

This episode is 5 minutes long.

When was this Learning Bayesian Statistics episode published?

This episode was published on July 22, 2026.

Can I download this Learning Bayesian Statistics episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.