EPISODE · May 15, 2025 · 12 MIN
Enough Coin Flips Can Make LLMs Act Bayesian
from Best AI papers explained · host Enoch H. Kang
This paper investigates whether Large Language Models (LLMs) utilize in-context learning (ICL) to perform reasoning consistent with a Bayesian framework. By using a simplified setting of biased coin flips and dice rolls, the authors analyze how LLMs update their internal probabilities based on provided examples. They find that LLMs often start with inherent biases (miscalibrated priors) but demonstrate behavior that broadly follows Bayesian updates when given sufficient evidence through ICL. The study indicates that deviations from true Bayesian inference primarily stem from initial poor priors rather than flawed updating mechanisms. Furthermore, the research suggests that attention magnitude has minimal impact on the Bayesian inference process in these models and that instruction-tuned models may exhibit shorter temporal horizons in their updates.
What this episode covers
This paper investigates whether Large Language Models (LLMs) utilize in-context learning (ICL) to perform reasoning consistent with a Bayesian framework. By using a simplified setting of biased coin flips and dice rolls, the authors analyze how LLMs update their internal probabilities based on provided examples. They find that LLMs often start with inherent biases (miscalibrated priors) but demonstrate behavior that broadly follows Bayesian updates when given sufficient evidence through ICL. The study indicates that deviations from true Bayesian inference primarily stem from initial poor priors rather than flawed updating mechanisms. Furthermore, the research suggests that attention magnitude has minimal impact on the Bayesian inference process in these models and that instruction-tuned models may exhibit shorter temporal horizons in their updates.
NOW PLAYING
Enough Coin Flips Can Make LLMs Act Bayesian
No transcript for this episode yet
Similar Episodes
Mar 31, 2026 ·54m
Mar 27, 2026 ·14m
Mar 24, 2026 ·42m
Mar 20, 2026 ·42m
Mar 17, 2026 ·41m
Mar 13, 2026 ·44m