#169 Bayesian Workflow for Beginners, Alex at Princeton
from Learning Bayesian Statistics Hosted by Alexandre Andorra
Support & Resources → Support the show on Patreon → Bayesian Modeling Course (first 2 lessons free) Our theme music is « Good Bayesian », by Baba Brinkman (feat MC Lars and Mega Ran). Check out his awesome work! Takeaways: Q: What does a mixed model in psychology actually assume? A: Written as a generative story, a standard repeated-measures model says participants are drawn from a shared population, each has one latent ability that stays stable across sessions, a condition effect shifts everyone by the same amount, and trials within a session are independent. Each of those is testable and can be wrong: abilities drift between sessions, some participants react more to a hard condition than others (an interaction), and performance can be autocorrelated, the way a tennis player losing several points in a row starts getting in their own head. Frequentist software like lme4 makes these assumptions silently; writing the model in PyMC puts them in the open. Q: What is the generative story of a model and why does it matter? A: The generative story is a description of how the data you observe came to be. In a typical psychology experiment: a population of people, participants drawn from it, several sessions per participant, and several trials per session. That nesting already tells you the model needs a hierarchy. In the Bayesian workflow, you build the model by translating that story line by line into distributions, which is why you can write the whole model before seeing any data, and why the model can simulate data on its own. Q: Should you start with a complex model or a simple one? A: Start with the simplest, dumbest model you can find. A complex model that doesn't sample is very hard to debug, while a simple model that samples well gives you a base to add one layer of complexity at a time. Likewise, don't start with interactions: fit the main effects first, and if one comes out with a very large posterior mean or a very wide posterior, that can be a sign an interaction is missing (unless there's a difference of scale in the data), especially in the social sciences where effects are usually modest. Full takeaways here! Chapters: 02:45 What assumptions hide under the hood of the mixed models psychologists fit? 05:58 What does Bayes' theorem say about updating your beliefs with data? 09:14 What is the generative story of your data and how does it become a model? 11:59 How do lme4, brms and PyMC express the same mixed model?15:02 How does a hierarchical model encode participants and their population? 18:03 Which hidden assumptions does a standard mixed model make? 27:17 What is a prior predictive check and why are default priors like Normal(0, 10) risky? 36:39 How do you read posterior distributions and credible intervals?40:53 What is partial pooling and why do hierarchical models shrink extreme participants? 46:02 How do you check whether a Bayesian model is well calibrated?52:20 How can an AI agent skill help you run the Bayesian workflow? Thank you to my Patrons for making this episode possible! Full links from the show here.
Embed this episode
No transcript yet.