24: Bayes' Rule - The Formula For Learning Everything

If you've ever debugged a program, looked for lost socks or tried to figure out why red spots are developing on your skin, then Bayes' rule was almost certainly used to help you on your journey. Even if you don't know anything about it. Humans have evolved to solve problems but along the way, we as a species sometimes fall for traps or fail to consider all the evidence when figuring things out.In this episode, Wolf explains what Bayes' rule is, how we use it and how we could use it better to solve our mysteries.One sentenceBayes' Rule is the formula that tells you how to update what you believe when you get new evidence — it combines what was already true with what you just learned.The mathThe probability of A given B equals the probability of B given A, times the probability of A, divided by the probability of BP(A | B) = P(B | A) * P(A) / P(B)Key conceptsBayes' Rule — the formula for updating what you believe when you get new evidenceRepresentativeness heuristic — substituting "how well does this match?" for "how likely is this?" (ignoring base rates)Base rate neglect — the tendency to ignore population-level frequencies when evaluating specific casesPrior / likelihood / posterior — what you believed before, how likely the evidence is, what you should believe nowSystem 1 / System 2 — Kahneman's framework for fast intuitive thinking vs. slow deliberate reasoningThe Tom W problemFrom Kahneman's Thinking, Fast and Slow, Chapter 14. A personality description that tricks you into ignoring base rates. The Sin of Representativeness — Unearned WisdomThe cab problemAlso from Kahneman. A witness, a hit-and-run, and the surprising math of why 80% reliability doesn't mean 80% probability. Kahneman's Bayesian inference exampleBooks:Daniel Kahneman, Thinking, Fast and Slow (2011) — the Tom W problem, the cab problem, System 1/System 2, representativenessSharon Bertsch McGrayne, The Theory That Wouldn't Die (2011) — the history of Bayes' theorem from its discovery through the frequentist wars to its modern resurgenceDouglas Hofstadter, Gödel, Escher, Bach: An Eternal Golden Braid (1979) — a Pulitzer-winning exploration of how self-reference and formal systems connect mathematics, art, and musicErnest Nagel and James R. Newman, Gödel's Proof (1958) — a concise, accessible walkthrough of Gödel's Incompleteness Theorems for non-mathematiciansHistorical:Thomas Bayes (1701–1761) — Presbyterian minister who first derived the theorem; never published it. Richard Price submitted it posthumously.Pierre-Simon Laplace — independently derived and generalized Bayes' work; arguably did the heavier mathematical liftingTools (if you want to go deeper):PyMC — Python library for Bayesian statistical modelingBayes' theorem — WikipediaThinking, Fast and Slow — WikipediaHosts:Jim McQuillan can be reached at [email protected] can be reached at [email protected] us on Mastodon: @[email protected] you have feedback for us, please send it to [email protected] our webpage at http://RuntimeArguments.fmTheme music:Dawn by nuer self, from the album Digital Sky