The Scientific Ritual

Science feels like the most reliable thing we have. The opposite of belief. But it’s a belief system itself—a ritual, with all the failure modes that rituals have. And the receipts are right there in the replication crisis. Further reading The Scientific Ritual — the article this lecture is based on Problems with p-values — the technical companion: Fisher, Neyman-Pearson, the hybrid mess The trap of scientific evidence — on the “no evidence” tension and the homeopathy/parachute paradox Everything is ideology — science as one belief system among several In praise of the sage — other ways of knowing; the MD/PhD distinction Scientific fact — on what science actually does The value of ritual — ritual as a knowledge-production strategy Meditation — on the dinner-table meditation example Beyond System 1 and System 2 — on Kahneman’s dual-process framework The placebo effect — on why “works for some, not for others” is a feature, not a bug Grit — positive-psychology critique Overengineering calming down (lecture) — the broader positive-psychology audit Bias is good (lecture) — the cognitive-bias series Life is worse (lecture) — the previous episode; a worked example of reading a literature References The replication crisis itself Open Science Collaboration (2015), Estimating the reproducibility of psychological science, Science 349 (6251) Wikipedia: replication crisis American Statistical Association: Wasserstein, Schirm & Lazar (2019), Moving to a World Beyond “p < 0.05” Statistical ritualism Gerd Gigerenzer (2018), Statistical Rituals: The Replication Delusion and How We Got There, Advances in Methods and Practices in Psychological Science Philip B. Stark & Andrea Saltelli (2018), Cargo-cult statistics and scientific crisis, Significance 15 (4) Andrew Gelman & Eric Loken (2014), The Statistical Crisis in Science — the “garden of forking paths” paper Andrew Gelman, Why I don’t like so-called Bayesian hypothesis testing p-values, Bayes factors, and software Wikipedia: p-value, Bayes factor Ronald A. Fisher (1925), Statistical Methods for Research Workers — where the 5% threshold appears as an illustrative example Harold Jeffreys (1939), Theory of Probability — where the Bayes-factor thresholds (BF > 3 substantial, BF > 10 strong) come from JASP — the open-source Bayesian statistics software with default priors Specific replication-crisis casualties Cuddy, Wilmuth & Carney (2010) original power posing paper; Carney’s later statement withdrawing support Hagger et al. (2016), A Multilab Preregistered Replication of the Ego-Depletion Effect Bargh, Chen & Burrows (1996) original elderly priming paper; failed Doyen et al. (2012) replication Brown, Sokal & Friedman (2013), The Complex Dynamics of Wishful Thinking — demolishing the 3:1 positivity ratio Carol Dweck, growth mindset — replication concerns documented in Sisk et al. (2018) and Bahník & Vranka (2017) Angela Duckworth, grit — meta-analytic critique in Credé, Tynan & Harms (2017) Books cited in the lecture Daniel Kahneman, Thinking, Fast and Slow Stephen J. Gould, Adam’s Navel and Other Essays Yann Martel, Life of Pi Bill Mollison, Permaculture: A Designer’s Manual Other Richard Dawkins on militant atheism (TED) — the “evidence vs. faith” framing Reform efforts: preregistration, open data, multi-lab replication consortia (e.g. ManyLabs)