EPISODE · Dec 1, 2025 · 1H 2M
Holiday Survival Guide Part 2: The survey study edition
from Normal Curves: Sexy Science, Serious Statistics · host Regina Nuzzo and Kristin Sainani
Does the temperature of your coffee six months ago really predict whether you feel gassy today? This week we dissect a new nutrition survey study on hot and cold beverage habits that claims to connect drink temperature with gut symptoms, anxiety, and more—despite relying on year-old memories and a blizzard of statistical tests. It’s the perfect case study for our Holiday Survival Guide Part 2, where we teach you how to talk with Uncle Joe at the dinner table about one of the most common—and most fraught—study designs in science: cross-sectional surveys. We walk through our easy checklist for making sense of results, show how recall bias and measurement error can skew the story, and reacquaint you with nonmonogamous Multiple-Testing Dude, who’s been very busy in this dataset. A friendly, practical guide to spotting when researchers are just torturing the data until it confesses.Statistical topicsConfoundingCross-sectional studiesFalse positivesMeasurement errorMultiple testingPICOT / PIVOT frameworkRecall biasResearch hypothesesSample size and powerSignal vs. noiseSMART frameworkStatistical significanceSubgroup analysesSurvey designTransparency and trustworthinessMethodological morals“When your measurement starts with ‘think back to last winter’ you might as well use a random number generator.”“If the effect is only significant in certain subgroups in certain seasons for certain outcomes, it might just be a bad case of gas.”ReferencesWu T, Doyle C, Ito J, et al. Cold Exposures in Relation to Dysmenorrhea among Asian and White Women. Int J Environ Res Public Health. 2023;21(1):56. Published 2023 Dec 30. doi:10.3390/ijerph21010056Wu T, Ramesh N, Doyle C, Hsu FC. Cold and hot consumption and health outcomes among US Asian and White populations. Br J Nutr. Published online September 18, 2025. doi:10.1017/S000711452510514XKristin and Regina’s online courses: Demystifying Data: A Modern Approach to Statistical Understanding Clinical Trials: Design, Strategy, and Analysis Medical Statistics Certificate Program Writing in the Sciences Epidemiology and Clinical Research Graduate Certificate Program Programs that we teach in:Epidemiology and Clinical Research Graduate Certificate Program Find us on:Kristin - LinkedIn & Twitter/XRegina - LinkedIn & ReginaNuzzo.com(00:00) - Intro (04:36) - Did they have real research hypotheses? (10:29) - Observational or randomized experiment? (19:25) - PICOT and PIVOT (25:36) - Memory problems (31:19) - Five outcomes and measurement problems therein (36:12) - SMART (40:31) - Multiple Testing Dude is having a great time (51:17) - How big is the effect? (57:47) - Wrap-up and Irish Coffee rating scale
What this episode covers
Does the temperature of your coffee six months ago really predict whether you feel gassy today? This week we dissect a new nutrition survey study on hot and cold beverage habits that claims to connect drink temperature with gut symptoms, anxiety, and more—despite relying on year-old memories and a blizzard of statistical tests. It’s the perfect case study for our Holiday Survival Guide Part 2, where we teach you how to talk with Uncle Joe at the dinner table about one of the most common—and most fraught—study designs in science: cross-sectional surveys. We walk through our easy checklist for making sense of results, show how recall bias and measurement error can skew the story, and reacquaint you with nonmonogamous Multiple-Testing Dude, who’s been very busy in this dataset. A friendly, practical guide to spotting when researchers are just torturing the data until it confesses.Statistical topicsConfoundingCross-sectional studiesFalse positivesMeasurement errorMultiple testingPICOT / PIVOT frameworkRecall biasResearch hypothesesSample size and powerSignal vs. noiseSMART frameworkStatistical significanceSubgroup analysesSurvey designTransparency and trustworthinessMethodological morals“When your measurement starts with ‘think back to last winter’ you might as well use a random number generator.”“If the effect is only significant in certain subgroups in certain seasons for certain outcomes, it might just be a bad case of gas.”ReferencesWu T, Doyle C, Ito J, et al. Cold Exposures in Relation to Dysmenorrhea among Asian and White Women. Int J Environ Res Public Health. 2023;21(1):56. Published 2023 Dec 30. doi:10.3390/ijerph21010056Wu T, Ramesh N, Doyle C, Hsu FC. Cold and hot consumption and health outcomes among US Asian and White populations. Br J Nutr. Published online September 18, 2025. doi:10.1017/S000711452510514XKristin and Regina’s online courses: Demystifying Data: A Modern Approach to Statistical Understanding Clinical Trials: Design, Strategy, and Analysis Medical Statistics Certificate Program Writing in the Sciences Epidemiology and Clinical Research Graduate Certificate Program Programs that we teach in:Epidemiology and Clinical Research Graduate Certificate Program Find us on:Kristin - LinkedIn & Twitter/XRegina - LinkedIn & ReginaNuzzo.com(00:00) - Intro (04:36) - Did they have real research hypotheses? (10:29) - Observational or randomized experiment? (19:25) - PICOT and PIVOT (25:36) - Memory problems (31:19) - Five outcomes and measurement problems therein (36:12) - SMART (40:31) - Multiple Testing Dude is having a great time (51:17) - How big is the effect? (57:47) - Wrap-up and Irish Coffee rating scale
NOW PLAYING
Holiday Survival Guide Part 2: The survey study edition
No transcript for this episode yet
Similar Episodes
Jun 15, 2022 ·8m
May 25, 2022 ·20m
May 19, 2022 ·16m
May 15, 2022 ·34m
May 12, 2022 ·1m