Episode 6: Stochastic Parrot Galactica, November 23, 2022

Emily and Alex discuss MetaAI's bullshit science paper generator, Galactica, along with its defenders. Plus, where could AI actually help scientific research? And more Fresh AI Hell. Watch the video of this episode on PeerTube. References:Imre Lakatos on research programsShah, Chirag and Emily M. Bender. 2022. Situating Search. Proceedings of the 2022 ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR ’22). UW RAISE (Responsibility in AI Systems and Experiences)Stochastic Parrots:Bender, Emily M., Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜. In Proceedings of FAccT 2021, pp.610-623. The Octopus Paper:Bender Emily M. and Alexander Koller. 2020. Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data. ACL 2020Palestinian man arrested because of bad machine translation.Katherine McKittrick, Dear Science and Other StoriesThe Sokal HoaxSafiya Noble, Algorithms of OppressionLatanya Sweeney, "Discrimination in Online Ad Delivery"Mehtab Khan and Alex Hanna, The Subjects and Stages of AI Dataset Development: A Framework for Dataset Accountability(What is 'sealioning'?)http://wondermark.com/1k62/Grover:Raji, Inioluwa Deborah, Emily M. Bender, Amandalynne Paullada, Emily Denton and Alex Hanna. 2021. AI and the Everything in the Whole Wide World Benchmark. Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS 2021) Track on Datasets and Benchmarks.Ben Dickson's coverage of Grover:Why we must rethink AI benchmarks, Ben Dickson, TechTalks, December 6, 2021.Sociology of science work by Roberta SinatraCheck out future streams on Twitch. Meanwhile, send us any AI Hell you see.Find our book The AI Con here, and MAIHT3k merch here.Subscribe to our newsletter via Buttondown.Follow us!EmilyBluesky: emilymbender.bsky.socialMastodon: dair-community.social/@EmilyMBenderAlexBluesky: alexhanna.bsky.socialMastodon: dair-community.social/@alexTwitter: @alexhannaMusic by Toby Menon.Artwork by Naomi Pleasure-Park. Production by Ozzy Llinas Goodman.