Episode 36: About That 'Dangerous Capabilities' Fanfiction (feat. Ali Alkhatib), June 24 2024

When is a research paper not a research paper? When a big tech company uses a preprint server as a means to dodge peer review -- in this case, of their wild speculations on the 'dangerous capabilities' of large language models. Ali Alkhatib joins Emily to explain why a recent Google DeepMind document about the hunt for evidence that LLMs might intentionally deceive us was bad science, and yet is still influencing the public conversation about AI.Ali Alkhatib is a computer scientist and former director of the University of San Francisco’s Center for Applied Data Ethics. His research focuses on human-computer interaction, and why our technological problems are really social – and why we should apply social science lenses to data work, algorithmic justice, and even the errors and reality distortions inherent in AI models.References:Google DeepMind paper-like object: Evaluating Frontier Models for Dangerous CapabilitiesFresh AI Hell:Hacker tool extracts all the data collected by Windows' 'Recall' AIIn NYC, ShotSpotter calls are 87 percent false alarms"AI" system to make callers sound less angry to call center workersAnthropic's Claude Sonnet 3.5 evaluated for "graduate level reasoning"OpenAI's Mira Murati says "AI" will have 'PhD-level' intelligenceOpenAI's Mira Murati also says AI will take some creative jobs, maybe they shouldn't have been there to start out withCheck out future streams on Twitch. Meanwhile, send us any AI Hell you see.Find our book The AI Con here, and MAIHT3k merch here.Subscribe to our newsletter via Buttondown.Follow us!EmilyBluesky: emilymbender.bsky.socialMastodon: dair-community.social/@EmilyMBenderAlexBluesky: alexhanna.bsky.socialMastodon: dair-community.social/@alexTwitter: @alexhannaMusic by Toby Menon.Artwork by Naomi Pleasure-Park. Production by Ozzy Llinas Goodman.