Ten Weirdest AI Evals Ever Published
from My Weird Prompts
What happens when researchers need to measure machine intelligence and end up grading puns, emoji, and a joke with the punchline surgically removed? This episode counts down ten of the strangest AI evaluations ever published — from BIG-bench's "ruin names" task to a high schooler's Minecraft model leaderboard, from a canary string that detects cheating to Claude stuck in Mount Moon for eighty hours. Along the way: why someone hand-built a joke detector, why a cryobiology quiz is written in Spanish, and what a pelican that was never drawn tells us about how we actually test these systems. Episode 6213 — myweirdprompts.com/n/6213 (permanent short link: myweirdprompts.com/264324) Listen on Android: MWP FM Player — myweirdprompts.com/apk
Embed this episode
Chapters
No transcript yet.