An AI Designed Its Own Psychology Studies, Then Confirmed What It Found

An AI Designed Its Own Psychology Studies, Then Confirmed What It Found Source: https://arxiv.org/abs/2606.26448 Paper was published on June 24, 2026 This episode was AI-generated on June 26, 2026. The script was written by an AI language model and the host voices were synthesized by Eleven Labs. The producer is not affiliated with Anthropic or Eleven Labs. A system called AutoCog designed psychology experiments, paid 250 real people to take them, diagnosed why its own theories failed, and rediscovered one of the deepest principles in decision science — then locked in predictions and confirmed them. It's the first time an AI has closed the full scientific loop with no researcher in the chair. But the headline runs ahead of the evidence, and the honest version turns out to be more interesting than either the hype or the cynicism. Key Takeaways: - How AutoCog closes the full discovery loop — designing experiments, paying online participants, diagnosing failures, and revising theories — with no human in the chair - Why the system scores theories by whether they can generate human-like behavior rather than by fitting data, and why that guards against overfitting - How three classic decision rules — Take-the-Best, Tallying, and WADD — collapse into endpoints of a single tunable dial - The flagship 'discovery,' Diminishing Returns WADD, turns out to be a fresh instance of Kahneman and Tversky's prospect theory - Where the headline overreaches: a friendly domain, a search that stayed local, an unaudited gap between the verbal theory and its code, and one thin confirmation - Why the durable result may not be the finding itself, but the idea that theory-building can become an auditable, resumable trace instead of a private flash of insight 00:04 - Can you automate the creative leap?: Sets up the frontier question — whether the irreducibly human act of inventing a better theory can be handed to an AI. 03:24 - Two blenders and three rival rules: Grounds the task in multi-attribute decision-making and lays out the three classic strategies the AI works with. 05:42 - Two lawyers, one self-correcting wheel: Walks through the four-stage loop of advocate agents, a simulate-before-collect gate, and a neutral arbiter that rebuilds the loser. 07:58 - Why it refuses to Photoshop the data: Explains the generate-don't-fit scoring metric and the unification pressure that quietly forces parsimony. 10:52 - Can it find an answer it was never given?: The validation phase — recovering hidden strategies, even deliberately bizarre anti-theories, and reporting noise honestly instead of inventing structure. 14:52 - When three theories become one knob: The first live deployment on 250 real people, where the winning model unifies the three rivals as settings of a single dial. 17:33 - The discovery it didn't go looking for: One small change surfaces Diminishing Returns WADD — a concave curve that turns out to be prospect theory, confirmed by a preregistered study. 22:19 - Where the headline runs out ahead: The skeptic's case — a friendly domain, a local search, a known mechanism, and an unaudited verbal-to-code gap. 26:46 - The creative leap, finally logged: Why an auditable, resumable trace of the discovery process may matter more than the specific finding, and what comes next. Recommended Reading: - Centaur: a foundation model of human cognition: The Helmholtz Munich line of work on behavioral foundation models that simulate human choices, the in-silico stand-in the episode flags as the conditional forward path for cheaper discovery loops. (https://arxiv.org/abs/2410.20268)