How a Frozen Model Went From 2% to 77% on Physics Puzzles — Without Retraining

How a Frozen Model Went From 2% to 77% on Physics Puzzles — Without Retraining Source: https://arxiv.org/abs/2606.29315 Paper was published on June 28, 2026 This episode was AI-generated on June 30, 2026. The script was written by an AI language model and the host voices were synthesized by Eleven Labs. The producer is not affiliated with Anthropic or Eleven Labs. The same Claude Sonnet model that solves 2% of a 2D physics puzzle climbs to roughly three-quarters — and not a single weight changes. The trick is letting the agent keep a lab notebook of its own experiments, and it turns out that frozen-brain-plus-evolving-notes can beat gradient training when you're short on reps. We unpack how it works, why failures teach more than lucky wins, and where the headline number is partly a story about how low the starting line was. Key Takeaways: - Why a model that can recite catapult physics still fails 98% of the time on one specific configuration — the gap between recalling and experimenting - How HExA wraps a standard ReAct loop in an outer 'actor / evolver / retriever' loop that turns batches of attempts into a capped, ranked notebook of reusable skills - Why a thorough, exploratory failure is rewarded more than an early quit — and how that solves the credit-assignment problem a lucky win can't - How in-context notes beat weight-updating GRPO at a matched 50-seed budget, with GRPO needing 3x the seeds just to catch up - The transfer result: 8% to 44% on the catapult using abstractions distilled from easier levels the agent never actually probed - The honest caveats — the benchmark was built by the same team with tidy experimentation tools, 2% is the floor, results are noisy (67% ±9), and a wrong-priors domain could let the evolver distill confident, plausible, incorrect skills 02:11 - Why does knowing physics not help?: Sets up the Interphyre catapult puzzle and the gap between a model that understands catapults in the abstract and one that can solve a specific unseen configuration. 03:39 - The agent that walks into the same wall fifty times: Explains the ReAct baseline and Reflexion, and the killer limitation: no lasting, reusable memory between puzzles. 04:41 - A frozen brain with a lab notebook: Introduces HExA's three hats — actor, evolver, retriever — and the analogy of a scientist who can't get smarter but keeps a compounding notebook. 06:09 - The single best line in the paper: Walks through seed 45 side-by-side, where ReAct micro-tunes for 25 turns and HExA pulls a skill that says to stop tuning the radius and reposition entirely. 09:06 - Why a thorough failure beats a lucky win: Breaks down the two-pass distillation, partial skills mined from failures, the credit-assignment problem, and why rewards use discrete buckets for an LLM reader rather than a gradient. 13:26 - Does it hold up beyond one seed?: The headline results: catapult ~67% (up to 77%), an open model from 0 to 54%, fewer turns per seed, and a 44% transfer result from abstractions on unprobed levels. 15:40 - Notebook versus muscle memory: Compares HExA to gradient-based GRPO, showing in-context learning wins at low data because insights are usable on the very next attempt — while conceding GRPO eventually catches up. 17:14 - Where the skeptic should push back: The reservations: a benchmark built by the same authors with handy tools, 2% being the floor, noisy variance, and the risk of an evolver distilling wrong principles in a domain with bad priors. 20:12 - The idea that outlives the method: The durable takeaway — agents should remember search strategy and meta-knowledge over answers — and the open question of whether editable memory is a bootstrap or the destination. Recommended Reading: - ReAct: Synergizing Reasoning and Acting in Language Models: The exact baseline agent the episode pits against HExA — the memoryless thought-then-action loop that 'walks into the same dead ends fifty times.' (https://arxiv.org/abs/2210.03629) - Reflexion: Language Agents with Verbal Reinforcement Learning: The 'close cousin' the hosts name explicitly — self-reflection between attempts that, unlike HExA's notebook, never builds a lasting reusable library. (https://arxiv.org/abs/2303.11366) - DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models: Introduces GRPO, the gradient-based RL method HExA is benchmarked against in the 'notes beat muscle memory' low-data comparison. (https://arxiv.org/abs/2402.03300) - PHYRE: A New Benchmark for Physical Reasoning: The original 2D physics-puzzle benchmark that Interphyre is built on top of, including the catapult-style mechanics the episode dissects seed by seed. (https://arxiv.org/abs/1908.05656)