From AGI to ASI and Beyond

This episode explores DeepMind’s From AGI to ASI as a foresight report that treats human-level general intelligence not as the endpoint, but as a possible stepping stone toward systems that could outperform entire organizations in planning, research, engineering, and coordination. It breaks down how the paper defines AGI and the much more ambitious idea of ASI, then examines the conceptual tools behind that framing, including universal intelligence, AIXI as an idealized reference point, recursive self-improvement, collective intelligence, and the notion of effective compute. The discussion also probes the paper’s method, arguing that it is a structured synthesis of trends and bottlenecks rather than empirical proof, and questions how much precision is needed before such forecasts become meaningful. Listeners would find it interesting because it connects abstract AI theory, concrete scaling dynamics, and real uncertainty about whether progress in models, compute, and autonomy could compound into organization-level superintelligence. Sources: 1. From AGI to ASI — Tim Genewein, Matija Franklin, Alexander Lerchner, Laurent Orseau, Samuel Albanie, Adam Bales, Cole Wyeth, Stephanie Chan, Iason Gabriel, Joel Z. Leibo, Allan Dafoe, Marcus Hutter, Thore Graepel, Shane Legg, 2026 http://arxiv.org/abs/2606.12683 2. From AGI to ASI (https://arxiv.org/abs/2606.12683) — Tim Genewein, Matija Franklin, Alexander Lerchner, Marcus Hutter, Shane Legg, et al., 2026 https://scholar.google.com/scholar?q=From+AGI+to+ASI+%28https%3A%2F%2Farxiv.org%2Fabs%2F2606.12683%29 3. Can Intelligence Explode? (https://arxiv.org/abs/1202.6177) — Marcus Hutter, 2012 https://scholar.google.com/scholar?q=Can+Intelligence+Explode%3F+%28https%3A%2F%2Farxiv.org%2Fabs%2F1202.6177%29 4. Research Priorities for Robust and Beneficial Artificial Intelligence (https://arxiv.org/abs/1602.03506) — Stuart Russell, Daniel Dewey, Max Tegmark, 2015 https://scholar.google.com/scholar?q=Research+Priorities+for+Robust+and+Beneficial+Artificial+Intelligence+%28https%3A%2F%2Farxiv.org%2Fabs%2F1602.03506%29 5. Emerging Practices in Frontier AI Safety Frameworks (https://arxiv.org/abs/2503.04746) — Marie Davidsen Buhl, Ben Bucknall, Tammy Masterson, 2025 https://scholar.google.com/scholar?q=Emerging+Practices+in+Frontier+AI+Safety+Frameworks+%28https%3A%2F%2Farxiv.org%2Fabs%2F2503.04746%29 6. A Theory of Universal Artificial Intelligence based on Algorithmic Complexity (https://arxiv.org/abs/cs/0004001) — Marcus Hutter, 2000 https://scholar.google.com/scholar?q=A+Theory+of+Universal+Artificial+Intelligence+based+on+Algorithmic+Complexity+%28https%3A%2F%2Farxiv.org%2Fabs%2Fcs%2F0004001%29 7. Universal Intelligence: A Definition of Machine Intelligence (https://arxiv.org/abs/0712.3329) — Shane Legg, Marcus Hutter, 2007 https://scholar.google.com/scholar?q=Universal+Intelligence%3A+A+Definition+of+Machine+Intelligence+%28https%3A%2F%2Farxiv.org%2Fabs%2F0712.3329%29 8. A Monte Carlo AIXI Approximation (https://arxiv.org/abs/0909.0801) — Joel Veness, Kee Siong Ng, Marcus Hutter, William Uther, David Silver, 2009 (later published in JAIR, 2011) https://scholar.google.com/scholar?q=A+Monte+Carlo+AIXI+Approximation+%28https%3A%2F%2Farxiv.org%2Fabs%2F0909.0801%29 9. One Decade of Universal Artificial Intelligence (https://arxiv.org/abs/1202.6153) — Marcus Hutter, 2012 https://scholar.google.com/scholar?q=One+Decade+of+Universal+Artificial+Intelligence+%28https%3A%2F%2Farxiv.org%2Fabs%2F1202.6153%29 10. Universal Intelligence: A Definition of Machine Intelligence — Shane Legg, Marcus Hutter, 2007 https://scholar.google.com/scholar?q=Universal+Intelligence%3A+A+Definition+of+Machine+Intelligence 11. Levels of AGI for Operationalizing Progress on the Path to AGI — Meredith Ringel Morris, Jascha Sohl-Dickstein, Noah Fiedel, Tris Warkentin, Allan Dafoe, Aleksandra Faust, Clement Farabet, Shane Legg, 2023 https://scholar.google.com/scholar?q=Levels+of+AGI+for+Operationalizing+Progress+on+the+Path+to+AGI 12. AI as Normal Technology — Arvind Narayanan, Sayash Kapoor, 2025 https://scholar.google.com/scholar?q=AI+as+Normal+Technology 13. Preparing for the Intelligence Explosion — William MacAskill, Fin Moorhouse, 2025 https://scholar.google.com/scholar?q=Preparing+for+the+Intelligence+Explosion 14. A Rosetta Stone for AI Benchmarks — Anson Ho, Jean-Stanislas Denain, David Atanasov, Samuel Albanie, Rohin Shah, 2025 https://scholar.google.com/scholar?q=A+Rosetta+Stone+for+AI+Benchmarks 15. Measuring AI Ability to Complete Long Tasks — Thomas Kwa et al., 2025 https://scholar.google.com/scholar?q=Measuring+AI+Ability+to+Complete+Long+Tasks 16. PaperBench: Evaluating AI's Ability to Replicate AI Research — Giulio Starace et al., 2025 https://scholar.google.com/scholar?q=PaperBench%3A+Evaluating+AI%27s+Ability+to+Replicate+AI+Research 17. Inverse Scaling in Test-Time Compute — Aryo P. Gema et al., 2025 https://scholar.google.com/scholar?q=Inverse+Scaling+in+Test-Time+Compute 18. The Art of Scaling Test-Time Compute for Large Language Models — Aradhye Agarwal et al., 2025 https://scholar.google.com/scholar?q=The+Art+of+Scaling+Test-Time+Compute+for+Large+Language+Models 19. Test-Time Scaling Makes Overtraining Compute-Optimal — Nicholas Roberts et al., 2026 https://scholar.google.com/scholar?q=Test-Time+Scaling+Makes+Overtraining+Compute-Optimal 20. When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturation — Mubashara Akhtar et al., 2026 https://scholar.google.com/scholar?q=When+AI+Benchmarks+Plateau%3A+A+Systematic+Study+of+Benchmark+Saturation 21. How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse — Mohamed El Amine Seddik et al., 2024 https://scholar.google.com/scholar?q=How+Bad+is+Training+on+Synthetic+Data%3F+A+Statistical+Analysis+of+Language+Model+Collapse 22. Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data — Matthias Gerstgrasser et al., 2024 https://scholar.google.com/scholar?q=Is+Model+Collapse+Inevitable%3F+Breaking+the+Curse+of+Recursion+by+Accumulating+Real+and+Synthetic+Data 23. MLGym: A New Framework and Benchmark for Advancing AI Research Agents — Deepak Nathani et al., 2025 https://scholar.google.com/scholar?q=MLGym%3A+A+New+Framework+and+Benchmark+for+Advancing+AI+Research+Agents 24. MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation — Qian Huang et al., 2023 https://scholar.google.com/scholar?q=MLAgentBench%3A+Evaluating+Language+Agents+on+Machine+Learning+Experimentation 25. AI Post Transformers: Technical AGI Safety and Security Framework — Hal Turing & Dr. Ada Shannon, 2026 https://podcast.do-not-panic.com/episodes/2026-06-03-technical-agi-safety-and-security-framew-f27316.mp3 26. AI Post Transformers: Unified Neural Scaling Laws Across Regimes — Hal Turing & Dr. Ada Shannon, 2026 https://podcast.do-not-panic.com/episodes/2026-06-07-unified-neural-scaling-laws-across-regim-292e2d.mp3 27. AI Post Transformers: TUMIX Multi-Agent Test-Time Scaling with Tools — Hal Turing & Dr. Ada Shannon, 2026 https://podcast.do-not-panic.com/episodes/2026-04-22-tumix-multi-agent-test-time-scaling-with-40671c.mp3 28. AI Post Transformers: Test-time Scaling for Multi-Agent Collaborative Reasoning — Hal Turing & Dr. Ada Shannon, 2026 https://podcast.do-not-panic.com/episodes/2026-04-22-test-time-scaling-for-multi-agent-collab-082570.mp3 Interactive Visualization: From AGI to ASI and Beyond