“AI 2040: Is it Actually a Deal?” by 1a3orn

The "AI Futures Project" has released their AI 2040: Plan A scenario. While their previous scenario AI 2027 was a forecast of what they thought a future with many powerful AIs would look like, AI 2040 is intended to be normative -- it's a description of what one ought to do, granted the likelihood of a future with many powerful AIs. I'm going to review some objections I have to their proposal as a normative plan. Some are within-frame objections -- reasons that I expect trying for the AI 2040 plan that would fail to accomplish the goals of the authors. Others are my own objections -- reasons that I expect trying for the AI 2040 plan would destroy things that I, personally, care about. Before I start: two elements of the plan that I like. First, in their "incremental AI policy wishlist" -- the ideal policy that one should execute soon -- AI 2040 recommends trying to limit the gap between the intelligence of internal and external model deployments, i.e., the gap between the "intelligence" accessible to Anthropic / OpenAI employees and to everyone else. I'm a fan of efforts in this direction; equality [...] ---Outline:(02:07) 1. The Deal Has No Actual Decision Procedures(09:50) 2. China Will Likely See Such a Deal as Unnecessary(12:25) 3. The Surveillance Possibilities are Actually Quite Bad(15:30) Conclusion --- First published: July 20th, 2026 Source: https://www.lesswrong.com/posts/LtmbQN9mzmFWxWtrD/ai-2040-is-it-actually-a-deal --- Narrated by TYPE III AUDIO.