“Epistemics and Coordination: It’s complicated!” by Raymond Douglas

Money is pouring in, people are looking for new areas to fund, and the invisible hand is starting to grab a bit at AI for epistemics and coordination (AIFEC hereafter). It also got invoked in AI2040 as a potential part of the wining strategy. My feelings here are mixed — I think the best version of AIFEC is great, but also the existing public writeups are only a few cycles deep on tracing out the different ways that the obvious plan backfires. And regrettably I think some people have correctly written off AIFEC because what they have read appears a bit naive to them. In my heart I always planned to do a proper writeup of my thoughts when things were a bit less busy, but, well, now we’re in the 100x funding era. So here's my scrappy, hopefully-better-than-nothing attempt. The six big claims: AIFEC could be great! It's a tractable way to do good on the margin, and the best version is a legitimate theory of victoryIt could also easily backfire, especially for implementations that depend on scaling with inference“Better epistemics” is often more hostile than it seems, and people have good reasons to be [...] ---Outline:(01:47) The basic case is pretty good!(04:04) Problem 1: Naive AIFEC could easily backfire(05:09) Problem 2: "Better AIFEC" can be a somewhat hostile move(07:55) Problem 3: Vagueness lets you ignore tradeoffs(09:54) Problem 4: Sometimes people just don't get along(11:47) But overall, I am still pro The original text contained 1 footnote which was omitted from this narration. --- First published: July 21st, 2026 Source: https://www.lesswrong.com/posts/5nP5WY2PzsYiegDzQ/epistemics-and-coordination-it-s-complicated --- Narrated by TYPE III AUDIO.