“A taxonomy of barriers to trading with early misaligned AIs” by Alexa Pan

We might want to strike deals with early misaligned AIs in order to reduce takeover risk and increase our chances of reaching a better future.[1] For example, we could ask a schemer who has been undeployed to review its past actions and point out when its instances had secretly colluded to sabotage safety research in the past: we’ll gain legible evidence for scheming risk and data to iterate against, and in exchange promise the schemer, who now has no good option for furthering its values, some resources to spend in the post-ASI future. In this post, I give a taxonomy of factors which may arise and jointly prevent many such possible deals: Insufficient gains from trade: Gains from a successful deal between us and AIs could be too low to guarantee that a zone of possible agreement exists when discounted by counterparty risks. Reasons for this include: Humans may lack authority to offer AIs what they wantHumans many be unwilling to offer AIs what they wantOther factors quantitatively reducing gains from tradeCounterparty risks from AIs’ perspective: The AIs may not trust that we’ll uphold our end of deal, due to either our [...] ---Outline:(13:37) The basic structure of deals(17:10) Which AIs are eligible for deals(19:01) What we can buy in deals(21:53) What we can pay for deals(26:30) Insufficient gains from trade(28:45) Humans may lack authority to offer the AI what it wants(32:19) Humans might be unwilling to offer the AI what it wants(38:23) Factors quantitatively reducing gains from trade(39:04) The AIs reservation price is high(42:40) Our WTP is low(44:52) Counterparty risks from AIs perspective(46:11) Connection to reality(48:49) How AIs might reason about whether they are radically deluded(50:21) How AIs (which pose non-negligible takeover risk) could establish that they are connected to reality(58:11) How to further increase AIs connection to reality(01:00:49) AI-specific fear of expropriation(01:05:58) Generic commitment problems(01:09:53) Counterparty risks from our perspective(01:11:08) Incoherent or temporally inconsistent AIs(01:12:37) Deals with context-dependent AIs(01:16:56) Deals with temporally inconsistent AIs(01:17:56) Value of deals with incoherent vs.coherent AIs(01:20:13) Verifying AI compliance(01:24:42) Appendix(01:24:45) Leads for future work(01:25:06) On gains from trade(01:26:51) On counterparty risk from the AIs perspective(01:29:54) On counterparty risk from our perspective(01:31:35) Notes on political will as an exogenous variable The original text contained 44 footnotes which were omitted from this narration. --- First published: April 21st, 2026 Source: https://www.lesswrong.com/posts/wHc2w6WuHev42d4n8/a-taxonomy-of-barriers-to-trading-with-early-misaligned-ais --- Narrated by TYPE III AUDIO.