What Buddhist Epistemology Teaches Us about The Future of AI

For Part Two in our Continual Learning mini-series, Dr. Nadav Amir of The University of Ottowa joins me and guest co-host Dr. Adam Safron of Tufts University for a trialogue on the fundamental role of goals in intelligence.Nadav draws on behavioral neuroscience, Buddhist philosopher Dharmakirti, control and information theory, and John Vervaeke’s work on “relevance realization” to formally articulate what many wisdom traditions have argued for millennia: what you take to be “reality” — your world and self — is a function of goal-based representations, state descriptions and reward functions create each other, and “what is” and “what do I care about” are two sides of the same thing. You can’t separate what you want from what you experience and what you are capable of. But this means that if you change your goals your reality will shift, and your sense of self with it. The opposite is also true: change the way you organize the the ineffable structure of reality into a conceptual framework, and you’ll care about very different things. Thanks to the pace of change we’re living through this kind of transformative experience now, so it’s a good time to ask:If selves are a kind of falsification, why do we have them in the first place? Does AI need goals of its own in order to be truly intelligent?And perhaps the deepest question we ask in this episode:How if at all can we develop better frameworks for deciding what we want our intelligent machines to become — and who we want to become alongside them?Thanks to the Survival and Flourishing Fund for their support of this mini-series. If you’d like more of this kind of work in the world:✨ Support my research, writing, and podcast with a tax-deductible donation✨ Become a founding member to access my online courses, including Jurassic Worlding and How To Live In The Future✨ Hire me for public speaking or advisory work with your organization✨ Browse and buy the books we mention on the show at Bookshop.org✨ Stream and download my tunes at artist-owned music co-op Subvert.fm✨ Continual Learning theme song: “Substrate Dynamics” by Neon Chameleon✨ Learn more about how we’re applying these ideas at Atlas Research Group, my team building sovereign infrastructure for collective intelligenceChapters0:00:00 Teaser0:01:48 Intro0:05:42 Who is Nadav Amir?0:08:15 Why we can’t separate goals from descriptions0:15:32 The Ugly Duckling Theorem & why taxonomy is not reality0:25:07 Evolution, karma, and non-conceptual awareness0:31:04 If selfhood is wrong, when is it practical?0:34:14 Suffering without a self & the value of wrong descriptions0:38:14 Non-conceptual experience & active inference0:41:07 Multiple nested hierarchical descriptions in minds & economies0:46:07 What’s missing from current approaches?0:50:09 Does greater agency mean better optimization or greater adaptability?0:57:04 Empowerment-based causal learning without a ground truth0:59:06 Intermission0:59:58 Transformative experiences & multiple levels of granularity1:05:19 No fixed models, no fixed environments1:10:55 Fundamental tradeoffs1:16:57 Reverse-engineering the centered self1:19:54 Does AI need goals to be intelligent and can we control AI if it has them?1:26:47 Does Nadav’s framework pertain for all possible minds?1:32:37 Mind as a property of collectives & goals as embedded in environments1:41:33 How can Nadav’s framework improve AI governance discourse?1:48:59 OutroMentioned ResourcesA Dharmakīrtian Model of Relevance Realization in Cognitive Agentsby Nadav Amir & John DunneAn exchange of letters on the role of noise in collective intelligenceby Daniel Kahneman et al.Cognitive glues are shared models of relative scarcities: the economics of collective intelligenceby Michael Levin & Benjamin LyonsReverse-Engineering The Centered Selfby L.A. Paul et al.Agency and Experience: Buddhist and Cognitive Perspectivesa Yin Cheng Conference @ PrincetonKnowing and Guessingby Satosi WatanabeOntological Laughter: Comedy as Experimental Possibility Spaceby Timothy MortonSelves as Perspectives: From Biological Life to Superintelligence and a Bodhisattva Projectby Thomas Doctor et al.What The Frog’s Eye Tells The Frog’s Brainby J. Y. Lettvin et al.The Center for the Study of Apparent SelvesExplore the entire open-access special issue here:And whether you read the papers or not, be sure to check out this illuminating interactive discourse map by Van Bettauer of Ideoscopic to help you navigate where these researchers agree, disagree, and point toward future study:(Long-time fans will also want to check out his re-imagined interface for AskFutureFossils.com, complete with simulated debates between my guests!) This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.humansontheloop.com/subscribe