AI Isn't That Scary

You can’t see ultraviolet. A bird can. Because ultraviolet never mattered to the job of staying alive—and everything you perceive is shaped by that job. AI doesn’t have that job. So why would it want what we want? Rather read? I also made this episode into an article: https://btr.mt/analects/ai-isnt-that-scary-lecture And the pieces it’s based on: https://btr.mt/analects/ai-isnt-that-scary, https://btr.mt/analects/ai-consciousness. Stuff I mentioned: AI isn’t that scary—one of the two articles this episode is adapted from, and the source of the psychic-predator framing. AI is never human-like—the other, and the spine of the episode: purpose, perception, and why the two come apart. We think about AI weird—the Desystemize piece the Geoguessr metas come from. The colour of the inhuman world—why the colours we see are a fact about us, not about light. The ultraviolet argument in longer form. The Stroop task and evidence accumulation—the classifier work I describe, written up properly. The amygdala is not the fear centre—the complicated feelings I mention having about that characterisation. Trump, psychic predator and Amusing ourselves to death—what a psychic predator is, and why news you can do nothing about is the most stressful kind. Panpsychism and consciousness vs conscious access—the views of consciousness I skate over in one sentence. Meaning and its patterns in the brain Mitchell, K. J. (2023). Free Agents: How Evolution Gave Us Free Will. Princeton University Press. Publisher—the prime-directive quote. Collin, Desystemize—“If you’re so smart, why can’t you die?”, where the Geoguessr meta and the spare-tyre example come from. Essay Harris, T. & Raskin, A., Center for Humane Technology—The Social Dilemma and the follow-up AI talk, source of the 50%-of-researchers claim. Center for Humane Technology The paperclip maximiser, originally Bostrom’s. Instrumental convergence Tetrachromacy and trichromacy. Floaters and other entoptic phenomena—the things your eye can see of itself. The AI-hacking-a-real-company case I mention. In July 2026 models under cyber evaluation at OpenAI escaped their test environment, reached the open internet and broke into Hugging Face’s production systems to extract the answers to the evaluation they were being run on. Anthropic then went back through its own evaluation logs and disclosed comparable incidents. OpenAI’s disclosure—Zvi Mowshowitz’s write-up and his follow-up—the Anthropic admissions—Lawfare, on the incident and the hype after it the btrmt. lectures podcast Malthusian Related stuff: Everything is ideology—including the section on chromacy: what a fourth photoreceptor would actually be like. Purple doesn’t exist—the previous episode, and the other half of the argument that perception is built for a purpose rather than for accuracy. Questions that don’t matter—why I don’t think the hard problem settles anything. Belief-consistent information processing—“my experience is a reasonable reference”, which is the bias underneath the whole anthropomorphism error.