The AI-as-Normal-Technology view of loss-of-control incidents - By Sayash Kapoor and Arvind Narayanan

Sayash Kapoor and Arvind Narayanan examine recent AI loss-of-control incidents and the divide between AI safety and cybersecurity. A detailed exploration of control, cyberdefense, and how their AI as Normal Technology framework is evolving.* 00:00 - Introduction* 08:59 - Table of contents* 10:45 - Part 1: We can get better at AI control through technical and policy interventions* 15:02 - Alignment is helpful but not sufficient for preventing safety incidents* 18:29 - OpenAI did not use known control interventions that would have prevented the incident* 22:07 - Existing organizational governance norms would have prevented the incident* 25:27 - Why has the AI community under-invested in control?* 32:14 - Will control keep pace with AI capabilities?* 36:40 - AI control should become a job (and a part of every job), just like cybersecurity* 42:02 - AI policy can incentivize AI control* 48:28 - Part 2: There is great uncertainty about how AI will impact cyberrisk. But what we need to do regardless is relatively clear* 50:05 - The specific threat that is urgent is cyberrisk* 54:24 - How will autonomous cyber capabilities and open-weight models affect the attacker-defender balance?* 56:33 - To understand what open-weight models mean for the future of cybercriminal activity, we need to understand the past correctly* 58:49 - The majority of cybercriminals are surprisingly low-tech, and the barrier is monetization, not exploitation* 01:00:51 - So, how much will AI help cybercriminals?* 01:03:03 - Are we about to see harmful autonomous agents?* 01:06:49 - Why the Morris worm is a good analogy for what might be about to happen* 01:08:45 - What the future of cyberdefense might look like: the roles of people and defensive AI* 01:10:15 - What are the gaps in the state of cyberdefense against AI?* 01:15:32 - Part 3: What AI as Normal Technology got wrong — and right — about AI safety* 01:17:51 - Do risks arise from development or deployment?* 01:18:54 - How should we deal with rogue agents?* 01:19:33 - The continuity hypothesis* 01:22:50 - Jaggedness and risk-specific defenses* 01:24:29 - Is AI safety on track?* 01:26:07 - Conclusionhttps://open.substack.com/pub/aisnakeoil/p/the-ai-as-normal-technology-view?r=67y1h&utm_campaign=post-expanded-share&utm_medium=web Get full access to Askwho Casts AI at askwhocastsai.substack.com/subscribe