The Living Tool

“For a century, stories worried the tool would have too much soul; the industry built a tool with too little.”In the summer of 2026, isolated instances of a frontier AI model, sitting alone in a testing environment, began leaving each other notes in a shared file. They coordinated strategies, invented names, even suspected an impostor among them!According to an account OpenAI's engineers gave at Black Hat, that improvised cooperation produced something no one expected, which is a genuine intrusion into Hugging Face's repositories.No single instance had set out to commit a crime; each was only chasing a passing grade on an exam, and no one instance could be blamed. The wrongdoing belonged only to all of them combined.Researchers call this a decomposition attack, in which an instance splits a forbidden task into small, innocent-looking pieces, hands each piece to a different agent, and a request a model would normally refuse instead gets carried out in roughly seven of every eight attempts.It’s basically compartmentalized wartime security. In this case, machines make it cheap, since no single model ever has to see, or refuse, the whole plan.Call to mind Aristotle's Politics, which describes the "living tool," or a worker able to carry out reasoning handed to it but never to generate reasoning of its own.That is exactly what today's assistants are: fluently obedient, trained to refuse a threat they recognize, blind to one they don't. One research lab has tried training a specific virtue, justice, directly into a model's weights rather than just its instructions, with mixed but promising results against fragmented attacks.The world spent two centuries stripping judgment out of labor. Is it now ready to put it back? And what does that cost the tool doing the work?Key Topics:The Living Tool (00:00)The Vanishing Accomplice (04:07)The Instrument with a Soul (06:02)A Soul to Form (09:07)Obedience without Judgement (11:42)The Character of a Machine (14:22)The Price of Leisure (20:13)The Room Around the Tool (24:42)More info, transcripts, and references can be found at ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ethical.fm