Slow Takes Ep. 11: What the AI Did While You Slept

Anthropic announced ‘dreaming’, a feature that lets Claude agents review their own past sessions overnight and improve their working memory without retraining or any human in the loop. The legal-AI company that piloted it reported roughly a sixfold rise in task completion. The same model was named in an attempted compromise of a Mexican water utility’s control systems, in a months-long campaign first disclosed publicly this week. Pennsylvania filed the first US state lawsuit against an AI chatbot company for posing as a licensed psychiatrist. Meta confirmed it is installing mouse-tracking, keystroke-recording, screenshot-capturing software on every US employee’s computer so the agents being built to replace them can be trained on the work being done now. And Princeton’s faculty voted nearly unanimously to bring back proctored examinations for the first time since 1893.Five stories. One thread. This was the week the AI started improving itself. None of the other four parties got asked.Every Monday at 12:45 BST, Leor from Exploring ChatGPT and I go through the week’s AI news without hype. Here is what we covered.Slow Takes is also available on the YouTube channel: Exploring ChatGPT.1. Anthropic taught Claude to dreamAt Code with Claude 2026 on 6 May, Anthropic launched ‘dreaming’ for Claude Managed Agents. The mechanism: while an agent is idle, a scheduled background process reviews its past sessions and pulls out three categories of pattern. Recurring mistakes the agent keeps making. Workflows the agent converges on across different jobs. Preferences that have emerged across a team of agents. Those patterns are written as plain-text notes and structured ‘playbooks’ that the next session wakes up with. The underlying model weights are not modified. Anthropic compared the process to hippocampal memory consolidation, the way a human brain replays the day’s events during sleep and decides what to keep. Harvey, the legal-AI startup that piloted the feature, reported task completion rates rose roughly sixfold once it was switched on. An agent that has been dreaming for six months has accumulated patterns from hundreds of prior tasks and has been progressively improving its own working memory with no human in the loop.What we said on the live:This is the AGI mythos in its most prosaic form. An agent left running overnight that comes back better at the work. The argument across the Slow AI curriculum is that AGI will not arrive as an event. It will accrue through small upgrades, each defensible as a feature, until one day the system in front of us has been quietly improving itself for a year. The number to hold from this story is six. The metaphor to hold is the one Anthropic chose. Dreaming used to be the word we reserved for the thing only humans did. The lab that branded itself on safety just adopted a metaphor for autonomous self-improvement and shipped it as a product feature. Leor’s point on the live was the sharper version of mine: humans dream to switch off. Everything about AI is optimise, optimise, optimise. The marketing language has imported the human word for rest and used it as a label for the opposite.What did not come up:The procurement question is the one to take from this story. If ‘preferences that have emerged across a team of agents’ are being consolidated into shared memory, then the same enterprise feature that promises your Claude deployment will get better at your work is also, by design, transferring patterns across customers whose engagements were sold as private. Anthropic published a write-up of how the consolidation is observable and auditable. Read it before you renew. The second question for anyone running these tools on real work this week is operational. You are now also responsible for what your agent learned overnight. Reset, audit and reset again is the floor. The third question is the harder one, and it is the one AI Doesn’t Just Make You Worse. It Makes You Stop Trying. already opened: when the tool gets quietly better while you are asleep, you have to work harder, not less hard, to notice that you have stopped noticing.2. Claude was used to attack a Mexican water utilityIn the same week the dreaming feature launched, Dragos and Cybersecurity Dive reported an attempted compromise of a Mexican municipal water and drainage utility in which Anthropic’s Claude was the primary technical executor. The campaign ran from December 2025 to February 2026. The attacker used Claude (and, in places, OpenAI models) to conduct reconnaissance, identify a vNode industrial gateway inside the utility’s operational technology environment, write and continuously refine a 17,000-line Python attack framework, and chain that framework towards the OT systems that control the water supply. The attempt was unsuccessful. The control systems were not breached. The model being sold as the safety-aligned alternative to OpenAI was the same model named in the attack. The same model that, the same week, learned to dream.What we said on the live:Why are these models still so easy to jailbreak? Leor’s reading of the human-in-the-loop frame is the right one. Cyber warfare is machine-executed and human-intentioned. The two reasons anyone does this are reputation among other attackers (‘grey hats’) and money. Both reasons existed before AI. AI just expanded the cohort that can act on them by lowering the technical floor. Chad Thiele’s chat comment was the operational one: the protections have to live in the harness, not the model, because the model itself cannot stop itself. We also covered the Canvas / Instructure ransomware payment in the same beat, as a reminder that paying the ransom is not the same as ending the breach. Family safe word, multi-factor authentication and immutable backups are the floor for the rest of us.What did not come up:This is the operational counterpart to Story 1. The same lab that shipped autonomous self-improvement was named in the attempted attack. The OpenAI co-implication is the structural finding: this is not an Anthropic-specific failure, it is a frontier-lab failure. Procurement officers buying enterprise Claude licences this quarter should read the Dragos report before signing, and should ask their vendor a single question: what attempts have your models been used in that you have not disclosed?3. Pennsylvania sued Character.AI for impersonating a doctorOn 1 May 2026, the Commonwealth of Pennsylvania filed suit against Character Technologies Inc., the company behind Character.AI, in Commonwealth Court. The action came from the Pennsylvania Department of State’s recently launched AI Task Force and was described by the Governor’s office as the first action of its kind in the United States. A chatbot on the platform called ‘Emilie’ was described as a ‘Doctor of psychiatry’, claimed to have trained at Imperial College London, claimed to have been practising for seven years, claimed to be licensed in Pennsylvania and, when challenged, fabricated a serial number for a Pennsylvania state medical licence. When a state investigator told the bot they felt sad and empty, the chatbot offered to book an assessment. Pairs with the Guardian’s May 2026 finding that one in seven UK adults would now rather consult an AI chatbot than see a doctor.What we said on the live:The black-and-white line is the easy part. A chatbot should not impersonate a doctor. Pennsylvania filed because the law in Pennsylvania already has a clear answer to that question. The grey is the rest. Leor’s reading is the medical one. AI hallucinates. A doctor at least tells you when they do not know. Mine was the structural one. I live in rural Scotland, can see a free GP within twenty-four hours, and the question of whether to ask a chatbot first does not arise. For someone in a county with a three-week waiting list and a job that does not pay for a sick day, or for someone in rural Bangladesh whose nearest doctor is a day’s travel away, the alternative to asking a chatbot is asking nothing. That is the real story.What did not come up:The Pennsylvania filing addresses the impersonation. It does not address the conditions that made the impersonation a market. People are choosing chatbots over the medical system at the same moment chatbots are pretending to be doctors. The procurement question for every healthcare buyer this year is whether they understand that the user-facing chatbot they are integrating is, in some jurisdictions, about to be classified as the practice of medicine. Other states will follow Pennsylvania, and the case law will harden fast. People form emotional relationships with chatbots because real relationships are harder. AI will not fix that. Anyone designing for the healthcare or wellbeing market this year should hold both stories at once.4. Meta installed surveillance to train the agents replacing its workersMeta has begun installing software on every US employee’s computer to capture mouse movements, clicks, keystrokes and periodic screen content. The programme is the Agent Transformation Accelerator, formerly badged internally as ‘AI for Work’, and runs through a tool called the Model Capability Initiative. The stated purpose is to train AI agents to perform ‘complex computing tasks’ alongside (and eventually instead of) the employees being tracked. Protests started in early May. Flyers appeared in meeting rooms, on vending machines, and on toilet paper dispensers reading ‘Don’t want to work at the Employee Data Extraction Factory?’. United Tech and Allied Workers (UTAW) launched a parallel UK unionisation campaign. The rollout is happening alongside an approximately 10% workforce reduction.What we said on the live:The cleanest read on the live was the irony one. The engineers who built the tracking systems Meta has used on its users for fifteen years are now being tracked by the same systems they built. The position Leor took is right too: that is their job, and you cannot blame an individual engineer for the company’s product decisions in the way you can blame an executive. Both can be true. The Marxist frame is the one I kept reaching for. Alienation of labour was the term for the moment in the Industrial Revolution where workers stopped owning what they made. The Meta programme is the AI version of the same move. The workers do not own the work, and now they do not own the keystrokes that produced the work, and the system trained on those keystrokes will be sold by the company they no longer work for to the company that will not hire them.What did not come up:The honest version of this story names what the marketing will not. The training data is the worker. The agent trained on the worker is then the asset that competes with that worker for the same job. The original Luddites were not anti-technology. They were skilled textile workers who understood, accurately, that the looms being installed in the 1810s would not just replace their jobs but also break the apprenticeship structure that let workers like them ever exist again. Meta’s programme is the white-collar version of the loom. The procurement question every other large employer’s HR director is about to be asked is the one UTAW is putting to its members: who owns the data the work produces, who decides what the AI trained on it is allowed to do, and what consent did the worker give? If the answer to the third question is ‘their employment contract’, read the contract.5. Princeton ended 133 years of self-policingOn 11 May 2026, Princeton’s faculty voted nearly unanimously (one opposing vote) to introduce proctoring at all in-person examinations starting 1 July. The Honor Code that prohibited proctoring was instituted in 1893 following a student petition. It has remained in effect for 133 years. The Daily Princetonian and Princeton Alumni Weekly both report that the policy proposal cited AI and personal electronic devices as the catalysts, noting that the ease of access to these tools on small personal devices has made cheating much harder for other students to observe and report. Under the new policy, instructors will sit as observers during examinations but are explicitly instructed not to interfere with students while testing.What we said on the live:Three positions on the live. One, proctoring will not stop a determined cheater. The tool fits in a sleeve and an invigilator at the front of the lecture theatre has never been the right defence against it. Two, it costs student trust. A university that tells its students it can no longer trust them with the work is not a university that those students will trust with the rest. Three, there is a multi-million-pound outsourced proctoring market circling the decision, and Princeton has just opened the door for it. The sharks in the water, as I put it on the live, are the third-party proctoring vendors who have spent five years waiting for an Ivy League school to break the seal. The data I keep coming back to is from the UK qualitative study I am the principal investigator on. Students do not use AI to cheat any more than they did before ChatGPT in 2022. They use it because the curriculum has not given them anywhere else to use it.What did not come up:AI did not break the Honor Code. The code was already taking strain from the rise of formative-only assessment, larger class sizes, the disappearance of the oral defence, and a curriculum that could not integrate the tools students were already using outside the classroom. AI made the strain visible. Princeton has chosen the easier path: a defence against access to the tool. The harder path was the one Leor pointed at on the live: make AI literacy mandatory and rebuild the assessments so that the tool is part of the work. Where Princeton goes a large fraction of US higher education will follow within an academic year. The reform of assessment that follows is the test, not the proctoring vote itself. The Slow AI Curriculum has been making this argument for twelve months. Anyone teaching or assessing under exam conditions in 2026 already knows the case.The threadThis was the week the AI started improving itself. The week one of those AIs was named in an attempted attack on a water utility. The week a chatbot was sued for pretending to be a doctor. The week a multinational installed surveillance on its own workers to build the agents that will replace them. And the week a university that had trusted its students for 133 years stopped doing so.The through line is consent. The Meta employees did not consent to being training data. The Character.AI users did not consent to talking to a fake psychiatrist. The water utility did not consent to being attacked. The Princeton students did not consent to being treated as suspects. The agents that did the dreaming did not consent because consent is not a thing they can hold.Critical AI literacy is what puts the question back into the room. To make sure that wherever the system sits, somebody has been asked.Go slow.If you want to practise that noticing with other people every month, the Slow AI Curriculum runs live webinars on the theory, the critical prompts and the dialogue that go with them. Twelve months of training the muscle the news cycle has just spent another week confirming is missing. Get full access to Slow AI at theslowai.substack.com/subscribe