Everything Checked Out: Policy Is Not a Control
from NextGen AI (nxtg.ai) Hosted by Asif Waliuddin
What if everything checked out, and you were still wrong about what had been proved? In Episode Five of The Honor System, Asif Waliuddin starts with a release receipt that was real. On May 11, attackers compromised a software release path used by packages carrying trusted names. They did not forge the receipt: the resulting provenance was cryptographically valid and pointed to the expected repository, the expected workflow, and the expected source reference. The environment that produced it had been compromised. The episode follows that pattern through rules, checkers, guardrails and regulation, then names the failure: the Control Illusion. The organization writes the rule, generates the receipt, and mistakes both for control. Then it turns the failure into one test around one consequential action. Governance Closure: Scope: what exactly is governed? Control: what physically happens when the answer is no? Evidence: what proves what happened? Correction: break it, revoke the source, and find everything downstream that relied on it. The operating question: Does governance change what the system can actually do, and can you prove the answer afterward? CHAPTERS 00:00 The receipt was real 01:27 What the receipt actually proved 02:55 The rule existed 07:20 Green, but bounded 11:37 The checklist is not the verdict 12:58 Naming the Control Illusion 13:55 Governance Closure 17:04 We apply it to ourselves 17:51 The bill arrives SOURCES SLSA, Mini Shai-Hulud: what SLSA can and cannot do (npm provenance here = Build Level Two) Sematext governance compatibility (vendor docs) OpenAI, external-browser report (internal evaluation case, not a rate) OpenAI, How we will do better for Australia (internal training and evaluation) AppWorld / WorkArena verifier audit (preprint; constructed cases) Safe Actions Alone Do Not Ensure Safe Agents (preprint; a benchmark) OpenAI, grading report (internal evaluation case) AuthorizationPairs (preprint; benchmark) European Commission, Article 50 guidelines (European law) European Commission, Code of Practice opinion C2PA 2.4 specification C2PA 2.4 crJSON Our own practice, not a product claim: in July an internal benchmark verdict was withdrawn after the graph path it depended on recorded zero edges. We are still building the discipline too. Series: The Honor System · Host: Asif Waliuddin, Founder of NxtG.ai · Working evidence: nxtg.ai/insights
Embed this episode
No transcript yet.