S1 E1: Your Model Came From Where?

The Pentagon buys a missile and every part has a paper trail. The alloy, the guidance chip, the guy in Ohio who machined the housing. Somebody can tell you where all of it came from.The Pentagon buys an AI model. Ask where it came from and the honest answer is a shrug.That gap is what Season One is about, and this is the episode the rest of it hangs on. If you can’t answer where your model came from, nothing built on top of it can be trusted.The oldest trick, in three actsWe start in Switzerland. Crypto AG sold cipher machines to roughly a hundred and twenty countries, including Iran and the Vatican. Neutral country, no agenda, gold standard.And from 1970 it was secretly owned by American and West German intelligence. The machines worked perfectly. They scrambled exactly as advertised. The people who built them had rigged the math from the start and could read everything.Andy takes it back further, to Enigma. The Germans trusted the machine. The British were reading the mail. As he puts it: to betray, you first have to belong.Act two is 2015. Juniper Networks, whose firewalls sit in front of government and corporate networks, discovers unauthorized code that let whoever knew the secret decrypt protected traffic. The twist is that the mechanism may have been American-built and quietly repurposed by someone else.You build the skeleton key, and you don’t get to choose who ends up holding it.Act three is this month. Attackers didn’t go after LiteLLM, the gateway that routes AI requests for a huge share of the industry. They went after Trivy, a security scanner, the tool you install in order to be safe.Poison it upstream, and LiteLLM’s build pulled it in automatically without checking the version. Two infected releases went live for about forty minutes.Forty minutes was enough to reach 2,500 organizations and 400,000 automated build pipelines, and to walk off with cloud keys, server credentials, and the credentials companies use to publish their own software. What you steal in a supply chain attack is the ability to run the next one.Nobody clicked anything. Machines pulling from machines at three in the morning.Fifty years, three eras, one play: you don’t attack the target. You compromise what the target already trusts and never re-checks.What changed isn’t the strategy. It’s the price. Crypto AG took decades and control of a company. LiteLLM took one upstream tool and a lunch break, and the victims’ own automation handled distribution.Four layers, and where Washington isn’t lookingWe map the season across four layers: the data, the model, the compute, and the integration layer where everything connects.Then the policy record. A Defense Department AI policy, executive orders, a NIST risk framework, chip export controls, and a serious new order this June covering classified model testing and early government access.None of it addresses where the training data came from or whether the supply chain is intact. Five agencies, five frameworks, no thread connecting them.So we get into what to actually do. An allied export pact for AI modeled on the Cold War’s Wassenaar Arrangement. Cryptographic provenance and a bill of materials for models, the way we already do for software. Certification incentives along the lines of FedRAMP and CMMC. And allied threat-intelligence sharing.Andy doesn’t let that stand. Verification isn’t free, and the cost of proving you’re trustworthy is exactly what squeezes out the small innovator and hands the field to the large, slow incumbents.Then he asks the fair question directly: is the guy who does compliance for a living recommending a world with more compliance in it? We have that argument on the record rather than editing it out.His sharper point is one we didn’t see coming. We keep mirror-imaging our adversaries. China can let things run wild because China hasn’t adopted this technology at the scale we have.Our openness bought us enormous advantage and an enormous attack surface at the same time. They aren’t playing the same game because they aren’t exposed the same way.And then OpenAI changed the termsThree days before we recorded, OpenAI shipped Daybreak: frontier models for vetted defenders with the guardrails taken off, and a cyber model trained to refuse less.The standard model completes about 1.5% of advanced attack requests. The cyber model completes 95%. Pointed at Chrome, it found previously unknown vulnerabilities, the kind attackers pay a fortune for.Ryan’s position is that this is uncomfortable and correct, because the asymmetry already favored whoever was willing to download an uncensored model. Andy’s is that everything becomes a weapon eventually, and the question isn’t the tool, it’s what you assume about the people holding it.We don’t resolve it. Episode 6 comes back to it.Where we landWe don’t get to treat AI as a black box built on a gray market. Not when it’s writing the targeting assessment.The playbook exists. Nobody’s running it, and the clock is a machine that doesn’t sleep.Next month: the data layer. Garbage in, doctrine out.If you think we got something wrong, come tell us. As Andy put it on the way out, your first opinion may not be your best. If you don’t ferment the wine, all you have is grape juice.In this episode01:00 The missile has a paper trail. The model doesn’t.02:20 The pour04:40 Why AI is different from every other supply chain07:00 Openness versus assurance08:30 Compromising the vendor, and why all warfare is asymmetric11:30 Crypto AG, Enigma, and the facade of trust15:00 Juniper, and the human side of the supply chain16:10 LiteLLM, Trivy, and forty minutes18:40 Double agents and the indirect approach20:00 The four layers21:10 The Farewell operation, and why both sides had reason to lie23:30 Is Washington asleep?25:50 The Innovator’s Dilemma27:00 Four plays30:30 What verification costs the small guy32:45 Mirror-imaging our adversaries35:05 Daybreak37:30 Trust but verifySourcesSecuring the AI Supply Chain, USF Global and National Security Institute decision brief: https://digitalcommons.usf.edu/gnsi_decision_briefs/35Crypto AG / Operation Rubicon: joint Washington Post and ZDF investigation, February 2020LiteLLM supply chain attack, SecurityWeek: https://www.securityweek.com/over-2500-organizations-impacted-by-litellm-supply-chain-attack/OpenAI, Expanding Daybreak as the Cyber Defense Window Narrows: https://openai.com/index/expanding-daybreak-as-the-cyber-defense-window-narrows/Executive Order 14409, Congress.gov: https://www.congress.gov/crs-product/IF13268 This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit fermentedopinions.substack.com