Digital Qualified Immunity: When Software Decides and Nobody Answers

When the government hands a decision about your life to software, who answers for it — and how does anyone check its work? Joshua and Will borrow a real legal doctrine, qualified immunity, to name something newer: digital qualified immunity. The agency says the software decided. The vendor says the software is a trade secret. The person who lost their benefits or their job is left with nobody to question. In this episode: Arkansas ARChoices. A formula replaced nurses’ judgment and cut home care hours for disabled and elderly Medicaid recipients. A judge blocked it in 2018, not because the math was wrong, but because nobody was told the rules changed. Houston ISD. Teachers were rated and fired using SAS’s EVAAS scores, a model so secret that the district itself couldn’t check it. The case settled for $237,000, and the district stopped firing teachers on those scores. Michigan’s MiDAS. An automated system falsely accused tens of thousands of people of unemployment fraud. The state’s own review found a 93% error rate. That led to Bauserman, a $20 million settlement, and Cahoo v. SAS Analytics, where the vendors were sued by name. Due process 101. Goldberg v. Kelly, the Mathews v. Eldridge balancing test, and why private prison guards can’t claim qualified immunity. California SB 947, the “No Robo Bosses” Act. What “relies primarily” might mean, and whether a manager skimming an AI-written summary counts as human review. What the contract should say. No trade-secret shield against the person affected, reproducible versioned models that auditors can run, vendors sharing liability, and treating any change to scoring weights as a rule change. Consistent isn’t the same as correct. If software made a decision about your life this year, how would you know? Note: We ran into some audio/video sync issues during recording. You may notice drift or more crosstalk than usual in places. We fixed what we could in editing. Thanks for bearing with us. https://fof.foundation [email protected]