Episode 016: Can an AI Judge Train You Better Than the Real One?

What if you could lose your case to an AI judge tonight, so you don't lose it to a real one tomorrow?For generations, lawyers learned advocacy the hard way: draft the brief, argue the motion, get knocked down by the judge, learn why you were wrong. Litigation partner Chris Ryan built a different path. BenchSim AI lets lawyers upload their brief and opposing counsel's brief, then argue out loud in real time against an AI judge who pushes back, interrupts, and grades the performance. The question this episode keeps circling: is this the future of how lawyers get their reps in, or is the courtroom apprenticeship something AI can never actually replace?In this episode:Why COVID permanently reduced young lawyers' opportunities to argue in front of a real judge, and what that "reps problem" means for the next generation of litigatorsHow Chris built BenchSim AI as a litigation partner with no modern coding background, using "vibe coding" to go from idea to working product in about six weeksWhat vibe coding actually is, and why it's the same process as building a custom GPT, Gem, or Claude skillHow BenchSim works: upload your brief and opposing counsel's brief, choose a judge temperament (quiet, neutral, or hot bench), and argue out loudWhy BenchSim deliberately skips video rendering of the judge to avoid latency that would kill the realism of rapid-fire argumentHow the AI judge develops counterpoints from the opposing brief and is programmed to interrupt when an advocate is talking in circlesThe SOC 2 certification process BenchSim is going through before marketing to law firms, and why that matters for adoptionWhether AI will actually save lawyers time, including the "airport test" framework for evaluating whether a tool is worth the overheadUsing AI as an adversary instead of a cheerleader — prompting it to argue against your own complaint or brief before opposing counsel doesThe Flintstones/Simpsons/Jetsons breakdown of how to stress-test a brief at every level of AI adoptionWe also discuss:Chris's recent "wow moment" using AI to play defense counsel against his own drafted complaintHeather's experience having Claude Code build software overnight while she sleepsWhether AI simulation training could expand beyond litigation into bar exam prep and other legal trainingA Practice Signal segment on a deeply inappropriate mentorship moment a young associate experienced, and whether AI could have helped an older partner communicate the underlying (legitimate) concern without the inappropriate framingChris's plans for BenchSim's feature roadmap, including potential expansion into opening statements and direct examination practiceKey TakeawayAvailability is not authority, and a simulation is not a verdict. BenchSim doesn't tell a lawyer whether they'll win or lose; it tells them where their argument is weak before a real judge finds out for them. That distinction matters. The value isn't in the AI replacing judgment, it's in creating reps that don't exist anymore because courtrooms don't generate them the way they used to.This episode lands differently depending on where you sit on the FSJ spectrum. A Flintstones lawyer can start by asking any AI tool to summarize their argument and flag weaknesses. A Simpsons lawyer can go further, prompting AI to act as opposing counsel and attack the brief. A Jetsons lawyer is already running full bench simulations, treating AI as an adversary that prepares them for the real fight rather than a cheerleader that tells them what they want to hear.Mentioned in This Episode:BenchSim AI (benchsimai.com)Taft (Taft Stettinius & Hollister)HarveyLegoraAnthropic Claude / Claude CodeChatGPTReddit (Practice Signal segment source)MPRE (Multistate Professional Responsibility Examination)[email protected]