From Supervising AI to Building Systems for It

What this episode covers

Dan and Eleanor open by discussing how fast software engineering has changed. In the last six months, Eleanor's practice flipped from treating AI as a messy assistant that needs close supervision to building systems that put the agents on the path to success. She now writes essentially no code herself, arguing that the models have become good enough that her involvement mostly makes the results worse.This journey starts from babysitting agents locally to delegating to async, cloud-based agents like GitHub Copilot, Cursor, Devin, OpenHands, or Factory. Eleanor warns that the home-grown terminal "loops" everyone is building right now are great for learning but too brittle to scale.Next up, what does an agent engineering system actually need? Eleanor recommends starting with a sandboxed, execution environment (usually containers), careful configuration over how the agent reaches the outside world (MCP servers and selective network access), a way to see across multiple repositories, and layered rules via AGENTS.md and skills.Eleanor makes the case that async delegation is a forcing function for better specifications. Deterministic feedback like static analysis and test suites are the single biggest factor in work quality because "you can't control AI with AI." She has moved to fully test-driven development and notes that current-generation models no longer find unintended workarounds to tests (e.g., deleting them) the way Claude 4 and early GPT-5 once did.Dan and Eleanor turn to adoption and skills, including how to get better at using AI with deliberate practice. Eleanor explains why she moved using Python, which she was most familiar with from use over her career, to statically typed languages like TypeScript and Go for agent work, why supply chain risk at her healthcare company has her questioning every dependency, and why she dislikes the term "junior developer." Curiosity and systems thinking, not tenure, are what matter now.The episode closes on verification and scale. Eleanor distrusts any output she can't verify, doesn't miss hand-writing code, and argues that inventing new ways to verify, including more formal methods, is the real bottleneck now that models are cheap and strong. On team size, she pushes back on the "small teams" consensus, pointing to the success of large open-source communities. Eleanor remarks that software development has become a sub-branch of systems engineering, and anyone not practicing this now will be shocked in a matter of months.(00:00) - Introduction (00:58) - The flip: from supervising AI to getting out of its way (03:14) - Cloud-based agents vs. rolling your own (06:08) - The primitives every agent system needs (07:43) - Why async delegation beats local babysitting (11:02) - Writing specs: Codex, Repo Prompt, and markdown (12:21) - Guardrails: AGENTS.md, skills, and deterministic checks (14:19) - Going fully test-driven (17:19) - How engineers really adopt (and hide) AI (19:38) - Getting better through deliberate practice (21:12) - From experiment to reusable skill to library (24:26) - Choosing a language: Python, TypeScript, Go (26:56) - Supply chain risk and distributing specs (29:06) - Beyond 'junior': curiosity over tenure (31:03) - Systems thinking as the durable skill (38:02) - Where Eleanor still doesn't trust AI (39:40) - Not missing the keyboard (42:43) - Keeping up with a fast-moving field (44:50) - What teaching reveals (48:27) - Verification as the real bottleneck (50:41) - Team size and open source at scale (55:48) - Closing: take agents seriously Click here to view the episode transcript.

PodParley-generated summary based on available episode metadata and transcript content.