Testing Is Phase Zero for AI Agents

Episode Summary Dave and Dan dig into service readiness — the industry calls it "harness engineering" — and why testing just moved to the #1 spot of their 13 repo-readiness pillars. Before you let agents ship code in the background, the question isn't 90% coverage; it's whether anyone actually trusts the tests. They walk through the onboarding plugin that gets a repo test-ready and how test decisions get baked into the agent's planning phase instead of left to human best-effort. Key Topics Why "no one has confidence in the tests" is universal — slow, flaky, and thin coverage everywhere Testing as pillar #1 of 13 for agentic-engineering readiness ("phase zero") The test-onboarding plugin: research versions → design the greenfield ideal → reconcile what to keep vs. throw out → a reviewed task list All four layers in scope: unit, integration, end-to-end, smoke — plus migrations, seeds, and integrations Baking "which tests does this need?" into the agent's planning phase rather than relying on a human to remember Coverage dogma vs. reality: test what matters, but keep 90% in CI as the practical proxy The daily cron that catches coverage gaps and staleness; local-first test execution for agents Notable Quotes "Every organization we've pretty much ever worked for — no one has confidence in the test. The tests are slow, the tests are flaky, and they don't cover enough stuff." "How do you know that you can trust your agent to ship code that's gonna work?" "We said 90% coverage — great philosophy. Three months later we realized maybe the 10% is the one that matters." About The Velocity Lab Dave O'Dell and Dan McAulay work inside engineering organizations every day helping them ship faster with AI. No hype, no BS — just what's working in the field. Subscribe: RSS