How Do You A/B Test SEO and GEO? Title Tags, Traffic, and the Truth

What does it really mean to run SEO as an experiment instead of a checklist?In this episode, we’re joined by Will Critchlow and Mat Reiss to explore the evolution of SEO and AEO/GEO A/B testing — from enterprise-level experimentation frameworks to a super simple way for any team to get started: title tags.Will breaks down the fundamentals of SEO and AEO/GEO testing and how it differs from traditional user-based A/B testing. Because Google effectively acts as the “audience,” experiments must split pages rather than users and be implemented server-side so crawlers can see the changes. We discuss why organic traffic is a stronger measure of impact than rankings alone, how counterfactual forecasting helps isolate true lift, and why modern SEO increasingly feels like experimenting against a black-box, AI-driven system.Then I called up Mat to help us get real practical, really quickly — with “title tags”, what he calls the “last mile” of search performance. He explains why small copy changes can meaningfully increase click-through rate without improving rankings, shares a case study that drove a 30% traffic lift from a simple tweak, and outlines how teams of different sizes can begin testing title tags without heavy engineering resources.In this episode:How SEO and AEO/GEO A/B testing differs from traditional experimentationWhy organic traffic is often a more reliable KPI than rankingsThe statistical and structural realities of page-based testingWhy title tags are an underused but high-impact growth leverA real-world example of a 30% traffic lift from a simple title changeWhen SEO testing becomes viable for small and mid-sized e-commerce teams