17# Kelly Wortham (Forward Digital, Test & Learn Community, Experimentation Island) on Why Agentic Commerce Breaks Experimentation, Measuring What Visitors No Longer Do, and Optimizing for Machines

In this episode of Knowledge Distillation, Katrin Ribant speaks with Kelly Wortham – founder of Forward Digital and the Test & Learn Community, a network of several thousand experimentation practitioners, and curator of the Experimentation Island conference. Kelly came into the industry from academia and social sciences, where running experiments meant clipboards in shopping malls and t-tests on hiring processes. That grounding in the messy real world is exactly what makes her take on agentic commerce so sharp: she sees the current shift not as a technology problem, but as a measurement crisis hiding in plain sight. The core of the episode is a problem most experimentation teams haven’t fully reckoned with yet. As AI-referred traffic grows, visitors arrive at websites already persuaded – they’ve done their comparison shopping in ChatGPT or Perplexity and are now just confirming what they already know. A/B tests designed to persuade don’t work on people who’ve already been persuaded. And because most companies have no clean way to separate AI-referred from non-AI-referred traffic, results get blended into noise. Kelly’s practical advice: start segmenting AI-referred traffic now, even if the data is messy, because building that muscle early is more valuable than waiting for a clean solution that doesn’t exist yet. The episode closes on a category Kelly calls brand impact tests – experiments that happen entirely off your website, in the third-party content ecosystem that AI models are trained on: reviews, product descriptions, social mentions. These are the inputs that shape what an AI recommends before a customer ever lands on your page. And on a provocation both find genuinely exciting: after years of optimizing for clicks and dark patterns, agentic commerce may be forcing brands back to clarity and human-first design – because optimizing for machines increasingly means optimizing for humans. All episodes on our website: www.ask-y.ai/knowledge-distillation-podcast Learn more about ASK-Y: www.ask-y.ai