How Etsy Tests A-B Variants Without Slowing Down Engineers

In this episode, Lucas and Luna look at how Etsy runs thousands of A/B tests per year without bogging down their engineering team. The secret? A self-serve experimentation platform with a 'feature flag' system that lets product managers launch variants without touching code. We walk through a specific 2023 case: Etsy tested whether showing 'Free shipping over $35' in search results vs. only on the product page lifted conversion. The winning variant added $8.2 million in annual revenue. But the real lesson is in the infrastructure — how Etsy handles statistical significance, prevents peeking, and kills losing variants in under 48 hours. We also discuss the trade-off: self-serve speed can lead to sloppy test design if guardrails aren't hard-coded. Tune in for a practical look at experimentation at scale. #Etsy #ABTesting #Experimentation #FeatureFlags #SelfServePlatform #ConversionOptimization #Ecommerce #ProductManagement #StatisticalSignificance #PeekingProblem #RevenueGrowth #CROStrategy #DataDriven #Marketing #FexingoBusiness #BusinessPodcast #LucasAndLuna #May2026 Keep every episode free: buymeacoffee.com/fexingo