EP 75: How to A/B test emails when the old playbook is broken

Bad tests don't just waste time. They send you in the wrong direction with confidence.In this episode of Send It, Jimmy Kim and Chase Dimond break down why traditional email testing is broken in 2026; and exactly how to fix it. Between Apple's Mail Privacy Protection inflating open rates and smaller segments making statistical significance harder to reach, the old way of testing leads to bad decisions disguised as data. This episode is the updated framework that actually works right now.EP 75: Creative testing that actually works in 2026Here's what they cover:-Why Apple MPP destroyed open rate as a reliable testing metric-The single biggest testing mistake brands are still making in 2026-The four layers of testing in order of impact; from positioning down to button colors-Three approaches to testing when your list is too small for traditional A/B splits-How to build a creative library that makes every future test smarter-The learning velocity mindset that separates the best email teams from everyone else-The HCE framework: hypothesis, control, evaluate-How to know when you actually have a winner and when it's just noiseThank you our friends at Omnisend, Inboox and eCom Email Certified for sponsoring the podcast!Omnisend is an all in one eCommerce email and SMS marketing automation platform that enables marketers to engage customers and drive sales through personalized experiences. https://www.omnisend.com/sendit/eCom Email Certified is a hands-on, results-driven program with a step-by-step guide to mastering retention marketing and driving serious results. Access over 25 hours of learnings and become a certified email & SMS marketer. Use code: SENDIT! for $300 OFF https://www.ecomemailcertified.com/Explore Inboox.ai, the AI-powered database of 1M+ real ecommerce emails with full HTML, performance data, and trend insights from the fastest-growing Shopify brands. https://inboox.ai/______00:00 Welcome to the show01:13 Why email testing got so much harder in 202604:05 How Apple MPP destroyed open rate as a reliable testing metric05:20 The metrics you should actually be testing against08:16 The one variable rule and why it matters10:25 Layer 1: Positioning - the highest leverage test most brands skip12:10 Layer 2: Structure - how the email is built and why it matters14:15 Layer 3: Creative - where most brands start but shouldn't15:21 Layer 4: Timing - when to send and why it matters for certain emails16:16 How to test effectively when your list is small20:45 Building a creative library that makes every future test smarter23:50 Learning velocity: why the best teams learn faster than everyone else27:58 The HCE framework: hypothesis, control, evaluate30:13 Recap of everything and final thoughts______Enjoying the pod? Subscribe to The eCom Email Marketer Newsletter for your weekly dose of retention strategy, actionable insights, & exclusive industry news! http://ecomemailmarketer.com/______Follow Jimmy & Chase on social!Follow Jimmy Kim at: https://x.com/yojimmykimhttps://www.linkedin.com/in/yojimmykim/https://www.instagram.com/yojimmykim/https://www.tiktok.com/@yojimmykim Follow Chase Dimond at: https://x.com/ecomchasedimondhttps://www.linkedin.com/in/chasedimond/https://www.instagram.com/chasehunterdimond/https://www.tiktok.com/@chasehunterdimondContact us at [email protected] for sponsorships, appearance requests, interviews, or general information!