IntelliJAMS EP 056: Instead of planning your tests, test your plan

Conversion rate went up, but did profit? Most testing programs optimize for one metric in isolation, and that's exactly where they go wrong. In this episode, Amanda Siegel (Director of E-commerce at Scale Media) breaks down why she stopped calling what she does "CRO" five years ago and started treating testing as a tool to validate business strategy, not just chase website wins.In this episode, Alex and Amanda explore what happens when you balance conversion rate, AOV, and subscription adoption in every test instead of picking one, why the old UX tricks (spin-to-wins, slashed prices, CTA color swaps) are losing their edge, and how to get buy-in for short-term metric dips that lead to long-term growth.Timestamps:0:00 - Intro and why Amanda calls herself "a little crazy"1:20 - The limits of traditional CRO and flat testing3:00 - What's changed in the last 10 years of e-commerce UX4:17 - Consumers are catching on to optimization tricks6:10 - Why reducing discounts is actually improving conversion7:36 - Shifting from product sales to business strategy (bundles, subscriptions)9:02 - No silver bullets: what works for your business vs. someone else's10:20 - Conversion rate is one-dimensional and easy to game11:43 - The growth experiment mindset: testing for business growth13:58 - Intentional trade-offs and why short-term losses can be worth it15:10 - The two most important questions in commerce16:19 - Getting buy-in when a metric temporarily looks bad19:15 - "Testing is a tool, not a job title"21:14 - Diminishing returns of small, isolated tests22:58 - "Don't plan your tests. Test your plan."24:03 - Amanda's advice: be bold, back-plan from ideal outcomes25:39 - Why testing has to be cyclical, not one-and-done27:06 - Where to connect with AmandaTopics covered:Why conversion rate optimization (CRO) is too narrow for modern e-commerceBalancing conversion rate, AOV, and subscription adoption in a single testHow consumer behavior has evolved and why old UX tricks are losing effectivenessThe counterintuitive finding that reducing discounts can improve conversionUsing A/B testing and experimentation to validate business strategy, not just website changesGetting executive buy-in for short-term metric trade-offsThe growth experiment mindset for Shopify brandsWhy testing needs to be cyclical and revisited over time---Want to think bigger about your testing program? Join GEM Academy for free courses and a community of e-commerce operators sharing what actually works: https://www.skool.com/intelligems-academy-1535Connect with Amanda Siegel: https://www.linkedin.com/in/amandadsiegel/Connect with Intelligems:Website: https://intelligems.ioBlog: https://intelligems.io/blogLinkedIn: https://www.linkedin.com/company/intelligems