The One Metric That Makes A-B Tests Lie episode artwork

EPISODE · Jun 5, 2026 · 9 MIN

The One Metric That Makes A-B Tests Lie

from Conversion Rate Optimization with Fexingo: A/B Testing, Landing Pages, and CRO Strategy · host Fexingo

Lucas and Luna dig into one of the most misunderstood pitfalls in A-B testing: how your chosen success metric can make a perfectly run experiment give you the wrong answer. Using Booking.com's infamous failed test of a 'Book Now' button — where conversion rate went up but revenue per visitor went down — they explain the difference between a metric that's easy to measure and one that actually matters. They walk through why 'click-through rate' can be a vanity metric, how Amazon prioritizes 'units per session' over 'purchase rate', and why the right north-star metric forces you to define what 'better' really means before you start. No jargon, just a concrete framework you can apply to your next test. #ABTesting #VanityMetrics #ConversionRate #SuccessMetric #Bookingcom #RevenuePerVisitor #NorthStarMetric #MarketingExperiments #DataDriven #LandingPageOptimization #ClickThroughRate #CROStrategy #BusinessPodcast #MarketingPodcast #FexingoBusiness #ConversionRateOptimization #Experimentation #AmazonStrategy Keep every episode free: buymeacoffee.com/fexingo

Lucas and Luna dig into one of the most misunderstood pitfalls in A-B testing: how your chosen success metric can make a perfectly run experiment give you the wrong answer. Using Booking.com's infamous failed test of a 'Book Now' button — where conversion rate went up but revenue per visitor went down — they explain the difference between a metric that's easy to measure and one that actually matters. They walk through why 'click-through rate' can be a vanity metric, how Amazon prioritizes 'units per session' over 'purchase rate', and why the right north-star metric forces you to define what 'better' really means before you start. No jargon, just a concrete framework you can apply to your next test. #ABTesting #VanityMetrics #ConversionRate #SuccessMetric #Bookingcom #RevenuePerVisitor #NorthStarMetric #MarketingExperiments #DataDriven #LandingPageOptimization #ClickThroughRate #CROStrategy #BusinessPodcast #MarketingPodcast #FexingoBusiness #ConversionRateOptimization #Experimentation #AmazonStrategy Keep every episode free: buymeacoffee.com/fexingo

NOW PLAYING

The One Metric That Makes A-B Tests Lie

0:00 9:00

No transcript for this episode yet

We transcribe on demand. Request one and we'll notify you when it's ready — usually under 10 minutes.

Frequently Asked Questions

How long is this episode of Conversion Rate Optimization with Fexingo: A/B Testing, Landing Pages, and CRO Strategy?

This episode is 9 minutes long.

When was this Conversion Rate Optimization with Fexingo: A/B Testing, Landing Pages, and CRO Strategy episode published?

This episode was published on June 5, 2026.

What is this episode about?

Lucas and Luna dig into one of the most misunderstood pitfalls in A-B testing: how your chosen success metric can make a perfectly run experiment give you the wrong answer. Using Booking.com's infamous failed test of a 'Book Now' button — where...

Can I download this Conversion Rate Optimization with Fexingo: A/B Testing, Landing Pages, and CRO Strategy episode?

Yes, you can download this episode by clicking the download button on the episode player, or subscribe to the podcast in your preferred podcast app for automatic downloads.
URL copied to clipboard!