Making Informed Decisions in A/B Tests with Multiple Metrics [Spotify]
An episode of the Snacks Weekly on Data Science podcast, hosted by Pan Wu, titled "Making Informed Decisions in A/B Tests with Multiple Metrics [Spotify]" was published on July 29, 2024 and runs 13 minutes.
July 29, 2024 ·13m · Snacks Weekly on Data Science
Summary
In this episode, we will touch on the importance of A/B testing in the product decision-making process. We will share the four types of metrics and the decision-making framework used by the Data Science team at Spotify, as well as the necessary statistical adjustments that need to be incorporated into experimentation to ensure a solid statistical foundation. For more details, you can refer to their published tech blog, linked here for your reference:https://engineering.atspotify.com/2024/03/risk-aware-product-decisions-in-a-b-tests-with-multiple-metrics/
Episode Description
In this episode, we will touch on the importance of A/B testing in the product decision-making process. We will share the four types of metrics and the decision-making framework used by the Data Science team at Spotify, as well as the necessary statistical adjustments that need to be incorporated into experimentation to ensure a solid statistical foundation.
For more details, you can refer to their published tech blog, linked here for your reference:
https://engineering.atspotify.com/2024/03/risk-aware-product-decisions-in-a-b-tests-with-multiple-metrics/
Similar Episodes
Jun 19, 2025 ·46m
Jun 13, 2025 ·40m
May 20, 2025 ·80m
May 13, 2025 ·74m
May 7, 2025 ·64m