TL;DR Sometimes we cannot estimate the required sample size needed to power an experiment before starting it. To alleviate this [...]
TL;DR We summarize the findings in our recent paper, Schultzberg, Ankargren, and Frånberg (2024), where we explain how Spotify’s decision-making [...]
When we want to determine the causal effect of a product or business change at Spotify, A/B testing is the [...]
At Spotify, we run a lot of A/B tests. Most of these tests follow a standard design, where we assign [...]
As companies mature, it’s easy to believe that the core experience and most user needs have been resolved, and all [...]
TL;DR: Spotify is releasing a new commercial product for software development teams: a version of our homegrown experimentation platform that [...]
In Part 1 of this series, we introduced the within-unit peeking problem that we call the “peeking problem 2.0”. We [...]
Spotify’s approach to challenges in sequential testing with longitudinal data At Spotify, we’re constantly improving our data infrastructure, which means [...]
Messaging at Spotify At Spotify, we use messaging to communicate with our listeners all over the world. Our Messaging team [...]
In the fast-paced world of streaming, personalization plays a vital role in enhancing user experiences. At Spotify, our Home serves [...]
TL;DR Sequential tests are the bread and butter for any company conducting online experiments. The literature on sequential testing has [...]
TL;DR: Using the properties of the Poisson bootstrap algorithm and quantile estimators, we have been able to reduce the computational [...]
At Spotify, we aim to build and improve our product in a data-informed way. To do that, teams are encouraged [...]
At Spotify we run hundreds of experiments at any given time. Coordinating these experiments, i.e., making sure the right user [...]
So you’ve read Part I of our two-part series about the new Experimentation Platform we’ve built at Spotify, and now [...]
At Spotify we try to be as scientific as possible about how we build our products. Teams generate hypotheses that [...]
