TL;DR  LLM evals, automated judges that assess relevance, coherence, and quality at scale, are a powerful new...

The technical and practical rationale for a clear separation between these domains.

TL;DR Spotify’s experimentation platform, Confidence, scaled product decision-making across hundreds of...

TL;DR Sometimes we cannot estimate the required sample size needed to power an experiment before starting it....

TL;DR We summarize the findings in our recent paper, , where we explain how Spotify’s decision-making engine...

When we want to determine the causal effect of a product or business change at Spotify, A/B testing is the...

At Spotify, . Most of these tests follow a standard design, where we assign users randomly to control and...

As companies mature, it’s easy to believe that the core experience and most user needs have been resolved,...

TL;DR: Spotify is releasing a new commercial product for software development teams: a version of our...

In of this series, we introduced the within-unit peeking problem that we call the “peeking problem 2.0”. We...

At Spotify, we’re constantly improving our data infrastructure, which means we can get feedback on...

At Spotify, we use messaging to communicate with our listeners all over the world. Our Messaging team powers...