May 15, 2024
Fixed-Power Designs: It’s Not IF You Peek, It’s WHAT You Peek at
TL;DR Sometimes we cannot estimate the required sample size needed [...]
Tag Archive: experimentation
-
-
March 5, 2024Risk-Aware Product Decisions in A/B Tests with Multiple Metrics
TL;DR We summarize the findings in our recent paper, Schultzberg, [...]
-
September 28, 2023How to Accurately Test Significance with Difference in Difference Models
When we want to determine the causal effect of a [...]
-
August 24, 2023Encouragement Designs and Instrumental Variables for A/B Testing
At Spotify, we run a lot of A/B tests. Most [...]
-
August 16, 2023Experimentation at Spotify: Three Lessons for Maximizing Impact in Innovation
As companies mature, it’s easy to believe that the core [...]
-
August 3, 2023Coming Soon: Confidence — An Experimentation Platform from Spotify
TL;DR: Spotify is releasing a new commercial product for software [...]
-
July 25, 2023Bringing Sequential Testing to Experiments with Longitudinal Data (Part 2): Sequential Testing
In Part 1 of this series, we introduced the within-unit [...]
-
July 18, 2023Bringing Sequential Testing to Experiments with Longitudinal Data (Part 1): The Peeking Problem 2.0
Spotify’s approach to challenges in sequential testing with longitudinal data [...]
-
June 28, 2023Experimenting with Machine Learning to Target In-App Messaging
Messaging at Spotify At Spotify, we use messaging to communicate [...]
-
June 15, 2023Experimenting at Scale, the Spotify Home Way
In the fast-paced world of streaming, personalization plays a vital [...]
-
March 21, 2023Choosing a Sequential Testing Framework — Comparisons and Discussions
TL;DR Sequential tests are the bread and butter for any [...]
-
March 23, 2022Comparing Quantiles at Scale in Online A/B Testing
TL;DR: Using the properties of the Poisson bootstrap algorithm and [...]
-
February 28, 2022Search Journey Towards Better Experimentation Practices
At Spotify, we aim to build and improve our product [...]
-
March 10, 2021Spotify’s New Experimentation Coordination Strategy
At Spotify we run hundreds of experiments at any given [...]
-
November 2, 2020Spotify’s New Experimentation Platform (Part 2)
So you’ve read Part I of our two-part series about [...]
-
October 29, 2020Spotify’s New Experimentation Platform (Part 1)
At Spotify we try to be as scientific as possible [...]
