Exclude from Your Taste Profile

What is “Exclude from your taste profile”?

Are you a parent forced to put the Bluey theme song on repeat? Do you work from home and play lofi beats or ambient piano music? Do you fall asleep to peaceful ambient noises?

Are you bummed out when these songs come up as your most listened to song each year or pop up on your Discover Weekly? If so, you’re not alone!

Most of our users say that they choose Spotify over other platforms because they absolutely love our personalization. But for a long time, that personalization has struggled to accurately reflect certain users’ music preferences because of one main factor: our systems have treated everything a listener streams as “representative” of their musical taste. 

This fundamental assumption generally works, but it fails due to three behavioral patterns we observed in how listeners use Spotify:

• Listeners play music for functional purposes (e.g., sleeping, working out, learning a new instrument or K-pop dance), and they don’t want this listening activity reflected in their recommendations.
• Listeners stream music for other people (e.g., playing music for kids, setting the mood for a party, etc.), and they don’t want this listening activity reflected in their recommendations.
• Listeners share their accounts — and they don’t want this shared listening reflected in their recommendations.

In February 2023, Spotify responded to these complaints by launching a new feature: Exclude from your taste profile. This feature introduced a simple new button in the playlist menu that would let users tell Spotify when they wanted a playlist to have less impact on their music recommendations.

How it started

You may be wondering, What is a “taste profile”? “Taste profile” is another way to say “what influences your recommendations”. It’s not something tangible that can be retrieved or looked at, but rather it’s Spotify’s interpretation of your taste based on what you listen to and how you listen to it. Your taste profile informs your recommendations and your taste summaries, like Blend.

Back in 2019, I was one of the struggling Spotify users mentioned above — I am a fervent sleep-music listener, and every week my music recommendations were filled with sleepy piano sounds and atmospheric synth tracks. Meanwhile, during my day-to-day, I typically vibed out to upbeat EDM, groovy funk jams, and West African tracks, but these rarely appeared in my recs.

Despite working on the core music recommendation systems behind Spotify’s personalization, there was little I could do to stop my Discover Weekly from being Discover Sleepy. And so, during Hack Week 2019, I decided to tackle this problem head-on.

I started out by looking at my own listening history and found out that more than 30% of my listening came from my sleep music — it’s no wonder Spotify thought I was a huge ambient music stan! I then polled around the Spotify employee base and found that this problem was common beyond just sleep music; kids’ music, lofi beats, and workout pump-up jams were among the most common types of music that employees were frustrated with finding in their recommendations. We came to call this type of listening “functional music listening”: music that users listened to for functional purposes but that didn’t reflect the type of music that they were interested in discovering more of.

To understand the problem at scale, a few of us performed an analysis of the broader Spotify user base and identified that there were millions of users who consumed functional music. Most importantly, we determined that users who consumed this type of music utilized Spotify’s recommendations less than users who didn’t consume functional music.

To address this issue, I worked with a team of analysts and engineers every Hack Week for the next three years with the goal of building something we could put out into the world.

“It’s just a button, right? What could be so hard about that?”

To a user, this feature appears as a simple button in the playlist context menu. 

However, simplicity hides complexity. If you peek behind the curtain of the “Spotify Algorithm”, you won’t find a single all-knowing AI. Instead, you’ll find an intricate collection of algorithms and machine learning models working in concert _{[pun intended]} to produce the right music recommendations for every user, in every moment. When this “simple button” is pushed, we need to tell all these machines, “Any time user X streams music from Playlist Y, just pretend those streams didn’t happen”. So how did we do that?

Nearly all the surfaces that recommend music throughout Spotify rely on a few core systems to provide them with the user’s taste profile. These key systems keep track of your recent listens, your top tracks and artists, where you stand as a user in the abstract musical universe (your “user embeddings”), and even your interactions on the Spotify app. We strategically decided to apply the Exclude from your taste profile preferences directly within these taste profile systems. 

By taking this approach, the feature can impact recommendations on Home, Search, Discover Weekly, Radio, etc. — without the teams that own those surfaces having to make changes to their own systems. For some specific surfaces such as Home Shortcuts and Recently Played, they actually don’t want to apply the filtering logic, so we also made sure they could fetch an unfiltered view of the data mentioned above.

Why playlists?

Because I get this question a lot, I would be remiss to not address it here. There are a lot of entities (tracks, artists, podcasts, etc.) on Spotify, so why did we implement this just for playlists?

Analysis and user research showed us that playlists were the source of most functional music, and with limited resources to start, it made the most sense to pick one entity and do it right. Playlists would be the proving ground to expand to other entities down the road.

Once we had chosen playlists as our target entity, we still had a number of decisions we had to make about how this feature would work. When we exclude a playlist, should we exclude all tracks from that playlist, or just any plays that came from it? What happens when the tracks in that playlist change — do we stop ignoring them?  Should we exclude plays from the past or just the ones after the user clicked the button? Should we infer the type of music that the playlist comprises and exclude all that type of music from the user’s taste profile? 

For most of these choices, we ultimately went with the option that was clearest from an implementation standpoint, but would also have the most impact on the user’s recommendations:

• We chose to exclude any plays from the excluded playlist, but plays of those tracks outside the playlist would still count toward the taste profile. 
• To have a noticeable effect, we decided to exclude future and past plays from the given playlist. If you’ve been listening to a white noise playlist every night for a year and we only excluded plays that happened after you clicked the Exclude button, then it potentially would take many months for that music to show impact on your recommendations.
• Since we aren’t mind readers, we decided that it would be best not to assume what other content a user might want excluded just from a single action. So we went as explicit as possible, which made it relatively straightforward to implement in the taste profile systems.

The A/B test

After numerous Spotify Hack Weeks, we finally had the feature working and ready to put in front of some users. However, because this was a bootstrapped initiative, it wasn’t fully built behind the scenes. Of the systems mentioned above, we had only affected the user embeddings, with the hope that it would be enough to show us some improvement. Additionally, we decided to implement the filtering directly on one specific recommended set: On Repeat.

Throughout the development, we put the feature in front of employees and took feedback from them to quickly iterate on it before it went out to users. This was an essential step in both debugging the UI and confirming that the effect on the recommendations was significant.

Even though the feature was well-received around Spotify, we knew that we couldn’t just go in and start tweaking the core recommendation systems. In order to launch this feature to Spotify users around the world, we needed to prove that our approach would actually improve the listening experience of our users. There’s one clear way to get that kind of evidence: an A/B test with real users.

We identified about 100 Spotify editorial playlists that indicated a user may have a high propensity for wanting this feature (e.g., Peaceful Piano, Deep Sleep, Sing Along, etc.). During the test, when users visited these playlists, they were greeted by a targeted message informing them about the feature with a call to action allowing them to exclude that playlist from their taste profile.

When the test was completed, the results exceeded our expectations! The data showed that users of the feature started consuming more recommended content, as well as more content overall on Spotify. The targeted message also had a massive influence on the feature, with about 12% of users clicking the Open Menu button, indicating that even if they didn’t use the feature, there was a ton of interest in improving their recommendations.

The final stretch

Having the cold hard data™ to show this approach worked allowed us to get the remaining work prioritized by the teams that we depended on for productionization.

As noted above, for the A/B test, we only implemented the influence of the feature on the user embeddings, but over the next few months, the production teams came in and implemented the logic on the recent listening, top tracks and artists, and interactions. And of course, we wanted this feature to work globally, so we translated the UI into 88 different languages.

The last detail we had to iron out was where Exclude from your taste profile would influence. In Spotify, our algorithmic sets and surfaces each have their own unique content proposition. While playlists like Discover Weekly are purely focused on discovery, we also have Your Shortcuts and Recently Played lists on the Home screen, which are the essence of what we call “retrieval”. In between, you have features like Daily Mix and Search, which are intended to help you find new music while mixing it with music you’re familiar with,

The “taste profile” is an abstract concept that is used differently by each surface in Spotify. Ultimately, we left it up to the teams that owned those surfaces to determine whether or not their use cases were more on the ”discovery” side or the “retrieval” side. In general, places in the app that focused on the retrieval use case tended not to filter out these “excluded” plays when looking at a user’s listening, because they aimed to reflect factual information rather than interpretations of a user’s taste.

In February 2023, the feature finally shipped! As of today, it’s been used by more than 20 million users, and on average they’re consuming  about 4% more music from recommendations than they previously were. In fact, listeners who use the Exclude from your taste profile feature consume nearly 10% more music on Spotify overall. While most features we ship move these numbers by fractions of a percent, this feature clearly has had a major impact on user experience.

Wrapping it up

If you have a big idea that you’d like to bring to your company, but it requires more resources than what exist, here are some tips to reaching your objectives:

1. Prepare yourself! Be ready to wear any hat needed for the project and expect to spend some evenings burning the midnight oil.
2. Perform an offline analysis of existing data to understand potential impact. Figure out which metrics your feature should try to move — if they’re metrics that the company already cares about, even better!
3. Hack together the simplest prototype you can — don’t worry about moving metrics just yet. It’s okay to just build a “painted door” if you can only build the UI and not implement the backend.
4. Build an MVP which, if tested, could move one or two of the metrics you care about. Aim small, but aim for actual impact. Then, run an A/B test.
5. Put on your PM hat (if that’s not your normal hat) and build a roadmap for the feature. Identify what is needed from whom, and put everything in a one-page product brief.
6. Use the cold hard data from the A/B test plus the roadmap to get buy-in. Escalate through leadership to get help coordinating the necessary teams.
7. Try to put a target launch date on the calendar — this will probably be a straw man, but it helps for prioritization. Work with everyone to build the thing(s) and ship it!

By applying these tips, we’ve been able to launch this feature and give users more control over their recommendations, but we know that it’s not yet perfect. We’ve been getting a ton of great feedback from users on where to go next with the feature and rest assured, we will continue working to make Spotify’s music recommendations better than ever!

Acknowledgements

This feature wouldn’t have been possible without the help of many people and teams around Spotify.  Although there are too many to name, a few people deserve a truly special thank you for their roles in Exclude from your taste profile:  Dominique Izbicki (Product), Manizeh Khan (Product), Erica Leh (Data Science), Mat Budelman (Design), Pierre Thomain (Mobile Eng), Erik Price (Mobile Eng), Nandini Singh (Data Science), Joni Martens (User Research), Jeremy Salfen (Data Science), Bindia Kalra (Data Eng), April Gustafson (Data Eng), Deepa Chawla (Data Eng), Baturalp Torun (Data/Backend Eng), Anton Vietrov (Backend Eng), Daniel Byrne (PR), Brittney Le Roy (PR), Alexandera Wei (Product Marketing), Jill Pollock (Legal), and Chris Bennett (Legal), and anyone else that I missed!

Tags: machine learning