Bringing the Climate Fight to Your IDE

TL;DR As part of Spotify’s goal to reach net zero emissions by 2030, we’ve made climate action one of our five technology pillars and have been integrating sustainability into our everyday engineering practices. This year, climate action was the theme of our company-wide Hack Week, we made climate responsibility part of our official engineering handbook, and we partnered with Thoughtworks to release the Cloud Carbon Footprint plugin for our flagship open source project Backstage. We believe that by providing our developer teams with the knowledge, tools, and support for reducing our greenhouse gas (GHG) emissions from streaming, we can continue to unlock climate action solutions across Spotify’s internal R&D community and across business units. And by publicly sharing our methods and solutions, we hope to scale our impact to developer communities beyond Spotify, as well. 

Sharing our climate tech and culture at the Google Cloud Sustainability Summit

This summer, we were invited to the Google Cloud Sustainability Summit to share our engineering approach to climate action. Spotify was founded in Sweden, a country that continues to lead in its commitment to conservation and sustainability — and you’ll find those same values permeating Spotify’s engineering culture, as well. 

A bottom-up approach to finding solutions

From our headquarters in Stockholm to 40 other Spotify offices worldwide — and thousands more home offices all around the globe — we’re empowering our engineering teams to address climate issues the same way we approach other tech issues: from the bottom up. We provide our developers with the resources they need (knowledge, tools, time, and space), and allow them to tackle the challenges of the future, each in their own way. The more ideas, the better. The faster we can invent and iterate, the faster we can scale and share the solutions that work.

How to hack our way to “Making the Planet Cooler”

It’s with that spirit that we kicked off our annual, company-wide Hack Week. This year’s theme: “Making the Planet Cooler”. We wanted to see what could happen if we gave our dev teams — our engineers, data scientists, designers, and product managers — the time and space to focus on climate solutions that leverage both our technology and the reach of the Spotify platform. 

Knowledge inspires

We also wanted to inspire our teams with the passion and knowledge of climate experts — people who could give us a crash course in climate issues and fuel our imaginations. So we created online content hubs and hosted talks by special guests, including podcasters who produce and report on climate issues, and Saul Griffith, an inventor and MacArthur fellow, who shared his practical blueprint for decarbonizing the planet by electrifying everything.

Five days, a hundred solutions

Equipped with knowledge and inspiration, 2,441 Spotifiers spent five days hacking around the clock from over one hundred locations worldwide. They came up with hundreds of ideas and demos, from a green tour planner for artists to a climate footprint calculator for listeners. You can read more about four of the Hack Week projects here. Two other ideas that came out of that Hack Week — a climate engineering handbook and open source software for tracking your cloud emissions — are great examples of how we’re making climate action more accessible to engineers, both inside and outside Spotify. 

Adding a new chapter to Spotify’s Engineering Handbook

The Spotify Engineering Handbook contains the core principles and practices of Spotify’s internal R&D community. Along with our tech architecture and strategies, it has what every engineer needs to know about building software at Spotify, including best practices for reliability, compliance, security, cost management, and documentation. To that list of fundamental engineering priorities, we’ve now added climate awareness and responsibility.

Necessary knowledge

Written with our specific infrastructure and ways of working in mind, the climate engineering section of the handbook first helps engineers understand basic concepts (what are C02 emissions and how much is a metric ton of C02e, anyway) before explaining the main sources of emissions from Spotify’s streaming business. These fall into two broad categories:

• Direct emissions from using our product, including all our computing resources running in the cloud, as well all the phones, laptops, TVs, game consoles, and other devices running the Spotify app. 
• Indirect emissions caused by related business activities, including emissions from our upstream partners, such as CDNs and mobile networks.

From there, we keep digging deeper into the details. For example, we ask, what is the emissions cost of activating the radio chip on a phone? For cloud computing, we evaluate the power consumption of the server’s various components: how many watts come from CPU usage vs disk usage vs running the cooling fan? Learning everything we can about our sources of emissions helps our engineers target their efforts and maximize their impact, as well as discover unexploited opportunities.

Defining the scope of our climate impact remains an important focus area for our climate teams. And we hope to share more of what we learn about how to account for emissions from streaming in the future.

Actionable advice

The handbook also includes hands-on guides with the most impactful actions a Spotify engineer can take to reduce those emissions in their daily work, from prioritizing hardware acceleration to remembering to clean up their cloud resources after they’re done running an experiment (or a Hack Week project). Engineers can find recommendations specific to their discipline, for example:

• If you’re a backend engineer, the handbook explains the importance and impact of autoscaling and rightsizing your cloud resource usage instead of relying on alerts.
• If you’re a mobile developer, the handbook reminds you that, when you account for how many times the app is downloaded and updated, reducing the app size by just 1 MB could save as much as 500 tons of C02 a year.

While the climate crisis can feel overwhelming at times, the handbook breaks the problem down the way you would break down any other large-scale engineering problem: into knowable parts and rational solutions.

Taking on the climate fight as part of your dayjob

In some ways, engineers have always been doing this work when they optimize for efficiency and reducing waste: the climate-responsible solution is also often the “better-engineered” solution — more efficient, more reliable, easier to maintain. 

The difference is prioritizing climate responsibility as an engineering goal from the start and making it part of your developer’s everyday practice instead of a separate routine. We have dedicated climate squads, just as we have dedicated security squads, but we want all of our engineers to have both security and climate responsibility on their minds as they do their work. We think making it an official part of the Spotify Engineering Handbook is a first important step.

Cloud Carbon Footprint plugin for Backstage

But what about outside Spotify? Can our priorities and ways of working translate beyond our walls to other organizations? We think that by appealing to the engineer’s mindset our approach to climate action becomes more universal. Developers are developers, and developers want to solve problems. A great place to test that theory: Backstage, the open platform for developer portals that we donated to the Cloud Native Computing Foundation. 

Saving the earth, starting with the clouds

During Hack Week, we partnered with Thoughtworks to build the Cloud Carbon Footprint plugin for Backstage. Built with support for AWS, Azure, and Google Cloud infrastructures, the CCF plugin helps your dev teams understand how their cloud usage impacts the environment and what they can do about it.

• Converting cloud usage into emissions estimates: The plugin incorporates best-practice methodologies for converting cloud utilization into estimated energy usage and carbon emissions, displaying metrics and carbon savings in easy-to-read dashboards. 
• Equivalency calculator: C02 emissions estimates are also converted into relatable comparisons, converting emissions usage into one-way flights between New York and London, millions of smartphones charged, tree seedlings grown. When the numbers can start to feel too abstract, the equivalency calculator brings your impact back down to earth.
• Plus, forecasts, recommendations, carbon intensity maps, and more: The data-rich displays in the plugin’s dashboard follows the same strategy as our other climate efforts: if we provide the knowledge and the tools, action will naturally follow. 

Want engineering solutions? Go where the engineers are

The CCF plugin shares many similarities with our Cost Insights plugin for Backstage — shifting resource management left — closer to the engineer who actually knows what the resources are being used for and how to optimize them.

This also points to one of the advantages of building this functionality for a platform like Backstage. Internal developer portals serve as the hub for engineering organizations. It’s where our engineers spend their days, so if you want your engineers to have climate on their minds, it makes sense to put it where they manage all of their other engineering responsibilities.

An open marketplace, a platform for change

Distributing the plugin as part of the open Backstage ecosystem also allows it to reach a wide variety of engineering organizations. Since its release, Backstage has been publicly adopted by over 200 companies, from American Airlines to Zalando, Expedia to HBO Max, DAZN to TELUS, Wayfair to VMware. As a free plugin in the Backstage Plugin Marketplace, the CCF plugin has the potential to reach tens of thousands of engineers.

Climate responsibility as engineering practice

Carbon offsets are a useful way for reducing the impact of emissions — but it’s not where an engineer’s mind goes first as a solution. We want solutions that are themselves sustainable and whose impact can scale. Ideally, we are building a better, brighter future, not just buying our way out of the past.

A technical challenge unlike any other

From our efforts so far, we’ve recognized two equally important things: (1) sustainability can be treated just like any other engineering problem, something that we can invent, build, and optimize solutions for; and (2) the climate fight can be a deeply personal one, not just another engineering problem at all. It’s a cause that can’t be ignored any longer, and many of us feel compelled to contribute to it because of that sense of urgency. Approaching the problem from an engineer’s point of view allows us to turn our concerns into action.

As engineers, we always want our work to have as much impact as possible. Even when we write a simple script that saves a couple of mouse clicks, we consider how two mouse clicks multiplied across hundreds of millions of users and billions of interactions scales to make a meaningful difference. We love to optimize as much as we love to invent: striving for 99.9999999% uptime, a website that’s a millisecond faster, an app download that’s 1 MB lighter.

We already apply this impact-focused, problem-solving mindset to feature development, security, reliability, resource management, and other engineering priorities. Now it’s time to apply it to climate responsibility, too. Whether it’s how we build a new feature, fix a bug, design an API, or architect a system, sustainability should become part of our job descriptions as engineers, not a side gig; a specification, not an afterthought; a measure that distinguishes a good hack from a poor one.

All hands on deck

While our bottom-up approach is familiar — tapping into the speed of our many distributed but aligned teams — the enormity of the climate issue brings a passion and urgency to our efforts. Reducing our app’s download size by 1 MB takes on new significance when we realize the impact that 1 MB multiplied by hundreds of millions of listeners has on our carbon footprint. 

We believe that climate change isn’t going to come from a single silver bullet to solve the problem. Instead it’s going to come from many different people united in making many changes. 

Of course, not every piece of code we write can live up to some elegant ideal of engineering. Hacking is just as important, especially for finding new solutions to old problems. As we’ve seen from our bottom-up approach, a single week of hacks, and the creativity, ingenuity, and passion of Spotify’s distributed and diverse developer community, sometimes even the ugliest code can show us the best way forward.

Tags: backend, engineering leadership