Happy Birthday, Backstage: Spotify’s Biggest Open Source Project Grows Up Fast
TLDR: As Backstage turns one, we’re doubling down on our commitment to the open source project and the community we’re building it with.
From Hack Week hunch to CNCF Sandbox
Last year, a small team of Spotifiers had a hunch about our homegrown developer portal: if Backstage could help our 1,600+ engineers manage the 14,000+ software components we use at Spotify, then couldn’t it do the same for other growing tech companies, too?
The team began building a proof of concept for an external version of Backstage during Hack Week. Just six weeks later Backstage was out in the wild — making its official open source debut one year ago today. A few months and a few thousand pull requests later, what started as a hunch became an early stage Sandbox project at the CNCF (also home to Kubernetes, Envoy, and Helm).
Looking back, the Backstage open source project feels like it has come incredibly far in a short amount of time. But on its first anniversary — as we prepare Backstage for a more stable release and wider adoption — we’re even more excited for what lies ahead.
— Expedia Group Developer Experience Team
Companies as varied as Expedia, Zalando, TELUS, American Airlines, and DoorDash have already started using Backstage. And we remain committed to our long-term vision of seeing Backstage become the standard for all kinds of companies. We think the past year has given us a good head start.
Why a developer portal?
To understand the problems Backstage solves, let’s go back to its beginnings at Spotify — and why we built it in the first place. (If you’ve heard this story before, feel free to skip ahead.)
In March 2020, our internal version of Backstage was already a mature product; our developers had started using a primitive version of it four years earlier. During that period, we were growing fast. We seemed to be adding new developers, new software components, and new tooling at an equally breakneck pace.
Our small, autonomous developer teams have always been our strength. But as we scaled, we didn’t have one way to create a microservice, we had a dozen. We didn’t have one new developer trying to find their way around our stack, we had hundreds.
The faster we grew, the more this fragmentation slowed us down again.
A single pane of glass
Designed first as a basic service catalog, our engineering teams began to gravitate to Backstage on their own — recognizing its ability to streamline workflows, help them align with work being done across the organization, and reduce the daily frustrations that slow developers down.
It became the “single pane of glass” for all our tooling. Everything our developers needed to create, manage, and monitor their projects was in one place. We began to rely on Backstage more and more — from managing data pipelines to software migrations — until it became the hub for all our development work.
With Backstage, infrastructure tooling got out of our engineers’ way so they could build and test faster. And since it simplified discovery — from ownership and documentation to best practices — we could onboard new developers faster, too.
Speed was the key. We saw firsthand that faster developers aren’t just more productive developers, they’re happier developers.
From internal portal to open platform
What’s the biggest difference between the internal version of Backstage and the version we released a year ago? We didn’t want to ship you Spotify’s developer portal. We wanted to ship the best platform for you to build your own developer portal — one that fits your particular needs and use cases.
Unlike the internal version of Backstage, which has more than 120 different plugins built by 60 different teams, the first open source version was mostly an empty shell. Shiny, new, and full of potential — yes. But less like a brand new car and more like a blank canvas.
Since that first day, the promise of that empty shell has been filled in and shaped into a full-featured product, thanks to feedback from early adopters and contributions from the open source community. In the last year:
- We introduced four core features: the Service Catalog, Software Templates, TechDocs, and our new Kubernetes monitoring tool. This is functionality that we think defines the Backstage experience and that everyone would want out of the box.
- We launched the Plugin Marketplace: the ecosystem for open source plugins for Backstage continues to grow, including contributions from individuals, from other tech companies, and software providers, like Snyk.
- We created the Backstage Design System: consistent frontend design is integral to creating a seamless experience inside Backstage, so we developed tools and guidelines anyone can use, including non-designers.
Stabilizing the core
The work we did last year — identifying the core features and iterating on them quickly — has prepared us for what’s next: stabilizing those features and APIs so that more companies can adopt the platform for production use.
In the coming weeks, our team will:
- Bring both the Service Catalog and the Software Templates scaffolder into beta, resulting in a more stable release ready for wider adoption.
- Create an easy, standardized way for developers to build plugins that will encourage contributions and lead to a richer ecosystem for everyone.
- Update other parts of the core app — notably, improving search and incorporating GraphQL systemwide.
You can learn more in the project roadmap.
Adopters: Backstage in the wild!
Beyond the official adopters list, we’ve consulted with hundreds of other companies evaluating Backstage — from digital natives to Fortune 50’s undergoing digital transformations. Our rule of thumb has been that once your org reaches 100 engineers, it’s time to stop managing your infrastructure solely with spreadsheets and Slack channels.
- Early adopters Zalando and SDA SE shared their adoption experiences last month at our first community session.
- Expedia has a team dedicated to rolling out Backstage.
- American Airlines has 20 teams using their version of Backstage, which they named Runway. They’re already seeing some good internal traction:
“We now get upwards of 500+ hits a day from people using not only “Create an App” but also consuming other components in Runway, like Catalog, and our custom plugins. Just a few months ago, this was maybe 50/day.” — Jason Walker, Director, Technology Transformation, American Airlines
- DoorDash is one of our most recent adopters and we’ve been working closely to get them up and running.
“The support we received from the Spotify team, GitHub collaborators, and Discord members enabled us to stand up our initial environment quickly and painlessly, while also inspiring a robust roadmap that will make Backstage our engineering hub.” — Adam Rogal, Director, Developer Platform, DoorDash
A world of contributors grows into a community
Of course, none of this would have been possible without our ever-growing community of contributors from around the world. Since the project’s beginning, the project has averaged two new contributors a week.
This year, we’ve given the global community of maintainers, contributors, adopters, and an official home on the Backstage Community page. As our excitement for Backstage open source continues to grow at Spotify, we hope you will join us there — and in the main repo.
Tags: engineering leadership
