Achieving Team Purpose and Pride with Scrum
Team purpose and pride — my team hit those high marks, but it was a long journey to get there from where we started.
At Spotify, we strive for “aligned autonomy” among our teams. Meaning: we align on what it is we set out to do, but preserve flexibility to choose how we’ll achieve those goals. Letting teams adjust their processes to work for them promises many benefits (innovation, lower overhead, team happiness, speed, etc.), but it takes intentional team effort to make these adjustments.
While this international effort towards aligned autonomy has shown dazzling success and efficiency across the company, my team was struggling to make it work, finding ourselves with a process that wasn’t working for us. This is the story of how we changed that.
Our problem
Our squad had long been following a process comprising bits and pieces from the Scrum framework, an agile methodology developed in the 1990s by Ken Schwaber and Jeff Sutherland. However, we hadn’t connected the Scrum practices we were using — like stand-ups, two-week sprints, and retros — to the principles behind them, and we hadn’t woven them together cohesively as a system. As a result, we found ourselves with a surprising lack of structure and clarity: our meetings often felt purposeless, we never finished our sprints, and our product manager had a difficult time knowing what could reasonably be expected to be delivered at any given time. We, as engineers, also had little sense of how our day-to-day work fit into a larger quarterly picture, or how close our team was to achieving its goals. This left many of us with a gnawing feeling that our team rhythm could be better, though we weren’t quite sure how to get there.
The goals we ultimately wanted our process to achieve were:
- Continuous improvement: We wanted to iterate better — to easily and fluidly understand our work and find opportunities where we could improve.
- Shared understanding and transparency: We wanted everyone on the team to know at any given time what work was happening, and what it entailed.
- Confidence: We wanted to be able to more confidently plan our long-term trajectory and communicate with stakeholders about what they could expect.
Our approach
To help us reach our goals, we sought the help of a Spotify Agile coach, who first guided us through an assessment of our existing ways of working. Since our team generally liked the Scrum framework but wasn’t using it holistically, our Agile coach helped us dig deeper into how the Scrum elements work together as a whole. Each piece has a specific role to play and interacts with each other piece. Ultimately, we unanimously agreed to adopt Scrum more or less “by the book”: that is to say, following the entire framework laid out in the Scrum Guide, rather than just disconnected bits of it.
Backlog refinement
Goal: Create a shared understanding of each ticket, as well as how “large” it is, so that the PM can prioritize accordingly.
Before these process changes, we were itching for a succinct way to size our stories; sometimes stories would get pointed during a planning meeting, but more often than not, we were bringing many unsized stories into a sprint. This meant that we had virtually no gauge of how much work we were bringing in or committing to.
With the help of our coach, we began holding a weekly backlog refinement meeting. We alternate each week between “coarse refinement” — in which we hone in on tickets, ask questions, and find collective understanding — and “fine refinement”, in which we actually point those tickets.
This system ensures that everyone has an opportunity to ask questions and shares a basic understanding of every ticket. We all know how much work we are committing to when we begin a sprint, and it also allows us to compare, sprint by sprint, how many points we are finishing as a team.
Sprint planning
Goal: Create a sprint full of stories ready to be picked up, and which we feel confident we can deliver on time.
Previously, our sprint planning process didn’t allow for us to share a collective grasp of each of the tickets in the backlog before our sprint planning ceremony, so we spent most of the two hours reading about the tickets and trying to arrive at an agreement about which ones felt important to bring in.
Now, because all the tickets are pointed and prioritized in the backlog ahead of time, the process is very simple: we go down the backlog — full of tickets we’ve already pointed and discussed — and simply do any subtasking to get clearer on the actual work we’ll be doing. After each ticket we review and bring into the sprint, we check whether the team feels we can take on more. By the end, we have a sprint full of fully subtasked stories we thoroughly understand, and that we’re confident we can deliver within two weeks.
Sprint review
Goal: Review the sprint’s work, celebrate achievements, and note what new tasks came out of this sprint.
While we already had a retro in which we talked vaguely about the successes and challenges of the sprint, we didn’t evaluate the work in terms of our team’s product prioritization.
In a 30-minute sprint review, we demo the features completed, and ask ourselves some basic questions:
- What work did we complete?
- Is there anything we need to extend or add to what we’ve done?
- Did we discover any tech debt?
- Are we on track to meet our longer-term goals?
This allows us to regroup and reprioritize work accordingly for the next sprint, which begins the following day.
Retro
Goal: Bring team celebrations and concerns to the table; arrive at an action item to implement in order to improve team process.
In previous retros, we all jotted down our notes and talked a little bit about the many things that had come up during the sprint, but we didn’t discuss action items sufficiently in order to implement them.
Now, we continue to create those notes, but then vote on a single issue to spend the majority of the retro discussing and ideating to solve.
By the end of the retro, we now have an implementable action item that can be tracked throughout the next few sprints. These action items allow us to actively resolve pain points and, in turn, make progress toward our broader goal of continuous self-improvement.
Stand-ups
Goal: Establish a shared understanding of the day-to-day state of the team’s work, and make any adjustments needed to unblock any team member.
Incorporating a key question to the end of stand-ups has helped the team prioritize and make adjustments where needed: “How likely are we to complete this sprint, on a scale of 1 to 5?” All at once, each team member holds up 1 to 5 fingers to communicate their answer. If anyone holds up three or fewer fingers, we invite a deeper discussion. This helps us catch and swarm on problems early, even if only one person has noticed them.
Recommendations
With simple adjustments to our Agile process, we found a meaningful change in our working rhythm. If you’re thinking about revamping your team’s Agile process, you can give these steps a try:
1. Try out a system holistically before making adjustments.
Agile systems are designed with a lot of intention. Honoring all of the different parts will allow you to experience the originally intended benefits, before fine-tuning the nuances to your specific use case.
2. Ask the “stand-up question”.
Asking “How confident are we that we will finish this sprint?” gives team members the opportunity to voice their concerns and offer potential solutions.
3. Focus on a single issue in retros.
Allow team members to vote on one or two issues to discuss at length, so there’s time and space to brainstorm actionable solutions.
4. Plan sprints you can finish, and commit to finishing them.
Create multiple decision points during the sprint planning process where team members can decline work. Planning accurately sized sprints and committing to finishing them will help teams run like a well-oiled machine.
These changes allowed our team to finally experience the great feeling of actually finishing a sprint and celebrating what we’ve accomplished, as well as giving us increased confidence when communicating our deliverables to stakeholders. We also found expanded opportunities to learn and collaborate, as backend and frontend engineers became more T-shaped to finish the sprint’s work in time.
Additionally, as we implemented these changes, the average time we took to complete a work item dropped from 8.1 days to just 3.9 days, and we were able to increase our product load from one product to three products, tripling our monthly active users (MAU) without any change in the number of engineers on our team. These quantitative improvements aligned with our impression that, with the help of our improved process, we were working with greater efficiency.
My team’s practical work of recommitting to the principles behind our Agile practices speaks to a larger theme here at Spotify: finding the right level of alignment to help navigate the flexibility of autonomy. By increasing the structure in our team processes (through adoption of Scrum, in our case), we found enhanced clarity in our work, which allowed us to ensure we always felt aligned towards our shared goals. Ultimately, we finished our process upgrade with an increased sense of pride, direction, and responsibility for our success.
Acknowledgments
Many thanks are in order to our Agile coach, Matthieu Cornillon, for guiding us through every step of this process! And of course to my teammates: Isaac Ezer, Joshua Freeberg, Rishabh Jain, Linda Liu, Yani Metaxas, Nithya Muralidharan, Sabrina Siu, Jim Thomson, Hui Yuan, and Veronica Yurovsky.
Tags: engineering leadership
