How we do large scale retrospectives

Foreword: This post was initiated by Andy Park, former agile coach here at Spotify. For years we’ve been experimenting with how to do “big retrospectives”. That is, how to capture and spread learnings from big complex multi-site efforts involving dozens of teams. We used to do the traditional “get everyone into one big room” version, but now with hundreds of people spread across different continents that’s a logistical nightmare. After some experimentation we’ve converged on a decentralized approach that scales much better. Andy has helped facilitate several of these retrospectives, and thankfully took the time to write an experience report about it. Here it is, with only minor additions from me.
Enjoy! /Henrik

At Spotify, Kaizen is hugely valued and retros are one of our goto tools to help us reflect and continuously improve.  Not just as part of squad level activities but also as a company level activity. Which sounds awesome in writing.  In practice what we found is that when you grow like crazy, the size of company initiatives could scale just as crazy and retros for programs becomes larger and larger running into a diminishing return problem.

Does this look familiar to some of your large program retros?

In late 2014 we had an opportunity to run a program level retro for an innovation initiative that spanned across 80-90 people in NYC and Stockholm.  Instead of a large session, we opted to try something bit different – and we found it to work better for these types of initiatives.

It worked well enough for us that we refined it further to apply it to our latest programs Moments and Running and Sony PlayStation™ Music which involved hundreds of people spread globally across many departments ranging from TPD (tech, product, design) to legal, business, licensing, partners, and much more.

In order for us to learn more effectively from programs of such size, here’s what we are doing for retro at scale, designed by Henrik Kniberg, and refined through couple executions!

How we do decentralized big retros

In summary:

How do we do this really

First we let out a deep sigh and deploy the fiery agile evangelists to wage a holy war to forcibly convert management over and secure company commitment.

Thankfully that isn’t the case at Spotify, so Step 1 begins with emailing agile coaches to seek volunteers to form a small working group of facilitators. Once the group is formed, most of the activities revolve around setting expectations, creating an objective for the retro, communicating to the company and participants about the retro as heads-up. We also craft a high level retro plan, a google spreadsheet that looks something like this:

As part of this we identify a handfull of “themes” that can be investigated independently (that’s the scaling trick!). For example “program management”, “new content”, “architecture”, etc. The choice of themes varies a lot dependending on context, so that’s one of the key decisions for the working group.

When the plan is good enough to start, the working group transitions into Step 2 – doing the actual retrospectives. We don’t need everyone involved in every conversation, instead we organize a number of smaller conversations. The facilitators spread out, pick a theme, gather a cross-section of people that have insights into that theme, and run a retrospective. Smaller groups make for better conversations, and also easier logistics (easier to find a time that works, a room that fits people, etc).

As for the specific format, we used to leave that up to each facilitator. Lately, though, we’ve started converging on a “standard” format, which gives the faciliator a starting point and also makes for easier merging later:

• Introduction (purpose of this retro, introduce each other)
• Timeline (draw a timeline together, annotate with milestones, what worked well, what didn’t, and any “elephants”)’
• Cluster (find patterns and label them)
• Recommendations (distill into concrete insights and recommendations, both for the how we work now, as well as how we might work in the future)
• Closing (round-table final thoughts from people, and quick feedback about the retro itself)

Pretty standard retro format, so I’ll spare you the details.

The working group meets up from time to time, using the spreadsheet above as a sync and status tool. They support each other and address problems such as lack of facilitators.

As individual retros are wrapping up (after a few weeks typically), the working group gradually transitions into Step 3 – discussing and merging the collected data. The goal is to find patterns and condence it into a short and coherent summary with key insights and recommendations.

When ready, the working group enters the last phase Step 4 – spreading the word. That includes things like sharing the summary with stakeholders and retro participants; meeting in small groups to share learnings; speaking during town hall sessions; and so forth to really help the organization internalize and act and upon the learnings.

Benefits of this format

What we learned with this distributed retro model at scale is that it enables different groups to focus on different topics deeply to generate cross functional and impactful learnings across the company.

On the human level it provides a great venue and increased engagement for folks to have their say, get their voice heard, and just as importantly a neutral safe place to air dirty laundry. This helps strengthen the Spotify DNA and the transparent learning organization we’re trying to build.

Retro the retro

We learn as we go and continuously try to improve the way we do cross-functional retrospectives.  Here are some plain yet sometimes overlooked points that have helped us do these more effectively since our initial attempts back in 2012.

• Have a “Road manager” for the retro to help organize and drive the effort and schedule various regular check-ins with retro working group, stakeholders, etc.
• Ask the The Spice girl question (“What do you want, really really want”) when carving out objectives of the retro.
• Be explicit about expectations and priority. Facilitators are volunteers, and when signing up they need to understand the level of commitment and work involved.
• If you want to move quickly, communicate the priority and importance of the retro to the company.  That way the participants understand what’s expected of them and will make the time to attend. Without that, don’t be surprised if retros get moved around to find time rather than folks making time, which add up to individual retros for themes taking longer and overall retro effort taking longer.  Ultimately diluting the learning as time passes.

On the whole, this format is the best we’ve found so far for large scale retros. We’ll keep experimenting though, so stay turned.

Feedback

We hope this gives you some ideas for how you might retrospectives, regardless of your context and scale!  We would love to hear your thoughts and maybe you have done something similar.  Please share your thoughts and comments below if you’d like.

– Andy & Henrik

Tags: agile, retrospective