How to Use the Improvement Rating to Make the Most of Your Retrospectives

For Agile teams on the road to ultimate high performance, retrospectives represent the team meeting with the most potential to smooth out kinks in the process.

But, how do teams know whether they’re truly tapping into retros in the best way possible?

If you’re hosting retros in ScatterSpoke -- you’re in luck, because determining what makes the difference between a retro for the sake of one and a productive retro is our not-so-secret sauce.

Being the data-driven bunch that we are, we’ve developed a surefire metric to quantify how well you retro, so you and your team can continually improve how you retro together over time.

That’s right. We created a special rating system with 3 key indicators to help teams track the process through which they reflect as a group and the outcomes of their retros with confidence.

What is the Improvement Rating?

We believe that you can never truly manage what you can’t measure. We reasoned that if we want ScatterSpokers everywhere to make the most of their retrospectives, on the team level as well as the organizational level, we’re going to need to quantify great retros vs. unproductive ones on a numerical scale.

That’s how the one-of-a-kind Improvement Rating metric inside the tool was born.

The Improvement Rating on your dashboard tracks three key criteria over time to show anyone on your team how well your team is leveraging retrospectives at any given moment in time.

What Indicators Does It Account For?

This rating out of 10 points is averaged using data from the following indicators:

1) Feedback Stat

When you’ve been a part of as many retros as we have been, you begin to see some patterns emerge. Our team has observed two primary deterrents to making retrospective meetings work their hardest for the team.

The first, and most dangerous, pitfall of team retros is collecting and implementing ideas from only a subset of extroverts on the team, while great ideas from introverts don’t get a platform.

The second common trip-up during retros is that team leads reflect on the process in isolation. With the best intentions, team leads leave the team out of the retro format (to be respectful of their calendars) and generate ideas for process improvements in a silo by themselves.

To help teams retroing in ScatterSpoke avoid these common pitfalls, we made balanced feedback a factor by rewarding proactive ideation from all members of the team. It contributes to the overall improvement rating of the group as it does IRL.

Your feedback score is based on each and every team member adding at least one card to every column on the retro board. Each retro is given a score and the average of those scores is reflected in this component of the scorecard.

2) Frequency Stat

Teams that are new to Agile ways of working opt to run more frequent retrospectives in their working groups in order to work on developing the habits required to reach high performance. Agile newbies schedule retros at least once every two weeks and some as frequently as once every week, depending on the pace of their work process.

As teams mature, they often move to a more dispersed schedule of monthly retrospective meetings. The optimal cadence for retrospectives is monthly. Anything rarer than that and we are beginning to risk limiting our opportunities to improve the process and exacting positive changes too late.

Your frequency score is based on having a minimum of one retro created every 30 days. Each month, starting from your first month using this invaluable practice, is allotted a score.

Each month’s score contributes to an average that generates your frequency stat up until this point, from first retro to most recent retro.

3) Follow-Through Stat

Running retros as a way to generate ideas for process improvement is great. We can’t rave about it enough! But, simply generating good ideas will only get us half of the way to success. We actually have to make better processes happen.

The action items generated during a team retro should be scoped, sized, assigned and prioritized alongside other strategic tasks in the backlog. In this way, we encourage timely follow-through on those crucial eureka moments post-retro. Without follow-through, our plans for process improvement will just live on as great ideas that never came to fruition.

Your follow-through score is based on the number of days it takes to close an individual action item. Each action item is given a score and the average of those scores is reflected here.

How to Get that Perfect 10 out of 10

For high-performing, mature Agile teams, getting the highly coveted perfect 10 on their improvement rating is a breeze. Chances are, they are already in a healthy rhythm of hosting regular retros, collecting feedback from all areas of the team and following through on their action items.

However, new Agile teams have to work a little bit harder to build the series of habits that allows process retrospectives to really make a difference in their working groups.

The good news is that hosting retros at the regular monthly cadence will get you a third of the way to optimal process improvement. 30% of the credit in the Improvement Rating inside ScatterSpoke is awarded to the team just for showing up.

Gathering diverse feedback from your quieter team members might present itself as the greater challenge. To encourage introverted team members to speak up and share their ideas during the time carved out for retros is an important component of best results. Get all your team members contributing by:

• Hosting anonymous retros that don’t require individuals to link their names to their ideas. It’s more important to get the good ideas on the table for the team to discuss, not which team members came up with them.
• Experiment with diverse, creative retro formats that shift the team away from the traditional Keep Doing, Stop Doing, Start Doing structure. It’s likely that some team members just ideate better on a canvas, drawing their ideas as they go.
• Encourage team members to add their ideas as they come to them. Many individuals get their lightbulb moments during the work process and second guess themselves or outright forget their suggestions by the time the next retro rolls around.

Busy teams working towards hard deadlines also often have a problem with following through on team process improvements raised during retros. In order to encourage follow-through from the team:

• Integrate action items from your retro into your team’s common backlog to avoid having them fall off the map when the meeting is over.
• Prioritize process improvements against BAU and strategic tasks in the team to-do list to position these action items with existing priorities for the team.
• As much as possible, carve out team capacity to execute on process improvement. Changes to the process won’t be implemented if the team members feel like they can’t step away from other tasks in the workflow even for a moment.
• Hold team members accountable for action items from the retro. At the end of the meeting, ask for volunteers to take ownership of some of the ideas that were particularly compelling. Assign team members to 1-2 action items from retro and ask them to give a quick update on their owned tasks at the start of your next retrospective.

Improve Your Retros in 2021

We’ve heard it said that the moment the retrospective meeting becomes skippable is the moment a team’s quest for success begins to stagnate.

This coming year, make sure you are consistently making the most of the valuable time you’ve set aside to reflect with your team. Leverage all of the resources at your disposal to keep your team proactive and engaged during these problem-solving sessions.

Tracking your Improvement Rating in ScatterSpoke can go a long way to maintaining a data-driven, quantitative approach to how well you’re using your time together to improve how you deliver value.

By tracking your IR, you’ll actually be keeping an eye on the time it takes for you to achieve resolutions as a team, your participation and how many opportunities for reflection you’re creating in your busy schedules.

Coupled with querying your team on their attitudes towards the retro format in place, timings and facilitation, you’ll be armed with both quantitative and qualitative data to make adjustments to the way you improve with your team.

Keep improving HOW you’re improving with your team using this invaluable retro metric.

‍