Any Agile team needs quantitative feedback to steer their continuous improvement efforts. But how do you choose Agile metrics that work best for your project? Is it better to improve an existing metric or start from scratch?
One way to think about your metrics involves thinking about the feedback you need to move forward. First, the feedback needs to be timely so that actions can be taken regularly. Actions should also be balanced across the three “accountabilities” of Scrum, as described by Henrik Kniberg:*
Beyond the "accountabilities" of Scrum lies a fourth dimension that is critically important for high-performing teams – psychological safety. This dimension is often influenced heavily by leadership, including the team Scrum Master and Product Owner.
Avoiding Common Problems With Agile Metrics
As you start considering your Agile metrics, you want to plan ahead and think about how they might be used. In a common scenario, a team would like to improve, and there may be existing measures already, but there is disagreement on which measures are important. Data integrity is questioned, obstacles are artificially constructed that delay implementation, and people innately don’t want to be measured – measurements remove a degree of autonomy, and they can be debated as anti-self-organization.
Worse, Agile metrics for leadership can be “weaponized,” and they are no longer balanced across the four dimensions. This leads to imbalanced performance. For example, “speed” or velocity becomes the focus, and “building it right” or quality is subordinated. This example is often the single most difficult problem to objectively manage with waterfall fixed scope / fixed deadline programs.
The Solution: 4 Dimensions of Agile Metrics
Transparency, Inspection, and Adaptation are the pillars of Scrum, and they are key values for leadership to support, not weaponize. Iterative development and feedback across all the dimensions is important for most effective execution and improvement along the way.
Begin by collecting a few metrics where the data is already available every sprint. Encourage metric-related discussion in the Sprint Review and Retrospective. It’s natural that metrics evolve based on the discussions above; therefore, it’s better to start using metric information for continuous improvement than design the perfect measure from the beginning. Predictability may initially be % of stories done / stories planned, and it may evolve to % story points done / planned, then to sprint goals delivered / planned, and so on.
Continue until each of your four balanced metrics are updated and discussed every sprint:
Don’t wait for automation; implement these ASAP in your next sprint review. If you must automate, the recommendation is to wait for the metric to stabilize, and then automate. Be careful; averages often hide the information in outliers, metrics will continue to evolve, and automation will require maintenance. The balanced metrics are most valuable to create conversations on improvement. Same-team trends, inflection points, and outliers generate the most productive conversations.
Presenting Agile Metrics for Leadership
When Agile performance metrics are discussed based on a singular dimension, we encourage that the discussion includes balancing dimensions — a “balanced scorecard.”
A troubling pattern often manifests as a discussion on velocity. I can provide a real-life example. After a period of pressure to justify Agile, a Scrum Master calculates and shares 3 sprints of Velocity trends with leadership. The team velocity grows nearly 100% sprint over sprint and again sprint over sprint. The Agile Transformation Office discusses that this is not a balanced picture of the team performance. However, it’s too late; leadership starts talking in the hallway, reinforcing Velocity improvements as the measure of Agile success. The very next sprint Velocity was half, with other teams mostly stable. The result? Damage control with leadership and eventually a Scrum Master looking for a job.
A second anti-success pattern to be aware of is when conversation leads to comparison of teams. Every team is unique and has different impediments. Team-to-team comparisons are almost always unwarranted. Teams by their nature have different individual skill sets and often different technology stacks. If your leadership is compelled to compare teams, the best comparison is to have them work from the same Product Backlog — this implies the same Product Owner. A productivity measure would compare the teams based on velocity (points) per sprint / normalized for each team by cost per point. A continuous improvement measure would examine % improvement over time.
A better solution is to align all teams and leadership on each other’s objectives and key results (OKRs, which are typically a quarter measure) and bonus based on overall team-of-teams success, not on individual performance and not on functional performance. This approach, coupled with the concept of balanced metrics, can help keep your project on track.
Want to dig deeper? Learn more about the roles of flow, value, and quality in Agile metrics, and then start thinking about value- and outcome-based Agile metrics.
Other Sources
- Kniberg, Henrik. "Agile Product Ownership in a Nutshell." YouTube, https://www.youtube.com/watch?v=502ILHjX9EE. Accessed 6 July 2021.
- https://rework.withgoogle.com/print/guides/5721312655835136/ Accessed 6 July 2021.
