Introduction
Managers need the ability to see how far along projects are within their organizations. Managers use this information to make strategic decisions about their companies. However, many managers forget (ignore?) the fundamental difference between Agile organizations and non-Agile organizations—work is about outcomes, not output. In this article I will address how measuring projects based on the percent of user stories completed is a terrible idea, and I will argue why this is bad for your sanity, and the sanity of your Agile teams.
I understand, but you’re doing it wrong
There are two basic questions any manager wants to know about the state of in-flight projects: a) Are we almost done?, and b) When will we be done? If you’ve ever remodeled your house and had to live through the cacophony and disruption that is home renovation, then you understand the need for these questions. The questions are not bad questions. They are legitimate questions to ask. The way we (i.e. Agile practitioners) answer these questions is contra to the ways managers are taught to think. That’s when conflict comes in.
In Agile we strive to measure success in outcomes rather than in output. User stories represent small bite-sized pieces of work. If a project has a given number of user stories in its backlog, then people infer that when all of the user stories are completed, the project will be completed. So it is understandable why people would look to the percentage of user stories completed by which to gauge project duration.
An example
Let’s say a project starts with 20 users stories. The team finishes its first iteration and completes 5 user stories. There are 15 user stories left in the project. Based on these numbers, the team completed 25% of the project work. The team, or likely the project manager, happily reports that 25% of the project is done. Management is happy.
The team prepares for its second iteration and realizes some stories are too big. They break down some of the stories, adding 10 more user stories to the backlog. The team didn’t add scope, they merely added additional stories of smaller size. The project backlog now sits at 25 user stories remaining. The team finishes their second iteration, completing 5 more stories, for a total of 10 stories completed to date. The project backlog is back to 20 user stories remaining with 10 users stories completed, there is now a total of 30 user stories for the project. Remember, scope has not changed! If a manager knows the team completed another 5 users stories, for a total of 10 user stories, the manager will assume the project is 50% completed (based on the original 20 total user stories). Imagine the manager’s surprise when he or she perceives the project as only being 33% complete (10 completed stories / 30 total user stories = .3333 or 33%)!
Fallout
The above is a very simple example of what happens all too frequently in organizations across the Agile spectrum. Most projects, particularly enterprise-wide projects, consist of a lot more than 20 user stories at their onset. It is not uncommon for a single project to have hundreds, maybe even thousands, of user stories over the course of its lifetime. Calculating the numbers becomes much more difficult to do on the fly when there are so many user stories in play. However, management still wants the calculations, and the large number of user stories causes the percentage complete to fluctuate—seemingly out of control. Most non-Agile managers interpret the conflicting percentages as an out of control project. Bad things can happen when managers think a project is out of control.
Management may assume something is wrong with the team. From the manager’s perspective, the team is not performing up to expectations.
Management assumes something is wrong with Agile. After all, “We already defined the user stories before we started. Why is it they are just learning about these things now?”
Given the expectation that managers deliver their projects on time, the natural reaction for most managers is to clamp down on their Agile team in the hopes that micromanaging the team will correct the problem. Managers will often introduce more meetings (as if talking about things produces value for customers), more status reports, or demand people work longer shifts to make up for the project being behind schedule (allegedly). These are not reactions to reality, they are reactions to a perceived problem. However, there isn’t necessarily a problem. This is Agile working as Agile should.
It is possible the project is behind schedule, but you can’t assess the schedule by simply looking at the number of user stories.
In Agile, we base our success on value produced, not busy-ness of teams. It is possible the 10 user stories completed in the above example represent 75% of customer value. Managers should be concerned about value produced, not the number of user stories completed. When you look solely at the number of user stories remaining versus user stories completed, you are focused on the wrong thing.
Conclusion: Learn to calculate value
We have to keep in mind that managers need information to ensure their organizations are moving in the right direction. It is unrealistic to tell managers they cannot have information to run their companies. Managers of Agile teams need to disconnect themselves from measuring busy-ness and embrace measuring value produced.
If a team produces one million dollars in value over six months, do you care if that team completed 10 user stories or 50? If you said you care about the value produced over the number of user stories, move to the head of the class. Which would you rather put on your resume?: a) Led an Agile team that completed 1,000 user stories; or b) Led an Agile team that produced $10,000,000 in customer value.
Focus on customer value and stop trying to measure projects by the percent of user stories completed. But by all means, continue to focus on user stories if you want to drive you, and your employees, crazy.
Ramirez, O. M. (2021). Stop using user stories to calculate project % complete. omramirez.com