Its been a while since i've posted, and that has been due to partly due to the pressures of work, but but also because i didnt think i had anything significant to share. That is until this week when i had a moment of WTF!!.
I want to share with you something that is seemingly innocuous but actually very toxic; namley using velocity as a performance metric!
I have never been a big fan of metrics for metrics sake, because a lot of them a pretty meaningless. We have a focus, and that is the delivery of software. All the business wants to know is that what they have asked for is to be delievered, on time and hopefully to budget. Why bother producing Burn down or burn up charts if we dont factor in defects or Tech tasks and besides what is the point of any of it if one story can block 20 others?
The old software delivery problem still exists right?
For me metrics fall into two piles:
1. Diagnostics Metrics: These are informative indicators chosen by the team that are used to evaluate and improve the teams process. We collect diagnostic metrics to give us some insight into where we can improve, and to track whether the improvements have any effect. Simply informative, never to acertain how much value is being delieverd and of a fairly short life; if the process improves, we move on and identify other sub-optimal areas. The measurmnent is dropped a that point, its served its purpose.
2. Performance Metrics: This the measurement that details how much value is being delievered. These are used to track our performance, and are chosen carefully. There should be fewer rather than more of these metrics. Wherever possible these metrics should be outside of the teams control, that way the numbers can not be perversed. E.g User or customer feedback. "how satisfied are you with the service/product" or "how likely are you to recommend the service offered by the team to another?".
The problem i have at the moment is that our velocity is being used as a performance metric rather than a diagnostic aid. Incredible i know, but true. Why? Because velocity cant be used as a performance measure, because it is unique to the team being measured at that point in time and only within that context. It changes, like the ebb and flow of a tide.
Not only is someone trying to use velocity as an indication of the teams performance, i have also heard the following remarks made.
Velocity is little more than a health-check that help inform us about our team’s capability. It can not tell us about how much value is being delivered or for that matter how fast its being delivered. We can deliver a lot of story points but only through making a trade-offs with our quality which, however we measure it, will impact our ability to deliver more. As uncle Bob Martin says:
I want to share with you something that is seemingly innocuous but actually very toxic; namley using velocity as a performance metric!
I have never been a big fan of metrics for metrics sake, because a lot of them a pretty meaningless. We have a focus, and that is the delivery of software. All the business wants to know is that what they have asked for is to be delievered, on time and hopefully to budget. Why bother producing Burn down or burn up charts if we dont factor in defects or Tech tasks and besides what is the point of any of it if one story can block 20 others?
The old software delivery problem still exists right?
Project Manager: How much have you got left to test?
Tester: Just one story out of the 40 we had.
Project Manager: Cool, the testing guys are nearly done.
- One week later....
Project Manager: How much have you got left to test?
Tester: Just that one story, but boy-oh-boy its a doozy.
For me metrics fall into two piles:
1. Diagnostics Metrics: These are informative indicators chosen by the team that are used to evaluate and improve the teams process. We collect diagnostic metrics to give us some insight into where we can improve, and to track whether the improvements have any effect. Simply informative, never to acertain how much value is being delieverd and of a fairly short life; if the process improves, we move on and identify other sub-optimal areas. The measurmnent is dropped a that point, its served its purpose.
2. Performance Metrics: This the measurement that details how much value is being delievered. These are used to track our performance, and are chosen carefully. There should be fewer rather than more of these metrics. Wherever possible these metrics should be outside of the teams control, that way the numbers can not be perversed. E.g User or customer feedback. "how satisfied are you with the service/product" or "how likely are you to recommend the service offered by the team to another?".
The problem i have at the moment is that our velocity is being used as a performance metric rather than a diagnostic aid. Incredible i know, but true. Why? Because velocity cant be used as a performance measure, because it is unique to the team being measured at that point in time and only within that context. It changes, like the ebb and flow of a tide.
Not only is someone trying to use velocity as an indication of the teams performance, i have also heard the following remarks made.
- Why is Team A slower than Team B? Wow, where do i start. Because they estimate in different scales, or maybe their iteration length is different, possibly the team composition is different? There are so many factors that can influence velocity that it’s only useful to compare it within the same team, and even then just to identify trends. The absolute value doesn’t mean anything.
- We only did 28 last iteration, can we push for 38, and then 58? The beginning of a project can have a naturaly low velocity, and this should be expected, however it should begin to increase after a number of iterations. However we have been asked to push and push to improve velocity when we reach a natural plateau, and we have pushed, after all, we want to go faster, because we rock, right? Because velocity indicates the capability of our teams to deliver it will settle down itself (because our process is stable, and we haven't employed any creative accounting). One my heroes is Deming who noted that in a stable process, the way to improve is to change the process.
Velocity is little more than a health-check that help inform us about our team’s capability. It can not tell us about how much value is being delivered or for that matter how fast its being delivered. We can deliver a lot of story points but only through making a trade-offs with our quality which, however we measure it, will impact our ability to deliver more. As uncle Bob Martin says:
“The way to go fast, is to go well"Never use velocity to measure performance. After all the thing being tracked is a gumi bear, or pink elephant. Simply look at velocity as a diagnostic metric to help us improve our software delivery process.
No comments:
Post a Comment