Story Points are Lame?

Ah yes, my second favorite topic when it comes to all things agile — story points. My favorite agile topic? Agile project management tools. But that topic is easy to sum up in one sentence: All the agile project management tools I’ve ever used or evaluated tend to suck in their own unique ways. Maybe one day I’ll elaborate on that enlightening thought, but right now I want to focus on story points.

ScoreboardStory points are one way agile software development teams can estimate the size of user stories. For example, the product detail page for an ecommerce app is estimated to be 8 story points, while the report showing overall ecommerce sales vs. in-store sales is estimated to be 16 story points. We’re not equating story points to how long it will take. Take an experienced, cross-functional agile ecommerce software development team. They say the product detail page is an 8, but the newly formed team consisting mostly of COBOL and FORTRAN software developers estimates the product details page is a 128. Who is right? They both are.

This leads me to an email posted by “macdonald1976” yesterday to the Scrum Development Yahoo! Group with the subject line of “Why points are lame…”

First of all, anything you can make up your own definition for is lame and suspect to say the least. My CTO doesn’t care to hear that my dev team has 150 story points left. He wants to know how many hours they have left. Hours are not subject to opinion. There are 60 minutes in a hour and 24 hours in a day. My CTO can relate to this. My CTO cannot relate to some story point with weights assigned arbitrarily assigned. Sure, some of you will argue that it is not arbitrary and you are entitled to your opinion just like you are entitled to your opinion as to what constitutes a point. This opinion on a unit of measure and the unscientific approach to defining it, no matter how scientific you contend it is, is the reason it has no value in defining remaining effort. Points = opinion and like the old saying goes, ‘opinions are like butt holes, everyone has one’.

Oh no he didn’t. Sending an email like this to the Scrum Development list is like crashing a Star Wars party wearing a Spock costume and talking smack about Yoda in Klingon. I’m happy old macdonald1976 wrote this email because it reflects the opinion of many who don’t quite grasp the concept behind story points. Even better though, it caused Andreas Axelsson to give this spot on response (with my comments in between paragraphs):

Points are intended to give a relative measure of size for stories, not a valid estimate of duration. Mike Cohn (author of Agile Estimating and Planning) says: “Estimate Size; Derive Duration”. The rationale is that it’s much more likely that a team can agree on the relative amount of work for a story compare to another, than on how many hours it’ll take to implement it.

Straightforward definition that gets right to the point.

Compare with two people running a track; The first person will insist it’s a ten minute track, the other a five minute one. Who’s right? They are much more likely to agree that it’s a one mile track (or whatever one can run in five minutes, I honestly have no clue), and that a two mile track is twice as long. Any two people running the track are likely to get a different time, but it’s still the same size.

The most popular explanation for story points that I know of. It’s popular for a reason. It makes sense.

So, estimate in points, validate how many points you can complete in an iteration, using an average over a number of iterations, or historical numbers if you’ve got them, and use that to derive the total project duration. I.e. our velocity of the last three iterations was 27, then the remaining 189 points could be finished in 7 iterations. You do need to add a range though. Or to quote “It’s better to be roughly right, than precisely wrong”. If you’ve got an experienced team on a known domain, you may want to adjust the estimate -10/+5%, if it’s a fresh team learning a new domain the figure might be -50/+25%. (Values from Mike’s book) Using the first figure that gives us a velocity of between 24-28 meaning our 189 points could be done in between 7 and 8 iterations. The inexperienced newbie team would get an initial estimate of between 6 and 14 iterations. (It’s probably not likely that an inexperienced team would average the same initial velocity as the experienced team but let’s ignore that for this exercise, the issue here is the range) There’s really no way to know very early in a project. Experience will tighten the range as the project moves along. So your estimate may narrow from “1st quarter” to “February” to “Last week of February”, all the time giving the Product Owner an accurate estimate just not a precise one.

Practical and concise advice on how story points should be used within a project.

Something you shouldn’t do, however, is to try to transform the hours of your completed iterations back onto the remaining stories. Estimates are pretty much bound to follow the normal distribution and thus you’ll end up with “easy” fives and “hard” fives. Imagine the disappointment if you end up spending 76 hours on an “easy” five from your first iteration and then apply it to every five point story in the backlog. It just won’t work and you’re back at trying to estimate duration directly.

Man, Andreas is hitting a key issue here. Equating story points back to actual hours is particularly tempting for teams to do. Do not do it!

Also, there’s no promise that ten points for one team can be related to ten points for another. It’s just a number, that makes sense to that particular team, and the CEO should not care about its meaning in relation to duration or any other unit. It’s a unit-less relative measure of size for a given team. Period.

So, you give your boss the date range, derived above, not the points themselves. The points are to be used by the PO to prioritize stories along with estimated ROI and any other factors that may contribute.

And the final rebuttal to macdonald1976’s original statement that story points are just opinions puts the executive’s focus where it should be: on the value being delivered. The CXO doesn’t care about points, nor should she care about hours. The CXO should care about what value is being delivered and when the rest of the value is likely to be completed by. We fall back on bean counting hours when we take our eyes off what matters most in the project. We can still do budgets with story points. We can still estimate when our project will likely be completed. We can still know how our teams are performing. We can still determine if the project is on track to deliver the goods. The bonus is that we don’t get caught up in spreadsheet hell, calculating every hour down to the penny. Instead, we can focus on making sure our businesses are delighting customers.

I absolutely recommend Mike’s book if you’re interested in a more in-depth explanation of the above thoughts.

Cheers,

Andreas Axelsson

Cheers is right Andreas. Cheers indeed. A full round of applause goes out to you for that excellent reply! It made my day.

  • Brian R. Robinson

    Two questions:

    1. How do you know the total number of story points to begin with? I thought that Agile teams generally only estimate stories for the next few sprints or so, thus you’d never have a number like 189, you’d have a number like 81 (27 velocity times 3 sprints). Without a total number of story points, you can’t even answer the question “How long until the work is done?”

    2. Suppose you have a prospective customer who wants a solution 8 months from now. Let’s say the solution needed is a hospital patient management system, and your team is only very well versed in financial applications. How would you be able to determine whether or not your team can deliver for that prospective customer?

  • Joshua Hoover

    Bria, good questions. :) I’ll do my best to answer them.

    1. Typically, you’re product backlog should have solid user stories for at least 2-3 iterations/sprints out. From there, you can use “epics” which are typically huge user stories that will eventually need to be broken down into smaller user stories. Epics can be estimated on just like stories, except you need to acknowledge that these estimates are subject to change as you get closer to taking on the work. Let’s also remember that by the time you reach these “epics” they may change or go away completely due to the customer’s change in priorities.

    2. I’d say you’d need to evaluate the high level of risk in the hypothetical situation you presented. You’re likely going to need to supplement the team with subject matter experts at the very least. Of course, you’ve also touched on another tricky subject with this hypothetical and that is fixed bid projects. There is a great post by Udi Dahan that addresses this issue.

  • Ryan McKeon

    Great post Josh! I’ll throw in my $0.01.

    1. You hit it on the nose. Epics compose the remainder of a backlog, and will be estimated quite large. However to plan accurately, the epics should be broken down enough so as to gauge to some degree of accuracy how much work they will take. A 1000 point epic is not very helpful for planning purposes, but sets of 32 point epics can fill out a schedule for years to come.

    2. Again you are exactly right, this is coming to the questions of fixed bid projects. This was one of the more interesting parts that Ken and Mike described a solution to in Certified Product Owner training.

    To determine work that can be completed in a fixed price/fixed scope contract, you take your historical velocities and a broken out, prioritized backlog for the upcoming work. Using the lowest historical velocities, determine the amount of work that fits in given the team’s members and time period for the contract. This is the amount of work that can almost certainly be done in the contract. Then use the teams highest historical velocities, and find out how much more of the work will fit. This is the work that might be able to be completed. Any additional items on the backlog will almost certainly not be able to be completed. Somewhere in between the minimum amount and the maximum amount will be the amount of work expected to be completed. Gauge how much work you will bid on based on how much risk you are willing to take on the contract.