Agile and Estimates and Contracts! Oh My!
Over the last six months, I’ve been mulling over one of those subjects that has often been a stumbling block for IT providers considering adoption of agile for their customers’ software development projects. The dilemma revolves around contracts and estimating projects. The two tend to go together. Once you have one figured out it seems you’re left with even more questions with the one remaining. I have some ideas on how to make agile contracts and estimating better based on what I’ve learned and observed over the years.
WARNING: This is a long post. I try not to write long posts. I think this one is worth it, mainly because it’s building off the ideas of those much smarter than me.
Fixed Price Contracts
Before I get into how I think you can make agile contracts work, let’s look at one of the most popular types of IT contracts out there – fixed price. Fixed price contracts are (at least in this context) those that lock in scope, schedule, and cost. From the customer’s perspective, it seems like fixed price contracts are the only way to go. IT providers essentially tell customers, in a legally binding document, that the customer’s project is sure to succeed. They’ll get what they want, when they want it, and for a predetermined price. Unfortunately, success is far from guaranteed.
I was once told by an ERP software salesman that contracts are like fences between neighbors — strong fences make good neighbors. I’m afraid that fixed price contracts on software development projects look more like medieval moats than strong, yet inviting picket fences. One neighbor, the IT provider, feels pressure to meet the contractual obligations, even if they’ve gotten in way over their heads. The other neighbor, the customer, at some point in the project feels the pressure to get what they actually need versus what they thought they needed initially, and that often means requesting changes to a project locked in on cost, schedule, and scope. Changes on a fixed price contract leads to the path of strict change management on the part of the IT provider, who is going to be weary of any changes; considering the huge amount of risk they’ve taken on by guaranteeing they can provide what the customer originally asked for on time and on budget.
Scott Ambler wrote an excellent article for Dr. Dobb’s on this topic of fixed price contracts in July this year titled, Is Fixed-price Software Development Unethical? Those who aren’t familiar with Ambler’s work might quickly write him off as just another agile zealot spewing the ideals of agile in an imaginary world where lions lay down with lambs and large financial institutions understand their own books. Not so fast. Ambler works for one of the biggest and oldest of IT vendors, IBM, which is pretty far from the land of agile zealots. Scott has not hesitated taking agile evangelists to task for their one line answers to tough questions that need more than just bumper sticker slogans. In his Dr. Dobb’s article, Ambler argues that we in the IT field need to educate customers on the hidden dangers of fixed price contracts and steer them towards better alternatives that share risk more realistically and give the customer a better chance at accomplishing the goals they set out to accomplish with their project.
Fixed price contracts lead to big requirements, architecture, and design up front (BRADUF – a new acronym I just made up because we need more acronyms). BRADUF is waste. The need for doing so much up front analysis work on a fixed price contract leads us to the topic of estimating. As Ambler points out in his Dr. Dobb’s article:
Up-front estimation motivates Big Requirements Up Front (BRUF), which in turn results in significant waste. To reduce the risk of uncertainty in your estimates, you need to increase the level of information that goes into those estimates. This motivates you to do detailed modeling of your requirements and potentially even your architecture. This unfortunately ignores the facts that people aren’t very good at defining up front what they want and even if they were, the requirements will evolve anyway. The implication is that the sizing measures that are input into your estimate are also on shifting ground. The “Chaos Report” shows that projects that took a BRUF approach ended up delivering systems (when they delivered anything at all) where 45 percent of the functionality was never used by stakeholders.
Estimating on fixed price contracts instinctively leads to the path of waterfall project management. Going down the path of locking everything in up front and spending a lot of time trying to make sure that estimates are as accurate as possible right at the start sets up a perfect path for using processes that are well defined, sequential, and can be carried out by groups focused on a particular process or phase of the project. Once so much work is completed up front it’s almost pointless to use a more adaptable approach like agile moving forward. Everyone has their marching orders and may as well march towards the goal. This domino effect is one that I’m not sure a lot of agile supporters see when they support fixed price contracts or the in-depth estimating methods used to support fixed price projects.
A Possible Alternative to Fixed Price for Agile Projects
Where does this leave an IT provider desiring to make the leap to agile? In a tough spot, I’m afraid. Not a hopeless position by any stretch, but still tough. First, let’s review one of the key values in the Agile Manifesto:
Customer collaboration over contract negotiation
Remember the last statement in the manifesto, “That is, while there is value in the items on the right, we value the items on the left more.” Contracts are important, just not as important as collaborating with customers. Fixed price contracts try to make up for a lack of collaboration between the IT provider and the customer, not to mention the fact that fixed price contracts try to put trust in place between the two parties. (The irony is that fixed price contracts have a tendency to erode trust on software development projects.) The fixed price contract with its locked scope, schedule, and cost gets the boot as a contract option for the IT provider considering or doing agile. Great, I’ve taken this long to get to a conclusion that many have made before in the agile world. I’m not done.
Bob Martin of Object Mentor posted an interesting variant to get around this problem: a base fee per story point, plus a lower-than-usual (close-to or below cost) fee per hour. This biases the contracted company’s to deliver early, but gives them some protection in case work proceeds slower than expected. Bob Martin described it this way:
”[A]gree to pay a certain amount for each point completed, plus a certain amount for each hour worked. For example, let’s say you’ve got a project of 1000 points. Let’s also say that a team of four has established an estimated velocity of 50 points per week. This looks like about an 80 man-week job. At $100/hour this would be a $320,000 job. So lets reduce the hourly rate to $30/hour, and ask the customer for $224 per point.
This sets up a very interesting dynamic. If the job really does take 80 man-weeks, then it will cost the same. If it takes 100 man-weeks then it will cost $344,000. If it takes 70 man-weeks it will cost $308,000. Notice that this is a small difference for a significant amount of time. Notice also that you, as developer feel strong motivation to be done early, since that increases your true hourly rate.”
I like this idea for a number of reasons:
- It satisfies the customer’s need for a decent number/range to budget ($$$) for
- There is a built-in incentive for the IT provider to complete the work in less time – higher margins ($$$)
- The pain (in $$$) of going over on the schedule is shared between the customer and the IT provider
- Estimating can still be relatively lightweight, reducing waste found in big up front estimating techniques – saves the customer and IT provider ($$$)
- The focus on the team(s), velocity, and high level (yet not too deep) planning is a good starting point for an agile project versus alternatives that focus on just about everything but those items
The dilemmas and questions this idea leaves us with that I can see right away are:
- We still have to estimate! How do we estimate with so many unknowns?
- How do we figure out how many people we need and what our teams are going to be for the project?
- How will we know the teams’ velocities if they’ve never worked together before?
- What happens when the customer adds new work and changes priorities during the project?
- How do we make all this a repeatable process?
My feeble attempt to answer each of these questions follows.
We still have to estimate! How do we estimate with so many unknowns?
Mike Cohn explains this in his excellent book, Agile Estimating and Planning. Chapter 16, Estimating Velocity – Make a Forecast, is especially relevant when it comes to estimating for contract work. I won’t detail everything here, but at a high level we need to:
- Have a product backlog defined well enough to cover the project at least at the “epic” level (very large user stories), then break the highest priority epics down to user stories until you have enough to estimate a few iterations (which you’ll know how much that is in the steps that follow).
- Estimate all the items in the product backlog, preferably with at least one person who is likely to work on the project as a team member (not just a product owner or scrum master/agile project manager).
- Figure out the available hours per iteration for that team (deducting a certain percentage for overhead, which can vary greatly from company to company).
- Take a sample set of user stories for the project (some small, medium, and large) and do task break down on these, assigning hours to the tasks.
- Stop doing step 4 once we hit or get close to the number of hours calculated in step 1.
- Add up the story points for all the stories that seemed to fit in one iteration and now apply a range to that velocity (e.g. multiply by 60% and 160%) to analyze how that will affect the project in terms of schedule and cost.
- Velocity for the team can be determined based on the velocity range analysis from step 6 and can be applied to figuring out how many teams will be needed to best meet the customer’s goals for the project, in addition to putting together a draft of a release plan.
How do we figure out how many people we need and what our teams are going to be for the project?
In my mind, this question is the toughest to answer. Bob Martin’s idea uses an example where you have an existing team, you know their velocity, and we assume this team and their velocity is still relevant in terms of the potential project. For smaller IT providers (“boutique software development shops”), Martin’s example is probably realistic, but for larger IT providers that take on larger projects that demand multiple teams, and have more people to manage and keep on billable projects, it’s not as realistic. As it is with most things, I think solving this problem comes down to people. We need people who have enough experience and expertise on the project proposal/bid that can estimate the project with a certain amount of confidence. I think on bigger projects where we know we’ll need more than one team (but aren’t sure how many more than one), then we should assemble one team on paper and then move onto estimating things based on that one team. (Of course, we should have at least one person involved in the estimating who will be doing the work.) The estimates, deadlines, risks, and other factors will drive out how many people/teams are needed and what the make up of the teams need to be in terms of skill sets and levels of experience.
How will we know the teams’ velocities if they’ve never worked together before?
We won’t, but the method for estimating outlined above will get you close enough. So many factors go into high performing teams that it is foolish to think that following some simple steps on paper will get us to a place where we can predict (for example) whether a team will perform at the level we need them to.
What happens when the customer adds new work and changes priorities during the project?
Adding new work shouldn’t be too problematic as long as the customer understands that this contract is for X number of points worth of work plus a low hourly fee. This means that something has to give with new work being added to the project. For example, if after a sprint review the customer decides they would like to see some new functionality added that they hadn’t thought of before, the IT provider can provide a point estimate on that new functionality and then work with the customer on re-prioritizing the backlog, with the understanding that anything past the X number of agreed upon points is out of scope under the original contract. One might think this smells a bit like “change request hell” experienced by far too many customers of fixed price software development projects, but it’s not. The customer is not paying an additional price for new backlog items being added, but they are asked to prioritize the new items and potentially push lower priorities out of scope of the original contract. Any substantial change request on a fixed price would require some serious maneuvering on the part of the IT provider in most cases, and often leads to the IT provider coming back with a quote on how much it will cost to incorporate the change request. In this case the IT provider is telling the customer they can only have their newly discovered functionality if they pay more money. The agile project allows the customer to decide whether the newly discovered functionality is more important than some previously defined functionality.
How do we make all this a repeatable process?
Repeat steps 1-7 above? 😉 Look, I know everyone wants to get awesome at estimating software development projects. We want this sure-fire, can’t miss process to make our estimates precise AND accurate. I don’t think it exists. Sure, if you’re doing a lot of the same types of work over and over, then you should be able to get pretty good with estimating those types of projects. It still comes down to having the people with the right level of experience and expertise working on the projects. Yes, historical data is useful and we should use it as a reference for future work. But we don’t want to blindly follow historical data as our golden path to “estimating nirvana”. Remember, at least one person ACTUALLY DOING THE WORK should be involved with estimating. This will help keep the estimates grounded in a bit of reality, not just past history alone.
I’ve outlined a way you can do agile contracts and estimate them but the hardest part is getting buy-in. The fixed price contract is a sacred cow. Enter stage left this oddity called agile software development. Everyone seems to love certain parts of agile, but contracts and how to estimate projects, especially in relation to pricing a potential project for a contract, are two issues not typically tackled in a manner consistent with agile principles. The challenge is adapting to a different way of estimating and structuring contracts. It’s pretty easy to sell the customer on “we’ll deliver potentially shippable software every 2-4 weeks.” It’s quite another feat to sell the customer on a new way of buying solutions. Actually, the customer might be the easy sell compared to the IT provider. It’s hard enough landing a deal without throwing in a new contract structure. The tendency for IT providers is going to be to stick with the “standard”. It’s one less roadblock to a sale. I understand. But I also know that if a company wants to truly amaze its customers and rise to the top, then it has to sometimes break away from the herd. It is possible to break away from the herd when it comes to estimating and contracts for the agile IT provider, and I believe it can pay big dividends. Customers will get more of what they want, less of what they don’t, and can properly budget the project while remaining flexible enough to account for changes. IT providers take on less of the risk, don’t waste tons of time up front (yet still can feel comfortable with estimates and planning), and continue to have a direct monetary incentive to complete work more efficiently. Both the customer and IT provider will benefit from the agile contract as it leads to the path of a partnership versus a cantankerous relationship built around change requests and contract nitpicking.