Are We There Yet?
Are we there yet? – Donkey from Shrek 2
If you are a parent with kids any older than three or four years of age you’ve heard it: “Are we there yet – How much farther is it?” If you’ve ever developed software, you’ve heard the same questions, but this time, rather than coming from your four year old in the back seat of the car as you are driving to Granny’s house, it’s a stakeholder wondering when your project will be finished. You likely know the answer to the question from your four year old, but chances are, you will have a very tough time answering the project stakeholder’s question with any degree of accuracy. The answer to this simple question is extremely important because product launches, project funding, marketing initiatives, budgeting, staffing and so much more all depend on this seemingly simple question.
Why are 21% of companies experiencing schedule overruns exceeding one year, and why are 11% overrunning their budget by more than US $1 million? As I have mentioned in several of my previous articles, estimating how long it will take to complete software projects is extremely difficult. In my view, it is the single most challenging part of the entire software development process. How can we do better? This article builds on the concepts presented in my three most recent articles: (Estimating Effort For Your Agile Stories, Agile Planning Poker, Calculating the Velocity of Your Agile Project) Using these articles as the platform upon which we will base this methodology, I will demonstrate how you can develop a reasonably accurate estimate of the completion date of your project.
There is a wide range of methods for estimating how long it will take to complete a software project. Just a few of the many ways include:
• Analogous: Similar projects of a similar size and scope. For projects where you can draw a strong analogy to one (preferably many) projects of similar size, scale and functionality. Likely the most accurate way to do it.
• Wishful Thinking: This is a remarkably popular method. It is often used when project sponsors or developers are driven by cost or time constraints and don’t want to spend the time up front to do proper planning or estimating. As one might expect, this method rarely (if ever) produces desired results and most often ends in significant overruns or outright failure.
• Waterfall: The most prevalent method for medium to large scale projects. Statistics show about a 1/3 success rate. 1/3 of projects are significantly late, over budget or lacking significantly in features and functionality. 1/3 of projects fail or are cancelled.
• Function Point Analysis: This method was in vogue for several years in the 1990s. It breaks a project down into a collection of smaller tasks. Each task is assigned a function point value. A function point has a corresponding time estimate associated with it. The time to complete the project is that total of all of the function point assessments.
• Mandate: The project manager is told when the project will be completed, what the budget will be, and what features and capabilities will be included in the final result. This is really just another form of Wishful Thinking only worse because the demoralizing death march to complete such projects destroys morale. The most talented (those who can leave) leave as soon as possible to find a better work environment.
• Agile: Agile is not an estimation methodology in and of itself, and it does not generally develop estimate in units of time. Instead, estimated units of work are used. The units of work do not directly map to time until you have calculated your team velocity. Once the velocity is determined, it can give you a pretty good idea as to how long it will take to complete your project – that is for the body of work in the project that you know of and understand.
There are many other methods, and no method is perfect. I use two approaches – Analogous and Agile. Whenever possible, I try to compare the project I am attempting to estimate to projects I have done in the past that are similar in size and complexity and have similar teams. The more similar the projects and teams, the more accurate this method will be. That approach is, however, not the topic for this article. When I am not able to use the Analogous approach, the method that I use to forecast is based on the Observed Velocity of an Agile Team or Project. For the approach to be effective, we need to understand a few fundamental things.
Specificity and Accuracy
“A little inaccuracy sometimes saves a ton of explanation.” – H. H. Munro (Saki) (1870-1916)
Specificity and Accuracy are not the same things. Guessing the number of jellybeans in a jar and saying 500 is a specific answer, but it is likely not accurate. Saying that the number of jellybeans in the jar is 500 plus or minus 50 (10%) provides a range states the accuracy of your estimate. This answer is less specific, but more useful because the stated accuracy provides people with an understanding of the uncertainty or range of accuracy of your estimate. Stakeholders of software development projects tend to want specific answers for when a project will be delivered, and if you want to win the jellybean guessing contest, you will also need to provide a specific answer. However, the challenge for estimation software project duration is that while the number of jellybeans in the jar remains constant, a software project usually changes its size and its requirements many times over the duration of the project. It’s not unlike going to the grocery store with your family and a list of items to purchase and an estimate of the amount of money you need to bring with you to pay for your purchases. As you move through the store, your spouse and your children remove items from the cart and add items to the cart that are not on your list. By the time you get to the checkout, the contents of the cart are very different from the items on your shopping list, and now your shopping budget is likely significantly off. Some of the items put into the cart might be less expensive than some that were removed, but similarly, they could be more expensive. You may now have extra money in your pocket to pay the bill, but it is more likely that you don’t have enough. To deal with this scenario, you could take several approaches:
1. Not bother with the list and get whatever you want and worry about the cost at the checkout. That could become very expensive very quickly, and you may discover after you are home that purchased what you wanted and didn’t purchase what you really needed.
2. Refuse to make any changes – stick to the list. Your stakeholders (family) might be really unhappy with you.
3. Focus on the important things on your list while shopping, but in your budget, plan for some unknowns.
The third option is likely the best one, and this is the approach I use when estimating how much time it will take to complete a project. The process of estimation is broken down into three parts and is based on the following three premises:
1. It is relatively easy to estimate what you know.
2. It is difficult to estimate what you know you don’t know.
3. It is very difficult to estimate what you don’t know you don’t know.
I refer to these items as Class 1, Class 2 and Class 3 respectively.
Download a copy of the spreadsheet used to perform these calculations here: Excel Spreadsheet: Estimate How Long It Will Take To Complete Your Agile Project
Before we get into the calculations, we need some basic Settings or Project information.
The first Velocity entry comes directly from the Velocity you have observed for your team. (To determine your team or project Velocity, see three previous articles mentioned in the first paragraph of this submission). Sprint duration is the number of working days in your sprint. Work days per month is the number of working days in a typical month. You can adjust this value to accommodate for an average number of days in a month that you expect to lose to vacation time, sick leave etc… over the course of the project. Velocity / Day is a simple calculation that divides the Sprint Velocity by the number of work days in a Sprint.
Class 1 – What You Know You Know
To calculate how long it will take to complete your project you simply divide Estimated Effort by Velocity in Units of Effort per time period. The following chart displays Velocity in Days, Weeks and Months using the values from the Settings table to adjust the time base accordingly. The Confidence value is the degree of confidence you have both in your Effort Estimates and the completeness of your collection of things that you know. Adjusting the confidence level changes the High and Low range of the estimate as a percentage. Spread is the difference between the High and Low range of your estimate. The higher your confidence level, the smaller the Spread will be, and the closer the High and Low values will be to the Probable.
So in the above example, the Estimate for all of the Story Cards that you know about is 600. Using the Team Velocity from the Settings, the Probable project completion is in 128 working days. With a 90% confidence level of 90%, the High and Low ranges of the estimate are 140 and 115 days respectively. So there you have it, a reasonably accurate estimate of how long your project will take to complete… but wait… there’s more…
Class 2 – What You Know You Don’t Know
These are the things in your project that you know about, but you don’t have a high degree of confidence in your estimates. This may be due to technical challenges, skill sets that the team lacks, new technologies that must be learned, high risk experimental work, etc… It could also be work you are aware of, but don’t know for sure if your will need to complete it. I use a confidence level of about 50% for this body of work.
Class 3 – What You Don’t Know You Don’t Know
These are the things in your project that you don’t know about at all. They exist in almost every project. They can be the sudden change in direction, the new essential feature that appeared out of nowhere and no one had ever discussed it, or it could even be something suddenly dropped from the project because it is deemed no longer needed. I separate this portion of the estimate out of the other two area so that I know I have explicitly accommodated for it, and in doing so, I have not distorted other areas of the Estimate in unseen ways. Because this is truly the area of the unknown, I use a confidence level of 25% or even lower. As you seen from the calculations below, the Range of Days (16) is greater than the Probable number of Days.
The Summary section of your estimate is simply the addition of Class 1, Class 2 and Class 3 values. The one exception is the Confidence level which is a weighted average of the Confidence levels from the three classes weighted by Effort for each.
In this example, the project will be completed in 178 working days plus or minus 27 days (54/2) with a confidence level of 82% that it will be completed in roughly 151 days.
By keeping track of the changes to Stories in all three Classes over the duration of the project, you will be able to do some benchmarking and have a good body of statistics for use in future projects. As well, you will have a significant body of information to share with Stakeholders who want to know why the project is slipping. If you’ve accommodated 50 Units of Effort in Class 3, but due to change requests and scope creep it becomes 300 Units, you have a mechanism to reign things in or to go back to the stakeholders and obtain more budget, time extension or whatever is needed to complete the project.
So there you have it, a simple, easy, and reasonably accurate means of calculating how long it will take to complete your project.
As a closing note: Remember to do the difficult and high risk things first. This means try to do as much of the Class 2 stuff as early as possible in the project. As well, do any of the items identified in your Estimates as complex or high risk early in the project. Some project will in spite of best efforts, a great plan and a superb team fail, so if you are going to fail, then fail early. Chances are that if you follow good process and have good people on your team, you will succeed.
“We didn’t lose the game; we just ran out of time.” – Vince Lombardi
As always, I look forward to your comments.
Tags: Accurate Estimate, Agile, Agile Development, Agile Project, Completion Date, Effort, Estimation, Methodology, Project Management, Project Velocity, Scope, Software, Software Development, Stakeholder