What is Creativity

Lightbulb (flickr: yikerlarooni )

Don’t think. Thinking is the enemy of creativity. It’s self-conscious, and anything self-conscious is lousy. You can’t try to do things. You simply must do things. – Ray Bradbury

A hunch is creativity trying to tell you something. – Frank Capra

What does creativity have to do with software development – actually a lot. Software development is the process of creating software solutions that have never before been created. If your solution has already been created, then you shouldn’t be doing it because you are either plagiarizing, or you are needlessly reproducing work that you should be reusing.  Not only should your work be original, it should also be useful. Clearly if you are creating something that has never before been created, you are well… doing something creative, and thus knowing how to be creative is clearly a good thing. But before discussing creativity as it applied to software we must first set a foundation and define what creativity is.

Creativity is more than just being different. Anybody can be plain weird; that’s easy. What’s hard is to be as simple as Bach. Making the simple, awesomely simple, that’s creativity. – Charles Mingus

There is nothing new in the world. Actually there is very little that is new, and the things that many of us consider to be new are not in fact new. The iPhone at its introduction contained little, if anything that was truly new.  There is no denying that the iPhone was a revolutionary product when it was introduced, but its genesis was that of evolution, not revolution. Wireless phones, digital cameras, touch screens, GUIs, MP3 players, SMS clients, and installable applications have all been around for years. What the iPhone brought us that was new, was the combination of all of the aforementioned technologies (and more) in a way that had previously never been seen. There were countless other things that could have been included in the iPhone, and many devices already included most of what the iPhone offered. In fact as a phone, the iPhone wasn’t even a particularly good phone. What made it unique was there was so much capability in one package, and it was so elegant and simple to use. Examine innovation in any field be it art, science, theatre, and even software, and you will find that most great innovations were mostly evolutionary, not revolutionary.

So what is creativity? Creativity is a difficult thing to define. You can’t see it, hear it or measure it in any way, yet we know it exists and we often (not always) recognize and appreciate its results. It is that bright light bulb that pierces the darkness, illuminating a solution that is often so simple that you wonder how it could have ever been overlooked. But these solutions are overlooked every day, and it is because we all view the world with our own set of filters that prevents us from seeing what other see.

Many people believe that so called “Creative People” dream up new ideas and new products out of the blue. While that may be true in a few rare cases, it is remarkably uncommon. Creativity is the process of breaking out of established patterns. It is an accretive process that builds upon the many innovations that have already occurred over the millennia. It is the reorganization, recombination or reinterpretation of any number of entities, concepts or ideas [elements] either by altering, removing, or adding new elements to the existing set in new and different ways. The result is something unique and previously unknown to the person or people creating it. It has nothing to do with aesthetics, practicality, usefulness, whether or not you or someone else likes or dislikes the results or even whether or not it has already been done by someone else. All that matters is that it is different than what was previously known to the creator. What is created does not even need to be manifested in the physical world. Physical manifestation is only evidence to the rest of the world of the uniqueness of the creative idea.

Picasso did not invent paint, canvas or even the techniques for laying paint onto canvas. Instead, he simply asked “What if on a single two dimensional canvas I could represent more than one view of a person or object?” Monet’s failing eyesight caused him to use larger blocks of colour so that he could actually see what he was painting. The principles of flight were well known long before the Wright Brothers first took to the air. What they did was find the right combination of bicycle parts, weight, wing surface, engine power, wood and canvas and the right balance point to make the first flight possible. Edison’s introduction of the light bulb to the world was no different. There were actually many people working on the creation of artificial light starting with Joseph Swan who obtained a UK patent for the first light bulb in 1860. Canadian Henry Woodward created his own version and obtained a US patent in 1874. Woodward’s invention was considered crazy in Canada and he sold the patent to Edison who saw the commercial potential and refined the design so that it became a viable product. There is no question that each of these individuals were highly creative and innovative people who were able arrive at solutions that had eluded so many. None of these creations rose out of nothing. It was their specific and unique organization of elements in a way that allowed them to arrive at the solution that had eluded so many that made them successful.

Productive Creativity

Creativity has more to do with the elimination of the inessential than with inventing something new. – Helmut Jahn

It is one thing to be creative, but is another thing to create something that is unique and also useful or wanted. We live in a hyper-competitive world where the race to have the next creative innovation has become extremely intense. Hundreds of billions of dollars are spent worldwide every year on R&D. Even with so much effort being expended by so many organizations, and with creativity itself being nothing more than discovering new combinations of existing elements, it is clear that creativity – the ability to find that magic combination that is both unique and wanted or needed is very difficult, and something that few of us do well.

Vincent Van Gogh was unable to sell a single painting while he was alive, but seven of his paintings are now among the 40 most expensive paintings ever sold at auction. Pablo Picasso with ten of the top 40 is the only artist to have more. Together, these two masters hold 17 of the top 40 spots – nearly half of the total. Two people are creators of nearly half of the 40 most valued pieces of art in history (at least the ones that have come to auction). So why was Picasso’s work revered during his lifetime, and Van Gogh’s work considered near worthless when created, yet now command prices in excess of $100 million. Why do some creative software ideas succeed, while most never see the light of day? In Van Gogh’s case, his timing was just lousy; the world was not ready for his kind of creativity. No doubt timing can also be lousy for technology innovators; Apple failed with the Newton and succeeded with the iPhone. Business factors are certainly relevant, but putting business factors aside, most creative efforts that do not catch on are because the idea is just not sufficiently useful to enough people. Sometimes timing is the issue, but more often than not, the idea is just not of value. The difference between the two is Productive Creativity.

People who are productively creative focus their effort on bringing something of value to the market. Van Gogh and Picasso were both brilliant artists, and each was passionate about his own work. While Van Gogh did not understand how to convert his brilliant paintings into a commercial success (spending time in an asylum and cutting off part of his own ear didn’t help); Picasso did. Certainly there are also artists with little or no artistic talent who understand how to commercialize their work, and some with neither talent or business acumen. Parallels can be drawn with software developers because in many respects, software development is at the intersection of Art and Commerce. Sadly, this intersection is not understood or appreciated as such. I have also encountered more than a few brilliant developers whose contributions are marginalized because they didn’t know how to frame their ideas in a business context or because the business itself regards that software development as purely a technology effort with no art involved. To some degree, I think this attitude is because software development entered the world in large corporations to run payrolls, manage accounts receivable and many other important, but nonetheless mundane tasks. Software development now has very little to do with the data centres, servers and networking that enables it and has more to do with things like human perception, human cognition, language, ontology and taxonomy. Software development is the art of abstraction of things, processes and interactions in the real world into structures in code resembling sentences, paragraphs and chapters of a book. There is an infinite number of ways to develop any software application and none of them are either right or wrong unless the numbers don’t add up. Product Creativity is the art of figuring out ways to create software that are better than what anyone else creates. Anyone who doubts the importance of creativity and inspiration and its intersection with commerce need only look at the mobile space where the Android and iPhone platform each have more than 200,000 unique and innovative applications available.

Fostering Creativity in Your Team

Creativity Killed (flickr: Anant Nath Sharma

But the person who scored well on an SAT will not necessarily be the best doctor or the best lawyer or the best businessman. These tests do not measure character, leadership, creativity, perseverance. – William J. Wilson

The truly creative people in this world are the outliers – they are rare. They are not identified by résumés that say they have ten years of Java experience, or perhaps the equivalent in .Net. These are simply measures indicating that they continued to show up for work each day. Sadly, these metrics are the primary filters used when recruiters and HR specialists begin the search process. The creative people you want on your team are unique in their ability to see the world differently from the rest of us. That unique vision is essential, but it is not enough. What is also needed is the ability to practically apply that unique vision within the structure and processes that are necessary to support the act of developing complex software projects. To be certain, an important part of that is experience. Very experienced people have a body of knowledge that gives them a distinct advantage over less experienced individuals, but the very filters we tend to use end up filtering out the people with the more important broad experience in favour of people with less useful narrow experience.

And it has become a kind of a truism in the study of creativity that you can’t be creating anything with less than 10 years of technical knowledge immersion in a particular field. – Mihaly Csikszentmihalyi

I don’t entirely agree with the previous quote because there are people who have decades of experience who will never be creative, and there are those who simply “get it” far sooner than most. That being said, a sound technical background is important – not measured in years, but measured in how they’ve used their experience to innovate. Sometimes lack of experience can even be an advantage because the lack of preconceived notions can make obvious solutions that are missed by people who are used to the status quo. Experience inherently builds patterns of efficient behaviour; creativity necessitates the breaking of patterns. Understanding “why” leads to new solutions; just knowing “how” simply leads to more of the same.

Not only is it important to find and hire people who develop productive creative solutions, you need to provide them with an environment where they can be productively creative. The onus for this is on your company and on you as the person leading the team. Finding the unique combination of elements that produces the innovative new solutions requires experimentation. Experimentation means trial and error, and thus some of what is attempted will not work. Many corporations discourage such things, but if you and your organization really do want to innovate, there is no alternative path. Innovation does not come from process, protocol, structure, long status meetings or administrative trivia. While structure and process are important (without them anarchy reigns), do your best to free your team from such things and instead, spend that time experimenting and innovating. The benefits will far outweigh any any that could come out of the meetings and administrative noise. Help your team by giving them some structure within which to succeed. Set some objectives for them such as each team member must present their innovation work once per quarter to the whole team. It’s amazing what this kind of peer pressure can produce. It can create a competitive environment where each team member tries to produce the best they can dream up. You will be amazed at what comes out of this.

If creative people are rare, the person who can be creative and function well within a structured process is even rarer. To their great loss, many organizations also have a difficult time working with creative people. Some companies fear creativity, and some simply don’t want anything to do with it. I’ve had conversations with several recruiters who say that employers have come to them and said “Please don’t send me anyone with ideas. I just want someone who is going to sit in his/her seat and program what I tell them to.” Without a doubt, there are other companies that are innovating, while companies like these stagnate and die. If you’ve gotten this far in the article, you are clearly not of that mind, so what do you do?

Here are some things you can do:

1. Try to find and hire those creative geniuses. I would rather have a small team of inspired out of the box thinkers that may be hard to manage than a team of drones. My team of creative geniuses will be more work for me to manage, but we will always finish first, and we will always have the best solution.
2. Make your team as diverse as possible. Try to find the best group of people possible that includes the broadest range of backgrounds as possible. Mix Java programmers with .Net programmers; seasoned pros with the kid just out of school; web developers with main framers; and financial application developers with smart phone developers. The greater the diversity, the more ideas people will be able to bring to the table.
3. Look for people to bring into your team with backgrounds in something completely different like art, literature, philosophy, music or languages. The more things they have done (particularly creative things) the more ideas, unique approaches and experience they have to offer. They still need to be excellent programmers, but excellent programmers with a broader range of experience have more to bring to the creativity table.
4. When hiring your team, pay little attention to things like the number of years a person has spent with a particular technology or worked in a particular industry. Instead, ask them about their innovations and inventions. Find out what they’ve dreamed and created, and why it was successful and what they’ve learned from the things that were not successful. When asking questions like this during an interview, be sure that you explain why. People are trained to filter their responses in interviews to provide what they think the corporation is looking for. Rarely is creativity one of those things, so help your candidates help you to uncover their creative talents.
5. Give creativity a home and a place to thrive. Creativity is a fragile thing that must be nurtured. Give your team one afternoon per week where they get to work on their own idea. Even better, give them an entire day each week. What they produce must be related to the business you are in, and they must present their idea to the rest of your team for possible inclusion in your project, or maybe something entirely new. Either way, it must be something its creator believes has value to the group and the organization as a whole. Celebrate the successes, and have some fun (not ridicule) with the ones that are odd or off the wall. Countless studies have shown that the companies that formally support creativity through such programs are consistently the ones with the best new idea, the highest productivity, and the best morale. Google is one of them. If you are running an Agile team – actually any development team, then schedule the creative time. The work that comes out of this time is important  (it could be the big break your company needs), so don’t let all of the other urgent things encroach on it.
6. Encourage experimentation, and yes, failure. Most organizations tend to have little tolerance for failure of any sort, and this causes people to take the safe route, to not experiment, and to not be creative. Failure means that something new was tried, and not all new things succeed. If nothing new is tried, then all you are doing is repeating what you’ve already done, and you can bet that while you continue to do the same thing, the competition will be innovating while you are not. Minimize the downside potential of the failure by limiting the scope of what is being attempted to a series of smaller pieces that can each be assessed to determine if continuing down that path is worthwhile.
7. Always ask Why, and then ask Why again. Keep peeling back the layers of onion until you find the real Why. You will often find that this makes the right solution evident, and this will provide your How.

Clearly the companies that innovate are the ones that win. This is especially true in the field of technology. For a company to be innovative requires both innovative people, and a company that encourages and even helps their people to be innovative and creative. This means you as a manager need to hire the best, most productively creative people you can find, and you need to create an environment in which they can be creative. Do this and your company has a decent chance of being a leader; not doing this means that you and your company will always be followers.

As always, I look forward to your comments.

Michael

Are We There Yet?

Are We There Yet? (Flickr - Lorant Zsabo)

Are we there yet? – Donkey from Shrek 2

If you are a parent with kids any older than three or four years of age you’ve heard it: “Are we there yet – How much farther is it?” If you’ve ever developed software, you’ve heard the same questions, but this time, rather than coming from your four year old in the back seat of the car as you are driving to Granny’s house, it’s a stakeholder wondering when your project will be finished. You likely know the answer to the question from your four year old, but chances are, you will have a very tough time answering the project stakeholder’s question with any degree of accuracy. The answer to this simple question is extremely important because product launches, project funding, marketing initiatives, budgeting, staffing and so much more all depend on this seemingly simple question.

Why are 21% of companies experiencing schedule overruns exceeding one year, and why are 11% overrunning their budget by more than US $1 million? As I have mentioned in several of my previous articles, estimating how long it will take to complete software projects is extremely difficult. In my view, it is the single most challenging part of the entire software development process. How can we do better? This article builds on the concepts presented in my three most recent articles: (Estimating Effort For Your Agile Stories, Agile Planning Poker, Calculating the Velocity of Your Agile Project) Using these articles as the platform upon which we will base this methodology, I will demonstrate how you can develop a reasonably accurate estimate of the completion date of your project.

Methods

There is a wide range of methods for estimating how long it will take to complete a software project. Just a few of the many ways include:
•    Analogous: Similar projects of a similar size and scope. For projects where you can draw a strong analogy to one (preferably many) projects of similar size, scale and functionality. Likely the most accurate way to do it.
•    Wishful Thinking: This is a remarkably popular method. It is often used when project sponsors or developers are driven by cost or time constraints and don’t want to spend the time up front to do proper planning or estimating. As one might expect, this method rarely (if ever) produces desired results and most often ends in significant overruns or outright failure.
•    Waterfall: The most prevalent method for medium to large scale projects. Statistics show about a 1/3 success rate. 1/3 of projects are significantly late, over budget or lacking significantly in features and functionality. 1/3 of projects fail or are cancelled.
•    Function Point Analysis: This method was in vogue for several years in the 1990s. It breaks a project down into a collection of smaller tasks. Each task is assigned a function point value. A function point has a corresponding time estimate associated with it. The time to complete the project is that total of all of the function point assessments.
•    Mandate: The project manager is told when the project will be completed, what the budget will be, and what features and capabilities will be included in the final result. This is really just another form of Wishful Thinking only worse because the demoralizing death march to complete such projects destroys morale. The most talented (those who can leave) leave as soon as possible to find a better work environment.
•    Agile: Agile is not an estimation methodology in and of itself, and it does not generally develop estimate in units of time. Instead, estimated units of work are used. The units of work do not directly map to time until you have calculated your team velocity. Once the velocity is determined, it can give you a pretty good idea as to how long it will take to complete your project – that is for the body of work in the project that you know of and understand.

There are many other methods, and no method is perfect. I use two approaches – Analogous and Agile. Whenever possible, I try to compare the project I am attempting to estimate to projects I have done in the past that are similar in size and complexity and have similar teams. The more similar the projects and teams, the more accurate this method will be. That approach is, however, not the topic for this article. When I am not able to use the Analogous approach, the method that I use to forecast is based on the Observed Velocity of an Agile Team or Project. For the approach to be effective, we need to understand a few fundamental things.

Specificity and Accuracy

“A little inaccuracy sometimes saves a ton of explanation.” – H. H. Munro (Saki) (1870-1916)

Specificity and Accuracy are not the same things. Guessing the number of jellybeans in a jar and saying 500 is a specific answer, but it is likely not accurate. Saying that the number of jellybeans in the jar is 500 plus or minus 50 (10%) provides a range states the accuracy of your estimate. This answer is less specific, but more useful because the stated accuracy provides people with an understanding of the uncertainty or range of accuracy of your estimate. Stakeholders of software development projects tend to want specific answers for when a project will be delivered, and if you want to win the jellybean guessing contest, you will also need to provide a specific answer. However, the challenge for estimation software project duration is that while the number of jellybeans in the jar remains constant, a software project usually changes its size and its requirements many times over the duration of the project. It’s not unlike going to the grocery store with your family and a list of items to purchase and an estimate of the amount of money you need to bring with you to pay for your purchases. As you move through the store, your spouse and your children remove items from the cart and add items to the cart that are not on your list. By the time you get to the checkout, the contents of the cart are very different from the items on your shopping list, and now your shopping budget is likely significantly off. Some of the items put into the cart might be less expensive than some that were removed, but similarly, they could be more expensive. You may now have extra money in your pocket to pay the bill, but it is more likely that you don’t have enough. To deal with this scenario, you could take several approaches:

1.    Not bother with the list and get whatever you want and worry about the cost at the checkout. That could become very expensive very quickly, and you may discover after you are home that purchased what you wanted and didn’t purchase what you really needed.
2.    Refuse to make any changes – stick to the list. Your stakeholders (family) might be really unhappy with you.
3.    Focus on the important things on your list while shopping, but in your budget, plan for some unknowns.

The third option is likely the best one, and this is the approach I use when estimating how much time it will take to complete a project. The process of estimation is broken down into three parts and is based on the following three premises:

1.    It is relatively easy to estimate what you know.
2.    It is difficult to estimate what you know you don’t know.
3.    It is very difficult to estimate what you don’t know you don’t know.
I refer to these items as Class 1, Class 2 and Class 3 respectively.

Estimation

Settings

Download a copy of the spreadsheet used to perform these calculations here: Excel Spreadsheet: Estimate How Long It Will Take To Complete Your Agile Project

Before we get into the calculations, we need some basic Settings or Project information.

Settings

The first Velocity entry comes directly from the Velocity you have observed for your team. (To determine your team or project Velocity, see three previous articles mentioned in the first paragraph of this submission). Sprint duration is the number of working days in your sprint. Work days per month is the number of working days in a typical month. You can adjust this value to accommodate for an average number of days in a month that you expect to lose to vacation time, sick leave etc… over the course of the project. Velocity / Day is a simple calculation that divides the Sprint Velocity by the number of work days in a Sprint.

Class 1 – What You Know You Know

To calculate how long it will take to complete your project you simply divide Estimated Effort by Velocity in Units of Effort per time period. The following chart displays Velocity in Days, Weeks and Months using the values from the Settings table to adjust the time base accordingly. The Confidence value is the degree of confidence you have both in your Effort Estimates and the completeness of your collection of things that you know. Adjusting the confidence level changes the High and Low range of the estimate as a percentage. Spread is the difference between the High and Low range of your estimate. The higher your confidence level, the smaller the Spread will be, and the closer the High and Low values will be to the Probable.

What You Know You Know

So in the above example, the Estimate for all of the Story Cards that you know about is 600. Using the Team Velocity from the Settings, the Probable project completion is in 128 working days. With a 90% confidence level of 90%, the High and Low ranges of the estimate are 140 and 115 days respectively. So there you have it, a reasonably accurate estimate of how long your project will take to complete… but wait… there’s more…

Class 2 – What You Know You Don’t Know

These are the things in your project that you know about, but you don’t have a high degree of confidence in your estimates. This may be due to technical challenges, skill sets that the team lacks, new technologies that must be learned, high risk experimental work, etc… It could also be work you are aware of, but don’t know for sure if your will need to complete it. I use a confidence level of about 50% for this body of work.

What You Know You Don't Know

Class 3 – What You Don’t Know You Don’t Know

These are the things in your project that you don’t know about at all. They exist in almost every project. They can be the sudden change in direction, the new essential feature that appeared out of nowhere and no one had ever discussed it, or it could even be something suddenly dropped from the project because it is deemed no longer needed. I separate this portion of the estimate out of the other two area so that I know I have explicitly accommodated for it, and in doing so, I have not distorted other areas of the Estimate in unseen ways. Because this is truly the area of the unknown, I use a confidence level of 25% or even lower. As you seen from the calculations below, the Range of Days (16) is greater than the Probable number of Days.

What You Don't Know You Don't Know

Summary

The Summary section of your estimate is simply the addition of Class 1, Class 2 and Class 3 values. The one exception is the Confidence level which is a weighted average of the Confidence levels from the three classes weighted by Effort for each.

Summary

In this example, the project will be completed in 178 working days plus or minus 27 days (54/2) with a confidence level of 82% that it will be completed in roughly 151 days.

By keeping track of the changes to Stories in all three Classes over the duration of the project, you will be able to do some benchmarking and have a good body of statistics for use in future projects. As well, you will have a significant body of information to share with Stakeholders who want to know why the project is slipping. If you’ve accommodated 50 Units of Effort in Class 3, but due to change requests and scope creep it becomes 300 Units, you have a mechanism to reign things in or to go back to the stakeholders and obtain more budget, time extension or whatever is needed to complete the project.

So there you have it, a simple, easy, and reasonably accurate means of calculating how long it will take to complete your project.

As a closing note: Remember to do the difficult and high risk things first. This means try to do as much of the Class 2 stuff as early as possible in the project. As well, do any of the items identified in your Estimates as complex or high risk early in the project. Some project will in spite of best efforts, a great plan and a superb team fail, so if you are going to fail, then fail early. Chances are that if you follow good process and have good people on your team, you will succeed.

“We didn’t lose the game; we just ran out of time.”  – Vince Lombardi

As always, I look forward to your comments.

Michael

A Simple Way to Calculate the Velocity of Your Agile Teams and Projects

Speed (flickr - Sean Tubridy)

Whenever the work is itself light, it becomes necessary, in order to economize time, to increase the velocity. Charles Babbage

What is Agile Velocity?

The calculation of Velocity in physics is pretty straightforward, and if memory serves me correctly, it is the first formula I learned. Simply put:

Velocity = Distance / Time

In truth, Velocity is a vector and requires a directional component to fully qualify as a Vector. Velocity without a directional component is Speed – a Scalar. What we have in the above formula and what we will actually be measuring is a Scalar – the speed of the team. Velocity as a term has, however, become so widely used in the Agile community that it makes no sense for me to refer to it as Speed. So with apologies to Sir Isaac Newton for the misused term, and for my liberal interpretation (as you shall see) of his first two laws of motion, we shall accelerate into the topic.

So if Velocity is Distance divided by Time, what do we use as the numerator or distance unit in our equation? For our purposes, the distance is Units of Effort. My previous post “Agile Planning Poker” explains how to accurately develop estimates for Units of Effort. Time (the denominator) is the length of our Sprint which in my projects is two weeks. Velocity is thus Units of Effort Completed / Sprint. As an example: a team that regularly completes an average of 70 units of Effort in a two week Sprint has a Velocity of 70 Units of Effort per Sprint, or is simply stated as just 70. I prefer to keep the Sprint as the denominator because the team will tend to think in terms of Sprints, but for reporting, or forecasting, I will sometimes use Weeks as the baseline. Whenever I do, I am careful to note the different units.

There is much discussion and debate online about using Story Points vs. Ideal Days vs. Actual Days to calculate Velocity. There are other ways I’ve seen that work with a variety of inputs that are formulaicly balanced. I firmly subscribe to the Occam’s Razor approach to things and thus avoid such complexities. The number and variety of factors that can affect the Velocity of a team are both significant in number, and difficult to quantify (See calibration below). Many of the factors will cancel each other out (electronics systems depend on this principle for noise reduction), but the net effect will be a bias on the team Velocity. To illustrate the difference, think about all of the things that you could quantify to determine their effect on the speed of an automobile. You could measure fuel consumption, manifold pressure, acceleration and a myriad of other things and plug them into equations to calculate your speed. Or you could simply measure how far you’ve travelled and divide by the time it took you to travel that distance. So rather than attempting to measure and accommodate for a myriad of changing and difficult to quantify input values, I prefer to take the simple path and use Observed Velocity. The Observed Velocity is simply how many Units of Effort your team completes in a typical Sprint. For this to be useful, however, you need to calibrate your Velocity.

Calibration

Optimization

Optimization is a process that should be completed before you begin Calibration. There are two things of primary interest to us in our calibration. They are:

1. The Friction or consistent forces that are a constant drag on productivity and reduce Project Velocity.
2. The Variable or Dynamic Forces that decelerate the project or team members and cause the Project Velocity to be irregular.

Optimizing both of these factors prior to Calibration will improve the stability of your Velocity calculation. As the Velocity is the basis for many of the metrics we use in Agile and Scrum, it is important to have a predictable Velocity.

Friction

Newton’s First Law States that “Every object will remain at rest or in uniform motion in a straight line unless compelled to change its state by the action of an external force.” Forces that do not propel your project will slow it down. By minimizing the forces that slow down your project, you reduce the “Friction” that reduces your Project Velocity. Less friction means higher Velocity and greater productivity.

In Software development, there are countless forces that can affect the Velocity of your team. As a team leader, project manager or executive, it is up to you to minimize the external forces that negatively impact on the team’s velocity. Friction forces are more or less constant. You can’t eliminate all of them, but you can reduce many of them. Optimizing them before Calibration is important. Friction forces include:

• Team composition: Are the right people with the right skills on the team.
• Process: Changes to your processes: Agile methods, build, release, testing, etc…
• Environmental factors: Interruptions, noise, poor ventilation, poor lighting, uncomfortable seating and desks, inadequate hardware or software, etc…
• Team dynamics: Some team members may not play nicely with others.

As you can see, most Friction type forces are largely environmental. Their effects are long term. They are also, often the easiest to address. Individually, Friction forces are usually weak forces. Cumulatively, they can have a significant impact. Obtaining optimal Team Velocity requires that they be eliminated or reduced.

Variable or Dynamic Forces

Variable or Dynamic forces are often unpredictable and unexpected. They decelerate your project and cause a loss of Velocity. Their effects are sometimes dramatic, but their influence is often brief.

Newton’s Second Law states that that “The acceleration of an object as produced by a net force is directly proportional to the magnitude of the net force, in the same direction as the net force, and inversely proportional to the mass of the object.” For our purposes, we are viewing this from the perspective of the cost (in terms of productivity) on individuals in the team, or the team as a whole. If you can’t eliminate a force that reduces Velocity (deceleration), then do your best to make it as consistent and predictable as possible  (minimal and infrequent deceleration). The more consistent and predictable the force, the more consistent your Velocity will be. Just like in the real world, accelerating and decelerating objects requires force (energy). Unnecessary acceleration and deceleration not only wastes energy and slows down your team, it also introduces variability. The variations in Velocity make your predictions more difficult. To put it simply: The more you allow your team is influenced by Dynamic Forces and things that change their ability to focus on their work, the more variability there will be in their Velocity  (productivity). This variability makes your forecasts less accurate. Some examples include:

• Team changes: Adding member, losing members, changing roles and responsibilities.
• New tools: Introduction of new development tools, database technologies, languages, etc… require learning, and reduce Velocity until learned.
• Vendor defects: Defects in third party tools and software requiring developer workarounds eat into productivity and Velocity.
• Responsibilities outside of the project: Team members assuming additional responsibilities outside of the project. Shifting between projects can have a dramatic effect on productivity.
• Personal issues: Colicky baby at home, personal health, family dynamics, etc…
• Stakeholders: Stakeholders may not be responsive to requests for information from the developers or tester and thus create delays. They may also have unreasonable expectations of the team.
• Unclear requirements: Lack of clarity or detail in requirements cause unnecessary churn and rework.
• Changing requirements: New project specifications might require skills that are non-existent or weak in the team. Acquiring the skills, either by introducing new team members, or by an existing team member acquiring the skills will impact productivity.
• Relocation: Moving the team to a new physical location upsets the rhythm and impacts their Velocity.

One cautionary note: Velocity is specific to a team and comparisons to other teams are irrelevant. There are many reasons for this, but perhaps the most obvious one is that different teams will develop different Estimates for the same task i.e. Developers on Team A might rate the Effort of a particular story at 6 while Team B might rate it at 15. Further, teams have different skill sets and strengths that make them very effective in some areas, but not in others. Because of this, resist inter-team comparisons – they are simply not fair or accurate.

The Calibration Process

While going through the following Calibration exercise, if your project is affected by Dynamic Forces like those listed above, do not compensate for them in your Velocity calculations. The reason for this is simple: In every Sprint you go through, one or more of these factors will likely raise its ugly head and reduce the Velocity. Some methodologies attempt to compensate for them through calculations. With software developers being the analytical type we tend to be, the temptation to do so can be great, but resist the temptation as in my experience, these calculations introduce additional complexity, with little value. By observing your Velocity averaged over three Sprints as described in the next section, and leaving the effects of Dynamic Forces in your Velocity calculations, you will automatically be developing a Velocity baseline that is biased to implicitly include a Dynamic Force compensation weighting. Indeed, there will be some variability between Sprints, but you should find the weighting to be surprisingly accurate, and most definitely simpler.

Sprint 1

So with the changes you’ve made to reduce Friction to moderate the effects of Dynamic Forces, you are ready to start your Calibration. Calibration is fortunately a very simple process, but it takes a bit of time. Calibration is simply observing your Velocity (Units of Effort Completed per Sprint) and adjusting your estimate to match the observed value. There is no formula for this, and there is no right or wrong way to do it. There is also only one objective to the Calibration Process, and that is to use a feedback loop through a Sprint Retrospective to review your Effort Estimates and adjust them so that they are consistent and accurate. The important thing to remember is that you are looking for a consensus from your team so that your team can then use the feedback to create more accurate estimates.

In your Retrospective for your first Sprint, you and your team will examine your Effort estimates for the Sprint and see if you agree as a team that the original Effort estimates were reflective of reality. Remember, that your estimates of Effort are relative. This means that per my previous article, you should find that your results roughly follow as Gaussian distribution. Further, relative sizing between Stories for both Estimate and Actual should be consistent. In other words, if Story A has an Estimated Effort of 5, and Story B has an Estimated Effort of 12, then the Observed Efforts at the end of the Sprint should remain the same relative to each other. If not, change the estimates to align with your newfound insight. Remember, that you and your team are attempting to calibrate or optimize is your ability to assess the Effort of any Story relative to other Stories in your project.

The result of this exercise produces a baseline Velocity that represents the Velocity of your team. You may also want to look at Velocity of individual team members, but I find this to be less useful as there are many factors affecting individual velocity i.e. Do you always give the really gnarly problems (the ones no one else can solve) to a specific person, and the really easy ones to someone else. If you do, you may see that the person with the easy problems has a higher velocity than the person with though ones. This is just one of many possible distortions in the measurement of individual Velocity that tend to make individual Velocities less useful.

Once you have completed your Velocity adjustments, and in the same session, I recommend that you jump immediately into doing your Estimates for the next Sprint. Doing so serves two purposes:

1. The team will get better at estimating Effort because the Calibration exercise will be fresh in their minds.
2. The freshly created baseline becomes the context for your Effort estimates for the next Sprint.

Sprint 2

After you complete your second Sprint, repeat the review process described in Sprint 1 and compare your results to others in both Sprint 1 and Sprint 2. Remembering that you are looking for your Effort estimates and observed results for individual Stories to be consistent relative to all other Stories, you should see significantly less divergence between Estimated and Observed results in your second Sprint. As in your first Sprint, adjust your Estimates to adjust with observed results.

Sprint 3

As in Sprint 2, compare the results the just completed Sprint (3) to the previous Sprint (2), as well as that of Sprint 1. Again, make your adjustments. After the third Sprint, you and your team should feel very comfortable about the ability of your team to deliver consistent Estimates and maintain a constant Velocity. If you think there is still room for improvement or you are struggling, go through an additional iteration. The more iterations you do, the more accurate your baseline will be, but keep in mind there is a law of diminishing returns – three Sprints should be all you need.

Your Velocity

With three Sprints behind you, take the average Velocity across all three Sprints. This is your Team Velocity. By reducing Friction and minimizing unnecessary Velocity changes prior to Calibration, you have optimized the individual and team Velocities. Over the course of the project, and assuming there are no significant factors introduced such as losing a team member, you should observe a very stable Velocity for the duration of the project. Some might say that you should continually try to increase the Velocity of the team – that should be your goal. While developers constantly try to improve their skills, and tools improve, it is unlikely that any of the team members will suddenly become significantly smarter or more productive, or the introduction of a new tool will suddenly make a drastic reduction in effort. If you can continue to reduce Friction or external forces, you may be able to increase Velocity, but remember, you should have gone through that exercise before you began your Calibration exercise. Unless your developers are slacking off (in my experience this is rare), you likely won’t be able to squeeze much more performance out of the individual team members. I am not suggesting that you not attempt to increase Velocity, but if you have done your job properly before you began the Calibration process, you should have already realized most of the gains possible. In addition to improving productivity, the purpose of the Optimization exercise was to obtain Velocity value that is more or less constant so you can begin developing accurate forecasts which is the topic for next week’s article.

As always, I look forward to your comments.

Michael

Developing Accurate Effort Estimates By Playing Games

Counting (flickr - marina moia)

Background

The poker player learns that sometimes both science and common sense are wrong; that the bumblebee can fly; that, perhaps, one should never trust an expert; that there are more things in heaven and earth than are dreamt of by those with an academic bent.  (David Mamet)

If you’ve read my previous post (recommended so that the following will make sense) you now have a mechanism for sizing the Effort of your Stories. But how do you actually come up with the numbers for Complexity and Size? Over the years that I’ve been leading software development projects, I’ve used a number of methods including comparison of projects of similar size and complexity, Microsoft Project, Function Point Analysis or even simply guessing. There have been a few other trendy and perhaps oddball approaches thrown in for good measure. Curiously, regardless of method, the results have tended to be pretty much the same. There has, however, been one exception. I’ve gotten noticeably better results with Agile methods using Agile Planning Poker.  Poker? Yes poker – well not really poker. It’s actually a process that is gaining in popularity as a means for sizing Agile Stories.

Agile Planning Poker is generally attributed to James Grenning in 2002 and popularized by Mike Cohn, owner of Mountain Goat Software in his book Agile Estimating and Planning. The process involves the members of the team independently developing quick Effort estimates and then comparing their estimate, discussing the differences and arriving at a consensus based on the discussions. Their independent assessments are all shared concurrently with the group by holding up flash cards with the numeric values of their respective assessments. The process relates to the game of poker only in the sense that the participants make their assessments discreetly, concealing them from the other participants until they all show their hands together. It is Important to the process that the participants conceal their assessments from each other and all participants reveal together. While this may seem silly, it serves the definite purpose of having each person arrive at their own conclusion without being influenced by the thinking of other members. The process is quick, simple, and in my experience, very effective.

The Process

The Moderator

The role of the moderator in this process is essential. The moderator sets the tone and the pace for the discussion. The moderator is typically the Project Manager or Scrum Master. The moderator:

• Reads the stories.
• Maintains pace to make sure that the process does not bog down.
• Moderates discussion, tempers the naturally competitive nature of some to ensure that nobody dominates with their perspective and that people who may not normally voice their opinions are also heard.
• Maintains a focus on good results – not on individuals having the winning hand.
• Ensures that people do not cluster their estimates. Some team members may want to be seen as correct, or at least not too different from the pack and always cluster their results in the middle i.e. always guessing 3/3. If your results follow a standard Gaussian Distribution, your safest bet would always be 3/3 if your motivation is to be with the pack, or to be viewed as being correct. Remember that although you are hoping to have a Gaussian Distribution, it is neither your motivation nor that of the team. What you are really doing with this exercise is trying to discover the outliers – the Stories that are actually much larger or much smaller than originally thought when the Story was first written. Following the herd will not uncover the outliers and consequently you will be exposing your project and team to risk. The Moderator must watch for this behaviour and encourage different viewpoints from all team members to make sure the outliers are uncovered.
• Helps the team to avoid Groupthink. If everyone always agrees, then you can be almost certain that you are doing something wrong. Diversity of opinion is essential and it must be encouraged. If there is too much consensus, the moderator should play Devil’s Advocate to stimulate alternative thinking.

Participants

Often it is only the developers involved in the sessions, but I like to have the Product Owner involved. At a minimum, a Planning Poker Session should include all of the developers on the team. For several reasons, it is a mistake to leave this work to a few of the senior team members:

• Smaller groups are more likely to have a group biases that can become Groupthink. More people means more opinions.
• Junior or newer team members learn a lot from these sessions. Experienced developers have insight that comes from years of experience and years of making and then recovering from mistakes. This experience is of incredible value to the team as a whole and can help to quickly elevate the capabilities of the entire team.
• Ensures that estimation is not inadvertently biased by the viewpoint, knowledge or experience of any one team member.
• When Product Managers / Owners are involved and participate in the estimation, they gain significant insight into, and a greater appreciation for the issues related to the development of each Story. Similarly, developers gain better insight into the views of the Product Owners. This can be very useful for creating and enhancing good team dynamics and communication.

The Tools

Some teams use cards with numbers printed on them. This can be particularly useful if you are using the Fibonacci Scaling, but is less important than when using the T-Shirt Sizing (SML) or Scale of 1:5 which I use. I prefer to not use cards because I don’t want to get bogged down at the beginning of a session while people are finding or sorting their cards. It is not a coincidence that I use two vectors and a scale of 1…5 for everything including Ranking of Stories, Defects, Risk and Effort. Simply put, we have five fingers on each hand (at least most of us do), and we have two hands. With no muss or fuss, all team members can walk into a Planning Meeting prepared with all of the tools they need, as long as they can count to five and know which hand is their left hand, and which hand is their right hand. As well, because our result (Effort) is the product of two numbers (Complexity x Size) we keep things nice and simple having nothing more complex than the ten fingers we were born with.

No Bluffing

It is up to the moderator to make sure there is no bluffing. Even though this is called Poker, honesty is the rule. It should be no one’s objective to “be right”. Rather, you are attempting as a team to create accurate estimates – not win the game.

The Six Steps

1.     Moderator Reads Story Description

The moderator reads the story description to the team.

2.     Ask Moderator Questions

The team asks the moderator questions about the Story and the moderator provides clarification about the Story. The moderator makes sure that the questions are relevant to the estimation of Complexity and Size. Q and A should be no longer than two or three minutes. Ideally you want to stay under two minutes per Story.

3.     Estimate Cards (Fingers)

The moderator finalizes discussion on the Story and asks each team member to make their estimates of Complexity and Size (see previous blog post for details) and write down their estimates for both Complexity and Size on a piece of paper or on their notepads. Estimates are not to be revealed until everyone has made their individual determinations.

4.     Show Hands Simultaneously

Once all team members have decided upon their estimates, the moderator asks the team to hold up their hands with the number of fingers on their hands representing Complexity and Size. For ease of processing the results make sure that the team has already decided upon consistent use of hands for both Complexity and Size. I use the left hand for Complexity and the right hand for size. It doesn’t matter which you choose as long as your team is consistent.

5.     Discuss Low / High Estimates

If everyone has exactly the same results (highly unlikely) then there is not much to discuss other than the possibility that there is a groupthink happening or that something is being missed. Assuming that there are differences of opinion, discuss the high and low values for both Complexity and Size. The objective is to tease out the issues that are perhaps not being considered by all team members. This is where team diversity is essential. The broader the range of experience of the team members, the more value there is to the exercise.

6.     Consensus or Pessimist Wins

Timebox the discussion and move to a conclusion. You are attempting to gain consensus, but consensus is not always possible. Decide as a guideline how you will deal with situations where you cannot achieve consensus. You really have three options:

1. Take the average. I don’t like this approach because in my view it tends to bias the results towards the optimistic view (see previous post) and may lead to underestimation.
2. Accept the most pessimistic estimate. This is my preferences as in my experience, estimation tends to be optimistic and to counter that bias, the pessimistic view seems to balance things out. This isn’t a license for the people with a consistent “glass half empty” view to rule the day, but it is a means of counteracting people’s generally, and natuarlly optimistic estimation tendencies. Your team may have a different bias so adjust accordingly.
3. Defer discussion to a separate meeting on particularly gnarly ones. I do this as a last resort. I find it is best to keep the estimation in your planning session. If you find a great deal of disagreement on many of the issues, then decide to consistently go with the pessimistic view and then review the estimates in your retrospective. Use that information to recalibrate. I will talk more about recalibration and velocity in my post next week.

7.     Finishing Up

Once you have made a decision, write the Complexity and Size numbers in the upper right hand corner of the story card. Multiply the two numbers together. The product will be a number in the range [1..25]. Using the colour coding from the previous post, you now have your estimate of effort. Repeat this process for each of the remaining Stories in your Sprint.

Conclusion

Agile Planning Poker is a quick, simple way to estimate the Effort required to complete your Agile Stories. It engages the team and encourage them to bring into the open issues that might impact Effort estimates that may not otherwise be discovered or considered. With a new team, or when you are introducing this process to your organization, don’t be tempted to size all of your stories right away. For the first two or three sprints, size the Stories in the Sprint you are about to begin. Use your Sprint Retrospective to recalibrate each time until you are comfortable with your consistency. The consistency is important because it will be the basis for calculating Velocity – more on that next week.

As usual, I look forward to your comments.

Old card players never die, they just shuffle away.  (Author Unknown)

Michael

The best we can do is size up the chances, calculate the risks involved, estimate our ability to deal with them, and then make our plans with confidence. (Henry Ford)

The greatest of all gifts is the power to estimate things at their true worth. (François de la Rochefoucauld)

The Problem

Estimation is the calculated approximation of a result which is usable even if input data may be incomplete or uncertain. (Wikipedia)

Estimating the size, effort, complexity and cost of software projects is possibly the most difficult task in all of Software Development and Project Management. Even estimating the time required to complete seemingly small and straightforward tasks can be annoyingly, or even dangerously difficult to do. Software Project Success is determined in large part by the ability of the team to meet stakeholder expectations. To be predictive, you need data and most prediction models typically use historical data as the basis of their forecasts. Stock market analysis and weather forecasting are classic examples. In spite of mountains of historical data, advanced algorithms and supercomputers to perform the calculations, weather forecasters are accurate less than 50% of the time. There is even more historical data, incredibly sophisticated algorithms and and nearly the same computing power available to market analysts. In the markets, success is somewhat better for some than others, and fortunately, to be successful in the stock market, you need only be right more often than you are wrong. Some win very big, but most, however, struggle to hit that 50% mark.

There are aspects unique to software development that makes software estimation inherently difficult and different from other forms of forecasting. Some of the reasons for this are obvious while many are not. When I set out to write this essay, I planned to speak in depth about the reasons for this, but soon realized (not without a bit of irony) that I had seriously underestimated the scope and complexity of the task. This topic alone could easily be the basis of an entire book, and there is a lot of information available on the web if you are really interested. So instead of focusing on why we fail so miserably at estimating software development effort, I will simply focus on a purely pragmatic view of how we can do a better job of it in Agile projects. That being said, here are just a few of the factors that can impact the accuracy of your estimates:

• In 1979 Kahneman and Tversky found that human judgment is generally optimistic due to overconfidence and insufficient consideration of distributional information about outcomes. Further, risks typically are underestimated and benefits overestimated. This is a human bias resulting from our “inside view” of the project.
• There is a human tendency for us to want to please others and in so doing we bias our estimates optimistically in the name of pleasing our stakeholders. Changing the estimate does not, however, change the amount of work that needs to be done, and in the longer term, shortcuts rarely turn out to be shortcuts. Ultimately we disappoint the stakeholders by delivering late.
• External forces (budget, customer deadlines, completion, etc…) pressure us to complete things as quickly as possible. Again, this is an external pressure that creates internal tensions which the team tries to satisfy. It usually distorts the estimates, but rarely changes the reality.
• Things change – the project requirements shift, the client needs more features to be added or removed.
• Technical uncertainty causes us to sometimes take a wrong path and have to redo work.
• The tools and technologies are constantly changing, causing developers to continually learn and adapt to the latest releases. In each new release of tools that they use from vendors, they will encounter bugs that are fixed, potentially causing old workarounds to break, while at the same time introducing new bugs.
• Interruptions and distractions affect productivity: Noisy workplace, ineffective meetings, poor lighting, uncomfortable seating, inefficient processes, etc…
• In many, if not most cases, there is no baseline. How can you estimate how long it will take you to do something that you’ve never done before? This leads to the question: If you’ve done it before, why are you doing it again?
• The tasks are often too many, too large and too complex, and with too many interdependencies to fully understand their implications.
• Software is part science, but a large part of it is art. Because a lot of it is art, creativity and productivity of individual team members will vary dramatically and the quality and quantity of their input will vary correspondingly. Somewhat counter intuitively, the productivity is not related to levels of education or years of experience. Given two individuals with essentially identical education and work experience, researchers have measured differences in productivity of as much as 100 times. These are the intangibles of insight, creativity and commitment, and they are far more important than education or certification, but nearly impossible to measure.

One of the most significant influences affecting accuracy of estimates is illustrated in the following example.

Focused attention distorts perceptions of time. I remember years ago reading about a study that was performed on highly trained fighter pilots to determine how their ability to estimate the passing of time was affected by their degree of mental focus applied to the tasks they were performing. As I recall, the test was set up so that the pilots were placed in flight simulators and the people running the test started a stopwatch and requested that the pilot indicate when ten minutes had passed. There were several scenarios of varying complexity that required pilot engagement ranging from routine in-flight functions like communication with the control tower to full air combat simulation. As the complexity of the tasks and corresponding need for focused attention increased, their perception of time became increasingly and dramatically distorted. In the simple task tests, the pilots routinely estimated the duration within a few seconds accuracy. At the extreme end (full air combat simulation) the estimates were dramatically off – sometimes in excess of 300%. In other words, when the greatest attention and focus was required, highly trained pilots let as much as 30 minutes pass, while believing that only ten minutes had elapsed.

Productive programming requires similar levels of focused concentration. Thus as a programmer’s focus and corresponding productivity increase, their ability to determine how long it takes to do something declines. For anyone who has done extensive software development this effect is clearly evident, yet very difficult to compensate for in estimates as it is unique to each individual. My wife has for years witnessed me disappear into “The Zone” and has come to refer to this effect as “Programmer Time” vs. “Real Time”. The irony in this situation is that we want developers to spend as much time as possible in The Zone where their productivity is maximized, but while in The Zone, their estimates of time are dramatically distorted. We then somehow expect them to use this hugely distorted perception of time as the basis for their estimates.

What Can We Do About It?

Clearly we cannot contain and/or compensate for many (if not most) of the factors that influence the accuracy of our estimates. That being said, we still need to have some degree of predictability in our work; “I don’t know” is not a good enough answer. Obviously the larger and more complex the task we are trying to estimate, the more challenging it will be to produce an accurate estimate. We also want to make sure that we pay the greatest attention to the  things that are most important to the success of our project. Fortunately Agile helps us in this respect because appropriate use of the methodology decomposes a project into small units of work, and by definition, focuses our effort and attention on the things that yield the greatest value to the users of the software. Agile practitioners and Scrum practitioners in particular have proposed a number of scales for calibrating estimated effort in projects including:

• Ranking effort on a scale of one to three – one being the smallest, and three being the largest.
• Using a Fibonacci Sequence [1, 2, 3, 5, 8]. A Story ranked as an eight is a Story that is too large to accurately estimate and should likely be classified as an Epic and decomposed into a smaller set of Stories.

There are other methods, but these are the two most common ones that I have encountered. Of note in both cases, the estimates are not produced in terms of units of time. Rather they are merely expressions of Relative Effort. There are several good reasons for this approach, but principally it is recognition of the variations of team dynamics, experience and productivity. For example: Using the Fibonacci Sequence scale, a task ranked as a five for a highly efficient and very experienced developer might take one day to complete whereas it might take a junior developer five days to complete. Alternatively, the same time differences may exist between two senior developers with very similar experience. This may have nothing to do with the overall aptitude of the individuals, but may be due to a personal problem solving style that is more effective in that specific instance. Or one developer may have solved a similar problem in the past that caused the solution of this particular problem to be obvious. Relative Effort is thus a good comparative yardstick.

While both of these methods are effective and widely used, I believe they do not take into account the underlying elements that affect effort and uncertainty. I have thus developed a different model that I find to be very effective. This model is also consistent with the way I develop rankings of Stories, Defects and Risk. If you’ve already read my articles on these topics you will know where this is headed, so here we go…

Developing The Effort Matrix

As I enumerated (in part) above by the multitude of factors that affect our ability to accurately estimate effort, it is clear that accurate estimation requires a multidimensional view to produce accurate and effective estimates. The challenge, however, is which dimensions do we measure? If we were to classify the possibilities using a SWOT (see my article on managing Risk: Five Simple Steps To Agile Risk Management) according to Internal vs. External influences, we can eliminate many of the candidates by simply focusing our attention on the things over which we have influence and conversely paying less attention to those that we can’t. I also keep the vectors to two so as to keep the process as simple as possible so that we actually use the process and don’t try to sidestep it because it is too cumbersome. Using two vectors also maintains a consistency with the other areas of the methodology. Here is what I’ve found works best:

Story Size

Story size is an estimate of the relative scale of the work in terms of actual development effort.

The wording provided here is a suggestion. Develop wording that works best for your team. Remember that these are guidelines – not rules.

Complexity (Story and Technical)

This is complexity of either or both the requirements of the Story and or its technical complexity. Complexity introduces uncertainty to the estimate – more complexity means more uncertainty.

The wording provided here is a suggestion. Develop wording that works best for your team. Not all factors listed need to be met – remember, these are guidelines, not rules.

Effort

Using these two vectors, I determine effort using the following simple formula:

Effort = Complexity x Size

Effort Matrix (click on image to enlarge)

Are The Estimates Accurate?

The Metrics

How do you know if you are doing a good job of estimating Effort? The answer is simple, but it may require a bit of work to come up with the answer. It is a straightforward exercise in basic statistics but without very much math. To determine if you doing a good job of estimating, you need to look for two key things:

1. Distribution: This is a measure of how well the distribution of your estimates map to a bell curve.
2. Velocity: A measure of how many units of Effort are completed per sprint as compared to how many units of Effort were forecast. Velocity is a large topic on its own and will not be discussed in this article. The process of calculating and comparing Velocity to forecasts is part of a larger Calibration process. Calibration is the feedback loop used in the setting of baseline metrics that are at the core of making all of your processes and estimates more predictable. We will use external mechanisms (to be discussed in a future article) to calibrate the Intrinsic measure of Relative Effort (as determined by the team) and use an Extrinsic view to measure Velocity (Relative Effort Compared to Time) and turn it into Time Estimates. This will help us offset The Zone effect in our time estimates and improve the accuracy. Look for a discussion on this in the next few weeks.

Distribution

If you are estimating well, and your Stories are scoped appropriately, then there should be a distribution of Effort approximating the distribution of the classic Bell Curve. While one could go into tremendous detail calculating Standard Deviation and performing other analysis, what we are looking for here is a simple litmus test to act as a reality check on our estimates. This is a quick and easy exercise with a spreadsheet to gather the raw data and chart it. Your distribution should look something like this.

Effort Distribution (click on image to enlarge)

What we are looking for is a clustering of estimates in the range of [4..15]. Ideally, you do not want to have anything in the [1,2,20,25] ranges.

The two most significant factors in how your estimates will be distributed are:

1. Appropriate scoping of stories: If your stories are written so that your distribution is weighted in the [10..20] range, your Stories are likely too large, and or too complex. If the distribution is weighted in the [2..9] range, then they are likely too small or not sufficiently complex. In either case, work through the Stories in successive iterations (Sprints) to re-scope and refine them. The sweet spot is to try and have your stories scoped so that the vast majority of your Stories scale in the  [4..15] range.
2. Accuracy of your estimates: This article defines the structure for managing Estimates. An upcoming article will describe the actual real-world mechanism for determining individual estimates. Look for it in the weeks to come.

Every team and every organization will be unique and everything in this article should be considered a guideline and suggestive rather than prescriptive. As the person leading your team, you will have to work with the team to find the balance point that works best for accurate calibration of your team. In both examples listed above, use an iterative process to discover and refine the mechanisms that work best for your organization.

As always, I look forward to your comments.

Michael

An Open Space Conference for Agile Coaches

Adding Topics To The Grid (click on image to enlarge)

Background

On June 11^th and 12^th I was fortunate to be able to attend Agile Coach Camp Canada 2010 in Waterloo Ontario. The event was held in the William G. Davis Computer Research Centre (“DC” building) at the University of Waterloo. I was invited to the camp by Declan Whelan who is a well recognized Agile Coach and one of the primary organizers of the camp. I checked out the site and saw already signed up were a couple first rate Agile Coaches Michael Sahota and Gil Broza whom I already know as well as a lot of other well known Agile Coaches whom I wanted to meet, so I figured it would be worth attending. I was not disappointed. This blog entry contains my observations as well as some photos and video I shot during the day and a half long event.

Over the years I have attended many conferences but this is the first Open Space conference I have ever attended. As I discovered, Open Space conferences are driven by an agenda created at the conference by the attendees.

You can check out the tweet stream for the conference at: #accca

Friday

Lightning Talks

After the Un-Keynote and a game of Ball Point (a lesson in Scrum and iterative process improvement) came the Lightning Talks.  Lightning Talks are a series of rapid succession 90 second long presentations on any topic related to Agile. Here are some quick notes I made on ideas that struck me during the talks. I don’t necessarily agree with all of the following, but much of it is thought provoking.

• Not a lot has been written about creating aligned optimized teams. Agile is about optimizing teams.
• An important part of the process of optimizing your team is to get it invigorated and focused.
• Core hours – A concept of setting aside 5½ hours during the working day where developers do not check e-mail, Twitter, Facebook or take phone calls. This part of the day is focused solely on the project. Outside of the Core Hours, developers can spend their time on those other things that are distractions. Focused attention minimizes distractions, keeps people focused. Check out “Does the Internet Make You Dumber?”
• The results are not the point. Check out Igniting The Third Factor by Dr. Peter Jensen the Sports Psychology Coach of the gold medal winning Women’s Canadian Olympic Hockey Team. It’s about getting teams excited and delivering under budget, ahead of schedule and exceeding expectations.
• Many of the metaphors in coaching and leadership including martial arts metaphors such as Sensei are silly, a waste of time and irritate people. Arguing the metaphor misses the point.
• Pop psychology – Left Brain (artistic) / Right Brain (analytical) is old neurology and it is wrong.
• Listening is the willingness to change. You don’t necessarily need to change, but you need to be open to the possibility.
• The Agile Manifesto still holds true.
• What about the people in the Agile world? Why are we making people fit into the methodologies? Not everyone can adapt to the new ways of doing things. The Agile coaches need to think about the people, not just the process.
• Testers are often excluded from the Agile process as the Agile process seems to include only the developers. There is a big disconnect between the development team and their story cards and the people testing. What about Session Based Test Management? Testers need to be involved in all phases including charter, scheduling, reviewing of results and debrief.
• Can’t, won’t, wouldn’t, shouldn’t, etc… all victim words and no team can become a high performance team if they use these words. This victimization approach is the team being victims of themselves. “It must really suck when Joe comes in and micro-manages you…” commiserate to build trust, then ban the words from the team. Check out Chris Avery responsibility model – move from the blame game to “What are we going to do as a team to improve?” The Agile Coach must help the team get past this.

Saturday

Sessions

As I mentioned, I have never attended an Un Conference. I was curious about how a successful conference could be held without any pre determined sessions, and without people making presentations. The notion behind an Un Conference is that attendees decide what they want to discuss, suggest topics and then attend the sessions that are of interest to them. If a session isn’t working for you and you are not getting out of it what you want, or if something else may be more interesting at that moment, then move to a new one. This is not just your right to do so, it is your obligation as a participant. Each session has a facilitator to lead the session, but the participants take it in the direction that is of interest to them. Everyone is involved as little or as much as they want. What I observed and participated in was an interesting, engaging and oft times very animated set of discussions on a variety of topics.

Here is a list of the sessions topic decided upon at the conference. Many of the links have summaries of the sessions that were submitted by the session facilitator. At the end of the morning and afternoon sessions, each session facilitator presented the findings to the entire group. Their presentations were limited to three minutes.

Creating Session Topics (click on image to enlarge)

The Grid (click on image to enlarge)

Session In Progress (click on image to enlarge)

Session Results (click on image to enlarge)

Session Summaries

Presenting Session Results (click on image to enlarge)

Presenting Session Results 2 (click on image to enlarge)

Session Summaries

Here is some video I shot of some of the summaries. I apologize for the shaky handheld camera and the sometimes awkward start and end to some of the presentations. I was using a Nikon D90 that I’ve only owned for a couple of days and consequently still have to learn a lot about before I can pull off smooth video. Solution: Fire the camera operator (me) and get a better one…

Session Summary Part One of Four

Session Summary Part Two of Four

Session Summary Part Three of Four

Session Summary Part Four of Four

Closing Circle

Closing Circle Video Part One of Three

Closing Circle Video Part Two of Three

Closing Circle Video Part Three of Three

Summary

I didn’t quite know what to expect from this event, nor did I have any idea what an Open Space conference would be like. My conclusion is that it was a superb event that was thought provoking and that challenged a lot of the preconceived notions of everyone there. For me, one of the most interesting observations was that in a day and a half long conference whose premise is how to manage technology (software) projects, there was no discussion of technology, and only cursory discussion of process or project management. Instead, I experienced a user driven agenda that focused on communication and how to get the best out of software development teams – team being the operative word here.

I was impressed with the passion, dedication and willingness of everyone to maintain an open mind and to share their experience. I left at the end of the day on Saturday with many new ideas, some new friends and an expanded awareness not just of Agile processes, but of the Agile Community. The next Canadian Agile Coach Camp will be held in 2011 in Montreal – I hope to attend, and I hope to see you there.

As always, I look forward to your comments.

Michael

Managing Risk Across Multiple Agile Projects

(From flickr: gagilas)

All human endeavours involve Risk. Software development projects as evidenced by their high failure rates are certainly no exception. A successful project is not the result of luck, but of anticipating, understanding and managing the Risks of the project. A successful portfolio of projects is therefore the result of anticipating, understanding and managing the cumulative Risk of all of the projects in your portfolio.

In my previous post “Five Simple Steps to Agile Risk Management”, I presented a method for Agile Risk Management. Experience has shown me that the approach works well for almost any type of project as well as for projects of pretty much any size. It is, however, a rare organization that has a single project underway at any given time, so while managing Risk in your individual projects is a good thing, we clearly need “more good”. “More good” is to have a comprehensive, consistent and up to date view of Risk across all projects. This article outlines a process for aggregating the Risks associated with a portfolio of projects so that Risk across multiple projects can easily be identified, reported, tracked and managed.

In many organizations, there are monthly or quarterly reviews of a portfolio or program by senior management to ascertain the progress. Risk assessment is an important facet of project management, and assessing and reporting on the Risks of projects is thus an essential part of this regular review. This high level view of Risk must have a level of detail and organization of the information appropriate to the target audience and also for the program manager(s) so both groups can keep tabs on all of the projects in the portfolio. In particular, senior management needs to be able to quickly understand the issues and determine which (if any) require their attention. For these purposes, the full detail of all Risks in all projects is far too much information so we need an effective means of aggregating, grouping and filtering information so that anything unnecessary is removed and the important information is brought into focus.

At a high level, program managers and executives are most interested in the external effects of the projects. In other words, they worry about the impact of the project risk on the company, its finances, customers, partners, reputation (brand), and so forth. Fortunately, the Agile Risk Management methodology I presented in my previous post provides us with everything we need to create such a summary. In that article, I suggested the following categories into which your various project Risks might be placed.

These categories tend to work well in most organizations, but they may not be right for yours so adjust them to your specific needs. In my previous post I also presented a Project Risk Registry for tracking the Risks in a project. Below is an Enhanced Risk Registry that I use in situations where I need to track more than one project, and where I need to report an aggregated view of Risks across those projects.

Enhanced Project Risk Registry

Enhanced Project Risk Registry (Click on image to enlarge)

In this Enhanced Risk Registry, I track how each identified Risk affects the seven Risk Classes (Solution…Scope). Each of these values are the combined Risk Vectors (Probability x Impact = Risk) for the individual Risks identified in the project. For practical reasons, I would not normally include Green Risks in the Registry. Individual Risks can change dramatically over the life of a project, and once a Risk has been recorded, it remains in the Registry until the project closes – even if the Risk turns Green. For example: In Risk SP-006 (Customer issue LX113 resolved) this issue may return because the fix itself was flawed. Keeping it in the registry reminds us to keep an eye on it to make sure it doesn’t return. As well, I maintain an additional column (not shown here) that contains the Risk Mitigation Strategy for each of the tracked Risks. This column will be of particular interest to management as they will want to know what you are doing about managing the identified Risks.

For each of the tracked Risks, I use a simple formula for each tracked Risk that takes the maximum value across the seven Risk Classes. In SP001 we see values ranging from 1 through 25, but the Risk for this item is 25 – the maximum of the seven Risk Class values. The reason I use the maximum instead of an average (or some other formula) is that if all values will tend towards a mean (as an average would do), then the true Risks would be obscured. I mentioned previously, but it is worth restating that the Risk values must be entered by the respective SMEs and not the PM. This ensures that the category expert’s assessment is captured. The next thing to note is that for each of the Risk Class columns, I also calculate the maximum display it by Class below its respective column. This is the Project Risk for each Risk Class.

So far what we have is the SME view of Risk for this particular project, and by now you are probably wondering what Adjusted Risk is in the above chart. This is the PM’s adjustment of Risk for this particular project. This is not an attempt to hide or obfuscate Risk. Instead, it is the PM’s assessment in consultation with the rest of the team to adjust the Risk rating based on additional information that may not be available to the SMEs that the time of their assessment. It may be also an adjustment of the Risk based on organizational focus. For example, a company that develops guidance systems for NASA might place the highest value on the Solution, whereas a company in the rapidly changing world of social networking might consider Time-line to be the most important Risk Class. There is no easy or formulaic way to make these adjustments as they are based on experience (been there done that) of the people making the adjustments, and on the culture of the organization. In no case should such adjustments be done without consultation. Whenever you make adjustments to the Risk, make sure you record the rationale for the adjustments.

Program/Portfolio Risk

Now that you have entered your project Risk in your enhanced Risk Register, expand your assessment by creating a Risk Register for each of the projects in your portfolio. Once you have done that, you can roll up the Adjusted Risk for all of your projects into the Portfolio or Program view of Risk. Here is an example:

Portfolio Risk Registry (Click on image to enlarge)

With this register, we can see at a glance the Risks for the entire portfolio of projects. The comparison can be easily made not only for the Risk of each project, but also comparing each Risk Class across all of the projects. The bottom line called Category Risk is the Organizational Risks across all projects. From this particular example, it is easy for management to see that projects SP and CP need particular attention. SP looks like the whole project is tipping strongly in the wrong direction and project failure may be imminent. CP looks like it has a serious Risk associated with Privacy (think about the recent TJX credit card info theft). AP6 might also be a concern, and the management team may want to have a closer look at it.

No doubt when you present this information, you will be faced with questions about the specifics of the ratings.  Because you have individual project Risk Registries, you will have the information at hand (including SME assessments) to provide as part of your presentation. Keep in mind that you may also be asked about ratings that appear to be too low, and you should be able to explain the rationale for those as well. Again, the SME assessments will be of tremendous value to you in your presentation.

How Often?

Project Risks can change dramatically during the course of a project so a portfolio assessment should be done at least once per quarter – ideally every month. Each time you do a review, you may want to identify one or two projects for a deep dive assessment. The choice to do this has a lot do with the size and culture of your organization, but give it consideration for inclusion as part of the standard process in your organization. Regardless of the frequency, remember to focus on the difficult (usually Risky) things first and if a project is going to fail, then fail early by dealing with those issues early. Your job is to try to beat the odds and not be in the one third of all projects that fail outright, or even the one third that are seriously challenged. You want to be in that top one third and your chances of success are much better if you identify and manage the Risks early. This process will help you stay on top of the issues and thus help you reduce your Risk.

Good luck!

As  always, I look forward to reading your comments.

Michael

Risk Factory (flickr: kyz)

This report, by its very length, defends itself against the risk of being read.  (Winston Churchill)

Background

In my post on Agile Project Charters I outlined the embarrassingly high failure rate of software projects. Success rates today are only marginally better than they were when the Standish Group released its first Chaos report in 1995. Recognizing the tremendous misalignment between project expectations and project results, a variety of tools and methods have evolved to help improve the odds of success. Chief among them is Project Management methodologies. Even with fifteen years of experience combined with improved software development tools and better methods, software project success rate have eked out only marginal gains. This is not a vilification of project management methodologies. Rather, it is a statement that software development is an inherently and increasingly complex undertaking with many uncertainties. With Risk Management, we attempt to identify the things we don’t know (the uncertainties) and quantify them so that they can be managed. This sounds like a paradox – how can you quantify what you don’t know- but it is a paradox we can manage.

Agile Methods such as Scrum are a relatively new entrant into the field of project management. A basic tenet of Agile Methods is that teams produce a continuous series of useable software builds in very short cycles called Sprints. Each build is assessed, issues identified and the backlog of tasks is reviewed and prioritized and the most important tasks are scheduled for the next sprint. It sounds like an ideal approach. For many teams it works extremely well as Agile teams tend to claim higher project success rates than do teams using more traditional methods. There is not a lot of empirical data available that makes effective comparisons of Agile project success rates to other methodologies, but what data that does exist tends to support those claims.

Most methodologies place a fairly high importance on Risk Management. Agile approaches tend to implicitly manage Risk. That might not be a bad approach if the only things that affected the outcome of the project were the decisions that the developers made to implement the solution, but as we shall shortly see, there exist a multitude of factors that can have a significant impact on the success of a project. Further, I maintain the position that explicit Risk identification and management can further improve on the success rate of Agile projects. In this article I will outline a Risk Management methodology I use that is quick, simple, pretty comprehensive and very Agile friendly. As the title of the article implies, I have broken the process down into five steps:

1. Identify
2. Classify
3. Quantify
4. Plan
5. Act
6. Repeat

Oops I lied… there are six steps. Actually there are only five steps but it is worth stating Repeat as a sixth step to emphasize that our Agile Risk Management Process defines a virtuous circle of continuous improvement.

The Basics

Yes, risk taking is inherently failure-prone.  Otherwise, it would be called sure-thing-taking.  (Tim McMahon)

• Risks are influencing factors that might adversely affect the outcome of a project.
• Risk is the direct result of uncertainty. If there is no uncertainty, it is not a risk – it is a certainty.
• Risk analysis is used to help a team understand uncertainty that could affect the outcome of the project.
• Risk management (sometimes called Risk Mitigation) is the plan that the team puts into place to pre-empt, contain or mitigate the effects of risk to a project.

The important thing to remember is that even in simple projects, things can and will go wrong, and that you need to make plans to minimize the impact of those events when they occur.

1. Identify

The Dimensions of Risk

Risk has two dimensional influences. The first Helpful/Harmful is a simple assessment of factors that have a potentially positive or negative influence on the success of our project:

1. Helpful: Factors that advance the objectives of the project
2. Harmful:  Factors that hinder or imperil the outcome of the project.

The second dimension of Risk is the identification of the source of the Risk:

1. Internal: Factors originating inside the organization or within the sphere of influence of the project.
2. External:  Factors originating outside of the organization or project that cannot usually be influenced by the project.

Combining these factors into a two dimensional assessment provides us with the classic SWOT Analysis view of our project: Strengths, Weaknesses, Opportunities, and Threats.  In the diagram below we see the two dimensions (four factor categories) arranged in a matrix with Helpful/Harmful dimension represented as columns, and Internal/External dimension represented as rows.

Strengths Weaknesses Opportunities and Threats

Risk Management is primarily interested in the Harmful column and that is what we will focus on in this article.

Examples of Weakness

• Insufficient resources
• Limited budget
• Aggressive timeline
• Important skills lacking in the team
• Technological uncertainties
• Lack of stakeholder consensus
• Lack of a disaster recovery plan

Examples of Threats

• Rapid and significant changes in the economy
• Pandemics
• Geopolitical tensions
• Economic uncertainty
• Changing legislation
• Changing competitive landscape
• Trade tariffs

Weaknesses are factors over which we tend to have some degree of control. Threats, however, are factors over which we tend to have little or no control. It is important to understand that even though we may have no control over a factor such as a pandemic, there are usually things we can do to manage or minimize the Risk effects on our project.

2. Classify

Each of the Risks needs to be categorized as to the affected area, likelihood and level of Impact it may have on the project. Risk Classes are used primarily for organizing, summarizing and reporting of Risks to management and stakeholders. Some Risks you identify may impact more than one Class, and if they do, they should be reflected in the summaries of those Classes.

The next chart is a list of Risks Classes I typically use. These categories are not prescriptive and you may wish to add others such as Reputation, Environmental Impact, etc… to suit your project or company needs. Solution, Timeline, Budget, Privacy and Security should be of interest to everyone with a stake in the outcome of the project. Resources and Scope are primarily relevant to the development team, but they can have a significant impact on the other categories and are as such included in the set. Some Risks may affect multiple Risk Classes and that effect should be reflected in your Risk Classification. I will show how the Risk Classes are summarized later in this article.

What Do You Assess?

I maintain the Risk ratings of each Story or Defect directly on the Cards. I use the same pattern of recording the three numbers Probability, Impact and Risk Rating and use a highlighter to colour code the risks. At a story level, most risk will likely be pretty benign, so don’t obsess and spend a lot of time on the low risk items. Focus on the ones that are genuine threats. Defects are areas that may require more attention if only because as Defects they likely have higher visibility in the organization. In both cases, write a few details about the Risk directly on the card. An added benefit of having developers assess Risk associated with the Stories and Defects is that it encourage a new dimension for their thinking about the work they are doing and helps them to be cognizant of the effects their work has to the overall success of the project.

Tracking Risk associated with Stories and Defects is insufficient – especially for Threats (factors external to the project) and for any identified Risk that is not a Story or a Defect I use a Risk Register (more on that later). The Stories and Defects that receive a high Risk Rating are also tracked in the Risk Register.

3. Quantify

Great – so now we know what to measure, but how do we go about doing that? If you’ve read my three previous blog posts, you’ve likely already guessed that we will use a matrix based on two vectors. The two vectors we will use in this case will be Probability and Impact. The Risks you identify must each be assessed according to these two vectors.

The assessment of each risk must be performed by the respective SME (Subject Matter Expert). A project manager is not qualified to perform an assessment of system security unless he/she is also a security SME. The same is likely true for assessing Risks relative to system performance, quality and privacy. Scrum is about teamwork so depend on the team to bring their expertise to the table. Another reason for SMEs to do the assessment is that I have in some organizations witnessed political pressure applied to PMs to produce Risk Reports to reflect a particular or desired Risk profile. This may force the PM to game the numbers to produce the desired results. The ethics of such practices are highly questionable. If you experience a situation like this, you’ve got much bigger issues on your hands than managing the Risk in the project and should perhaps consider looking for a new job. Having the SMEs do the assessment does help insulate the PM from such pressures. Once the SMEs have performed their assessments, it is useful to discuss the assessments as a team to ensure that there is a consistent approach and weighting applied across all assessments. This also allows the thinking and assumptions behind the assessments to be shared amongst the team and brings the team’s collective wisdom to bear on evolving potential solutions. It may even uncover additional Risks due to Risk interdependencies.

Impact

The Impact of a Risk is a measure of its affect on the project. It ranges from Minimal (1) at the low end where the consequences would be very small up to Extreme (5) at the high end. You and your team should devise wording to describe each Impact level to suit the realities of your organization. Whatever you decide upon should be consistent throughout the entire organization so at to minimize confusion. The wording should not be viewed as a set of rules – instead, it is a set of guidelines. Here is some suggested wording:

Probability

If there is a very high probability that a Risk may be realized, then it is clear that it should have the attention of the team. Conversely, if there is a very low probability of the risk being realized, then it is likely that it should receive less attention from the team. We thus need to ensure that the greatest attention is focused on the Risks with the highest occurrence probability. The following chart provides a suggested scale for assessing the probability of Risk manifestation.

Enter the Matrix

We now have two Risk Vectors and as we did in the prioritization of Stories and Defects (see my previous blogs), we take the two vectors and multiply them together to obtain the simple product which is the Risk value. Using the same thresholds for Stories and Defects as well as the corresponding colour system we end up with a Risk Matrix that looks like this:

Risk Matrix

4. Plan

Risk Rating

Now that you’ve identified the important Risks that threaten the success of your project, what should you do about them? You can make your Risk Planning as comprehensive as you wish, but like most things in life, the simplest approach is often the best approach. Unlike Impact and Probability Assessment, your wording should not be considered a guideline.  For each of the various Risk Ratings, we want specific things to occur because the risk thresholds are triggers to mobilize the team or stakeholder to take action to mitigate the Risk. Here is some suggested wording for your Risk Planning. The wording you use in your company should be different than mine and reflect the realities of your organization, but it is important that the wording be focused on Actions to manage the Risk:

Risk Ratings

Risk Register

To track and manage Risk on a project I use a Risk Register. To do this, I use a spreadsheet. Each time I do a Risk Assessment (ideally each sprint planning session) I add a new page to the spreadsheet and each page is a Risk Assessment corresponding to a particular Sprint. This way I can track how a Risk has changed over the course of a project. I can also monitor how Risks are added and removed from the Register. As you near the end of the project, you should see all of your Risks gradually move into the green or minimal range. If this does not happen, you are definitely doing something wrong because if you still have Orange or Red risk in the late stages of your project, you have not been managing the Risk and you are rolling the dice on project success. All of the time, effort and money invested up to that point is at Risk of being lost.

Risk Registry

5. Act

Insanity: doing the same thing over and over again and expecting different results.
(Albert Einstein)

First Things First

Act is simply that. It is the implementation of the defined Risk Mitigation Strategies. Well it’s actually not that simple. Human nature is such that we tend to put off the things that aren’t fun, interesting or that might be just plain hard work. This is project suicide when it comes to Risk. It is imperative that you deal with the high Risk items first. Deferring performance testing and finding out a week before implementation that you can’t possibly achieve the requisite transactional throughput may be the death of your project. At a minimum, someone is going to have to do a lot of explaining – don’t let that be you.

Fail Early

This is important

The “Fail Early” phrase is becoming very popular in the world of venture capital. In essence it means figure out as early as possible in the process as to whether or not what you are doing will succeed. These findings are essentially the go-no go for your project. If success is not possible, either stop (kill the project) and move on to something else, or rethink the project and come at it from a different angle. Either way, do the difficult, gnarly, risky stuff up front. An added benefit is that it helps you define the boundaries of your system and sets expectations as to what is possible/realistic and what is impossible/unrealistic. It could even bring to light unrealistic success criteria and the definition of project success many need to change. When this happens, the project may still live, but under a revised and possibly more realistic set of stakeholder expectations. It may also stimulate commitments like a larger budget or access to key people.

As simple and as obvious as this may sound, it is amazing how often such critical, high Risk items are left until the final stages of the project. From my own observations over the years, this is one of the biggest reasons for project failure. Do the Risky stuff first and Fail Early.

5. Repeat

This process is very lightweight and very quick to perform. Identifying Risks early, and implementing appropriate Risk Mitigation Strategies for each is essential to the success of projects. Done properly, it is a continuous virtuous circle of Assessment and Action to constantly identify, manage and minimize Risk.

Your Risk Plan should be reviewed at a minimum quarterly. Better still, your review should coincide with your sprint planning sessions. At your these sessions, you have access to your team where everyone is already looking at the stories, reviewing effort estimate, etc…  You don’t need to do an exhaustive review each time, but pay particular attention to the Risks you are tracking in your Risk Register. Also look for any new Risks that might start appearing as the team progresses through the project and learns more about the challenges. As always, if you discover Risks that are high, deal with them early.

Summary

In this article I have presented a simple, easy five step process for assessing and managing Risk in an Agile process. My next post will approach how you can aggregate the Risks of multiple Projects into a Program view of Risk.

As always, I look forward to your comments.

Defect Card (from flickr - listentoreason)

All software has defects of some sort – we know that. If left unresolved, some defects can have cataclysmic consequences while others are so minor that they go unnoticed by virtually everyone. Like most things in this universe there is a law of diminishing returns when it applies to the correction of software defects. Unless you have unlimited resources to assign to bug fixes, you have to focus your attention on the ones  that have the highest ROI. The question is “How do you make those determinations?” There are multiple drivers in any organization that concurrently push and pull the development team in any number of directions. Those drivers could be the Sales Team, QA Team, Finance, End Users, Customers, and Online Media  such as Blogs, Twitter and Internet Forums as well as Traditional Media. The list seems endless and all of it is important (or at least most of it is). Some issues are much more important than others and you can bet, if a software defect is featured on prime time TV news broadcasts that it will be the first thing your CEO will want addressed. So assuming you don’t have the media telling your CEO what your top priorities are, you need a process for focusing your attention on the most important things first.

In my previous post How To Easily Prioritize Your Agile Stories I presented a simple ranking mechanism for assessing and prioritizing Agile Stories and using those rankings to determine their sequence and into which sprint a Story will be placed. This post proposes a nearly identical mechanism and use of the same two vector matrix. The vectors for ranking bugs, however, are different. Instead of Urgency and Business Value, I use Scope and Severity for ranking defects. this is because Urgency and Business Value do not accurately reflect the fact that there are material differences between adding functionality to a system and fixing something that is broken. To illustrate the difference, consider that while prime time news is unlikely to run stories about features missing from your product, it might run a story about the devastating impact of a software defect in your product. Clearly there are different drivers in play here and thus, different vectors are needed. For the purposes of assessing the priority of software defects, I have found that the following two vectors provide the right balance:

• Scope: How many users are affected or how much of the system is affected
• Severity: How critical is the defect i.e. data loss, data corruption, cosmetic issues, etc…

While these are the vectors I use, you are welcome to use your own as they may more accurately reflect your specific business reality.

Scope

The following table provides suggested wording for the ranking of the Scope of a defect.

Severity

The following table provides suggested wording for the ranking of the Severity of a defect.

Priority

As I did with the Ranking of Stories, I multiply the two vectors (Scope and Severity) and the product of the two is the Priority.

Defect Matrix

Actions

Without a set of predetermined actions associated with each priority range, all that has been accomplished in the shuffling of some paper.This is where many systems seem to fall down.  To be effective, appropriate actions must be determined and described. The needs of your organization might be different from mine, and thus the wording you use in your Actions will likely be different.

Defect Priority Ranking

For consistency, I use the same thresholds and the same colour coding as the Story ranking. As with Stories, I print both vectors and the resulting Priority in the top right hand corner of the card and use a highlighter to colour code the card according to the table below. The scale and rankings map directly to the Story rankings and the result is a Defect Card that allows me to easily mix bug fixes into my Backlog with Stories and balance the priorities between the two types. A weakness of most systems I’ve used is that defects and features are ranked using different methods. Doing so makes it difficult to compare the priority of defects relative to features. This system uses the same patterns for both defects and features, but assesses each according to their relevant criteria (vectors), and produces scales that are completely compatible with each other.

Stakeholder Buy In

Consensus on the meaning of the thresholds is important. Selecting the appropriate wording describing the Scope, Severity and Priority may result in wording different than mine. It is also important to keep in mind that it is unrealistic to think that all possible scenarios can be foreseen and described with this system. It is thus important to remember that these are guidelines, not rules. Once agreed upon, however, I have witnessed how the system expedites the assessment process and sidesteps many situations where stakeholders push their agenda as being more important than anyone else’s. Sometimes simply having assessment guidelines (any guidelines) is the most important part of the process, and if nothing else, it can oft times move potentially contentious debates out of the emotional realm into a more empirical one. If you and your stakeholders can at least agree on the basics, and make the process transparent and simple, you’ve made your life and the lives of your team easier.

I’m looking forward to your feedback. Please leave your comments below.

Michael

What Comes First?

Index cards for Saturday morning chores (alandd from flickr)

You’ve spoken to all of your stakeholders, maybe had a workshop or two, gathered all of the input, defined requirements and converted it all into stories, and now you have writer’s cramp from printing them all out onto index cards. You’ve now taken over the conference room and spread your cards out on the table so that you can organize them and figure what goes into your first sprint. So really – what stories come first? Marketing, Sales, Finance, and Operations all have their priorities. Your lead architect says that none of it can be done until the architecture is nailed down – and then of course there is the CEO and her view of what needs to be done first – Oh, and don’t forget the customers. It’s all important; otherwise it wouldn’t be on your index cards would it? Some things are definitely more important than others – but which ones? Scott Ambler says that 85% of all features built into software products are either rarely used, or never used at all. So as much as some stakeholders might think that a particular product feature is essential, it is highly likely it will end up being one of the 85% that go unused. One of the fundamental reasons for using an Agile methodology is to focus on the things that bring the greatest value, so how do you actually determine which stories provide the greatest value and are thus scheduled first? To be effective and obtain consistent, repeatable results requires a system or process that is quick, easy to use and consistently applied.

The quick answer is to rank all of the cards on a scale of 1 to 10, or 1 to 100 if you need finer differentiation, and then sort them in order from highest to lowest priority. This is a pretty common approach, but it is challenging to apply a single numerical value to what is usually a multidimensional question. Do you rank your stories by the CEO’s sense of urgency, the amount of money a customer will pay you to do it, what marketing says is essential, or to placate a cranky stakeholder? Any of these criteria might be valid, but how do you know which ones are more important than the others, and when does one criterion apply but not another? I found that looking at these choices from a two dimensional perspective, and by having guidelines for the application of each of two vectors makes the decision making process much more straightforward. You could in actually use any number of dimensions or vectors you wish, but using more than two causes the process to get bogged down in the fine details and makes the otherwise simple calculations more cumbersome. Perhaps more importantly, it is very difficult to represent three or more dimensions in a two dimensional world such as a computer screen or on an index card. So the number in this approach is two.

So now that you’ve decided to be merely two-dimensional, what are you going to choose as your assessment vectors? In truth, you can use any two you wish. I use Time as the first vector because time is almost inevitably the fundamental constraint on all projects and it is the one finite resource. Most importantly, Time is the basis for arranging and sequencing our work. Time for the purposes of this methodology, is also thought of as Urgency.

The following table lists a set of guidelines that are suggestions for selecting each one of the five possible Urgency (or Time) vector values. It is important to keep in mind that guidelines are suggestions, and you are free to define your own criteria. Regardless of what you choose, keep your guidelines as general as possible because the more specific and detailed your guidelines become, the less useful they will be.

Urgency

So what about vector number two? I use Business Value. The category and the guidelines are purposely generalized so as to avoid debates about whether or not something is in the list and is thus applicable or not applicable as a criterion. It also leaves rating open to interpretation by different stakeholders so that the numbering system does not get gamed or used as a lever by different stakeholders to manipulate the process. Business Value could thus mean the amount of revenue that might be generated or lost, effect on brand or reputation, performance issues, security concerns, etc… It is up to the team to decide in each case what the Business Value guidelines should be, and how to classify each item.

Business Value

If you run an internal IT shop, some of the Business Value suggestions may not be relevant. If, however, you run a small bakery and your project is opening a new location, these suggestions might all be relevant. In either case, use what makes the most sense in your situation.

Priority

Now what? We have some nice numbers and suggestions for how to match our backlog with these numbers, but what does that tell us? By themselves, not much, but if we put them together, we can use the two vectors to assess the Priority or ranking of the story. To do this, we simply multiply Urgency x Business Value and the product of these two numbers is the Priority. The result is a number ranging between 1 and 25.

Priority= Urgency x Business Value

Priority = Urgency x Business Value

Priority Ranges

I then break the values into ranges with a colour assigned to each range. For each range, I apply a ranking value as shown in the chart below. I use four ranges. Four seems to be enough to cover the necessary set of Actions and it has the advantage of fitting nicely into a chromatic scale from green to red. Experience shows that using five ranges causes people to obsess a bit more about their Business Value and Urgency ratings and slows down the process while not adding any real value. Five steps would also require an intermediate colour like an orangey-red or a greeny-yellow. Forcing people into colour discussions like this detracts from the ease and clarity this system tries to promote. Three steps are not enough – in particular because there is no way to accommodate the red or critical valuation. The particular selection of colours also maps to the mechanism I use for prioritizing bugs which I will discuss in an upcoming issue.

Agile Story Priority Ranges

Using the System

So now you have this nice way of ranking the importance of your stories and a nice colourful chart – how do you use it? At first blush, this may appear to be a lot of work with little value. In practice, this is a very quick and simple exercise and it gets you to a decision point pretty quickly. The fact that it is a system makes it predictable, and the 10 minutes it takes to learn and implement will be recouped in your first sprint planning session.

The way I approach using the system is very simple. I leave a space in the upper right hand corner of my story cards. In that corner, I write the three numbers representing Urgency, Business Value and Priority; always in that order. When prioritizing a story, I do a quick assessment of Urgency and Business Value, and then simply multiply the two numbers together to get the Priority number. Then using a highlighter, I colour the Priority number according to the chart above. That’s it. It should take about 10-20 seconds per card. Some cards will require substantial debate amongst the stakeholders and that might be appropriate, but don’t waste your time on differentiating between green and yellow ratings. Instead focus on the red and orange, and then the obvious yellows. Once you’ve got them all rated, simply sort the cards according to their numeric value and you are done.

As you plan for each new sprint, review the ratings of each card in your backlog and change the ratings as appropriate. Some things will increase or decrease in their valuation, and there will most likely be new stories introduced, but again, do not spend much time at the green end of the spectrum. Instead stay focused on the high priority items

There you have it, a quick, easy to learn, easy to use system that will give you consistent, repeatable results.

I’m looking forward to your feedback. Please leave your comments below.

Michael

That Was Easy!