What is Creativity

Lightbulb (flickr: yikerlarooni )

Don’t think. Thinking is the enemy of creativity. It’s self-conscious, and anything self-conscious is lousy. You can’t try to do things. You simply must do things. – Ray Bradbury

A hunch is creativity trying to tell you something. – Frank Capra

What does creativity have to do with software development – actually a lot. Software development is the process of creating software solutions that have never before been created. If your solution has already been created, then you shouldn’t be doing it because you are either plagiarizing, or you are needlessly reproducing work that you should be reusing.  Not only should your work be original, it should also be useful. Clearly if you are creating something that has never before been created, you are well… doing something creative, and thus knowing how to be creative is clearly a good thing. But before discussing creativity as it applied to software we must first set a foundation and define what creativity is.

Creativity is more than just being different. Anybody can be plain weird; that’s easy. What’s hard is to be as simple as Bach. Making the simple, awesomely simple, that’s creativity. – Charles Mingus

There is nothing new in the world. Actually there is very little that is new, and the things that many of us consider to be new are not in fact new. The iPhone at its introduction contained little, if anything that was truly new.  There is no denying that the iPhone was a revolutionary product when it was introduced, but its genesis was that of evolution, not revolution. Wireless phones, digital cameras, touch screens, GUIs, MP3 players, SMS clients, and installable applications have all been around for years. What the iPhone brought us that was new, was the combination of all of the aforementioned technologies (and more) in a way that had previously never been seen. There were countless other things that could have been included in the iPhone, and many devices already included most of what the iPhone offered. In fact as a phone, the iPhone wasn’t even a particularly good phone. What made it unique was there was so much capability in one package, and it was so elegant and simple to use. Examine innovation in any field be it art, science, theatre, and even software, and you will find that most great innovations were mostly evolutionary, not revolutionary.

So what is creativity? Creativity is a difficult thing to define. You can’t see it, hear it or measure it in any way, yet we know it exists and we often (not always) recognize and appreciate its results. It is that bright light bulb that pierces the darkness, illuminating a solution that is often so simple that you wonder how it could have ever been overlooked. But these solutions are overlooked every day, and it is because we all view the world with our own set of filters that prevents us from seeing what other see.

Many people believe that so called “Creative People” dream up new ideas and new products out of the blue. While that may be true in a few rare cases, it is remarkably uncommon. Creativity is the process of breaking out of established patterns. It is an accretive process that builds upon the many innovations that have already occurred over the millennia. It is the reorganization, recombination or reinterpretation of any number of entities, concepts or ideas [elements] either by altering, removing, or adding new elements to the existing set in new and different ways. The result is something unique and previously unknown to the person or people creating it. It has nothing to do with aesthetics, practicality, usefulness, whether or not you or someone else likes or dislikes the results or even whether or not it has already been done by someone else. All that matters is that it is different than what was previously known to the creator. What is created does not even need to be manifested in the physical world. Physical manifestation is only evidence to the rest of the world of the uniqueness of the creative idea.

Picasso did not invent paint, canvas or even the techniques for laying paint onto canvas. Instead, he simply asked “What if on a single two dimensional canvas I could represent more than one view of a person or object?” Monet’s failing eyesight caused him to use larger blocks of colour so that he could actually see what he was painting. The principles of flight were well known long before the Wright Brothers first took to the air. What they did was find the right combination of bicycle parts, weight, wing surface, engine power, wood and canvas and the right balance point to make the first flight possible. Edison’s introduction of the light bulb to the world was no different. There were actually many people working on the creation of artificial light starting with Joseph Swan who obtained a UK patent for the first light bulb in 1860. Canadian Henry Woodward created his own version and obtained a US patent in 1874. Woodward’s invention was considered crazy in Canada and he sold the patent to Edison who saw the commercial potential and refined the design so that it became a viable product. There is no question that each of these individuals were highly creative and innovative people who were able arrive at solutions that had eluded so many. None of these creations rose out of nothing. It was their specific and unique organization of elements in a way that allowed them to arrive at the solution that had eluded so many that made them successful.

Productive Creativity

Creativity has more to do with the elimination of the inessential than with inventing something new. – Helmut Jahn

It is one thing to be creative, but is another thing to create something that is unique and also useful or wanted. We live in a hyper-competitive world where the race to have the next creative innovation has become extremely intense. Hundreds of billions of dollars are spent worldwide every year on R&D. Even with so much effort being expended by so many organizations, and with creativity itself being nothing more than discovering new combinations of existing elements, it is clear that creativity – the ability to find that magic combination that is both unique and wanted or needed is very difficult, and something that few of us do well.

Vincent Van Gogh was unable to sell a single painting while he was alive, but seven of his paintings are now among the 40 most expensive paintings ever sold at auction. Pablo Picasso with ten of the top 40 is the only artist to have more. Together, these two masters hold 17 of the top 40 spots – nearly half of the total. Two people are creators of nearly half of the 40 most valued pieces of art in history (at least the ones that have come to auction). So why was Picasso’s work revered during his lifetime, and Van Gogh’s work considered near worthless when created, yet now command prices in excess of $100 million. Why do some creative software ideas succeed, while most never see the light of day? In Van Gogh’s case, his timing was just lousy; the world was not ready for his kind of creativity. No doubt timing can also be lousy for technology innovators; Apple failed with the Newton and succeeded with the iPhone. Business factors are certainly relevant, but putting business factors aside, most creative efforts that do not catch on are because the idea is just not sufficiently useful to enough people. Sometimes timing is the issue, but more often than not, the idea is just not of value. The difference between the two is Productive Creativity.

People who are productively creative focus their effort on bringing something of value to the market. Van Gogh and Picasso were both brilliant artists, and each was passionate about his own work. While Van Gogh did not understand how to convert his brilliant paintings into a commercial success (spending time in an asylum and cutting off part of his own ear didn’t help); Picasso did. Certainly there are also artists with little or no artistic talent who understand how to commercialize their work, and some with neither talent or business acumen. Parallels can be drawn with software developers because in many respects, software development is at the intersection of Art and Commerce. Sadly, this intersection is not understood or appreciated as such. I have also encountered more than a few brilliant developers whose contributions are marginalized because they didn’t know how to frame their ideas in a business context or because the business itself regards that software development as purely a technology effort with no art involved. To some degree, I think this attitude is because software development entered the world in large corporations to run payrolls, manage accounts receivable and many other important, but nonetheless mundane tasks. Software development now has very little to do with the data centres, servers and networking that enables it and has more to do with things like human perception, human cognition, language, ontology and taxonomy. Software development is the art of abstraction of things, processes and interactions in the real world into structures in code resembling sentences, paragraphs and chapters of a book. There is an infinite number of ways to develop any software application and none of them are either right or wrong unless the numbers don’t add up. Product Creativity is the art of figuring out ways to create software that are better than what anyone else creates. Anyone who doubts the importance of creativity and inspiration and its intersection with commerce need only look at the mobile space where the Android and iPhone platform each have more than 200,000 unique and innovative applications available.

Fostering Creativity in Your Team

Creativity Killed (flickr: Anant Nath Sharma

But the person who scored well on an SAT will not necessarily be the best doctor or the best lawyer or the best businessman. These tests do not measure character, leadership, creativity, perseverance. – William J. Wilson

The truly creative people in this world are the outliers – they are rare. They are not identified by résumés that say they have ten years of Java experience, or perhaps the equivalent in .Net. These are simply measures indicating that they continued to show up for work each day. Sadly, these metrics are the primary filters used when recruiters and HR specialists begin the search process. The creative people you want on your team are unique in their ability to see the world differently from the rest of us. That unique vision is essential, but it is not enough. What is also needed is the ability to practically apply that unique vision within the structure and processes that are necessary to support the act of developing complex software projects. To be certain, an important part of that is experience. Very experienced people have a body of knowledge that gives them a distinct advantage over less experienced individuals, but the very filters we tend to use end up filtering out the people with the more important broad experience in favour of people with less useful narrow experience.

And it has become a kind of a truism in the study of creativity that you can’t be creating anything with less than 10 years of technical knowledge immersion in a particular field. – Mihaly Csikszentmihalyi

I don’t entirely agree with the previous quote because there are people who have decades of experience who will never be creative, and there are those who simply “get it” far sooner than most. That being said, a sound technical background is important – not measured in years, but measured in how they’ve used their experience to innovate. Sometimes lack of experience can even be an advantage because the lack of preconceived notions can make obvious solutions that are missed by people who are used to the status quo. Experience inherently builds patterns of efficient behaviour; creativity necessitates the breaking of patterns. Understanding “why” leads to new solutions; just knowing “how” simply leads to more of the same.

Not only is it important to find and hire people who develop productive creative solutions, you need to provide them with an environment where they can be productively creative. The onus for this is on your company and on you as the person leading the team. Finding the unique combination of elements that produces the innovative new solutions requires experimentation. Experimentation means trial and error, and thus some of what is attempted will not work. Many corporations discourage such things, but if you and your organization really do want to innovate, there is no alternative path. Innovation does not come from process, protocol, structure, long status meetings or administrative trivia. While structure and process are important (without them anarchy reigns), do your best to free your team from such things and instead, spend that time experimenting and innovating. The benefits will far outweigh any any that could come out of the meetings and administrative noise. Help your team by giving them some structure within which to succeed. Set some objectives for them such as each team member must present their innovation work once per quarter to the whole team. It’s amazing what this kind of peer pressure can produce. It can create a competitive environment where each team member tries to produce the best they can dream up. You will be amazed at what comes out of this.

If creative people are rare, the person who can be creative and function well within a structured process is even rarer. To their great loss, many organizations also have a difficult time working with creative people. Some companies fear creativity, and some simply don’t want anything to do with it. I’ve had conversations with several recruiters who say that employers have come to them and said “Please don’t send me anyone with ideas. I just want someone who is going to sit in his/her seat and program what I tell them to.” Without a doubt, there are other companies that are innovating, while companies like these stagnate and die. If you’ve gotten this far in the article, you are clearly not of that mind, so what do you do?

Here are some things you can do:

1. Try to find and hire those creative geniuses. I would rather have a small team of inspired out of the box thinkers that may be hard to manage than a team of drones. My team of creative geniuses will be more work for me to manage, but we will always finish first, and we will always have the best solution.
2. Make your team as diverse as possible. Try to find the best group of people possible that includes the broadest range of backgrounds as possible. Mix Java programmers with .Net programmers; seasoned pros with the kid just out of school; web developers with main framers; and financial application developers with smart phone developers. The greater the diversity, the more ideas people will be able to bring to the table.
3. Look for people to bring into your team with backgrounds in something completely different like art, literature, philosophy, music or languages. The more things they have done (particularly creative things) the more ideas, unique approaches and experience they have to offer. They still need to be excellent programmers, but excellent programmers with a broader range of experience have more to bring to the creativity table.
4. When hiring your team, pay little attention to things like the number of years a person has spent with a particular technology or worked in a particular industry. Instead, ask them about their innovations and inventions. Find out what they’ve dreamed and created, and why it was successful and what they’ve learned from the things that were not successful. When asking questions like this during an interview, be sure that you explain why. People are trained to filter their responses in interviews to provide what they think the corporation is looking for. Rarely is creativity one of those things, so help your candidates help you to uncover their creative talents.
5. Give creativity a home and a place to thrive. Creativity is a fragile thing that must be nurtured. Give your team one afternoon per week where they get to work on their own idea. Even better, give them an entire day each week. What they produce must be related to the business you are in, and they must present their idea to the rest of your team for possible inclusion in your project, or maybe something entirely new. Either way, it must be something its creator believes has value to the group and the organization as a whole. Celebrate the successes, and have some fun (not ridicule) with the ones that are odd or off the wall. Countless studies have shown that the companies that formally support creativity through such programs are consistently the ones with the best new idea, the highest productivity, and the best morale. Google is one of them. If you are running an Agile team – actually any development team, then schedule the creative time. The work that comes out of this time is important  (it could be the big break your company needs), so don’t let all of the other urgent things encroach on it.
6. Encourage experimentation, and yes, failure. Most organizations tend to have little tolerance for failure of any sort, and this causes people to take the safe route, to not experiment, and to not be creative. Failure means that something new was tried, and not all new things succeed. If nothing new is tried, then all you are doing is repeating what you’ve already done, and you can bet that while you continue to do the same thing, the competition will be innovating while you are not. Minimize the downside potential of the failure by limiting the scope of what is being attempted to a series of smaller pieces that can each be assessed to determine if continuing down that path is worthwhile.
7. Always ask Why, and then ask Why again. Keep peeling back the layers of onion until you find the real Why. You will often find that this makes the right solution evident, and this will provide your How.

Clearly the companies that innovate are the ones that win. This is especially true in the field of technology. For a company to be innovative requires both innovative people, and a company that encourages and even helps their people to be innovative and creative. This means you as a manager need to hire the best, most productively creative people you can find, and you need to create an environment in which they can be creative. Do this and your company has a decent chance of being a leader; not doing this means that you and your company will always be followers.

As always, I look forward to your comments.

Michael

An Outlier (Flickr - Christian Beirle González)

High Performance People

“Imagination is more important than knowledge.” Albert Einstein

Software development is an inherently human endeavour. It is not a process (although process helps), and it is not a science (although science also helps). If it simply were a process, then anyone could in a very short period of time learn to do it. If it were a science, you could give the same problem to a dozen programmers to solve and you would get the same results from each of them. Experience tells us that neither of these things is in fact true. If it is not a process, and it is not a science, then what is it? Before answering that, consider that no two software solutions are ever alike – or at least they shouldn’t be. If you find that you are creating a software solution that is exactly like something else that already exists, you should seriously reconsider what you are doing because either you are either a) infringing on someone else’s copyright or patent or b) wasting time and money solving a problem that someone else has already solved. Implicitly this means that for each new project you begin, you must “create” a new solution. In other words, you are dreaming up a new way to solve a problem. This means that to “create” a new solution, you need – well… creativity. Creativity is an intangible. It is highly unique to each individual. It cannot be measured and for the most part cannot be taught. We often refer to this intangible as art.

We tend to think of art in a very narrow way. We all know of art in its traditional forms: painting, music, dance, theatre and so forth. Take a look, however, at business, sports, automobile design and many other endeavours and you will see how the very best like Wayne Gretzky, Michael Jordan, Richard Branson,  Steve Jobs and Enzo Ferrari have elevated what they do to an art. Even in traditional sciences like physics, who can deny the inspired creativity of Einstein and Newton. These are just a few of the truly exceptional people the world has seen. They redefined what was possible and the likes of them will never work for you – but not for the reasons you might think. They are the outliers – the ones that don’t fit neatly into nice little packages with items that can be ticked off on their résumés and if you were reviewing their résumés before they achieved their great successes, you would almost certainly pass over each one of them.

Hiring Processes Eliminate The Wrong People

“Insanity: doing the same thing over and over again and expecting different results.” – Albert Einstein

The processes used by most organizations to hire and build software development teams are not designed to find the outliers. They are in fact designed to weed out the outliers and instead create a homogenous mix of sameness. Most companies make their hiring decisions based on some mix of the following criteria:

• Does the person have a degree
• Number of years of work experience
• Number of years experience with a particular tool or language
• Has the candidate built something like this before
• Does everyone on the team give the person the “thumbs up” when they interview the candidate

What you have here is a recipe for homogeneity – a soup of sameness where you have unwittingly filtered out anyone who can bring new ideas to your project. You have blindly targeted the middle of the Bell Curve of innovation and have excluded the possibility of bringing fresh new ideas to bear on your project. As well, none of the criteria listed above can in any way be correlated to success either of an individual or a team. All you have is a specious set of criteria that gives you (the hiring manager) a means to cope with the process of hiring and a way to save yourself if the candidate turns out to be a bad hire… “Well he had a great résumé and the team really liked him…”

Hiring top notch people is more important on Agile teams because so much more of the decision making is handled by the development team. They need to make good decisions (not necessarily safe decisions) and they need to devise creative solutions. If you are in a software company, the need to find the outliers and the artists is even greater. You cannot afford to create a soup of sameness.

I have over the years interviewed hundreds of people. I have in turn been interviewed maybe 100 times. It is my experience that very few people know how to interview prospective candidates, and the truth is: why should they? The typical hiring manager has to hire a new employee a couple of times each year. They have no training in how to conduct an interview, what to look for in an interview or how to assess the results of an interview.  So what do they do? They fall back to things they can put in lists and check off and it is likely the same list of questions that was used to assess and hire the existing team members. They then turn to the others on the team (who likely have even less hiring experience) and look for confirmation. How can you end up with anything but more of the same of what you already have?

Stop Hiring Dogs

The fear of things that are different is a tremendous driver for people. I recently attended an Agile Conference where I struck up a hallway conversation with a group of Attendees. I asked the question “On an Agile Team, how do you deal with a super bright architect who simply gets things before anyone else and has that crystal clear vision of what needs to be done?” I was shocked by the response. There were about ten people in the conversation and the comments ranged from “get rid of him – he will unbalance the team” to “it’s all about the team and he needs to go” to the most extreme “cut him out like a cancer.” The reactions troubled me for a couple of reasons: 1. there was an automatic assumption that really bright people can’t participate in a team environment and 2. the nice friendly team atmosphere where everyone agrees on everything was the most important thing. I believe that both assumptions are dead wrong.

A few years ago, I stumbled upon the following article. It was written by Miki Saxon of RampUp Solutions Inc. and posted on the Notre Dame Mendoza College of Business website. The article is unfortunately no longer available but I was lucky to have saved a copy and I am thus be able to share it with you here. I think the article provides a valuable perspective.

A few days ago an executive I’ll call Dan called me to bemoan the lack of creativity in his organization and I told him to stop hiring dogs. He informed me that he had great people and when I agreed he demanded to know why I called them dogs.

The problem is that Dan hires people he likes who fall inside his comfort zone, so his organization gets along well. And while it’s well diversified from an HR point of view it has little mental diversity. It’s a happy place, kind of like a dog park with a large variety of breeds and mutts all well socialized to play together and those that don’t play nice are asked to leave. That kind of peace may be good for a dog park, but it can mean death for a company’s innovation efforts.

Unfortunately, people have been moving away from thought diversity for quite awhile now. The attitude has a name, homophily, it’s been around forever and it’s an attitude I run into frequently when it comes to hiring, although it’s rarely intentional. It’s a word you should learn just so you can avoid it. It’s what makes it difficult for Dan’s people to be creative; when something is suggested it’s often accepted with little discussion and even when a counter idea is presented it has similar DNA.

It’s not that Dan needs to toss a bunch of cats in the middle, but he does need to start hiring people that come from a variety of companies and industries, with different experiences and with whom he may not be as comfortable as he is now. It also means that Dan will have to work harder. Not because his people won’t get along, but because diversity of thought does foster exactly what Dan wants—higher creativity.

Creativity means multiple ideas with no common DNA leading to passionate champions, intense discussions and heated meetings. Dan will have to actively manage the various elements if he wants to harness that energy for the benefit of the organization.

Whether you consider yourself a manager, a leader or a manager-who-leads the more mentally diverse your organization the more difficult to manage, but the rewards are high for doing it well.

How I Hire Great People

When I hire people, I rarely look at their formal education. I find it is largely irrelevant unless they’ve done something out of the norm, in which case I pay closer attention. Richard Branson, Michael Dell, Steve Jobs, Paul Allen, Larry Ellison and Bill Gates seem to have all done fairly well without a formal education. The best programmer I have ever hired did not finish grade 11.

Neither do I look for people who have worked on projects that are the same as the current one – in fact I often specifically look for people who have not. Even better, I look for people who come from different companies, different industries and have worked with different technologies. Why – because I want new thinking and I want innovation. That never comes from doing more and more of the same. I remember debating the VP of a company in the financial services sector who was trying to get a new product off of the ground. He was pressing to have the best, most innovative product on the market. He wanted new thinking, and to get it, he was trying to hire the lead architect from his direct competition. I call that irony.

Years of experience is somewhat relevant to me, but more relevant is diversity of experience. My father, who spent decades, managing a data processing department once said about hiring: “So this guy has ten years of COBOL experience. So does he have ten years of experience, or one year of experience repeated ten times?” Ten years with a wide range of technologies and projects is far more valuable. How many years of experience with Java or C# did you look for in your last hire, and be honest – how much relevance did it have to the success of the person in his/her new role?

“Truly successful decision-making relies on a balance between deliberate and instinctive thinking.” — Malcolm Gladwell

So if you can’t follow the cookie-cutter process that most companies use, then how do you hire great people? In large part, avoid the pitfalls I’ve mentioned and try some of the suggestions I’ve made above, but more specifically, try doing things that will help you to find the people that are different – not the same. I mean finding the outliers. I look for the creative thinkers. I look for diversity – not homogeneity. I look for people who have innovated and who look like they will continue to innovate, and I look for people who can really think. This is a tough thing to uncover, but to do this I ask questions in the interview that place their thought process in the open. Here is an example:

I am going to give you a list of three things. I want you to put them in order of importance, from most important to least important, and I want you to tell me why. The three things are:

• Hard work or work ethic
• Technical aptitude
• Communication

I repeat the question to make sure it is clear, and that is all I give the candidate. There is no right or wrong answer to the question, but I do not let onto that because I am studying their problem solving process. I want to see how they get to the answer, and I want to know why they got there. I have asked this question hundreds of times and as such, I have been able to calibrate the responses. It is amazing to observe what happens. I’ve seen everything from sheer panic to an answer rattled off in seconds. Others might take a full five minutes to provide a response. What I like to see is the candidates ask questions of me – questions like “is the question asked as it applies to this particular role, or is it a general question as it might apply to the organization?” This demonstrates the person trying to place the question within the right context. Placing decisions in an appropriate context is a subtle, but very important component to the decision making process. One candidate looked me straight in the eye and said “There is no correct answer to this problem is there?” to which I responded “You are correct.” He continued “So what you are doing is trying to see ‘HOW’ I answer the question – correct?” To which I also said “Yes.” He went on to provide a superb answer and not surprisingly, gave a great overall interview. I ended up hiring him, and he was a superb addition to the team.

Even though there is no right or wrong answer to the question, I do have a preferred response. It generally goes something like this:

1. Communication. Because without a complete understanding of the challenges at hand including understanding customer needs and having great team collaboration, you can work hard and brilliantly solve the wrong problem.
2. Hard work. Because hard work and a strong work ethic will drive you to learn anything you need to accomplish the task at hand.
3. Technical aptitude.  Although still very important it is lowest of the three because it is the easiest to acquire and compensate for.

Incidentally, the candidate I mentioned above who figured out what I was up to, listed the items in reverse order, but he did so with brilliant reasoning and a hypothetical context that he supplied. The truth is that he was extremely bright, and very creative – one of the outliers I was looking for, and I suspect he was playing me at my own question to see how I would react. He produced great work and it was the art I was seeking.

The best teams I’ve led or participated in have had a very high percentage of the team members that were outliers. To be sure, they were in some respects harder to manage. Sometimes tensions ran high, but everyone had great respect for each other. There was always brilliant work being produced, and the pace at which work was produced was often astounding. These teams were fun and stimulating to work on because each member was continually raising the bar with innovative ideas, and thus challenging the rest of the team to contribute at the same level.

So if you want to build a great team, find great people – find the outliers – not people who are the ingredients for the soup of sameness. Develop questions for your interview process that help you filter out average instead of filtering out brilliant. If you have questions you’ve developed that you think help you uncover the outliers and the artists, please share them in the comments below so we can all benefit.

As always, I look forward to your feedback.

The Simplest Solution is Almost Always The Best Solution

Art Gallery of Ontario Staircase (Flickr - paccpass)

“Everything should be made as simple as possible, but not simpler.” – Albert Einstein

Who the heck is Occam and what do he and his razor have to do with software design– oh… and creating software is a science, not an art – right?

There is much written about the varying levels of productivity of programmers. Research has shown that productivity differences between two programmers of nearly identical experience and education can easily vary by an order of magnitude (10x) and differences in debugging productivity can vary by a factor of 25x. The very productive ones write simple, elegant, easily understood code that accomplishes a lot with very little. These “Star” programmers “get it” in a way that most people can’t even comprehend. That being said, there are things that Mere Mortals can do to elevate their own productivity levels to approach that of the Star. In part, the Star programmers accomplish more by doing less. This is at first counter-intuitive, but if you consider that if a programmer spends time writing unnecessary code, or has to continually refactor code because the design doesn’t easily handle changing requirements they will have less time to produce the “needed” code and thus be less productive. The effectiveness of the top produces can be achieved in part by embodying the principles of Occam’s Razor and doing less.

William of Occam was a 14^th century Franciscan philosopher and monk who knew nothing of software. He did, however, understand the issues of complexity and the corresponding effectiveness of simplicity in all things. The principle behind Occam’s Razor admonishes complexity in favour of simplicity and accomplishes it by shaving off anything that does not directly support the conclusion. Occam’s Razor was originally stated as:

“Plurality should not be posited without necessity”

or

“Entities must not be multiplied beyond necessity”

Modern interpretation is often quoted as “The simplest solution is almost always the best”. The modern interpretation, however, misses the important point that “simplest” as meant by Occam means the “fewest assumptions”. The principle is also often referred to as the Principle of Parsimony: “The use of the simplest or most frugal route of explanation available”.

The statement “The lady down the street is a witch – she did it” is a very “simple”, unambiguous statement that introduces very few entities, and provides a simple explanation – or so it first seems. Examined closely, the statement, is built on a few very large assumptions and Occam would not have approved. The statement illustrates Occam’s notion of plurality and the unnecessary introduction of entities to support a conclusion. It assumes: 1) Witches exist 2) The lady is a witch 3) She did it 4) Her supposedly being a witch has something to do with whether or not she did it (whatever “it” might be). In truth, none of these elements are connected in any way to her purported guilt, but accepting them as support without questioning their veracity will likely lead to an incorrect conclusion – her guilt. History has shown us that many innocent people have been sentenced to horrible deaths based on such specious reasoning. When it comes to software design, are you sure your reasoning is not similarly flawed?  Are you certain that everything you are doing is truly being done in support of the project objectives? Have you unnecessarily introduced elements thinking that they support your planned outcome, but when closely examined they are not?

Fortunately one of the advantages of Scrum and other Agile Methods used in software development is that we focus on the most important things first. Specifically, we concentrate less on the technology and more on delivering value to the end users. Estimates vary, but in general it is thought that 80% of the functionality built into software application is either never, or rarely used. Using Scrum and Agile methods ensure that we are constantly assessing and re-prioritizing features and functionality as we go. Often, as we approach the end of the project, we realize that much of what was originally on the list of requirements is no longer relevant, but because we have delivered the elements of highest value first, stakeholders are often satisfied earlier in the development. Often they will decide that many of the capabilities originally requested are no longer relevant. Good development methodology that focuses effort on delivering value goes a long way towards the principles espoused by Occam, but it’s not enough.

The business schools reward difficult complex behaviour more than simple behaviour, but simple behaviour is more effective. -Warren Buffett

Software solutions are ever increasing in size and complexity as the requirements of application solutions also expand. Complexity introduces uncertainty, and uncertainty means risk; the greater the risk, the higher the probability of failure. Plurality (adding Entities) in support of a conclusion does not simply add complexity, it multiplies the complexity. This is illustrated by the following exercise: Take a blank piece of paper and a pencil and on the paper draw two dots with a straight line connecting the dots to each other. Now add a third dot and connect it to each of the other two dots. You should now have three dots and three lines. Now add a fourth dot and connect it to the previous three dots. You should now have four dots and six lines. Jump to a 100 dots and you will have 4,950 lines or interconnections. The number of lines connecting the dots to each other increases dramatically with each new dot added. There is a formula to calculate the number of lines connecting the dots. If you really want the formula to calculate the number of connections it is:

 nCr = n!/r!(n-r)!

where n is the number of items in the subset – 2 in our case because we are looking for the number of connections between any two dots, and r, which is the number of dots.

The complexity here is not in the dots, but the fact that each additional dot multiplies the number of lines that connect the dots. If you think of the entities in your software (classes, object instances, modules, components, databases, tables, etc…) you can easily see that the complexity of the system increases as the number of entities increases. But wait a minute… in software these things don’t all connect to each other. You are probably right but the real question is “Are you sure?” Can you say with absolute certainty that a change in any one entity will not have an impact on any other entity? What happens if you have hundreds or even thousands of these entities – far more than any one person can realistically understand – are you still sure that you understand all of the interactions? What about the collective effect of all of these entities? Now what if the system is not well designed with an appropriate separation of concerns and is not self-documenting with well chosen class names thus causing the function of each of the entities to be less clear? The “what if’s” could be added endlessly and they increase with the number of entities you add. These “What if’’s” represent assumptions in Occam’s view, and the more assumptions you have, the more complex your system and thus the higher the risk and the greater probability of failure.

My assertion is that “Assumptions are unknowns masquerading as knowns” and as such represent a hidden, and very real danger to the integrity of your system and the probability of success of your project. Assumptions are a necessary part of life. I assume that the sun will rise tomorrow; I assume that my Internet connection will work; I assume that milk will be available in the grocery store the next time I go shopping. Assumptions help us to bundle up the complexity of life into nice neat little packages that help us to manage the massive amounts of information we must deal with every minute of every day. Assumptions are very useful, but incorrect assumptions can have catastrophic effects. Clearly we can’t test every single assumptions and this is where experience has no peer. Knowing which assumptions to test comes from a heuristic “been there done that” body of knowledge and there is no substitute for that experience.

So it’s clear that complexity must be reduced and assumptions challenged – but how? This is where the art is involved. If you look at the field of architecture, you can find many strong parallels with the world of creating software. Both fields are very technical. In both fields simplicity and elegance and the facility with which humans interact with the result are key elements of success, and in both fields there are many ways and forms in which to deliver a solution – all of them satisfying the requirements. Some solutions, however, are clearly superior to others, and some are simply brilliant. In architecture, the results of the “art” are usually more evident and very clearly visible because they are on the exterior – and well – the seemingly gravity defying cantilevered design doesn’t fall down. There is no less inspiration, intuition or vision in great software than there is in great architecture, but it is not so clearly evident. A great software solution is simple, elegant, enjoyable to use and it doesn’t fall down. I believe it is no coincidence that the people who mastermind the designs in both fields are called architects, and in both fields, brilliant results require uniquely talented people. In some ways this notion is at odds with the Agile Approach as Agile focuses heavily on the team.

Most of the fundamental ideas of science are essentially simple, and may, as a rule, be expressed in a language comprehensible to everyone. – Albert Einstein

I have worked with several terrific architects over the years. They’ve had amazingly diverse education and work backgrounds. Perhaps the best I’ve worked with is a guy who did not have a computer science degree; in fact he didn’t finish grade 11. One of the traits I noticed that was common to him and the rest of the talented architects I’ve worked is that they ruthlessly asked the question “Why”. They questioned the key assumptions of the design, looking for conclusive justification for both its existence and its functions. A great architect and a great software architect will both have a solid reason for everything in the design. The Why for each element will be well reasoned, and can be simply explained. If the existence and function of an entity cannot be simply explained, then it is likely that much like the woman at the end of the street who is reasoned guilty based on the assumption that she is a witch, your design likely suffers from the “Plurality” admonished by Occam. Something I’ve noticed by constantly asking Why is that many requirements are expressed in terms of a solution, not the actual requirement. Asking Why can remove multiple layers of assumptions and take you to the real requirement which may be very different from what was originally stated. Entities with clear purposes for their existence tend to fit together well in large designs because the absence of duplication (plurality) contributes to a simpler and more elegant design – especially on the grand scale.

Clearly there is far more to good software design than simply using Occam’s Razor, and not all teams can have a great architect. For that matter, not all teams can even have any architect, but everyone on the team should strive to think like a great architect and of everything in the design ask “Why”. Like Occam, ruthlessly shave off the unnecessary entities and produce designs that are simple, elegant, easily explained and contain no assumptions.

A word of caution: Occam used the Razor to posit that “God’s existence could not be deduced by reason alone.” This was too much for Pope John XXII who excommunicated poor William. So temper your questioning with sound judgment so that you do not get excommunicated from the team.

They say the world has become too complex for simple answers. They are wrong. – Ronald Reagan

As always, I look forward to your comments.

Michael

Are We There Yet?

Are We There Yet? (Flickr - Lorant Zsabo)

Are we there yet? – Donkey from Shrek 2

If you are a parent with kids any older than three or four years of age you’ve heard it: “Are we there yet – How much farther is it?” If you’ve ever developed software, you’ve heard the same questions, but this time, rather than coming from your four year old in the back seat of the car as you are driving to Granny’s house, it’s a stakeholder wondering when your project will be finished. You likely know the answer to the question from your four year old, but chances are, you will have a very tough time answering the project stakeholder’s question with any degree of accuracy. The answer to this simple question is extremely important because product launches, project funding, marketing initiatives, budgeting, staffing and so much more all depend on this seemingly simple question.

Why are 21% of companies experiencing schedule overruns exceeding one year, and why are 11% overrunning their budget by more than US $1 million? As I have mentioned in several of my previous articles, estimating how long it will take to complete software projects is extremely difficult. In my view, it is the single most challenging part of the entire software development process. How can we do better? This article builds on the concepts presented in my three most recent articles: (Estimating Effort For Your Agile Stories, Agile Planning Poker, Calculating the Velocity of Your Agile Project) Using these articles as the platform upon which we will base this methodology, I will demonstrate how you can develop a reasonably accurate estimate of the completion date of your project.

Methods

There is a wide range of methods for estimating how long it will take to complete a software project. Just a few of the many ways include:
•    Analogous: Similar projects of a similar size and scope. For projects where you can draw a strong analogy to one (preferably many) projects of similar size, scale and functionality. Likely the most accurate way to do it.
•    Wishful Thinking: This is a remarkably popular method. It is often used when project sponsors or developers are driven by cost or time constraints and don’t want to spend the time up front to do proper planning or estimating. As one might expect, this method rarely (if ever) produces desired results and most often ends in significant overruns or outright failure.
•    Waterfall: The most prevalent method for medium to large scale projects. Statistics show about a 1/3 success rate. 1/3 of projects are significantly late, over budget or lacking significantly in features and functionality. 1/3 of projects fail or are cancelled.
•    Function Point Analysis: This method was in vogue for several years in the 1990s. It breaks a project down into a collection of smaller tasks. Each task is assigned a function point value. A function point has a corresponding time estimate associated with it. The time to complete the project is that total of all of the function point assessments.
•    Mandate: The project manager is told when the project will be completed, what the budget will be, and what features and capabilities will be included in the final result. This is really just another form of Wishful Thinking only worse because the demoralizing death march to complete such projects destroys morale. The most talented (those who can leave) leave as soon as possible to find a better work environment.
•    Agile: Agile is not an estimation methodology in and of itself, and it does not generally develop estimate in units of time. Instead, estimated units of work are used. The units of work do not directly map to time until you have calculated your team velocity. Once the velocity is determined, it can give you a pretty good idea as to how long it will take to complete your project – that is for the body of work in the project that you know of and understand.

There are many other methods, and no method is perfect. I use two approaches – Analogous and Agile. Whenever possible, I try to compare the project I am attempting to estimate to projects I have done in the past that are similar in size and complexity and have similar teams. The more similar the projects and teams, the more accurate this method will be. That approach is, however, not the topic for this article. When I am not able to use the Analogous approach, the method that I use to forecast is based on the Observed Velocity of an Agile Team or Project. For the approach to be effective, we need to understand a few fundamental things.

Specificity and Accuracy

“A little inaccuracy sometimes saves a ton of explanation.” – H. H. Munro (Saki) (1870-1916)

Specificity and Accuracy are not the same things. Guessing the number of jellybeans in a jar and saying 500 is a specific answer, but it is likely not accurate. Saying that the number of jellybeans in the jar is 500 plus or minus 50 (10%) provides a range states the accuracy of your estimate. This answer is less specific, but more useful because the stated accuracy provides people with an understanding of the uncertainty or range of accuracy of your estimate. Stakeholders of software development projects tend to want specific answers for when a project will be delivered, and if you want to win the jellybean guessing contest, you will also need to provide a specific answer. However, the challenge for estimation software project duration is that while the number of jellybeans in the jar remains constant, a software project usually changes its size and its requirements many times over the duration of the project. It’s not unlike going to the grocery store with your family and a list of items to purchase and an estimate of the amount of money you need to bring with you to pay for your purchases. As you move through the store, your spouse and your children remove items from the cart and add items to the cart that are not on your list. By the time you get to the checkout, the contents of the cart are very different from the items on your shopping list, and now your shopping budget is likely significantly off. Some of the items put into the cart might be less expensive than some that were removed, but similarly, they could be more expensive. You may now have extra money in your pocket to pay the bill, but it is more likely that you don’t have enough. To deal with this scenario, you could take several approaches:

1.    Not bother with the list and get whatever you want and worry about the cost at the checkout. That could become very expensive very quickly, and you may discover after you are home that purchased what you wanted and didn’t purchase what you really needed.
2.    Refuse to make any changes – stick to the list. Your stakeholders (family) might be really unhappy with you.
3.    Focus on the important things on your list while shopping, but in your budget, plan for some unknowns.

The third option is likely the best one, and this is the approach I use when estimating how much time it will take to complete a project. The process of estimation is broken down into three parts and is based on the following three premises:

1.    It is relatively easy to estimate what you know.
2.    It is difficult to estimate what you know you don’t know.
3.    It is very difficult to estimate what you don’t know you don’t know.
I refer to these items as Class 1, Class 2 and Class 3 respectively.

Estimation

Settings

Download a copy of the spreadsheet used to perform these calculations here: Excel Spreadsheet: Estimate How Long It Will Take To Complete Your Agile Project

Before we get into the calculations, we need some basic Settings or Project information.

Settings

The first Velocity entry comes directly from the Velocity you have observed for your team. (To determine your team or project Velocity, see three previous articles mentioned in the first paragraph of this submission). Sprint duration is the number of working days in your sprint. Work days per month is the number of working days in a typical month. You can adjust this value to accommodate for an average number of days in a month that you expect to lose to vacation time, sick leave etc… over the course of the project. Velocity / Day is a simple calculation that divides the Sprint Velocity by the number of work days in a Sprint.

Class 1 – What You Know You Know

To calculate how long it will take to complete your project you simply divide Estimated Effort by Velocity in Units of Effort per time period. The following chart displays Velocity in Days, Weeks and Months using the values from the Settings table to adjust the time base accordingly. The Confidence value is the degree of confidence you have both in your Effort Estimates and the completeness of your collection of things that you know. Adjusting the confidence level changes the High and Low range of the estimate as a percentage. Spread is the difference between the High and Low range of your estimate. The higher your confidence level, the smaller the Spread will be, and the closer the High and Low values will be to the Probable.

What You Know You Know

So in the above example, the Estimate for all of the Story Cards that you know about is 600. Using the Team Velocity from the Settings, the Probable project completion is in 128 working days. With a 90% confidence level of 90%, the High and Low ranges of the estimate are 140 and 115 days respectively. So there you have it, a reasonably accurate estimate of how long your project will take to complete… but wait… there’s more…

Class 2 – What You Know You Don’t Know

These are the things in your project that you know about, but you don’t have a high degree of confidence in your estimates. This may be due to technical challenges, skill sets that the team lacks, new technologies that must be learned, high risk experimental work, etc… It could also be work you are aware of, but don’t know for sure if your will need to complete it. I use a confidence level of about 50% for this body of work.

What You Know You Don't Know

Class 3 – What You Don’t Know You Don’t Know

These are the things in your project that you don’t know about at all. They exist in almost every project. They can be the sudden change in direction, the new essential feature that appeared out of nowhere and no one had ever discussed it, or it could even be something suddenly dropped from the project because it is deemed no longer needed. I separate this portion of the estimate out of the other two area so that I know I have explicitly accommodated for it, and in doing so, I have not distorted other areas of the Estimate in unseen ways. Because this is truly the area of the unknown, I use a confidence level of 25% or even lower. As you seen from the calculations below, the Range of Days (16) is greater than the Probable number of Days.

What You Don't Know You Don't Know

Summary

The Summary section of your estimate is simply the addition of Class 1, Class 2 and Class 3 values. The one exception is the Confidence level which is a weighted average of the Confidence levels from the three classes weighted by Effort for each.

Summary

In this example, the project will be completed in 178 working days plus or minus 27 days (54/2) with a confidence level of 82% that it will be completed in roughly 151 days.

By keeping track of the changes to Stories in all three Classes over the duration of the project, you will be able to do some benchmarking and have a good body of statistics for use in future projects. As well, you will have a significant body of information to share with Stakeholders who want to know why the project is slipping. If you’ve accommodated 50 Units of Effort in Class 3, but due to change requests and scope creep it becomes 300 Units, you have a mechanism to reign things in or to go back to the stakeholders and obtain more budget, time extension or whatever is needed to complete the project.

So there you have it, a simple, easy, and reasonably accurate means of calculating how long it will take to complete your project.

As a closing note: Remember to do the difficult and high risk things first. This means try to do as much of the Class 2 stuff as early as possible in the project. As well, do any of the items identified in your Estimates as complex or high risk early in the project. Some project will in spite of best efforts, a great plan and a superb team fail, so if you are going to fail, then fail early. Chances are that if you follow good process and have good people on your team, you will succeed.

“We didn’t lose the game; we just ran out of time.”  – Vince Lombardi

As always, I look forward to your comments.

Michael

A Simple Way to Calculate the Velocity of Your Agile Teams and Projects

Speed (flickr - Sean Tubridy)

Whenever the work is itself light, it becomes necessary, in order to economize time, to increase the velocity. Charles Babbage

What is Agile Velocity?

The calculation of Velocity in physics is pretty straightforward, and if memory serves me correctly, it is the first formula I learned. Simply put:

Velocity = Distance / Time

In truth, Velocity is a vector and requires a directional component to fully qualify as a Vector. Velocity without a directional component is Speed – a Scalar. What we have in the above formula and what we will actually be measuring is a Scalar – the speed of the team. Velocity as a term has, however, become so widely used in the Agile community that it makes no sense for me to refer to it as Speed. So with apologies to Sir Isaac Newton for the misused term, and for my liberal interpretation (as you shall see) of his first two laws of motion, we shall accelerate into the topic.

So if Velocity is Distance divided by Time, what do we use as the numerator or distance unit in our equation? For our purposes, the distance is Units of Effort. My previous post “Agile Planning Poker” explains how to accurately develop estimates for Units of Effort. Time (the denominator) is the length of our Sprint which in my projects is two weeks. Velocity is thus Units of Effort Completed / Sprint. As an example: a team that regularly completes an average of 70 units of Effort in a two week Sprint has a Velocity of 70 Units of Effort per Sprint, or is simply stated as just 70. I prefer to keep the Sprint as the denominator because the team will tend to think in terms of Sprints, but for reporting, or forecasting, I will sometimes use Weeks as the baseline. Whenever I do, I am careful to note the different units.

There is much discussion and debate online about using Story Points vs. Ideal Days vs. Actual Days to calculate Velocity. There are other ways I’ve seen that work with a variety of inputs that are formulaicly balanced. I firmly subscribe to the Occam’s Razor approach to things and thus avoid such complexities. The number and variety of factors that can affect the Velocity of a team are both significant in number, and difficult to quantify (See calibration below). Many of the factors will cancel each other out (electronics systems depend on this principle for noise reduction), but the net effect will be a bias on the team Velocity. To illustrate the difference, think about all of the things that you could quantify to determine their effect on the speed of an automobile. You could measure fuel consumption, manifold pressure, acceleration and a myriad of other things and plug them into equations to calculate your speed. Or you could simply measure how far you’ve travelled and divide by the time it took you to travel that distance. So rather than attempting to measure and accommodate for a myriad of changing and difficult to quantify input values, I prefer to take the simple path and use Observed Velocity. The Observed Velocity is simply how many Units of Effort your team completes in a typical Sprint. For this to be useful, however, you need to calibrate your Velocity.

Calibration

Optimization

Optimization is a process that should be completed before you begin Calibration. There are two things of primary interest to us in our calibration. They are:

1. The Friction or consistent forces that are a constant drag on productivity and reduce Project Velocity.
2. The Variable or Dynamic Forces that decelerate the project or team members and cause the Project Velocity to be irregular.

Optimizing both of these factors prior to Calibration will improve the stability of your Velocity calculation. As the Velocity is the basis for many of the metrics we use in Agile and Scrum, it is important to have a predictable Velocity.

Friction

Newton’s First Law States that “Every object will remain at rest or in uniform motion in a straight line unless compelled to change its state by the action of an external force.” Forces that do not propel your project will slow it down. By minimizing the forces that slow down your project, you reduce the “Friction” that reduces your Project Velocity. Less friction means higher Velocity and greater productivity.

In Software development, there are countless forces that can affect the Velocity of your team. As a team leader, project manager or executive, it is up to you to minimize the external forces that negatively impact on the team’s velocity. Friction forces are more or less constant. You can’t eliminate all of them, but you can reduce many of them. Optimizing them before Calibration is important. Friction forces include:

• Team composition: Are the right people with the right skills on the team.
• Process: Changes to your processes: Agile methods, build, release, testing, etc…
• Environmental factors: Interruptions, noise, poor ventilation, poor lighting, uncomfortable seating and desks, inadequate hardware or software, etc…
• Team dynamics: Some team members may not play nicely with others.

As you can see, most Friction type forces are largely environmental. Their effects are long term. They are also, often the easiest to address. Individually, Friction forces are usually weak forces. Cumulatively, they can have a significant impact. Obtaining optimal Team Velocity requires that they be eliminated or reduced.

Variable or Dynamic Forces

Variable or Dynamic forces are often unpredictable and unexpected. They decelerate your project and cause a loss of Velocity. Their effects are sometimes dramatic, but their influence is often brief.

Newton’s Second Law states that that “The acceleration of an object as produced by a net force is directly proportional to the magnitude of the net force, in the same direction as the net force, and inversely proportional to the mass of the object.” For our purposes, we are viewing this from the perspective of the cost (in terms of productivity) on individuals in the team, or the team as a whole. If you can’t eliminate a force that reduces Velocity (deceleration), then do your best to make it as consistent and predictable as possible  (minimal and infrequent deceleration). The more consistent and predictable the force, the more consistent your Velocity will be. Just like in the real world, accelerating and decelerating objects requires force (energy). Unnecessary acceleration and deceleration not only wastes energy and slows down your team, it also introduces variability. The variations in Velocity make your predictions more difficult. To put it simply: The more you allow your team is influenced by Dynamic Forces and things that change their ability to focus on their work, the more variability there will be in their Velocity  (productivity). This variability makes your forecasts less accurate. Some examples include:

• Team changes: Adding member, losing members, changing roles and responsibilities.
• New tools: Introduction of new development tools, database technologies, languages, etc… require learning, and reduce Velocity until learned.
• Vendor defects: Defects in third party tools and software requiring developer workarounds eat into productivity and Velocity.
• Responsibilities outside of the project: Team members assuming additional responsibilities outside of the project. Shifting between projects can have a dramatic effect on productivity.
• Personal issues: Colicky baby at home, personal health, family dynamics, etc…
• Stakeholders: Stakeholders may not be responsive to requests for information from the developers or tester and thus create delays. They may also have unreasonable expectations of the team.
• Unclear requirements: Lack of clarity or detail in requirements cause unnecessary churn and rework.
• Changing requirements: New project specifications might require skills that are non-existent or weak in the team. Acquiring the skills, either by introducing new team members, or by an existing team member acquiring the skills will impact productivity.
• Relocation: Moving the team to a new physical location upsets the rhythm and impacts their Velocity.

One cautionary note: Velocity is specific to a team and comparisons to other teams are irrelevant. There are many reasons for this, but perhaps the most obvious one is that different teams will develop different Estimates for the same task i.e. Developers on Team A might rate the Effort of a particular story at 6 while Team B might rate it at 15. Further, teams have different skill sets and strengths that make them very effective in some areas, but not in others. Because of this, resist inter-team comparisons – they are simply not fair or accurate.

The Calibration Process

While going through the following Calibration exercise, if your project is affected by Dynamic Forces like those listed above, do not compensate for them in your Velocity calculations. The reason for this is simple: In every Sprint you go through, one or more of these factors will likely raise its ugly head and reduce the Velocity. Some methodologies attempt to compensate for them through calculations. With software developers being the analytical type we tend to be, the temptation to do so can be great, but resist the temptation as in my experience, these calculations introduce additional complexity, with little value. By observing your Velocity averaged over three Sprints as described in the next section, and leaving the effects of Dynamic Forces in your Velocity calculations, you will automatically be developing a Velocity baseline that is biased to implicitly include a Dynamic Force compensation weighting. Indeed, there will be some variability between Sprints, but you should find the weighting to be surprisingly accurate, and most definitely simpler.

Sprint 1

So with the changes you’ve made to reduce Friction to moderate the effects of Dynamic Forces, you are ready to start your Calibration. Calibration is fortunately a very simple process, but it takes a bit of time. Calibration is simply observing your Velocity (Units of Effort Completed per Sprint) and adjusting your estimate to match the observed value. There is no formula for this, and there is no right or wrong way to do it. There is also only one objective to the Calibration Process, and that is to use a feedback loop through a Sprint Retrospective to review your Effort Estimates and adjust them so that they are consistent and accurate. The important thing to remember is that you are looking for a consensus from your team so that your team can then use the feedback to create more accurate estimates.

In your Retrospective for your first Sprint, you and your team will examine your Effort estimates for the Sprint and see if you agree as a team that the original Effort estimates were reflective of reality. Remember, that your estimates of Effort are relative. This means that per my previous article, you should find that your results roughly follow as Gaussian distribution. Further, relative sizing between Stories for both Estimate and Actual should be consistent. In other words, if Story A has an Estimated Effort of 5, and Story B has an Estimated Effort of 12, then the Observed Efforts at the end of the Sprint should remain the same relative to each other. If not, change the estimates to align with your newfound insight. Remember, that you and your team are attempting to calibrate or optimize is your ability to assess the Effort of any Story relative to other Stories in your project.

The result of this exercise produces a baseline Velocity that represents the Velocity of your team. You may also want to look at Velocity of individual team members, but I find this to be less useful as there are many factors affecting individual velocity i.e. Do you always give the really gnarly problems (the ones no one else can solve) to a specific person, and the really easy ones to someone else. If you do, you may see that the person with the easy problems has a higher velocity than the person with though ones. This is just one of many possible distortions in the measurement of individual Velocity that tend to make individual Velocities less useful.

Once you have completed your Velocity adjustments, and in the same session, I recommend that you jump immediately into doing your Estimates for the next Sprint. Doing so serves two purposes:

1. The team will get better at estimating Effort because the Calibration exercise will be fresh in their minds.
2. The freshly created baseline becomes the context for your Effort estimates for the next Sprint.

Sprint 2

After you complete your second Sprint, repeat the review process described in Sprint 1 and compare your results to others in both Sprint 1 and Sprint 2. Remembering that you are looking for your Effort estimates and observed results for individual Stories to be consistent relative to all other Stories, you should see significantly less divergence between Estimated and Observed results in your second Sprint. As in your first Sprint, adjust your Estimates to adjust with observed results.

Sprint 3

As in Sprint 2, compare the results the just completed Sprint (3) to the previous Sprint (2), as well as that of Sprint 1. Again, make your adjustments. After the third Sprint, you and your team should feel very comfortable about the ability of your team to deliver consistent Estimates and maintain a constant Velocity. If you think there is still room for improvement or you are struggling, go through an additional iteration. The more iterations you do, the more accurate your baseline will be, but keep in mind there is a law of diminishing returns – three Sprints should be all you need.

Your Velocity

With three Sprints behind you, take the average Velocity across all three Sprints. This is your Team Velocity. By reducing Friction and minimizing unnecessary Velocity changes prior to Calibration, you have optimized the individual and team Velocities. Over the course of the project, and assuming there are no significant factors introduced such as losing a team member, you should observe a very stable Velocity for the duration of the project. Some might say that you should continually try to increase the Velocity of the team – that should be your goal. While developers constantly try to improve their skills, and tools improve, it is unlikely that any of the team members will suddenly become significantly smarter or more productive, or the introduction of a new tool will suddenly make a drastic reduction in effort. If you can continue to reduce Friction or external forces, you may be able to increase Velocity, but remember, you should have gone through that exercise before you began your Calibration exercise. Unless your developers are slacking off (in my experience this is rare), you likely won’t be able to squeeze much more performance out of the individual team members. I am not suggesting that you not attempt to increase Velocity, but if you have done your job properly before you began the Calibration process, you should have already realized most of the gains possible. In addition to improving productivity, the purpose of the Optimization exercise was to obtain Velocity value that is more or less constant so you can begin developing accurate forecasts which is the topic for next week’s article.

As always, I look forward to your comments.

Michael

Developing Accurate Effort Estimates By Playing Games

Counting (flickr - marina moia)

Background

The poker player learns that sometimes both science and common sense are wrong; that the bumblebee can fly; that, perhaps, one should never trust an expert; that there are more things in heaven and earth than are dreamt of by those with an academic bent.  (David Mamet)

If you’ve read my previous post (recommended so that the following will make sense) you now have a mechanism for sizing the Effort of your Stories. But how do you actually come up with the numbers for Complexity and Size? Over the years that I’ve been leading software development projects, I’ve used a number of methods including comparison of projects of similar size and complexity, Microsoft Project, Function Point Analysis or even simply guessing. There have been a few other trendy and perhaps oddball approaches thrown in for good measure. Curiously, regardless of method, the results have tended to be pretty much the same. There has, however, been one exception. I’ve gotten noticeably better results with Agile methods using Agile Planning Poker.  Poker? Yes poker – well not really poker. It’s actually a process that is gaining in popularity as a means for sizing Agile Stories.

Agile Planning Poker is generally attributed to James Grenning in 2002 and popularized by Mike Cohn, owner of Mountain Goat Software in his book Agile Estimating and Planning. The process involves the members of the team independently developing quick Effort estimates and then comparing their estimate, discussing the differences and arriving at a consensus based on the discussions. Their independent assessments are all shared concurrently with the group by holding up flash cards with the numeric values of their respective assessments. The process relates to the game of poker only in the sense that the participants make their assessments discreetly, concealing them from the other participants until they all show their hands together. It is Important to the process that the participants conceal their assessments from each other and all participants reveal together. While this may seem silly, it serves the definite purpose of having each person arrive at their own conclusion without being influenced by the thinking of other members. The process is quick, simple, and in my experience, very effective.

The Process

The Moderator

The role of the moderator in this process is essential. The moderator sets the tone and the pace for the discussion. The moderator is typically the Project Manager or Scrum Master. The moderator:

• Reads the stories.
• Maintains pace to make sure that the process does not bog down.
• Moderates discussion, tempers the naturally competitive nature of some to ensure that nobody dominates with their perspective and that people who may not normally voice their opinions are also heard.
• Maintains a focus on good results – not on individuals having the winning hand.
• Ensures that people do not cluster their estimates. Some team members may want to be seen as correct, or at least not too different from the pack and always cluster their results in the middle i.e. always guessing 3/3. If your results follow a standard Gaussian Distribution, your safest bet would always be 3/3 if your motivation is to be with the pack, or to be viewed as being correct. Remember that although you are hoping to have a Gaussian Distribution, it is neither your motivation nor that of the team. What you are really doing with this exercise is trying to discover the outliers – the Stories that are actually much larger or much smaller than originally thought when the Story was first written. Following the herd will not uncover the outliers and consequently you will be exposing your project and team to risk. The Moderator must watch for this behaviour and encourage different viewpoints from all team members to make sure the outliers are uncovered.
• Helps the team to avoid Groupthink. If everyone always agrees, then you can be almost certain that you are doing something wrong. Diversity of opinion is essential and it must be encouraged. If there is too much consensus, the moderator should play Devil’s Advocate to stimulate alternative thinking.

Participants

Often it is only the developers involved in the sessions, but I like to have the Product Owner involved. At a minimum, a Planning Poker Session should include all of the developers on the team. For several reasons, it is a mistake to leave this work to a few of the senior team members:

• Smaller groups are more likely to have a group biases that can become Groupthink. More people means more opinions.
• Junior or newer team members learn a lot from these sessions. Experienced developers have insight that comes from years of experience and years of making and then recovering from mistakes. This experience is of incredible value to the team as a whole and can help to quickly elevate the capabilities of the entire team.
• Ensures that estimation is not inadvertently biased by the viewpoint, knowledge or experience of any one team member.
• When Product Managers / Owners are involved and participate in the estimation, they gain significant insight into, and a greater appreciation for the issues related to the development of each Story. Similarly, developers gain better insight into the views of the Product Owners. This can be very useful for creating and enhancing good team dynamics and communication.

The Tools

Some teams use cards with numbers printed on them. This can be particularly useful if you are using the Fibonacci Scaling, but is less important than when using the T-Shirt Sizing (SML) or Scale of 1:5 which I use. I prefer to not use cards because I don’t want to get bogged down at the beginning of a session while people are finding or sorting their cards. It is not a coincidence that I use two vectors and a scale of 1…5 for everything including Ranking of Stories, Defects, Risk and Effort. Simply put, we have five fingers on each hand (at least most of us do), and we have two hands. With no muss or fuss, all team members can walk into a Planning Meeting prepared with all of the tools they need, as long as they can count to five and know which hand is their left hand, and which hand is their right hand. As well, because our result (Effort) is the product of two numbers (Complexity x Size) we keep things nice and simple having nothing more complex than the ten fingers we were born with.

No Bluffing

It is up to the moderator to make sure there is no bluffing. Even though this is called Poker, honesty is the rule. It should be no one’s objective to “be right”. Rather, you are attempting as a team to create accurate estimates – not win the game.

The Six Steps

1.     Moderator Reads Story Description

The moderator reads the story description to the team.

2.     Ask Moderator Questions

The team asks the moderator questions about the Story and the moderator provides clarification about the Story. The moderator makes sure that the questions are relevant to the estimation of Complexity and Size. Q and A should be no longer than two or three minutes. Ideally you want to stay under two minutes per Story.

3.     Estimate Cards (Fingers)

The moderator finalizes discussion on the Story and asks each team member to make their estimates of Complexity and Size (see previous blog post for details) and write down their estimates for both Complexity and Size on a piece of paper or on their notepads. Estimates are not to be revealed until everyone has made their individual determinations.

4.     Show Hands Simultaneously

Once all team members have decided upon their estimates, the moderator asks the team to hold up their hands with the number of fingers on their hands representing Complexity and Size. For ease of processing the results make sure that the team has already decided upon consistent use of hands for both Complexity and Size. I use the left hand for Complexity and the right hand for size. It doesn’t matter which you choose as long as your team is consistent.

5.     Discuss Low / High Estimates

If everyone has exactly the same results (highly unlikely) then there is not much to discuss other than the possibility that there is a groupthink happening or that something is being missed. Assuming that there are differences of opinion, discuss the high and low values for both Complexity and Size. The objective is to tease out the issues that are perhaps not being considered by all team members. This is where team diversity is essential. The broader the range of experience of the team members, the more value there is to the exercise.

6.     Consensus or Pessimist Wins

Timebox the discussion and move to a conclusion. You are attempting to gain consensus, but consensus is not always possible. Decide as a guideline how you will deal with situations where you cannot achieve consensus. You really have three options:

1. Take the average. I don’t like this approach because in my view it tends to bias the results towards the optimistic view (see previous post) and may lead to underestimation.
2. Accept the most pessimistic estimate. This is my preferences as in my experience, estimation tends to be optimistic and to counter that bias, the pessimistic view seems to balance things out. This isn’t a license for the people with a consistent “glass half empty” view to rule the day, but it is a means of counteracting people’s generally, and natuarlly optimistic estimation tendencies. Your team may have a different bias so adjust accordingly.
3. Defer discussion to a separate meeting on particularly gnarly ones. I do this as a last resort. I find it is best to keep the estimation in your planning session. If you find a great deal of disagreement on many of the issues, then decide to consistently go with the pessimistic view and then review the estimates in your retrospective. Use that information to recalibrate. I will talk more about recalibration and velocity in my post next week.

7.     Finishing Up

Once you have made a decision, write the Complexity and Size numbers in the upper right hand corner of the story card. Multiply the two numbers together. The product will be a number in the range [1..25]. Using the colour coding from the previous post, you now have your estimate of effort. Repeat this process for each of the remaining Stories in your Sprint.

Conclusion

Agile Planning Poker is a quick, simple way to estimate the Effort required to complete your Agile Stories. It engages the team and encourage them to bring into the open issues that might impact Effort estimates that may not otherwise be discovered or considered. With a new team, or when you are introducing this process to your organization, don’t be tempted to size all of your stories right away. For the first two or three sprints, size the Stories in the Sprint you are about to begin. Use your Sprint Retrospective to recalibrate each time until you are comfortable with your consistency. The consistency is important because it will be the basis for calculating Velocity – more on that next week.

As usual, I look forward to your comments.

Old card players never die, they just shuffle away.  (Author Unknown)

Michael

The best we can do is size up the chances, calculate the risks involved, estimate our ability to deal with them, and then make our plans with confidence. (Henry Ford)

The greatest of all gifts is the power to estimate things at their true worth. (François de la Rochefoucauld)

The Problem

Estimation is the calculated approximation of a result which is usable even if input data may be incomplete or uncertain. (Wikipedia)

Estimating the size, effort, complexity and cost of software projects is possibly the most difficult task in all of Software Development and Project Management. Even estimating the time required to complete seemingly small and straightforward tasks can be annoyingly, or even dangerously difficult to do. Software Project Success is determined in large part by the ability of the team to meet stakeholder expectations. To be predictive, you need data and most prediction models typically use historical data as the basis of their forecasts. Stock market analysis and weather forecasting are classic examples. In spite of mountains of historical data, advanced algorithms and supercomputers to perform the calculations, weather forecasters are accurate less than 50% of the time. There is even more historical data, incredibly sophisticated algorithms and and nearly the same computing power available to market analysts. In the markets, success is somewhat better for some than others, and fortunately, to be successful in the stock market, you need only be right more often than you are wrong. Some win very big, but most, however, struggle to hit that 50% mark.

There are aspects unique to software development that makes software estimation inherently difficult and different from other forms of forecasting. Some of the reasons for this are obvious while many are not. When I set out to write this essay, I planned to speak in depth about the reasons for this, but soon realized (not without a bit of irony) that I had seriously underestimated the scope and complexity of the task. This topic alone could easily be the basis of an entire book, and there is a lot of information available on the web if you are really interested. So instead of focusing on why we fail so miserably at estimating software development effort, I will simply focus on a purely pragmatic view of how we can do a better job of it in Agile projects. That being said, here are just a few of the factors that can impact the accuracy of your estimates:

• In 1979 Kahneman and Tversky found that human judgment is generally optimistic due to overconfidence and insufficient consideration of distributional information about outcomes. Further, risks typically are underestimated and benefits overestimated. This is a human bias resulting from our “inside view” of the project.
• There is a human tendency for us to want to please others and in so doing we bias our estimates optimistically in the name of pleasing our stakeholders. Changing the estimate does not, however, change the amount of work that needs to be done, and in the longer term, shortcuts rarely turn out to be shortcuts. Ultimately we disappoint the stakeholders by delivering late.
• External forces (budget, customer deadlines, completion, etc…) pressure us to complete things as quickly as possible. Again, this is an external pressure that creates internal tensions which the team tries to satisfy. It usually distorts the estimates, but rarely changes the reality.
• Things change – the project requirements shift, the client needs more features to be added or removed.
• Technical uncertainty causes us to sometimes take a wrong path and have to redo work.
• The tools and technologies are constantly changing, causing developers to continually learn and adapt to the latest releases. In each new release of tools that they use from vendors, they will encounter bugs that are fixed, potentially causing old workarounds to break, while at the same time introducing new bugs.
• Interruptions and distractions affect productivity: Noisy workplace, ineffective meetings, poor lighting, uncomfortable seating, inefficient processes, etc…
• In many, if not most cases, there is no baseline. How can you estimate how long it will take you to do something that you’ve never done before? This leads to the question: If you’ve done it before, why are you doing it again?
• The tasks are often too many, too large and too complex, and with too many interdependencies to fully understand their implications.
• Software is part science, but a large part of it is art. Because a lot of it is art, creativity and productivity of individual team members will vary dramatically and the quality and quantity of their input will vary correspondingly. Somewhat counter intuitively, the productivity is not related to levels of education or years of experience. Given two individuals with essentially identical education and work experience, researchers have measured differences in productivity of as much as 100 times. These are the intangibles of insight, creativity and commitment, and they are far more important than education or certification, but nearly impossible to measure.

One of the most significant influences affecting accuracy of estimates is illustrated in the following example.

Focused attention distorts perceptions of time. I remember years ago reading about a study that was performed on highly trained fighter pilots to determine how their ability to estimate the passing of time was affected by their degree of mental focus applied to the tasks they were performing. As I recall, the test was set up so that the pilots were placed in flight simulators and the people running the test started a stopwatch and requested that the pilot indicate when ten minutes had passed. There were several scenarios of varying complexity that required pilot engagement ranging from routine in-flight functions like communication with the control tower to full air combat simulation. As the complexity of the tasks and corresponding need for focused attention increased, their perception of time became increasingly and dramatically distorted. In the simple task tests, the pilots routinely estimated the duration within a few seconds accuracy. At the extreme end (full air combat simulation) the estimates were dramatically off – sometimes in excess of 300%. In other words, when the greatest attention and focus was required, highly trained pilots let as much as 30 minutes pass, while believing that only ten minutes had elapsed.

Productive programming requires similar levels of focused concentration. Thus as a programmer’s focus and corresponding productivity increase, their ability to determine how long it takes to do something declines. For anyone who has done extensive software development this effect is clearly evident, yet very difficult to compensate for in estimates as it is unique to each individual. My wife has for years witnessed me disappear into “The Zone” and has come to refer to this effect as “Programmer Time” vs. “Real Time”. The irony in this situation is that we want developers to spend as much time as possible in The Zone where their productivity is maximized, but while in The Zone, their estimates of time are dramatically distorted. We then somehow expect them to use this hugely distorted perception of time as the basis for their estimates.

What Can We Do About It?

Clearly we cannot contain and/or compensate for many (if not most) of the factors that influence the accuracy of our estimates. That being said, we still need to have some degree of predictability in our work; “I don’t know” is not a good enough answer. Obviously the larger and more complex the task we are trying to estimate, the more challenging it will be to produce an accurate estimate. We also want to make sure that we pay the greatest attention to the  things that are most important to the success of our project. Fortunately Agile helps us in this respect because appropriate use of the methodology decomposes a project into small units of work, and by definition, focuses our effort and attention on the things that yield the greatest value to the users of the software. Agile practitioners and Scrum practitioners in particular have proposed a number of scales for calibrating estimated effort in projects including:

• Ranking effort on a scale of one to three – one being the smallest, and three being the largest.
• Using a Fibonacci Sequence [1, 2, 3, 5, 8]. A Story ranked as an eight is a Story that is too large to accurately estimate and should likely be classified as an Epic and decomposed into a smaller set of Stories.

There are other methods, but these are the two most common ones that I have encountered. Of note in both cases, the estimates are not produced in terms of units of time. Rather they are merely expressions of Relative Effort. There are several good reasons for this approach, but principally it is recognition of the variations of team dynamics, experience and productivity. For example: Using the Fibonacci Sequence scale, a task ranked as a five for a highly efficient and very experienced developer might take one day to complete whereas it might take a junior developer five days to complete. Alternatively, the same time differences may exist between two senior developers with very similar experience. This may have nothing to do with the overall aptitude of the individuals, but may be due to a personal problem solving style that is more effective in that specific instance. Or one developer may have solved a similar problem in the past that caused the solution of this particular problem to be obvious. Relative Effort is thus a good comparative yardstick.

While both of these methods are effective and widely used, I believe they do not take into account the underlying elements that affect effort and uncertainty. I have thus developed a different model that I find to be very effective. This model is also consistent with the way I develop rankings of Stories, Defects and Risk. If you’ve already read my articles on these topics you will know where this is headed, so here we go…

Developing The Effort Matrix

As I enumerated (in part) above by the multitude of factors that affect our ability to accurately estimate effort, it is clear that accurate estimation requires a multidimensional view to produce accurate and effective estimates. The challenge, however, is which dimensions do we measure? If we were to classify the possibilities using a SWOT (see my article on managing Risk: Five Simple Steps To Agile Risk Management) according to Internal vs. External influences, we can eliminate many of the candidates by simply focusing our attention on the things over which we have influence and conversely paying less attention to those that we can’t. I also keep the vectors to two so as to keep the process as simple as possible so that we actually use the process and don’t try to sidestep it because it is too cumbersome. Using two vectors also maintains a consistency with the other areas of the methodology. Here is what I’ve found works best:

Story Size

Story size is an estimate of the relative scale of the work in terms of actual development effort.

The wording provided here is a suggestion. Develop wording that works best for your team. Remember that these are guidelines – not rules.

Complexity (Story and Technical)

This is complexity of either or both the requirements of the Story and or its technical complexity. Complexity introduces uncertainty to the estimate – more complexity means more uncertainty.

The wording provided here is a suggestion. Develop wording that works best for your team. Not all factors listed need to be met – remember, these are guidelines, not rules.

Effort

Using these two vectors, I determine effort using the following simple formula:

Effort = Complexity x Size

Effort Matrix (click on image to enlarge)

Are The Estimates Accurate?

The Metrics

How do you know if you are doing a good job of estimating Effort? The answer is simple, but it may require a bit of work to come up with the answer. It is a straightforward exercise in basic statistics but without very much math. To determine if you doing a good job of estimating, you need to look for two key things:

1. Distribution: This is a measure of how well the distribution of your estimates map to a bell curve.
2. Velocity: A measure of how many units of Effort are completed per sprint as compared to how many units of Effort were forecast. Velocity is a large topic on its own and will not be discussed in this article. The process of calculating and comparing Velocity to forecasts is part of a larger Calibration process. Calibration is the feedback loop used in the setting of baseline metrics that are at the core of making all of your processes and estimates more predictable. We will use external mechanisms (to be discussed in a future article) to calibrate the Intrinsic measure of Relative Effort (as determined by the team) and use an Extrinsic view to measure Velocity (Relative Effort Compared to Time) and turn it into Time Estimates. This will help us offset The Zone effect in our time estimates and improve the accuracy. Look for a discussion on this in the next few weeks.

Distribution

If you are estimating well, and your Stories are scoped appropriately, then there should be a distribution of Effort approximating the distribution of the classic Bell Curve. While one could go into tremendous detail calculating Standard Deviation and performing other analysis, what we are looking for here is a simple litmus test to act as a reality check on our estimates. This is a quick and easy exercise with a spreadsheet to gather the raw data and chart it. Your distribution should look something like this.

Effort Distribution (click on image to enlarge)

What we are looking for is a clustering of estimates in the range of [4..15]. Ideally, you do not want to have anything in the [1,2,20,25] ranges.

The two most significant factors in how your estimates will be distributed are:

1. Appropriate scoping of stories: If your stories are written so that your distribution is weighted in the [10..20] range, your Stories are likely too large, and or too complex. If the distribution is weighted in the [2..9] range, then they are likely too small or not sufficiently complex. In either case, work through the Stories in successive iterations (Sprints) to re-scope and refine them. The sweet spot is to try and have your stories scoped so that the vast majority of your Stories scale in the  [4..15] range.
2. Accuracy of your estimates: This article defines the structure for managing Estimates. An upcoming article will describe the actual real-world mechanism for determining individual estimates. Look for it in the weeks to come.

Every team and every organization will be unique and everything in this article should be considered a guideline and suggestive rather than prescriptive. As the person leading your team, you will have to work with the team to find the balance point that works best for accurate calibration of your team. In both examples listed above, use an iterative process to discover and refine the mechanisms that work best for your organization.

As always, I look forward to your comments.

Michael

An Open Space Conference for Agile Coaches

Adding Topics To The Grid (click on image to enlarge)

Background

On June 11^th and 12^th I was fortunate to be able to attend Agile Coach Camp Canada 2010 in Waterloo Ontario. The event was held in the William G. Davis Computer Research Centre (“DC” building) at the University of Waterloo. I was invited to the camp by Declan Whelan who is a well recognized Agile Coach and one of the primary organizers of the camp. I checked out the site and saw already signed up were a couple first rate Agile Coaches Michael Sahota and Gil Broza whom I already know as well as a lot of other well known Agile Coaches whom I wanted to meet, so I figured it would be worth attending. I was not disappointed. This blog entry contains my observations as well as some photos and video I shot during the day and a half long event.

Over the years I have attended many conferences but this is the first Open Space conference I have ever attended. As I discovered, Open Space conferences are driven by an agenda created at the conference by the attendees.

You can check out the tweet stream for the conference at: #accca

Friday

Lightning Talks

After the Un-Keynote and a game of Ball Point (a lesson in Scrum and iterative process improvement) came the Lightning Talks.  Lightning Talks are a series of rapid succession 90 second long presentations on any topic related to Agile. Here are some quick notes I made on ideas that struck me during the talks. I don’t necessarily agree with all of the following, but much of it is thought provoking.

• Not a lot has been written about creating aligned optimized teams. Agile is about optimizing teams.
• An important part of the process of optimizing your team is to get it invigorated and focused.
• Core hours – A concept of setting aside 5½ hours during the working day where developers do not check e-mail, Twitter, Facebook or take phone calls. This part of the day is focused solely on the project. Outside of the Core Hours, developers can spend their time on those other things that are distractions. Focused attention minimizes distractions, keeps people focused. Check out “Does the Internet Make You Dumber?”
• The results are not the point. Check out Igniting The Third Factor by Dr. Peter Jensen the Sports Psychology Coach of the gold medal winning Women’s Canadian Olympic Hockey Team. It’s about getting teams excited and delivering under budget, ahead of schedule and exceeding expectations.
• Many of the metaphors in coaching and leadership including martial arts metaphors such as Sensei are silly, a waste of time and irritate people. Arguing the metaphor misses the point.
• Pop psychology – Left Brain (artistic) / Right Brain (analytical) is old neurology and it is wrong.
• Listening is the willingness to change. You don’t necessarily need to change, but you need to be open to the possibility.
• The Agile Manifesto still holds true.
• What about the people in the Agile world? Why are we making people fit into the methodologies? Not everyone can adapt to the new ways of doing things. The Agile coaches need to think about the people, not just the process.
• Testers are often excluded from the Agile process as the Agile process seems to include only the developers. There is a big disconnect between the development team and their story cards and the people testing. What about Session Based Test Management? Testers need to be involved in all phases including charter, scheduling, reviewing of results and debrief.
• Can’t, won’t, wouldn’t, shouldn’t, etc… all victim words and no team can become a high performance team if they use these words. This victimization approach is the team being victims of themselves. “It must really suck when Joe comes in and micro-manages you…” commiserate to build trust, then ban the words from the team. Check out Chris Avery responsibility model – move from the blame game to “What are we going to do as a team to improve?” The Agile Coach must help the team get past this.

Saturday

Sessions

As I mentioned, I have never attended an Un Conference. I was curious about how a successful conference could be held without any pre determined sessions, and without people making presentations. The notion behind an Un Conference is that attendees decide what they want to discuss, suggest topics and then attend the sessions that are of interest to them. If a session isn’t working for you and you are not getting out of it what you want, or if something else may be more interesting at that moment, then move to a new one. This is not just your right to do so, it is your obligation as a participant. Each session has a facilitator to lead the session, but the participants take it in the direction that is of interest to them. Everyone is involved as little or as much as they want. What I observed and participated in was an interesting, engaging and oft times very animated set of discussions on a variety of topics.

Here is a list of the sessions topic decided upon at the conference. Many of the links have summaries of the sessions that were submitted by the session facilitator. At the end of the morning and afternoon sessions, each session facilitator presented the findings to the entire group. Their presentations were limited to three minutes.

Creating Session Topics (click on image to enlarge)

The Grid (click on image to enlarge)

Session In Progress (click on image to enlarge)

Session Results (click on image to enlarge)

Session Summaries

Presenting Session Results (click on image to enlarge)

Presenting Session Results 2 (click on image to enlarge)

Session Summaries

Here is some video I shot of some of the summaries. I apologize for the shaky handheld camera and the sometimes awkward start and end to some of the presentations. I was using a Nikon D90 that I’ve only owned for a couple of days and consequently still have to learn a lot about before I can pull off smooth video. Solution: Fire the camera operator (me) and get a better one…

Session Summary Part One of Four

Session Summary Part Two of Four

Session Summary Part Three of Four

Session Summary Part Four of Four

Closing Circle

Closing Circle Video Part One of Three

Closing Circle Video Part Two of Three

Closing Circle Video Part Three of Three

Summary

I didn’t quite know what to expect from this event, nor did I have any idea what an Open Space conference would be like. My conclusion is that it was a superb event that was thought provoking and that challenged a lot of the preconceived notions of everyone there. For me, one of the most interesting observations was that in a day and a half long conference whose premise is how to manage technology (software) projects, there was no discussion of technology, and only cursory discussion of process or project management. Instead, I experienced a user driven agenda that focused on communication and how to get the best out of software development teams – team being the operative word here.

I was impressed with the passion, dedication and willingness of everyone to maintain an open mind and to share their experience. I left at the end of the day on Saturday with many new ideas, some new friends and an expanded awareness not just of Agile processes, but of the Agile Community. The next Canadian Agile Coach Camp will be held in 2011 in Montreal – I hope to attend, and I hope to see you there.

As always, I look forward to your comments.

Michael

Managing Risk Across Multiple Agile Projects

(From flickr: gagilas)

All human endeavours involve Risk. Software development projects as evidenced by their high failure rates are certainly no exception. A successful project is not the result of luck, but of anticipating, understanding and managing the Risks of the project. A successful portfolio of projects is therefore the result of anticipating, understanding and managing the cumulative Risk of all of the projects in your portfolio.

In my previous post “Five Simple Steps to Agile Risk Management”, I presented a method for Agile Risk Management. Experience has shown me that the approach works well for almost any type of project as well as for projects of pretty much any size. It is, however, a rare organization that has a single project underway at any given time, so while managing Risk in your individual projects is a good thing, we clearly need “more good”. “More good” is to have a comprehensive, consistent and up to date view of Risk across all projects. This article outlines a process for aggregating the Risks associated with a portfolio of projects so that Risk across multiple projects can easily be identified, reported, tracked and managed.

In many organizations, there are monthly or quarterly reviews of a portfolio or program by senior management to ascertain the progress. Risk assessment is an important facet of project management, and assessing and reporting on the Risks of projects is thus an essential part of this regular review. This high level view of Risk must have a level of detail and organization of the information appropriate to the target audience and also for the program manager(s) so both groups can keep tabs on all of the projects in the portfolio. In particular, senior management needs to be able to quickly understand the issues and determine which (if any) require their attention. For these purposes, the full detail of all Risks in all projects is far too much information so we need an effective means of aggregating, grouping and filtering information so that anything unnecessary is removed and the important information is brought into focus.

At a high level, program managers and executives are most interested in the external effects of the projects. In other words, they worry about the impact of the project risk on the company, its finances, customers, partners, reputation (brand), and so forth. Fortunately, the Agile Risk Management methodology I presented in my previous post provides us with everything we need to create such a summary. In that article, I suggested the following categories into which your various project Risks might be placed.

These categories tend to work well in most organizations, but they may not be right for yours so adjust them to your specific needs. In my previous post I also presented a Project Risk Registry for tracking the Risks in a project. Below is an Enhanced Risk Registry that I use in situations where I need to track more than one project, and where I need to report an aggregated view of Risks across those projects.

Enhanced Project Risk Registry

Enhanced Project Risk Registry (Click on image to enlarge)

In this Enhanced Risk Registry, I track how each identified Risk affects the seven Risk Classes (Solution…Scope). Each of these values are the combined Risk Vectors (Probability x Impact = Risk) for the individual Risks identified in the project. For practical reasons, I would not normally include Green Risks in the Registry. Individual Risks can change dramatically over the life of a project, and once a Risk has been recorded, it remains in the Registry until the project closes – even if the Risk turns Green. For example: In Risk SP-006 (Customer issue LX113 resolved) this issue may return because the fix itself was flawed. Keeping it in the registry reminds us to keep an eye on it to make sure it doesn’t return. As well, I maintain an additional column (not shown here) that contains the Risk Mitigation Strategy for each of the tracked Risks. This column will be of particular interest to management as they will want to know what you are doing about managing the identified Risks.

For each of the tracked Risks, I use a simple formula for each tracked Risk that takes the maximum value across the seven Risk Classes. In SP001 we see values ranging from 1 through 25, but the Risk for this item is 25 – the maximum of the seven Risk Class values. The reason I use the maximum instead of an average (or some other formula) is that if all values will tend towards a mean (as an average would do), then the true Risks would be obscured. I mentioned previously, but it is worth restating that the Risk values must be entered by the respective SMEs and not the PM. This ensures that the category expert’s assessment is captured. The next thing to note is that for each of the Risk Class columns, I also calculate the maximum display it by Class below its respective column. This is the Project Risk for each Risk Class.

So far what we have is the SME view of Risk for this particular project, and by now you are probably wondering what Adjusted Risk is in the above chart. This is the PM’s adjustment of Risk for this particular project. This is not an attempt to hide or obfuscate Risk. Instead, it is the PM’s assessment in consultation with the rest of the team to adjust the Risk rating based on additional information that may not be available to the SMEs that the time of their assessment. It may be also an adjustment of the Risk based on organizational focus. For example, a company that develops guidance systems for NASA might place the highest value on the Solution, whereas a company in the rapidly changing world of social networking might consider Time-line to be the most important Risk Class. There is no easy or formulaic way to make these adjustments as they are based on experience (been there done that) of the people making the adjustments, and on the culture of the organization. In no case should such adjustments be done without consultation. Whenever you make adjustments to the Risk, make sure you record the rationale for the adjustments.

Program/Portfolio Risk

Now that you have entered your project Risk in your enhanced Risk Register, expand your assessment by creating a Risk Register for each of the projects in your portfolio. Once you have done that, you can roll up the Adjusted Risk for all of your projects into the Portfolio or Program view of Risk. Here is an example:

Portfolio Risk Registry (Click on image to enlarge)

With this register, we can see at a glance the Risks for the entire portfolio of projects. The comparison can be easily made not only for the Risk of each project, but also comparing each Risk Class across all of the projects. The bottom line called Category Risk is the Organizational Risks across all projects. From this particular example, it is easy for management to see that projects SP and CP need particular attention. SP looks like the whole project is tipping strongly in the wrong direction and project failure may be imminent. CP looks like it has a serious Risk associated with Privacy (think about the recent TJX credit card info theft). AP6 might also be a concern, and the management team may want to have a closer look at it.

No doubt when you present this information, you will be faced with questions about the specifics of the ratings.  Because you have individual project Risk Registries, you will have the information at hand (including SME assessments) to provide as part of your presentation. Keep in mind that you may also be asked about ratings that appear to be too low, and you should be able to explain the rationale for those as well. Again, the SME assessments will be of tremendous value to you in your presentation.

How Often?

Project Risks can change dramatically during the course of a project so a portfolio assessment should be done at least once per quarter – ideally every month. Each time you do a review, you may want to identify one or two projects for a deep dive assessment. The choice to do this has a lot do with the size and culture of your organization, but give it consideration for inclusion as part of the standard process in your organization. Regardless of the frequency, remember to focus on the difficult (usually Risky) things first and if a project is going to fail, then fail early by dealing with those issues early. Your job is to try to beat the odds and not be in the one third of all projects that fail outright, or even the one third that are seriously challenged. You want to be in that top one third and your chances of success are much better if you identify and manage the Risks early. This process will help you stay on top of the issues and thus help you reduce your Risk.

Good luck!

As  always, I look forward to reading your comments.

Michael

Defect Card (from flickr - listentoreason)

All software has defects of some sort – we know that. If left unresolved, some defects can have cataclysmic consequences while others are so minor that they go unnoticed by virtually everyone. Like most things in this universe there is a law of diminishing returns when it applies to the correction of software defects. Unless you have unlimited resources to assign to bug fixes, you have to focus your attention on the ones  that have the highest ROI. The question is “How do you make those determinations?” There are multiple drivers in any organization that concurrently push and pull the development team in any number of directions. Those drivers could be the Sales Team, QA Team, Finance, End Users, Customers, and Online Media  such as Blogs, Twitter and Internet Forums as well as Traditional Media. The list seems endless and all of it is important (or at least most of it is). Some issues are much more important than others and you can bet, if a software defect is featured on prime time TV news broadcasts that it will be the first thing your CEO will want addressed. So assuming you don’t have the media telling your CEO what your top priorities are, you need a process for focusing your attention on the most important things first.

In my previous post How To Easily Prioritize Your Agile Stories I presented a simple ranking mechanism for assessing and prioritizing Agile Stories and using those rankings to determine their sequence and into which sprint a Story will be placed. This post proposes a nearly identical mechanism and use of the same two vector matrix. The vectors for ranking bugs, however, are different. Instead of Urgency and Business Value, I use Scope and Severity for ranking defects. this is because Urgency and Business Value do not accurately reflect the fact that there are material differences between adding functionality to a system and fixing something that is broken. To illustrate the difference, consider that while prime time news is unlikely to run stories about features missing from your product, it might run a story about the devastating impact of a software defect in your product. Clearly there are different drivers in play here and thus, different vectors are needed. For the purposes of assessing the priority of software defects, I have found that the following two vectors provide the right balance:

• Scope: How many users are affected or how much of the system is affected
• Severity: How critical is the defect i.e. data loss, data corruption, cosmetic issues, etc…

While these are the vectors I use, you are welcome to use your own as they may more accurately reflect your specific business reality.

Scope

The following table provides suggested wording for the ranking of the Scope of a defect.

Severity

The following table provides suggested wording for the ranking of the Severity of a defect.

Priority

As I did with the Ranking of Stories, I multiply the two vectors (Scope and Severity) and the product of the two is the Priority.

Defect Matrix

Actions

Without a set of predetermined actions associated with each priority range, all that has been accomplished in the shuffling of some paper.This is where many systems seem to fall down.  To be effective, appropriate actions must be determined and described. The needs of your organization might be different from mine, and thus the wording you use in your Actions will likely be different.

Defect Priority Ranking

For consistency, I use the same thresholds and the same colour coding as the Story ranking. As with Stories, I print both vectors and the resulting Priority in the top right hand corner of the card and use a highlighter to colour code the card according to the table below. The scale and rankings map directly to the Story rankings and the result is a Defect Card that allows me to easily mix bug fixes into my Backlog with Stories and balance the priorities between the two types. A weakness of most systems I’ve used is that defects and features are ranked using different methods. Doing so makes it difficult to compare the priority of defects relative to features. This system uses the same patterns for both defects and features, but assesses each according to their relevant criteria (vectors), and produces scales that are completely compatible with each other.

Stakeholder Buy In

Consensus on the meaning of the thresholds is important. Selecting the appropriate wording describing the Scope, Severity and Priority may result in wording different than mine. It is also important to keep in mind that it is unrealistic to think that all possible scenarios can be foreseen and described with this system. It is thus important to remember that these are guidelines, not rules. Once agreed upon, however, I have witnessed how the system expedites the assessment process and sidesteps many situations where stakeholders push their agenda as being more important than anyone else’s. Sometimes simply having assessment guidelines (any guidelines) is the most important part of the process, and if nothing else, it can oft times move potentially contentious debates out of the emotional realm into a more empirical one. If you and your stakeholders can at least agree on the basics, and make the process transparent and simple, you’ve made your life and the lives of your team easier.

I’m looking forward to your feedback. Please leave your comments below.

Michael