Michael Lant

Software Archictecture, Development, Agile Methods and the Intersection of People Process and Technology

Agile, Defect Management, Project Management, Scrum, Software Development, Technology

A Simple Agile Defect Management Process

Defect Card

Defect Card (from flickr - listentoreason)

All software has defects of some sort – we know that. If left unresolved, some defects can have cataclysmic consequences while others are so minor that they go unnoticed by virtually everyone. Like most things in this universe there is a law of diminishing returns when it applies to the correction of software defects. Unless you have unlimited resources to assign to bug fixes, you have to focus your attention on the ones  that have the highest ROI. The question is “How do you make those determinations?” There are multiple drivers in any organization that concurrently push and pull the development team in any number of directions. Those drivers could be the Sales Team, QA Team, Finance, End Users, Customers, and Online Media  such as Blogs, Twitter and Internet Forums as well as Traditional Media. The list seems endless and all of it is important (or at least most of it is). Some issues are much more important than others and you can bet, if a software defect is featured on prime time TV news broadcasts that it will be the first thing your CEO will want addressed. So assuming you don’t have the media telling your CEO what your top priorities are, you need a process for focusing your attention on the most important things first.

In my previous post How To Easily Prioritize Your Agile Stories I presented a simple ranking mechanism for assessing and prioritizing Agile Stories and using those rankings to determine their sequence and into which sprint a Story will be placed. This post proposes a nearly identical mechanism and use of the same two vector matrix. The vectors for ranking bugs, however, are different. Instead of Urgency and Business Value, I use Scope and Severity for ranking defects. this is because Urgency and Business Value do not accurately reflect the fact that there are material differences between adding functionality to a system and fixing something that is broken. To illustrate the difference, consider that while prime time news is unlikely to run stories about features missing from your product, it might run a story about the devastating impact of a software defect in your product. Clearly there are different drivers in play here and thus, different vectors are needed. For the purposes of assessing the priority of software defects, I have found that the following two vectors provide the right balance:

  • Scope: How many users are affected or how much of the system is affected
  • Severity: How critical is the defect i.e. data loss, data corruption, cosmetic issues, etc…

While these are the vectors I use, you are welcome to use your own as they may more accurately reflect your specific business reality.

Scope

The following table provides suggested wording for the ranking of the Scope of a defect.

Value Guidelines
5
  • Affects most or all users and/or a very larger range of system functionality
4
  • Affects a large set of users and/or large range of system functionality
3
  • Affects a moderate set of users and/or moderate range of system functionality
2
  • Affects a small set of users and/or a small range of system functionality
1
  • Affects a minimal set of users and/or a very small range of system functionality

Severity

The following table provides suggested wording for the ranking of the Severity of a defect.

Value Guidelines
5
  • Data loss, data corruption or system unavailable
4
  • Important functionality is unavailable with no workaround
3
  • Important functionality is unavailable but has a reasonable workaround
2
  • Secondary functionality is unavailable but has a reasonable workaround
1
  • Cosmetic issues or some functionality unavailable but has a simple workaround

Priority

As I did with the Ranking of Stories, I multiply the two vectors (Scope and Severity) and the product of the two is the Priority.

Defect Matrix

Defect Matrix

Actions

Without a set of predetermined actions associated with each priority range, all that has been accomplished in the shuffling of some paper.This is where many systems seem to fall down.  To be effective, appropriate actions must be determined and described. The needs of your organization might be different from mine, and thus the wording you use in your Actions will likely be different.

Defect Priority Ranking

Defect Priority Ranking

For consistency, I use the same thresholds and the same colour coding as the Story ranking. As with Stories, I print both vectors and the resulting Priority in the top right hand corner of the card and use a highlighter to colour code the card according to the table below. The scale and rankings map directly to the Story rankings and the result is a Defect Card that allows me to easily mix bug fixes into my Backlog with Stories and balance the priorities between the two types. A weakness of most systems I’ve used is that defects and features are ranked using different methods. Doing so makes it difficult to compare the priority of defects relative to features. This system uses the same patterns for both defects and features, but assesses each according to their relevant criteria (vectors), and produces scales that are completely compatible with each other.

Stakeholder Buy In

Consensus on the meaning of the thresholds is important. Selecting the appropriate wording describing the Scope, Severity and Priority may result in wording different than mine. It is also important to keep in mind that it is unrealistic to think that all possible scenarios can be foreseen and described with this system. It is thus important to remember that these are guidelines, not rules. Once agreed upon, however, I have witnessed how the system expedites the assessment process and sidesteps many situations where stakeholders push their agenda as being more important than anyone else’s. Sometimes simply having assessment guidelines (any guidelines) is the most important part of the process, and if nothing else, it can oft times move potentially contentious debates out of the emotional realm into a more empirical one. If you and your stakeholders can at least agree on the basics, and make the process transparent and simple, you’ve made your life and the lives of your team easier.

I’m looking forward to your feedback. Please leave your comments below.

Michael

3 Comments

  1. Scrum, the hard parts - ShareTheLinks

    December 20, 2022 at 1:11 am

    […] the mix, you break that focus. Sometimes those hotfixes are required, but it is important to have a clear grid of severity so that managers understand why their bugs will not be done in this […]

  2. Mike Hollosi

    August 16, 2010 at 11:49 am

    I have successfully worked with only 3 grades of Severity
    1 (Emergency) – corresponding to your 4-5
    2 (Detrimental) – your 3
    3 (Inconvenient) – your 2

    In reality, the lowest category, cosmetic issues, never gets much attention or discussion, so differentiating is not purposeful.

    1. Michael

      August 17, 2010 at 12:39 am

      Hi Mike,

      I’ve used three levels, and I’ve also used five. My experience is that three levels does not provide sufficient differentiation while five is too granular – your mileage may vary… Irrespective of the number of levels used, the importance thing is that there is an easily understood, and easily managed process in place to provide a means for ranking and managing issues.

      Thanks for your comments!

      Michael

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.