Managing Risk Across Multiple Agile Projects

(From flickr: gagilas)

All human endeavours involve Risk. Software development projects as evidenced by their high failure rates are certainly no exception. A successful project is not the result of luck, but of anticipating, understanding and managing the Risks of the project. A successful portfolio of projects is therefore the result of anticipating, understanding and managing the cumulative Risk of all of the projects in your portfolio.

In my previous post “Five Simple Steps to Agile Risk Management”, I presented a method for Agile Risk Management. Experience has shown me that the approach works well for almost any type of project as well as for projects of pretty much any size. It is, however, a rare organization that has a single project underway at any given time, so while managing Risk in your individual projects is a good thing, we clearly need “more good”. “More good” is to have a comprehensive, consistent and up to date view of Risk across all projects. This article outlines a process for aggregating the Risks associated with a portfolio of projects so that Risk across multiple projects can easily be identified, reported, tracked and managed.

In many organizations, there are monthly or quarterly reviews of a portfolio or program by senior management to ascertain the progress. Risk assessment is an important facet of project management, and assessing and reporting on the Risks of projects is thus an essential part of this regular review. This high level view of Risk must have a level of detail and organization of the information appropriate to the target audience and also for the program manager(s) so both groups can keep tabs on all of the projects in the portfolio. In particular, senior management needs to be able to quickly understand the issues and determine which (if any) require their attention. For these purposes, the full detail of all Risks in all projects is far too much information so we need an effective means of aggregating, grouping and filtering information so that anything unnecessary is removed and the important information is brought into focus.

At a high level, program managers and executives are most interested in the external effects of the projects. In other words, they worry about the impact of the project risk on the company, its finances, customers, partners, reputation (brand), and so forth. Fortunately, the Agile Risk Management methodology I presented in my previous post provides us with everything we need to create such a summary. In that article, I suggested the following categories into which your various project Risks might be placed.

These categories tend to work well in most organizations, but they may not be right for yours so adjust them to your specific needs. In my previous post I also presented a Project Risk Registry for tracking the Risks in a project. Below is an Enhanced Risk Registry that I use in situations where I need to track more than one project, and where I need to report an aggregated view of Risks across those projects.

Enhanced Project Risk Registry

Enhanced Project Risk Registry (Click on image to enlarge)

In this Enhanced Risk Registry, I track how each identified Risk affects the seven Risk Classes (Solution…Scope). Each of these values are the combined Risk Vectors (Probability x Impact = Risk) for the individual Risks identified in the project. For practical reasons, I would not normally include Green Risks in the Registry. Individual Risks can change dramatically over the life of a project, and once a Risk has been recorded, it remains in the Registry until the project closes – even if the Risk turns Green. For example: In Risk SP-006 (Customer issue LX113 resolved) this issue may return because the fix itself was flawed. Keeping it in the registry reminds us to keep an eye on it to make sure it doesn’t return. As well, I maintain an additional column (not shown here) that contains the Risk Mitigation Strategy for each of the tracked Risks. This column will be of particular interest to management as they will want to know what you are doing about managing the identified Risks.

For each of the tracked Risks, I use a simple formula for each tracked Risk that takes the maximum value across the seven Risk Classes. In SP001 we see values ranging from 1 through 25, but the Risk for this item is 25 – the maximum of the seven Risk Class values. The reason I use the maximum instead of an average (or some other formula) is that if all values will tend towards a mean (as an average would do), then the true Risks would be obscured. I mentioned previously, but it is worth restating that the Risk values must be entered by the respective SMEs and not the PM. This ensures that the category expert’s assessment is captured. The next thing to note is that for each of the Risk Class columns, I also calculate the maximum display it by Class below its respective column. This is the Project Risk for each Risk Class.

So far what we have is the SME view of Risk for this particular project, and by now you are probably wondering what Adjusted Risk is in the above chart. This is the PM’s adjustment of Risk for this particular project. This is not an attempt to hide or obfuscate Risk. Instead, it is the PM’s assessment in consultation with the rest of the team to adjust the Risk rating based on additional information that may not be available to the SMEs that the time of their assessment. It may be also an adjustment of the Risk based on organizational focus. For example, a company that develops guidance systems for NASA might place the highest value on the Solution, whereas a company in the rapidly changing world of social networking might consider Time-line to be the most important Risk Class. There is no easy or formulaic way to make these adjustments as they are based on experience (been there done that) of the people making the adjustments, and on the culture of the organization. In no case should such adjustments be done without consultation. Whenever you make adjustments to the Risk, make sure you record the rationale for the adjustments.

Program/Portfolio Risk

Now that you have entered your project Risk in your enhanced Risk Register, expand your assessment by creating a Risk Register for each of the projects in your portfolio. Once you have done that, you can roll up the Adjusted Risk for all of your projects into the Portfolio or Program view of Risk. Here is an example:

Portfolio Risk Registry (Click on image to enlarge)

With this register, we can see at a glance the Risks for the entire portfolio of projects. The comparison can be easily made not only for the Risk of each project, but also comparing each Risk Class across all of the projects. The bottom line called Category Risk is the Organizational Risks across all projects. From this particular example, it is easy for management to see that projects SP and CP need particular attention. SP looks like the whole project is tipping strongly in the wrong direction and project failure may be imminent. CP looks like it has a serious Risk associated with Privacy (think about the recent TJX credit card info theft). AP6 might also be a concern, and the management team may want to have a closer look at it.

No doubt when you present this information, you will be faced with questions about the specifics of the ratings.  Because you have individual project Risk Registries, you will have the information at hand (including SME assessments) to provide as part of your presentation. Keep in mind that you may also be asked about ratings that appear to be too low, and you should be able to explain the rationale for those as well. Again, the SME assessments will be of tremendous value to you in your presentation.

How Often?

Project Risks can change dramatically during the course of a project so a portfolio assessment should be done at least once per quarter – ideally every month. Each time you do a review, you may want to identify one or two projects for a deep dive assessment. The choice to do this has a lot do with the size and culture of your organization, but give it consideration for inclusion as part of the standard process in your organization. Regardless of the frequency, remember to focus on the difficult (usually Risky) things first and if a project is going to fail, then fail early by dealing with those issues early. Your job is to try to beat the odds and not be in the one third of all projects that fail outright, or even the one third that are seriously challenged. You want to be in that top one third and your chances of success are much better if you identify and manage the Risks early. This process will help you stay on top of the issues and thus help you reduce your Risk.

Good luck!

As  always, I look forward to reading your comments.

Michael

Risk Factory (flickr: kyz)

This report, by its very length, defends itself against the risk of being read.  (Winston Churchill)

Background

In my post on Agile Project Charters I outlined the embarrassingly high failure rate of software projects. Success rates today are only marginally better than they were when the Standish Group released its first Chaos report in 1995. Recognizing the tremendous misalignment between project expectations and project results, a variety of tools and methods have evolved to help improve the odds of success. Chief among them is Project Management methodologies. Even with fifteen years of experience combined with improved software development tools and better methods, software project success rate have eked out only marginal gains. This is not a vilification of project management methodologies. Rather, it is a statement that software development is an inherently and increasingly complex undertaking with many uncertainties. With Risk Management, we attempt to identify the things we don’t know (the uncertainties) and quantify them so that they can be managed. This sounds like a paradox – how can you quantify what you don’t know- but it is a paradox we can manage.

Agile Methods such as Scrum are a relatively new entrant into the field of project management. A basic tenet of Agile Methods is that teams produce a continuous series of useable software builds in very short cycles called Sprints. Each build is assessed, issues identified and the backlog of tasks is reviewed and prioritized and the most important tasks are scheduled for the next sprint. It sounds like an ideal approach. For many teams it works extremely well as Agile teams tend to claim higher project success rates than do teams using more traditional methods. There is not a lot of empirical data available that makes effective comparisons of Agile project success rates to other methodologies, but what data that does exist tends to support those claims.

Most methodologies place a fairly high importance on Risk Management. Agile approaches tend to implicitly manage Risk. That might not be a bad approach if the only things that affected the outcome of the project were the decisions that the developers made to implement the solution, but as we shall shortly see, there exist a multitude of factors that can have a significant impact on the success of a project. Further, I maintain the position that explicit Risk identification and management can further improve on the success rate of Agile projects. In this article I will outline a Risk Management methodology I use that is quick, simple, pretty comprehensive and very Agile friendly. As the title of the article implies, I have broken the process down into five steps:

1. Identify
2. Classify
3. Quantify
4. Plan
5. Act
6. Repeat

Oops I lied… there are six steps. Actually there are only five steps but it is worth stating Repeat as a sixth step to emphasize that our Agile Risk Management Process defines a virtuous circle of continuous improvement.

The Basics

Yes, risk taking is inherently failure-prone.  Otherwise, it would be called sure-thing-taking.  (Tim McMahon)

• Risks are influencing factors that might adversely affect the outcome of a project.
• Risk is the direct result of uncertainty. If there is no uncertainty, it is not a risk – it is a certainty.
• Risk analysis is used to help a team understand uncertainty that could affect the outcome of the project.
• Risk management (sometimes called Risk Mitigation) is the plan that the team puts into place to pre-empt, contain or mitigate the effects of risk to a project.

The important thing to remember is that even in simple projects, things can and will go wrong, and that you need to make plans to minimize the impact of those events when they occur.

1. Identify

The Dimensions of Risk

Risk has two dimensional influences. The first Helpful/Harmful is a simple assessment of factors that have a potentially positive or negative influence on the success of our project:

1. Helpful: Factors that advance the objectives of the project
2. Harmful:  Factors that hinder or imperil the outcome of the project.

The second dimension of Risk is the identification of the source of the Risk:

1. Internal: Factors originating inside the organization or within the sphere of influence of the project.
2. External:  Factors originating outside of the organization or project that cannot usually be influenced by the project.

Combining these factors into a two dimensional assessment provides us with the classic SWOT Analysis view of our project: Strengths, Weaknesses, Opportunities, and Threats.  In the diagram below we see the two dimensions (four factor categories) arranged in a matrix with Helpful/Harmful dimension represented as columns, and Internal/External dimension represented as rows.

Strengths Weaknesses Opportunities and Threats

Risk Management is primarily interested in the Harmful column and that is what we will focus on in this article.

Examples of Weakness

• Insufficient resources
• Limited budget
• Aggressive timeline
• Important skills lacking in the team
• Technological uncertainties
• Lack of stakeholder consensus
• Lack of a disaster recovery plan

Examples of Threats

• Rapid and significant changes in the economy
• Pandemics
• Geopolitical tensions
• Economic uncertainty
• Changing legislation
• Changing competitive landscape
• Trade tariffs

Weaknesses are factors over which we tend to have some degree of control. Threats, however, are factors over which we tend to have little or no control. It is important to understand that even though we may have no control over a factor such as a pandemic, there are usually things we can do to manage or minimize the Risk effects on our project.

2. Classify

Each of the Risks needs to be categorized as to the affected area, likelihood and level of Impact it may have on the project. Risk Classes are used primarily for organizing, summarizing and reporting of Risks to management and stakeholders. Some Risks you identify may impact more than one Class, and if they do, they should be reflected in the summaries of those Classes.

The next chart is a list of Risks Classes I typically use. These categories are not prescriptive and you may wish to add others such as Reputation, Environmental Impact, etc… to suit your project or company needs. Solution, Timeline, Budget, Privacy and Security should be of interest to everyone with a stake in the outcome of the project. Resources and Scope are primarily relevant to the development team, but they can have a significant impact on the other categories and are as such included in the set. Some Risks may affect multiple Risk Classes and that effect should be reflected in your Risk Classification. I will show how the Risk Classes are summarized later in this article.

What Do You Assess?

I maintain the Risk ratings of each Story or Defect directly on the Cards. I use the same pattern of recording the three numbers Probability, Impact and Risk Rating and use a highlighter to colour code the risks. At a story level, most risk will likely be pretty benign, so don’t obsess and spend a lot of time on the low risk items. Focus on the ones that are genuine threats. Defects are areas that may require more attention if only because as Defects they likely have higher visibility in the organization. In both cases, write a few details about the Risk directly on the card. An added benefit of having developers assess Risk associated with the Stories and Defects is that it encourage a new dimension for their thinking about the work they are doing and helps them to be cognizant of the effects their work has to the overall success of the project.

Tracking Risk associated with Stories and Defects is insufficient – especially for Threats (factors external to the project) and for any identified Risk that is not a Story or a Defect I use a Risk Register (more on that later). The Stories and Defects that receive a high Risk Rating are also tracked in the Risk Register.

3. Quantify

Great – so now we know what to measure, but how do we go about doing that? If you’ve read my three previous blog posts, you’ve likely already guessed that we will use a matrix based on two vectors. The two vectors we will use in this case will be Probability and Impact. The Risks you identify must each be assessed according to these two vectors.

The assessment of each risk must be performed by the respective SME (Subject Matter Expert). A project manager is not qualified to perform an assessment of system security unless he/she is also a security SME. The same is likely true for assessing Risks relative to system performance, quality and privacy. Scrum is about teamwork so depend on the team to bring their expertise to the table. Another reason for SMEs to do the assessment is that I have in some organizations witnessed political pressure applied to PMs to produce Risk Reports to reflect a particular or desired Risk profile. This may force the PM to game the numbers to produce the desired results. The ethics of such practices are highly questionable. If you experience a situation like this, you’ve got much bigger issues on your hands than managing the Risk in the project and should perhaps consider looking for a new job. Having the SMEs do the assessment does help insulate the PM from such pressures. Once the SMEs have performed their assessments, it is useful to discuss the assessments as a team to ensure that there is a consistent approach and weighting applied across all assessments. This also allows the thinking and assumptions behind the assessments to be shared amongst the team and brings the team’s collective wisdom to bear on evolving potential solutions. It may even uncover additional Risks due to Risk interdependencies.

Impact

The Impact of a Risk is a measure of its affect on the project. It ranges from Minimal (1) at the low end where the consequences would be very small up to Extreme (5) at the high end. You and your team should devise wording to describe each Impact level to suit the realities of your organization. Whatever you decide upon should be consistent throughout the entire organization so at to minimize confusion. The wording should not be viewed as a set of rules – instead, it is a set of guidelines. Here is some suggested wording:

Probability

If there is a very high probability that a Risk may be realized, then it is clear that it should have the attention of the team. Conversely, if there is a very low probability of the risk being realized, then it is likely that it should receive less attention from the team. We thus need to ensure that the greatest attention is focused on the Risks with the highest occurrence probability. The following chart provides a suggested scale for assessing the probability of Risk manifestation.

Enter the Matrix

We now have two Risk Vectors and as we did in the prioritization of Stories and Defects (see my previous blogs), we take the two vectors and multiply them together to obtain the simple product which is the Risk value. Using the same thresholds for Stories and Defects as well as the corresponding colour system we end up with a Risk Matrix that looks like this:

Risk Matrix

4. Plan

Risk Rating

Now that you’ve identified the important Risks that threaten the success of your project, what should you do about them? You can make your Risk Planning as comprehensive as you wish, but like most things in life, the simplest approach is often the best approach. Unlike Impact and Probability Assessment, your wording should not be considered a guideline.  For each of the various Risk Ratings, we want specific things to occur because the risk thresholds are triggers to mobilize the team or stakeholder to take action to mitigate the Risk. Here is some suggested wording for your Risk Planning. The wording you use in your company should be different than mine and reflect the realities of your organization, but it is important that the wording be focused on Actions to manage the Risk:

Risk Ratings

Risk Register

To track and manage Risk on a project I use a Risk Register. To do this, I use a spreadsheet. Each time I do a Risk Assessment (ideally each sprint planning session) I add a new page to the spreadsheet and each page is a Risk Assessment corresponding to a particular Sprint. This way I can track how a Risk has changed over the course of a project. I can also monitor how Risks are added and removed from the Register. As you near the end of the project, you should see all of your Risks gradually move into the green or minimal range. If this does not happen, you are definitely doing something wrong because if you still have Orange or Red risk in the late stages of your project, you have not been managing the Risk and you are rolling the dice on project success. All of the time, effort and money invested up to that point is at Risk of being lost.

Risk Registry

5. Act

Insanity: doing the same thing over and over again and expecting different results.
(Albert Einstein)

First Things First

Act is simply that. It is the implementation of the defined Risk Mitigation Strategies. Well it’s actually not that simple. Human nature is such that we tend to put off the things that aren’t fun, interesting or that might be just plain hard work. This is project suicide when it comes to Risk. It is imperative that you deal with the high Risk items first. Deferring performance testing and finding out a week before implementation that you can’t possibly achieve the requisite transactional throughput may be the death of your project. At a minimum, someone is going to have to do a lot of explaining – don’t let that be you.

Fail Early

This is important

The “Fail Early” phrase is becoming very popular in the world of venture capital. In essence it means figure out as early as possible in the process as to whether or not what you are doing will succeed. These findings are essentially the go-no go for your project. If success is not possible, either stop (kill the project) and move on to something else, or rethink the project and come at it from a different angle. Either way, do the difficult, gnarly, risky stuff up front. An added benefit is that it helps you define the boundaries of your system and sets expectations as to what is possible/realistic and what is impossible/unrealistic. It could even bring to light unrealistic success criteria and the definition of project success many need to change. When this happens, the project may still live, but under a revised and possibly more realistic set of stakeholder expectations. It may also stimulate commitments like a larger budget or access to key people.

As simple and as obvious as this may sound, it is amazing how often such critical, high Risk items are left until the final stages of the project. From my own observations over the years, this is one of the biggest reasons for project failure. Do the Risky stuff first and Fail Early.

5. Repeat

This process is very lightweight and very quick to perform. Identifying Risks early, and implementing appropriate Risk Mitigation Strategies for each is essential to the success of projects. Done properly, it is a continuous virtuous circle of Assessment and Action to constantly identify, manage and minimize Risk.

Your Risk Plan should be reviewed at a minimum quarterly. Better still, your review should coincide with your sprint planning sessions. At your these sessions, you have access to your team where everyone is already looking at the stories, reviewing effort estimate, etc…  You don’t need to do an exhaustive review each time, but pay particular attention to the Risks you are tracking in your Risk Register. Also look for any new Risks that might start appearing as the team progresses through the project and learns more about the challenges. As always, if you discover Risks that are high, deal with them early.

Summary

In this article I have presented a simple, easy five step process for assessing and managing Risk in an Agile process. My next post will approach how you can aggregate the Risks of multiple Projects into a Program view of Risk.

As always, I look forward to your comments.