Evaluating Risk Strategies with Effective KPIs

by Adam Sandman on

Introduction

Risk management is a critical area in project and product management, overseeing changes across people, processes, technology, and organizational domains. However, many practitioners often overlook its importance, only to address it reactively with corrective actions later on. Dr. Sriram Rajagopalan, Inflectra’s Global Head for Agile Strategy and Training & Learning Services, notes that in a rush to develop products quickly or manage projects with poorly developed key performance indices, the crucial link between quality and risk is often missed.

A common misconception is that risk only means threats, leading to a negative perception. However, a brief review of risk management literature reveals that risks encompass both positive and negative uncertainties that can impact any aspect of a product or project. Therefore, risk management principles should be applied to every release, phase, wave, iteration, or sprint, irrespective of which project management approach is used: planned, adaptive, or hybrid. It is essential to understand and engage with the risk management lifecycle comprehensively.

Risk Management Lifecycle

Uncertain risk events can happen independently of the project itself. It could come from within the project and can also originate from outside the project variables. As a result, risk management (n.d.) focuses on multiple categories of risks depending upon the industry and the nature of the products developed. These risk categories are described as the Risk Breakdown Structure (RBS), within which the product and project management teams identify additional risk types to set an agreed foundation for the various stages of the risk management lifecycle. Therefore, this entire risk management process is a continuous and iterative lifecycle process involving identification, analysis, evaluation, treatment, and monitoring.

Risk Identification

The types of risks agreed upon in the risk breakdown structure by the delivery team and stakeholders give them a structure for brainstorming specific risks. This collaborative engagement facilitates the identification of the various internal and external factors that can impact project outcomes. Such risks are not limited to projects but can extend to programs and portfolios. One of the early artifacts created because of risk identification is the risk register.

An effectively led project means that the project manager, product manager, product owner, scrum master, or risk owner requires everyone in the delivery team and the stakeholders to identify and weigh risks in a cross-functional manner. A developer identifies risks for test development, or a procurement manager identifies vendor risks from a people and talent perspective.

Risk Analysis

Risk analysis often involves a qualitative assessment using high-level scales for probability and impact. The key prerequisite is evaluating and addressing identified risks and ensuring the proper application of agreed-upon scales.

Not all risks are equally urgent or important, so it's crucial to apply and, if necessary, reevaluate these scales. For example:

  • What does a critical impact mean for the project or business?
  • How does a high probability aid in prioritization?
  • Should the scales be adjusted to better align with current workflows?

Without consensus on these scales, risk analysis can be flawed, leading to further risks and undermining the risk management process.

In projects that involve life-or-death situations, such as healthcare, aviation, fintech, or construction, the cost of correction is extremely high. In commercial aviation projects, an incorrect sensor calibration may involve more detailed analysis, and a simple high, medium, and low scale may be inadequate. In such initiatives, increased levels of granular scales for probability and impact, an additional Failure Mode Effect Analysis (FMEA) involving the detectability measure (Spotlight on SpiraPlan 7.0, 2022), or even quantitative assessments involving simulation, scenario analysis, and sensitivity analysis may be involved. An important outcome of this stage is the agreed-upon scales to be used in the risk register.

Risk Evaluation

After identifying risks and agreeing on the scales for analysis, the next step is evaluating each risk to determine its exposure. Risk exposure is calculated by multiplying the numerical scores for probability and impact. For example, a risk with a critical probability score of 5 and a catastrophic impact score of 4 results in an exposure of 5*4 = 20.

These evaluated risks are then plotted on a two-dimensional graph, known as a risk heat map or risk summary, with probability on one axis and impact on the other. This visual aid helps prioritize risks for treatment. The risk register is continuously updated based on these evaluations.

Risk Treatment

In this step, risks are categorized based on the organization's risk appetite, stakeholders' risk tolerance, and the project's risk threshold. Here are some examples of Risk Treatment:

  • Urgent Risks: Any risk score above 75% of the project's threshold (e.g., risk exposure > 15 out of a maximum of 20) is treated as urgent. These risks are prioritized for immediate controls and follow-up.
  • Watch List: Risks scoring between 25% and 75% of the highest risk exposure (e.g., risk exposure between 5 and 15) are monitored and treated based on high probability or high impact scales. Continuous monitoring ensures additional controls are applied if needed.
  • Non-Urgent Risks: Risks below 25% of the highest risk exposure (e.g., risk exposure < 5) are occasionally reviewed to determine if they need escalation to the watch list or urgent category.

This categorization ensures that resources are focused on the most critical risks, allowing for effective risk management.

KPIs To Monitor During The Risk Management Lifecycle

Dr. Rajagopalan (2020) says, "If one doesn't manage risks, risks will manage them." The art of risk management comes to life when one effectively monitors the risks in all these stages. This involves multiple metrics, like the following. Each of the following measures helps you evaluate how effectively the team is identifying, analyzing, evaluating, and treating the risks throughout the lifecycle, including things that may appear innocuous at the surface but may inadvertently impact the delivery.

  • Treatment Options by the Risk Status
  • Mitigations Not Updated in 2 Weeks
  • Mitigation Review Unaligned with Risk Review
  • Mitigations with Past Review Date
  • Risks without Control Tasks
  • Risks with Unassigned Control Tasks
  • Missed Risks in a Risk Type
  • Unassigned Owner for a Risk
  • Missing Impact Assessment for a Risk
  • Missing Probability Assessment for a Risk
  • Missing Review Date
  • Long Review cycle

Summary

Understanding the risk management lifecycle is essential for delivering value in projects and products. International standards like ISO31000 emphasize the importance of continuous risk monitoring. Using application lifecycle management tools to track these KPIs ensures tighter control over the risk management process, ultimately leading to better project outcomes.

References

Rajagopalan, S. (2023, March 10). Quality is a function of risk. Retrieved March 21, 2024, from https://agilesriram.blogspot.com/2023/03/quality-is-function-of-risk.html

Rajagopalan, S. (2020). Organized Common Sense: Why do project management skills apply to everyone? Parker, CO: Outskirts Press

Risk Management (n.d.). Inflectra Corporation. Retrieved March 21, 2024 from https://www.inflectra.com/Products/SpiraPlan/Highlights/Risk-Management.aspx

Spotlight on SpiraPlan 7.0: FMEA Risk Management Comes to Town (2022). Retrieved March 22, 2024, from https://www.inflectra.com/Ideas/Entry/spotlight-on-spiraplan-70-fmea-risk-management-1304.aspx

Spira Helps You Deliver Quality Software, Faster and with Lower Risk.

Get Started with Spira for Free

And if you have any questions, please email or call us at +1 (202) 558-6885

Free Trial