Using Risk-Driven Development (RDD) in Software Delivery

by Adam M. Sandman on Saturday, March 16, 2024

Using Risk Driven Development (RDD) in Software Delivery

Dr. Sriram Rajagopalan, Enterprise Agile Evangelist

Introduction

There is not a single industry today that does not use software to manage their projects or its portfolio of operations. Whether it is developing software that the company sells for their clients or assuring quality in the continuous realization of value through the operational value delivery pipeline, risk management is critical. The influence of risks and risk management on prioritizing features for product development or ongoing evaluation of operational process controls is highlighted in many frameworks as listed below.

The traditional plan-driven project delivery frameworks recognize early identification and analysis of risk management before the project itself is put into motion!

The agile approaches to product development use risks inherent in the features, user stories, and spikes as part of their “timeboxing” principle to create value.

The Kanban and Lean approaches focus on avoiding queues promoting the need for cross-functional multidisciplinary teams.

The total quality management frameworks use thresholds and tolerance to observe patterns during operations to evaluate operational quality issues.

“If you don’t manage risks, risks will manage you!” says Dr. Rajagopalan, Inflectra’s Enterprise Agile Evangelist (Rajagopalan, 2020a). So, risks are at the core of any project, product, or operational initiative directly aligning the total cost of ownership ensuring that the value maximization principle does not compromise quality!

Early Predecessors to Risk Driven Development

The concepts of risk driven development are not new. For instance, the model-based risk management standards focused on the end-state in software development. This allowed an appropriate level of abstraction to be created that helped people understand the various failure states without needlessly worrying about the implementation details. The foundations of object-oriented programming, therefore, focused on the principles of encapsulation, inheritance, and polymorphism. Even within a specific procedure being developed, the ideas of input validation before applying any logic on the data and evaluating return data type of the values are inherent in such risk driven development.

To visually evaluate the impact of scenarios that could adversely affect the intended outcome, the unified modeling language (UML) was created (Rumbaugh, Jacobson, Booch, 2005). Contrary to the big upfront design and development methods, the idea behind these approaches was to progressively elaborate on requirements (Rajagopalan, 2014) based on the level of threat to implementing the logic. This approach is pervasive in the way design patterns were created in object creation, object communication, concurrency control, etc. (Vlissides, 1995). These concepts were not limited to the development community alone because the singleton pattern of a class having only one object instance can be seen in a similar singular focus on requirement articulation in a user-story (Cohn, 2004).

In the testing stages, the development community embraced test-driven development (TDD) to ensure development was done to make the tests written to verify the requirements pass (Cohn, 2004) rather than author test cases based on development. Such approaches to reduce the impact of failure in the results achieved, the management community also incorporated acceptance test driven development (ATDD) and hypothesis driven development (HDD) by introducing behavior driven development (BDD) approaches. Such ideas can be seen even in the International Standards Organizations’ recommendation of the overarching risk management standard, ISO 31000, where one of its eight principles (Hutchinson, 2018) emphasizes that the focus of risk management is in creating and protecting the organizational value.

Rise of Risk Driven Development

There is an intense focus on value generation in today’s business community. The emergence of markets based on the fourth industrial revolution technologies (Rajagopalan, 2020b) have increased the type, number, and complexity of the software interfaces. No longer are users expected to interface with the software with a keyboard and mouse. The use of artificial intelligence and technological sophistication has made many different keyboard-less interfaces possible such as touch, voice, and local environment in the case of the Internet of Things (IoT). This has allowed us to communicate in semi-autonomous cars and smart homes.

The ubiquity of the software along with the severity of the risks can have serious economic consequences. Some of these risks could also be fatal to the users and the people employed in managing the development and operational value streams in regulated fields like healthcare. Therefore, various regulations are on the rise to keep controls in the software development industry and the software delivery disciplines such as the product development, project management, and operational excellence.

Implementing Risk Driven Development

In order to practice risk driven development, the software development mindset must change. Instead of starting with requirements (as is typically done with traditional linear or sequential development), the software development lifecycle must change to be a results driven approach. As results translate into creating or sustaining value, the mindset should start with the risks impacting the delivery of value. In other words, the simplistic approach to implementing risk driven development is proposed as illustrated in Figure 1.

Figure 1: Approach to Implementing Risk Driven Development

Please keep in mind that the following steps are sequential for the most part. This is particularly true until test scenarios cases are developed to evaluate whether these test scenarios will meet the requirements or specifications - leading to reduced risk exposure. However, as test scenarios themselves are evaluated, additional risks may emerge or the scenarios may be associated with other existing risks leading to a reprioritization of the risks. For instance, in order to give additional confidence that the requirement controls reduce risk exposure, different levels of controls validation may be put in place:

Software in regulated industries like financial services and the healthcare sector may need to adhere to FDA’s latest FDA-2022-D-0795 guidelines (2022) on computer system assurance (CSA) and the associated 21 CFR Part 820 regulations require risk based testing that involves unscripted exploratory testing.

The Healthcare industry requires additional validation in the form of performance qualification (PQ) tests to verify user requirements, operational qualification (OQ) tests to functional specifications, and installation qualification (IQ) to test for design specifications (Sandle, 2018). Furthermore, these tests may be executed by a different group outside of the product delivery team.

Certain industries, like the manufacturing, retail food chains, or allied industries that should comply with occupational safety hazards, will additionally require manual inspections (Womack, 2011), such as the Gemba walks and unscheduled supervisions, to check against policy adherence.. The risk management frameworks may vary in these scenarios. For instance, the manufacturing industry uses Failure Mode Effects Analysis (FMEA), Failure Mode Effects Controls Analysis (FMECA), and Fault Tree Analysis (FTA). The retail food chain uses Hazard Analysis and Critical Care Points (HACCP) or Hazard and Operability Analysis (HAZOP).

While the above considerations are beyond the scope of our discussion, these risk frameworks should be evaluated to identify, evaluate, and analyze risks based on the risk driven development proposed below before releasing the final product increment.

Architecture for adopting Risk Driven Development

Based on the proposal illustrated in Figure 1, it is only natural to evolve an architecture for adopting Risk Driven Development. Maintaining industry agnostic terms, this proposed architecture is illustrated in Figure 2. While it is self-explanatory, let us briefly explain this proposed architecture. Here, blue dotted lines represent optional implementation based on the risk being addressed or the development approach utilized. The yellow dotted lines indicate an optional business process based on the level of risk exposure reduction required, such as the use of the Failure Mode Effect Analysis (FMEA), General Automation Manufacturing Process (GAMP), or a specific ISO policy like the ISO14971 for medical devices. The 0, 1, M, N represent the standard Entity-Relationship notations. For instance, 1:0/N between Business Risk and Functional Specification mean a business risk may be addressed by zero to many functional specifications.

Figure 2: RDD Proposed Architecture

The various dependencies and relationships captured in Figure 2 are explained below.

Once business risks are analyzed and prioritized, the requirements are drafted to reduce these business risks.

One or more requirements here serve as controls to reduce the risk exposure. Each requirement may produce a deliverable (output).

The user requirements may be broken down into smaller specifications, such as the high-level specification as requirements (sub-deliverables) or activities (tasks) to evaluate progress to completion.

Sometimes, the business risk may require an internal functional control, like a procedure to follow, more than external controls. In this case, a business risk may be reduced by zero or more functional specifications.

Either the user requirements or the functional specifications may involve a detailed design or a document describing the process or approval required.

As the functional specifications are evaluated by the team, technical risks may arise as well.

The requirements and specifications are verified by several test scenarios. Several challenges to executing these test scenarios may emerge as well, such as the availability of a technical resource, which may be associated with risks as well.

These scenarios could be prescriptive test cases, exploratory test cases, or a collection of test cases to verify that the product developed or the processes implemented work as expected to address the requirements or specifications ultimately reducing the business risk.

As test cases are executed, they verify quality in the form of both conformance to requirements and specifications as well as fitness for use.

The observations during the test execution are logged and evaluated. These observations can be deviations from the expected behavior or potential enhancement ideas. As these observations are triaged, these observations can create new requirements or be associated with new or existing technical or business risks.

Steps to Implementing Risk Driven Development in SpiraPlan

One of the first steps in implementing Risk Driven Development in SpiraPlan is to map the proposed architecture in Figure 2 to SpiraPlan specific artifacts as documented in Figure 3.

Figure 3: SpiraPlan Artifact Mapping to implement RDD

In this section, let us elaborate on the ideas and best practices to keep in mind while implementing the risk driven development. To illustrate these steps, let us use an example of logging into a website, which is a very simplistic use case for any industry. This simplistic use case also exemplifies how the risk driven development is industry agnostic.

For example, this approach can be used in strategic business initiatives like mergers and acquisitions. It can also be used in managing employee retirement benefits by allowing multiple products like a mobile app to track investments, a web application to allow exchanges between funds, and an artificial-intelligence (AI) driven background process to look for fraud detection with all the three products simultaneously working off of the same data.

Similarly, a clinical lab can use the same approach to ensure good lab practices are being upheld while engaging in drug discovery (new product development) or drug production (employing GxP processes in operations).

1. Categorize Risks

Not all risks are equal. It is better to group related risks so that the pertinent risks are addressed first, and any dependencies identified. Among the various risks identified, organizations frequently have some classification of risks and this is called a risk breakdown structure (RBS). Within these risk categories, various people with relevant domain experience are consulted to identify specific subtypes of risks. The RBS is very helpful so that the delivery team can focus their energy on risks that need to be addressed by them at the project level while delegating other risks to the program, portfolio, or enterprise levels. For instance, we identified security, resource, and privacy risks as part of business risks.

In SpiraPlan, these risk types are defined at the template level because many related projects can share these risk types. SpiraPlan also allows specific workflows that can be applied at these risk types to allow specific change management or governance protocols as required by the organization. Additionally, SpiraPlan also provides “components” at the project level where risks can also be categorized from a different perspective.

For the purposes of our discussion, we will focus mainly on the business risks.

Figure 4: Risk Categorization

2. Assess Risks

The assessment of the risks goes through three stages, namely the risk identification, risk analysis, and risk evaluation.

The risk identification is facilitated by various stakeholders depending upon the nature of the product developed or operational excellence considerations. A good RBS, an internal risk management process, and associated governance process is frequently critical to facilitate the identification of risks within each risk category.

Using the SpiraPlan risk module supporting this functionality, let us enter some risks as seen in Figure 5.

Figure 5: Identification of Project Specific Risks

The risk analysis can involve qualitative or quantitative analysis. It focuses on the likelihood (probability) of any risk events or the severity or consequence (impact) of risk on project goals and objectives. Some organizations may even involve detectability as another factor to risk analysis. Therefore, having a good rating scale on what the probability and impact means helps organizations consistently and objectively analyze risks. SpiraPlan has definitions for the probability and impact and has extensions to support detectability. Given below in Figure 5 is the risk probability scale in SpiraPlan. A similar scale can be put in place in SpiraPlan for Risk Impact also. Consequently, SpiraPlan computes Risk Exposure as a product of probability and impact.

Figure 6: Risk Probability Scale

Applying these probability and impact scales on the risks identified in Figure 5, the risk score computed on each risk is illustrated in Figure 7. In the first risk identified below, risk probability is associated with Certain (score 5 in Figure 6) and risk impact is associated with Critical (score 3 not shown in Figures). So, the risk exposure for this risk is 5x3 = 15.

Figure 7: Risk Analysis with Risk Exposure

The risk evaluation involves the knowledge gained from the risk analysis to make decisions about what risks need to be acted upon immediately, periodically monitored, or actively accepted. This area is a precursor to risk response planning by looking at the risk heat map (probability and severity). By looking at the risk heat map with probability on the X-Axis and impact on the Y-Axis, it is possible to look at which risks should be addressed and what should be on the watch list.

Figure 8: Risk Heat Map for Risk Prioritization

3. Prioritize work on based on the initial risk exposure

The risk heat map discussed in Figure 8 assists with the risk-based prioritization. The concept of risk-based prioritization has been present in some of the popular prioritization techniques like MoSCoW (Must do, Should do, Could do, Won’t do) that emerged from Dynamic Systems Development Methodology (DSDM).

According to Software Quality: State of the Art in Management, Testing, and Tools (2011), this principle-based prioritization is directed to ensure that development aligns their efforts to the highest value delivery. Frequently, teams evaluate the risks and review what controls must be in place. Depending upon the nature of product being developed or the operational controls required to sustain the benefits, the accountability rests with the delivery team to come up with controls. Even if requirements are created without these risks, it is possible to associate these requirements with the risks once they are assessed as illustrated earlier. In such cases, a report can be run to evaluate which requirements with the highest risk score should be worked on.

4. Create requirement as controls to address these risks

For instance, based on the security risks identified with the use of plain-text credentials, the team can introduce some controls such as having every member of the team review the organizational security policy, incorporate them in their software design principles, and also update and confirm the code review guidelines. In Figure 9, these three controls are identified as requirements. Additionally, another requirement is also added to illustrate how the risk exposure drives the requirement prioritization. SpiraPlan provides a robust requirements module to support this risk and requirement association.

Figure 9: Prioritized Requirements to Address Highest Risks

5. Create tasks for tracking of progress towards completion

This is an optional step. However, in today’s world of adaptive project management and incremental product delivery approaches, the tasks representing the activities are critical to evaluate progress. SpiraPlan provides a task module to support different types of tasks to ensure and evaluate progress to completion. The breakdown of one of the requirements into various tasks and the progress of these tasks towards completion as part of work in progress (shown in green on the task progress bar) are illustrated in Figures 10 and 11 below.

Figure 10: Task Breakdown

Figure 11: Task Progress

6. Verify controls and document deviations

In this step, verification involves various test scenarios to evaluate if the controls are working as desired. This test scenario could be individual test cases or a collection of predefined test cases as test sets. These test scenarios could also be scripted with test steps, sample data, expected results, and parameters. Or, it could be exploratory testing to support unscripted and adhoc testing. SpiraPlan supports extremely versatile testing modules to meet all these needs. For example, the following Figure 12 illustrates two test cases for the quality team to verify and evaluate if the controls are working as expected. SpiraPlan also supports risk-based testing where risks can be associated with test cases to prioritize verification but also deviations associated during testing to evaluate clarity of the requirements.

Figure 12: Verification with Test Cases

7. Triage deviations and reanalyze for final risk exposure

As part of the verification, the quality team generates a snapshot of their test runs. During this testing phase, it is possible that the controls work exactly as expected or deviate. These deviations could take on a wide spectrum of observations from defects or futuristic ideas. If they are defects, they are issues which indicate risks that have materialized. In such cases, new controls can be created or existing controls modified depending on the organizational processes. If they are not defects but ideas for future enhancement, then, these could be documented for future work. SpiraPlan has a powerful “Incident” module to support such deviations for triaging purposes.

Once the test cases or test sets are run successfully and marked complete with any incidents triaged appropriately, the original risks associated with these controls need to be reassessed. SpiraPlan supports custom properties that can be added to capture these post-implementation risk assessments where probability, impact, and exposure must be added as custom fields as illustrated in Figure 13. If the FMEA process is used, the detectability and RPN also must be added.

Figure 13: Revised Risk Score Computation

8. Accept the residual risk or iterate with additional requirement controls

At this point, it is evident from Figures 7, 9, and 13 that a specific security risk associated with the use of plain-text access credentials has been associated with three different requirements as control measures. The tasks monitor the progress to the requirement controls as development controls. The test cases and test sets verify that the controls are separately evaluated for compliance. As a result, the original risk exposure score of this security risk has been reduced from 15 to the revised final exposure score of 6. Because risk driven requirements use requirements as controls, the revised final exposure also is the residual risk that is now accepted. Should this residual risk not be accepted, then, risk driven development can continue to iterate with additional controls to reduce this risk further.

Summary

It is critical to emphasize that in today’s emerging world of product development and project management, there is a constant pressure to accelerate the delivery of value. While value delivery using adaptive approaches generates competitive advantage in terms of product positioning in the market, the market is also unforgiving with escaped defects that compromise brand equity. Hence, the value of risk management is pivotal to value delivery that this risk management is recognized in all the standards for quality management, project management, program management, and portfolio management. The risk driven development suggested here becomes the necessary and sufficient condition for organizations to deliver and sustain value.

References

Cohn, M. (2004). User Stories Applied: For Agile Software Development. United Kingdom: Pearson Education.

Hutchins, G. (2018). ISO 31000-: 2018 Enterprise Risk Management. (n.p.): Certified Enterprise Risk Manager Academy.

Rajagopalan, S. (2014). Review of the myths on original software development model. International Journal of Software Engineering & Applications, 5(16), 103-111.

Rajagopalan, S. (2020a). Organized Common Sense. Outskirts Press. Parker, CO: Outskirts Press.

Rajagopalan, S. (2020). The role of Project Management in Drug Development. Retrieved September 24, 2021 from https://www.youtube.com/watch?v=alxSRZpOJKk