Preparing for the GAMP Transition to Computer Software Assurance

What is Changing with GAMP?

Traditionally, software validation has often been accomplished via software testing and other verification activities conducted at each stage of the software development lifecycle.  However, as explained in FDA’s Software Validation guidance, software testing alone is often insufficient to establish confidence that the software is fit for its intended use. Instead, the Software Validation guidance recommends that “software quality assurance” focus on preventing the introduction of defects into the software development process, and it encourages use of a risk-based approach for establishing confidence that software is fit for its intended use.

The FDA believes that applying a risk-based approach to computer software used as part of production or the quality system would better focus manufacturers’ assurance activities to help ensure product quality while helping to fulfill the validation requirements of 21 CFR 820.70(i). For these reasons, the FDA is now providing recommendations on computer software assurance for computers and automated data processing systems used as part of medical device production or the quality system. The FDA believes that these recommendations will help foster the adoption and use of innovative technologies that promote patient access to high-quality medical devices and help manufacturers to keep pace with the dynamic, rapidly changing technology landscape, whilst promoting compliance with laws and regulations implemented by the FDA. Consequently the medical device industry is looking to move from a Computer Software Validation approach to one of Computer Software Assurance.

What is Computer Software Validation (CSV)?

Computer Software Validation (CSV) is a requirement of the Quality System regulation, which was published in the Federal Register on October 7, 1996 and took effect on June 1, 1997. (21 CFR 820 and 61 FR 52602). Validation requirements apply to software used as components in medical devices, to software that is itself a medical device, and to software used in production of the device or in implementation of the device manufacturer's quality system. Software validation is a part of the design validation for a finished device, but is not separately defined in the Quality System regulation. The FDA considers software validation to be “confirmation by examination and provision of objective evidence that software specifications conform to user needs and intended uses, and that the particular requirements implemented through software can be consistently fulfilled.”

In practice, software validation activities may occur both during, as well as at the end of the software development life cycle to ensure that all requirements have been fulfilled. Since software is usually part of a larger hardware system, the validation of software typically includes evidence that all software requirements have been implemented correctly and completely and are traceable to system requirements. A conclusion that software is validated is highly dependent upon comprehensive software testing, inspections, analyses, and other verification tasks performed at each stage of the software development life cycle. Testing of device software functionality in a simulated use environment, and user site testing are typically included as components of an overall design validation program for a software automated device.

What is Computer Software Assurance (CSA)?

Computer Software Assurance (CSA) is a risk-based approach for establishing and maintaining confidence that software is fit for its intended use. This approach considers the risk of compromised safety and/or quality of the device (should the software fail to perform as intended) to determine the level of assurance effort and activities appropriate to establish confidence in the software. Because the computer software assurance effort is risk-based, it follows a least-burdensome approach, where the burden of validation is no more than necessary to address the risk. Such an approach supports the efficient use of resources, in turn promoting product quality.

In addition, computer software assurance establishes and maintains that the software used in production or the quality system is in a state of control throughout its lifecycle (“validated state”). This is important because manufacturers increasingly rely on computers and automated processing systems to monitor and operate production, alert responsible personnel, and transfer and analyze production data, among other uses.  By allowing manufacturers to leverage principles such as risk-based testing, unscripted testing, continuous performance monitoring, and data monitoring, as well as validation activities performed by other entities (e.g., developers, suppliers), the computer software assurance approach provides flexibility and agility in helping to assure that the software maintains a validated state consistent with 21 CFR 820.70(i).

What Are The Core Requirements for CSA?

Although the guidance from the FDA is still in draft form, when you look at the original specifications for CSV and how they are evolving to meet the needs of CSA, the following core activities are paramount:

• Requirements Management - Design input requirements must be documented, and that specified requirements must be verified. Requirements reflect the stated or implied needs of the customer, and may be market-based, contractual, or statutory, as well as an organization's internal requirements.
• Software Verification -  provides objective evidence that the design outputs of a particular phase of the software development life cycle meet all of the specified requirements for that phase.
• Risk Analysis & Management - Once a manufacturer has determined that a software feature, function, or operation is intended 221 for use as part of production or the quality system, FDA recommends using a risk-based analysis 222 to determine appropriate assurance activities
• Assurance Activity Planning - Once the manufacturer has determined whether a software feature, function, or operation poses a high process risk (a quality problem that may foreseeably compromise safety), the manufacturer should identify the assurance activities commensurate with the medical device risk or the process risk.

The types of assurance activities commonly performed by manufacturers include, but are not limited to, the following:

• Unscripted testing – Dynamic testing in which the tester’s actions are not prescribed by written instructions in a test case. It includes exploratory testing, ad-hoc testing, and error-guessing.
• Scripted testing – Dynamic testing in which the tester’s actions are prescribed by written instructions in a test case. Scripted testing includes both robust and limited scripted testing. It can be a mixture of automated testing and manual, depending on what is possible and mitigates the identified technical risk.

When establishing the recorded results of assurance activities, the manufacturer should capture sufficient objective evidence to demonstrate that the software feature, function, or operation was assessed and performs as intended. In general, the record should include the following:

• the intended use of the software feature, function, or operation;
• the determination of risk of the software feature, function, or operation;
• documentation of the assurance activities conducted, including:
  • description of the testing conducted based on the assurance activity;
  • issues found (e.g., deviations, failures) and the disposition;
  • conclusion statement declaring acceptability of the results;
  • the date of testing/assessment and the name of the person who conducted the testing/assessment;
  • established review and approval when appropriate (e.g., when necessary, a signature and date of an individual with signatory authority)

How Does SpiraPlan Help?

SpiraPlan has been designed from day one to offer a unique platform for managing the quality of software, systems and devices. With its built in support for key quality management features such as requirements management, test case management and traceability, risk management, risk analysis and risk-based testing, as well as defect/problem management and verification and validation, SpiraPlan has been designed to support current-state CSV activities as well as be ready to assist organizations as they transition to CSA. To better illustrate this, we shall consider the example scenario from the FDA CSA guidance:

A medical device manufacturer has decided to implement a commercial business intelligence solution for data mining, trending, and reporting. The software is intended to better understand product and process performance over time, in order to provide identification of improvement opportunities. The following features, functions, or operations were considered by the manufacturer in developing a risk-based assurance strategy.

Requirements Management

The following features and functions were identified in the system being considered (just a subset of the total):

Inside each requirement, we have clearly identified the intended use of the feature. This could be done either as a section of the main text (as shown above) or as a separate text field if preferred:

Now that we've identified the Features, Functions, or Operations and the associated Intended Use of the Features, Functions or Operations, we can now do a risk-based analysis to determine if this functions pose a high process risk.

Risk-Based Analysis

Once a manufacturer has determined that a software feature, function, or operation is intended for use as part of production or the quality system, the FDA recommends using a risk-based analysis to determine appropriate assurance activities. Broadly, this risk-based approach entails systematically identifying reasonably foreseeable software failures, determining whether such a failure poses a high process risk, and systematically selecting and performing assurance activities commensurate with the medical device or process risk, as applicable.

In this case, using SpiraPlan's risk management module, we identified the following risks associated with this feature:

We have analyzed the risk and due to the data impacts to safety, the impact was rated 5 - Catastrophic and the Probability was rated 3 - Possible, giving an overall Exposure of 12.

As a first step in documenting the necessary assurance activities, we captured the risk mitigations that will be useful in reducing the probability of such a failure:

The final step in this part of the process is to associate this risk to the originally identified requirements. In this example, we only had a single risk, but in most cases there will be more than one:

With this done, we can now run a requirements - risk exposure report to get the aggregate risk for each of the features in the system:

This has identified both requirements as having a relatively high process risk. Therefore we need to determine and apply the appropriate assurance activities...

Determine Assurance Activities

Once the manufacturer has determined whether a software feature, function, or operation poses a high process risk (a quality problem that may foreseeably compromise safety), the manufacturer should identify the assurance activities commensurate with the medical device risk or the process risk. In cases where the quality problem may foreseeably compromise safety (high process risk), the level of assurance should be commensurate with the medical device risk.

The manufacturer determined assurance activities commensurate with the medical device risk and has performed an assessment of the system capability, supplier evaluation, and installation activities. Additionally, the manufacturer establishes a detailed scripted test protocol that exercises the possible interactions and potential ways the functions could fail.

Using SpiraPlan, in this example, we would define the various test cases (both unscripted and scripted, both automated and manual) that adequately mitigate the identified risk:

Furthermore, using the risk-based testing approach, we can even see the total risk mitigated by each test case being executed:

This is useful for verifying the coverage of risks by test cases.

Note: Manufacturers are responsible for determining the appropriate assurance activities for ensuring the software features, functions, or operations maintain a validated state. The assurance activities and considerations noted above are some possible ways of providing assurance and are not intended to be prescriptive or exhaustive. Manufacturers may leverage any of the activities or a combination of activities that are most appropriate for risk associated with the intended use.

Establishing the Appropriate Record

Finally, once this has all been put in place, SpiraPlan lets you execute the defined test cases and log results along with associated defects. As per the FDA, documentation of assurance activities need not include more evidence than necessary to show that the software feature, function, or operation performs as intended for the risk identified. The FDA recommends the record retain sufficient details of the assurance activity to serve as a baseline for improvements or as a reference point if issues occur.

Typically you would include the following:

SpiraPlan provides all of these necessary features and data records as part of its test execution and defect tracking modules:

when executing you can record the necessary results:

and log defects / non-conformities at the same time:

Furthermore, for specific types of testing, other Inflectra products can assist:

• Exploratory Testing - SpiraCapture is a free add-on that streamlines the capture of exploratory testing sessions and observations.
• Automated Scripted Testing - Rapise is a rapid and easy to use automated testing tool for web, desktop, mobile and API applications.

Where Can I Learn More?

• To learn more about the FDA's draft guidance on GAMP and Computer Software Assurance (CSA) please refer to their publication: https://www.fda.gov/media/162627/download
• To learn more about GAMP, risk management and how SpiraPlan can be used to assist you in your move to CSA, please attend the upcoming webinar.