The Business Case for Test Management With SpiraTest

May 5th, 2020 by Adam Sandman

Often we get asked the question, how do you determine the Return on Investment (ROI) of purchasing and using SpiraTest to manage a team's requirements, test cases and defects. If the organization in question is using another tool (for example Micro Focus / HP ALM) it is easy, you simply compare features and license cost. However what is the ROI and business case for teams that are using manual methods, using MS-Word and MS-Excel instead of a tool? This is a bit trickier to calculate, so don't worry - we've put together a handy spreadsheet to help you out!

The Benefits of Test Management Tools

As described in our introductory article - What is Test Management Software? - test management is the process of taking your project's requirements, building a test plan, writing the tests, planning the test activities and capturing the results. Those test cases, test plans, and test activities can consist of manual tests, automated functional tests, API tests, load/performance tests, security tests, and frankly any other kind of test and experimentation activity that improves the quality of the final product.

Whether you choose to formally use a test management tool or not, you are in effect performing the functions of test management, namely:

Gathering and synthesising the requirements for the system
Figuring out how you will test it, and what tests need to be done
Performing the actual testing, and seeing what works and what is broken
Recording all of the defects, problems and issues that need to be fixed
Determining when you can safely release the system into production with an acceptable number of defects

(note that I did not perpetuate the fallacy that it will be defect free).

Now if you don't have a test management tool, then you are probably using:

Microsoft Word, Google Docs or a Wiki to write the requirements
Microsoft Excel, Google Sheets or some other format to write your tests
A bug tracker to track the defects

Plus... Lots of people to review these documents, determine what tests need to be run, and see how close you are to being ready to release.

Whereas if you are using an integrated test management tool (like SpiraTest) you have all of this information in one place, up to date in real-time:

So it clearly makes sense to invest in an easy to use requirements and test management tool that can manage your entire software quality assurance process. If you are interested in knowing what features we think you should look for, here's a good whitepaper - How To Choose A Test Management Tool. However you probably are wondering, how do I justify the cost of such a tool, what is the Return on Investment (ROI) over my existing manual methods?

Measuring Return on Investment (ROI)

We have helped several of our customers put together business cases over the years to help them justify the cost of a tool like SpiraTest, and usually what we do is create two scenarios:

What are activities I have to do today to manage my requirements, test cases, test results and defects?
How do those activities change look, after I have implemented a test management tool?

Current State

In the current state, we need to determine which roles are performing each of these tasks:

Requirements Management
- Writing requirements in MS-Word
- Managing documents in SharePoint, etc.
Test Case Editing
- Writing test cases in Excel
- Reviewing test cases
- Updating and distributing test cases
Manual Testing
- Executing tests and recording results in Excel
- Manually updating the test results into a central sheet
- Creating new defects in bug-tracker and linking to results
Bug Tracking
- Reviewing bugs in old Bug Tracker
- Reviewing fixed bugs in old bug tracker
- Bug triaging and prioritization
Reporting
- Compiling requirements coverage metrics
- Compiling testing metrics and making graphs
- Compiling defect metrics and making graphs

For each activity you need to assign the number and types of role that perform that activity, and in a given month, how much time they spend on it.

Future State

For the future state, we do a similar activity, but now we consider the test management tool to be in place, and estimate (for the same number of requirements) how much time each of the roles would spend on the new activities:

Requirements Management
- Writing requirements in SpiraTest
- Managing requirements in SpiraTest
Test Case Editing
- Writing test cases in SpiraTest
- Reviewing test cases in SpiraTest
- Updating and distributing test cases
Manual Testing
- Executing tests and recording results in SpiraTest
- Creating new defects in bug-tracker and linking to results
Bug Tracking
- Reviewing bugs in SpiraTest Bug Tracker
- Reviewing fixed bugs in SpiraTest bug tracker
- Bug triaging and prioritization
Reporting
- Reviewing requirements coverage metrics
- Reviewing testing metrics and dashboards/graphs
- Reviewing defect metrics and dashboards/graphs

Finally, you need to add in the cost of the tool (software, hosting and any training) and add that to the total for the future state.

That gives you the total future and current states, from which you can determine the cost savings (assuming the future state is cheaper!!) and ultimately the ROI.

Using Our Business Case Estimator

To make this process a bit easier, we have created this handy SpiraTest Business Case Estimation spreadsheet. This spreadsheet should not be used 'as-is', but should be customized to meet the specifics of your organization.

Metrics Tab

In the first Metrics tab, you need to specify how many months you need to demonstrate the ROI for. The longer the time, the greater the ROI, but you'll need to figure out what is a reasonable investment horizon for your organization. You might want to use 3-months for a Proof of Concept (POC) project, or 6-months to a year for a real implementation.

You should also define the various roles you have, the number of people in those roles, and an approximate, notional salary. It doesn't have to be 100% accurate (or include payroll tax costs, etc.), it's just used for comparison purposes.

Current State Tab

Then in the current state tab, you will assign the different roles to the various activities, and determine how many hours they would spend on a project of the specified length.

If your team works differently, feel free to adjust then names, number and types of activities, the ones in the spreadsheet are just a guide.

The final row will contain the total cost for the project if you were to do it using your current team and tools.

Future State Tab

Then in the future state tab, estimate how many people and how many hours they would need to perform the test management tasks illustrated, for a project of the same length. Unlike the current state, for the most part these activities are pretty standard when using SpiraTest, so you won't need to add/change the activities themselves, just the actors, quantity, and # hours.

Finally, the spreadsheet includes built-in formulas for determining the number of SpiraTest licenses you will need, the duration of the subscription, and the commensurate cost. It will add on the future state personnel cost and use that to determine the % ROI of the software investment.