April 6th, 2023 by Adam Sandman
by Experimentus, an Inflectra Partner
I was in a client call earlier in the week, discussing test process improvement, and they asked me what the most common improvement areas identified are. As an individual and as an organisation we have assessed hundreds of organisations so I think that I’m well qualified to comment based on our experience.
In this article I am going to discuss the three most common areas where we see room for improvement, as well as why they are important:
- Risk management
- Review Process
- Metrics and KPIs
Risk Management
Firstly, and most obviously, risk identification, prioritisation and reporting. Testing is really nothing more than a risk mitigation exercise, so understanding what the risks inherent in your product are is vital to successful testing. We would always recommend gaining an understanding of the risks and planning and reporting on your testing accordingly.
If you don’t fully understand the risks there are a few pitfalls that you can fall into. Firstly, you may not adequately test against the most important risks, either because you didn’t identify them at all or because you didn’t prioritise them highly enough. This directly impacts on the effectiveness of your testing, raising the possibility of defects leaking into the production environment with all the cost and adverse impacts that can incur.
Conversely, a lack of understanding of risk can lead to over-testing. Extensive testing of areas that are very low risk may be a waste of time and will have a direct impact on the efficiency of your testing. Cost and speed to market are almost always important factors and any unnecessary testing can affect those.
Finally, when making the decision to release to production, it is important that the testing carried out gives the information required to make that a fully informed decision. If you’re not taking risk into account this just isn’t the case.
Many, many words have been written about risk and testing and there are many ways of assessing it, based on your organisation’s ways of working. There is no one-size-fits-all solution so we help with context-appropriate advice where we see potential gaps in the process.
Review Process
The second area I’d like to talk about is reviews, effectively the “testing” of requirements and other documentation. Having the techniques to adequately review the documentation that you are basing your testing on is vital to stopping defects at source – shifting defect detection left.
You need to be absolutely certain that the documents that you are basing your testing on (e.g. functional and technical designs, requirements, user stories etc..) don’t themselves contain errors, inconsistencies or ambiguities. If they do then the likelihood of defects being introduced during design and coding is vastly increased.
There are a number of types of review; distributed, walkthroughs and inspections for example that can be used and they often have increasing levels of formality and rigour. Having the right people carry out reviews is also key, as is ensuring that people aren’t wasting time reviewing details in documentation that they are not expected to understand – asking people with specific expertise to only review the sections of a document that are relevant to them is a more efficient way of carrying out reviews.
Recording outputs from reviews can be a useful way of spotting trends in documentation and making that documentation better and more usable.
We often observe that organisations don’t provide enough guidance on how reviews should be carried out (who should be involved and how, what type of review should be carried out, how observations should be recorded) and as a result defects that could have been identified much earlier make it through to later phases of testing, perhaps even into production, with all the additional cost that causes.
In short, an effective review process is one of the best means of shifting your test effort left and streamlining your route to production.
Metrics and KPIs
The third area I’d like to talk about is metrics, also known as Key Performance Indicators (KPIs). What do you measure? How? And, why?
There are a number of aspects of metrics that we observe and comment on. Firstly, “why”. Why are you measuring what you measure? What insight will each metric give you and how can you use it to improve your processes? Most organisations today are data rich, but unless you use that data to support your organisational aims, what is the point of it? We still see organisations collecting data “because they can” and producing enormous reports with dozens of data points that nobody ever reads. Pick metrics that are useful to your organisation and will be understood and, most importantly, acted upon by your audience.
It is important to understand that your metrics needs and capabilities may change over time as your process maturity changes. Ensuring that your metrics evolve as your processes mature is vital to maintaining their relevance and ability to support your goals.
Consistency of metrics gathering across your organisation is also key. We see programs or projects capturing different metrics or using different definitions of metrics (the criteria for something like “priority”, for example, need to be defined). What this can mean is that organisations aren’t comparing like with like when trying to make comparisons across projects and that any consolidated organisation-wide measures are flawed.
In short: more is not always better; ensure value is gained through relevance to your current state; be consistent in measurement and definitions.
Our intelligent Test Method (iTM) includes definitions of a number of suggested metrics. We work with our clients to identify and implement those relevant to them at that point in their process evolution and to potentially identify any more bespoke metrics that are right for them.
How Experimentus Can Help?
Experimentus has a wealth of experience in test process improvement and our proven assessment method identifies opportunities for improvement in all areas, including the key areas of risk management, reviews and metrics.
Our Intelligent Test Method (iTM) includes pre-built processes, templates and guidance for all of the three areas discussed in this article and even if you have existing processes it can act as a set of exemplars on which to build changes.
Our expert consultants have helped clients to implement process in these three key areas at many organisations worldwide and our experience of working across organisations of different sizes, industries and geographies means that we can tailor a solution that suits your individual needs.
> Please take our quick test assessment survey today.
How Inflectra Can Help?
Inflectra’s mission is to help you deliver quality software, faster and with lower risk. When you analyse the three areas identified in this article, you can see right away how it aligns with our mission.
Our SpiraPlan quality project management platform includes built in risk management functionality, that lets you identify, analyse and manage risks, and use them as the basis for your test plans and backlogs. Our requirements and document management functionality brings quality assurance to the world of specifications, user stories and documentation. With SpiraPlan, we make sure your review processes are efficient and comprehensive.
Finally, the customizable dashboards, flexible reporting and built-in analytics allow you to define your metrics and KPIs and ensure that the entire team is working together to achieve them. Please sign up for a trial of SpiraPlan today.
Experimentus and Inflectra Together
With the partnership between Experimentus and Inflectra, you have a capable team that is committed to your success and ensuring quality at all stages of the software development lifecycle.