July 15th, 2014 by Adam Sandman
requirements management bug tracking
Defects generally come from two sources:
- internal testing processes and
- issues reported by external users.
These are sufficiently different that they often require separate processes.
Defects from Internal Tests
When internal tests fail they lead to defect reports and some tools can make the link between the two automatically as part of the defect recording process. In some cases the failed test itself acts as the record of the defect, although this can become problematic when defects from other sources are factored in. It is the discrepancy between the expected test result and the actual test result that indicates that there is a defect, therefore the most important relationship is that between the defect and the test, not the code; after all, at the time of the defect discovery the defective piece of code has not yet been identified.
Investment in Traceability
If a project has invested in traceability, the tests will probably already be linked to the requirements - in whatever form they exist, whether it is shall statements, user stories, or any other form, it doesn’t matter. This traceability offers an immediate opportunity to verify that the observed behavior is indeed in error, after all, it is also possible that the expected test result that was written incorrectly, not the code. This three-piece chain of information helps verify the defect, define how the software should work and also which tests must be run again once the problem is resolved.
Note that in this case a link between the defect and the requirement is not necessary because the tests and the requirements are linked, providing a traceability path from defect to requirement (and notably to code.) Defects from users will need links to requirements because:
- Initially, all that is known is which capability (requirement) is broken, and
- Externally discovered defects also suggest that some tests must have failed, but not which ones.
Defects Reported by Users
Defects come from users as well as test engineers. These defects start out as issues and must be verified as true defects somehow; usually by support engineers or test engineers duplicating the problem. This is done by comparing the software behavior with the requirements. (As we said earlier, turning to test cases would not be very helpful as the tests must have failed in order for the software to have been released with the defect.) From this perspective, if the defect is verified then the best initial link to create is the one from the confirmed defect to the requirements not properly fulfilled. (If the requirements are faulty, then a business analysis will be required.)
With the pre-established requirement-test relationship, it is also possible to see where the error slipped through the test process and therefore where the tests are deficient and must be improved. With the code fixed and tests updated, it is once again a simple matter of traceability to see those tests that must be rerun – or in some cases, which new tests must be run for the first time.
In conclusion, relationships from a defect are best made to tests or requirements, but not the code. The relationship of the defect to the code can be deduced through other traceability. When code runs it’s not failing, but that doesn’t mean it satisfies the requirements.
You may also be interested in:
The Cost of Quality Assurance
10 Questions You Should Ask When Choosing a Tool for Requirements Management
Testing in Agile Projects: Familiarity Breeds Consent
Principles of Requirements Engineering (or Requirements 101)
What is a Software Defect?
Testing Lifecycle: Don’t be a fool, use a proper tool.