What are Testing Methodologies?
Testing methodologies are the strategies and approaches used to test a particular product to ensure it is fit for purpose. Testing methodologies usually involve testing that the product works in accordance with its specification, has no undesirable side effects when used in ways outside of its design parameters and worst case will fail-safely (e.g. a nuclear reactor will shut down on failure).
What are Software Testing Methodologies?
Software testing methodologies are the different approaches and ways of ensuring that a software application in particular is fully tested. Software testing methodologies encompass everything from unit testing individual modules, integration testing an entire system to specialized forms of testing such as security and performance.
Importance of Testing Methodologies
As software applications get ever more complex and intertwined and with the large number of different platforms and devices that need to get tested, it is more important than ever to have a robust testing methodology for making sure that software products/systems being developed have been fully tested to make sure they meet their specified requirements and can successfully operate in all the anticipated environments with the required usability and security.
The functional testing part of a testing methodology is typically broken down into four components - unit testing, integration testing, system testing and acceptance testing – usually executed in this order. Each of them is described below:
The Unit testing part of a testing methodology is the testing of individual software modules or components that make up an application or system. These tests are usually written by the developers of the module and in a test-driven-development methodology (such as Agile, Scrum or XP) they are actually written before the module is created as part of the specification. Each module function is tested by a specific unit test fixture written in the same programming language as the module.
SpiraTest includes support for the management and execution of automated unit tests. With its library of plugins and extensions for all of the major unit test frameworks (xUnit, JUnit, NUnit, PyUnit, etc.), SpiraTest allows the test manager to ensure that there is full coverage of all program code and that all unit tests have passed.
The Integration testing part of a testing methodology is the testing of the different modules/components that have been successfully unit tested when integrated together to perform specific tasks and activities (also known as scenario testing or End to End testing (E2E)). This testing is usually done with a combination of automated functional tests and manual testing depending on how easy it is to create automated tests for specific integrated components.
SpiraTest includes support for storing, managing and coordinating integration tests across various modules and components. In addition, specifically for E2E and scenario tests, you can assign different test cases in a single test scenario to different testers so that they can work together on an integrated test.
With its library of plugins and extensions for different automated functional testing tools including our Rapise automation platform, SpiraTest is the perfect solution for managing your integration testing.
The system testing part of a testing methodology involves testing the entire system for errors and bugs. This test is carried out by interfacing the hardware and software components of the entire system (that have been previously unit tested and integration tested), and then testing it as a whole. This testing is listed under the black-box testing method, where the software is checked for user-expected working conditions as well as potential exception and edge conditions.
SpiraTest includes support for storing, managing and coordinating system tests across all the modules and components that make up a system. SpiraTest support data-driven testing where test cases are defined with input parameters and different combinations of test data can be passed to the manual and automated tests. This ensures that both expected and exception cases can be tested using the same test frameworks.
The acceptance testing part of a testing methodology is the final phase of functional software testing and involves making sure that all the product/project requirements have been met and that the end-users and customers have tested the system to make sure it operates as expected and meets all their defined requirements:
User Acceptance Testing (UAT)
This is an activity that is vitally important, since in many cases until you have completed UAT, the customer or organization will not accept the software and sign off on it. Therefore, having a smooth and successful UAT is very important.
The key aspect of UAT is that the testers will be end users, who may have significant functional expertise and knowledge, but may be used to how previous systems or processes work, rather than the new system being tested. Therefore the test cases they will need to execute, will need to be more prescriptive and outline the discrete steps / tasks to be carried out.
It is important to have a tool for them to use that makes the testing process as easy as possible. For example, making it easy to follow the script, ideally interact with the system under test in the same window, and record results, screenshots without unnecessary complexity.
SpiraTest provides a powerful manual testing solution that helps you coordinate and manage you acceptance testing activities, with all the test scripts, assignments, test results and associated defects/bugs all tracked in a single unified system. With SpiraTest’s requirements test coverage capabilities you can validate that all of your requirements have been fully tested to your customer’s satisfaction.
In most testing methodologies, functional testing involves testing the application against the business requirements. Functional testing is done using the functional specifications provided by the client or by using the design specifications like use cases provided by the design team.
On the other hand, non-functional testing involves testing the application against the non-functional requirements, which typically involve measuring/testing the application against defined technical qualities (also known as the ‘-ilities’ because they all end in ‘-ility), for example: vulnerability, scalability, usability. Some examples of non-functional testing are described below:
Performance, Load, Stress Testing
There are several different types of performance testing in most testing methodologies, for example: performance testing is measuring how a system behaves under an increasing load (both numbers of users and data volumes), load testing is verifying that the system can operate at the required response times when subjected to its expected load, and stress testing is finding the failure point(s) in the system when the tested load exceeds that which it can support.
SpiraTest includes support for storing, managing and coordinating your performance, load and stress testing activities. With its library of plugins and extensions for different automated performance testing tools including LoadRunner, JMeter and NeoLoad, SpiraTest is the perfect solution for managing your performance testing.
Security, Vulnerability Testing
Previously, security was something that was tested after-the-fact. With the rise in cyber-crime and the awareness of the risks associated with software vulnerabilities, application security is now something that needs to be designed and developed at the same time as the desired business functionality. Security testing tests the software for confidentiality, integrity, authentication, availability, and non-repudiation. Individual tests are conducted to prevent any unauthorized access to the software code.
The usability testing part of a testing methodology looks at the end-user usability aspect of the software. The ease with which a user can access the product forms the main testing point. Usability testing looks at five aspects of testing, - learnability, efficiency, satisfaction, memorability, and errors.
The compatibility part of a testing methodology tests that the product or application is compatible with all the specified operating systems, hardware platforms, web browsers, mobile devices, and other designed third-party programs (e.g. browser plugins). Compatibility tests check that the product works as expected across all the different hardware/software combinations and that all functionality is consistently supported.
When you can perform compatibility testing using automated tests, one feature that you should really consider having, is a way to do data-driven testing:
This lets you create sets of different test configurations that need to be tested, which are automatically generated by a system (e.g. give it all the browsers and all the platforms, and it creates an N by N matrix of all possible permutations). Then you can run all your tests quickly and easily through all the combinations to ensure there are no missed ‘edge’ cases.
What About Agile Testing Methods?
Agile projects have to test their systems in the same was as traditional waterfall projects, in terms of what kinds of test have to be accounted for. However, the activities will occur in smaller bursts as part of the iterations or sprints. In addition, the people doing the work may be testing specialists (e.g. performance, security) but will be fully embedded in the agile teams.
In addition, because agile projects deal with a might greater degree of uncertainty and change, you may not have a fully working system, written test cases (apart from high level acceptance tests), or even fully formed requirements (user stories tend to be quite high level). So there are some techniques that can help.
Exploratory / Session Based Testing
Exploratory testing is a useful approach, used in software testing that is about exploring - finding out about the software, what it does, what it doesn’t do, what works and what doesn’t work. The tester is required to make decisions about what to test next and where to spend a specified amount of time. Exploratory testing is most useful when there are unclear requirements (e.g. early in an agile sprint) or where you want to find the unknown issues that have not been uncovered by more formal testing methods.
Session-based testing is a variation of exploratory testing, where you have test charters and specific time-boxed “sessions” to conduct your activities.
SpiraCapture is a free tool from Inflectra that lets you record your web testing activities directly inside the Chrome browser.
Another aspect of agile projects is the move to development & operations (DevOps). As organizations embrace DevOps and have and to end pipelines of features from inception to production, testing needs to occur continuously throughout the DevOps pipeline:
Why Use SpiraTest for Your Test Management?
SpiraTest® manages your project's requirements, test cases, bugs and issues in one integrated environment, with full traceability throughout the testing lifecycle.
- It is a complete solution that includes requirements management, test , release management and defect tracking all fully integrated from day one.
- Highly intuitive web application that provides a complete picture of a project’s status and health yet requires only a web-browser.
- Ability to leverage your existing technology investments. SpiraTestintegrates with many automated testing solutions and third-party defect-management systems.
SpiraTest makes the managing and tracking of your cross-platform testing easy, it enabled you to quickly configure different test plans for the different hardware/software combinations and make sure that all of the requirements and test cases have been executed (and passed) against all of the specified combinations and configurations.
How do I Get Started?
To learn more about SpiraTest and how can be used to implement your testing methodologies please: