What is Sanity Testing? Definition, Examples, Process, & More
One of the various software testing types, sanity tests are an important tool in developers’ toolkits. Keep reading to learn more about what sanity testing is, how it differs from similar concepts like smoke or regression testing, key attributes, how to execute sanity tests, and more.
What is Sanity Testing?
Sanity testing in software development is a subset of the QA process that runs narrow tests after minor changes are made in order to check if specific functionalities, fixes, and changes are working as expected. This narrow focus on very targeted areas allows testers to verify functionality without exhaustively testing the entire system.
What are Sanity Tests Used For?
Sanity testing is best for small updates to code, but what specific situations are they typically used for? Common use cases of sanity tests include:
- Verifying bug fixes work as intended without side effects
- Confirming integrations with existing modules
- Checking regressions and stability before wider testing
- Validating minor UI changes for usability and functionality
- Confirming that APIs, databases, etc. are functional after an update
- Serving as a “checkpoint” in continuous workflows like DevOps
When Should Sanity Tests be Done?
Because of the inherently narrow focus of sanity testing, they’re typically performed after minor code changes or bug fixes. They can also be useful after smaller enhancements or updates are made to specific areas of the system (but not for small enhancements that apply to the entire system).
Sanity Testing Examples
Example #1: Bug Fix for Login Functionality
Situation/Context: There was a bug preventing certain users from logging into their accounts.
- Purpose of Sanity Test: Validate that the bug fix is functioning properly and confirm that it didn’t introduce new issues
- Scope: Test login with valid credentials as well as edge cases such as invalid username/password combinations
Components:
- Input Scenarios: Valid username and password, invalid username and password, blank username and password
- Expected Outcome: Successful login with valid credentials, error message for invalid or blank credentials
- Additional Checks: Successful login redirects users to the correct page, session starts properly without errors
Example #2: Improve Checkout Price Calculation
Situation/Context: Discounts weren’t being visibly applied properly during checkout.
- Purpose of Sanity Test: Confirm that discount amounts are visible to users during checkout and that the final price reflects this discount
- Scope: Validating discount calculation and that it displays in the checkout process, as well as payment integration to support this
Components:
- Input Scenarios: Add discounted product to cart, add multiple products with different discounts to cart, add full-price items
- Expected Outcome: The correct discounts are applied to all eligible products, total price reflects sum after discounts
- Additional Checks: Verify that the tax calculation is accurate, removing discounted items updates the total price correctly
Smoke Testing vs. Sanity Testing
While smoke and sanity tests both check for functionality and stability, smoke testing is done earlier in the build process to verify core functionality before further testing. Sanity testing happens later in the process and more narrowly concentrates on ensuring that specific changes haven’t led to new issues.
Smoke tests are broader, simpler, and earlier while sanity tests occur later and are narrower (but deeper).
Regression Testing vs. Sanity Testing
Sanity testing is a subset of regression testing — both check the impact of new changes to a system to confirm that they didn’t affect existing features and modules. However, regression testing verifies this at the application level (that the entire system works) vs. sanity testing verifies at the feature or functionality level. The scope of their testing also reflects the triggers for their use, as regression is typically done after major changes while sanity testing is done after smaller changes.
Attributes of Sanity Tests
Effective sanity tests typically share several key characteristics. These maximize their value and work together to form valuable tests:
- Narrow and deep
- Unscripted
- Undocumented
- Executed by testers
- Subset of regression testing
Narrow & Deep
We mentioned how sanity tests are narrower in focus and deeper in thoroughness than standard regression tests and smoke tests, and this is because it allows testers to comprehensively confirm that no unintended side effects came from tweaks and minor changes. The narrow focus means that tests and checks can be made quickly, not spending excessive resources on larger-scale testing for minimal updates. Because of this, effective sanity tests are the go-to verification method when small changes are made to modules or features.
Executed by Testers
Having test engineers with extensive knowledge and familiarity with the system perform sanity testing means that they can be more efficient. This also strips out the need for formal test cases and documentation (more on this below), further speeding up the process. More experienced testers intuitively know what parts might be affected by changes, allowing for more targeted testing and identification.
Unscripted
The purpose of sanity tests is to be quick, and spending time creating detailed test cases and scripts would hamper this. As a result of sanity testing being done by experienced testers, they can use their knowledge to dynamically perform tests rather than leaning on a predetermined script or agenda. This agility is incredibly helpful for modern software development teams, where adaptability and continuous updates are the norm.
Undocumented
Another benefit of having well-versed testers handle sanity testing is that the need for extensive documentation is removed. The goal is simply to ensure that the relevant pieces still function as intended, so additional documentation is largely unnecessary. The faster cycle times achieved are once again particularly useful for Agile team methodologies and fast-paced development.
Subset of Regression Testing
Lastly, it’s important to remember that sanity testing is a subset of regression testing. While they may focus on different areas (sanity tests on less critical components and regression tests on the entire application), they work together to enhance your test coverage at bird’s-eye and granular levels. Picking and choosing when to employ one vs. the other is critical for resource optimization.
Benefits of Sanity Testing
Some of the advantages that sanity testing provides include:
- Improves test efficiency: As we’ve discussed, the narrow scope of sanity tests allows for minimal documentation and preparation, meaning that they can quickly catch issues pertaining to recent updates.
- Offers rapid feedback: Because of their speed, sanity tests provide much faster feedback than larger-scale testing efforts, providing clear and rapid confirmation of whether a change was successful or not.
- Validates critical fixes: Before moving on to system testing, sanity tests ensure that critical defects have been resolved and relevant functionalities have remained intact.
- Manages risk of major issues: Acting as a checkpoint, these tests catch broken functionality before deeper testing or deployment begins, minimizing future downtime that is often more expensive and difficult to fix.
- Complements other testing: Sanity tests balance speed and thoroughness, offering a more targeted approach to evaluating more narrow areas when larger-scale testing would be an inefficient use of resources.
- Increases team confidence: Targeted efforts by testers who are deeply familiar with the system and functionalities reassure the team that the application is stable.
- Fits Agile workflows: The speed of sanity tests makes them ideal for Agile environments where development is fast-paced and iterative, building off rapid feedback.
Disadvantages of Sanity Tests
There are some downsides to using sanity tests — however, most of the drawbacks are by design and can help determine whether sanity testing is the best type to use. While the narrower scope can help speed, it can miss other important areas and lead to defects remaining undetected. The lack of documentation also speeds up the process but can limit learnings for future testing and make it more difficult to track exactly what was tested.
Another disadvantage of relying on tester intuition for speed is that depending on their knowledge and background, issues can be overlooked. Less experienced testers may miss edge cases, which reduces the effectiveness of sanity tests. These tests also primarily focus on functional issues — non-functional testing is still necessary to cover other areas of quality assurance.
Sanity Testing Process
Like all software testing, sanity tests follow a few basic steps from identification to execution. Using these as a guide will help maximize the value of your tests and minimize the risk of overlooked issues.
1. Identify
Begin by considering what changes, fixes, or updates need to be validated in order to determine the scope of testing. Testers will need to fully understand the changes they’re testing, so review the code updates, any associated development notes, change logs, etc. From there, identify what modules or functionalities are affected by the changes (and make sure to consider any dependencies). The last piece of this step is establishing what the goal of the sanity tests will be, based on the information gathered. This could be confirming a bug fix works without introducing new defects or verifying that a minor feature integrates properly with the existing API.
2. Evaluate
From there, confirm that the current build is stable enough for effective testing. It should have basic functionality and critical/impacted workflows need to be operational. Check if there are any obvious defects that could prevent test execution (such as a disabled login). If the system is fit to be tested, prioritize the areas that need immediate validation.
3. Test
Once you have a prioritized list of features or modules, begin executing tests to validate their functionality. Confirm that all relevant areas work as expected and haven’t caused any new concerns. Although sanity tests have minimal documentation and are relatively informal, it’s important to share the results of testing with the rest of the team and any other stakeholders.
Enhance Your Software Testing
Are you struggling to keep track of all the necessary testing types and steps for your software projects? Tools like Rapise are designed to streamline and enhance your testing process with powerful features such as automation, self-healing tests, validation, AI-powered improvements, and intuitive dashboards. Want to learn how Rapise can supercharge your testing process? Learn more about its suite of features, hear from current users, or start a free trial today!