June 29th, 2021 by Adam Sandman
Two of the major features in our most recent release of Rapise are the completely overhauled Spira Dashboard and the new support for handling the bane of test automation engineers everywhere - flaky tests! In this article we shed some light on how Rapise 7.0 can make flaky tests a thing of the past.
Flaky Tests
If you find yourself in a situation that some of your tests pass or fail randomly then you have the curse of the "Flaky Test".
In Rapise 7.0 we have introduce a new feature that allows Rapise and Spira to work together to avoid this curse. In the new Rapise 7.0 Spira Dashboard, there is now a way to mark it as Flaky. Simply find it in Test Cases View and use the Actions menu to set the flag. Flaky test cases are indicated with a special icon (red flag):
When a test case marked as Flaky is executed by RapiseLauncher - its test run is marked as Flaky too.
Note
The Flaky flag makes it easier to distinguish between random and important test run failures.
Automatic Rerun
To instruct RapiseLauncher to rerun a failed test case use the Max rerun attempts test set custom property. If a test case keeps failing it will be additionally executed a specified number of times (by default it is zero). To stop rerunning tests if many of them finally failed use the Do not rerun if N tests failed property. By default this property is zero and it means rerun all the tests in a test set if Max rerun attempts is set.
Test runs that correspond to reruns are also decorated with special icons. The number in the circle designates the rerun attempt:
We hope you enjoy the new Flaky Test / Rerun features in Rapise 7.0, hopefully they should ease the curse of the dreaded flaky test!