Introduction
In her provocative but timely article on codeless test automation, Angie Jones provides the history of previous efforts at codeless test automation (often called record and play test automation) giving insights into why they have failed, and suggesting ten features that the new batch of codeless (or scriptless as some would prefer) tools need to have to avoid a similar fate.
Angie also correctly divines that without such tools, the problem is that in the competitive market for programmers, it will become increasingly hard to find enough automation engineers (read programmers who write test automation frameworks using APIs such as Selenium WebDriver) to satisfy the demand for reliable automated testing. When you couple that with the problem of maintaining someone else’s test automation framework, the benefit of easier to maintain codeless tools becomes apparent.
Rapise – Powerful Codeless Test Automation
Before we look at each of the ten features that such tools need to have, we firstly need to introduce Inflectra’s codeless test automation tool – Rapise:
Rapise is a leading “next-generation” test automation platform, designed specifically for agile projects where you have interdisciplinary teams, short release cycles, and constantly changing applications.
Although the focus of the original article (and this whitepaper) is on web testing, Rapise applies the same tenets and features to all forms of UI testing – mobile and desktop – as well as web applications.
As we look at the ten features that test automation tools should have, we will be using our sample web application (www.libraryinformationsystem.org) as our example. This is a public web application hosted by Inflectra that you are welcome to try testing with Rapise or any tool of your choosing.
So, to answer the question in Angie’s article: “How do we enable testers who aren't programmers to do test automation?”, with Rapise you can empower your testers to do test automation, and where necessary involve your programmers and automation engineers to help them out with some of the edge cases.
As a codeless automation supplier, we agree that for Rapise (or any tool) to have any chance of long-term success, it needs to solve the pain points of the older “record and play” tools and provide the following capabilities to ensure that history does not repeat itself:
1. Smart Element Locators
One of the biggest problems with test automation is that testers develop against an application at a given moment in time. When the application changes, those automation scripts may break. (This is true of both coded and codeless automation frameworks.)
There's an opportunity for codeless solutions to have an advantage over coded frameworks by gathering more than one element locator during the recording. In the event that a locator is no longer found during playback, the alternative locators for that element can be tried. This gives the tester a far better chance to develop stable scripts that don't require as much maintenance.
Unlike hand-written test automation code, everything in Rapise is “object-based”, so regardless of whether the application being tested is web, mobile, or desktop, Rapise defines a list of reusable UI objects that are used in your test scenario. This separation of object from test flow is key, it allows you to create maintainable tests. However, that is only part of the story; each object uses a series of “smart locators” to identify the object on the page, allowing Rapise to more easily cope with changes in the application:
In addition, Rapise uses multiple locators for finding an object on the page. Rapise will try each of the locators that it has learned in turn, so that even if the application has changed, and some of the locators fail, Rapise will still be able to find it:
Rapise goes a step further, it allows you to define “application profiles” that allow you to “train” Rapise to understand the specific nuances of your application, so that you can tell it what attributes and elements are significant, and which can be ignored. For example, in some applications the ID values of elements are stable and make good anchor points for locating elements on the page, whereas in other applications, the IDs contain session values which change on every page load (!).
Finally, for more complex applications such as ERP and CRM systems (e.g. Microsoft Office365, Dynamics365 and Salesforce), Rapise includes “custom libraries” where Inflectra automation engineers have prepared “accelerators” that allows Rapise to recognize complex controls on the page such as grids, treeviews and menus, making them easier to automate and much simpler to maintain.
For example, instead of Rapise recording that you clicked on a hyperlink inside a table, it records that you added a row to new Salesforce Lead entry line. These custom libraries are completely extensible and can be written by your own automation engineers and then used by non-programmers.
Finally, we have a feature you can enable, called “self healing tests”. When running in this mode, Rapise learns multiple possible ways to find an element on the screen, and then using its machine-learning algorithms, determines probabilistically during playback if there is a match or not:
This means that over time, Rapise will begin to learn when a design change in the application caused the change in behavior, as opposed to the application not working correctly.
2. Conditional Waiting
Record-and-playback tools that require script authors to insert pauses between steps not only are tedious to use, but promote a test automation anti-pattern as well.
A more modern approach, called conditional waiting, has proved quite reliable. In this approach, scripts don't blindly wait x number of seconds before continuing to the next step; instead they wait until a condition is true and then proceed as soon as possible.
This drastically cuts down on the execution time of the automation suite while also preventing flaky tests. Codeless tools should, at the very least, allow script authors to insert conditional waiting, but it would be even better if the tools took care of this behind the scenes.
Rapise includes various options for handling the playback speed of your tests; firstly, it lets you define the standard playback speed, together with the number of times to retry finding an object, as well as the time to spend between each attempt:
However, it is also very useful to be able to have Rapise wait conditionally for either an object to become available, or for a specific attribute to be set on the property (e.g. wait until the button is visible or enabled). With Rapise you can easily do this by dragging on the Global utility methods to your test:
In addition, Rapise has some automated checks built-in where it knows to wait until an element is available in the application, connected to the browser DOM, and not in a hidden layer so that in may cases you can simply record events and actions and let Rapise do the rest.
3. Control Structures
One of the major limitations of older record-and-playback tools was the absence of control structures, such as loops and conditional clauses. A script author who needed to repeat an action 10 times would need to record that action 10 times and maintain each one of the actions individually.
Also, if there were multiple viable alternatives within a step and the test author wanted to take one action if a certain condition was true, and a different action if it was not, there was no support for such logic.
In coded automation frameworks, testers use loops and if-else clauses to control the flow of scripts. This is a necessary requirement for codeless solutions if the project is to be scalable.
Rapise’s unique “Rapise Visual Language” (called RVL) was designed from the outset to provide the key control structures for writing tests that can test multiple branches and handle varying test data. Firstly, within Rapise you have the ability to define a set of data called a “Map”:
When you create a Map, Rapise generates a data set that you can populate with different values that need to be tested:
Now you can simply wrap the appropriate sections of your test scenario with a loop and associate it with the Map:
Rapise will then convert the section of test into a looped section, associated with the Map. You then just need to use the dropdown menus to replace the hardcoded values with the columns from the Map:
Using if…then…else branches to control the flow in the test is just as easy, you use the option to wrap selection into if to specify what happens in each branch:
Rapise includes a wide variety of control conditions that let you do various types of comparison, “fuzzy” matches and range checks, all without any code or programming script:
4. Easy Assertions
Recording a scenario is fairly easy to do. As the user takes action on an application, the tool records those actions. However, when a user validates the actions, that does not usually include a web action. It’s something that you do with your eyes.
Because of this, adding assertions within codeless automation tools has always been a pain, requiring manual entry by the user to describe what needed to be validated. This is neither intuitive nor easily expressed.
The process of adding assertions should be as simple as adding the navigational steps. Great consideration should be given to this feature, since it's the most important part of the test script.
Rapise makes assertions and verification as easy as recording the initial steps. In fact you simply use the Verify button (or the convenient keyboard shortcut CTRL+1) during recording to add verification steps. For example, after logging into our sample application we want to verify that the user is in fact logged in by checking the profile text:
Rapise takes these verification actions and converts them into codeless assert statements:
You can then modify the assert directly in the test scenario by simply changing the dropdown. For example, you may want to change the comparison of login names to be a text broad match rather than an exact match:
Rapise makes it easy to adapt anything that was recorded by simple drag and drop or menu operations.
5. Modification Without Redo
Most older record-and-playback tools did not allow authors to easily edit recorded scenarios. If the flow of a scenario changed and you needed to insert a new action in the middle of the scenario, you had to re-record the entire scenario. That's not ideal.
As the application under test changes, the test scripts need to change as well. This may require adding or removing steps from a flow, or slightly changing existing ones. Codeless automation tools should recognize this as inevitable and make it as simple as possible to accomplish this.
Rapise’s Visual Language makes the maintenance and modification of test scripts very easy.
Firstly, if the objects on the page have changed sufficiently that you need to remap an object in Rapise to an object on the page, you don’t even need to change your test script at all, just “re-learn” the object:
Next, when you record new steps, Rapise gives you the option of either appending them to the current test script or inserting them at a specific location:
Finally, you can of course just drag and drop objects and actions from the object tree or choose them from the test script grid to create additional steps or modify existing steps without any recording at all:
Whichever option you prefer, modifying test scripts in Rapise never require redoing the entire recording.
6. Reusable Steps
Some steps exist in multiple scenarios. For example, logging into an application may be something that the majority of tests must do. Recording this action in each of those tests is what leads to maintenance nightmares. If anything changes with the login step, the test author would need to update every test that contains this.
Codeless tools should allow authors to record common steps that they can then insert into any test flow. This would not only save time in authoring, but also promote reusability and make maintenance much simpler. Similarly, the tools should support the use of variables within tests to allow for reuse of state between the steps.
Rapise provides many options for reusable steps, including the ability to break a single test project into multiple smaller test projects called “Sub-Tests”:
You can then simply call sections of the sub-test from the main test:
However, one of the simplest options that Rapise provides is the ability to break down your test scenario into reusable sections (called test sheets) that can be called from the main test flow:
Rapise of course lets you reuse variables and test data between the various different test sections.
7. Cross-Browser Support
Many record-and-playback tools come in the form of a browser extension, which means they may be tied to a specific browser. But many product teams must run tests across multiple browsers. Test authors should be able to record a scenario once on a given browser, and be able to play that recording on any other major browser.
Rapise lets you record and execute tests in multiple browsers without any changes to your test script using its extensible set of browser libraries and profiles. Rapise lets you record in either Firefox, Chrome or Internet Explorer:
In addition, for playback, Rapise lets you play back your tests against a greater selection of web browsers (e.g. Edge, Safari, Opera) using the industry standard Selenium Web Driver API:
For further flexibility, Rapise lets you use different web browser profiles in case you want to test different versions of the same browser or launch it with a specific setting (e.g. run IE in safe mode with no add-ons, or run Chrome in Private browsing mode):
Of course, Rapise can test more than just web applications: its codeless methodology and object based recording can be used to also test mobile apps (iOS or Android) and desktop applications that use a variety of technologies (Windows, Java, RCP, Qt, PowerBuilder).
8. Reporting
Adequate reporting is a must-have for test automation, especially when testers are executing hundreds—or thousands—of tests. Triaging any failures should not require reruns or extensive debugging. An automatically generated report with details on what occurred, screenshots, and even a video of the recording would be very helpful.
Rapise comes with out of the box automated reporting that captures the pass/fail from every action and assertion / condition in your test script:
When tests fail, it is of course useful to see screenshots of what happened when the test run. Rapise includes built-in screenshot capture with the ability to display the actions as a “screen flow”:
In addition, when you use Rapise with SpiraTest, our enterprise test management system, you can securely store and version your automated tests as well as have a central web-based repository of all your test resulst:
SpiraTest also gives you world class test reporting, metrics and dashboards:
9. Ability to Insert Code
No codeless tool can incorporate everything that's possible. So, test authors should have the ability to let testers insert code within the tests for those edge cases. For extra points, don't limit this to UI-specific code, such as JavaScript. Many teams are looking at models such as the Test Automation Pyramid and exercising certain actions as web services instead of UI actions. This will allow for faster, more reliable test scripts.
In addition to its primary codeless test creation methodology (RVL), Rapise also provides a full-blown Integrated Development Environment (IDE) for creating functions and modules in code that can be executed as steps from within the codeless test scenarios:
These coded functions can be written by skilled test automation engineers as reusable elements, and then incorporated into the main test scenarios in exactly the same way as any of the standard learned objects:
The Rapise IDE provides all the key features needed to write code fragments, including code-completion, code-folding, intellisense, and an integrated debugger with breakpoints and watches:
In many cases, testers want to be able to test multiple layers of the application from within the same test scenario. To that end, Rapise includes a full web service and API testing module that lets developers write object-based web service (REST and SOAP) tests which can be called from the primary codeless interface:
10. Continuous Integration
Older record-and-playback tools required testers to click a button within the tool to execute their tests. In the era of DevOps, test automation is a key ingredient in continuous integration and continuous deployment. Tests should integrate with such pipelines and automatically execute when triggered. For faster execution times, tests should be capable of running in parallel as well.
Rapise can be integrated with a wide variety of DevOps, ALM, CI and test management systems. For example, Rapise integrates with our SpiraTeam application lifecycle management (ALM) solution which allows distributed scheduling and execution as well as integration into DevOps pipelines and toolchains:
In addition, Rapise comes with pre-build libraries and add-ons for integrating automated UI tests into the most common Continuous Integration (CI) platforms such as Jenkins, TeamCity, TFS, and Bamboo:
Rapise also integrates with a wide variety of cloud-based testing platforms that simplify the management of test environments and devices (especially when dealing with native app or web testing on mobile platforms), including SauceLabs, BrowserStack and Kobiton.
Conclusion
We believe that Rapise provides the benefits of codeless test automation in terms of empowering testers who are not programmers by trade to be able to gain the benefits of test automation without the pain of hard to maintain, ossified test recordings that cannot be modified, reused, componentized or scaled.
Rapise lets testers do what they do best, which is - understand the application and the users expectations and ensure that all of the possible paths and journeys are tested – and lets programmers and automation engineers do what they do best, which is – write testing logic, write extensions and functions for the more complex edge cases that cannot be done (yet) without any code.