June 20th, 2017 by Thea Maisuradze

Web applications are a perfect example of a rapidly moving target for UI test automation tools. Agile and DevOps drive small changes and frequent releases. This leads to certain challenges in QA field. How to create tests that survive after small UI changes? How to create tests that produce same results if ran 10 times in a row on an unmodified application? How to create tests that check exactly those features they were designed to check?

Answers to these questions are not entirely simple. Test automation is still an art.

In this blog we try to figure out what every test automation engineer can do to find the right answers. First I'll give a few examples that illustrate common challenges in Web UI test automation.

Common Challenges

Selectors

Assume that you have a page with Log In button and the test relies on

/html/body[1]/form[1]/div[3]/div[2]/div[2]/p[1]/input

xpath expression to find it. Imagine that a new element is inserted before this button. The xpath expression (selector) becomes invalid and the test breaks. Is it a good selector? I would say No. Can we do better? Sure, if the button has ID attribute specified we can use

//input[@id="MainContent_LoginUser_LoginButton"]

With this selector no matter what is inserted/modified on the page - if Log In button is there - the test will find it.

So, selectors matter.

Delays

Every Web application loads data from somewhere. It can be a database, third-party REST service or a banner network. And nobody knows how fast a web page is loaded and its elements are ready to use. So, an automated test must take into account various delays and be patient. Otherwise, the test may be useless as it will provide pass and fail results under the same circumstances.

Assertions

Tests perform checks. Aristotle would call it telos of tests. Let's compare two assertions:

Purchase button is displayed in the middle of a screen.
Purchase button center is at x=1440, y=900.

Which one is better? Of course, the second assertion may lead to many confusions with the test. It will fail on a screen with a smaller resolution, it may produce a false positive result on a 5K Retina display.

The second assertion is weak. And the first one is strong.

Practice

A good test is a resilient test with strong assertions, it must meet the challenges presented above. Of course, there may be exceptions, but they are a minority.

Resilient Tests

What may help with building resilient tests for Web applications?

Private Browsing

All modern browsers have private/incognito mode that makes you feel that you are running a web application for the first time. No cache, no cookies. It may help an automated test to enter the same water again and again and produce a steady result.

XPATH Generators

In Rapise - test automation tool that we develop - we have an XPATH generator integrated into Spy which automatically produces several xpath alternatives for a given DOM node. For example:

/html/body[1]/form[1]/div[3]/div[2]/div[2]/p[1]/input
(//input)[9]
//input[@id="MainContent_LoginUser_LoginButton"]
//input[@value='Log In']

By looking at xpath variants a test engineer can quickly choose the best one or produce a derivative.

Alternative Selectors

It is nice to have an ability to specify a set of selectors that may be used to identify a particular element in the application. If some selectors are broken there is a chance that at least one is still valid.

Wait Statements

Use wait statements preferably with a timeout to delay test execution util an element is available for interaction. Such waits can be explicit or implicit if, for example, a test engine tries to find an element on a page several times with a specific delay between lookup attempts.

Physical and DOM Click

Sometimes passing a click using DOM event does not work. In this case, it is useful to have a way to simulate a physical click.

Automation Friendly Application

Software developers may also help with building resilient tests. They can make life of QA teams much easier by

Assigning IDs to key elements on a page.
Using wait cursors, status bars, progress bars, beach balls.
Using ARIA

Conclusion

Building resilient tests is a key to successful UI test automation. If your tests are brittle better go with manual testing approach otherwise you'll spend too much time fixing those failing tests. Of course, right selection of an automation tool is also important. It should support you down the road of resiliency.