Sequential Test Execution on Unique Hosts
RemoteLaunch was originally designed (along with SpiraTest) to have each unique test set (containing test cases) assigned to a specific automation host:
This meant that if you had two sets to execute, you would assign each test set to run on a unique machine (automation host) individually.
When that happens, RemoteLaunch downloads the entire test set and proceeds to execute each of its test cases in turn sequentially (based on the order of test cases in the test set):
- Each host downloads the entire test set and executes from start to finish
- If you schedule two test sets for the same host, it will execute them one at a time depending on which one it downloads first
- This is a reliable and deterministic case
Test Execution on Shared Hosts
However, some of our customers have more sophisticated test environments where they have:
- Multiple machines available to run the automation test cases
- Large numbers of automated test cases that can be run in parallel, independent of each other.
There was a partial solution available in previous versions of RemoteLaunch...
Standard Mode (RemoteLaunch 5.0)
This is not how RemoteLaunch was designed, but a side effect use case allows you to run a test set on a choice of multiple machines:
- You can assign multiple machines to the same ‘host token’
- Both hosts will check for the same tests and one will get the test set first.
However if the second host gets the test set before the first one has a chance to mark it as in-process, sometimes they will both try and run the same test set at the same time, with unpredictable results.
So this is not always a reliable use case, and although it could work in some very simple cases, it was not a reliable solution.
So, with RemoteLaunch 6.0 we introduce a new, load balanced execution option.
Load-Balancing Mode (RemoteLaunch 6.0)
Inside RemoteLaunch 6.0 we have added a new flag in the Client Setup screen:
To use this new option, you will need to enable this flag on all of the RemoteLaunch machines that will take part in the "test pool".
Now, inside Spira you simply create an automation host token for the pool (rather than individual machines).
You then assign a single test set (containing many test cases) to the test pool:
Now, each of the automation hosts (running RemoteLaunch) that belong to that pool will execute each of those test cases in turn (following the sequence) as the machines become available:
The tests will always be taken from the test set in the correct, deterministic sequential order, but they will be distributed dynamically to each of the available automation hosts in the specific automation host pool.