Overview
We have had many customers over the past 7 years use our free TestRail migration tool to migrate projects from various versions of TestRail to SpiraTest. However, in the past two months we have seen a massive uptick in the number of prospective customers looking to make the switch. There are many reasons why SpiraTest is a superior test management alternative to TestRail, but recently (based on what we've heard from the customers), it seems that the stability and usability of the platform has suffered a lack of investment. Whatever the reason, we realized that we need to improve our existing migration tool to bring over additional data as well as handle some API discrepancies between the different versions of TestRail.
In the following sections, we'll outline the enhancements and changes we have made in this new release.
Custom Property Support
The new importer adds support for importing the various types of TestRail test case custom field as SpiraTest custom properties:
The importer will correctly map the TestRail custom field type into the corresponding SpiraTest custom property type. It handles both global custom fields and those that are project-specific.
Similarly, the new importer also imports TestRail test result custom fields as SpiraTest test run custom properties:
For both test cases and test results there is special handling for the steps_separated and step_results fields. These are mapped into the SpiraTest native test step fields instead.
Customizable Execution Status Mapping
In the previous version of the importer, only the standard (out of the box) TestRail execution statues could be handled. However many customers had customized the statuses used in their projects. So in the updated importer we have added a new dialog box where you can customize the mapping of execution status between the two systems:
Of course if you haven't customized the values in TestRail, no change is necessary, we have populated the dialog with the out of the box TestRail values.
Mapping of Test Case Fields
The new importer automatically retrieves the list of test case templates in a project and uses those to create corresponding test case types in SpiraTest.
We chose to map TestRail test templates to SpiraTest test case types as they drive the workflows and the required fields in SpiraTest, similar to how fields are handled in TestRail.
Since we have already mapped SpiraTest test case types to TestRail templates, the new importer will create a new custom property in SpiraTest to store the list of TestRail test case types:
The new importer will also automatically import and map the test case priorities defined in the TestRail project.
This is in contrast to the old importer that assumed the test case priorities were on a scale of 1-4 only.
Populating Test Set Test Cases
In SpiraTest, you have the ability to create test sets and associate test cases with them. When you run the test sets in SpiraTest, the logged test runs are then associated with the test set and test case in question. TestRail on the other hand has test plans that you can execute, creating test results and associated tests. These test executions are linked to the test plan, not the test cases themselves. Due to this logical difference in how data is related in the two systems, the old import tool would import TestRail test plans as SpiraTest test sets, but would not include any of the test cases.
The new importer tool addresses this limitation by importing the test runs for a test case and when it finds an associated test plan, adds the test case in question to the corresponding test set test case instance in SpiraTest. As a result, when you look at a test set in SpiraTest, you will now see the various test cases that were executed as a part of it.
Attachment Handling
One of the biggest challenges migrating data from TestRail is that they have changed the API and storage format for attachments multiple times during its development. This means that there is no single, standardized way to retrieve attachments or screenshots related to a test case. In some cases, the test case would have image URLs embedded that referred to files that were not actually attached to the test case. We have worked with our customers who were running many different versions of TestRail (6.x, 7.x and 8.x) to identify all of the possible document storage and API options, and make sure the importer can handle all of them.
Cloud Rate Limiting Support
Finally, for customers using TestRail on the cloud, there is an API rate limit of 500 transactions per minute (unlike the cloud version of SpiraTest which has no such imposed limits). This rate limit prevented our old importer from successfully importing large projects that required lots of API calls. In the new version of the importer we have added code to identify when a rate limit message is returned by TestRail, and intelligently wait for 1 minute and then resume the import. This prevents the importer from failing dye to the TestRail API rate limits.