Unit Test Frameworks

The first stage of ensuring a quality software product is to ensure that the development process makes use of automated, repeatable unit tests. SpiraTeam can integrate with all the major xUnit unit test frameworks to ensure that all features are being adequately covered.

.NET - NUnit


NUnit is a unit test framework for the Microsoft .NET platform that is based on the popular xUnit paradigm. Using NUnit allows you to develop automated unit test fixtures that can be used to either regression test an existing application or to define the intended functionality prior to development (using the Agile recommended test-driven development (TDD) approach).

When you use the SpiraTeam Add-In for NUnit it enables you to run automated NUnit tests against a .NET application and have the results be recorded as test runs inside SpiraTeam. This allows you to track your automated unit tests against the different releases in SpiraTeam and to have complete visibility of the overall requirements test coverage.

Note: This integration requires at least version 2.2 of SpiraTeam and version 2.5.5 of NUnit.

Integrating the Add-In With Your Tests

The NUnit add-in works by looking for additional test attributes that map the NUnit test fixtures to SpiraTeam test cases:

/// <summary>

/// Sample test that asserts a failure

/// </summary>

[

Test,

SpiraTestCase (<test case id>)

]

public void _01_SampleFailure()

{

//Failure Assertion

Assert.AreEqual (1, 0);

}

Then when the test is executed by NUnit, the Add-In will capture the results from the unit tests and report them back as automated test executions in SpiraTeam.

.NET - Visual Studio Unit Testing


Microsoft Visual Studio Team System (MS VSTS) Test (also known as MS-Test) is a unit test framework for the Microsoft .NET platform that comes built-in to Visual Studio Professional Edition. Using MS-Test allows you to develop automated unit test fixtures that can be used to either regression test an existing application or to define the intended functionality prior to development (using the Agile recommended test-driven development (TDD) approach).

When you use the SpiraTeam Extension for MS-Test it enables you to run automated unit tests against a .NET application and have the results be recorded as test runs inside SpiraTeam. This allows you to track your automated unit tests against the different releases in SpiraTeam and to have complete visibility of the overall requirements test coverage.

Note: This integration requires at least version 2.3 of SpiraTeam and Visual Studio 2008 Professional Edition.

Integrating the Extension With Your Tests

The MS-Test extension works by looking for additional test attributes that map the unit test fixtures to SpiraTeam test cases:

/// <summary>

/// Sample test that asserts a failure and overrides the default configuration

/// </summary>

[

TestMethod,

SpiraTestCase(5),

SpiraTestConfiguration("http://localhost/SpiraTest", "fredbloggs", "fredbloggs", 1, 1, 2)

]

public void SampleFail()

{

//Verify the state

Assert.AreEqual(2, testFixtureState, "*Real Error*: State not persisted");

//Failure Assertion

Assert.AreEqual(1, 0, "Failed as Expected");

}

Then when the test is executed by MS-Test, the extension will capture the results from the unit tests and report them back as automated test executions in SpiraTeam.

Java – Junit


JUnit is a unit test framework for the Java platform that is based on the popular xUnit paradigm. Using JUnit allows you to develop automated unit test fixtures that can be used to either regression test an existing application or to define the intended functionality prior to development (using the Agile recommended test-driven development (TDD) approach).

When you use the SpiraTeam Extension for JUnit it enables you to run automated JUnit tests against a Java application and have the results be recorded as test runs inside SpiraTeam. This allows you to track your automated unit tests against the different releases in SpiraTeam and to have complete visibility of the overall requirements test coverage.

Note: This integration requires at least version 3.0 of SpiraTeam and version 4.0 of JUnit.

Integrating the Extension With Your Tests

The JUnit extension works by looking for additional test annotations that map the JUnit test fixtures to SpiraTeam test cases:

/**

* Tests the addition of the two values

*/

@Test

@SpiraTestCase(testCaseId=5)

public void testAdd()

{

double result = fValue1 + fValue2;

// forced failure result == 5

assertTrue (result == 6);

}

Then when the test is executed by JUnit, the extension will capture the results from the unit tests and report them back as automated test executions in SpiraTeam.

Java – TestNG


TestNG is a unit test framework for the Java platform that is based on the popular xUnit paradigm. Using TestNG allows you to develop automated unit test fixtures that can be used to either regression test an existing application or to define the intended functionality prior to development (using the Agile recommended test-driven development (TDD) approach).

When you use the SpiraTeam Listener for TestNG it enables you to run automated TestNG tests against a Java application and have the results be recorded as test runs inside SpiraTeam. This allows you to track your automated unit tests against the different releases in SpiraTeam and to have complete visibility of the overall requirements test coverage.

Note: This integration requires at least version 3.0 of SpiraTeam and version 1.0 of TestNG running under JDK 1.5 or later.

Integrating the Listener With Your Tests

The TestNG listener works by looking for additional test annotations that map the TestNG test fixtures to SpiraTeam test cases:

/**

* Tests the addition of the two values

*/

@Test

@SpiraTestCase(testCaseId=5)

public void testAdd()

{

double result = fValue1 + fValue2;

// forced failure result == 5

assertTrue (result == 6);

}

Then when the test is executed by TestNG, the listener will capture the results from the unit tests and report them back as automated test executions in SpiraTeam.

Python – PyTest


pytest is a mature full-featured Python testing tool that helps you write better programs. The pytest framework makes it easy to write small tests, yet scales to support complex functional testing for applications and libraries.

When you use the SpiraTeam Extension for PyTest it enables you to have a simple way to ensure your code works properly. We provide fine-tuned control over your test runs as well! You can associate a test function in pytest with a test case inside SpiraTeam, and even add releases and test sets to your test run as well!

Integrating the Extension With Your Tests

Getting started is quick and simple – it should take less than five minutes! All you need is to install the SpiraTeam PyPI package and create a config file, and you’re done! No longer do you need to worry about spending the time to put in test results manually – we’ll handle it for you!


Then when the test is executed by pytest, the extension will capture the results from the unit tests and report them back as automated test executions in SpiraTeam.

Python – PyUnit


PyUnit is a unit test framework for the Python scripting language that is based on the popular xUnit paradigm. Using PyUnit allows you to develop automated unit test fixtures that can be used to either regression test an existing application or to define the intended functionality prior to development (using the Agile recommended test-driven development (TDD) approach).

When you use the SpiraTeam Extension for PyUnit it enables you to run automated PyUnit tests against a Python application and have the results be recorded as test runs inside SpiraTeam. This allows you to track your automated unit tests against the different releases in SpiraTeam and to have complete visibility of the overall requirements test coverage.

Note: This integration requires at least version 2.3 of SpiraTeam.

Integrating the Extension With Your Tests

The PyUnit extension works by looking for a special numeric suffix on each of the test methods that map the PyUnit test fixtures to SpiraTeam test cases:

def testchoice__3(self):

element = random.choice(self.seq)

self.assert_(element in self.seq)

releaseId = 1

testSetId = 1

spiraTestExtension = spiratestextension.SpiraTestExtension()

spiraTestExtension.projectId = 1

spiraTestExtension.server = "localhost"

spiraTestExtension.port = 80

spiraTestExtension.path = "SpiraTest"

spiraTestExtension.userName = "fredbloggs"

spiraTestExtension.password = "fredbloggs"

spiraTestExtension.recordResults(TestSequenceFunctions, testResult, releaseId, testSetId)

Then when the test is executed by PyUnit, the extension will capture the results from the unit tests and report them back as automated test executions in SpiraTeam.

NodeJS – UnitJS


UnitJS is an assertion library for JavaScript, running on Node.js or a web browser. It works with various test runner and unit testing frameworks, including Mocha, Jasmine, Karma, Protractor, and QUnit.

SpiraTeam currently includes a pre-built extension for the MochaJS test runner and our sample code illustrates it working with UnitJS. However, we supply the source code to the extension, so you can easily adapt it for other runners such as Jasmine and Protractor.

NodeJS – Mocha


Mocha is a feature-rich JavaScript test framework running on Node.js and in the browser, making asynchronous testing simple and fun. Mocha tests run serially, allowing for flexible and accurate reporting, while mapping uncaught exceptions to the correct test cases.

SpiraTeam currently includes a pre-built extension for the MochaJS test runner and our sample code illustrates it working with UnitJS. However, it can be used as a general extension that works with any unit test framework that supports the MochaJS runner.

NodeJS - JasmineJS

Our extension for Jasmine lets you integrate SpiraTeam seamlessly into your web browser and/or NodeJS testing environment. You can tell Jasmine to report back the results of your automated tests directly to SpiraTeam in addition to logging to the console of the machine running the test.


Jasmine is a behavior-driven development framework for testing JavaScript code. It does not depend on any other JavaScript frameworks. It does not require a DOM. And it has a clean, obvious syntax so that you can easily write tests.

The new Jasmine reporter will integrate JasmineJS with SpiraTeam. It will create a new test run in SpiraTeam for each test spec executed in Jasmine. All you need to is install the NodeJS package and then define the credentials and tests to be integrated using simple JSON:

describe("Test having two specs", () => {

var SpiraReporter = require('jasmine-spiratest');

jasmine.getEnv().addReporter(new SpiraReporter({

"url": "https://doctor/SpiraPlan",

"username": "fredbloggs",

"token": "{XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX}",

"projectId": 1,

"releaseId": 1,

"testSetId": 1,

"testCases": {

"default": 20,

"equality works": 21,

"addition works": 16

}

}));

describe("Test basic JavaScript", () => {

it("Equality works...", () => {

expect(2).toEqual(2);

});

it("Addition works", () => {

expect(3 + 2).toEqual(5);

});

it("Multiplication works", () => {

expect(4 * 5).toEqual(20);

});

});

});

NodeJS – Jest

Jest is a delightful JavaScript Testing Framework with a focus on simplicity. It works with projects using: Babel, TypeScript, Node, React, Angular, Vue and more!


Jest is a JavaScript testing framework designed to ensure correctness of any JavaScript codebase. It allows you to write tests with an approachable, familiar and feature-rich API that gives you results quickly.

The new Jest reporter will integrate Jest with SpiraTeam. It will create a new test run in SpiraTeam for each test executed in Jest. All you need to is install the NodeJS package and then define the credentials and tests to be integrated using simple JSON:

"reporters": [

"default",

[

"jest-spiratest",

{

"url": "https://doctor/SpiraPlan",

"username": "fredbloggs",

"token": "{XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX}",

"projectId": 1,

"releaseId": 1,

"testSetId": 1,

"testCases": {

"default": 20,

"storesCorrectly": 21,

"javascriptWorks": 16

}

}

]

]

Ruby – Test::Unit


Test::Unit is a unit test framework for the Ruby scripting language that is based on the popular xUnit paradigm. Using Test::Unit allows you to develop automated unit test fixtures that can be used to either regression test an existing application or to define the intended functionality prior to development (using the Agile recommended test-driven development (TDD) approach).

When you use the SpiraTeam custom runner for Test::Unit it enables you to run automated Test::Unit tests against a Ruby application and have the results be recorded as test runs inside SpiraTeam. This allows you to track your automated unit tests against the different releases in SpiraTeam and to have complete visibility of the overall requirements test coverage.

Note: This integration requires at least version 2.3 of SpiraTeam.

Integrating the Custom Runner With Your Tests

The Test::Unit custom runner works by looking for a special numeric suffix on each of the test methods that map the Test::Unit test fixtures to SpiraTeam test cases:

#this is a test case that tests addition operations

class TC_Adder < Test::Unit::TestCase

def setup

@adder = Adder.new(5)

end

def test_add__2

assert_equal(7, @adder.add(2), "Should have added correctly")

end

def test_addfail__3

assert_equal(7, @adder.add(3), "Test failure")

end

def teardown

@adder = nil

end

end

#this is a test suite that calls the test case

class TS_Examples

def self.suite

suite = Test::Unit::TestSuite.new

suite << TC_Adder.suite

return suite

end

end

projectId = 1

releaseId = 2

testSetId = -1

testRunner = Test::Unit::SpiraTest::TestRunner.new(TS_Examples, "http://servername/SpiraTest", "fredbloggs", "fredbloggs", projectId, releaseId, testSetId)

testRunner.start

Then when the test is executed by Test::Unit, the custom runner will capture the results from the unit tests and report them back as automated test executions in SpiraTeam.

Perl – Test Anything Protocol (TAP)


Perl’s Test Anything Protocol (TAP) is a unit test framework built into the Perl scripting language. Using TAP allows you to develop automated unit test harnesses that can be used to either regression test an existing application or to define the intended functionality prior to development (using the Agile recommended test-driven development (TDD) approach).

When you use the SpiraTeam Extensions for TAP it enables you to run automated TAP test harnesses against a Perl application and have the results be recorded as test runs inside SpiraTeam. This allows you to track your automated unit tests against the different releases in SpiraTeam and to have complete visibility of the overall requirements test coverage.

Note: This integration requires at least version 2.3 of SpiraTeam.

Integrating the Extensions With Your Test Harnesses

To use SpiraTeam with TAP, each of the TAP test case files needs to have a corresponding test case in SpiraTeam. These can be either existing test cases that have manual test steps or they can be new test cases designed specifically for automated testing and therefore have no defined test steps. In either case, no changes need to be made to the individual test cases:

#this is a test case that tests addition operations

#!/usr/bin/perl -w

use Inflectra::SpiraTest::Addons::SpiraHarness::Harness;

#instantiate the harness

my $harness = Inflectra::SpiraTest::Addons::SpiraHarness::Harness->new;

#specify the SpiraTeam custom harness properties

$spira_args = {};

$spira_args->{"base_url"} = "http://localhost/SpiraTest";

$spira_args->{"user_name"} = "fredbloggs";

$spira_args->{"password"} = "fredbloggs";

$spira_args->{"project_id"} = 1;

$spira_args->{"release_id"} = 1;

$spira_args->{"test_set_id"} = 1;

$harness->{"spira_args"} = $spira_args;

#define the list of tests and their SpiraTeam Mapping

#Hash is of the format: TestFile => Test Case ID

my $tests = {};

$tests->{"SampleTest1.pl"} = 2;

$tests->{"SampleTest2.pl"} = 3;

$harness->runtests($tests);

Firstly you need to use the SpiraTeam specific harness rather than the general TAP::Harness library. This new class is actually a subclass of the standard one, so it supports all the same methods, with the exception of the runtests command, which now accepts a Perl hashref rather than a simple array. This hashref includes the mapping between the Perl test harnesses and the corresponding SpiraTeam test cases.

Then when the test is executed by TAP, the extension will capture the results from the unit tests and report them back as automated test executions in SpiraTeam.

PHP – PHPUnit


PHPUnit is a unit test framework for the PHP scripting language that is based on the popular xUnit paradigm. Using PHPUnit allows you to develop automated unit test fixtures that can be used to either regression test an existing application or to define the intended functionality prior to development (using the Agile recommended test-driven development (TDD) approach).

When you use the SpiraTeam Extension for PHPUnit it enables you to run automated PHPUnit tests against a PHP application and have the results be recorded as test runs inside SpiraTeam. This allows you to track your automated unit tests against the different releases in SpiraTeam and to have complete visibility of the overall requirements test coverage.

Note: This integration requires at least version 2.3 of SpiraTeam.

Integrating the Extension With Your Tests

The PHPUnit extension works by looking for a special numeric suffix on each of the test methods that map the PHPUnit test fixtures to SpiraTeam test cases:

/**

* Tests the addition of the two values

*/

public function testAdd__2()

{

$result = $this->fValue1 + $this->fValue2;

// forced failure result == 5

$this->assertTrue ($result == 6);

}

// Create a test suite that contains the tests

// from the ArrayTest class

$suite = new PHPUnit_Framework_TestSuite('SimpleTest');

//Set the timezone identifier to match that used by the SpiraTeam server

date_default_timezone_set ("US/Eastern");

//Create a new SpiraTeam listener instance and specify the connection info

$spiraListener = new SpiraListener_Listener;

$spiraListener->setBaseUrl ('http://localhost/SpiraTeam');

$spiraListener->setUserName ('fredbloggs');

$spiraListener->setPassword ('fredbloggs');

$spiraListener->setProjectId (1);

$spiraListener->setReleaseId (1);

$spiraListener->setTestSetId (1);

// Create a test result and attach the SpiraTeam listener

// object as an observer to it (as well as the default console text listener)

$result = new PHPUnit_Framework_TestResult;

$textPrinter = new PHPUnit_TextUI_ResultPrinter;

$result->addListener($textPrinter);

$result->addListener($spiraListener);

// Run the tests and print the results

$result = $suite->run($result);

$textPrinter->printResult($result);

Then when the test is executed by PHPUnit, the extension will capture the results from the unit tests and report them back as automated test executions in SpiraTeam.

Selenium


Selenium is a test tool that allows you to write automated web application user interface tests in any programming language against any HTTP website using any mainstream JavaScript-enabled browser. Selenium comes in two parts.

  • A server which can automatically launch and kill supported browsers, and acts as a HTTP proxy for web requests from those browsers. There are two versions of Selenium available, Selenium-RC (Remote Control) and Selenium WebDriver.
  • Client libraries for various computer languages - these are referred to as ‘client drivers’.

Therefore to use SpiraTeam with Selenium-RC or Selenium WebDriver, you need to decide which client driver you want to use, and then use the appropriate integration between SpiraTeam and that driver’s underlying platform/language and unit test framework (e.g. JUnit, NUnit, PHPUnit, etc.). Once you have chosen the language, you can refer to the previous section of this page that describes how SpiraTeam integrates with that unit test framework.

Cypress.io


Cypress IO is a user-friendly test automation tool for end to end testing, UI testing, regression suites, and integration and unit testing. It's simple to install and doesn't require any changes to your code. Cypress prides itself on writing tests quickly and in real-time as you build your web application.

SpiraTeam comes with a free custom reporter extension for Cypress that lets you execute your Cypress scripts as normal and have the results automatically reported back into SpiraTeam as test runs against mapped test cases. In addition to the detailed test results, the custom reporter automatically attaches any screenshots generated by Cypress to the SpiraTeam test run.

Robot Framework


Robot Framework is a generic open-source automation framework. It can be used for test automation and robotic process automation (RPA). Robot Framework is supported by Robot Framework Foundation. Many industry-leading companies use the tool in their software development.

SpiraTeam includes a free integration with Robot Framework. This integration lets you execute Robot Framework test suites, and have the results automatically transmitted back to SpiraTeam. It can be run locally or as part of a CI/CD pipeline.

Other xUnit Compatible Frameworks

xUnit is the collective name for several unit testing frameworks that derive their structure and functionality from Smalltalk's Sunit. The names of many of these frameworks are a variation on "SUnit," usually replacing the "S" with the first letter (or letters) in the name of their intended language ("JUnit" for Java, "RUnit" for R, etc.). These frameworks and their common architecture are collectively known as "xUnit".

The xUnit XML report output format was developed by the Ant project as part of their JUnit runner. The format includes a tree of test cases organized by test suite, and includes counts of test cases with each status, and the stack trace produced by test cases which failed or errored. Other testing frameworks based on xUnit have adopted this common XML reporting as standard as well as many other custom frameworks not based on xUnit.


SpiraTeam includes a free xUnit generic parser (implemented in both Python and NodeJS) that can be used to read the results of any test framework that can generate an xUnit style XML report file. This parser also offers the ability to create builds in SpiraTeam and include attachments with the test results.

Try SpiraTeam free for 30 days, no credit cards, no contracts

Start My Free Trial

And if you have any questions, please email or call us at +1 (202) 558-6885