Unit Test Frameworks

.NET - NUnit

NUnit is a unit test framework for the Microsoft .NET platform that is based on the popular xUnit paradigm. Using NUnit allows you to develop automated unit test fixtures that can be used to either regression test an existing application or to define the intended functionality prior to development (using the Agile recommended test-driven development (TDD) approach).

When you use the SpiraTest Add-In for NUnit it enables you to run automated NUnit tests against a .NET application and have the results be recorded as test runs inside SpiraTest. This allows you to track your automated unit tests against the different releases in SpiraTest and to have complete visibility of the overall requirements test coverage.

Integrating the Add-In With Your Tests

The NUnit add-in works by looking for additional test attributes that map the NUnit test fixtures to SpiraTest test cases:

/// <summary>

/// Sample test that asserts a failure

/// </summary>

[

Test,

SpiraTestCase (<test case id>)

]

public void _01_SampleFailure()

{

//Failure Assertion

Assert.AreEqual (1, 0);

}

Then when the test is executed by NUnit, the Add-In will capture the results from the unit tests and report them back as automated test executions in SpiraTest. Learn More.

.NET - Visual Studio Unit Testing

Microsoft Visual Studio Team System (MS VSTS) Test (also known as MS-Test) is a unit test framework for the Microsoft .NET platform that comes built-in to Visual Studio Professional Edition. Using MS-Test allows you to develop automated unit test fixtures that can be used to either regression test an existing application or to define the intended functionality prior to development (using the Agile recommended test-driven development (TDD) approach).

When you use the SpiraTest Extension for MS-Test it enables you to run automated unit tests against a .NET application and have the results be recorded as test runs inside SpiraTest. This allows you to track your automated unit tests against the different releases in SpiraTest and to have complete visibility of the overall requirements test coverage.

Integrating the Extension With Your Tests

The MS-Test extension works by looking for additional test attributes that map the unit test fixtures to SpiraTest test cases:

/// <summary>

/// Sample test that asserts a failure and overrides the default configuration

/// </summary>

[

TestMethod,

SpiraTestCase(5),

SpiraTestConfiguration("http://localhost/SpiraTest", "fredbloggs", "fredbloggs", 1, 1, 2)

]

public void SampleFail()

{

//Verify the state

Assert.AreEqual(2, testFixtureState, "*Real Error*: State not persisted");

//Failure Assertion

Assert.AreEqual(1, 0, "Failed as Expected");

}

Then when the test is executed by MS-Test, the extension will capture the results from the unit tests and report them back as automated test executions in SpiraTest. Learn More.

Java – Junit

JUnit is a unit test framework for the Java platform that is based on the popular xUnit paradigm. Using JUnit allows you to develop automated unit test fixtures that can be used to either regression test an existing application or to define the intended functionality prior to development (using the Agile recommended test-driven development (TDD) approach).

When you use the SpiraTest Extension for JUnit it enables you to run automated JUnit tests against a Java application and have the results be recorded as test runs inside SpiraTest. This allows you to track your automated unit tests against the different releases in SpiraTest and to have complete visibility of the overall requirements test coverage.

Integrating the Extension With Your Tests

The JUnit extension works by looking for additional test annotations that map the JUnit test fixtures to SpiraTest test cases:

/**

* Tests the addition of the two values

*/

@Test

@SpiraTestCase(testCaseId=5)

public void testAdd()

{

double result = fValue1 + fValue2;

// forced failure result == 5

assertTrue (result == 6);

}

Then when the test is executed by JUnit, the extension will capture the results from the unit tests and report them back as automated test executions in SpiraTest. Learn More.

Java – TestNG

TestNG is a unit test framework for the Java platform that is based on the popular xUnit paradigm. Using TestNG allows you to develop automated unit test fixtures that can be used to either regression test an existing application or to define the intended functionality prior to development (using the Agile recommended test-driven development (TDD) approach).

When you use the SpiraTest Listener for TestNG it enables you to run automated TestNG tests against a Java application and have the results be recorded as test runs inside SpiraTest. This allows you to track your automated unit tests against the different releases in SpiraTest and to have complete visibility of the overall requirements test coverage.

Integrating the Listener With Your Tests

The TestNG listener works by looking for additional test annotations that map the TestNG test fixtures to SpiraTest test cases:

/**

* Tests the addition of the two values

*/

@Test

@SpiraTestCase(testCaseId=5)

public void testAdd()

{

double result = fValue1 + fValue2;

// forced failure result == 5

assertTrue (result == 6);

}

Then when the test is executed by TestNG, the listener will capture the results from the unit tests and report them back as automated test executions in SpiraTest. Learn More.

Python – PyTest

pytest is a mature full-featured Python testing tool that helps you write better programs. The pytest framework makes it easy to write small tests, yet scales to support complex functional testing for applications and libraries.

When you use the SpiraTest Extension for PyTest it enables you to have a simple way to ensure your code works properly. We provide fine-tuned control over your test runs as well! You can associate a test function in pytest with a test case inside SpiraTest, and even add releases and test sets to your test run as well!

Integrating the Extension With Your Tests

Getting started is quick and simple – it should take less than five minutes! All you need is to install the SpiraTest PyPI package and create a config file, and you’re done! No longer do you need to worry about spending the time to put in test results manually – we’ll handle it for you!

Then when the test is executed by pytest, the extension will capture the results from the unit tests and report them back as automated test executions in SpiraTest. Learn More.

Python – PyUnit

PyUnit is a unit test framework for the Python scripting language that is based on the popular xUnit paradigm. Using PyUnit allows you to develop automated unit test fixtures that can be used to either regression test an existing application or to define the intended functionality prior to development (using the Agile recommended test-driven development (TDD) approach).

When you use the SpiraTest Extension for PyUnit it enables you to run automated PyUnit tests against a Python application and have the results be recorded as test runs inside SpiraTest. This allows you to track your automated unit tests against the different releases in SpiraTest and to have complete visibility of the overall requirements test coverage.

Integrating the Extension With Your Tests

The PyUnit extension works by looking for a special numeric suffix on each of the test methods that map the PyUnit test fixtures to SpiraTest test cases:

def testchoice__3(self):

element = random.choice(self.seq)

self.assert_(element in self.seq)

releaseId = 1

testSetId = 1

spiraTestExtension = spiratestextension.SpiraTestExtension()

spiraTestExtension.projectId = 1

spiraTestExtension.server = "localhost"

spiraTestExtension.port = 80

spiraTestExtension.path = "SpiraTest"

spiraTestExtension.userName = "fredbloggs"

spiraTestExtension.password = "fredbloggs"

spiraTestExtension.recordResults(TestSequenceFunctions, testResult, releaseId, testSetId)

Then when the test is executed by PyUnit, the extension will capture the results from the unit tests and report them back as automated test executions in SpiraTest. Learn More.

NodeJS – UnitJS

UnitJS is an assertion library for JavaScript, running on Node.js or a web browser. It works with various test runner and unit testing frameworks, including Mocha, Jasmine, Karma, Protractor, and QUnit.

SpiraTest currently includes a pre-built extension for the MochaJS test runner and our sample code illustrates it working with UnitJS. However, we supply the source code to the extension, so you can easily adapt it for other runners such as Jasmine and Protractor. Learn More.

NodeJS – Mocha

Mocha is a feature-rich JavaScript test framework running on Node.js and in the browser, making asynchronous testing simple and fun. Mocha tests run serially, allowing for flexible and accurate reporting, while mapping uncaught exceptions to the correct test cases.

SpiraTest currently includes a pre-built extension for the MochaJS test runner and our sample code illustrates it working with UnitJS. However, it can be used as a general extension that works with any unit test framework that supports the MochaJS runner. Learn More.

NodeJS - JasmineJS

Our extension for Jasmine lets you integrate SpiraTest seamlessly into your web browser and/or NodeJS testing environment. You can tell Jasmine to report back the results of your automated tests directly to SpiraTest in addition to logging to the console of the machine running the test.

Jasmine is a behavior-driven development framework for testing JavaScript code. It does not depend on any other JavaScript frameworks. It does not require a DOM. And it has a clean, obvious syntax so that you can easily write tests.

The new Jasmine reporter will integrate JasmineJS with SpiraTest. It will create a new test run in SpiraTest for each test spec executed in Jasmine. All you need to is install the NodeJS package and then define the credentials and tests to be integrated using simple JSON:

describe("Test having two specs", () => {

var SpiraReporter = require('jasmine-spiratest');

jasmine.getEnv().addReporter(new SpiraReporter({

"url": "https://doctor/SpiraTest",

"username": "fredbloggs",

"token": "{XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX}",

"projectId": 1,

"releaseId": 1,

"testSetId": 1,

"testCases": {

"default": 20,

"equality works": 21,

"addition works": 16

}

}));

describe("Test basic JavaScript", () => {

it("Equality works...", () => {

expect(2).toEqual(2);

});

it("Addition works", () => {

expect(3 + 2).toEqual(5);

});

it("Multiplication works", () => {

expect(4 * 5).toEqual(20);

});

});

});

For more information, consult the online documentation.

NodeJS – Jest

Jest is a delightful JavaScript Testing Framework with a focus on simplicity. It works with projects using: Babel, TypeScript, Node, React, Angular, Vue and more!

Jest is a JavaScript testing framework designed to ensure correctness of any JavaScript codebase. It allows you to write tests with an approachable, familiar and feature-rich API that gives you results quickly.

The new Jest reporter will integrate Jest with SpiraTest. It will create a new test run in SpiraTest for each test executed in Jest. All you need to is install the NodeJS package and then define the credentials and tests to be integrated using simple JSON:

"reporters": [

"default",

[

"jest-spiratest",

{

"url": "https://doctor/SpiraTest",

"username": "fredbloggs",

"token": "{XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX}",

"projectId": 1,

"releaseId": 1,

"testSetId": 1,

"testCases": {

"default": 20,

"storesCorrectly": 21,

"javascriptWorks": 16

}

}

]

]

For more information, consult the online documentation.

Ruby – Test::Unit

Test::Unit is a unit test framework for the Ruby scripting language that is based on the popular xUnit paradigm. Using Test::Unit allows you to develop automated unit test fixtures that can be used to either regression test an existing application or to define the intended functionality prior to development (using the Agile recommended test-driven development (TDD) approach).

When you use the SpiraTest custom runner for Test::Unit it enables you to run automated Test::Unit tests against a Ruby application and have the results be recorded as test runs inside SpiraTest. This allows you to track your automated unit tests against the different releases in SpiraTest and to have complete visibility of the overall requirements test coverage.

Integrating the Custom Runner With Your Tests

The Test::Unit custom runner works by looking for a special numeric suffix on each of the test methods that map the Test::Unit test fixtures to SpiraTest test cases:

#this is a test case that tests addition operations

class TC_Adder < Test::Unit::TestCase

def setup

@adder = Adder.new(5)

end

def test_add__2

assert_equal(7, @adder.add(2), "Should have added correctly")

end

def test_addfail__3

assert_equal(7, @adder.add(3), "Test failure")

end

def teardown

@adder = nil

end

end

#this is a test suite that calls the test case

class TS_Examples

def self.suite

suite = Test::Unit::TestSuite.new

suite << TC_Adder.suite

return suite

end

end

projectId = 1

releaseId = 2

testSetId = -1

testRunner = Test::Unit::SpiraTest::TestRunner.new(TS_Examples, "http://servername/SpiraTest", "fredbloggs", "fredbloggs", projectId, releaseId, testSetId)

testRunner.start

Then when the test is executed by Test::Unit, the custom runner will capture the results from the unit tests and report them back as automated test executions in SpiraTest. Learn More.

Perl – Test Anything Protocol (TAP)

Perl’s Test Anything Protocol (TAP) is a unit test framework built into the Perl scripting language. Using TAP allows you to develop automated unit test harnesses that can be used to either regression test an existing application or to define the intended functionality prior to development (using the Agile recommended test-driven development (TDD) approach).

When you use the SpiraTest Extensions for TAP it enables you to run automated TAP test harnesses against a Perl application and have the results be recorded as test runs inside SpiraTest. This allows you to track your automated unit tests against the different releases in SpiraTest and to have complete visibility of the overall requirements test coverage.

Integrating the Extensions With Your Test Harnesses

To use SpiraTest with TAP, each of the TAP test case files needs to have a corresponding test case in SpiraTest. These can be either existing test cases that have manual test steps or they can be new test cases designed specifically for automated testing and therefore have no defined test steps. In either case, no changes need to be made to the individual test cases:

#this is a test case that tests addition operations

#!/usr/bin/perl -w

use Inflectra::SpiraTest::Addons::SpiraHarness::Harness;

#instantiate the harness

my $harness = Inflectra::SpiraTest::Addons::SpiraHarness::Harness->new;

#specify the SpiraTest custom harness properties

$spira_args = {};

$spira_args->{"base_url"} = "http://localhost/SpiraTest";

$spira_args->{"user_name"} = "fredbloggs";

$spira_args->{"password"} = "fredbloggs";

$spira_args->{"project_id"} = 1;

$spira_args->{"release_id"} = 1;

$spira_args->{"test_set_id"} = 1;

$harness->{"spira_args"} = $spira_args;

#define the list of tests and their SpiraTest Mapping

#Hash is of the format: TestFile => Test Case ID

my $tests = {};

$tests->{"SampleTest1.pl"} = 2;

$tests->{"SampleTest2.pl"} = 3;

$harness->runtests($tests);

Firstly you need to use the SpiraTest specific harness rather than the general TAP::Harness library. This new class is actually a subclass of the standard one, so it supports all the same methods, with the exception of the runtests command, which now accepts a Perl hashref rather than a simple array. This hashref includes the mapping between the Perl test harnesses and the corresponding SpiraTest test cases.

Then when the test is executed by TAP, the extension will capture the results from the unit tests and report them back as automated test executions in SpiraTest. Learn More.

PHP – PHPUnit

PHPUnit is a unit test framework for the PHP scripting language that is based on the popular xUnit paradigm. Using PHPUnit allows you to develop automated unit test fixtures that can be used to either regression test an existing application or to define the intended functionality prior to development (using the Agile recommended test-driven development (TDD) approach).

When you use the SpiraTest Extension for PHPUnit it enables you to run automated PHPUnit tests against a PHP application and have the results be recorded as test runs inside SpiraTest. This allows you to track your automated unit tests against the different releases in SpiraTest and to have complete visibility of the overall requirements test coverage.

Integrating the Extension With Your Tests

The PHPUnit extension works by looking for a special numeric suffix on each of the test methods that map the PHPUnit test fixtures to SpiraTest test cases:

/**

* Tests the addition of the two values

*/

public function testAdd__2()

{

$result = $this->fValue1 + $this->fValue2;

// forced failure result == 5

$this->assertTrue ($result == 6);

}

// Create a test suite that contains the tests

// from the ArrayTest class

$suite = new PHPUnit_Framework_TestSuite('SimpleTest');

//Set the timezone identifier to match that used by the SpiraTest server

date_default_timezone_set ("US/Eastern");

//Create a new SpiraTest listener instance and specify the connection info

$spiraListener = new SpiraListener_Listener;

$spiraListener->setBaseUrl ('http://localhost/SpiraTest');

$spiraListener->setUserName ('fredbloggs');

$spiraListener->setPassword ('fredbloggs');

$spiraListener->setProjectId (1);

$spiraListener->setReleaseId (1);

$spiraListener->setTestSetId (1);

// Create a test result and attach the SpiraTest listener

// object as an observer to it (as well as the default console text listener)

$result = new PHPUnit_Framework_TestResult;

$textPrinter = new PHPUnit_TextUI_ResultPrinter;

$result->addListener($textPrinter);

$result->addListener($spiraListener);

// Run the tests and print the results

$result = $suite->run($result);

$textPrinter->printResult($result);

Then when the test is executed by PHPUnit, the extension will capture the results from the unit tests and report them back as automated test executions in SpiraTest. Learn More.

Selenium

Selenium is a test tool that allows you to write automated web application user interface tests in any programming language against any HTTP website using any mainstream JavaScript-enabled browser. Selenium comes in two parts.

• A server which can automatically launch and kill supported browsers, and acts as a HTTP proxy for web requests from those browsers. There are two versions of Selenium available, Selenium-RC (Remote Control) and Selenium WebDriver.

• Client libraries for various computer languages - these are referred to as ‘client drivers’.

Therefore, to use SpiraTest with Selenium-RC or Selenium WebDriver, you need to decide which client driver you want to use, and then use the appropriate integration between SpiraTest and that driver’s underlying platform/language and unit test framework (e.g. JUnit, NUnit, PHPUnit, etc.). Once you have chosen the language, you can refer to the previous section of this page that describes how SpiraTest integrates with that unit test framework. Learn More.

Cypress.io

Cypress IO is a user-friendly test automation tool for end to end testing, UI testing, regression suites, and integration and unit testing. It's simple to install and doesn't require any changes to your code. Cypress prides itself on writing tests quickly and in real-time as you build your web application.

SpiraTest comes with a free custom reporter extension for Cypress that lets you execute your Cypress scripts as normal and have the results automatically reported back into SpiraTest as test runs against mapped test cases. In addition to the detailed test results, the custom reporter automatically attaches any screenshots generated by Cypress to the SpiraTest test run. Learn More.

Robot Framework

Robot Framework is a generic open-source automation framework. It can be used for test automation and robotic process automation (RPA). Robot Framework is supported by Robot Framework Foundation. Many industry-leading companies use the tool in their software development.

SpiraTest includes a free integration with Robot Framework. This integration lets you execute Robot Framework test suites, and have the results automatically transmitted back to SpiraTest. It can be run locally or as part of a CI/CD pipeline. Learn More.

Other xUnit Compatible Frameworks

xUnit is the collective name for several unit testing frameworks that derive their structure and functionality from Smalltalk's Sunit. The names of many of these frameworks are a variation on "SUnit," usually replacing the "S" with the first letter (or letters) in the name of their intended language ("JUnit" for Java, "RUnit" for R, etc.). These frameworks and their common architecture are collectively known as "xUnit".

The xUnit XML report output format was developed by the Ant project as part of their JUnit runner. The format includes a tree of test cases organized by test suite, and includes counts of test cases with each status, and the stack trace produced by test cases which failed or errored. Other testing frameworks based on xUnit have adopted this common XML reporting as standard as well as many other custom frameworks not based on xUnit.

SpiraTest includes a free xUnit generic parser (implemented in both Python and NodeJS) that can be used to read the results of any test framework that can generate an xUnit style XML report file. This parser also offers the ability to create builds in SpiraTest and include attachments with the test results. Learn More.