Unit Test Frameworks

.NET - NUnit

NUnit is a unit test framework for the Microsoft .NET platform that is based on the popular xUnit paradigm. Using NUnit allows you to develop automated unit test fixtures that can be used to either regression test an existing application or to define the intended functionality prior to development (using the Agile recommended test-driven development (TDD) approach).

When you use the SpiraPlan Add-In for NUnit it enables you to run automated NUnit tests against a .NET application and have the results be recorded as test runs inside SpiraPlan. This allows you to track your automated unit tests against the different releases in SpiraPlan and to have complete visibility of the overall requirements test coverage.

Integrating the Add-In With Your Tests

The NUnit add-in works by looking for additional test attributes that map the NUnit test fixtures to SpiraPlan test cases:

/// <summary>

/// Sample test that asserts a failure

/// </summary>

[

Test,

SpiraTestCase (<test case id>)

]

public void _01_SampleFailure()

{

//Failure Assertion

Assert.AreEqual (1, 0);

}

Then when the test is executed by NUnit, the Add-In will capture the results from the unit tests and report them back as automated test executions in SpiraPlan. Learn More.

.NET - Visual Studio Unit Testing

Microsoft Visual Studio Team System (MS VSTS) Test (also known as MS-Test) is a unit test framework for the Microsoft .NET platform that comes built-in to Visual Studio Professional Edition. Using MS-Test allows you to develop automated unit test fixtures that can be used to either regression test an existing application or to define the intended functionality prior to development (using the Agile recommended test-driven development (TDD) approach).

When you use the SpiraPlan Extension for MS-Test it enables you to run automated unit tests against a .NET application and have the results be recorded as test runs inside SpiraPlan. This allows you to track your automated unit tests against the different releases in SpiraPlan and to have complete visibility of the overall requirements test coverage.

Integrating the Extension With Your Tests

The MS-Test extension works by looking for additional test attributes that map the unit test fixtures to SpiraPlan test cases:

/// <summary>

/// Sample test that asserts a failure and overrides the default configuration

/// </summary>

[

TestMethod,

SpiraTestCase(5),

SpiraTestConfiguration("http://localhost/SpiraTest", "fredbloggs", "fredbloggs", 1, 1, 2)

]

public void SampleFail()

{

//Verify the state

Assert.AreEqual(2, testFixtureState, "*Real Error*: State not persisted");

//Failure Assertion

Assert.AreEqual(1, 0, "Failed as Expected");

}

Then when the test is executed by MS-Test, the extension will capture the results from the unit tests and report them back as automated test executions in SpiraPlan. Learn More.

Java – Junit

JUnit is a unit test framework for the Java platform that is based on the popular xUnit paradigm. Using JUnit allows you to develop automated unit test fixtures that can be used to either regression test an existing application or to define the intended functionality prior to development (using the Agile recommended test-driven development (TDD) approach).

When you use the SpiraPlan Extension for JUnit it enables you to run automated JUnit tests against a Java application and have the results be recorded as test runs inside SpiraPlan. This allows you to track your automated unit tests against the different releases in SpiraPlan and to have complete visibility of the overall requirements test coverage.

Integrating the Extension With Your Tests

The JUnit extension works by looking for additional test annotations that map the JUnit test fixtures to SpiraPlan test cases:

/**

* Tests the addition of the two values

*/

@Test

@SpiraTestCase(testCaseId=5)

public void testAdd()

{

double result = fValue1 + fValue2;

// forced failure result == 5

assertTrue (result == 6);

}

Then when the test is executed by JUnit, the extension will capture the results from the unit tests and report them back as automated test executions in SpiraPlan. Learn More.

Java – TestNG

TestNG is a unit test framework for the Java platform that is based on the popular xUnit paradigm. Using TestNG allows you to develop automated unit test fixtures that can be used to either regression test an existing application or to define the intended functionality prior to development (using the Agile recommended test-driven development (TDD) approach).

When you use the SpiraPlan Listener for TestNG it enables you to run automated TestNG tests against a Java application and have the results be recorded as test runs inside SpiraPlan. This allows you to track your automated unit tests against the different releases in SpiraPlan and to have complete visibility of the overall requirements test coverage.

Integrating the Listener With Your Tests

The TestNG listener works by looking for additional test annotations that map the TestNG test fixtures to SpiraPlan test cases:

/**

* Tests the addition of the two values

*/

@Test

@SpiraTestCase(testCaseId=5)

public void testAdd()

{

double result = fValue1 + fValue2;

// forced failure result == 5

assertTrue (result == 6);

}

Then when the test is executed by TestNG, the listener will capture the results from the unit tests and report them back as automated test executions in SpiraPlan. Learn More.

Python – PyTest

pytest is a mature full-featured Python testing tool that helps you write better programs. The pytest framework makes it easy to write small tests, yet scales to support complex functional testing for applications and libraries.

When you use the SpiraPlan Extension for PyTest it enables you to have a simple way to ensure your code works properly. We provide fine-tuned control over your test runs as well! You can associate a test function in pytest with a test case inside SpiraPlan, and even add releases and test sets to your test run as well!

Integrating the Extension With Your Tests

Getting started is quick and simple – it should take less than five minutes! All you need is to install the SpiraPlan PyPI package and create a config file, and you’re done! No longer do you need to worry about spending the time to put in test results manually – we’ll handle it for you!

Then when the test is executed by pytest, the extension will capture the results from the unit tests and report them back as automated test executions in SpiraPlan. Learn More.

Python – PyUnit

PyUnit is a unit test framework for the Python scripting language that is based on the popular xUnit paradigm. Using PyUnit allows you to develop automated unit test fixtures that can be used to either regression test an existing application or to define the intended functionality prior to development (using the Agile recommended test-driven development (TDD) approach).

When you use the SpiraPlan Extension for PyUnit it enables you to run automated PyUnit tests against a Python application and have the results be recorded as test runs inside SpiraPlan. This allows you to track your automated unit tests against the different releases in SpiraPlan and to have complete visibility of the overall requirements test coverage.

Integrating the Extension With Your Tests

The PyUnit extension works by looking for a special numeric suffix on each of the test methods that map the PyUnit test fixtures to SpiraPlan test cases:

def testchoice__3(self):

element = random.choice(self.seq)

self.assert_(element in self.seq)

releaseId = 1

testSetId = 1

spiraTestExtension = spiratestextension.SpiraTestExtension()

spiraTestExtension.projectId = 1

spiraTestExtension.server = "localhost"

spiraTestExtension.port = 80

spiraTestExtension.path = "SpiraTest"

spiraTestExtension.userName = "fredbloggs"

spiraTestExtension.password = "fredbloggs"

spiraTestExtension.recordResults(TestSequenceFunctions, testResult, releaseId, testSetId)

Then when the test is executed by PyUnit, the extension will capture the results from the unit tests and report them back as automated test executions in SpiraPlan. Learn More.

NodeJS – UnitJS

UnitJS is an assertion library for JavaScript, running on Node.js or a web browser. It works with various test runner and unit testing frameworks, including Mocha, Jasmine, Karma, Protractor, and QUnit.

SpiraPlan currently includes a pre-built extension for the MochaJS test runner and our sample code illustrates it working with UnitJS. However, we supply the source code to the extension, so you can easily adapt it for other runners such as Jasmine and Protractor. Learn More.

NodeJS – Mocha

Mocha is a feature-rich JavaScript test framework running on Node.js and in the browser, making asynchronous testing simple and fun. Mocha tests run serially, allowing for flexible and accurate reporting, while mapping uncaught exceptions to the correct test cases.

SpiraPlan currently includes a pre-built extension for the MochaJS test runner and our sample code illustrates it working with UnitJS. However, it can be used as a general extension that works with any unit test framework that supports the MochaJS runner. Learn More.

NodeJS - JasmineJS

Our extension for Jasmine lets you integrate SpiraPlan seamlessly into your web browser and/or NodeJS testing environment. You can tell Jasmine to report back the results of your automated tests directly to SpiraPlan in addition to logging to the console of the machine running the test.

Jasmine is a behavior-driven development framework for testing JavaScript code. It does not depend on any other JavaScript frameworks. It does not require a DOM. And it has a clean, obvious syntax so that you can easily write tests.

The new Jasmine reporter will integrate JasmineJS with SpiraPlan. It will create a new test run in SpiraPlan for each test spec executed in Jasmine. All you need to is install the NodeJS package and then define the credentials and tests to be integrated using simple JSON:

describe("Test having two specs", () => {

var SpiraReporter = require('jasmine-spiratest');

jasmine.getEnv().addReporter(new SpiraReporter({

"url": "https://doctor/SpiraPlan",

"username": "fredbloggs",

"token": "{XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX}",

"projectId": 1,

"releaseId": 1,

"testSetId": 1,

"testCases": {

"default": 20,

"equality works": 21,

"addition works": 16

}

}));

describe("Test basic JavaScript", () => {

it("Equality works...", () => {

expect(2).toEqual(2);

});

it("Addition works", () => {

expect(3 + 2).toEqual(5);

});

it("Multiplication works", () => {

expect(4 * 5).toEqual(20);

});

});

});

For more information, consult the online documentation.

NodeJS – Jest

Jest is a delightful JavaScript Testing Framework with a focus on simplicity. It works with projects using: Babel, TypeScript, Node, React, Angular, Vue and more!

Jest is a JavaScript testing framework designed to ensure correctness of any JavaScript codebase. It allows you to write tests with an approachable, familiar and feature-rich API that gives you results quickly.

The new Jest reporter will integrate Jest with SpiraPlan. It will create a new test run in SpiraPlan for each test executed in Jest. All you need to is install the NodeJS package and then define the credentials and tests to be integrated using simple JSON:

"reporters": [

"default",

[

"jest-spiratest",

{

"url": "https://doctor/SpiraPlan",

"username": "fredbloggs",

"token": "{XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX}",

"projectId": 1,

"releaseId": 1,

"testSetId": 1,

"testCases": {

"default": 20,

"storesCorrectly": 21,

"javascriptWorks": 16

}

}

]

]

For more information, consult the online documentation.

Ruby – Test::Unit

Test::Unit is a unit test framework for the Ruby scripting language that is based on the popular xUnit paradigm. Using Test::Unit allows you to develop automated unit test fixtures that can be used to either regression test an existing application or to define the intended functionality prior to development (using the Agile recommended test-driven development (TDD) approach).

When you use the SpiraPlan custom runner for Test::Unit it enables you to run automated Test::Unit tests against a Ruby application and have the results be recorded as test runs inside SpiraPlan. This allows you to track your automated unit tests against the different releases in SpiraPlan and to have complete visibility of the overall requirements test coverage.

Integrating the Custom Runner With Your Tests

The Test::Unit custom runner works by looking for a special numeric suffix on each of the test methods that map the Test::Unit test fixtures to SpiraPlan test cases:

#this is a test case that tests addition operations

class TC_Adder < Test::Unit::TestCase

def setup

@adder = Adder.new(5)

end

def test_add__2

assert_equal(7, @adder.add(2), "Should have added correctly")

end

def test_addfail__3

assert_equal(7, @adder.add(3), "Test failure")

end

def teardown

@adder = nil

end

end

#this is a test suite that calls the test case

class TS_Examples

def self.suite

suite = Test::Unit::TestSuite.new

suite << TC_Adder.suite

return suite

end

end

projectId = 1

releaseId = 2

testSetId = -1

testRunner = Test::Unit::SpiraTest::TestRunner.new(TS_Examples, "http://servername/SpiraTest", "fredbloggs", "fredbloggs", projectId, releaseId, testSetId)

testRunner.start

Then when the test is executed by Test::Unit, the custom runner will capture the results from the unit tests and report them back as automated test executions in SpiraPlan. Learn More.

Perl – Test Anything Protocol (TAP)

Perl’s Test Anything Protocol (TAP) is a unit test framework built into the Perl scripting language. Using TAP allows you to develop automated unit test harnesses that can be used to either regression test an existing application or to define the intended functionality prior to development (using the Agile recommended test-driven development (TDD) approach).

When you use the SpiraPlan Extensions for TAP it enables you to run automated TAP test harnesses against a Perl application and have the results be recorded as test runs inside SpiraPlan. This allows you to track your automated unit tests against the different releases in SpiraPlan and to have complete visibility of the overall requirements test coverage.

Integrating the Extensions With Your Test Harnesses

To use SpiraPlan with TAP, each of the TAP test case files needs to have a corresponding test case in SpiraPlan. These can be either existing test cases that have manual test steps or they can be new test cases designed specifically for automated testing and therefore have no defined test steps. In either case, no changes need to be made to the individual test cases:

#this is a test case that tests addition operations

#!/usr/bin/perl -w

use Inflectra::SpiraTest::Addons::SpiraHarness::Harness;

#instantiate the harness

my $harness = Inflectra::SpiraTest::Addons::SpiraHarness::Harness->new;

#specify the SpiraPlan custom harness properties

$spira_args = {};

$spira_args->{"base_url"} = "http://localhost/SpiraTest";

$spira_args->{"user_name"} = "fredbloggs";

$spira_args->{"password"} = "fredbloggs";

$spira_args->{"project_id"} = 1;

$spira_args->{"release_id"} = 1;

$spira_args->{"test_set_id"} = 1;

$harness->{"spira_args"} = $spira_args;

#define the list of tests and their SpiraPlan Mapping

#Hash is of the format: TestFile => Test Case ID

my $tests = {};

$tests->{"SampleTest1.pl"} = 2;

$tests->{"SampleTest2.pl"} = 3;

$harness->runtests($tests);

Firstly you need to use the SpiraPlan specific harness rather than the general TAP::Harness library. This new class is actually a subclass of the standard one, so it supports all the same methods, with the exception of the runtests command, which now accepts a Perl hashref rather than a simple array. This hashref includes the mapping between the Perl test harnesses and the corresponding SpiraPlan test cases.

Then when the test is executed by TAP, the extension will capture the results from the unit tests and report them back as automated test executions in SpiraPlan. Learn More.

PHP – PHPUnit

PHPUnit is a unit test framework for the PHP scripting language that is based on the popular xUnit paradigm. Using PHPUnit allows you to develop automated unit test fixtures that can be used to either regression test an existing application or to define the intended functionality prior to development (using the Agile recommended test-driven development (TDD) approach).

When you use the SpiraPlan Extension for PHPUnit it enables you to run automated PHPUnit tests against a PHP application and have the results be recorded as test runs inside SpiraPlan. This allows you to track your automated unit tests against the different releases in SpiraPlan and to have complete visibility of the overall requirements test coverage.

Integrating the Extension With Your Tests

The PHPUnit extension works by looking for a special numeric suffix on each of the test methods that map the PHPUnit test fixtures to SpiraPlan test cases:

/**

* Tests the addition of the two values

*/

public function testAdd__2()

{

$result = $this->fValue1 + $this->fValue2;

// forced failure result == 5

$this->assertTrue ($result == 6);

}

// Create a test suite that contains the tests

// from the ArrayTest class

$suite = new PHPUnit_Framework_TestSuite('SimpleTest');

//Set the timezone identifier to match that used by the SpiraPlan server

date_default_timezone_set ("US/Eastern");

//Create a new SpiraPlan listener instance and specify the connection info

$spiraListener = new SpiraListener_Listener;

$spiraListener->setBaseUrl ('http://localhost/SpiraPlan');

$spiraListener->setUserName ('fredbloggs');

$spiraListener->setPassword ('fredbloggs');

$spiraListener->setProjectId (1);

$spiraListener->setReleaseId (1);

$spiraListener->setTestSetId (1);

// Create a test result and attach the SpiraPlan listener

// object as an observer to it (as well as the default console text listener)

$result = new PHPUnit_Framework_TestResult;

$textPrinter = new PHPUnit_TextUI_ResultPrinter;

$result->addListener($textPrinter);

$result->addListener($spiraListener);

// Run the tests and print the results

$result = $suite->run($result);

$textPrinter->printResult($result);

Then when the test is executed by PHPUnit, the extension will capture the results from the unit tests and report them back as automated test executions in SpiraPlan. Learn More.

Selenium

Selenium is a test tool that allows you to write automated web application user interface tests in any programming language against any HTTP website using any mainstream JavaScript-enabled browser. Selenium comes in two parts.

• A server which can automatically launch and kill supported browsers, and acts as a HTTP proxy for web requests from those browsers. There are two versions of Selenium available, Selenium-RC (Remote Control) and Selenium WebDriver.

• Client libraries for various computer languages - these are referred to as ‘client drivers’.

Therefore, to use SpiraPlan with Selenium-RC or Selenium WebDriver, you need to decide which client driver you want to use, and then use the appropriate integration between SpiraPlan and that driver’s underlying platform/language and unit test framework (e.g. JUnit, NUnit, PHPUnit, etc.). Once you have chosen the language, you can refer to the previous section of this page that describes how SpiraPlan integrates with that unit test framework. Learn More.

Cypress.io

Cypress IO is a user-friendly test automation tool for end to end testing, UI testing, regression suites, and integration and unit testing. It's simple to install and doesn't require any changes to your code. Cypress prides itself on writing tests quickly and in real-time as you build your web application.

SpiraPlan comes with a free custom reporter extension for Cypress that lets you execute your Cypress scripts as normal and have the results automatically reported back into SpiraPlan as test runs against mapped test cases. In addition to the detailed test results, the custom reporter automatically attaches any screenshots generated by Cypress to the SpiraPlan test run. Learn More.

Robot Framework

Robot Framework is a generic open-source automation framework. It can be used for test automation and robotic process automation (RPA). Robot Framework is supported by Robot Framework Foundation. Many industry-leading companies use the tool in their software development.

SpiraPlan includes a free integration with Robot Framework. This integration lets you execute Robot Framework test suites, and have the results automatically transmitted back to SpiraPlan. It can be run locally or as part of a CI/CD pipeline. Learn More.

Other xUnit Compatible Frameworks

xUnit is the collective name for several unit testing frameworks that derive their structure and functionality from Smalltalk's Sunit. The names of many of these frameworks are a variation on "SUnit," usually replacing the "S" with the first letter (or letters) in the name of their intended language ("JUnit" for Java, "RUnit" for R, etc.). These frameworks and their common architecture are collectively known as "xUnit".

The xUnit XML report output format was developed by the Ant project as part of their JUnit runner. The format includes a tree of test cases organized by test suite, and includes counts of test cases with each status, and the stack trace produced by test cases which failed or errored. Other testing frameworks based on xUnit have adopted this common XML reporting as standard as well as many other custom frameworks not based on xUnit.

SpiraPlan includes a free xUnit generic parser (implemented in both Python and NodeJS) that can be used to read the results of any test framework that can generate an xUnit style XML report file. This parser also offers the ability to create builds in SpiraPlan and include attachments with the test results. Learn More.