NUnit Vs XUnit Vs MSTest: Core Differences

Unit testing is essential to check whether the single independent block of code is working as expected or not. While there are different Unit Testing Frameworks to choose from, .Net has three popular Unit Testing frameworks for creating and managing tests

• NUnit
• xUnit
• MSTest

This article discussed these .Net Unit Testing Frameworks, and compared them with a comprehensive table of NUnit vs xUnit vs MSTest differences by highlighting salient features each provides.

What is NUnit?

NUnit is a unit testing tool ported initially from JUnit for .Net Framework and is completely an Open Source Project. NUnit was released in the early 2000s, though the initial Nunit was ported from Junit, the recent .Net version 3 is completely rewritten from scratch.

To run the Nunit test we need to add attributes to our methods. An example, attribute [Test], Indicates the Test method. The Attributes in Nunit are similar to JUnit/TestNG Annotations.

Below are the NuGet Packages required by NUnit

• NUnit
• NUnit3TestAdapter
• Microsoft.NET.Test.Sdk

NUnit Attributes with Example

Here are different Attributes in NUnit with their usage and example

• [Test]

With this attribute, we can create tests. Any method marked with [Test] attribute is considered a test.

public class SuccessTests
{
[Test] public void Add()
{ /* ... */ }

public void TestSubtract()
{ /* backwards compatibility */ }
}

• [TestFixture]

Marking the class with the [TestFixture] attribute will mark the class as a container for tests.

[TestFixture]
public class SuccessTests
{
// ...
}

• [Setup], [TearDown]

[TestFixture]
public class SuccessTests
{
[SetUp] public void Init()
{ /* create webdriver instance */ }

[Test] public void Add()
{ /* Add actions and assertions */ }

[TearDown] public void Dispose()
{ /* Destroy the webdriver instance */ }
}
}

• [OneTimeSetUp], [OneTimeTearDown]

[TestFixture] 
public class SuccessTests 
{ 
[OneTimeSetUp] 
public void Init() 
{ /* Initialise DB or Setup test data before running all the tests */ }

[OneTimeTearDown] 
public void Cleanup() 
{ /* Close the DB connection or destroy objects after running all tests */ }

[Test] 
public void Add() 
{ /* ... */ }

}

• [Ignore]

[Ignore] attribute can be used at the test level and class level. Using this attribute at the test level will skip that particular test, and using this attribute at the class level will skip all the tests within the class during the execution.

[TestFixture]
public class SuccessTests
{
[Test]
[Ignore("Ignore a test")]
public void IgnoredTest()
{ /* ... */ }
}

NUnit Test: Example

Running NUnit Tests using BrowserStack’s Cloud Selenium Grid on real devices helps get real user conditions into account for better accuracy of test results.

Run NUnit Tests on Real Devices

To run Selenium tests on BrowserStack Automate using NUnit Attributes, use the code below:

using System;
using OpenQA.Selenium;
using OpenQA.Selenium.Remote;
using OpenQA.Selenium.Support.UI;
using NUnit.Framework;
using System.Collections.Generic;
using OpenQA.Selenium.Chrome;

namespace SingleTest
{
[TestFixture]
public class AttributeTest
{
IWebDriver driver;

[OneTimeSetUp]

public void oneTimeSetup()
{
Dictionary<string, object> browserStackOptions = new Dictionary<string, object>();
browserStackOptions.Add("userName", <user name>);
browserStackOptions.Add("accessKey", <access key>);
ChromeOptions options = new ChromeOptions();
options.AddAdditionalOption("bstack:options", browserStackOptions);
driver = new RemoteWebDriver(new Uri("https://hub-cloud.browserstack.com/wd/hub/"), options);
}

[SetUp]
public void setup()
{
driver.Manage().Window.Maximize();
driver.Url = "https://bstackdemo.com/";

}

[Test]
public void test1()
{
DefaultWait<IWebDriver> fluentWait = new DefaultWait<IWebDriver>(driver);
fluentWait.Timeout = TimeSpan.FromSeconds(5);
fluentWait.PollingInterval = TimeSpan.FromMilliseconds(250);
fluentWait.Until(mydriver => mydriver.Title == "StackDemo");
}

[Test]
public void test2()
{
DefaultWait<IWebDriver> fluentWait = new DefaultWait<IWebDriver>(driver);
fluentWait.Timeout = TimeSpan.FromSeconds(5);
fluentWait.PollingInterval = TimeSpan.FromMilliseconds(250);
fluentWait.Until(mydriver => mydriver.Title == "StackDemo1");
}

}
}

What is xUnit?

xUnit is a unit testing tool for .Net Framework which was released in 2007 as an alternative for Nunit. xUnit has attributes for the execution of tests but is not similar to NUnit. [Fact] and [Theory] attributes are similar to [Test]

Below are the NuGet Packages required by xUnit

• xunit
• xunit.runner.visualstudio
• Microsoft.NET.Test.Sdk

xUnit Attributes with Example

• [Fact]

Method marked with [Fact] attribute is considered as a test method.

public class UnitTest1
{
[Fact]
public void PassingTest()
{
Assert.Equal(4, Add(2, 2));
}

int Add(int x, int y)
{
return x + y;
}
}

• [Theory]

Method marked as Theory will accept multiple data sets, and we can reuse the same test. This is helpful when we want to perform the same check for various combinations of test data.

public class UnitTest1
{
[Theory]
[InlineData(3)]
[InlineData(5)]
[InlineData(6)]
public void MyFirstTheory(int value)
{
Assert.True(IsOdd(value));
}

bool IsOdd(int value)
{
return value % 2 == 1;
}
}

xUnit Test: Example

Here’s an example of how xUnit Test would look like

//Import all the required libraries

namespace SingleTest
{

public class AttributeTest
{
IWebDriver driver;

[Fact]
public void test()
{
Dictionary<string, object> browserStackOptions = new Dictionary<string, object>();
browserStackOptions.Add("userName", <user name>);
browserStackOptions.Add("accessKey", <access key>);
ChromeOptions options = new ChromeOptions();
options.AddAdditionalOption("bstack:options", browserStackOptions);
driver = new RemoteWebDriver(new Uri("https://hub-cloud.browserstack.com/wd/hub/"), options);
driver.Manage().Window.Maximize();
driver.Url = "https://bstackdemo.com/";
DefaultWait<IWebDriver> fluentWait = new DefaultWait<IWebDriver>(driver);
fluentWait.Timeout = TimeSpan.FromSeconds(5);
fluentWait.PollingInterval = TimeSpan.FromMilliseconds(250);
fluentWait.Until(mydriver => mydriver.Title == "StackDemo");
}

}
}

What is MSTest?

MSTest is a unit testing framework developed by Microsoft and ships with Visual Studio. However, Microsoft made version 2 open-source which can easily be downloaded. MSTest has attributes range similar to NUnit and provides a wide range of attributes along with parallel run support at the Class and Method level.

MSTest Attributes with Example

• [TestMethod], [TestClass]

Methods marked with TestMethod attribute are considered as tests. The TestClass attribute is marked at class level and is a container for tests.

[TestClass]
public class TestClass
{
[TestMethod]
[DataRow(1, "message", true, 2.0)]
public void TestMethod1(int i, string s, bool b, float f) {}

[TestMethod]
[DataRow(new string[] { "line1", "line2" })]
public void TestMethod2(string[] lines) {}
}

• [TestInitialize], [TestCleanup]

TestInitialize is typically used when we need to set up an object or perform pre-requisites before running the test. TestCleanup will be run after the test, to clean up the data, driver objects.

[TestClass]
public class MyTestClass
{
[TestInitialize]
public void TestInitialize()
{
// Initialize driver object
}

[TestCleanup]
public void TestCleanup()
{
// Destroy the driver object
}
}

MSTest Test: Example

Here’s an example of how MSTest Test would look like

//Import required libraries

namespace SingleTest
{
[TestClass]
public class AttributeTest
{
IWebDriver driver;

[ClassInitialize]

public static void oneTimeSetup()
{
Dictionary<string, object> browserStackOptions = new Dictionary<string, object>();
browserStackOptions.Add("userName", <user name>);
browserStackOptions.Add("accessKey", <access key>);
ChromeOptions options = new ChromeOptions();
options.AddAdditionalOption("bstack:options", browserStackOptions);
driver = new RemoteWebDriver(new Uri("https://hub-cloud.browserstack.com/wd/hub/"), options);
}

[TestInitialize]
public void setup()
{
driver.Manage().Window.Maximize();
driver.Url = "https://bstackdemo.com/";

}

[TestMethod]
public void test1()
{
DefaultWait<IWebDriver> fluentWait = new DefaultWait<IWebDriver>(driver);
fluentWait.Timeout = TimeSpan.FromSeconds(5);
fluentWait.PollingInterval = TimeSpan.FromMilliseconds(250);
fluentWait.Until(mydriver => mydriver.Title == "StackDemo");
}

[TestMethod]
public void test2()
{
DefaultWait<IWebDriver> fluentWait = new DefaultWait<IWebDriver>(driver);
fluentWait.Timeout = TimeSpan.FromSeconds(5);
fluentWait.PollingInterval = TimeSpan.FromMilliseconds(250);
fluentWait.Until(mydriver => mydriver.Title == "StackDemo1");
}

}
}

NUnit vs xUnit vs MSTest: Feature Differences

On a closing note

NUnit Version 3.x is completely rewritten and it supports parallel test execution, a wide range of attributes for various level setup, and teardown that can help leverage flexibility while designing our Automated tests. 

Whichever Unit Testing framework you choose, it is important to test the application on real device cloud for more accurate test results. By testing under real user conditions you can identify the bottlenecks in the real user experience and rectify them in time before release. Using NUnit as your Unit Testing Framework, you can test your application on real devices and browsers.

Run Unit Test on Real Devices