How to run parallel test cases in TestNG

When working with Selenium for automated web testing, especially in combination with TestNG, generating clear and structured test reports becomes an essential part of the testing process. While Selenium handles browser automation effectively, it does not offer any built-in reporting capabilities.

This is where TestNG plays a key role, not only for test organization and execution, but also for integrating reporting mechanisms that provide insights into test results, failures, and overall execution status.

This article provides an overview of reporting in Selenium with TestNG, along with key concepts such as parallel execution, thread management, WebDriver handling and performance comparison.

What is reporting in Selenium, and Why is it required?

Reporting is necessary for manual as well as automated testing. Without creating good summary report one cannot showcase the health of the application under test. Reports give a statistic of passed, failed, and skipped test cases. In Selenium automation reports, along with pass/fail/skipped statistics, QAs would require also some more details of test automation metrics like total execution time, start and end time of execution, screenshots, text, video, and console logs of failed test cases, environment details of the test execution, etc.

To produce such reports with this extensive data, it is required to integrate Selenium tests with a reporting library. There are many reporting libraries available, out of which TestNG is one of the most widely used.

What is TestNG?

TestNG is a testing framework inspired by JUnit and NUnit but introduces some new functionalities that make it more powerful and easier to use. TestNG helps organize the tests and produce the test reports. You need to integrate this by adding the TestNG library to the automation framework. After adding the TestNG library, you can easily add the necessary TestNG annotations and execute the test automation script.

TestNG helps to efficiently organise the tests and maintain readability of the test cases. TestNG makes it easy to perform parallel test execution by defining parallel attributes in testing.xml file.

What is Parallel Execution in TestNG?

Parallel testing is a process where multiple tests are executed simultaneously/in parallel in different thread processes. With respect to Selenium and TestNG, it allows you to execute multiple tests on different browsers, devices, environments in parallel and at the same time, instead of running it sequentially.

The main purpose of running tests in parallel mode is to reduce execution time and do maximum environment coverage (browsers/devices/environment) in less time.

Suppose, for an application you need to execute a sanity automation suite of 50 test cases in Chrome and Firefox browser. If you go with the traditional sequential flow, you need to execute the suite for Chrome browser first which would take 1 hr and then you need to execute for Firefox browser which takes another 1 hr. So, in total you would need 2 hrs to test in both the browsers. By using a parallel mechanism, you can run simultaneously for both the browsers in just 1 hr thereby reducing the execution time by 50%.

TestNG helps to run test methods/classes/tests in parallel. Using the testng.xml file, one can specify parallel attributes to classes, tests, and methods. Java’s multi-thread feature can also be applied by defining the number of threads for parallel testing in the thread attribute.

<suite name="Parallel Test Suite" thread-count="2" parallel="methods" >

How to Perform Parallel Execution in TestNG

TestNG provides built-in support for parallel testing. This section explains how you can run parallel tests in TestNG.

Before setting it up, there are a few key prerequisites to ensure smooth and effective parallel execution.

Prerequisites

• Add TestNG to your project (via Maven/Gradle or JAR).
• Set up a test framework (e.g., Selenium).
• Create a testng.xml file to configure parallel execution
• Ensure thread safety, especially for WebDriver instances.
• Use proper TestNG annotations to structure tests.

Download required Maven dependencies

1. Install Java 8 or higher and set JAVA_HOME in system environment variables.
2. For a Maven project, add Selenium Java, TestNG and WebDriverManager dependencies. Save the pom.xml file to download all the dependencies.
3. Add the TestNG library to classpath.

<dependencies>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>4.5.0</version>
</dependency>

<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>7.6.1</version>
<scope>test</scope>
</dependency>

<dependency>
<groupId>io.github.bonigarcia</groupId>
<artifactId>webdrivermanager</artifactId>
<version>5.2.1</version>
</dependency>
</dependencies>

Executing Test Methods Sequentially in TestNG

Below program runs the test methods sequentially:

Step 1 Under src/test/java create a package and under that create a class as ParallelTest

public class ParallelTest {

WebDriver driver;

@Test(priority = 1)
public void testChrome() throws InterruptedException {
System.out.println("The thread ID for Chrome is "+ Thread.currentThread().getId());
WebDriverManager.chromedriver().setup();
driver = new ChromeDriver();
driver.get("https://www.bstackdemo.com/");
driver.manage().window().maximize();
Assert.assertEquals(driver.getTitle(), "StackDemo");
}

@Test(priority = 2)
public void testFirefox() throws InterruptedException {
System.out.println("The thread ID for Firefox is "+ Thread.currentThread().getId());
WebDriverManager.firefoxdriver().setup();
driver = new FirefoxDriver();
driver.get("https://www.bstackdemo.com/");
driver.manage().window().maximize();
Assert.assertEquals(driver.getTitle(), "StackDemo"); 
}

@AfterClass
public void close() {
driver.quit();
} 
}

Step 2 Right click on the class and select Run As >> TestNG Test. Observe the time taken to execute both the methods in a sequential manner. It takes 18486 ms as seen below

Now let us run both the methods in parallel.

To do so you need to first create a testing.xml file and add a parallel attribute for the test suite with value as methods.

Parallel Execution at different levels in TestNG

TestNG allows you Parallel Execution at different levels such as:

• Parallel Execution of Test Methods
• Parallel Execution of Test Classes and
• Parallel Execution of Test Suites

You can understand these different levels of Parallel Execution in TestNG using examples as demonstrated in the sections below.

Executing Parallel Test Methods in TestNG

Step 1 To create a testing.xml file, right click on the ParallelTest class and select TestNG >> Convert To TestNG.

Step 2 You may select Parallel mode and ThreadCount value of your choice while creating the testing.xml file or you may update it later as per the requirement change. I have selected Parallel mode as methods and ThreadCount as 2.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Parallel Test Suite" parallel="methods" thread-count="2">
<test name="Parallel Test" >
<classes>
<class name="com.qa.testcases.ParallelTest"/>
</classes>
</test> <!-- Test -->
</suite> <!-- Suite -->

Step 3 Right click on the testing.xml file and select “Run As” -> “TestNG Suite”. Observe the time taken to execute both the methods in parallel mode(10204 ms, decreased the execution timeby 8282 ms).

Executing Test Classes in Parallel using TestNG

Step 1 To run classes in parallel mode, create two class files as ChromeTest and FirefoxTest with three test methods.

public class ChromeTest {
WebDriver driver;

@BeforeTest
public void setUp() {
WebDriverManager.chromedriver().setup();
driver = new ChromeDriver();
driver.get("https://www.bstackdemo.com/");
driver.manage().window().maximize();
}

@Test(priority = 1)
public void testTitle() {
System.out.println("The thread ID for testTitle Chrome is "+ Thread.currentThread().getId());
Assert.assertEquals(driver.getTitle(), "StackDemo");
}

@Test(priority = 2)
public void clickOffers() throws InterruptedException {
System.out.println("The thread ID for clickOffers Chrome is "+ Thread.currentThread().getId());
WebElement offers=driver.findElement(By.cssSelector("a#offers"));
offers.click();
Thread.sleep(2000);
WebElement loginBtn=driver.findElement(By.cssSelector("button#login-btn"));
Assert.assertTrue(loginBtn.isDisplayed()); 
}

@Test(priority = 3)
public void clickOrders() throws InterruptedException {
driver.navigate().to("https://www.bstackdemo.com/");
System.out.println("The thread ID for clickOrders Chrome is "+ Thread.currentThread().getId());
WebElement orders=driver.findElement(By.cssSelector("a#orders"));
orders.click();
Thread.sleep(2000);
WebElement loginBtn=driver.findElement(By.cssSelector("button#login-btn"));
Assert.assertTrue(loginBtn.isDisplayed());
}

@AfterTest
public void tearDown() {
driver.close();
}
}


public class FirefoxTest {
WebDriver driver;

@BeforeTest
public void setUp() {
WebDriverManager.firefoxdriver().setup();
driver = new FirefoxDriver(); 
driver.get("https://www.bstackdemo.com/");
driver.manage().window().maximize();
}

@Test(priority = 1)
public void testTitle() { 
System.out.println("The thread ID for testTitle Firefox is " + Thread.currentThread().getId());
Assert.assertEquals(driver.getTitle(), "StackDemo");
}

@Test(priority = 2)
public void clickOffers() throws InterruptedException {
System.out.println("The thread ID for clickOffers Firefox is " + Thread.currentThread().getId());
WebElement offers = driver.findElement(By.cssSelector("a#offers"));
offers.click();
Thread.sleep(3000);
WebElement loginBtn = driver.findElement(By.cssSelector("button#login-btn")); 
Assert.assertTrue(loginBtn.isDisplayed());
}

@Test(priority = 3)
public void clickOrders() throws InterruptedException {
driver.navigate().to("https://www.bstackdemo.com/");
System.out.println("The thread ID for clickOrders Firefox is "+ Thread.currentThread().getId());
WebElement orders=driver.findElement(By.cssSelector("a#orders"));
orders.click();
Thread.sleep(2000);
WebElement loginBtn=driver.findElement(By.cssSelector("button#login-btn"));
Assert.assertTrue(loginBtn.isDisplayed());
}
@AfterTest
public void tearDown() {
driver.quit();
}

}

Step 2 In the testing.xml file, add class names and update parallel value as classes and run it.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Parallel Test Suite" parallel="classes" thread-count="2">
<test name="Parallel Test" >
<classes>
<class name="com.qa.testcases.ChromeTest"/>
<class name="com.qa.testcases.FirefoxTest"/>
</classes>
</test> <!-- Test -->
</suite> <!-- Suite -->

To demonstrate that two classes are running on different threads I have added a code to print the current Thread Id. Note that ChromeTest is running on Thread ID 14 and FirefoxTest is on Thread ID 15.

Obviously, as demonstrated in parallel methods, execution time while running classes in parallel is lesser when classes are run sequentially.

Executing Test Suites in Parallel using TestNG

To run all the tests available inside the suite tag in parallel mode, you need to update the “parallel” value as “tests” in testing.xml file.

Here thread-count value plays an important role because if thread count is less than tests, tests need to wait for other tests to execute.

Let us understand this by following example:

Step 1 Create one more class as EdgeTest. ChromeTest, FirefoxTest, and EdgeTest classes have 3 test methods.

public class EdgeTest {
WebDriver driver;

@BeforeTest
public void setUp() {
WebDriverManager.edgedriver().setup();
driver = new EdgeDriver();
driver.get("https://www.bstackdemo.com/");
driver.manage().window().maximize();
}

@Test(priority = 1)
public void testTitle() {
System.out.println("The thread ID for testTitle Edge is "+ Thread.currentThread().getId());
Assert.assertEquals(driver.getTitle(), "StackDemo");
}

@Test(priority = 2)
public void clickOffers() throws InterruptedException {
System.out.println("The thread ID for clickOffers Edge is "+ Thread.currentThread().getId());
WebElement offers=driver.findElement(By.cssSelector("a#offers"));
offers.click();
Thread.sleep(2000);
WebElement loginBtn=driver.findElement(By.cssSelector("button#login-btn"));
Assert.assertTrue(loginBtn.isDisplayed()); 
}

@Test(priority = 3)
public void clickOrders() throws InterruptedException {
driver.navigate().to("https://www.bstackdemo.com/");
System.out.println("The thread ID for clickOrders Edge is "+ Thread.currentThread().getId());
WebElement orders=driver.findElement(By.cssSelector("a#orders"));
orders.click();
Thread.sleep(2000);
WebElement loginBtn=driver.findElement(By.cssSelector("button#login-btn"));
Assert.assertTrue(loginBtn.isDisplayed());
}

@AfterTest
public void tearDown() {
driver.close();
}
}

Step 2 Update parallel value as tests and add all the 3 tests in testing.xml file. Keep thread count as 2 and run the testing.xml file.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Parallel Test Suite" thread-count="2" parallel="tests" >
<test name="Parallel Test Chrome" >
<classes>
<class name="com.qa.testcases.ChromeTest"/>
</classes>
</test> <!-- Test -->

<test name="Parallel Test Firefox" >
<classes>
<class name="com.qa.testcases.FirefoxTest"/>
</classes>
</test> <!-- Test -->

<test name="Parallel Test Edge" >
<classes>
<class name="com.qa.testcases.EdgeTest"/>
</classes>
</test> <!-- Test -->
</suite> <!-- Suite -->

When thread count is set to 2, ChromeTest and FireTest will execute in parallel on 2 Threads. EdgeTest will wait for either one of them to execute so that it can execute on the freed thread.

You can see from the above console logs, that “FirefoxTest” and “ChromeTest” ran in parallel on Thread 14 and 15 respectively. Later “EdgeTest” ran on Thread 15 as it was freed earlier than 14.

If thread count would have been “3”, all three tests would have run on Thread 14, 15 and 16 respectively in parallel thereby decreasing execution time.

Talk to an Expert

Cons of parallel testing using TestNG:

1. Parallel testing is considerate while testing independent modules/classes. It fails for modules which are dependent on another module thereby giving flaky results.
2. Tester should have a detailed understanding of the product under test and the workflow of the testcases to apply parallelism. If any dependent module is run in parallel mode, complete test execution may go for a toss.

Threads in TestNG

In TestNG, a thread refers to a separate path of execution that allows multiple test cases to run at the same time. Instead of running tests one after another (sequentially), threads help execute them in parallel, which speeds up the overall test run, especially useful for large test suites.

Each thread can run a test method, a class, or even an entire test block independently, without waiting for others to finish. This makes better use of system resources like CPU and memory, reducing total execution time.

The number of threads can be controlled using the thread-count attribute in the testng.xml file.

For example, setting thread-count=”3″ tells TestNG to run up to three tests at the same time.

Threads are the foundation of parallel execution in TestNG. When used correctly, they help scale testing efficiently across methods, classes, or even browsers. However, proper handling is important to avoid issues like shared resource conflicts or inconsistent results.

Performance comparison between Serialized and Parallelized test execution in TestNG

Below are the key differences between Serialized and Parallelized test execution in TestNG:

How to Convert a Static WebDriver to Non-Static for Parallel Test Execution in TestNG

The following steps can be used to convert Static WebDriver to Non-Static:

1. Declare WebDriver as a Non-Static Instance Variable: This ensures that each object of the test class holds its own WebDriver instance.

public WebDriver driver;

2. Initialize WebDriver in @BeforeMethod or @BeforeClass: The @BeforeMethod annotation ensures a new browser instance is created for each test method when running in parallel.

@BeforeMethod

public void setup() {

    driver = new ChromeDriver();

}

3. Use Instance Driver Inside Test Methods: This ensures that each test method operates independently, using the correct browser instance created specifically for its own execution thread.

@Test

public void openHomePage() {

    driver.get("https://example.com");

    // additional test steps

}

4. Quit WebDriver in @AfterMethod: This ensures that each browser session is closed properly after test execution.

@AfterMethod

public void tearDown() {

    if (driver != null) {

        driver.quit();

    }

}

5. Using ThreadLocal for Safer Parallel Tests: For more control in multi-threaded execution, especially in frameworks using Page Object Model, ThreadLocal<WebDriver> can be used to isolate WebDriver instances per thread:

public class DriverManager {

    private static ThreadLocal<WebDriver> driver = new ThreadLocal<>();




    public static WebDriver getDriver() {

        return driver.get();

    }




    public static void setDriver(WebDriver driverInstance) {

        driver.set(driverInstance);

    }




    public static void quitDriver() {

        if (driver.get() != null) {

            driver.get().quit();

            driver.remove();

        }

    }

}

To set up ThreadLocal WebDriver, use the following steps:

Step 1: Initialize in Setup

@BeforeMethod

public void setup() {

    WebDriver localDriver = new ChromeDriver();

    DriverManager.setDriver(localDriver);

}

Step 2: Access in Test

DriverManager.getDriver().get("https://google.com");

Parallel Test Execution Using DataProviders in TestNG

Follow these steps to run parallel tests using DataProviders in TestNG:

Step 1: Create a DataProvider with Parallel Set to True

Define the data in a method annotated with @DataProvider. Set parallel = true so that TestNG knows to run each data set in a separate thread.

@DataProvider(name = "loginData", parallel = true)

public Object[][] getData() {

    return new Object[][] {

        {"user1", "pass1"},

        {"user2", "pass2"},

        {"user3", "pass3"}

    };

}

Step 2: Link the Test Method to the DataProvider

Use the dataProvider attribute in the @Test annotation to feed the test method with the data. Each test run gets a unique username-password pair and runs in parallel.

@Test(dataProvider = "loginData")

public void testLogin(String username, String password) {

    WebDriver driver = new ChromeDriver();

    driver.get("https://google.com/login");

    // perform login steps

    driver.quit();

}

Step 3: Set Thread Count in testng.xml (Optional but Recommended)

Define how many threads can run in parallel by configuring the suite file. This allows TestNG to run multiple test methods with different data at the same time.

<suite name="ParallelDataProvider" parallel="methods" thread-count="3">

    <test name="LoginTest">

        <classes>

            <class name="com.test.LoginTest"/>

        </classes>

    </test>

</suite>

Parallel Test Execution in Multiple Browsers Using TestNG for Selenium Automation Testing

Running the same test cases across different browsers is a common requirement in cross-browser testing. TestNG makes this easy by allowing parallel execution using XML configuration and parameterization, so tests can be executed in multiple browsers at the same time, saving time and ensuring broader coverage.

Follow These Steps to Set Up Parallel Test Execution Across Browsers:

Step 1: Create a Test Method That Accepts the Browser Name

Use @Parameters to pass the browser type dynamically and launch the appropriate driver.

@Parameters("browser")

@BeforeMethod

public void setup(String browser) {

    if (browser.equalsIgnoreCase("chrome")) {

        driver = new ChromeDriver();

    } else if (browser.equalsIgnoreCase("firefox")) {

        driver = new FirefoxDriver();

    } else if (browser.equalsIgnoreCase("edge")) {

        driver = new EdgeDriver();

    }

}

Step 2: Write the Test Case

Use the driver instance in the test as usual.

@Test

public void openHomePage() {

    driver.get("https://google.com");

    // perform actions or validations

}

Step 3: Clean Up After Test Execution

Always close the browser after test completion.

@AfterMethod

public void tearDown() {

    if (driver != null) {

        driver.quit();

    }

}

Step 4: Define Browsers in testng.xml and Enable Parallel Execution

In the XML file, define each browser under a separate <test> block and enable parallel execution.

<suite name="CrossBrowserSuite" parallel="tests" thread-count="3">




    <test name="ChromeTest">

        <parameter name="browser" value="chrome"/>

        <classes>

            <class name="com.test.CrossBrowserTest"/>

        </classes>

    </test>




    <test name="FirefoxTest">

        <parameter name="browser" value="firefox"/>

        <classes>

            <class name="com.test.CrossBrowserTest"/>

        </classes>

    </test>




    <test name="EdgeTest">

        <parameter name="browser" value="edge"/>

        <classes>

            <class name="com.test.CrossBrowserTest"/>

        </classes>

    </test>




</suite>

Challenges of Parallel Test Execution in TestNG

Some of the challenges of parallel test execution in TestNG include:

• Shared WebDriver conflicts: Multiple threads accessing a single driver instance can cause unexpected failures.
• Data collisions: Tests modifying the same data files, databases or config settings may lead to inconsistent results.
• Thread safety issues: Shared objects or non-thread-safe utilities can behave unpredictably when accessed by multiple threads.
• Complex debugging: Simultaneous failures can generate overlapping logs, making it difficult to trace issues.
• High resource usage: Running too many threads can strain system memory and CPU, leading to crashes or slowdowns.
• Timing and synchronization problems: Dynamic elements may not load as expected when tests execute simultaneously.
• Test order unpredictability: Dependent tests may break if executed in a different order due to parallel scheduling.
• Improper Test Isolation: Tests that rely on shared states, static variables, or global config can interfere with each other when run in parallel, leading to flaky results.
• Setup and Teardown Conflicts: If setup (@BeforeClass, @BeforeMethod) or teardown (@AfterClass, @AfterMethod) logic isn’t thread-safe, it can lead to unexpected behavior or resource cleanup issues during parallel runs.

Read More: How to Automate TestNG in Selenium

Conclusion

Sequential testing is time consuming and therefore, parallel testing is required to decrease the execution time and cover more devices, browsers, platforms under test. With enhancement and innovations in the technology field, companies are frequently launching new devices, browser/ updated browsers and platforms to give a better user experience. It becomes challenging for a tester to test against all such combinations and on the other hand it is obligatory to test that too.

BrowserStack offers 3500+ real devices and browsers to help achieve cross browser testing in parallel using Selenium through its Cloud Selenium Grid. Sign up today to have a seamless experience while running parallel tests!

Run Parallel Tests on BrowserStack

Frequently Asked Questions

1. How to reuse the drivers in testNG parallel execution?

To reuse WebDriver instances in TestNG parallel execution, initialize the driver using @BeforeClass and use it across all test methods within that class. Driver instances can also be managed using ThreadLocal to ensure thread safety, allowing each thread to maintain its own separate driver instance during parallel execution.

Useful Resources

TestNG and Selenium

• Database Testing using Selenium and TestNG
• How to use DataProvider in Selenium and TestNG?
• All about TestNG Listeners in Selenium
• How to run parallel test cases in TestNG
• How to use TestNG Reporter Log in Selenium: Tutorial
• Prioritizing tests in TestNG with Selenium