In automated testing, the order of test execution plays an essential role in identifying critical issues early and optimizing test cycles. TestNG, a popular Java testing framework, offers a straightforward way to control this order through its priority attribute.
Overview
In TestNG, priority defines the order in which test methods are executed. Lower values indicate higher priority, meaning tests with the smallest priority value run first. If no priority is set, the default value is 0.
How to Set Priority in TestNG
- Use the priority attribute in the @Test annotation.
- Lower values run before higher values (e.g., priority = 1 runs before priority = 2).
- Negative priority values are allowed and executed before positive values.
- If no priority is specified, TestNG assigns a default priority of 0.
This article explains how to set test priorities in TestNG, explores the execution behavior of prioritized tests, and highlights best practices for using prioritization effectively in Selenium test automation.
What is Test Case Prioritization?
Test case prioritization is a key strategy in software testing. It helps testers execute high-priority test cases first, ensuring that critical issues are identified and resolved early in the testing lifecycle.
Prioritization is especially useful during regression testing of large and complex applications, where exhaustive testing of every scenario is impractical. It also helps manage large test suites that require significant effort to maintain.
In test automation, configuring the test suite to control the order of test execution is often necessary. TestNG provides built-in support for setting test case priority, enabling testers to specify the execution order of test methods in Selenium automation.
What is TestNG?
TestNG is a Java-based test automation framework that overcomes JUnit’s limitations and additional functionalities. This makes it more powerful and easier to use. It covers a range of test categories such as Unit testing, Functional testing, Integration testing, etc.
This framework is very popular among developers and testers for test case creation and execution since it helps them to organize the test cases in a structured way. This helps in maintaining the readability of the scripts.
Priority in TestNG
Priority is an attribute that tells TestNG which order the tests need to follow. When there are multiple test cases and you want to execute them in a particular order, the TestNG priority attribute helps in executing the test cases in that order.
- The test cases get executed in ascending order of the priority list. Thus, test cases with lower priority get executed first.
- One test method is allowed to have only one test priority in TestNG.
- If test priority is not defined explicitly while running multiple cases, TestNG assigns all cases with a Default test priority, i.e., zero (0).
Read More: How to Automate TestNG in Selenium
Why set Priority to Test Cases?
Test case priority helps define the sequence in which test methods are executed. In large test suites, especially for complex applications, running tests in a specific order ensures that critical functionalities are validated early, reducing the risk of late-stage failures.
Key reasons for setting test case priority include:
- Early detection of critical issues: High-priority tests identify blockers and severe defects early in the testing cycle.
- Efficient test execution: Prioritizing tests helps save time by focusing on the most important scenarios first.
- Optimized regression testing: Ensures that vital features are tested first during regression, even when time constraints prevent running all tests.
- Improved test suite management: Helps manage large test suites by grouping and ordering tests logically.
Using cloud-based platforms like BrowserStack can further enhance test execution by providing scalable infrastructure that supports prioritized test runs across multiple environments. This integration helps teams maintain speed and reliability throughout the testing process.
Syntax for using TestNG Priority
The syntax for test priority is @Test (priority = x), where x can be any integer – negative, zero, or positive.
For Example,
@Test (priority = 1)
public void function(){
//test code
}Here, the test method function has a test priority of 1.
Negative Priority in TestNG
Negative Priority can be assigned to any test method when you want to have higher precedence over the test methods with default priority.
Consider the example below:
@Test
public void loginTest() {
System.out.println("Testcase with default priority");
}
@Test(priority = -1)
public void logoutTest() {
System.out.println("Testcase with negative priority");
}The output of the above example will be:
Test Order without Priority in TestNG
In cases where there are multiple test methods without any Test priority, the test methods run in an alphabetical order based on the names of the test methods.
Consider the following example:
package ui;
import org.testng.annotations.Test;
public class Login {
@Test
public void cloginTest() {
System.out.println("Login successful");
}
@Test
public void bregisterTest() {
System.out.println("Register successful");
}
@Test
public void alogoutTest() {
System.out.println("Logout successful");
}
}The output of the above example will be:
From the output, it is clear that the TestNG executed the test methods in alphabetical order of their method names irrespective of their place of implementation in the code. So, in order to execute test methods as per your needs, you will have to set priority with @Test annotation.
Test execution order with priority in TestNG
Consider the same example as above but with test priority being set:
package ui;
import org.testng.annotations.Test;
public class Login {
@Test(priority = 1)
public void cloginTest() {
System.out.println("Login successful");
}
@Test(priority = 2)
public void bregisterTest() {
System.out.println("Register successful");
}
@Test(priority = 3)
public void alogoutTest() {
System.out.println("Logout successful");
}
}The output of the above example will be:
As per the above output, it is clearly visible that the output has changed after assigning the test methods with test priority. Test method with lower priority i.e. cloginTest() was executed first followed by bregisterTest() and clogoutTest(). Hence, the alphabetical order of test method names is not taken while providing the test priority to these test methods.
Test execution order with the same priority in TestNG
To understand how test cases with same priority in TestNG are executed, consider the following example:
package ui;
import org.testng.annotations.Test;
public class Login {
@Test(priority = 1)
public void loginTest() {
System.out.println("Login successful");
}
@Test(priority = 2)
public void registerTest() {
System.out.println("Register successful");
}
@Test(priority = 2)
public void sendEmail() {
System.out.println("Sent email successfully");
}
@Test(priority = 3)
public void logoutTest() {
System.out.println("Logout successful");
}
}The output of the above example will be:
In the above example, you can see that the two test methods have the same priority. In such cases, TestNG will execute the test methods with the same priority in alphabetical order, i.e., TestNG considers the alphabetical order of ‘r’ and ‘s’ and executes them accordingly. Hence, registerTest() was executed before sendEmail().
Read More: Test Case Reduction and Techniques to Follow
Test execution order with a combination of Prioritized and Non-prioritized methods in TestNG
To understand how test execution works when there are Test Cases with priority and without priority running together, consider the following example:
package ui;
import org.testng.annotations.Test;
public class Login {
@Test
public void loginTest() {
System.out.println("Login successful");
}
@Test(priority = 0)
public void registerTest() {
System.out.println("Register successful");
}
@Test(priority = -1)
public void sendEmail() {
System.out.println("Sent email successfully");
}
@Test(priority = 1)
public void checkCalendar() {
System.out.println("Calendar checked successfully");
}
@Test
public void meetUp() {
System.out.println("Zoom Meet successful");
}
@Test(priority = 1)
public void logoutTest() {
System.out.println("Logout successful");
}
}In the above example, the sendEmail() test method with a negative priority. Thus, it gets executed first. This is followed by loginTest() and meetUp() test methods which are non-prioritized methods that are executed based on alphabetical order ‘l’ and then ‘m’.
Next, registerTest() method is executed since it has a zero priority. This is then followed by test methods checkCalendar() and logoutTest() methods that have the same priority. TestNG considered the alphabetical order of their method names and executed checkCalendar() before logoutTest() test method.
Following is the output obtained:
Running Prioritized Tests in TestNG using Selenium
Now, automate Google’s homepage and print the homepage title using the TestNG priority feature.
package testNGPriority;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.annotations.Test;
public class FirefoxTest
{
WebDriver driver; // Creating reference of WebDriver.
@Test(priority = 1)
public void driverSetup()
{
System.out.println("Running Firefox");
driver = new FirefoxDriver(); // Create an object of ChromeDriver class.
}
@Test(priority = 2)
public void getURL()
{
driver.get("https://www.google.com");
}
@Test(priority = 3)
public void getTitle()
{
String title = driver.getTitle();
System.out.println(title);
}
@Test(priority = 4)
public void closeBrowser()
{
driver.close();
System.out.println("Test successfully passed");
}
}The output will be something like this:
Note: TestNG Prioritization does not work when running tests parallelly since all the tests are executed simultaneously during parallel execution.
Common Mistakes to Avoid
Common mistakes can undermine the effectiveness of test prioritization in TestNG. Being aware of these pitfalls help ensure a more reliable and maintainable test suite.
- Assigning the Same Priority to All Tests: Setting the same priority value for all tests defeats the purpose of prioritization and can cause unpredictable execution order.
- Ignoring Default Execution Order: Tests without an explicit priority are assigned a default priority of zero. Not accounting for this can result in unexpected test runs.
- Using Priority as a Substitute for Test Design: Priority should not be used to fix poorly designed or dependent tests. Tests should be as independent and atomic as possible.
- Setting Negative Priorities Without Clear Intent: While TestNG allows negative priorities, using them without clear reasoning can confuse the test execution flow.
Read More: How to run parallel test cases in TestNG
Best Practices for Test Prioritization
Following proven best practices for test prioritization helps maximize test efficiency and maintainability while ensuring critical scenarios are always tested first.
- Prioritize Based on Business Impact: Assign higher priority to tests covering critical functionalities that directly affect users or business outcomes.
- Keep Tests Independent and Atomic: Design tests to be self-contained so they don’t rely heavily on execution order, minimizing inter-test dependencies.
- Avoid Overcomplicating Priority Levels: Use a simple and consistent priority scheme. Avoid too many priority levels to reduce confusion and maintenance overhead.
- Regularly Review and Update Priorities: As the application evolves, revisit test priorities to ensure they remain aligned with current business needs and system behavior.
- Leverage Tools That Support Prioritization Efficiently: Use cloud testing platforms like BrowserStack to run prioritized test suites seamlessly across different environments, enhancing speed and reliability.
Conclusion
TestNG provides an important feature of Test Priority which is very useful in running the code in the sequence we want with bare minimum or no changes to the existing code. TestNG acts as a run tool here, enabling this feature for the users.
However, it is recommended to run the TestNG with Selenium tests on real devices to get more accurate results. BrowserStack’s real device cloud allows QA to run tests under real user conditions with access to 3000+ real devices and browsers.
Featured Article
