Prioritizing tests in TestNG with Selenium

What is Test Case Prioritization?

Test Case Prioritization is a crucial step followed in Software Testing. It allows the testers to test the highest priority test cases first, which eventually helps them in resolving the critical issues in the initial phase of the testing life cycle. 

This approach is especially beneficial during the Regression testing of Complex and Big applications where it is impossible and impractical to test each scenario exhaustively.

Test Case Priority is also used in cases where the size of the test suite is quite big and requires more effort for maintenance. In the case of test automation, it is often required to configure our test suite so that some test methods have more precedence over others or the test methods need to be run in a specific order. 

TestNG plays a crucial role here, allowing us to handle such scenarios. This article will cover how to set test case priority in TestNG with Selenium automation testing.

What is TestNG?

TestNG is a Java-based test automation framework that overcomes JUnit’s limitations and additional functionalities. This makes it more powerful and easier to use. It covers a range of test categories such as Unit testing, Functional testing, Integration testing, etc. 

This framework is very popular among developers and testers for test case creation and execution since it helps them to organize the test cases in a structured way. This helps in maintaining the readability of the scripts. The current version of TestNG is 7.6.0.

Priority in TestNG

Priority is an attribute that tells TestNG which order the tests need to follow. When we have multiple test cases and want to execute them in a particular order, the TestNG priority attribute helps in executing the test cases in that order.

• The test cases get executed in ascending order of the priority list. Thus, test cases with lower priority get executed first. 
• One test method is allowed to have only one test priority in TestNG. 
• If test priority is not defined explicitly while running multiple cases, TestNG assigns all cases with a Default test priority, i.e., zero (0).

Syntax for using TestNG Priority

The syntax for test priority is @Test (priority = x), where x can be any integer – negative, zero, or positive.

For Example,

@Test (priority = 1)
public void function(){
//test code
}

Here, the test method function has a test priority of 1.

Negative Priority in TestNG

Negative Priority can be assigned to any test method when you want to have higher precedence over the test methods with default priority.

Consider the example below:

@Test

public void loginTest() {

       System.out.println("Testcase with default priority");

 }  

@Test(priority = -1)

public void logoutTest() {

       System.out.println("Testcase with negative priority");

}

The output of the above example will be:

Test Order without Priority in TestNG

In cases where there are multiple test methods without any Test priority, the test methods run in an alphabetical order based on the names of the test methods. 

Consider the following example:

package ui;

import org.testng.annotations.Test;

public class Login {

@Test

public void cloginTest() {

System.out.println("Login successful");

}

@Test

public void bregisterTest() {

System.out.println("Register successful");

}

@Test

public void alogoutTest() {

System.out.println("Logout successful");

}

}

The output of the above example will be:

From the output, it is clear that the TestNG executed the test methods in alphabetical order of their method names irrespective of their place of implementation in the code. So, in order to execute test methods as per your needs, you will have to set priority with @Test annotation.

Test execution order with priority in TestNG

Consider the same example as above but with test priority being set:

package ui;

import org.testng.annotations.Test;

public class Login {



@Test(priority = 1)

public void cloginTest() {

System.out.println("Login successful");

}

@Test(priority = 2)

public void bregisterTest() {

System.out.println("Register successful");

}

@Test(priority = 3)

public void alogoutTest() {

System.out.println("Logout successful");

}

}

The output of the above example will be:

As per the above output, it is clearly visible that the output has changed after assigning the test methods with test priority. Test method with lower priority i.e. cloginTest() was executed first followed by bregisterTest() and clogoutTest(). Hence, the alphabetical order of test method names is not taken while providing the test priority to these test methods.

Test execution order with the same priority in TestNG

To understand how test cases with same priority in TestNG are executed, consider the following example:

package ui;
import org.testng.annotations.Test;
public class Login {
@Test(priority = 1)
public void loginTest() {
System.out.println("Login successful");

}
@Test(priority = 2)
public void registerTest() {
System.out.println("Register successful");
}

@Test(priority = 2)
public void sendEmail() {
System.out.println("Sent email successfully");
}

@Test(priority = 3)
public void logoutTest() {
System.out.println("Logout successful");
}
}

The output of the above example will be:

In the above example, you can see that the two test methods have the same priority. In such cases, TestNG will execute the test methods with the same priority in alphabetical order, i.e., TestNG considers the alphabetical order of ‘r’ and ‘s’ and executes them accordingly. Hence, registerTest() was executed before sendEmail().

Test execution order with a combination of Prioritized and Non-prioritized methods in TestNG

To understand how test execution works when there are Test Cases with priority and without priority running together, consider the following example:

package ui;

import org.testng.annotations.Test;

public class Login {

@Test

public void loginTest() {

System.out.println("Login successful");

}

@Test(priority = 0)

public void registerTest() {

System.out.println("Register successful");

}

@Test(priority = -1)

public void sendEmail() {

System.out.println("Sent email successfully");

}

@Test(priority = 1)

public void checkCalendar() {

System.out.println("Calendar checked successfully");

}

@Test

public void meetUp() {

System.out.println("Zoom Meet successful");

}

@Test(priority = 1)

public void logoutTest() {

System.out.println("Logout successful");

}

}

In the above example, the sendEmail() test method with a negative priority. Thus, it gets executed first. This is followed by loginTest() and meetUp() test methods which are non-prioritized methods that are executed based on alphabetical order ‘l’ and then ‘m’. 

Next, registerTest() method is executed since it has a zero priority. This is then followed by test methods checkCalendar() and logoutTest() methods that have the same priority. TestNG considered the alphabetical order of their method names and executed checkCalendar() before logoutTest() test method.

Following is the output obtained:

Running Prioritized Tests in TestNG using Selenium

Now, let’s automate Google’s Home page and print the Home Page title using the TestNG Priority feature.

package testNGPriority;
import org.openqa.selenium.WebDriver; 
import org.openqa.selenium.firefox.FirefoxDriver; 
import org.testng.annotations.Test; 
public class FirefoxTest 
{ 
WebDriver driver; // Creating reference of WebDriver. 
@Test(priority = 1) 
public void driverSetup()
{ 
System.out.println("Running Firefox"); 
driver = new FirefoxDriver(); // Create an object of ChromeDriver class. 
} 
@Test(priority = 2) 
public void getURL()
{ 
driver.get("https://www.google.com"); 
} 
@Test(priority = 3) 
public void getTitle()
{ 
String title = driver.getTitle(); 
System.out.println(title); 
} 
@Test(priority = 4) 
public void closeBrowser()
{ 
driver.close(); 
System.out.println("Test successfully passed"); 
} 
}

The output will be something like this:

Conclusion

TestNG provides an important feature of Test Priority which is very useful in running the code in the sequence we want with bare minimum or no changes to the existing code. TestNG acts as a run tool here, enabling this feature for the users.

However, it is recommended to run the TestNG with Selenium tests on real devices to get more accurate results. BrowserStack’s real device cloud allows QA to run tests under real user conditions with access to 3000+ real devices and browsers. 

Try BrowserStack for Free