How to run TestNG from the Command line

TestNG is a popular testing framework inspired by JUnit and NUnit. It is designed to simplify a wide range of testing requirements and provide advanced features for test automation.

While most developers are familiar with running TestNG tests through IDEs like Eclipse or IntelliJ IDEA, running tests from the command line offers greater flexibility, especially in continuous integration environments, build automation, and remote servers.

Why Run TestNG from the Command Line?

Running TestNG from the command line provides several advantages over IDE-based execution. It enables seamless integration with build tools like Maven and Gradle, supports automated testing pipelines, and allows for better control over test execution parameters.

Command line execution is particularly valuable in DevOps environments where tests need to be triggered automatically as part of deployment processes.

The command line approach also offers superior flexibility when it comes to test configuration, parallel testing, and report generation. Whether you’re working on a large enterprise project or a simple test automation suite, understanding how to execute TestNG tests from the command line is an essential skill for any test automation engineer.

Prerequisites

Before diving into command line execution, ensure you have the following components properly installed and configured on your system:

Java Development Kit (JDK)

It must be installed and properly configured. TestNG requires Java 8 or higher versions to function correctly. Verify your Java installation by running java -version in your command prompt or terminal. The JAVA_HOME environment variable should point to your JDK installation directory.

TestNG JAR files

These files need to be available in your classpath. Download the latest TestNG JAR from the official TestNG website or Maven Central Repository. The core TestNG library typically includes dependencies like JCommander for command line parsing. If you’re using additional TestNG modules or plugins, ensure those JAR files are also accessible.

Compiled Test Classes

Compiled test classes are essential since TestNG executes compiled Java bytecode, not source files. Your test classes should be compiled using javac or through your preferred build tool. The compiled .class files should be organized in the proper package structure within your classpath.

TestNG XML configuration file (optional but recommended)

It helps organize your test execution. This XML file defines test suites, groups, parameters, and other execution configurations. While not strictly mandatory, it significantly simplifies command line execution for complex test scenarios.

Classpath Configuration

Proper classpath configuration ensures that TestNG can locate all necessary dependencies, including your test classes, TestNG libraries, and any third-party libraries your tests depend on. This is crucial for successful test execution from the command line.

Steps to Run TestNG from Command Line

The steps below illustrates how to run TestNg from command line:

Step 1: Create a TestNG Test class

Create a Java class with various TestNG annotations like @Test, @BeforeSuite, @BeforeMethod, @AfterMethod, and @AfterSuite.

import org.testng.annotations.Test;

import org.testng.annotations.BeforeSuite;

import org.testng.annotations.AfterSuite;




public class LoginTest {

    @BeforeSuite

   public void beforeSuite() {

        System.out.println("Before Suite");

   }




    @Test

    public void testMethod1() {

        System.out.println("Test Method 1");

    }




    @Test

    public void testMethod2() {

        System.out.println("Test Method 2");

    }




    @AfterSuite

    public void afterSuite() {

        System.out.println("After Suite");

    }

}

Setp 2: Create a TestNG XML configuration (testng.xml) file.

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">

<suite name="Suite">

  <test name="Test">

     <classes>

        <class name="com.example.tests.LoginTest"/>

    </classes>

  </test>

</suite>

Step 3: Compile the Test class using the command below:

javac -cp "path/to/testng.jar" -d bin 

src/com/example/LoginTest.java

NB: Change the directory to your project directory

Step 4: Run using the command below:

java -cp "path/to/testng.jar;bin" org.testng.TestNG testng.xml

Understanding the TestNG Command Line Options

TestNG provides an extensive set of command line options that gives control over test execution behavior and output formatting.

Basic Execution Options

This includes fundamental parameters for specifying test sources and execution modes.

• The -testclass option allows you to specify individual test classes
• The -packages enable package-level test discovery.
• The -groups parameter lets you execute specific test groups defined in your test methods using the groups attribute in TestNG annotations.

Configuration and Setup

These options help customize the test execution environment.

• The -d option specifies the output directory for generated reports and results.
• The -listener parameter allows you to attach custom test listeners for enhanced reporting or logging.
• The -parallel option enables parallel test execution at different levels such as methods, classes, or tests.

Filtering and Selection

These options provide powerful mechanisms for controlling which tests execute.

• The -excludegroups option allows you to skip specific test groups.
• The -methods lets you run particular test methods.
• The -testnames parameter enables execution of specific test definitions from your TestNG XML file.

Reporting and Output

These options control the format and verbosity of test results.

• The -verbose option increases the detail level of console output, helping with debugging and monitoring test progress.
• The -reporter parameter allows you to specify custom report generators for specialized output formats.

Advanced Configuration

These options include parameters for advanced test scenarios.

• The -threadcount option controls the number of concurrent threads when running tests in parallel.
• The -dataproviderthreadcount parameter manages parallel execution of data-driven tests.
• The -suitethreadpoolsize option configures thread pool sizes for suite-level parallel execution.

XML Configuration Override

These options enable you to modify XML-defined parameters from the command line without editing the XML file.

• The -Dproperty=value syntax allows you to set system properties that your tests can access. This approach is particularly useful for environment-specific configurations or dynamic parameter injection.

Debugging and Troubleshooting

These options assist in identifying and resolving execution issues.

• The -log option controls TestNG’s internal logging level.
• The -debug provides additional diagnostic information.

These options are invaluable when troubleshooting classpath issues or test discovery problems.

Integration Options

These options facilitate connectivity with external tools and systems.

• The -port option enables TestNG’s remote execution capabilities
• The -host specifies connection endpoints for distributed testing scenarios.

These features support advanced testing architectures and continuous integration workflows.

Talk to an Expert

Conclusion

Understanding how to run TestNG from the command line is an important skill that enhances test automation capabilities and opens doors to advanced testing scenarios. This approach provides the flexibility needed for continuous integration, automated deployment pipelines, and sophisticated test execution strategies that go beyond simple IDE-based testing.

The command line interface offers extensive control over test execution parameters, enabling you to fine-tune everything from parallel execution settings to custom report generation. Whether it is a simple test class or complex test suites involving multiple modules and dependencies, the command line approach scales effectively to meet your requirements.

For developers and testers looking for a platform to scale TestNG tests effortlessly, BrowserStack Automate is an ideal choice. It offers built-in TestNG support and instant access to real devices and browsers with no infrastructure setup required.