TestNG Features: Annotations, Groups, Parallel Runs, Reports

TestNG Features at a Glance

Each feature maps to one annotation, attribute, or suite-file setting. The table lists the ones you will use on almost every project.

Annotations for Setup, Tests, and Teardown

Annotations are the feature everything else builds on. @Test marks a method as a test. The Before and After annotations run setup and teardown at four scopes: method, class, test (a test tag in testng.xml), and suite. TestNG runs them from the outside in, so @BeforeSuite runs first and @AfterSuite last, and a test class never has to call setup code explicitly.

public class LoginTests {
    private WebDriver driver;

    @BeforeClass
    public void openBrowser() {
        driver = new ChromeDriver();
    }

    @BeforeMethod
    public void openLoginPage() {
        driver.get("https://ecommerce-playground.lambdatest.io/index.php?route=account/login");
    }

    @Test
    public void validLoginShowsAccountPage() {
        // fill the form, submit, assert the account page title
    }

    @AfterClass
    public void closeBrowser() {
        driver.quit();
    }
}

The browser opens once for the class and every test starts on a fresh login page, which is the pattern most Selenium suites settle on. The TestNG annotations chapter lists all eleven annotations, their order of execution, and the attributes each one accepts.

Suite Configuration With testng.xml

A TestNG suite is an XML file, not a Java class. It names the test classes or packages to run, the groups to include or exclude, the parameters to pass in, and the thread settings. Because it sits outside the code, a CI job can switch from a five-minute smoke run to a full regression run by pointing at a different file.

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Smoke" parallel="classes" thread-count="4">
    <parameter name="browser" value="chrome"/>
    <test name="Checkout smoke">
        <groups>
            <run>
                <include name="smoke"/>
            </run>
        </groups>
        <classes>
            <class name="tests.LoginTests"/>
            <class name="tests.CheckoutTests"/>
        </classes>
    </test>
</suite>

The same file also carries listeners, method-level include and exclude rules, and a suite-wide timeout. The TestNG XML file chapter builds one from scratch and runs it from Eclipse, Maven, and the command line.

Groups, Priorities, and Dependencies

Three attributes on @Test control which tests run and in what order. groups tags a method with one or more names, so a suite file can run only the smoke group or everything except the slow group. priority orders methods within a class when the default alphabetical order is wrong for the flow. dependsOnMethods and dependsOnGroups declare that a test only makes sense after another one passed; when the prerequisite fails, TestNG marks the dependent test skipped instead of failed.

@Test(groups = "smoke", priority = 1)
public void login() { }

@Test(groups = {"smoke", "checkout"}, dependsOnMethods = "login")
public void addToCart() { }

@Test(groups = "regression", dependsOnGroups = "smoke")
public void applyCoupon() { }

Groups are the feature to learn first, because they replace copying test classes into separate projects. The TestNG groups chapter covers nesting, regular expressions, and group-level setup, and priority in TestNG explains how priority interacts with dependencies.

Data-Driven Testing With DataProvider and Parameters

TestNG feeds data into tests in two ways. @DataProvider is a method that returns a two-dimensional array; every row becomes one invocation of the test method, with its own line in the report. @Parameters reads named values from the parameter tags in testng.xml, which suits settings that change per environment rather than per case.

@DataProvider(name = "credentials")
public Object[][] credentials() {
    return new Object[][] {
        {"user1", "correct-password", true},
        {"user1", "wrong-password", false},
        {"", "", false}
    };
}

@Test(dataProvider = "credentials")
public void loginOutcome(String user, String password, boolean expectSuccess) {
    // one run per row; the report shows all three
}

@Parameters("browser")
@BeforeClass
public void openBrowser(String browser) {
    // "chrome" from testng.xml
}

A DataProvider can also read rows from a CSV, Excel, or JSON file, and it can run its rows in parallel. See DataProvider in TestNG for the file-backed versions and parameterization in TestNG for when to pick which mechanism.

Parallel Execution

Parallel execution is one attribute in the suite file: parallel set to methods, classes, tests, or instances, plus a thread-count. TestNG then schedules the work across a thread pool without any change to the test code. This is the feature that makes a cloud grid worthwhile, because a suite that opens forty browsers in sequence can open them eight at a time instead.

The condition is that tests share no mutable state. A WebDriver field on the class breaks the moment two methods run at once, so the usual pattern is a ThreadLocal driver or one driver per class with parallel set to classes. The parallel execution in TestNG chapter shows the thread-safe setup and how to size thread-count against grid capacity.

Listeners and Reports

Listeners are TestNG's extension points. A class that implements ITestListener gets a callback when a test starts, passes, fails, or is skipped; ISuiteListener wraps the whole suite; IAnnotationTransformer can change annotation values at run time, such as adding a retry analyzer to every test. Register a listener once with @Listeners on a class or a listeners block in testng.xml.

public class ScreenshotOnFailure implements ITestListener {
    @Override
    public void onTestFailure(ITestResult result) {
        WebDriver driver = DriverFactory.current();
        File shot = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE);
        // copy shot to test-output/screenshots/<method name>.png
    }
}

Reporting comes free. After every run TestNG writes index.html, emailable-report.html, and testng-results.xml, and Reporter.log lets a test add its own lines to them. ExtentReports and Allure plug into the same listener API when a team wants screenshots and history in the report itself. The TestNG listeners chapter compares the interfaces, TestNG Reporter log covers custom log lines, and TestNG reports walks through the default files and the Jenkins plugin.

Retries, Timeouts, and Repeated Runs

Browser tests fail for reasons that have nothing to do with the application, and TestNG has three features for that. IRetryAnalyzer re-runs a failed method up to a count you choose and reports only the final result. timeOut on @Test kills a method that runs past its limit and marks it failed, so a page that never loads cannot hang the build. invocationCount runs a method several times in one go, which is the simplest way to confirm a fix for a flaky test actually holds.

@Test(retryAnalyzer = RetryTwice.class, timeOut = 30000)
public void placeOrder() { }

@Test(invocationCount = 10)
public void searchStaysFast() { }

Retries hide real bugs when the count is high, so keep it at one or two and log every retry. IRetryAnalyzer in TestNG shows the full implementation and how to apply it suite-wide, and TestNG exception tests covers the related expectedExceptions attribute for tests that should throw.

Hard and Soft Assertions

TestNG ships its own assertion class, so a project needs no extra library for the basics. Assert methods are hard: the first failed check throws and the test stops. SoftAssert records every failed check and reports them all when assertAll is called, which is what you want when verifying several fields on one page.

SoftAssert soft = new SoftAssert();
soft.assertEquals(page.title(), "Checkout");
soft.assertTrue(page.totalIsVisible(), "total missing");
soft.assertEquals(page.itemCount(), 3);
soft.assertAll(); // fails once, listing every mismatch

The TestNG assertions chapter lists every Assert method, explains when a soft assert is the wrong choice, and shows how assertion messages surface in the report.

Integrations With Selenium, Maven, Gradle, and CI

TestNG has no opinion about what a test does, which is why it fits so many stacks. Selenium WebDriver is the most common pairing and the reason most of the features above exist in the form they do; TestNG in Selenium covers that combination end to end. Appium tests use the same annotations for mobile apps, and Cucumber can hand its scenarios to TestNG to get parallel runs and TestNG reports for BDD suites.

On the build side, Maven runs TestNG through the Surefire plugin and Gradle through useTestNG, so Jenkins, GitHub Actions, and GitLab CI pick up the results without extra configuration. TestNG also runs existing JUnit tests inside a TestNG suite, which makes a gradual migration possible. Start with the TestNG Maven dependency chapter for the build setup, then Appium with TestNG, Cucumber with TestNG, and JUnit with TestNG for the specific integrations.

Conclusion

The features of TestNG are the reason a Java team picks it over a plain runner: annotations for life cycle, testng.xml for composition, groups and dependencies for selection, DataProvider for data, parallel threads for speed, listeners and retries for resilience, and reports without setup. Learn them in that order. Annotations and the suite file carry a small project; groups, data providers, and parallel execution are what keep a large one fast and readable.

The next chapter, Install TestNG, puts the framework on your machine so you can try each feature as you read. If you are choosing between frameworks first, TestNG vs JUnit sets these features against their JUnit 5 equivalents.