JUnit 5 vs. TestNG: Choosing The Right Framework For Selenium Automation Testing

Difference between JUnit 5 vs. TestNG

To choose the perfect framework between JUnit vs. TestNG for Selenium automation, let us first understand the differences between JUnit 5 and TestNG. It is important to note that we are comparing JUnit 5 (and not JUnit 4) with TestNG. Also, some features like parallel execution with JUnit 5 are still in the Beta stage; though the features work as per the expectations!

JUnit 5 and TestNG: Setup and Installation

In this section of the JUnit vs TestNG framework comparison, we look at how to install and configure JUnit and TestNG framework.

1. Installing and setting up JUnit5

Step 1: JUnit 5 requires Java version 8 or above versions for running the tests. You can download Java from the official website https://www.oracle.com/java/technologies/downloads/.

Step 2: In the case of a Java project, download the JUnit 5 jars from https://junit.org/junit5/.

Step 3: Open the project and add these external jars through build path configuration.

Step 4: In the case of the Maven project, download maven from the official website https://maven.apache.org/download.cgi.

Step 5: Set up the environment variables.

Step 6: Add the below dependencies in pom.xml for running our JUnit 5 tests.

<dependencies>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
                  <version>5.7.1</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-runner</artifactId>
            <version>5.7.1</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>5.7.1</version>
                  <scope>test</scope>
              </dependency>
    </dependencies>

You can also refer to our JUnit video tutorial on how to install JUnit.

2. Installing and Setting up of TestNG

Step1: Download Java from the official website https://www.oracle.com/java/technologies/downloads/.

Step 2: In the case of a Java project, download the jar from https://mvnrepository.com/artifact/org.TestNG/TestNG/6.7.

Step 3: Open the project and add these external jars through build path configuration.

Step 4: In the case of the Maven project, download maven from the official website https://maven.apache.org/download.cgi.

Step 5: Set up the environment variables.

Step 6: Add the TestNG dependency in the pom.xml file.

<dependency>
    <groupId>org.TestNG</groupId>
    <artifactId>TestNG</artifactId>
    <version>7.4.0</version>
    <scope>test</scope>
</dependency>

Once the setup is completed, you are ready to go…!

‘Test Suites’ in JUnit 5 and TestNG

A test suite is a collection of tests to be executed in a framework. The test suite defines the test cases and can be quickly executed.

1. JUnit 5 suite

In JUnit 5, the test suite is defined with @RunWith and @Suite annotations.

When we annotate a class with @RunWith annotation, the framework will invoke the given class as the test runner instead of running the default one. The below example specifies a group of test classes to be run along with the class.

@RunWith(Suite.class)
@Suite.SuiteClasses(
{
         JUnitTest1.class,
         JUnitTest2.class
})

2. TestNG suite

In TestNG, the suite is defined in an XML file.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://TestNG.org/TestNG-1.0.dtd">
	<suite name="TestSuite" parallel="tests">
		<test name="DemoTest">
		   <classes>
			<class name="com.pages.LoginPageTest"></class>
			<class name="com.pages.HomePageTest"></class>
			<class name="com.pages.ProductPageTest"></class>
		   </classes>
		</test>
</suite>

2M+ Devs and QAs rely on TestMu AI

Deliver immersive digital experiences with Next-Generation Mobile Apps and Cross Browser Testing Cloud

Sign up with googleSign up with Email

With TestNG certification, you can challenge your skills in performing automated testing with TestNG and take your career to the next level.

Here’s a short glimpse of the TestNG certification from TestMu AI:

‘Annotations’ in JUnit 5 and TestNG

Annotations give additional information about the class or test method in the automation testing framework. Read our detailed tutorial on TestNG annotations for Selenium automation to know more about the usage of annotations in TestNG.

1. JUnit 5 annotations

The annotations in JUnit 5 are a bit different, even from its previous versions of JUnit. Shown below are some of the popular JUnit 5 annotations:

• @Test – This annotation is used for declaring a test.
• @TestFactory – This annotation is used for defining a method which is a test factory. It is used for the runtime generation of dynamic tests.
• @RepeatedTest – This annotation is used to specify that the method is a template for the tests that can be repeated a specific number of times. For example, the below test method is annotated with @RepeatedTest for a specific number 5, which indicates that this test will be repeated automatically five times.

@RepeatedTest(5)

public void print(){

  // write some code to be executed

}

• @ParameterizedTest – This annotation is used to indicate that the method is a parameterized test. These parameterized tests are similar to standard test methods but we have to specify a source to provide parameters that would be used for every test.
• @TestMethodOrder – This annotation is used to define the order of the test execution.
• @DisplayName – This annotation is used to specify a customized display name for the method or class.
• @Tag – This annotation is used for filtering the tests at the method or class level by defining the tags.
• @Disabled – This annotation is used to disable a test method or class.
• @BeforeEach – This annotation is used to specify that the said test method has to be executed before each @Test, @RepeatedTest, @ParameterizedTest, or @TestFactory method.
• @AfterEach – This annotation is used to specify that the specific test method has to be executed after each @Test, @RepeatedTest, @ParameterizedTest, or @TestFactory method.
• @BeforeAll –  This annotation is used to specify that the specific test method has to be executed before all  @Test, @RepeatedTest, @ParameterizedTest, or @TestFactory methods.
• @AfterAll – This annotation is used to specify that the specific test method has to be executed after all  @Test, @RepeatedTest, @ParameterizedTest, or @TestFactory methods.

2. TestNG annotations

Below are the various annotations used in TestNG: 

• @BeforeMethod – This will be executed before every @Test annotated method.
• @AfterMethod: This annotation will be executed after every @Test annotated method.
• @BeforeClass – This will be executed before the first @Test method execution. It will be executed one only time throughout the test case.
• @AfterClass – This will be executed after all test methods in the current class have been run
• @BeforeTest – This will be executed before the first @Test annotated method. It can be executed multiple times before the test case.
• @AfterTest – A method with this annotation will be executed when all @Test annotated methods complete the execution of those classes inside the <test> tag in the testng.xml file.
• @BeforeSuite – It will run only once before all tests in the suite are executed.
• @AfterSuite – A method with this annotation will run once after the execution of all tests in the suite is complete.
• @BeforeGroups – This method will run before the first test run of that specific group.
• @AfterGroups – This method will run after all test methods of that group complete their execution.
• @Test  – This is used for declaring a test.

Watch this video to learn about the TestNG Annotations and how they help provide better structure and readability to the code.

Quick Comparison of JUnit 5 and TestNG Annotations

Let’s compare the annotations between JUnit 5 vs. TestNG.

2M+ Devs and QAs rely on TestMu AI

Deliver immersive digital experiences with Next-Generation Mobile Apps and Cross Browser Testing Cloud

Sign up with googleSign up with Email

This JUnit certification establishes testing standards for those who wish to advance their careers in Selenium automation testing with JUnit.

Here’s a short glimpse of the JUnit certification from TestMu AI:

‘Handling Exceptions’ in JUnit 5 and TestNG

In this section of the JUnit 5 vs TestNG comparison, we deep dive into handling exceptions in JUnit 5 and TestNG.

1. Handling exceptions in JUnit 5

To handle the exceptions thrown in the tests, we could use Assertions.assertThrows() API. The assertThrows() method asserts that execution of the code throws an exception of the given type.

This assertion would fail if there is no exception or some other type of exception is thrown. The assertion will pass only if an exception is thrown as provided.

Read: Common Exceptions in Selenium Automation Testing

NumberFormatException is thrown when the input is not a valid number. In the sample shown below, we have provided an invalid number due to which NumberFormatException is thrown.

@Test
void testExpectedException() {

    //First argument - specifies the expected exception.
    //Here it expects that code block will throw NumberFormatException
    //Second argument - is used to pass an executable code block or lambda expression
    Assertions.assertThrows(NumberFormatException.class, () -> {
        Integer.parseInt("Hello");
    });

}

2. Handling exceptions in TestNG

To handle the exceptions thrown in the tests, we can use the expectedExceptions parameter along with the @Test annotation in TestNG.

In the below example, we expect an ArithmeticException to be thrown when the number is divided by zero. Hence, we use expectedExpections in the test so that when the test is executed, it would be marked as passed. This is because the expected condition is met.

@Test(expectedExceptions = ArithmeticException.class)
public void DivideByZeroTest() {
    int i = 20/0;
}

‘Ignoring Tests’ in JUnit 5 and TestNG

In our automation framework, we would have some test cases which have to be avoided for some reason. For example, it would have been obsolete or changed the requirement. In such cases, those cases have to be ignored so that they are not considered in the test run. Let us now see how to handle such cases in JUnit 5 and TestNG.

1. Ignoring tests in JUnit 5

To ignore a test, we can use @Ignore annotation.

@Ignore
public void oldTest(){
    System.out.println("This test has to be ignored");
}

@Test
public void newTest(){
    System.out.println("Test has been executed");
}

Running the above tests would ignore the first test, which is annotated with @Ignore annotation.

Console Output:

2. Ignoring tests in TestNG

To ignore a test in TestNG, we can pass a parameter in the @Test method.

package com.pages;

import org.TestNG.annotations.Test;

public class SimpleTest {
	
	@Test
	public void test1() {
		System.out.println("This is test1");
	}
	
	@Test(enabled = false)
	public void test2() {
		System.out.println("This is test2");
	}

}

The second test has been disabled, so running the class would ignore the second case.

Console Output:

‘Grouping Tests’ in JUnit 5 and TestNG

Grouping of tests helps us to identify the tests and execute them quickly. Let us now see how to group the tests in TestNG and JUnit 5.

1. Grouping tests in JUnit 5

In JUnit 5 we can use an annotation @Tag which helps us to group our tests.

package demo;

import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;

public class SimpleTest {

    @Test
    @Tag("Smoke")
    public void Test_1(){
        System.out.println("Test1 has been executed");
    }

    @Test
    @Tag("Smoke")
    public void Test_2(){
        System.out.println("Test2 has been executed");
    }
    @Test
    @Tag("Regression")
    public void Test_3(){
        System.out.println("Test3 has been executed");
    }
}

We add the tag in the Run configurations to execute our tests specific to a tag and then execute it.

In the example, we have grouped only one test under Regression, and hence only one will be executed when the Regression tag is specified.

Console Output

2. Grouping tests in TestNG

For grouping tests in TestNG under the same category, we use groups parameters along with @Test annotation. 

package com.pages;


import org.TestNG.annotations.AfterTest;
import org.TestNG.annotations.BeforeTest;
import org.TestNG.annotations.Test;

public class DemoTest1 {

    @BeforeTest
    public static void start() {
        System.out.println("=======Starting TestNG tests========");
    }

    @Test(groups = { "Sanity", "Regression" })
    public void test_method1()
    {
        //Test implementation
    }

    @Test(groups = {"Sanity"} )
    public void test_method2()
    {
        //Test implementation
    }

    @Test(groups = {"Regression"})
    public void test_method3()
    {
        //Test implementation
    }


    @AfterTest
    public static void end() {
        System.out.println("All the tests are executed successfully");

    }
}

To execute the tests belonging to a specific category, we have to add the group name in the testng.xml file. We can include (or exclude) multiple groups before executing the tests.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://TestNG.org/TestNG-1.0.dtd">
<suite name="TestSuite" parallel="methods" thread-count="4">
    <test name="DemoTest">
            <groups>
                <run>
                    <include name = "Sanity"></include>
                    <exclude name="Regression"></exclude>
                </run>
            </groups>
            <classes>
                <class name="com.pages.DemoTest1"></class>
            </classes>
    </test>
</suite>

In the above snippet, we have two groups – Sanity and Regression. In the testng.xml, we have included the Sanity group and excluded the Regression group. After test execution, we could only see the second test being executed as it belongs only to the Sanity group.

Console Output:

Watch this video to learn how TestNG has become one of the most robust test automation frameworks and all you need to know to get started with TestNG in Selenium.

‘Parameterizing Tests’ in JUnit 5 and TestNG

Parameterizing the tests helps us provide different sets of inputs to test the same test scenario. Now let us see how to parameterize the tests using JUnit 5 and TestNG.

1. Parameterizing tests in TestNG

There are two ways to parameterize the tests in TestNG.

• Using @Parameters and passing the value of the parameter in the TestNG XML file
• Using @DataProvider annotation in TestNG

• Parameterizing the tests using @Parameters

package com.pages;

import org.TestNG.annotations.Parameters;
import org.TestNG.annotations.Test;

public class SimpleTest {


    @Test()
    @Parameters("username")
    public void test1(String username) {
        System.out.println("The username "+username + " is passed");
    }
}

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://TestNG.org/TestNG-1.0.dtd">
<suite name="TestSuite" parallel="tests">
    <test name="DemoTest">
        <parameter name = "username" value = "Alex"/>

        <classes>
            <class name="com.pages.SimpleTest">
            </class>
        </classes>
    </test>
</suite>

In the above example, we have used @Parameter annotation to provide a username. The value will be specified in the testng.xml file.

• Parameterizing using @DataProvidersThe @DataProviders annotation is used to provide multiple input values to the test. It has a separate data provider method that is used to pass the parameters.Scenario: To test the login functionality of a webpage using different sets of credentials.
  In the above example, we have added a test case for the login functionality in a webpage.First, we would locate the webelements for entering the username and password.

WebElement name = driver.findElement(By.xpath("//input[@name="email"]"));
 
WebElement passwd = driver.findElement(By.xpath("//input[@name="password"]"));

name.clear();
        name.sendKeys(username);
        passwd.clear();
        passwd.sendKeys(password);
        WebElement loginButton =      driver.findElement(By.xpath("//button[text()='Login']"));
        loginButton.click();

@DataProvider(name = "credentials")
    public Object[][] getUserInput() {
        return new Object[][]{
                {"[email protected]", "xaxxdssc"},
                {"[email protected]", "mypasswd"},
                {"[email protected]", "textmypass"}
        };
    }

@Test(dataProvider = "credentials")
    public void LoginTest(String username,String password) {

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://TestNG.org/TestNG-1.0.dtd">
<suite name="TestSuite" parallel="tests">
    <test name="DemoTest">
        <classes>
            <class name="com.pages.SimpleTest">
            </class>
        </classes>
    </test>
</suite>

After locating the elements we need to pass different sets of inputs to username and password.

As mentioned earlier, in order to pass different parameters we would use @Dataproviders.

We implement a Dataprovider method to pass different sets of input and will provide a unique name to identify the data provider.

To use these inputs in our tests, we provide a parameter dataprovider along with the unique name within the @Test annotation.

FileName – testng.xml

2M+ Devs and QAs rely on TestMu AI

Deliver immersive digital experiences with Next-Generation Mobile Apps and Cross Browser Testing Cloud

Sign up with googleSign up with Email

2. Parameterizing tests in JUnit 5

JUnit provides different ways to parameterize the tests, akin to parameterizing in JUnit 4 framework.

As this article primarily focuses on JUnit 5 let us see how to parameterize the tests in JUnit 5. You will have to add the below dependency to parameterize the tests in JUnit 5.

<dependency>
   <groupId>org.junit.jupiter</groupId>
   <artifactId>junit-jupiter-params</artifactId>
   <version>5.7.1</version>
   <scope>test</scope>
</dependency>

Below is the code snippet to run our tests in different browsers by parameterizing the browsers.

import org.junit.jupiter.api.*;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.MethodSource;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.firefox.FirefoxDriver;

import java.util.concurrent.TimeUnit;
import java.util.stream.Stream;

import static org.junit.jupiter.params.provider.Arguments.arguments;

public class JUnitParamTest {


    String urlToTest = "https://www.lambdatest.com/selenium-playground/simple-form-demo.php";
    WebDriver driver;

    public void browser_setup(String browser) {
        System.out.println("Setting up the drivers and browsers");

        if(browser.equalsIgnoreCase("Chrome")) {
            System.setProperty("webdriver.chrome.driver", "C:\Users\Shalini\Downloads\chromedriver_latest94\chromedriver.exe");
            driver = new ChromeDriver();
              }
        if(browser.equalsIgnoreCase("Firefox")) {
            System.setProperty("webdriver.gecko.driver","C:\Users\Shalini\Downloads\geckodriver\geckodriver.exe");
                    driver = new FirefoxDriver();
              }

    }



    @ParameterizedTest
    @MethodSource("browser")
    public void enterAndDisplayMessage(String browser) {
        browser_setup(browser);
        String inputString ="Hello";
        String methodName = Thread.currentThread()
                .getStackTrace()[1]
                .getMethodName();
        System.out.println("********Execution of "+methodName+" has been started********");
        System.out.println("Launching LambdaTest selenium playground website started..");
        driver.get(urlToTest);
        driver.manage().window().maximize();
        driver.manage().timeouts().pageLoadTimeout(10, TimeUnit.SECONDS);

        WebElement textBox = driver.findElement(By.xpath("//input[@id='user-message']"));
        textBox.sendKeys(inputString);

        WebElement showButton = driver.findElement(By.id("showInput"));
        showButton.click();

        WebElement messageDisplayed = driver.findElement(By.xpath("//div[@id='user-message']//span[@id = 'message']"));
        String message = messageDisplayed.getText();
        Assertions.assertEquals(inputString,message);
        System.out.println("********Execution of "+methodName+" has ended********");
    }


    @AfterEach
    public void tearDown() {
        System.out.println("Quitting the browsers has started");
        driver.quit();
        System.out.println("Quitting the browsers has ended");
    }

    @AfterAll
    public static void end() {
        System.out.println("Tests ended");
    }

    static Stream<Arguments> browser() {
        return Stream.of(
                arguments("Chrome"),
                arguments("Firefox")
        );
    }
}

As we have parameterized two different browsers, our tests will be executed in two browsers.

Console Output

‘Prioritizing tests’ in JUnit 5 and TestNG

Setting priority for the test cases is pretty important during the test execution as it helps find the critical defects in essential features. Though developing atomic test cases is considered as one of the Selenium best practices, there could be scenarios where you may have to devise inter-dependent tests.

1. Prioritizing the tests in JUnit 5

In JUnit 5, the execution order of tests or priority of the tests can be set using @Order annotation. We also have to use @TestMethodOrder(MethodOrderer.OrderAnnotation.class)  to execute the tests in a pre-defined order.

In the below example, we have two tests – enterAndDisplayMessage() and addAndDisplayResult(). The test enterAndDisplayMessage() has the order set to 2 and the test addAndDisplayResult() has the order set to 1.

import org.junit.jupiter.api.*;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;

import java.util.concurrent.TimeUnit;

@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class OrderTests {

    String urlToTest = "https://www.lambdatest.com/selenium-playground/simple-form-demo.php";
    WebDriver driver;

    @BeforeAll
    public static void start() {
        System.out.println("=======Starting junit 5 tests========");
    }

    @BeforeEach
    public void setup() {
        System.out.println("Setting up the drivers");
        System.setProperty("webdriver.chrome.driver", "C:\Users\Shalini\Downloads\chromedriver_latest94\chromedriver.exe");
        driver = new ChromeDriver();
    }
    @Test
    @Order(2)
    public void enterAndDisplayMessage() {
        String inputString ="Hello World";
        String methodName = Thread.currentThread()
                .getStackTrace()[1]
                .getMethodName();
        System.out.println("********Execution of "+methodName+" has been started********");
        System.out.println("Launching LambdaTest selenium playground website started..");
        driver.get(urlToTest);
        driver.manage().window().maximize();
        driver.manage().timeouts().pageLoadTimeout(10, TimeUnit.SECONDS);

        WebElement textBox = driver.findElement(By.xpath("//input[@id='user-message']"));
        textBox.sendKeys(inputString);

        WebElement showButton = driver.findElement(By.id("showInput"));
        showButton.click();

        WebElement messageDisplayed = driver.findElement(By.xpath("//div[@id='user-message']//span[@id = 'message']"));
        String message = messageDisplayed.getText();
        Assertions.assertEquals(inputString,message);
        System.out.println("********Execution of "+methodName+" has ended********");
    }


    @Test
    @Order(1)
    public void addAndDisplayResult(){
        String methodName = Thread.currentThread()
                .getStackTrace()[1]
                .getMethodName();

        System.out.println("********Execution of "+methodName+" has been started********");
        System.out.println("Launching LambdaTest selenium playground website started..");
        driver.get(urlToTest);
        driver.manage().window().maximize();
        driver.manage().timeouts().pageLoadTimeout(10, TimeUnit.SECONDS);

        WebElement input1 = driver.findElement(By.id("sum1"));
        input1.sendKeys("1");
        WebElement input2 = driver.findElement(By.id("sum2"));
        input2.sendKeys("3");

        WebElement addButton = driver.findElement(By.xpath("//*[@id='gettotal']//button[contains(@class,'selenium_btn')]"));
        addButton.click();
        WebElement resultDisplayed = driver.findElement(By.id("addmessage"));
        String result = resultDisplayed.getText();

        Assertions.assertEquals("4",result);
        System.out.println("********Execution of "+methodName+" has ended********");
    }
    @AfterEach
    public void tearDown() {
        System.out.println("Quitting the browsers has started");
        driver.quit();
        System.out.println("Quitting the browsers has ended");
    }

    @AfterAll
    public static void end() {
        System.out.println("Tests ended");
    }

}

Console Output

Upon execution, we could see the tests being executed in the order provided.

2. Prioritizing tests in TestNG

In TestNG, we can set the execution order or priority by providing the parameter @ priority and the @Test annotation.

package com.pages;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.TestNG.Assert;
import org.TestNG.annotations.AfterTest;
import org.TestNG.annotations.BeforeTest;
import org.TestNG.annotations.Test;

import java.util.concurrent.TimeUnit;

public class TestNGOrder {

    String urlToTest = "https://www.lambdatest.com/selenium-playground/simple-form-demo.php";
    WebDriver driver;

    @BeforeTest
    public void setup() {
        System.out.println("Setting up the drivers");
        System.setProperty("webdriver.chrome.driver", "C:\Users\Shalini\Downloads\chromedriver_latest94\chromedriver.exe");
        driver = new ChromeDriver();
    }
    @Test(priority = 1)
    public void enterAndDisplayMessage() {
        String inputString ="Hello World";
        String methodName = Thread.currentThread()
                .getStackTrace()[1]
                .getMethodName();
        System.out.println("********Execution of "+methodName+" has been started********");
        System.out.println("Launching LambdaTest selenium playground website started..");
        driver.get(urlToTest);
        driver.manage().window().maximize();
        driver.manage().timeouts().pageLoadTimeout(10, TimeUnit.SECONDS);

        WebElement textBox = driver.findElement(By.xpath("//input[@id='user-message']"));
        textBox.sendKeys(inputString);

        WebElement showButton = driver.findElement(By.id("showInput"));
        showButton.click();

        WebElement messageDisplayed = driver.findElement(By.xpath("//div[@id='user-message']//span[@id = 'message']"));
        String message = messageDisplayed.getText();
        Assert.assertEquals(inputString,message);
        System.out.println("********Execution of "+methodName+" has ended********");
    }


    @Test(priority = 2)
    public void addAndDisplayResult(){
        String methodName = Thread.currentThread()
                .getStackTrace()[1]
                .getMethodName();

        System.out.println("********Execution of "+methodName+" has been started********");
        System.out.println("Launching LambdaTest selenium playground website started..");
        driver.get(urlToTest);
        driver.manage().window().maximize();
        driver.manage().timeouts().pageLoadTimeout(10, TimeUnit.SECONDS);

        WebElement input1 = driver.findElement(By.id("sum1"));
        input1.sendKeys("1");
        WebElement input2 = driver.findElement(By.id("sum2"));
        input2.sendKeys("3");

        WebElement addButton = driver.findElement(By.xpath("//*[@id='gettotal']//button[contains(@class,'selenium_btn')]"));
        addButton.click();
        WebElement resultDisplayed = driver.findElement(By.id("addmessage"));
        String result = resultDisplayed.getText();

        Assert.assertEquals("4",result);
        System.out.println("********Execution of "+methodName+" has ended********");
    }

    @AfterTest
    public void tearDown() {
        driver.quit();
    }



}

Console Output

If there is no priority given for the tests, the tests will be executed in alphabetical order.

Consider the same code used above, and we don’t provide any priority explicitly. In that case, you can see the tests executed in alphabetical order.

You can also read this to understand the concept of setting the priority in TestNG – Prioritizing test cases in TestNG

Console Output

‘Creating Test Dependency’ in JUnit 5 and TestNG

Creating inter-dependent tests is one of the non-recommended practices in Selenium. However, it can be useful when you want to avoid (or skip) the execution of tests depending on the result of some other tests.

Read – Scenarios you cannot automate with Selenium

1. Creating dependency in JUnit 5

The @TestMethodOrder annotation that is used to configure the test execution order is also used for creating test dependencies in JUnit 5. It provides similar functionality that is offered by the @FixMethodOrder annotation in the JUnit 4 framework.

Fixing the execution order is particularly useful when writing functional tests or integration tests. It is majorly used in conjunction with @TestInstance(Lifecycle.PER_CLASS).

For controlling the execution order of the test methods, test classes (or test methods) are annotated with the @TestMethodOrder annotation. The desired order is specified using the MethodOrderer implementation.

2. Creating dependency in TestNG

In the below example, we have written three tests – editProfileTest(), selectProduct(), and loginTest(). The former two tests are dependent on loginTest().

So when the main test i.e., loginTest() itself, fails, the dependent tests can be skipped as there is no point in testing when the login functionality is not working.

package com.pages;

import org.TestNG.annotations.Test;

public class dependentTest {

    @Test(dependsOnMethods = {"loginTest"})
    public void editProfileTest(){
        System.out.println("The user has successfully edited his profile");
    }
    @Test(dependsOnMethods = {"loginTest"})
    public void selectProduct(){
        System.out.println("The product has been added into the cart");
    }
    @Test
    public void loginTest(){
        System.out.println("The user is successfully logged in");
    }

}

Console Output

Parallel Test Execution in JUnit 5 and TestNG

Parallel testing is one of the crucial features to be kept in mind when implementing Selenium test scenarios. It has to be utilized from the early stages of testing. Parallel execution of tests would highly help in cutting the execution time.

Now let us run our tests in parallel using JUnit 5 and TestNG. We can utilize the TestMu AI cloud Selenium Grid to execute our tests in parallel.

JUnit 5 Parallel Test Execution

Until JUnit 4, the parallel test execution wasn’t supported, which was the biggest drawback of using the Junit in the automation framework. But this was overcome in JUnit 5.

Let’s understand how to perform parallel testing in JUnit with an example. 

Scenario 1: Verify the login functionality of TestMu AI Selenium playground.

Step 1: Launch https://www.lambdatest.com/selenium-playground/ website.

Step 2: Enter the valid username and password.

Step 3: Click Login.

Scenario 2: Verify if the user is able to see the message entered in the form.

Step 1: Launch https://www.lambdatest.com/selenium-playground/ website.

Step 2: On the main page, click the Simple Form Demo option.

Step 3: Once the page is navigated to a simple form, enter the message box.

Step 4: Click the Show Message button.

Scenario 3: Verify if the user can select the first option in the checkbox

Step 1: Launch https://www.lambdatest.com/selenium-playground/ website.

Step 2: On the main page, click the CheckBox Demo option.

Step 3: Upon navigation, select the first checkbox.

Step 4: Verify if the first checkbox is selected or not.

We have three tests to be run in parallel in TestMu AI Selenium Grid. Let us now run these tests across different browsers in parallel.

package ParallelTestsInJnit;

import org.junit.jupiter.api.*;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.MethodSource;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.List;
import java.util.concurrent.TimeUnit;
import java.util.stream.Stream;

import static org.junit.jupiter.params.provider.Arguments.arguments;

public class ParallelTestsInGrid {

   String username = "YOUR_USERNAME"; //Enter your username
   String accesskey = "YOUR_ACCESSKEY"; //Enter your accesskey

   static RemoteWebDriver driver = null;
   String gridURL = "@hub.lambdatest.com/wd/hub";
   String urlToTest = "http://labs.lambdatest.com/selenium-playground/";

   @BeforeAll
   public static void start() {
       System.out.println("=======Running junit 5 tests in parallel in LambdaTest Grid has started========");
   }

   public void setup(String browser) {
       System.out.println("Setting up the drivers and browsers");

       DesiredCapabilities capabilities = new DesiredCapabilities();

       if(browser.equals("Chrome")) {
           capabilities.setCapability("platform", "Windows 10");// To specify the OS
           capabilities.setCapability("browserName", "Chrome"); //To specify the browser
           capabilities.setCapability("version","94.0");     //To specify the browser
           capabilities.setCapability("build", "ChromeTests");               //To identify the test
           capabilities.setCapability("name", "Parallel_JUnit5Tests");
       }

       else if (browser.equals("Firefox")){
           capabilities.setCapability("browserName", "Firefox");    //To specify the browser
           capabilities.setCapability("version", "93.0");        //To specify the browser version
           capabilities.setCapability("platform", "Windows 10");     // To specify the OS
           capabilities.setCapability("build", "FirefoxTests");               //To identify the test
           capabilities.setCapability("name", "Parallel_JUnit5Tests");
       }

       else if (browser.equals("Edge")){
           capabilities.setCapability("browserName", "MicrosoftEdge");
           capabilities.setCapability("platform", "Windows 10");
           capabilities.setCapability("version","94.0");     // To specify the OS
           capabilities.setCapability("build", "EdgeTests");               //To identify the test
           capabilities.setCapability("name", "Parallel_JUnit5Tests");
       }
       capabilities.setCapability("network", true);      // To enable network logs
       capabilities.setCapability("visual", true);          // To enable step by step screenshot
       capabilities.setCapability("video", true);       // To enable video recording
       capabilities.setCapability("console", true);         // To capture console logs
       try {
           driver = new RemoteWebDriver(new URL("https://" + username + ":" + accesskey + gridURL), capabilities);
       } catch (MalformedURLException e) {
           System.out.println("Invalid grid URL");
       } catch (Exception e) {
           System.out.println(e.getMessage());
       }
   }


   @ParameterizedTest
   @MethodSource("browser")
   @DisplayName("LoginTest")
   @Order(1)
   public void test1(String browser){
       setup(browser);
       String methodName = Thread.currentThread()
               .getStackTrace()[1]
               .getMethodName();
       System.out.println("********Execution of "+methodName+" has been started********");

       driver.get(urlToTest);
       driver.manage().window().maximize();
       driver.manage().timeouts().pageLoadTimeout(10, TimeUnit.SECONDS);

       List<WebElement> elements = driver.findElements(By.xpath("//a[@class='nav-link']"));
       for(WebElement e : elements){
           if (e.getText().equals("Login")){
               e.click();
               WebElement username = driver.findElement(By.id("email"));
               username.sendKeys("testuser");

               WebElement password = driver.findElement(By.xpath("//input[@name='password']"));
               password.sendKeys("mypassword");

               WebElement loginBtn = driver.findElement(By.xpath("//button[@id='login-button']"));
               loginBtn.click();
           }
       }
       System.out.println("********Execution of "+methodName+" has ended********");

   }

   @ParameterizedTest
   @MethodSource("browser")
   @DisplayName("FormTest")
   @Order(2)
   public void test2(String browser){
       setup(browser);
       String methodName = Thread.currentThread()
               .getStackTrace()[1]
               .getMethodName();
       System.out.println("********Execution of "+methodName+" has been started********");
       driver.get(urlToTest);
       driver.manage().window().maximize();
       driver.manage().timeouts().implicitlyWait(10,TimeUnit.SECONDS);
       driver.findElement(By.xpath("//a[contains(@class,'btn_close_ck')]")).click();
       WebElement element = driver.findElement(By.linkText("Simple Form Demo"));
       element.click();
       driver.manage().timeouts().pageLoadTimeout(10,TimeUnit.SECONDS);
       WebElement textBox = driver.findElement(By.xpath("//input[@id='user-message']"));
       textBox.sendKeys("Hello World");

       WebElement submitBtn = driver.findElement(By.id("showInput"));
       submitBtn.click();
       System.out.println("********Execution of "+methodName+" has ended********");


   }

   @ParameterizedTest
   @MethodSource("browser")
   @DisplayName("CheckBoxTest")
   @Order(3)
   public void test3(String browser){
       setup(browser);
       String methodName = Thread.currentThread()
               .getStackTrace()[1]
               .getMethodName();
       System.out.println("********Execution of "+methodName+" has been started********");
       driver.get(urlToTest);
       driver.manage().window().maximize();
       driver.manage().timeouts().implicitlyWait(10,TimeUnit.SECONDS);
       driver.findElement(By.xpath("//a[contains(@class,'btn_close_ck')]")).click();

       WebElement option = driver.findElement(By.linkText("Checkbox Demo"));
       option.click();
       driver.manage().timeouts().pageLoadTimeout(10,TimeUnit.SECONDS);
       WebElement checkBox1 = driver.findElement(By.xpath("//input[@id='ex1-check1']"));
       checkBox1.click();

       boolean flag = checkBox1.isSelected();
       if(!flag){
           Assertions.fail("The checkbox is not selected");
       }


       System.out.println("********Execution of "+methodName+" has ended********");

   }


   static Stream<Arguments> browser() {
       return Stream.of(
               arguments("Chrome"),
               arguments("Firefox"),
               arguments("Edge")
       );
   }

   @AfterEach
   public void tearDown() {
       System.out.println("Quitting the browsers has started");
       driver.quit();
       System.out.println("Quitting the browsers has ended");

   }

   @AfterAll
   public static void end() {
       System.out.println("Tests ended");

   }
}

To run the tests in parallel, you can add the below arguments in VM options.

-Djunit.jupiter.execution.parallel.enabled=true

-Djunit.jupiter.execution.parallel.mode.default=concurrent

Console Output:

We will see the tests running in different browsers in parallel upon execution.

Go to the TestMu AI Automation dashboard; you will notice the three builds run in parallel. 

You may also read this article Running JUnit 5 tests in parallel to understand the concept of running the tests in parallel in JUnit5 in detail.

TestNG Parallel Test Execution

Parallel test execution in TestNG can be achieved with ease. However, first, you need to define the parallel execution in testng.xml.

<suite name="Parallel_Testing" parallel="methods" thread-count="8">

Parallelism can be achieved at the methods, classes, tests, and instances level. In addition, you can define the number of threads in which the tests have to be executed in parallel. Defining the thread- count, like 8 in the above example, will use eight threads for execution. When the number of tests or methods is greater, the tests will run in a serial manner and wait for the completion of other tests.

Here is a brief video to help you with Parallel Testing in TestNG.

For demonstration, we will use the same tests as those in JUnit 5 parallel testing with minor changes for running tests in parallel in TestNG.

package com.pages;

import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.TestNG.Assert;
import org.TestNG.annotations.AfterTest;
import org.TestNG.annotations.BeforeTest;
import org.TestNG.annotations.Parameters;
import org.TestNG.annotations.Test;


import java.net.MalformedURLException;
import java.net.URL;
import java.util.List;
import java.util.concurrent.TimeUnit;


public class DemoTest1 {

   String username = "YOUR_USERNAME"; //Enter your username
   String accesskey = "YOUR_ACCESSKEY"; //Enter your accesskey

   static RemoteWebDriver driver = null;
   String gridURL = "@hub.lambdatest.com/wd/hub";
   String urlToTest = "http://labs.lambdatest.com/selenium-playground/";


   @BeforeTest
   @Parameters("browser")
   public void setup(String browser) {
       System.out.println("Setting up the drivers and browsers");

       DesiredCapabilities capabilities = new DesiredCapabilities();

       if(browser.equals("Chrome")) {
           capabilities.setCapability("platform", "Windows 10");// To specify the OS
           capabilities.setCapability("browserName", "Chrome"); //To specify the browser
           capabilities.setCapability("version","94.0");     //To specify the browser
           capabilities.setCapability("build", "ChromeTests");               //To identify the test
           capabilities.setCapability("name", "Parallel_JUnit5Tests");
       }

       else if (browser.equals("Firefox")){
           capabilities.setCapability("browserName", "Firefox");    //To specify the browser
           capabilities.setCapability("version", "93.0");        //To specify the browser version
           capabilities.setCapability("platform", "Windows 10");     // To specify the OS
           capabilities.setCapability("build", "FirefoxTests");               //To identify the test
           capabilities.setCapability("name", "Parallel_JUnit5Tests");
       }

       else if (browser.equals("Edge")){
           capabilities.setCapability("browserName", "MicrosoftEdge");
           capabilities.setCapability("platform", "Windows 10");
           capabilities.setCapability("version","94.0");     // To specify the OS
           capabilities.setCapability("build", "EdgeTests");               //To identify the test
           capabilities.setCapability("name", "Parallel_JUnit5Tests");
       }
       capabilities.setCapability("network", true);      // To enable network logs
       capabilities.setCapability("visual", true);          // To enable step by step screenshot
       capabilities.setCapability("video", true);       // To enable video recording
       capabilities.setCapability("console", true);         // To capture console logs
       try {
           driver = new RemoteWebDriver(new URL("https://" + username + ":" + accesskey + gridURL), capabilities);
       } catch (MalformedURLException e) {
           System.out.println("Invalid grid URL");
       } catch (Exception e) {
           System.out.println(e.getMessage());
       }
   }


   @Test
   public void test1(){
       // Code Implementation
}

   @Test
   public void test2(){
       
     //Code Implementation
   }

   @Test
   public void test3(){
          //Code Implementation       
   }

   @AfterTest
   public void tear(){
       driver.close();
   }

}

TestNG.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://TestNG.org/TestNG-1.0.dtd">
<suite name="TestSuite" parallel="tests" thread-count="4">
   <test name="ChromeBrowserTest">
       <parameter name="browser" value="Chrome"/>
       <classes>
           <class name="com.pages.DemoTest1">
           </class>
       </classes>
   </test>
   <test name="FirefoxBrowserTest">
       <parameter name="browser" value="Firefox" />
       <classes>
           <class name="com.pages.DemoTest1">
           </class>
       </classes>
   </test>

   <test name="EdgeBrowserTest">
       <parameter name="browser" value="Edge" />
       <classes>
           <class name="com.pages.DemoTest1">
           </class>
       </classes>
   </test>
</suite>

In the TestNG.xml file, we pass different browsers as parameters for cross browser testing, and we use parallel = “tests” and thread-count=”4” parameters to run our tests in parallel.

Console Output:

‘Customizing Test Names’ in JUnit 5 and TestNG

The TestNG framework does not allow for the customization of test names. However, we can give the tests a customized name in JUnit 5.

Customizing the test name in JUnit 5

JUnit 5 provides an annotation @DisplayName which aids in customizing the test names for better readability.

Consider the below class with multiple tests, namely DemoTest1_A, DemoTest1_B, and DemoTest1_C. Most of the time, the method name wouldn’t be easier to depict the actual purpose of the test. However, in such cases, we can use the @DisplayName annotation followed by a customized name which would help identify and understand the feature of the test.

package ParallelTestsInJUnit;

import org.junit.jupiter.api.*;

public class DemoTest1 {

    @BeforeAll
    public static void start() {
        System.out.println("=======Starting junit 5 tests========");
    }

    @Test
    @DisplayName("Login_Test")
    void DemoTest1_A() {
        System.out.println(Thread.currentThread().getStackTrace()[1].getMethodName()+" => executed successfully");
    }

    @Test
    @DisplayName("Product_Test")
    void DemoTest1_B() {
        System.out.println(Thread.currentThread().getStackTrace()[1].getMethodName()+" => executed successfully");
    }

    @Test
    @DisplayName("Home_Test")
    void DemoTest1_C() {
        System.out.println(Thread.currentThread().getStackTrace()[1].getMethodName()+" => executed successfully");
    }

    @AfterAll
    public static void end() {
        System.out.println("All the tests are executed successfully");

    }
}

The console output shows the tests being named after the customized names given in the @DisplayName annotation. 

Console Output

Unfortunately, TestNG doesn’t support customizing the names of the test.

‘Reporting’ in JUnit 5 and TestNG

JUnit 5 doesn’t support any built-in report. But it can be integrated with plugins to generate reports.

While that’s not the case in TestNG, it has an inbuilt HTML report generated after the test execution. Once the tests are executed by running the TestNG.xml file, a test- output folder will be created, which can be found after refreshing the project.

The report will be named emailable-report.html. Right-click the report 🡪 Open With 🡪 Web Browser.

The TestNG report would appear to be similar to the one below.

The TestNG report would appear to be similar to the one below.

AnotherTestNG report – index.html, can be viewed by right-clicking the test-output folder 🡪 Open With 🡪 Web Browser.

Index.html report