How To Use Assertions In TestNG With Selenium

What are Assertions in TestNG?

Every test automation framework (e.g., TestNG, JUnit, PyTest, etc.) provides a mechanism to raise asserts. Hence, the generic concept of Asserts and Verify in Selenium is synonymous across different automation frameworks.

As far as the TestNG framework is concerned, asserts in TestNG are used for validating the scenario under test. The outcome of the test can be decided based on the assertion status. TestNG assertion is raised if the ‘achieved test result’ is not the same as the ‘expected test result.’

Assert Class in TestNG provides different methods that can be used to raise asserts. For using TestNG assertions, all you need to do is import the org.testng.Assert package.

Let’s take a simple test scenario where the page title of the TestMu AI homepage has to be validated before further tests can be executed.

Here expected result would be ‘Most Powerful Cross Browser Testing Tool Online | TestMu AI.’ The following cases can arise during the process of test execution:

Case 1

• Fetched Title: Most Powerful Cross Browser Testing Tool Online | TestMu AI
• Expected Title: Most Powerful Cross Browser Testing Tool Online | TestMu AI
• Test case result: Passed, as fetched title matches the expected title

Case 2

• Fetched Title: Most Powerful Cross Browser Testing Tool Online | TestMu AI
• Expected Title: Most Powerful Cross Browser Testing Tool Online
• Test case result: Failed, as fetched title does not match with the expected title

The above example that demonstrated TestNG assertions gives an overall idea about the what & how of assertions. Let’s look at the syntax and other internals of assertions in TestNG.

Syntax of TestNG assertions

The Assert package in TestNG provides methods(or options) to raise assertions. Shown below is the generic syntax of TestNG assertions:

Assert.methodName(actual, expected);

• Assert is the Class provided by the TestNG framework
• methodName is the name of the method that can be used for implementing TestNG assertions
• actual is the first parameter that describes the value that the user gets during the execution of the test script
• expected is the second parameter that describes what the user should get to validate the functionality of the test case

Watch this video to learn how to set up and use TestNG with Selenium to automate your testing process. We will also introduce the TestNG Priority method, which allows you to write easy-to-read and maintainable tests.

What are Assertion errors?

An Assertion Error, which is a subclass of the Error class, is thrown whenever an issue is encountered during the process of Selenium automation testing.

Read – Asserts in NUnit using Selenium

Let’s take a look at a sample implementation on TestNG assertions where the Selenium test raises an Assertion Error due to an error in the code.

The example shown below is run on the TestMu AI cloud Selenium Grid. A cloud-based Selenium Grid helps improve browser and test coverage by helping run tests on a range of browser and OS combinations. Here’s how you can perform cross browser testing using TestNG.

In addition, the overall process of Selenium Java automation testing using TestNG can be expedited by leveraging parallel testing in TestNG with Selenium.

Here is a brief video to help you with Parallel Testing in TestNG.

package com.lambdatest;
import java.net.URL;
import java.util.ArrayList;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Assert;
import org.testng.annotations.AfterTest;
import org.testng.annotations.Test;
 
public class AssertionTestOnLambdatest
{
   private static RemoteWebDriver driver;
   private static String status;
 
   @Test
   public void hardAssertion() throws Exception
   {
       String username = "Enter your user name";
       String accesskey = "Enter your access key";
       String gridURL = "@hub.lambdatest.com/wd/hub";
 
       DesiredCapabilities capabilities = new DesiredCapabilities();
 
       capabilities.setCapability("browserName", "chrome");
       capabilities.setCapability("version", "latest");
       capabilities.setCapability("platform", "win10");
       capabilities.setCapability("build", "AssertionError Test");
       capabilities.setCapability("name", "AssertionError Test");
       capabilities.setCapability("visual", true);
       driver=new RemoteWebDriver(new URL("https://"+username+":"+accesskey+gridURL), capabilities);
 
       driver.get("https://www.lambdatest.com/");
 
       String expectedtitle="Most Powerful Cross Browser Testing Tool Online | LambdaTe";
 
       String actualtitle = driver.getTitle();
       ArrayList<String> exceptionCapture = new ArrayList<>();
       try
       {
           Assert.assertEquals(actualtitle, expectedtitle);
           status="passed";
       }
       catch(AssertionError e)
       {
           status = "failed";
           exceptionCapture.add(e.getMessage());
           ((JavascriptExecutor) driver).executeScript("lambda-exceptions", exceptionCapture);
       }
   }
 
   @AfterTest
   public void tearDown() throws Exception
   {
 
       if (driver != null) {
           ((JavascriptExecutor) driver).executeScript("lambda-status=" + status);
           driver.quit();
       }
 
   }
}

Since the tests are running on a cloud Selenium Grid, an instance of Remote WebDriver in Selenium is created before implementing the actual test logic. Then, relevant Selenium methods are used to access and perform operations on the WebElements in the DOM.

The respective methods are placed under relevant TestNG annotations, thereby improving the readability of the tests. However, as seen in the implementation, the fetched title does not match with the expected title, which eventually results in an assertion error.

The rest of the implementation is pretty straightforward, hence we are deep diving into those aspects of the code 🙂

Invoke the following command on the terminal to kick-start the test execution:

mvn test –P single

The test resulted in a failure as a TestNG assertion was raised since the read title did not match with the expected title.

Navigate to the Exception tab in TestMu AI Automation Dashboard to check the details of the TestNG Assertion error.

In the above images, we can clearly see that the expected title and actual title did not match; hence AssertionError is displayed. The noticeable point here is we do not see any clear failure message in all the above outputs. Let’s resolve this problem in the next section.

Austin Siewert

Co-Founder, Steadfast Systems

Discovered @TestMu AI yesterday. Best browser testing tool I've found for my use case. Great pricing model for the limited testing I do 👏

2M+ Devs and QAs rely on TestMu AI

Deliver immersive digital experiences with Next-Generation Mobile Apps and Cross Browser Testing Cloud

Sign up with googleSign up with Email

With TestNG certification, you can challenge your skills in performing automated testing with TestNG and take your career to the next level.

Here’s a short glimpse of the TestNG certification from TestMu AI:

Different Methods in TestNG assertions

TestNG provides different methods to validate the test result during script execution. In this section, we will see the most commonly used assertion methods available for TestNG assertions

Methods

Description

assertEquals(String actual, String expected, String message);

Asserts that two strings are equal. If not, then AssertionError is thrown.

Parameters

actual: actual value

expected: expected value

message: message that we want to display in case of failure

assertEquals(boolean actual, boolean  expected, String message);

Asserts that two boolean values are equal. If not, then AssertionError is thrown.

Parameters

actual: actual value

expected: expected value

message: message that we want to display in case of failure

assertEquals(Collection <?> actual, Collection <?>  expected, String message);

Asserts that two Collection type objects hold the same elements and in the same order. If not, an AssertionError is thrown.

Parameters

actual: actual value

expected: expected value

message: message that we want to display in case of a failure

assertEqualsNoOrder(Object[] actual, Object[]  expected, String message);

Asserts that two arrays contain the same elements in no particular order. If not, an AssertionError is thrown.

Parameters

actual: actual value

expected: expected value

message: message that we want to display in case there is a failure

assertTrue(boolean condition, String message);

Asserts that condition is true. If not, then AssertionError is thrown.

Parameters:

condition: evaluate condition

message: Assertion error message in case of a failure.

assertFalse(boolean condition, String message);

Asserts that condition is false. If not, then AssertionError is thrown.

Parameters

condition: evaluate the condition

message: Assertion error message in case of a failure

assertNotNull(Object object, String message);

Asserts that an object is not null. If not, then AssertionError is thrown.

Parameters

object: assertion object

message: message that we want to display in case of a failure

assertNull(Object object, String message);

Asserts that an object is null. If not, then AssertionError is thrown.

Parameters

object: assertion object

message: message that we want to display in case of a failure

assertNotSame(Object actual, Object expected, String message);

Asserts that two objects do not refer to the same objects. If not, an AssertionError is thrown.

Parameters

actual: actual value

expected: expected value

message: message that we want to display in case of a failure

assertSame(Object actual, Object expected, String message);

Asserts that two objects refer to the same objects. If not, an AssertionError is thrown.

Parameters

actual: actual value

expected: expected value

message: message that we want to display in case of a failure

The “message” parameter is optional in all the above methods.

Watch this video to learn how TestNG has become one of the most robust test automation frameworks and all you need to know to get started with TestNG in Selenium.

Assertion Messages in TestNG Tests

TestNG provides the flexibility to add a custom message as a parameter inside the test method. The message helps in better understanding and tracking the reason behind the test failure.

Assert.methodName(actual, expected, message);

We have already discussed the major parameters in the earlier section. The newest parameter, ‘message’ is used for defining the custom message that is displayed in case the test results in a failure.

We now modify the already demonstrated example where a custom ‘message’ is displayed when the TestNG assertion is raised.

package com.lambdatest;
 
import java.net.URL;
import java.util.ArrayList;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Assert;
import org.testng.annotations.AfterTest;
import org.testng.annotations.Test;
 
public class AssertionTestOnLambdatest
{
 
   private static RemoteWebDriver driver;
   private static String status;
 
   @Test
   public void hardAssertion() throws Exception
   {
       String username = "Enter your user name";
       String accesskey = "Enter your access key";
       String gridURL = "@hub.lambdatest.com/wd/hub";
 
       DesiredCapabilities capabilities = new DesiredCapabilities();
 
       capabilities.setCapability("browserName", "chrome");
       capabilities.setCapability("version", "latest");
       capabilities.setCapability("platform", "win10");
       capabilities.setCapability("build", "AssertionError Test");
       capabilities.setCapability("name", "AssertionError Test");
       capabilities.setCapability("visual", true);
       driver=new RemoteWebDriver(new URL("https://"+username+":"+accesskey+gridURL), capabilities);
 
       driver.get("https://www.lambdatest.com/");
 
       String expectedtitle="Most Powerful Cross Browser Testing Tool Online | LambdaTe;
 
       String actualtitle = driver.getTitle();
       ArrayList<String> exceptionCapture = new ArrayList<>();
       try
       {
           Assert.assertEquals(actualtitle, expectedtitle);
           status="passed";
       }
       catch(AssertionError e)
       {
           status = "failed";
           exceptionCapture.add("Title not matched"+" "+e.getMessage());
           ((JavascriptExecutor) driver).executeScript("lambda-exceptions", exceptionCapture);
       }
   }
 
   @AfterTest
   public void tearDown() throws Exception
   {
 
       if (driver != null) {
           ((JavascriptExecutor) driver).executeScript("lambda-status=" + status);
           driver.quit();
       }
 
   }
}

As seen in the execution snapshot, the custom message is printed when the corresponding TestNG assert is raised.

As seen in the ‘Exception’ Tab of the Automation dashboard, the custom message ‘Title not matched’ is printed instead of the rudimentary message that we witnessed in the earlier screenshot.

Custom Soft Assertions in TestNG

In the previous sections, we demonstrated the implementation of different TestNG asserts, but there might be a need to write our assertions (called Custom Assertions).

We will first create a CustomAssertion class, which extends the SoftAssert class so that we can use all assertion methods in TestNG. For demonstration purposes, we will take the example of the TestMu AI website in which we will validate whether the “Free start testing” button is a button web element or not.

package com.lt.assertions;
 
import org.openqa.selenium.WebElement;
import org.testng.asserts.SoftAssert;
 
public class CustomAssertion extends SoftAssert
{
   // We created assertButton in which we will validate whether element is button or not
   public void assertButton(WebElement element)
   {
       // Extract the tagname as well as type of element
       String tagname = element.getTagName();
       String type = element.getAttribute("type");
      
       // Verify whether tag is button and type is submit
       if(tagname.equals("button") && type.equals("submit"))
       {
           System.out.println("Element is Button, So Assertion Passed");
       }
       else
       {
           // If assertion is failed then we will throw AssertionError
           throw new AssertionError("Element is not Button, So Assertion Failed");
       }
   }
}

Now, we will create the main class where we test the recently created custom assertion. Here, we will inspect elements that we want to test and create the object of the above-created custom assertion class. Finally, with the help of the created object, we will call the method that we created in the custom assertion.

package com.lt.assertions;
 
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
 
public class TestAssertion
{
   public static void main(String[] args)
   {
       RemoteWebDriver driver;
 
       String username = "user_name";
      //Enter your access key
      String accesskey = "access_key";
      //Enter Grid URL
      String gridURL = "@hub.lambdatest.com/wd/hub";
      //Set Desired capabilities to run on Lambdatest platform
      DesiredCapabilities capabilities = new DesiredCapabilities();
      capabilities.setCapability("browserName", "chrome"); //Set Browser Name
      capabilities.setCapability("version", "latest"); //Set browser version
      capabilities.setCapability("platform", "win10"); // If this cap isn't specified, it will just get the any available one
      capabilities.setCapability("build", "Hard Assert Test");
      capabilities.setCapability("name", "Hard Assert Test");
      capabilities.setCapability("visual", true);
      driver = new RemoteWebDriver(new URL("https://"+username+":"+accesskey+gridURL), capabilities);
 
       driver.get("https://www.lambdatest.com/");
      
       // Inspect the “Free Testing Button” element
       WebElement freetestingbutton = driver.findElement(By.xpath("(//button[contains(text(),"Start Free Testing")])[1]"));
          
       // Creating the object of CustomAssertion class
       CustomAssertion customassert = new CustomAssertion();
 
       // Calling the custom created method with the help of object
       customassert.assertButton(freetestingbutton);
   }
}

If we run the main class, then we can see our assertion is passed, and the success message is displayed on the console.

If we want to fail the above TestNG assertion, add the following change in the existing implementation:

// Inspect the “Free Testing Button” element
WebElement freetestingbutton = driver.findElement(By.tagName("p"));

Now if your test fail you can learn how to capture screenshot of failed tests easily through this video.

Conclusion

In this TestNG tutorial with a focus on TestNG assert, we looked at different asserts and constraints helpful in Selenium automation testing. The reason for using assert in test code is to pause the execution as soon as assert is encountered.

Selenium testing is an essential element in ensuring the effectiveness of test automation. With the combination of cloud-based Selenium testing tools like TestMu AI, development teams can verify product functionalities across several browsers & operating system combinations. However, the core business logic remains intact, with minimal changes required to port the TestNG test code to the remote Selenium Grid.

Let us know the TestNG asserts you have been using in the comments section to learn from each other’s experience. If you are preparing for an interview you can learn more through TestNG interview questions.

Happy Testing