TestNG | Soft and hard asserts

How do you pick the proper assert for your test? In short, if you need your @Test Method to fail straight away after one of its Asserts fails, use Hard Asserts. If you want the execution to proceed even if an assert fails, if you want to see how the following asserts behave, and if you want to see the full result at the end of the test, you need to use Soft Asserts.

Learn more in the following sections.

Hard Asserts

Different Asserts statements:

• Assert.assertTrue() & Assert.assertFalse()

  An Assert true statement fails the test and stops the execution of the test if the actual output is false. Assert.assertFalse() works opposite of Assert.assertTrue(). If you want your test to continue only if a certain element is not present on the page, use an Assert false statement so it will fail the test if the element is present on the page.

• Assert.assertEquals()

  This statement works similar to assert true and assert fail, but it will stop the execution if the value is not equal and carry on the execution if the value is equal.

The first argument in these asserts is the actual result that we need to compare with and the second one is the value expected from us.

This is how it looks:

import org.testng.Assert;
import org.testng.annotations.Test;
import appcode.SomeClassToTest;

public class TestNG_Asserts {
@Test
    public void testSum() {
        System.out.println("\nRunning Test -> testSum");
        SomeClassToTest obj = new SomeClassToTest();
        int result = obj.sumNumbers(1, 2);
        Assert.assertEquals(result, 3);
    }
    
    @Test
    public void testStrings() {
        System.out.println("\nRunning Test -> testStrings");
        String expectedString = "Hello World";
        SomeClassToTest obj = new SomeClassToTest();
        String result = obj.addStrings("Hello", "World");
        Assert.assertEquals(result, expectedString);
    }
}

When such an assert fails, the whole test fails. In other words, a Hard Assert throws an AssertException immediately when an assert statement fails, and the test suite continues with next @Test because it marks a method as failed if the assert condition gets failed and the remaining statements inside the method will be aborted.

If you need to put an assert that just checks the current state, without forcing the test to fail, you need to use soft asserts.

Soft Assert

Soft Assert collects errors during @Test is running. It does not throw an exception when an assert fails. The execution will continue with the next step after the assert statement. If you need to throw an exception (if such occurs), you need to use assertAll() method as a last statement in the @Test. The test suite again continues with next @Test as it is.

Unlike Hard Results, for Soft Asserts, we need to create an object to use them. This is how it looks:

//Please note that they use different import
import org.testng.asserts.SoftAssert;
import org.testng.annotations.Test;
import appcode.SomeClassToTest;

public class TestNG_SoftAsserts {
    
    @Test
    public void testSum() {
        // we need to create an object in order to use them
        SoftAssert sa = new SoftAssert();

        System.out.println("\nRunning Test -> testSum");
        SomeClassToTest obj = new SomeClassToTest();
        int result = obj.sumNumbers(1, 2);
        // we need to use that object when we are calling them
        sa.assertEquals(result, 2);
        System.out.println("\nLine after assert 1");
        sa.assertEquals(result, 3);
        System.out.println("\nLine after assert 2");


        // the asserts used till now will not throw any exception if they fail. The @Test will not fail either. 
        // If you need the test method to fail at the end, showing all exceptions, you need to use assertAll()
        sa.assertAll();
    }
}

Using assertAll() at the end will check if any of the asserts during the test failed, and will fail your test with a proper summary.