TestNG | Change test results dynamically

Use the IInvokedMethodListener to update your test results based on your own conditions. Before you continue, see TestNG | Custom TestNG listeners using ITestListener to familiarize yourself with the custom listeners.

TestNG will track test results for you as it runs: Pass, Failed, Skipped. There are times, however, where you may not want to rely on TestNG to make the deciding factor of if a test failed or passed and you may want to apply your own result to the test. In this case, first implement the IInvokedMethodListener in the Listener class you created and implement the missing methods.

Copy

public class TestListener implements IInvokedMethodListener {
    @Override
    public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {
        // TODO Auto-generated method stub


    }


    @Override
    public void afterInvocation(IInvokedMethod method, ITestResult arg0) {
        // TODO Auto-generated method stub
    }
}

Next, within the afterInvocation method, we will add the code to update the test status. The examples show how to change from Failed to Skipped based on the exception thrown, from Passed to Failed when a custom exception is thrown, and finally from Skipped to Failed when another custom exception is thrown.

Copy

    @Override
    public void afterInvocation(IInvokedMethod method, ITestResult arg0) {
        if (arg0.getMethod().isTest()) {
            //Change Failed to Skipped based on exception text
            if (arg0.getStatus() == ITestResult.FAILURE) {
                if (arg0.getThrowable() != null) {
                    if (arg0.getThrowable().getStackTrace() != null) {
                        StringWriter sw = new StringWriter();
                        arg0.getThrowable().printStackTrace(new PrintWriter(sw));
                        if (sw.toString().contains("visible")) {
                            ITestContext tc = Reporter.getCurrentTestResult().getTestContext();
                            tc.getFailedTests().addResult(arg0, Reporter.getCurrentTestResult().getMethod());
                            tc.getFailedTests().getAllMethods().remove(Reporter.getCurrentTestResult().getMethod());
                            Reporter.getCurrentTestResult().setStatus(ITestResult.SKIP);
                            tc.getSkippedTests().addResult(arg0, Reporter.getCurrentTestResult().getMethod());
                        }
                    }
                }
            }


            //Change Pass to failure and throw custom exception error
            if (arg0.getStatus() == ITestResult.SUCCESS) {
                ITestContext tc = Reporter.getCurrentTestResult().getTestContext();
                tc.getPassedTests().addResult(arg0, Reporter.getCurrentTestResult().getMethod());
                tc.getPassedTests().getAllMethods().remove(Reporter.getCurrentTestResult().getMethod());
                Reporter.getCurrentTestResult().setStatus(ITestResult.FAILURE);
                Reporter.getCurrentTestResult().setThrowable(new Exception("test Fail"));
                tc.getSkippedTests().addResult(arg0, Reporter.getCurrentTestResult().getMethod());
            }


            //Change skip to fail with custom failure
            if (arg0.getStatus() == ITestResult.SKIP) {
                ITestContext tc = Reporter.getCurrentTestResult().getTestContext();
                tc.getSkippedTests().addResult(arg0, Reporter.getCurrentTestResult().getMethod());
                tc.getSkippedTests().getAllMethods().remove(Reporter.getCurrentTestResult().getMethod());
                Reporter.getCurrentTestResult().setStatus(ITestResult.FAILURE);
                Reporter.getCurrentTestResult().setThrowable(new Exception("test Fail"));
                tc.getFailedTests().addResult(arg0, Reporter.getCurrentTestResult().getMethod());
            }
        }
    }

You should always wrap your decisions in the if condition. Otherwise, you will end up with configuration errors.

Copy

if (arg0.getMethod().isTest()) {

}

Due to the internal logic of TestNG reporting, it is also necessary to add and remove your current test results context from the reporting class before adding your final decision. Otherwise, you will end up with multiple entries in your final report. Remember to update the "getPassedTests", "getSkippedTests", "getFailedTests" to the appropriate object you want to modify.

Copy

tc.getPassedTests().addResult(arg0, Reporter.getCurrentTestResult().getMethod());   tc.getPassedTests().getAllMethods().remove(Reporter.getCurrentTestResult().getMethod());

Finally, re-add the new result (along with the exception for failures) to the Reporter class. Just like in previous examples, make sure you get the appropriate list when you add the result at in the end ("getPassedTests", "getSkippedTests", "getFailedTests").

Copy

Reporter.getCurrentTestResult().setStatus(ITestResult.FAILURE);
Reporter.getCurrentTestResult().setThrowable(new Exception("test Fail"));
tc.getFailedTests().addResult(arg0, Reporter.getCurrentTestResult().getMethod());