When analyzing test executions- especially at a bigger scope- it is vital to immediately see whether a test passed or failed.
However, sometimes we observe tests with Unknown status in the reporting- which almost always requires analysis to understand what happened.
Thus Unknown is an undesirable result, mildly put.
Why are tests with status Unknown present in my reporting?
Causes for "Unknown" test status
-
The test was run without Reporting context
When a logical test is run, it needs to have implemented reporting to log the test details in Smart Reporting. See Implementing Reporting for further details.
If this is not present, a test with generic name like RemoteWebDriver will be created, likely with Unknown result unless the test failed at driver instantiation.
Copy//** Below step creates the general context of the execution. Reports of all tests run with this driver will share this metadata
PerfectoExecutionContext perfectoExecutionContext = new PerfectoExecutionContext.PerfectoExecutionContextBuilder()
.withProject(new Project("My Project", "1.0"))
.withJob(new Job("My Job", 45))
.withContextTags("tag1")
.withWebDriver(driver)
.build();
//** Below step is vital to name the actual test
ReportiumClient reportiumClient = new ReportiumClientFactory().createPerfectoReportiumClient(perfectoExecutionContext);
try {
reportiumClient.testStart("My test name", new TestContext("tag2", "tag3"));
...
reportiumClient.testStop(TestResultFactory.createSuccess());
}
... -
No test status was sent to Smart Reporting upon ending the test.
Often this just means, that a possible outcome of the test is not implemented. In below example, the test is started with a logical name.
However an exception as part of the script is not caught. In the example, a specific Webdriver exception would be caught, but a plain nullpointer exception is not- so in this case neither failed nor passed is sent to reporting as test result.
See Implementing Reporting for further details.
Copy...
//** Below step is vital to name the actual test
ReportiumClient reportiumClient = new ReportiumClientFactory().createPerfectoReportiumClient(perfectoExecutionContext);
try {
reportiumClient.testStart("My test name", new TestContext("tag2", "tag3"));
//** Below will throw a java.lang.NullPointerException (which is NOT caught, so no result will be passed)
Integer aNumber=null;
aNumber+=3;
//** This part is essential to assign a test result to the test- success
reportiumClient.testStop(TestResultFactory.createSuccess());
} catch (WebDriverException e) {
//** This part is essential to assign a test result to the test - failure
reportiumClient.testStop(TestResultFactory.createFailure(e.getMessage(), e));
e.printStackTrace();
} finally {
try {
driver.quit();
...
