Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The bar across the top of the display (just below the top-level DigitalZoom Test Analysis tabs) displays:

  • Saved view selector - If you have any pre-configured views - select to view the current data based on the configured selections.
  • Active filters - lists all of the currently selected filters and groupings for the list.
  • Search by test name field (see next section) - field supports listing only the results whose test name matches the request (if any exist).

...

  • Start typing the name (or any sub-string of the name) and DigitalZoom will present . Smart Reporting presents a list of suggested available test names.
  • Either select the test name from the list of suggestions or complete typing the requested name, and activate the search.
  • The list refreshes to display only the tests whose name matches the search term.
    • If no tests match the search term, the Report Library will notify you that there were no matches.

...

  • Report Name - the script name as supplied in the PerfectoReportiumClient testStart() method.
    • The column includes a checkbox, used to select test reports for the Cross Report view.
  • Status -the final status reported by the specific test run. The test status is one of: Passed, Failed, Blocked, Unknown.
  • Test-set indicator - indicator that the test report is included in a test-set.
  • History - indicates when this test run was executed relative to other runs of the same test. (See below for more detail.)
  • Failure/Blocked reason - Indicates a reason detected, either automatically by DigitalZoom Smart Reporting heuristics or by the testers (see Implement the reporting framework), why the test failed. For more information, see below.
  • Platform -Indicates if tests were executed on Mobile or Web devices. May include and indication of the number of devices executed on.
    • Form factor - identifies (in icon format) if the testing device was a Mobile or Desktop Web device.
    • Device - List of the devices used in the test. Each device separated by comma.
    • Device ID - for mobile devices provides the ID number of the testing device. (not displayed by default)
    • Browser -browser version for Web devices.
    • OS - List of operating system versions, coordinated with devices list.
    • Resolution - List of device resolutions, coordinated with devices list. (not displayed by default)
  • CI Job Information - details of the CI Job reported for this test. (not displayed by default)
    • Name - Job name as reported in the execution context. (not displayed by default)
    • Number - Job number as reported in the execution context. (not displayed by default)
    • Branch - Job branch as reported in the execution context. (not displayed by default)
  • Time - Provides details of when test was run and duration.
    • Start - Start time of the test run
    • Duration - duration time of the test.
  • Tags - Indicates number of tags associated with the test run.
  • Lab - indicates if test was run in a Perfecto CQ Lab. (not displayed by default)
  • Automation Framework - indicates the automation framework supported by the device used by automation script. (not displayed by default)
  • Additional columns may be displayed containing the values of the custom fields associated with the tests.

...

Anchor
failBl
failBl
Failure/Blocked Reason

DigitalZoom Smart Reporting is designed to not only provide the test data from your test runs, but also to be tool that allows you - the tester or the test-manager - to better understand the results of your tests. If your test ends with a success status then you know that all is well. However, when a test fails, DigitalZoom Reporting Zoom may analyze the test data and provide a failure reason that indicates what caused the test to fail.

The DigitalZoom Reporting system Smart Reporting system provides functionality that supports generating this failure reason classification, either:

  • Manually by the test script, based on the stage in the script execution when the test is determined to have failed.
  • Automatically by DigitalZoom, that analyzes . Smart Reporting analyzes the entire test information and generates the reason based on a heuristic classification generates the reason.

In either case, the failure reason is color-coded and displayed in the grid of the test reports, in the Failure Reason column to allow a quick overview of the different failed tests.

Blocked Reasons

The DigitalZoom system Smart Reporting system will automatically identify instances when the test failed before the test was able to start. When these failures occur, the report will be marked with a blocked failure reason (that will appear in the table as the Failure Reason, color of reason in yellow) and the Status column will show a Blocked status (). These blocked failure reasons are completely controlled (failure reason text and color) by the DigitalZoom systemSmart Reporting system. Some examples of blocked failure reasons include:

...

Use the tags defined by the different context classes to better focus the analysis of the test results by filtering the tests to the subset of interest. See DigitalZoom Best Practices Test analysis: best practices document on how to optimize your test tags for efficient test analysis.

...