Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

In this section:

Table of Contents
maxLevel3

Introduction

The DigitalZoom The Test Analysis views provide a very good overview of the trends and highlights of your test programming. To get more specific information about a particular test, dive drill down to the Single Test Report (STR) , by selecting clicking the test report from in the Report Library table.

In this section:

Table of Contents
maxLevel3

Anchor
str
str
Single Test Report

...

overview

Clicking on the name of a test report in the Tests List Report Library opens the specific Single Test Report (STR):, as shown in the following figure.

Image Modified

Logical

...

steps

The left panel of the STR View shows the a list of the logical steps that comprise the test (as named in the testStep() method).

Reports for Native native automation executions that activate nested scripts , will include the steps of both the main script as well as the steps of the and the nested script. The commands of the nested scripts will be are identified by a special symbol ("</>").

Image RemovedImage Added

Clicking on any of the logical steps will present a logical step reveals a view of the artifacts (video, screenshots, expected vs. actual values) associated with the particular command/step.

Command

...

information

Click on the commands of the test a command to display detailed information regarding about the command execution, including -:

  • Timer information - : Displays the Perfecto Timer and UX Timer values when the command was executed.
  • Parameter information - : Identifies -Device the following:
    • The device used for the command
    • If The UI element (if the command accessed a UI Element - identifies the element
    • If the command inserted text - provides the text The text sent to the UI element . (if the command inserted text 
      Note: If the text was sent as a Secured String - then , the text value will appear appears as: "***"
    • Other information - may include , such as parameters for visual analysis, assertion information, or UI Element element attribute values.

Image Modified

...

Visual artifact area

The right panel of the STR View view presents visual artifacts, for example screenshots or videos from the test run.

When viewing you view the video, the timeline includes indicators that highlight the corresponding times that at which the logical steps occur at. Hovering occurred. Moving the pointer over any of these points display displays a tooltip that identifies the corresponding logical step.

Error

...

messages

When the test script displayed in the STR generated an Error message or Failed - the error message will be error or failure message, the message is displayed at the top of the Artifact Visual visual artifact area.

At first, only the header line of the error message is displayed on a red background:

Image Modified

To see the more complete error full message, together with a stack dump (if relevant), click the "Pull Down" button at the bottom of the error message to reveal the full messagedown arrow below the error message:

Image Modified

STR

...

header area

The Single Test Report header includes the top two rows of the STR. The header shows the following:

Image Modified

  • The top line includes:
    • Back to Report Grid button: This button reverts Reverts the display to Report Library View, regardless of any navigation to other test report views.
    • Name of the current test.
  • Second The second line includes:
    • Test status - shows the status of the test run of this STR.
    • History graph - shows five runs similar to the history graph in the Report Library View. Selecting one of the nodes of the graph navigates to the STR of the selected run. Tooltip provides information on the run for each node.
    • Run information - Start time and duration information of the test's run.
    • Device information - information on the device or devices used for the test run.
      • Activate interactive session icon - Opens the device in a Perfecto Lab interactive session. If the device is not available, the Perfecto Lab will notify the user to select another device.
    • Tags - list of tags associated with the test run.
    • JIRA bug reporting icon - appears if DigitalZoom Reporting Smart Reporting is integrated with JIRA. Supports entering bug reports directly as a JIRA issue.
    • Report Details button - displays detailed information on the test run data, and device(s) data.
    • Open Support Case - Connects directly to Perfecto Support to allow you to open a new incident.
    • Download button - supports accessing and downloading the artifacts (video, log files) associated with the test run.

Anchor
failR
failR
Failure Reason display

When the test script or DigitalZoom heuristics Smart Reporting heuristics have identified a Failure reason it will appear in the second line next to the Test status (only if the status is Failed). The Failure reason will be one of those configured for the CQ Lab by the Administrator.

If no Failure reason was identified for a failed test - , the Failure reason will be displayed as Add failure reason. This allows you to update (see below) the test report with a valid Failure reason.

...

Add (

...

update) the

...

failure reason

If no Failure reason was assigned to the test that failed, you can add a new Failure reason (or update the current Failure reason) by:

...

  1. If a Failure reason is assigned already, the menu will include an entry (Clear failure reason) that removes the assigned Failure reason (leaving the test report without any Failure reason).
  2. If the assigned reason is a Blocked reason - you cannot update the reason.

...

View report details

Use the Report Details button in the upper right corner of the STR View (see in figure above). This displays a popup window with information details related to the Test Run.

...

Excerpt
hiddentrue

Note: If provided, the EXECUTION tab information for a test run that failed during Driver creation, prior to ReportiumClient instantiation, is taken from the capabilities settings (for Selenium/Appium scripts).

Anchor
vcs
vcs

...

Access source code from the STR

Sometimes the test run did not complete as expected and there is a failure status. In many of these cases it is easier to understand what went wrong or what needs to be fixed in the test script, if you could view the source code. This functionality is dependent upon the tester supplying the information as described in the article here.

...

  • If the STR displays an error message, open the error message, and at the bottom there are links to open the source file display (see below).
  • For all STR click, on the Report Details Button - At the bottom of the Details popup window there are links to open the source file display (see below).

There are three configured links to access source code information:

...

Hovering over link 3 will display a tooltip encouraging the user to set the custom fields for future test runs.

















Link display

The links are displayed either:

  • Error Message, when you pull down to see the full error message, at bottom of the display (as shown above).
  • In the Report Details window, at the bottom

...

Display the source code

When clicking on one of the Open commit or Open source file links, DigitalZoom opens Smart Reporting opens a new tab in the browser and browses directly to the VCS at the display of either the commit or the source file.

...

Download artifacts

You can download a PDF version of the test report to your local workstation through the Download menu, by clicking on the download button.

Test reports include the basic information on the test execution displayed in the STR, parts of which may be downloaded to your local workstation using the the Download menu. In addition, the test may attach additional log files - of the device activity, network activity, or other vitals information - as artifacts.

Anchor
appCrash
appCrash
Application

...

crash report

There are times when a test "fails" because the application under test crashes - actually a case where the test succeeded in uncovering an application fault. When this occurs the mobile operating system generates a crash log that includes information that the application developer can use to identify the cause of the crash.

The Perfecto system identifies these situations and retrieves the crash log from the device and notifies the DigitalZoom analytics Smart Reporting analytics of the status. DigitalZoom will Smart Reporting will identify the failure reason for this type of test as Application crashed, and will add the crash log as an artifact to the test report.

...

  • Click the download menu button for the test report.
  • Click on the App-Crash-Report entry to download the log file.

Anchor
multidev
multidev
Tests on multiple devices

When a Perfecto Native Automation test script allocated multiple devices to run the test, the reporting system gathers artifacts (screenshots, video) from all of the devices involved. At the completion of the test execution the report for the test run generates a single report that includes the artifacts from all devices.

Info
iconfalse

Note: Displaying a single report for Multiple Devices is available only for Perfecto Native Automation.

Multiple

...

devices in the Report Library

...

view

Test-runs that activated multiple devices will be listed in the Report Library View with the following indications that multiple devices were involved:

  • Platform type column - indication of number of devices used.
  • Device column - list of all devices used.
  • OS column - list of all OS versions used, corresponding to the devices listed.
  • Resolution column - list of the device resolution for each device used.

Multiple

...

devices in the Single Test Report

...

view

When drilling down to the STR of a test that activated multiple devices -

...