Page tree
Skip to end of metadata
Go to start of metadata

Last updated: Oct 02, 2019 17:15

The Smart Reporting feature is available via the Test Analysis view. This section walks you through setting up, accessing, and navigating available resources.

Important: The Smart Reporting feature saves the full execution report (including screenshots and video) for 45 days. After 45 days, basic reporting details (test name, status, device & device properties, execution start & duration) are saved for an additional 45 days. Test steps and commands with screenshots or recording are deleted.

90 days after the initial test execution, the report is completely removed from the Perfecto Test Analysis view. This configuration is valid for all Perfecto clouds and is not subject to change. We recommend that you save your execution reports locally if you need them for a longer period of time, as follows:

  1. On the Report Library tab, click the report you want to view.
  2. When the report displays, click the download icon in the top right.
  3. From the menu that opens, under Export to PDF, select Full report.

Step 1 | Set up Smart Reporting

  1. Download the Repording SDK client for your programming languages and framework.
  2. Implement reporting by integrating the tool into your test, for JavaJavaScriptC#PythonUFT, or Ruby.

On this page:

Step 2 | Access the Test Analysis view

  1. Log in to Perfecto at: https://<CQLab>.app.perfectomobile.com.

  2. On the UI landing page, click Analyze Reports.

  3. Export the generated Execution Summary or Single Test report (optional) as:
    1. A PDF report
    2. JSON data that can be formatted (using the Public API)

Step 3 | View reports

The Test Analysis portal presents your test results on the following tabs (see in red box below) that present different views of the result data:

  • CI Dashboard: Displays the history of the test results, grouped based on the CI jobname identifier. Here, you can track your CI/CD pipeline and scheduled builds/jobs.
  • Heatmap: Displaysa graphic overview of different cross-sections of the test results. You can group test results by two levels of characterizations – devices or specific tags. The different groups are color-coded based on the distribution of passed or failed tests.
  • Report Library: Displays a list of all single test reports for the Perfecto Lab, along with recorded videos and screenshots. You can filter the listing to focus on a set of tests with a particular tag or drill down to view the full report. The tags may be execution-level context tags or specific test tags. You can also export reports in PDF format or as JSON data.
  • Live Stream: Displays a list of test executions currently in progress and allows you to stop one or more executions at once if needed. 
  • Admin : Provides access to admin-level tasks. This option is only available for users with administration privileges.

In addition, you can drill down to view the Single Test Report (STR) for a specific test. You can also export the generated Execution Summary or Single Test report (optional) as:

The following pages provide more information on these tabs:

Step 4 | Group or filter reports

The Heatmap and Report Library tabs display a sidebar panel on the right side of the window. The sidebar supports filtering, grouping, and saving custom views.

Filtering

You can filter the reports displayed based on any combination of the following parameters: 

  • When the test was run. Under Time Frame, select a range of dates. The default is to only display tests run in the past 24 hours.
  • Specific test information: 
    • Tags assigned to the tests
      By default, only the first 8 tags are displayed. If more tags are available, you can view them by clicking + Select another tag.

      Note: If less than 8 tags were used in the executions that ran during the selected time frame, the + Select another tag option is not available.
    • Custom Field values assigned to the tests
    • Device identification used by the tests, such as:
      • Device ID
      • Operating system
      • Browser
      • Form factor to view only tests run on Mobile devices or Desktop Web devices
    • Continuous Integration Job information
  • Final reported Status of the test - select to show only the tests that FailedPassed, or final status is Unknown.

Grouping

You can group jobs by the following parameters:

  • Browser
  • Device ID
  • Device model
  • Form factor
  • Job name
  • Operating system of the device
  • Project name
  • Project version
  • Resolution of the device display
  • Status of execution
  • Tags
  • Test
  • Custom Field values

You can select two levels of groupings, where the second grouping is applied within each group (by the first parameter).

Note: When grouping the Heatmap view by Job name, executions that did not provide the job information are not included in the Heatmap display although they are still available as a single test report (STR).

Custom views

After selecting group or filter parameters (see below) to focus on a subset of available test reports, you can save these settings as a custom view. A custom view allows you to:

  • Understand the progression of the test results by viewing sets of comparable test reports but from different time frames.
  • Share a focused view with other users.

To save the set of selected parameters as a custom view:

  1. In the sidebar header, click the save icon .
  2. In the Save View As form, do the following:
    • Enter a name for the view.
    • (Optional) If you want others to be able to access this view, turn on Shared view.
  3. Click Save.

To switch to a custom view:

  • In the upper left, expand the list to view existing custom and shared views.


To delete or share a custom view:

  1. In the sidebar, click the menu icon .
  2. Select Delete or turn on Shared view.

Video Tutorials

Introduction to Perfecto Reporting

How to add reporting to your test and view the resulting reports