DOCUMENTATION
Page tree
Skip to end of metadata
Go to start of metadata

Last updated: Dec 02, 2019 18:05

The Perfecto Smart Reporting feature optimizes the visibility of test results for enhanced analysis. To gain the maximum benefits, this section provides an overview of the basic concepts.

Tags

Smart Reporting allows you to attach a set of tags to a test or a specific subset of the test. You may later use these tags to create different cross-sections of the test results and to focus on a specific subset of test reports.

Best Practice

It is important to use tags that are meaningful and valuable to your teams. Tags should be classified into different buckets:

  • Execution level tags: Configured across all tests in a particular execution.
  • Single test tags: Identify specific tests that may exercise a particular subset of the application.
  • Logical step names: Identify the different logical execution steps of the test.

To learn more about how to optimize your tags for easier report analysis, see Smart Reporting Best Practices.

On this page:

Context classes

The Reporting SDK includes the following Context classes used to declare parameters associated with different sets of tests:


PerfectoExecutionContext class

The PerfectoExecutionContext class is used to associate different settings with general text execution. You create instances of this class using a builder method - PerfectoExecutionContextBuilder. Using the builder method, you can also declare the following settings with this class:

  • Project identifier - use the "withProject()" method to set a label for the execution to associate with a particular testing project.
  • Job identifier - use the "withJob()" method to associate the job identifier (especially when running tests within a CI tool) of a particular run with an execution.
  • Tags - use the "withContextTags()" method to declare a set of tags to be associated with all tests run in this execution.
  • Custom Fields - use the "withCustomFields()" method to define a collection of <"name", "value"> pairs of information associated with all tests run in this execution.

All of these settings are associated with all tests executed within the entire Execution unit. This association could be used to filter all of the tests relative to a particular execution when examining the actual reports (see below for more information).

In addition, the class supports the following methods:

  • withWebDriver() - defines the automation driver that the Reporting system will use to collect the execution data and artifacts from.
  • build() - activates the builder to configure the Reporting execution.
TestContext class

The TestContext class supports defining a separate set of data associated with a particular test execution. Instances of the TestContext class are configured using the Builder() method. You may provide:

  • A list of specific tags, using the withExecutionTags() method, as a comma separated list of tags, to the instance builder.
  • Custom field <name, value> pairs, using the withCustomFields() method, as a collection of CustomField objects.
Using the TestContext Constructor
reportiumClient.testStart("myTest", new TestContext.Builder()
                   .withTestExecutionTags("Sanity", "NewNote", "Android")
                   .withCustomFields(new CustomField("team", "dbApps"))
                   .build());

Custom fields

You may need to add information to a specific test that requires separate values for a tag. For example, to identify a particular tester, you may want to define a "tester" tag that should be "John" for some tests and "Judy" for other tests. Custom fields allow you to add tags that are <name, value> pairs. You can use them to identify and filter your different test runs, either setting them at the Execution Context instance, the Test Context instance, or (if you are using a CI tool) as part of the JVM command line parameters. The order of precedence of conflicting custom field values is:

  1. JVM values (highest)
  2. TestContext values (medium)
  3. ExecutionContext values (lowest)

Job identification

When using Continuous Integration tools to automatically generate and run your tests, for example when performing regression testing, the CI tool associates a Job Name and Job Number with each run of the process. In addition, you may have different versions of the tests defined in different branches of the repository. Import these identifiers into your tests and tag the tests to associate the Single Test Reports with these CI Jobs. Learn how to import the job identifiers for different CI Tools here. Then view the Test summary reports in the CI Dashboard.

Test steps

Within each test, it is possible to group the different executed commands into logical steps. You can identify each step by a step name, and the step name displays in the test report together with any data and artifact (for example, a screenshot or video) associated with the commands executed by the step. Identifying the steps is part of the ReportiumClient stepStart() method, and attaching a result for the step is part of the stepEnd() method.