Page tree
Skip to end of metadata
Go to start of metadata

Last updated: Oct 14, 2019 13:29

The Perfecto Gradle Plugin allows you, the tester, to:

  • Select a device or multiple devices from the Perfecto Lab to run Apple XCTest/XCUITest.
  • Install the application and test files onto the selected devices.
  • Run the test methods on the devices.
  • See the progress of the test set on the console.
  • Access the Report Library to view the results of the tests.

The following steps assume that the application and test files are available in the local disc storage and the plugin jar file is not installed. In this example, we use Apple Xcode to prepare the application files.

On this page:

Setting up the Gradle plugin involves the following:

  • Installing the Gradle plugin to prepare the build.gradle file
  • Configuring parameters through a configuration file, the command line, or encoded in the build.gradle file
  • Activating the plugin, understanding the output, and connecting to Smart Reporting execution reports

The Gradle plugin supports Gradle tasks for iOS environments. Use the task named "perfecto-xctest".

Each task supports the following actions:

  • Reading of Perfecto configuration parameters that select the devices to install and run the instrumentation tests
  • When working inside Android Studio, checking if there are tasks to build the application and testing apk files (and if any apk files are present, activating them)
  • Installing the application, testing apk files on Perfecto Lab devices, and running the test methods
  • Generating output to the Gradle console and the Test analysis with Smart Reporting report.
1 | Prepare the application and application test runner files

Executing XCUITest and XCTest tests involves the following application files:

  • The application .ipa file. This includes the XCTest unit tests.
  • The UI Runner .ipa file. This includes the XCUITest UI test methods.

To create the files and convert them to .ipa files:

  1. In Apple Xcode, change the scheme destination to Generic IOS device.

  2. Click Product > Build For > Testing. Once this is complete, open up the Organizer and select the latest version.

  3. Right-click the generated .app file in the Project Navigator (in the Products folder) and select Show in Finder.

    Finder displays the path to the Products folder and shows the following .app packages:

    • The application package

    • An app package for the UI tests named *UITests-Runner
  4. To convert the files to .ipa files, do one of the following:
    • Supply the path to the application package path to this file as the testAppPath configuration parameter. The Gradle plugin will convert it to an .ipa file.
    • Do the following to convert each of the files to .ipa files:
      1. Create an empty "Payload" (case sensitive) folder and copy the .app bundle to this folder.

        Note: If the Payload folder already exists, delete it and create a new, empty folder.

      2. Right-click the Payload folder and Compress it to a zip file.
      3. Rename the .zip file to <testName>.ipa.
2 | Add the Perfecto plugin dependency to your build.gradle file
  1. Create a new file named build.gradle.

  2. Add the lines to the build gradle file that define the location of the plugin library and the dependency on the plugin. Gradle will look to verify that the plugin is installed before performing the task.

    This is the recommended configuration method. However, if there are problems using the automatic download method described here, you can download the Perfecto Gradle plugin manually.


    1. To configure Gradle to automatically locate and download the plugin library, add the following lines to build.gradle file:

      buildscript {
          repositories {
              jcenter()
              maven {
                  url "https://repo1.perfectomobile.com/public/repositories/maven"
              }
          }
        
          dependencies {
              classpath "com.perfectomobile.instrumentedtest.gradleplugin:plugin:+"
          }
      }
    2. If the plugin library is already downloaded to a folder (for example the libs sub-folder), add the following lines to build.gradle file:

      buildscript {
          repositories {
              jcenter()
        
              flatDir dirs: 'libs'
          }
        
          dependencies {
              classpath "com.perfectomobile.instrumentedtest.gradleplugin:plugin:+"
          }
      }
  3. Add the line that defines the plugin task:

    apply plugin: 'com.perfectomobile.instrumentedtest.gradleplugin'
  4. Add the plugin configuration settings by including the following Lab authentication parameters:
    • The URL for the Perfecto Lab
    • Your personal Security Token generated for the Perfecto Lab

      perfectoGradleSettings {
          cloudURL "mobilecloud.perfectomobile.com"
          securityToken "AAABNg0ODAoPeNqtkT1PwzAQhnf/CkssM...JxQ3HEI8NsX02ff"
      }



      Note: You could also supply the configuration file location parameter in this gradle-file clause.
3 | Create an XCUITest configuration file

In this step, you select which devices to run on and define other configurations in a configuration file.

  1. Create a configuration file (ConfigFile.json) as a JSON text file in a known folder (you will need the full path later).
  2. Add the device selection parameters to the configuration file.

    1. If specifying a set of devices.

      "devices": [
          {
              "platformName":"IOS",
      		"platformVersion":"^[9]\\.[3-9].*|^[1-9][0-9].*"
          },
          {}
      ]
      
      

      This example selects two devices - one that is an iOS device of version 9.3 and up (using a regular expression), and one is randomly selected by the plugin.

    2. If specifying a number of random iOS devices

      "numOfDevices": 20


      NOTE:
       If a device has not been explicitly selected (using the deviceName property) or when numOfDevices is used (without explicitly setting platformVersion), the system will automatically select an iOS device with accordance to the minimum iOS version as defined in your application.

  3. Add the reporting parameter settings, to add tags to the DigitalZoom execution report.

    "tags": ["plugin", "xctest", "demo"],
    "projectName": "playground",
    "projectVersion": "1.5",
    "jobName": "newFeature",
    "jobNumber": "45",
  4. Add the application parameters - identifying where the application files are located.

    • Using the appPath and testAppPath fields - for iOS applications

      "testAppPath": "/Users/myUser/root/XcTest/Playground/app-Runner.ipa",
      "appPath": "/Users/myUser/root/XcTest/Playground/app.ipa"
  5. Save the configuration file that should look something like the following:

    {
        "devices": [
            {
    			"deviceName": "FA6BR0304878"
    		},
            {
                "platformName":"IOS",
    			"platformVersion":"^[9]\\.[3-9].*|^[1-9][0-9].*"
            },
            {}
        ],
        "testAppPath": "/Users/myUser/root/XcTest/Playground/app-Runner.ipa",
    	"appPath": "/Users/myUser/root/XcTest/Playground/app.ipa",
        "tags": ["plugin", "xctest", "demo"],
        "projectName": "playground",
        "projectVersion": "1.5",
        "jobName": "newFeature",
        "jobNumber": "45"
    }

You can see a full list of configuration parameters here.

4 | Execute the plugin and access the execution report
  1. Open a command-line (terminal) window in the folder where you want to execute the plugin. 

  2. Execute the plugin using the following command in the command-line window:
    Here we also supply the full path to the Configuration File created in step 2. Other configuration parameters (except for device selection) could be added to the command line.

    For iOS apps
    gradle perfecto-xctest -PconfigFileLocation="C:\temp\Testing\ConfigFile.json"

    This will:
     - Select the devices as specified in the Device selection parameters of the configuration file [or a random device if no specification provided].
     - Install the application and test apk files onto the device
     - Run the test methods (based on the configuration parameters)
     - Send output to the console window
     - Generate an execution report that can be viewed in the Test analysis with Smart Reporting interface.

    The command line parameters can set any of the configuration parameters, except for device selection parameters. For more information on the configuration parameters, see Configuration parameters for the Gradle Plugin.

     Click here for information on running the Gradle plugin over a proxy connection

    If you run the Gradle plugin over a proxy connection, you need to supply the proxy information in form of the following Java parameters when activating the plugin:

    • http.proxyHost - IP address of the proxy
    • http.proxyPort - IP port used for connection
    • http.proxyUser - username for connection to proxy server
    • http.proxyPassword - password for connection to proxy server.

    For proxies supporting SSL encryption, use the following Java Proxy parameters:

    • https.proxyHost - IP address of the proxy
    • https.proxyPort - IP port used for connection
    • https.proxyUser - username for connection to proxy server
    • https.proxyPassword - password for connection to proxy server.

    For example:

    gradle perfecto-xctest -PconfigFileLocation="C:\temp\Espresso\ConfigFile.json" -Dhttp.proxyHost=10.0.0.100 -Dhttp.proxyPort=8800 -Dhttp.proxyUser=someUserName -Dhttp.proxyPassword=somePassword




    During the execution, the plugin will report on the progress of the execution, and the completion of each test method to the command-line window.

     Click view sample output

    At the end of the execution, a high-level summary report of the completion status for each device used will be presented in the command-line window. The report includes:

    • Configuration settings resolution: The configuration file used and the list of the configuration settings for the test run
    • Progress notifications as the test is configured, installed, and executed, including notifications of the start and completion of:
      • Allocating the device
      • Resigning the application
      • Installing the application and UI runner files
      • Executing the test
      • Connecting the video recording
    • Completion status for each test method on each device
     Click to view a sample summary report
    Parsing configuration file: configFile.json
    
    Parsed configuration file configFile.json successfully
    
    Configuration:
    cloudURL='YourLab.perfectomobile.com'
    appName='app'
    tags='[test_1, test2]'
    projectName='IOS-XCUITest'
    projectVersion='v10.9-test'
    jobName='some_job'
    jobNumber='2'
    branch='master'
    failBuildOnFailure='false'
    debug='false'
    devices='[DeviceDetails{deviceName='047CEB33AF9306ED58A002DAA400DF726C38AAC1', platformName='IOS', platformVersion='null', manufacturer='null', model='null', resolution='null', network='null', location='null', description='null', selectionCriteria='null', index='1'}]'
    acquireDeviceRetryNumber='0'
    acquireDeviceRetryInterval='30'
    postExecution='PostExecutionDetails{uninstall=false}'
    preCleanUp='false'
    resign='true'
    takeScreenshotOnTestEnd='false'
    takeScreenshotOnTestFailure='false'
    shard='false'
    ipaPath='/Users/.../root/xcodeSampleApp/ipas/SimpleAppForTesting.ipa'
    testIpaPath='/Users/.../root/xcodeSampleApp/ipas/Payload.ipa'
    runUiTests='true'
    runUnitTests='true'
    
    
    Device: 00083F5BF9243A97E19FA80FAD9D8649653F6819 Apple iPhone-7 Plus: Operating on device {
    	"deviceId":"00083F5BF9243A97E19FA80FAD9D8649653F6819",
    	"osVersion":"11.3",
    	"screenResolution":"1242*2208",
    	"location":"NA-US-PHX",
    	"manufacturer":"Apple",
    	"model":"iPhone-7 Plus",
    	"firmware":"11.3",
    	"distributor":"MacCam11",
    	"selectionCriteria":{
    		"platformVersion":"^[9]\\.[3-9].*|^[1-9][0-9].*",
    		"platformName":"IOS"},
    	"os":"iOS"} 
    according to device details {
    	"platformName":"IOS",
    	"platformVersion":"^[9]\\.[3-9].*|^[1-9][0-9].*",
    	"automationName":"PerfectoMobile",
    	"index":2}
    
    Device: 00083F5BF9243A97E19FA80FAD9D8649653F6819 Apple iPhone-7 Plus: Resigning application
    
    Device: 00083F5BF9243A97E19FA80FAD9D8649653F6819 Apple iPhone-7 Plus: Completed resigning application
    
    Device: 00083F5BF9243A97E19FA80FAD9D8649653F6819 Apple iPhone-7 Plus: Uploading IPA to device server
    
    Device: 00083F5BF9243A97E19FA80FAD9D8649653F6819 Apple iPhone-7 Plus: Completed uploading IPA to device server
    
    Device: 00083F5BF9243A97E19FA80FAD9D8649653F6819 Apple iPhone-7 Plus: Starting video recording
    
    Device: 00083F5BF9243A97E19FA80FAD9D8649653F6819 Apple iPhone-7 Plus: Enabling instrumented test mode
    
    Device: 00083F5BF9243A97E19FA80FAD9D8649653F6819 Apple iPhone-7 Plus: Uninstalling IPA
    
    Device: 00083F5BF9243A97E19FA80FAD9D8649653F6819 Apple iPhone-7 Plus: Uninstalling test bundle
    
    Device: 00083F5BF9243A97E19FA80FAD9D8649653F6819 Apple iPhone-7 Plus: Installing ipa SimpleAppForTesting.ipa
    
    Device: 00083F5BF9243A97E19FA80FAD9D8649653F6819 Apple iPhone-7 Plus: IPA  C:\Windows\TEMP\ITES\4b918fc2-f1a1-4d7c-9241-1730cfe471cb\00083F5BF9243A97E19FA80FAD9D8649653F6819\2\Apps\SimpleAppForTesting.ipa Installed
    
    Device: 00083F5BF9243A97E19FA80FAD9D8649653F6819 Apple iPhone-7 Plus: Installing ipa SimpleAppForTesting-Runner.ipa
    
    Device: 00083F5BF9243A97E19FA80FAD9D8649653F6819 Apple iPhone-7 Plus: IPA  C:\Windows\TEMP\ITES\4b918fc2-f1a1-4d7c-9241-1730cfe471cb\00083F5BF9243A97E19FA80FAD9D8649653F6819\2\Apps\SimpleAppForTesting-Runner.ipa Installed
    
    Device: 00083F5BF9243A97E19FA80FAD9D8649653F6819 Apple iPhone-7 Plus: Starting execution of bundle id com.igor.SimpleAppForTesting
     Click to view a sample error message

    The following error message appears upon failure to allocate a device:

    Failed to execute command handset open: Cannot open device FA6BR0304878 for user myuser@mycompany.com, device is in use.
    Command duration or timeout: 1.20 seconds
    Build info: version: '2.53.1', revision: 'a36b8b1cd5757287168e54b817830adce9b0158d', time: '2016-06-30 19:26:09'
    System info: host: 'nb-user-macosx.local', ip: '172.30.200.19', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.12.6', java.version: '1.8.0_131'
    Driver info: org.openqa.selenium.remote.RemoteWebDriver
  3. As part of the summary report, the plugin provides the URL of the single test report (STR) for the execution run. Copy the report URL from the summary report on your console:

    View the detailed report at: https://demo.reporting.perfectomobile.com/library?startExecutionTime[1]=lastMonth&tags[0]=5af27a82-54cc-405a-8c6e-fa46fcae874b 
    Finished flow execution
  4. Open the URL in your browser to access the execution report. Here, you can drill down to see the screenshots of the different test method executions and what went wrong, if anything. For more information on Smart Reporting, see the Reporting overview. For details on the STR, see Single test report (STR).

Samples of plugin use

The Perfecto GitHub repository includes different samples that demonstrate how you can use the Perfecto Gradle plugin for the following different modes/configurations:

  • defaultAndroidProjectSample: An Android project with the plugin Json config file located in the default place
  • localJarSample: An Android project configured to run with the Perfecto plugin jar file locally
  • pluginConfigurationSample: An Android project with the cloudURL/security token configured in the build.gradle file
  • remoteRunSample: How to run the plugin without the source code, only with .apk files

Demo video

The following video demonstrates the above procedure.