Last updated: Dec 31, 2018 17:01
Preparing the Application Files
Executing XCUITest and XCTest tests involve two application files:
- The application ipa file - this includes the XCTest unit tests.
- The UI Runner ipa file - this includes the XCUITest UI test methods.
Use the following steps to create the files as ipa files :
Step 1: Change scheme destination to Generic IOS device.
Step 2: Click Product > Build For > Testing. Once this is complete, open up the Organiser and select the latest version.
Step 3: Right click on the generated .app file in the Project Navigator (in the Products folder) and select Show in Finder
Step 4: Finder will display the path to the Products folder, and sill show two .app packages -
The application package
- An app package for the UI Tests named *UITests-Runner.
There are two options for using these files:
- Supply the path to the application package path to this file as the testAppPath configuration parameter. The Gradle plugin will convert to .ipa file.
- Use the following steps to convert each of the files to ipa files:
Step 1: Create an empty "Payload" (case sensitive) folder and copy the .app bundle to this folder.
- Step 2: Right click on the Payload folder and Compress it to a zip file.
- Step 3: Rename the .zip file to <testName>.ipa
Activating the Plugin Task
To activate the plugin - Open a command-line window in the project's folder and run the perfecto-xctest task in gradle:
- Select the devices as specified in the Device selection parameters of the configuration file [
The command line parameters can set any of the configuration parameters except for device selection parameters. For more information on the configuration parameters read this page.
Running over a proxy connection
If the Perfecto Lab is connected via a proxy, you will need to supply the proxy information as Java parameters when activating the plugin. Use the following Java Proxy parameters:
- http.proxyHost - IP address of the proxy
- http.proxyPort - IP port used for connection
- http.proxyUser - username for connection to proxy server
- http.proxyPassword - password for connection to proxy server.
For proxies supporting SSL encryption, use the following Java Proxy parameters:
- https.proxyHost - IP address of the proxy
- https.proxyPort - IP port used for connection
- https.proxyUser - username for connection to proxy server
- https.proxyPassword - password for connection to proxy server.
Screenshots (XCTest methods only)
XCTest test methods may snap screenshots at key points during the test run. These screenshots will be retrieved by the plugin and included in the execution report. The plugin supports screenshots generated by either:
- KIF system KIFSystemTestActor class
- EarlGrey screenshots
To take screenshots using the KIFSystemTestActor captureScreenshotWithDescription() method:
Use the instrumentationArgs configuration parameter to indicate using KIF for screenshots
This will preset the KIF_SCREENSHOTS environment variable for the test run.
- Supply a description parameter that includes the test's className and testMethod in the following format:
"testClass#testMethod" (full description may include a prefix or suffix to this string). This format allows to associate the screenshot to the correct test method
To take screenshots using the EarlGrey GREYScreenshotUtil class:
Use the instrumentationArgs configuration parameter to indicate using EarlGrey for screenshots
- The screenshot should be in a "png" file and the name should include a string that includes the test's className and testMethod in the following format:
"testClass#testMethod.png" (full description may include a prefix or suffix to this string). This format allows to associate the screenshot to the correct test method.
The following XCode snippet provides an example of creating the screenshot:
Summary report on console
During the execution of the test methods, the plugin reports to the console:
- Configuration settings resolution - the configuration file used and the list of the configuration settings for the test run.
- Progress notifications as the test is configured, installed, and executed including notifications of the start and completion of:
- Allocating the device
- Resigning the application
- Installing the application and UI runner files
- Executing the test
- Connecting the video recording
- Completion status for each test method on each device.
The following is a sample of this report:
The following is a sample of an error message (failure to allocate device) as it appears in the report:
When execution completes on all devices, the plugin produces a summary report of the total execution status that includes:
- Total summary - indicates whether there were any errors reported for the tests, and on how many devices the test ran successfully.
- Per Device summary - for each device used - indicates
- Final test run status
- Device information
- Number of test methods that reported success and failure on that device.
- URL for the Execution report on the DigitalZoom Reporting server.
The following is a sample of this report:
As part of the summary report (see above) - the plugin provides the URL of the DigitalZoom STR for the execution run. Use the URL to connect to the reporting server and drill down to see the screenshots of the different test method executions and see what went wrong. For more information on the STR see the Reporting overview.
Samples of plugin use
The Perfecto GitHub repository includes
The different samples demonstrate using the plugin for the following different modes/configurations:
- defaultAndroidProjectSample - an Android project with the plugin Json config file located in the default place
- localJarSample - an Android project configured to run with the Perfecto plugin jar file locally
- pluginConfigurationSample - an Android project with the cloudURL/security token configured in the build.gradle file
- remoteRunSample - how to run the plugin without the source code and only with apk's