Page tree
Skip to end of metadata
Go to start of metadata

When running your Android instrumentation tests with the Perfecto Gradle Plugin, you can add configuration information for the test execution at three levels:

  • Configuration file - a Json formatted file listing parameter assignments - (see below).
  • Within the module's build.gradle file - in the perfectoGradleSettings clause, include Authentication parameters and indicate the location of the configuration file.
  • On the command-line - include any of the parameters in format: -P<paramName>="<paramvalue>".

Configuration Parameters

The configuration parameters are grouped into the following categories:

Perfecto Lab Authentication

ParameterPossible ValuesMeaning
cloudURL URL of the Perfecto Lab to connect to. For example, mobilecloud.perfectomobile.com
securityTokenSecurity TokenTester's personal security token for the Perfecto Lab.

These parameters may be set:

  • In the configuration file, in the following format:

    "cloudURL": "mobilecloud.perfectomobile.com"
    "securityToken": "AAABNg0ODAoPeNqtkT1PwzAQhnf/CkssM...JxQ3HEI8NsX02ff"
  • In the build.gradle file, in the following format:

    perfectoGradleSettings { 
        cloudURL "mobilecloud.perfectomobile.com" 
        securityToken "AAABNg0ODAoPeNqtkT1PwzAQhnf/CkssM...JxQ3HEI8NsX02ff" 
    } 
  • In the command-line in the following format:

    > gradle perfecto-android-inst -PcloudURL="mobilecloud.perfectomobile.com" -PsecurityToken="AAABNg0ODAoPeNqtkT1PwzAQhnf/CkssM...JxQ3HEI8NsX02ff"

Device Selection

Obtain the values for the following parameters based on the information in the Perfecto Lab Launch Mobile window as listed for the specific devices to select.

ParameterPossible ValuesMeaning
deviceName The device ID
description The device description as defined in the Perfecto Lab.
location Physical location of the device.
manufacturer For mobile devices manufacturer of the model. For example: Samsung, HTC.
model Device model - for example: Galaxy S III, HTC One
network Device network, For example AT&T, Verizon
platformNameAndroidDevice operating system
platformVersion Operating system or browser version
resolution Screen resolution of device or browser
Note: If no device selection parameters are set, plugin will select a random Android device, from the devices connected to the Perfecto Lab.

These parameters may be set in the configuration file, in the following format:

  • Single device selection:

    "devices": {
    	"platformName" : "Android"
    	"platformVersion": "^[678].*"
    	"manufacturer" : "Samsung"
    }
  • Multiple device selection (in the following example, four devices are selected):

    "devices" : [
    	{"deviceName" : "FA6BR0304878"},
    	{
    		"platformName" : "Android",
    		"platformVersion": "^[678].*",
    		"manufacturer" : "Samsung"
    	},
    	{
    		"model": "HTC One",
    		"location": "US-MA-BOS",
    		"network": "AT&T, Verizon"
    	},
    	{}
    ]

    Note: The fourth device in this example - "{}" is chosen randomly from all available Android devices.

Selection Retry Parameters

If the plugin was unsuccessful in allocating the device, you can use the following parameters to configure a retry mechanism built into the plugin:

ParameterPossible ValuesMeaning
acquireDeviceRetryNumber default = 0Number of retries that plugin should attempt to allocate the device.
acquireDeviceRetryInterval default = 30Number of seconds to wait between each retry attempt

These parameters may be set in either the configuration file, the perfectoGradleSettings clause of the build.gradle file, or on the command line, in the following format:

Configuration File format
"acquireDeviceRetryNumber" : 5
"acquireDeviceRetryInterval" : 45
build.gradle file
perfectoGradleSettings { 
	...
    acquireDeviceRetryNumber  5
	acquireDeviceRetryInterval  45 
} 
Command Line format
gradle perfecto-android-inst ... -PacquireDeviceRetryNumber=5 -PacquireDeviceRetryInterval=45

Network Virtualization

The Gradle plugin supports configuration of a virtual network environment to execute the Espresso/Robotium tests. The virtual network can be configured using the same parameters supported by the Perfecto Network virtualization start command. The following is the list of parameters supported:

ParameterPossible ValuesMeaning
latencyIn the range 0-8000 ms.Latency applied on packets in the Network.
packetLossIn the range 0-100%Network packet loss.
A reasonable packet loss value should not exceed 5%.
bandwidthInIn range 3-100,000 Kbps
or unlimited.
Limitation on the allowed download network bandwidth into the device.
bandwidthOutIn range 3-100,000 Kbps
or unlimited.
Limitation on the allowed upload network bandwidth from the device.
packetCorruptionIn the range 0-100%Network packet corruption.
A reasonable packet loss value should not exceed 5%
packetReorderingIn the range 0-100%The percentage of network packets sent immediately without any delay.
Used alongside the latency parameter. Moreover, the packets sent
immediately will arrive earlier than the packets that were delayed by the
defined latency value, essentially creating a packet reordering.

A reasonable packet reordering value should not exceed 10%.
packetDuplicationIn the range 0-100%Network packets duplicated.
A reasonable packet duplication value should not exceed 3%.
delayJitterIn the range 0-8000 ms.Random latency variation; the actual latency between latency +- jitter.
Used alongside the Latency parameter.

For example, if the latency is defined to be 100 ms and
the jitter is 10 ms, this causes the added delay to be 100ms - 10ms.
correlationIn the range 0-100%Network packet value correlation, affecting the Latency, Corruption,
Reordering and Duplication.

For Latency, Corruption, Reordering and Duplication, the current packet
n value will be correlated by this % to previous packet n-1 value.
In other words, the current packet value is correlated to the previous
packet value.

For example, if the correlation is defined to be 25%, the packet loss is 5%,
and the previous packet was lost, then the updated packet loss value
(for the current packet) would be 75% * 5% + 25%, which equals 28.75%.
However, if the previous packet was not lost, then the packet loss value
would be 75% * 5%, which equals 3.75%.
blockedDestinationsList of StringsNetwork packet block, to specific destinations, defined by domain name,
IP address, and IP range destinations in IP Prefix (Slash) notation.
blockedPortsList of StringsNetwork packet block, to specific ports.
For example, to block http, define port number 80.
To unblock, prefix the value with a '-'. For example, -80.
generateHarFiletrue | false

Indicates if a HTTP Archive (HAR) file should be generated for
each test to analyze the traffic of the virtual network. See Note below.

profile Suggested network virtualization profiles.
See the Network Conditions for supported values 

Note: To use the generateHarFile parameter, first install the certificate and configure the device using Perfecto Automation. This feature is currently  in limited release, contact support

These parameters may be set:

  • In the configuration file, in the following format:

    "networkEmulation": {
        "latency": 89,
        "packetLoss": 7,
        "bandwidthIn": 5,
        "bandwidthOut": 40
    }
  • In the perfectoGradleSettings clause of the build.gradle file:

    perfectoGradleSettings { 
    	...
    	networkEmulation {
    		packetLoss 5
    		profile    4g_lte_good
    	}
    	...
    }

Application Parameters

ParameterPossible ValuesMeaning
instrumentationArgs 

Custom arguments to pass to the Instrumentation Runner.

format:

  • Configuration file:
    "instrumentationArgs" : ["key"="value","key"="value",...]
  • Command line:
    -PinstrumentationArgs="key=value;key=value;..."
failBuildOnFailure true | falseReport a failure to the Gradle console if any test fails. Default is false.
debug Run the task in debug mode to output more verbose messages.
apkName The APK name of the application to test. If the run is part of your application build and If you don’t specify path to the APK, you should set that field so the Plugin should take the APK from your build (by default set to “app”).
apkPath Path to the APK. Use this field if the run is not part of your application build or your APK is output to a different location than the default.
testApkPath Path to the test APK. Use this field if the run is not part of your application build or your APK is output to a different location location than the default.

When activating the plugin:

  • Within an Android Studio project, supply the apkName parameter.
  • As a standalone activation, supply both the apkPath and testApkPath parameters.

Note: The apkName of the Espresso/Robotium test application, is the same as the apkName, with a "-AndroidTest" suffix.

In addition, there are filter parameters that limit the test scenarios executed during the run:

ParameterPossible ValuesMeaning
testSize large | medium | smallRuns a test method annotated by size. The annotations are @LargeTest,
@MediumTest, and @SmallTest .
See examples below.
testClassNames 

 Array of class names that you wish to run. If not specified, all the classes will run. 

format:

  • Configuration file -
    "testClassNames":["Class","Class",...]
  • Command line -
    -PtestClassNames="Class;Class;..."
 testMethodNames 

 Array of method names that you wish to run. If not specified, all methods will run. 

format:

  • Configuration file -
    "testMethodNames":["Class#Method","Class#Method",...]
  • Command line -
    -PtestMethodNames="Class#Method;Class#Method;..."

Note: Use only either testClassNames or testMethodNames, not both.

 

These parameters may be set:

  • In the configuration file, in the following format:

    "testApkPath": "Users/usern/AndroidStudioProjects/Playground/app/build/outputs/apk/app-debug-AndroidTest.apk"
    "apkPath": "Users/usern/AndroidStudioProjects/Playground/app/build/outputs/apk/app-debug.apk"
    "testSize": "small"
    "testMethodNames": ["com.perfectomobile.com.DemoClass#demoMethod","com.perfectomobile.com.DemoClass#demoMethod2"]
  • In the perfectoGradleSettings clause of the build.gradle file:

    perfectoGradleSettings { 
    	...
        testApkPath "Users/usern/AndroidStudioProjects/Playground/app/build/outputs/apk/app-debug-AndroidTest.apk"
    	apkPath "Users/usern/AndroidStudioProjects/Playground/app/build/outputs/apk/app-debug.apk" 
    } 
  • In the command line, in the following format:

    gradle perfecto-android-inst ... -PapkPath="Users/usern/AndroidStudioProjects/Playground/app/build/outputs/apk/app-debug.apk" -PtestSize="large" 

Pre-Execution and Post-Execution Parameters

The following parameters control whether the application and test apk files are uninstalled from the devices, either prior to installing the latest version or at the end of the tests:

ParameterSub-parameterPossible ValuesMeaning
installationDetails  The parameters in this Clause affect the installation phase of the test run.
-preCleanUptrue | falseIf the parameter is set (true) then the apk files of the application and test-application are uninstalled before installing the new version (supplied by above parameters)
postExecution  The parameters in this Clause affect the post-execution phase of the test run.
-uninstalltrue | falseIf the parameter is set (true) then the apk files of the application and test-application are uninstalled after the test has completed execution.

These parameters may be configured:

  • In the configuration file, in the following format:

    "installationDetails" : {"preCleanUp" : true}
    "postExecution" : {"uninstall" : false}
  • In the perfectoGradleSettings clause of the module's build.gradle file:

    perfectoGradleSettings { 
    	...
        postExecution {
    		uninstall false
    	}
    	installationDetails {
    		preCleanUp true
    	}
     } 

Test Execution Parameters

The following parameters affect particular test executions:

ParameterPossible ValuesMeaning
takeScreenshotOnTestEndtrue | falseSave screenshot of device at end of the test executions, regardless of the test result status.
takeScreenshotOnTestFailuretrue | falseSave screenshot of device at end of the test executions, if the test result status is a failure status.
shardtrue | false

Indicates that test runs should use the Sharding tests capability of the Android JUnitRunner. The number of shards to create is equal to the number of devices selected by the configuration.

Default: false

These parameters may be set in the configuration file, build.gradle file, or the command-line.

Reporting Parameters

The following parameters are used by DigitalZoom Reporting to classify the execution report and make it easy to identify the reports for the execution.

ParameterPossible ValuesMeaning
tags Set of tags to associate with the execution
jobName 

CI identification of the build, used for classification of the report in the CI Dashboard.

jobNumber  CI Job Number of the build
projectName Name of the project - for classification
projectVersion Version number assigned to the project for this build
branch Branch name. like additional tag.

These parameters may be set:

  • In the configuration file, in the following format:

    "tags" : ["plugin", "espresso", "demo"]
    "projectName": "playground"
    "projectVersion": "1.5"
    "jobName": "newFeature"
    "jobNumber": "45"
  • In the perfectoGradleSettings clause of the build.gradle file:

    perfectoGradleSettings { 
    	...
        projectName "playground"
    	projectVersion "1.5"
    } 
  • In the command line, in the following format:

    gradle perfecto-android-inst ... -Ptags="plugin;espresso;demo" -PjobName="newFeature"

Configuration File Location

The plugin execution reads the configuration file whose location is indicated by the configFileLocation parameter that may be supplied:

  • In the perfectoGradleSettings clause of the build.gradle file.

    perfectoGradleSettings { 
    	...
        configFileLocation "C:\temp\Espresso\ConfigFile.json" 
    } 
  • Or in the command line

    gradle perfecto-android-inst ... -PconfigFileLocation="C:\temp\Espresso\ConfigFile.json"

If the parameter is not supplied -

  • If running in Android Studio project - default location is <project dir>/src/main/assets/perfecto/instrumentedTest/configFile.json
  • If running in Standalone mode or the file is not found at the default location, then tests will run on a randomly selected available Android device.
  • The application and test-application identification parameters (apkPath and testApkPath) must be supplied in either the build.gradle file or on the command line.