Page tree
Skip to end of metadata
Go to start of metadata

Last updated: Jan 19, 2020 11:32

This article explains how to configure the testNG XML for device redundancy when using Quantum to execute and feed monitor data to Splunk.

Anytime you schedule the execution of a "monitor," it should be done in a redundant manner. Redundancy, in this case, means you should always execute the test case against at least two different devices of the same type or two different browser instances. The reason behind redundancy is to get rid of any potential flakiness due to environmental issues and to ensure you are getting legitimate results.

 Duplicate the Test Node

The way to schedule redundant executions is to add two separate "test" nodes to the testNG XML file. The tests should have the exact same configurations with the exception of the device selection capabilities.

<test name="Android Amazon Search Test" enabled="true">
        <parameter name="perfecto.env.resources" value="src/main/resources/android" />
        <parameter name="perfecto.capabilities.deviceName" value="04157DF43A656B1A" />
        <parameter name="perfecto.capabilities.browserName" value="mobileOS"></parameter>

        <groups>
            <run>
                <include name="@amazonSearch" />
            </run>
        </groups>

        <classes>
            <class
                name="com.qmetry.qaf.automation.step.client.gherkin.GherkinScenarioFactory" />
        </classes>
 </test>
 
        <test name="iOS Amazon Search Test" enabled="true">
        <parameter name="perfecto.env.resources" value="src/main/resources/ios" />
        <parameter name="perfecto.capabilities.deviceName" value="1C3B401545D2CDBEC9D323460D914AD7319F31D9" />
        <parameter name="perfecto.capabilities.browserName" value="mobileOS"></parameter>


        <groups>
            <run>
                <include name="@amazonSearch" />
            </run>
        </groups>


        <classes>
            <class
                name="com.qmetry.qaf.automation.step.client.gherkin.GherkinScenarioFactory" />
        </classes>
 </test>

Device Selection Capabilities

In the below example XML you'll find I have hardcoded two separate device ids however you could simply add the capability to generically pick model types or platform types/versions like so

<parameter name="perfecto.capabilities.model" value="iPhone-.*" />


or
<parameter name="perfecto.capabilities.platformName" value="iOS" />
<parameter name="perfecto.capabilities.platformVersion" value="10.*|10.*.*" />

Read more on device selection capabilities here.

Parallel Execution

Next, you can execute the test in parallel so that you are capturing the results at the exact same time (or close to it), to do this we need to enable parallel executions in the XML. To enable parallel execution of the test cases we simply add thread-count="2" (in the case that we are only using two devices for redundancy if you are using more modify this number to match the number of redundant tests you have set) and add parallel="tests" to the suite node as shown at the top of the below XML.


<suite name="Appium Demo Suite" verbose="0" thread-count="2"
 parallel="tests">
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Appium Demo Suite" verbose="0" thread-count="2"
	parallel="tests">
	<listeners>
		<listener class-name="com.quantum.listerners.QuantumReportiumListener" />
		<listener class-name="com.quantum.listerners.TestListener" />
	</listeners>

	<parameter name="driver.name" value="perfectoRemoteDriver" />


	 <test name="Android Amazon Search Test" enabled="true">
		<parameter name="perfecto.env.resources" value="src/main/resources/android" />
		<parameter name="perfecto.capabilities.deviceName" value="04157DF43A656B1A" />
		<parameter name="perfecto.capabilities.browserName" value="mobileOS"></parameter>

		<groups>
			<run>
				<include name="@amazonSearch" />
			</run>
		</groups>

		<classes>
			<class
				name="com.qmetry.qaf.automation.step.client.gherkin.GherkinScenarioFactory" />
		</classes>
	</test>
	
		<test name="iOS Amazon Search Test" enabled="true">
		<parameter name="perfecto.env.resources" value="src/main/resources/ios" />
		<parameter name="perfecto.capabilities.deviceName" value="1C3B401545D2CDBEC9D323460D914AD7319F31D9" />
		<parameter name="perfecto.capabilities.browserName" value="mobileOS"></parameter>

		<groups>
			<run>
				<include name="@amazonSearch" />
			</run>
		</groups>

		<classes>
			<class
				name="com.qmetry.qaf.automation.step.client.gherkin.GherkinScenarioFactory" />
		</classes>
	</test>