TestNG | Parallel execution

After adding the TestNG framework and passing parameters from the testng.xml file, we can now leverage our framework to run multiple tests, on different devices in parallel.

Important: This document includes references to a third-party product, TestNG. The user interface and usage of third-party products are subject to change without notice. For the latest published information about TestNG, see https://testng.org/doc/documentation-main.html.

Parallel execution

First, we need to be aware of some parallel execution rules:

  • Only 1 test can run on 1 device at a time, which means we need to create a different driver for every thread.
  • If we have dependencies between our tests, we must respect it and run those tests in sequence, not in parallel.
  • With parallel execution, we can run the same test on different device models to achieve more coverage or run different tests on the same device models to cut the execution time and get faster results for specific model.

Running the same test on different device models

Going back to the testng.xml file from our previous post:

Copy
<suite name="Suite" parallel="false">
    <test name="Test"> 
        <parameter name="mcm" value="demo.perfectomobile.com" />
        <parameter name="mcmUser" value="avnerg@perfectomobile.com" />
        <parameter name="mcmPassword" value="xxxxxx" />
        <parameter name="deviceModel" value="iPhone-6" />
        <classes>
            <parameter name="testUsername" value="John" />
            <parameter name="testPassword" value="Perfecto1" />
            <class name="TestNG.PerfectoAppTest"/>
        </classes>
    </test>
</suite>
  1. Because the mobileCloud url, username and password are the same for all devices, we are going to move them to the suite level.
  2. The deviceModel parameter will remain in its current location.
  3. The testUsername and testPassword parameter will also move to the suite level, since they are the same for both tests.
  4. We will add 2 more tests, each will get a different device as a parameter.
  5. Last thing - in the suite tag, change the parallel param from "false" to "tests".
Copy
<suite name="Suite" parallel="tests">
    <parameter name="mcm" value="demo.perfectomobile.com" />
    <parameter name="mcmUser" value="avnerg@perfectomobile.com" />
    <parameter name="mcmPassword" value="xxxxxx" />
    <parameter name="testUsername" value="John" />
    <parameter name="testPassword" value="Perfecto1" />
    <test name="Test iPhone-6">
        <parameter name="deviceModel" value="iPhone-6" />
        <classes>
            <class name="TestNG.PerfectoAppTest"/>
        </classes>
    </test>
    <test name="Test Galaxy 5">
        <parameter name="deviceModel" value="Galaxy S5 SM-G900A" />
        <classes>
            <class name="TestNG.PerfectoAppTest"/>
        </classes>
    </test>
    <test name="Test iPad Air">
        <parameter name="deviceModel" value="iPad Air" />
        <classes>
            <class name="TestNG.PerfectoAppTest"/>
        </classes>
    </test>
</suite>

Because TestNG starts a different thread for each test, the @BeforeClass method will create a dedicated RemoteWebDriver for each test, which will be dedicated to only 1 device. The @AfterClass method then runs the driver.quit() function, which will close each of the drivers separately after it has finished the tests.

Our example shows that running the same test on 3 different devices helps cut the run time from 174 seconds to only 64 seconds due to parallel execution.