Last updated: Mar 01, 2021 10:42
Apache JMeter™ is open-source software and 100% pure Java, designed to load test functional behavior and measure performance. It was originally intended for testing web applications but has since expanded to other test functions.
Perfecto has e2e integration with JMeter to perform SUT (Single User Performance) testing, with JUnit supporting both web and mobile applications.
In this exercise, we will perform client-side performance testing while server being stressed with JMeter threads for the Etihad web application. While measuring responses on the few real devices, we will ramp up virtual users to add increase load on the system. Following is an architecture diagram.
We have selected the following transactions:
- Launch App: The time it takes to load the app
- Search Flights: The time it takes to list the available flights on the given itinerary
The following thread groups are required:
Perform the following steps to set up JMeter and a Java Perfecto environment:
In this example, we have implemented a sample Java script to perform SUT Performance Testing.
- Download the Sample Java project here - https://github.com/PerfectoCode/Community-Samples/tree/master/RemoteWebDriverJmeterSample.
- Import the project as a normal Java project in Eclipse or IntelliJ.
Export the project as a JAR file.Click to view image
- Place the generated JAR file into JMeter's lib/JUnit.
- Download the Java dependencies here - https://github.com/PerfectoCode/Community-Samples/tree/master/JMeterSamples/Dependencies/libs
- Place the downloaded dependencies into JMeter’s lib folder
- Download the latest JMeter.
- Extract the downloaded JMeter file.
- Download the Plugins Manager JAR file and put it into JMeter's lib/ext.
In the JMeter, from the Options menu, select Plugins Manager.Click to view image
Under Available Plugins, search for Dummy Sampler. Then click Apply Changes and Restart JMeter.Click to view image
- Download the Etihad Perfecto User Experience Jmx file - EtihadPerformance.jmx and open it in JMeter.
Get the Perfecto security token and, under User Defined Variables, assign the value to the securityToken, host and reportingUrl parameters.Click to view image
For each JUnit request, select the ClassName and respective TestMethod, for example:
Get Driver > setUpBeforeClass
Get App Launch Time > launchApp
Get Search Flight Time > searchFlights
Close Driver > oneTimeTearDownClick to view image
Under User Parameters, enter the device and platform details as a separate user.Click to view image
On the toolbar, click the (green) Run button to execute the tests on the given device.Click to view image
JMeter gets the Perfecto user experience time (UXT) for the given transactions from the device.Click to view image
In this case, Android took on average of 2s to launch the app and 7.9s to search the flights. iOS took on average of 5.5 seconds to launch the app and 7.9s seconds to search the fights.
To load the server, we will use JMeter threads. Virtual users need network traffic or APIs to run the script. Perform the following steps to capture the traffic.
Method 1 | Use Perfecto Network Traffic Capture
- As part of the client-side tests SUT Performance Testing, we already captured the network traffic with the HAR file. Use this file and convert it into JMX using the following tool: https://converter.blazemeter.com
Load the converted JMX file.
Add the same transaction names as given in client-side performance testing (Launch App and Search Flight).Click to view image
Method 2 | Capture traffic directly from the web or mobile device
- Perform the steps provided at the following link to record the traffic: https://jmeter.apache.org/usermanual/jmeter_proxy_step_by_step.html
- Add the same transaction names as given in client-side performance testing (Launch App and Search Flight).
Merge both Perfecto user experience and server-side scripts in the test plan. A sample e2e JMX test plan file is available here: EtihadPerformance.jmx
Because Etihad is the production website, let’s run Performance testing with 7 users: 2 on the client-side with a real device (in Perfecto) and 5 on the server-side.
To execute the performance tests:
On the toolbar, click the (green) Run button. The test will load the server and execute client-side tests in Perfecto. The summary report contains the transaction time of both server- and client-side.
From the summary, we see that as per JMeter threads (server-side), the system took an average of 28s to load the launch page and 7.8s to search the flights.
From the Perfecto user experience side (client-side), it took 2s on Android and 6s on iOS devices. More time was reported on the server-side because the server page load time depends on the processing time of all the APIs (including third-party APIs that are not visible on the screen, whereas the Perfecto page load time depends on specific text to display on the page, ignoring background and third-party APIs.
The following video shows a demo execution.