Last updated: Mar 01, 2021 10:39
This article provides comprehensive information on Single User Performance Testing with Perfecto.
Client-side performance testing is also called Single User Performance Testing (SUT) because in this case, no matter what the state of the servers is, it evaluates client-side KPIs like response time and device vitals.
SUT needs to take into account all of the factors that could impact the end-user experience across all tiers, including last-mile networks and client devices: network changes and signal degradation, changes in location, usage of onboard resources and sensors (that consume CPU, memory, and so on), background applications competing for device resources, and more. The following image shows external factors influencing application performance. So, simple functional testing is not enough.
All of these variables should be considered in test scenarios, and they should be automated because the time to test is limited. Service level agreements (SLA) for acceptable user experience should be set (for example, time to log in, time to accomplish a task) and measured in all scenarios.
The following example evaluates Etihad web application performance metrics.
The following transactions are under test:
- Launch App: The time it takes to load the app
- Search Flights: The time it takes to list the available flights on the given itinerary
The Perfecto predefined function helps to simulate end-user conditions and offers the following abilities to extend coverage to real user conditions when testing user journeys or flows. The test includes the following sections:
End-user conditions that simulate network, location, background applications, and other properties
Points of interest that identify crucial application flows, such as AppLaunch and Search Flight
Timer reporting ability to track timings and performance of the application
An example might be how long it took for a page to load or a process to happen, and what that actually looks like for a user (so the UX or user experience on the device glass). In this test, we have chosen app launch time and flight search transactions.
Perfecto provides page load time as User Experience Time (UX) with the help of visual functions (OCR and image analysis). In other words, the timer will wait for a specific text or image to actually appear (be rendered) on the screen. In our example, upon launching, the Etihad web timer will wait for the text One Way to appear on the screen.
The Perfecto page load time includes:
For details, see Network traffic capture and analysis (HAR) | Performance testing.
- A detailed report about the result of the script that is easy to understand and provides the following details:
Functional Validation: The feature is functional with detailed steps and a recorded videoClick to view image
- Performance validation, which includes:
Page Load Time: Measurement of responsiveness of the application’s rendered page load time retrieved by visual analysisClick to view image
- Network Traffic: Report including a detailed (HAR) network traffic log. For details, see
Device Vitals: Recorded device vitals to provide visibility into network, CPU, and memory consumption during the execution of the test scriptClick to view image
A sample Java project is available here: https://github.com/PerfectoCode/Community-Samples/blob/master/JavaPerfectoUserExperience
The following video shows a sample execution.