Last updated: Aug 02, 2017 11:16
UX Perfecto Mobile timer via Remote Web Driver and measuring a sequence of commands
by Amir Rozenberg
Measuring your application quality from an end user experience is a key part of ensuring success in production. In adopting the end user perspective, you want to be able to define key steps in the test case, and measure the desired outcome in terms of availability and response time. According to Gartner, Amazon loses $6.79M for every second of delay in end user experience. Hence, obtaining accurate user experience timers is a key factor in your performance testing.
In the original article about Measuring User Experience we've discussed the basics of setting the checkpoint parameters to get the most accurate user experience timer for each event in your script.
In this article let's discuss a few variations on this topic: getting an accurate timer from Selenium as well as getting accurate timers for a sequence of commands.
Getting an accurate user experience timing from Selenium via Remote Web Driver
Measuring user experience in Perfecto cloud is the most accurate way to conduct the measurement because it takes place on the HSS, which is closest to the device. Many external IDEs (such as Eclipse) have their own timer management. The challenge is that the timer measurement may be inaccurate considering the measurement is taking place remotely from the mobile cloud. i.e., network time is added to the timer value.
The best way to measure timer accuracy from Selenium is to A- follow the checkpoint best practices described here and B- get the native UX timer for the last command (typically a checkpoint). Here are a few code samples to accomplish that:
In the following example we are triggering a transaction, adding a checkpoint to measure how long the transaction took, and then collecting the UX timer natively for that checkpoint:
Here is the C# implementation of the timer collection function:
Getting an accurate user experience timing for a sequence of events
Sometimes you need to understand how long it took from the minute you launched the application, went through a login process and saw the account page. There is a UX timer available for that purpose:
- In the Native environment, you can group commands. The UX timer for the group will be the SUM of the UX timers of the commands inside the group
- In Selenium, the best practice is to sum all of the UX timers collected from the native environment into the actual timer you want to measure. Each individual command in the sequence will potentially have its own value so you need to retrieve the UX timer after each command executed:
Another scripting feature to be aware of is ‘composite commands’. For example in C# (and Java as well as other languages) it is valid to write one line of code that finds an element by its identifier and then calls the ‘click()’ method as follows:
When executed this results in 2 separate native command calls EACH with its own timer values. The first command is the equivalent of :
And the second command:
The implication of this is that you should not use composite commands in Performance scripts within the sequence of commands that you are measuring in a particular performance scenario or to verify that one or other of the individual commands always returns a zero timer.