Page tree
Skip to end of metadata
Go to start of metadata

UX Perfecto Mobile timer via Remote Web Driver and measuring a sequence of commands


by Amir Rozenberg

Measuring your application quality from an end user experience is a key part of ensuring success in production. In adopting the end user perspective, you want to be able to define key steps in the test case, and measure the desired outcome in terms of availability and response time. According to Gartner, Amazon loses $6.79M for every second of delay in end user experience. Hence, obtaining accurate user experience timers is a key factor in your performance testing.

In the original article about Measuring User Experience we've discussed the basics of setting the checkpoint parameters to get the most accurate user experience timer for each event in your script.

In this article let's discuss a few variations on this topic: getting an accurate timer from Selenium as well as getting accurate timers for a sequence of commands.

Getting an accurate user experience timing from Selenium via Remote Web Driver

Measuring user experience in Perfecto cloud is the most accurate way to conduct the measurement because it takes place on the HSS, which is closest to the device. Many external IDEs (such as Eclipse) have their own timer management. The challenge is that the timer measurement may be inaccurate considering the measurement is taking place remotely from the mobile cloud. i.e., network time is added to the timer value.

The best way to measure timer accuracy from Selenium is to A- follow the checkpoint best practices described here and B- get the native UX timer for the last command (typically a checkpoint). Here are a few code samples to accomplish that:

In the following example we are triggering a transaction, adding a checkpoint to measure how long the transaction took, and then collecting the UX timer natively for that checkpoint:

/ Drive an action
driver.FindElement(By.XPath("/html[1]/body[1]/div[6]/section[1]/form[1]/label[1]/input[1]")).SendKeys("Formula 1");
// Find an element, equivalent of a checkpoint
IWebElement el2 = driver.FindElement(By.XPath("/html[1]/body[1]/div[6]/section[1]/div[1]/ul[1]/li[1]/em[1]"));
// Get the UX timer natively for the last checkpoint
long uxTimer = PerfectoFunctionLibrary.timerGet(driver, "UX", "xxxxx");

Here is the C# implementation of the timer collection function:

public static long timerGet(RemoteWebDriver driver, string timerType, string timerId)
 {
 string units = "milliseconds";
 string command = "mobile:timer:info";
 Dictionary<string, object> Parms = new Dictionary<string, object>();
 Parms.Add("timerId", timerId);
 Parms.Add("type", timerType);
 Parms.Add("units", units);
 long result = (long)driver.ExecuteScript(command, Parms);
 return result;
}

Java:

public static long timerGet(RemoteWebDriver driver, String timerType, String timerId)
 {
 String units = "milliseconds";
 String command = "mobile:timer:info";
 Map Parms = new HashMap();
 Parms.put("timerId", timerId);
 Parms.put("type", timerType);
 Parms.put("units", units);
 long result = (long)driver.executeScript(command, Parms);
 return result;
}

Getting an accurate user experience timing for a sequence of events

Sometimes you need to understand how long it took from the minute you launched the application, went through a login process and saw the account page. There is a UX timer available for that purpose:

  • In the Native environment, you can group commands. The UX timer for the group will be the SUM of the UX timers of the commands inside the group

  • In Selenium, the best practice is to sum all of the UX timers collected from the native environment into the actual timer you want to measure. Each individual command in the sequence will potentially have its own value so you need to retrieve the UX timer after each command executed:
// Drive step 1
driver.FindElement(By.XPath("/html[1]/body[1]/div[6]/section[1]/form[1]/label[1]/input[1]")).SendKeys("Formula 1");
IWebElement el2 = driver.FindElement(By.XPath("/html[1]/body[1]/div[6]/section[1]/div[1]/ul[1]/li[1]/em[1]"));
// Measure UX timer 1
long uxTimer1 = PerfectoFunctionLibrary.timerGet(driver, "UX", "xxxxx");
// Drive step 2
el2.Click();
// Measure UX timer 2
long uxTimer2 = PerfectoFunctionLibrary.timerGet(driver, "UX", "xxxxx");
// Drive step 3
driver.FindElement(By.XPath("/html[1]/body[1]/div[6]/section[1]/div[1]/ul[1]/li[1]/em[1]"));
// Measure UX timer 3
long uxTimer3 = PerfectoFunctionLibrary.timerGet(driver, "UX", "xxxxx");
Console.WriteLine("Timer: uxTimer1 " + uxTimer1);
Console.WriteLine("Timer: uxTimer2 " + uxTimer2);
Console.WriteLine("Timer: uxTimer3 " + uxTimer3);
// Combine the timers
long NavUX3 = uxTimer1 + uxTimer2 + uxTimer3;
Console.WriteLine("Timer: Combined UX timer " + NavUX3);

Another scripting feature to be aware of is ‘composite commands’. For example in C# (and Java as well as other languages) it is valid to write one line of code that finds an element by its identifier and then calls the ‘click()’ method as follows:

driver.FindElement(By.XPath(".//*[@id='orb-nav-links']/ul/li[2]/a")).Click();

When executed this results in 2 separate native command calls EACH with its own timer values. The first command is the equivalent of :

IWebElement el1 = driver.FindElement(By.XPath(".//*[@id='orb-nav-links']/ul/li[2]/a"));

And the second command:

el1.Click();

The implication of this is that you should not use composite commands in Performance scripts within the sequence of commands that you are measuring in a particular performance scenario or to verify that one or other of the individual commands always returns a zero timer.