Page tree
Skip to end of metadata
Go to start of metadata

Last updated: Nov 01, 2018 11:38

By Nate Stuyvesant

Optimize performance and prevent disconnections in your network. Understand the protocols used, effects of proxy servers and VPNs.

The purpose of this article is to summarize Perfecto’s cloud network requirements and best practices. Perfecto clouds use video streaming to give users access to devices from anywhere in the world (similar to Remote Desktop). Video streaming is more sensitive to latency and bandwidth limitations than other types of Internet use such as web browsing. When you interact with a device in your cloud, a short delay of 1 to 1.5 seconds is expected. To prevent greater latency, planning is required.

Requirements

TCP Ports: Your firewall and other security appliances must allow HTTPS access (TCP port 443) to the cloud and RTMP (Realtime Multimedia Protocol, TCP port 1935) to the streamer. If TCP port 1935 to the streamer is not permitted in your environment, your IT group will need to enable TCP port 443 (RTMP tunneled in HTTPS) or 80 (RTMP tunneled in HTTP) access to the Streaming Server (STS). This will result in encapsulation and encryption overhead giving you slower performance (see article). The static IP addresses for the STS and cloud are provided during onboarding (but can be requested at any time by opening a Support case). If video streaming goes through intermediate proxies, deep packet or SSL inspection will add further latency.

Whitelisting: Security appliances must whitelist your cloud's website (MCM) and STS IP addresses.

Note: To access the Perfecto Continuous Quality Lab UI - whitelist the URL: <CQL Name>.app.perfectomobile.com - the gateway to all Perfecto UI applications.

        where: <CQL Name> is the "private" name of your installation, for example - the name for demo lab (demo.app.perfectomobile.com) is demo.

Proxy Servers: Many corporate environments use proxy servers in order to conserve bandwidth and hide the IP address of client computers. Proxy server settings in the browser are ignored by RTMP which blocks streaming. Excluding traffic between end-users and the STS will solve this problem. Additionally, Proxy servers must have connection timeouts of no less than 180 seconds. Long-running commands in HP UFT have been known to be interrupted with a "Bad response from cloud" because of low connection timeouts set in WebSense or proxy servers. Additionally, explicit proxy (common with Blue Coat ProxySG) is not compatible with RTMP.

Bandwidth: Should exceed the [number of devices] * [256 kilobits per second] + [1.5 megabits per second]. For example, 10 devices requires 10 * 256 Kbps + 1.5 Mbps = 4 Mbps of bandwidth. While your site may have enough bandwidth, if many users share the same Internet connection as those who need cloud access, this can greatly affect the latency of the video stream. Bandwidth spikes from activities such as large downloads may cause intermittent problems. Many companies throttle bandwidth especially for RTMP video traffic to discourage employees from impacting available bandwidth as well as viewing sites with video such as YouTube, news, and other content that may not be work-related. Your IT department may not know that you need video streaming to use Perfecto clouds. This can cause your cloud to be treated as unwanted video streaming.

Several strategies are available to ensure that your teams have adequate bandwidth:

  • Isolate project teams in each location to their own VLANs. This allows bandwidth to be more easily controlled and measured.
  • Have your IT team assign a higher Quality of Service (QoS) priority to traffic from the cloud's streamer to your project team's VLANs. For IEEE P802.1Q, priority 5 is recommended. For IP Differentiated Services Code Point (DSCP), please use Expedited Forwarding (EF). Perfecto Support can provide the IP addresses of your MCM and Streaming Server for you to share with your IT organization.
  • Ensure that your IT department is not throttling bandwidth for the project team below the required level based on the number of devices you are accessing.
  • If your project teams are in several locations, measure the performance from user's workstations to the cloud as provided below. Generic speed tests should not be used as these do not measure the bandwidth of the network path from the user to the lab (often only to a local ISP).
  • Consider a separate Internet connection for the project teams that will not be impacted by other users. Some customers use LTE gateways for their testing team on an isolated network as a workaround.

What do I do if I have a performance or connectivity problem?

  1. Determine what kind of problem you have:
    1a. If you have already opened a Support Ticket, please utilize the Perfecto's Network Connectivity Test. Then download the results and send them to us via the email chain started when you opened the Support Ticket. 
    1. Interactive latency - when manually interacting with a device, each click takes too much time.
      1. If you see a delay between a click (such as on a number button in the calculator) and a response of...
        1. Less than 2 seconds, you are getting good performance. Remember, this device may be thousands of miles away. Do not expect performance to be the same as though it's sitting in your hand. Nothing in this article will help you improve interactive latency below 2 seconds.
        2. 2 to 3 seconds - Fair performance. You will notice that it is a remote device but you are able to get your work done. It may be possible to improve the performance slightly by applying best practices found in this article.
        3. 3 to 4 seconds - Less than optimal. Waiting to see responses is beginning to impact your productivity. Definitely have your IT team look at this article.
        4. Greater than 4 seconds - Unacceptable performance. Your network configuration is having a significant impact on your use of the Perfecto platform. Your IT team must provide a solution.
      2. Are you in Image Mode? If you are getting a small blue diamond-shaped icon overlaid on the top right corner of the device screen, a series of still images are being sent by your cloud because the video streamer is not accessible from your computer. When you open a device, the cloud tells your computer to open a connection to the video streamer. Usually, this is a different FQDN and IP address from your cloud. In most cases, your IT department is blocking access to this host because it has not been whitelisted yet. This results in unacceptable performance. When your cloud was setup, the person who placed the order received a list of fully qualified domain names and IP addresses for the components of your Perfecto cloud. If you do not have access to this information, please open a Support case with Perfecto requesting it. Your IT department will need to make sure access to those hosts is enabled for your users at each location where they use the Perfecto cloud.
      3. If you're able, please connect to your cloud from home (without using a VPN connection to your organization) or your favorite coffee shop. If you see better interactive performance away from your organization's network, this indicates the problem will require the involvement of your IT organization. If you open a case, please note this.
      4. Depending upon where your cloud is hosted, please run a Perfecto speed test to determine whether the issue may be related to network bandwidth or latency:
        1. Testing your connection to some of our data centers
          1. Use the Perfecto Network Connectivity Test

        2. The results of your test will look like this...



          1. The example shown is from a user at home with a gigabit connection. While not everyone is fortunate to have such fast speeds, if the blue line is more than 1.5 megabits per second, you have enough bandwidth to use Perfecto. If your download speed is less than 1.5 megabits per second, the delays you are experiencing are due to poor bandwidth. Please open a case with Perfecto Support and ask for the bandwidth requirements of your cloud to be lowered by increasing compression. This will improve performance though visual artifacts from compression may be more noticeable. If your manager can get your bandwidth increased by your IT department, this will yield the best results.
          2. Your base latency and should always be below 100ms. If the line goes above 100ms, your performance problem is due to latency. This can be caused by slow wide area network connections, overhead from VPN tunnel encryption, overloaded proxy servers, deep packet inspection, and more. Perfecto will need to work with your IT department to help them isolate and remove the source of the latency. Please open a case with Perfecto Support.
      5. Here are some steps that may improve performance:
        1. Open a device, click More menu (top right) -> Settings -> Streaming -> Connections (you should get a dialog like this)
        2. Change the connection type to see if there is an effect on performance:
          1. In the row associated with your device, change the Connection dropdown to Safe, Encrypted or Direct (instead of Default). Your network configuration affects which option will be best for you. Normally, Direct (rtmp://) is the fastest option if it is not blocked, but there are scenarios where a different option will be better.
          2. Click the reconnect button. You should see the beginning of the URL in the Actual URI column change (rtmp:// for Direct, rtmps:// for Encrypted and rtmpt:// for Safe).
          3. Click Close and then test the device interactively by clicking number buttons in the Calculator or Dialer. If there's no improvement, repeat steps 1 and 2 with a different Connection selection. 
        3. Click More menu (top right) -> Settings -> Streaming -> Settings and select the Connection type that gave you best performance as the Default then click the Close button.
    2. Open device latency - you select a device and click Open and it takes a long time before you can see its screen and interact with it. High-resolution devices take longer to open. If the device is in another country, this will add time.
      1. Typical times it takes to open devices:
        1. Apple's iPad Pro typically takes 8 seconds to open even on the fastest network connection.
        2. An Apple iPhone 6S generally opens in less than 6 seconds on fast network connections.
      2. If the problem you are experiencing is the amount of time it takes for the device to appear, the system is trying to open the video stream according to the default (which is usually set to Direct); however, it will try to use other modes if the default is not available. This can take up to 40 seconds. To solve, try selecting one of the other modes (as described in steps under 1.a.v in the Interactive latency section above) to determine your optimal selection and make it your default.

    3. Slow test execution - the duration of a test execution is longer than you expect. Many factors affect this kind of performance including:
      1. The performance of the app under test. Test environments are notoriously scaled down versions of production. The hardware used is usually less powerful and test versions of the app may have extra overhead to enable debugging. In some cases, Perfecto-hosted devices access the test environment though a VPN tunnel from our data center. VPN tunnels add latency. Additionally, the network configuration for access on your side of the tunnel may need to be optimized.
      2. How you wrote your test. Tests can often be optimized by eliminating unnecessary steps like reinstallation of the app under test. Hard-coded waits to synchronize the script with app responses should be avoided. Visual analysis should be used for verifications in combination with object-based navigation. Perfecto University (https://perfecto.litmos.com) has online courses that can help you.
  2. Finally, if you and your IT team cannot resolve the issue on your own, please open a case with Perfecto Support. Provide us with as much information as you can especially the type(s) of problem(s) you are encountering. Are you experiencing delays when manually interacting with hosted devices (touch and swipe)? Is your problem primarily with the speed at which your automated scripts run? Are you experiencing problems with particular types of devices? Is everyone in your organization affected or only users at a certain location? Does performance vary according to the time of day? Performance issues have varying types of root causes and the more information you provide, the faster we'll be able to identify and resolve your issue.

Depending upon the type of issue you are experiencing, Support may need your help to coordinate a meeting between Perfecto IT and your organization's firewall, proxy and network infrastructure teams to help them meet the networking requirements outlined above.

Network protocols we use

When users interact with devices or hosted desktop browsers connected to a cloud with a web browser, development or CI tool, several protocols are used:

  • HTTPS (TCP port 443 - encrypted HTTP) - Used for your login and interaction with the web UI of the MobileCloud Manager website (MCM) as well as REST calls from external tools and technologies like Appium, Selenium, HP Unified Functional Testing (UFT), and Jenkins.
  • RTMP (TCP port 1935), RTMP/S (RTMP encapsulated in HTTPS) or RTMP/T(encapsulated in HTTP) - Used for your interactive video session with mobile devices. The Realtime Messaging Protocol was designed by Adobe for Flash streaming video and audio. When TCP port 1935 is blocked by your organization's firewall, video falls back from RTMP -> RTMP/S -> RTMP/T. Encapsulation of RTMP video streams in HTTPS or HTTP adds latency. See http://blogs.adobe.com/connectsupport/tunneling-with-rtmp-encapsulated-in-http-rtmpt-should-be-avoided-as-it-causes-latency for more information. Device video in clouds is served by a separate computer called a Streaming Server (STS).
  • HTTP (TCP port 80) - This protocol is rarely used other than to redirect website requests from http:// to https:// and when RTMP falls back to RTMP/T.

Conclusion

Following our networking best practices for cloud will ensure the best possible experience for your users. Should you or your IT organization have any questions, please contact your Value Realization Consultant to schedule a call with our network experts.