Page tree
Skip to end of metadata
Go to start of metadata

Last updated: Sep 22, 2019 17:31

Use iOS standard UIAutomation to identify UI Objects with PerfectoMobile devices

Appium provides support for UIAutomation selector syntax to identify UI Objects of iOS applications. This provides an alternative to XPath identification.

The syntax used is defined by Apple and fully documented at UIAutomationRef in the Apple developer site. In general, the syntax describes the full tree path to the specific UI Object, each level in the tree is separated by a dot ('.'). For example, the following path indicates the first button that appears on the display in the main interaction section of the display window:

UIATarget.localTarget().frontMostApp().mainWindow().buttons()[0]

On this page:

The first three levels of the description indicate that the selector should search for the object in the display of the current (front most) application. The mainWindow() level indicates that the object is on the main interaction frame (as opposed to, for example, the navigationBar, keyboard, or statusBar). The buttons() level indicates the array of all buttons on the main interaction frame, and the [0] selects the first button in the array.

Aside from indexing the objects within a collection of similar elements (buttonstextFieldsimages, etc.) by number index, UIAutomation supports indexing by the label. For example, the following would select the button with the label "Exit".

UIATarget.localTarget().frontMostApp().mainWindow().buttons()["Exit"]

UIAutomation additionally supports a Predicate syntax to select objects based on the values of the element attributes. For example, the following selector uses withPredicate to select a button object based on its name attribute value:

UIATarget.localTarget().frontMostApp().mainWindow().buttons.withPredicate("name == 'Add'")

Additional examples of Predicate use can be found here.

Using UIAutomation in Appium Script

The Appium iosDriver class supports findElementByIosUIAutomation and findElementsByIosUIAutomation methods that return a single WebElement or a List of WebElements, respectively. The use of these methods is similar to the findElementByXpath or findElementByLinkText methods but accept an UIAutomation selector string as its parameter.

Special Notes

UIAutomation selector strings require the complete tree path for a collection of elements.

  •  This means that there is no UIAutomation selector for an Xpath specification that does not specify a depth in the object tree. For example, the Xpath: //*[@label="Exit"]
  •  Appium supports a special feature that supports UIAutomation selectors that do not include the UIATarget.localTarget().frontMostApp().mainWindow() prefix. The string ".tableViews().staticTexts()" is interpreted with the prefix.
  •  Skipping levels in the tree path is not supported. This means that if a button is defined within a View collection on the mainWindow, it will not be included in the collection:

    UIATarget.localTarget().frontMostApp().mainWindow().buttons()

    But would need a specification:

    UIATarget.localTarget().frontMostApp().mainWindow().tableViews().buttons()

Supported Tags

When using Perfecto for Appium, the UIAutomation identifies the object, operations on the object are performed with Appium calls.. 

The selector tags and their associated UI Object Class supported by the Perfecto Appium implementation:

UIAWindow level Selector tag

UI Object Class
activityIndicatorsUIAActivityIndicator
activityViewUIAActivityView
ancestryUIAElement
buttonsUIAButton
collectionViewsUIACollectionView
elementsUIAElement, UIAElementArray 
imagesUIAImage
linksUIALink
navigationBarUIANavigationBar
navigationBarsUIANavigationBar
parentUIAElement
progressIndicatorsUIAProgressIndicator
pickersUIAPicker
popoverUIAPopover
scrollViewsUIAScrollView
searchBarsUIASearchBar
secureTextFieldsUIASecureTextField
segmentedControlsUIASegmentedControl
slidersUIASlider
staticTextsUIAStaticText
switchesUIASwitch
tabBarUIATabBar
tabBarsUIATabBar
tableViewsUIATableView
textFieldsUIATextField
textViewsUIATextView
toolbarUIAToolbar
toolbarsUIAToolbar
webViewsUIAWebView

UIAApplication level
Selector tag

UI Object Class
actionSheetUIAActionSheet
alertUIAAlert
editingMenuUIAEditingMenu
keyboardUIAKeyboard
mainWindowUIAWindow
statusBarUIAStatusBar
windowsUIAWindow
Special Selector tagUI Object Class
cellsUIATableCell, for UIATableView, UIACollectionView 
keysUIAKey, for UIAKeyboard 
segmentedControlsUIASegmentedControl, for UIATextView 
  • The UIAApplication level selector tags may appear as the child of the frontMostApp() tag.
  • The UIAWindow level selector tags appear as descendants of the UIAApplication level tags.