Legacy | Screen text

  • Command: screen

  • Subcommand: text

  • Supported Platforms: Native (legacy), UFT (legacy), Selenium, Appium

    All commands must be executed using the same framework. Sharing the execution ID between different frameworks is not possible. This means that if you work with a Selenium/Appium driver, you need to call the commands using the Selenium/Appium driver (or, in the case of Reporting commands, through the Reporting SDK that works with the driver) and not as part of a UFT test or another framework.

  • Supported OS: Android, iOS

Description

Returns the text that appears on the screen of the device (but does not select the text). The retrieved screen text can be stored in a defined String variable or to a location in the Perfecto Repository (see key parameter) and then used with the string compare and string attribute checkpoints.

Parameters

Name Value Default Description
deviceId*     The device for this command.
context

all (all)

body (body)

lowerPanel ()

upperPanel ()

all

Defines the screen region where to retrieve the text.

variable

    

The variable name to receive the screen text.

The text will always be available in the report.

source

primary (Primary)

native (Native)

camera (Camera)

auto (Auto)

secondary ()

  

The source for retrieving the screen content.

Primary - Screenshot taken from the device (higher resolution but slower)

Native - Native device controls

Camera - Screenshot taken from the video stream (lower resolution but faster)

Auto - Native device controls, if available. Else, digital screen image, if available. Else, analog screen image.

report

all (All)

all-on-error (All on error)

screenshot (Screenshot)

screenshot-on-error (Screenshot on error)

text (Text)

none (None)

All

The value to display in the report.

Screenshot - display screenshot

Screenshot on error - display screenshots only for actions in error

Text - display screen text

All - display screenshot and text

All on error - display text always and screenshot only for actions in error

None - display nothing

It is recommended to reduce the resolution in very long scripts that result in large report files.

profile

default (Default)

DocumentConversion_Accuracy (Document Conversion Accuracy)

DocumentConversion_Speed (Document Conversion Speed)

DocumentArchiving_Accuracy (Document Archiving Accuracy)

DocumentArchiving_Speed (Document Archiving Speed)

BookArchiving_Accuracy (Book Archiving Accuracy)

BookArchiving_Speed (Book Archiving Speed)

TextExtraction_Accuracy (Text Extraction Accuracy)

TextExtraction_Speed (Text Extraction Speed)

FieldLevelRecognition (Field Level Recognition)

BarcodeRecognition (Barcode Recognition)

  

Use when text cannot be found on the device screen.

Try all the profiles with the word accuracy. Per profile, try changing the DPI (96 or 300 or 120).

ocr

    

A generic OCR parameter.

Use this to provide additional OCR specifications that are not exposed as function parameters.

Example - "isolate=true" It is possible to provide multiple specifications.

This parameter is not relevant when using native screen source.

analysis

full (Full)

automatic (Automatic)

manual (Manual)

  

The image analysis mode.

The device screen may contain text and images.

Automatic - OCR distinguishes between text and images, filtering out images.

Manual - OCR assumes text only. Use alongside the Inverted text parameter.

Full - OCR tries both automatic and manual.

This parameter is not relevant when using native screen source.

inverse

any (Any)

yes (Yes)

no (No)

  

The text appearance, likely to appear in inverted colors (dark text on light background). Use only if 'Analysis mode' is set to 'Manual'.

Yes - Text is inverted

No - Text is normal, not inverted

Any - Text might be inverted or not, checked automatically by the system

This parameter is not relevant when using native screen source.

relation.direction

left (Left)

right (Right)

above (Above)

below (Below)

left-above (Left-Above)

left-below (Left-Below)

right-above (Right-Above)

right-below (Right-Below)

inside (Inside)

  

The direction of the related region, stored in memory from the last found needle, in relation to the searched needle.

For more information, see the Visual Relations series.

Left - The related region is to the left of the searched needle

Right - The related region is to the right of the searched needle

Above - The related region is above the searched needle

Below - The related region is below the searched needle

Left-Above - The related region is to the left and above the searched needle

Left-Below - The related region is to the left and below the searched needle

Right-Above - The related region is to the right and above the searched needle

Right-Below - The related region is to the right and below to the searched needle

Inside - The searched needle is inside the related region boundaries of the related region

relation.inline

horizontal (Horizontal)

vertical (Vertical)

  

The search area is in-line with the related region, stored in memory from the last found needle.

For more information, see the Visual Relations series.

Horizontal - The needle is within the height (top to button) boundaries of the related region

Vertical - The needle is within the width (left to right) boundaries of the related region

lines

    

The number of text lines to be retrieved from the screen.

This parameter is only applicable with visual relations.

report.resolution

high (High)

medium (Medium)

low (Low)

High

The image resolution.

It is recommended to reduce the resolution in very long scripts that result in large report files.

key

    

The full repository path, including directory and file name, where to save the screen text. Example - PRIVATE:dir1/screenText.txt. The text will always be available in the report.

language

English (English)

Spanish (Spanish)

Japanese (Japanese)

German (German)

French (French)

Dutch (Dutch)

Italian (Italian)

ChinesePRC (Chinese)

Russian (Russian)

PortugueseStandard (Portuguese)

  

The OCR language to use.It is possible to select multiple languages.

For languages that do not appear in the drop down menu, contact Perfecto Support to enable them.

This parameter is not relevant when using native screen source.

policy

auto (Auto)

accuracy (Accuracy)

performance (Performance)

Auto

The text recognition accuracy policy.

Selecting the Accuracy value will increase the operation time.

Auto - shifted toward recognition accuracy with a moderate impact on the recognition speed.

Accuracy - shifted toward maximum recognition accuracy.

Performance - shifted toward recognition speed.

This parameter is not relevant when using native screen source.

screen.top

    

The top value of the screen area where to search for the needle.

Values can be in pixels or in percentage of screen width (0-100). For percentage, use the % sign. Example: 20%

Default is to search the entire screen.

It is recommended to use the percentage value as it does not rely on the screen resolution.

screen.height

    

The height value of the screen area where to search for the needle.

Values can be in pixels or in percentage of screen width (0-100). For percentage, use the % sign. Example: 20%

Default is to search the entire screen.

It is recommended to use the percentage value as it does not rely on the screen resolution.

screen.left

    

The left value of the screen area where to search for the needle.

Values can be in pixels or in percentage of screen width (0-100). For percentage, use the % sign. Example: 20%

Default is to search the entire screen.

It is recommended to use the percentage value as it does not rely on the screen resolution.

screen.width

    

The width value of the screen area where to search for the needle.

Values can be in pixels or in percentage of screen width (0-100). For percentage, use the % sign. Example: 20%

Default is to search the entire screen.

It is recommended to use the percentage value as it does not rely on the screen resolution.

usecache

use (Use)

nouse (No Use)

Use

Indicates whether the text recognition application should use a cached picture of the screen if available (this value should not be changed by the user, under normal circumstances).

grid

    

The column name to be used for grid view in the report.

Selecting this parameter creates the grid report.

* Mandatory parameter

Request & Response

Copy

Request

https://mycloud.perfectomobile.com/services/executions/john@perfectomobile.com_controller_16-11-07_19_58_34_522?operation=command&securityToken=<your_token>&command=screen&subcommand=text&param.deviceId=6370E775 
Copy

JSON response

 {
  "reason": "Success",
  "timer.elapsed": "2094",
  "returnValue": "m <$> A 9 s 9 El t ,,ill 100% 1 3:07 PM\nTHE\nGRIDZ!\nThe Grid dn iaii\non nnw] .tun nnnx on l''n\n(: nnnzi nix ah!? npi ivpyn\nRate now Later No, thanks\nZ\nto EQ oo DO may",
  "testGridReportUrl": "https://mycloud.reporting-stg.perfectomobile.com?externalId[0]=john@perfectomobile.com_controller_16-11-07_19_58_34_522",
  "description": "Success",
  "timer.system": "2094",
  "completed": "true",
  "flowEndCode": "SUCCEEDED",
  "executionId": "john@perfectomobile.com_controller_16-11-07_19_58_34_522",
  "timer.ux": "0",
  "singleTestReportUrl": "https://mycloud.perfectomobile.com/nexperience/singletest/report/?reportRepositoryKey=PRIVATE:controller/161107/controller_16-11-07_19_58_34_522.xml&ownerId=john@perfectomobile.com&sharingCode=3eff0387-e7b0-4c0a-af43-419f95dc36ec",
  "reportKey": "PRIVATE:controller/161107/controller_16-11-07_19_58_34_522.xml",
  "screenAnalysis.grade": "0",
  "timer.device": "0"}