You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
ORPA-pyOpenRPA/Wiki/ENG_Guide/markdown/Robot/02_Defs.md

677 lines
19 KiB

# 2. Defs
4 years ago
## Desktop app UI access (win32 and UI automation dlls)
### Definitions
* **UIO** - UI Object (class of pywinauto UI object) [pywinauto.base_wrapper]
* **UIOSelector** - List of dict (key attributes)
* **PWA** - PyWinAuto
* **PWASpecification** - List of dict (key attributes in pywinauto.find_window notation)
* **UIOTree** - Recursive Dict of Dict … (UI Parent -> Child hierarchy)
* **UIOInfo** - Dict of UIO attributes
* **UIOActivity** - Activity of the UIO (UI object) from the Pywinauto module
* **UIOEI** - UI Object info object
### What is UIO?
UIO is a User Interface Object (pyOpenRPA terminology). For maximum compatibility, this instance is inherited from the object model developed in the [pywinauto library (click to get a list of available class functions)]([https://pywinauto.readthedocs.io/en/latest/code/pywinauto.base_wrapper.html](https://pywinauto.readthedocs.io/en/latest/code/pywinauto.base_wrapper.html)).
This approach allows us to implement useful functionality that has already been successfully developed in other libraries, and Supplement it with the missing functionality. In our case, the missing functionality is the ability to dynamically access UIO objects using UIO selectors.
### UIOSelector structure & example
<a name=”UIOSelector_Structure_Examples”></a>
UIOSelector is the list of condition items for the UIO in GUI. Each item has condition attributes for detect applicable UIO. Here is the description of the available condition attributes in item.
```
**
```
Desciption\*\*<br>
```
``
```
```
`
```
[
{
“depth_start” :: [int, start from 1] :: the depth index, where to start check the condition list (default 1),
“depth_end” :: [int, start from 1] :: the depth index, where to stop check the condition list (default 1),
“ctrl_index” || “index” :: [int, starts from 0] :: the index of the UIO in parent UIO child list,
“title” :: [str] :: the condition for the UIO attribute *title*,
“title_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute *title*,
“rich_text” :: [str] :: the condition for the UIO attribute *rich_text*,
“rich_text_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute *rich_text*,
“class_name” :: [str] :: the condition for the UIO attribute *class_name*,
“class_name_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute *class_name*,
“friendly_class_name” :: [str] :: the condition for the UIO attribute *friendly_class_name*,
“friendly_class_name_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute *friendly_class_name*,
“control_type” :: [str] :: the condition for the UIO attribute *control_type*,
“control_type_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute *control_type*,
“is_enabled” :: [bool] :: the condition for the UIO attribute *is_enabled*. If UI object is enabled on GUI,
“is_visible” :: [bool] :: the condition for the UIO attribute *is_visible*. If UI object is visible on GUI,
“backend” :: [str, “win32” || “uia”] :: the method of UIO extraction (default “win32”). ATTENTION! Current option can be only for the first item of the UIO selector. For the next items this option will be implemented from the first item.
},
{ … specification next level UIO }
]
```
``
```
```
`
```
**The UIO selector example**
```
``
```
```
`
```
[
{“class_name”:”CalcFrame”, “backend”:”win32”}, # 1-st level UIO specification
{“title”:”Hex”, “depth_start”:3, “depth_end”: 3} # 3-rd level specification (because of attribute depth_start|depth_stop)
]
```
``
```
```
`
```
### The UIDesktop module (OpenRPA/Robot/UIDesktop.py)
The UIDesktop is extension of the pywinauto module which provide access to the desktop apps by the **win32** and **ui automation** dll frameworks (big thx to the Microsoft :) ).
```
# EXAMPLE 1
from pyOpenRPA.Robot import UIDesktop
UIDesktop.UIOSelector_Get_UIO(
inSpecificationList=[
{"title":"notepad.exe"},{"title":"OK"}],
inElement=None,
inFlagRaiseException=True)
```
**Functions:**
| `Get_OSBitnessInt`()
| Detect OS bitness.
|
| `PWASpecification_Get_PWAApplication`(…)
| #Backend selection - attribute “backend” (“win32” || “uia”) in 1-st list element
|
| `PWASpecification_Get_UIO`(…)
| #Backend def selection - attribute “backend” (“win32” || “uia”) in 1-st list element #old name - GetControl
|
| `UIOSelectorSecs_WaitAppear_Bool`(…)
| Wait for UI object will appear in GUI for inWaitSecs seconds.
|
| `UIOSelectorSecs_WaitDisappear_Bool`(…)
| Wait for UI object will disappear in GUI for inWaitSecs seconds.
|
4 years ago
| `UIOSelectorUIOActivity_Run_Dict`(…[, …])
| Run the activity in UIO (UI Object)
|
| `UIOSelector_Exist_Bool`(inUIOSelector)
| Check if object is exist by the UIO selector.
|
4 years ago
| `UIOSelector_FocusHighlight`(inUIOSelector)
| Set focus and highlight (draw outline) the element (in app) by the UIO selector.
|
| `UIOSelector_GetChildList_UIOList`([…])
| Get list of child UIOs by the parent UIOSelector
|
| `UIOSelector_Get_BitnessInt`(inSpecificationList)
| Detect process bitness by the UI Object UIO Selector.
|
| `UIOSelector_Get_BitnessStr`(inSpecificationList)
| Detect process bitness by the UI Object UIO Selector.
|
| `UIOSelector_Get_UIO`(inSpecificationList[, …])
| Get the pywinauto object by the UIO selector.
|
4 years ago
| `UIOSelector_Get_UIOActivityList`(inUIOSelector)
| Get the list of the UI object activities
|
| `UIOSelector_Get_UIOInfo`(inUIOSelector)
| Get the UIO dict of the attributes
|
| `UIOSelector_Get_UIOList`(inSpecificationList)
| Get the UIO list by the selector
|
4 years ago
| `UIOSelector_Highlight`(inUIOSelector)
| Highlight (draw outline) the element (in app) by the UIO selector.
|
| `UIOSelector_SafeOtherGet_Process`(inUIOSelector)
| Safe get other process or None if destination app is the other/same bitness
|
| `UIOSelector_SearchChildByMouse_UIO`(…)
| UIOSelector (see description on the top of the document) #old name - AutomationSearchMouseElement
|
| `UIOSelector_SearchChildByMouse_UIOTree`(…)
| !!!!Safe call is included (you can set activity and UIDesktop will choose the bitness and return the result)!!!!!
|
4 years ago
| `UIOSelector_TryRestore_Dict`(inSpecificationList)
| Try to restore (maximize) window, if its minimized.
|
| `UIOSelectorsSecs_WaitAppear_List`(…[, …])
| Wait for many UI object will appear in GUI for inWaitSecs seconds.
|
| `UIOSelectorsSecs_WaitDisappear_List`(…[, …])
| Wait for many UI object will disappear in GUI for inWaitSecs seconds.
|
### pyOpenRPA.Robot.UIDesktop.Get_OSBitnessInt()
Detect OS bitness.
* **Returns**
int 32 or int 64
### pyOpenRPA.Robot.UIDesktop.PWASpecification_Get_PWAApplication(inControlSpecificationArray)
#Backend selection - attribute “backend” (“win32” || “uia”) in 1-st list element
* **Parameters**
**inControlSpecificationArray** List of dict, dict in pywinauto.find_windows notation
* **Returns**
process application object
### pyOpenRPA.Robot.UIDesktop.PWASpecification_Get_UIO(inControlSpecificationArray)
#Backend def selection - attribute “backend” (“win32” || “uia”) in 1-st list element
#old name - GetControl
* **Parameters**
**inControlSpecificationArray** List of dict, dict in pywinauto.find_windows notation
* **Returns**
list of UIO object
### pyOpenRPA.Robot.UIDesktop.UIOSelectorSecs_WaitAppear_Bool(inSpecificationList, inWaitSecs)
Wait for UI object will appear in GUI for inWaitSecs seconds.
* **Parameters**
* **inSpecificationList** UIOSelector. Example: [{“title”:”notepad”},{“title”:”OK”}]
* **inWaitSecs** Float value (seconds) for wait UI element appear in GUI
* **Returns**
True - UI object will appear. False - else case
### pyOpenRPA.Robot.UIDesktop.UIOSelectorSecs_WaitDisappear_Bool(inSpecificationList, inWaitSecs)
Wait for UI object will disappear in GUI for inWaitSecs seconds.
* **Parameters**
* **inSpecificationList** UIOSelector.
Example: [{“title”:”notepad”},{“title”:”OK”}]
* **inWaitSecs** Float value (seconds) for wait UI element disappear in GUI
* **Returns**
True - UI object will disappear. False - else case
4 years ago
### pyOpenRPA.Robot.UIDesktop.UIOSelectorUIOActivity_Run_Dict(inUIOSelector, inActionName, inArgumentList=None, inkwArgumentObject=None)
Run the activity in UIO (UI Object)
* **Parameters**
* **inUIOSelector** UIOSelector - List of items, which contains condition attributes
* **inActionName** UIOActivity (name) activity name string from Pywinauto
* **inArgumentList**
* **inkwArgumentObject**
* **Returns**
### pyOpenRPA.Robot.UIDesktop.UIOSelector_Exist_Bool(inUIOSelector)
Check if object is exist by the UIO selector.
* **Parameters**
**inUIOSelector**
* **Returns**
True - Object is exist. False - else case
4 years ago
### pyOpenRPA.Robot.UIDesktop.UIOSelector_FocusHighlight(inUIOSelector)
Set focus and highlight (draw outline) the element (in app) by the UIO selector.
* **Parameters**
**inUIOSelector** UIOSelector - List of items, which contains condition attributes
* **Returns**
### pyOpenRPA.Robot.UIDesktop.UIOSelector_GetChildList_UIOList(inUIOSelector=None, inBackend='win32')
Get list of child UIOs by the parent UIOSelector
* **Parameters**
* **inUIOSelector** UIOSelector - List of items, which contains condition attributes
* **inBackend** “win32” or “uia”
* **Returns**
### pyOpenRPA.Robot.UIDesktop.UIOSelector_Get_BitnessInt(inSpecificationList)
Detect process bitness by the UI Object UIO Selector.
* **Parameters**
**inSpecificationList** UIOSelector. Example: [{“title”:”notepad”},{“title”:”OK”}]
* **Returns**
int 32 or int 64
### pyOpenRPA.Robot.UIDesktop.UIOSelector_Get_BitnessStr(inSpecificationList)
Detect process bitness by the UI Object UIO Selector.
* **Parameters**
**inSpecificationList** UIOSelector. Example: [{“title”:”notepad”},{“title”:”OK”}]
* **Returns**
str “32” or str “64”
### pyOpenRPA.Robot.UIDesktop.UIOSelector_Get_UIO(inSpecificationList, inElement=None, inFlagRaiseException=True)
Get the pywinauto object by the UIO selector.
* **Parameters**
* **inSpecificationList**
* **inElement**
* **inFlagRaiseException**
* **Returns**
4 years ago
### pyOpenRPA.Robot.UIDesktop.UIOSelector_Get_UIOActivityList(inUIOSelector)
Get the list of the UI object activities
* **Parameters**
**inUIOSelector** UIOSelector - List of items, which contains condition attributes
* **Returns**
### pyOpenRPA.Robot.UIDesktop.UIOSelector_Get_UIOInfo(inUIOSelector)
Get the UIO dict of the attributes
* **Parameters**
**inUIOSelector** UIOSelector - List of items, which contains condition attributes
* **Returns**
### pyOpenRPA.Robot.UIDesktop.UIOSelector_Get_UIOList(inSpecificationList, inElement=None, inFlagRaiseException=True)
Get the UIO list by the selector
* **Parameters**
* **inSpecificationList** UIO Selector
* **inElement** Входной элемент - показатель, что не требуется выполнять коннект к процессу
* **inFlagRaiseException** Флаг True - выкинуть ошибку в случае обнаружении пустого списка
* **Returns**
4 years ago
### pyOpenRPA.Robot.UIDesktop.UIOSelector_Highlight(inUIOSelector)
Highlight (draw outline) the element (in app) by the UIO selector.
* **Parameters**
**inUIOSelector** UIOSelector - List of items, which contains condition attributes
* **Returns**
### pyOpenRPA.Robot.UIDesktop.UIOSelector_SafeOtherGet_Process(inUIOSelector)
Safe get other process or None if destination app is the other/same bitness
* **Parameters**
**inUIOSelector** UIO Selector of the UI object
* **Returns**
None or process (of the other bitness)
### pyOpenRPA.Robot.UIDesktop.UIOSelector_SearchChildByMouse_UIO(inElementSpecification)
UIOSelector (see description on the top of the document)
#old name - AutomationSearchMouseElement
* **Parameters**
**inElementSpecification** UIOSelector of the UI Object
* **Returns**
pywinauto element wrapper instance or None
### pyOpenRPA.Robot.UIDesktop.UIOSelector_SearchChildByMouse_UIOTree(inUIOSelector)
!!!!Safe call is included (you can set activity and UIDesktop will choose the bitness and return the result)!!!!!
* **Parameters**
**inUIOSelector** UIOSelector of the UI Object
* **Returns**
?
4 years ago
### pyOpenRPA.Robot.UIDesktop.UIOSelector_TryRestore_Dict(inSpecificationList)
Try to restore (maximize) window, if its minimized. (!IMPORTANT! When use UIA framework minimized windows doesnt appear by the UIOSelector. You need to try restore windows and after that try to get UIO)
* **Parameters**
**inSpecificationList** UIOSelector - List of items, which contains condition attributes
* **Returns**
### pyOpenRPA.Robot.UIDesktop.UIOSelectorsSecs_WaitAppear_List(inSpecificationListList, inWaitSecs, inFlagWaitAllInMoment=False)
Wait for many UI object will appear in GUI for inWaitSecs seconds.
* **Parameters**
* **inSpecificationListList** UIOSelector list.
Example: [
> [{“title”:”notepad”},{“title”:”OK”}],
> [{“title”:”notepad”},{“title”:”Cancel”}]
]
* **inWaitSecs** Float value (seconds) for wait UI element appear in GUI
* **inFlagWaitAllInMoment** True - Wait all UI objects from the UIOSelector list to be appeared
* **Returns**
List of index, which UI object UIO will be appeared. Example: [1] # Appear only UI object with UIO selector: [{“title”:”notepad”},{“title”:”Cancel”}]
### pyOpenRPA.Robot.UIDesktop.UIOSelectorsSecs_WaitDisappear_List(inSpecificationListList, inWaitSecs, inFlagWaitAllInMoment=False)
Wait for many UI object will disappear in GUI for inWaitSecs seconds.
* **Parameters**
* **inSpecificationListList** UIOSelector list.
Example: [
> [{“title”:”notepad”},{“title”:”OK”}],
> [{“title”:”notepad”},{“title”:”Cancel”}]
]
* **inWaitSecs** Float value (seconds) for wait UI element disappear in GUI
* **inFlagWaitAllInMoment** True - Wait all UI objects from the UIOSelector list to be disappeared.
* **Returns**
List of index, which UI object UIO will be disappeared. Example: [1] # Disappear only UI object with UIO selector: [{“title”:”notepad”},{“title”:”Cancel”}]
* **Returns**
## References
[reStructuredText](http://docutils.sourceforge.net/rst.html)