Ivan Maslov
4 years ago
parent
be1d43b6b0
commit
654c9a0b94
@ -1,22 +0,0 @@
|
|||||||
# Content
|
|
||||||
- About
|
|
||||||
- How to use
|
|
||||||
|
|
||||||
# About
|
|
||||||
The OpenRPA support web app manipulation (by the Selenium lib).
|
|
||||||
More docs about selenium you can find [here](https://selenium-python.readthedocs.io/)
|
|
||||||
|
|
||||||
# How to use
|
|
||||||
To start use selenium just import selenium modules in [the robot tool](https://gitlab.com/UnicodeLabs/OpenRPA/wikis/04.2.-Tool-Robot:-How-to-use). Here is the example of the usage.
|
|
||||||
> from selenium import webdriver <br>
|
|
||||||
> from selenium.webdriver.common.keys import Keys <br>
|
|
||||||
> <br>
|
|
||||||
> driver = webdriver.Chrome() <br>
|
|
||||||
> driver.get("http://www.python.org") <br>
|
|
||||||
> assert "Python" in driver.title <br>
|
|
||||||
> elem = driver.find_element_by_name("q") <br>
|
|
||||||
> elem.clear() <br>
|
|
||||||
> elem.send_keys("pycon") <br>
|
|
||||||
> elem.send_keys(Keys.RETURN) <br>
|
|
||||||
> assert "No results found." not in driver.page_source <br>
|
|
||||||
> driver.close() <br>
|
|
||||||
@ -1,270 +0,0 @@
|
|||||||
Here you can find the docs and examples of the OpenRPA desktop (GUI) app access.
|
|
||||||
|
|
||||||
# Definitions
|
|
||||||
|
|
||||||
**UIO** - UI Object (class of pywinauto UI object) [pywinauto.base_wrapper]<br>
|
|
||||||
**UIOSelector** - List of dict (key attributes)<br>
|
|
||||||
**PWA** - PyWinAuto<br>
|
|
||||||
**PWASpecification** - List of dict (key attributes in pywinauto.find_window notation)<br>
|
|
||||||
**UIOTree** - Recursive Dict of Dict ... (UI Parent -> Child hierarchy)<br>
|
|
||||||
**UIOInfo** - Dict of UIO attributes<br>
|
|
||||||
**UIOActivity** - Activity of the UIO (UI object) from the Pywinauto module<br>
|
|
||||||
**UIOEI** - UI Object info object
|
|
||||||
|
|
||||||
# What is UIO?
|
|
||||||
UIO is a User Interface Object (pyOpenRPA terminology). For maximum compatibility, this instance is inherited from the object model developed in the [pywinauto library (click to get a list of available class functions)](https://pywinauto.readthedocs.io/en/latest/code/pywinauto.base_wrapper.html).
|
|
||||||
|
|
||||||
This approach allows us to implement useful functionality that has already been successfully developed in other libraries, and Supplement it with the missing functionality. In our case, the missing functionality is the ability to dynamically access UIO objects using UIO selectors.
|
|
||||||
|
|
||||||
# UIOSelector structure & example
|
|
||||||
<a name="UIOSelector_Structure_Examples"></a>
|
|
||||||
UIOSelector is the list of condition items for the UIO in GUI. Each item has condition attributes for detect applicable UIO. Here is the description of the available condition attributes in item.
|
|
||||||
|
|
||||||
**Desciption**<br>
|
|
||||||
```
|
|
||||||
[
|
|
||||||
{
|
|
||||||
"depth_start" :: [int, start from 1] :: the depth index, where to start check the condition list (default 1),
|
|
||||||
"depth_end" :: [int, start from 1] :: the depth index, where to stop check the condition list (default 1),
|
|
||||||
"ctrl_index" || "index" :: [int, starts from 0] :: the index of the UIO in parent UIO child list,
|
|
||||||
"title" :: [str] :: the condition for the UIO attribute *title*,
|
|
||||||
"title_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *title*,
|
|
||||||
"rich_text" :: [str] :: the condition for the UIO attribute *rich_text*,
|
|
||||||
"rich_text_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *rich_text*,
|
|
||||||
"class_name" :: [str] :: the condition for the UIO attribute *class_name*,
|
|
||||||
"class_name_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *class_name*,
|
|
||||||
"friendly_class_name" :: [str] :: the condition for the UIO attribute *friendly_class_name*,
|
|
||||||
"friendly_class_name_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *friendly_class_name*,
|
|
||||||
"control_type" :: [str] :: the condition for the UIO attribute *control_type*,
|
|
||||||
"control_type_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *control_type*,
|
|
||||||
"is_enabled" :: [bool] :: the condition for the UIO attribute *is_enabled*. If UI object is enabled on GUI,
|
|
||||||
"is_visible" :: [bool] :: the condition for the UIO attribute *is_visible*. If UI object is visible on GUI,
|
|
||||||
"backend" :: [str, "win32" || "uia"] :: the method of UIO extraction (default "win32"). ATTENTION! Current option can be only for the first item of the UIO selector. For the next items this option will be implemented from the first item.
|
|
||||||
},
|
|
||||||
{ ... specification next level UIO }
|
|
||||||
]
|
|
||||||
```
|
|
||||||
**The UIO selector example**
|
|
||||||
```
|
|
||||||
[
|
|
||||||
{"class_name":"CalcFrame", "backend":"win32"}, # 1-st level UIO specification
|
|
||||||
{"title":"Hex", "depth_start":3, "depth_end": 3} # 3-rd level specification (because of attribute depth_start|depth_stop)
|
|
||||||
]
|
|
||||||
```
|
|
||||||
# The UIDesktop module (OpenRPA/Robot/UIDesktop.py)
|
|
||||||
The UIDesktop is extension of the pywinauto module which provide access to the desktop apps by the **win32** and **ui automation** dll frameworks (big thx to the Microsoft :) ).
|
|
||||||
|
|
||||||
## Functions
|
|
||||||
*Naming convention: \<InArgument\>\_\<ActivityName\>\_\<OutArgument - if exist>*<br>
|
|
||||||
### List
|
|
||||||
- [UIOSelector_Get_UIOList](#UIOSelector_Get_UIOList)
|
|
||||||
- [UIOSelector_Get_UIO](#UIOSelector_Get_UIO)
|
|
||||||
- [UIOSelector_Exist_Bool](#UIOSelector_Exist_Bool)
|
|
||||||
- [UIOSelectorsSecs_WaitAppear_List](#UIOSelectorsSecs_WaitAppear_List)
|
|
||||||
- [UIOSelectorsSecs_WaitDisappear_List](#UIOSelectorsSecs_WaitDisappear_List)
|
|
||||||
- [UIOSelectorSecs_WaitAppear_Bool](#UIOSelectorSecs_WaitAppear_Bool)
|
|
||||||
- [UIOSelectorSecs_WaitDisappear_Bool](#UIOSelectorSecs_WaitDisappear_Bool)
|
|
||||||
- [UIOSelector_Get_BitnessInt](#UIOSelector_Get_BitnessInt)
|
|
||||||
- [UIOSelector_SearchChildByMouse_UIO](#UIOSelector_SearchChildByMouse_UIO)
|
|
||||||
- [UIOSelector_SearchChildByMouse_UIOTree](#UIOSelector_SearchChildByMouse_UIOTree)
|
|
||||||
- [UIOSelector_Get_UIOInfoList](#UIOSelector_Get_UIOInfoList)
|
|
||||||
- [UIOSelector_IsExist_Bool](#UIOSelector_IsExist_Bool)
|
|
||||||
- [UIOSelector_WaitAppear_Dict](#UIOSelector_WaitAppear_Dict)
|
|
||||||
- [UIOSelector_TryRestore_Dict](#UIOSelector_TryRestore_Dict)
|
|
||||||
- [UIOSelector_Get_UIOActivityList](#UIOSelector_Get_UIOActivityList)
|
|
||||||
- [UIOSelectorUIOActivity_Run_Dict](#UIOSelectorUIOActivity_Run_Dict)
|
|
||||||
- [UIOSelector_Get_UIOInfo](#UIOSelector_Get_UIOInfo)
|
|
||||||
- [UIOSelector_GetChildList_UIOList](#UIOSelector_GetChildList_UIOList)
|
|
||||||
- [UIOSelector_SearchUIONormalize_UIOSelector](#UIOSelector_SearchUIONormalize_UIOSelector)
|
|
||||||
- [UIOSelector_SearchProcessNormalize_UIOSelector](#UIOSelector_SearchProcessNormalize_UIOSelector)
|
|
||||||
- [UIOSelector_FocusHighlight](#UIOSelector_FocusHighlight)
|
|
||||||
- [UIOSelector_Highlight](#UIOSelector_Highlight)
|
|
||||||
### Description
|
|
||||||
**`UIOSelector_Get_UIOList (inUIOSelector,inParentUIO=None,inFlagRaiseException=True)`**<br>
|
|
||||||
<a name="UIOSelector_Get_UIOList"></a>
|
|
||||||
___
|
|
||||||
Get the list of the UIO items<br>
|
|
||||||
___
|
|
||||||
**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))<br>
|
|
||||||
**inParentUIO** - Technical argument. Applicable only if you has UIO object, which is parent for the futher search the child UIO.<br>
|
|
||||||
**inFlagRaiseException** - if True - raise exception if UIO hasn't been detected. False - don't raise the exception - return None.<br>
|
|
||||||
___
|
|
||||||
|
|
||||||
**`UIOSelector_Get_UIO (inUIOSelector,inParentUIO=None,inFlagRaiseException=True)`**<br>
|
|
||||||
<a name="UIOSelector_Get_UIO"></a>
|
|
||||||
___
|
|
||||||
Get first (if more than one UIO are applied) UIO (UI Object) <br>
|
|
||||||
___
|
|
||||||
**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))<br>
|
|
||||||
**inParentUIO** - Technical argument. Applicable only if you has UIO object, which is parent for the futher search the child UIO.<br>
|
|
||||||
**inFlagRaiseException** - if True - raise exception if UIO hasn't been detected. False - don't raise the exception - return None.<br>
|
|
||||||
___
|
|
||||||
|
|
||||||
**`UIOSelector_Exist_Bool (inUIOSelector)`**<br>
|
|
||||||
<a name="UIOSelector_Exist_Bool"></a>
|
|
||||||
___
|
|
||||||
Check if UIO exist (Identified by the UIOSelector) <br>
|
|
||||||
___
|
|
||||||
**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))<br>
|
|
||||||
___
|
|
||||||
|
|
||||||
**`UIOSelectorsSecs_WaitAppear_List (inUIOSelectorList,inWaitSecs,inFlagWaitAllInMoment=False)`**<br>
|
|
||||||
<a name="UIOSelectorsSecs_WaitAppear_List"></a>
|
|
||||||
___
|
|
||||||
Wait for UIO is appear (at least one of them or all at the same time). return: \[0,1,2\] - index of UIOSpecification, which is appear <br>
|
|
||||||
___
|
|
||||||
**inUIOSelectorList** - !ATTENTION! Current argument is not the UIOSelector. This is the !list! of the many UIO selectors. ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))<br>
|
|
||||||
**inWaitSecs** - Time in seconds to wait the UIO will appear.<br>
|
|
||||||
**inFlagWaitAllInMoment** - True - wait when UIO of the all UIOSelectors will appear. False - at least one UIO of the UIOSelector list.<br>
|
|
||||||
___
|
|
||||||
|
|
||||||
**`UIOSelectorsSecs_WaitDisappear_List (inUIOSelectorList,inWaitSecs,inFlagWaitAllInMoment=False)`**<br>
|
|
||||||
<a name="UIOSelectorsSecs_WaitDisappear_List"></a>
|
|
||||||
___
|
|
||||||
Wait for UIO is Disappear (at least one of them or all at the same time). return: \[0,1,2\] - index of UIOSpecification, which is Disappear <br>
|
|
||||||
___
|
|
||||||
**inUIOSelectorList** - !ATTENTION! Current argument is not the UIOSelector. This is the !list! of the many UIO selectors. ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))<br>
|
|
||||||
**inWaitSecs** - Time in seconds to wait the UIO will appear.<br>
|
|
||||||
**inFlagWaitAllInMoment** - True - wait when UIO of the all UIOSelectors will appear. False - at least one UIO of the UIOSelector list.<br>
|
|
||||||
___
|
|
||||||
|
|
||||||
**`UIOSelectorSecs_WaitAppear_Bool (inUIOSelector,inWaitSecs)`**<br>
|
|
||||||
<a name="UIOSelectorSecs_WaitAppear_Bool"></a>
|
|
||||||
___
|
|
||||||
Wait for UIO is appear (at least one of them or all at the same time). return: Bool - True - UIO is appear. <br>
|
|
||||||
___
|
|
||||||
**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))<br>
|
|
||||||
**inWaitSecs** - Time in seconds to wait the UIO will appear.<br>
|
|
||||||
___
|
|
||||||
|
|
||||||
|
|
||||||
**`UIOSelectorSecs_WaitDisappear_Bool (inUIOSelector,inWaitSecs)`**<br>
|
|
||||||
<a name="UIOSelectorSecs_WaitDisappear_Bool"></a>
|
|
||||||
___
|
|
||||||
Wait for UIO is disappear (at least one of them or all at the same time) . return: Bool - True - UIO is Disappear. <br>
|
|
||||||
___
|
|
||||||
**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))<br>
|
|
||||||
**inWaitSecs** - Time in seconds to wait the UIO will appear.<br>
|
|
||||||
___
|
|
||||||
|
|
||||||
**`UIOSelector_Get_BitnessInt (inUIOSelector)`**<br>
|
|
||||||
<a name="UIOSelector_Get_BitnessInt"></a>
|
|
||||||
___
|
|
||||||
Get process bitness (32 or 64). return None (if Process not found), int 32, or int 64<br>
|
|
||||||
___
|
|
||||||
**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))<br>
|
|
||||||
___
|
|
||||||
|
|
||||||
|
|
||||||
**`UIOSelector_SearchChildByMouse_UIO(inUIOSelector)`**<br>
|
|
||||||
<a name="UIOSelector_SearchChildByMouse_UIO"></a>
|
|
||||||
___
|
|
||||||
Run the search UIO by mouse hover event. result = UIO element wrapper instance or None <br>
|
|
||||||
___
|
|
||||||
**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))<br>
|
|
||||||
___
|
|
||||||
|
|
||||||
**`UIOSelector_SearchChildByMouse_UIOTree(inUIOSelector)`**<br>
|
|
||||||
<a name="UIOSelector_SearchChildByMouse_UIOTree"></a>
|
|
||||||
___
|
|
||||||
Run the search UIO by mouse hover event. result = UIO element wrapper instance or None <br>
|
|
||||||
___
|
|
||||||
**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))<br>
|
|
||||||
___
|
|
||||||
|
|
||||||
**`UIOSelector_Get_UIOInfoList (inUIOSelector,inElement=None)`**<br>
|
|
||||||
<a name="UIOSelector_Get_UIOInfoList"></a>
|
|
||||||
___
|
|
||||||
Get the UIOEI object.<br>
|
|
||||||
___
|
|
||||||
**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))<br>
|
|
||||||
___
|
|
||||||
|
|
||||||
**`UIOSelector_IsExist_Bool (inUIOSelector)`**<br>
|
|
||||||
<a name="UIOSelector_IsExist_Bool"></a>
|
|
||||||
___
|
|
||||||
Check is the UIO/UIO's by the UIOSelector exist <br>
|
|
||||||
___
|
|
||||||
**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))<br>
|
|
||||||
___
|
|
||||||
|
|
||||||
**`UIOSelector_WaitAppear_Dict(inUIOSelector,inTimeout=60)`**<br>
|
|
||||||
<a name="UIOSelector_WaitAppear_Dict"></a>
|
|
||||||
___
|
|
||||||
Wait for the UIO by the UIOSelector appear <br>
|
|
||||||
___
|
|
||||||
**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))<br>
|
|
||||||
___
|
|
||||||
|
|
||||||
**`UIOSelector_TryRestore_Dict(inUIOSelector)`**<br>
|
|
||||||
<a name="UIOSelector_TryRestore_Dict"></a>
|
|
||||||
___
|
|
||||||
Try to restore (maximize) window, if it's minimized. (!IMPORTANT! When use UIA framework minimized windows doesn't appear by the UIOSelector. You need to try restore windows and after that try to get UIO)<br>
|
|
||||||
___
|
|
||||||
**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))<br>
|
|
||||||
___
|
|
||||||
|
|
||||||
**`UIOSelector_Get_UIOActivityList (inUIOSelector)`**<br>
|
|
||||||
<a name="UIOSelector_Get_UIOActivityList"></a>
|
|
||||||
___
|
|
||||||
Get the list of the UI object activities<br>
|
|
||||||
___
|
|
||||||
**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))<br>
|
|
||||||
___
|
|
||||||
|
|
||||||
**`UIOSelectorUIOActivity_Run_Dict(inUIOSelector,inActionName,inArgumentList=\[\],inkwArgumentObject={})`**<br>
|
|
||||||
<a name="UIOSelectorUIOActivity_Run_Dict"></a>
|
|
||||||
___
|
|
||||||
Run the activity in UIO (UI Object) <br>
|
|
||||||
___
|
|
||||||
**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))<br>
|
|
||||||
**inActionName** - UIOActivity (name) from Pywinauto<br>
|
|
||||||
___
|
|
||||||
|
|
||||||
**`UIOSelector_Get_UIOInfo(inUIOSelector)`**<br>
|
|
||||||
<a name="UIOSelector_Get_UIOInfo"></a>
|
|
||||||
___
|
|
||||||
Get the UIO dict of the attributes <br>
|
|
||||||
___
|
|
||||||
**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))<br>
|
|
||||||
___
|
|
||||||
|
|
||||||
**`UIOSelector_GetChildList_UIOList(inUIOSelector=\[\],inBackend=mDefaultPywinautoBackend)`**<br>
|
|
||||||
<a name="UIOSelector_GetChildList_UIOList"></a>
|
|
||||||
___
|
|
||||||
Get list of child UIO's by Parent UIOSelector<br>
|
|
||||||
___
|
|
||||||
**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))<br>
|
|
||||||
___
|
|
||||||
|
|
||||||
**`UIOSelector_SearchUIONormalize_UIOSelector (inUIOSelector)`**<br>
|
|
||||||
<a name="UIOSelector_SearchUIONormalize_UIOSelector"></a>
|
|
||||||
___
|
|
||||||
Technical def. Do UIOSelector normalization for the search processes.<br>
|
|
||||||
___
|
|
||||||
**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))<br>
|
|
||||||
___
|
|
||||||
|
|
||||||
**`UIOSelector_SearchProcessNormalize_UIOSelector (inUIOSelector)`**<br>
|
|
||||||
<a name="UIOSelector_SearchProcessNormalize_UIOSelector"></a>
|
|
||||||
___
|
|
||||||
Technical def. Do UIOSelector normalization for the search processes.<br>
|
|
||||||
___
|
|
||||||
**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))<br>
|
|
||||||
___
|
|
||||||
|
|
||||||
**`UIOSelector_FocusHighlight(inUIOSelector)`**<br>
|
|
||||||
<a name="UIOSelector_FocusHighlight"></a>
|
|
||||||
___
|
|
||||||
Set focus and highlight (draw outline) the element (in app) by the UIO selector.<br>
|
|
||||||
___
|
|
||||||
**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))<br>
|
|
||||||
___
|
|
||||||
|
|
||||||
**`UIOSelector_Highlight(inUIOSelector)`**<br>
|
|
||||||
<a name="UIOSelector_Highlight"></a>
|
|
||||||
___
|
|
||||||
Highlight (draw outline) the element (in app) by the UIO selector.<br>
|
|
||||||
___
|
|
||||||
**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))<br>
|
|
||||||
___
|
|
||||||
@ -1 +0,0 @@
|
|||||||
**How to automate Keyboard and Mouse device in PC**
|
|
||||||
@ -1,7 +0,0 @@
|
|||||||
**How to automate image recognition on PC**
|
|
||||||
|
|
||||||
Here you can find any ways you need to use in your business case:
|
|
||||||
* Find the exact match on the screen with the other image
|
|
||||||
* Use text recognition module (OCR)
|
|
||||||
* Use computer vision (CV) to identify the objects on screen/image/video
|
|
||||||
* Use artificial intelligence (AI) to make custom identification/classification/text recognition
|
|
||||||
Binary file not shown.
File diff suppressed because one or more lines are too long
Loading…
Reference in new issue