From 654c9a0b94baddfc6adad0cd326275b35bce9057 Mon Sep 17 00:00:00 2001 From: Ivan Maslov Date: Tue, 9 Mar 2021 20:36:34 +0300 Subject: [PATCH] GUIDE ENG update --- Sources/GuideSphinx/Robot/02_Defs.rst | 66 +++- Sources/GuideSphinx/Robot/03_HowToUse.rst | 151 +++++++-- Sources/pyOpenRPA/Robot/UIDesktop.py | 49 +++ ...Web-app-access-(Chrome,-Firefox,-Opera).md | 22 -- ...I-access-(win32-and-UI-automation-dlls).md | 270 ---------------- ...practice.-Keyboard-&-mouse-manipulation.md | 1 - ...ice.-Screen-capture-&-image-recognition.md | 7 - .../html/03_Copyrights_Contacts.html | 3 +- Wiki/ENG_Guide/html/Robot/01_Robot.html | 106 +++++++ Wiki/ENG_Guide/html/Robot/02_Defs.html | 227 +++++++++++++- Wiki/ENG_Guide/html/Robot/03_HowToUse.html | 178 +++++++++-- .../_modules/pyOpenRPA/Robot/UIDesktop.html | 77 ++++- .../_sources/03_Copyrights_Contacts.rst.txt | 4 +- .../html/_sources/Robot/02_Defs.rst.txt | 66 +++- .../html/_sources/Robot/03_HowToUse.rst.txt | 151 +++++++-- Wiki/ENG_Guide/html/genindex.html | 16 +- Wiki/ENG_Guide/html/index.html | 12 +- Wiki/ENG_Guide/html/objects.inv | Bin 1160 -> 1222 bytes Wiki/ENG_Guide/html/searchindex.js | 2 +- .../markdown/03_Copyrights_Contacts.md | 5 +- Wiki/ENG_Guide/markdown/Robot/01_Robot.md | 119 +++++++ Wiki/ENG_Guide/markdown/Robot/02_Defs.md | 292 +++++++++++++++++- Wiki/ENG_Guide/markdown/Robot/03_HowToUse.md | 201 ++++++++++-- Wiki/ENG_Guide/markdown/index.md | 12 +- 24 files changed, 1583 insertions(+), 454 deletions(-) delete mode 100644 Wiki/05.1.-Theory-&-practice.-Web-app-access-(Chrome,-Firefox,-Opera).md delete mode 100644 Wiki/05.2.-Theory-&-practice.-Desktop-app-UI-access-(win32-and-UI-automation-dlls).md delete mode 100644 Wiki/05.3.-Theory-&-practice.-Keyboard-&-mouse-manipulation.md delete mode 100644 Wiki/05.4.-Theory-&-practice.-Screen-capture-&-image-recognition.md diff --git a/Sources/GuideSphinx/Robot/02_Defs.rst b/Sources/GuideSphinx/Robot/02_Defs.rst index 533d2155..54d02d33 100644 --- a/Sources/GuideSphinx/Robot/02_Defs.rst +++ b/Sources/GuideSphinx/Robot/02_Defs.rst @@ -2,9 +2,69 @@ 2. Defs #################################### -************************************************** -pyOpenRPA.Robot.UIDesktop -************************************************** +**************************************************************************************************** +Desktop app UI access (win32 and UI automation dlls) +**************************************************************************************************** + +Definitions +############################################ + +- **UIO** - UI Object (class of pywinauto UI object) [pywinauto.base_wrapper] +- **UIOSelector** - List of dict (key attributes) +- **PWA** - PyWinAuto +- **PWASpecification** - List of dict (key attributes in pywinauto.find_window notation) +- **UIOTree** - Recursive Dict of Dict ... (UI Parent -> Child hierarchy) +- **UIOInfo** - Dict of UIO attributes +- **UIOActivity** - Activity of the UIO (UI object) from the Pywinauto module +- **UIOEI** - UI Object info object + + +What is UIO? +############################################ +UIO is a User Interface Object (pyOpenRPA terminology). For maximum compatibility, this instance is inherited from the object model developed in the [pywinauto library (click to get a list of available class functions)](https://pywinauto.readthedocs.io/en/latest/code/pywinauto.base_wrapper.html). + +This approach allows us to implement useful functionality that has already been successfully developed in other libraries, and Supplement it with the missing functionality. In our case, the missing functionality is the ability to dynamically access UIO objects using UIO selectors. + + +UIOSelector structure & example +############################################ + +UIOSelector is the list of condition items for the UIO in GUI. Each item has condition attributes for detect applicable UIO. Here is the description of the available condition attributes in item. + +**Desciption**
+ ``` +[ + { + "depth_start" :: [int, start from 1] :: the depth index, where to start check the condition list (default 1), + "depth_end" :: [int, start from 1] :: the depth index, where to stop check the condition list (default 1), + "ctrl_index" || "index" :: [int, starts from 0] :: the index of the UIO in parent UIO child list, + "title" :: [str] :: the condition for the UIO attribute *title*, + "title_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *title*, + "rich_text" :: [str] :: the condition for the UIO attribute *rich_text*, + "rich_text_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *rich_text*, + "class_name" :: [str] :: the condition for the UIO attribute *class_name*, + "class_name_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *class_name*, + "friendly_class_name" :: [str] :: the condition for the UIO attribute *friendly_class_name*, + "friendly_class_name_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *friendly_class_name*, + "control_type" :: [str] :: the condition for the UIO attribute *control_type*, + "control_type_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *control_type*, + "is_enabled" :: [bool] :: the condition for the UIO attribute *is_enabled*. If UI object is enabled on GUI, + "is_visible" :: [bool] :: the condition for the UIO attribute *is_visible*. If UI object is visible on GUI, + "backend" :: [str, "win32" || "uia"] :: the method of UIO extraction (default "win32"). ATTENTION! Current option can be only for the first item of the UIO selector. For the next items this option will be implemented from the first item. + }, + { ... specification next level UIO } +] + ``` +**The UIO selector example** + ``` +[ + {"class_name":"CalcFrame", "backend":"win32"}, # 1-st level UIO specification + {"title":"Hex", "depth_start":3, "depth_end": 3} # 3-rd level specification (because of attribute depth_start|depth_stop) +] + ``` +The UIDesktop module (OpenRPA/Robot/UIDesktop.py) +######################################################################################## +The UIDesktop is extension of the pywinauto module which provide access to the desktop apps by the **win32** and **ui automation** dll frameworks (big thx to the Microsoft :) ). .. code-block:: python diff --git a/Sources/GuideSphinx/Robot/03_HowToUse.rst b/Sources/GuideSphinx/Robot/03_HowToUse.rst index bc4ec41a..70ef27b1 100644 --- a/Sources/GuideSphinx/Robot/03_HowToUse.rst +++ b/Sources/GuideSphinx/Robot/03_HowToUse.rst @@ -2,21 +2,10 @@ 3. How to use #################################### -************************************************** -Content -************************************************** -- `About <#about>`__ -- `How to use <#way-to-use>`__ -- `Create python script <#create-python-script>`__ -- `Execute python script <#execute-python-script>`__ - -************************************************** -About -************************************************** The Robot tool is the main module for production process automation. It has no graphic/console interface. All low-level actions to OS are perfoming by the Robot tool in OpenRPA. ************************************************** -Way to use +How to execute RPA script ************************************************** You can use the robot by the several ways: @@ -39,9 +28,8 @@ In order to use robot just add Robot tool folder in work directory and add line import cv2 # [Computer vision](https://gitlab.com/UnicodeLabs/OpenRPA/wikis/05.4.-Theory-&-practice:-Screen-capture-&-image-recognition)
import keyboard #[Keyboard manipulation](https://gitlab.com/UnicodeLabs/OpenRPA/wikis/05.3.-Theory-&-practice:-Keyboard-&-mouse-manipulation)
-************************************************** Execute python script -************************************************** +############################################ The OpenRPA is fully portable solution. It contains own python enviroment both 32 and 64 bit versions. So, you can execute your python script in several ways: - Execute in python x32 (\OpenRPA\Resources\WPy32-3720\python-3.7.2) @@ -80,9 +68,9 @@ In order to simplify the execution process you can write several code lines in f .\..\Resources\WPy32-3720\python-3.7.2\OpenRPAOrchestrator.exe orchestratorMain.py pause >nul -************************************************** + Use in studio script (n/a) -************************************************** +############################################ .. code-block:: python import sys @@ -92,12 +80,129 @@ Use in studio script (n/a) import subprocess import time - #Highlight the UI Object in Folder explorer
- GUI.UIOSelector_FocusHighlight([{"class_name":"CabinetWClass","backend":"uia"},{"ctrl_index":2},{"ctrl_index":0},{"ctrl_index":2},{"ctrl_index":0}])
+ #Highlight the UI Object in Folder explorer + GUI.UIOSelector_FocusHighlight([{"class_name":"CabinetWClass","backend":"uia"},{"ctrl_index":2},{"ctrl_index":0},{"ctrl_index":2},{"ctrl_index":0}]) + + #Wait 2 seconds + time.sleep(3) + + #Loop: get child element of UI List + for lItem in GUI.UIOSelector_Get_UIO([{"class_name":"CabinetWClass","backend":"uia"},{"ctrl_index":2},{"ctrl_index":0},{"ctrl_index":2},{"ctrl_index":0}]).children(): + print(str(lItem)) + + + +Here you can find the docs and examples of the OpenRPA desktop (GUI) app access. + +**************************************************************************************************** +Theory & practice. Desktop app UI access (win32 and UI automation dlls) +**************************************************************************************************** + +Definitions +############################################ + +**UIO** - UI Object (class of pywinauto UI object) [pywinauto.base_wrapper]
+**UIOSelector** - List of dict (key attributes)
+**PWA** - PyWinAuto
+**PWASpecification** - List of dict (key attributes in pywinauto.find_window notation)
+**UIOTree** - Recursive Dict of Dict ... (UI Parent -> Child hierarchy)
+**UIOInfo** - Dict of UIO attributes
+**UIOActivity** - Activity of the UIO (UI object) from the Pywinauto module
+**UIOEI** - UI Object info object + + +What is UIO? +############################################ +UIO is a User Interface Object (pyOpenRPA terminology). For maximum compatibility, this instance is inherited from the object model developed in the [pywinauto library (click to get a list of available class functions)](https://pywinauto.readthedocs.io/en/latest/code/pywinauto.base_wrapper.html). + +This approach allows us to implement useful functionality that has already been successfully developed in other libraries, and Supplement it with the missing functionality. In our case, the missing functionality is the ability to dynamically access UIO objects using UIO selectors. + + +UIOSelector structure & example +############################################ + +UIOSelector is the list of condition items for the UIO in GUI. Each item has condition attributes for detect applicable UIO. Here is the description of the available condition attributes in item. + +**Desciption** + ``` +[ + { + "depth_start" :: [int, start from 1] :: the depth index, where to start check the condition list (default 1), + "depth_end" :: [int, start from 1] :: the depth index, where to stop check the condition list (default 1), + "ctrl_index" || "index" :: [int, starts from 0] :: the index of the UIO in parent UIO child list, + "title" :: [str] :: the condition for the UIO attribute *title*, + "title_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *title*, + "rich_text" :: [str] :: the condition for the UIO attribute *rich_text*, + "rich_text_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *rich_text*, + "class_name" :: [str] :: the condition for the UIO attribute *class_name*, + "class_name_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *class_name*, + "friendly_class_name" :: [str] :: the condition for the UIO attribute *friendly_class_name*, + "friendly_class_name_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *friendly_class_name*, + "control_type" :: [str] :: the condition for the UIO attribute *control_type*, + "control_type_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *control_type*, + "is_enabled" :: [bool] :: the condition for the UIO attribute *is_enabled*. If UI object is enabled on GUI, + "is_visible" :: [bool] :: the condition for the UIO attribute *is_visible*. If UI object is visible on GUI, + "backend" :: [str, "win32" || "uia"] :: the method of UIO extraction (default "win32"). ATTENTION! Current option can be only for the first item of the UIO selector. For the next items this option will be implemented from the first item. + }, + { ... specification next level UIO } +] + ``` +**The UIO selector example** + ``` +[ + {"class_name":"CalcFrame", "backend":"win32"}, # 1-st level UIO specification + {"title":"Hex", "depth_start":3, "depth_end": 3} # 3-rd level specification (because of attribute depth_start|depth_stop) +] + ``` +The UIDesktop module (OpenRPA/Robot/UIDesktop.py) +######################################################################################## +The UIDesktop is extension of the pywinauto module which provide access to the desktop apps by the **win32** and **ui automation** dll frameworks (big thx to the Microsoft :) ). + +*Naming convention: \\_\\_\*
+ + +**************************************************************************************************** +Theory & practice. WEB app UI access (selenium) +**************************************************************************************************** + +About +############################################### +The pyOpenRPA support web app manipulation (by the Selenium lib). +More docs about selenium you can find here (https://selenium-python.readthedocs.io/) + +How to use +############################################### +To start use selenium just import selenium modules in the robot tool. Here is the example of the usage. + +.. code-block:: python + + from selenium import webdriver + from selenium.webdriver.common.keys import Keys + + driver = webdriver.Chrome() + driver.get("http://www.python.org") + assert "Python" in driver.title + elem = driver.find_element_by_name("q") + elem.clear() + elem.send_keys("pycon") + elem.send_keys(Keys.RETURN) + assert "No results found." not in driver.page_source + driver.close() + +**************************************************************************************************** +Theory & practice. Keyboard & mouse manipulation +**************************************************************************************************** + + +**************************************************************************************************** +Theory & practice. Screen capture & image recognition +**************************************************************************************************** - #Wait 2 seconds
- time.sleep(3)
+How to automate image recognition on PC +########################################### - #Loop: get child element of UI List
- for lItem in GUI.UIOSelector_Get_UIO([{"class_name":"CabinetWClass","backend":"uia"},{"ctrl_index":2},{"ctrl_index":0},{"ctrl_index":2},{"ctrl_index":0}]).children():
-         print(str(lItem)) +Here you can find any ways you need to use in your business case: +- Find the exact match on the screen with the other image +- Use text recognition module (OCR) +- Use computer vision (CV) to identify the objects on screen/image/video +- Use artificial intelligence (AI) to make custom identification/classification/text recognition \ No newline at end of file diff --git a/Sources/pyOpenRPA/Robot/UIDesktop.py b/Sources/pyOpenRPA/Robot/UIDesktop.py index f82cc34b..0109cbdd 100644 --- a/Sources/pyOpenRPA/Robot/UIDesktop.py +++ b/Sources/pyOpenRPA/Robot/UIDesktop.py @@ -782,6 +782,12 @@ def UIOSelector_Get_UIOInfoList (inUIOSelector, inElement=None): #inSpecificationList - UIOSelector #old name - PywinautoExtTryToRestore def UIOSelector_TryRestore_Dict(inSpecificationList): + """ + Try to restore (maximize) window, if it's minimized. (!IMPORTANT! When use UIA framework minimized windows doesn't appear by the UIOSelector. You need to try restore windows and after that try to get UIO) + + :param inSpecificationList: UIOSelector - List of items, which contains condition attributes + :return: + """ lResult={} try: #Подготовка взодного массива @@ -800,6 +806,12 @@ def UIOSelector_TryRestore_Dict(inSpecificationList): #inControlSpecificationArray - UIOSelector #old name - ElementActionGetList def UIOSelector_Get_UIOActivityList (inUIOSelector): + """ + Get the list of the UI object activities + + :param inUIOSelector: UIOSelector - List of items, which contains condition attributes + :return: + """ #Check the bitness lSafeOtherProcess = UIOSelector_SafeOtherGet_Process(inUIOSelector) if lSafeOtherProcess is None: @@ -838,6 +850,15 @@ def UIOSelector_Get_UIOActivityList (inUIOSelector): #inActionName - UIOActivity (name) from Pywinauto #old name - ElementRunAction def UIOSelectorUIOActivity_Run_Dict(inUIOSelector, inActionName, inArgumentList=None, inkwArgumentObject=None): + """ + Run the activity in UIO (UI Object) + + :param inUIOSelector: UIOSelector - List of items, which contains condition attributes + :param inActionName: UIOActivity (name) activity name string from Pywinauto + :param inArgumentList: + :param inkwArgumentObject: + :return: + """ if inArgumentList is None: inArgumentList=[] # 2021 02 22 Minor fix by Ivan Maslov if inkwArgumentObject is None: inkwArgumentObject={} # 2021 02 22 Minor fix by Ivan Maslov lResult={} @@ -887,6 +908,12 @@ def UIOSelectorUIOActivity_Run_Dict(inUIOSelector, inActionName, inArgumentList= #!!!!!Safe call is included (you can set activity and UIDesktop will choose the bitness and return the result)!!!!! #old name - ElementGetInfo def UIOSelector_Get_UIOInfo(inUIOSelector): + """ + Get the UIO dict of the attributes + + :param inUIOSelector: UIOSelector - List of items, which contains condition attributes + :return: + """ #Check the bitness lSafeOtherProcess = UIOSelector_SafeOtherGet_Process(inUIOSelector) if lSafeOtherProcess is None: @@ -1012,6 +1039,15 @@ def UIOXY_SearchChild_ListDict(inRootElement,inX,inY,inHierarchyList=None): #inControlSpecificationArray- UIOSelector #old name - ElementGetChildElementList def UIOSelector_GetChildList_UIOList(inUIOSelector=None, inBackend=mDefaultPywinautoBackend): + """ + Get list of child UIO's by the parent UIOSelector + + :param inUIOSelector: UIOSelector - List of items, which contains condition attributes + :param inBackend: "win32" or "uia" + :return: + """ + + if inUIOSelector is None: inUIOSelector = [] #mRobotLogger.info(f"File!!!!") #mRobotLogger.info(f"inSelector:{str(inUIOSelector)}, inBackend:{str(inBackend)}") @@ -1301,6 +1337,12 @@ def BackendStr_GetTopLevelList_UIOInfo(inBackend=mDefaultPywinautoBackend): #!!!!!Safe call is included (you can set activity and UIDesktop will choose the bitness and return the result)!!!!! #old name - ElementDrawOutlineNew def UIOSelector_Highlight(inUIOSelector): + """ + Highlight (draw outline) the element (in app) by the UIO selector. + + :param inUIOSelector: UIOSelector - List of items, which contains condition attributes + :return: + """ #Check the bitness lSafeOtherProcess = UIOSelector_SafeOtherGet_Process(inUIOSelector) if lSafeOtherProcess is None: @@ -1326,6 +1368,13 @@ def UIOSelector_Highlight(inUIOSelector): #!!!!!Safe call is included (you can set activity and UIDesktop will choose the bitness and return the result)!!!!! #old name - ElementDrawOutlineNewFocus def UIOSelector_FocusHighlight(inUIOSelector): + """ + Set focus and highlight (draw outline) the element (in app) by the UIO selector. + + :param inUIOSelector: UIOSelector - List of items, which contains condition attributes + :return: + """ + #Check the bitness lSafeOtherProcess = UIOSelector_SafeOtherGet_Process(inUIOSelector) if lSafeOtherProcess is None: diff --git a/Wiki/05.1.-Theory-&-practice.-Web-app-access-(Chrome,-Firefox,-Opera).md b/Wiki/05.1.-Theory-&-practice.-Web-app-access-(Chrome,-Firefox,-Opera).md deleted file mode 100644 index 4f4c4892..00000000 --- a/Wiki/05.1.-Theory-&-practice.-Web-app-access-(Chrome,-Firefox,-Opera).md +++ /dev/null @@ -1,22 +0,0 @@ -# Content -- About -- How to use - -# About -The OpenRPA support web app manipulation (by the Selenium lib). -More docs about selenium you can find [here](https://selenium-python.readthedocs.io/) - -# How to use -To start use selenium just import selenium modules in [the robot tool](https://gitlab.com/UnicodeLabs/OpenRPA/wikis/04.2.-Tool-Robot:-How-to-use). Here is the example of the usage. -> from selenium import webdriver
-> from selenium.webdriver.common.keys import Keys
->
-> driver = webdriver.Chrome()
-> driver.get("http://www.python.org")
-> assert "Python" in driver.title
-> elem = driver.find_element_by_name("q")
-> elem.clear()
-> elem.send_keys("pycon")
-> elem.send_keys(Keys.RETURN)
-> assert "No results found." not in driver.page_source
-> driver.close()
diff --git a/Wiki/05.2.-Theory-&-practice.-Desktop-app-UI-access-(win32-and-UI-automation-dlls).md b/Wiki/05.2.-Theory-&-practice.-Desktop-app-UI-access-(win32-and-UI-automation-dlls).md deleted file mode 100644 index 2588417f..00000000 --- a/Wiki/05.2.-Theory-&-practice.-Desktop-app-UI-access-(win32-and-UI-automation-dlls).md +++ /dev/null @@ -1,270 +0,0 @@ -Here you can find the docs and examples of the OpenRPA desktop (GUI) app access. - -# Definitions - -**UIO** - UI Object (class of pywinauto UI object) [pywinauto.base_wrapper]
-**UIOSelector** - List of dict (key attributes)
-**PWA** - PyWinAuto
-**PWASpecification** - List of dict (key attributes in pywinauto.find_window notation)
-**UIOTree** - Recursive Dict of Dict ... (UI Parent -> Child hierarchy)
-**UIOInfo** - Dict of UIO attributes
-**UIOActivity** - Activity of the UIO (UI object) from the Pywinauto module
-**UIOEI** - UI Object info object - -# What is UIO? -UIO is a User Interface Object (pyOpenRPA terminology). For maximum compatibility, this instance is inherited from the object model developed in the [pywinauto library (click to get a list of available class functions)](https://pywinauto.readthedocs.io/en/latest/code/pywinauto.base_wrapper.html). - -This approach allows us to implement useful functionality that has already been successfully developed in other libraries, and Supplement it with the missing functionality. In our case, the missing functionality is the ability to dynamically access UIO objects using UIO selectors. - -# UIOSelector structure & example - -UIOSelector is the list of condition items for the UIO in GUI. Each item has condition attributes for detect applicable UIO. Here is the description of the available condition attributes in item. - -**Desciption**
- ``` -[ - { - "depth_start" :: [int, start from 1] :: the depth index, where to start check the condition list (default 1), - "depth_end" :: [int, start from 1] :: the depth index, where to stop check the condition list (default 1), - "ctrl_index" || "index" :: [int, starts from 0] :: the index of the UIO in parent UIO child list, - "title" :: [str] :: the condition for the UIO attribute *title*, - "title_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *title*, - "rich_text" :: [str] :: the condition for the UIO attribute *rich_text*, - "rich_text_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *rich_text*, - "class_name" :: [str] :: the condition for the UIO attribute *class_name*, - "class_name_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *class_name*, - "friendly_class_name" :: [str] :: the condition for the UIO attribute *friendly_class_name*, - "friendly_class_name_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *friendly_class_name*, - "control_type" :: [str] :: the condition for the UIO attribute *control_type*, - "control_type_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *control_type*, - "is_enabled" :: [bool] :: the condition for the UIO attribute *is_enabled*. If UI object is enabled on GUI, - "is_visible" :: [bool] :: the condition for the UIO attribute *is_visible*. If UI object is visible on GUI, - "backend" :: [str, "win32" || "uia"] :: the method of UIO extraction (default "win32"). ATTENTION! Current option can be only for the first item of the UIO selector. For the next items this option will be implemented from the first item. - }, - { ... specification next level UIO } -] - ``` -**The UIO selector example** - ``` -[ - {"class_name":"CalcFrame", "backend":"win32"}, # 1-st level UIO specification - {"title":"Hex", "depth_start":3, "depth_end": 3} # 3-rd level specification (because of attribute depth_start|depth_stop) -] - ``` -# The UIDesktop module (OpenRPA/Robot/UIDesktop.py) -The UIDesktop is extension of the pywinauto module which provide access to the desktop apps by the **win32** and **ui automation** dll frameworks (big thx to the Microsoft :) ). - -## Functions -*Naming convention: \\_\\_\*
-### List -- [UIOSelector_Get_UIOList](#UIOSelector_Get_UIOList) -- [UIOSelector_Get_UIO](#UIOSelector_Get_UIO) -- [UIOSelector_Exist_Bool](#UIOSelector_Exist_Bool) -- [UIOSelectorsSecs_WaitAppear_List](#UIOSelectorsSecs_WaitAppear_List) -- [UIOSelectorsSecs_WaitDisappear_List](#UIOSelectorsSecs_WaitDisappear_List) -- [UIOSelectorSecs_WaitAppear_Bool](#UIOSelectorSecs_WaitAppear_Bool) -- [UIOSelectorSecs_WaitDisappear_Bool](#UIOSelectorSecs_WaitDisappear_Bool) -- [UIOSelector_Get_BitnessInt](#UIOSelector_Get_BitnessInt) -- [UIOSelector_SearchChildByMouse_UIO](#UIOSelector_SearchChildByMouse_UIO) -- [UIOSelector_SearchChildByMouse_UIOTree](#UIOSelector_SearchChildByMouse_UIOTree) -- [UIOSelector_Get_UIOInfoList](#UIOSelector_Get_UIOInfoList) -- [UIOSelector_IsExist_Bool](#UIOSelector_IsExist_Bool) -- [UIOSelector_WaitAppear_Dict](#UIOSelector_WaitAppear_Dict) -- [UIOSelector_TryRestore_Dict](#UIOSelector_TryRestore_Dict) -- [UIOSelector_Get_UIOActivityList](#UIOSelector_Get_UIOActivityList) -- [UIOSelectorUIOActivity_Run_Dict](#UIOSelectorUIOActivity_Run_Dict) -- [UIOSelector_Get_UIOInfo](#UIOSelector_Get_UIOInfo) -- [UIOSelector_GetChildList_UIOList](#UIOSelector_GetChildList_UIOList) -- [UIOSelector_SearchUIONormalize_UIOSelector](#UIOSelector_SearchUIONormalize_UIOSelector) -- [UIOSelector_SearchProcessNormalize_UIOSelector](#UIOSelector_SearchProcessNormalize_UIOSelector) -- [UIOSelector_FocusHighlight](#UIOSelector_FocusHighlight) -- [UIOSelector_Highlight](#UIOSelector_Highlight) -### Description -**`UIOSelector_Get_UIOList (inUIOSelector,inParentUIO=None,inFlagRaiseException=True)`**
- -___ -Get the list of the UIO items
-___ -**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))
-**inParentUIO** - Technical argument. Applicable only if you has UIO object, which is parent for the futher search the child UIO.
-**inFlagRaiseException** - if True - raise exception if UIO hasn't been detected. False - don't raise the exception - return None.
-___ - -**`UIOSelector_Get_UIO (inUIOSelector,inParentUIO=None,inFlagRaiseException=True)`**
- -___ -Get first (if more than one UIO are applied) UIO (UI Object)
-___ -**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))
-**inParentUIO** - Technical argument. Applicable only if you has UIO object, which is parent for the futher search the child UIO.
-**inFlagRaiseException** - if True - raise exception if UIO hasn't been detected. False - don't raise the exception - return None.
-___ - -**`UIOSelector_Exist_Bool (inUIOSelector)`**
- -___ -Check if UIO exist (Identified by the UIOSelector)
-___ -**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))
-___ - -**`UIOSelectorsSecs_WaitAppear_List (inUIOSelectorList,inWaitSecs,inFlagWaitAllInMoment=False)`**
- -___ -Wait for UIO is appear (at least one of them or all at the same time). return: \[0,1,2\] - index of UIOSpecification, which is appear
-___ -**inUIOSelectorList** - !ATTENTION! Current argument is not the UIOSelector. This is the !list! of the many UIO selectors. ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))
-**inWaitSecs** - Time in seconds to wait the UIO will appear.
-**inFlagWaitAllInMoment** - True - wait when UIO of the all UIOSelectors will appear. False - at least one UIO of the UIOSelector list.
-___ - -**`UIOSelectorsSecs_WaitDisappear_List (inUIOSelectorList,inWaitSecs,inFlagWaitAllInMoment=False)`**
- -___ -Wait for UIO is Disappear (at least one of them or all at the same time). return: \[0,1,2\] - index of UIOSpecification, which is Disappear
-___ -**inUIOSelectorList** - !ATTENTION! Current argument is not the UIOSelector. This is the !list! of the many UIO selectors. ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))
-**inWaitSecs** - Time in seconds to wait the UIO will appear.
-**inFlagWaitAllInMoment** - True - wait when UIO of the all UIOSelectors will appear. False - at least one UIO of the UIOSelector list.
-___ - -**`UIOSelectorSecs_WaitAppear_Bool (inUIOSelector,inWaitSecs)`**
- -___ -Wait for UIO is appear (at least one of them or all at the same time). return: Bool - True - UIO is appear.
-___ -**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))
-**inWaitSecs** - Time in seconds to wait the UIO will appear.
-___ - - -**`UIOSelectorSecs_WaitDisappear_Bool (inUIOSelector,inWaitSecs)`**
- -___ -Wait for UIO is disappear (at least one of them or all at the same time) . return: Bool - True - UIO is Disappear.
-___ -**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))
-**inWaitSecs** - Time in seconds to wait the UIO will appear.
-___ - -**`UIOSelector_Get_BitnessInt (inUIOSelector)`**
- -___ -Get process bitness (32 or 64). return None (if Process not found), int 32, or int 64
-___ -**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))
-___ - - -**`UIOSelector_SearchChildByMouse_UIO(inUIOSelector)`**
- -___ -Run the search UIO by mouse hover event. result = UIO element wrapper instance or None
-___ -**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))
-___ - -**`UIOSelector_SearchChildByMouse_UIOTree(inUIOSelector)`**
- -___ -Run the search UIO by mouse hover event. result = UIO element wrapper instance or None
-___ -**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))
-___ - -**`UIOSelector_Get_UIOInfoList (inUIOSelector,inElement=None)`**
- -___ -Get the UIOEI object.
-___ -**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))
-___ - -**`UIOSelector_IsExist_Bool (inUIOSelector)`**
- -___ -Check is the UIO/UIO's by the UIOSelector exist
-___ -**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))
-___ - -**`UIOSelector_WaitAppear_Dict(inUIOSelector,inTimeout=60)`**
- -___ -Wait for the UIO by the UIOSelector appear
-___ -**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))
-___ - -**`UIOSelector_TryRestore_Dict(inUIOSelector)`**
- -___ -Try to restore (maximize) window, if it's minimized. (!IMPORTANT! When use UIA framework minimized windows doesn't appear by the UIOSelector. You need to try restore windows and after that try to get UIO)
-___ -**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))
-___ - -**`UIOSelector_Get_UIOActivityList (inUIOSelector)`**
- -___ -Get the list of the UI object activities
-___ -**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))
-___ - -**`UIOSelectorUIOActivity_Run_Dict(inUIOSelector,inActionName,inArgumentList=\[\],inkwArgumentObject={})`**
- -___ -Run the activity in UIO (UI Object)
-___ -**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))
-**inActionName** - UIOActivity (name) from Pywinauto
-___ - -**`UIOSelector_Get_UIOInfo(inUIOSelector)`**
- -___ -Get the UIO dict of the attributes
-___ -**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))
-___ - -**`UIOSelector_GetChildList_UIOList(inUIOSelector=\[\],inBackend=mDefaultPywinautoBackend)`**
- -___ -Get list of child UIO's by Parent UIOSelector
-___ -**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))
-___ - -**`UIOSelector_SearchUIONormalize_UIOSelector (inUIOSelector)`**
- -___ -Technical def. Do UIOSelector normalization for the search processes.
-___ -**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))
-___ - -**`UIOSelector_SearchProcessNormalize_UIOSelector (inUIOSelector)`**
- -___ -Technical def. Do UIOSelector normalization for the search processes.
-___ -**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))
-___ - -**`UIOSelector_FocusHighlight(inUIOSelector)`**
- -___ -Set focus and highlight (draw outline) the element (in app) by the UIO selector.
-___ -**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))
-___ - -**`UIOSelector_Highlight(inUIOSelector)`**
- -___ -Highlight (draw outline) the element (in app) by the UIO selector.
-___ -**inUIOSelector** - UIOSelector - List of items, which contains condition attributes ([see UIOSelector Structure&Examples](#UIOSelector_Structure_Examples))
-___ \ No newline at end of file diff --git a/Wiki/05.3.-Theory-&-practice.-Keyboard-&-mouse-manipulation.md b/Wiki/05.3.-Theory-&-practice.-Keyboard-&-mouse-manipulation.md deleted file mode 100644 index cc6998e9..00000000 --- a/Wiki/05.3.-Theory-&-practice.-Keyboard-&-mouse-manipulation.md +++ /dev/null @@ -1 +0,0 @@ -**How to automate Keyboard and Mouse device in PC** \ No newline at end of file diff --git a/Wiki/05.4.-Theory-&-practice.-Screen-capture-&-image-recognition.md b/Wiki/05.4.-Theory-&-practice.-Screen-capture-&-image-recognition.md deleted file mode 100644 index d7bc5681..00000000 --- a/Wiki/05.4.-Theory-&-practice.-Screen-capture-&-image-recognition.md +++ /dev/null @@ -1,7 +0,0 @@ -**How to automate image recognition on PC** - -Here you can find any ways you need to use in your business case: -* Find the exact match on the screen with the other image -* Use text recognition module (OCR) -* Use computer vision (CV) to identify the objects on screen/image/video -* Use artificial intelligence (AI) to make custom identification/classification/text recognition \ No newline at end of file diff --git a/Wiki/ENG_Guide/html/03_Copyrights_Contacts.html b/Wiki/ENG_Guide/html/03_Copyrights_Contacts.html index 68fc0a25..6e5045f0 100644 --- a/Wiki/ENG_Guide/html/03_Copyrights_Contacts.html +++ b/Wiki/ENG_Guide/html/03_Copyrights_Contacts.html @@ -193,8 +193,9 @@ If you need IT help feel free to contact me (prefer e-mail or skype).

diff --git a/Wiki/ENG_Guide/html/Robot/01_Robot.html b/Wiki/ENG_Guide/html/Robot/01_Robot.html index bde6eb68..f811535c 100644 --- a/Wiki/ENG_Guide/html/Robot/01_Robot.html +++ b/Wiki/ENG_Guide/html/Robot/01_Robot.html @@ -260,6 +260,25 @@ Example: [{“title”:”notepad”},{“title”:”OK”}]

+
+
+pyOpenRPA.Robot.UIDesktop.UIOSelectorUIOActivity_Run_Dict(inUIOSelector, inActionName, inArgumentList=None, inkwArgumentObject=None)[source]
+

Run the activity in UIO (UI Object)

+
+
Parameters
+
    +

  • inUIOSelector – UIOSelector - List of items, which contains condition attributes

    • +

  • inActionName – UIOActivity (name) activity name string from Pywinauto

    • +

  • inArgumentList –

    • +

  • inkwArgumentObject –

    • +
+
+
Returns
+

+
+
+
+
pyOpenRPA.Robot.UIDesktop.UIOSelector_Exist_Bool(inUIOSelector)[source]
@@ -274,6 +293,37 @@ Example: [{“title”:”notepad”},{“title”:”OK”}]

+
+
+pyOpenRPA.Robot.UIDesktop.UIOSelector_FocusHighlight(inUIOSelector)[source]
+

Set focus and highlight (draw outline) the element (in app) by the UIO selector.

+
+
Parameters
+

inUIOSelector – UIOSelector - List of items, which contains condition attributes

+
+
Returns
+

+
+
+
+ +
+
+pyOpenRPA.Robot.UIDesktop.UIOSelector_GetChildList_UIOList(inUIOSelector=None, inBackend='win32')[source]
+

Get list of child UIO’s by the parent UIOSelector

+
+
Parameters
+
    +

  • inUIOSelector – UIOSelector - List of items, which contains condition attributes

    • +

  • inBackend – “win32” or “uia”

    • +
+
+
Returns
+

+
+
+
+
pyOpenRPA.Robot.UIDesktop.UIOSelector_Get_BitnessInt(inSpecificationList)[source]
@@ -320,6 +370,34 @@ Example: [{“title”:”notepad”},{“title”:”OK”}]

+
+
+pyOpenRPA.Robot.UIDesktop.UIOSelector_Get_UIOActivityList(inUIOSelector)[source]
+

Get the list of the UI object activities

+
+
Parameters
+

inUIOSelector – UIOSelector - List of items, which contains condition attributes

+
+
Returns
+

+
+
+
+ +
+
+pyOpenRPA.Robot.UIDesktop.UIOSelector_Get_UIOInfo(inUIOSelector)[source]
+

Get the UIO dict of the attributes

+
+
Parameters
+

inUIOSelector – UIOSelector - List of items, which contains condition attributes

+
+
Returns
+

+
+
+
+
pyOpenRPA.Robot.UIDesktop.UIOSelector_Get_UIOList(inSpecificationList, inElement=None, inFlagRaiseException=True)[source]
@@ -338,6 +416,20 @@ Example: [{“title”:”notepad”},{“title”:”OK”}]

+
+
+pyOpenRPA.Robot.UIDesktop.UIOSelector_Highlight(inUIOSelector)[source]
+

Highlight (draw outline) the element (in app) by the UIO selector.

+
+
Parameters
+

inUIOSelector – UIOSelector - List of items, which contains condition attributes

+
+
Returns
+

+
+
+
+
pyOpenRPA.Robot.UIDesktop.UIOSelector_SafeOtherGet_Process(inUIOSelector)[source]
@@ -382,6 +474,20 @@ Example: [{“title”:”notepad”},{“title”:”OK”}]

+
+
+pyOpenRPA.Robot.UIDesktop.UIOSelector_TryRestore_Dict(inSpecificationList)[source]
+

Try to restore (maximize) window, if it’s minimized. (!IMPORTANT! When use UIA framework minimized windows doesn’t appear by the UIOSelector. You need to try restore windows and after that try to get UIO)

+
+
Parameters
+

inSpecificationList – UIOSelector - List of items, which contains condition attributes

+
+
Returns
+

+
+
+
+
pyOpenRPA.Robot.UIDesktop.UIOSelectorsSecs_WaitAppear_List(inSpecificationListList, inWaitSecs, inFlagWaitAllInMoment=False)[source]
diff --git a/Wiki/ENG_Guide/html/Robot/02_Defs.html b/Wiki/ENG_Guide/html/Robot/02_Defs.html index 996400f2..2042b5bb 100644 --- a/Wiki/ENG_Guide/html/Robot/02_Defs.html +++ b/Wiki/ENG_Guide/html/Robot/02_Defs.html @@ -96,7 +96,13 @@
  • 1. Description
  • 2. Defs
    • @@ -183,8 +189,77 @@

    2. Defs

    -
    -

    pyOpenRPA.Robot.UIDesktop

    +
    +

    Desktop app UI access (win32 and UI automation dlls)

    +
    +

    Definitions

    +
      +

    • UIO - UI Object (class of pywinauto UI object) [pywinauto.base_wrapper]

      • +

    • UIOSelector - List of dict (key attributes)

      • +

    • PWA - PyWinAuto

      • +

    • PWASpecification - List of dict (key attributes in pywinauto.find_window notation)

      • +

    • UIOTree - Recursive Dict of Dict … (UI Parent -> Child hierarchy)

      • +

    • UIOInfo - Dict of UIO attributes

      • +

    • UIOActivity - Activity of the UIO (UI object) from the Pywinauto module

      • +

    • UIOEI - UI Object info object

      • +
    +
    +
    +

    What is UIO?

    +

    UIO is a User Interface Object (pyOpenRPA terminology). For maximum compatibility, this instance is inherited from the object model developed in the [pywinauto library (click to get a list of available class functions)](https://pywinauto.readthedocs.io/en/latest/code/pywinauto.base_wrapper.html).

    +

    This approach allows us to implement useful functionality that has already been successfully developed in other libraries, and Supplement it with the missing functionality. In our case, the missing functionality is the ability to dynamically access UIO objects using UIO selectors.

    +
    +
    +

    UIOSelector structure & example

    +

    <a name=”UIOSelector_Structure_Examples”></a> +UIOSelector is the list of condition items for the UIO in GUI. Each item has condition attributes for detect applicable UIO. Here is the description of the available condition attributes in item.

    +
    +
    **Desciption**<br>

    ```

    +
    +
    +
    +
    [
    +
    {

    “depth_start” :: [int, start from 1] :: the depth index, where to start check the condition list (default 1), +“depth_end” :: [int, start from 1] :: the depth index, where to stop check the condition list (default 1), +“ctrl_index” || “index” :: [int, starts from 0] :: the index of the UIO in parent UIO child list, +“title” :: [str] :: the condition for the UIO attribute title, +“title_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute title, +“rich_text” :: [str] :: the condition for the UIO attribute rich_text, +“rich_text_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute rich_text, +“class_name” :: [str] :: the condition for the UIO attribute class_name, +“class_name_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute class_name, +“friendly_class_name” :: [str] :: the condition for the UIO attribute friendly_class_name, +“friendly_class_name_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute friendly_class_name, +“control_type” :: [str] :: the condition for the UIO attribute control_type, +“control_type_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute control_type, +“is_enabled” :: [bool] :: the condition for the UIO attribute is_enabled. If UI object is enabled on GUI, +“is_visible” :: [bool] :: the condition for the UIO attribute is_visible. If UI object is visible on GUI, +“backend” :: [str, “win32” || “uia”] :: the method of UIO extraction (default “win32”). ATTENTION! Current option can be only for the first item of the UIO selector. For the next items this option will be implemented from the first item.

    +
    +
    +

    }, +{ … specification next level UIO }

    +
    +
    +
    +
    ]

    ```

    +
    +
    The UIO selector example

    ```

    +
    +
    +
    +
    [

    {“class_name”:”CalcFrame”, “backend”:”win32”}, # 1-st level UIO specification +{“title”:”Hex”, “depth_start”:3, “depth_end”: 3} # 3-rd level specification (because of attribute depth_start|depth_stop)

    +
    +
    +
    +
    ]

    ```

    +
    +
    +
    +
    +

    The UIDesktop module (OpenRPA/Robot/UIDesktop.py)

    +

    The UIDesktop is extension of the pywinauto module which provide access to the desktop apps by the win32 and ui automation dll frameworks (big thx to the Microsoft :) ).

    # EXAMPLE 1
 from pyOpenRPA.Robot import UIDesktop
 
@@ -217,21 +292,39 @@
 
    UIOSelectorSecs_WaitDisappear_Bool(…)
    
 
    Wait for UI object will disappear in GUI for inWaitSecs seconds.
    
 
-
    UIOSelector_Exist_Bool(inUIOSelector)
    
+
    UIOSelectorUIOActivity_Run_Dict(…[, …])
    
+
    Run the activity in UIO (UI Object)
    
+
+
    UIOSelector_Exist_Bool(inUIOSelector)
    
 
    Check if object is exist by the UIO selector.
    
 
-
    UIOSelector_Get_BitnessInt(inSpecificationList)
    
+
    UIOSelector_FocusHighlight(inUIOSelector)
    
+
    Set focus and highlight (draw outline) the element (in app) by the UIO selector.
    
+
+
    UIOSelector_GetChildList_UIOList([…])
    
+
    Get list of child UIO’s by the parent UIOSelector
    
+
+
    UIOSelector_Get_BitnessInt(inSpecificationList)
    
 
    Detect process bitness by the UI Object UIO Selector.
    
 
-
    UIOSelector_Get_BitnessStr(inSpecificationList)
    
+
    UIOSelector_Get_BitnessStr(inSpecificationList)
    
 
    Detect process bitness by the UI Object UIO Selector.
    
 
-
    UIOSelector_Get_UIO(inSpecificationList[, …])
    
+
    UIOSelector_Get_UIO(inSpecificationList[, …])
    
 
    Get the pywinauto object by the UIO selector.
    
 
-
    UIOSelector_Get_UIOList(inSpecificationList)
    
+
    UIOSelector_Get_UIOActivityList(inUIOSelector)
    
+
    Get the list of the UI object activities
    
+
+
    UIOSelector_Get_UIOInfo(inUIOSelector)
    
+
    Get the UIO dict of the attributes
    
+
+
    UIOSelector_Get_UIOList(inSpecificationList)
    
 
    Get the UIO list by the selector
    
 
+
    UIOSelector_Highlight(inUIOSelector)
    
+
    Highlight (draw outline) the element (in app) by the UIO selector.
    
+
 
    UIOSelector_SafeOtherGet_Process(inUIOSelector)
    
 
    Safe get other process or None if destination app is the other/same bitness
    
 
@@ -241,10 +334,13 @@
 
    UIOSelector_SearchChildByMouse_UIOTree(…)
    
 
    !!!!Safe call is included (you can set activity and UIDesktop will choose the bitness and return the result)!!!!!
    
 
-
    UIOSelectorsSecs_WaitAppear_List(…[, …])
    
+
    UIOSelector_TryRestore_Dict(inSpecificationList)
    
+
    Try to restore (maximize) window, if it’s minimized.
    
+
+
    UIOSelectorsSecs_WaitAppear_List(…[, …])
    
 
    Wait for many UI object will appear in GUI for inWaitSecs seconds.
    
 
-
    UIOSelectorsSecs_WaitDisappear_List(…[, …])
    
+
    UIOSelectorsSecs_WaitDisappear_List(…[, …])
    
 
    Wait for many UI object will disappear in GUI for inWaitSecs seconds.
    
 
 
@@ -324,6 +420,25 @@ Example: [{“title”:”notepad”},{“title”:”OK”}]
+
+
+pyOpenRPA.Robot.UIDesktop.UIOSelectorUIOActivity_Run_Dict(inUIOSelector, inActionName, inArgumentList=None, inkwArgumentObject=None)[source]
+

Run the activity in UIO (UI Object)

+
+
Parameters
+
    +

  • inUIOSelector – UIOSelector - List of items, which contains condition attributes

    • +

  • inActionName – UIOActivity (name) activity name string from Pywinauto

    • +

  • inArgumentList –

    • +

  • inkwArgumentObject –

    • +
+
+
Returns
+

+
+
+
+
pyOpenRPA.Robot.UIDesktop.UIOSelector_Exist_Bool(inUIOSelector)[source]
@@ -338,6 +453,37 @@ Example: [{“title”:”notepad”},{“title”:”OK”}]

+
+
+pyOpenRPA.Robot.UIDesktop.UIOSelector_FocusHighlight(inUIOSelector)[source]
+

Set focus and highlight (draw outline) the element (in app) by the UIO selector.

+
+
Parameters
+

inUIOSelector – UIOSelector - List of items, which contains condition attributes

+
+
Returns
+

+
+
+
+ +
+
+pyOpenRPA.Robot.UIDesktop.UIOSelector_GetChildList_UIOList(inUIOSelector=None, inBackend='win32')[source]
+

Get list of child UIO’s by the parent UIOSelector

+
+
Parameters
+
    +

  • inUIOSelector – UIOSelector - List of items, which contains condition attributes

    • +

  • inBackend – “win32” or “uia”

    • +
+
+
Returns
+

+
+
+
+
pyOpenRPA.Robot.UIDesktop.UIOSelector_Get_BitnessInt(inSpecificationList)[source]
@@ -384,6 +530,34 @@ Example: [{“title”:”notepad”},{“title”:”OK”}]

+
+
+pyOpenRPA.Robot.UIDesktop.UIOSelector_Get_UIOActivityList(inUIOSelector)[source]
+

Get the list of the UI object activities

+
+
Parameters
+

inUIOSelector – UIOSelector - List of items, which contains condition attributes

+
+
Returns
+

+
+
+
+ +
+
+pyOpenRPA.Robot.UIDesktop.UIOSelector_Get_UIOInfo(inUIOSelector)[source]
+

Get the UIO dict of the attributes

+
+
Parameters
+

inUIOSelector – UIOSelector - List of items, which contains condition attributes

+
+
Returns
+

+
+
+
+
pyOpenRPA.Robot.UIDesktop.UIOSelector_Get_UIOList(inSpecificationList, inElement=None, inFlagRaiseException=True)[source]
@@ -402,6 +576,20 @@ Example: [{“title”:”notepad”},{“title”:”OK”}]

+
+
+pyOpenRPA.Robot.UIDesktop.UIOSelector_Highlight(inUIOSelector)[source]
+

Highlight (draw outline) the element (in app) by the UIO selector.

+
+
Parameters
+

inUIOSelector – UIOSelector - List of items, which contains condition attributes

+
+
Returns
+

+
+
+
+
pyOpenRPA.Robot.UIDesktop.UIOSelector_SafeOtherGet_Process(inUIOSelector)[source]
@@ -446,6 +634,20 @@ Example: [{“title”:”notepad”},{“title”:”OK”}]

+
+
+pyOpenRPA.Robot.UIDesktop.UIOSelector_TryRestore_Dict(inSpecificationList)[source]
+

Try to restore (maximize) window, if it’s minimized. (!IMPORTANT! When use UIA framework minimized windows doesn’t appear by the UIOSelector. You need to try restore windows and after that try to get UIO)

+
+
Parameters
+

inSpecificationList – UIOSelector - List of items, which contains condition attributes

+
+
Returns
+

+
+
+
+
pyOpenRPA.Robot.UIDesktop.UIOSelectorsSecs_WaitAppear_List(inSpecificationListList, inWaitSecs, inFlagWaitAllInMoment=False)[source]
@@ -499,12 +701,13 @@ Example: [

+

References

-

reStructuredText 1

+

reStructuredText 1

-
1
+
1

http://docutils.sourceforge.net/rst.html

diff --git a/Wiki/ENG_Guide/html/Robot/03_HowToUse.html b/Wiki/ENG_Guide/html/Robot/03_HowToUse.html index 8e67620a..cedd22e7 100644 --- a/Wiki/ENG_Guide/html/Robot/03_HowToUse.html +++ b/Wiki/ENG_Guide/html/Robot/03_HowToUse.html @@ -97,19 +97,32 @@
  • 1. Description
  • 2. Defs
  • 3. How to use
    • -

    Execute python script

    +

    Execute python script

    The OpenRPA is fully portable solution. It contains own python enviroment both 32 and 64 bit versions. So, you can execute your python script in several ways: - Execute in python x32 (OpenRPAResourcesWPy32-3720python-3.7.2) - Execute in python x64 (OpenRPAResourcesWPy64-3720python-3.7.2.amd64) - Execute from .cmd file

    +

    Execute in the Python x32

    To execute your python script in x32 bit version just write in command line from x32 python directory:

    @@ -260,9 +261,8 @@
    -
    -

    Use in studio script (n/a)

    +

    Use in studio script (n/a)

    import sys
 sys.path.append('../../')
 import GUI
@@ -270,17 +270,131 @@
 import subprocess
 import time
 
-#Highlight the UI Object in Folder explorer<br>
-GUI.UIOSelector_FocusHighlight([{"class_name":"CabinetWClass","backend":"uia"},{"ctrl_index":2},{"ctrl_index":0},{"ctrl_index":2},{"ctrl_index":0}])<br>
+#Highlight the UI Object in Folder explorer
+GUI.UIOSelector_FocusHighlight([{"class_name":"CabinetWClass","backend":"uia"},{"ctrl_index":2},{"ctrl_index":0},{"ctrl_index":2},{"ctrl_index":0}])
 
-#Wait 2 seconds<br>
-time.sleep(3)<br>
+#Wait 2 seconds
+time.sleep(3)
 
-#Loop: get child element of UI List<br>
-for lItem in GUI.UIOSelector_Get_UIO([{"class_name":"CabinetWClass","backend":"uia"},{"ctrl_index":2},{"ctrl_index":0},{"ctrl_index":2},{"ctrl_index":0}]).children():<br>
-  &nbsp; &nbsp; &nbsp; &nbsp; print(str(lItem))
+#Loop: get child element of UI List
+for lItem in GUI.UIOSelector_Get_UIO([{"class_name":"CabinetWClass","backend":"uia"},{"ctrl_index":2},{"ctrl_index":0},{"ctrl_index":2},{"ctrl_index":0}]).children():
+    print(str(lItem))
    +

    Here you can find the docs and examples of the OpenRPA desktop (GUI) app access.

    +
    + +
    +

    Theory & practice. Desktop app UI access (win32 and UI automation dlls)

    +
    +

    Definitions

    +

    UIO - UI Object (class of pywinauto UI object) [pywinauto.base_wrapper]<br> +UIOSelector - List of dict (key attributes)<br> +PWA - PyWinAuto<br> +PWASpecification - List of dict (key attributes in pywinauto.find_window notation)<br> +UIOTree - Recursive Dict of Dict … (UI Parent -> Child hierarchy)<br> +UIOInfo - Dict of UIO attributes<br> +UIOActivity - Activity of the UIO (UI object) from the Pywinauto module<br> +UIOEI - UI Object info object

    +
    +
    +

    What is UIO?

    +

    UIO is a User Interface Object (pyOpenRPA terminology). For maximum compatibility, this instance is inherited from the object model developed in the [pywinauto library (click to get a list of available class functions)](https://pywinauto.readthedocs.io/en/latest/code/pywinauto.base_wrapper.html).

    +

    This approach allows us to implement useful functionality that has already been successfully developed in other libraries, and Supplement it with the missing functionality. In our case, the missing functionality is the ability to dynamically access UIO objects using UIO selectors.

    +
    +
    +

    UIOSelector structure & example

    +

    <a name=”UIOSelector_Structure_Examples”></a> +UIOSelector is the list of condition items for the UIO in GUI. Each item has condition attributes for detect applicable UIO. Here is the description of the available condition attributes in item.

    +
    +
    Desciption

    ```

    +
    +
    +
    +
    [
    +
    {

    “depth_start” :: [int, start from 1] :: the depth index, where to start check the condition list (default 1), +“depth_end” :: [int, start from 1] :: the depth index, where to stop check the condition list (default 1), +“ctrl_index” || “index” :: [int, starts from 0] :: the index of the UIO in parent UIO child list, +“title” :: [str] :: the condition for the UIO attribute title, +“title_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute title, +“rich_text” :: [str] :: the condition for the UIO attribute rich_text, +“rich_text_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute rich_text, +“class_name” :: [str] :: the condition for the UIO attribute class_name, +“class_name_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute class_name, +“friendly_class_name” :: [str] :: the condition for the UIO attribute friendly_class_name, +“friendly_class_name_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute friendly_class_name, +“control_type” :: [str] :: the condition for the UIO attribute control_type, +“control_type_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute control_type, +“is_enabled” :: [bool] :: the condition for the UIO attribute is_enabled. If UI object is enabled on GUI, +“is_visible” :: [bool] :: the condition for the UIO attribute is_visible. If UI object is visible on GUI, +“backend” :: [str, “win32” || “uia”] :: the method of UIO extraction (default “win32”). ATTENTION! Current option can be only for the first item of the UIO selector. For the next items this option will be implemented from the first item.

    +
    +
    +

    }, +{ … specification next level UIO }

    +
    +
    +
    +
    ]

    ```

    +
    +
    The UIO selector example

    ```

    +
    +
    +
    +
    [

    {“class_name”:”CalcFrame”, “backend”:”win32”}, # 1-st level UIO specification +{“title”:”Hex”, “depth_start”:3, “depth_end”: 3} # 3-rd level specification (because of attribute depth_start|depth_stop)

    +
    +
    +
    +
    ]

    ```

    +
    +
    +
    +
    +

    The UIDesktop module (OpenRPA/Robot/UIDesktop.py)

    +

    The UIDesktop is extension of the pywinauto module which provide access to the desktop apps by the win32 and ui automation dll frameworks (big thx to the Microsoft :) ).

    +

    *Naming convention: <InArgument>_<ActivityName>_<OutArgument - if exist>*<br>

    +
    +
    +
    +

    Theory & practice. WEB app UI access (selenium)

    +
    +

    About

    +

    The pyOpenRPA support web app manipulation (by the Selenium lib). +More docs about selenium you can find here (https://selenium-python.readthedocs.io/)

    +
    +
    +

    How to use

    +

    To start use selenium just import selenium modules in the robot tool. Here is the example of the usage.

    +
    from selenium import webdriver
+from selenium.webdriver.common.keys import Keys
+
+driver = webdriver.Chrome()
+driver.get("http://www.python.org")
+assert "Python" in driver.title
+elem = driver.find_element_by_name("q")
+elem.clear()
+elem.send_keys("pycon")
+elem.send_keys(Keys.RETURN)
+assert "No results found." not in driver.page_source
+driver.close()
+
    +
    +
    +
    +
    +

    Theory & practice. Keyboard & mouse manipulation

    +
    +
    +

    Theory & practice. Screen capture & image recognition

    +
    +

    How to automate image recognition on PC

    +

    Here you can find any ways you need to use in your business case: +- Find the exact match on the screen with the other image +- Use text recognition module (OCR) +- Use computer vision (CV) to identify the objects on screen/image/video +- Use artificial intelligence (AI) to make custom identification/classification/text recognition

    +
    diff --git a/Wiki/ENG_Guide/html/_modules/pyOpenRPA/Robot/UIDesktop.html b/Wiki/ENG_Guide/html/_modules/pyOpenRPA/Robot/UIDesktop.html index b1b90a2c..07331fb6 100644 --- a/Wiki/ENG_Guide/html/_modules/pyOpenRPA/Robot/UIDesktop.html +++ b/Wiki/ENG_Guide/html/_modules/pyOpenRPA/Robot/UIDesktop.html @@ -957,7 +957,13 @@ #(особенность uia backend - он не может прицепиться к окну, если оно свернуто) #inSpecificationList - UIOSelector #old name - PywinautoExtTryToRestore -def UIOSelector_TryRestore_Dict(inSpecificationList): +
    [docs]def UIOSelector_TryRestore_Dict(inSpecificationList): + """ + Try to restore (maximize) window, if it's minimized. (!IMPORTANT! When use UIA framework minimized windows doesn't appear by the UIOSelector. You need to try restore windows and after that try to get UIO) + + :param inSpecificationList: UIOSelector - List of items, which contains condition attributes + :return: + """ lResult={} try: #Подготовка взодного массива @@ -969,13 +975,19 @@ lRPAApplication.top_window().restore() except Exception: True==False - return lResult + return lResult
    #################################################################################################### #Get the list of the UI object activities #!!!!!Safe call is included (you can set activity and UIDesktop will choose the bitness and return the result)!!!!! #inControlSpecificationArray - UIOSelector #old name - ElementActionGetList -def UIOSelector_Get_UIOActivityList (inUIOSelector): +
    [docs]def UIOSelector_Get_UIOActivityList (inUIOSelector): + """ + Get the list of the UI object activities + + :param inUIOSelector: UIOSelector - List of items, which contains condition attributes + :return: + """ #Check the bitness lSafeOtherProcess = UIOSelector_SafeOtherGet_Process(inUIOSelector) if lSafeOtherProcess is None: @@ -1005,7 +1017,7 @@ f"Exception was occured in child process (message): {lPIPEResponseDict['ErrorMessage']}, (traceback): {lPIPEResponseDict['ErrorTraceback']}") else: lResult = lPIPEResponseDict["Result"] - return lResult + return lResult
    #################################################################################################### #Run the activity in UIO (UI Object) @@ -1013,7 +1025,16 @@ #inUIOSelector #inActionName - UIOActivity (name) from Pywinauto #old name - ElementRunAction -def UIOSelectorUIOActivity_Run_Dict(inUIOSelector, inActionName, inArgumentList=None, inkwArgumentObject=None): +
    [docs]def UIOSelectorUIOActivity_Run_Dict(inUIOSelector, inActionName, inArgumentList=None, inkwArgumentObject=None): + """ + Run the activity in UIO (UI Object) + + :param inUIOSelector: UIOSelector - List of items, which contains condition attributes + :param inActionName: UIOActivity (name) activity name string from Pywinauto + :param inArgumentList: + :param inkwArgumentObject: + :return: + """ if inArgumentList is None: inArgumentList=[] # 2021 02 22 Minor fix by Ivan Maslov if inkwArgumentObject is None: inkwArgumentObject={} # 2021 02 22 Minor fix by Ivan Maslov lResult={} @@ -1056,13 +1077,19 @@ raise Exception(f"Exception was occured in child process (message): {lPIPEResponseDict['ErrorMessage']}, (traceback): {lPIPEResponseDict['ErrorTraceback']}") else: lResult = lPIPEResponseDict["Result"] - return lResult + return lResult
    #################################################################################################### #Get the UIO dict of the attributes #!!!!!Safe call is included (you can set activity and UIDesktop will choose the bitness and return the result)!!!!! #old name - ElementGetInfo -def UIOSelector_Get_UIOInfo(inUIOSelector): +
    [docs]def UIOSelector_Get_UIOInfo(inUIOSelector): + """ + Get the UIO dict of the attributes + + :param inUIOSelector: UIOSelector - List of items, which contains condition attributes + :return: + """ #Check the bitness lSafeOtherProcess = UIOSelector_SafeOtherGet_Process(inUIOSelector) if lSafeOtherProcess is None: @@ -1091,7 +1118,7 @@ f"Exception was occured in child process (message): {lPIPEResponseDict['ErrorMessage']}, (traceback): {lPIPEResponseDict['ErrorTraceback']}") else: lResultList = lPIPEResponseDict["Result"] - return lResultList + return lResultList
    #################################################################################################### #Search child UIO by the: Parent UIO, X, Y #inHierarchyList: [{"index":<>,"element":<>}] - technical argument for internal purpose @@ -1187,7 +1214,16 @@ #!!!!!Safe call is included (you can set activity and UIDesktop will choose the bitness and return the result)!!!!! #inControlSpecificationArray- UIOSelector #old name - ElementGetChildElementList -def UIOSelector_GetChildList_UIOList(inUIOSelector=None, inBackend=mDefaultPywinautoBackend): +
    [docs]def UIOSelector_GetChildList_UIOList(inUIOSelector=None, inBackend=mDefaultPywinautoBackend): + """ + Get list of child UIO's by the parent UIOSelector + + :param inUIOSelector: UIOSelector - List of items, which contains condition attributes + :param inBackend: "win32" or "uia" + :return: + """ + + if inUIOSelector is None: inUIOSelector = [] #mRobotLogger.info(f"File!!!!") #mRobotLogger.info(f"inSelector:{str(inUIOSelector)}, inBackend:{str(inBackend)}") @@ -1235,7 +1271,7 @@ f"Exception was occured in child process (message): {lPIPEResponseDict['ErrorMessage']}, (traceback): {lPIPEResponseDict['ErrorTraceback']}") else: lResultList = lPIPEResponseDict["Result"] - return lResultList + return lResultList
    #################################################################################################### #Подготовить массив для обращшения к поиску элемементов @@ -1476,7 +1512,13 @@ #Highlight the UI object #!!!!!Safe call is included (you can set activity and UIDesktop will choose the bitness and return the result)!!!!! #old name - ElementDrawOutlineNew -def UIOSelector_Highlight(inUIOSelector): +
    [docs]def UIOSelector_Highlight(inUIOSelector): + """ + Highlight (draw outline) the element (in app) by the UIO selector. + + :param inUIOSelector: UIOSelector - List of items, which contains condition attributes + :return: + """ #Check the bitness lSafeOtherProcess = UIOSelector_SafeOtherGet_Process(inUIOSelector) if lSafeOtherProcess is None: @@ -1495,13 +1537,20 @@ f"Exception was occured in child process (message): {lPIPEResponseDict['ErrorMessage']}, (traceback): {lPIPEResponseDict['ErrorTraceback']}") else: return lPIPEResponseDict["Result"] - return True + return True
    ################################################################################################### #inSpecificationArray - UIOSelector #!!!!!Safe call is included (you can set activity and UIDesktop will choose the bitness and return the result)!!!!! #old name - ElementDrawOutlineNewFocus -def UIOSelector_FocusHighlight(inUIOSelector): +
    [docs]def UIOSelector_FocusHighlight(inUIOSelector): + """ + Set focus and highlight (draw outline) the element (in app) by the UIO selector. + + :param inUIOSelector: UIOSelector - List of items, which contains condition attributes + :return: + """ + #Check the bitness lSafeOtherProcess = UIOSelector_SafeOtherGet_Process(inUIOSelector) if lSafeOtherProcess is None: @@ -1520,7 +1569,7 @@ f"Exception was occured in child process (message): {lPIPEResponseDict['ErrorMessage']}, (traceback): {lPIPEResponseDict['ErrorTraceback']}") else: return lPIPEResponseDict["Result"] - return True + return True
    ################################################################################################### #old name - draw_outline_new diff --git a/Wiki/ENG_Guide/html/_sources/03_Copyrights_Contacts.rst.txt b/Wiki/ENG_Guide/html/_sources/03_Copyrights_Contacts.rst.txt index b9f01159..72920e6c 100644 --- a/Wiki/ENG_Guide/html/_sources/03_Copyrights_Contacts.rst.txt +++ b/Wiki/ENG_Guide/html/_sources/03_Copyrights_Contacts.rst.txt @@ -17,10 +17,12 @@ Ivan Maslov (founder) - Skype: MegaFinder -- Facebook: https://www.facebook.com/RU.Ivan.Maslov +- Facebook: https://www.facebook.com/RU.IT4Business - LinkedIn: https://www.linkedin.com/in/RU-IvanMaslov/ +- WhatsApp | Telegram: +7 906 722 39 25 + ************************************************** 3-rd party components license dependencies diff --git a/Wiki/ENG_Guide/html/_sources/Robot/02_Defs.rst.txt b/Wiki/ENG_Guide/html/_sources/Robot/02_Defs.rst.txt index 533d2155..54d02d33 100644 --- a/Wiki/ENG_Guide/html/_sources/Robot/02_Defs.rst.txt +++ b/Wiki/ENG_Guide/html/_sources/Robot/02_Defs.rst.txt @@ -2,9 +2,69 @@ 2. Defs #################################### -************************************************** -pyOpenRPA.Robot.UIDesktop -************************************************** +**************************************************************************************************** +Desktop app UI access (win32 and UI automation dlls) +**************************************************************************************************** + +Definitions +############################################ + +- **UIO** - UI Object (class of pywinauto UI object) [pywinauto.base_wrapper] +- **UIOSelector** - List of dict (key attributes) +- **PWA** - PyWinAuto +- **PWASpecification** - List of dict (key attributes in pywinauto.find_window notation) +- **UIOTree** - Recursive Dict of Dict ... (UI Parent -> Child hierarchy) +- **UIOInfo** - Dict of UIO attributes +- **UIOActivity** - Activity of the UIO (UI object) from the Pywinauto module +- **UIOEI** - UI Object info object + + +What is UIO? +############################################ +UIO is a User Interface Object (pyOpenRPA terminology). For maximum compatibility, this instance is inherited from the object model developed in the [pywinauto library (click to get a list of available class functions)](https://pywinauto.readthedocs.io/en/latest/code/pywinauto.base_wrapper.html). + +This approach allows us to implement useful functionality that has already been successfully developed in other libraries, and Supplement it with the missing functionality. In our case, the missing functionality is the ability to dynamically access UIO objects using UIO selectors. + + +UIOSelector structure & example +############################################ + +UIOSelector is the list of condition items for the UIO in GUI. Each item has condition attributes for detect applicable UIO. Here is the description of the available condition attributes in item. + +**Desciption**
    + ``` +[ + { + "depth_start" :: [int, start from 1] :: the depth index, where to start check the condition list (default 1), + "depth_end" :: [int, start from 1] :: the depth index, where to stop check the condition list (default 1), + "ctrl_index" || "index" :: [int, starts from 0] :: the index of the UIO in parent UIO child list, + "title" :: [str] :: the condition for the UIO attribute *title*, + "title_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *title*, + "rich_text" :: [str] :: the condition for the UIO attribute *rich_text*, + "rich_text_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *rich_text*, + "class_name" :: [str] :: the condition for the UIO attribute *class_name*, + "class_name_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *class_name*, + "friendly_class_name" :: [str] :: the condition for the UIO attribute *friendly_class_name*, + "friendly_class_name_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *friendly_class_name*, + "control_type" :: [str] :: the condition for the UIO attribute *control_type*, + "control_type_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *control_type*, + "is_enabled" :: [bool] :: the condition for the UIO attribute *is_enabled*. If UI object is enabled on GUI, + "is_visible" :: [bool] :: the condition for the UIO attribute *is_visible*. If UI object is visible on GUI, + "backend" :: [str, "win32" || "uia"] :: the method of UIO extraction (default "win32"). ATTENTION! Current option can be only for the first item of the UIO selector. For the next items this option will be implemented from the first item. + }, + { ... specification next level UIO } +] + ``` +**The UIO selector example** + ``` +[ + {"class_name":"CalcFrame", "backend":"win32"}, # 1-st level UIO specification + {"title":"Hex", "depth_start":3, "depth_end": 3} # 3-rd level specification (because of attribute depth_start|depth_stop) +] + ``` +The UIDesktop module (OpenRPA/Robot/UIDesktop.py) +######################################################################################## +The UIDesktop is extension of the pywinauto module which provide access to the desktop apps by the **win32** and **ui automation** dll frameworks (big thx to the Microsoft :) ). .. code-block:: python diff --git a/Wiki/ENG_Guide/html/_sources/Robot/03_HowToUse.rst.txt b/Wiki/ENG_Guide/html/_sources/Robot/03_HowToUse.rst.txt index bc4ec41a..70ef27b1 100644 --- a/Wiki/ENG_Guide/html/_sources/Robot/03_HowToUse.rst.txt +++ b/Wiki/ENG_Guide/html/_sources/Robot/03_HowToUse.rst.txt @@ -2,21 +2,10 @@ 3. How to use #################################### -************************************************** -Content -************************************************** -- `About <#about>`__ -- `How to use <#way-to-use>`__ -- `Create python script <#create-python-script>`__ -- `Execute python script <#execute-python-script>`__ - -************************************************** -About -************************************************** The Robot tool is the main module for production process automation. It has no graphic/console interface. All low-level actions to OS are perfoming by the Robot tool in OpenRPA. ************************************************** -Way to use +How to execute RPA script ************************************************** You can use the robot by the several ways: @@ -39,9 +28,8 @@ In order to use robot just add Robot tool folder in work directory and add line import cv2 # [Computer vision](https://gitlab.com/UnicodeLabs/OpenRPA/wikis/05.4.-Theory-&-practice:-Screen-capture-&-image-recognition)
    import keyboard #[Keyboard manipulation](https://gitlab.com/UnicodeLabs/OpenRPA/wikis/05.3.-Theory-&-practice:-Keyboard-&-mouse-manipulation)
    -************************************************** Execute python script -************************************************** +############################################ The OpenRPA is fully portable solution. It contains own python enviroment both 32 and 64 bit versions. So, you can execute your python script in several ways: - Execute in python x32 (\OpenRPA\Resources\WPy32-3720\python-3.7.2) @@ -80,9 +68,9 @@ In order to simplify the execution process you can write several code lines in f .\..\Resources\WPy32-3720\python-3.7.2\OpenRPAOrchestrator.exe orchestratorMain.py pause >nul -************************************************** + Use in studio script (n/a) -************************************************** +############################################ .. code-block:: python import sys @@ -92,12 +80,129 @@ Use in studio script (n/a) import subprocess import time - #Highlight the UI Object in Folder explorer
    - GUI.UIOSelector_FocusHighlight([{"class_name":"CabinetWClass","backend":"uia"},{"ctrl_index":2},{"ctrl_index":0},{"ctrl_index":2},{"ctrl_index":0}])
    + #Highlight the UI Object in Folder explorer + GUI.UIOSelector_FocusHighlight([{"class_name":"CabinetWClass","backend":"uia"},{"ctrl_index":2},{"ctrl_index":0},{"ctrl_index":2},{"ctrl_index":0}]) + + #Wait 2 seconds + time.sleep(3) + + #Loop: get child element of UI List + for lItem in GUI.UIOSelector_Get_UIO([{"class_name":"CabinetWClass","backend":"uia"},{"ctrl_index":2},{"ctrl_index":0},{"ctrl_index":2},{"ctrl_index":0}]).children(): + print(str(lItem)) + + + +Here you can find the docs and examples of the OpenRPA desktop (GUI) app access. + +**************************************************************************************************** +Theory & practice. Desktop app UI access (win32 and UI automation dlls) +**************************************************************************************************** + +Definitions +############################################ + +**UIO** - UI Object (class of pywinauto UI object) [pywinauto.base_wrapper]
    +**UIOSelector** - List of dict (key attributes)
    +**PWA** - PyWinAuto
    +**PWASpecification** - List of dict (key attributes in pywinauto.find_window notation)
    +**UIOTree** - Recursive Dict of Dict ... (UI Parent -> Child hierarchy)
    +**UIOInfo** - Dict of UIO attributes
    +**UIOActivity** - Activity of the UIO (UI object) from the Pywinauto module
    +**UIOEI** - UI Object info object + + +What is UIO? +############################################ +UIO is a User Interface Object (pyOpenRPA terminology). For maximum compatibility, this instance is inherited from the object model developed in the [pywinauto library (click to get a list of available class functions)](https://pywinauto.readthedocs.io/en/latest/code/pywinauto.base_wrapper.html). + +This approach allows us to implement useful functionality that has already been successfully developed in other libraries, and Supplement it with the missing functionality. In our case, the missing functionality is the ability to dynamically access UIO objects using UIO selectors. + + +UIOSelector structure & example +############################################ + +UIOSelector is the list of condition items for the UIO in GUI. Each item has condition attributes for detect applicable UIO. Here is the description of the available condition attributes in item. + +**Desciption** + ``` +[ + { + "depth_start" :: [int, start from 1] :: the depth index, where to start check the condition list (default 1), + "depth_end" :: [int, start from 1] :: the depth index, where to stop check the condition list (default 1), + "ctrl_index" || "index" :: [int, starts from 0] :: the index of the UIO in parent UIO child list, + "title" :: [str] :: the condition for the UIO attribute *title*, + "title_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *title*, + "rich_text" :: [str] :: the condition for the UIO attribute *rich_text*, + "rich_text_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *rich_text*, + "class_name" :: [str] :: the condition for the UIO attribute *class_name*, + "class_name_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *class_name*, + "friendly_class_name" :: [str] :: the condition for the UIO attribute *friendly_class_name*, + "friendly_class_name_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *friendly_class_name*, + "control_type" :: [str] :: the condition for the UIO attribute *control_type*, + "control_type_re" :: [str] :: regular expression (python ver) for the condition for the UIO attribute *control_type*, + "is_enabled" :: [bool] :: the condition for the UIO attribute *is_enabled*. If UI object is enabled on GUI, + "is_visible" :: [bool] :: the condition for the UIO attribute *is_visible*. If UI object is visible on GUI, + "backend" :: [str, "win32" || "uia"] :: the method of UIO extraction (default "win32"). ATTENTION! Current option can be only for the first item of the UIO selector. For the next items this option will be implemented from the first item. + }, + { ... specification next level UIO } +] + ``` +**The UIO selector example** + ``` +[ + {"class_name":"CalcFrame", "backend":"win32"}, # 1-st level UIO specification + {"title":"Hex", "depth_start":3, "depth_end": 3} # 3-rd level specification (because of attribute depth_start|depth_stop) +] + ``` +The UIDesktop module (OpenRPA/Robot/UIDesktop.py) +######################################################################################## +The UIDesktop is extension of the pywinauto module which provide access to the desktop apps by the **win32** and **ui automation** dll frameworks (big thx to the Microsoft :) ). + +*Naming convention: \\_\\_\*
    + + +**************************************************************************************************** +Theory & practice. WEB app UI access (selenium) +**************************************************************************************************** + +About +############################################### +The pyOpenRPA support web app manipulation (by the Selenium lib). +More docs about selenium you can find here (https://selenium-python.readthedocs.io/) + +How to use +############################################### +To start use selenium just import selenium modules in the robot tool. Here is the example of the usage. + +.. code-block:: python + + from selenium import webdriver + from selenium.webdriver.common.keys import Keys + + driver = webdriver.Chrome() + driver.get("http://www.python.org") + assert "Python" in driver.title + elem = driver.find_element_by_name("q") + elem.clear() + elem.send_keys("pycon") + elem.send_keys(Keys.RETURN) + assert "No results found." not in driver.page_source + driver.close() + +**************************************************************************************************** +Theory & practice. Keyboard & mouse manipulation +**************************************************************************************************** + + +**************************************************************************************************** +Theory & practice. Screen capture & image recognition +**************************************************************************************************** - #Wait 2 seconds
    - time.sleep(3)
    +How to automate image recognition on PC +########################################### - #Loop: get child element of UI List
    - for lItem in GUI.UIOSelector_Get_UIO([{"class_name":"CabinetWClass","backend":"uia"},{"ctrl_index":2},{"ctrl_index":0},{"ctrl_index":2},{"ctrl_index":0}]).children():
    -         print(str(lItem)) +Here you can find any ways you need to use in your business case: +- Find the exact match on the screen with the other image +- Use text recognition module (OCR) +- Use computer vision (CV) to identify the objects on screen/image/video +- Use artificial intelligence (AI) to make custom identification/classification/text recognition \ No newline at end of file diff --git a/Wiki/ENG_Guide/html/genindex.html b/Wiki/ENG_Guide/html/genindex.html index 326a3c00..2121296e 100644 --- a/Wiki/ENG_Guide/html/genindex.html +++ b/Wiki/ENG_Guide/html/genindex.html @@ -353,22 +353,34 @@
  • UACUpdate() (in module pyOpenRPA.Orchestrator.__Orchestrator__)
  • UIOSelector_Exist_Bool() (in module pyOpenRPA.Robot.UIDesktop), [1] +
    • +
  • UIOSelector_FocusHighlight() (in module pyOpenRPA.Robot.UIDesktop), [1]
  • UIOSelector_Get_BitnessInt() (in module pyOpenRPA.Robot.UIDesktop), [1]
  • UIOSelector_Get_BitnessStr() (in module pyOpenRPA.Robot.UIDesktop), [1]
  • UIOSelector_Get_UIO() (in module pyOpenRPA.Robot.UIDesktop), [1] +
    • +
  • UIOSelector_Get_UIOActivityList() (in module pyOpenRPA.Robot.UIDesktop), [1] +
    • +
  • UIOSelector_Get_UIOInfo() (in module pyOpenRPA.Robot.UIDesktop), [1] +
    • +
  • UIOSelector_Get_UIOList() (in module pyOpenRPA.Robot.UIDesktop), [1]
    • diff --git a/Wiki/ENG_Guide/html/index.html b/Wiki/ENG_Guide/html/index.html index 57270cd8..a747eba2 100644 --- a/Wiki/ENG_Guide/html/index.html +++ b/Wiki/ENG_Guide/html/index.html @@ -287,16 +287,16 @@ in QUEUE

  • 2. Defs
  • 3. How to use
  • 4. Dependencies
    • diff --git a/Wiki/ENG_Guide/html/objects.inv b/Wiki/ENG_Guide/html/objects.inv index 1e1d8600e92c16b82e955d4dc462e36ceab50003..3f44f1653112162864344fcd87041103a16994e9 100644 GIT binary patch delta 1117 zcmV-j1fu(h3C0PKcz;>VZsRr(zUwIjbkSSfI0<^#n}50+qj7Cm_WCF=(%2?UlLATY zMz_93U$0NHL+i(qt7OwwqeBpq!}&gv8qN=$WQzpkYB?G#baVh?HM2?&!m!>ALqEwr zo>ZI)@NfOP=$G5Oa0)v!IM|c;DW7ul!N2j6_W=(^dysY%S%3bO+iYgxG>T)#wo4@r zD(Mg3;QQA^i(oRJQo?`Y0>)f2olTi#V`kvjhrsHI24?2Shja}u)=|>#@^b*z za=AC7)T&4Tlb~sdeuwOd8~cTc6h;Z^#{Kbq2HPJQs-ERbk(&Nk1EMX`-Ec(K5To65 zCd?Pme3#Mi)PK4wY%p!fc>*v7u>k`{lndMbh-LeP^q58eZd z5;0L-?tgd;=$YIps^q?;Fc%A^qUp3RFs-WzCQ^3 z>vX+nB9j9bv5Sb#R^0!N@x$&e(JRP-)mX{()1`!8T&3U5Wsf5S`tGJ*bWx< z0Q&4#^TfYXd~iYPd{@TVT%`uOwvl#*wFc;tWvw&!po(>D*S^nO2ZxUB#m`Q(6`Fx`LKtOE5+hDOB%}6Xs&lR@Lm7J z67TjtoTwzz&eRPyrDc&d4dvDuIne#UzsUTb{Zi^Rv!Y=mv#Nu?HLuzuo^zym;?B-% zOSfJJtR1l1wCh|U+=u(}SC6uod>yW3a({IWmBcSaS4&80 z?M-1Gwr*vW%KPl>>?AIP#ZqfrTM&^tfm>eXuKxb>FXMmXNA6)w(X!ekyD0@Z$UHAg j@^#mwKTykGzMI;_Z_$tB6V z<6B-Mua_rD3)#jdp4r(1`DUrr-zT;EPs)mU24cAwhI5r1KpRDkRAC%9hjC1@;=@U% zIR|o2nxdcYAL0q@ba*f){)1a`_anIrviAXp!#xNyObq+NOn)&oa2lqmW!vQfC#4{R zH~7ArXdaDc6QA(EF%Baps46DZ&=J+}%Tr|3SOGQABXMCs_d~jZ7h@^?p7LV^#xSwh z!`#S(0~MhB68#RvGt=fX=I;(rHK#yd|0S|fHuzxWB?g^~$UOm$?B1F26E>tenFsHBLz{gj$OW8+E`W1YD0aYi*_^vSi(Qg2*^IbeKx^Sk9_QRVl7B zfOp>wz>)iDEZ4(v*@62r9&$uEUMN=b2*P7ML>`am8`zE)e(~4K=duVQgnts7!Fynl zJoR)hw|_hu)Ku)Gujam@&=(K($%8KZiQ&~7g~fkA(o$}v3D?sJ=wq4d>xa6t>>V$3 zR~RAgCZ0zlW`w=7iR-O(z`l*Q*bJi#8M^_H*^ z-ND=(K%Ha8HFY<|2b+M#2gcJ;F_XE5TN|dEVWj|W$fB>K^Tz){cZBErDeedHC;4?H zr$QUbIngHlkjjMI4ak+~tV0GN!AW8yVQxA=$fcy|jAkw6de-SZgv5?dGz;z@hu%e48-J$UJjQc{@Py#@B@slzpo_h|pf*EuGKfr` zGHFlgzM3%CvQ86Jiyk`9^KWTwjK+cXfLHVsUe21FZh>3fYSw8??PsvprF)pBm(?oo z^Rsi3xDb|GZE$UXC-MY#dv&_{`_I2L`O1zgz^1Naw^?yl2^`4G^@VQV4e2-3+!)hi ZTU(!VdRx)WIUSet$%5`7_y;ib>ET*P8fX9j diff --git a/Wiki/ENG_Guide/html/searchindex.js b/Wiki/ENG_Guide/html/searchindex.js index 02d8846a..6fff7139 100644 --- a/Wiki/ENG_Guide/html/searchindex.js +++ b/Wiki/ENG_Guide/html/searchindex.js @@ -1 +1 @@ -Search.setIndex({docnames:["01_HowToInstall","02_RoadMap","03_Copyrights_Contacts","Orchestrator/01_Orchestrator","Orchestrator/02_Defs","Orchestrator/03_gSettingsTemplate","Orchestrator/04_HowToStart","Robot/01_Robot","Robot/02_Defs","Robot/03_HowToUse","Robot/04_Dependencies","Studio/01_Studio","Studio/02_HowToUse","index"],envversion:{"sphinx.domains.c":2,"sphinx.domains.changeset":1,"sphinx.domains.citation":1,"sphinx.domains.cpp":3,"sphinx.domains.index":1,"sphinx.domains.javascript":2,"sphinx.domains.math":2,"sphinx.domains.python":2,"sphinx.domains.rst":2,"sphinx.domains.std":1,"sphinx.ext.todo":2,"sphinx.ext.viewcode":1,sphinx:56},filenames:["01_HowToInstall.rst","02_RoadMap.rst","03_Copyrights_Contacts.rst","Orchestrator\\01_Orchestrator.rst","Orchestrator\\02_Defs.rst","Orchestrator\\03_gSettingsTemplate.rst","Orchestrator\\04_HowToStart.rst","Robot\\01_Robot.rst","Robot\\02_Defs.rst","Robot\\03_HowToUse.rst","Robot\\04_Dependencies.rst","Studio\\01_Studio.rst","Studio\\02_HowToUse.rst","index.rst"],objects:{"pyOpenRPA.Orchestrator":{__Orchestrator__:[4,0,0,"-"]},"pyOpenRPA.Orchestrator.__Orchestrator__":{AgentActivityItemAdd:[4,1,1,""],AgentOSCMD:[4,1,1,""],AgentOSFileBinaryDataBase64StrCreate:[4,1,1,""],AgentOSFileBinaryDataBytesCreate:[4,1,1,""],AgentOSFileTextDataStrCreate:[4,1,1,""],GSettingsAutocleaner:[4,1,1,""],GSettingsKeyListValueAppend:[4,1,1,""],GSettingsKeyListValueGet:[4,1,1,""],GSettingsKeyListValueOperatorPlus:[4,1,1,""],GSettingsKeyListValueSet:[4,1,1,""],OSCMD:[4,1,1,""],OSCredentialsVerify:[4,1,1,""],OrchestratorRestart:[4,1,1,""],OrchestratorSessionSave:[4,1,1,""],ProcessIsStarted:[4,1,1,""],ProcessListGet:[4,1,1,""],ProcessStart:[4,1,1,""],ProcessStop:[4,1,1,""],ProcessorActivityItemAppend:[4,1,1,""],ProcessorActivityItemCreate:[4,1,1,""],ProcessorAliasDefCreate:[4,1,1,""],ProcessorAliasDefUpdate:[4,1,1,""],PythonStart:[4,1,1,""],RDPSessionCMDRun:[4,1,1,""],RDPSessionConnect:[4,1,1,""],RDPSessionDisconnect:[4,1,1,""],RDPSessionDublicatesResolve:[4,1,1,""],RDPSessionFileStoredRecieve:[4,1,1,""],RDPSessionFileStoredSend:[4,1,1,""],RDPSessionLogoff:[4,1,1,""],RDPSessionMonitorStop:[4,1,1,""],RDPSessionProcessStartIfNotRunning:[4,1,1,""],RDPSessionProcessStop:[4,1,1,""],RDPSessionReconnect:[4,1,1,""],RDPSessionResponsibilityCheck:[4,1,1,""],RDPTemplateCreate:[4,1,1,""],SchedulerActivityTimeAddWeekly:[4,1,1,""],UACKeyListCheck:[4,1,1,""],UACSuperTokenUpdate:[4,1,1,""],UACUpdate:[4,1,1,""],WebCPUpdate:[4,1,1,""],WebURLConnectDef:[4,1,1,""],WebURLConnectFile:[4,1,1,""],WebURLConnectFolder:[4,1,1,""],WebUserInfoGet:[4,1,1,""],WebUserIsSuperToken:[4,1,1,""],WebUserUACHierarchyGet:[4,1,1,""]},"pyOpenRPA.Robot":{UIDesktop:[8,0,0,"-"]},"pyOpenRPA.Robot.UIDesktop":{Get_OSBitnessInt:[8,1,1,""],PWASpecification_Get_PWAApplication:[8,1,1,""],PWASpecification_Get_UIO:[8,1,1,""],UIOSelectorSecs_WaitAppear_Bool:[8,1,1,""],UIOSelectorSecs_WaitDisappear_Bool:[8,1,1,""],UIOSelector_Exist_Bool:[8,1,1,""],UIOSelector_Get_BitnessInt:[8,1,1,""],UIOSelector_Get_BitnessStr:[8,1,1,""],UIOSelector_Get_UIO:[8,1,1,""],UIOSelector_Get_UIOList:[8,1,1,""],UIOSelector_SafeOtherGet_Process:[8,1,1,""],UIOSelector_SearchChildByMouse_UIO:[8,1,1,""],UIOSelector_SearchChildByMouse_UIOTree:[8,1,1,""],UIOSelectorsSecs_WaitAppear_List:[8,1,1,""],UIOSelectorsSecs_WaitDisappear_List:[8,1,1,""]}},objnames:{"0":["py","module","Python module"],"1":["py","function","Python function"]},objtypes:{"0":"py:module","1":"py:function"},terms:{"0643":6,"100":5,"1050":[4,5],"120":5,"127":4,"1680":[4,5],"1680x1050":[4,5],"1992":6,"2008":0,"2012":0,"2019":13,"2020":1,"2021":1,"222":[3,5],"2999226":0,"300":5,"3389":[4,5],"3600":5,"3720":[0,9],"3720python":9,"4100115560661986":13,"412":4,"600":5,"640x480":[4,5],"77767775":4,"77777sdfsdf77777dsfdfsf77777777":4,"8081":5,"\u0432":[7,8,13],"\u0432\u0445\u043e\u0434\u043d\u043e\u0439":[7,8],"\u0432\u044b\u043a\u0438\u043d\u0443\u0442\u044c":[7,8],"\u0432\u044b\u043f\u043e\u043b\u043d\u044f\u0442\u044c":[7,8],"\u0434\u0435\u043c\u043e\u043d\u0430":5,"\u0434\u0435\u043c\u043e\u043d\u0443":5,"\u043a":[5,7,8],"\u043a\u043e\u043d\u043d\u0435\u043a\u0442":[7,8],"\u043a\u043e\u0442\u043e\u0440\u043e\u043c\u0443":5,"\u043b\u043e\u0433\u0433\u0435\u0440\u0430":5,"\u043b\u043e\u0433\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f":5,"\u043c\u043e\u0436\u043d\u043e":5,"\u043d\u0435":[7,8],"\u043e\u0431\u043d\u0430\u0440\u0443\u0436\u0435\u043d\u0438\u0438":[7,8],"\u043e\u0448\u0438\u0431\u043a\u0443":[7,8],"\u043f\u0435\u0440\u0435\u0439\u0442\u0438":13,"\u043f\u043e":5,"\u043f\u043e\u0434\u0433\u043e\u0442\u043e\u0432\u043a\u0430":5,"\u043f\u043e\u0434\u043a\u043b\u044e\u0447\u0438\u0442\u044c\u0441\u044f":5,"\u043f\u043e\u043a\u0430\u0437\u0430\u0442\u0435\u043b\u044c":[7,8],"\u043f\u043e\u0440\u0442":5,"\u043f\u0440\u043e\u0446\u0435\u0441\u0441\u0443":[7,8],"\u043f\u0443\u0441\u0442\u043e\u0433\u043e":[7,8],"\u0440\u0430\u0437\u0434\u0435\u043b":13,"\u0440\u0430\u0441\u043f\u043e\u043b\u043e\u0436\u0435\u043d\u0438\u0435":5,"\u0441\u0435\u0440\u0432\u0435\u0440\u0430":5,"\u0441\u0435\u0442\u0435\u0432\u043e\u0435":5,"\u0441\u043b\u0443\u0447\u0430\u0435":[7,8],"\u0441\u043e\u0437\u0434\u0430\u0442\u044c":5,"\u0441\u043f\u0438\u0441\u043a\u0430":[7,8],"\u0442\u0440\u0435\u0431\u0443\u0435\u0442\u0441\u044f":[7,8],"\u0442\u0443\u0442\u043e\u0440\u0438\u0430\u043b\u043e\u0432":13,"\u0444\u0430\u0439\u043b":5,"\u0444\u043b\u0430\u0433":[7,8],"\u0447\u0442\u043e":[7,8],"\u044d\u043b\u0435\u043c\u0435\u043d\u0442":[7,8],"case":[3,4,5,7,8],"default":[4,5,12],"float":[3,5,7,8],"function":[4,5,8],"import":[3,4,5,6,8,9],"int":[3,4,7,8],"new":[1,3,4,5,6],"return":[4,5,7,8],"switch":10,"true":[4,5,6,7,8],"try":[4,6],"var":4,Abs:5,Are:0,For:[0,1,6,12],Has:5,NOT:4,RUS:[1,13],The:[0,3,9],USEFUL:4,USe:5,Use:[2,4,6,13],Will:[4,5],__agentdictitemcreate__:5,__create__:5,__main__:6,__name__:6,__orchestrator__:13,__uacclientadmincreate__:[5,6],_sessionlast_rdplist:4,a2o:4,abl:13,about:[3,4,5],abs:4,absolut:[2,4,5,6,13],abspath:6,access:[4,5,6,9,13],accessus:5,action:[1,9,13],activ:[3,4,5,7,8,12],activitydict:5,activityitem:4,activityitemdict:4,activitylist:5,activitylistappendprocessorqueuebool:5,activitylistexecut:4,activitylistexecutebool:5,activitytimelist:5,actual:[4,13],add:[4,5,6,9],addhandl:5,address:[4,5],admindict:[5,6],administr:4,after:[3,5,6,12],agent:13,agentactivityitemadd:4,agentdict:[4,5],agentkeydict:[5,6],agentkeystr:5,agentoscmd:4,agentosfilebinarydatabase64strcr:4,agentosfilebinarydatabytescr:4,agentosfiletextdatastrcr:4,algorithm:3,algorythm:[3,5,13],alia:[3,4,5],aliasdefdict:[3,4,5],all:[2,3,4,5,6,7,8,9,13],allow:[4,5,6,13],alreadi:4,amd64:[0,9],analyz:13,ani:[0,3,4,12,13],anoth:[4,12],apach:2,app:[4,7,8,9,13],appear:[7,8],append:[3,4,5,9],appli:[3,5],applic:[3,4,5,7,8],architectur:13,archiv:[0,5],arg:[3,4,5],argdict:[3,4,5],arggset:[3,4,5],arglist:[3,4,5],arglogg:[3,4,5],argument:[1,4,5],argvaluestr:4,asctim:5,associ:4,asweigart:2,async:3,asynchonu:3,attent:[3,4,6,12],attribut:[3,5,7,8],authent:5,authentif:4,authtoken:5,authtokensdict:5,auto:4,autoclean:[3,5],autom:[9,10,13],automat:[3,5,12],automationsearchmouseel:[7,8],avail:[4,5],b4ff:6,backend:[7,8,9],backward:[1,4],base64:4,base:[3,4,13],basic:[3,4,5,6],becaus:[3,4],been:[0,4,5,13],befor:5,beginwith:[4,5],below:[1,3,6,12],best:13,between:[3,4,5,6],big:13,binari:4,bit:[2,4,7,8,9],block:5,bool:[3,4,5],boppreh:2,both:[4,9],branch:[0,1],browser:[5,12],bsd:2,build:6,built:0,busi:[3,4,13],button:5,cabinetwclass:9,cach:5,call:[3,4,6,7,8],callabl:4,can:[3,4,5,7,8,9,13],cancel:[7,8],cant:3,captur:[9,13],central:3,chang:[5,13],check:[4,5,6,7,8,13],checkintervalsecfloat:5,checktasknam:5,child:9,children:9,choos:[7,8],chrome:[9,13],class_nam:9,claus:2,cleaner:4,clear:[4,5],client:[4,5],clientrequesthandl:5,clipboard:4,close:[4,5],cmd:[4,5,12,13],cmdinputbool:[5,6],code:[3,9,12],collect:5,com:[0,2,9,10],comma:3,command:[4,5,9],commerci:13,commun:4,compani:13,compat:[1,4],compex:3,complet:[0,5],complex:3,compon:13,comput:[9,13],concept:13,config:6,configur:[4,5,6,13],congratul:13,connect:[4,5],connectioncountint:5,connectionfirstqueueitemcountint:5,consist:3,consol:[5,9,12,13],consolid:[3,13],contact:13,contain:[4,5,9,12],content:[1,4,5],continu:4,control:[3,4,5,6],controlpanel:[5,6],controlpaneldict:5,controlpanelkeyallowedlist:5,controlpanelrefreshintervalsecfloat:5,cooki:5,copi:9,copyright:13,core:[3,13],cp_test:6,cp_versioncheck:6,cpdict:5,cpkei:5,cpkeydict:[5,6],cpkeystr:5,creat:[2,3,4,5,6,13],credenti:4,crosscheck:4,css:10,ctrl_index:9,current:[3,4,5,6],custom:3,cv2:9,daemon:4,dai:4,data:5,datasetlast:5,date:1,datetim:[5,6],dear:13,decentr:3,decid:13,def:[3,5,6,7,13],defaliastest:[3,5],defnamestr:5,defsettingsupdatepathlist:5,del:6,depend:13,deploi:4,deprec:5,depthbit:[4,5],descript:[5,8,13],desktop:[3,4,5,6,9,13],desktopus:4,destin:[4,7,8],detail:3,detect:[5,7,8],determin:4,dev:[1,4],develop:[4,13],dict:[4,5,7,8,13],dictionari:[3,5],differ:3,directori:[4,9],disappear:[7,8],disc:4,disconnect:[4,5],distribut:13,divis:13,dll:[9,13],document:[7,8],docutil:[4,8],doe:12,doen:4,domain:5,domainadstr:5,domainupperstr:4,don:4,dont:[4,5],doubl:12,download:0,dp0:9,drive:[4,5],dsd:[3,5],dump:5,dumploglist:5,dumploglistcountint:5,dumploglisthashstr:5,dumploglistrefreshintervalsecfloat:5,duplic:4,durat:5,each:4,edit:13,editor:12,element:[7,8,9],els:[4,5,6,7,8],empti:5,encod:4,end:3,eng:[1,13],enterpris:13,env:5,enviro:9,environ:4,equal:[4,5],equalcas:[4,5],everi:[4,5],everydai:4,exampl:[3,4,5,6,7,8],except:[0,4,6],exe:[0,4,5,8,9],execut:[3,4,5,11,13],executebool:5,exist:[4,5,7,8],expens:13,expir:4,explor:[9,12],extens:[4,9],extra:4,extract:13,facebook:2,fail:1,fals:[4,5,7,8],fast:13,featur:[3,4,5,13],feel:2,field:[4,5],file:[4,5],filehandl:5,filemanag:5,filemod:5,fileurl:5,fileurlfilepathdict:5,fileurlfilepathdict_help:5,fill:[4,5],find:[3,13],find_window:[7,8],firefox:[9,13],first:13,flag:[4,5],flagaccess:5,flagaccessdefrequestglobalauthent:5,flagcredentialsask:5,flagdonotexpir:5,flagforc:5,flagsessionisact:[4,5],flaguseallmonitor:[4,5],flexibl:3,folder:[4,6,9],follow:[0,12],forc:[4,5],forget:5,formatt:5,founder:13,framework:[10,13],free:[2,13],fridai:4,from:[0,3,4,5,6,7,8,13],full:[4,5],fulli:9,fullscreen:[4,5],fullscreenbool:5,fullscreenrdpsessionkeystr:5,functional:6,further:4,garbag:4,gener:[4,5,13],get:[4,5,7,8,9],get_osbitnessint:[7,8],getcontrol:[7,8],getlogg:5,git:[0,4,5],github:2,gitlab:[0,9],give:[4,5],given:4,global:[4,13],goe:3,good:3,graphic:[9,13],great:13,group:13,gset:[3,6,13],gsettingsautoclean:4,gsettingsdict:4,gsettingskeylistvalueappend:4,gsettingskeylistvalueget:4,gsettingskeylistvalueoperatorplu:4,gsettingskeylistvalueset:4,gui:[3,4,7,8,9,10,13],guid:[1,5],gurbag:5,handl:5,handlebar:10,handler:5,hard:[4,5],has:[0,3,4,9],have:4,height:[4,5],help:[0,2,4,13],here:13,hex:[4,5],hidden:4,hierarchi:[4,12],highlight:[9,12],hightlight:12,host:[4,5],hostnameupperstr:5,how:13,html:[4,8,12],htmlrenderdef:5,http:[0,2,3,4,5,8,9,10,13],human:4,identif:4,ignor:[4,5],ignorebool:5,imag:[9,13],imaslov:6,inactivityitemdict:4,inactivitylist:4,inadisdefaultbool:[4,6],inadloginstr:[4,6],inadstr:[4,6],inaliasstr:4,inarg1str:4,inargdict:4,inarggset:4,inarggsettingsstr:4,inarglist:4,inargloggerstr:4,inbreaktriggerprocesswoexelist:4,incloseforcebool:4,includ:[7,8],incmdstr:4,incontenttypestr:4,incontrolspecificationarrai:[7,8],incpkeystr:4,indef:4,indefnamestr:4,indepthbitint:4,index:[3,4,5,7,8],indict:5,indomainstr:4,inel:[7,8],inelementspecif:[7,8],inencodingstr:4,infiledatabase64str:4,infiledatabyt:4,infiledatastr:4,infilepathstr:4,inflagforceclosebool:4,inflaggetabspathbool:4,inflagraiseexcept:[7,8],inflagwaitallinmo:[7,8],info:[4,5,6],infolderpathstr:4,inform:[3,4],infrastructur:13,ingset:[3,4,6],ingsettingsclientdict:5,inhashkeystr:5,inheightpxint:4,inhostfilepathstr:4,inhostnamestr:4,inhoststr:[4,5],inhtmlrenderdef:4,init:[3,4,5,6],initdatetim:5,initi:4,injsinitgeneratordef:4,injsongeneratordef:4,inkeylist:4,inkeystr:5,inlogg:[4,5],inloginstr:[4,5],inmatchtypestr:4,inmethodstr:4,inmodestr:[3,4,5,6],inmodulepathstr:4,inpasswordstr:[4,5],inpathstr:4,inportint:4,inportstr:[4,5],inprocessnamewexestr:4,inprocessnamewoexelist:4,inprocessnamewoexestr:4,input:5,inrdpfilepathstr:4,inrdpsessionkeystr:[4,5],inrdptemplatedict:4,inrequest:4,inrolehierarchyalloweddict:[4,6],inrolekeylist:4,inrowcountint:5,inrunasyncbool:4,insert:6,inshareddrivelist:4,inspecificationlist:[7,8],inspecificationlistlist:[7,8],instal:13,instanc:[3,4,7,8],instopprocessnamewoexestr:4,insupertokenstr:[4,6],interact:[3,4],interest:4,interfac:[3,9,13],internet:12,interpret:3,interv:[4,5],intervalsecfloat:5,intimehhmmstr:4,inuioselector:[7,8],inurllist:[4,6],inurlstr:4,inusebothmonitorbool:4,inusernamestr:4,inuserstr:4,invalu:4,inwaitsec:[7,8],inweekdaylist:4,inwidthpxint:4,islistenbool:5,isresponsiblebool:4,it4busi:2,item:[3,4],iter:5,ivan:13,ivanmaslov:2,join:6,jsinitgeneratordef:5,json:[3,4,5],jsongeneratordef:5,jsrender:10,jsview:10,just:9,kb2999226:0,keep:3,kei:4,keyboard:[2,9,10,13],keystr:5,kill:4,killer:13,know:3,known:13,kwarg:3,lactivityitem:4,laliasstr:4,last:5,launch:4,left:3,len:5,let:13,level:[9,13],levelnam:5,licens:13,lifetim:5,lifetimerequestsecfloat:5,lifetimesecfloat:5,light:3,like:[3,13],line:9,link:[3,4,5,13],linkedin:2,list:[3,4,5,7,8,9],listen:4,listenport:5,listenport_:5,listenurllist:5,listread:5,litem:9,load:5,local:[4,5],localhost:5,locat:4,log:[4,5,6,13],logger:[4,5,6],loggerdumploghandleradd:5,loggerhandlerdumploglist:5,login:[4,5,6],logoff:[4,5],logviewerbool:[5,6],lol:12,look:[3,4,5,12,13],lookmachinescreenshot:5,loop:9,low:9,lowercas:5,lprocessisstartedbool:4,lprocesslist:4,lpyopenrpasourcefolderpathstr:6,lrdpitemdict:4,lrdptemplatedict:4,lresult:5,lresultdict:[4,5],luacclientdict:6,lxml:10,machin:[0,4,5,6,13],machina:5,mail:2,main:[3,4,5,6,9],maintain:13,makedir:5,manag:4,mani:[3,7,8],manipul:[4,9,13],march:1,maslov:13,master:0,matchtyp:5,max:5,mayb:4,mechan:3,mega:3,megafind:2,memori:4,merg:4,messag:5,method:5,methodmatchurl:5,methodmatchurlbeforelist:5,mhandlerdumploglist:5,microsoft:0,mit:[2,13],mmstr:5,mode:12,model:2,modul:[3,4,5,6,9],moduletocal:4,mondai:4,monei:13,monitor:4,more:3,mous:[9,13],mrobotlogg:5,mrobotloggerfh:5,mrobotloggerformatt:5,must:5,name:[3,4,5,6,7,8],namewoexestr:4,namewoexeupperstr:4,nbsp:9,need:[0,2,3,4,5,13],nest:4,net:[4,8],never:4,newkeydict:4,newkeylist:4,newvalu:4,non:13,none:[3,4,5,7,8],notat:[7,8],notepad:[4,5,7,8],noth:4,nothingbool:5,now:[4,5],nul:9,object:[3,4,5,7,8,9,13],occupi:4,octet:4,off:5,old:[4,5,6,7,8],one:[5,6],onli:[0,1,4,5,7,8],onlin:13,open:[12,13],opencv:[0,10,13],openrpa52zzz:6,openrpa:[0,5,9],openrpa_32:12,openrpa_64:12,openrpaorchestr:9,openrparesourceswpy32:9,openrparesourceswpy64:9,openrparobotdaemon:5,opensourc:13,oper:[0,4,6],opera:[9,13],option:5,orc:5,orchestr:[5,6],orchestratormain:9,orchestratorrestart:4,orchestratorsessionsav:4,orchestratorstart:5,order:[3,9,12],org:2,oscmd:4,oscredentialsverifi:4,other:[7,8,13],out:4,output:5,outstr:4,over:2,overwrit:5,own:[3,9,13],packag:[0,6,7],page:[4,5],pai:[3,6],panel:[4,5,6,12],paramet:[3,4,5,7,8],parent:12,parti:13,pass:[4,5],password:[4,5],path:[4,5,6,9],paus:9,pdb:5,per:6,perfom:[9,13],perform:13,period:5,phone:3,pid:4,pil:10,pipupgrad:5,pixel:[4,5],plan:[1,4],platform:13,pleas:13,plu:4,port:[4,5],portabl:[0,9],possibl:13,post:[4,5],postfix:4,power:13,practic:[9,13],prefer:2,previou:4,print:[6,9],process:[3,5,7,8,9,11,13],processdetaillist:4,processisstart:4,processlistget:4,processnam:4,processor:[5,13],processoractivityitemappend:4,processoractivityitemcr:4,processoraliasdefcr:4,processoraliasdefupd:4,processordict:5,processstart:[4,5],processstartifturnedoff:5,processstop:[4,5],processwoexelist:4,processwoexeupperlist:4,product:9,program:[4,5],progress:1,project:[3,13],properti:13,protocol:3,psutil:[6,10],pull:5,purpos:[1,2],pwaspecification_get_pwaappl:[7,8],pwaspecification_get_uio:[7,8],pyautogui:[2,9,10,13],pymupdf:10,pyopenrpa:[0,1,2,3,5,6,11],pyopenrpadict:[5,6],pyrobot_cp:6,python:[0,3,5,7,10,12,13],pythonstart:4,pywin32:[2,10],pywinauto:[2,7,8,10],queue:[1,3,4,5,13],queuelist:5,r01:5,r01_integrationorderout:5,r01_orchestratortorobot:5,rais:4,rdp:[3,4,5],rdpactiv:5,rdpconfigurationdict:5,rdpkei:4,rdpkeydict:[5,6],rdpkeystr:5,rdplist:[4,5],rdpsession:13,rdpsessioncmdrun:4,rdpsessionconnect:[4,5],rdpsessiondisconnect:[4,5],rdpsessiondublicatesresolv:4,rdpsessionfilereciev:5,rdpsessionfilesend:5,rdpsessionfilestoredreciev:4,rdpsessionfilestoredsend:4,rdpsessionkei:5,rdpsessionkeystr:5,rdpsessionlogoff:[4,5],rdpsessionmonitorstop:4,rdpsessionprocessstart:5,rdpsessionprocessstartifnotrun:4,rdpsessionprocessstop:4,rdpsessionreconnect:[4,5],rdpsessionresponsibilitycheck:4,rdptemplatecr:4,read:5,readi:0,reciev:[4,5],recognit:[9,13],reconnect:4,reconnectbool:5,reestr_otgruzok:5,refactor:1,refer:13,refresh:5,refreshsecond:5,regener:4,rel:[4,5,6],reliabl:13,rememb:5,remot:[4,5,13],render:12,renderfunct:5,renderrobotr01:5,report:5,reqir:3,request:[4,5,10],requesttimeoutsecfloat:5,requir:[4,13],resolut:[4,5],resourc:[0,9],respons:[4,5],responsecontenttyp:5,responsedefrequestglob:5,responsefilepath:5,responsefolderpath:5,responsibilitycheckintervalsec:5,restart:[4,5],restartorchestr:5,restartorchestratorbool:[5,6],restartorchestratorgitpullbool:[5,6],restartpcbool:[5,6],restrict:13,restructuredtext:[4,8],result:[4,5,7,8],returnbool:5,roadmap:13,robot:[3,4,5,6,9],robot_r01:5,robot_r01_help:5,robotlist:5,robotrdpact:[4,5],rolehierarchyalloweddict:5,root:[4,5],row:5,rpa:[4,5,13],rpatestdirtest:4,rst:[4,8],ruledomainuserdict:5,rulemethodmatchurlbeforelist:5,run:[0,4,5,6,13],russia:[2,13],russian:13,safe:[4,7,8],same:[7,8],save:4,schedul:[3,13],scheduleractivitytimeaddweekli:4,schedulerdict:5,scopesrcul:5,screen:[4,5,9,13],screenshot:[0,5,13],screenshotviewerbool:[5,6],script:[3,13],search:[4,13],sec:5,second:[4,5,7,8,9],section:4,see:[0,1,4,5,6,7,8],select:[7,8],selector:[7,8,12,13],selenium:[2,9,10,13],semant:[2,10],send:[4,5],sent:4,sequenc:3,server:[0,3,5,12],serverdict:5,serverset:5,sesion:[4,5],session:[4,5],sessionguidstr:5,sessionhex:[4,5],sessionisignoredbool:[4,5],sessioniswindowexistbool:[4,5],sessioniswindowresponsiblebool:[4,5],set:[4,5,6,7,8,13],set_trac:5,setformatt:5,setlevel:5,settingstempl:[3,6],settingsupd:6,setup:5,sever:[3,9,13],share:4,shareddrivelist:[4,5],shell:[4,13],should:5,show:5,side:[4,5],signal:4,simplifi:9,sinc:13,singl:3,singleton:4,skype:2,sleep:9,socket:3,solut:[0,9,13],some:[3,4,5,13],sort:4,sourc:[3,4,6,7,8,13],sourceforg:[4,8],space:3,special:5,sphinx:3,standart:5,start:[0,4,5,12,13],statu:4,stdout:[5,6],stop:[4,13],storag:[5,13],store:5,str:[3,4,5,7,8,9],stream:4,streamhandl:5,strftime:5,string:4,struct:5,structur:[3,4,5],studio:[3,11,12],subprocess:9,success:4,successfulli:[0,4,13],sundai:4,supertoken:[4,6],superus:6,supetoken:4,support:[0,3,6,12,13],symbol:3,sync:3,sys:[5,6,9],system:13,tablet:3,technic:[3,5],technicalsessionguidcach:5,templat:[6,13],tesseract:10,test2:5,test:[4,5,6],testcontrolpanelkei:5,testdef:4,testdefalia:4,testdir:4,testdirtest:4,testrdp:5,text:[4,12],than:3,thank:[2,13],theori:[9,13],thi:[4,5,13],thought:4,thread:[3,5],threadidint:5,thursdai:4,time:[4,5,9,13],timehh:5,titl:[7,8],todo:4,token:4,tokendatetim:5,too:[4,5],tool:[3,9,12],top:[7,8],tor:13,track:4,transmiss:4,transmit:[3,4],tree:13,trigger:[4,5],ttt:[3,5],turn:5,turpl:3,tutori:1,txt:4,type:[4,5,6],uac:13,uackeylistcheck:4,uacsupertokenupd:[4,6],uacupd:[4,6],uia:[7,8,9],uidesktop:[1,7,13],uio:[7,8,12,13],uioselector:[7,8],uioselector_exist_bool:[7,8],uioselector_focushighlight:9,uioselector_get_bitnessint:[7,8],uioselector_get_bitnessstr:[7,8],uioselector_get_uio:[7,8,9],uioselector_get_uiolist:[7,8],uioselector_safeotherget_process:[7,8],uioselector_searchchildbymouse_uio:[7,8],uioselector_searchchildbymouse_uiotre:[7,8],uioselectorsecs_waitappear_bool:[7,8],uioselectorsecs_waitdisappear_bool:[7,8],uioselectorssecs_waitappear_list:[7,8],uioselectorssecs_waitdisappear_list:[7,8],under:13,unicodelab:[0,2,9],univers:3,unzip:0,updat:[4,5],upper:5,url:[4,5],url_:5,urllist:5,usag:4,use:[0,3,4,5,13],used:4,user:[3,4,5,6,13],user_99:4,user_pass_her:4,useradstr:5,usernam:[4,5],usernameupperstr:4,userupperstr:5,using:[4,13],utf:4,util:5,valu:[4,7,8],variant:4,verifi:4,version:[1,6,9,12],versioncheck:6,versionstr:5,viewer:[5,13],virtual:5,vision:[9,13],vista:0,visual:13,vms:4,wai:[4,6,13],wait:[7,8,9,12],want:[3,5,12,13],warn:6,web:[3,5,9,13],webcpupd:4,weburlconnectdef:4,weburlconnectfil:4,weburlconnectfold:4,webuserinfoget:4,webuserissupertoken:4,webuseruachierarchyget:4,wednesdai:4,week:4,weekdai:[4,5],weekdaylist:5,well:13,were:13,when:[3,4,5,6,12],where:[4,12],which:[3,4,5,7,8,13],who:5,why:3,width:[4,5],wiki:9,win32:[7,8,9,13],win32api:10,win:4,window:[0,4,5,10],winpython:2,without:[0,4,5,13],wmi:10,work:[4,5,9],workingdirectorypathstr:5,world:[2,13],wpy32:[0,9],wpy64:[0,9],wrapper:[7,8],write:[3,9,13],www:[2,10],x32:[0,10,12],x64:[0,10,12],xlsx:5,yandex:13,you:[0,2,3,4,5,7,8,9,13],your:[0,9],zip:0},titles:["1. How to install","2. Roadmap","3. Copyrights & Contacts","1. Description","2. Defs","3. gSettings Template","4. How to start process","1. Description","2. Defs","3. How to use","4. Dependencies","1. Description","2. How to use","Welcome to pyOpenRPA\u2019s wiki"],titleterms:{The:[12,13],Use:9,__orchestrator__:4,about:[9,13],action:12,agent:4,architectur:3,button:12,check:0,choos:12,click:12,cmd:9,compon:[2,3],concept:3,configur:3,contact:2,content:[9,12,13],copyright:2,creat:9,ctrl:12,def:[4,8],depend:[2,10],descript:[3,7,11,12],dict:3,donat:13,execut:9,expand:12,extract:12,file:9,founder:2,from:9,global:3,group:4,gset:[4,5],gui:12,guid:13,has:13,hold:12,hover:12,how:[0,3,6,9,12],instal:0,interest:12,ivan:2,kei:12,licens:2,list:12,main:13,maslov:2,mous:12,object:12,orchestr:[3,4,13],parti:2,process:[4,6],processor:[3,4],progress:13,properti:12,pyopenrpa:[4,7,8,13],python:[4,9],rdpsession:4,refer:[3,4,8],requir:0,result:12,roadmap:1,robot:[7,8,13],run:12,schedul:4,screenshot:12,script:9,search:12,second:12,select:12,set:3,shown:12,start:6,structur:13,studio:[9,13],system:0,templat:5,tool:13,tree:12,turn:12,tutori:13,uac:4,uidesktop:8,use:[9,12],viewer:12,wai:9,web:4,welcom:13,wiki:13,x32:9,x64:9,you:12}}) \ No newline at end of file +Search.setIndex({docnames:["01_HowToInstall","02_RoadMap","03_Copyrights_Contacts","Orchestrator/01_Orchestrator","Orchestrator/02_Defs","Orchestrator/03_gSettingsTemplate","Orchestrator/04_HowToStart","Robot/01_Robot","Robot/02_Defs","Robot/03_HowToUse","Robot/04_Dependencies","Studio/01_Studio","Studio/02_HowToUse","index"],envversion:{"sphinx.domains.c":2,"sphinx.domains.changeset":1,"sphinx.domains.citation":1,"sphinx.domains.cpp":3,"sphinx.domains.index":1,"sphinx.domains.javascript":2,"sphinx.domains.math":2,"sphinx.domains.python":2,"sphinx.domains.rst":2,"sphinx.domains.std":1,"sphinx.ext.todo":2,"sphinx.ext.viewcode":1,sphinx:56},filenames:["01_HowToInstall.rst","02_RoadMap.rst","03_Copyrights_Contacts.rst","Orchestrator\\01_Orchestrator.rst","Orchestrator\\02_Defs.rst","Orchestrator\\03_gSettingsTemplate.rst","Orchestrator\\04_HowToStart.rst","Robot\\01_Robot.rst","Robot\\02_Defs.rst","Robot\\03_HowToUse.rst","Robot\\04_Dependencies.rst","Studio\\01_Studio.rst","Studio\\02_HowToUse.rst","index.rst"],objects:{"pyOpenRPA.Orchestrator":{__Orchestrator__:[4,0,0,"-"]},"pyOpenRPA.Orchestrator.__Orchestrator__":{AgentActivityItemAdd:[4,1,1,""],AgentOSCMD:[4,1,1,""],AgentOSFileBinaryDataBase64StrCreate:[4,1,1,""],AgentOSFileBinaryDataBytesCreate:[4,1,1,""],AgentOSFileTextDataStrCreate:[4,1,1,""],GSettingsAutocleaner:[4,1,1,""],GSettingsKeyListValueAppend:[4,1,1,""],GSettingsKeyListValueGet:[4,1,1,""],GSettingsKeyListValueOperatorPlus:[4,1,1,""],GSettingsKeyListValueSet:[4,1,1,""],OSCMD:[4,1,1,""],OSCredentialsVerify:[4,1,1,""],OrchestratorRestart:[4,1,1,""],OrchestratorSessionSave:[4,1,1,""],ProcessIsStarted:[4,1,1,""],ProcessListGet:[4,1,1,""],ProcessStart:[4,1,1,""],ProcessStop:[4,1,1,""],ProcessorActivityItemAppend:[4,1,1,""],ProcessorActivityItemCreate:[4,1,1,""],ProcessorAliasDefCreate:[4,1,1,""],ProcessorAliasDefUpdate:[4,1,1,""],PythonStart:[4,1,1,""],RDPSessionCMDRun:[4,1,1,""],RDPSessionConnect:[4,1,1,""],RDPSessionDisconnect:[4,1,1,""],RDPSessionDublicatesResolve:[4,1,1,""],RDPSessionFileStoredRecieve:[4,1,1,""],RDPSessionFileStoredSend:[4,1,1,""],RDPSessionLogoff:[4,1,1,""],RDPSessionMonitorStop:[4,1,1,""],RDPSessionProcessStartIfNotRunning:[4,1,1,""],RDPSessionProcessStop:[4,1,1,""],RDPSessionReconnect:[4,1,1,""],RDPSessionResponsibilityCheck:[4,1,1,""],RDPTemplateCreate:[4,1,1,""],SchedulerActivityTimeAddWeekly:[4,1,1,""],UACKeyListCheck:[4,1,1,""],UACSuperTokenUpdate:[4,1,1,""],UACUpdate:[4,1,1,""],WebCPUpdate:[4,1,1,""],WebURLConnectDef:[4,1,1,""],WebURLConnectFile:[4,1,1,""],WebURLConnectFolder:[4,1,1,""],WebUserInfoGet:[4,1,1,""],WebUserIsSuperToken:[4,1,1,""],WebUserUACHierarchyGet:[4,1,1,""]},"pyOpenRPA.Robot":{UIDesktop:[8,0,0,"-"]},"pyOpenRPA.Robot.UIDesktop":{Get_OSBitnessInt:[8,1,1,""],PWASpecification_Get_PWAApplication:[8,1,1,""],PWASpecification_Get_UIO:[8,1,1,""],UIOSelectorSecs_WaitAppear_Bool:[8,1,1,""],UIOSelectorSecs_WaitDisappear_Bool:[8,1,1,""],UIOSelectorUIOActivity_Run_Dict:[8,1,1,""],UIOSelector_Exist_Bool:[8,1,1,""],UIOSelector_FocusHighlight:[8,1,1,""],UIOSelector_GetChildList_UIOList:[8,1,1,""],UIOSelector_Get_BitnessInt:[8,1,1,""],UIOSelector_Get_BitnessStr:[8,1,1,""],UIOSelector_Get_UIO:[8,1,1,""],UIOSelector_Get_UIOActivityList:[8,1,1,""],UIOSelector_Get_UIOInfo:[8,1,1,""],UIOSelector_Get_UIOList:[8,1,1,""],UIOSelector_Highlight:[8,1,1,""],UIOSelector_SafeOtherGet_Process:[8,1,1,""],UIOSelector_SearchChildByMouse_UIO:[8,1,1,""],UIOSelector_SearchChildByMouse_UIOTree:[8,1,1,""],UIOSelector_TryRestore_Dict:[8,1,1,""],UIOSelectorsSecs_WaitAppear_List:[8,1,1,""],UIOSelectorsSecs_WaitDisappear_List:[8,1,1,""]}},objnames:{"0":["py","module","Python module"],"1":["py","function","Python function"]},objtypes:{"0":"py:module","1":"py:function"},terms:{"0643":6,"100":5,"1050":[4,5],"120":5,"127":4,"1680":[4,5],"1680x1050":[4,5],"1992":6,"2008":0,"2012":0,"2019":13,"2020":1,"2021":1,"222":[3,5],"2999226":0,"300":5,"3389":[4,5],"3600":5,"3720":[0,9],"3720python":9,"4100115560661986":13,"412":4,"600":5,"640x480":[4,5],"722":2,"77767775":4,"77777sdfsdf77777dsfdfsf77777777":4,"8081":5,"906":2,"\u0432":[7,8,13],"\u0432\u0445\u043e\u0434\u043d\u043e\u0439":[7,8],"\u0432\u044b\u043a\u0438\u043d\u0443\u0442\u044c":[7,8],"\u0432\u044b\u043f\u043e\u043b\u043d\u044f\u0442\u044c":[7,8],"\u0434\u0435\u043c\u043e\u043d\u0430":5,"\u0434\u0435\u043c\u043e\u043d\u0443":5,"\u043a":[5,7,8],"\u043a\u043e\u043d\u043d\u0435\u043a\u0442":[7,8],"\u043a\u043e\u0442\u043e\u0440\u043e\u043c\u0443":5,"\u043b\u043e\u0433\u0433\u0435\u0440\u0430":5,"\u043b\u043e\u0433\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f":5,"\u043c\u043e\u0436\u043d\u043e":5,"\u043d\u0435":[7,8],"\u043e\u0431\u043d\u0430\u0440\u0443\u0436\u0435\u043d\u0438\u0438":[7,8],"\u043e\u0448\u0438\u0431\u043a\u0443":[7,8],"\u043f\u0435\u0440\u0435\u0439\u0442\u0438":13,"\u043f\u043e":5,"\u043f\u043e\u0434\u0433\u043e\u0442\u043e\u0432\u043a\u0430":5,"\u043f\u043e\u0434\u043a\u043b\u044e\u0447\u0438\u0442\u044c\u0441\u044f":5,"\u043f\u043e\u043a\u0430\u0437\u0430\u0442\u0435\u043b\u044c":[7,8],"\u043f\u043e\u0440\u0442":5,"\u043f\u0440\u043e\u0446\u0435\u0441\u0441\u0443":[7,8],"\u043f\u0443\u0441\u0442\u043e\u0433\u043e":[7,8],"\u0440\u0430\u0437\u0434\u0435\u043b":13,"\u0440\u0430\u0441\u043f\u043e\u043b\u043e\u0436\u0435\u043d\u0438\u0435":5,"\u0441\u0435\u0440\u0432\u0435\u0440\u0430":5,"\u0441\u0435\u0442\u0435\u0432\u043e\u0435":5,"\u0441\u043b\u0443\u0447\u0430\u0435":[7,8],"\u0441\u043e\u0437\u0434\u0430\u0442\u044c":5,"\u0441\u043f\u0438\u0441\u043a\u0430":[7,8],"\u0442\u0440\u0435\u0431\u0443\u0435\u0442\u0441\u044f":[7,8],"\u0442\u0443\u0442\u043e\u0440\u0438\u0430\u043b\u043e\u0432":13,"\u0444\u0430\u0439\u043b":5,"\u0444\u043b\u0430\u0433":[7,8],"\u0447\u0442\u043e":[7,8],"\u044d\u043b\u0435\u043c\u0435\u043d\u0442":[7,8],"case":[3,4,5,7,8,9],"class":[8,9],"default":[4,5,8,9,12],"float":[3,5,7,8],"function":[4,5,8,9],"import":[3,4,5,6,7,8,9],"int":[3,4,7,8,9],"new":[1,3,4,5,6],"return":[4,5,7,8,9],"switch":10,"true":[4,5,6,7,8],"try":[4,6,7,8],"var":4,Abs:5,Are:0,For:[0,1,6,8,9,12],Has:5,NOT:4,RUS:[1,13],The:[0,3],USEFUL:4,USe:5,Use:[2,4,6],Will:[4,5],__agentdictitemcreate__:5,__create__:5,__main__:6,__name__:6,__orchestrator__:13,__uacclientadmincreate__:[5,6],_sessionlast_rdplist:4,a2o:4,abil:[8,9],abl:13,about:[3,4,5],abs:4,absolut:[2,4,5,6,13],abspath:6,access:[4,5,6,13],accessus:5,action:[1,9,13],activ:[3,4,5,7,8,9,12],activitydict:5,activityitem:4,activityitemdict:4,activitylist:5,activitylistappendprocessorqueuebool:5,activitylistexecut:4,activitylistexecutebool:5,activitynam:9,activitytimelist:5,actual:[4,13],add:[4,5,6,9],addhandl:5,address:[4,5],admindict:[5,6],administr:4,after:[3,5,6,7,8,12],agent:13,agentactivityitemadd:4,agentdict:[4,5],agentkeydict:[5,6],agentkeystr:5,agentoscmd:4,agentosfilebinarydatabase64strcr:4,agentosfilebinarydatabytescr:4,agentosfiletextdatastrcr:4,algorithm:3,algorythm:[3,5,13],alia:[3,4,5],aliasdefdict:[3,4,5],all:[2,3,4,5,6,7,8,9,13],allow:[4,5,6,8,9,13],alreadi:[4,8,9],amd64:[0,9],analyz:13,ani:[0,3,4,9,12,13],anoth:[4,12],apach:2,app:[4,7,13],appear:[7,8],append:[3,4,5,9],appli:[3,5],applic:[3,4,5,7,8,9],approach:[8,9],architectur:13,archiv:[0,5],arg:[3,4,5],argdict:[3,4,5],arggset:[3,4,5],arglist:[3,4,5],arglogg:[3,4,5],argument:[1,4,5],argvaluestr:4,artifici:9,asctim:5,assert:9,associ:4,asweigart:2,async:3,asynchonu:3,attent:[3,4,6,8,9,12],attribut:[3,5,7,8,9],authent:5,authentif:4,authtoken:5,authtokensdict:5,auto:4,autoclean:[3,5],autom:[10,13],automat:[3,5,12],automationsearchmouseel:[7,8],avail:[4,5,8,9],b4ff:6,backend:[7,8,9],backward:[1,4],base64:4,base:[3,4,13],base_wrapp:[8,9],basic:[3,4,5,6],becaus:[3,4,8,9],been:[0,4,5,8,9,13],befor:5,beginwith:[4,5],below:[1,3,6,12],best:13,between:[3,4,5,6],big:[8,9,13],binari:4,bit:[2,4,7,8,9],block:5,bool:[3,4,5,8,9],boppreh:2,both:[4,9],branch:[0,1],browser:[5,12],bsd:2,build:6,built:0,busi:[3,4,9,13],button:5,cabinetwclass:9,cach:5,calcfram:[8,9],call:[3,4,6,7,8],callabl:4,can:[3,4,5,7,8,9,13],cancel:[7,8],cant:3,captur:13,central:3,chang:[5,13],check:[4,5,6,7,8,9,13],checkintervalsecfloat:5,checktasknam:5,child:[7,8,9],children:9,choos:[7,8],chrome:[9,13],class_nam:[8,9],class_name_r:[8,9],classif:9,claus:2,cleaner:4,clear:[4,5,9],click:[8,9],client:[4,5],clientrequesthandl:5,clipboard:4,close:[4,5,9],cmd:[4,5,12,13],cmdinputbool:[5,6],code:[3,8,9,12],collect:5,com:[0,2,9,10],comma:3,command:[4,5,9],commerci:13,common:9,commun:4,compani:13,compat:[1,4,8,9],compex:3,complet:[0,5],complex:3,compon:13,comput:[9,13],concept:13,condit:[7,8,9],config:6,configur:[4,5,6,13],congratul:13,connect:[4,5],connectioncountint:5,connectionfirstqueueitemcountint:5,consist:3,consol:[5,9,12,13],consolid:[3,13],contact:13,contain:[4,5,7,8,9,12],content:[1,4,5],continu:4,control:[3,4,5,6],control_typ:[8,9],control_type_r:[8,9],controlpanel:[5,6],controlpaneldict:5,controlpanelkeyallowedlist:5,controlpanelrefreshintervalsecfloat:5,convent:9,cooki:5,copi:9,copyright:13,core:[3,13],cp_test:6,cp_versioncheck:6,cpdict:5,cpkei:5,cpkeydict:[5,6],cpkeystr:5,creat:[2,3,4,5,6,13],credenti:4,crosscheck:4,css:10,ctrl_index:[8,9],current:[3,4,5,6,8,9],custom:[3,9],cv2:9,daemon:4,dai:4,data:5,datasetlast:5,date:1,datetim:[5,6],dear:13,decentr:3,decid:13,def:[3,5,6,7,13],defaliastest:[3,5],defnamestr:5,defsettingsupdatepathlist:5,del:6,depend:13,deploi:4,deprec:5,depth:[8,9],depth_end:[8,9],depth_start:[8,9],depth_stop:[8,9],depthbit:[4,5],descipt:[8,9],descript:[5,8,9,13],desktop:[3,4,5,6,13],desktopus:4,destin:[4,7,8],detail:3,detect:[5,7,8,9],determin:4,dev:[1,4],develop:[4,8,9,13],dict:[4,5,7,8,9,13],dictionari:[3,5],differ:3,directori:[4,9],disappear:[7,8],disc:4,disconnect:[4,5],distribut:13,divis:13,dll:13,doc:9,document:[7,8],docutil:[4,8],doe:12,doen:4,doesn:[7,8],domain:5,domainadstr:5,domainupperstr:4,don:4,dont:[4,5],doubl:12,download:0,dp0:9,draw:[7,8],drive:[4,5],driver:9,dsd:[3,5],dump:5,dumploglist:5,dumploglistcountint:5,dumploglisthashstr:5,dumploglistrefreshintervalsecfloat:5,duplic:4,durat:5,dynam:[8,9],each:[4,8,9],edit:13,editor:12,elem:9,element:[7,8,9],els:[4,5,6,7,8],empti:5,enabl:[8,9],encod:4,end:3,eng:[1,13],enterpris:13,env:5,enviro:9,environ:4,equal:[4,5],equalcas:[4,5],everi:[4,5],everydai:4,exact:9,exampl:[3,4,5,6,7],except:[0,4,6],exe:[0,4,5,8,9],execut:[3,4,5,11,13],executebool:5,exist:[4,5,7,8,9],expens:13,expir:4,explor:[9,12],express:[8,9],extens:[4,8,9],extra:4,extract:[8,9,13],facebook:2,fail:1,fals:[4,5,7,8],fast:13,featur:[3,4,5,13],feel:2,field:[4,5],file:[4,5],filehandl:5,filemanag:5,filemod:5,fileurl:5,fileurlfilepathdict:5,fileurlfilepathdict_help:5,fill:[4,5],find:[3,9,13],find_element_by_nam:9,find_window:[7,8,9],firefox:[9,13],first:[8,9,13],flag:[4,5],flagaccess:5,flagaccessdefrequestglobalauthent:5,flagcredentialsask:5,flagdonotexpir:5,flagforc:5,flagsessionisact:[4,5],flaguseallmonitor:[4,5],flexibl:3,focu:[7,8],folder:[4,6,9],follow:[0,12],forc:[4,5],forget:5,formatt:5,found:9,founder:13,framework:[7,8,9,10,13],free:[2,13],fridai:4,friendly_class_nam:[8,9],friendly_class_name_r:[8,9],from:[0,3,4,5,6,7,8,13],full:[4,5],fulli:9,fullscreen:[4,5],fullscreenbool:5,fullscreenrdpsessionkeystr:5,functional:6,further:4,garbag:4,gener:[4,5,13],get:[4,5,7,8,9],get_osbitnessint:[7,8],getcontrol:[7,8],getlogg:5,git:[0,4,5],github:2,gitlab:[0,9],give:[4,5],given:4,global:[4,13],goe:3,good:3,graphic:[9,13],great:13,group:13,gset:[3,6,13],gsettingsautoclean:4,gsettingsdict:4,gsettingskeylistvalueappend:4,gsettingskeylistvalueget:4,gsettingskeylistvalueoperatorplu:4,gsettingskeylistvalueset:4,gui:[3,4,7,8,9,10,13],guid:[1,5],gurbag:5,handl:5,handlebar:10,handler:5,hard:[4,5],has:[0,3,4,8,9],have:4,height:[4,5],help:[0,2,4,13],here:[8,9,13],hex:[4,5,8,9],hidden:4,hierarchi:[4,8,9,12],highlight:[7,8,9,12],hightlight:12,host:[4,5],hostnameupperstr:5,how:13,html:[4,8,9,12],htmlrenderdef:5,http:[0,2,3,4,5,8,9,10,13],human:4,identif:[4,9],identifi:9,ignor:[4,5],ignorebool:5,imag:13,imaslov:6,implement:[8,9],inactionnam:[7,8],inactivityitemdict:4,inactivitylist:4,inadisdefaultbool:[4,6],inadloginstr:[4,6],inadstr:[4,6],inaliasstr:4,inarg1str:4,inargdict:4,inarggset:4,inarggsettingsstr:4,inarglist:4,inargloggerstr:4,inargu:9,inargumentlist:[7,8],inbackend:[7,8],inbreaktriggerprocesswoexelist:4,incloseforcebool:4,includ:[7,8],incmdstr:4,incontenttypestr:4,incontrolspecificationarrai:[7,8],incpkeystr:4,indef:4,indefnamestr:4,indepthbitint:4,index:[3,4,5,7,8,9],indict:5,indomainstr:4,inel:[7,8],inelementspecif:[7,8],inencodingstr:4,infiledatabase64str:4,infiledatabyt:4,infiledatastr:4,infilepathstr:4,inflagforceclosebool:4,inflaggetabspathbool:4,inflagraiseexcept:[7,8],inflagwaitallinmo:[7,8],info:[4,5,6,8,9],infolderpathstr:4,inform:[3,4],infrastructur:13,ingset:[3,4,6],ingsettingsclientdict:5,inhashkeystr:5,inheightpxint:4,inherit:[8,9],inhostfilepathstr:4,inhostnamestr:4,inhoststr:[4,5],inhtmlrenderdef:4,init:[3,4,5,6],initdatetim:5,initi:4,injsinitgeneratordef:4,injsongeneratordef:4,inkeylist:4,inkeystr:5,inkwargumentobject:[7,8],inlogg:[4,5],inloginstr:[4,5],inmatchtypestr:4,inmethodstr:4,inmodestr:[3,4,5,6],inmodulepathstr:4,inpasswordstr:[4,5],inpathstr:4,inportint:4,inportstr:[4,5],inprocessnamewexestr:4,inprocessnamewoexelist:4,inprocessnamewoexestr:4,input:5,inrdpfilepathstr:4,inrdpsessionkeystr:[4,5],inrdptemplatedict:4,inrequest:4,inrolehierarchyalloweddict:[4,6],inrolekeylist:4,inrowcountint:5,inrunasyncbool:4,insert:6,inshareddrivelist:4,inspecificationlist:[7,8],inspecificationlistlist:[7,8],instal:13,instanc:[3,4,7,8,9],instopprocessnamewoexestr:4,insupertokenstr:[4,6],intellig:9,interact:[3,4],interest:4,interfac:[3,8,9,13],internet:12,interpret:3,interv:[4,5],intervalsecfloat:5,intimehhmmstr:4,inuioselector:[7,8],inurllist:[4,6],inurlstr:4,inusebothmonitorbool:4,inusernamestr:4,inuserstr:4,invalu:4,inwaitsec:[7,8],inweekdaylist:4,inwidthpxint:4,is_en:[8,9],is_vis:[8,9],islistenbool:5,isresponsiblebool:4,it4busi:2,item:[3,4,7,8,9],iter:5,ivan:13,ivanmaslov:2,join:6,jsinitgeneratordef:5,json:[3,4,5],jsongeneratordef:5,jsrender:10,jsview:10,just:9,kb2999226:0,keep:3,kei:[4,8,9],keyboard:[2,10,13],keystr:5,kill:4,killer:13,know:3,known:13,kwarg:3,lactivityitem:4,laliasstr:4,last:5,latest:[8,9],launch:4,left:3,len:5,let:13,level:[8,9,13],levelnam:5,lib:9,librari:[8,9],licens:13,lifetim:5,lifetimerequestsecfloat:5,lifetimesecfloat:5,light:3,like:[3,13],line:9,link:[3,4,5,13],linkedin:2,list:[3,4,5,7,8,9],listen:4,listenport:5,listenport_:5,listenurllist:5,listread:5,litem:9,load:5,local:[4,5],localhost:5,locat:4,log:[4,5,6,13],logger:[4,5,6],loggerdumploghandleradd:5,loggerhandlerdumploglist:5,login:[4,5,6],logoff:[4,5],logviewerbool:[5,6],lol:12,look:[3,4,5,12,13],lookmachinescreenshot:5,loop:9,low:9,lowercas:5,lprocessisstartedbool:4,lprocesslist:4,lpyopenrpasourcefolderpathstr:6,lrdpitemdict:4,lrdptemplatedict:4,lresult:5,lresultdict:[4,5],luacclientdict:6,lxml:10,machin:[0,4,5,6,13],machina:5,mail:2,main:[3,4,5,6,9],maintain:13,make:9,makedir:5,manag:4,mani:[3,7,8],manipul:[4,13],march:1,maslov:13,master:0,match:9,matchtyp:5,max:5,maxim:[7,8],maximum:[8,9],mayb:4,mechan:3,mega:3,megafind:2,memori:4,merg:4,messag:5,method:[5,8,9],methodmatchurl:5,methodmatchurlbeforelist:5,mhandlerdumploglist:5,microsoft:[0,8,9],minim:[7,8],miss:[8,9],mit:[2,13],mmstr:5,mode:12,model:[2,8,9],modul:[3,4,5,6],moduletocal:4,mondai:4,monei:13,monitor:4,more:[3,9],mous:13,mrobotlogg:5,mrobotloggerfh:5,mrobotloggerformatt:5,must:5,name:[3,4,5,6,7,8,9],namewoexestr:4,namewoexeupperstr:4,need:[0,2,3,4,5,7,8,9,13],nest:4,net:[4,8],never:4,newkeydict:4,newkeylist:4,newvalu:4,next:[8,9],non:13,none:[3,4,5,7,8],notat:[7,8,9],notepad:[4,5,7,8],noth:4,nothingbool:5,now:[4,5],nul:9,object:[3,4,5,7,8,9,13],occupi:4,ocr:9,octet:4,off:5,old:[4,5,6,7,8],one:[5,6],onli:[0,1,4,5,7,8,9],onlin:13,open:[12,13],opencv:[0,10,13],openrpa52zzz:6,openrpa:[0,5],openrpa_32:12,openrpa_64:12,openrpaorchestr:9,openrparesourceswpy32:9,openrparesourceswpy64:9,openrparobotdaemon:5,opensourc:13,oper:[0,4,6],opera:[9,13],option:[5,8,9],orc:5,orchestr:[5,6],orchestratormain:9,orchestratorrestart:4,orchestratorsessionsav:4,orchestratorstart:5,order:[3,9,12],org:[2,9],oscmd:4,oscredentialsverifi:4,other:[7,8,9,13],our:[8,9],out:4,outargu:9,outlin:[7,8],output:5,outstr:4,over:2,overwrit:5,own:[3,9,13],packag:[0,6,7],page:[4,5],page_sourc:9,pai:[3,6],panel:[4,5,6,12],paramet:[3,4,5,7,8],parent:[7,8,9,12],parti:13,pass:[4,5],password:[4,5],path:[4,5,6,9],paus:9,pdb:5,per:6,perfom:[9,13],perform:13,period:5,phone:3,pid:4,pil:10,pipupgrad:5,pixel:[4,5],plan:[1,4],platform:13,pleas:13,plu:4,port:[4,5],portabl:[0,9],possibl:13,post:[4,5],postfix:4,power:13,practic:13,prefer:2,previou:4,print:[6,9],process:[3,5,7,8,9,11,13],processdetaillist:4,processisstart:4,processlistget:4,processnam:4,processor:[5,13],processoractivityitemappend:4,processoractivityitemcr:4,processoraliasdefcr:4,processoraliasdefupd:4,processordict:5,processstart:[4,5],processstartifturnedoff:5,processstop:[4,5],processwoexelist:4,processwoexeupperlist:4,product:9,program:[4,5],progress:1,project:[3,13],properti:13,protocol:3,provid:[8,9],psutil:[6,10],pull:5,purpos:[1,2],pwa:[8,9],pwaspecif:[8,9],pwaspecification_get_pwaappl:[7,8],pwaspecification_get_uio:[7,8],pyautogui:[2,9,10,13],pycon:9,pymupdf:10,pyopenrpa:[0,1,2,3,5,6,8,9,11],pyopenrpadict:[5,6],pyrobot_cp:6,python:[0,3,5,7,8,10,12,13],pythonstart:4,pywin32:[2,10],pywinauto:[2,7,8,9,10],queue:[1,3,4,5,13],queuelist:5,r01:5,r01_integrationorderout:5,r01_orchestratortorobot:5,rais:4,rdp:[3,4,5],rdpactiv:5,rdpconfigurationdict:5,rdpkei:4,rdpkeydict:[5,6],rdpkeystr:5,rdplist:[4,5],rdpsession:13,rdpsessioncmdrun:4,rdpsessionconnect:[4,5],rdpsessiondisconnect:[4,5],rdpsessiondublicatesresolv:4,rdpsessionfilereciev:5,rdpsessionfilesend:5,rdpsessionfilestoredreciev:4,rdpsessionfilestoredsend:4,rdpsessionkei:5,rdpsessionkeystr:5,rdpsessionlogoff:[4,5],rdpsessionmonitorstop:4,rdpsessionprocessstart:5,rdpsessionprocessstartifnotrun:4,rdpsessionprocessstop:4,rdpsessionreconnect:[4,5],rdpsessionresponsibilitycheck:4,rdptemplatecr:4,read:5,readi:0,readthedoc:[8,9],reciev:[4,5],recognit:13,reconnect:4,reconnectbool:5,recurs:[8,9],reestr_otgruzok:5,refactor:1,refer:13,refresh:5,refreshsecond:5,regener:4,regular:[8,9],rel:[4,5,6],reliabl:13,rememb:5,remot:[4,5,13],render:12,renderfunct:5,renderrobotr01:5,report:5,reqir:3,request:[4,5,10],requesttimeoutsecfloat:5,requir:[4,13],resolut:[4,5],resourc:[0,9],respons:[4,5],responsecontenttyp:5,responsedefrequestglob:5,responsefilepath:5,responsefolderpath:5,responsibilitycheckintervalsec:5,restart:[4,5],restartorchestr:5,restartorchestratorbool:[5,6],restartorchestratorgitpullbool:[5,6],restartpcbool:[5,6],restor:[7,8],restrict:13,restructuredtext:[4,8],result:[4,5,7,8,9],returnbool:5,rich_text:[8,9],rich_text_r:[8,9],roadmap:13,robot:[3,4,5,6],robot_r01:5,robot_r01_help:5,robotlist:5,robotrdpact:[4,5],rolehierarchyalloweddict:5,root:[4,5],row:5,rpa:[4,5,13],rpatestdirtest:4,rst:[4,8],ruledomainuserdict:5,rulemethodmatchurlbeforelist:5,run:[0,4,5,6,7,8,13],russia:[2,13],russian:13,safe:[4,7,8],same:[7,8],save:4,schedul:[3,13],scheduleractivitytimeaddweekli:4,schedulerdict:5,scopesrcul:5,screen:[4,5,13],screenshot:[0,5,13],screenshotviewerbool:[5,6],script:[3,13],search:[4,13],sec:5,second:[4,5,7,8,9],section:4,see:[0,1,4,5,6,7,8],select:[7,8],selector:[7,8,9,12,13],selenium:[2,10,13],semant:[2,10],send:[4,5],send_kei:9,sent:4,sequenc:3,server:[0,3,5,12],serverdict:5,serverset:5,sesion:[4,5],session:[4,5],sessionguidstr:5,sessionhex:[4,5],sessionisignoredbool:[4,5],sessioniswindowexistbool:[4,5],sessioniswindowresponsiblebool:[4,5],set:[4,5,6,7,8,13],set_trac:5,setformatt:5,setlevel:5,settingstempl:[3,6],settingsupd:6,setup:5,sever:[3,9,13],share:4,shareddrivelist:[4,5],shell:[4,13],should:5,show:5,side:[4,5],signal:4,simplifi:9,sinc:13,singl:3,singleton:4,skype:2,sleep:9,socket:3,solut:[0,9,13],some:[3,4,5,13],sort:4,sourc:[3,4,6,7,8,13],sourceforg:[4,8],space:3,special:5,specif:[8,9],sphinx:3,standart:5,start:[0,4,5,8,9,12,13],statu:4,stdout:[5,6],stop:[4,8,9,13],storag:[5,13],store:5,str:[3,4,5,7,8,9],stream:4,streamhandl:5,strftime:5,string:[4,7,8],struct:5,structur:[3,4,5],studio:[3,11,12],subprocess:9,success:4,successfulli:[0,4,8,9,13],sundai:4,supertoken:[4,6],superus:6,supetoken:4,supplement:[8,9],support:[0,3,6,9,12,13],symbol:3,sync:3,sys:[5,6,9],system:13,tablet:3,technic:[3,5],technicalsessionguidcach:5,telegram:2,templat:[6,13],terminolog:[8,9],tesseract:10,test2:5,test:[4,5,6],testcontrolpanelkei:5,testdef:4,testdefalia:4,testdir:4,testdirtest:4,testrdp:5,text:[4,9,12],than:3,thank:[2,13],theori:13,thi:[4,5,8,9,13],thought:4,thread:[3,5],threadidint:5,thursdai:4,thx:[8,9],time:[4,5,9,13],timehh:5,titl:[7,8,9],title_r:[8,9],todo:4,token:4,tokendatetim:5,too:[4,5],tool:[3,9,12],top:[7,8],tor:13,track:4,transmiss:4,transmit:[3,4],tree:13,trigger:[4,5],ttt:[3,5],turn:5,turpl:3,tutori:1,txt:4,type:[4,5,6],uac:13,uackeylistcheck:4,uacsupertokenupd:[4,6],uacupd:[4,6],uia:[7,8,9],uidesktop:[1,7],uio:[7,12,13],uioactiv:[7,8,9],uioei:[8,9],uioinfo:[8,9],uioselector:7,uioselector_exist_bool:[7,8],uioselector_focushighlight:[7,8,9],uioselector_get_bitnessint:[7,8],uioselector_get_bitnessstr:[7,8],uioselector_get_uio:[7,8,9],uioselector_get_uioactivitylist:[7,8],uioselector_get_uioinfo:[7,8],uioselector_get_uiolist:[7,8],uioselector_getchildlist_uiolist:[7,8],uioselector_highlight:[7,8],uioselector_safeotherget_process:[7,8],uioselector_searchchildbymouse_uio:[7,8],uioselector_searchchildbymouse_uiotre:[7,8],uioselector_structure_exampl:[8,9],uioselector_tryrestore_dict:[7,8],uioselectorsecs_waitappear_bool:[7,8],uioselectorsecs_waitdisappear_bool:[7,8],uioselectorssecs_waitappear_list:[7,8],uioselectorssecs_waitdisappear_list:[7,8],uioselectoruioactivity_run_dict:[7,8],uiotre:[8,9],under:13,unicodelab:[0,2,9],univers:3,unzip:0,updat:[4,5],upper:5,url:[4,5],url_:5,urllist:5,usag:[4,9],use:[0,3,4,5,7,8,13],used:4,useful:[8,9],user:[3,4,5,6,8,9,13],user_99:4,user_pass_her:4,useradstr:5,usernam:[4,5],usernameupperstr:4,userupperstr:5,using:[4,8,9,13],utf:4,util:5,valu:[4,7,8],variant:4,ver:[8,9],verifi:4,version:[1,6,9,12],versioncheck:6,versionstr:5,video:9,viewer:[5,13],virtual:5,visibl:[8,9],vision:[9,13],vista:0,visual:13,vms:4,wai:[4,6,9,13],wait:[7,8,9,12],want:[3,5,12,13],warn:6,web:[3,5,13],webcpupd:4,webdriv:9,weburlconnectdef:4,weburlconnectfil:4,weburlconnectfold:4,webuserinfoget:4,webuserissupertoken:4,webuseruachierarchyget:4,wednesdai:4,week:4,weekdai:[4,5],weekdaylist:5,well:13,were:13,whatsapp:2,when:[3,4,5,6,7,8,12],where:[4,8,9,12],which:[3,4,5,7,8,9,13],who:5,why:3,width:[4,5],wiki:9,win32:[7,13],win32api:10,win:4,window:[0,4,5,7,8,10],winpython:2,without:[0,4,5,13],wmi:10,work:[4,5,9],workingdirectorypathstr:5,world:[2,13],wpy32:[0,9],wpy64:[0,9],wrapper:[7,8],write:[3,9,13],www:[2,9,10],x32:[0,10,12],x64:[0,10,12],xlsx:5,yandex:13,you:[0,2,3,4,5,7,8,9,13],your:[0,9],zip:0},titles:["1. How to install","2. Roadmap","3. Copyrights & Contacts","1. Description","2. Defs","3. gSettings Template","4. How to start process","1. Description","2. Defs","3. How to use","4. Dependencies","1. Description","2. How to use","Welcome to pyOpenRPA\u2019s wiki"],titleterms:{The:[8,9,12,13],Use:9,__orchestrator__:4,about:[9,13],access:[8,9],action:12,agent:4,app:[8,9],architectur:3,autom:[8,9],button:12,captur:9,check:0,choos:12,click:12,cmd:9,compon:[2,3],concept:3,configur:3,contact:2,content:[12,13],copyright:2,creat:9,ctrl:12,def:[4,8],definit:[8,9],depend:[2,10],descript:[3,7,11,12],desktop:[8,9],dict:3,dll:[8,9],donat:13,exampl:[8,9],execut:9,expand:12,extract:12,file:9,founder:2,from:9,global:3,group:4,gset:[4,5],gui:12,guid:13,has:13,hold:12,hover:12,how:[0,3,6,9,12],imag:9,instal:0,interest:12,ivan:2,kei:12,keyboard:9,licens:2,list:12,main:13,manipul:9,maslov:2,modul:[8,9],mous:[9,12],object:12,openrpa:[8,9],orchestr:[3,4,13],parti:2,practic:9,process:[4,6],processor:[3,4],progress:13,properti:12,pyopenrpa:[4,7,13],python:[4,9],rdpsession:4,recognit:9,refer:[3,4,8],requir:0,result:12,roadmap:1,robot:[7,8,9,13],rpa:9,run:12,schedul:4,screen:9,screenshot:12,script:9,search:12,second:12,select:12,selenium:9,set:3,shown:12,start:6,structur:[8,9,13],studio:[9,13],system:0,templat:5,theori:9,tool:13,tree:12,turn:12,tutori:13,uac:4,uidesktop:[8,9],uio:[8,9],uioselector:[8,9],use:[9,12],viewer:12,web:[4,9],welcom:13,what:[8,9],wiki:13,win32:[8,9],x32:9,x64:9,you:12}}) \ No newline at end of file diff --git a/Wiki/ENG_Guide/markdown/03_Copyrights_Contacts.md b/Wiki/ENG_Guide/markdown/03_Copyrights_Contacts.md index b6097950..6af79196 100644 --- a/Wiki/ENG_Guide/markdown/03_Copyrights_Contacts.md +++ b/Wiki/ENG_Guide/markdown/03_Copyrights_Contacts.md @@ -17,11 +17,14 @@ Thank you! * Skype: MegaFinder -* Facebook: [https://www.facebook.com/RU.Ivan.Maslov](https://www.facebook.com/RU.Ivan.Maslov) +* Facebook: [https://www.facebook.com/RU.IT4Business](https://www.facebook.com/RU.IT4Business) * LinkedIn: [https://www.linkedin.com/in/RU-IvanMaslov/](https://www.linkedin.com/in/RU-IvanMaslov/) + +* WhatsApp | Telegram: +7 906 722 39 25 + ## 3-rd party components license dependencies diff --git a/Wiki/ENG_Guide/markdown/Robot/01_Robot.md b/Wiki/ENG_Guide/markdown/Robot/01_Robot.md index 94fdb182..4a844be5 100644 --- a/Wiki/ENG_Guide/markdown/Robot/01_Robot.md +++ b/Wiki/ENG_Guide/markdown/Robot/01_Robot.md @@ -89,6 +89,31 @@ Wait for UI object will disappear in GUI for inWaitSecs seconds. +### pyOpenRPA.Robot.UIDesktop.UIOSelectorUIOActivity_Run_Dict(inUIOSelector, inActionName, inArgumentList=None, inkwArgumentObject=None) +Run the activity in UIO (UI Object) + + +* **Parameters** + + + * **inUIOSelector** – UIOSelector - List of items, which contains condition attributes + + + * **inActionName** – UIOActivity (name) activity name string from Pywinauto + + + * **inArgumentList** – + + + * **inkwArgumentObject** – + + + +* **Returns** + + + + ### pyOpenRPA.Robot.UIDesktop.UIOSelector_Exist_Bool(inUIOSelector) Check if object is exist by the UIO selector. @@ -105,6 +130,40 @@ Check if object is exist by the UIO selector. +### pyOpenRPA.Robot.UIDesktop.UIOSelector_FocusHighlight(inUIOSelector) +Set focus and highlight (draw outline) the element (in app) by the UIO selector. + + +* **Parameters** + + **inUIOSelector** – UIOSelector - List of items, which contains condition attributes + + + +* **Returns** + + + + +### pyOpenRPA.Robot.UIDesktop.UIOSelector_GetChildList_UIOList(inUIOSelector=None, inBackend='win32') +Get list of child UIO’s by the parent UIOSelector + + +* **Parameters** + + + * **inUIOSelector** – UIOSelector - List of items, which contains condition attributes + + + * **inBackend** – “win32” or “uia” + + + +* **Returns** + + + + ### pyOpenRPA.Robot.UIDesktop.UIOSelector_Get_BitnessInt(inSpecificationList) Detect process bitness by the UI Object UIO Selector. @@ -159,6 +218,36 @@ Get the pywinauto object by the UIO selector. +### pyOpenRPA.Robot.UIDesktop.UIOSelector_Get_UIOActivityList(inUIOSelector) +Get the list of the UI object activities + + +* **Parameters** + + **inUIOSelector** – UIOSelector - List of items, which contains condition attributes + + + +* **Returns** + + + + +### pyOpenRPA.Robot.UIDesktop.UIOSelector_Get_UIOInfo(inUIOSelector) +Get the UIO dict of the attributes + + +* **Parameters** + + **inUIOSelector** – UIOSelector - List of items, which contains condition attributes + + + +* **Returns** + + + + ### pyOpenRPA.Robot.UIDesktop.UIOSelector_Get_UIOList(inSpecificationList, inElement=None, inFlagRaiseException=True) Get the UIO list by the selector @@ -181,6 +270,21 @@ Get the UIO list by the selector +### pyOpenRPA.Robot.UIDesktop.UIOSelector_Highlight(inUIOSelector) +Highlight (draw outline) the element (in app) by the UIO selector. + + +* **Parameters** + + **inUIOSelector** – UIOSelector - List of items, which contains condition attributes + + + +* **Returns** + + + + ### pyOpenRPA.Robot.UIDesktop.UIOSelector_SafeOtherGet_Process(inUIOSelector) Safe get other process or None if destination app is the other/same bitness @@ -231,6 +335,21 @@ UIOSelector (see description on the top of the document) +### pyOpenRPA.Robot.UIDesktop.UIOSelector_TryRestore_Dict(inSpecificationList) +Try to restore (maximize) window, if it’s minimized. (!IMPORTANT! When use UIA framework minimized windows doesn’t appear by the UIOSelector. You need to try restore windows and after that try to get UIO) + + +* **Parameters** + + **inSpecificationList** – UIOSelector - List of items, which contains condition attributes + + + +* **Returns** + + + + ### pyOpenRPA.Robot.UIDesktop.UIOSelectorsSecs_WaitAppear_List(inSpecificationListList, inWaitSecs, inFlagWaitAllInMoment=False) Wait for many UI object will appear in GUI for inWaitSecs seconds. diff --git a/Wiki/ENG_Guide/markdown/Robot/02_Defs.md b/Wiki/ENG_Guide/markdown/Robot/02_Defs.md index 1fc659dc..6939178f 100644 --- a/Wiki/ENG_Guide/markdown/Robot/02_Defs.md +++ b/Wiki/ENG_Guide/markdown/Robot/02_Defs.md @@ -1,6 +1,142 @@ # 2. Defs -## pyOpenRPA.Robot.UIDesktop +## Desktop app UI access (win32 and UI automation dlls) + +### Definitions + + +* **UIO** - UI Object (class of pywinauto UI object) [pywinauto.base_wrapper] + + +* **UIOSelector** - List of dict (key attributes) + + +* **PWA** - PyWinAuto + + +* **PWASpecification** - List of dict (key attributes in pywinauto.find_window notation) + + +* **UIOTree** - Recursive Dict of Dict … (UI Parent -> Child hierarchy) + + +* **UIOInfo** - Dict of UIO attributes + + +* **UIOActivity** - Activity of the UIO (UI object) from the Pywinauto module + + +* **UIOEI** - UI Object info object + +### What is UIO? + +UIO is a User Interface Object (pyOpenRPA terminology). For maximum compatibility, this instance is inherited from the object model developed in the [pywinauto library (click to get a list of available class functions)]([https://pywinauto.readthedocs.io/en/latest/code/pywinauto.base_wrapper.html](https://pywinauto.readthedocs.io/en/latest/code/pywinauto.base_wrapper.html)). + +This approach allows us to implement useful functionality that has already been successfully developed in other libraries, and Supplement it with the missing functionality. In our case, the missing functionality is the ability to dynamically access UIO objects using UIO selectors. + +### UIOSelector structure & example + + +UIOSelector is the list of condition items for the UIO in GUI. Each item has condition attributes for detect applicable UIO. Here is the description of the available condition attributes in item. + + + +``` +** +``` + +Desciption\*\*
    + + + + ``` + `` + ``` + + + + ``` + ` + ``` + + +[ + + { + + “depth_start” :: [int, start from 1] :: the depth index, where to start check the condition list (default 1), + “depth_end” :: [int, start from 1] :: the depth index, where to stop check the condition list (default 1), + “ctrl_index” || “index” :: [int, starts from 0] :: the index of the UIO in parent UIO child list, + “title” :: [str] :: the condition for the UIO attribute *title*, + “title_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute *title*, + “rich_text” :: [str] :: the condition for the UIO attribute *rich_text*, + “rich_text_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute *rich_text*, + “class_name” :: [str] :: the condition for the UIO attribute *class_name*, + “class_name_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute *class_name*, + “friendly_class_name” :: [str] :: the condition for the UIO attribute *friendly_class_name*, + “friendly_class_name_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute *friendly_class_name*, + “control_type” :: [str] :: the condition for the UIO attribute *control_type*, + “control_type_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute *control_type*, + “is_enabled” :: [bool] :: the condition for the UIO attribute *is_enabled*. If UI object is enabled on GUI, + “is_visible” :: [bool] :: the condition for the UIO attribute *is_visible*. If UI object is visible on GUI, + “backend” :: [str, “win32” || “uia”] :: the method of UIO extraction (default “win32”). ATTENTION! Current option can be only for the first item of the UIO selector. For the next items this option will be implemented from the first item. + + }, + { … specification next level UIO } + +] + + + + ``` + `` + ``` + + + + ``` + ` + ``` + + +**The UIO selector example** + + + + ``` + `` + ``` + + + + ``` + ` + ``` + + +[ + + {“class_name”:”CalcFrame”, “backend”:”win32”}, # 1-st level UIO specification + {“title”:”Hex”, “depth_start”:3, “depth_end”: 3} # 3-rd level specification (because of attribute depth_start|depth_stop) + +] + + + + ``` + `` + ``` + + + + ``` + ` + ``` + + +### The UIDesktop module (OpenRPA/Robot/UIDesktop.py) + +The UIDesktop is extension of the pywinauto module which provide access to the desktop apps by the **win32** and **ui automation** dll frameworks (big thx to the Microsoft :) ). ``` # EXAMPLE 1 @@ -40,11 +176,26 @@ UIDesktop.UIOSelector_Get_UIO( | Wait for UI object will disappear in GUI for inWaitSecs seconds. | +| `UIOSelectorUIOActivity_Run_Dict`(…[, …]) + + | Run the activity in UIO (UI Object) + + | | `UIOSelector_Exist_Bool`(inUIOSelector) | Check if object is exist by the UIO selector. | +| `UIOSelector_FocusHighlight`(inUIOSelector) + + | Set focus and highlight (draw outline) the element (in app) by the UIO selector. + + | +| `UIOSelector_GetChildList_UIOList`([…]) + + | Get list of child UIO’s by the parent UIOSelector + + | | `UIOSelector_Get_BitnessInt`(inSpecificationList) | Detect process bitness by the UI Object UIO Selector. @@ -60,11 +211,26 @@ UIDesktop.UIOSelector_Get_UIO( | Get the pywinauto object by the UIO selector. | +| `UIOSelector_Get_UIOActivityList`(inUIOSelector) + + | Get the list of the UI object activities + + | +| `UIOSelector_Get_UIOInfo`(inUIOSelector) + + | Get the UIO dict of the attributes + + | | `UIOSelector_Get_UIOList`(inSpecificationList) | Get the UIO list by the selector | +| `UIOSelector_Highlight`(inUIOSelector) + + | Highlight (draw outline) the element (in app) by the UIO selector. + + | | `UIOSelector_SafeOtherGet_Process`(inUIOSelector) | Safe get other process or None if destination app is the other/same bitness @@ -80,6 +246,11 @@ UIDesktop.UIOSelector_Get_UIO( | !!!!Safe call is included (you can set activity and UIDesktop will choose the bitness and return the result)!!!!! | +| `UIOSelector_TryRestore_Dict`(inSpecificationList) + + | Try to restore (maximize) window, if it’s minimized. + + | | `UIOSelectorsSecs_WaitAppear_List`(…[, …]) | Wait for many UI object will appear in GUI for inWaitSecs seconds. @@ -175,6 +346,31 @@ Wait for UI object will disappear in GUI for inWaitSecs seconds. +### pyOpenRPA.Robot.UIDesktop.UIOSelectorUIOActivity_Run_Dict(inUIOSelector, inActionName, inArgumentList=None, inkwArgumentObject=None) +Run the activity in UIO (UI Object) + + +* **Parameters** + + + * **inUIOSelector** – UIOSelector - List of items, which contains condition attributes + + + * **inActionName** – UIOActivity (name) activity name string from Pywinauto + + + * **inArgumentList** – + + + * **inkwArgumentObject** – + + + +* **Returns** + + + + ### pyOpenRPA.Robot.UIDesktop.UIOSelector_Exist_Bool(inUIOSelector) Check if object is exist by the UIO selector. @@ -191,6 +387,40 @@ Check if object is exist by the UIO selector. +### pyOpenRPA.Robot.UIDesktop.UIOSelector_FocusHighlight(inUIOSelector) +Set focus and highlight (draw outline) the element (in app) by the UIO selector. + + +* **Parameters** + + **inUIOSelector** – UIOSelector - List of items, which contains condition attributes + + + +* **Returns** + + + + +### pyOpenRPA.Robot.UIDesktop.UIOSelector_GetChildList_UIOList(inUIOSelector=None, inBackend='win32') +Get list of child UIO’s by the parent UIOSelector + + +* **Parameters** + + + * **inUIOSelector** – UIOSelector - List of items, which contains condition attributes + + + * **inBackend** – “win32” or “uia” + + + +* **Returns** + + + + ### pyOpenRPA.Robot.UIDesktop.UIOSelector_Get_BitnessInt(inSpecificationList) Detect process bitness by the UI Object UIO Selector. @@ -245,6 +475,36 @@ Get the pywinauto object by the UIO selector. +### pyOpenRPA.Robot.UIDesktop.UIOSelector_Get_UIOActivityList(inUIOSelector) +Get the list of the UI object activities + + +* **Parameters** + + **inUIOSelector** – UIOSelector - List of items, which contains condition attributes + + + +* **Returns** + + + + +### pyOpenRPA.Robot.UIDesktop.UIOSelector_Get_UIOInfo(inUIOSelector) +Get the UIO dict of the attributes + + +* **Parameters** + + **inUIOSelector** – UIOSelector - List of items, which contains condition attributes + + + +* **Returns** + + + + ### pyOpenRPA.Robot.UIDesktop.UIOSelector_Get_UIOList(inSpecificationList, inElement=None, inFlagRaiseException=True) Get the UIO list by the selector @@ -267,6 +527,21 @@ Get the UIO list by the selector +### pyOpenRPA.Robot.UIDesktop.UIOSelector_Highlight(inUIOSelector) +Highlight (draw outline) the element (in app) by the UIO selector. + + +* **Parameters** + + **inUIOSelector** – UIOSelector - List of items, which contains condition attributes + + + +* **Returns** + + + + ### pyOpenRPA.Robot.UIDesktop.UIOSelector_SafeOtherGet_Process(inUIOSelector) Safe get other process or None if destination app is the other/same bitness @@ -317,6 +592,21 @@ UIOSelector (see description on the top of the document) +### pyOpenRPA.Robot.UIDesktop.UIOSelector_TryRestore_Dict(inSpecificationList) +Try to restore (maximize) window, if it’s minimized. (!IMPORTANT! When use UIA framework minimized windows doesn’t appear by the UIOSelector. You need to try restore windows and after that try to get UIO) + + +* **Parameters** + + **inSpecificationList** – UIOSelector - List of items, which contains condition attributes + + + +* **Returns** + + + + ### pyOpenRPA.Robot.UIDesktop.UIOSelectorsSecs_WaitAppear_List(inSpecificationListList, inWaitSecs, inFlagWaitAllInMoment=False) Wait for many UI object will appear in GUI for inWaitSecs seconds. diff --git a/Wiki/ENG_Guide/markdown/Robot/03_HowToUse.md b/Wiki/ENG_Guide/markdown/Robot/03_HowToUse.md index 7fc99c49..f1edbef6 100644 --- a/Wiki/ENG_Guide/markdown/Robot/03_HowToUse.md +++ b/Wiki/ENG_Guide/markdown/Robot/03_HowToUse.md @@ -1,24 +1,8 @@ # 3. How to use -## Content - - -* [About](#about) - - -* [How to use](#way-to-use) - - -* [Create python script](#create-python-script) - - -* [Execute python script](#execute-python-script) - -## About - The Robot tool is the main module for production process automation. It has no graphic/console interface. All low-level actions to OS are perfoming by the Robot tool in OpenRPA. -## Way to use +## How to execute RPA script You can use the robot by the several ways: @@ -42,7 +26,7 @@ import cv2 # [Computer vision](https://gitlab.com/UnicodeLabs/OpenRPA/wikis/05.4 import keyboard #[Keyboard manipulation](https://gitlab.com/UnicodeLabs/OpenRPA/wikis/05.3.-Theory-&-practice:-Keyboard-&-mouse-manipulation)
    ``` -## Execute python script +### Execute python script The OpenRPA is fully portable solution. It contains own python enviroment both 32 and 64 bit versions. So, you can execute your python script in several ways: - Execute in python x32 (OpenRPAResourcesWPy32-3720python-3.7.2) @@ -78,7 +62,7 @@ copy /Y ..\Resources\WPy32-3720\python-3.7.2\python.exe ..\Resources\WPy32-3720\ pause >nul ``` -## Use in studio script (n/a) +### Use in studio script (n/a) ``` import sys @@ -88,13 +72,178 @@ import keyboard import subprocess import time -#Highlight the UI Object in Folder explorer
    -GUI.UIOSelector_FocusHighlight([{"class_name":"CabinetWClass","backend":"uia"},{"ctrl_index":2},{"ctrl_index":0},{"ctrl_index":2},{"ctrl_index":0}])
    +#Highlight the UI Object in Folder explorer +GUI.UIOSelector_FocusHighlight([{"class_name":"CabinetWClass","backend":"uia"},{"ctrl_index":2},{"ctrl_index":0},{"ctrl_index":2},{"ctrl_index":0}]) -#Wait 2 seconds
    -time.sleep(3)
    +#Wait 2 seconds +time.sleep(3) -#Loop: get child element of UI List
    -for lItem in GUI.UIOSelector_Get_UIO([{"class_name":"CabinetWClass","backend":"uia"},{"ctrl_index":2},{"ctrl_index":0},{"ctrl_index":2},{"ctrl_index":0}]).children():
    -         print(str(lItem)) +#Loop: get child element of UI List +for lItem in GUI.UIOSelector_Get_UIO([{"class_name":"CabinetWClass","backend":"uia"},{"ctrl_index":2},{"ctrl_index":0},{"ctrl_index":2},{"ctrl_index":0}]).children(): + print(str(lItem)) ``` + +Here you can find the docs and examples of the OpenRPA desktop (GUI) app access. + +## Theory & practice. Desktop app UI access (win32 and UI automation dlls) + +### Definitions + +**UIO** - UI Object (class of pywinauto UI object) [pywinauto.base_wrapper]
    +**UIOSelector** - List of dict (key attributes)
    +**PWA** - PyWinAuto
    +**PWASpecification** - List of dict (key attributes in pywinauto.find_window notation)
    +**UIOTree** - Recursive Dict of Dict … (UI Parent -> Child hierarchy)
    +**UIOInfo** - Dict of UIO attributes
    +**UIOActivity** - Activity of the UIO (UI object) from the Pywinauto module
    +**UIOEI** - UI Object info object + +### What is UIO? + +UIO is a User Interface Object (pyOpenRPA terminology). For maximum compatibility, this instance is inherited from the object model developed in the [pywinauto library (click to get a list of available class functions)]([https://pywinauto.readthedocs.io/en/latest/code/pywinauto.base_wrapper.html](https://pywinauto.readthedocs.io/en/latest/code/pywinauto.base_wrapper.html)). + +This approach allows us to implement useful functionality that has already been successfully developed in other libraries, and Supplement it with the missing functionality. In our case, the missing functionality is the ability to dynamically access UIO objects using UIO selectors. + +### UIOSelector structure & example + + +UIOSelector is the list of condition items for the UIO in GUI. Each item has condition attributes for detect applicable UIO. Here is the description of the available condition attributes in item. + +**Desciption** + + + + ``` + `` + ``` + + + + ``` + ` + ``` + + +[ + + { + + “depth_start” :: [int, start from 1] :: the depth index, where to start check the condition list (default 1), + “depth_end” :: [int, start from 1] :: the depth index, where to stop check the condition list (default 1), + “ctrl_index” || “index” :: [int, starts from 0] :: the index of the UIO in parent UIO child list, + “title” :: [str] :: the condition for the UIO attribute *title*, + “title_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute *title*, + “rich_text” :: [str] :: the condition for the UIO attribute *rich_text*, + “rich_text_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute *rich_text*, + “class_name” :: [str] :: the condition for the UIO attribute *class_name*, + “class_name_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute *class_name*, + “friendly_class_name” :: [str] :: the condition for the UIO attribute *friendly_class_name*, + “friendly_class_name_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute *friendly_class_name*, + “control_type” :: [str] :: the condition for the UIO attribute *control_type*, + “control_type_re” :: [str] :: regular expression (python ver) for the condition for the UIO attribute *control_type*, + “is_enabled” :: [bool] :: the condition for the UIO attribute *is_enabled*. If UI object is enabled on GUI, + “is_visible” :: [bool] :: the condition for the UIO attribute *is_visible*. If UI object is visible on GUI, + “backend” :: [str, “win32” || “uia”] :: the method of UIO extraction (default “win32”). ATTENTION! Current option can be only for the first item of the UIO selector. For the next items this option will be implemented from the first item. + + }, + { … specification next level UIO } + +] + + + + ``` + `` + ``` + + + + ``` + ` + ``` + + +**The UIO selector example** + + + + ``` + `` + ``` + + + + ``` + ` + ``` + + +[ + + {“class_name”:”CalcFrame”, “backend”:”win32”}, # 1-st level UIO specification + {“title”:”Hex”, “depth_start”:3, “depth_end”: 3} # 3-rd level specification (because of attribute depth_start|depth_stop) + +] + + + + ``` + `` + ``` + + + + ``` + ` + ``` + + +### The UIDesktop module (OpenRPA/Robot/UIDesktop.py) + +The UIDesktop is extension of the pywinauto module which provide access to the desktop apps by the **win32** and **ui automation** dll frameworks (big thx to the Microsoft :) ). + + + +``` +* +``` + +Naming convention: __\*
    + +## Theory & practice. WEB app UI access (selenium) + +### About + +The pyOpenRPA support web app manipulation (by the Selenium lib). +More docs about selenium you can find here ([https://selenium-python.readthedocs.io/](https://selenium-python.readthedocs.io/)) + +### How to use + +To start use selenium just import selenium modules in the robot tool. Here is the example of the usage. + +``` +from selenium import webdriver +from selenium.webdriver.common.keys import Keys + +driver = webdriver.Chrome() +driver.get("http://www.python.org") +assert "Python" in driver.title +elem = driver.find_element_by_name("q") +elem.clear() +elem.send_keys("pycon") +elem.send_keys(Keys.RETURN) +assert "No results found." not in driver.page_source +driver.close() +``` + +## Theory & practice. Keyboard & mouse manipulation + +## Theory & practice. Screen capture & image recognition + +### How to automate image recognition on PC + +Here you can find any ways you need to use in your business case: +- Find the exact match on the screen with the other image +- Use text recognition module (OCR) +- Use computer vision (CV) to identify the objects on screen/image/video +- Use artificial intelligence (AI) to make custom identification/classification/text recognition diff --git a/Wiki/ENG_Guide/markdown/index.md b/Wiki/ENG_Guide/markdown/index.md index 442cd465..5d2de696 100644 --- a/Wiki/ENG_Guide/markdown/index.md +++ b/Wiki/ENG_Guide/markdown/index.md @@ -166,7 +166,7 @@ in QUEUE * 2. Defs - * pyOpenRPA.Robot.UIDesktop + * Desktop app UI access (win32 and UI automation dlls) * References @@ -175,19 +175,19 @@ in QUEUE * 3. How to use - * Content + * How to execute RPA script - * About + * Theory & practice. Desktop app UI access (win32 and UI automation dlls) - * Way to use + * Theory & practice. WEB app UI access (selenium) - * Execute python script + * Theory & practice. Keyboard & mouse manipulation - * Use in studio script (n/a) + * Theory & practice. Screen capture & image recognition * 4. Dependencies