OPEN_RPA_PUBLIC
/
ORPA-pyOpenRPA
2
0
Fork 1
Code Issues 2 Pull Requests Packages Projects Releases Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '2ad58efa4f'
${ noResults }
ORPA-pyOpenRPA/Wiki/ENG_Guide/html/Robot/03_HowToUse.html

459 lines
30 KiB
Raw Blame History

<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>3. How to use &mdash; pyOpenRPA v1.2.0 documentation</title>
<link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<link rel="shortcut icon" href="../_static/favicon.ico"/>
<!--[if lt IE 9]>
<script src="../_static/js/html5shiv.min.js"></script>
<![endif]-->
<script type="text/javascript" id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
<script src="../_static/jquery.js"></script>
<script src="../_static/underscore.js"></script>
<script src="../_static/doctools.js"></script>
<script type="text/javascript" src="../_static/js/theme.js"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="4. Dependencies" href="04_Dependencies.html" />
<link rel="prev" title="2. Defs" href="02_Defs.html" />
</head>
<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search" >
<a href="../index.html" class="icon icon-home"> pyOpenRPA
</a>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="../search.html" method="get">
<input type="text" name="q" placeholder="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
<p class="caption"><span class="caption-text">GENERAL</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../01_HowToInstall.html">1. How to install</a></li>
<li class="toctree-l1"><a class="reference internal" href="../02_RoadMap.html">2. Roadmap</a></li>
<li class="toctree-l1"><a class="reference internal" href="../03_Copyrights_Contacts.html">3. Copyrights &amp; Contacts</a></li>
</ul>
<p class="caption"><span class="caption-text">ROBOT</span></p>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="01_Robot.html">1. Description</a></li>
<li class="toctree-l1"><a class="reference internal" href="02_Defs.html">2. Defs</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">3. How to use</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#how-to-execute-rpa-script">How to execute RPA script</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#create-python-script">Create python script</a></li>
<li class="toctree-l3"><a class="reference internal" href="#execute-python-script">Execute python script</a></li>
<li class="toctree-l3"><a class="reference internal" href="#execute-in-the-python-x32">Execute in the Python x32</a></li>
<li class="toctree-l3"><a class="reference internal" href="#execute-in-the-python-x64">Execute in the Python x64</a></li>
<li class="toctree-l3"><a class="reference internal" href="#execute-from-cmd-file">Execute from .cmd file</a></li>
<li class="toctree-l3"><a class="reference internal" href="#use-in-studio-script-n-a">Use in studio script (n/a)</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#desktop-app-ui-access-win32-and-ui-automation-dlls">Desktop app UI access (win32 and UI automation dlls)</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#definitions">Definitions</a></li>
<li class="toctree-l3"><a class="reference internal" href="#what-is-uio">What is UIO?</a></li>
<li class="toctree-l3"><a class="reference internal" href="#uioselector-structure-example">UIOSelector structure &amp; example</a></li>
<li class="toctree-l3"><a class="reference internal" href="#the-uidesktop-module-openrpa-robot-uidesktop-py">The UIDesktop module (OpenRPA/Robot/UIDesktop.py)</a></li>
<li class="toctree-l3"><a class="reference internal" href="#id1">The UIDesktop module (OpenRPA/Robot/UIDesktop.py)</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#theory-practice-web-app-ui-access-selenium">Theory &amp; practice. WEB app UI access (selenium)</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#about">About</a></li>
<li class="toctree-l3"><a class="reference internal" href="#id4">How to use</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#theory-practice-keyboard-mouse-manipulation">Theory &amp; practice. Keyboard &amp; mouse manipulation</a></li>
<li class="toctree-l2"><a class="reference internal" href="#theory-practice-screen-capture-image-recognition">Theory &amp; practice. Screen capture &amp; image recognition</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#how-to-automate-image-recognition-on-pc">How to automate image recognition on PC</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="04_Dependencies.html">4. Dependencies</a></li>
</ul>
<p class="caption"><span class="caption-text">STUDIO</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../Studio/01_Studio.html">1. Description</a></li>
<li class="toctree-l1"><a class="reference internal" href="../Studio/02_HowToUse.html">2. How to use</a></li>
</ul>
<p class="caption"><span class="caption-text">ORCHESTRATOR</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../Orchestrator/01_Orchestrator.html">1. Description</a></li>
<li class="toctree-l1"><a class="reference internal" href="../Orchestrator/02_Defs.html">2. Defs</a></li>
<li class="toctree-l1"><a class="reference internal" href="../Orchestrator/03_gSettingsTemplate.html">3. gSettings Template</a></li>
<li class="toctree-l1"><a class="reference internal" href="../Orchestrator/04_HowToUse.html">4. How to use</a></li>
<li class="toctree-l1"><a class="reference internal" href="../Orchestrator/05_UAC.html">5. UAC - User Access Control</a></li>
<li class="toctree-l1"><a class="reference internal" href="../Orchestrator/06_Defs%20Managers.html">6. Defs Managers</a></li>
</ul>
<p class="caption"><span class="caption-text">AGENT</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../Agent/02_Defs.html">2. Defs</a></li>
</ul>
</div>
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
<nav class="wy-nav-top" aria-label="top navigation">
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="../index.html">pyOpenRPA</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="breadcrumbs navigation">
<ul class="wy-breadcrumbs">
<li><a href="../index.html" class="icon icon-home"></a> &raquo;</li>
<li>3. How to use</li>
<li class="wy-breadcrumbs-aside">
<a href="../_sources/Robot/03_HowToUse.rst.txt" rel="nofollow"> View page source</a>
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<div class="section" id="how-to-use">
<h1>3. How to use<a class="headerlink" href="#how-to-use" title="Permalink to this headline"></a></h1>
<p>The Robot tool is the main module for production process automation. It has no graphic/console interface. All low-level actions to OS are perfoming by the Robot tool in pyOpenRPA.</p>
<div class="section" id="how-to-execute-rpa-script">
<h2>How to execute RPA script<a class="headerlink" href="#how-to-execute-rpa-script" title="Permalink to this headline"></a></h2>
<p>You can use the robot by the several ways:</p>
<ul class="simple">
<li><p>In Python script</p></li>
<li><p>In Studio script (n/a)</p></li>
</ul>
<div class="section" id="create-python-script">
<h3>Create python script<a class="headerlink" href="#create-python-script" title="Permalink to this headline"></a></h3>
<p>In order to use robot just add Robot tool folder in work directory and add line “import GUI” in your script.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">sys</span>
<span class="n">sys</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s1">&#39;../../&#39;</span><span class="p">)</span>
<span class="kn">import</span> <span class="nn">selenium</span> <span class="c1"># [Web app access](https://gitlab.com/UnicodeLabs/OpenRPA/wikis/05.1.-Theory-&amp;-practice:-Web-app-access-(Chrome,-Firefox,-Opera))&lt;br&gt;</span>
<span class="kn">import</span> <span class="nn">GUI</span> <span class="c1"># [Win32 &amp; UI Automation access](https://gitlab.com/UnicodeLabs/OpenRPA/wikis/05.2.-Theory-&amp;-practice:-Desktop-app-UI-access-(win32-and-UI-automation-dlls)) &lt;br&gt;</span>
<span class="kn">import</span> <span class="nn">pyautogui</span> <span class="c1">#[Screen capture/recognition](https://gitlab.com/UnicodeLabs/OpenRPA/wikis/05.4.-Theory-&amp;-practice:-Screen-capture-&amp;-image-recognition) [#Mouse manipulation](https://gitlab.com/UnicodeLabs/OpenRPA/wikis/05.3.-Theory-&amp;-practice:-Keyboard-&amp;-mouse-manipulation)&lt;br&gt;</span>
<span class="kn">import</span> <span class="nn">cv2</span> <span class="c1"># [Computer vision](https://gitlab.com/UnicodeLabs/OpenRPA/wikis/05.4.-Theory-&amp;-practice:-Screen-capture-&amp;-image-recognition)&lt;br&gt;</span>
<span class="kn">import</span> <span class="nn">keyboard</span> <span class="c1">#[Keyboard manipulation](https://gitlab.com/UnicodeLabs/OpenRPA/wikis/05.3.-Theory-&amp;-practice:-Keyboard-&amp;-mouse-manipulation)&lt;br&gt;</span>
</pre></div>
</div>
</div>
<div class="section" id="execute-python-script">
<h3>Execute python script<a class="headerlink" href="#execute-python-script" title="Permalink to this headline"></a></h3>
<p>The pyOpenRPA is fully portable solution. It contains own python enviroment both 32 and 64 bit versions. So, you can execute your python script in several ways:
- Execute in python x32 (OpenRPAResourcesWPy32-3720python-3.7.2)
- Execute in python x64 (OpenRPAResourcesWPy64-3720python-3.7.2.amd64)
- Execute from .cmd file</p>
</div>
<div class="section" id="execute-in-the-python-x32">
<h3>Execute in the Python x32<a class="headerlink" href="#execute-in-the-python-x32" title="Permalink to this headline"></a></h3>
<p>To execute your python script in x32 bit version just write in command line from x32 python directory:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">cd</span> <span class="s2">&quot;\OpenRPA\Resources\WPy32-3720\python-3.7.2&quot;</span>
<span class="n">python</span><span class="o">.</span><span class="n">exe</span> <span class="s2">&quot;path to your python script.py&quot;</span>
</pre></div>
</div>
</div>
<div class="section" id="execute-in-the-python-x64">
<h3>Execute in the Python x64<a class="headerlink" href="#execute-in-the-python-x64" title="Permalink to this headline"></a></h3>
<p>To execute your python script in x32 bit version just write in command line from x32 python directory:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">cd</span> <span class="s2">&quot;\OpenRPA\Resources\WPy64-3720\python-3.7.2.amd64&quot;</span>
<span class="n">python</span><span class="o">.</span><span class="n">exe</span> <span class="s2">&quot;path to your python script.py&quot;</span>
</pre></div>
</div>
</div>
<div class="section" id="execute-from-cmd-file">
<h3>Execute from .cmd file<a class="headerlink" href="#execute-from-cmd-file" title="Permalink to this headline"></a></h3>
<p>In order to simplify the execution process you can write several code lines in file with the .cmd extension:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">cd</span> <span class="o">%~</span><span class="n">dp0</span>
<span class="n">copy</span> <span class="o">/</span><span class="n">Y</span> <span class="o">..</span>\<span class="n">Resources</span>\<span class="n">WPy32</span><span class="o">-</span><span class="mi">3720</span>\<span class="n">python</span><span class="o">-</span><span class="mf">3.7</span><span class="o">.</span><span class="mi">2</span>\<span class="n">python</span><span class="o">.</span><span class="n">exe</span> <span class="o">..</span>\<span class="n">Resources</span>\<span class="n">WPy32</span><span class="o">-</span><span class="mi">3720</span>\<span class="n">python</span><span class="o">-</span><span class="mf">3.7</span><span class="o">.</span><span class="mi">2</span>\<span class="n">OpenRPAOrchestrator</span><span class="o">.</span><span class="n">exe</span>
<span class="o">.</span>\<span class="o">..</span>\<span class="n">Resources</span>\<span class="n">WPy32</span><span class="o">-</span><span class="mi">3720</span>\<span class="n">python</span><span class="o">-</span><span class="mf">3.7</span><span class="o">.</span><span class="mi">2</span>\<span class="n">OpenRPAOrchestrator</span><span class="o">.</span><span class="n">exe</span> <span class="n">orchestratorMain</span><span class="o">.</span><span class="n">py</span>
<span class="n">pause</span> <span class="o">&gt;</span><span class="n">nul</span>
</pre></div>
</div>
</div>
<div class="section" id="use-in-studio-script-n-a">
<h3>Use in studio script (n/a)<a class="headerlink" href="#use-in-studio-script-n-a" title="Permalink to this headline"></a></h3>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">sys</span>
<span class="n">sys</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s1">&#39;../../&#39;</span><span class="p">)</span>
<span class="kn">import</span> <span class="nn">GUI</span>
<span class="kn">import</span> <span class="nn">keyboard</span>
<span class="kn">import</span> <span class="nn">subprocess</span>
<span class="kn">import</span> <span class="nn">time</span>
<span class="c1">#Highlight the UI Object in Folder explorer</span>
<span class="n">GUI</span><span class="o">.</span><span class="n">UIOSelector_FocusHighlight</span><span class="p">([{</span><span class="s2">&quot;class_name&quot;</span><span class="p">:</span><span class="s2">&quot;CabinetWClass&quot;</span><span class="p">,</span><span class="s2">&quot;backend&quot;</span><span class="p">:</span><span class="s2">&quot;uia&quot;</span><span class="p">},{</span><span class="s2">&quot;ctrl_index&quot;</span><span class="p">:</span><span class="mi">2</span><span class="p">},{</span><span class="s2">&quot;ctrl_index&quot;</span><span class="p">:</span><span class="mi">0</span><span class="p">},{</span><span class="s2">&quot;ctrl_index&quot;</span><span class="p">:</span><span class="mi">2</span><span class="p">},{</span><span class="s2">&quot;ctrl_index&quot;</span><span class="p">:</span><span class="mi">0</span><span class="p">}])</span>
<span class="c1">#Wait 2 seconds</span>
<span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span>
<span class="c1">#Loop: get child element of UI List</span>
<span class="k">for</span> <span class="n">lItem</span> <span class="ow">in</span> <span class="n">GUI</span><span class="o">.</span><span class="n">UIOSelector_Get_UIO</span><span class="p">([{</span><span class="s2">&quot;class_name&quot;</span><span class="p">:</span><span class="s2">&quot;CabinetWClass&quot;</span><span class="p">,</span><span class="s2">&quot;backend&quot;</span><span class="p">:</span><span class="s2">&quot;uia&quot;</span><span class="p">},{</span><span class="s2">&quot;ctrl_index&quot;</span><span class="p">:</span><span class="mi">2</span><span class="p">},{</span><span class="s2">&quot;ctrl_index&quot;</span><span class="p">:</span><span class="mi">0</span><span class="p">},{</span><span class="s2">&quot;ctrl_index&quot;</span><span class="p">:</span><span class="mi">2</span><span class="p">},{</span><span class="s2">&quot;ctrl_index&quot;</span><span class="p">:</span><span class="mi">0</span><span class="p">}])</span><span class="o">.</span><span class="n">children</span><span class="p">():</span>
<span class="nb">print</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">lItem</span><span class="p">))</span>
</pre></div>
</div>
<p>Here you can find the docs and examples of the OpenRPA desktop (GUI) app access.</p>
</div>
</div>
<div class="section" id="desktop-app-ui-access-win32-and-ui-automation-dlls">
<h2>Desktop app UI access (win32 and UI automation dlls)<a class="headerlink" href="#desktop-app-ui-access-win32-and-ui-automation-dlls" title="Permalink to this headline"></a></h2>
<div class="section" id="definitions">
<h3>Definitions<a class="headerlink" href="#definitions" title="Permalink to this headline"></a></h3>
<ul class="simple">
<li><p><strong>UIO</strong> - UI Object (class of pywinauto UI object) [pywinauto.base_wrapper]</p></li>
<li><p><strong>UIOSelector</strong> - List of dict (key attributes)</p></li>
<li><p><strong>PWA</strong> - PyWinAuto</p></li>
<li><p><strong>PWASpecification</strong> - List of dict (key attributes in pywinauto.find_window notation)</p></li>
<li><p><strong>UIOTree</strong> - Recursive Dict of Dict … (UI Parent -&gt; Child hierarchy)</p></li>
<li><p><strong>UIOInfo</strong> - Dict of UIO attributes</p></li>
<li><p><strong>UIOActivity</strong> - Activity of the UIO (UI object) from the Pywinauto module</p></li>
<li><p><strong>UIOEI</strong> - UI Object info object</p></li>
</ul>
</div>
<div class="section" id="what-is-uio">
<h3>What is UIO?<a class="headerlink" href="#what-is-uio" title="Permalink to this headline"></a></h3>
<p>UIO is a User Interface Object (pyOpenRPA terminology). For maximum compatibility, this instance is inherited from the object model developed in the [pywinauto library (click to get a list of available class functions)](<a class="reference external" href="https://pywinauto.readthedocs.io/en/latest/code/pywinauto.base_wrapper.html">https://pywinauto.readthedocs.io/en/latest/code/pywinauto.base_wrapper.html</a>).</p>
<p>This approach allows us to implement useful functionality that has already been successfully developed in other libraries, and Supplement it with the missing functionality. In our case, the missing functionality is the ability to dynamically access UIO objects using UIO selectors.</p>
</div>
<div class="section" id="uioselector-structure-example">
<h3>UIOSelector structure &amp; example<a class="headerlink" href="#uioselector-structure-example" title="Permalink to this headline"></a></h3>
<p>UIOSelector is the list of condition items for the UIO in GUI. Each item has condition attributes for detect applicable UIO. Here is the description of the available condition attributes in item.</p>
<p><strong>Desciption</strong></p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span>[
{
&quot;depth_start&quot; :: [int, start from 1] :: the depth index, where to start check the condition list (default 1),
&quot;depth_end&quot; :: [int, start from 1] :: the depth index, where to stop check the condition list (default 1),
&quot;ctrl_index&quot; || &quot;index&quot; :: [int, starts from 0] :: the index of the UIO in parent UIO child list,
&quot;title&quot; :: [str] :: the condition for the UIO attribute *title*,
&quot;title_re&quot; :: [str] :: regular expression (python ver) for the condition for the UIO attribute *title*,
&quot;rich_text&quot; :: [str] :: the condition for the UIO attribute *rich_text*,
&quot;rich_text_re&quot; :: [str] :: regular expression (python ver) for the condition for the UIO attribute *rich_text*,
&quot;class_name&quot; :: [str] :: the condition for the UIO attribute *class_name*,
&quot;class_name_re&quot; :: [str] :: regular expression (python ver) for the condition for the UIO attribute *class_name*,
&quot;friendly_class_name&quot; :: [str] :: the condition for the UIO attribute *friendly_class_name*,
&quot;friendly_class_name_re&quot; :: [str] :: regular expression (python ver) for the condition for the UIO attribute *friendly_class_name*,
&quot;control_type&quot; :: [str] :: the condition for the UIO attribute *control_type*,
&quot;control_type_re&quot; :: [str] :: regular expression (python ver) for the condition for the UIO attribute *control_type*,
&quot;is_enabled&quot; :: [bool] :: the condition for the UIO attribute *is_enabled*. If UI object is enabled on GUI,
&quot;is_visible&quot; :: [bool] :: the condition for the UIO attribute *is_visible*. If UI object is visible on GUI,
&quot;backend&quot; :: [str, &quot;win32&quot; || &quot;uia&quot;] :: the method of UIO extraction (default &quot;win32&quot;). ATTENTION! Current option can be only for the first item of the UIO selector. For the next items this option will be implemented from the first item.
},
{ ... specification next level UIO }
]
</pre></div>
</div>
<p><strong>The UIO selector example</strong></p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="p">[</span>
<span class="p">{</span><span class="s2">&quot;class_name&quot;</span><span class="p">:</span><span class="s2">&quot;CalcFrame&quot;</span><span class="p">,</span> <span class="s2">&quot;backend&quot;</span><span class="p">:</span><span class="s2">&quot;win32&quot;</span><span class="p">},</span> <span class="c1"># 1-st level UIO specification</span>
<span class="p">{</span><span class="s2">&quot;title&quot;</span><span class="p">:</span><span class="s2">&quot;Hex&quot;</span><span class="p">,</span> <span class="s2">&quot;depth_start&quot;</span><span class="p">:</span><span class="mi">3</span><span class="p">,</span> <span class="s2">&quot;depth_end&quot;</span><span class="p">:</span> <span class="mi">3</span><span class="p">}</span> <span class="c1"># 3-rd level specification (because of attribute depth_start|depth_stop)</span>
<span class="p">]</span>
</pre></div>
</div>
</div>
<div class="section" id="the-uidesktop-module-openrpa-robot-uidesktop-py">
<h3>The UIDesktop module (OpenRPA/Robot/UIDesktop.py)<a class="headerlink" href="#the-uidesktop-module-openrpa-robot-uidesktop-py" title="Permalink to this headline"></a></h3>
<p>The UIDesktop is extension of the pywinauto module which provide access to the desktop apps by the <strong>win32</strong> and <strong>ui automation</strong> dll frameworks (big thx to the Microsoft :) ).</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># EXAMPLE 1</span>
<span class="kn">from</span> <span class="nn">pyOpenRPA.Robot</span> <span class="kn">import</span> <span class="n">UIDesktop</span>
<span class="n">UIDesktop</span><span class="o">.</span><span class="n">UIOSelector_Get_UIO</span><span class="p">(</span>
<span class="n">inSpecificationList</span><span class="o">=</span><span class="p">[</span>
<span class="p">{</span><span class="s2">&quot;title&quot;</span><span class="p">:</span><span class="s2">&quot;notepad.exe&quot;</span><span class="p">},{</span><span class="s2">&quot;title&quot;</span><span class="p">:</span><span class="s2">&quot;OK&quot;</span><span class="p">}],</span>
<span class="n">inElement</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span>
<span class="n">inFlagRaiseException</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</pre></div>
</div>
</div>
<div class="section" id="id1">
<h3>The UIDesktop module (OpenRPA/Robot/UIDesktop.py)<a class="headerlink" href="#id1" title="Permalink to this headline"></a></h3>
<p>The UIDesktop is extension of the pywinauto module which provide access to the desktop apps by the <strong>win32</strong> and <strong>ui automation</strong> dll frameworks (big thx to the Microsoft :) ).</p>
<p><a href="#id2"><span class="problematic" id="id3">*</span></a>Naming convention: &lt;InArgument&gt;_&lt;ActivityName&gt;_&lt;OutArgument - if exist&gt;*&lt;br&gt;</p>
</div>
</div>
<div class="section" id="theory-practice-web-app-ui-access-selenium">
<h2>Theory &amp; practice. WEB app UI access (selenium)<a class="headerlink" href="#theory-practice-web-app-ui-access-selenium" title="Permalink to this headline"></a></h2>
<div class="section" id="about">
<h3>About<a class="headerlink" href="#about" title="Permalink to this headline"></a></h3>
<p>The pyOpenRPA support web app manipulation (by the Selenium lib).
More docs about selenium you can find here (<a class="reference external" href="https://selenium-python.readthedocs.io/">https://selenium-python.readthedocs.io/</a>)</p>
</div>
<div class="section" id="id4">
<h3>How to use<a class="headerlink" href="#id4" title="Permalink to this headline"></a></h3>
<p>To start use selenium just import selenium modules in the robot tool. Here is the example of the usage.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">selenium</span> <span class="kn">import</span> <span class="n">webdriver</span>
<span class="kn">from</span> <span class="nn">selenium.webdriver.common.keys</span> <span class="kn">import</span> <span class="n">Keys</span>
<span class="n">driver</span> <span class="o">=</span> <span class="n">webdriver</span><span class="o">.</span><span class="n">Chrome</span><span class="p">()</span>
<span class="n">driver</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;http://www.python.org&quot;</span><span class="p">)</span>
<span class="k">assert</span> <span class="s2">&quot;Python&quot;</span> <span class="ow">in</span> <span class="n">driver</span><span class="o">.</span><span class="n">title</span>
<span class="n">elem</span> <span class="o">=</span> <span class="n">driver</span><span class="o">.</span><span class="n">find_element_by_name</span><span class="p">(</span><span class="s2">&quot;q&quot;</span><span class="p">)</span>
<span class="n">elem</span><span class="o">.</span><span class="n">clear</span><span class="p">()</span>
<span class="n">elem</span><span class="o">.</span><span class="n">send_keys</span><span class="p">(</span><span class="s2">&quot;pycon&quot;</span><span class="p">)</span>
<span class="n">elem</span><span class="o">.</span><span class="n">send_keys</span><span class="p">(</span><span class="n">Keys</span><span class="o">.</span><span class="n">RETURN</span><span class="p">)</span>
<span class="k">assert</span> <span class="s2">&quot;No results found.&quot;</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">driver</span><span class="o">.</span><span class="n">page_source</span>
<span class="n">driver</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>
</div>
</div>
</div>
<div class="section" id="theory-practice-keyboard-mouse-manipulation">
<h2>Theory &amp; practice. Keyboard &amp; mouse manipulation<a class="headerlink" href="#theory-practice-keyboard-mouse-manipulation" title="Permalink to this headline"></a></h2>
</div>
<div class="section" id="theory-practice-screen-capture-image-recognition">
<h2>Theory &amp; practice. Screen capture &amp; image recognition<a class="headerlink" href="#theory-practice-screen-capture-image-recognition" title="Permalink to this headline"></a></h2>
<div class="section" id="how-to-automate-image-recognition-on-pc">
<h3>How to automate image recognition on PC<a class="headerlink" href="#how-to-automate-image-recognition-on-pc" title="Permalink to this headline"></a></h3>
<p>Here you can find any ways you need to use in your business case:
- Find the exact match on the screen with the other image
- Use text recognition module (OCR)
- Use computer vision (CV) to identify the objects on screen/image/video
- Use artificial intelligence (AI) to make custom identification/classification/text recognition</p>
</div>
</div>
</div>
</div>
</div>
<footer>
<div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
<a href="04_Dependencies.html" class="btn btn-neutral float-right" title="4. Dependencies" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
<a href="02_Defs.html" class="btn btn-neutral float-left" title="2. Defs" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
</div>
<hr/>
<div role="contentinfo">
<p>
&#169; Copyright 2021, Ivan Maslov.
</p>
</div>
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<script type="text/javascript">
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
</script>
</body>
</html>
Reference in new issue View Git Blame Copy Permalink