RPA With Open Source
RPA is specialized application for business process and workflow automation. Four open source technologies are presented here to demonstrate RPA.
Join the DZone community and get the full member experience.Join For Free
We humans are lazy and were habitually attempting to do more with less. For thousands of years, we have built machines to perform arduous tasks for us. In the era of computerization and software, a new terminology emerges in automation – Robotic Process Automation (RPA).
While automation has a broad connotation of any labor-saving technique or technology, RPA is a specialized application for business process and workflow automation. RPA is particularly aimed at mimicking essential human actions for business development. The goal of RPA is to reduce the use of human labor for mundane tasks, thus making it possible to reuse their capabilities towards gainful business activities.
An abundance of RPA software is available – commercial, freeware, and open source. Here, I look into some of the ways to leverage open source to bring RPA bots to life. These bots can be headed or headless. Headed refers to bots that mimic user actions (including mouse and keyboard) precisely to achieve the task needed. Headless bots perform tasks in the background. Both types of automation are used in the RPA world.
Portfolio value calculation is a common need of fund managers, finance professions, and investors. This task is demonstrated here with open source. Yahoo Finance is the source used. It is publicly accessible; however, the interface is subject to change without notice. A robust approach is to use an assured production-ready API service.
Four open-source technologies are presented here to demonstrate the portfolio bot. We look at both headed and headless variants.
- AutoHotKey (AHK)
- Robot Framework
- Shell scripts
AutoHotKey is an open-source (GNU GPL) scripting language for Windows OS. Among its several capabilities, a noticeable feature is mouse & keypad emulation. RPA is being promoted as no-code/low-code automation technique to replace the unspecialized workforce performing low-key tasks. A commonly used automation approach is to record user mouse and keyboard actions and repeat them.
While user action emulation is an easy and simple approach, it's prone to incongruity. For instance, mouse positions are not absolute and vary with dependency on display resolution, font, and hardware. Thus, a script generated on one system is not always assured to work on another or even on the same system all the time. Auto-generated scripts are also sequential in nature with readability and maintainability challenges.
This is an auto-generated AHK script for a portfolio value calculation task. Screen actions are here captured and auto-generated by Auto Script Writer (ASW) accompanying AHK. (Note: ASW, which was earlier part of the AHK package, is no longer distributed officially. AHK remains open-source and independently usable for automation. ASW is required only for user action capture and script generation.)
The recommended approach is to use an action recorder as the start point and align the generated script for functional or OO design. Absolute mouse positions in generated scripts are also recommended to be replaced with relative positions.
The above auto-generated script is here revised for readability and maintainability:
Upon execution of the script, stock values and networth are updated in Excel spreadsheet.
See execution of the automation here.
This is headed portfolio bot with AHK. A keen reader may notice this script is not completely scalable with its use of sleep, fixed mouse positions and predetermined spreadsheet cell numbers. This is intended for demonstration of AHK automation capabilities and not intended for production.
Selenium is a widely used open-source (Apache License 2.0) web application test framework. With its strong capabilities, platform independence, support for several programming languages and browsers, it is a good candidate for RPA. The portfolio task automated earlier with a mouse and keypad approach is now automated with scripts. This scripting methodology is more robust and not dependent on the execution system as the earlier approach is.
This is a Java Selenium portfolio bot. This falls in between headed and headless where the browser is used to navigate to the finance website and Selenium’s WebDriver function is used in the background to retrieve the stock value.
On execution of the script, spreadsheet is updated as earlier.
A demo is available here.
Robot Framework is an open-source (Apache License 2.0) dual-dialect RPA family. The core execution engine is Python. Robot Framework further extends the core engine with RPA libraries implemented also in Python. It also provides an easy-to-use and conversational English-like scripting language. This dual capability is a unique advantage where a skilled python developer can add extended capability and a low-skilled user can automate with an easy language.
The structure of the net worth bot in Robot Framework looks like this:
ExcelUtil.py is a Python component that provides general-purpose spreadsheet operations:
FinUtil.py is a Python utility to retrieve the current quote of a stock from a financial website.
These libraries are used in the robot script networth.robot. This is English-like and no deep technical skill is needed to create this script. In this script, Python libraries are imported with the Library keyword. Bot operations are invoked in English-like language. For example, the Python function get_data is invoked as Get Data and get_quote as Get Quote.
Any other Python library can also be invoked from this script. You may notice here that the Python DateTime library is used.
This is a headless RPA. Headed is also possible by using appropriate libraries. On execution of this .robot script, we can see the spreadsheet is updated.
Note: Robot Framework needs to be installed in Python to execute this script. A package installer such as pip can be used (pip install robotframework).
This is a shell script that uses CSV file format to automate networth information:
Holdings to query are provided in the CSV file:
$ cat holdings.csv
When the script networth.sh is executed:
Networth value is added to the CSV:
$ cat holdings.csv
It is prudent to mention shell scripting is platform-independent. The above automation is performed on Windows® OS with MinGW.
There are several RPA tools. Open source is a bellwether with rich possibilities. A few engaging ones are presented here. Did you like what we covered here? Are you a fan of automation and RPA bots? How would you attempt the problem differently? Share with us in comments below.
Opinions expressed by DZone contributors are their own.