Automated Acceptance Testing With Robot Framework
Automated Acceptance Testing With Robot Framework
With Robot Framework, we can automate acceptance tests. In this first part of a series about Robot Framework, we will explore some of the basic concepts. Enjoy!
Join the DZone community and get the full member experience.Join For Free
Robot Framework is an open source test automation framework. It is keyword driven and therefore very suitable to describe and automate acceptance tests. In this post, we will introduce Robot Framework and dive into the basic concepts.
Robot Framework knows its origin at Nokia Networks, but has been open sourced since 2008. It is a framework for creating automated acceptance tests. Test scripts written with Robot Framework are keyword driven, making them very readable and understandable for everyone (assuming best practices are being used). Whether you are creating Test Cases based on Test Designs or based on User Story acceptance criteria, a Robot Framework test script will be your Test Case documentation and automated test script in one. Besides that, it is written in plain text and therefore very suitable for adding them to your version control system (like Subversion or Git). Get rid of all those tests described in Excel Workbooks! After running your test script, a test report is automatically generated, which gives you your test evidence.
Enough introduction, let’s see how this looks like and start exploring Robot Framework by using some examples. The sources used in this post can be found at GitHub.
2. Installation of Robot Framework
Robot Framework is implemented in Python, so you will need a Python interpreter on your system. We are using Ubuntu 18.04, the instructions for other platforms can be found at the Python website. We will use the Python 3 version which is available in our Ubuntu release. You can also use Python 2, but it is recommended not to use this Python version anymore because it has reached its end of life and Robot Framework support will also end for this Python version. Elaborate installation instructions can also be found here.
Verify the Python installation:
Install pip in order to be able to install Python packages easily:
Verify the pip installation:
Install the Robot Framework library:
Verify the Robot Framework installation:
We are now ready for creating our first Robot Framework script.
3. Application Under Test
Before getting started, we need an application which we want to test. We will make use of a small Python script which will allow us to add an employee to a CSV file, retrieve the list of employees from the CSV file and to delete the complete list. We are using a Python script, but it must be clear that any kind of application can be tested by means of Robot Framework. The
employee.py script is the following:
The application is for your information only and only for demonstration purposes. More important is how to use it. The next commands will add an employee
John Doe and an employee
Monty Python, the employees are retrieved, the employees are deleted (actually, the CSV file is deleted) and the employees are retrieved again, showing an empty list.
4. Explore the Basics
A good starting point for learning Robot Framework, is the documentation section of the Robot Framework website. In this paragraph, we will highlight some of the basics but when you want to explore more features, it is really recommended to read and learn how to use the official documentation. The user guide contains very good documentation for general usage of Robot Framework, the standard libraries which are available and some built-in tools.
4.1 Sections and Reporting
In our first test, we will verify the output when we retrieve an empty list of employees.
The test script contains several sections (there are more, but we will cover that later on):
- Settings: used for importing libraries and for adding metadata;
- Variables: used for defining variables which can be used elsewhere in the test script;
- Test Cases: the actual test cases to be executed. All test cases together are contained in a Test Suite.
A test script is formatted like a table. You can choose to use whitespace characters in order to separate the columns, but we prefer using a pipe. Pipes are more clear as a separator and when using whitespace, it is not always clear whether the separator is present and this can cause a lot of analysis time when a script returns errors.
Let’s take a look at our first test script
employee.robot (the script is located in a subdirectory test):
The Settings section contains a documentation setting which describes the purpose of the test suite and we import the
OperatingSystem library. We need this library in order to be able to invoke the Python script. A library contains several ready-made functions. In Robot Framework terms, they are called keywords.
The Variables section contains one variable
APPLICATION, which contains the command we need to invoke for running the Python script.
The Test Cases section contains one test
Empty Employees List. It also contains a documentation tag which describes the purpose of the test. In the next line, we invoke the command for retrieving the list by using the keyword
Run and Return RC and Output from the
OperatingSystem library. We capture the output of the command and also its return code. With the built-in keywords
Should be Equal As Integers and
Should Be Equal, we can verify the return code and the output.
Run the test:
After running the test, we notice that the test case has passed and three files have been created. The
output.xml file contains the results in XML format and can be used for processing the results into other applications. That is all you need to know for now about this output file.
report.html provides us an overview of the test results. In case of one test it might seem a bit trivial, but this is really useful when your test suite contains a lot of tests.
The most important file is the
log.html file. This file contains the details of the test execution. Especially when something goes wrong during the test, this file contains the detailed information for analyzing what has gone wrong but it also serves as your test evidence.
4.2 Setup and Keywords
Let’s add another test and test the adding of an employee. We add the employee, verify the return code and then retrieve the list of employees to verify whether the employee is really added.
We run both tests again and both tests pass. But when we run the tests again, both tests fail.
We assume in our tests that the list of employees is empty from the start on. This is not true anymore once we have ran the tests because during the tests we add an employee to the list. We do get clear messages why our tests fail. When the console output is not clear enough, just check the
log.html file where more detailed information can be found of what went wrong.
We can solve our failed tests by adding a
Setup to the tests. In the setup we can add all the prerequisites for running our test. We need to empty the list of employees and we also have a command for doing so. This brings us to the following problem: the
Setup can only contain 1 keyword and calling the command and verifying the return code requires two lines. We can solve this by creating our own keyword in the Keywords section.
We add the
Setup to both tests and both tests execute successfully when running more than once.
Our tests do what they have to do and they execute successfully but they are not very readable and they contain duplicate items. It would be better if we create keywords in order to improve readability and in order to have more maintainable tests.
First, we create a keyword for retrieving the list of employees. We need the output of the list in our test and therefore we
Return the output value, just like you would do in any other programming language.
Empty Employees List test case is changed as follows:
This test case looks a lot better now. We have a clear distinction between the setup of the test case and the test itself and because keywords are used, the test itself is very readable.
Something similar can be done for adding an employee. The only difference here, is that we need to be able to pass
Arguments to the keyword. It is possible to add arguments next to each other separated with pipes, but it is also allowed to place them on a new line. Be aware that you must use ellipsis when a new line is being used.
Add Employee test case is changed as follows:
We also added a
Teardown to this test, in the
Teardown you bring the application back to its state before the test. It is often used to restore configuration settings to its original value when they have been changed during the
In this post, we gave an introduction to Robot Framework for creating automated acceptance tests. It is an easy to learn, stable framework and moreover, when the test cases are set up by using best practices, they are even readable to your business users. This last remark about using best practices is very important. If you do not follow best practices, the test cases will become unreadable and unmaintainable and you will end up with spaghetti code. See also How To Write Good Test Cases and Robot Framework Do’s and Don’ts.
Published at DZone with permission of Gunter Rotsaert , DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.