How to Write Data Driven Tests With Robot Framework

In this post, we are going to explore how to write data driven tests with Robot Framework. We will take a look at how data driven tests can be written with standard Robot Framework syntax and we will take a look at the Data Driver Library which offers some interesting features.

1. Introduction

In our previous post, we looked at some basic concepts of Robot Framework. This time, we will explore how to write data driven tests with Robot Framework.

We will continue using the examples of our previous post, it is therefore advised to take a look at that post but it is not required if you already have knowledge of the basic Robot Framework concepts. The source code used in this post can be found at GitHub.

2. How to Write Data Driven Tests

Data driven tests allow us to repeat the same test over and over again but with different input and/or output data. Of course, we can write separate test cases in order to accomplish this, but data driven tests will make this more concise, less error prone and more clear.

We will explore some of the capabilities by means of the demo application of our previous post. We created an employee.py Python script which allows us to add an employee to a CSV file, to retrieve the list of employees and to remove the entire list of employees. We also created a Robot Framework test script employee.robot for this. One of the test cases verifies whether adding an employee functions properly.

| Add Employee                  | [Documentation] | Verify adding an employee

| | [Setup]                     | Clear Employees List

| | Add Employee                | first_name=John | last_name=Doe

| | ${output} =                 | Retrieve Employees List

| | Should Be Equal             | ${output}       | ['John Doe']

| | [Teardown]                  | Clear Employees List

The test contains the following steps:

• We clear the entire list of employees in the Setup;
• We add an employee John Doe to the list;
• We retrieve the list of employees and verify whether the employee has been added to the list;
• We clear the entire list of employees again in the Teardown in order to restore the list of employees to its initial state.

2.1 The Standard Robot Framework Way

Let’s assume we want to verify whether adding an employee also works fine for other employee names. We therefore need to rewrite our test case as a keyword in order to be used as a template for input data. The test remains the same, we only need to change it to accept the data as arguments and use the arguments in our tests. We also have to remove the Setup tag at the beginning of the test. Setup is not supported in keywords, see here for the explanation why.

If you do use Setup in a keyword, the following error is raised:

xxxxxxxxxx

[ ERROR ] Error in file '.../MyRobotFrameworkPlanet/test/data_driven.robot' on line 24: Non-existing setting 'Setup'.

The Add Employee Template keyword which contains the test, is the following (the data_driven.robot file in the test directory of the repository contains the full contents of the test script):

xxxxxxxxxx

| Add Employee Template         | [Documentation] | Template for verifying adding an employee

|                               | [Arguments]     | ${first_name}

|                               | ...             | ${last_name}

| |                             | Clear Employees List

| | Add Employee                | first_name=${first_name} | last_name=${last_name}

| | ${output} =                 | Retrieve Employees List

| | Should Be Equal             | ${output}       | ['${first_name} ${last_name}']

| | [Teardown]                  | Clear Employees List

The template keyword takes the first_name and last_name as arguments and the hard coded values for John Doe are replaced with the argument values.

The Add Employee test only contains the Template tag which refers to the keyword which has to be used as a template for the data driven test. The remainder of the test case consists of the data we would like to test.

xxxxxxxxxx

| Add Employee                  | [Documentation] | Verify adding an employee

| | [Template]                  | Add Employee Template

| | first_name=John             | last_name=Doe

| | first_name=Monty            | last_name=Python

| | first_name=Knight           | last_name=Ni

Run the test:

xxxxxxxxxx

$ robot employee.robot

The log.html results shows us clearly that for each data record the template is executed. For each data record, the results can be reviewed.

2.2 Data Driver Library

The above works just fine when your data set is limited. But what to do when you need to verify tens, hundreds or thousands of input data? This would make your *.robot file unreadable and unmaintainable. This is where the Data Driver Library offers a solution for. The Data Driver Library allows us to use a CSV or Excel file for the input data. This way, the data is decoupled from the test script. Let’s see how this works! First, we need to install the library:

xxxxxxxxxx

$ pip3 install --upgrade robotframework-datadriver

Currently, the Data Driver Library has some limitations in its usage. These are not insurmountable, but it is important to know them:

• Only the first test case will be used as a template, all other test cases will be deleted;
• Test cases have to be defined with a Test Template at the test suite level (the template is used for all test cases whereas the Template tag we used before is only applicable for one specific test case);
• The keyword which is used as Test Template must be defined within the same *.robot file and may not be defined in a resource file (we will explain resource files in a next blog. For now, it is enough to know that a resource file contains keywords which can be shared between several test scripts, similar to a library).

The rewritten test is available in the data_driver.robot file located in the test directory:

xxxxxxxxxx

| *** Settings ***   |

| Documentation      | Test the employee Python script, data driven approach

| Library            | OperatingSystem

| Library            | DataDriver | file=data_driver.csv | dialect=unix

| Test Setup         | Clear Employees List

| Test Teardown      | Clear Employees List

| Test Template      | Add Employee Template

| *** Variables ***  |

| ${APPLICATION}     | python3 ../employee.py

| *** Test Cases ***

| Add Employee ${first_name} ${last_name}

| *** Keywords ***              |

| Add Employee Template         | [Documentation] | Template for verifying adding an employee

|                               | [Arguments]     | ${first_name}

|                               | ...             | ${last_name}

| | Add Employee                | first_name=${first_name} | last_name=${last_name}

| | ${output} =                 | Retrieve Employees List

| | Should Be Equal             | ${output}       | ['${first_name} ${last_name}']

...

We added the Data Driver library with the input data file data_driver.csv and the used dialect as arguments. We use unix as a dialect because we are using Ubuntu. See the website of the Data Driver library for a complete list of options which can be set. We moved the Setup and Teardown to the Test Suite level and removed it from the Add Employee Template. It is more clear to do so because also the template needs to be defined at the suite level and the result will be the same because only one test case may be added anyway. The test case itself only contains the name of the test case. Ensure that you make the name unique by using variables or another unique identifier, otherwise a warning is raised during test execution.

xxxxxxxxxx

[ WARN ] Multiple test cases with name 'Add Employee' executed in test suite 'Data Driver'.

The data_driver.csv contains the data we want to be tested, the first line must contain the arguments to be used for calling the template keyword:

xxxxxxxxxx

${first_name},${last_name}

John,Doe

Monty,Python

Knight,Ni

We notice after test execution that for each data record in the CSV file, a test is being created and executed:

xxxxxxxxxx

$ robot data_driver.robot 

==============================================================================

Data Driver :: Test the employee Python script, data driven approach 

==============================================================================

Add Employee John Doe                                                 | PASS |

------------------------------------------------------------------------------

Add Employee Monty Python                                             | PASS |

------------------------------------------------------------------------------

Add Employee Knight Ni                                                | PASS |

------------------------------------------------------------------------------

Data Driver :: Test the employee Python script, data driven approach  | PASS |

3 critical tests, 3 passed, 0 failed

3 tests total, 3 passed, 0 failed

==============================================================================

The log.html file also contains this result and we can verify that the Setup, Teardown and the Template are invoked with the data from the CSV file.

3. Conclusion

In this post, we explored how we can write data driven tests by means of Robot Framework. We used the standard Robot Framework approach by using templates and we explored the Data Driver Library which offers some interesting functionality by decoupling the test script from the test data. This can be extremely useful when large datasets need to be tested.