Increase Coverage and Reduce Cost With Data-Driven Testing in Jira

As complexity increases, it becomes a considerable challenge to guarantee quality and reliability in the software industry. Despite an increase in test automation due to a power shift toward agile approaches, manual test case design and execution are still the most common testing practices.

However, extensive use of manual testing is widely considered to be a significant contributor to increasing project costs. A new approach is required to accelerate test case design and reduce the time spent on manual maintenance and execution. In this post, we’ll explore data-driven testing and how by leveraging test effectiveness, coverage, and reusability, it can contribute to an improvement in testing efficiency.

What Is Data-Driven Testing?

Data-driven testing (DDT) is a test design approach where the test data (input and output values) is separated from the actual test case. The test data can be stored in one or more central data sources and shared across different test cases. By storing your test data in a central repository (local storage, excel spreadsheet, XML file or SQL database) you can run the same test with a new set of data each time, avoiding the need to design and execute repetitive tests cases. Towards the end of the test cycle, you can store test data to provide a clear audit trail of what was and wasn't covered by the test.

DDT can be used in both manual or automated test scenarios, enabling test scripts to run together alongside their respective data set. In DDT, the following operations are performed in a loop for each row of test data in the data source:

• Retrieving the test data.
• Inputting the test data in the system under test (SUT) and simulating other actions.
• Verifying the expected values.
• Continuing the test execution with the next set of test data.

In a nutshell, DDT can be visualized as follows:

Adopting a Data-Driven Approach to Testing

Commonly, test cases are designed using variables in the test step level. A variable placeholder is added in the test step to receive the values from the data source instead of using hard-coded or fixed values. By doing so, each time you execute the test case, the variable is replaced with the test data in the corresponding data source. This approach reduces time and effort involved in creating test cases, as often a new test case requires just a new row of test data in the data source. Additionally, it provides a clear separation of testing logic from the actual test data, reducing the time spent on maintenance (i.e. any changes to the test step do not affect the test data and vice-versa). If you would like to find out more about how to get up and running with data-driven testing, check out our documentation for more details.

Data-Driven Testing in Action

Depending on the business rules you have to validate during the test case design, you may need to repeat the test case several times with different test data to ensure that the application works as expected for various input values. For example, if you have to design test cases to test the search and book a flight functionality of an airline company, you'll need to check different combinations of input values for the search screen (i.e. from, to, depart, return, class, etc). To prevent duplication, along with the test case you can provide a data source with test data to map each combination you need to address to leverage the coverage, as seen in the image below:

When designing test cases using the DDT approach, the focus should be in mapping the test data and its different combinations and variations (negative and positive) to meet the requirements and business rules and ensure adequate test coverage. By doing so, during the test execution, the variable placeholders in the test step level are replaced by the values available in the test data table, ensuring complete reusability of the steps instead of using hard-coded or fixed values as displayed in the image below:

Benefits of Data-Driven Testing

Adopting DDT is a way of ensuring repeatability and reusability of existing test cases. DDT leverages test coverage by feeding different types of test data – either valid or invalid (positive and negative test cases). As a result, you can benefit from a dramatic reduction of redundant test cases and bugs in production. Other benefits of adopting DDT include:

• Increased test coverage
• Reduction in time spent designing and maintaining test cases
• Repeatability and reusability of existing test cases
• Reduction in the number of test cases

Why Does DDT Make a Difference?

DDT enables developers and testers to have a clear separation of the logic of their test cases/scripts from the test data while making it possible to have the test scripts in a single repository. Making it easier to understand, maintain and manage tests for higher efficiency. Repeatability enables you to execute any test script hundreds of times with different data assets, saving time by freeing up your test team to focus on more challenging exploratory and user experience testing.