This post comes from Sowmya Krishnan at the Open@Citrix blog.
Data-driven tests (DDT) are useful for generating more tests with less code. If you have lots of assertions to be repeated with related data sets or parameters, data-driven tests are the way to go because they save a lot of redundant code. The data sets provided as input will be used iteratively in the test scripts in effect, running different test cases. I’ve recently tried to implement this with Marvin, CloudStack’s Integration Test framework, which is written in Python.
Before beginning to code test suites in data driven fashion, it’s best to understand the entities which drive the tests. For example, we may want to test the same set of VM life cycle tests for different network offerings such as a simple isolated offering, an offering with persistent network feature and an offering with Netscaler as a service provider for LB. These offerings then become the data sets which are supplied as input to the test script. And the assertions for VM life cycle are repeated for each of these network offerings.
DDT in Python
Python provides a powerful and efficient way to achieve this with the DDT library. It comes with a class decorator, @ddt, and method decorators @data and @file_data. Use the class decorator with the TestCase class and specify the data sets with which the tests need to be run with the @data or @file_data method decorator. While @data takes in the arguments directly to be passed to the test, @file_data will load the data set from a JSON file.
CloudStack Test Case Example
Here’s an example:
Consider a case where we need to verify various network operations in a VPC with two different VPC offerings: the default VPC offering and the default VPC offering with NS as the LB.
First, Install DDT (pip install DDT) if it's not already installed. More info here.
Then, start by defining the class decorator with the test class:
@ddt class TestVPCNetwork(cloudstackTestCase):Then, define the data sets using the method decorator:
@data("network_offering", "network_offering_vpcNS") def test_01_create_network(self, value):(Where the network offerings are already defined prior to starting the tests)
The result will be that test_01_create_network will be iterated for both the network offerings. The complete test suite can be found here.
There’s another way of writing data-driven tests in Python via generative tests. Nose (we use Nose as the test runner with Marvin) supports test generators. Details here. Using this, we can write test functions that are generators. However, it doesn’t work if we are using unittest.TestCase subclasses, which is the case with Marvin.
So, for now, I am sticking with DDT and hoping to implement this for all VPC tests and explore the possibility of using this with other test suites as well.