Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Data Driven Tests in Marvin

DZone's Guide to

Data Driven Tests in Marvin

· DevOps Zone
Free Resource

Best practices for getting to continuous deployment faster and with dramatic results in reduced outage minutes, development costs, and QA testing cycles. Brought to you by Rainforest QA.

This post comes from Sowmya Krishnan at the Open@Citrix blog.

Data-driven tests (DDT) are useful for generating more tests with less code. If you have lots of assertions to be repeated with related data sets or parameters, data-driven tests are the way to go because they save a lot of redundant code. The data sets provided as input will be used iteratively in the test scripts in effect, running different test cases. I’ve recently tried to implement this with Marvin, CloudStack’s Integration Test framework, which is written in Python. 

Before beginning to code test suites in data driven fashion, it’s best to understand the entities which drive the tests.  For example, we may want to test the same set of VM life cycle tests for different network offerings such as a simple isolated offering, an offering with persistent network feature and an offering with Netscaler as a service provider for LB. These offerings then become the data sets which are supplied as input to the test script. And the assertions for VM life cycle are repeated for each of these network offerings.

DDT in Python

Python provides a powerful and efficient way to achieve this with the DDT library. It comes with a class decorator, @ddt, and method decorators @data and @file_data. Use the class decorator with the TestCase class and specify the data sets with which the tests need to be run with the @data or @file_data method decorator. While @data takes in the arguments directly to be passed to the test, @file_data will load the data set from a JSON file.

CloudStack Test Case Example

Here’s an example:

Consider a case where we need to verify various network operations in a VPC with two different VPC offerings: the default VPC offering and the default VPC offering with NS as the LB.

First, Install DDT (pip install DDT) if it's not already installed. More info here. 

Then, start by defining the class decorator with the test class:

@ddt

class TestVPCNetwork(cloudstackTestCase):
Then, define the data sets using the method decorator:
@data("network_offering", "network_offering_vpcNS")

def test_01_create_network(self, value):
(Where the network offerings are already defined prior to starting the tests)

The result will be that test_01_create_network will be iterated for both the network offerings. The complete test suite can be found here.

There’s another way of writing data-driven tests in Python via generative tests. Nose (we use Nose as the test runner with Marvin) supports test generators. Details here. Using this, we can write test functions that are generators. However, it doesn’t work if we are using unittest.TestCase subclasses, which is the case with Marvin.

So, for now, I am sticking with DDT and hoping to implement this for all VPC tests and explore the possibility of using this with other test suites as well.



Discover how to optimize your DevOps workflows with our on-demand QA solution, brought to you in partnership with Rainforest QA.

Topics:

Published at DZone with permission of Mark Hinkle, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}