Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Creating Parametrized Tests to Allow for Simpler Builds

DZone's Guide to

Creating Parametrized Tests to Allow for Simpler Builds

Having parametrized tests greatly improves your ability to run tests during build without creating complex requirements for agents.

· DevOps Zone ·
Free Resource

DevOps involves integrating development, testing, deployment and release cycles into a collaborative process. Learn more about the 4 steps to an effective DevSecOps infrastructure.

When running unit tests in a TFS or TeamCity Build, you may often face the problem of running tests with options different from those used in Developer Machine. For example, there are tons of tests that require a MongoDb and an ElasticSearch or Solr integration.

While it is quite normal for developers to have everything installed in the local dev box, it can be annoying to install MongoDb and ElasticSearch on all agent machines. This approach complicates the setup of build servers and creates a situation that isn't quite manageable.

While it is possible to create a dedicated pool composed only of agents with MongoDb and ElasticSearch installed, I prefer being able to run my tests in all available agents without any restriction.

The best solution is to have parameterized tests so that you can execute tests with different parameters during the build. For example, it can be helpful to parameterize connection strings.

Having parametrized tests greatly improves your ability to run tests during build without creating complex requirements for agents.

In a .NET environment, it is quite common to use an app.settings file to contain all of the connection strings. Here is an example from one of our projects:








All Tests use the ConfigurationManager object to access connection strings from the configuration file, and there is a single point where we specified all of the connection strings used by the tests.

The obvious problem of having the string setting in app.config is that this file is source-controlled and it is not possible to have different settings for different machines and developers.

A possible solution to this approach is using a PowerShell script that modifies all of the configuration files in bin directories before running the test. Here is a naive approach with PowerShell:

param(
    [string] $baseMongoConnection = "mongodb://admin:xxxxxx##localhost/{0}",
    [string] $connectionQueryString = "?authSource=admin",
    [string] $configuration = "debug"
)
 
##Logging tests
$configFileName = "..\Logging\Jarvis.Framework.LoggingTests\bin\$configuration\Jarvis.Framework.LoggingTests.dll.config"
Write-Output "Config File Name Is: $configFileName"
 
$xml = (Get-Content $configFileName)
  
Edit-XmlNodes $xml -xpath "/configuration/connectionStrings/add[@name='testDb']/@connectionString" -value "$baseMongoConnection$connectionQueryString"
 
$xml.save($configFileName)
 
##main tests
$configFileName = "..\Jarvis.Framework.Tests\bin\$configuration\Jarvis.Framework.Tests.dll.config"
Write-Output "Config File Name Is: $configFileName"
 
$xml = (Get-Content $configFileName)
  
Edit-XmlNodes $xml -xpath "/configuration/connectionStrings/add[@name='eventstore']/@connectionString" -value ($baseMongoConnection -f "jarvis-framework-es-test" + $connectionQueryString)
Edit-XmlNodes $xml -xpath "/configuration/connectionStrings/add[@name='saga']/@connectionString" -value ($baseMongoConnection -f "jarvis-framework-saga-test" + $connectionQueryString)
Edit-XmlNodes $xml -xpath "/configuration/connectionStrings/add[@name='readmodel']/@connectionString" -value ($baseMongoConnection -f "jarvis-framework-readmodel-test" + $connectionQueryString)
Edit-XmlNodes $xml -xpath "/configuration/connectionStrings/add[@name='system']/@connectionString" -value ($baseMongoConnection -f "jarvis-framework-system-test" + $connectionQueryString)
Edit-XmlNodes $xml -xpath "/configuration/connectionStrings/add[@name='engine']/@connectionString" -value ($baseMongoConnection -f "jarvis-framework-engine-test" + $connectionQueryString)
Edit-XmlNodes $xml -xpath "/configuration/connectionStrings/add[@name='rebus']/@connectionString" -value ($baseMongoConnection -f "jarvis-rebus-test" + $connectionQueryString)
 
$xml.save($configFileName)
 
function Edit-XmlNodes {
param (
     $doc = $(throw "doc is a required parameter"),
    [string] $xpath = $(throw "xpath is a required parameter"),
    [string] $value = $(throw "value is a required parameter"),
    [bool] $condition = $true
)    
    if ($condition -eq $true) {
        $nodes = $doc.SelectNodes($xpath)
          
        foreach ($node in $nodes) {
            if ($node -ne $null) {
                if ($node.NodeType -eq "Element") {
                    $node.InnerXml = $value
                }
                else {
                    $node.Value = $value
                }
            }
        }
    }
}
-

This is a quick and dirty script that can edit configuration files, but this is not a good approach because if a developer wants to change this setting in the machine, it is difficult to instruct VS to run this script with parameter before running the test. Additionally, it leads to unnecessarily complicated builds because it introduces another point of failure. Furthermore, it is not possible to have agent-dependent settings, so you cannot specify that Agent X should run the test against mongo instance Y.

A better solution is using environment variables to override the app.config connectionStrings and create an NUnit SetupFixture that is executed before the first test.

NUnit has the ability to run a global setup that is run before the very first is run. It is the perfect place where to put logic and change the configuration of the tests. In the following example, the init script checks some environment variables, then changes the connectionString:

[SetUpFixture]
public class GlobalSetup
{
    [SetUp]
    public void ShowSomeTrace()
    {
        var overrideTestDb = Environment.GetEnvironmentVariable("TEST_MONGODB");
        if (String.IsNullOrEmpty(overrideTestDb)) return;
 
        var overrideTestDbQueryString = Environment.GetEnvironmentVariable("TEST_MONGODB_QUERYSTRING");
        var config = ConfigurationManager.OpenExeConfiguration(ConfigurationUserLevel.None);
        var connectionStringsSection = (ConnectionStringsSection)config.GetSection("connectionStrings");
        connectionStringsSection.ConnectionStrings["eventstore"].ConnectionString = overrideTestDb.TrimEnd('/') + "/jarvis-framework-es-test" + overrideTestDbQueryString;
        connectionStringsSection.ConnectionStrings["saga"].ConnectionString = overrideTestDb.TrimEnd('/') + "/jarvis-framework-saga-test" + overrideTestDbQueryString;
        connectionStringsSection.ConnectionStrings["readmodel"].ConnectionString = overrideTestDb.TrimEnd('/') + "/jarvis-framework-readmodel-test" + overrideTestDbQueryString;
        connectionStringsSection.ConnectionStrings["system"].ConnectionString = overrideTestDb.TrimEnd('/') + "/jarvis-framework-system-test" + overrideTestDbQueryString;
        connectionStringsSection.ConnectionStrings["engine"].ConnectionString = overrideTestDb.TrimEnd('/') + "/jarvis-framework-engine-test" + overrideTestDbQueryString;
        connectionStringsSection.ConnectionStrings["rebus"].ConnectionString = overrideTestDb.TrimEnd('/') + "/jarvis-rebus-test" + overrideTestDbQueryString;
        config.Save();
        ConfigurationManager.RefreshSection("connectionStrings");
    }
}

This approach has numerous advantages. Firstly, it works even for developers' workstations. If you want to use a different connectionString, simply populate the corresponding Environment Variable and you are ready to go. Secondly, you can simply define variables that are valid for all agents on the build definition or set up environment variables different for each build agent. Doing this will allow each build agent to run tests against different Mongo instances or databases.

This means that one of the best approaches is parametrizing the tests with defaults that are good for developer machines, then allowing the override of configuration with environment variables to allow for easy configuration for build agents.

Read the 4-part DevOps testing eBook to learn how to detect problems earlier in your DevOps testing processes.

Topics:
paramaterized tests ,nunit ,testing

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}