Arrange - Act - Assert: Intuitive Testing for .NET

Today I have a new module to introduce you to. It’s a relatively simple module for testing, and you can pick it up in short order and start testing your scripts, modules, and even compiled .Net code. If you put it together with WASP you can pretty much test anything

The basis for the module is the arrange-act-assert model of testing. First we arrange the things we’re going to test: set up data structures or whatever you need for testing. Then we act on them: we perform the actual test steps. Finally, we assert the expected output of the test. Normally, the expectation is that during the assert step we’ll return $false if the test failed, and that’s all there is to it. Of course, there’s plenty more to testing, but lets move on to my new module.

The module is called PSaint (pronounced “saint”), and it stands, loosely, for PowerShell Arrange-Act-Assert in testing. Of course, what it stands for isn’t important, just remember the name is PSaint :)

PSaint is really a very simple module, with only a few functions. There are two major functions which we’ll discuss in detail: Test-Code and New-RhinoMock, and then a few helpers which you may or may not even use:

Set-TestFilter

Sets filters (include and/or exclude) for the tests by name or category.

Set-TestSetup (alias “Setup”)

Sets the test setup ScriptBlock which will be run before each test.

Set-TestTeardown (alias “Teardown”)

Sets the test teardown ScriptBlock which will be run after each test.

Assert-That

Assserts something about an object (or the output of a scriptblock) and throws if that assertion is false. This function supports asserting that an exception should be thrown, or that a test is false … and supports customizing the error message as well.

Assert-PropertyEqual

This is a wrapper around Compare-Object to compare the properties of two objects.

How to test with PSaint: Test-Code

Test-Code (alias “Test”) is the main driver of functionality in PSaint, and you use it to define the tests that you want to run. Let’s jump to an example or two so you can see the usefulness of this module.

Let’s start with an extremely simple function that we want to write: New-Guid. We want a function that generates a valid random GUID as a string. We’ll start by writing a couple of tests. First we’ll test that the output of the function is a valid GUID.

test "New-Guid outputs a Guid" {
   act {
      $guid = New-Guid
   }
   assert {
      $guid -is [string]
      New-Object Guid $guid
   }
}

Now, to verify that the test works, you should define this function (the GUID-looking thing is one letter short) and then run that test:

function New-Guid { "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaa" }

Another proof that it works would be that it should fail on this function too, because “x” is not a valid character in a Guid:

function New-Guid { "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" }

So, let’s write a minimal New-Guid that actually generates a valid Guid:

function New-Guid { "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa" }

If you run our test on that, you will see:

If you don’t like the fact that the Category is empty, you could add a category or two to the end of our test. We should also switch to using Assert-That if we want to know which test in the assert failed. Finally, we want to write another test which would test that New-Guid doesn’t just return the same Guid every time, the way ours does right now:

test "New-Guid outputs a Guid" {
   act {
      $guid = New-Guid
   }
   assert {
      Assert-That { $guid -is [string] } -FailMessage "New-Guid returned a $($guid.GetType().FullName)"
      New-Object Guid $guid  # Throws relevant errors already
   }
} -Category Output, ValidGuid

test "New-Guid outputs different Guids" {
   arrange {
      $guids = @()
      $count = 100
   }
   act {
      # generate a bunch of Guids
      for($i=0; $i -lt $count; $i++) {
         $guids += New-Guid
      }
   }
   assert {
      # compare each guid to all the ones after it
      for($i=0; $i -lt $count; $i++) {
         for($j=$i+1; $j -lt $count; $j++) {
            Assert-That ($guids[$i] -ne $guids[$j]) -FailMessage "There were equal Guids: $($guids[$i])"
         }
      }
   }
} -Category Output, RandomGuids

Now, we have to actually fix our New-Guid function to generate real random Guids:

function New-Guid { [System.Guid]::NewGuid().ToString() }

And at that point, we should have a function, and a couple of tests that verify it’s functionality…

The finer points of assertions

One thing you’ll notice the first time you use Get-Member after loading the PSaint module is a few script properties have been added to everything. I did this because I found myself writing the same Assert-That calls over and over and decided that it would be slicker to make these extension methods than to write new functions for each one:

MustBeA([Type]$Expected,[string]$Message)
MustBeFalse([string]$Message)
MustBeTrue([string]$Message)
MustEqual([Object]$Expected,[string]$Message)
MustNotEqual([Object]$Expected,[string]$Message)

There’s also a

MustThrow([Type]$Expected, [string]$Message)

which can be used on script blocks (note that this function executes the ScriptBlock immediately, so be careful how you use it).

We can use these to tidy up our tests quite a bit, while still getting good error messages when tests fail:

test "New-Guid outputs a Guid String" {
   act {
      $guid = New-Guid
   }
   assert {
      $guid.MustBeA( [string] )
      New-Object Guid $guid # Throws relevant errors already
   }
} -Category Output, ValidGuid

test "New-Guid outputs different Guids" {
   arrange {
      $guids = @()
      $count = 100
   }
   act {
      # generate a bunch of Guids
      for($i=0; $i -lt $count; $i++) {
         $guids += New-Guid
      }
   }
   assert {
      # compare each guid to all the ones after it
      for($i=0; $i -lt $count; $i++) {
         for($j=$i+1; $j -lt $count; $j++) {
            $guids[$i].MustNotEqual($guids[$j])
         }
      }
   }
} -Category Output, RandomGuids

COM Objects

PSaint also has a wrapper for COM objects to help with testing them. It adds GetProperty and SetProperty methods to allow you to access COM object properties which don’t show up on boxed COM objects (a common problem when working with MSOffice, for instance). It also adds InvokeMethod for COM objects to invoke methods that don’t show up for similar reasons. These, of course, only help you if you’re already fairly literate with the COM object in question.

Mock Objects

PSaint includes New-RhinoMock, a function for generating a new mock object using RhinoMocks (which is included). Rhino Mocks is a BSD-licensed dynamic mock object framework for the .Net platform. Its purpose is to ease testing by allowing the developer to create mock implementations of custom objects and verify the interactions using unit testing.

I have to admit that this New-RhinoMock function is incomplete, and exposes only a fraction of the options and power in RhinoMocks, but it’s been sufficient for the few times when I’ve wanted to actually mock objects from PowerShell, so I’m including it here.

For those of you (developers) who want to know why RhinoMocks instead of your favorite mocking framework, the answer is astonishingly simple: it had the least number of necessary generic methods (which are impossible to call in PowerShell 2).

Source: http://huddledmasses.org/arrange-act-assert-intuitive-testing/