Practical PHP Testing Patterns: Parameterized Test

Sometimes the mechanics of different tests are really the same, and the only thing that changes between them are input and expected output data.

As an example, consider testing mathematical calculations functions, or any kind of stateless method: you invoke it always in the same way, and then check the returned value.

Another case is that of triangulation in Test-Driven Development. When an implementation is not obvious, triangulating it leads to the creation of multiple tests with different data, but usually the exact same procedure.

To eliminate the duplication between these tests, we can code a method which models the whole test, and that accepts input (and possible initial state) and expected output as parameters. This method will perform the arrange, act, assert, and even teardown phases if needed. It will eliminate the need for maintaining the same test code in different places, of course.

An advantage of these refactorings that we already cited in Test Utility Method is that adding a new test becomes equivalent to adding one or two lines of code. This pattern is really a specialized, standalone Test Utility Method.

Implementation

A simple form of Parameterized Test is a private method which is delegated from in the various test*() ones, which are called by the testing framework.

The advantage of this first approach is to present more clear names to the code reader, and to support possible additional work to perform after the method is executed (it may return something which we can make further assertions on.)

A second way of implementing Parameterized Test is with PHPUnit's @dataProvider annotation. Is it completely automatic, and displays data used in the test instance in case of failure in order to recognize which iteration we are in.

You can also prepare input and output for @dataProvider programmatically (read: with code), so with a process as flexible and complex as you want.

Variations

Some older versions of Parameterized Test exist.

A Loop-Driven Test uses nested loops that verify every combination of inputs/outputs. For example, it may check pairs of x and y coordinates in an image. Often exhaustive testing is an overkill and contains logic for the generation of data which may contain bugs.

In a Tabular Test, you have a table, represented as code or in some configuration file, where each row corresponds to one test run and contains the necessary input/output relation. One Test Method works on everything here. This mechanism is obsolete as PHPUnit with @dataProvider executes each instance of the method into a different Testcase Object. You won't observe tests influencing each other, no problem with localization of which row is causing a failure.

Example

The code sample shows you how to go from a strong duplication between tests to a single copy of the code that uses @dataProvider to be executed N times.

It also shows, in the third Testcase Class, how easy is to produce data for the tests with PHP code.

<?php
/**
 * This class contains many duplicated tests.
 * Yet if we kept them in a single testSquareRootIsCalculated() method,
 * the first failure would prevent the other from running.
 */
class NotParameterizedTest extends PHPUnit_Framework_TestCase
{
    public function testSquareRootIsCalculatedCorrectlyByPHPFor0()
    {
        $this->assertEquals(0, sqrt(0));
    }

    public function testSquareRootIsCalculatedCorrectlyByPHPFor1()
    {
        $this->assertEquals(1, sqrt(1));
    }

    public function testSquareRootIsCalculatedCorrectlyByPHPFor4()
    {
        $this->assertEquals(2, sqrt(4));
    }

    public function testSquareRootIsCalculatedCorrectlyByPHPFor9()
    {
        $this->assertEquals(3, sqrt(9));
    }

    public function testSquareRootIsCalculatedCorrectlyByPHPFor16()
    {
        $this->assertEquals(4, sqrt(16));
    }

    public function testSquareRootIsCalculatedCorrectlyByPHPForMinusOne()
    {
        $this->assertEquals('i', sqrt(-1));
    }
}

class ParameterizedTest extends PHPUnit_Framework_TestCase
{
    /**
     * This should be a static method, returning an array of arrays.
     * Each row of this table-like array is a set of arguments
     * for the Test Method.
     */
    public static function squaresAndRoots()
    {
        return array(
            array(0, 0),
            array(1, 1),
            array(4, 2),
            array(9, 3),
            array(16, 4),
            array(-1, 'i')
        );
    }

    /**
     * The annotation requires as unique argument the name of a public method
     * in this Testcase Class.
     * @dataProvider squaresAndRoots
     */
    public function testSquareRootIsCalculatedCorrectlyByPHP($square, $root)
    {
        $this->assertEquals($root, sqrt($square));
    }
}

class ProgrammaticDataProviderTest extends PHPUnit_Framework_TestCase
{
    private static $width = 5;
    private static $height = 10;

    /**
     * I don't know why these methods are required to be static. It makes
     * no difference however as we are never going to call it directly.
     */
    public static function everyCoupleOfCoordinates()
    {
        $testParameters = array();
        for ($x = 1; $x <= self::$width; $x++) {
            for ($y = 1; $y <= self::$height; $y++) {
                $testParameters[] = array($x, $y);
            }
        }
        return $testParameters;
    }

    /**
     * I don't advise you to test an image at every pixel, but in some cases
     * you may need an exhaustive search. The programmatic generation of data
     * keeps the test code really short, but don't forget that the generation
     * logic may hide bugs if it becomes too complex.
     * @dataProvider everyCoupleOfCoordinates
     */
    public function testImageHasCorrectTransparencyValue($x, $y)
    {
        $this->fail("This test will be executed in isolation with the x=$x and y=$y values.");
    }
}