Practical PHP Testing Patterns: Test Discovery

The Test Discovery pattern describes automated mechanisms for the testing framework to define which of your tests should be run at the user's request.

Usually, a framework that implements this pattern should discover all the tests of an application, since only by running all the tests you are sure that you have not make a regression which you could find. There is however a contract to conform in your tests code to make sure that PHPUnit finds all your tests.

Lost tests are the most common case of regressions, when running all the test scattered throughout the codebase is so difficult that developers often forget to execute one or more of them before checking in their code. Ideally, you should be able to run all the tests at the push of a button, and PHPUnit can help with that.
Now drop ideally from that sentence: you must be able to run them all with a single command when it is necessary, with the exceptions of tests that cannot be executed on the current machine, for example because they require Selenium or a particular database instance (integration tests).

What conventions should I follow?

Here is PHPUnit's contract, which you should follow with your test classes:

• thou shalt put all of your tests in a single folder or in subfolders of that (won't matter if you define a configuration file: in that case, a finite number of folders whose list you'll have to maintain is the maximum PHPUnit can work on.)
• Thou shalt name all your test files with something that ends in Test.php; for example, tests/Zend/Filter/IntTest.php.
• These files shalt contain classes that extend PHPUnit_Framework_TestCase, and should not be abstract.
• Thou shalt name your public test methods in the classes starting with the test* prefix; for example, testShouldFillTheCollectionWithZeroValues() is a good method name.
• thou shalt run phpunit with that folder as the working directory.

If you follow this very simple rules, you would be able to run all your tests with no mandatory configuration file whatsoever.

In literature (mainly Meszaros's book xUnit Test Patterns), the variations of this pattern are called Testcase Class Discovery and Test Method Discovery, and PHPUnit implements both of them.

What PHPUnit does for a living?

• PHPUnit scans the folder looking for conformant source files and includes all of them. The base class you extend is already available from the framework.
• It lists the newly defined classes and see which fulfills the contract (concrete subclasses of PHPUnit_Framework_TestCase).
• It lists the methods that start with test*, and instance the class one time for each of them.
• It executes your tests and shows the results with a nice green bar or a red bar, which shows that the code needs more work.

Configuration sample

The phpunit.xml file, which is searched in the working directory during execution, makes you able define a test suite that spans over more than one folder, along with different prefixes. It's not mandatory, but it's handy.

<phpunit>
    <testsuite name="MyTestSuite">
        <!--  a single file which won't be otherwise be found -->
        <file>DummyTesting.php</file>

        <!--  the current directory, probably a tests/ folder -->
        <directory>.</directory>

        <!--  an additional directory of other tests we want to execute along with these ones -->
        <directory suffix="AcceptanceTest">../application/tests</directory>
    </testsuite>
</phpunit>

While I can think of cases where you want to aggregate more than one test folder, I see no reason to change for example the Test suffix. Only when you have an external convention you can't modify you should interfere with the suffix, while for the majority of cases sticking to the convention will make it easier for the developers to grasp what classes are tests, and easier to maintain in the long run.

Don't reinvent the wheel by pushing your own convention and new developers will be happy to meet your codebase, getting up to speed very quickly. Once you have a configuration, you can instead put in the name of a bootstrap file, or directives for code coverage. In this way, you'll still be able to run 'phpunit' without worrying of additional arguments, which are read from this conventional XML file.