The different kinds of testing

Automated testing supports your constant effort in design and refactoring, and besides that ensures that your application actually works in a reliable and repeatable way. Tests at every level of detail are a form of executable specification and documentation. They give you immediate feedback and confidence that your code works, plus a satisfying green bar many times a day.

I've been consulting on a Zend Framework application, with the goal of repairing the test suite and expanding it. In this article I'll describe the different categories of testing, as applied to a Zend Framework 1 application, but this classification pertains to every web application based on object-oriented programming.

Since this kind of applications is obviously PHP-based, PHPUnit will be the tool of choice along with some of its standard extensions. For a panoramic of PHPUnit and its features, feel free to download my free ebook on the subject, which condenses much of the technical informations about it to a mere 50 pages.
Let's start with the most debated and simple kind of testing - the one at the unit level.

Unit testing

Each unit tests target a unit of code in isolation - usually a class, and thus one or more objects instantiated from this class. The isolation property is what defines a unit test: its code must not have dependencies on other classes than the one under test, since they should be tested independently, by their own test classes.
Since PHPUnit models a test case for a production code class as another class extending PHPUnit_Framework_TestCase, implementing unit testing leads very often to a parallel hierarchy of classes, where every Foo_Bar class has a corresponding Foo_BarTest test case.

Given these premises, a unit test that fails tells you immediately where the error is: in the class it exercises. Moreover, it will be very fast to execute, since it works on only a single object at the time.

Unit test should target mostly your models, and any code written by you that is not framework-specified: these would also be the classes that contain the majority of the business logic, and the most interesting to test. This code is usually composed of Plain Old PHP Objects and of subclasses of framework or library base classes when when they leave no other choice for integration.

For writing unit tests, usually no external library other than PHPUnit is necessary. In a Zend Framework application you can usually reuse the bootstrap files, which set up things like autoloading, in the phpunit --bootstrap option or by defining it in the phpunit.xml configuration file. This way it will be executed only once for each test suite run.

I prefer to leave initialization of the single components to test in the test cases itself, to ensure maximum isolation. However, a simpler and standard solution is to just run the whole Bootstrap class, with a custom configuration (application/config/application.ini), which 'testing' environment section is created by default by Zend_Tool.

Pragmatic unit testing

That's not a standard name.

In some cases, you should also be pragmatic: you cannot usually mock all the external resources, nor you should since mocking a contract which you can't change can lead you to madness. You should configure a lightweight version of your dependencies and test with them.

For example, if you're using the Doctrine Object-Relational Mapper, you must test the interaction with the database somewhere, and mocking the whole Doctrine infrastructure will be prohibitive and unuseful. The standard practice here is to use the real Doctrine infrastructure to test database-coupled classes, like Repositories and Data Access Objects, but to instantiate a lightweight database like an sqlite in-memory one which is much faster in its operations than a production one. This database can then be discarded or truncated at the end of each test to ensure no global state is shared between test cases.

The downside in this approach is that sqlite is not the real database; one time I was testing with it and due to a bug (feature?) in Doctrine 1 the code failed in MySQL while passing with Sqlite. The reason was sqlite does not support foreign key constraints and was simply ignoring them, while MySQL correctly throwed exceptions when they were violated. Moreover, these tests are never fast as the ones totally isolated from external libraries.

The upside is that the tests for classes interacting with the database via Doctrine or another ORM still have the benefit of the unit level: when the test fail, it is clear that the related production code class has encountered a regression, because the ORM code is only imported in discrete, distant points of time, when the test suite is green, and so could never change while you're expanding your code.

Nevertheless this kind of testing should be applied only to the adapters of your application, which constitute the boundary of the object graph towards external components like databases, web services or the filesystem.

Functional testing

Functional testing's goal is to exercise a medium-sized object graph, without instantiating the whole application, to a cover a full functionality and make sure the classes adhere to the same contract.

For example, these tests can target a service layer built upon your Domain Model, if you want to enhance to cover your factories or DI mechanisms. In other cases, they can target the controllers: this happens when you have supplemental logic on the client side.

In case of functional testing on plain old classes, PHPUnit suffices again. In case you target controllers instead, the Zend_Test component gives you a Zend_Test_PHPUnit_ControllerTestCase class which you can extend to gain helper functionalities. Basically, every test method of a Zend_Test test case makes at least a HTTP request. The helper test case sets up a fake HTTP request and response objects in every setUp(), and lets you check the result, being it written HTML (via querying and asserting), XML or JSON.

Integration testing

Integration tests target an external component such as a library to ensure the expectations of the developers on it are met.

Integration tests are usually started as exploratory tests, which are used to learn about the library and to encapsulate this knowledge into a repeatable, executable form. With time, they become regression tests, which allow you to upgrade the library to a new release or version by catching the changes in behavior. Some of these tests target the PHP runtime itself, to check for example that an extension assumed as present is really available.

For example, this week we were surprised when a === check inside a Domain Model class was failing. We started writing integration tests for Doctrine_Query, and it turned out that PDO and Doctrine returned strings for numeric fields on their Active Record. By having a specific test to cover our expectations, we understood where our assumption was wrong, and cease to suspect a bug in our own code where the === resided.
For this kind of test, again only PHPUnit is necessary; moreover, you'll have to bootstrap the involved library, but it can be simply a matter of adding it to the include_path.

Acceptance testing

Acceptance tests are end-to-end tests, which see the application as a black box. They exercise the behavior of the whole application, from the user inserting data to the reports created and the actions performed as a consequence.

These tests are much slower, but they work on the end result of your work, and define what the user will see and interact with.

For old-style applications, which do not involve rich clients, Zend_Test is usually enough for these kinds of tests. A thin layer of CSS expression built over it in order to check the pages without duplicating the same selectors all over the suite may help.

However, for Javascript-rich apps, a tool like Selenium is necessary. Selenium drives a real web browser to a fresh instance of your application, and execute your tests, which can be defined manually or via a record-and-replay browser extension. Many PHPUnit extensions offer the means for connecting to a Selenium server, which manages the browsers, and navigate the web application.

As a result of its focus on real web browsers such as Firefox and Chrome, Selenium tests are much slower than Zend_Test ones. However, they are the only tool available to execute acceptance tests which involve JavaScript.

Conclusion

Note that everyone of these kinds of tests (except the integration ones) can be written before the production code it exercises. Unit tests ahead of their referred class; functional tests ahead of the Facade they target; acceptance tests before a whole vertical slice of functionality is implemented. Moreover, if you're doing Test-Driven Development you should in general start at the higher level of abstraction (acceptance) and descending into the lower levels as needed.

These different types of testing are always present, maybe as a small part of the suite, in every web application of moderate size. Learning to recognize them when they emerge will help you organizing the test suite better and maintaining it productive and responsive to change.