Test-Driven Development with OSGi

For my thesis in Computer Engineering I'm working on a project built over an OSGi framework. As always, I'm guided by test since I apply Test-Driven Development all the time at the acceptance and unit level. I thought it would be nice to share my experiences in dealing with a framework (sometimes an heavy burden) when doing TDD.
For the rest of this article, I'll refer to Equinox plugin tests (commonly run through the Eclipse IDE) and to basic JUnit tests, but the concepts for implementing proficiently OSGi bundles are the same in every framework.

Test-Driven Development has to be fast for the developer to be able to change the design at will. The moment the tests become too slow is the moment the developers stop to run them. Unit tests should run in the order of seconds, not minutes: their speed is a component of their effectiveness. Thus, execution speed is the main metric we have to tune.

The OSGi platform

My approach was dictated by the project I'm expanding: use test bundles, which are specific bundle containing only tests. They import the package(s) under test and perform black box testing of the bundle. Although testing only to the packages exported by the bundle may seem a strong limitation, actually if you need to test-drive an internal class it is probably the sign that you need to separate the bundle responsibilities (breaking it up in more than one bundle).

The subjects of the test written in the TDD process are usually the application's end points (in my case the web interface) or OSGi services. OSGi is not invasive if used right, and Declarative Services are definitely the way to go if you want to TDD your bundles. Dependency Injection is favored between Declarative Services, and it is configured by XML component descriptors so that your services remain Plain Old Java Objects while the responsibility to build a Factory is left to the OSGi framework.

In every Declarative Service, you have to include at least the default empty constructor which will be called by the framework, and that has to instantiate the default collaborators not implemented as Declarative Services; however, you are free to add other constructors to use in tests to inject stubs and mocks.

The lifecycle of the service will be managed by the framework in production. The dependencies towards other Declarative Services are resolved by the OSGi framework, and injected through bind methods defined via XML. Setter injection is a good way of breaking dependencies, almost as constructor injection.

Unit tests

Unit testing has usually a single class as a target: once the basic framework is running, there is no need to start an OSGi service even if the class under test implements one. The test bundle should share the necessary code with the parallel production code bundle, so that the service can be instantiated in the test case class. Since the Declarative Service is a POJO, if the test bundle imports the necessary interfaces from sibling bundles it can provide stubs and mocks to the class under test.

The inherent dependency towards JUnit of the test cases can be dealt with a specific bundle which contains a JUnit jar (included in the Eclipse distribution) and exports the Api package.The alternative is including the jar in every Bundle-ClassPath, a solution that does not scale.

Testing of components that do not cross the bundle boundary are even simpler than for Declarative Services, as long as their package is exported. You can instance them in a JUnit test case and exercise them as much as you want, without even starting the OSGi framework (I assume the IDE is smart enough to solve the dependencies; Eclipse is).

Functional tests

Functional testing is focused on a subset of the object graph, and exercise the complex of more than one class (and usually more than one bundle). This means that the Declarative Service used as a Facade should be started along with all the sibling bundles that contain its collaborators. In Eclipse, the test should be run as a JUnit Plugin Test instead of as a simple JUnit Test.

However, there is still no need to start the whole application: you should define a subset of bundles to be started and communicate that subset to your OSGi framework. In the case of Equinox/Eclipse, this takes the form of a custom config.ini, usually contained in the test bundle, which defines the property osgi.bundles. Eclipse is capable of even generating a custom config.ini which contains the necessary dependencies, but a) I won't trust it to always work and b) a config.ini is needed anyway to run the functional tests in an environment different from the IDE.

Acceptance tests

Acceptance tests, which exercise the whole system, are necessarily another story. The whole application should be started in a sandbox, and exercised in its functionalities which may itself take some minutes to operate. Add to this the burden of starting the framework and the bundles before every test and you get the heaviness picture I was talking about.

To avoid this situation, it is considered good practice to execute all the test cases in a bundle's test suite while starting the application only one time (again, the AllTests suite should be run as a JUnit Plugin Test.) This leaves open the possibility of test that interfere with each other, and you'll have to keep an eye open to maintain the tests well isolated.

By the way, with this approach only one startup for test bundle is needed. Running acceptance tests in a specific environment, like a Continuos Integration server, is probably the best approach to avoid staring at an IDE for minutes to have the tests run.