TDD for multithreaded applications

This article describes some practices for test-driving multithreaded and distributed applications written in Java. The example I worked on and we will use is a peer-to-peer application composed of many Nodes (clients) and of a few Supernodes (servers).

The ultimate goal it to build an application composed of all these entities, but the first tests target a Supernode serving one or more Nodes.

The walking skeleton

TDD is mostly iterative, but needs a starting point. The simplest story we can think of is that of a Node connecting to a Supernode.

Client and servers usually run in their own threads (in the case of the server, multiple ones), but initially the Node object can just be a POJO and run in the test's thread because we do not need to manage multiple Nodes yet.

The Supernode object instead is a Thread (or a Runnable) and so we already face a simplified version of the synchronization problem: how to make sure the Supernode is ready to answer to connections once we have started its thread?

The JUnit test is the following:

    @Test
    public void aNodeCanConnectToASupernode() throws Exception {
        Supernode supernode = new Supernode(8888);
        supernode.start();
        supernode.ensureStartupIsFinished();
        Node n = new Node();
        n.connect("127.0.0.1", 8888);
        assertEquals(1, supernode.getNodes());
    }

supernode.start() runs the new thread, while the call to ensureStartupIsFinished() will have to block until the other thread is ready. Then, we create a Node object and tell it to connect; after it has finished this operation, we count how many nodes have connected to the Supernode.

To satisfy this test, the Supernode can be a single-threaded server:

public class Supernode extends Thread {

    private int port;
    private boolean startupCompleted;
    private int nodes = 0;

    public Supernode(int port) {
        this.port = port;
        this.startupCompleted = false;
    }

    public void run() {
        ServerSocket sock;
        try {
            sock = new ServerSocket(this.port);
            // ...networking setup...

        synchronized (this) {
            this.startupCompleted = true;
            notify();
        }
          while (true) {
            // ...accepting new connections on sock and other stuff
        }
    }

    public int getNodes() {
        return nodes;
    }

    synchronized public void ensureStartupIsFinished() throws InterruptedException {
        while (!this.startupCompleted) {
            wait();
        }
    }

}

What's in this first example?

• Thread objects are manageable as POJOs from a single JVM: as long as we write them with this API it will be simple to instantiate and terminate them, and to add primitives for synchronization.
• The startupCompleted field, which is an example of this synchronization behavior added to the production code. Adding production code just for end-to-end testing purposes is not uncommon.

The test thread blocks inside ensureStartupIsFinished() until it is woken up via notification. Even then, startupCompleted must be true or it will wait more. This is Plain Old Java Synchronization: note the synchronized blocks around this.wait() and this.notify(). The problem with frameworks and containers is you have to hope they provide the synchronization facilities to test your code once it's inside them: have you ever tried to wait for Tomcat to start?

There are some noticeable missing parts in this code:

• the threads for each node. The current test does not require them as only one Node is connecting for now.
• Thread.sleep() calls: at least for the happy paths I have covered until now, I never need to introduce them and considered them a smell.
• Configuration files: if we had to read configuration, the tests would take really long to write and would refer continuously to external resources. This is the case when testing with external tools which are not embeddable (Tomcat requiring configuration files while Jetty allowing configuration to be passed in Java test code). You can always add file-based configuration later, but for now it will slow us down.

Evolution

By adding one test at the time with a larger scope, we can try to evolve the code and add the difficult networking, multithreading part one bit at a time.

After some iterations, the test becomes:

public class FileSharingNetworkTest {
    Supernode supernode;
    
    @Before
    public void setUp() throws Exception {
        supernode = new Supernode(8888);
        supernode.start();
        supernode.ensureStartupIsFinished();
    }
    
    @After
    public void tearDown() throws Exception {
        supernode.ensureStop();
    }
    
    @Test
    public void aNodeCanConnectToASupernode() throws Exception {
        Node n = newNode(Arrays.asList("1.txt", "2.txt"));
        n.ensureConnectionIsFinished();
        assertEquals(1, supernode.getNodes());
        assertEquals(2, supernode.getDocuments());
    }
    
    @Test
    public void multipleNodesCanConnectToASupernodeSimultaneously() throws Exception {
        Node n1 = newNode();
        Node n2 = newNode();
        n1.ensureConnectionIsFinished();
        n2.ensureConnectionIsFinished();
        assertEquals(2, supernode.getNodes());
    }
    
    private Node newNode() {
        Node n = new Node("127.0.0.1", 8888);
        n.start();
        n.setDocumentList(Arrays.asList("1.txt", "2.txt"));
        return n;
    }
    
    private Node newNode(List<String> documentList) {
        Node n = new Node("127.0.0.1", 8888);
        n.start();
        n.setDocumentList(documentList);
        return n;
    }
}

The server-side code doesn't have multiple threads yet. What is the test case that will call for them? You have to find it and write it. This workflow will ensure that there is a test that targets this case. In my case, it was the first test requiring interaction between the two clients, where one had to see the documents listed by the other after both had connected.

Even if you know where you will end up, you can test-drive the implementation: the advantage is that you understand better a standard design and ensure its test coverage. After a few more tests, I have reached a multithreaded server with a main thread and chidren for managing the connections; and Node objects implemented as independent threads.

Conclusions

When working with TDD at a system scale that includes asynchronous behavior, we should strive for a test suite that is:

• fast; even with multiple threads to wait for, a single end to end test should take less than a second to complete.
• Comprehensive; TDD makes us only write tested code instead of copying down snippets from the web.
• Robust: totally deterministic, as every run will either pass or fail, even when repeated dozens of times. There should be no sleeping calls for all the happy paths; there should be synchronization and stopping facilities built into the system.
• Featuring unit tests: along with the end to end tests we should write unit tests for the objects we need to extract (and that will be single threaded). It was easy for me to get caught up into covering more and more cases with a full scale test, but unit tests are better at pointing out where a bug resides.

We also have to keep in mind how to design our objects and interfaces:

• not starting with N threads but with at most 1 more than the test's one (the server or a remote peer).
• Evolving them: adding a few lines of verbose Java networking code each time. My example has evolved to N client threads, a server main thread and N server children threads talking with each client. I will now have to evolve it to a network of supernodes, being this about a file sharing network; to introduce secure channels and certificates. The difficult part is to constantly refactor to support new stories without having to rework the whole system for a single one.
• Not only extracting methods (an automated operation), but also to extract interfaces and most importantly objects; targeting the longest and complex classes and chopping them down into basic responsibilities.