Running JUnit tests in Parallel with Maven

A little-known but very useful feature slipped into JUnit 4 and recent versions of the Maven Surefire Plugin: support for parallel testing. This feature has been around for a while in TestNG, but has been missing in JUnit. And now, if you are a JUnit user, you too can run your tests in parallel!

Running your tests in parallel is a powerful way to speed up your tests. Good automated tests should be independent, isolated and reproducible, making them ideal candidates for being run concurrently. However in practice not all test classes are designed with concurrency in mind, and aspects such as shared mutable instance variables, shared file or database access, or the use of embedded web servers may make running the tests in parallel trickier. Nevertheless, running your tests in parallel is decidedly a very neat trick!

Let's see how it works. As of JUnit 4.7, you can configure Maven to run your tests in parallel, just like in TestNG. The following configuration will run your test methods in parallel:

<plugin>   <groupId>org.apache.maven.plugins</groupId>   <artifactId>maven-surefire-plugin</artifactId>   <version>2.5</version>   <configuration>     <parallel>methods</parallel>   </configuration></plugin>

Another useful trick is to run your test classes in parallel, rather than the methods. You can do this by setting the configuration item to 'classes' instead of 'methods'. This may or faster depending on the number of test classes you have, but it might also be safer for some test cases not designed with concurrency in mind:

<plugin>  <groupId>org.apache.maven.plugins</groupId>  <artifactId>maven-surefire-plugin</artifactId>  <version>2.5</version>  <configuration>    <parallel>classes</parallel>  </configuration></plugin>

For finer control over the number of threads, you should check out the the configurable-parallel-computer library (see also this blog entry from the author of the library). (This actually made little difference on my dual-core laptop, but I would expect more significant differences on a large build server). Once you add this dependency to your dependencies (it's not in the standard Maven repositories, so you have to install it yourself), you could run your tests in parallel using 4 threads as shown here:

<plugin>   <groupId>org.apache.maven.plugins</groupId>   <artifactId>maven-surefire-plugin</artifactId>   <version>2.5</version>   <configuration>     <parallel>methods</parallel>     <threadCount>4</threadCount>   </configuration></plugin>

You can also configure the maximum number of threads per CPU core, which allows for a bit more flexibility when the tests are run on different machines:

<plugin>  <groupId>org.apache.maven.plugins</groupId>  <artifactId>maven-surefire-plugin</artifactId>  <version>2.5</version>  <configuration>    <parallel>methods</parallel>    <threadCount>4</threadCount>    <perCoreThreadCount>true</perCoreThreadCount>  </configuration></plugin>

The 'useUnlimitedThreads' will create as many threads as there are classes or methods, and attempt to run them all concurrently. This works fine for unit tests, but when I used this option for web tests, it quickly saturated the server.

<plugin>  <groupId>org.apache.maven.plugins</groupId>  <artifactId>maven-surefire-plugin</artifactId>  <version>2.5</version>  <configuration>    <parallel>classes</parallel>    <useUnlimitedThreads>true</useUnlimitedThreads>  </configuration></plugin> 

Mileage will vary, depending on your machine capacity and on the nature of your tests, so you should experiment with different configurations and see what works best for you. You will generally get more significant gains from slower tests involving I/O, or web/database access, such as functional or web tests, than from large numbers of small, fast unit tests. Here are some examples I got for a medium-sized web application (running on my Macbook pro, which has 2 cores):

Using nothing:

$ mvn verify:...Tests run: 150, Failures: 0, Errors: 0, Skipped: 0[INFO] [jetty:stop {execution: stop-jetty}][INFO] Stopping server 0[INFO] ------------------------------------------------------------------------[INFO] BUILD SUCCESSFUL[INFO] ------------------------------------------------------------------------[INFO] Total time: 1 minute 2 seconds

Using 'methods':

$ mvn verify:...Tests run: 150, Failures: 0, Errors: 0, Skipped: 0[INFO] [jetty:stop {execution: stop-jetty}][INFO] Stopping server 0[INFO] ------------------------------------------------------------------------[INFO] BUILD SUCCESSFUL[INFO] ------------------------------------------------------------------------[INFO] Total time: 35 seconds

Using 'classes':

$ mvn verify:...Tests run: 150, Failures: 0, Errors: 0, Skipped: 0[INFO] [jetty:stop {execution: stop-jetty}][INFO] Stopping server 0[INFO] ------------------------------------------------------------------------[INFO] BUILD SUCCESSFUL[INFO] ------------------------------------------------------------------------[INFO] Total time: 32 seconds

Using 'both':

$ mvn verify:...Tests run: 150, Failures: 0, Errors: 0, Skipped: 0[INFO] [jetty:stop {execution: stop-jetty}][INFO] Stopping server 0[INFO] ------------------------------------------------------------------------[INFO] BUILD SUCCESSFUL[INFO] ------------------------------------------------------------------------[INFO] Total time: 1 minutes 1 seconds 

Whether 'classes' or 'methods' works best depends largely on the number of test classes you have, and the number of methods in each class. For these tests, 'both' had no notable effect.

As mentioned above, in these tests, and on my machine, increasing the threadCount had little effect. But this, again, will depend on the nature of your tests.

From http://weblogs.java.net/blog/johnsmart/archive/2010/07/06/running-junit-tests-parallel-maven