Modular Java 9 Apps With Gradle and Chainsaw

For the last few months, I have observed the development and adoption of the Java 9 module system, also known as Project Jigsaw. The final result is impressive, however, I also see a lot of confusion among regular developers in how to actually use modules. The tool support does not help, either. The final days of Jigsaw development were really hot, and some important decisions were made at the last minute. No doubt the authors of many popular tools had little time to make the necessary changes and, in my opinion, on September 21, we woke up a bit unprepared for Jigsaw.

My first attempt to add a module descriptor to an existing, small application was unsuccessful. I spent four hours figuring out how to solve package splits among third-party dependencies and how to add the necessary CLI switches to my Gradle build to make everything work. I found the experimental-jigsaw Gradle plugin, but I realized, that it has its own limitations, too (e.g. I could not add a mocking library to my tests, and I was restricted to JUnit 4!). However, the experience was worth it — I realized that I can use it to make a better Jigsaw plugin for Gradle and bring modules closer to developers. So, here we go with the Gradle Chainsaw Plugin!

Sample Project

Let's begin with a sample project structure:

The initial build script allows us to work on a regular Java 9 application without modules, using JUnit 5 for testing:

buildscript {
    repositories {
        mavenLocal()
        mavenCentral()
    }
    dependencies {
        classpath 'org.junit.platform:junit-platform-gradle-plugin:1.0.0'
    }
}

plugins {
    id 'java'
    id 'idea'
}

apply plugin: 'org.junit.platform.gradle.plugin'

group 'com.example.foo'
version '0.1.0-SNAPSHOT'

ext.log4jVersion = '2.6.2'
ext.junitVersion = '5.0.0'
ext.mockitoVersion = '2.10.0'

sourceCompatibility = 1.9
targetCompatibility = 1.9

repositories {
    mavenLocal()
    jcenter()
}

junitPlatform {
    filters {
        engines {
            include 'junit-jupiter'
        }
    }
    logManager 'org.apache.logging.log4j.jul.LogManager'
}

dependencies {
    testCompile('org.junit.jupiter:junit-jupiter-api:' + junitVersion)
    testCompile('org.mockito:mockito-core:' + mockitoVersion)
    testRuntime('org.junit.jupiter:junit-jupiter-engine:' + junitVersion)
    testRuntime('org.apache.logging.log4j:log4j-core:' + log4jVersion)
    testRuntime('org.apache.logging.log4j:log4j-jul:' + log4jVersion)
}

The last file is settings.gradle:

rootProject.name = 'modular'

Creating a Module Descriptor

The first step to introducing modules is creating a module descriptor in our /src/main/java/module-info.java file:

module com.example.foo {
   exports com.example.foo;
}

Module descriptors use a Java-like meta-language. From the language point of view, a module is a collection of related packages and controls the visibility rules for them. Because of that, the module name should always be derived from the root package — it's just like saying ,"I'm taking control over this package name space." This is also the official recommendation for naming modules given by chief Java architect Mark Reinhold and other Java experts. In our example, all the classes are in the com.example.foo package, so this will also be the name of our module.

Note that this recommendation is not enforced by the Java compiler. The first reason is that modules are added to a language with a 20+ year history, and they must work with the existing code base. The second reason is that, originally, the idea for naming modules was different, and it was changed in the final months of development. We'll get back to that later.

When choosing the module name and creating packages, we must pay attention to a couple of rules:

• In Java 9, a package cannot exist in more than one module. If we try to use two modules with the same package inside them, we get a compilation error. This is the so-called package split.

• Modules allow you to control the visibility of individual packages. If you make the package com.example.foo available to other modules, the subpackages are not opened — unless you tell Java to open them, too.

• We should not change the name of our module once we release it or we risk a module hell.

To explain module hell, imagine the following situation:

1. You release foo-1.0 that uses com.example.abc as a module name.

2. To use the classes of another module, we must declare it in the module descriptor with the requires clause. Someone creates a project, bar, that depends on foo-1.0 and requires com.example.abc in the module descriptor.

3. We release foo-1.1 and \ change the module name to com.example.def.

4. Someone else creates a project, joe, and requires com.example.def in the module descriptor.

5. Yet another person creates a project, moo, that depends both on bar and joe.

The build system should deal with the version conflict and perhaps use foo-1.1. However, the module descriptors refer to foo as both com.example.abc and com.example.def. From the perspective of Java, these are two distinct modules with exactly the same packages inside. What's this? It's a package split, and we know it's illegal. Project moo doesn't compile, and its authors can do nothing about it... So, never change the name of a released module.

Let's see what we can do within module descriptors. There are a couple of available constructs, summarized by the following table:

The most commonly used statements will be, of course, requires and exports. If we want to use some class from another module, we must require it (and make sure that the class is in the exported package). If we want to publish some API for another module, we must export the package with it.

Keeping private things private is one of the main reasons for getting interested in modules. Ask yourself: How many times you released a new version of a library and someone else complained that you broke something, because he used some internal API? How many times did you import ImmutableList from jersey.repackaged.com.google.guava instead of the "official" Guava implementation? How many times did you fix such imports in others' code? How many times did something break out because of it? Modules solve this issue in a similar way — the private keyword keeps class internals hidden from the outside world.

Now it should be obvious what our module does — and what to do if we want to extend it.

How to Build It

OK, it's finally time for Gradle. Currently, it does not offer any official support for compiling Jigsaw modules, so we need an extra plugin: Gradle Chainsaw. All we need to do is to load it and select the module name:

plugins {
    id 'java'
    id 'idea'
    id 'com.zyxist.chainsaw'
    version '0.1.3'
}

// ...

javaModule.name = 'com.example.foo'

The plugin needs the module name to configure the compiler and the JVM with the necessary CLI switches. This name can be — of course — extracted from the module descriptor. The only issue is that Gradle must run on earlier Java versions, and we don't have the access to the official JDK APIs for modules.

The plugin does one extra thing that is not done by JVM — it verifies that the module name matches the root package name, effectively enforcing the official recommendation.

Let's Add Tests

Unit tests are an interesting use case for the Java module system. The /src/test/java directory doesn't contain any module descriptor. In the test phase, both the compiler and JVM see tests as a part of our module (of course, they are not packaged into the final JAR). Thanks to that, the tests can use the same package names and have unrestricted access to all the classes and interfaces.

There is, however, a small problem with using additional testing libraries. They are modules, too, so we need to require them, but we don't want to put them into our module descriptor for obvious reasons.

Chainsaw helps us here in two ways. Firstly, it allows you to specify additional test modules in your build.gradle file. They are added dynamically to the compiler and the JVM so that we can use their APIs in our tests. Secondly, it automatically detects JUnit 4 and JUnit 5 and configures the necessary modules for us.

Let's try to add a Mockito module to our tests. The necessary dependency is already present in the build file, so we just need to configure the module:

// note: this module name was auto-generated by Java
// starting from version 2.10.5, Mockito will use org.mockito module name.
javaModule.extraTestModules = ['mockito.core']

dependencies {
// ...
testCompile('org.mockito:mockito-core:' + mockitoVersion)
    // ...
}

Let's write a simple unit test to make sure everything works:

package com.example.foo;

import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.Mockito.mock;

public class FooTest {
   @Test
   public void shouldDoSomething() {
      Object obj = mock(Object.class);
      assertTrue(true);
   }
}

The project should compile and we should see the test report after running it.

Problematic Dependencies

One could say: Okay, all problems solved! Unfortunately, not this time. Remember the section about package splits and adding modules to a language with 20+ years of history? It means that there are plenty of JARs out there that break Jigsaw's rules, and if we try to use them in our application, we can run into trouble.

When we try to use a non-modular JAR archive in our modular application, Java creates a so-called automatic module from it:

• It exports all the packages to everyone and has the access to all other modules (+ the good old classpath, which is not normally used in Jigsaw).

• The module name is chosen automatically:

  • From the Automatic-Module-Name entry in the JAR manifest.

  • Or (when missing) it is generated from the archive name — this is the remnant of the old idea for naming modules I mentioned earlier.

The first issue that can break our code is missing the Automatic-Module-Name entry in your JAR manifest. It was added to Jigsaw at the last minute, and many developers simply didn't hear about it. Basically, it allows you to choose a stable module name for the future before you migrate to Jigsaw. If it is missing, Java generates the module name from the JAR archive, which should be considered as pretty random, and it has nothing to do with the actual content of the module. Even worse, if the JAR archive uses some name that is a restricted keyword in Java, e.g. foo-native, we won't be able to require such a module in our descriptor because the compiler would complain that native cannot be used as an identifier. And if we make a dependency on such a name, and later the authors choose another name, we'll run into module hell.

So, here's the simple rule we should follow:

Never publish a module in public repositories (Maven Central, JCenter, corporate Artifactory, etc.) that depends on JARs with neither module descriptors, nor Automatic-Module-Name entries in the manifest.

Unfortunately, Chainsaw won't help us with module naming issues because it would be strange to extract JAR archives and modify them on the fly. However, there is one more issue with legacy third-party dependencies. They can produce package splits! The most unfortunate example of such a dependency is jsr305.jar. The artifact was produced by the team behind the FindBugs static analysis tool and is a random collection of annotations inspired by JSR-305. The JSR itself was rejected and abandoned, but the archive "illegally" inherited the claimed package name javax.annotation. Unfortunately, there is another JSR (250) that was released and uses the same package, too. Java 9 refuses to compile and run our application if both JARs appear on the module path. JSR-305 is a dependency of several popular libraries, such as Google Guava, so there is a good chance that your application will have it.

To deal with this issue, we must patch JSR-250 module with the annotations from jsr305.jar:

javaModule.patchModules 'com.google.code.findbugs:jsr305': 'javax.annotation:jsr250-api'

dependencies {
    patch 'com.google.code.findbugs:jsr305:1.3.9'

    compile 'javax.annotation:jsr250-api:1.0'
    compile 'com.google.guava:guava:23.1-jre'
}

How it works:

1. We instruct Chainsaw to patch the jsr250-api dependency with jsr305.

2. We add the jsr305 dependency to a special configuration called patch. Gradle still needs the actual JAR, but it must be removed from all other configurations so that, e.g., the compiler couldn't see it.

3. The plugin generates the necessary CLI switches to the compiler and JVM.

4. The annotations from jsr305 are visible as a part of the jsr250.api module.

The last thing we need is to require the module in our descriptor:

module com.example.foo {
    requires jsr250.api;  // note: auto-generated, unstable name!
    requires guava; // note: auto-generated, unstable name!
}

Gradle remembers the patch during compilation, test execution, and when running the application with the run task. However, if we are going to deploy our JAR somewhere and start it manually (e.g. with some scripts), we must remember that the patch is not preserved in the final archive and we must add the necessary --patch-module CLI switches on our own. Chainsaw helps us during the build because the --patch-module switch requires the full path to the JAR archive, and it's hard to imagine hardcoding them into the build script.

Patching is intended as a temporary solution until the authors of third-party libraries solve the issues with their code. It is expected that the need for patching is going to decrease over time. However, for now, it's the only way to use many popular tools.

Summary

It will take some time until the Java ecosystem gets used to modules. I expect that the tool support will improve over time and eventually, Gradle will get first-class support for Jigsaw. For now, you can use the Chainsaw plugin to play with modules and see how they fit into your applications.

• Plugin Portal: https://plugins.gradle.org/plugin/com.zyxist.chainsaw

• GitHub: https://github.com/zyxist/chainsaw

The Polish version of the article is also published on my blog!