Benchmarking a Java/Maven Monorepo

maven is the build technology that’s dominant in the java community — but it also has a "love to hate" aspect to it. if you’re in java development, you’re going to encounter it again and again, so on that basis, i’ll discuss a public monorepo that uses it: the excellent jooby web microframework.

note: i found this technology after a production usage of sparkjava that was cool, but that left me with dissatisfaction because of the static nature of it. i went as far as to complete a refactoring to make it non-static in nature that was also mostly backward-compatible. the author probably felt that it was wrong in a number of ways. anyway, then i found jooby as i was trying to stay way from springboot. apart from anything else, jooby uses a monorepo for the organization of its dependent integration modules.

cloning jooby

git clone git@github.com:jooby-project/jooby.git

that takes 20 seconds and delivers a .git folder that is 46mb in size and a checkout that is 61mb.

building it

there are three runs of mvn install below, each with different parameters. each takes a different elapsed time and consumes disk space differently. importantly, all these builds are from the project root.

in order to test the impact on the acquisition of jars from maven central that go into a local artifact cache, i’m doing a rm -rf ~/.m2/repository/ before each mvn command below.

jooby itself but no modules that depend on jooby

mvn clean install -pl jooby -am

this gets a total of 268 jars from maven central consuming 54mb of disk space and takes 1 minute and 41 seconds to complete. much of those 268 are plugins for maven itself. after the build, there is an additional 6mb of items in target/ folders.

the results in tabular form:

jooby’s mongodb integration module

there are things jooby-mongo depends on, but no modules that depend on it.

mvn clean install -pl jooby-mongodb -am

the results:

if you cd into jooby-mongo and run straight mvn clean install from there, the build time is just 17 seconds. at least, if the dependencies are already cached. recursive build systems like maven allow you to build that way, as an option. it turns out the -am -pl business is lesser-known, though, than straight mvn install after traversing to the directory/module you’re interested in.

building everything

yup, jooby itself and all dependent modules including jooby-mongo (again).

perhaps only jooby committers themselves are interested in this build. the results:

working in an ide

the pinnacle of java ides is intellij idea, of course.

mvn idea:idea seems to work, but it doesn’t obey the -pl and -am flags. that was the official maven plugin for intellij idea, but it is abandoned now.

i raised a feature request with the excellent jetbrains folks, as they have a competing (and recommended) way of loading maven projects into the ide. until that’s implemented, all the modules are going to be shown regardless of the intention of the developer.

asking jooby’s author edgar espina

edgar sharing his thinking on motivations, pros/cons, reflecting back now he’s accomplished it:

it was easy to start with… especially when i need to make a new release. all the modules are updated to follow the release. this simplifies a lot my work as framework developer but also helps framework users. on upgrades, they just change a single property/value.

cons?

today, it is very very hard to just see the number of submodules in github. it could be considered a github ux issue, too, but also i think jooby has too many modules and they need to be moved to their own repository. having modules in their own repository helps when someone needs to change something… they just need to check out the subproject and not the entire repo.

the code coverage project was also hard to implement… it has references to all the existing modules. i need this for producing a single jacoco (code coverage) report. jacoco doesn’t work well on multi-module projects.

conclusion and my opinion

edgar’s monorepo setup gives him a bunch of nebulous benefits without sacrificing build speed and disk space, given maven’s build options. maven’s command line is a little bit cryptic, though; specialist knowledge has to be shared.

consistency is one of the nebulous benefits. while the wordsmith emerson is generally right when he says “ a foolish consistency is the hobgoblin of little minds ,” in this case, he’s wrong — it was worth it.

expanding and contracting monorepos

you may be interested in a previous article of mine on jooby’s monorepo — specifically, a modification to it that allowed it to expanded and contract (like google’s) based on the intention of the developer who’s about to do work.