Benchmarking a Java/Maven Monorepo
Ever wonder how a Maven monorepo stacks up when benchmarked? Take an example from Jooby, which uses a monorepo to organize its dependent integration modules.
Join the DZone community and get the full member experience.Join For Free
maven is the build technology that’s dominant in the java community — but it also has a "love to hate" aspect to it. if you’re in java development, you’re going to encounter it again and again, so on that basis, i’ll discuss a public monorepo that uses it: the excellent jooby web microframework.
note: i found this technology after a production usage of sparkjava that was cool, but that left me with dissatisfaction because of the static nature of it. i went as far as to complete a refactoring to make it non-static in nature that was also mostly backward-compatible. the author probably felt that it was wrong in a number of ways. anyway, then i found jooby as i was trying to stay way from springboot. apart from anything else, jooby uses a monorepo for the organization of its dependent integration modules.
git clone email@example.com:jooby-project/jooby.git
that takes 20 seconds and delivers a
folder that is 46mb in size and a checkout that is 61mb.
there are three runs of
below, each with different parameters. each takes a different elapsed time and consumes disk space differently. importantly, all these builds are from the project root.
in order to test the impact on the acquisition of jars from maven central that go into a local artifact cache, i’m doing a
rm -rf ~/.m2/repository/
jooby itself but no modules that depend on jooby
mvn clean install -pl jooby -am
this gets a total of 268 jars from maven central consuming 54mb of disk space and takes 1 minute and 41 seconds to complete. much of those 268 are plugins for maven itself. after the build, there is an additional 6mb of items in
the results in tabular form:
|jars from maven central||268|
|disk space for those jars||54mb|
|build time||1m 41s|
jooby’s mongodb integration module
there are things jooby-mongo depends on, but no modules that depend on it.
mvn clean install -pl jooby-mongodb -am
|jars from maven central||286|
|disk space for those jars||61mb|
|build time||2m 10s|
if you cd into
and run straight
mvn clean install
from there, the build time is just 17 seconds. at least, if the dependencies are already cached. recursive build systems like maven allow you to build that way, as an option. it turns out the
business is lesser-known, though, than straight
after traversing to the directory/module you’re interested in.
yup, jooby itself and all dependent modules including jooby-mongo (again).
perhaps only jooby committers themselves are interested in this build. the results:
|jars from maven central||809|
|disk space for those jars||347mb|
|build time||13m 30s|
working in an ide
the pinnacle of java ides is intellij idea, of course.
seems to work, but it doesn’t obey the
flags. that was the official maven plugin for intellij idea, but it is abandoned now.
i raised a feature request with the excellent jetbrains folks, as they have a competing (and recommended) way of loading maven projects into the ide. until that’s implemented, all the modules are going to be shown regardless of the intention of the developer.
asking jooby’s author edgar espina
edgar sharing his thinking on motivations, pros/cons, reflecting back now he’s accomplished it:
it was easy to start with… especially when i need to make a new release. all the modules are updated to follow the release. this simplifies a lot my work as framework developer but also helps framework users. on upgrades, they just change a single property/value.
today, it is very very hard to just see the number of submodules in github. it could be considered a github ux issue, too, but also i think jooby has too many modules and they need to be moved to their own repository. having modules in their own repository helps when someone needs to change something… they just need to check out the subproject and not the entire repo.
the code coverage project was also hard to implement… it has references to all the existing modules. i need this for producing a single jacoco (code coverage) report. jacoco doesn’t work well on multi-module projects.
conclusion and my opinion
edgar’s monorepo setup gives him a bunch of nebulous benefits without sacrificing build speed and disk space, given maven’s build options. maven’s command line is a little bit cryptic, though; specialist knowledge has to be shared.
consistency is one of the nebulous benefits. while the wordsmith emerson is generally right when he says “ a foolish consistency is the hobgoblin of little minds ,” in this case, he’s wrong — it was worth it.
expanding and contracting monorepos
you may be interested in a previous article of mine on jooby’s monorepo — specifically, a modification to it that allowed it to expanded and contract (like google’s) based on the intention of the developer who’s about to do work.
Published at DZone with permission of Paul Hammant, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.