jar-with-deps don’t like META-INF/services

recently, i was preparing a connection checker for deployit’s powerful remote execution framework overthere . to make the checker, as compact as possible, i put together a jar-with-deps ¹ for distribution.
tests and trial runs from the ide worked, so i was expected the dry-run of the distribution to be a quick formality. instead: boom!
turns out that one of the libraries used by overthere, truezip – or indeed any code that utilizes java’s spi mechanism ² – doesn’t play well with the jar-with-deps idea.

seeing double

truezip uses a de.schlichtherle.truezip.fs.spi.fsdriverservice provider configuration file in meta-inf/services to register drivers for various archive formats, and the checker was failing because one of the required drivers wasn’t being loaded, even though the code was correctly included in the jar-with-deps .

the duplicate option of ant’s jar task ³ and gradle issue 1050 , which talks about merging files during archive creation, hint at what the problem is: once packaged up in a single jar-with-deps archive, only one of truezip’s provider configuration files was being found.

maven, gradle and java

once i examined the jar-with-deps constructed by maven ⁴ , it was pretty clear why: the jar only contains one of the configuration files – looks like the first one encountered in my case, but that may be non-deterministic. presumably, this happens because maven uses a temporary directory to prepare the archive contents, which of course can’t hold multiple files of the same name.

with gradle ⁵ , things are a bit more interesting because the archive does indeed contain all the configuration files – the zip format supports duplicate entries ⁶ . however, classloader.getresources doesn’t play along, so again only one entry is found.

there can be only one

basically, it seems that merging the affected files is the only feasible way around this. here’s a possible gradle snippet ⁷ :

task jarwithdeps(type: jar, dependson: classes) {
    mergedir = "${builddir}/merge"
    // might need a lazy var in a multi-module project where deps are inherited
    runtimedeps = configurations.runtime.collect { ziptree(it) }
 
    dofirst {
        new file(mergedir).delete()
        mergefiles(mergedir, runtimedeps, file-to-merge)
        // ... possibly other files too
    }
 
    // this project's classes and all deps
    from sourcesets*.classesdir
    from(runtimedeps) {
        exclude file-to-merge
    }
    from mergedir
}
 
private def mergefiles(targetdir, filetrees, relativepath) {
  // prepare the merge
  mergedfile = new file("${targetdir}/${relativepath}")
  new file(mergedfile.parent).mkdirs()
 
  filetrees*.matching({ include "**/${relativepath}" })*.each {
    mergedfile << it.bytes
  }
}

unfortunately, i haven’t been able to find a similar option for the maven archiver , but i guess there must be something out there in the maven ecosystem

from http://blog.xebia.com/2011/07/jar-with-deps-dont-like-meta-infservices/