Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

jar-with-deps don’t like META-INF/services

DZone's Guide to

jar-with-deps don’t like META-INF/services

· Java Zone
Free Resource

Check out this 8-step guide to see how you can increase your productivity by skipping slow application redeploys and by implementing application profiling, as you code! Brought to you in partnership with ZeroTurnaround.

Recently, I was preparing a connection checker for Deployit’s powerful remote execution framework Overthere. To make the checker, as compact as possible, I put together a jar-with-deps1 for distribution.
Tests and trial runs from the IDE worked, so I was expected the dry-run of the distribution to be a quick formality. Instead: boom!
Turns out that one of the libraries used by Overthere, TrueZIP – or indeed any code that utilizes Java’s SPI mechanism2 – doesn’t play well with the jar-with-deps idea.

Seeing Double

TrueZIP uses a de.schlichtherle.truezip.fs.spi.FsDriverService provider configuration file in META-INF/services to register drivers for various archive formats, and the checker was failing because one of the required drivers wasn’t being loaded, even though the code was correctly included in the jar-with-deps.

The duplicate option of Ant’s Jar task3 and Gradle issue 1050, which talks about merging files during archive creation, hint at what the problem is: once packaged up in a single jar-with-deps archive, only one of TrueZIP’s provider configuration files was being found.

Maven, Gradle and Java

Once I examined the jar-with-deps constructed by Maven4, it was pretty clear why: the JAR only contains one of the configuration files – looks like the first one encountered in my case, but that may be non-deterministic. Presumably, this happens because Maven uses a temporary directory to prepare the archive contents, which of course can’t hold multiple files of the same name.

With Gradle5, things are a bit more interesting because the archive does indeed contain all the configuration files – the ZIP format supports duplicate entries6. However, ClassLoader.getResources doesn’t play along, so again only one entry is found.

There Can Be Only One

Basically, it seems that merging the affected files is the only feasible way around this. Here’s a possible Gradle snippet7:

task jarWithDeps(type: Jar, dependsOn: classes) {
    mergeDir = "${buildDir}/merge"
    // might need a lazy var in a multi-module project where deps are inherited
    runtimeDeps = configurations.runtime.collect { zipTree(it) }
 
    doFirst {
        new File(mergeDir).delete()
        mergeFiles(mergeDir, runtimeDeps, file-to-merge)
        // ... possibly other files too
    }
 
    // this project's classes and all deps
    from sourceSets*.classesDir
    from(runtimeDeps) {
        exclude file-to-merge
    }
    from mergeDir
}
 
private def mergeFiles(targetDir, fileTrees, relativePath) {
  // prepare the merge
  mergedFile = new File("${targetDir}/${relativePath}")
  new File(mergedFile.parent).mkdirs()
 
  fileTrees*.matching({ include "**/${relativePath}" })*.each {
    mergedFile << it.bytes
  }
}

Unfortunately, I haven’t been able to find a similar option for the Maven Archiver, but I guess there must be something out there in the Maven ecosystem ;-)

Footnotes
  1. A.k.a. “farJar”. For an equivalent Gradle task definition, see here.
  2. See MultiSPI for an approach that provides more flexibility and “modern alternatives” for service providers.
  3. Note that the jar utility itself doesn’t provide a similar option.
  4. run mvn clean package
  5. run gradle clean jarWithDeps
  6. although the ZipOutputStream evidently wasn’t too happy with them and, judging by the 1.6.0_20 implementation, will still throw a ZipException
  7. Would be interesting to try to do the merging at JAR construction time, perhaps by keeping “merge-so-far(s)” in the mergeDir and using eachFile to replace the unmerged files and update the “merge-so-far(s)” at the same time.

 

From http://blog.xebia.com/2011/07/jar-with-deps-dont-like-meta-infservices/

The Java Zone is brought to you in partnership with ZeroTurnaround. Check out this 8-step guide to see how you can increase your productivity by skipping slow application redeploys and by implementing application profiling, as you code!

Topics:

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

SEE AN EXAMPLE
Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.
Subscribe

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}