Measure Code Coverage by Integration Tests with Sonar

You probably know already that JaCoCo is the most performant code coverage engine. But you might not know that you can now combine it with Sonar to assess the code coverage by integration tests. This was the most voted Sonar issue (SONAR-613) and the latest version of the Sonar JaCoCo Plugin solves it. I am now going to explain how.

Let's first come back to the needs :

• There should be an easy way to assess code coverage by integration / functional / acceptance / UI tests (let's say integration tests), whatever system is used to execute those tests : Maven plugin (maven-surefire-plugin, maven-osgi-test-plugin, maven-failsafe-plugin...), Ant script, GreenPepper, Selenium... and also whatever the package format : jar, ear, war... This can really be tricky and even a nightmare with coverage engines that use source code instrumentation (Clover) or offline bytecode instrumentation (Cobertura or Emma). Indeed an application is usually made of multiples java libraries and doing static instrumentation of each java libraries is often too complex to automate. The only suitable approach for this usage is to do on-the-fly byte code instrumentation like JaCoCo does.
• Code coverage by unit tests and code coverage by integration tests must not be mixed as the information provided by each one is not the same. For instance, having a very good code coverage by integration tests but a poor code coverage by unit tests is a good start but not really a good news for the productivity of the development team. On the other side, having a very good code coverage by unit tests but a poor code coverage by integration tests will not prevent regressions to appear in production environment. This is why there should be a new set of dedicated metrics for integration tests.
• For multi-modules Maven projects, there should be a way to compute overall code coverage by unit tests across modules. This case clearly lies between unit tests and integration tests but this is widely used and a lot of users have requested the feature.

The Sonar JaCoCo plugin covers the 3 needs in two steps :

• Launch your integration tests after having configured the JaCoCo agent to be launched with the integration tests and to dump a Jacoco result file at the end of the execution
• Configure and launch Sonar to reuse the JaCoCo result file. The Sonar plugin will extract only the required code coverage information

Let's take an example with a Maven project containing three modules A, B and C. The C module is only used to execute integration tests with the Maven Failsafe Plugin and we would like to get the code coverage on module A and B by integration tests contained in module C. The following line must be added to the configuration of the Maven Failsafe Plugin in the pom.xml file of the C module :

<argLine>-javaagent:${jacoco.agent.path}=destfile=${jacoco.file.path}</argLine>

The integration tests on module C must then be launched with the following command line :

mvn -Djacoco.agent.path="PATH_TO_AGENT" -Djacoco.file.path="PATH_TO_DUMP" -Prun-its clean install

After having run integration tests you can execute Sonar analyzes on the overall project. Note that you still can use Clover, Cobertura or Emma to assess code coverage by unit tests together with JaCoCo for integration tests. mvn -Dsonar.jacoco.itReportPath="PATH_TO_DUMP" sonar:sonar And here is what you will get in Sonar in addition to code coverage measures by unit tests :

Of course, it's possible to drilldown to the source code in order to see which lines are covered or uncovered by integration tests :


To get the code coverage by unit tests across all Maven modules, the configuration steps are pretty similar :

• Add the argline property to your Maven Surefire Plugin configuration
• execute mvn clean install -Djacoco.agent.path="PATH_TO_AGENT" -Djacoco.file.path="PATH_TO_DUMP"
• execute mvn -Dsonar.jacoco.itReportPath="PATH_TO_DUMP" sonar:sonar"

It is now time for you to give a try to new version of Sonar JaCoCo Plugin. We are waiting for your feedback !