It’s known that having automated tests as part of your build process improves the software quality and reduces the number of bugs.
Do you know if you need more unit tests? Or if your tests cover all possible branches of an if or switch statements? Or if your code coverage is decreasing over time? Especially after you join a team to work on an on-going project.
Code coverage helps to answer these questions. This post covers reporting code coverage using Maven’s jacoco-maven-plugin, a library that adds minimal overhead with a normal build.
- Java 7+
- Maven 3.2+
- Overview of my previous post, Splitting Unit and Integration Tests using Maven and Surefire plugin because this post uses the same source code.
The example application has two unit test classes,
DefaultSomeBusinessServiceTest, and two integration tests classes,
ApplicationTests; this is similar to those discussed in Splitting Unit and Integration Tests Using Maven and Surefire plugin section.
Configuring jacoco-maven-plugin and Coverage ThresholdWarning: According to the JaCoCo documentation, do not set forkCount to
neveras it would prevent executing the tests with the JaCoCo
javaagentand no coverage would be recorded.
Let’s configure jacoco-maven-plugin in
prepare-agent goal sets up the property
argLine (for most packaging types), pointing to the JaCoCo runtime agent. You can also pass
argLine as a VM argument. Maven-surefire-plugin uses
argLine to set the JVM options to run the tests.
If you are explicitly setting
argLine, make sure it allows late replacements like:
This is so that the maven-surefire-plugin picks up changes made by other Maven plugins such as jacoco-maven-plugin.
The JaCoCo Java agent will collect coverage information when maven-surefire-plugin runs the tests. It will write it to
destFile property value, if set, or
target/jacoco.exec by default. Read more at https://www.eclemma.org/jacoco/trunk/doc/prepare-agent-mojo.html.
report goal creates code coverage reports for tests in HTML, XML, CSV formats. Stay tuned, I’ll cover uploading code coverage reports to SonarQube in another post. This goal reads the
dataFile property value, if set, or
target/jacoco.exec. And, it writes the resulting reports to
outputDirectory property value or
target/site/jacoco. Read more at https://www.eclemma.org/jacoco/trunk/doc/report-mojo.html.
check goal validates that the coverage rules (discussed later) are met. In case they are not, it interrupts and fails the build unless the
haltOnFailure property is set to
false. Read more at https://www.eclemma.org/jacoco/trunk/doc/check-mojo.html.
Running the Tests and Creating the Coverage Reports
Let’s build the application and analyze the Maven command output:
Right after the
clean phase completes, jacoco-maven-plugin’s
prepare-agent goal (bound to the Maven’s Build Default Lifecycle’s
initialize phase) sets the
argLine property pointing to the JaCoCo Java agent.
Unit and Integration tests ran separately as covered in a previous post.
Next, not included in this log output, the Maven artifact is built and repackaged.
After that, jacoco-maven-plugin’s
coverage-report goal (bound to the Maven’s Build Default Lifecycle’s
post-integration-test phase) generates HTML, XML and CSV reports.
Opening the HTML report at
target/site/jacoco/index.html results in:
Code coverage report for a successful build
check goal (bound to the Maven’s Build Default Lifecycle’s
verify phase) checks the code coverage metrics are met.
Let’s take a closer look at the jacoco-maven-plugin’s
coverage-checkrules configuration in pom.xml:
Setting the rule element to
CLASS means every Java class from the application would need to meet each counter limit for the build to pass.
In this example, there is only one limit, a
LINE counter that needs coverage of at least 80 percent. I’ll cover JaCoCo Counters later.
Other JaCoCo rules you could use are:
|BUNDLE||The set of counter limits would have to be met at the application as a whole|
|PACKAGE||The set of counter limits would have to be met for all packages (e.g.
|CLASS||The set of counter limits would have to be met for every Java class|
|METHOD||The set of counter limits would have to be met for every class method|
Notice that as you move down in the JaCoCo rules table, the
check goal becomes more constraining.
As an example, if you remove
com.asimio.demo.Application from the
excludes sections, the build fails because the
LINE counter doesn’t reach 80 percent for said class:
|INSTRUCTION||The amount of code that can be executed or missed|
|BRANCH||The total number of branches (if and switch statements) in a method that can be executed or missed.
|LINE||Executed when at least one instruction that is assigned to this line has been executed.
|METHOD||Executed when at least one instruction has been executed|
|CLASS||Executed when at least one of its methods has been executed|
Notice that as you move down in the JaCoCo counters table, the
check goal becomes less constraining.
Examples of associating a counter to a rule are:
value is one of:
and a numeric
Although not a silver bullet, code coverage helps to measure what percentage of code is executed when running the test suites. And thus, it helps to reduce the number of bugs and improve the software release quality.
Keeping a certain threshold might get difficult over time as a development team adds edge cases or implement defensive programming.
JaCoCo adds minimal overhead to the build process. Jacoco-maven-plugin’s
prepare-agentgoal, bound to the
initialize phase, sets the agent responsible for instrumenting the Java code before maven-surefire-plugin runs.
Coverage-report goal is bound to the
post-integration-test phase. And
coverage-report goal is bound to the
verify phase. Read more at Maven’s Build Default Lifecycle. This means, unlike other libraries, JaCoCo doesn’t need to run the tests twice.
Thanks for reading and sharing. If you found this post helpful and would like to receive updates when content like this gets published, sign up to the newsletter.
Accompanying source code for this blog post can be found on BitBucket.