DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workkloads.

Secure your stack and shape the future! Help dev teams across the globe navigate their software supply chain security challenges.

Releasing software shouldn't be stressful or risky. Learn how to leverage progressive delivery techniques to ensure safer deployments.

Avoid machine learning mistakes and boost model performance! Discover key ML patterns, anti-patterns, data strategies, and more.

Related

  • Practical Use of Weak Symbols
  • Creating a Web Project: Refactoring
  • Generate Unit Tests With AI Using Ollama and Spring Boot
  • Understanding the Two Schools of Unit Testing

Trending

  • AI, ML, and Data Science: Shaping the Future of Automation
  • Measuring the Impact of AI on Software Engineering Productivity
  • Building Scalable and Resilient Data Pipelines With Apache Airflow
  • Microsoft Azure Synapse Analytics: Scaling Hurdles and Limitations
  1. DZone
  2. Software Design and Architecture
  3. Performance
  4. Code Coverage Isn't Only for Unit Tests

Code Coverage Isn't Only for Unit Tests

A look at the basics and subtypes of code and test coverage, and a tutorial on how to measure it.

By 
Jonatan Ivanov user avatar
Jonatan Ivanov
·
Jun. 02, 16 · Tutorial
Likes (2)
Comment
Save
Tweet
Share
23.6K Views

Join the DZone community and get the full member experience.

Join For Free

What is Code (or Test) Coverage?

Code coverage (or test coverage) shows which lines of the code were (or were not) being executed by the tests. It is also a metric which helps you to find out the percentage of your covered (executed) code by the tests. E.g.: It tells you that your codebase consists of 10 lines, 8 lines were being executed by your tests, so your coverage is 80%. (It tells you nothing about the quality of your software or how good your tests are.)

If you want to read more about code coverage, check these links:

  • Wiki - Code Coverage

  • Martin Fowler - Test Coverage

What Are the Subtypes of Test Coverage?

Basically, test coverage can be measured for all levels of tests, like unit-, integration-, acceptance tests, etc. For example, unit test coverage is a subtype of test coverage, it shows which lines of the source code were (or were not) being executed by the unit tests.

How Do You Measure Code Coverage?

The usual way (at least in the Java world) to measure code coverage is instrumenting the code. Instrumentation is a technique to track the execution of the code at runtime. This can be done in different ways:

  • Offline instrumentation

    • Source code modification

    • Byte code manipulation

  • On-the-fly instrumentation

    • Using an instrumenting ClassLoader

    • Using a JVM agent

Offline Instrumentation

Offline instrumentation is a technique to modify the source code or the byte code at compile time in order to track the execution of the code at runtime. In practice, this means that the code coverage tool injects data collector calls into your source or byte code to record if a line was executed or not.

Offline Instrumentation Example (Clover)

Clover uses source code instrumentation but I only show you the decompiled code because it is easier to get the byte code.

The original method is the following:

public static void main(String[] args) {
    SpringApplication.run(CoverageDemoApplication.class, args);
}

After instrumenting-compiling-decompiling the code, we get something like this:

public static void main(String[] var0) {
    CoverageDemoApplication.__CLR4_1_144in3j646t.R.inc(4);
    CoverageDemoApplication.__CLR4_1_144in3j646t.R.inc(5);
    SpringApplication.run(CoverageDemoApplication.class, var0);
}

The instrumented code is a bit more sophisticated, here you can check the whole file: instrumented and decompiled code

Offline Instrumentation Demo Project

I created a GitHub repo where I hack the Gradle Clover plugin in order to see the instrumented files. So if you want the red pill, I can show you how deep the rabbit hole goes: what the Clover plugin does under the hood and how to get the instrumented class files: Offline instrumentation demo

On-the-fly Instrumentation

This instrumentation process happens on-the-fly during class loading by using a Java Agent or a special Class Loader so the source/byte code remains untouched.

Picking the "Right" Coverage Tool

This question becomes more interesting above unit testing level. For example, in order to run integration tests, you need to:

  1. Start the application

  2. Run the tests

  3. Stop the application

This process is a bit more complicated than running unit tests. Suppose that we have a classic, servlet-based web application which runs in Tomcat (or in any other Servlet Container) and we want to measure the integration test coverage. In this case, the integration testing consists of the following steps:

  1. Compile the code and create the application package (a war file in our case)

  2. Start the app in Tomcat

  3. Run the integration tests and track execution data

  4. Stop the app

  5. Generate report from the collected data

If our coverage tool instruments the code offline, we will end up with two packages: one for the test environments (in order to be able to measure test coverage) and one for non-testing environments (e.g.: production). Having different packages for the same application is something that we really want to avoid.

If our coverage tool instruments the code on-the-fly, we can deploy the same package to each of our environments. The only difference will be the configuration for the different environments, e.g.: an additional JVM option where we configure the coverage tool which seems much more convenient.

Demo Project

I created a GitHub repo to show this scenario in action. So here is what you need:

  • Separate source sets for unit- and functional tests: functionalTest.gradle

  • To use the JaCoCo plugin and get the JaCoCo Agent: jacoco.gradle

  • To deploy and undeploy the application using the agent: deploy.gradle

  • Configure the JaCoCo plugin to create coverage report for unit- and functional tests: jacoco.gradle, functionalTest.gradle

  • Configure the JaCoCo plugin to create a merged report which aggregates the coverage data of the unit- and functional tests: functionalTest.gradle

  • Wire up the tasks (set task dependencies)

So if you clone this repo and run ./gradlew build, you should see something like this in the console:

$ ./gradlew build

[...]
:jar
:bootRepackage
:assemble

:compileTestJava
[...]
:test

:unZipJacocoAgent
:deploy
:compileFunctionalTestJava
[...]
:functionalTest
:undeploy

:jacocoFunctionalTestReport
:jacocoTestReport
:jacocoMerge
:mergedReport
:check
:build

And you should see three reports in the build/reports/jacoco directory. Or in the resources directory of this repo.

Covered Lines (Unit Tests, Functional Tests, Merged)

Covered lines for unit tests

Covered lines for functional tests

Covered lines

Stats (Unit Tests, Functional Tests, Merged)

Stats for unit tests

Stats for functional tests

Merged stats

Featured image: vladstudio.com

unit test Code coverage

Published at DZone with permission of Jonatan Ivanov. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • Practical Use of Weak Symbols
  • Creating a Web Project: Refactoring
  • Generate Unit Tests With AI Using Ollama and Spring Boot
  • Understanding the Two Schools of Unit Testing

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!