Mutation Testing: Covering Your Code With the Right Test Cases (Part 1)

DZone 's Guide to

Mutation Testing: Covering Your Code With the Right Test Cases (Part 1)

Want to learn more about mutation testing?

· Java Zone ·
Free Resource

Image title

Mutation Testing Basics

The concept of mutation testing is to modify code in a small way and verify that tests detect that modification. Undetected modification gives a hint to what test is likely missing.

Mutation Procedure Next Steps:

  1. Modify code in small way (creating mutant)
  2. Execute existing test suites (killing mutant)
  3. Verify that at least one existent test failed (mutant killed)
  4. If all test succeded (mutant survived), there is likely a missing test case

But why do we have to consider testing the result of the modified code? Let's take a look at an example:

Image title

Initially, the system was fully covered by two test cases. First, two mutations have been detected by failed tests. But what about the third one? The last modification wasn't detected for two reasons:

  1. There is a missing test case.
  2. Last modification made the equivalent version of the same code

Covering Code by Tests Without Mutations: Registration API Example

To demonstrate the power of mutation testing, we create tests based on simple line coverage (old classical way). To count the line coverage, we use the JoCoCo plugin.

Assume that we have client and registration service with two functions:

  1. Register new client
  2. Check that client is already registred

Image title

The old good way to check your test coverage is to verify that every line was executed during the test suite (at least once). Lets see default test scenario:

Image title

Image title

Now, it seems that all cases are counted, right? To double check, execute JoCoCo and check code coverage rate using Maven:

mvn clean test

mvn jacoco:report

Let's see generated JoCoCo report located in target/site/jacoco/index.html file

Image title

Hmm. It seems that the coverage rate is 100 percent. But can we rely on it?

No. No, we cannot. Stay tuned for the next installment where we explore why!

java ,java testing ,mutation testing ,test coverage ,tutorial

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}