Mutation Testing: Covering Your Code With the Right Test Cases (Part 2)

DZone 's Guide to

Mutation Testing: Covering Your Code With the Right Test Cases (Part 2)

Check out our second installment on how to mutate code and obtain quality code coverage.

· Java Zone ·
Free Resource

Image title

In part one, we achieved 100 percent coverage. In this installment, we mutate our code and check how good our tests are.

There are several mutation testing frameworks. In this case, we will use PITEST.

Image title

What mutation might be applied to the code? Here is a list of pitest mutations here:

Image title

Let me introduce our mutants (just few of them). First is the line of origin, and the following are mutants (modified code is highlighed by red):Image title

Image title

Now, let's execute the pitest Maven plugin and check the mutation testing result:

mvn clean test

mvn org.pitest:pitest-maven:mutationCoverage

Open report located in target/pit-reports/201902032155/index.html

Image title

So, the report says:

  1. Line coverage 100 percent, but we knew it from the JoCoCo report earlier
  2. There are two mutation that survived and 23 were killed.
  3. Mutant 1 was created by changing conditional boundary at line 14 and the second by replacing return with true.

Let's clarify what mutants looks like:

Image title

So, what are the missing test cases that might eliminate our mutants?

  1. For our first mutant, we create a boundary case and assert that the client is created as far as zero passport id is valid
  2. For the second mutant, we check that the service doesn't have an unexistent client.

Image title

Results give us 100 percent killed mutants, but can we rely on it?

Unfortunately not. Mutation testing gives us just some hints —what cases we might forget (especially for boundary conditions).If we count all possible application set of states, we found out there are more "missing" cases.

Criticism of Mutation Testing

  • Time-consuming. Due big number of execution: number of mutations * number of tests
  • Creates impression of fully tested app if no mutations been found 
  • Don't provide clear metrics of test quality 


  • Mutation testing helps us to find possible missing test cases
  • Mutation testing can't find all missing test

Lastly, I didn't describe a case when mutation can be equivalent to the initial code, but it does not happen that often.

Code used in this article is available on GitHub.

code coverage, java, maven, mutation, mutation testing, pit, pitest

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}