Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Mutation Testing: Covering Your Code With the Right Test Cases (Part 2)

DZone 's Guide to

Mutation Testing: Covering Your Code With the Right Test Cases (Part 2)

Check out our second installment on how to mutate code and obtain quality code coverage.

· Java Zone ·
Free Resource

Image title

In part one, we achieved 100 percent coverage. In this installment, we mutate our code and check how good our tests are.

There are several mutation testing frameworks. In this case, we will use PITEST

What mutation might be applied to the code? Here is a list of pitest mutations here:

Let me introduce our mutants (just few of them). First is the line of origin, and the following are mutants (modified code is highlighed by red):

Now, let's execute the pitest Maven plugin and check mutation testing result:

mvn clean test

mvn org.pitest:pitest-maven:mutationCoverage


Open report located in target/pit-reports/201902032155/index.html

So, the report says:

  1. Line coverage 100 percent, but we knew it from the JoCoCo report earlier
  2. There are two mutation that survived and 23 were killed.
  3. Mutant 1 was created by changing conditional boundary at line 14 and the second by replacing return with true.

Let's clarify what mutants looks like:

So, what are the missing test cases that might eliminate our mutants?

  1. For our first mutant, we create a boundary case and assert that the client is created as far as zero passport id is valid
  2. For the second mutant, we check that the service doesn't have an unexistent client.

Results give us 100 percent killed mutants, but can we rely on it?

Unfortunatly not. Mutation testing give us just some hints —what cases we might forget (especially for boundary conditions).

If we count all possible application set of states, we found out there are more "missing" cases.

Image title

So only for passport id, there are seven reasonable test cases (we forget case B, C, E). Test case D was found by our survived mutant #1.

Registration service is also not fully covered. You can imagine for yourself what kind of scenarios you are missing, but let's just summarize.

Conclusion

  • Mutation testing help us to find possible missing test cases
  • Mutation testing can't find all missing test

Lastly, I didn't describe a case when mutation can be equivalent to the initial code, but it does not happen that often.

Code used in this article is available on GitHub.

Topics:
java ,code coverage ,mutation testing ,mutation ,maven ,pit ,pitest

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}