Mutation Testing: Covering Your Code With the Right Test Cases (Part 1)
Want to learn more about mutation testing?
Join the DZone community and get the full member experience.Join For Free
Mutation Testing Basics
The concept of mutation testing is to modify code in a small way and verify that tests detect that modification. Undetected modification gives a hint to what test is likely missing.
Mutation Procedure Next Steps:
- Modify code in small way (creating mutant)
- Execute existing test suites (killing mutant)
- Verify that at least one existent test failed (mutant killed)
- If all test succeded (mutant survived), there is likely a missing test case
But why do we have to consider testing the result of the modified code? Let's take a look at an example:
Initially, the system was fully covered by two test cases. First, two mutations have been detected by failed tests. But what about the third one? The last modification wasn't detected for two reasons:
- There is a missing test case.
- Last modification made the equivalent version of the same code
Covering Code by Tests Without Mutations: Registration API Example
To demonstrate the power of mutation testing, we create tests based on simple line coverage (old classical way). To count the line coverage, we use the JoCoCo plugin.
Assume that we have client and registration service with two functions:
- Register new client
- Check that client is already registred
The old good way to check your test coverage is to verify that every line was executed during the test suite (at least once). Lets see default test scenario:
Now, it seems that all cases are counted, right? To double check, execute JoCoCo and check code coverage rate using Maven:
mvn clean test mvn jacoco:report
Let's see generated JoCoCo report located in target/site/jacoco/index.html file
Hmm. It seems that the coverage rate is 100 percent. But can we rely on it?
No. No, we cannot. Stay tuned for the next installment where we explore why!
Opinions expressed by DZone contributors are their own.