Types Of Code Coverage- Examples In C#

Code coverage analysis is used to measure the quality of software testing, usually using dynamic execution flow analysis. There are many different types of code coverage analysis, some very basic and others that are very rigorous and complicated to perform without advanced tool support. Most people believe that they have tested everything when Visual Studio Code Coverage shows 100%. But this is so wrong! Here, I will give you a couple of examples of the different code coverage types.

Statement Code Coverage

The most basic type of code coverage is the Statement Code Coverage. Yeah, you guessed it right, this is the type of code coverage performed by most IDEs (Visual Studio including).

private void PeformAction(int x, int y)
{
    this.DoSomething();
    if (x > 3 && y < 6)
    {
        this.DoSomethingElse();
    }
    this.DoSomething();
}

The goal of the statement coverage is to execute all statements in our code with least possible tests. So, here you will need only one test- x = 4 and y = 5. You are not required to execute the different combinations of the condition in the IF.

Statement Coverage = (Number of executed statements)/(Total Number of Statements)* 100%

Branch Code Coverage (Decision Coverage)

Every decision is executed with both possible outcomes – TRUE and FALSE. For Loops- bypass it, execute the body and return to the beginning.

private void PeformAction(int x, int y)
{
    this.DoSomething();
    if (x > 3 && y < 6)
    {
        this.DoSomethingElse();
    }
    else
    {
        System.Console.WriteLine(x + y);
    }
    this.DoSomething();
}

In the above example, we need two tests to accomplish this type of code coverage.

1. x = 4 and y = 5 => IF Executed

2. x = 2 and y = 5 => ELSE Executed

Branch Coverage = (Number of Branches)/(Total Number of Branches)* 100%

The calculation counts every executed branch only ones.

private void PeformAction(int x, int y, bool shouldExecute)
{
    if (shouldExecute)
    {
        this.DoSomething();
        if (x > 3 && y < 6)
        {
            this.DoSomethingElse();
        }
        else
        {
            System.Console.WriteLine(x + y);
        }
        this.DoSomething();
    }
}

The branch of the “shouldExecute” will be executed twice in our tests, but it should be counted only once in the formula.

Note: 100% Branch Coverage guarantees 100% statement coverage. Also, it requires more tests.

Branch Condition Coverage

Sometimes your conditions depend on several smaller conditions, called atomic parts. An atomic is a condition that has no logical operations such as AND, OR, NOT, but can include “<“, “>” or “=”. One condition can consist of multiple atomic parts.

private void PeformAction(int x, int y, int z)
{
    this.DoSomething();
    if ((x > 3 && y < 6) || z == 20)
    {
        this.DoSomethingElse();
    }
    else
    {
        System.Console.WriteLine(x + y);
    }
    this.DoSomething();
}

The complexity of such expressions should be considered and tested. The coverage requires every atomic part to adopt the values – TRUE and FALSE.

In the example, there are 3 atomic parts: x > 3, y < 6 and z == 20. We need two tests to accomplish 100% Branch Condition Coverage.

1. x = 2, y = 4, z = 21 ((F && T) || F) => F

2. x = 4, y = 7, z = 20 ((T && F) || T) => T

The Branch Condition Coverage is weaker than Statement Coverage and Branch Coverage because it is not required to execute every statement.

private void PeformAction(int x, int y)
{
    this.DoSomething();
    if (x > 3 && y < 6)
    {
        this.DoSomethingElse();
    }
    else
    {
        System.Console.WriteLine(x + y);
    }
    this.DoSomething();
}

Here you can accomplish 100% Branch Condition Coverage with again 2 tests.

1. x = 2, y = 3 (F && T) => F

2. x = 4, y = 7 (T && F) => F

But you don’t have 100% Statement Coverage, the code in the IF won’t be executed.

Branch Condition Combination Code Coverage

All combination of the atomic parts should be applied.

1. x = 2, y = 3 (F && T) => F

2. x = 4, y = 7 (T && F) => F

3. x = 4, y = 3 (T && T) => T

4. x = 2, y = 7 (F && F) => F

It is more comprehensive than the Statement Coverage and Branch Coverage. However, the number of tests can grow exponentially. (2ⁿ with n atomic parts in the condition).

It is a very expensive technique due to the growing number of atomic conditions. Also, all combinations are not always possible.

private void PeformAction(int x, int y)
{
    this.DoSomething();
    if (3 <= x && x < 5)
    {
        this.DoSomethingElse();
    }
    else
    {
        System.Console.WriteLine(x + y);
    }
    this.DoSomething();
}

3 <= x (F) && x < 5 (F) – the combination not possible because the value x shall be smaller than 3 and greater or equal to 5 at the same time.

Condition Determination Coverage

This coverage eliminates the previously discussed problems. Not all combinations of the atomic parts should be applied. Only the combinations where the modification of the logical value of the atomic part can change the logical value of the whole condition.

private void PeformAction(int x, int y)
{
    this.DoSomething();
    if (4 <= x || y < 6)
    {
        this.DoSomethingElse();
    }
    else
    {
        System.Console.WriteLine(x + y);
    }
    this.DoSomething();
}

If we want to accomplish 100% Branch Condition Combination Coverage, we need 4 tests.

1. x = 2, y = 3 (F || T) => T

2. x = 4, y = 7 (T || F) =>T

3. x = 4, y = 3 (T || T) => T

4. x = 2, y = 7 (F || F) => F

We can exclude the 3rd and the 4th tests in order to accomplish 100% Condition Determination Coverage.  Because even if the first/second atomic part is changed from False to True and vice versa, the outcome won’t be changed.

References:

1. Software Testing Foundations

2. Code Coverage Wiki

3. Advanced Software Testing – Vol. 1

4. Advanced Software Testing – Vol. 2