Unit Tests Vs. Integration Tests: Battle Continues

One would expect that after being in the industry for almost 20 years, a trivial comparison like this would be years behind me, but I am probably less than correct. It seems that the right amount of unit tests is still a burning issue that needs some attention. At Knoldus, we have the policy of maintaining a very high code coverage so that we can objectively validate the quality of the code through our now publicly available tool CodeSquad.

We place a very high premium on writing very very effective unit tests. However, a recent discussion with one of our esteemed clients made us rethink this strategy. Let us be clear though that we do not believe that we should not be writing unit tests but the distribution between the unit and integration tests, the scenarios in which we can skip unit tests do need a review. The following is an attempt to do that:

Let us start with the advantages of the unit tests:

TDD and Unit Tests

1. Allow us to do a better design. (This is the one in which I believe the most) If the software is easily testable, it is modulated correctly, has the right structure, and follows SRP to allow for separation of concerns
2. Allow us to make big changes in the code easily since we know that we have a test harness to fall back on
3. Provide lightning visual feedback and are the fastest feedback cycles
4. Help with providing documentation and allow for easier code re-use
5. Help with coding constipation. When faced with a large and daunting piece of work ahead writing the tests will get you moving quickly.
6. Boost developer productivity and confidence

Now, let us look at scenarios in which unit tests might not be "as" useful

Unit Test Smells

1. Scenarios wherein the code is obvious in the first glance. There isn't a cyclomatic complexity to talk about. If the code is picking up the current RAM usage and passing it to the logging service, then there is less benefit of testing it
2. Scenarios wherein all the work that a piece of code is doing is to add coordinating logic. In these scenarios, there are a lot of units involved and the testing involves quite a few mocked objects, which might not add a lot of value. All that the piece of code is doing is delegating the flow or coordinating with various units without any algorithmic logic.
3. Tests are written just to improve the code coverage, but your gut would tell that they do not matter. These would be tests check whether a particular log statement is published or not. These are very low-level tests and would be very very brittle.

Now, let us look at scenarios in which unit tests are absolutely mandatory:

1. Scenarios that involve algorithmic logic: Typically, this means self-contained algorithms for business rules or for things like sorting or parsing data. This cost-benefit argument goes strongly in favor of unit testing this code because it's cheap to do and highly beneficial.
2. Complex code with many dependencies: This code is very expensive to write with unit tests but too risky to write without. The best option is to refactor this code so that it falls in the first category.

Let us look at the quadrants of unit testing with this post

As you would notice, it is less useful to write unit tests for code, which is either trivial or just does coordination.

So, how does this compare to integration tests and end-to-end tests?

In his blog, Martin Fowler described the following pyramid

As you would notice, it is easy, cost-wise, to write unit tests that run faster as well as compared to end-to-end tests, which are costly and run much slower. One of the parameters that is not brought about in the post explicitly is:

The Level of Confidence

With unit tests, we can ensure that the units are working fine; however, the level of confidence is lower as compared to an E2E test where we are assured that the system is working fine.

The sweet spot, however, is somewhere in between.

Integration tests strike a great balance on the trade-offs between confidence and speed/expense. This is why it's advisable to some of your effort here as well.

Integration tests would run across the units, provide more level of confidence, reduce the need for mocking at all places, and would run easily with your build cycles without causing undue stress on cost or confidence.

Conclusion

There are major advantages of unit tests, and the most important, in my perception, is the clarity of design. Well-designed software can be tested easily. If you are finding it hard to test a piece of software, then it needs a design re-look in most cases.

That said, going overboard with unit tests to improve code coverage is a unit test smell. When you find yourself making endless mock objects and the logic falls under the category of unit test smells, as described above, then we are getting diminishing returns.

A common myth is also that unit tests slow down the development. Done right, unit tests speed up the development cycles.

Hence, write unit tests for all the mandatory cases described above; for other scenarios, make sure that the pieces for which you might want to skip the unit tests are covered by integration tests. 

Happy testing!