My Simple View of Testing
Assumptions + important edge cases. That’s it. That’s all I consider when I’m testing my code. But knowing that you aren’t satisfied with such a trivial explanation, I’ll explain.
Since I discovered Test Driven Development (TDD) a handful of years ago I have followed it, practiced it, tried to adhere to its loose tenants. There’s no question that, in order for me to produce code that I am comfortable releasing to QA and users, TDD is the sanest path we software developers now have to follow. The outcome of my embrace of and consequent struggles with TDD is that I have developed an intuition about testing my code that only now can I explain simply as assumptions + important edge cases. That formula is how I determine when I’ve done enough testing.
Assumptions are the things you take to be true about how your code should work. That covers both the input and output side of the code you are currently working on. Since TDD drives your work down to the individual functions, your assumptions are the current list of facts you possess about what might be input to your function and what output each of those inputs should produce.
You cannot and should not test functionality that you do not need—that is, perhaps, the most fundamental tenant of TDD. The popular aphorism You’re Not Gonna Need It (YAGNI) applies here. So once you are satisfied that you have collected all the facts about some code you need to write, you only need to work on it until all of your assumptions are codified in that function. Then stop. If you don’t, you’ll start testing for functionality that only you dreamed up for the code to do. The assumptions you have gathered, most likely through Agile processes that are well documented elsewhere, represent the entire universe of outcomes that anyone, at a static point in time, care about the code producing. If the code does something outside of that tenuous list, so what? The assumption is that no one will ever ask the code to do that extra work anyway. You and the people who helped formulate the assumptions have a contract that only those assumptions matter.
Important Edge Cases
Of course there are always those conditions that your business sponsors and business analysts and such never think of—stuff that only programmers care about. Those are the edge cases. You have been trained to worry about the extremes in the input range when writing a function.
I worry about that too when I’m writing unit tests. Even when the edge cases aren’t covered by the list of assumptions. For every function there are those input values that just wreak havoc when not handled properly. Maybe, for instance, you know that letting a null value be operated on by your function is an edge case that you absolutely must deal with. Even though handling null values in some graceful manner might not be in your list of assumed behaviors for your code, you know that you have to write code to handle that situation in a reasonable manner (Want to make the world a better place? Give it fewer NullPointerExceptions to deal with).
The null value example is an instance of an important edge case. It’s just one of a small list of edge cases that you should care about though. If you wrote the code to match your assumptions already, then all you have left to do is handle the few very important edge cases that keep your code from behaving particularly badly. And that is a good place to stop testing and writing code.
There’s Always a Caveat (or two)
The caveat that has probably already caused you to think about some scathing comment to add to this post is that whole assumption part. It’s just so damn open-ended. That’s true but I’m ok with that (you don’t have to be though). The idea is that I can grasp the idea of figuring out what I, at any given time, assume about the operation of a piece of code. I can write out that finite list of assumptions and test each one. When I add the edge cases on the end of that list, I can test the code for everything I know about it’s expected functionality. In other words, assumptions + important edge cases is merely my heuristic for figuring out when I’m done writing and unit testing a piece of code.
The other thing about assumptions is that they can include more than just business-driven functionality. Depending on your development team, they could include standard programming practices around security, code monitoring, performance, etc…. You might have a common list of assumptions that applies to every function you write just to satisfy the software development standards in your organization.
You will get some overlap between the two concepts of assumptions and important edge cases. The Venn diagram (I love these things) below shows that some of your assumptions will cover some important edge cases. It’s up to you to sort all of that out and just pick the edge cases that are still important to consider but didn’t happen to be on the list of assumptions.
That’s the attitude I take towards writing code and unit testing it, for better or worse. It’s an internalization of a lot of ideas that I have both learned about from others and worked out for myself. It’s not a prescription for the right way to write code nor is it an admonishment of how anyone else does their work.