Lessons learned from JsonUnit

I'd like to share some lessons I have learned when working on JsonUnit. I started the project almost three years ago, when I needed something to compare JSON structures in unit tests. Today, I am glad I started the project - it's ideal for learning. On one hand it's small enough so I can come back after several months and still remember what's going on. On the other hand it's big enough to not be boring. Moreover, from time to time I get a feature request that gives me an opportunity to play.

Think twice before adding new assert* method

It all started with one static method assertJsonEquals(expected, actual). But new methods followed soon - assertJsonPartEquals(expected, actual, path) for comparing only part of the document, assertJsonStructureEquals for comparing only structure while ignoring values and obviously assertJsonPartStructureEquals combining the two methods together.

So far so good. Until you realize that it would be nice to have negative variants. Something like  assertPartNotEquals. So you just create four new methods with Not in the name and suddenly you have eight methods. Each new feature just multiplies the number of methods.

And what about parameter types? Shall we support String, Reader or InputStream for expected and actual values? The number of possible combinations is staggering.

It's easy to solve the parameter type problem. Just use good old Object and deal with the types inside the library. Luckily, we do not need type safety when comparing JSON document. Actually, the opposite is true, we may want to compare a number with a string like this

assertJsonPartEquals(
	2, 
	"{\"test\":[{\"value\":1},{\"value\":2}]}", 
	"test[1].value"
);

But what to do about the method name combinatoric explosion? It took me quite a long time to realize that it can be solved by parameters too. So instead of assertStructureEquals being an extra method, it can be just a parametrization of assertJsonEquals.

Use objects for parameters

It's quite tempting to add a new boolean parameter for such options. For example

assertJsonEquals(Object expected, Object actual, boolean structureOnly)

But it would not help much. Any additional configuration parameter would again make the API explode due to number of combinations. There is a better approach. We can use objects and do something like

assertJsonEquals(Object expected, Object actual, 
			Configuration configuration)

Now it's possible to keep the number of methods small and just extend the Configuration class. Additionally, if you provide a nice factory method, the code reads quite naturally

assertJsonEquals(
		"[{\"test\":1}, {\"test\":2}]", 
		"[{\n\"test\": 1\n}, {\"test\": 4}]", 
		when(COMPARING_ONLY_STRUCTURE)
);

Hamcrest matchers are great for composition

The problem with complexity explosion is caused by the fact that it is not possible to combine standard static assert methods. But if you use Hamcrest matchers, you can do that. You can even combine standard matchers with your own like this

assertThat(
	asList("{\"test\":1}"), 
	not(contains(jsonEquals("{\"test\":2}")))
);

Methods not(), contains() and jsonEquals() are each from different JAR file but they play well together.

Moreover, jsonEquals is basically a factory method that creates an instance. It gives you a possibility to extends the matcher and do stuff like this.

assertThat("{\"test\":1.00001}", 
	jsonEquals("{\"test\":1}").withTolerance(0.001));

Basically it's the same as the Configuration class above, but this time implemented on the matcher itself.

Java generics are crazy

Do you understand Java generics? If you think you do, tell me which of the following standard Hamcrest methods is correct

<T> Matcher<java.lang.Iterable<? extends T>> 
		contains(Matcher<? super T> itemMatcher)

<T> Matcher<java.lang.Iterable<? super T>> 
		hasItem(Matcher<? super T> itemMatcher)

Both of them accept a matcher and create another matcher which can be used on an Iterable. But one return type contains <? extends T> while the other <? super T>. PECS rule says that producer extends and consumer implies super, Matcher is a consumer, so hasItem is correct. Or not? Does PECS rule apply here? I will not tell you the solution, but it's described here.

The unfortunate result of this difference is that the first line of the following code compiles while the second one does not. And I did not manage to make them compile both

// compiles
assertThat(asList("{\"test\":1}"), contains(jsonEquals("{\"test\":1}")));

// does not compile
assertThat(asList("{\"test\":1}"), hasItem(jsonEquals("{\"test\":1}")));

Fluent assertions are great

While Hamcrest matchers are great for composition, my preferred style of assertions is the fluent one. If you are using AssertJ , you know what I am talking about. In the context of JSON assertions it can look like this

assertThatJson("{\"test\":[1,2,3]}").node("test").isEqualTo(new int[]{1, 2, 3});

I love this style as a a user. I only have to remember to write assertThatJson and the rest is hinted and auto-completed by the IDE. No more remembering whether equality matcher is created by equalTo(), eq() or is(). No more fighting with those pesky static imports.

What's more, I like this approach as library author too. There is only one static method and the rest is just normal object-oriented programming. It really easy to work with. I wonder why it's not the default choice for assertions.

If it hurts, you are doing something wrong

Java is a talkative language and sometime it even stands in your way. But more often than not, it is just a hint that you are doing something wrong. The language is trying to tell you something. It happened to me when I have started to use an EnumSet for managing options. The trouble with Java sets is their mutability. If you want to maintain immutability, you have to write three lines just to add a single element.

EnumSet<Option> optionsWith = EnumSet.copyOf(options); // copy the original set
optionsWith.add(option); // add option
options = optionsWith; // replace the original set with a new value

You have to repeat these lines in every place you want to add an option. It hurts. At least until you realize that it's not a good idea to use plain EnumSet. We have an object-oriented language, so let's create an object and hide the implementation details inside

public class Options {
    private final EnumSet<Option> options;
 
    ...
    public Options with(Option option) {
        EnumSet<Option> optionsWith = EnumSet.copyOf(options);
        optionsWith.add(option);
        return new Options(optionsWith);
    }
}

The three ugly lines are still there. But only once. Not only I have hidden them, but it forced me to introduce a new object and to realize that there are other methods that belong in the Options class. Now the code is more readable and maintainable. If EnumSet was easier to use, I might have never realized, I am supposed to introduce an object.

Feature test coverage is the king

Since I am working on a testing library, it should not be a surprise that I have unit tests. I do not know my test coverage but I think that I have 100% feature coverage. In other words, there is a test for every feature I am aware of. And it's great. I am not afraid to do a radical refactoring and if the tests pass I am quite confident I have not broken anything. But it's hard to get used to it. If the tests pass after large refactoring I just do not believe my eyes. I foolishly run them again just to see them pass again. And sometimes I even feel a bit disappointed. I just expect to enjoy long bug hunting and the damn code just works. Where's the fun?

Pet projects are great

I want to finish by last lesson I have learned. It's great to have a pet project. At work I have all those deadlines and pressure so it's not easy to learn. Usually I just end-up using the first solution that works. When working on JsonUnit I feel that I am still learning. Most of the requested features are surprising and require rethinking the project again and again in new and interesting ways. And that's what I like about programming.