Writing Compile Time Tests
How to avoid using JSON paths when testing API responses to save time.
Join the DZone community and get the full member experience.Join For Free
Wouldn’t it be great if your tests could find bugs before they were even run? Here is how you can do just that when validating data-type issues in REST responses. It takes just a small change, but when done thoughtfully, it can be a useful tool in your testing tool belt.
First, we will see an example of tests with the “normal” way to setup JSON assertions in API tests to show how they could be improved. Then, we will rewrite our tests to find the same bug, but the test will fail faster, letting us know at compile-time that something is wrong without having to even run the tests!
We will be using Kotlin for this because it allows for some nice coding style, but this technique should work with any compile-time type-checked language. You can write your code to do this, or you can use the same code as this article. The JAR for this code is open source and available so you can pull it in with Maven or Gradle. You can find setup instructions and the source code here if you are interested.
Background for the Tests
Suppose we are testing a REST API. That API probably has a response model object. That is, there should be some object that your API serializes into a response for your API under test. In this example, we have Pet.kt from the Petstore example OpenAPI spec. It is setup to be serialized by Kotlinx serialization, but this technique works with other serializers as well.
In our example, this object would be serialized and returned as a response by the API under test. We need to test this API call by checking the response format and content.
Writing Our Initial Test
Now, let’s set up a reasonable test in REST-assured. Pretend that we know one of our customers is using a long for the ID. We can test to make sure our API handles long Ids with this test. Note that 2,147,483,648 is one more than the max integer.
A simple test, validating a real customer use case. Now, fast-forward six months or a year later, to the next big feature. By this time, everyone has probably forgotten about this test and the reason for using longs for ids. Imagine that, to be consistent with another API, we have decided to change from Longs to Integers for IDs. Luckily, this is exactly what this test is supposed to save us from.
The change is made to our model object:
We then run our API test and find out that it fails! You will probably see an error like this from your code Unexpected JSON token: Failed to parse 'int'. Cool, we caught the issue! But we can catch it faster…
Make Our Test Fail Faster
It would be great if we did not have to wait for the tests to run before we noticed this error. At this point the development could be close to, if not already, complete. In some test environments, this type of test does not even run until after the code has been deployed to a shared development environment.
Finding issues before they get that far along, saves time, money, and headaches.
Check out this updated test. I will explain after what is going on and how to do this with only REST-assured if required.
The ‘checkBool’ block is running those assertions on the deserialized Pet object using a Kotlin receiver. Since the Pet object we are using is the same model object that generated the response for the API under test, the JSON schema should always match. Keep that in mind for later.
Now, imagine we are in the same situation as before. We are changing our Long ID to an int. How does that look with our new test format?
A compile error! We did not even have to run the tests to be alerted that something was off! Since ‘id’ is part of the Pet object, it is type checked. Changing the production Pet object, invalidates the test immediately, so we no long need to wait for the test to run. This is clearly a faster way to fail than relying on the untyped JSON path assertion.
You can do the same thing with just REST-assured if you would like. DockMatcher checkBool is essentially just a Kotlin DSL to help make the following code more readable. The important part is that the number we assert against is compared to the production model object.
What if I Don’t Want it to Break?
While this compile-time checking is useful, what if I want the test to behave like the JSON path matching? Going in the reverse direction, from int to long, tests generally will not care about the type change. Validating that type change is just extra maintenance. Luckily, we keep the ability to ignore type with this new testing style as well.
Even though we can use compile time checking, we do not have to. If we know that we will not need to check integer vs long, we can write the test in such a way that it will not break when we switch types.
This test will allow for the switch from integer to long. Our assertion library, in this case AssertJ, is smart enough to work with int or longs, but would still fail at compile-time if an incompatible type, like string, was used in our model object.
What’s the Catch?
Like most things in life, there is a downside to this approach as compared to the Json path or non-typed approached. Luckily though, it is not hard to avoid if you understand it.
The downside comes from re-using the production model response object in the test. If your tests are too strict, you will be faced with errors of inconsequential changes, like the switch from int to long above.
This just means you should think carefully about what fields you need to type check, and which you do not. In the first example, there is a reason I said, ‘pretend that we know one of our customers is using a long for the ID’. If we know a customer needs a long, it makes sense to check the type.
It might not make sense if all our customers actually use integer IDs. This is not as much a problem, as it is a new consideration to think about when writing tests. Having better testability, means you have more decisions to make about the tests you write.
Left shifting should not just mean automating tests. In the above examples I’ve shown how you can left shift to even before the tests run in some cases. These cases are narrow, only checking that data types match, but finding any error with the compiler has the potential to save time and headaches.
The key is to try to use your strictly typed production code in tests where possible. There will be situations where reusing production code invalidates your tests, but simple data classes without much logic will often be safe to use in tests.
Hopefully, you have some ideas about how you might be able to implement this in your tests. The examples above all came from my own implementation. You can either use that or write your own version from those examples.
Opinions expressed by DZone contributors are their own.