Hamcrest vs. AssertJ

During RESTful API testing, it’s not uncommon that you would need to call a REST endpoint and validate its response to check its functionality. But how you can validate if something is working correctly or not? The answer is — assertions. Assertions are the cornerstone of the complete testing process, whether RESTful APIs or other components. When we say “testing assertions”, we mean checking that a boolean expression is true, unless there is an issue or bug with the tested use case.

In RESTful API testing, you might deal with API specific assertions like response codes or length validation. In addition, there will always be plenty of situations where you would need to perform general assertions on a daily basis even if you are only working with APIs. These could be variable validation, dates comparison, and so on.

In this article, we are going to talk about two of the most popular Java assertion frameworks available with free access, which can be used in any of your projects. They can be used as part of your RESTful API testing, but also in any another check which you need to perform in your tests. The names of these frameworks are AssertJ and Hamcrest, and we will compare their open-source engagement, syntax, class structure and functionality.

Hamcrest is one of the most well-known Java frameworks, which helps writing tests in the Java programming language. An interesting fact about the name of the framework: It was created as an anagram to the word “matchers”, which are logically equal to “assertions”. Due to the high popularity of the framework, it was later ported to other programming languages like C++, C#, Objective-C, Python, ActionScript 3, PHP, JavaScript, Erlang, and R.

AssertJ is not as well-known as Hamcrest, but at the same time, its popularity has been growing pretty fast over the last few years. As opposed to Hamcrest’s classic assertion syntax, which was inherited from the default Java testing framework JUnit, the main idea of AssertJ is that it provides fluent syntax. The main goal of that is to improve code readability. It’s worth mentioning that AssertJ is a fork of the FEST Assert project, which was the first step of AssertJ creation.

Now let’s get started with our comparison.

Open Source Community Engagement and General Popularity

It’s always a good sign when you choose a framework that has an active community around it. The main sign of an active community is activity in the framework repository, which basically means ongoing development and maintenance. If a framework is open source, you can always find its code repository and verify how often it has new code.

hamcrest/JavaHamcrest

joel-costigliola/assertj

As you can see, AssertJ has had a more stable development pattern during the last years, and there are still commits on a monthly basis.

Now let’s take a look at the frameworks’ general popularity. Let's check out what Google Trends shows us over the past year.

As you can see, Hamcrest definitely has a wider audience. But on the other hand, it seems that there is growth (albeit small) in AssertJ’s popularity pattern over the last 12 months. Hamcrest, on the other hand, stays on the same level.

You might wonder why Hamcrest has gained wider popularity at this point. This is probably because Hamcrest has a much longer life cycle. The first version of Hamcrest was presented in the Maven repository as early as May 2007, while the first version of AssertJ was added to the Maven repository only in March 2013. So it might be fair to compare trends over the past 5 years:

Now you can see the reason for the difference. When the first version of AssertJ was developed, Hamcrest already had a large crowd and was the main assertion library, in addition to default tools provided by commonly used test frameworks. But why then would someone want to create one more framework when there is already a list of available options to achieve the same goals? That’s exactly what we are going to investigate.

Syntax

While both tools do their jobs quite well, the syntax difference is quite noticeable. Let’s go over the main syntax of both frameworks, based on some basic assertion examples:

Simple Assertions

assertThat(a, equalTo(b)); //Hamcrest
assertThat(a).isEqualTo(b); //AssertJ

Dates Assertions

assertThat(tomorrow, isAfter(today)); //Hamcrest
assertThat(tomorrow).isAfter(today); //AssertJ

List Assertions

assertThat(list, Matchers.<Collection<String>> allOf(CoreMatchers.hasItem("a"),
       CoreMatchers.not(CoreMatchers.hasItem("b"))
)); //Hamcrest 

assertThat(list).        
       contains("a").
       doesNotContain("b"); //AssertJ

Null Assertions

assertThat(a, nullValue()); //Hamcrest
assertThat(actual).isNull();  //AssertJ

Assertions With a Custom Message

assertThat("Error", a, equalTo(b)); //Hamcrest
assertThat(a).isEqualTo(b).overridingErrorMessage("Error"); //AssertJ

It’s up to you which syntax you like most, but for me, AssertJ feels more natural and easy to read.

Assertion Class Structure

In addition to the syntax difference, these frameworks have a different class hierarchy. While AssertJ provides only one static class that contains all available assertions, Hamcrest has different classes. If your file contains many assertions of different types, all your classes will be full of such imports:

import static org.hamcrest.CoreMatchers.equalTo;
import static org.hamcrest.Matchers.empty;
import static org.hamcrest.CoreMatchers.not;
import static org.hamcrest.Matchers.hasSize;
import static org.hamcrest.CoreMatchers.nullValue;
import static org.hamcrest.MatcherAssert.assertThat;

It doesn’t take too much effort from you to maintain all these imports, as all main development environments handle imports automatically. But it is still nice and simple when your code is cleaner, like in the case of AssertJ:

import static org.assertj.core.api.Assertions.*;

Due to the same reason, AssertJ provides more convenient autosuggestions, because you don’t need to memorize and guess which class matches you need to use to find the required assertion:

Soft Assertion Functionality

While both frameworks provide wonderful functionality to perform classic assertions, only AssertJ provides you with something extra. This “something” is the concept of soft assertions.

Soft assertions allow you to perform multiple checks and see results for all of them. For unit tests, there is a good pattern to keep one assertion for each unit test. But do not forget that the same assertion library can be used in different integration tests where multiple assertions are the common practice. AssertJ soft assertions can be used this way:

@Test
public void soft_assertion_assertj_test(){
   User user = new User("Yuri", "Bushnev", "bushnevyuri@gmail.com");

   SoftAssertions softly = new SoftAssertions();
   softly.assertThat(user).isNotNull();
   softly.assertThat(user.getName()).isEqualTo("Yuri");
   softly.assertThat(user.getSurename()).isEqualTo("");
   softly.assertThat(user.getEmail()).isEqualTo("bushnevyurigmail.com");
   softly.assertAll();
}

As you can see, it’s not too complicated. In the example above, we are expecting two assertion errors during the validation of the surname and the email. A standard way would require having three assertions one by one. Then, we would only see the first failed assertion about the surname while the issue in the email would be hidden until the surname would be fixed. Such issues are often reproducible in real life when regression testing complicated applications — when some bugs are caught quickly during the first test run, but many other bugs can be missed and found only later, because tests can not go further until the failed step has been fixed. But with soft assertions, you would see something like this:

org.assertj.core.api.SoftAssertionError: 
The following 2 assertions failed:

1) expected:<"[]"> but was:<"[Bushnev]">
at AssertJTests.soft_assertion_assertj_test(AssertJTests.java:16) expected:<"[]"> but was:<"[Bushnev]">
2) expected:<"bushnevyuri[]gmail.com"> but was:<"bushnevyuri[@]gmail.com">
at AssertJTests.soft_assertion_assertj_test(AssertJTests.java:17) expected:<"bushnevyuri[]gmail.com"> but was:<"bushnevyuri[@]gmail.com">

One more interesting feature provided by AssertJ is the migrator tool, which helps you to migrate from JMeter or other frameworks to AssertJ. It is definitely a useful feature if you decided to migrate all your assertions. But just keep in mind that there is no critical need to migrate from one assertion library to another. Take your time, do it step by step, and make sure that you do not produce any regression during refactoring.

Conclusion

The purpose of this article is not to urge you to use an exact library or compel a certain way of conducting tests. Rather, the purpose is to show you the most popular available Java assertion frameworks, if you want to extend the capabilities of standard assertions.

The most important thing is that you choose a tool you are comfortable with. Keep in mind that when writing good assertions, analytical skills are more helpful than tools. That being said, it’s very useful to take a couple of hours and try something new from time to time. From my point of view, AssertJ is definitely worth a look if you haven’t done so already.

If you are looking to learn more, you can look at more assertion libraries. Each programming language and every RESTful API verification tool has its own assertion libraries, helping users implement different kinds of assertions in an easy way.

These default and inbuilt libraries are a very good starting point since they have tons of documentation that also covers their history. This is the most stable way to go since these libraries have been used for many years, and the issues and bugs were already found and fixed a long time ago. But as soon as you gain more experience, you might feel that there might is something else in addition to default frameworks. After you start digging, you can find plenty of available solutions available for any kind of need.