To be honest, we were very surprised to see this blog from Hazelcast. Why wouldn’t Mr. Luck at least make an effort and reach out to the Ignite community or the GridGain team before making such outrageous accusations? The whole premise that someone would produce an open source benchmark for an open source project and “purposely” fake the results for the whole world to see sounds silly at best.
I should also mention that the Ignite community has made an effort to reach out to Hazelcast on their user group list, publishing the benchmark results together with the code, and asking Hazelcast team to review and provide feedback. Mr. Luck chose to purge the post from the Hazelcast public mailing list with a comment:
We do not think it is appropriate for you to post your perf numbers to our mailing list. We are going to remove your post. Please keep that to your own mailing list and websites
Purging an email from a public mailing list simply because you don’t like the content? This type of authoritative behavior clearly demonstrates the real difference between a commercial company and an Apache project.
Needless to say, after seeing the Hazelcast blog, we reran all the benchmarks and reconfirmed all the results on Amazon AWS with Ignite being over 50% to 100% faster in most of the tests. Here are some of the results:
Deadlock-Free Transactions (Optimistic, Serializable)
Hazelcast does not have support for deadlock-free transactions, so we are comparing Ignite deadlock-free transactions to Hazelcast pessimistic transactions.
These results, as well as many others, are published on the Ignite website here: https://ignite.apache.org/benchmarks/ignite-vs-hazelcast.html
As for Hazelcast’s accusations, here are my comments:
Map.put() -> Map.set()
We have updated the benchmark, but this change made no noticeable difference and did not affect the previous outcome.
Synchronous vs. Asynchronous Backups
The configuration we ran the benchmark with was 100% identical between Apache Ignite and Hazelcast, so Hazelcast statements are simply false. In addition, from Greg’s blog it seems that they ran the benchmarks on their own hardware and for some reason changed the test scenarios, making it absolutely impossible for anyone to verify. Our benchmarks, on the other hand, were run on Amazon AWS, the code is published on GitHub, and anyone can run and reproduce the results.
We take benchmarks very seriously and regularly run benchmarks to compare Ignite and GridGain performance to other products. In our runs, we have over 20 different configuration scenarios and use cases that we test. We reran all the benchmarks today and came out ahead on all of them.