Solid NoSQL Benchmarks from YCSB with a Side of HBase Bologna
Cassandra's numbers are most impressive, but Netflix didn't perform a side-by-side comparison of the available NoSQL platforms. Admirably, Yahoo! Cloud Serving Benchmark (YCSB) endeavored to perform just such a comparison. The results of that effort were recently published by Network World.
It's not surprising that Cassandra and HBase lead the pack in many of the metrics since both are based on Google's BigTable. It was surprising however to see HBase's latency near zero. This warranted some digging.
Now, side-by-side comparisons are always tough because they often depend highly on system configuration and the specifics of the use case / data model used. And in NetworkWorld's article, there is a key paragraph:
"During updates, HBase and Cassandra went far ahead from the main group with the average response latency time not exceeding two milliseconds. HBase was even faster. HBase client was configured with AutoFlush turned off. The updates aggregated in the client buffer and pending writes flushed asynchronously, as soon as the buffer became full. To accelerate updates processing on the server, the deferred log flush was enabled and WAL edits were kept in memory during the flush period"
With Autoflush disabled, writes are buffered until flush is called. See:
If I'm reading that correctly, with autoflush disabled, the durability of a put operation is not guaranteed until the flush occurs. This really sets up an unfair comparison with the other systems where durability is guaranteed on each write. When buffering the data locally, nothing is sent over the wire, which naturally results in near-zero latency!
The change required to YCSB to level the playing field can be seen here: