Learn how you can maximize big data in the cloud with Apache Hadoop. Download this eBook now. Brought to you in partnership with Hortonworks.
Today, I thought a lot about how to examine different databases. Choosing a database is often a daunting task. There's a lot of confusion, a 'theorem', and more than all, the immortal proverb 'not one size fits all'. As if it helps.
One of the first things that you realize, when examining NoSQL distributed databases (and how could you not)is that these days databases are like cars: they're all good. Old fashioned SQL databases can scale in and out, horizontally sharded over several machines to achieve high availability. NoSQL systems claim to be consistent. What difference then does it make what database would you choose?
The Availability and Consistency that I mentioned comes, of course, from the misunderstood CAP theorem
, that - so people say - states that you can only choose 2 out of the 3
- Consistency: every read would get you the most recent write
- Availability: every node (if not failed) always executes queries
- Partition-tolerance: even if the connections between nodes are down, the other two (A & C) promises, are kept.
Usually its depicted in a nicely equilaterl triangle, as this one fromOfirm
There's a nice proof and explanation of it in this 4 minute video here
. But if we think about it, and also see some of Brewer's (the theorem author) later remarks
, we'll see that the 2 out of 3 is really 1 out of 2:
It's really just A vs C!
And this is simply because:
- Availability is achieved by replicating the data across different machines
- Consistency is achieved by updating several nodes before allowing further reads
- Total partitioning, meaning failure of part of the system is rare. However, we could look at a delay, a latency, of the update between nodes, as a temporary partitioning. It will then cause a temporary decision between A and C:
- On systems that allow reads before updating all the nodes, we will get high Availability
- On systems that lock all the nodes before allowing reads, we will get Consistency
That's it! And since this decision is temporary, it exists only for the duration of the delay, some may say
that we are really contrasting Latency (another word for Availability) against Consistency.
By the way, there's no distributed system that wants to live with "Paritioning" - if it does, it's not distributed. That is why putting SQL in this triangle may lead to confusion.
Hortonworks DataFlow is an integrated platform that makes data ingestion fast, easy, and secure. Download the white paper now. Brought to you in partnership with Hortonworks.