When one is better than two: Collapsing data management layers for scalability and simplicity

Rick Grehan recently posted an in-depth product review of Couchbase and MongoDB. I was struck in particular by one of Rick’s observations:

“Couchbase’s full incorporation of Memcached has no counterpart in MongoDB, and Memcached is a powerful adjunct as general object caching system for high-throughput, data-intensive Internet and intranet applications. If your application needs a Memcache server with your database, then look no further than Couchbase.” (italics mine)

In computing, there is an interesting phenomenon that occurs over a period of time. We build layers of abstraction and components as layers – over time, these layers grow and become somewhat unwieldy until someone comes along and shows us that the layers can be designed away. Either intermediate layers are eliminated or different layers are consolidated, essentially providing an order of magnitude improvement in performance, simplicity and management.

For example, originally, many hardware chips started as discrete components (adders, multipliers, etc.) and over time, single chips consolidated many of these functions and interestingly, such chips are called Integrated Circuits (ICs) because they combine or “integrate” these components into a single consolidated chip.

Similarly, over time, we have seen how a small inexpensive box consolidates a router, a broadband modem, firewall, DHCP server – all components that existed as discrete software and hardware systems at some point in time. Or, interestingly, how PDAs and phones have consolidated – both the layers of software and the containers.

I see this same phenomenon emerging in the NoSQL database market. NoSQL databases are widely used to build distributed internet applications dealing with unstructured data and support a very large set of users. As we engage with companies considering NoSQL, we find that their topologies tend to front end the data store with a high-performance scalable distributed caching tier (such as memcached or Oracle Coherence) to deal with scaling a large number of users and a large data set. That’s two sets of software and hardware, and all the complexity of managing two disparate systems.

This scenario is ripe for layer consolidation – which is what Couchbase does. With Couchbase handling both tiers, users can simplify their application environments for greater scalability, efficiency and manageability. It’s the case that one is better than two.