Couchbase NoSQL at Tunewiki: A Billion Documents and Counting

The other day I had a chance to catch up with Chad Kouse, CTO and Co-Founder of TuneWiki, an awesome app built using Couchbase. TuneWiki is revolutionizing the social music experience giving music lovers the opportunity to share their favorite music, lyrics and photos with friends. It is the #1 app on Spotify with the world’s largest database of scrolling lyrics, providing an unsurpassed karaoke style listening experience.

With over a billion data objects and growing, TuneWiki uses Couchbase Server to serve lyrics to millions of users around the world. I wanted to share in this blog my conversation with Chad, as he explains why the TuneWiki team went with Couchbase Server, what they’re doing with Couchbase, and why he’s confident about the capabilities of Couchbase Server to support TuneWiki’s growing user base.

Chad, can you share more details about how your application uses Couchbase?
At TuneWiki, we use Couchbase for caching and storing data. We ran across Couchbase when we were looking for a replacement for memcached. If a memcached node suddenly failed or if we were adding a new memcached node into the cluster - the entire app would slowdown. MySQL just wasn’t able to keep up. There was also extra effort needed when we wanted to expire items from the cache and to keep MySQL in sync with the caching tier.

With Couchbase, we didn’t need to have two data stores -  just a single one. Couchbase was easy to setup, use and manage. Because the Couchbase API was compatible with memcached, we could leave the memcached code as is and we only had to replace all the MySQL calls with Couchbase. Couchbase gave us predictable performance we needed for our app. We also never had to worry about growing the cluster. Adding more nodes always gave us the performance boost we needed without diminishing returns. Using Couchbase Server simplified our backend architecture to a great extend.

What exactly does your app store in Couchbase?
We basically store everything our app needs in Couchbase. This includes lyrics, timelines, API keys to quickly validate user actions and OAuth tokens. Tunewiki has social elements like sharing photos and lyrics with friends. If you’re using this feature, the items you share are stored in Couchbase.

Tell me about your app workload.
We currently have over 1 billion objects stored in Couchbase with the system running at over 10K ops / sec. We have sized our cluster to keep around 80% of our total data in RAM. For example, we use counters to monitor API usage and throttle clients based on these counters. With a relational database, as you can imagine, this is not something you can do easily. With Couchbase, this counter gets shared across our entire API cluster of servers automatically without the need to write any extra application code. We use the atomic increment API’s to increment the counter in memory to track hits - it is lightweight because it is all in memory and you never have to go to disk. We don’t have any slowdowns and there’s never a bottleneck.

What advice do you have for other Couchbase users?
Couchbase is a great option even if you're starting out with a small project. You can experiment with just 1 or 2 nodes during development and when you’re ready to go into production, you can easily provision more nodes in your cluster to scale. We also got excellent support from Couchbase and I recommend users to checkout the Couchbase Google groups and other material that’s on the Couchbase site.

Any next steps? Are there new features or new apps you are planning to build with Couchbase Server 2.0?
Views in Couchbase Server 2.0 is a cool feature. We plan to use views in Couchbase Server for general housekeeping of our data and to improve app functionality. With a key-value store, you couldn’t support predicates on your data - you had to know the keys that you needed. This would involve scanning across your data to figure out what keys were relevant and what were not. With a billion objects in Couchbase, you can imagine how tedious that would be.

For example, if there are a bunch of OAuth keys that are not needed any more, it would be nice to use views to get a list of them so that they can be cleaned up. Well without views you could potentially set an expiration time on them but if you forgot to put a value for the expiration time, these keys would live forever.

Where can folks learn more about how you have used Couchbase?
Couchbase users can checkout our blog to learn more.