Over a million developers have joined DZone.

Deploying MongoDB for High Availability

DZone's Guide to

Deploying MongoDB for High Availability

· Database Zone
Free Resource

Learn NoSQL for free with hands-on sample code, example queries, tutorials, and more.  Brought to you in partnership with Couchbase.

This was a live blog for the recent MongoSV conference.

Going to go over HA best practices: keeping data online and safe.

What about a single node? This will have downtime. If the node crashes, intervention might be necessary. If it disappears will need a backup.

Replica set v1: Single datacenter, single switch, single power source. But automatically recovers from a single node crash. A good start but not great.

The next step up is still single datacenter, but w/ multiple power/network zones. Like EC2 in a single region but multiple AZs. Still some points of failure (datacenter / two node failure). With an arbiter, we can’t do w=2 writes and remain up. With 3 non-arbiters we can use w=2 but are still vulnerable to datacenter failure.

The next step up is multi datacenter w/ a single DR (disaster recovery) node in a different DC. We can’t always stay up, but we at least have a DR option now.

Now let’s look at the ideal: three datacenters, five nodes. One has a single delayed slave (that helps recover from fat-finger incidents like accidental db.drop). The other two DCs each have 2 active nodes. We can lose an entire DC and still have a majority w/ the other two DCs. Can do `w={dc: 2}` to guarantee write in 2 DCs.

Moving on to HA sharding

Each shard needs to be a replica set - same rules apply as above. Balancing can be run in a window (this is cool!) can set an activeWindow to only run the balancer at night. Sweet!

Config servers need to be on at least 2 different power/network zones. Ideally just put in three separate DCs. Use host names rather than IP addresses: much easier to move a config server. Take backups of config servers. Important note (saw this earlier too in Richard’s talk): not a replica set. To bring new nodes online you need to manually move the data. Not a problem if a config server is down for a day or something: just won’t do splits/migrates.

Run one mongos per app-server. Don’t need to worry about scaling mongos. Saves a network hop for many ops. If you really don’t like this, run a pool per power region with a load-balancer in front.

Application-level tips:

Handle spikes: queue non-synchronous writes, isolate components and features. Can your site handle going into a read-only mode? That helps a lot when dealing w/ issues.

Monitor! Load, disk, CPU, but most importantly I/O (iostat). Alerts go hand in hand w/ monitoring.Have good procedures for backups, adding replica set members, adding shards, etc. Practice (in staging) (which ought to be the same as prod). Randomly shut down boxes & load test as much as possible.


That’s all for today. Another great event by 10gen; I think my favorite talks of the day were Kyle’s and Richard’s, but all of them were great. Hope you enjoyed the blog posts!

Source: http://blog.fiesta.cc/post/13989346153/deploying-mongodb-for-high-availability-mongosv

The Getting Started with NoSQL Guide will get you hands-on with NoSQL in minutes with no coding needed. Brought to you in partnership with Couchbase.


Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}