Over a million developers have joined DZone.

Solving Code Deployment Scalability Problems with… Murder

DZone 's Guide to

Solving Code Deployment Scalability Problems with… Murder

· Integration Zone ·
Free Resource
Twitter recently open sourced some morbidly-named software that solves problems that growing websites are faced with when they need to deploy code changes to a growing application.  Named " Murder" after a group of crows, Twitter's newly available code is a Python-Ruby hybrid that facilitates rapid deployments across many nodes.

Sometimes explosive growth of a web property comes when you least expect it.  If you don't have the tools to handle large flows of data and the changes required in your software, you'll lose some of that growth while you figure things out (or get murdere d by the users :) ).  Twitter understands these challenges too well.  

As the company's server farm grows, it gets tougher and tougher to roll changes across each and every server.  Thanks to Murder, the changes that once took 40 minutes to deploy now take only 12 seconds.  Twitter developer Larry Gadea recently gave a talk on how the company came up with Murder:

Twitter - Murder Bittorrent Deploy System from Larry Gadea on Vimeo.

Murder actually uses a technology that file sharers know quite well: BitTorrent.  Murder is built on a more advanced version of BitTorrent called BitTornado.  Twitter took this technology and built features on top of it that would allow it to function smoothly in a datacenter.  Murder was developer to include high bandwidth, low-latency server access, no NAT/Firewall issues, no ISP traffic shaping, only trusted peers, and more.  

However, Murder didn't do it all on its own.  It was paired with another app, Capistrano, to bring the time of deployment way down.  They were eventually able to reduce code deployment times by 99.5%!

Gadea encourages other websites with a decent server farm to play with Murder and see how they can make it work for their system.  He believes that it's very important for young companies to think about scalability before it becomes an issue.

Pick up Murder at GitHub to give it a whirl.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}