Ceph vs. Swift for OpenStack Object Storage

For a casual outside observer, there’s a lot in common between Ceph and Swift: they are both open source projects, they have both enjoyed major and ongoing increases in the number of developers actively engaged in improving them, they are both mature, and they both have a legion of fans with serious engineering skills and live deployment experience. Each camp extolls the virtues of their preferred approach and acts as cheerleaders encouraging its adoption. Supporting either has to be viewed as a win for the open source community overall.

Ceph – if you can forgive the pun – was out of the blocks first in this two-horse race, launching in 2006. Swift launched two years later in 2008 and has been playing catch-up ever since. Ceph delivers unified storage, supporting File, Block, and Object. Swift is Object only. Ceph is an independent open source project. Swift was originally part of the Open Stack project – though the company that owns it, SwiftStack – is moving it on from this heritage. The general consensus is that Ceph is something of a ‘jack of all trades’, complete with the accompanying inference of ‘master of none’, whereas Swift does one thing well, but one thing only – giving it the polar opposite of inferences – that of the ‘one trick pony’ – SwiftStack is working on file-based services, they haven’t arrived yet. So, when it comes to the specialty of Swift, surely the choice is obvious. It's the Object specialist and part of OpenStack, and therefore the best choice when looking at this configuration, right?

Well no, not really. It's not that simple. There are two strong reasons to prefer Ceph to Swift – reasons which those legions of fans (on both sides) overlook because they have pretty much nothing to do with engineering virtues and everything to do with human behavior, the efficient use of skilled engineering resources, and support contract cost management in the enterprise. Before I get to that, let’s take a shallowish dive into the major differences – just for the sake of form.

In Favor of Ceph

• The obvious point of File, Block, and Object in the same wrapper. It might be an obvious point, but it’s a pretty damn important one.
• Better transfer speed and lower latency – because traffic to and from the Swift cluster goes through proxy servers, which slow it down.
• Swift can have further latency problems, as replicas are not necessarily updated at the same time, so requesters retrieving data can access old – wrong/outdated – versions.

Against Ceph

• Ceph’s multi-region support — usually touted as an advantage — is in a master-slave configuration, but as replication is only possible from master to slave, in a deployment with 2+ regions, you can get uneven load distribution. Not a problem in Swift.
• In Ceph, you should only write to the master... but there is nothing to stop you from writing to the slave, which can mean poor execution, resulting in inconsistencies and, in extreme circumstances, complete corruption. Not a problem in Swift.
• There can also be a security issue, as RADOS clients on the cloud compute node communicate directly with the RADOS servers over the same network Ceph uses for unencrypted replication traffic. So, potentially, if Ceph client node is compromised, the attacker can see all traffic on the storage network. Not a problem in Swift.

The security problem is a bit of a straw man, as best practice demands a separate network, and in any case, I’m knit picking the problems – working hard to find the cons.

But, really, none of these pros and cons are relevant. And in any case, as both approaches can work alongside each other comfortably, should you be making an ‘either/or' choice in the first place? Well, as I said earlier, there are two concrete reasons why Ceph is the winning approach.

Running Open Source Technology Requires Skilled Engineers

Anybody in the proprietary camp will tell you that the money you save by avoiding software costs can come back in additional engineering skills costs: paying for the support contracts or skilled headcount required, and keeping that skilled headcount up to speed with developments comes at a cost. Just how many different skill sets can you actually master? When there are two different ways of doing an open source approach, smart enterprises will adopt the tech that makes this headache as small as possible.

APIs

Because Swift is busy working on proprietary APIs that not only differ from Ceph, but also from Amazon Simple Storage System, it can potentially lead to widespread resistance to ‘yet another storage interface’.

In reality, the choice is simple, albeit uncomfortable for enterprises and individuals who have invested a lot of time and resource into getting good at Swift. Ceph is a Swiss army knife, complete with the Swiss army knife’s array of potential use cases: corkscrew, screwdriver, saw, bottle opener, even a needle. Meanwhile, Swift is a really great pen knife. Who cares if the blade is sharper? When you’re in the shop getting ready for the camping trip, who even checks?

Who can rationally choose the lower number of use cases? Don’t ask the fans – the support of fans is simply not rational.