Building an OpenStack Swift to WebDAV Proxy

When I first started using OpenStack Swift, the object storage service within OpenStack, I immediately wanted the ability to mount it as a drive on my system. At the time I was working on the HP public cloud and wanted to mount a public cloud storage as a local drive. While not a Dropbox alternative, this would let me have a rather large remote drive and there are many ways to use this.

The first attempt to do this, back in early 2012, didn't go so well. I used some existing open source projects and tried to plug swift in. Where it worked to read small files and a small number of files it failed when files got larger or you wanted to upload files.

From that attempt, Matt Butcher tried a second attempt built on SabreDAV, a PHP library. While it was able to read and write files without an issue there were problems with larger files. This isn't a case of building a way to mount a server via WebDAV but to build a pass through proxy to swift. The issue wasn't in SabreDAV or in PHP. Rather, the issue is in the way PHP sits behind a web server and how web servers pass files to PHP. It was a fundamental technology issue that we explored working around.

The third attempt was far more successful. Matt used Node.js to build the WedDAV to swift pass-through proxy. The pass-through element was important as files don't reside on the proxy. Rather, the information passes through and is simply translated from WebDAV to swift and vice versa. The server hosting the application doesn't need to have a large file store to cache the files locally. This was enabled by the low level access to files and information within Node.js.

The project was a success in many ways and we'd hoped to make it available in the HP public cloud. Unfortunately, that didn't happen for a number of reasons and the codebase was shelved for some time. Now that code is available on GitHub under an open source license.

So, if you're interested in mounting OpenStack Swift as a drive or you'd like to interface with swift via WebDAV checkout the Swift WebDAV Proxy.

Note, this is the first in a series talking about this project. More details coming soon.