What is New in RavenDB 3.0: RavenFS

a frequent request from ravendb users was the ability to store binary data. be that actual documents (pdf, word), images (user’s photo, accident images, medical scans) or very large items (videos, high resolution aerial photos).

ravendb can do that, sort of, with attachments. but attachments were never a first class feature in ravendb.

with ravenfs, files now have first class support. here is a small screen shot, i’ve a detailed description of how it works below.

the raven file system exposes a set of files, which are binary data with a specific key. however, unlike a simple key/value store, ravenfs does much more than just store the binary values.

it was designed upfront to handle very large files (multiple gbs) efficiently at api and storage layers level. to the point where it can find common data patterns in distinct files (or even in the same file) and just point to it, instead of storing duplicate information. ravenfs is a replicated and highly available system, updating a file will only send the changes made to the file between the two nodes, not the full file. this lets you update very large files, and only replicate the changes. this works even if you upload the file from scratch, you don’t have to deal with that manually.

files aren’t just binary data. files have metadata associated with them, and that metadata is available for searching. if you want to find all of joe’s photos from may 2014, you can do that easily. the client api was carefully structured to give you full functionality even when sitting in a backend server, you can stream a value from one end of the system to the other without having to do any buffering.

let us see how this works from the client side, shall we?

var filestore = new filesstore()
{
    url = "http://localhost:8080",
    defaultfilesystem = "northwind-assets",
};

using(var filesession = filestore.openasyncsession())
{
    var stream = file.openread("profile.png");
    var metadata = new ravenjobject
    {
        {"user", "users/1345"},
        {"formal": true}
    };
    filesession.registerupload("images/profile.png", stream, metadata);
    await filesession.savechangesasync(); // actually upload the file
}

using(var filesession = filestore.openasyncsession())
{
    var file = await session.query()
                    .whereequals("formal", true)
                    .firstordefaultasync();

    var stream = await session.downloadasync(file.name);

    var file = file.create("profile.png");

    await stream.copytoasync(file);
}

first of all, you start by creating a filestore, similar to ravendb’s documentstore, and then create a session. ravenfs is fully async, and we don’t provide any sync api. the common scenario is using for large files, where blocking operations are simply not going cut it.

now, we upload a file to the server, note that at no point do we need to actually have the file in memory. we open a stream to the file, and register that stream to be uploaded. only when we call savechangesasync will we actually read from that stream and write to the file store. you can also see that we are specifying metadata on the file. later, we are going to be searching on that metadata. the results of the search is a fileheader object, which is useful if you want to show the user a list of matching files. to actually get the contents of the file, you call downloadasync. here, again, we don’t load the entire file to memory, but rather will give you a stream for the contents of the file that you can send to its final destination.

pretty simple, and highly efficient process, overall.

ravenfs also has all the usual facilities you need from a data storage system, including full & incremental backups, full replication and high availability features. and while it has the usual file system folder model, to encourage familiarity, the most common usage is actually as a metadata driven system, where you locate a desired file based searching.