Recursive Indexing in RavenDB

Consider the graph below. I already talked about this graph when I wrote about permission-based graph queries.

In this post, I want to show off another way to deal with the same problem, but without using graph queries and using only the capabilities that we have in RavenDB 4.1.

The idea is that given a user, I want to be able to issue a query for all the issues that this user has access to, either directly (like Sunny in the graph), via a group (like Max, via project-x group) or via a recursive group, like (Nati, via project-x –> team-nati groups).

As you can imagine from the name of this post, this requires recursion. You can read the documentation about this, but I thought to spice things up and use several features all at once.

Let’s look at the following index (Issues/Permissions):

// Issues/Permissions index definition

map("Issues", issue =>{
   var groups = issue.Groups.reduce(recurse_groups, {});
   return  { Groups: Object.keys(groups), Users: issue.Users };
});

function recurse_groups(accumulator, grpId) {
    if(grpId == null || accumulator.hasOwnProperty(grpId))
        return accumulator;
    accumulator[grpId] = null; 
    var grp = load(grpId, "Groups");
    if(grp == null || grp.Parents == null)
        return accumulator;
    return grp.Parents.reduce(recurse_groups, accumulator);
}

This is an JS index, which has a map() function over the Issues collection. For each of the issues, we index the Users for the issue and the groups (recursively) that are allowed to access it.

Here is what the output of this index will be for our issue in the graph:

Now let’s look at how we can query this, shall we?

This query has two clauses; either we are assigned directly or via a group. The key here is in the recurse_groups() and inside that, the load() call in the index. It scans upward through the defined groups and their parents until we have a simple structure in the index that is easily searchable.

RavenDB will ensure that whenever a document that is referenced by a load() in the index is updated, all the documents that are referencing it will be re-indexed. In the case we have here, whenever a group is updated, we’ll re-index all the relevant issues to match the new permissions structure.

One of the core principles of RavenDB is that you can push more work to the indexing and keep your queries fast and simple. This is a good example of how we can arrange the data in such a way that we can push work to background indexing in a pretty elegant manner.