The Cost of Getting Data from LevelDB

we are currently investigating the usage of leveldb as a storage engine in ravendb. some of the things that we feel very strongly about is transactions (leveldb doesn’t have it) and performance (for a different definition of the one usually bandied about).

leveldb does have atomicity, and the rest of cid can be built atop of that without too much complexity (already done, in fact). but we run into an issue when looking at the performance of reading. i am not sure if that is unique or not, but in our scenario, we typically deal with relatively large values. documents of several mb are quite common. that means that we are pretty sensitive to memory allocations. it doesn’t help that we have very little control on the large object heap, so it was with great interest that we looked at how leveldb did things.

reading the actual code make a lot of sense (more on that later, i will probably go through a big review of that). but there was one story that really didn’t make any sense to us, reading a value by key.

we started out using leveldb sharp :

database.get("users/1");

this in turn result in the following getting called:

a few things to note here. all from the point of view of someone who deals with very large values.

• valueptr is not released, even though it was allocated by us.
• we copy the value from valueptr into a string, resulting in two copies of the data and twice the memory usage.
• there is no way to get just partial data.
• there is no way to get binary data (for example, encrypted)
• this is going to be putting a lot of pressure on the large object heap.

but wait, it actually gets better. let us look at the leveldb method that get called:

so we are actually copying the data multiple times now. for fun, the db->rep->get() call also copy the data. and that is pretty much where we stopped looking.

we are actually going to need to write a new c api and export that to be able to make use of that in our c# code. fun, or not.