Reviewing LevelDB: Part XI, Reading from Sort String Tables via the TableCache

in the previous post, i focused mostly on reading the code for writing a sst to disk. but i have to admit that i am not really following how you read them back in a way that would be easy to read.

in order to understand that, i think that the right place in the code would be the tablecache. the api for that is pretty slick, here are just the header (comments were stripped).

class tablecache {
 public:
  tablecache(const std::string& dbname, const options* options, int entries);
  ~tablecache();

  iterator* newiterator(const readoptions& options,
                        uint64_t file_number,
                        uint64_t file_size,
                        table** tableptr = null);

  status get(const readoptions& options,
             uint64_t file_number,
             uint64_t file_size,
             const slice& k,
             void* arg,
             void (*handle_result)(void*, const slice&, const slice&));

  void evict(uint64_t file_number);

 private:

  status findtable(uint64_t file_number, uint64_t file_size, cache::handle**);
};

there are a couple of interesting things here that show up right away. how do you know what is the number of entries. i guess that this is stored externally, but i am not sure where. i am going to figure that question out first.

and the answer is strange:

it isn't number of entries in the table , it is the number of files? that actually say something very important, since this means that the table cache is reading multiple sst files, rather than just one per cache. looking at the rest of the api, it makes even more sense. we need to pass the file number that we are going to be reading. that one is going to be got from the version we are using.

side note: i just spend an hour or so setting up a tryout project for leveldb, so i can actually execute the code and see what it does. this had me learning cmake (i am using kdevelop to read the code) and i finally got it. still haven't figure out how to step into leveldb 's code. but that is a good step.

also, the debug experience is about 2/10 compared to vs.

and damn, i am so not a c++ developer any longer. then again, never was a real dev on linux, anyway.

the first thing the tablecache does is to setup a cache. this is interesting, and i followed the code, it create 16 internal caches and has between them. i think that this is done to get concurrency because all the internal cache methods looks like this:

note the mutex lock. that is pretty much how it works for all of the cache work. looking deeper into the code, i can see another reason why i should be grateful for staying away from c++. leveldb comes with its own hash table functionality. at any rate, the cache it using lru to evict items, and it get the number of entries from the value that was passed in the ctor. that make me think that it hold the opened files.

speaking of the cache, here is an example of the code using it:

the cache is also used to do block caching, this is why it takes a slice as an argument. i'm going to go over that later, because this looks quite interesting. the rest of this method looks like this:

so the only very interesting bit is the table::open. the rest is just opening the mmap file and storing it in the cache. note that the actual file size is passed externally. i am not really sure what this means yet. i'll go over the table code later, right now i want to focus on the table cache.

looking over the tablecache interface, we can see that we always get the file size from outside. that got me curious enough to figure out why. and the answer is that we always have it in the filemetadata that we previously explored. i am not sure why that is so important, but i'll ignore it for now.

the rest of tablecache is pretty simple, although this made me grin:

more specifically, look at the registercleanup, this is basically registering for the delete event, so they can unref the cache. pretty nice, all around.

the rest of the code is just forwarding calls to the table, so that is what i'll be reading next...