DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
View Events Video Library
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Integrating PostgreSQL Databases with ANF: Join this workshop to learn how to create a PostgreSQL server using Instaclustr’s managed service

Mobile Database Essentials: Assess data needs, storage requirements, and more when leveraging databases for cloud and edge applications.

Monitoring and Observability for LLMs: Datadog and Google Cloud discuss how to achieve optimal AI model performance.

Automated Testing: The latest on architecture, TDD, and the benefits of AI and low-code tools.

Related

  • Java Module Benefits With Example
  • JQueue: A Library to Implement the Outbox Pattern
  • A Camunda Greenhouse: Part 3
  • OpenTelemetry Moves Past the Three Pillars

Trending

  • How TIBCO Is Evolving Its Platform To Embrace Developers and Simplify Cloud Integration
  • Running Unit Tests in GitHub Actions
  • AWS Amplify: A Comprehensive Guide
  • How To Simplify Multi-Cluster Istio Service Mesh Using Admiral
  1. DZone
  2. Data Engineering
  3. Databases
  4. Reviewing Lightning Memory-Mapped Database Library: Partial

Reviewing Lightning Memory-Mapped Database Library: Partial

Oren Eini user avatar by
Oren Eini
·
Jul. 12, 13 · Interview
Like (0)
Save
Tweet
Share
4.95K Views

Join the DZone community and get the full member experience.

Join For Free

continuing in my quest to learn more, i decided to go over the lmdb codebase .

lmdb is:

lmdb is an ultra-fast, ultra-compact key-value data store developed by symas for the openldap project. it uses memory-mapped files, so it has the read performance of a pure in-memory database while still offering the persistence of standard disk-based databases.

it has some interesting feature set, and a really small codebase. i am anxious to see how they managed to do so much.

interestingly, the data model for lmdb is quite different from the usual append-only / transaction log. instead, it allows only a single concurrent writer, and modify the data in place. there also appears to be a lot of dire warnings regarding usage of long transactions, since they would result in increased file size, presumably because the db couldn’t find the pages the scavenge because they are locked by ongoing transactions.

one thing that i should note already, the code (i am currently about 1/3 of the way of lmdb.h file) is very well commented, and it explains a lot about what is going on. if the rest of the code is like that, this is going to be really nice to read. okay, i take it back. the main files seems to be lmdb.h and mdb.c. the first one is 1,300 lines and the second is over 7,500 lines. admittedly, this is impressive in the sense that pretty much everything is done there, and there are a lot of docs. but damn , i wish this was better organized. right now my head feels like i need to pop up and take a breath.

i read ~2,000 lines of code so far, and i haven’t found anything that does something. it is all headers or comments or macros up until now. i skipped over to the code, and it is really hard to understand what is going on.

take this code snippet:

image

it shows a lot of the things that make it hard to work with. effectively random naming convention (under_score, pascal_naming, shout_naming, etc), the goto trick is used way too much, lovely variable names such as n2. this is part of a method that goes on for something like 150 – 200 lines. and it include the following code:

image

quick, can you tell me how many variables are declared here? and note that they are all local variables. i counted it twice, once getting to 17 and once getting 18.

that is just too much, i am not going to go any deeper. the leveldb codebase was easy to follow, it had structure . this codebase is just a code dump. it might be a really good codebase, for what it needs to do, but i literally can’t follow it.

Database Lightning Memory-Mapped Database Lightning (software) Library

Published at DZone with permission of Oren Eini, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • Java Module Benefits With Example
  • JQueue: A Library to Implement the Outbox Pattern
  • A Camunda Greenhouse: Part 3
  • OpenTelemetry Moves Past the Three Pillars

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends: