DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
View Events Video Library
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Integrating PostgreSQL Databases with ANF: Join this workshop to learn how to create a PostgreSQL server using Instaclustr’s managed service

Mobile Database Essentials: Assess data needs, storage requirements, and more when leveraging databases for cloud and edge applications.

Monitoring and Observability for LLMs: Datadog and Google Cloud discuss how to achieve optimal AI model performance.

Automated Testing: The latest on architecture, TDD, and the benefits of AI and low-code tools.

Related

  • A Guide to Data-Driven Design and Architecture
  • A Data-Driven Approach to Application Modernization
  • How To Design Reliable IIoT Architecture
  • Design Patterns for Microservices: Ambassador, Anti-Corruption Layer, and Backends for Frontends

Trending

  • LTS JDK 21 Features
  • Exploring Edge Computing: Delving Into Amazon and Facebook Use Cases
  • A Better Web3 Experience: Account Abstraction From Flow (Part 2)
  • Causes and Remedies of Poison Pill in Apache Kafka
  1. DZone
  2. Data Engineering
  3. AI/ML
  4. Architecture and Design Guidance: Make the Machine Do Your Work for You

Architecture and Design Guidance: Make the Machine Do Your Work for You

Oren Eini user avatar by
Oren Eini
·
Aug. 14, 13 · Interview
Like (0)
Save
Tweet
Share
4.87K Views

Join the DZone community and get the full member experience.

Join For Free

i am currently playing around with persistent b+ trees. this is done using memory-mapped files and pointer arithmetic in c#. i consider this a non-trivial exercise, and certainly quite outside of my usual haunts. obviously, as you can imagine, there are bugs. those can be pretty hard to figure out because they can arise from any number of issues (usually it is me, but still …).

after fighting for a few hours with a set of bugs that just wouldn’t let me be, i decided that i had enough of that and set out to design the debug hooks i need to actually figure it out. because i am dealing with pointers, i couldn’t just use the normal methods that i would use when debugging c# code. to top that, i think that most of you would agree that explaining a data structure ( any data structure) without a whiteboard is nigh impossible. in the same manner, trying to figure out a bug in a b+ tree by following the member references was too hard. i wanted to actually visualize it.

i could throw it into a treeview in a few minutes, but i wanted to take the time to do it right. i have a feeling that this is going to be useful for the future as well, so i learned how to use graphviz’s dot language and wrote the code to dump the structure of the b+tree into a properly formatted dot file, after which i could ask dot.exe to turn that into an image. the piece of code that i was having trouble with was the page split function.

here is the before snapshot. note that we have nearly run out of room in this page.

output

and here is the output of the after snapshot. as you can see, this really makes no sense at all. we have a page split, but it all went into the same page?

output

what actually happened is that i had a bug where i wired both the left and right pages in the split to the original page. once that was fixed (and once i actually saw the data), it was very easy to see what was wrong. it was a matter of changing a single parameter. after the fix, i could visually see that everything was fine.

output

leaving aside the details about the actual bug and fix, the lesson that i wish to impart today is that building those sort of capabilities into your systems is important. now that i have this ability, i fully expect to be start using it when i need to debug the next issue.

in fact, i ran into the next issue quite quickly. after inserting quite a lot of data, i messed up the pointers and returned what looked very much like corrupted data. it was pretty hard to isolate (as usual with pointer issues) and, since the actual error occurred at some point during the process, i wasn’t really clear on where and how to figure it out. i ended up just dumping the tree content into a file after every operation, and then checking the operations until i found the one where we started to have corruption.

that allowed me to figure out that the corruption began at step 98 and iteration four.

image

as it turned out, that was pretty hard to figure out because i didn’t realize that when i was splitting the page i needed to compact the data on the original page. too much thinking in list<t> and not enough thinking about the layout of the problem in memory.



Design Machine Architecture

Published at DZone with permission of Oren Eini, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • A Guide to Data-Driven Design and Architecture
  • A Data-Driven Approach to Application Modernization
  • How To Design Reliable IIoT Architecture
  • Design Patterns for Microservices: Ambassador, Anti-Corruption Layer, and Backends for Frontends

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends: