DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones AWS Cloud
by AWS Developer Relations
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones
AWS Cloud
by AWS Developer Relations

Trending

  • Cypress Tutorial: A Comprehensive Guide With Examples and Best Practices
  • Testing Applications With JPA Buddy and Testcontainers
  • How to Handle Secrets in Kubernetes
  • Fun Is the Glue That Makes Everything Stick, Also the OCP
  1. DZone
  2. Data Engineering
  3. Databases
  4. "Biggy: SQLite for Documents and .NET"

"Biggy: SQLite for Documents and .NET"

Greg Duncan user avatar by
Greg Duncan
·
Feb. 28, 14 · Interview
Like (0)
Save
Tweet
Share
4.02K Views

Join the DZone community and get the full member experience.

Join For Free

wekeroad - Hello Biggy

image

A File-based Document Store for .NET

I've been using NeDB (a file-based document store for Node) for a few projects and I utterly love it. Such a simple idea, so fast, so elegant and many times just what I need! I had assumed that such a thing must be around for .NET because there are about 100 different kinds of lists in C#... someone must have made one with a persistent backing store!

But I looked around and couldn't find it, so I made it (as I'll need this in the coming months).

The idea is basically this: I want to use LINQ, I like Dynamics, and I like speed. So that's it, and here's Biggy:

...

Reading and Writing

So, by now you should be wondering why this is useful. The simple answer is that if you have a high-read application (like a blog, CMS, etc) then something like Biggy could speed things up.

Whenever you instantiate a new BiggyList it tries to read it's data from disk - this is good, and it's bad. It's good because from that point on whenever you try to query your data (using LINQ) it's an in-memory operation and you can't get much faster than that.

It's bad because this means you probably want to have a single DB instance around for the life of your app. This might be easy for some, might be repulsive to others. I'm used to doing this kind of thing with Node (all modules in Node are cached which means you always hit the same module instance).

For a blog engine, this might be a very fun thing to have - no database installs, superfast, easy to use. For a Twitter clone... not so much.

...

robconery / biggy

Biggy: SQLite for Documents and .NET

This is just a goofy idea at this point, inspired by NeDB which is basically the same thing, but with Node.

I like the idea of SQLite (a file-based relational data-store), but wouldn't it be fun to have this kind of thing for a Document database too? One nice thing about C# (among many) is the built-in LINQ stuff, another nice thing is that C# has Dynamics now too. Biggy is simply an implementation of ICollection with a JSON backing store. I added a few helpy things in there (like events and a few other things) and this might be completely dumb but I like the idea.

...

What It's Good For

The only disk activity occurs when you call "Save()" and when you instantiate the List itself - everything else happens in memory. This makes Biggy incredibly fast but it also means we're doing file management - which can be tricky.

This is one place that I hope I can get a PR for - I'm dropping the entire contents to disk on every save and YES if you try this will millions of records it will probably cause you some problems. But with 100 or so, it shouldn't be that big of a problem.

That makes Biggy compelling for high-read situations, such as a blog, product catalog, etc. At least that's what I've used NeDB for and it works great.

Performance

In the Tasks project (a Console app) there are simple loops that write 1000 records to disk at once (in a batch) as well as a simple read. You can see the results for yourself... they are OK.

Writing 1000 records in a batch takes about 30ms (give or take), writing in a loop takes about 4 seconds (!), but reading records out is too small to record :):):).

There's a lot to do to make this a bit more functional, but for now it does what I envisioned.

Wanna Help?

..."

Now that's an interesting project to play with... hum... and it looks like of fun too. [i.e. queued for a futureCoding4Fun blog post ;]

Database Document SQLite

Published at DZone with permission of Greg Duncan, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Trending

  • Cypress Tutorial: A Comprehensive Guide With Examples and Best Practices
  • Testing Applications With JPA Buddy and Testcontainers
  • How to Handle Secrets in Kubernetes
  • Fun Is the Glue That Makes Everything Stick, Also the OCP

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com

Let's be friends: