DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
View Events Video Library
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Integrating PostgreSQL Databases with ANF: Join this workshop to learn how to create a PostgreSQL server using Instaclustr’s managed service

Mobile Database Essentials: Assess data needs, storage requirements, and more when leveraging databases for cloud and edge applications.

Monitoring and Observability for LLMs: Datadog and Google Cloud discuss how to achieve optimal AI model performance.

Automated Testing: The latest on architecture, TDD, and the benefits of AI and low-code tools.

Related

  • Creating Customized and Bootable Disk Images of Host Systems
  • SeaweedFS vs. JuiceFS Design and Features
  • A Complete Guide to AWS File Handling and How It Is Revolutionizing Cloud Storage
  • Improving Read Performance by ~30% In AI Speech and Text Processing by a Distributed Storage System

Trending

  • How To Validate Archives and Identify Invalid Documents in Java
  • Agile Metrics and KPIs in Action
  • The Convergence of Testing and Observability
  • Spring WebFlux Retries

What is New in RavenDB 3.0: RavenFS

Oren Eini user avatar by
Oren Eini
·
Sep. 16, 14 · Interview
Like (0)
Save
Tweet
Share
4.33K Views

Join the DZone community and get the full member experience.

Join For Free

a frequent request from ravendb users was the ability to store binary data. be that actual documents (pdf, word), images (user’s photo, accident images, medical scans) or very large items (videos, high resolution aerial photos).

ravendb can do that, sort of, with attachments. but attachments were never a first class feature in ravendb.

with ravenfs, files now have first class support. here is a small screen shot, i’ve a detailed description of how it works below.

image

the raven file system exposes a set of files, which are binary data with a specific key. however, unlike a simple key/value store, ravenfs does much more than just store the binary values.

it was designed upfront to handle very large files (multiple gbs) efficiently at api and storage layers level. to the point where it can find common data patterns in distinct files (or even in the same file) and just point to it, instead of storing duplicate information. ravenfs is a replicated and highly available system, updating a file will only send the changes made to the file between the two nodes, not the full file. this lets you update very large files, and only replicate the changes. this works even if you upload the file from scratch, you don’t have to deal with that manually.

files aren’t just binary data. files have metadata associated with them, and that metadata is available for searching. if you want to find all of joe’s photos from may 2014, you can do that easily. the client api was carefully structured to give you full functionality even when sitting in a backend server, you can stream a value from one end of the system to the other without having to do any buffering.

let us see how this works from the client side, shall we?

var filestore = new filesstore()
{
    url = "http://localhost:8080",
    defaultfilesystem = "northwind-assets",
};

using(var filesession = filestore.openasyncsession())
{
    var stream = file.openread("profile.png");
    var metadata = new ravenjobject
    {
        {"user", "users/1345"},
        {"formal": true}
    };
    filesession.registerupload("images/profile.png", stream, metadata);
    await filesession.savechangesasync(); // actually upload the file
}

using(var filesession = filestore.openasyncsession())
{
    var file = await session.query()
                    .whereequals("formal", true)
                    .firstordefaultasync();

    var stream = await session.downloadasync(file.name);

    var file = file.create("profile.png");

    await stream.copytoasync(file);
}

first of all, you start by creating a filestore, similar to ravendb’s documentstore, and then create a session. ravenfs is fully async, and we don’t provide any sync api. the common scenario is using for large files, where blocking operations are simply not going cut it.

now, we upload a file to the server, note that at no point do we need to actually have the file in memory. we open a stream to the file, and register that stream to be uploaded. only when we call savechangesasync will we actually read from that stream and write to the file store. you can also see that we are specifying metadata on the file. later, we are going to be searching on that metadata. the results of the search is a fileheader object, which is useful if you want to show the user a list of matching files. to actually get the contents of the file, you call downloadasync. here, again, we don’t load the entire file to memory, but rather will give you a stream for the contents of the file that you can send to its final destination.

pretty simple, and highly efficient process, overall.

ravenfs also has all the usual facilities you need from a data storage system, including full & incremental backups, full replication and high availability features. and while it has the usual file system folder model, to encourage familiarity, the most common usage is actually as a metadata driven system, where you locate a desired file based searching.


File system

Published at DZone with permission of Oren Eini, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • Creating Customized and Bootable Disk Images of Host Systems
  • SeaweedFS vs. JuiceFS Design and Features
  • A Complete Guide to AWS File Handling and How It Is Revolutionizing Cloud Storage
  • Improving Read Performance by ~30% In AI Speech and Text Processing by a Distributed Storage System

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends: