DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones AWS Cloud
by AWS Developer Relations
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones
AWS Cloud
by AWS Developer Relations
Securing Your Software Supply Chain with JFrog and Azure
Register Today

Trending

  • What to Pay Attention to as Automation Upends the Developer Experience
  • DevOps vs. DevSecOps: The Debate
  • Deploying Smart Contract on Ethereum Blockchain
  • Getting Started With the YugabyteDB Managed REST API

Trending

  • What to Pay Attention to as Automation Upends the Developer Experience
  • DevOps vs. DevSecOps: The Debate
  • Deploying Smart Contract on Ethereum Blockchain
  • Getting Started With the YugabyteDB Managed REST API

The Design of RavenDB 4.0: Over the Wire Protocol

We have a look at the protocol used via RavenDB 4.0 as it communicates over-the-wire. For this database, HTTP is the only way!

Oren Eini user avatar by
Oren Eini
·
May. 02, 16 · Analysis
Like (1)
Save
Tweet
Share
2.45K Views

Join the DZone community and get the full member experience.

Join For Free

So the first thing to tackle is the over the wire protocol. RavenDB is a REST based system, working purely within HTTP. A lot of that was because at the time of conception, REST was pretty much the thing, so it was natural to go ahead with that. For many reasons, that has been the right decision.

The ability to easily debug the over the wire protocol with a widely available tool like Fiddler makes it very easy to support in production, script it using curl or wget, and in general make it easier to understand what is going on.

On the other hand, there are a bunch of things that we messed up on. In particular, RavenDB is using the HTTP headers to pass the document metadata. At the time, that seemed like an elegant decision and something that is easily in line with how REST is supposed to work. In practice, this limited the kind of metadata that we can use to stuff that can pass through HTTP headers, and forced some constraints on us (case insensitivity, for example), and there have been several cases where proxies of various kind inserted their own metadata that ended up in RavenDB, sometimes resulting in bad things happening.

When looking at RavenDB 4.0, we made the following decisions:

  • We are still going to use HTTP as the primary communication mechanism.
  • Unless we have a good reason to avoid it, we are going to be human readable over the wire.

So now, instead of sending document metadata as HTTP headers, we send them inside the document and use headers only to control HTTP itself. That said, we have taken the time to analyze our protocol and found multiple places where we can do much better. In particular, the existence of web sockets means that there are certain scenarios that just became tremendously simpler for us. RavenDB has several needs for bidirectional communications. Bulk inserts, Change API, subscriptions, etc.

The fact that web sockets are now pretty much available across the board means that we have a much easier time dealing with those scenarios. In fact, the presence of web sockets, in general, is going to have a major impact on how we are doing replication, but that will be the topic of another post.

Beyond the raw over the wire protocol, we also looked at a common source for bugs, issues, and trouble: IIS. Hosting under IIS means that you are playing by IIS rules, which can be really hard to do if you aren’t a classic web application. In particular, IIS has certain ideas about size of requests, their duration, the time you have to shut down or start up, etc. In certain configurations, IIS will just decide to puke on you (if you have a long URL, it will return 404, with no indication why, for example), resulting in escalated support calls. One particular support call happened because a database took too long to open (as a result of a rude shutdown by IIS), which resulted in IIS aborting the thread and hanging the entire server because of an abandoned mutex. Fun stuff like that.

In RavenDB 4.0, we are going to be using Kestrel exclusively, which simplify our life considerably. You can still front it with IIS, of course (or ngnix, etc), for operational, logging, etc. reasons. But this means that RavenDB will be running its own process, worry about its own initialization, shutdown, etc. It makes our life much easier.

That is about it for the protocol bits, in the next post, I’ll talk about the most important aspect of a database, how the data actually gets stored on disk.   

More posts in "The design of RavenDB 4.0" series:

  1. (12 May 2016) Separation of indexes and documents
  2. (10 May 2016) Voron has a one track mind
  3. (05 May 2016) Physically segregating collections
  4. (03 May 2016) Making Lucene reliable
  5. (28 Apr 2016) The implications of the blittable format
  6. (26 Apr 2016) Voron takes flight
  7. (22 Apr 2016) Over the wire protocol
  8. (20 Apr 2016) We already got stuff out there
  9. (18 Apr 2016) The general idea
Wire protocol Protocol (object-oriented programming) Design

Published at DZone with permission of Oren Eini, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Trending

  • What to Pay Attention to as Automation Upends the Developer Experience
  • DevOps vs. DevSecOps: The Debate
  • Deploying Smart Contract on Ethereum Blockchain
  • Getting Started With the YugabyteDB Managed REST API

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com

Let's be friends: