Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Our Own Multi-Model Database (Part 1)

DZone's Guide to

Our Own Multi-Model Database (Part 1)

Multi-model databases are easy to get wrong, so how about doing one right? Follow along with this series and merge a KV store, a graph database, and an object DB.

· Database Zone
Free Resource

Check out the IT Market Clock report for recommendations on how to consolidate and replace legacy databases. Brought to you in partnership with MariaDB.

shittydb

I may be remembering this wrong, but I think it was Henry Rollins who once asked, “What came first, the sh*tty multi-model databases or the drugs?” His confusion was over whether:

  • Option A: There were a bunch of developers dicking around with their Mac laptops and they wrote a sh*tty database, put it on GitHub, posted on Hacker News, and then other developers who were on drugs started using it or…

  • Option B: There were a bunch of developers on ketamine and ecstasy and somebody said, "Let's write a sh*tty database."

I think “A” is what probably happens and how we end up with over 300 databases on DB-Engines. But what about “B?” Well, I don’t have any good stuff lying around, but I did hurt my foot the other day and the doctors gave me some Tramadol, so let's down some of that and see what happens.

Alright, most multi-model databases start out from some key-value backing. There are plenty of choices to choose from, but I’m gonna go embedded first (an homage to Neo4j, which started life as embedded only) so I’ll use ChronicleMap.

chronicle-map-diagram_04

Chronicle Map is built by the Open HFT folks (Roman Leventov, Rob Austin, Peter Lawrey, and others) and is known to be pretty damn fast, so we’ll use that.

Now, I’ll need to handle Keys and their values, but those values can be Objects, and those Objects can be connected together. Yes, our multi-model database will be a KV store, object database, and graph database rolled into one sh*tty mess. Let’s call this thing ChronicleGraph and set up our storage.

public class ChronicleGraph {

    private static ChronicleMap<String, Object> nodes;


So we have this “nodes” map that accepts any string as a key and stores an object. Now onto our addNode method, we need to accept both empty nodes and nodes with properties, but we’ll force the users to give us a primary key of some sort. This will make getting the node back easy.

    public boolean addNode (String key) {
        return addNode(key,"");
    }

    public boolean addNode (String key, Object properties) {
        nodes.put(key, properties);
        return true;
    }


Let’s test this out… does this pass?

    @Test
    public void shouldAddNode() {
        boolean created = cg.addNode("key");
        Assert.assertTrue(created);
        Assert.assertEquals("", cg.getNode("key"));
    }


Yes… what about with complex properties?

    @Test
    public void shouldAddNodeWithObjectProperties() {
        HashMap<String, Object> address = new HashMap<>();
        address.put("Country", "USA");
        address.put("Zip", "60601");
        address.put("State", "TX");
        address.put("City", "Chicago");
        address.put("Line1 ", "175 N. Harbor Dr.");
        HashMap<String, Object> properties = new HashMap<>();
        properties.put("name", "max");
        properties.put("email", "maxdemarzi@hotmail.com");
        properties.put("address", address);
        boolean created = cg.addNode("complex", properties);
        Assert.assertTrue(created);
        Assert.assertEquals(properties, cg.getNode("complex"));
    }


Yes, no problem. OK, so that was the easy part. How do we join these guys together in relationships, though? First, let’s add another map for “relationships,” which will store the relationship properties if any. We want to be able to store relationship properties because, without those, we’d be reduced to the most useless of all NoSQL databases — the dreaded “RDFs.” Ewwwww.

    private static ChronicleMap<String, Object> relationships;
    private static HashMap<String, ChronicleMap<String, Set<String>>> related = new HashMap<>();


We are also adding a map of maps called “related” which will store our relationships by type and direction. To add a new relationship, we ask the user for the Type and the keys of the two nodes we’re connecting. If the related map doesn’t know about this relationship type, then we add the type before trying to create the relationship. This addRelationshipType will create two ChronicleMaps one for the Incoming and one for the Outgoing direction and our addEdge method is called twice to match each direction.

    public boolean addRelationship (String type, String from, String to) {
        if(!related.containsKey(type+"-out")) {
            addRelationshipType(type, DEFAULT_MAXIMUM_RELATIONSHIPS, 
                                DEFAULT_OUTGOING, DEFAULT_INCOMING);
        }
        addEdge(related.get(type + "-out"), from, to);
        addEdge(related.get(type + "-in"), to, from);

        return true;
    }


This means, of course, that when we delete a node, we have to delete it from the nodes map as well as from both its incoming and outgoing relationships and delete any relationship properties it may have had from both directions. That method was painful, so I’ll spare you the details, but take a look at it here if you want.

Now, what good would this database be if it could not traverse? Let’s add a new method to get the nodes linked by an outgoing relationship (and we’ll do the same for incoming) by first getting the node ids in the type+”out” ChronicleMap of “related” and then using “nodes” to get their properties:

    public Set<Object> getOutgoingRelationshipNodes(String type, String from) {
        Set<Object> results = new HashSet<>();
        for (String key : related.get(type+"-out").get(from) ) {
            HashMap<String, Object> properties = new HashMap<>();
            properties.put("_id", key);
            properties.put("properties", nodes.get(key));
            results.add(properties);
        }
        return results;
    }


… and now for a crazy test:

 @Test
    public void shouldGetNodeOutgoingRelationshipNodes() {
        cg.addNode("one", 1);
        cg.addNode("two", "node two");

        HashMap<String, Object> node3props = new HashMap<> ();
        node3props.put("property1", 3);
        cg.addNode("three", node3props);

        cg.addRelationshipType("FRIENDS", 10000, 100, 100);
        cg.addRelationship("FRIENDS", "one", "two");
        cg.addRelationship("FRIENDS", "one", "three");
        Set<Object> actual = cg.getOutgoingRelationshipNodes("FRIENDS", "one");

        Set<Object> expected = new HashSet<Object>() {{
            add( new HashMap<String, Object>() {{
                put("_id", "two");
                put("properties", "node two");
            }});
            add( new HashMap<String, Object>() {{
                put("_id", "three");
                put("properties", node3props);
            }});
        }};
        Assert.assertEquals(expected, actual);
    }


Beautiful, isn’t it? The Tramadol is wearing off, so we’ll stop here. Take a look at the source code on GitHub as always. I’ll try to continue this later maybe by adding a web server and an API to go with it. In the meantime, enjoy the sh*tty music and the drugs.

Interested in reducing database costs by moving from Oracle Enterprise to open source subscription?  Read the total cost of ownership (TCO) analysis. Brought to you in partnership with MariaDB.

Topics:
database ,multi-model databases ,key-value stores ,graph databases ,object database ,tutorial

Published at DZone with permission of Max De Marzi, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}