Neo4j Spatial, Part 1: Building a Recommendation Engine

one of my new year resolutions is to do a project with neo4j spatial , so we’ll kick off my first blog post of the year with a gentle introduction to this awesome plugin. i advise you to watch this very short 15 minute video by neo4j spatial creator craig taverner . the man is a genius level developer, you’ll gain iq points just listening, i swear.

the plan is to make a restaurant recommendation engine based on things you care about and your current location. yes, this is baby level stuff, but we’ll start with this and see where else neo4j spatial can take us later on.

so the first thing i always do when starting a new project is look for data. asking google, the all knowing and all powerful, i ran into…

factual has a massive amount of data accessible by api and specifically some restaurant data that i was interested in. let’s take a look at it shall we:

{
  "accessible_wheelchair": true,
  "address": "5951 n broadway st",
  "alcohol": true,
  "alcohol_bar": true,
  "alcohol_beer_wine": true,
  "alcohol_byob": true,
  "attire": "casual",
  "category_ids": [
    366
  ],
  "category_labels": [
    [
      "social",
      "food and dining",
      "restaurants",
      "sushi"
    ]
  ],
  "country": "us",
  "cuisine": [
    "cafe",
    "thai",
    "sushi",
    "asian",
    "japanese"
  ],
  "email": "indieincindiecafe@gmail.com",
  "factual_id": "8f4b9dff-d1e2-47d5-b761-1e63928f79ac",
  "groups_goodfor": true,
  "hours": "{\"monday\":[[\"15:00\",\"22:00\",\"dinner\"]],\"tuesday\":[[\"17:00\",\"22:00\",\"dinner\"]],\"wednesday\":[[\"15:00\",\"22:00\",\"dinner\"],[\"12:00\",\"15:00\"]],\"thursday\":[[\"15:00\",\"22:00\",\"dinner\"],[\"12:00\",\"15:00\"]],\"friday\":[[\"15:00\",\"22:30\",\"dinner\"],[\"12:00\",\"15:00\"]],\"saturday\":[[\"15:00\",\"22:30\",\"dinner\"],[\"12:00\",\"15:00\"]],\"sunday\":[[\"15:00\",\"22:00\",\"dinner\"],[\"12:00\",\"15:00\"]]}",
  "hours_display": "mon 3:00 pm-10:00 pm; tue 5:00 pm-10:00 pm; wed-thu 12:00 pm-10:00 pm; fri-sat 12:00 pm-10:30 pm; sun 12:00 pm-10:00 pm",
  "kids_goodfor": true,
  "latitude": 41.990326,
  "locality": "chicago",
  "longitude": -87.672907,
  "meal_cater": true,
  "meal_deliver": true,
  "meal_dinner": true,
  "meal_lunch": true,
  "meal_takeout": true,
  "name": "indie cafe",
  "neighborhood": [
    "edgewater",
    "far north side",
    "northside",
    "north edgewater"
  ],
  "open_24hrs": false,
  "options_healthy": true,
  "options_vegan": true,
  "options_vegetarian": true,
  "parking": true,
  "parking_street": true,
  "payment_cashonly": false,
  "postcode": "60660",
  "price": 2,
  "rating": 5,
  "region": "il",
  "reservations": true,
  "seating_outdoor": true,
  "smoking": false,
  "status": "1",
  "tel": "(773) 561-5577",
  "website": "http://www.indiecafe.us"
}

look at all that beautiful data. we know their hours, whether or not they have healthy, vegan and vegetarian options. if they are a good restaurant for kids or large groups, cuisines, rating, price… as well as latitude and longitude. we also know if they take reservations, but we have no easy way to just make a reservation (other than calling them and talking to a real human being, but who wants to do that). i know how to fix this…

i’ve used opentable before, and after spending some time poking around their site looking for an api i was unable to find one. not to fear, dan sosedoff was kind enough to build an unofficial opentable api for us.

what kind of data can we get here:

{
  "id": 68710,
  "name": "indie cafe",
  "address": "5951 north broadway",
  "city": "chicago",
  "state": "il",
  "area": "chicago / illinois",
  "postal_code": "60660",
  "country": "us",
  "phone": "7735615111",
  "reserve_url": "http://www.opentable.com/single.aspx?rid=68710",
  "mobile_reserve_url": "http://mobile.opentable.com/opentable/?restid=68710"
}

awesome, there is that magic reservation link i was looking for. now if only i had menus and menu items…

ah ha! found some, but they seem to have discontinued their data api in a recent enterprise pivot. no matter, i’ll e-mail them and ask for a sample of their data. with any luck we’ll have more data than we know what to do with.

now, back to neo4j spatial. the easiest thing we can do here, is create a spatial index and add the restaurants to it, then we’ll be able to query it. now, us ruby developers are a lazy lot, so instead of messing around with maven and building the plugin from source, i tweeked neography so that installing neo4j with the spatial plugin looks like this:

echo "require 'neography/tasks'" > rakefile
rake neo4j:install
rake neo4j:get_spatial
rake neo4j:start

easy right? now let’s try creating a restaurants spatial index and adding a restaurant to it:

@neo = neography::rest.new
@neo.create_spatial_index("restaurants")

the response we get back from the server is:

{ "template" => "http://localhost:7474/db/data/index/node/restaurants/{key}/{value}",
  "provider" => "spatial",
  "geometry_type" => "point",
  "lat" => "lat",
  "lon" => "lon" }

but what actually happened?

it created 4 nodes for us. a special 0th node called “spatial root”, node 1 which we’ll take a look at in a second, and the root of an r-tree and another node that holds the r-tree metadata. let’s take a closer look at node 1:

take a look at that name, “restaurants”… that’s right, we’re not using lucene or some other kind of external index engine, we are building the index inside of our own graph! alright, let’s add that restaurant.

node = @neo.create_node({:name => "indie cafe", :lat => 41.990326, :lon => -87.672907 })

it created node 4, not connected to anything else, just kind of hanging around…

so let’s now add this node to our spatial index:

@neo.add_node_to_spatial_index("restaurants", node)

an additional node 5 was created, with an “id” property that points back to our restaurant “node 4″.

to make sure this is working right, let’s try querying our index. we’ll execute a cypher query that looks to the node spatial index “restaurants” for anything that appears within 10.0 kilometers of latitude 41.99, longitude -87.67.

@neo.execute_query("start n = node:restaurants({location}) return n",
                   {:location => "withindistance:[41.99,-87.67,10.0]"})

… and we find it:

...{"data"=>{"lon"=>-87.672907, "lat"=>41.990326, "name"=>"indie cafe"}...

excellent, now let’s try adding a second restaurant.

node2 = @neo.create_node({:name => "la ciudad", :lat => 41.964239, :lon => -87.654758 })
@neo.add_node_to_spatial_index("restaurants", node2)

our r-tree gets another branch, and our tree root properties change to encompass both branch nodes:

we’ll query our graph one more time:

@neo.execute_query("start n = node:restaurants({location}) return n",
                   {:location => "withindistance:[41.99,-87.67,10.0]"})

… and we see we get back both restaurants.

...{"data"=>{"lon"=>-87.672907, "lat"=>41.990326, "name"=>"indie cafe"}}...
...{"data"=>{"lon"=>-87.654758, "lat"=>41.964239, "name"=>"la ciudad"}}...

so now you have a good handle on how the simplest part of the neo4j plugin works. on the next blog post, we’ll use what we have learned here to build the full application, and explore the more intricate parts of the neo4j spatial plugin in future blog posts.