Over a million developers have joined DZone.

The Neo4j-Slack Integration You’ve Been Waiting For (Is Here)

Build a Slack and Neo4j integration to demonstrate how useful a graph database backend would be. And, of course, how much fun.

· Database Zone

Sign up for the Couchbase Community Newsletter to stay ahead of the curve on the latest NoSQL news, events, and webinars. Brought to you in partnership with Coucbase.

Our colleague Andreas, who loves Slack and brought it into our company, suggested the other day that we could build a Slack and Neo4j integration to demonstrate how useful a graph database backend would be.

And, of course, how much fun.

Read All about Our Neo4j-Slack Integration That Generates Recommended Channels to Other Users

In the process of making the integration, we came up with some surprisingly useful benefits. More below.

Building Blocks

As it was only midnight we, the amazing Nicole and Michael, decided to put something together.

We set up a Neo4j 2.2.3 instance in the cloud.

Then, we created a Python app for our Slack-Neo4j server and pushed it to GitHub.

The application uses web.py for the webapp, requests to access the Slack APIs and py2neo to talk to Neo4j.

Next we pushed it to Heroku to make it available publicly so that Slack could connect to it.

You have to provide environment variables to your Neo4j server, your Slack API token and the team token configured with your slash command. Find the details in the project readme.

Slack Slash Command

With all that complete, we could then set up a slash command. For our integration, the ideas came from Andreas:

  • /graph import – import users, channels and membership into Neo4j
  • /graph cypher MATCH ... RETURN – execute read only cypher statement and return the results
  • /graph – provide an overview of the data that’s in the database

Implementing the app was straightforward. Parsing the POST payload and checking the team token, we then got the first word of the text parameter as “command” to dispatch on.

Getting Data from Slack into Neo4j

For the integration with Neo4j we sent Cypher statements to Neo4j using py2neo’s APIs. Example below:

from py2neo import Graph
graph = Graph(os.environ.get('NEO4J_URL'))

graph.cypher.execute("MATCH (u:User)-[:MEMBEROF]->(c:Channel) return u.screenname, c.name")


Sending requests to the Slack API with the token and getting the JSON response is straightforward with requests. We then passed the JSON response directly as parameters to a Cypher statement to create the graph structure in Neo4j.

res = requests.get("https://slack.com/api/channels.list?token={}".format(token))

query = """
UNWIND {channels} AS channel
MERGE (c:Channel {id:channel.id}) ON CREATE SET c.name = channel.name
"""
graph.cypher.execute_one(query, res.json())


As you can see, we can import users, channels and memberships, easy peasy.

/graph import channels

slackbot: 115 users uploaded. Only you can see this message

/graph import users

slackbot: 117 channels uploaded Only you can see this message

Graph All the Slack Things

And to show you that it worked, here is a graph of our Slack universe:

And here are some queries that show the most prolific people:

/graph cypher match (u:User)-->() return u, count(*) as memberships order by memberships desc limit 3`

slackbot:
   | u                                                            | memberships
---+--------------------------------------------------------------+-------------
1 | (n200:User {fullname:"Michael",id:"U02HVJ36",username:"mh"})  |          64
2 | (n151:User {fullname:"Chris",id:"U0KLMP5X",username:"cl"})    |          37
3 | (n210:User {fullname:"Philip",id:"U02HDEF0EX",username:"pr"}) |          37


Recommendations

Finally, the biggest surprise of all: We wanted to recommend new channels to people.

We used traditional collaborative filtering for this concept of “channels of your colleagues that are not yet your channels”. But we also filter out prolific users and channels so that they don’t distort the picture.

/graph cypher
MATCH (c:Channel) with toInt(count(*)*0.618) as channel_cutoff
MATCH (u:User) with toInt(count(*)*0.618) as user_cutoff, channel_cutoff

MATCH (u:User {username:"laeg"})-[:MEMBER_OF]->(c:Channel)
      <-[:MEMBER_OF]-(coll:User)-[:MEMBER_OF]->(reco:Channel)

WHERE size((c)<--())    < channel_cutoff
  AND size((reco)<--()) < channel_cutoff
  AND size((coll)-->()) < user_cutoff
  AND NOT (u)-[:MEMBER_OF]->(reco)

RETURN reco.name, count(*) AS freq
ORDER BY freq DESC
LIMIT 5;



slackbot:
  | reco.name           | freq
--+---------------------+------
1 | feedback            |  218
2 | dev-team            |  179
3 | sales_marketing     |  161
4 | marketing           |  142
5 | cypher-the-language |  125


We hope this integration between Neo4j as a open source graph database and the collaborative wonder-tool known as Slack will make it into the community Slack integrations page.

The Getting Started with NoSQL Guide will get you hands-on with NoSQL in minutes with no coding needed. Brought to you in partnership with Couchbase.

Topics:
database ,neo4j ,nosql ,slack ,integration

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

SEE AN EXAMPLE
Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.
Subscribe

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}