Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

The Neo4j-Slack Integration You’ve Been Waiting For (Is Here)

DZone's Guide to

The Neo4j-Slack Integration You’ve Been Waiting For (Is Here)

Build a Slack and Neo4j integration to demonstrate how useful a graph database backend would be. And, of course, how much fun.

· Database Zone
Free Resource

Find out how Database DevOps helps your team deliver value quicker while keeping your data safe and your organization compliant. Align DevOps for your applications with DevOps for your SQL Server databases to discover the advantages of true Database DevOps, brought to you in partnership with Redgate

Our colleague Andreas, who loves Slack and brought it into our company, suggested the other day that we could build a Slack and Neo4j integration to demonstrate how useful a graph database backend would be.

And, of course, how much fun.

Read All about Our Neo4j-Slack Integration That Generates Recommended Channels to Other Users

In the process of making the integration, we came up with some surprisingly useful benefits. More below.

Building Blocks

As it was only midnight we, the amazing Nicole and Michael, decided to put something together.

We set up a Neo4j 2.2.3 instance in the cloud.

Then, we created a Python app for our Slack-Neo4j server and pushed it to GitHub.

The application uses web.py for the webapp, requests to access the Slack APIs and py2neo to talk to Neo4j.

Next we pushed it to Heroku to make it available publicly so that Slack could connect to it.

You have to provide environment variables to your Neo4j server, your Slack API token and the team token configured with your slash command. Find the details in the project readme.

Slack Slash Command

With all that complete, we could then set up a slash command. For our integration, the ideas came from Andreas:

  • /graph import – import users, channels and membership into Neo4j
  • /graph cypher MATCH ... RETURN – execute read only cypher statement and return the results
  • /graph – provide an overview of the data that’s in the database

Implementing the app was straightforward. Parsing the POST payload and checking the team token, we then got the first word of the text parameter as “command” to dispatch on.

Getting Data from Slack into Neo4j

For the integration with Neo4j we sent Cypher statements to Neo4j using py2neo’s APIs. Example below:

from py2neo import Graph
graph = Graph(os.environ.get('NEO4J_URL'))

graph.cypher.execute("MATCH (u:User)-[:MEMBEROF]->(c:Channel) return u.screenname, c.name")


Sending requests to the Slack API with the token and getting the JSON response is straightforward with requests. We then passed the JSON response directly as parameters to a Cypher statement to create the graph structure in Neo4j.

res = requests.get("https://slack.com/api/channels.list?token={}".format(token))

query = """
UNWIND {channels} AS channel
MERGE (c:Channel {id:channel.id}) ON CREATE SET c.name = channel.name
"""
graph.cypher.execute_one(query, res.json())


As you can see, we can import users, channels and memberships, easy peasy.

/graph import channels

slackbot: 115 users uploaded. Only you can see this message

/graph import users

slackbot: 117 channels uploaded Only you can see this message

Graph All the Slack Things

And to show you that it worked, here is a graph of our Slack universe:

And here are some queries that show the most prolific people:

/graph cypher match (u:User)-->() return u, count(*) as memberships order by memberships desc limit 3`

slackbot:
   | u                                                            | memberships
---+--------------------------------------------------------------+-------------
1 | (n200:User {fullname:"Michael",id:"U02HVJ36",username:"mh"})  |          64
2 | (n151:User {fullname:"Chris",id:"U0KLMP5X",username:"cl"})    |          37
3 | (n210:User {fullname:"Philip",id:"U02HDEF0EX",username:"pr"}) |          37


Recommendations

Finally, the biggest surprise of all: We wanted to recommend new channels to people.

We used traditional collaborative filtering for this concept of “channels of your colleagues that are not yet your channels”. But we also filter out prolific users and channels so that they don’t distort the picture.

/graph cypher
MATCH (c:Channel) with toInt(count(*)*0.618) as channel_cutoff
MATCH (u:User) with toInt(count(*)*0.618) as user_cutoff, channel_cutoff

MATCH (u:User {username:"laeg"})-[:MEMBER_OF]->(c:Channel)
      <-[:MEMBER_OF]-(coll:User)-[:MEMBER_OF]->(reco:Channel)

WHERE size((c)<--())    < channel_cutoff
  AND size((reco)<--()) < channel_cutoff
  AND size((coll)-->()) < user_cutoff
  AND NOT (u)-[:MEMBER_OF]->(reco)

RETURN reco.name, count(*) AS freq
ORDER BY freq DESC
LIMIT 5;



slackbot:
  | reco.name           | freq
--+---------------------+------
1 | feedback            |  218
2 | dev-team            |  179
3 | sales_marketing     |  161
4 | marketing           |  142
5 | cypher-the-language |  125


We hope this integration between Neo4j as a open source graph database and the collaborative wonder-tool known as Slack will make it into the community Slack integrations page.

Align DevOps for your applications with DevOps for your SQL Server databases to increase speed of delivery and keep data safe. Discover true Database DevOps, brought to you in partnership with Redgate

Topics:
database ,neo4j ,nosql ,slack ,integration

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}