Java Network/Graph/Data Mining Algorithm 'Arsenal' on Neo4j

It’s nice to have an arsenal. In the world of graph databases, one such stock room is the Java Universal Network/Graph Framework(JUNG) which contains a cache of algorithms from graph theory, data mining, and social network analysis, such as routines for clustering, decomposition, optimization, random graph generation, statistical analysis, and calculation of network distances, flows, and importance measures (centrality, PageRank, HITS, etc.).

We can use JUNG via the Blueprints ouplementation and access it via Gremlin. It doesn’t come pre-packaged with Neo4j, but Michael Hunger playing the role of “Tank” was able to load up our stock room with a few key strokes.

We’ll create a directory for our project.

mkdir neojung
cd neojung

Let’s install Neography

echo "source 'http://rubygems.org'
gem 'neography' " > Gemfile
bundle install 

… add the tasks, and install noe4j.

echo "require 'neography/tasks'" >> Rakefile
rake neo4j:install

At this point you have a barebones Neo4j installation in the neojung/neo4j directory. To add JUNG we need to build a few jar files and add them to the neo4j lib directory.

We are going to be using Maven, so install it if you don’t have it.

sudo apt-get install maven2

First thing we need to do is grab blueprints and build it.

git clone git://github.com/tinkerpop/blueprints.git
cd blueprints/blueprints-graph-jung/
mvn clean -DskipTests install

We also need to get a copy of the dependencies.

mvn dependency:copy-dependencies

Now we can copy all these jar files into Neo4j.

cp target/*.jar ../../neo4j/lib
cp target/dependency/*.jar ../../neo4j/lib

Finally we need to add executable permissions to these jar files and start Neo4j.

cd ../..
chmod +x neo4j/lib/*.jar
rake neo4j:start

If you bring up the Neo4j Power Tool Console on localhost:7474 and go into the Gremlin web shell you are now able to use JUNG algorithms. Let’s import the pagerank algorithm:

import edu.uci.ics.jung.algorithms.scoring.PageRank  
==>import com.tinkerpop.gremlin.*
==>import com.tinkerpop.gremlin.java.*
==>import com.tinkerpop.gremlin.pipes.*
...
==>import edu.uci.ics.jung.algorithms.scoring.PageRank

We’ll now create a small toy graph and run pagerank on it.

j = new GraphJung(TinkerGraphFactory.createTinkerGraph());
==> graphjung[tinkergraph[vertices:6 edges:6]]
pr = new PageRank<Vertex,Edge>(j, 0.15d)
==> edu.uci.ics.jung.algorithms.scoring.PageRank@8f2588
pr.evaluate()
==>
j.getVertices().collect{ [it, pr.getVertexScore(it)] }
==> [v[3], 0.30472082661863664]
==> [v[2], 0.14598540145985392]
==> [v[1], 0.11375485828040566]
==> [v[6], 0.11375485828040566]
==> [v[5], 0.1757986539008436]
==> [v[4], 0.14598540145985392]

If you have large graphs or low memory servers, you’ll want to heed this warning by Marko Rodriguez.

NOTE: JUNG is a library that was developed for in-memory graph structures. As such, many of the aspects of its various classes are memory based. For instance, given the above PageRank example, the method pageRank.getVertexScore() is pulling from an in-memory Map that contains the score for each vertex. If the number of vertices in the graph is large, then such in-memory structures will ultimately throw an OutOfMemoryError. As such, be wary of such situations when using GraphJung. 

I’ll be writing more about JUNG and how to use some of its other algorithms. Be sure to
Follow @maxdemarzi to know as soon as they are posted.