Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Visualizing the News with Vivagraph.js

DZone's Guide to

Visualizing the News with Vivagraph.js

· Database Zone
Free Resource

Traditional relational databases weren’t designed for today’s customers. Learn about the world’s first NoSQL Engagement Database purpose-built for the new era of customer experience.


neo_news

Today I want to introduce you to VivaGraphJS – a JavaScript Graph Drawing Library made by Andrei Kashcha of Yasiv. It supports rendering graphs using WebGL, SVG or CSS formats and currently supports a force directed layout. The Library provides an API which tracks graph changes and reflect changes on the rendering surface which makes it fantastic for graph exploration.

Today we will be integrating it with Neo4j and the Alchemy API.

alchemyapi

The Alchemy API provides a set of natural language processing tools, and we’ll make use of their Entity Extraction capabilities.

We are going to take a look at the news of the world, extract the entities mentioned in them, connect them all together in Neo4j and visualize them. To get the news, we’ll use Feedzilla.

feedzilla

sidekiq job will run every six hours to collect the latest 100 news articles:

module Job
  class GetNews
    include Sidekiq::Worker
    sidekiq_options :retry => false
 
    def perform
      Job::GetNews.perform_in(6.hours)
 
      feed = HTTPClient.get("http://api.feedzilla.com/v1/categories/19/articles.json?order=date&count=100&title_only=1")
      parsed_feed = Oj.load(feed.body)
      parsed_feed["articles"].each do |article|
        Job::GetArticle.perform_async(article["url"])
      end
    end
 
  end
end

From each article, we’ll get the story and pass it on to Alchemy API and ask it to find the entities in the article.

@entities = Oj.load(HTTPClient.post_content("http://access.alchemyapi.com/calls/url/URLGetRankedNamedEntities",
      {:url => article_url, 
       :apikey => ENV['ALCHEMY_API'],
       : outputMode => "json"}),
       {:'Accept-encoding' => "gzip"})["entities"]

We’ll create a node for each article and connect each entity found by a “MENTIONED” relationship:

commands = []
@batch_result.each do |b|
  commands << [:create_relationship, "MENTIONED", @article_node, b["body"]["self"].split("/").last]
end

Some of these entities have interesting descriptions, so we’ll query dbpedia for more information.

dbpedia_logo

Moving on to our front-end application, we’ll provide an auto-complete search box which will query the Entities index of our graph:

get '/search' do
  content_type :json
  neo = Neography::Rest.new   
 
  cypher = "START me=node:entities({query}) 
            RETURN ID(me), me.text
            ORDER BY me.text
            LIMIT 15"
 
  neo.execute_query(cypher, {:query => "text:*#{params[:term]}* OR uri:*#{params[:term]}*" })["data"].map{|x| { label: x[1], value: x[0]}}.to_json   
end

We’ll also provide an end-point to pull a node and all the nodes connected to it and return a JSON object with this data:

get '/edges/:id' do
    content_type :json
    neo = Neography::Rest.new   
 
    cypher = "START me=node(#{params[:id]}) 
              MATCH me -- related
              RETURN ID(me), me.text, me.description, me.type, ID(related), related.text, related.description, related.type"
 
    connections = neo.execute_query(cypher)["data"]   
    connections.collect{|n| {"source" => n[0], "source_data" => {:label => n[1], 
                                                                 :description => n[2],
                                                                 :type => n[3] },
                             "target" => n[4], "target_data" => {:label => n[5], 
                                                                 :description => n[6],
                                                                 :type => n[7]}} }.to_json
  end

Our javascript will contain the calls to Vivagraph:

var graph = Viva.Graph.graph(); 
 
  var layout = Viva.Graph.Layout.forceDirected(graph, {
      springLength:100,
      springCoeff:0.0001,
      dragCoeff:0.02,
      gravity:-1
  });   

Our nodes will display both the Text found in the entity as well as an image to represent the entity type.

var ui = Viva.Graph.svg('g'),
        svgText = Viva.Graph.svg('text').attr('y', '-4px').text(node.data.label),
        img = Viva.Graph.svg('image')
           .attr('width', 32)
           .attr('height', 32)
           .link('/img/' + node.data.type + '.png');
 
    ui.append(svgText);
    ui.append(img);

We’ll add a mouseover event to update the side-panel with the description of our node:

$(ui).hover(function() { // mouse over
                        highlightRelatedNodes(node.id, true);
                        $('#explanation').html(node.data.description);
                    }, function() { // mouse out
                        highlightRelatedNodes(node.id, false);
                    });

…and to load the nodes related to this entity on click:

$(ui).click(function() { 
                        console.log("click", node);
                        if (!node || !node.position) return;
                        renderer.rerender();
                        loadData(graph,node.id);
            }
    );

As usual, the code is available on github, and you can see it running live on Heroku.

Check out some more of the Vivagraph.js demos:




Learn how the world’s first NoSQL Engagement Database delivers unparalleled performance at any scale for customer experience innovation that never ends.

Topics:

Published at DZone with permission of Max De Marzi, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}