DZone
Cloud Zone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
  • Refcardz
  • Trend Reports
  • Webinars
  • Zones
  • |
    • Agile
    • AI
    • Big Data
    • Cloud
    • Database
    • DevOps
    • Integration
    • IoT
    • Java
    • Microservices
    • Open Source
    • Performance
    • Security
    • Web Dev
DZone > Cloud Zone > Apache Giraph: Distributed Graph Processing in the Cloud

Apache Giraph: Distributed Graph Processing in the Cloud

René Pickhardt user avatar by
René Pickhardt
·
May. 22, 12 · Cloud Zone · Interview
Like (0)
Save
Tweet
8.74K Views

Join the DZone community and get the full member experience.

Join For Free
Claudio Martella introduces Apache Giraph which according to him is a loose implementation of Google Pregel which was introduced  on SIGMOD in 2010. He points out that Map Reduce cannot be used to do graph processing.

He then gave an example on how MapReduce can be used to to do page rank calculation. He points out that Pagerank can be calculated as a local property of a graph in a distributed way by calculating local pagerank from the knowledge of the neighbours. He did this to show what the Drawbacks of this method are in his oppinion:

  • job boostrap take some time
  • disk is hit about 6  times
  • Data is sorted
  • Graph is passed through

Like in the Pregel Paper he says that other Graphalgorithms like singlesource shortest paths have the same problems. 

 

Claudio Martella from Apache explains how giraph works at in the graph dev room @ Fosdem 2012
Claudio Martella from Apache explains how giraph works at in the graph dev room @ Fosdem 2012

 

After introducing more about implementing Pregle ontop of the existing MapReduce structure for distributing he says that this system has some advantages over MapReduce

  • it’s a stateful computation
  • Disk is hit if/only for checkpoints
  • No sorting is necessary
  • Only messages hit the network

He points out that the advantages of Giraph over other methods (Hama, GoldenOrb, Signal/Collect) are especially an active community (Facebook, Yahoo, Linkedin, Twitter) behind this project. I personally think another advantage is that it is run by Apache who already run MapReduce (Hadoop) with great success. So it is something that people trust…

Claudio points out explicitly that they are searching for more contributors and I think this is really an interesting topic to work on! So thank Claudio for your inspiring work!

here the video streams from the graph dev room:

 

 

Graph (Unix) Processing Cloud

Published at DZone with permission of René Pickhardt, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Ultra-Fast Microservices: When Microstream Meets Payara
  • Is Java Still Relevant?
  • Debugging Deadlocks and Race Conditions
  • How to Determine if Microservices Architecture Is Right for Your Business

Comments

Cloud Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • MVB Program
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends:

DZone.com is powered by 

AnswerHub logo