DZone
Big Data Zone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
  • Refcardz
  • Trend Reports
  • Webinars
  • Zones
  • |
    • Agile
    • AI
    • Big Data
    • Cloud
    • Database
    • DevOps
    • Integration
    • IoT
    • Java
    • Microservices
    • Open Source
    • Performance
    • Security
    • Web Dev
DZone > Big Data Zone > The World's Shortest MapReduce App

The World's Shortest MapReduce App

Dmitriy Setrakyan user avatar by
Dmitriy Setrakyan
·
Oct. 13, 10 · Big Data Zone · Interview
Like (0)
Save
Tweet
8.41K Views

Join the DZone community and get the full member experience.

Join For Free
We often say that GridGain is very simple to use, but we often forget that many of our users don't actually exploit the full power of GridGain functional APIs. In this blog I want to demonstrate several GridGain features that are available at your fingertips once you download GridGain.

1. Broadcasting
Here is a simple app that broadcasts execution of a closure to all participating nodes (in our simple scenario the closure simply prints out a string): 
G.grid().run(
    GridClosureCallMode.BROADCAST,
    F.println("Broadcasting This Message To All Nodes")
);

The "F.println()" method returns a simple closure that will print out a passed in argument. Then GridGain will take this closure and execute it on all available nodes.

2. Splitting Execution
Here is a simple app that splits a given phrase into words, creates closures that print individual words, and executes them on different nodes:

G.grid().run(
GridClosureCallMode.SPREAD,
F.yield("Splitting This Message Into Words".split(" "), F.println())
);

What happens here is that initial String is split into words using standard JDK split method. Then method "F.yield()" will take each word, give it to "F.println()" closure and return a collection of "F.println()" closures curried with a predefined argument (one for each word). GridGain will then take this collection of closures and execute them on the Grid sending each closure to a different node in round-robin fashion.

3. The World's Shortest MapReduce App
Now let's try to actually get a little more sophisticated and execute an app which will split a phrase into multiple words, have individual grid nodes count letters in individual words and then, in reduction step, add up all the counts. Here is how this app will look like:

int letterCnt = G.grid().forkjoin(
GridClosureCallMode.SPREAD,
F.yield("Counting Letters In This Phrase".split(" "),
new C1<String, Integer>() {
@Override public Integer apply(String word) {
return word.length();
}
}
),
F.sumIntReducer()
);

Here are the things to note here. The C1 is a convenience alias for a GridClosure class which in our case takes a string and returns a number of characters in that string. A collection of closures with different words as an argument will be distributed to individual grid nodes and each of those closures will return a number of characters in the word it received. Then the local node will use "F.sumIntReducer()" which simply adds all character counts given to it.

And to top it all, none of the code above requires any deployment. You simply startup a few bare bone GridGain nodes, write your code, hit the Run button, and your code just executes on the Grid (well, it's utilizing GridGain peer-class-loading mechanism underneath). There are no Ant or Maven scripts to execute, just write your code and run it. If you need to change your code, just change it and run it again.

I doubt there is another product out there that will let you execute fairly complex MapReduce applications in such a concise and elegant manner ;-)

From http://gridgain.blogspot.com/2010/10/worlds-shortest-mapreduce-app.html

app MapReduce hadoop

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Hyperautomation: The Beacon for Your Digital Transformation Journey
  • Progressive Delivery With Argo Rollouts: Blue-Green Deployment
  • Kubernetes Service Types Explained In-Detail
  • Complete Guide to TestOps

Comments

Big Data Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • MVB Program
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends:

DZone.com is powered by 

AnswerHub logo