Over a million developers have joined DZone.
Platinum Partner

GridGain: Master-Worker in Peer-To-Peer Architecture

· Java Zone

The Java Zone is brought to you in partnership with ZeroTurnaround. Check out this 8-step guide to see how you can increase your productivity by skipping slow application redeploys and by implementing application profiling, as you code!

GridGain - Grid Computing Made SimpleWe sometimes get questions from users on how to ensure Master-Worker pattern within peer-to-peer (P2P) architecture in GridGain. When designing our API and our deployment model, we purposely went with P2P architecture because we wanted to have ultimate freedom on how a grid node is used. As a result, in GridGain a node can act as master or worker or both, depending on your configuration. Moreover, you don't even have to change a single line of code to get this to work.

The following example shows on how it can be done. In GridGain every node has a notion of attributes which it gets at startup. Here is an example that shows how a node can get a "worker" attribute from Spring XML configuration file at startup:

<bean 
id="grid.cfg"
class="org.gridgain.grid.GridConfigurationAdapter"
scope="singleton">
...
<property name="userAttributes">
<map>
<!-- Make this node a worker node. -->
<entry key="segment.worker" value="true"/>
</map>
</property>
...
</bean>
Now, we need to make sure that only worker nodes are passed into GridTask.map(...) method on the master nodes. To do this, on the master nodes we need to configure GridAttributesTopologySpi, the purpose of which is to filter nodes based on their attributes. Here is how the configuration will look like:
<bean 
id="grid.cfg"
class="org.gridgain.grid.GridConfigurationAdapter"
singleton="true">
...
<property name="topologySpi">
<bean class="org.gridgain.grid.spi.topology.attributes.GridAttributesTopologySpi">
<property name="attributes">
<map>
<!-- Include only worker nodes. -->
<entry key="segment.worker" value="true"/>
</map>
</property>
</bean>
</property>
...
</bean>

That's it! To verify that it works, we can add assertion into our GridTask implementation to make sure that all included nodes are indeed "worker" nodes as follows:

public class FooBarGridTask 
extends GridTaskAdapter<String, String> {
...
public Map<GridJob, GridNode> map(
List<GridNode> topology, String arg) {

Map<GridJob, GridNode> jobs =
new HashMap<GridJob, GridNode>(topology.size());

for (GridNode node : topology) {
String workerAttr =
node.getAttribute("segment.worker");

// Assert that worker attribute is present and
// is assigned value "true".
assert workerAttr != null;
assert Boolean.getBoolean(workerAttr) == true;

jobs.put(new FooBarWorkerJob(arg), node);
}

return jobs;
}
...
}

Note, that although we only segmented grid into 2 segments, masters and workers, you can configure as many segments as you like by providing additional node attributes. For example, you can have several worker groups each responsible for processing only a certain subset of jobs.

For addtional examples take a look at Segmenting Grid Nodes article on our Wiki.

The Java Zone is brought to you in partnership with ZeroTurnaround. Check out this 8-step guide to see how you can increase your productivity by skipping slow application redeploys and by implementing application profiling, as you code!

Topics:

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}