Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Multi-Node Distributed Execution Using Ignite and Dexecutor

DZone's Guide to

Multi-Node Distributed Execution Using Ignite and Dexecutor

A good introduction for Java Developers using Dexecutor — a fast, light, distributed independent task engine for running a cluster of Apache Ignite in-memory nodes on a single machine for massive parallelism.

· Java Zone
Free Resource

Download Microservices for Java Developers: A hands-on introduction to frameworks and containers. Brought to you in partnership with Red Hat.

Today, we will try to execute Dexecutor in a distributed mode using Apache Ignite. For the demo, we would be setting up multiple Ignite nodes on a single machine.

Refer to Introducing Dexecutor, to get an introduction on Dexecutor and to understand the problem we would solve in a distributed fashion. In short:

We would be distributing the execution of dexecutor tasks on Apache Ignite nodes in a single machine.

To do that, one of the nodes would act as the master and submit the tasks to Ignite to be executed by other Ignite compute nodes using Dexecutor.

Here are the steps to do that.

Step 1: Add Dexecutor-Ignite Dependency

<dependency>
     <groupId>com.github.dexecutor<groupId>
     <artifactId>dexecutor-ignite<artifactId>
     <version>LATEST_RELEASE<version>
<dependency>

Step 2: Start Ignite

IgniteConfiguration cfg = new IgniteConfiguration();
cfg.setGridName(nodeName);

Ignite ignite = Ignition.start(cfg);

Step 3 : Create Dexecutor Using Ignite

if (isMaster) {
    DefaultDependentTasksExecutor<Integer, Integer> dexecutor = newTaskExecutor(ignite.compute());

    buildGraph(dexecutor);
    dexecutor.execute(ExecutionConfig.TERMINATING);
}
private DefaultDependentTasksExecutor<Integer, Integer> newTaskExecutor(final IgniteCompute igniteCompute) {
        DependentTasksExecutorConfig<Integer, Integer> config = new DependentTasksExecutorConfig<Integer, Integer>(
                new IgniteExecutionEngine<Integer, Integer>(igniteCompute), new SleepyTaskProvider());
        return new DefaultDependentTasksExecutor<Integer, Integer>(config);
}

Step 4: Execution

Open three terminals and execute the following :

Terminal #1

mvn test-compile exec:java -Djava.net.preferIPv4Stack=true -Dexec.mainClass="com.github.dexecutor.ignite.Node" -Dexec.classpathScope="test" -Dexec.args="s node-A"

Terminal #2

mvn test-compile exec:java -Djava.net.preferIPv4Stack=true -Dexec.mainClass="com.github.dexecutor.ignite.Node" -Dexec.classpathScope="test" -Dexec.args="s node-B"

Terminal #3

mvn test-compile exec:java  -Dexec.classpathScope="test" -Djava.net.preferIPv4Stack=true -Dexec.mainClass="com.github.dexecutor.ignite.Node" -Dexec.args="m node-C"

dexecutor-ignite-execution

Here is the Node Implementation.

References

Download Modern Java EE Design Patterns: Building Scalable Architecture for Sustainable Enterprise Development.  Brought to you in partnership with Red Hat

Topics:
node ,apache ignite ,java ,dexecutor

Published at DZone with permission of Mohammad Nadeem, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}