Over a million developers have joined DZone.

GridGain: To Split Or Not To Split

DZone's Guide to

GridGain: To Split Or Not To Split

· Java Zone
Free Resource

Microservices! They are everywhere, or at least, the term is. When should you use a microservice architecture? What factors should be considered when making that decision? Do the benefits outweigh the costs? Why is everyone so excited about them, anyway?  Brought to you in partnership with IBM.

GridGain - Grid Computing Made SimpleWhen designing your tasks for execution on the grid, you need to decide whether to split your task into smaller jobs for parallel execution, and what the size of your split should be.

When Not To Split

People often think of a grid as of infrastructure for execution of long running tasks. That is not always the case. Compute grids can add a lot of value even for quick, short running jobs. Imagine, for example, that your application constantly needs to calculate in real time a bunch of statistical metrics and averages on a financial portfolio, let's say for displaying them on UI. Although every single calculation may not take too long, having all calculations performed concurrently on the same server or a thick client can bring it to its knees fairly quickly. A good solution would be to take every calculation and execute it on a separate grid node.

The example above is a good case for when not to split your tasks. When deciding whether to split or not, you should take into consideration the time of local execution. If your task can execute locally fast enough, then don't split it and run it on the grid as a whole. By doing that you get the following benefits:

  • You remove a single point of failure. If a grid node crashes in a middle of calculation, then GridGain will automatically fail it over to another node.
  • You balance the load across your grid nodes. GridGain will automatically load balance your jobs to the least loaded nodes. You can also turn on job stealing and have less loaded nodes steal jobs from more loaded nodes.
  • You improve overall scalability of your application. Now you can add or remove grid nodes on demand whenever your application load peaks or slows down and, hence, keep the response times constant regardless of the load. For example, you can configure your grid to include more nodes into topology as application load grows.
  • You get GridGain's simplicity. Here is how simple it can be to execute some portfolio calculation on the grid. Note that all you have to do is attach @Gridify annotation to your Java method and that's it!
public double calculatePortfolioPosition() {


How To Split

Now, let's say you really need to split your task into smaller jobs in order to speed up execution. A good formula to decide on what the size of your split should be is to take the time your task takes to execute locally and divide it by the time you would like to achieve. So if your task executes in 2 seconds and you would like to achieve 100 milliseconds, then the number of jobs your task should split into should be (2000 ms / 100 ms = 20). In reality, the execution time will be slightly more than 100ms as most likely your jobs will not be absolutely equal, and there is also a slight network communication overhead.

That is not to say that for this example you would only need to have 20 nodes in the grid. Ideally you should have as many nodes as your application needs in order to handle the load - let GridGain pick the most available 20 nodes for execution of individual jobs within your task.

For more information visit GridGain Wiki or watch Grid Application In 15 Minutes screencast.

Discover how the Watson team is further developing SDKs in Java, Node.js, Python, iOS, and Android to access these services and make programming easy. Brought to you in partnership with IBM.


Published at DZone with permission of Dmitriy Setrakyan, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}