GridGain Drivers on Cruise Control

Latest GridGain releases have had a big focus on remote grid clients, or as we like to call them remote drivers. We have recently added Java/Android and C#/.NET clients with C++ and Objective-C clients right around the corner.  The goal of remote drivers is to provide ability to use Grid API from environments where it does not make sense to start a node and join the grid, but yet it is still needed to access objects from cache or execute a MapReduce task on remote grid.

The real trick when designing clients is to make sure that the API's are feature rich, but yet portable between different languages. We used Google Protocol Buffers to ensure portability. This way an object can be stored in cache using .NET C# client and retrieved, for example, using Java or C++ clients.

Another real important feature for remote drivers is data affinity, especially when working with partitioned caches. Partition caches have a concept of data ownership where every node owns a certain chunk of data. This way data can be split across many nodes and the whole grid becomes one shared RAM storage. However, such approach only works well with collocation of computations and data. All client requests to data should come exactly to the node where the data is cached - otherwise there will be lots of redundant data movement (a.k.a. data noise) which will have adverse effect on scalability.

At GridGain we took specific effort to make sure that all remote GridGain drivers have an embedded notion of affinity - this way you can store an object in cache using Java and then retrieve it using C# client which will know exactly where to go to get it.

Here is an example of how you would use GridGain remote Java or .NET C# clients for basic data and compute operations:

Java Client:

// Get all remote grid nodes
for (GridClientNode node: client.compute().nodes())
    System.out.println("External IP addresses: " + node.externalAddresses())
 
// Connect to partitioned cache.
GridClientData rmtCache = client.data("partitioned")
 
// Get and update 10 keys. All cache operations
// will go exactly to the node where data is cached.
for (int key = 0; key < 10; key++) {
    // Get data.
    String val = rmtCache.get(key);
 
    // Update data if it is equal to the one we just got.
    rmtCache.cas(key, val + i, val);
 
    // Asynchronously remove data.
    rmtCache.removeAsync(key);
}
 
// Connect to remote grid.
GridClientCompute rmtCompute = client.compute();
 
// Execute map-reduce task on remote grid.
GridClientFuture fut = rmtCompute.executeAsync("MyTask", "myarg");
 
// Wait for completion.
fut.get();
 
...

.NET C# Client:

using X = System.Console;
 
// Get all remote grid nodes.
foreach (IGridClientNode node in client.Compute().Nodes())
    X.WriteLine("External IP addresses: " + node.ExternalIpAddresses);
 
// Connect to partitioned cache.
IGridClientData rmtCache = client.Data("partitioned");
 
// Get and update 10 keys. All cache operations
// will go exactly to the node where data is cached.
for (int key = 0; key < 10; key++) {
    // Get data.
    String val = rmtCache.GetItem(key);
 
    // Update data if it is equal to the one we just got.
    rmtCache.Cas(key, val + i, val);
 
    // Asynchronously remove data.
    rmtCache.RemoveAsync(key);
}
 
// Connect to remote grid.
IGridClientCompute rmtCompute = client.Compute();
 
// Execute map-reduce task on remote grid.
IGridClientFuture fut = rmtCompute.ExecuteAsync("MyTask", "myarg");
 
// Wait for completion.
fut.Result;
 
...

Note how almost identical the client API's for Java and C# look. Also note that both, C# and Java clients use key affinity and send data operations exactly to the nodes where the data is.

To try GridGain and it's new remote drivers, download GridGain 4.0.2 here.