What's New in Apache Ignite .NET 2.1

DZone 's Guide to

What's New in Apache Ignite .NET 2.1

Take a look at what's new in the latest release of Apache Ignite 2.1 from a .NET perspective. With this relase, Ignite is a complete database with a unique feature set.

· Big Data Zone ·
Free Resource

With the release of Apache Ignite 2.1 last week, Ignite Persistent Store was introduced! This marks another huge step forward for the project, making it a complete database with a unique feature set. Check out the comparison table here. As usual, we’ll have a look at new features from a .NET standpoint.

Apache Ignite Persistent Store

Ignite Persistent Store

Ignite has been an in-memory system from the start: RAM is fast, disk is slow... we all know that. However, in most use cases, we still want to persist data in some non-volatile storage, i.e. in the case of full cluster restart, data center failures, and the like. The most common solution is an RDBMS serving as a cache store, which has a lot of downsides (i.e. poor performance, single point of failure, and the complexity of the overall solution).

Ignite 2.1 solves all these problems with a single line of configuration, which enables efficient automatic persistence of all cached data on disk. The following code demonstrates how cached data survives cluster restart (just run it repeatedly):

var cfg = new IgniteConfiguration 
    PersistentStoreConfiguration = new PersistentStoreConfiguration() 
using (var ignite = Ignition.Start(cfg))
    ignite.SetActive(true);  // Required with enabled persistence.
    var cache = ignite.GetOrCreateCache<Guid, string>("myCache");
    cache[Guid.NewGuid()] = "Hello, world!";
    Console.WriteLine("\nCache size: " + cache.GetSize());

There is a single NuGet package behind this code — nothing else — and you can run SQL, LINQ and full-text queries over arbitrary data.

By default, everything is stored in the Ignite work directory, which is in the system Temp folder, so you may want to change this (call IIgnite.GetConfiguration().WorkDirectory to see where is it on your machine).

Every Ignite node persists only a part of data, which is primary or backup for that node, so storage space and IO load are split between all machines (in contrast with the RDBMS cache store).

See more details in the documentation; a working example can be found in the full binary or source distribution.

Automatic Remote Assembly Loading

Ignite Compute jobs involve executing user code on remote machines (for example, ICompute.Broadcast will execute provided action on every node in the cluster). Before 2.1, we had to make sure that required assemblies (DLL and EXE files) were loaded on all cluster nodes. With nodes starting from, Apache.Ignite.exe we had to use the -assembly switch manually.

Ignite 2.1 automates this process. When a compute job arrives at some node and Ignite fails to locate an assembly with required types locally, it will request that assembly from other nodes in the cluster. This feature has to be enabled explicitly:

var cfg = new IgniteConfiguration
    PeerAssemblyLoadingMode = PeerAssemblyLoadingMode.CurrentAppDomain

Besides making the deployment simpler, this feature enables a whole set of new use cases where the cluster keeps running and client nodes perform computations:

  • Perform one-time computations (for example, engineers use the cluster to speed up some work).
  • Test new and updated computations (quick turnaround).
  • Gather information from remote machines (for example, some logs, system details).
  • Dynamically deploy services.

Ignite uses assembly-qualified type names for compute jobs, which includes assembly version, so you can change the existing compute job implementation and Ignite will pick it up. The assembly version auto-increment ([assembly: AssemblyVersion("1.0.*")]) can simplify the process.

More details can be found in the documentation.

Assembly Loading and LINQPad

LINQPad is great for testing things quickly (see Using Apache Ignite.NET in LINQPad), and automatic assembly loading works perfectly there.

You can start a bunch of Apache.Ignite.exe processes, then play around with ICompute in LINQPad. It produces an assembly with a unique name on each run, so you don’t need to worry about versions. Try this:

  • Install Ignite.NET somewhere (see NuGet Deployment below) and modify Apache.Ignite.exe.config: <igniteConfiguration peerAssemblyLoadingMode="CurrentAppDomain" />.
  • Run Apache.Ignite.exe a couple of times.
  • Run LINQPad and install the Ignite NuGet package.
  • Switch to “C# Program” and paste the code:
void Main() {
 var cfg = new IgniteConfiguration {
  PeerAssemblyLoadingMode = PeerAssemblyLoadingMode.CurrentAppDomain,
   ClientMode = true
 using(var ignite = Ignition.Start(cfg)) {
  ignite.GetCompute().Broadcast(new MyAction());

class MyAction: IComputeAction {
 public void Invoke() => Console.WriteLine("Hello, World!");

Run the program and observe the “Hello, World!” output on all server nodes. Change the text and run again: New code is picked up by Ignite!

Standalone NuGet Deployment

Ignite nodes can be started from code (Ignition.Start()) or with a standalone executable (Apache.Ignite.exe). However, the standalone executable was only available as a part of a full binary distribution.

2.1 fixes this discrepancy and includes the standalone executable in the Apache.Ignite NuGet package so you can easily install and run Ignite with the command line:

# Powershell
wget https://dist.nuget.org/win-x86-commandline/latest/nuget.exe -OutFile nuget.exe
nuget install Apache.Ignite

LINQ Improvements

SQL DML (Data Modification Language) has been introduced in Ignite 1.9, and LINQ is starting to catch up.

Conditional data removal (SQL DELETE FROM ... WHERE ...) is now possible:

var cache = ignite.GetCache<int, Deal>("deals").AsCacheQueryable();

cache.Where(p => p.Value.Company == "Foo").RemoveAll();

This is more optimal than loading relevant entries and removing them afterwards. Batch updates via UpdateAll are on the way in future versions.

Local collection joins provide efficient alternative to Contains and other similar cases. For example, search by multiple field values can be done like this:

int[] companies = new[] { "Foo", "Bar", "Baz" };

var entries = cache.Join(companies, entry => entry.Value.Company, name => name, (entry, name) => entry);

Which generates a temporary table join:

SELECT     _t0._key, 
FROM       "deals".deal AS _t0 
INNER JOIN table (f0 nvarchar = ?) _t1 
ON         ( 
                      _t1.f0 = _t0.company)

As you can see, this query is parametrised with a single ?, so varying number of elements in a local array does not affect the cached query plan.

.net ,apache ignite ,database ,linq ,persistent storage

Published at DZone with permission of Pavel Tupitsyn . See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}