DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

The software you build is only as secure as the code that powers it. Learn how malicious code creeps into your software supply chain.

Apache Cassandra combines the benefits of major NoSQL databases to support data management needs not covered by traditional RDBMS vendors.

Generative AI has transformed nearly every industry. How can you leverage GenAI to improve your productivity and efficiency?

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Related

  • Scalable Data Grid Using Apache Ignite
  • Split the Monolith: What, When, How
  • Microservice: Creating JPA Application Using Jakarta Persistence API in Payara Micro
  • Circuit Breaker Pattern With Netflix-Hystrix: Java

Trending

  • Memory Leak Due to Time-Taking finalize() Method
  • Operational Principles, Architecture, Benefits, and Limitations of Artificial Intelligence Large Language Models
  • Agile’s Quarter-Century Crisis
  • Building Reliable LLM-Powered Microservices With Kubernetes on AWS
  1. DZone
  2. Data Engineering
  3. Data
  4. Running Microservices on Top of In-Memory Data Grid: Part II

Running Microservices on Top of In-Memory Data Grid: Part II

Let's look into the Apache Ignite Cluster Layer, a GitHub project that includes the basic building blocks needed to implement a proposed microservices-based architecture.

By 
Denis Magda user avatar
Denis Magda
DZone Core CORE ·
Jan. 18, 17 · Opinion
Likes (7)
Comment
Save
Tweet
Share
12.3K Views

Join the DZone community and get the full member experience.

Join For Free

This post is the second in the series of articles that provides a step-by-step guide on how to build a fault-tolerant and scalable microservices-based solution relying on an In-Memory Data Grid (IMDG) like Apache Ignite In-Memory Data Fabric.

In the first post, one of the possible architectures was presented and explained as a system consisted of the following layers:

  • Apache Ignite cluster layer.
  • Persistent storage layer.
  • Layer used by external applications.

Microservices

In this post, we will look into the first layer (the Apache Ignite Cluster Layer), referring to a special GitHub project that includes basic building blocks needed to implement the proposed microservices-based architecture in life. In particular, the following will be covered:

  • Configuration and launching of data nodes.
  • Exemplary services' implementation using Apache Ignite Service Grid API. 
  • Configuration and launching of service nodes hosting specific Apache Ignite services.
  • A dummy application that connects to the cluster and triggers services execution.

Data Nodes When Microservices Are on Top of an In-Memory Data Grid Architecture

As it was learned from the first post, a data node is an Apache Ignite server node that holds a portion of the dataset and executes queries and computations coming from application logic side over this dataset. In general, this type of node is agnostic to application logic because the nodes simply need to store datasets and process them efficiently when an application talks to the data.

Let's have a look at how a data node can be defined at the implementation layer.

Download the provided GitHub project and locate data-node-config.xml, which is used to spawn a new data node. The configuration contains a number of data nodes related sections and parameters.

First, we need to set a special node filter for every Apache Ignite cache that is going to be deployed in the cluster. The filter is called upon a cache startup to define a subset of the cluster nodes where cache data will be stored — data nodes. The same filter will be called every time the cluster topology changes which happen when a new cluster node joins the cluster or an old one leaves it. An implementation of the filter has to be added to the classpath of every Apache Ignite node regardless of the fact whether is node going to be a data node or not.

<bean class="org.apache.ignite.configuration.CacheConfiguration">
...
  <property name="nodeFilter">
    <bean class="common.filters.DataNodeFilter"/>
  </property>
</bean>

Second, the filter has to be implemented. In this example, we use one of the most straightforward implementations. DataNodeFilter decides if an Apache Ignite node should be considered as a data node by checking the custom data.node parameter. If the node has this parameter set in its attributes map then it will be a data node and the cache data will be hosted on it. Otherwise, the node will be ignored.

 public boolean apply(ClusterNode node) {
  Boolean dataNode = node.attribute("data.node");
  return dataNode != null && dataNode;
}

Third, data-node-config.xml adds the data.node parameter to the attributes map of every node that uses the config at start time. This is how it's done.

<property name="userAttributes">
  <map key-type="java.lang.String" value-type="java.lang.Boolean">
    <entry key="data.node" value="true"/>
  </map>
</property>

At the end, start an instance of a data node use DataNodeStartup file from the example or pass data-node-config.xml to Apache Ignite's ignite.sh/bat scripts. If you prefer the latter then make sure to build a JAR file that will contain all the classes from java/app/common directory. The JAR has to be added to the classpath of every data node.

Service Nodes When Running Microservices on Top of an In-Memory Data Grid

The definition of a service node at the implementation layer is not much different from the approach used for data nodes in the previous section. Basically, we need to establish a way that will specify a subset of Apache Ignite nodes where a particular microservice supposed to be deployed.

Initially, we need to implement a microservice using Apache Ignite Service Grid APIs. For the sake of the blog post, let's review one of the existed implementations available in the provided GitHub example, the Maintenance Service.

The service allows scheduling a vehicle maintenance service and see a list of existed appointments. It implements all required Service Grid methods which are init(...), execute(...), and cancel(...), and introduces the new ones in its own unique Java interface:

 public interface MaintenanceService extends Service {
  public Date scheduleVehicleMaintenance(int vehicleId);
  public List<Maintenance> getMaintenanceRecords(int vehicleId);
}

There are several ways to configure and deploy Maintenance Service on a specific Apache Ignite nodes — service nodes. In our case, every node that might be considered for a Maintenance Service deployment has to be started with maintenance-service-node-config.xml. Let's look in the configuration.

First, to make sure that an instance of Maintenance Service is deployed on specific nodes only we need to set a node filter. 

<bean class="org.apache.ignite.services.ServiceConfiguration">
  <property name="nodeFilter">
    <bean class="common.filters.MaintenanceServiceFilter"/>
  </property>
</bean>

Second, the filter that is used for Maintenance Service allows deploying an instance of the service on a node only if the node has maintenance.service.node in its attributes map.

 public boolean apply(ClusterNode node) {
  Boolean dataNode = node.attribute("maintenance.service.node");
  return dataNode != null && dataNode;
}

Finally, every node starting with maintenance-service-node-config.xml will have this attribute in its map thanks to this XML section:

<property name="userAttributes">
  <map key-type="java.lang.String" value-type="java.lang.Boolean">
    <entry key="maintenance.service.node" value="true"/>
  </map>
</property>

That's it. Start one or more instances of a Maintenance Service node using the MaintenanceServiceNodeStartup file or pass maintenance-service-node-config.xml to Apache Ignite's ignite.sh/bat scripts. If you prefer the latter then make sure to build a jar file that will contain all the classes from java/app/common and java/services/maintenance directories. The JAR has to be added to the classpath of every node where the service might be deployed.

The example contains one more Apache Ignite Service for Vehicles management. Start at least one instance of a service node where this service can be deployed using the VehicleServiceNodeStartup file or ignite.sh/bat script with vehicle-service-node-config.xml configuration. Don't forget to assemble a JAR with required classes and add it to the classpath of the node if you go for with ignite.sh/bat approach.

Sample Application for Running Microservices on Top of an In-Memory Data Grid

Once we have our data nodes, Maintenance Service and Vehicles Service nodes up and running we're ready to run the first sample application that will talk to our distributed microservices.

Locate and start TestAppStartup file from the example. The application connects to the cluster, fills predefined caches in with dummy data and interacts with the developed services.

MaintenanceService maintenanceService = ignite.services().serviceProxy(MaintenanceService.SERVICE_NAME,
MaintenanceService.class, false);
int vehicleId = rand.nextInt(maxVehicles);
Date date = maintenanceService.scheduleVehicleMaintenance(vehicleId);

If you note, the application works with the services using service proxies. The beauty of the proxies is that the node which is started as a part of the application process doesn't need to have a service implementation in the local classpath and an instance of the service being deployed locally.

To Be Continued

In this post, we saw how to implement Apache Ignite Cluster Layer from the proposed microservices-based architecture. In the next posts in the series, we will look into the implementation details of the layers that are left:

  • Persistent storage layer.

  • Layer used by external applications.

Data (computing) microservice Data grid Apache Ignite application cluster Implementation

Published at DZone with permission of Denis Magda. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • Scalable Data Grid Using Apache Ignite
  • Split the Monolith: What, When, How
  • Microservice: Creating JPA Application Using Jakarta Persistence API in Payara Micro
  • Circuit Breaker Pattern With Netflix-Hystrix: Java

Partner Resources

×

Comments

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends: