DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Last call! Secure your stack and shape the future! Help dev teams across the globe navigate their software supply chain security challenges.

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Releasing software shouldn't be stressful or risky. Learn how to leverage progressive delivery techniques to ensure safer deployments.

Avoid machine learning mistakes and boost model performance! Discover key ML patterns, anti-patterns, data strategies, and more.

Related

  • Spring Boot With Kubernetes
  • Advanced Kubernetes Setup for Spring Boot App With PostgreSQL DB
  • Containerization and Helm Templatization Best Practices for Microservices in Kubernetes
  • [CSF] Using Metrics In Spring Boot Services With Prometheus, Graphana, Instana, and Google cAdvisor

Trending

  • DZone's Article Submission Guidelines
  • A Complete Guide to Modern AI Developer Tools
  • Vibe Coding With GitHub Copilot: Optimizing API Performance in Fintech Microservices
  • Is Agile Right for Every Project? When To Use It and When To Avoid It
  1. DZone
  2. Coding
  3. Frameworks
  4. Apache Ignite Cluster Together With Spring Boot

Apache Ignite Cluster Together With Spring Boot

Spring Boot is taking the Java world by storm, so let's take a look at how it's helping make the Big Data world a bit more accessible!

By 
Piotr Mińkowski user avatar
Piotr Mińkowski
·
Apr. 06, 18 · Tutorial
Likes (6)
Comment
Save
Tweet
Share
26.8K Views

Join the DZone community and get the full member experience.

Join For Free

I have already introduced Apache Ignite in one of my previous articles, In-Memory Data Grid With Apache Ignite. Apache Ignite can be easily launched locally together with a Spring Boot application. The only thing we have to do is include the artifact org.apache.ignite:ignite-spring-data in the project dependencies and then declare the Ignite instance @Bean. A sample @Bean declaration is below.

@Bean
public Ignite igniteInstance() {
  IgniteConfiguration cfg = new IgniteConfiguration();
  cfg.setIgniteInstanceName("ignite-cluster-node");
  CacheConfiguration ccfg1 = new CacheConfiguration("PersonCache");
  ccfg1.setIndexedTypes(Long.class, Person.class);
  CacheConfiguration ccfg2 = new CacheConfiguration("ContactCache");
  ccfg2.setIndexedTypes(Long.class, Contact.class);
  cfg.setCacheConfiguration(ccfg1, ccfg2);
  IgniteLogger log = new Slf4jLogger();
  cfg.setGridLogger(log);
  return Ignition.start(cfg);
}

In this article, I would like to show you a little more advanced sample where we will start multiple Ignite nodes inside the cluster, Ignite's web console for monitoring the cluster, and Ignite's agent for providing communication between the cluster's nodes and web console. Let's begin by looking at a picture with an architecture of our sample solution.

We have three nodes that are part of the cluster. If you carefully take a look at the picture illustrating an architecture, you have probably noticed that there are two nodes: a Server node and a Client node. By default, all Ignite nodes are started as Server nodes. The Client node needs to be explicitly enabled. Server nodes participate in caching, compute execution, and stream processing, while Client nodes provide the ability to connect to the servers remotely. However, they allow using the whole set of Ignite APIs, including near caching, transactions, compute, and streaming.

Here's Ignite's client instance @Bean declaration.

@Bean
public Ignite igniteInstance() {
  IgniteConfiguration cfg = new IgniteConfiguration();
  cfg.setIgniteInstanceName("ignite-cluster-node");
  cfg.setClientMode(true);
  CacheConfiguration ccfg1 = new CacheConfiguration("PersonCache");
  ccfg1.setIndexedTypes(Long.class, Person.class);
  CacheConfiguration ccfg2 = new CacheConfiguration("ContactCache");
  ccfg2.setIndexedTypes(Long.class, Contact.class);
  cfg.setCacheConfiguration(ccfg1, ccfg2);
  return Ignition.start(cfg);
}

The fact is that we don't have to do anything more to make our nodes work together within the cluster. Every new node is automatically detected by all other cluster's nodes using multicast communication. When starting our sample application, we only have to guarantee that each instance's server would listen of a different port by overriding the server.port Spring Boot property. Here's command that starts the sample application, which is available on GitHub under the branch cluster. Each node exposes the same REST API, which may be easily tested using Swagger2 just by opening its dashboard available under address http://localhost:port/swagger-ui.html.

java -jar -Dserver.port=8901 -Xms512m -Xmx1024m -XX:+UseG1GC -XX:+DisableExplicitGC -XX:MaxDirectMemorySize=256m target/ignite-rest-service-1.0-SNAPSHOT.jar

If you have successfully started a new node, you should see similar information in your application logs.

>>> +----------------------------------------------------------------------+
>>> Ignite ver. 2.4.0#20180305-sha1:aa342270b13cc1f4713382a8eb23b2eb7edaa3a5
>>> +----------------------------------------------------------------------+
>>> OS name: Windows 10 10.0 amd64
>>> CPU(s): 4
>>> Heap: 1.0GB
>>> VM name: 14132@piomin
>>> Ignite instance name: ignite-cluster-node
>>> Local node [ID=9DB1296A-7EEC-4564-BAAD-14E5D4A3A08D, order=2, clientMode=false]
>>> Local node addresses: [piomin/0:0:0:0:0:0:0:1, piomin/127.0.0.1, piomin/192.168.1.102, piomin/192.168.116.1, /192.168.226.1, /192.168.99.1]
>>> Local ports: TCP:8082 TCP:10801 TCP:11212 TCP:47101 UDP:47400 TCP:47501

Let's move back for a moment to the source code of our sample application. I assume you have already cloned a given repository from GitHub. There are two Maven modules available. The module ignite-rest-service is responsible for starting Ignite's cluster node in the Server node, while ignite-client-service for starting node in the Client node. Because we run only a single instance of the Client node, we would not override its default port set inside the application.yml file. You can build the project using the mvn clean install command and then start with java -jar or just run the main class IgniteClientApplication from your IDE.

There is also a JUnit test class inside module ignite-client-service, which defines one test responsible for calling HTTP endpoints (POST /person, POST /contact) that puts data into Ignite's cache. This test performs two operations: it puts some data into Ignite's in-memory cluster by calling endpoints exposed by the Client node and then checks if that data has been propagated through the cluster by calling the GET /person/{id}/withContacts endpoint exposed by one of the selected server nodes.

public class TestCluster {

  TestRestTemplate template = new TestRestTemplate();
  Random r = new Random();
  int[] clusterPorts = new int[] {8901, 8902};

  @Test
  public void testCluster() throws InterruptedException {
    for (int i=0; i<1000; i++) {
    Person p = template.postForObject("http://localhost:8090/person", createPerson(), Person.class);
    Assert.notNull(p, "Create person failed");
    Contact c1 = template.postForObject("http://localhost:8090/contact", createContact(p.getId(), 0), Contact.class);
    Assert.notNull(c1, "Create contact failed");
    Contact c2 = template.postForObject("http://localhost:8090/contact", createContact(p.getId(), 1), Contact.class);
    Assert.notNull(c2, "Create contact failed");
    Thread.sleep(10);
    Person result = template.getForObject("http://localhost:{port}/person/{id}/withContacts", Person.class, clusterPorts[r.nextInt(2)], p.getId());
    Assert.notNull(result, "Person not found");
    Assert.notEmpty(result.getContacts(), "Contacts not found");
    }
  }

  private Contact createContact(Long personId, int index) {
  ...
  }

  private Person createPerson() {
  ...
  }

}

Before running any tests, we should launch two additional elements being a part of our architecture: Ignite's web console and agent. The most suitable way to run Ignite's web console on the local machine is through its Docker image apacheignite/web-console-standalone. Here's Docker command that starts Ignite's web console and exposes it on port 80. Because I run Docker on Windows, it is now available under the default VM address http://192.168.99.100/.

docker run -d -p 80:80 -p 3001:3001 -v /var/data:/var/lib/mongodb --name ignite-web-console apacheignite/web-console-standalone

In order to access it, you should first register your user. Although the mail server is not available on the Docker container, you will be logged in after it. You can configure your cluster using Ignite's web console and also run some SQL queries on that cluster. Of course, we still need to connect our cluster consisting of three nodes with the instance of web console started on Docker container. To achieve it, you have to download a web agent. It is probably not very intuitive, but you have to click Start Demo, which is located in the right corner of Ignite's web console. Then, you will be redirected to the download page, where you can accept the download of the ignite-web-agent-2.4.0.zip file, which contains all needed libraries and configuration to start the web agent locally.

After downloading and unpacking the web agent, go to its main directory and change property server-uri to http://192.168.99.100 inside the default.properties file. Then, you may run the script ignite-web-agent.bat (or .sh if you are testing it on Linux), which starts the web agent. Unfortunately, it's not all that has to be done. Every server node's application should include the artifact ignite-rest-http in order to be able to communicate with the agent. It is responsible for exposing the HTTP endpoint that is accessed by a web agent. It is based on Jetty server, which causes some problems in conjunction with Spring Boot. Spring Boot sets default versions of Jetty libraries used inside the project. The problem is that ignite-rest-http requires older versions of that libraries, so we also have to override some default managed versions in the pom.xml file, according to the code below.

<dependencyManagement>
	<dependencies>
		<dependency>
			<groupId>org.eclipse.jetty</groupId>
			<artifactId>jetty-http</artifactId>
			<version>9.2.11.v20150529</version>
		</dependency>
		<dependency>
			<groupId>org.eclipse.jetty</groupId>
			<artifactId>jetty-server</artifactId>
			<version>9.2.11.v20150529</version>
		</dependency>
		<dependency>
			<groupId>org.eclipse.jetty</groupId>
			<artifactId>jetty-io</artifactId>
			<version>9.2.11.v20150529</version>
		</dependency>
		<dependency>
			<groupId>org.eclipse.jetty</groupId>
			<artifactId>jetty-continuation</artifactId>
			<version>9.2.11.v20150529</version>
		</dependency>
		<dependency>
			<groupId>org.eclipse.jetty</groupId>
			<artifactId>jetty-util</artifactId>
			<version>9.2.11.v20150529</version>
		</dependency>
		<dependency>
			<groupId>org.eclipse.jetty</groupId>
			<artifactId>jetty-xml</artifactId>
			<version>9.2.11.v20150529</version>
		</dependency>
	</dependencies>
</dependencyManagement>

After implementing the changes described above, we may finally proceed to run all the elements that are part of our sample system. If you start the Ignite web agent locally, it should automatically detect all running cluster nodes. Here's the screen with the logs displayed by the agent after startup.

At the same time, you should see that a new cluster has been detected by the Ignite web console.

You can configure a new or a currently existing cluster using the web console or just run a test query on the selected managed cluster. You have to include a name of the cache as a prefix to the table name when defining a query.

Similar queries have to be declared inside a repository interface. Here are additional methods used for finding entities stored in PersonCache. If you would like to include results stored in other caches, you have to explicitly declare its name together with the table name.

@RepositoryConfig(cacheName = "PersonCache")
public interface PersonRepository extends IgniteRepository {

  List findByFirstNameAndLastName(String firstName, String lastName);

  @Query("SELECT p.id, p.firstName, p.lastName, c.id, c.type, c.location FROM Person p JOIN \"ContactCache\".Contact c ON p.id=c.personId WHERE p.id=?")
  List<List> findByIdWithContacts(Long id);

  @Query("SELECT c.* FROM Person p JOIN \"ContactCache\".Contact c ON p.id=c.personId WHERE p.firstName=? and p.lastName=?")
  List selectContacts(String firstName, String lastName);

  @Query("SELECT p.id, p.firstName, p.lastName, c.id, c.type, c.location FROM Person p JOIN \"ContactCache\".Contact c ON p.id=c.personId WHERE p.firstName=? and p.lastName=?")
  List<List> selectContacts2(String firstName, String lastName);
}

We are nearing the end. Now, let's run our JUnit test TestCluster in order to generate some test data and put it into the clustered cache. You can monitor the size of a cache using the web console. All you have to do is to run the SELECT COUNT(*) query and set graph mode as the default mode for displaying results. The chart visible below illustrates the number of entities stored inside Ignite's cluster at 5s intervals.

Docker (software) Spring Framework Spring Boot Apache Ignite

Published at DZone with permission of Piotr Mińkowski, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • Spring Boot With Kubernetes
  • Advanced Kubernetes Setup for Spring Boot App With PostgreSQL DB
  • Containerization and Helm Templatization Best Practices for Microservices in Kubernetes
  • [CSF] Using Metrics In Spring Boot Services With Prometheus, Graphana, Instana, and Google cAdvisor

Partner Resources

×

Comments

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends: