{{announcement.body}}
{{announcement.title}}

Introducing Apache Ignite New Subsystems for Metrics and Monitoring

DZone 's Guide to

Introducing Apache Ignite New Subsystems for Metrics and Monitoring

Explore the Apache Ignite new monitoring opportunities and how to use different tools and technics to gather metrics for diagnosis.

· Database Zone ·
Free Resource

Apache Ignite as a distributed database and caching platform needs end-to-end monitoring to act on time. Historically, Apache Ignite provides a set of API and instrumentation to gather application-specific information and metrics by the external tools. In release 2.8.0, Apache Ignite improved the monitoring capabilities and introduced some nice features like "System views subsystem" and "Metrics subsystem."

In this short article, we are going to explore the Apache Ignite new monitoring opportunities and how to use different tools and technics to gather metrics for diagnosis. Anyway, the full release notes of version 2.8.0 can be found here.

As we mentioned before, in the 2.8.0 version, Apache Ignite out-of-the-box provides two new subsystems: the Metrics subsystem and System views subsystem. 

The Metrics subsystem gives the ability to use a few independent, pluggable metrics exporters for integration with different 3rd party monitoring tools such as Nagios, Zabbix, Grafana, etc.

On the other hand, the System views subsystem allows viewing a list of Ignite internal entities via several independent pluggable exporters.

An exporter is a special SPI implementation which exports metrics in different protocols.

For metrics subsystem, Apache Ignite provides the following exporters:

  1. JMX
  2. SQL View
  3. Log
  4. OpenCensus

For System views subsystem, only two exporters are available:

  1. JMX
  2. SQL

Note that, Exporters are configured during Ignite node startup and can't be changed in runtime.

Now, let's move to the practical part of the article. First, we are going to start an Ignite node, create a few entities with some dataset, and then use VisualVM and Ignite command line SQLCLI to gather metrics.

Before starting, ensure the JAVA_HOME environment variable is set and points to your JDK installation.

Having installed Java on our system, let’s install, and run an Apache Ignite node as follows:

Step 1. Download the distribution archive from the Ignite website.

Step 2. Extract the distribution archive in any directory you like and set an environment variable IGNITE_HOME to the created directory.

Step 3. Run the ignite.sh bash file in your favorite terminal. If you want to run an Ignite node with pre-configured caches, run the following command:

Shell
 




xxxxxxxxxx
1


 
1
ignite.sh $IGNITE_HOME/examples/config/example-cache.xml



Step 4. To connect SQLLINE to the Ignite node, run sqlline.sh -u jdbc:ignite:thin:{host} from your $IGNITE_HOME/bin directory. For instance:

Shell
 




xxxxxxxxxx
1


 
1
sqlline.sh --color=true --verbose=true -u jdbc:ignite:thin://127.0.0.1/



Step 5. Let's create two tables as shown in the next listing.

SQL
 




xxxxxxxxxx
1
11


 
1
CREATE TABLE City (
2
  id LONG PRIMARY KEY, name VARCHAR)
3
  WITH "template=replicated";
4
 
          
5
CREATE TABLE Person (
6
  id LONG, name VARCHAR, city_id LONG, PRIMARY KEY (id, city_id))
7
  WITH "backups=1, affinityKey=city_id";
8
 
          
9
CREATE INDEX idx_city_name ON City (name);
10
 
          
11
CREATE INDEX idx_person_name ON Person (name);



Step 6. Insert some example data into the tables as follows:

SQL
 




xxxxxxxxxx
1


1
INSERT INTO City (id, name) VALUES (1, 'Forest Hill');
2
INSERT INTO City (id, name) VALUES (2, 'Denver');
3
INSERT INTO City (id, name) VALUES (3, 'St. Petersburg');
4
 
          
5
INSERT INTO Person (id, name, city_id) VALUES (1, 'John Doe', 3);
6
INSERT INTO Person (id, name, city_id) VALUES (2, 'Jane Roe', 2);
7
INSERT INTO Person (id, name, city_id) VALUES (3, 'Mary Major', 1);
8
INSERT INTO Person (id, name, city_id) VALUES (4, 'Richard Miles', 2);



Step 7. Now, we are ready to query the data. Run the following query:

SQL
 




xxxxxxxxxx
1


 
1
SELECT p.name, c.name
2
FROM Person p, City c
3
WHERE p.city_id = c.id;



The above query will produce the following output.

Step 8. Now, Lunch the VisualVM application from the JDK bin directory. Note that, you can also run the VisualVM from the Ignitevisorcmd command line by using the following command.

Shell
 




xxxxxxxxxx
1


 
1
VVM



On the top-left corner of the application tab, you can see different options like Local, Remote and Snapshots. Select the org.apache.ignite.startup.cmdline.CommandLineStartup application from the Local section as shown below.

By default when Ignite node is started with ignite.sh!bat script, it picks up a random JMX port and binds to it. You can explicitly set the JMX port by setting the IGNITE_JMX_PORT environmental variable. In *nix system it can be done in the following way:

Note that, if you do not have the MBeans tab, it means that you have to install the MBeans plugin manually for VisualVM.

You can navigate through the next/previous button on the tabular value and check which SQL queries were executed before. Select the CACHES view, which will expose the information about the Ignite caches. In our cases, there are two user-defined caches and two system caches shown in the next screenshot. 

For now, you can select any view you want to exposes and monitor the Ignite system information which was executed before. Select the CACHES view, which exposes the detailed information about the Ignite caches. In our cases, there are two user-defined caches and two system caches.

In such a way, you can select any subsystem view you want to exposes and monitor the Ignite system information. Anyway, by using JConsole/VisualVM for monitoring Ignite node is useful for development or prototyping. Monitoring an Ignite cluster over 5 nodes by VisualVM or JConsole is unrealistic and time-consuming.

As we mentioned before, Ignite also provides SQL exporter for gathering metrics. Each system view exported as a SQL System view in the SYS schema.

Let's query the SQL history system view and observe which queries were executed before. All the SQL subsystem views are in the Ignite SYS schema. So you can query the SQL system view via Ignite SQLLINE CLI.

Step 9. Run the following SQL query to gather all the information about SQL subsystem views.

SQL
 




xxxxxxxxxx
1


 
1
Select NAME, SCHEMA, DESCRIPTION from SYS.VIEWS;


The above query should return the following output as shown in the following screenshot.

Let's query the SQL history system view and observe which queries were executed before. 

You can integrate all the above metrics exporters with different 3rd party monitoring tools for production information. If you are curious about how to use Grafana for monitoring the Ignite cluster, read the article which provides step-by-step instructions to install and configure the entire stack technology.

Anyway, if you want to learn more about the Apache Ignite, do not miss my book.

Topics:
apache ignite ,database ,gridgain ,jmx ,monitoring ,tutorial

Published at DZone with permission of Shamim Bhuiyan . See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}