Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Setting Up a Failover Scenario Using Apache Camel ZooKeeper

DZone's Guide to

Setting Up a Failover Scenario Using Apache Camel ZooKeeper

In this article, we'll go over how you, as a developer, can use the open source Camel ZooKeeper to deal with this annoying scenario.

Free Resource

Download Microservices for Java Developers: A hands-on introduction to frameworks and containers. Brought to you in partnership with Red Hat.

In this article, we will discuss using the Apache Camel ZooKeeper component, and demonstrate how easily we can set up a fail-over scenario for Apache Camel Routes. While working in a clustered environment, situations arise where a user wants to have a backup or slave route which will become active only when the master (or the currently active) route stops working. There are different ways to achieve this: one can use Quartz to configure a master-slave setup; JGroups can also be used. In a Fabric8 environment, there is a master component which can easily be set up as a failover scenario.

There is one more way to discuss here in detail using ZooKeeperRoutePolicy. This policy uses a common znode (zookeeper registry) path across all instances. Each instance will be registered with a unique-id in the znode path; thus there will be a list of unique-ids. Each unique-id represents one instance of a route deployed with ZooKeeperRoutePolicy route policy.

Whichever node is at the top of the list will be considered the master route. Other remaining nodes will be dormant and not active. Once the master route stops, one of the slaves will be elected as the master route. Then it will continue to serve requests.

Setting Up the Project and Configuring Camel Zookeeper as the Master-Slave Route

Now let's discuss the coding portion. I have tested this example in Red Hat JBoss Fuse 6.3.0. You can find the code base at my personal GitHub link.

1. Create a Maven project with following structure:

[cpandey@cpandey CamelZookeeperMasterSlave]$ tree
.
├── pom.xml
├── ReadMe.txt
└── src
 └── main
  ├── java
   └── com
    └── test
     └── pckg
      └── MyRouteBuilder.java
  └── resources
   ├── log4j.properties
   └── OSGI-INF
    └── blueprint
     └── blueprint.xml

2. The pom.xml should have following dependencies:

 <dependency>
 <groupId>org.apache.camel</groupId>
 <artifactId>camel-core</artifactId>
 </dependency>
 <dependency>
 <groupId>org.apache.camel</groupId>
 <artifactId>camel-blueprint</artifactId>
 </dependency>
 <dependency>
 <groupId>org.apache.camel</groupId>
 <artifactId>camel-zookeeper</artifactId>
 </dependency>

3. A version of these artifacts are picked from the BOM version mentioned in pom.xml:

 <dependencyManagement>
 <dependencies>
 <dependency>
 <groupId>org.jboss.fuse.bom</groupId>
 <artifactId>jboss-fuse-parent</artifactId>
 <version>6.3.0.redhat-310</version>
 <type>pom</type>
 <scope>import</scope>
 </dependency>
 </dependencies>
 </dependencyManagement>

4. Here, I have used JAVA DSL for the Camel route. Hence in blueprint.xml, I have defined a bean, which refers to the class com.test.pckg.MyRouteBuilder where the Camel route is defined.

<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.osgi.org/xmlns/blueprint/v1.0.0 https://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd">

 <camelContext id="_context1" xmlns="http://camel.apache.org/schema/blueprint">
  <routeBuilder ref="myBuilder"/>
 </camelContext>
 <bean class="com.test.pckg.MyRouteBuilder" id="myBuilder"/>
</blueprint>

5. The following camel route is defined in MyRouteBuilder.java:

@Override
public void configure() {
System.out.println("......Inside MyRouteBuilder.....");
ZooKeeperRoutePolicy policy = new ZooKeeperRoutePolicy("zookeeper:localhost:" + 2181 + "/someapp/somepolicy", 1);
from("file:/path/to/someFolder").routePolicy(policy).id("Master").log("Body... ${body} "+new Date());
}

The tasks performed by this piece of code are:

  • Define a ZooKeeperRoutePolicy(String uri, int enabledCount) which creates a znode at path the /someapp/somepolicy. The ZooKeeper Server is listening at localhost:2181.
  • We want to have one instance active at a time. For a master-slave scenario, the route is configured with one route instance and only the first entry in the listing will start its route. However, if there is a scenario when we need two or more route instances to be up at the same time, we can set enabledCount to 2 or more.
  • We then have a file based consumer polling at path/to/someFolder where routePolicy is set to ZooKeeperRoutePolicy. Finally, we log the body which is set with the content of the file polled or consumed.

How to Deploy in Red Hat JBoss Fuse With a Fabric8 Environment

1. Fabric8 uses Apache ZooKeeper (which is a highly reliable distributed coordination service) as its registry to store the cluster configuration and node registration. Hence, we don't need to setup a different ZooKeeper server. We can register znode in an existing ZooKeeper registry and use it to set up master-slave route.

2. Follow the steps below to deploy this application from the Karaf terminal of Red Hat JBoss Fuse:

profile-edit --feature fabric-zookeeper-commands default
container-create-child root abc
container-create-child root pqr 
profile-create testZK
profile-edit --feature camel-zookeeper --feature camel-blueprint testZK
profile-edit --bundle mvn:com.mycompany/camel-zookeeper-example/1.0.0-SNAPSHOT testZK
container-add-profile abc testZK
container-add-profile pqr testZK 

3. As per our route, at location /path/to/someFolder, create a simple text file and add some text data to it and save it.

4. In the logs for one of the nodes, we can check following, which will ensure that znode is created and that the route is active:

2018-03-25 17:30:30,075 | WARN | masterSlaveRoute | ZookeeperProducer | 191 - org.apache.camel.camel-core - 2.17.0.redhat-630329 | Node '/someapp/somepolicy/localhost.localdomain-cd65c741-4252-420a-88a4-f084a933fcaf' did not exist, creating it.
2018-03-25 17:30:30,080 | INFO | masterSlaveRoute | ZooKeeperElection | 195 - org.apache.camel.camel-zookeeper - 2.17.0.redhat-630329 | Candidate node '/someapp/somepolicy/localhost.localdomain-cd65c741-4252-420a-88a4-f084a933fcaf' has been created
2018-03-25 17:30:30,107 | INFO | masterSlaveRoute | BlueprintCamelContext | 191 - org.apache.camel.camel-core - 2.17.0.redhat-630329 | Route: election-route-localhost.localdomain-cd65c741-4252-420a-88a4-f084a933fcaf started and consuming from: Endpoint[zookeeper://localhost:2181/someapp/somepolicy]
2018-03-25 17:30:30,112 | INFO | masterSlaveRoute | Master | 191 - org.apache.camel.camel-core - 2.17.0.redhat-630329 | Body... Sun Mar 25 17:27:32 IST 2018

4. Using the following command from the Karaf terminal, we can easily identify which nodes are registered. This unique number we can be identified in logs also:

zk:list -r /someapp/somepolicy
#Output we get is like
/someapp/somepolicy/localhost.localdomain-cd65c741-4252-420a-88a4-f084a933fcaf0000000013

5. If the route or application is stopped, this znode entry will be deleted.

How to Deploy in Red Hat JBoss Fuse in a Standalone Environment

1. Download ZooKeepers latest release from here. I downloaded zookeeper-3.4.11.

2. Extract zookeeper-3.4.11.tar.gz. Now go to zookeeper-3.4.11/conf folder:

$ vi conf/zoo.cfg

tickTime = 2000
initLimit = 10
syncLimit = 5
dataDir = /path/to/zookeeper/data
clientPort = 2181

3. Finally, start the ZooKeeper server:

$ bin/zkServer.sh start

4. Now onthe Red Hat JBoss Fuse 6.3.0 Karaf terminal, run the following commands:

features:install camel-zookeeper
osgi:install -s mvn:com.mycompany/camel-zookeeper-example/1.0.0-SNAPSHOT

5. As per our route, at location /path/to/someFolder, create a simple text file and add some text data to it and save it. We will get similar logs (as mentioned above) for deployment in the Fabric8 environment. Here we can have multiple Red Hat JBoss Fuse standalone environments running at different nodes which have the same application deployment.

6. If deployment and testing are done, you can stop the ZooKeeper server with the following command:

bin/zkServer.sh stop

That's it. I hope this helps you to to set up a master-slave scenario with Camel routes. This might help you to achieve a successful failover scenario.

Download Building Reactive Microservices in Java: Asynchronous and Event-Based Application Design. Brought to you in partnership with Red Hat

Topics:
integration ,apache camel ,failover

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}