Over a million developers have joined DZone.

Using the UltraESB to proxy and load balance requests to Tomcat

DZone's Guide to

Using the UltraESB to proxy and load balance requests to Tomcat

· Integration Zone ·
Free Resource

WSO2 is the only open source vendor to be named a leader in The Forrester Wave™: API Management Solutions, Q4 2018 Report. Download the report now or try out our product for free.


The UltraESB from AdroitLogic has deep support for the HTTP/S protocols, and the ability to load balance or fail over requests between multiple backend systems, using round-robin, weighted or random algorithms. Thus, it can even be used to proxy requests to Tomcat servers, and load balance/failover between multiple instances - instead of using Apache + mod_jk or mod_proxy etc.

Refer to the full article that describes the scenario, and compares a Proxy service definition on the UltraESB, and a similar approach with Apache2+mod_jk and compare the simplicity and the power of using the UltraESB yourself. Additionally, the UltraESB has excellent support for REST, SOAP, XML, JSON, Cookies, Baic/Digest/AWS S3 authentication etc that makes it easy to manage complex integrations.


The requirement is to load balance requests from clients between multiple Tomcat instances that host distributable web applications performing session replication. However, sticky sessions are desired so that a session fail-over will only take place on an unexpected failure of a Tomcat node.

A typical deployment using Apache2 Web Server with mod_jk


An equivalent deployment using the UltraESB

Configuration of Tomcat instances

For this example, we used two Tomcat 6.X servers on an Ubuntu platform, and configured session replication and clustering as per article:


The Tomcat1 server listens for HTTP on port 8080 and for AJP on port 8009, while Tomcat2 listens for HTTP on port 8081 and for AJP on port 8010. The server.xml for the Tomcat1 can be found here, while the server.xml for Tomcat2 can be found here. Note that we specified "tomcat1" as the jvmRoute for the first server, and "tomcat2" as the jvmRoute for the second server.

e.g. <Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat1">

The above appends this "jvmRoute" to the JSESSIONID cookie or the jsessionid path parameter when using URL re-writing, to allow stick sessions. Then we configured the Tomcat example application to use session replication by specifying the <distributable/> tag into the end of the web.xml (found at webapps/examples/WEB-INF) as follows:

<web-app .....    

You could now start the Tomcat instances, and directly access them at http://localhost:8080/examples To test session replication, the following scenario could help.

Access http://localhost:8080/examples/servlets/servlet/SessionExample this will return a page where you could set a session attribute. If you set attribute "a" as say "100", you will be returned to a page such as shown below. Note that the URL now contains the jvmRoute id of the Tomcat instance that handled the request and established the session. Since we accesses Tomcat1 over port 8080, we can see ".tomcat1" at the end of the session id as shown below.


If you now access Tomcat2 at http://localhost:8081/examples/servlets/servlet/SessionExample you will still see the attribute a=100 since the session has been replicated between the instances. Note that configuring details for Tomcat is out of scope for this article.

Configuration of the UltraESB

The example scenario can be found in the configuration file at samples/conf/ultra-sample-111.xml, and the salient point of this configuration is shown below. The Proxy service "web-proxy" defines the URL pattern it accepts as "*" and thus handles any request arriving over its transport "http-8280" configured to listen on HTTP port 8280. Once a request arrives, the mediation.getJvmRoute() method extracts the jvmRoute of a JSESSIONID cookie or jsessionid path segment, or returns null if one is not found. We then use conditional routing to send the request to endpoint definition "tc1-failover" or "tc2-failover" depending on the instance holding the active session, and if a session is not found, we use round-robin load balancing to select between one of the instances.

    <u:proxy id="web-proxy">
        <u:transport id="http-8280">
            <u:property name="url" value="*"/>
                    String jvmRoute = mediation.getJvmRoute(msg);
                    logger.debug("JVM Route : {}", jvmRoute);
                    if ("tomcat1".equals(jvmRoute)) {
                        mediation.sendToEndpoint(msg, "tc1-failover");
                    } else if ("tomcat2".equals(jvmRoute)) {
                        mediation.sendToEndpoint(msg, "tc2-failover");
                    } else {
                        mediation.sendToEndpoint(msg, "round-robin-loadbalance");
                <u:address type="response"/>

    <u:endpoint id="tc1-failover" type="fail-over">
        <u:address type="prefix">http://localhost:8080</u:address>
        <u:address type="prefix">http://localhost:8081</u:address>
        <u:property name="switchLocationHeadersTo" value="http://localhost:8280/"/>

    <u:endpoint id="tc2-failover" type="fail-over">
        <u:address type="prefix">http://localhost:8081</u:address>
        <u:address type="prefix">http://localhost:8080</u:address>
        <u:property name="switchLocationHeadersTo" value="http://localhost:8280/"/>

    <u:endpoint id="round-robin-loadbalance" type="round-robin-with-fail-over">
        <u:address type="prefix">http://localhost:8081</u:address>
        <u:address type="prefix">http://localhost:8080</u:address>
        <u:property name="switchLocationHeadersTo" value="http://localhost:8280/"/>

The "tc1-failover" endpoint performs fail-over processing, and always selects the first available address to forward the message to. If this endpoint fails, the next available address in the list is selected. Thus the "tc1-failover" always prefers Tomcat1 server first, while "tc2-failover" always prefers Tomcat2 server first. If the preferred instance fails, the request is sent to the other. The "round-robin-loadbalance" endpoint is a round-robin load balancer with fail-over, and is used to allocate a request to a node using round-robin policy. If an address on this fails, the request fails-over to the next. The UltraESB supports weighted, and random load balancing with and without failover. Finally, the switchLocationHeadersTo property of the endpoints instruct the UltraESB to re-write any Location headers (e.g. when used with REST or RESTful calls etc) to point to the UltraESB, instead of the individual Tomcat instances.

Advanced options

Although not shown in this example, the UltraESB allows HTTP level failures, SOAP faults etc from a server to cause a fail-over to another instance; as well as validation of a successful response by a custom ResponseValidator - before accepting it as a valid response to be sent back to the client. This allows the UltraESB to detect for example that a HTTP 200 response that states "Service is not active" to be identified as a failure and fail-over the request to another instance. Additionally, the UltraESB could easily read or write cookies, HTTP headers etc much easily.

Configuration of the Apache2 Web Server with mod_jk

On Ubuntu 9.04, installing Apache2 with mod_jk required the following commands:

sudo apt-get install apache2
sudo apt-get install libapache2-mod-jk

Next we created a mod_jk.conf and workers.properties file at /etc/apache2 as per this article, and edited the /etc/apache2/apache2.conf to comment Virtual hosts, and include the mod_jk.conf at the very end as shown below.

#Include /etc/apache2/sites-enabled/
Include /etc/apache2/mod_jk.conf

Testing the scenarios

Both the Apache2 proxying, and the UltraESB proxying performs load balancing and failover with sticky sessions as expected. To test against the Apache2 frontend, access http://localhost/examples/servlets/servlet/SessionExample and to test against the UltraESB, test against the URL http://localhost:8280/examples/servlets/servlet/SessionExample In both cases, killing one of the Tomcat servers owning the session will failover to the other instance without loss of session contents.

To start the sample # 111 configuration of the UltraESB, start it as follows, and start the two Tomcat servers as described above.

user@host:~/java/ultraesb-1.0.0/bin$ ./ultraesb.sh -sample 111

To make the testing more meaningful, you may edit the SessionExample Servlet at webapps/examples/WEB-INF/classes/SessionExample.java to report the Tomcat instance name in the heading as follows:

Now, if the original request was handled by "tomcat1", kill that instance, and retry the request - and you will be taken to the Tomcat2 instance as follows:


IAM is now more than a security project. It’s an enabler for an integration agile enterprise. If you’re currently evaluating an identity solution or exploring IAM, join this webinar.


Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}