Application Clustering For Scalability and High Availability

application clustering is a sub-topic of “parallel computing”. today many types of software support parallel computing in some form. why do we need parallel, clustered applications? for 2 reasons; scalability and high availability. does every application need clustering? no. however, if your system is underperforming and not available enough to meet customer requirements, then that signs that you need clustering. there are many kind of clustering at each software tier, please note that application clustering is different. a stand-alone application may not run smoothly in some types of clustered environment and needs some tweaks that i’ll share here in this article.

which benefits can we gain from clustering? there are 2 main benefits:

1. load balancing (scalability): by distributing processing, we can make it possible vertical or horizontal scalability. in this way, we can leverage resources more effectively. one example is java gc advantage. if we distribute user sessions to different jvms then gc would run less and performance would be better. also we can overcome jvm heap size limit.

2. fail-over (high availability): fail-over means less and shorter service interruption. some slas may require some high availability percentages like two-nines availability. a server may be down because of many reasons; system failures, planned outage, hardware or network problems etc. to achieve ha, one important technique is clustering. fail-over may occur automatically, or manually. we may prepare a redundant backup system (cold failover) or our load balanced servers may also serve the fail-over function (hot failover). please note that there is an engineering branch for this topic; reliability engineering. moreover, fail-over is not only an issue solved by clustering but it is also related with your system design and characteristics. if you design your applications against interruptions then it is called “fault-tolerant”. when dealing with this issue, we shouldn’t forget the recovery time (disaster recovery). you may fail but it is important how fast you can make system up. downtime is costly and its cost should be calculated to get better understanding of ha.

there are many kinds of clustering in different tiers. in every tier, it may be named differently meaning the same thing like virtualization, partitioning, mirroring. let me try to list clustering tiers:

1. hardware clustering: raid is such technology used for both performance and reliable disks.

2. os (server) virtualization: we know many os-level virtualization programs that can run many logical servers within same physical server at the same time.

3. db clustering: many dbms supports clustered database instances via different topologies. we can add jdbc-level cluster libraries to this category that simulate clustered database feature.

4. jvm clustering: some vendors transparently cluster jvm without knowledge of the running applications. some of them use byte-code instrumentation for this.

5. as clustering: application servers (and http servers) support clustering with varying capabilities. some of them can make session state replication across as instances. some of them have load-balancers to distribute coming requests. some of them can have transparent fail-over feature etc.

6. application clustering: this category is what i want to explore today. application clustering is parallel with application server clustering but not the same thing. if you don’t make your application cluster-enabled, it won’t make use of as clustering. if you configure your servers for clustering, you may still need to change your application code to run on these clustered servers.

there are some adjustments when preparing a stand-alone application to run in a clustered environment. briefly, some of your application services need to be reorganized for state integrity. these services are probably using singleton pattern. if you are building your application with some frameworks or libraries, they also must support clustering in a consistent way. otherwise, you won’t make us of clustering by yourself.

to keep state integrity, you may choose one of following architectures (some ass may already support this infrastructure):

state replication: you may have multiple service instances on cluster nodes and synchronize the state by replicating it. there are many replication techniques you can use; port communication, rmi, javaspaces. this architecture has advantage in fail-over and disadvantage in performance.

hub-and-spoke (master-slave): in this architecture, you would have only one instance of services on master cluster node and child nodes will invoke master node services through for instance rmi method calls. this architecture has advantage in performance but disadvantage in fail-over capability (if master server fails, child nodes would lose service states).

let’s delve into some services that must support clustering:

1. locking: application locks should be shared across different cluster nodes. if db locks are used then it is cluster-safe. if a singleton lock service is doing locking, then we need to make lock acquire-release calls among cluster nodes synchronously. for example; a user is running a process on as cluster node1 and hold a pessimistic lock on entity a, then another user’s process running on cluster node2 must receive an exception when trying to update entity a.

2. caches: every cache service should be analyzed if a cluster-enablement is required. some caches may repeat on cluster nodes but that may not cause any problem. on the other hand, let’s say we have a persistent object cache service that runs according to update information to tables and checks if any table modified. a process on another cluster may update a table and that change information should be reflected to all caches on the other cluster nodes.

3. events: you may have designed some type of application events and those may need to be cluster-aware. for example, you have a notification event that triggers when a user sends a message to another user. if that event occurs in node01 and other user is on node02, he would never receive that event and message if that event is not cluster-enabled.

4. identities: let’s say we have a sequence generator service that has a cache to prevent accessing database every time a number is requested. if identity service is not cluster-enabled, identity services on different cluster nodes would generate same identity numbers and applications would get duplicate record errors.

5. authentication: if user can login on each cluster node then some authentication information need to be shared. one example is a feature such that a user account is locked after multiple failed login tries. if failed attempts are not shared, a hacker may continue his attempts on other nodes. i don’t add authorization here since we didn’t need an adjustment for it to work in clustered environment (your authorization service may need it).

6. scheduling: scheduled or batched tasks should be considered when clustering. same scheduled processes may try to run on each instance whereas only one execution is enough.

7. logging (file io): you have to deal with file operations in a clustered environment. one file issue is log files. multiple cluster nodes may try to write the same log file and make it corrupt or cause io errors if not cluster-enabled. our solution for this is to generate and use different log files for each cluster node.

8. messaging (network io): if you are using some messaging services, you have to make sure that message service is cluster-enabled. if you are using a jms product, you need to test in a clustered environment whether it behaves appropriately. if you wrote it and you plan to use multiple messaging service instances, then you have to use different ports if cluster nodes are on the same server. we just used only one instance of messaging service to overcome that kind of problems.

9. a service that need to be clustered: since there are many application services beyond this list, you may have a special application service that need to be cluster-aware. you should provide an api to execute in your cluster architecture compatibly. we did so. we have a very easy api to register services to run on cluster nodes seamlessly without thinking state sharing. thus, any programmer can utilize that service without dealing with clustering issues and details.

it is very important to evaluate if a service really needs cluster-enablement. for example, ordinary persistent objects should not work in distributed and clustered architecture since it would be a very big overhead to run across network (add extra code addition to this cost). sometimes, by just cluster-enabling some services is good enough to utilize parallel computing.

a sample cluster topology:

links:
http://www.theserverside.com/tt/articles/content/j2eeclustering/article.html
http://www.javaworld.com/jw-02-2001/jw-0223-extremescale.html
http://www.javaworld.com/jw-08-2001/jw-0803-extremescale2.html
http://www.redbooks.ibm.com/abstracts/sg246153.html