There is Safety in Numbers: How to Build Fail-Safe Clusters

Judah Johns

Apr. 10, 12 · Interview

Like (0)

Save

10.96K Views

very often, websites can't cope with any extra load, whether from high traffic or performance reducing tasks. as a result, companies lose money, as well as taking a reputation hit. programmers blame system administrators. system administrators blame managers. managers blame service providers. and vice versa. in the end, in this day and age, you can't afford down time: it will hurt your bottom line. so, how do you solve this issue?

one of the best solutions for such issues is using a cluster architecture for your applications. we'll show you how to create fail-safe clusters within glassfish and within jelastic .

1. background

you need two hardware nodes, n1 and n2 , for glassfish servers and then machine lb1 for load balancing and machine db1 for the database. make sure, that all machines can "see" each other (ping).

all software will work under linux control. also we'll use a preset image of a virtual machine with centos installed. glassfish uses ssh to control remote nodes. that's why we have to install ssh-server on each machine of the cluster. moreover we need separate accounts in every os (specify their names, for example: glassfish).

# adduser glassfish

# passwd glassfish

glassfish uses nonstandard ports, so you have to specify required rules in the firewall or just switch it off. let's choose the easiest variant:

# service iptables save

# service iptables stop

# chkconfig iptables off

and of course, install jvm .

unfortunately, there are a lot of configurations and settings, when you create a cluster manually. so, be patient!

2. configure the cluster

we configure the cluster via command line with the asadmin utility.

1. download glassfish distribution

wget download.java.net/glassfish/3.1.1/release/glassfish-3.1.1.zip

and unzip the package.

2. start the administrative domain instance (domain1):

./asadmin start-domain domain1

glassfish disables remote connections to the admin domain. that's why all cluster nodes, except das nodes, can't cooperate with it.

typical error:

failed to rendezvous with das on n1:4848. please check if this server is running, that the host and port are correct, and that this server is configured to allow remote access.

to fix it type the next string:

./asadmin enable-secure-admin

and restart administrative domain.

create your cluster:

./asadmin create-cluster c1

create two application servers on this hardware node:

./asadmin create-local-instance --cluster c1 i1 ./asadmin create-local-instance --cluster c1 i2

3. start the cluster:

./asadmin start-cluster c1

4. add new instances on the second hardware node :

./asadmin –host n1 –port 4848 create-local-instance –cluster c1 i3 ./asadmin –host n1 –port 4848 create-local-instance –cluster c1 i4

5. go to glassfish admin page, find the instance you have just created, change its type to ssh and change node host to n2 . also you have to specify the login and password for ssh access to machine n2 . as you can see below, all the clusters are working now:

3. load balancing and ha

1. install nginx :

#yum install nginx #nano /etc/nginx/nginx.conf

2. set round-robbin balancing for 4 servers on nginx with sticky session support:

upstream backend { ip_hash; server 192.168.0.1:28080 max_fails=5 fail_timeout=15s; server 192.168.0.1:28081 max_fails=5 fail_timeout=5s; server 192.168.0.2:28082 max_fails=5 fail_timeout=5s; server 192.168.0.2:28083 max_fails=5 fail_timeout=5s; }

3. hosts n1 and n2 should have their common parent domain, for example: cluster.com. edit the files /etc/hosts on both machines:

n1: 127.0.0.1 localhost 127.0.1.1 n1.cluster.com 127.0.0.1 n1.cluster.com 192.168.0.2 n2.cluster.com n2: 127.0.0.1 localhost 127.0.1.1 n2.cluster.com 127.0.0.1 n2.cluster.com 192.168.0.1 n1.cluster.com

4. check multicast on each hardware node:

./asadmin validate-multicast

5. change n2 to n2.cluster.com in das admin console. 6. web-descriptor of your application should contain a suitable instruction ( <distributable> ) for providing replication of http-sessions. don't forget to set the availability flag, when you deploy your app.

4. configure database

we'll use postgresql as an example.

install the database, create a new user and database:

# yum install postgresql-8.4 # service postgresql initdb # service postgresql start

specify allowed subnets in the pg_hba.conf (/var/lib/pgsql/data/pg_hba.conf) file :

host all all 192.168.0.0/24 md5

5. application deployment

now you can deploy your application (for example: jforum ).

obviously, this is time consuming!

an easier way is to do this in jelastic.

you can create fail-safe clusters with just a few clicks from within the dashboard:

if you found this article useful, please let me know.

cluster Build (game engine)

Opinions expressed by DZone contributors are their own.

Comments