Learn How to Configure the WSO2 Load Balancer for Auto Scaling
Join the DZone community and get the full member experience.
Join For FreeThis post assumes that the reader is
familiar at configuring the WSO2 Load Balancer without autoscaling, and
has configured the system already with the load balancer. Hence this
post focuses on setting up the load balancer with autoscaling. If you
are a newbie to setting up WSO2 Servers proxied by WSO2 Load Balancer,
please read the blog post, How to setup WSO2 Elastic Load Balancer to configure WSO2 Load Balancer without autoscaling.
autoscaler.xml
The autoscaling configurations are defined from CARBON_HOME/repository/deployment/server/synapse-configs/tasks/autoscaler.xml1) Task Definition
In WSO2 Load Balancer, the
autoscaling algorithm to be used is defined as a Task.
ServiceRequestsInFlightEC2Autoscaler is the default class that is used
for the autoscaler task.
<task xmlns="http://ws.apache.org/ns/synapse" class="org.wso2.carbon.mediator.autoscale.ec2autoscale.ServiceRequestsInFlightEC2Autoscaler" name="autoscaler">
2) loadbalancer.xml pointed from autoscaler.xml
This property points to the file loadbalancer.xml for further autoscaler configuration.
This property points to the file loadbalancer.xml for further autoscaler configuration.
<property name="configuration" value="$system:loadbalancer.xml"/>
3) Trigger Interval
The autoscaling task is triggered based on the trigger interval that is defined in the autoscaler.xml. This is given in seconds.
The autoscaling task is triggered based on the trigger interval that is defined in the autoscaler.xml. This is given in seconds.
<trigger interval="5"/>
Autoscale Mediators
autoscaleIn
and autoscaleOut mediators are the mediators involved in autoscaling
as we discussed above. As with the other synapse mediators, the
autoscaling mediators should be defined in the main sequence of the
synapse configuration, if you are going to use autoscaling. Load
Balancer-1.0.x comes with these mediators defined at the main sequence,
which can be found at
$CARBON_HOME/repository/deployment/server/synapse-configs/sequences/main.xml.
Hence you will need to modify main.xml, only if you are configuring the load balancer without autoscaling.
autoscaleIn mediator is defined as an in mediator. It gets the configurations from loadbalancer.xml, which is the single file that should be configured for autoscaling, once you have already got a system that is set up for load balancing.
<autoscaleIn configuration="$system:loadbalancer.xml"/>
Similarly autoscaleOut mediator is defined as an out mediator.
<autoscaleOut/>
loadbalancer.xml
loadbalancer.xml
contains the service cluster configurations for the respective
services to be load balanced and the load balancer itself. Here the
service-awareness of the load balancer makes it possible to manage the
load across multiple service clusters. The properties given in
loadbalancer.xml is used to provide the required configurations and
customizations for autoscaling and load balancing. These configurations
can also be taken from the system properties as shown below.
1) Properties common for all the instances
1.1) ec2AccessKey
The property 'ec2AccessKey' is used to provide the EC2 Access Key of the instance.
The property 'ec2AccessKey' is used to provide the EC2 Access Key of the instance.
<property name="ec2AccessKey" value="${AWS_ACCESS_KEY}"/>1.2) ec2PrivateKey
The certificate is defined by the properties 'ec2PrivateKey'.
<property name="ec2PrivateKey" value="${AWS_PRIVATE_KEY}"/>1.3) sshKey
The ssh key pair is defined by 'sshKey'.
<property name="sshKey" value="stratos-1.0.0-keypair"/>1.4) instanceMgtEPR
'instanceMgtEPR' is the end point reference of the web service that is called for the management of the instances.
<property name="instanceMgtEPR" value="https://ec2.amazonaws.com/"/>1.5) disableApiTermination
The
'disableApiTermination' property is set to true by default, and is
recommended to leave as it is. This prevents terminating the instances
via the AWS API calls.
<property name="disableApiTermination" value="true"/>
1.6) enableMonitoring
The 'enableMonitoring' property can be turned on, if it is preferred to monitor the instances.
<property name="enableMonitoring" value="false"/>
2) Configurations for the load balancer service group
These are defined under
<loadBalancer> .. </loadBalancer>2.1) securityGroups
The service group that
the load balancer belongs to is defined by the property
'securityGroups'. The security group will differ for each of the
service that is load balanced as well as the load balancers. Autoscaler
uses this property to identify the members of the same cluster.
<property name="securityGroups" value="stratos-appserver-lb"/>
2.2) instanceType
'instanceType' defines the EC2 instance type of the instance - whether they are m1.small, m1.large, or m1.xlarge (extra large).
'instanceType' defines the EC2 instance type of the instance - whether they are m1.small, m1.large, or m1.xlarge (extra large).
<property name="instanceType" value="m1.large"/>2.3) instances
The
property, 'instances' defines the number of the load balancer
instances. Multiple load balancers are used to prevent the single point
of failure - by providing a primary and a secondary load balancer.
<property name="instances" value="1"/>2.4) elasticIP
Elastic
IP address for the load balancer is defined by the property,
'elasticIP'. We will be able to access the service, by accessing the
elastic IP of the load balancer. The load balancer picks the value of
the elastic IP from the system property ELASTIC_IP.
<property name="elasticIP" value="${ELASTIC_IP}"/>In a public cloud, elastic IPs are public (IPV4) internet addresses, which is a scarce resource. Hence it is recommended to use the elastic IPs only to the load balancer instances that to be exposed to the public, and all the services that are communicated private should be associated to private IP addresses for an efficient use of this resource. Amazon EC2 provides 5 IP addresses by default for each customer, which of course can be increased by sending a request to increase elastic IP address limit.
2.5) availabilityZone
This defines in which availability zone the spawned instances should be.
<property name="availabilityZone" value="us-east-1c"/>2.6) payload
The file that is defined by 'payload' is uploaded to the spawned instances. This is often a zip archive, that extracts itself into the spawned instances.
<property name="payload" value="/mnt/payload.zip"/>payload.zip contains the necessary files such as the public and private keys, certificates, and the launch-params (file with the launch parameters) to download and start a load balancer instance in the spawned instances.
The launch-params includes the details for the newly spawned instances to function as the other instances. More information on this can be found from the EC2 documentations.
Sample Launch Parameters
Given below is a sample launch-params, that is used in StratosLive by the load balancer of the Application Server service.
AWS_ACCESS_KEY_ID=XXXXXXXXXXXX,AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx, AMI_ID=ami-xxxxxxxxx,ELASTIC_IP=xxx.xx.xxx.xxx, PRODUCT_MODIFICATIONS_PATH_S3=s3://wso2-stratos-conf-1.5.2/appserver/, COMMON_MODIFICATIONS_PATH_S3=s3://wso2-stratos-conf-1.5.2/stratos/, PRODUCT_PATH_S3=s3://wso2-stratos-products-1.5.2,PRODUCT_NAME=wso2stratos-as-1.5.2, SERVER_NAME=appserver.stratoslive.wso2.com, HTTP_PORT=9763,HTTPS_PORT=9443,STARTUP_DELAY=0;60
We will look more into these launch-params now.
Credentials
The credentials - access key ID and the secret access key are given to access the aws account.
S3 Locations
The service zip and the common modifications or patches are stored in an S3 bucket. The locations are given by a few properties in the launch-params shown above.
- PRODUCT_MODIFICATIONS_PATH_S3 - Points to the product specific changes, files, or patches are uploaded to a specific location.
- COMMON_MODIFICATIONS_PATH_S3 - Points to the patches and changes common to all the servers.
- PRODUCT_PATH_S3 - Points to the location where the relevant Stratos service zips are available.
- STARTUP_DELAY - Given in seconds. Provides some time to start the service that is downloaded on the newly spawned instance, such that it will join the service cluster and be available as a new service instance.
Apart from these, PRODUCT_NAME, SERVER_NAME, HTTP_PORT, and HTTPS_PORT for the application are also given.
3) Configurations for the application groups
These are defined under
<services> .. </services>
These too should be configured as we configured the properties for the load balancers above.
We define the default values of the properties for all the services under
We define the default values of the properties for all the services under
<defaults> .. </defaults>Some of these properties - such as the payload, host, and domain - will be specific to a particular service group, and should be defined separately for each of the services, under
<service> .. </service>Properties applicable to all the instances
payload, availabilityZone, securityGroups, and instanceType are
a few properties that are not specific to the application instances.
We have already discussed about these properties when setting the load
balancer properties above.
Properties specific to the application instances
These
properties are specific to the application clusters, and are not
applicable to the load balacer instances. We will discuss about these
properties now.
3.1) minAppInstances
The
property 'minAppInstances' shows the minimum of the application
instances that should always be running in the system. By default, the
minimum of all the application instances are set to 1, where we may go
for a higher value for the services that are of high demand all the
time, such that we will have multiple instances all the time serving the
higher load.
<property name="minAppInstances" value="1"/>3.2) maxAppInstances
'maxAppInstances'
defines the upper limit of the application instances. The respective
service can scale up till it reaches the number of instances defined
here.
<property name="maxAppInstances" value="5"/>
3.3) queueLengthPerNode
Here we also define the properties such as payload and availabilityZone, if they differ from the default values provided under
3.7) hosts
The property 'queueLengthPerNode' provides the maximum length of the message queue per node.
<property name="queueLengthPerNode" value="400"/>3.4) roundsToAverage
The
property 'roundsToAverage' indicates the number of attempts to be made
before the scaling the system up or down. When it comes to scaling
down, the algorithm makes sure that it doesn't terminate an instance
that is just spawned. This is because the spawned instances are billed
for an hour. Hence, even if we don't have much load, it makes sense to
wait for a considerable amount (say 58 minutes) of time before
terminating the instances.
<property name="roundsToAverage" value="10"/>3.5) instancesPerScaleUp
This
defines how many instances should be scaled up for each time. By
default, this is set to '1', such that a single instance is spawned
whenever the system scales. However, this too can be changed such that
multiple instances will be spawned each time the system scales up.
However it may not be cost-effective to set this to a higher value.
<property name="instancesPerScaleUp" value="1"/>3.6) messageExpiryTime
messageExpiryTime defines how long the message can stay without getting expired.
<property name="messageExpiryTime" value="60000"/>Properties specific to a particular service group
Properties
such as hosts and domain are unique to a particular service group,
among all the service groups that are load balanced by the given load
balancer. We should note that we can use a single load balancer set up
with multiple service groups, such as Application Server, Enterprise
Service Bus, Business Process Server, etc.
Here we also define the properties such as payload and availabilityZone, if they differ from the default values provided under
<defaults> .. </defaults>
Hence these properties should be defined under
<service>.. </service>
for each of the services.
3.7) hosts
'hosts'
defines the hosts of the service that to be load balanced. These will
be used as the access point or url to access the respective service.
Multiple hosts can be defined under
<hosts> .. </hosts>
Given below is a sample hosts configurations for the application server service
<hosts> <host>appserver.cloud-test.wso2.com</host> <host>as.cloud-test.wso2.com</host> </hosts>3.8) domain
Like
the EC2 autoscaler uses the security groups to identify the service
groups, 'domain' is used by the load balancer
(ServiceDynamicLoadBalanceEndpoint) to correctly identify the clusters
of the load balanced services.
<domain>wso2.manager.domain</domain>
Once you have configured the load balancer as above, with the
product/service instances, you will have the system that dynamically
scales.
Load balancing (computing)
Web Service
Property (programming)
application
Scaling (geometry)
Autoscaling
Mediator (software)
cluster
Host (Unix)
AI
Opinions expressed by DZone contributors are their own.
Comments