How to Quickly Launch a Cassandra Cluster on Amazon EC2
Join the DZone community and get the full member experience.
Join For Freecurator's note: this post is from 2011, so comment if you see any details that need updating!
if you have read my previous post, “ map-reduce with ruby using hadoop “, then you will know that firing up a hadoop cluster is really simple when you use whirr. without even ssh’ing on the machines in the cloud you can start-up your cluster and interact with it. in this post i’ll show you that it is just as easy to fire up a cassandra cluster on amazon ec2.
install whirr
i will fly through the setup of whirr quite quickly. all the commands you need are here, but if you want a more thorough explanation then see my other post, “ map-reduce with ruby using hadoop “.
i am assuming that you have homebrew installed .
sudo brew update sudo brew install maven mkdir ~/src/cloudera cd ~/src/cloudera wget http://archive.cloudera.com/cdh/3/whirr-0.1.0+23.tar.gz tar -xvzf whirr-0.1.0+23.tar.gz cd whirr-0.1.0+23 mvn clean install mvn package -ppackage
be patient with the above. there is a lot to install, so it will take some time. maven installs a lot of dependencies if it is your first time using it.
the good news is that from here on you are setup to easily fire-up your amazon ec2 cluster for cassandra, or hadoop if you choose.
whirr configuration file
we will need to make a configuration file for whirr to tell it that we want to launch a cassandra cluster with 3 nodes. if you are brave, patient and have the cash, then you could just as easily fire-up a 100 node cluster (leave a comment if you do – there may be prizes!).
you will need to create a cassandra.properties file with the following contents…
whirr.service-name=cassandra whirr.cluster-name=mycassandracluster whirr.instance-templates=3 cassandra whirr.provider=ec2 whirr.identity=<your_amazon_ec2_access_key_id_goes_here> whirr.credential=<your_amazon_ec2_secret_access_key_goes_here> whirr.private-key-file=${sys:user.home}/.ssh/id_rsa
replace the obvious fields with your amazon ec2 access key id and amazon ec2 secret access key.
launch your cluster
now you are ready to fire-up your cassandra cluster. simply use the following command and then be prepared to wait 5-10 minutes while amazon builds your machines. this time is variable. sometimes amazon is quick, sometimes not so quick.
bin/whirr launch-cluster --config cassandra.properties launching mycassandracluster cluster configuring template starting 3 node(s) nodes started: [[id=us-east-1/i-13f25e7f, providerid=i-13f25e7f, tag=mycassandracluster, name=null, location=[id=us-east-1d, scope=zone, description=us-east-1d, parent=us-east-1], uri=null, imageid=us-east-1/ami-74f0061d, os=[name=null, family=amzn-linux, version=2010.11.1-beta, arch=paravirtual, is64bit=true, description=amazon/amzn-ami-2010.11.1-beta.x86_64-ebs], usermetadata={}, state=running, privateaddresses=[10.204.99.163], publicaddresses=[50.16.155.106], hardware=[id=t1.micro, providerid=t1.micro, name=t1.micro, processors=[[cores=1.0, speed=1.0]], ram=630, volumes=[[id=vol-1657d47e, type=san, size=null, device=/dev/sda1, durable=true, isbootdevice=true]], supportsimage=hasrootdevicetype(ebs)]], [id=us-east-1/i-17f25e7b, providerid=i-17f25e7b, tag=mycassandracluster, name=null, location=[id=us-east-1d, scope=zone, description=us-east-1d, parent=us-east-1], uri=null, imageid=us-east-1/ami-74f0061d, os=[name=null, family=amzn-linux, version=2010.11.1-beta, arch=paravirtual, is64bit=true, description=amazon/amzn-ami-2010.11.1-beta.x86_64-ebs], usermetadata={}, state=running, privateaddresses=[10.117.43.129], publicaddresses=[50.16.85.79], hardware=[id=t1.micro, providerid=t1.micro, name=t1.micro, processors=[[cores=1.0, speed=1.0]], ram=630, volumes=[[id=vol-1457d47c, type=san, size=null, device=/dev/sda1, durable=true, isbootdevice=true]], supportsimage=hasrootdevicetype(ebs)]], [id=us-east-1/i-11f25e7d, providerid=i-11f25e7d, tag=mycassandracluster, name=null, location=[id=us-east-1d, scope=zone, description=us-east-1d, parent=us-east-1], uri=null, imageid=us-east-1/ami-74f0061d, os=[name=null, family=amzn-linux, version=2010.11.1-beta, arch=paravirtual, is64bit=true, description=amazon/amzn-ami-2010.11.1-beta.x86_64-ebs], usermetadata={}, state=running, privateaddresses=[10.117.46.170], publicaddresses=[184.73.100.203], hardware=[id=t1.micro, providerid=t1.micro, name=t1.micro, processors=[[cores=1.0, speed=1.0]], ram=630, volumes=[[id=vol-e857d480, type=san, size=null, device=/dev/sda1, durable=true, isbootdevice=true]], supportsimage=hasrootdevicetype(ebs)]]] authorizing firewall running configuration script completed launch of mycassandracluster started cluster of 3 instances cluster{instances=[instance{roles=[cassandra], publicaddress=/50.16.85.79, privateaddress=/10.117.43.129}, instance{roles=[cassandra], publicaddress=/50.16.155.106, privateaddress=/10.204.99.163}, instance{roles=[cassandra], publicaddress=/184.73.100.203, privateaddress=/10.117.46.170}], configuration={}}
you now have your very own cassandra cluster running in the cloud. not so hard, hey!
connect from ruby
i will be following this post with step-by-step guide on how you can interact with your new cluster from your ruby on rails application. i recommend subscribing to the rss feed to get updates to the blog.
shutdown the cluster
here is how you can shutdown your cluster.
bin/whirr destroy-cluster --config cassandra.properties destroying mycassandracluster cluster cluster mycassandracluster destroyed
conclusion
whirr makes it very easy to start and stop a cassandra cluster in the cloud without leaving the comfort of your laptop. what you do with that cluster is up to you, but i will be give you some ideas of what you could do in future posts.
Published at DZone with permission of Phil Whelan, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.
Comments