Over a million developers have joined DZone.

Puppet and CloudStack

DZone's Guide to

Puppet and CloudStack

· Cloud Zone ·
Free Resource

Learn how to migrate and modernize stateless applications and run them in a Kubernetes cluster.

Curator's Note: The content of this article was originally written by David Nalley over at the Build a Cloud blog .

Efficient use of CloudStack really demands configuration management (among other things). I've been a puppet user for many years predating my involvement with CloudStack, and thus I have a slight bias for puppet.

Thus it thrilled me to no end when I saw folks like Jason Hancock doing work around automating configuration of CloudStack instances. Jason really knows Puppet, and even operates a new Hosted Puppetmaster startup. Jason showed this off a few times at both PuppetCamp LA 2012, and at the CloudStack Collaboration Conference in 2012.

It's awesome work, and you should take the time to watch both of his videos and check out his blog.

The gist of what he was presenting is configuring instance metadata within CloudStack at deployment, having the instance read that metadata and set it as a fact, and then using case statements to apply different roles to the instances.

And then there was a knife.....plugin

Next I learned that the good folks at Edmunds.com had written a CloudStack plugin for knife. That was exciting in and of itself, especially as a CloudStack person. It wasn't just knife, which is a command-line tool for Chef, another configuration management tool. knife is commonly used to provision machines, and the folks at Edmunds.com had baked in some extra awesomeness. They had the ability to define an application stack based on a JSON definition of what the stack looked like.

So one could define a Hadoop Cluster like this in JSON, complete with network and firewall configuration:
"name": "hadoop_cluster_a",
"description": "A small hadoop cluster with hbase",
"version": "1.0",
"environment": "production",
"servers": [
    "name": "zookeeper-a, zookeeper-b, zookeeper-c",
    "description": "Zookeeper nodes",
    "template": "rhel-5.6-base",
    "service": "small",
    "port_rules": "2181",
    "run_list": "role[cluster_a], role[zookeeper_server]",
    "actions": [
      { "knife_ssh": ["role:zookeeper_server", "sudo chef-client"] }
    "name": "hadoop-master",
    "description": "Hadoop master node",
    "template": "rhel-5.6-base",
    "service": "large",
    "networks": "app-net, storage-net",
    "port_rules": "50070, 50030, 60010",
    "run_list": "role[cluster_a], role[hadoop_master], role[hbase_master]"
    "name": "hadoop-worker-a hadoop-worker-b hadoop-worker-c",
    "description": "Hadoop worker nodes",
    "template": "rhel-5.6-base",
    "service": "medium",
    "port_rules": "50075, 50060, 60030",
    "run_list": "role[cluster_a], role[hadoop_worker], role[hbase_regionserver]",
    "actions": [
      { "knife_ssh": ["role:hadoop_master", "sudo chef-client"] },
      { "http_request": "http://${hadoop-master}:50070/index.jsp" }
And then deploying a Hadoop Cluster is as simple as:
    knife cs stack create hadoop_cluster_a
As a CloudStack guy I thought this was awesome, complex applications were suddenly deployable with ease, this is exactly the kind of automation that CloudStack is supposed to enable.


As a puppet afficionado though, it made me a bit sad, nothing existed like this for folks using puppet, and I was jealous.

Then I went to FOSDEM

At FOSDEM some folks from a mapping company were talking about their use of CloudStack, and commented that they wanted CloudStack resource types. At first, I was confused. They then showed me some work by Ken Barber on OpenNebula resource and types, and I realized that this was exactly what would fill the role of knife-cloudstack's stack functionality. It also would in many ways, truly be infrastructure as code. It would not just be configuring the OS and software on a machine, but it would be configuring the underlying machine itself, and indeed many machines. You could literally have most of your infrastructure as code.

That was awesome, but it was for OpenNebula, and not CloudStack - I started bugging folks like Teyo Tyree and Dan Bode about this.


Then I attended Puppetconf, and saw the inimitable Dan Bode showing a similar (albeit evolved) resource type for Google Compute Engine. I was even more envious at this point.

And then just after Thanksgiving - Dan Bode unveiled resources and types for Apache CloudStack.

Now you can use puppet not only to query CloudStack, but you can also define your infrastructure.
    cloudstack_instance { 'foo1':
      ensure     => present,
      flavor     => 'M1.Medium',
      zone       => 'FMT-ACS-001',
      image      => 'CentOS 6.3(64-bit)',
      network    => 'davidsnetwork',
As with other resource types, you can define defaults:
  image   => 'CentOS 6.3(64bit)',
  flavor  => 'M1.Medium',
  zone    => 'FMT-ACS-001',
  network => 'davidsnetwork',
  keypair => 'davidskeys',
Which makes defining instances much easier, and looking something like this:
cloudstack_instance { 'bar':
  ensure  => $::ensure,
  group   => 'role=db'
Of course, that's still a single machine, so if we want to have a multi-machine LAMP stack.
class my_web_stack {
  cloudstack_instance { 'httpd':
    ensure => present,
    group  => 'role=httpd',

  cloudstack_instance { 'mysql':
    ensure => present,
    group  => 'role=mysql',
Hopefully this leaves you as excited, as you can now puppetize ALL of your CloudStack instances.

Join us in exploring application and infrastructure changes required for running scalable, observable, and portable apps on Kubernetes.


Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}