Turning a 5 Hour Manual Build and Deploy Routine Into a Single Code Commit - Part 1

I aim to deliver as a series of tutorials will help you move towards full testing, integration and deployment of your systems and turn what could be a five hour manual build and deploy routine into a single code commit.

The Toolbox

Whilst a bad workman may well always blame his tools, a good workman recognises the best tools out there and makes sure he’s using them.  This series will focus on the tools that I have found to be best for me, however (as always!) your mileage may vary.

Puppet

Written in ruby and with a very strong community around it, puppet is a centralised way of distributing and maintaining system state in a secure manner.

Git

Git is a distributed Source Coded Management system.  I find it works nicely for me as it means I can run local tests on check-in before pushing those changes to the master server and having the build system run against my code.

The Marionette Collective

The Marionette Collective (or MCollective for short) is the brain-child of R.I. Pienaar and is now owned by Puppet Labs.  It uses messaging systems to allow you to query and command remote servers without needing to log into them.  It also integrates nicely with Puppet.

Jenkins

Jenkins is a Continuous Integration (CI) Server.  If you’ve not used a CI server before, just think of it as a task scheduler with built-in reporting.  As we progress, you’ll see that it’s much more than that, however for the time being, that description will suffice.

Capistrano

Traditionally a tool used for deploying Ruby on Rails Applications, Capistrano is a tool to take code from a Source Code Management tool such as Git or Subversion and place it in a controlled, repeatable manner.  Capistrano also has an automated roll-back facility enabling us to deploy and roll-back our configurations to previous versions.

I’m going to assume that you either know how to setup all of the above or that you know how to use Google.  I’d suggest starting with GIT and Puppet before moving on to Jenkins with Capistrano and MCollective as the final step as this is the order that I’ll address them here.  Take time to get familiar with the tools and you’ll see just how powerful they can be.

The plan

Everyone loves it when a plan comes together so here’s the general approach to setting up the system:

1. All the configuration files and Puppet Manifests are stored in a git repository
2. Changes are made and then committed to the repository before being “pushed” to the CI server
3. The CI Server (Jenkins in our case) runs a series of tests on the manifests and configuration files
4. If the tests fail, the build breaks and notifications are sent via Jabber, Email or even a large robotic rabbit…
5. If the builds pass, the manifests and config files are deployed to the Puppetmaster by Capistrano
6. A puppet run is forced across all servers using MCollective
   

That gives you a flavour of what we’ll be doing over the coming weeks, so get your systems ready and we’ll get started in the next segment.