One area that I have been concentrating on for a while now is the automated delivery of software that actually works. This has recently become a hot topic with Continuous Integration and Continuous Delivery being discussed. The DevOps banner is being used for wider discussions which include the breaking down of barriers between teams to increase co-operation. This is all good stuff and a step in the right direction in an area which has long been neglected. Companies are now starting to invest in how they deliver their software. But they may be saving up problems for the future.
Configuration Management and Release Orchestration
The new technologies (Ansible, Chef, Puppet) in this space have the ability to drastically reduce the amount of time it takes to deliver a 'working' host. Many a time I've witnessed a host ping-ponging between Sys Admins and Dev teams before it's fit for use. The Dev teams don't specify exactly what they need and the System Admins don't automate it for future use. With DevOps we break this cycle and bring the efficiencies of automation to OS builds.
But these technologies leave fundamental gaps in the delivery of application software. Let's think this through a bit :
I have an application called Saturn that:
- Comprises of 10 different processes that each serve a different purpose;
- Each process has a different binary package;
- Uses Java 1.6 and has a list of dependant libraries;
- Is not resource intensive so it's processes can co-habitate with other applications on a host and;
- Has configuration that needs to differ between environments.
We don't have the luxury of a dedicated host so Saturn shares hosts with other applications. In fact, in Dev and UAT we have multiple instances of Saturn running and even many different versions of Saturn running on the same host. If we are to share it's essential for the application to be a good neighbour so it doesn't have to be isolated. But without the right tools, sharing of resources can cause conflicts. Here is why:
There is another team that are building Pluto and they are using Java 1.8. This is one of the applications that share hosts with Saturn. Instantly we see a conflict in the version of Java used. We could hardcode the Java version we use but that reduces our application portability.
We then discover that Saturn and Pluto use different versions of the same Java libraries. How do we cope with that? We could embed these libraries in our application but that seems rather clumsy, encourages duplication and reduces visibility of usage. This is a wider problem than just Java. There are two other teams that own Jupiter and Mars which are written in Python. They have very similar problems with the use of conflicting versions of Python and Python libraries. These problems stem from the reliance on the underlying OS build to fulfill runtime dependancies.
When resource sharing, we need a tool that prevents dependancy conflicts.
There are similar conflicts when we look at the application binaries. We are making our lives unneccessarily difficult if we don't have tooling that provides functionality to:
- easily manage multiple versions of the application software on a single host;
- select/update binary versions used in each environment;
- give transparancy and visibility of environment and configuration settings and;
- automate the removal of unused software for effective disk management.
Storing your application binaries and libraries in a repository makes perfect sense. Yum is frequently used to hold RPMs that are built containing the application software. But this is for RedHat-based Linux distros; what about Debian-based distros and of course Windows and OS X? This is highly restrictive! We need a repository that can store software for all flavours of OS.
We need our repository to have functionality to:
- Be OS and packaging standard independant.
- Distinguish between production and non-production packages.
- Retain meta data about each package:
- When was it created and by whom
- What level of testing it has had
- Which environments its used in
- Which hosts it exists on
- Be immutable so you know it's not been tampered with.
- Restrict who can add or remove packages.
Configuration management tools can make a big difference in release orchestration. Releases often comprise of many steps that need to be performed in a certain order over a number of different hosts. Software deployment is usually an essential part but there are frequently many other steps involved (ie process stop/start, database updates, batch scheduler changes etc).
There is however the temptation to mix up what is being released with how it is being released. The release runbook (sometimes called recipe or playbook) should contain all the steps involved and their dependancies. If, in the runbook, we start to include what is actually changing (ie versions of packages to install or remove) then it would need modifying for each release performed and needs to be tested, just like code. To lessen the testing burden it seems logical to separate what is being released from how it gets released. But what tools do we need to do this?
Using a configuration manager to quickly set the standard OS build and to orchestrate a release saves a lot of time and money and gives welcome consistency. But if your application is part of a larger business where resources are shared and is going to be changing frequently, a configuration manager has fundamental gaps. We need something more.