Evolving Enterprise Infrastructure Using Chef
Evolving Enterprise Infrastructure Using Chef
Join the DZone community and get the full member experience.Join For Free
In response to accelerated release cycles, a new set of testing capabilities is now required to deliver quality at speed. This is why there is a shake-up in the testing tools landscape—and a new leader has emerged in the just released Gartner Magic Quadrant for Software Test Automation.
People consider Chef as a configuration management tool. You specify the state using the infrastructure DSL that Chef provides. You just apply yoru configurations, and you get reproducible infrastructure. Here are the main topics:
- Chef is a modular infrastructure management toolkit. There are several way to look at chef: At its core chef is an API layer over day to day system administration. All other Chef components leverage this API.
- chef-client and chef-solo are two binaries that execute Chef scripts in client/server or serverless chef setups.
- Knife is a command line interface to chef-managed infrastructure.
All these things together give you traditional configuration management style workflows via a Chef managed infrastructure.
Now, there are ample one-off tasks that happen only once in a server's life time. Incident management, audits are another example. There are configurations related to third party hosted solutions. Enterprise's own service deployments. Most of these workflows don't model well in conventional configuration management style infrastructure management.
For such workflows chef can be easily extended. Chef provides multiple avenues to extend its capabilities:
- Chef internally distributed as a set of different libraries.
At its core chef has Mixlib::ShellOut , which is a separate rubygem. This is the component that provides portable command execution. Since 90% of the time configuration management systems dispatch command, this is the component that can be called the heart of Chef. If you have lots of shell-script based tools you can easily port them with mixlib-shellout. I have rarely seen bash scripts that check the return value of every command it invokes. It's more difficult to maintain long shell scripts. With mixlin-shellout its much easy to express safe guard measures.
- Chef uses mixlib-json to convert JSON data. This is also a separate gem, and you can independently use it. You might be wondering why this is important. JSON is the principle marshalling solution in chef and most other modern systems. mixlib-json facilitates the inclusion of a json conversion library from the available ones. You can use mixlib-json independently to grab raw data and serialize them and feed into other programs in a relatively portable way.
- mixlib-cli and mixlib-config . Mixlib::CLI lets you easily parse command line arguments. While mixlib-config lets you read, access override configuration values. Again both of them are distributed as separate gems. You can use these two and build your custom command line tool chains rapidly.
- knife-plugins. Knife can be extended using plugins. It's a more formal way of extending knife via sub-commands. The best part of Chef is that it's pure ruby everywhere, so you can pretty much use any rubygem (fog, aws-sdk, openvz) to have any cloud based infrastructure automation tool. There are many cases where your configurations are not applied against a server (linux or windows etc). They can be hosted solutions. Like IAM entries inside AWS, or ultraDNS based resource records, pagerduty escalation policies, salesforce settings, campfire notifications etc. The knife-plugin ecosystem along with the other libraries provide an elegant way to develop such tooling.
You can use an organization that can start automating smaller chunks of work and then refactor them into portable recipes so that a conventions-configuration management style infrastrcuture is possible. Chef's philosophy emphasizes resources as the primary building block and idempotency is a desired criteria. It is desired because there is an implicit assumption of repetetive invocation of chef runs within a system. Once you have your infrastructure to this level, there is lot magic that can be leveraged from chef, and also a lot other tools that work on top of chef can be adopted.
- Chef is also available as a single monolithic operating system package. Its called omnibus installers. They are present for most of the popular operating systems. For instance, the linux installer bundles everything on top of glibc. So you have an isolated, battle-tested working Chef binary that you can introduce to your system without inflicting major drifts.
This is crucial, as its lets you minimize iterative migration of manual or semi-automated infrastructure under full configuration management. Or, if not traditional configuration management, still reproducible and automatic infrastructure. Start with only the omnibus installer and then iteratively automate the most repetitive and time consuming tasks. Whatever it is.
- Once you have adopted chef significantly you'll realize the volume of chef-scripts is large. This is a by-product of treating large infrastructure as code. Code needs maintenance. And the maintenance can be easy if appropriate software development practices are applied. Having tests, keeping your chef scipts under a CI adds tremendous value. But the entire paradigm depends upon how easy it is to write tests against chef script. Last year has seen tremendous efforts on this aspect. Now we have decently effective unit testing, functional testing, and integration testing frameworks (chefspec, cucumber-chef, minitest-handler are examples respectively). Chef allows you to treat infrastructure as any other functional domain, like mobile or banking or health care. Any developers can work with domain experts to develop neat chef scripts (an operations guy is an example of a domain expert).
Together these things makes it much easier for an enterprise to iteratively improve and evolve its infrastructure to accommodate newer innovations (like cloud adoption, automatic failovers, disposable servers) with greater agility using Chef....
Published at DZone with permission of Ranjib Dey , DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.