Integration Testing Chef Cookbooks With Serverspec

Writing a server integration test using the Serverspec testing framework to verify your server configuration helps to ensure the consistency and dependability of Chef code as it is being developed. Serverspec tests will prove that the correct packages were installed, configured correctly, and tested on the various platforms (CentOS, Ubuntu, Windows, and more).

This article walks you through the process of creating and running Serverspec tests. We'll set up our workstation to run the Chef cookbook integration test on a local workstation environment. Then we'll run through some of the integration tests to ensure everything is working properly.

Workstation Prerequisite

To set up your local development environment, you'll first need to download the Chef Development Kit (DK), VirtualBox, and Vagrant. This prerequisite lets you specifically download Chef DK 0.15.15, VirtualBox 5.0.26, and Vagrant 1.8.1, which are compatible with each other. The versions will change as Chef DK, VirtualBox, and Vagrant get new updates and it's your responsibility to validate the changes are compatible. Once you've downloaded and installed Chef DK and Vagrant, open up your terminal window and check the Chef and Vagrant versions.

$ chef -v 
Chef Development Kit Version: 0.15.15 
chef-client version: 12.11.18 
delivery version: 0.0.23 
berks version: 4.3.5 
kitchen version: 1.10.0

Once you've downloaded and installed VirtualBox, open the application and check the VirtualBox version(VirtualBox>About VirtualBox).

Once you’ve downloaded and extracted nginx-pkg from Chef Supermarket, open up your terminal window and navigate to the nginx-pkg directory to finish preparing your local environment.

Integration Testing

Test Kitchen is part of the Chef DK suite and does not need to be installed separately. Install Serverspec by adding a line to your Chef cookbook gemfile. Then, it's time to test your Chef cookbook. Let's take a closer look at each of the frameworks used for integration testing.

Serverspec is a testing framework designed to check that your server has been configured correctly. Serverspec tests are used to verify your remote cloud or virtualization server’s actual state (resource types, port, package, command, HTTP status code, and more) through an SSH connection.

Test-Kitchen is a testing harness to execute and verify your infrastructure code on one or more platforms in isolation. It allows you to run your code on various cloud providers and virtualization technologies such as Amazon EC2, Docker, Vagrant and VirtualBox, and more.

Running kitchen test will execute kitchen create, kitchen converge, kitchen verify, and kitchen destroy, in that order. Serverspec tests will run during the kitchen verify phase. The kitchen.yml file tells VirtualBox which type of architecture and operating system to use, then allows you to bootstrap with Chef code.

$ chef exec bundle exec kitchen --help
Commands:
kitchen console          # Kitchen console
… 
kitchen converge         # Change instance state to converge to provision instances
Kitchen destroy          # Change instance state to destroy and delete all info
…
… 
kitchen list             # Lists one or more instances
kitchen login            # Log in to one instance
… 
kitchen test             # Test (destroy, create, converge, setup, verify, destroy)
kitchen verify           # Verify and run automated tests on one or more inst...

Reviewing Serverspec Tests For The Nginx Cookbook

Kitchen and Serverspec allow us to converge, verify, and destroy your Chef cookbook on various real virtualization technologies — no simulation here. I want to continue driving this message across all my testing blog posts; it's much easier and cheaper to catch bugs locally as opposed to having production apps crash and burn.

Every cookbook should include a suite of integration tests. The folder test/integration is where Serverspec integration tests live. Let's review the cookbook-nginx-pkg file structure for serverspec testing:

|- cookbook-ngnix-pkg/
|-- recipes/
|-- spec/unit
|-- test/integration
|---- default/
|---- vendor/serverspec/
|------ serverspec/
|-------- spec_helper.rb
|-------- vendor_spec.rb

When writing Serverspec tests for your cookbook, the standard is to create a separate spec test for each recipe in the test/integration folder.

Let's take a closer look at recipe/default.rb and attributes/default.rb, which install the Nginx package and latest stable version. 

# install the package
package node['nginx-pkg']['package']['name'] do
  version node['nginx-pkg']['package']['version']
end

Source.

7  default['nginx-pkg']['package']['name'] = 'nginx'
…
12 default['nginx-pkg']['package']['version'] = ''

Source.

It's a simple recipe, right? Checking if the virtual machine convergence includes the NGINX package shouldn't be too difficult.

1 require 'spec_helper'
2
3 describe package('nginx') do
4  it { should be_installed }
5 end

Source.

Ask yourself if this is adequate test coverage for this cookbook. It checks that the Nginx package has been installed. If you ask me, this isn't adequate test coverage. Let's add Serverspec tests to improve test coverage.

Improving Serverspec Test Coverage

If you are using any of the popular configuration management tools available, they only get you halfway there by automating the server configuration. It’s important to think about all possible scenarios for testing the recipe. Untested recipe scenarios are destined to fail in production. It’s more than checking that the package was installed on the server. I selected the nginx-pkg cookbook to demonstrate how to improve the integration testing by adding the following tests:

Service Resource Type: Checking that a service should not be enabled using the be_enabled matcher.

describe service('nginx') do
  it { should_not be_running }
end

Directory Resource Type: Checking that the package directory exists using the be_directory matcher.

describe file('/etc/nginx') do
  it { should be_directory }
end

File Resource Type: Checking that the file exists and that the file contains a given string, using the be_file and contain matchers. You can use more strict grammar syntax like be_a_file instead of be_file with all resource types.

describe file('/etc/nginx/conf.d/default.conf') do
  it { should be_a_file }
  its(:content) { should match(%r{listen\s+80;}) }
end

Package Resource Type: Checking the given package version is installed, using be_installed and with_version matchers.

describe package('nginx') do
  it { should be_installed.with_version('1.10.0') }
end

To test a specific package version, I highly recommend setting the package version attribute within your kitchen.yml file:

attributes: {
   nginx-pkg: {
       package: {
         version: "1.10.0-1.el7.ngx"
         }
    }
}

Command Resource Type: Executing commands from the line to check command results using stdout, stderr, and exit status, along with RSpec matchers.

describe 'NGINX Service - Status, Start, and Stop  do

I recommend referencing the Serverspec resource types when writing integration tests, or as a review of Chef cookbook. Getting familiar with the resource types will only improve your Chef integration test coverage.

Conclusion

In this post, we set up a clean development environment and reviewed the integration testing for the nginx-pkg cookbook, and demonstrated how to improve integration test coverage by adding more Serverspec tests. Your infrastructure code deserves testing, and before deploying those servers, prove that your cookbook works.

To continue learning about Serverspec, Test Kitchen, and RSpec testing extensions, you can reference these documentation links:

• Learn Chef.
• Test Kitchen.
• Serverspec.
• RSpec Expectations.