Using Onceover to Start Testing With rspec-puppet

I said this to my colleagues the other day:

“Am I correct to understand that if you want to run rspec tests on your control repo, you still need to define the location of every module in fixtures.yml  via something like this? Seems like there should be a better way.”

Most thought that was the only way, and we were stuck with creating duplicate information, storing repo locations in both our Puppetfile and the fixtures.yml file. Then I asked Dylan Ratcliffe the same question and he answered quickly.

“Nah, I use Onceover to automate all that. I’ll get the link.”

I was dumbfounded. I had heard of Onceover before, but I thought it was for Beaker testing and dismissed it as something I wouldn’t use. Turns out, Onceover is the perfect tool for getting started with rspec testing on your control repo.

I have a control repo out on GitHub that I created to help Puppet customers get started with Puppet Enterprise and I decided I should try using Onceover on my control repo right away.

So, I pull up the Onceover README.

Clone Down the Control Repo

git clone [https://github.com/puppet-rampupprogram/control-repo/](https://github.com/puppet-rampupprogram/control-repo/)
cd control-repo

Install the Onceover Gem

Install Bundler, make a Gemfile, and install Onceover.

gem install bundler
echo ‘gem onceover’ > Gemfile
bundle install

It's Time to Configure

Now that Onceover is installed, the README says I need to configure it. I can do so by running onceover init.

bundle exec onceover init

As the command runs, I can see that it created a number of files for me.

created spec
created spec/onceover.yaml
created spec/acceptance
created spec/acceptance/nodesets
created spec/acceptance/nodesets/onceover-nodes.yml
created spec/pre_conditions
created spec/pre_conditions/README.md
created spec/factsets
created spec/factsets/README.md

I’m curious what’s going to happen now if I just try to run the spec tests. Back to the README, where I discover that I can run onceover run spec.

bundle exec onceover run spec

Starting at the beginning of the output, I see that it’s deploying my modules from the Puppetfile.

INFO     -> Updating module /Users/nw/git_repos/npwalker-control-repo/.onceover/etc/puppetlabs/code/environments/production/modules/inifile
INFO     -> Updating module /Users/nw/git_repos/npwalker-control-repo/.onceover/etc/puppetlabs/code/environments/production/modules/stdlib
INFO     -> Updating module /Users/nw/git_repos/npwalker-control-repo/.onceover/etc/puppetlabs/code/environments/production/modules/concat
INFO     -> Updating module /Users/nw/git_repos/npwalker-control-repo/.onceover/etc/puppetlabs/code/environments/production/modules/hiera
INFO     -> Updating module /Users/nw/git_repos/npwalker-control-repo/.onceover/etc/puppetlabs/code/environments/production/modules/pe_code_manager_webhook
INFO     -> Updating module /Users/nw/git_repos/npwalker-control-repo/.onceover/etc/puppetlabs/code/environments/production/modules/pe_metric_curl_cron_jobs
INFO     -> Updating module /Users/nw/git_repos/npwalker-control-repo/.onceover/etc/puppetlabs/code/environments/production/modules/gms
INFO     -> Updating module /Users/nw/git_repos/npwalker-control-repo/.onceover/etc/puppetlabs/code/environments/production/modules/rbac
WARNING: Could not find hiera config file, continuing

Seeing a Couple of Failures

 1) role::all_in_one_pe using fact set CentOS-5.11-32 should compile into a catalogue without dependency cycles
     Failure/Error: it { should compile }
       error during compilation: Evaluation Error: Unknown variable: '::is_pe'. at     /Users/nw/git_repos/npwalker-control-repo/.onceover/etc/puppetlabs/code/environments/production/modules/hiera/manifests/params.pp:21:15 on node noway.local
     # ./spec/classes/role__all_in_one_pe_on_CentOS-5.11-32_spec.rb:7:in `block (3 levels) in     <top (required)>'

The Onceover gem doesn’t know that ::is_pe is. is_pe is a custom fact from the stdlib module, and I remember seeing something about “factsets” in the Onceover README.

I spin up a Vagrant machine with PE 2016.2.1 installed on Centos 6.6, and I run puppet facts. I copy that output into the appropriate file in spec/factsets.

cat spec/factsets/CentOS-6.6-64.json | grep is_pe
   "is_pe": false,

Now, when I run my spec tests, it should be able to use that fact. I get a new error:

 4) role::all_in_one_pe using fact set CentOS-6.6-64 should compile into a catalogue without     dependency cycles
     Failure/Error: it { should compile }
       error during compilation: Evaluation Error: Unknown variable:     'code_manager_auto_configure_nc_value'. at /Users/nw/git_repos/npwalker-control-repo/.onceover/etc/puppetlabs/code/environments/product    ion/modules/pe_code_manager_webhook/manifests/init.pp:11:51 on node noway.local
     # ./spec/classes/role__all_in_one_pe_on_CentOS-6.6-64_spec.rb:7:in `block (3 levels) in <top (required)>'

This is interesting. The spec tests must be running some extra flags above and beyond what happens in a default Puppet run because I use this module all the time. However, this error brings up a good point: I’m referencing a variable I have since commented out.

Okay, so I push a change to pe_code_manager_webhook to remove that variable from the pick statement, and then subsequently push a change to the control-repo to use the newest version of pe_code_manager_webhook. I’m ready to try running the spec tests again.

A New Error Emerges

 4) role::all_in_one_pe using fact set CentOS-6.6-64 should compile into a catalogue without dependency cycles
     Failure/Error: it { should compile }
       error during compilation: Could not retrieve dependency 'Service[pe-puppetserver]' of     Class[Hiera]
     # ./spec/classes/role__all_in_one_pe_on_CentOS-6.6-64_spec.rb:7:in `block (3 levels) in     <top (required)>'

The Onceover README referred directly to this case in the using workarounds section. Since my code is referencing a service that is provided by Puppet Enterprise, I need to trick the spec tests a little so that Onceover can complete the spec tests. I place a pre_conditions file copying the example that is provided. This example is a little complex, as it is designed to work for more advanced acceptance tests, too. I could get away with just passing in the ensure parameter if I wanted to save some code.

cat spec/pre_conditions/pe-puppetserver.pp 
service { 'pe-puppetserver': ensure => 'running', enable => false, hasrestart => false, #

Force Puppet to use start and stop to restart start => 'echo "Start"', #.

This will always exit: 0 stop => 'echo "Stop"', #.

This will also always exit: 0 has status => false, #.

Force Puppet to use our command for status => 'echo "Status"', #. This will always exit 0 and therefore Puppet will think the service is running provider => 'base', }.

Let’s Try This Again

role::all_in_one_pe
  using fact set CentOS-6.6-64
    should compile into a catalogue without dependency cycles

Finished in 2.4 seconds (files took 0.91583 seconds to load)
1 example, 0 failures

Victory! I’ve completed a successful set of spec tests. What do I have at this point? Well, if I run these specs on every pull request to my control repo, then at a minimum I know that the Puppet code being added is valid and will compile. This may be a humble start, but for the minimal work it took, I have much more confidence in changing my code.

Nick Walker is a principal technical customer success engineer at Puppet.