Ansible TDD Development Using Molecule 2.4

Introduction

Ansible is an agentless IT orchestration tool written in Python that simplifies infrastructure automation and deployment, similar to an agent-based tool like Puppet or Chef. 

Molecule contains a set of tools to help us in the development and testing of Ansible roles. Ansible role can be tested against multiple operating systems and distributions, virtualization providers such as docker and vagrant, test frameworks such as testinfra and Goss using Molecule.

This tutorial is to develop an Ansible role from scratch using the test-driven development approach using Molecule, testinfra, and Docker.

Requirements

Develop an Ansible role for installing an SOS report package on Ubuntu machines and generate a report using the TDD approach.

SOS report is a tool to capture the debugging information for the current system in a compressed tarball format that can be sent to technical support for further analysis.

Pre-Requisites

Install Ansible, Molecule, and docker-py using pip. Assume Docker for Mac is installed on your Mac OS.

pip install ansible
pip install molecule
pip install docker-py

Create Ansible Role Skeleton

We can initialize an Ansible role with Molecule using the following two approaches:

Approach 1: Create Ansible role with the ansible-galaxy command, then initialize with Molecule

We can generate an Ansible skeleton role using the ansible-galaxy init command without Molecule and later initialize through the molecule init command.

This approach can be used to add Molecule tests to any existing Ansible role.

ansible-galaxy init ansible-role-sosreport
cd ansible-role-sosreport/
molecule init scenario --scenario-name default --role-name ansible-role-sosreport

Approach 2: Create Ansible role with Molecule init command

Create a new role using the Molecule init command:

molecule init role --role-name ansible-role-sosreport

The Molecule init command creates the Molecule directory inside the newly created Ansible role with a default scenario. The tests are written under the Molecule/default/tests folder using testinfra.

Since we are doing a test-driven approach, we will write the tests initially, then go with the actual code.

First, we have to redefine the default Molecule configuration file, which is generated by the molecule init command with the driver and image, platforms, verifier, and the test sequence for our default scenario.

file: ansible-role-sosreport/molecule/molecule.yml

---
dependency:
  name: galaxy
driver:
  name: docker
lint:
  name: yamllint
platforms:
  - name: instance
    image: ubuntu:16.04
    privileged: true
provisioner:
  name: ansible
  lint:
    name: ansible-lint
    options:
      x: ANSIBLE0013
verifier:
  name: testinfra
  options:
    verbose: true
  lint:
    name: flake8
    enabled: False
scenario:
  name: default
  test_sequence:
    - destroy
    - syntax
    - create
    - prepare
    - converge
    - lint
    - side_effect
    - verify
    - destroy

We are using Docker driver in this example, which is mentioned in the driver section (line 4-5).

The platforms section (line 8-11) has the Docker image used by Molecule to test this role. Here we are using the Ubuntu image, but we can have multiple Docker images in this section.

In the provisioner section (line 12-17), we are excluding the Ansible line rule ANSIBLE0013 for the shell module.

We are overriding the scenario (line 25-36) section by specifying the test sequence that we required. For example, we removed the idempotence test from the test_sequence list.

1. Writing Tests

The intention of ansible-role-sosreport role is to make sure that the sosreport package is installed and generates the system report in a compressed tar format.

Use Case 1: Make sure that sos-report pre-requisites and sos-report package is installed.

Use Case 2: Make sure that sos report file is generated at a given location.

Tests are written under the scenario_name/tests folder. Currently, we have only one scenario, named default. To learn more about scenarios, please refer to the reference section.

file: ansible-role-sosreport/molecule/default/tests/test_default.py

import os
import testinfra.utils.ansible_runner

testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(
    os.environ['MOLECULE_INVENTORY_FILE']).get_hosts('all')


def test_sos_report_package(host):
    assert host.package("software-properties-common").is_installed
    apt_repo = int(host.check_output("find /etc/apt/ -type f -exec grep 'canonical-support/support-tools' '{}' \; -print | wc -l").strip())
    assert  apt_repo > 0
    assert host.package("sosreport").is_installed


def test_sos_report_file(host):
    assert host.check_output("find /tmp/sosreport-*/ -name 'sosreport-shoneslab*.tar.xz' | wc -l").strip() == '1'

Here, we have two functions to verify whether the sosreport package (line 8-12) is installed with its dependencies and the actual report is generated (line 15-16) after applying the ansible-role-sosreport role to the Ubuntu 16.04 Docker image.

2. Run the Test

After writing the test scenarios, we can run the test using molecule. Since the actual functionality is not yet implemented, the following commands will throw errors.

molecule --debug test

3. Implement the Code

Once the test has failed, we know that the functionality is missing in the Ansible role and yet to implement. We have to write the logic as Ansible tasks in the tasks folder under the module name ansible-role-sosreport.

File: ansible-role-sosreport/tasks/install.yml

This file is to install the sosreport package and its dependencies:

---
- name: install prereq
  apt:
    name: software-properties-common
    state: present
  become: true

- name: add canonical support tools apt repo
  apt_repository:
    state: present
    repo: "{{ sosreport_apt_repo }}"
  become: true

- name: update apt packages
  apt:
    update_cache: yes
    cache_valid_time: 86400
  become: true

- name: install sosreport
  apt:
    name: sosreport
    state: present
  become: true

file: ansible-role-sosreport/tasks/report.yml

This file is to generate the sosreport:

---
- name: get the timestamp
  set_fact:
    sosreport_dir_timestamp: "{{ lookup('pipe', 'date +%m%d%Y-%H%M%S') }}"

- name: ensure the temp directory
  file:
    path: "{{ sosreport_temp_dir }}/sosreport-{{ sosreport_dir_timestamp }}"
    state: directory
  become: true

- name: generate sosreport
  shell: >
    sosreport -a \
    --name={{ sosreport_customer_name }} \
    --case-id={{ sosreport_caseid }} \
    --tmp-dir="{{ sosreport_temp_dir }}/sosreport-{{ sosreport_dir_timestamp }}" \
    --batch
  become: true
  changed_when: false

- name: find the latest generated file
  find:
    path: "{{ sosreport_temp_dir }}/sosreport-{{ sosreport_dir_timestamp }}"
    patterns: "sosreport-{{ sosreport_customer_name }}.{{ sosreport_caseid }}-*.tar.xz"
  register: reg_sosreport

- name: stat for the sosreport
  stat:
    path: "{{ reg_sosreport.files[0]['path'] }}"
  register: reg_stat_sosreport
  when: reg_sosreport | length > 0

- name: change file permissions
  file:
    name: "{{ reg_sosreport.files[0]['path'] }}"
    mode: 0755
  become: true
  when:
    - reg_sosreport | length > 0
    - reg_stat_sosreport.stat.exists is defined
    - reg_stat_sosreport.stat.exists

- name: ensure the destination directory
  file:
    path: "{{ sosreport_final_dest_path }}"
    state: directory
    mode: 0755
  delegate_to: localhost
  become: false
  when:
    - reg_sosreport | length > 0
    - reg_stat_sosreport.stat.exists is defined
    - reg_stat_sosreport.stat.exists

- name: stat whether the file is already copied
  stat:
    path: "{{ sosreport_final_dest_path }}/{{ reg_sosreport.files[0]['path'] }}"
  register: reg_stat_sosreport_copied
  delegate_to: localhost
  when: reg_sosreport | length > 0

- name: fetch the remote file to local
  fetch:
    src: "{{ reg_sosreport.files[0]['path'] }}"
    dest: "{{ sosreport_final_dest_path }}"
    flat: yes
  become: true
  when:
    - reg_sosreport | length > 0
    - not reg_stat_sosreport_copied.stat.exists
    - reg_stat_sosreport.stat.exists is defined
    - reg_stat_sosreport.stat.exists

file: ansible-role-sosreport/tasks/main.yml

The main.yml is the starting point of an Ansible role. Since the logic is implemented in a modular approach, we are including the other two tasks files in main.yml.

---
# tasks file for ansible-role-sosreport
- include_tasks: install.yml
  tags: sosreport_install

- include_tasks: report.yml
  tags: sosreport_report

file: ansible-role-sosreport/defaults/main.yml

It is always a good practice to externalize the variables than hardcoding in the tasks file. Eventually, these values can be overridden when we use inventory (not the scope of this tutorial)

sosreport_apt_repo: "ppa:canonical-support/support-tools"
sosreport_temp_dir: /tmp
sosreport_final_dest_path: "{{ lookup('env','HOME') }}/temp/"

# Customer Name should be with out spaces
sosreport_customer_name: 'shoneslab'
sosreport_caseid: 42799

4. Run the Test

This is the iterative process. Once you see all the errors are gone, your Ansible module is completed with a TDD approach.

molecule --debug test

Conclusion

It is easy to develop an Ansible role using Molecule and can easily integrate with CI tools like Gitlab/Jenkins.  

This Ansible role can be downloaded using the following command:

ansible-galaxy install shoneslab.sosreport