Run RSpec on GitHub Actions in the Shortest Time Using Parallel Jobs

GitHub introduced their own CI server solution called GitHub Actions. You will learn how to set up your Ruby on Rails application on GitHub Actions with the YAML config file. To run your RSpec test suite faster, you will configure parallel jobs with matrix strategy on GitHub Actions.

Automate Your Workflow on GitHub Actions

GitHub Actions makes it easy to automate all your software workflows with world-class CI/CD. The building, testing, and deploying your code right from GitHub became available with simple YAML configuration.

You can even create a few YAML config files to run a different set of rules on your CI, like scheduling daily CI builds. But let’s focus strictly on how to get running tests for the Rails app on GitHub Actions.

Setup Ruby on Rails on GitHub Actions With YAML Config

In your project repository, you need to create a file: .github/workflows/main.yaml. Thanks to it, GitHub will run your CI build. You can find results of CI builds in the Actions tab for your GitHub repository.

In our case, the Rails application has a Postgres database, so you need to set up a service with a docker container to run Postgres DB.

# If you need DB like PostgreSQL then define service below.

# Example for Redis can be found here:

# https://github.com/actions/example-services/tree/master/.github/workflows

services:

  postgres:

    image: postgres:10.8

    env:

      POSTGRES_USER: postgres

      POSTGRES_PASSWORD: ""

      POSTGRES_DB: postgres

    ports:

    # will assign a random free host port

    - 5432/tcp

    # needed because the postgres container does not provide a healthcheck

    options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5

To be able to install the pg ruby gem from a project Gemfile, you will needlibpq-devlibrary in the Ubuntu system, hence the step to install it. libpq is a set of library functions that allow client programs to pass queries to the PostgreSQL backend server and to receive the results of these queries. We need it to compile pg gem. The next step will be installing our Ruby gems.

xxxxxxxxxx

# required to compile pg ruby gem

- name: install PostgreSQL client

  run: sudo apt-get install libpq-dev

- name: Build and create DB

  env:

    # use localhost for the host here because we have specified a container for the job.

    # If we were running the job on the VM this would be postgres

    PGHOST: localhost

    PGUSER: postgres

    PGPORT: ${ { job.services.postgres.ports[5432] }} # get randomly assigned published port

    RAILS_ENV: test

  run: |

    gem install bundler

    bundle config path vendor/bundle

    bundle install --jobs 4 --retry 3

    bin/rails db:setup

To run RSpec tests across parallel jobs, you need to set up the matrix feature, and thanks to that, run the whole test suite distributed across jobs.

Configuring a Build Matrix

A build matrix provides different configurations for the virtual environment to test. For instance, a workflow can run a job for more than one supported version of a language, operating system, etc. For each configuration, a copy of the job runs and reports status.

In case of running parallel tests, you want to run the Rails application on the same Ruby version and Ubuntu system. But you want to split the RSpec test suite into 2 sets, so half of the tests go to a first parallel job and the second half to another job.

To split tests, you can use Ruby gem Knapsack Pro that will split tests across parallel GitHub jobs in a dynamic way. Thanks to that, each parallel job will be consuming a set of tests fetched from Knapsack Pro API Queue to ensure each parallel job finishes work at a similar time. This allows for evenly distributed tests and no bottleneck in parallel jobs (no slow job). Your CI build will be as fast as possible.

In our case, you split tests across 2 parallel jobs, so you need to set 2 as matrix.ci_node_total. Then each parallel job should have an assigned index matrix.ci_node_index start from 0. The first parallel job gets index 0, and the second job gets index 1. This allows Knapsack Pro to know what tests should be executed on a particular job.

xxxxxxxxxx

# https://help.github.com/en/articles/workflow-syntax-for-github-actions#jobsjob_idstrategymatrix

strategy:

  fail-fast: false

  matrix:

    # Set N number of parallel jobs you want to run tests on.

    # Use higher number if you have slow tests to split them on more parallel jobs.

    # Remember to update ci_node_index below to 0..N-1

    ci_node_total: [2]

    # set N-1 indexes for parallel jobs

    # When you run 2 parallel jobs then first job will have index 0,

    # the second job will have index 1 etc

    ci_node_index: [0, 1]

You need to specify also API token for Knapsack Pro; for RSpec, it will be KNAPSACK_PRO_TEST_SUITE_TOKEN_RSPEC.

Then you can run tests with Knapsack Pro in Queue Mode for RSpec:

bundle exec rake knapsack_pro:queue:rspec

Full GitHub Actions config file for Rails tests

Here you can find the full YAML configuration file for GitHub Actions and Ruby on Rails project.

I also recorded a video showing how it all works and how CI builds with parallel jobs are configured on GitHub Actions.

xxxxxxxxxx

# .github/workflows/main.yaml

name: Main

on: [push]

jobs:

  vm-job:

    runs-on: ubuntu-latest

    # If you need DB like PostgreSQL then define service below.

    # Example for Redis can be found here:

    # https://github.com/actions/example-services/tree/master/.github/workflows

    services:

      postgres:

        image: postgres:10.8

        env:

          POSTGRES_USER: postgres

          POSTGRES_PASSWORD: ""

          POSTGRES_DB: postgres

        ports:

        # will assign a random free host port

        - 5432/tcp

        # needed because the postgres container does not provide a healthcheck

        options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5

    # https://help.github.com/en/articles/workflow-syntax-for-github-actions#jobsjob_idstrategymatrix

    strategy:

      fail-fast: false

      matrix:

        # Set N number of parallel jobs you want to run tests on.

        # Use higher number if you have slow tests to split them on more parallel jobs.

        # Remember to update ci_node_index below to 0..N-1

        ci_node_total: [2]

        # set N-1 indexes for parallel jobs

        # When you run 2 parallel jobs then first job will have index 0, the second job will have index 1 etc

        ci_node_index: [0, 1]

    steps:

    - uses: actions/checkout@v1

    - name: Set up Ruby 2.6

      uses: actions/setup-ruby@v1

      with:

        ruby-version: 2.6.5

    - uses: actions/cache@v1

      with:

        path: vendor/bundle

        key: ${{ runner.os }}-gems-${{ hashFiles('**/Gemfile.lock') }}

        restore-keys: |

          ${{ runner.os }}-gems-

    # required to compile pg ruby gem

    - name: install PostgreSQL client

      run: sudo apt-get install libpq-dev

    - name: Build and create DB

      env:

        # use localhost for the host here because we have specified a container for the job.

        # If we were running the job on the VM this would be postgres

        PGHOST: localhost

        PGUSER: postgres

        PGPORT: ${{ job.services.postgres.ports[5432] }} # get randomly assigned published port

        RAILS_ENV: test

      run: |

        gem install bundler

        bundle config path vendor/bundle

        bundle install --jobs 4 --retry 3

        bin/rails db:setup

    - name: Run tests

      env:

        PGHOST: localhost

        PGUSER: postgres

        PGPORT: ${{ job.services.postgres.ports[5432] }} # get randomly assigned published port

        RAILS_ENV: test

        KNAPSACK_PRO_TEST_SUITE_TOKEN_RSPEC: ${{ secrets.KNAPSACK_PRO_TEST_SUITE_TOKEN_RSPEC }}

        KNAPSACK_PRO_TEST_SUITE_TOKEN_CUCUMBER: ${{ secrets.KNAPSACK_PRO_TEST_SUITE_TOKEN_CUCUMBER }}

        KNAPSACK_PRO_TEST_SUITE_TOKEN_MINITEST: ${{ secrets.KNAPSACK_PRO_TEST_SUITE_TOKEN_MINITEST }}

        KNAPSACK_PRO_TEST_SUITE_TEST_UNIT: ${{ secrets.KNAPSACK_PRO_TEST_SUITE_TOKEN_TEST_UNIT }}

        KNAPSACK_PRO_TEST_SUITE_TOKEN_SPINACH: ${{ secrets.KNAPSACK_PRO_TEST_SUITE_TOKEN_SPINACH }}

        KNAPSACK_PRO_CI_NODE_TOTAL: ${{ matrix.ci_node_total }}

        KNAPSACK_PRO_CI_NODE_INDEX: ${{ matrix.ci_node_index }}

        # if you use Knapsack Pro Queue Mode you must set below env variable

        # to be able to retry CI build and run previously recorded tests

        KNAPSACK_PRO_FIXED_QUEUE_SPLIT: true

      run: |

        # run tests in Knapsack Pro Regular Mode

        bundle exec rake knapsack_pro:rspec

        bundle exec rake knapsack_pro:cucumber

        bundle exec rake knapsack_pro:minitest

        bundle exec rake knapsack_pro:test_unit

        bundle exec rake knapsack_pro:spinach

        # you can use Knapsack Pro in Queue Mode once recorded first CI build with Regular Mode

        bundle exec rake knapsack_pro:queue:rspec

        bundle exec rake knapsack_pro:queue:cucumber

        bundle exec rake knapsack_pro:queue:minitest

Dynamic Test Suite Split With Queue Mode

If you would like to better understand how Queue Mode works in Knapsack Pro and what other problems it solves, you will find useful information in the below video:

Also, check out another article describing the parallelisation setup for GitHub Actions for Rails with MySQL, Redis, and Elasticsearch or learn how to auto split slow RSpec test files by test examples.

I hope you find this helpful.

If you are currently considering moving to GitHub Actions, definitely check out our Comparison of GitHub Actions to other CI solutions. The most popular ones include Github Actions vs Circle CI, Github Actions vs Jenkins, and Github Actions vs Travis CI.