Pipelines on a Rails Environment With react_on_rails and Webpack
Pipelines on a Rails Environment With react_on_rails and Webpack
I thought it may be useful to walk through some of the basics of how Bitbucket Pipelines is set up and strategies to help debug your pipeline's configuration.
Join the DZone community and get the full member experience.Join For Free
I noticed a bit of a gap when it comes to documentation and setting up Bitbucket Pipelines when you have some modern webpack dependencies with yarn.
If you've gone through this process, you may have struggled to find helpful documentation, too. I thought it may be useful to walk through some of the basics of how Bitbucket Pipelines is set up and strategies to help debug your pipeline's configuration. Read on for my experience of going through the process of setting up Bitbucket Pipelines on a Rails 5.2 environment.
Setting Up a Docker Environment
The first thing that is useful to know about setting up your environment for Bitbucket Pipelines is that you're setting up a Docker container for your Rails app to run in.
Your default Docker container for your pipelines environment will be on Ubuntu 14.04, which is important to know because you will use Ubuntu command line tools to install your dependencies.
Choosing Your Ruby Image
Getting started, we used a base Ruby image so we had the correct version of Ruby that matched our Ruby version for our Rails application.
In the Pipelines documentation, they say you can find your list of official Ruby images. I didn't have much luck using those listed here with the alpine/slim namespacing.
If you're having trouble finding one that works here like I experienced, I would recommend just matching your Ruby image name with your Ruby version, like so:
Since we have a nice integration with yarn using Rails webpacker and react_on_rails, we need to make sure that we can install yarn and have those dependencies defined in your package.json when the application builds.
Unfortunately, the version of node that is installed in the default container is not high enough to support yarn. So before we can build you need to install an updated version of node.
An important thing to note: one thing we have to do in our Ubuntu environment is, first, download these packages via
curl and then install them in our command line with
It seemed with this example of node vs yarn, that I needed to have node successfully installed before I could
apt-get yarn successfully.
My pipelines.yml file looks like this, which I used to get a version of node installed that can support yarn:
image: ruby:2.4.0 pipelines: default: - step: script: - curl -sL https://deb.nodesource.com/setup_8.x | bash - - apt-get update && apt-get install -y nodejs And these lines get added to install yarn into our environment: - curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - - echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list - apt-get update - apt-get install -y yarn
From here we need to make sure we install our packages from our package.json with:
Setting Up Your Database Configs
In all of our Rails applications, we keep our database.yml under a .gitignore so each person can modify their own local dev environments.
Our Rails environment on Pipelines needs a database.yml to look at to tell it which database to use. In our case for Postgres, I created a database.ci.yml to use specifically just for Bitbucket pipelines. That file looks like this:
# this file is used in the build process of bitbucket pipelines CI default: &default adapter: postgresql encoding: unicode # For details on connection pooling, see rails configuration guide # http://guides.rubyonrails.org/configuring.html#database-pooling pool: 5 host: localhost development: <<: *default database: example_ci_development # Warning: The database defined as "test" will be erased and # re-generated from your development database when you run "rake". # Do not set this db to the same as development or production. test: <<: *default database: example_ci_test username: test_user password: test_user_password production: <<: *default database: example_ci_production
Now in my script, I'll want to make sure that we set our DATABASE_URL to what matches our database config as well as rename this file to database.yml so Rails knows that it's the file to use.
- export DATABASE_URL=postgresql://test_user:test_user_password@localhost/communityright_ci_test - cp config/database.ci.yml config/database.yml
You'll also need to make sure that you use a service that matches whatever database you are using. Here's a guide to using services and databases from Bitbucket support.
For our app that looks like this:
pipelines: default: - step: script: - all scripts run here services: - postgres
Definitions define what services we call from the main pipelines task.
definitions: services: postgres: image: postgres environment: POSTGRES_DB: example_ci_test POSTGRES_USER: test_user POSTGRES_PASSWORD: test_user_password
We also define 'Redis' in our services here since we are using Sidekiq for our ActiveJob workers in the application. You can see an example of that in the full file below.
Other .gitignored Files
We also .gitiginored our config/secrets.yml - so I made sure that we made a CI specific secrets.yml and copied that over for Rails to use (it will need it to run specs).
- cp config/secrets.ci.yml config/secrets.yml
Caching Your Packages and Bundler Files
Having to do all of this installation of yarn packages and Ruby gems on each Pipelines build eats up a lot of time.
To cut down on this time we can cache our node_modules and vendor directories. This way these packages are available if there are no updates.
You define them first in your definitions, and make sure you are caching the correct folders.
pipelines: default: - step: caches: - bundler - yarn script: - other scripts - yarn install - bundle install --path vendor definitions: caches: bundler: ./vendor yarn: ./node_modules
Running Your Feature Specs With Capybara-Webkit
If you're using capybara-webkit for your headless browser, you will run into issues trying to run your feature specs.
In order to get around this, you can install and configure xvfb and run your specs that way. If you are experiencing your specs timing out on your features, it's likely you need to install xvfb for your environment to run your specs.
In the full file shown below, you can see how we are installing xvfb in our Docker environment with
apt-get (along with our other dependencies).
You can then run your specs with it with:
xvfb-run -a bundle exec rspec
Full File for Pipelines
There are a few other dependencies in here that I didn't dig into in detail that you may or may not need for your Rails 5+ application. From this point, you should do a little trial and error to see which of these you need.
image: ruby:2.4.0 pipelines: default: - step: caches: - bundler - yarn script: - curl -sL https://deb.nodesource.com/setup_8.x | bash - - apt-get update && apt-get install -y nodejs apt-transport-https - curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - - echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list - apt-get update - apt-get install -y xvfb yarn qt5-default libqt5webkit5-dev gstreamer1.0-plugins-base gstreamer1.0-tools gstreamer1.0-x - yarn install - export DATABASE_URL=postgresql://test_user:test_user_password@localhost/example_ci_test - cp config/database.ci.yml config/database.yml - cp config/secrets.ci.yml config/secrets.yml - bundle install --path vendor - RAILS_ENV=development bundle exec rake db:setup - RAILS_ENV=test bundle exec rake db:test:prepare - xvfb-run -a bundle exec rspec services: - redis - postgres definitions: caches: bundler: ./vendor yarn: ./node_modules services: redis: image: redis postgres: image: postgres environment: POSTGRES_DB: example_ci_test POSTGRES_USER: test_user POSTGRES_PASSWORD: test_user_password
Running Into Issues Debugging?
If things start to get really hairy, you can actually boot up a Docker environment on your local machine and run through your scripts to see what's wrong with your syntax.
You'll first need to install Docker on your machine.
Once you get that up and running, you can use the command line tools and boot up an environment for your project using a syntax like so:
docker run -it --volume=/Users/corinnekunze/dev/example-app:/example-app -- workdir="/example-app" --memory=2048m ruby:2.4.0 /bin/bash
Then, you can start writing your scripts out line-by-line and see what you can get working.
Just know that if you are altering files here in your Docker environment it's going to modify your local directory, for example, renaming your database.yml files.
Published at DZone with permission of Corinne Kunze , DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.