GitLab CI With Docker: Environment Variable Quirks

Introduction

DevOps has been one of those things over the years that I've come to learn more about and implement more and more as time goes on, especially with my TDD leanings.

That said, while I've become fairly proficient with several key DevOps technologies — especially in the CI/CD space — I wouldn't necessarily consider myself an expert in all of them.

Recently, I've had occasion to make use of GitLab's CI functionality, and for those who want to avoid having to integrate with external CI services (let's face it: not all CI technologies are able to be integrated with certain repository solutions, or it can be an outright pain), it can definitely be a big step towards achieving a proper CI process.

Feature vs. Bug

In incorporating GitLab CI into a GitLab repository, I came across something that can either be described as a feature, a bug, or something in between.

To use GitLab CI's pipelines, all one needs to do is create a .gitlab-ci.yml file in the root of their Git repo and commit it. GitLab will take care of the rest.

These pipelines can take advantage of Docker which, these days, is almost a necessity.

So what could possibly go wrong?

Take, for example, a variation of the YAML file I created:

default:
  image:
    name: ruby:2.6

   services:
     - name: postgres:11.3-alpine
       alias: db

test:
  stage: test
  script:
    - bundle install
    - rake spec

Pretty straightforward, right?

And, in fairness, it didn't take much effort to get this script going.

However.

Something I came up against — purely by coincidence/accident — was that one or two of the environment variables the Ruby app uses was being inexplicably overwritten, in this case, the DB_PORT and DB_NAME environment variables (side note/peculiarity: DB_PORT was actually being set to something like "tcp://172.17.0.2:5432", which isn't much of a port at all).

The Cause

After much digging through the Ruby code and the supporting gems, I couldn't find a single thing that would cause such a change. I did come across one or two other similar incidents after Googling, but, not one of them could find a reasonable cause.

Then, after much experimentation, I found that the Postgres Docker image was setting the DB_PORT and DB_NAME environment variables. It looks as though GitLab CI makes environment variables set from services available to the GitLab CI script itself — something I wasn't expecting.

While I can see something like this potentially being useful in some situations, perhaps the biggest issue is that I could find no documentation anywhere about Postgres setting those particular environment variables.

So here we have a bit of a "perfect storm" of unfortunate variable naming and a feature I wasn't expecting.

The Solution

Fortunately, GitLab CI allows for a before_script block for each GitLab CI job, in which you can set/clear any environment variables you wish (as well as run other commands, of course).

By clearing the environment variables mentioned above, the Ruby application then ran as intended, making use of the environment variables as originally planned.

Conclusion

Hopefully, this short article will help the next person to come across a similar situation.

And, if you end up needing to rely on the feature of GitLab CI making available the environment variables to the rest of the GitLab CI script, but still need to use the same variables but with different values, you can always do the following in each of your jobs:

1. In before_script, create new environment variables and copy the affected ones over temporarily.

2. Execute your normal GitLab CI script  block.

3.  In after_script  (similar to before_script  but with the obvious difference), copy those environment variables from step 1 back to their original variables and unset the temporary ones.

Further Reading

Docker Environment Variables: How to Set and Configure Server Applications