Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Updating CI Environments With Travis

DZone's Guide to

Updating CI Environments With Travis

Using GitHub, TravisCI, Pull Approve, and SonarQube to update your CI environments.

· DevOps Zone
Free Resource

The DevOps Zone is brought to you in partnership with Sonatype Nexus. The Nexus Suite helps scale your DevOps delivery with continuous component intelligence integrated into development tools, including Eclipse, IntelliJ, Jenkins, Bamboo, SonarQube and more. Schedule a demo today

What would the process of updating the code and continuous integration environment look like in an ideal world?

A developer [1] pushes code changes to a repository branch and they are immediately tested on a build server. When (s)he is satisfied with the results, he [2] creates a pull request. Before a peer developer looks at the changes, they are already reviewed by an automated code analyser. And finally the reviewer [3] approves the pull request to be merged to the main branch.

GitHub Pull Request

So there is good news! There are already services available online that enable this process without you having to run a complex CI environment. If you continue reading this article you’ll learn about the systems we used to implement it for the minimesos project at Container Solutions and all the integration details.

In our project we use GitHub as our source code repository, Travis CI for running the builds, Pull Approve for structuring code reviews, and SonarQube for monitoring code quality.

When code changes are pushed to the source repository, GitHub uses webhooks to notify Pull Approve and Travis CI. Unless this push belongs to already existing pull request, Pull Approve ignores this event. However Travis CI pulls the code, runs a build and notifies the author of the changes about the results by email.

push flow

When a pull request is created in GitHub (PR), Pull Approve extends the PR with a required check: one or more authorised reviewers have to approve the changes by commenting the PR with certain keywords. Travis CI pulls the code again and starts building the PR. It also adds both the PR and latest related push builds to the PR as additional checks. Our PR build on Travis completes with a SonarQube preview scan. This scan analyses the updated code and compares results with the data, which is pulled from the SonarQube server. It places new findings as inline comments in the GitHub PR and adds the SonarQube check.

pull request flow

When a reviewer is satisfied with the code changes he/she comments on the PR. They are notified about the new comment, Pull Approve checks if its content contains pre-defined keywords, and, if yes, enables the Merge button by marking its check as passed. The merging of the PR to the main branch creates another commit. Travis CI detects the build on the main branch and completes it by updating the SonarQube server with new findings, so they can be used for reporting on future pull requests.

master merge flow

This is it. Clear and easy. And now we’ll dive into implementation details.

SonarQube

sonarqubeLet’s start with SonarQube – the only component which we host ourselves. SonarQube is a web application which stores data in a database. For simplicity and reproducibility we created a Docker Compose docker-compose.yml file.

sonarqube:
  image: "sonarqube:5.3"
  command: -Dsonar.web.context=/sonar
  ports:
   - "9000:9000"
   - "5432:5432"
  environment:
   - SONARQUBE_JDBC_URL=jdbc:postgresql://localhost:5432/sonar
   - SONARQUBE_JDBC_PASSWORD=[SONAR_DB_USER_PASSWORD]
  restart: always
  volumes:
   - sonarqube-extensions-plugins:/opt/sonarqube/extensions/plugins
   - /opt/sonarqube
db:
  image: postgres
  net: container:sonarqube
  environment:
   - POSTGRES_USER=sonar
   - POSTGRES_PASSWORD=[SONAR_DB_USER_PASSWORD]
  restart: always
  volumes:
   - postgresql-data:/var/lib/postgresql/data
   - postgresql-run:/run/postgresql

Update the file with a difficult to guess SONAR_DB_USER_PASSWORD and start SonarQube withsudo docker-compose up -d. When SonarQube starts, you should access it on http://localhost:9000 and change the default admin password from admin to something else. Database data is stored in mapped host directories, and therefore it’s persisted when containers are restarted. SonarQube plugins can be installed by downloading JAR files to the sonarqube-extensions-plugins directory and restarting the server. Install GitHub Plugin to enable integration with GitHub.

Important thing to add: for future integration with Travis CI, both application and database should be accessible from Travis CI build boxes.

Pull Approve

pull-approveIn order to integrate your project with Pull Approve, login with your GitHub account on https://pullapprove.com/ and you’ll be offered to enable integration for repositories you are authorised to control. Then you need to add .pullapprove.yml file with Pull Approve settings to the root of your repository. Here is the content of this file in our repository. All available configuration options are described on the Pull Approve site.

Travis CI

travisAnd now let’s review the integration with Travis CI. It starts the same way as integration with Pull Approve – login to the site, enable the integration and add .travis.yml file. The contents of the file heavily depends on the build tool, languages and technologies which are used in your project. Our version of the file became quite complicated because we need to use certain version of docker and making it available on build boxes deserves a separate blog post. Travis CI site contains a detailed description of available options and samples for nearly all commonly used technologies. Based on the images above you have already concluded that our build runs somewhat differently depending on what is being built. This logic is implemented in travis.sh, which is being called from .travis.yml.

travis.sh explained

For every build run, Travis CI provides a number of default environment variables, which enable determining what is being built. When using TRAVIS_PULL_REQUEST we detect PR build we extend the build with SonarQube preview scan and add system properties to configure connections with SonarQube server and GitHub. Installed GitHub Plugin for SonarQube enables commenting on pull request in GitHub.

Values of the system properties are taken from our repository specific environment variables:

travis env variables

The GitHub authentication token and SonarQube database user password should not be visible in Travis CI logs. Therefore they are marked as secure environment variables. Actually, because environment variables in Travis CI are not editable, only the person, who is adding these variables will know the values.

open and secure environment variables

Since pull requests in GitHub can be initiated from forks of your repository, Travis CI does not expose the secure variables to untrusted code. We use TRAVIS_SECURE_ENV_VARS default variable to detect builds on externally modified code and to avoid use of sensitive data. The environment variable TRAVIS_BRANCH is used to detect build on default branch and to refresh data in SonarQube database.

Conclusion

Although both Pull Approve and Travis CI are very new services and it took a while to understand the integration details, our setup seems to be very stable and flexible. Most importantly, it enables us to keep an eye on quality of our code. We use Gradle as the build tool for our project. However, none of the integrations of the components depend on it, and you should be able to use Maven, Ant, or shell scripts as long as you have a way to invoke SonarQube runner.

Do you think we can improve or simplify our setup even further? Do you want to share your experience of setting up a CI environment using online services? Please, do not hesitate to let us know by commenting or reaching out to @sadovnikov or @containersoluti on Twitter.

The DevOps Zone is brought to you in partnership with Sonatype Nexus. Use the Nexus Suite to automate your software supply chain and ensure you're using the highest quality open source components at every step of the development lifecycle. Get Nexus today

Topics:
travis ci ,github ,sonarqube ,jenkins ,code reivew ,continious integration

Published at DZone with permission of Viktor Sadovnikov. See the original article here.

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}