Deploying From CircleCI to Linode and Other VPS Providers

The ultimate goal of building and testing software is generally to deploy that software. The official CircleCI blog and many community blogs showcase examples on deploying to Kubernetes clusters, pushing to Docker registries, uploading to AWS S3, and even new serverless techniques. However, for solo developers or even small companies, using large deployment platforms like these is not always practical.

But there are other options.

In this blog post, we’re going to cover a few ways to deploy your software to traditional Virtual Private Servers (VPS) such as those provided by Linode, DigitalOcean, or Vultr.

SSH keys

All of the tools we’re going to cover will send your data over the internet in order to deploy your software. That being the case, having a secure connection is very important. Many examples in this post will use SSH keys to encrypt the connection. I suggest creating a brand new SSH keypair specifically for CircleCI. My favorite resource for creating SSH keys is by GitHub.

The private key should be stored on CircleCI via the web app. We have instructions you can follow here. The public key should be stored in the authorized_keys file for the user on the remote server you’d like to deploy to. For example, if you were deploying to a Linux server prod@example.com, the public key should be appended to the file /home/prod/.ssh/authorized_keys on that remote machine.

Rsync

rsync is a file transfer tool that has been around for a while. The name comes from “remote synchronization,” which means it allows you to transfer files from one directory or machine to another, but also keep them in sync. This is useful because this purpose means that rsync can compare files on a remote server and not transfer for them if they already exist, making for faster and more efficient deployments.

This works very well when you have several files to transfer, some updated and some not, such as when building a statically-generated website with Hugo or Jekyll. Basic Rails, Flask, and React apps can be deployed this way as well.

Usage

rsync -va --delete ./my-files/ user@example.com:/var/www/html

For rsync, you provide options, the file or directory you want to transfer, and then where you want to transfer to.

• -va - The ‘v’ means turn on verbosity so that we can see what’s happening in case something goes wrong, while the ‘a’ means archive mode. Archive mode turns on many useful options such as recursion, the copying of file permissions and ownsership, and more.
• --delete - This tells rsync that if a file has been deleted on CircleCI, it should be deleted from the remote server as well. Old files hanging around can lead to unsual issues where production doesn’t match your local environment, causing unwanted effects.
• ./my-files/ - The directory whose contents we want to copy. Notice that there’s a trailing slash, though with rsync, no trailing slash. ./my-files without the trailing slash means “copy this whole directory including everything in it.” ./my-files/ with the trailing slash means “only copy the contents of the directory and not the directory itself.”
• user@example.com - ‘user’ is the username on the remote server ‘example.com’ that we want to deploy to.
• :/var/www/html - This is the file path on the remote server where rsync should be sending the files to. For web servers, this should be the DocumentRoot.

Installing

While some Docker images may already have rsync installed, the official CircleCI Images and the third-party CI Builds Images don’t. However, installing it is fairly straightforward:

# Ubuntu & Debian based Docker images
sudo apt-get update && sudo apt-get install rsync

# Alpine based Docker images
apk update && apk add rsync

CircleCI example

Here’s a snippet of the CircleCI config for my personal website (Feliciano.Tech), which is deployed via rsync:

deploy:
    docker:
      - image: cibuilds/hugo:0.42.1
    steps:
      - attach_workspace:
          at: src
      - add_ssh_keys
      - run: |
          echo 'web01.revidian.net ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJiGRY6N9WYQ0vy6cTiwAgNbc6ueJmVo/EafBtmT7bcD6cQMbipYM/KfYQ2lCn2TxqWepZKYoyoVQXgArycCOns=' >> ~/.ssh/known_hosts
          rsync -va --delete src/public/ staticweb@web01.revidian.net:www/feliciano.tech/public_html

Related Tools

There are other options aside from rsync which work similarly. scp, known as Secure Copy, and sftp, known as Secure FTP, are two tools that both utilize SSH for encryption and handle the basic file transfer process that you can use to deploy to a VPS.
Take note of that last one.
Old school FTP is completely insecure and should never be used.
Whenever you think FTP, think SFTP.

Docker and SSH

If you’re running a Dockerized app on Linode or a similar provider, you likely don’t have a sophisticated orchestration tool such as Kubernetes or Docker Swarm running. Of course, this doesn’t mean you can’t, but that tends to be less common for VPS users. So how do we “deploy” a Docker app from CircleCI without manually logging into the server? Let’s look at an example:

deploy:
    docker:
      - image: circleci/golang:1.10
    steps:
      - attach_workspace:
          at: src
      - add_ssh_keys
      - run: |
          docker build .
          docker login -u $DOCKER_USER -p $DOCKER_PASS
          docker push myuser/myapp
          echo 'example.com ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJiGRY6N9WYQ0vy6cTiwAgNbc6ueJmVo/EafBtmT7bcD6cQMbipYM/KfYQ2lCn2TxqWepZKYoyoVQXgArycCOns=' >> ~/.ssh/known_hosts
          ssh root@example.com <<'ENDSSH'
          docker pull myuser/myapp
          docker stop my-app
          docker rm my-app
          docker run --name=my-app --restart=always -v $PWD:/app -d myuser/myapp
          ENDSSH

In this deploy job, we build a new Docker image for our example app and push it to Docker Hub. This makes the app available outside of CircleCI. We then use a feature of the SSH command that allows us to run commands on a remote machine, in this case our production machine. We SSH in, pull the new Docker image onto the production machine, stop, remove, and start the app over with the updated image.

To keep the CircleCI config light, we could also save the remote Docker commands into a Bash script called update-app.sh or similar, and then run the script from CircleCI. That would look like this:

ssh root@example.com '/var/app/update-app.sh`

If you give this a try, let me know how you do!

To wrap up, using a VPS to deploy your application is an often-overlooked but viable option if you don’t need the flexible orchestration of larger providers like Kubernetes and AWS S3. My intention with this post is to showcase some of the other deployment options you can use along with CircleCI for shipping applications.