When appropriate tools and technology are available in abundance, automation is considered the primary resource to improve the efficiency of any software development process. Faster innovation cycles and speed to market are crucial to a business. With the very same principle in mind, Shippable was founded so that software powered organizations could accelerate innovation. Shippable provides an integrated platform, using which you can automate your software delivery from source control to production, without needing to write complex, imperative code.
In this tutorial, we will learn how to create an end to end deployment for a single service, with source code maintained in a repository on Github. We will also see how to trigger tests every time the deployed environment is updated or your test cases are updated.
Before You Begin
Here are the links for the repositories used in this tutorial.
All our tasks, task definitions, and their dependencies are defined as resources and jobs. For this setup, we will use a separate repository and define resources and jobs there. This repository will act as a syncRepo. On adding a syncRepo, Shippable will read the jobs and resources ymls and create your pipeline.
We will need the following resources to successfully set up a pipeline
- syncRepo # to define our jobs and images
- gitRepo # this is where our app code resides
- An Integration of type hub # to push the app image
- An Integration of type deploy # to deploy our app
Before we begin with our code, we will create integrations through shippable UI. We will create a dockerhub integration where we will push our image. We will also create a docker data center integration and this is where we will deploy our app. You can push and deploy your image to many other integrations that are part of the platform.
Defining Resources and Jobs
Next, define your resources and jobs in shippable.resources.yml and shippable.jobs.yml, respectively. Here is how we will define resources and jobs for our pipeline.
Resources: # Pointer to a repository on source control - name: sampleapp_repo type: gitRepo integration: github pointer: sourceName: shipppableSamples/sampleApp branch: master # Points to docker registry where we will push our image - name: dockerhub type: integration integration: "dockerhub" # Points to the image that we will push - name: app-img type: image integration: dockerhub pointer: sourceName: "library/sampleapp" seed: versionName: latest # A cluster to deploy our app - name: ddc-cluster type: cluster integration: ddc-cluster # docker options that can be appended to a docker image to be pushed - name: options type: dockerOptions version: portMappings: - "51000:51000"
We will define the following 3 jobs in our jobs yml:
- A runSh job that builds our app image.
- A manifest job to generate a manifest.
- A deploy job that deploys our app.
jobs: - name: build-app type: runSh steps: - IN: sampleapp_repo - IN: dockerhub - OUT: app-img - TASK: - script: ./IN/sampleapp_repo/gitRepo/build.sh on_success: - script: echo 'on success !!!!!' on_failure: - script: echo 'Failed job .... :(' - name: sampleapp-man type: manifest steps: - IN: options - IN: app-img force: true versionName: latest - name: sampleapp-deploy type: deploy steps: - IN: sampleapp-man - IN: ddc-cluster
The first job is build-app, this job will take the resource
sampleapp_repo as the input and dockerhub integration. The git repository will be cloned into the build machine, and the dockerhub credentials are also available on the build machine. We have defined a task section which will run the script. The present at the specified location clones the repository at the location:
./IN/ /gitRepo/ . So
build.sh is the script present on the source control. This job builds a docker image and pushes it to dockerhub.
app-img is the output of this job.
A manifest job is triggered when the
build-app job runs successfully. The image is input to the manifest job and Manifest jobs generate a new version of the manifest each time anything in the manifest changes. Every time a new image is built a new manifest gets generated, or when you change the docker Options which is also an IN to this job.
When a manifest is generated successfully, the deploy rob is triggered and the app is deployed to the cluster which is an IN to the deploy job. The ddc-cluster in this case which is defined in
Adding a syncRepo
Now push these files and add this repo as syncRepo from Shippable UI. Shippable will read the jobs and resources ymls and create your pipeline.
Upon triggering the build job from the UI, the build.sh file will run and a docker image is built and pushed to docker hub. The instructions to build and push the image are in the build.sh file. Here is how your pipeline will look when all the jobs run successfully. At this point, our app is deployed to the Docker data center. On visiting the URL for it, we can see that our app has been deployed.
Running Tests in Pipelines
Now that our app is deployed we will hook in another job into the pipelines that will test our app. All the tests are written and pushed to another repo called
bat(build acceptance test). We will define another job called bat which is of type runCI. runCI jobs allow us to use CI jobs that will do acceptance tests in the pipelines.We will first go and enable this project on Shippable.
Then define this job in our yml:
- name: bat_runCI type: runCI steps: - IN: bat_params - IN: sampleapp-deploy
This job takes the
sampleapp-deploy job as input and another resource called
bat_params are some parameters that you want in your
runCI job. Tokens can be encrypted and passed as params from here. We will define bat_params in the resources yml as follows:
- name: bat_params type: params version: params: url: "www.example.com" port: 51000
If you are already using CI to run tests you can simply hook in our already existing CI job to your pipeline. Also, another reason to use CI jobs to run tests is that the build images comes with preinstalled services which can be used instead of setting them up yourself. Now on pushing these changes, the sync job will run and the runCI job is hooked into the pipeline. Now every time you deploy some changes to your app or commit some changes to your repo where you have defined your test (the repo bat in this case). The bat_runCI job is triggered and your changes are tested on your app.