When appropriate tools and technology are available in abundance, automation is considered the primary resource to improve the efficiency of any software development process. Faster innovation cycles and speed to market are crucial to a business. With the very same principle in mind, Shippable was founded so that software powered organizations could accelerate innovation. Shippable provides an integrated platform, using which you can automate your software delivery from source control to production, without needing to write complex, imperative code.

In this tutorial, we will learn how to create an end to end deployment for a single service, with source code maintained in a repository on Github. We will also see how to trigger tests every time the deployed environment is updated or your test cases are updated. 

Before You Begin

Here are the links for the repositories used in this tutorial.

• syncRepo
• sampleapp_repo
• bat_repo

All our tasks, task definitions, and their dependencies are defined as resources and jobs. For this setup, we will use a separate repository and define resources and jobs there. This repository will act as a syncRepo. On adding a syncRepo, Shippable will read the jobs and resources ymls and create your pipeline.  

We will need the following resources to successfully set up a pipeline

• syncRepo                                              # to define our jobs and images
• gitRepo                                                  # this is where our app code resides
• An Integration of type hub                     # to push the app image
• An Integration of type deploy                 # to deploy our app

Before we begin with our code, we will create integrations through shippable UI. We will create a dockerhub integration where we will push our image. We will also create a docker data center integration and this is where we will deploy our app. You can push and deploy your image to many other integrations that are part of the platform.

Defining Resources and Jobs

Next, define your resources and jobs in shippable.resources.yml and shippable.jobs.yml, respectively. Here is how we will define resources and jobs for our pipeline.

Resources:
  # Pointer to a repository on source control
  - name: sampleapp_repo
    type: gitRepo
    integration: github
    pointer:
      sourceName: shipppableSamples/sampleApp
      branch: master

  # Points to docker registry where we will push our image  
  - name: dockerhub
    type: integration
    integration: "dockerhub"

  # Points to the image that we will push
  - name: app-img
    type: image
    integration: dockerhub
    pointer:
      sourceName: "library/sampleapp"
    seed:
      versionName: latest

  # A cluster to deploy our app
  - name: ddc-cluster
    type: cluster
    integration: ddc-cluster

  # docker options that can be appended to a docker image to be pushed
  - name: options                  
    type: dockerOptions        
    version:
      portMappings:                         
        - "51000:51000"

We will define the following 3 jobs in our jobs yml:

• A runSh job that builds our app image.
• A manifest job to generate a manifest.
• A deploy job that deploys our app.

jobs:
  - name: build-app
    type: runSh
    steps:
      - IN: sampleapp_repo
      - IN: dockerhub
      - OUT: app-img
      - TASK:
        - script: ./IN/sampleapp_repo/gitRepo/build.sh
    on_success:
      - script: echo 'on success !!!!!'
    on_failure:
      - script: echo 'Failed job .... :('

  - name: sampleapp-man
    type: manifest
    steps:
      - IN: options
      - IN: app-img
        force: true
        versionName: latest

  - name: sampleapp-deploy
    type: deploy
    steps:
      - IN: sampleapp-man
      - IN: ddc-cluster

The first job is build-app, this job will take the resource sampleapp_repo  as the input and dockerhub integration. The git repository will be cloned into the build machine, and the dockerhub credentials are also available on the build machine. We have defined a task section which will run the script. The present at the specified location clones the repository at the location: ./IN/ /gitRepo/ . So  build.sh  is the script present on the source control. This job builds a docker image and pushes it to dockerhub.  app-img  is the output of this job.

A manifest job is triggered when the build-app  job runs successfully. The image is input to the manifest job and Manifest jobs generate a new version of the manifest each time anything in the manifest changes. Every time a new image is built a new manifest gets generated, or when you change the docker Options which is also an IN to this job.

When a manifest is generated successfully, the deploy rob is triggered and the app is deployed to the cluster which is an IN to the deploy job. The ddc-cluster in this case which is defined in  shippable.resources.yml .

Adding a syncRepo

Now push these files and add this repo as syncRepo from Shippable UI. Shippable will read the jobs and resources ymls and create your pipeline.

Running Jobs

Upon triggering the build job from the UI, the build.sh file will run and a docker image is built and pushed to docker hub. The instructions to build and push the image are in the build.sh file. Here is how your pipeline will look when all the jobs run successfully. At this point, our app is deployed to the Docker data center. On visiting the URL for it, we can see that our app has been deployed.

Running Tests in Pipelines

Now that our app is deployed we will hook in another job into the pipelines that will test our app. All the tests are written and pushed to another repo called bat(build acceptance test). We will define another job called bat which is of type runCI. runCI jobs allow us to use CI jobs that will do acceptance tests in the pipelines.We will first go and enable this project on Shippable.

Then define this job in our yml:

  - name: bat_runCI
    type: runCI
    steps:
      - IN: bat_params
      - IN: sampleapp-deploy

This job takes the  sampleapp-deploy job as input and another resource called  bat_params. bat_params are some parameters that you want in your  runCI job. Tokens can be encrypted and passed as params from here. We will define bat_params in the resources yml as follows:

 - name: bat_params
    type: params
    version:
      params:
        url: "www.example.com"
        port: 51000

If you are already using CI to run tests you can simply hook in our already existing CI job to your pipeline. Also, another reason to use CI jobs to run tests is that the build images comes with preinstalled services which can be used instead of setting them up yourself. Now on pushing these changes, the sync job will run and the runCI job is hooked into the pipeline. Now every time you deploy some changes to your app or commit some changes to your repo where you have defined your test (the repo bat in this case). The bat_runCI job is triggered and your changes are tested on your app.