Build, Test, and Deploy Apps Independently From a Monorepo

in our previous blog posts , we've shared our thoughts and experiences on our approach with monorepo. we received a few questions after that blog on how ci and deploys go with the monorepo.

in this article, we will learn how to run ci and build and deploy applications independently from a monorepo. on each pr and commit, we will run tests on the service that has changed build a docker image from it and push it to a registry. this image can then be deployed to a cluster on any supported container service . we will use shippable for this scenario.

setup

our monorepo has two projects. both of them are simple node js apps. www is a front-end app and will run selenium tests. api is the backend app, which we will test using superagent and chai. the initial setup can be a bit tricky and depends on how projects in your repository are structured. detecting what has changed in the repository is important. we will call a shell script from shippable.yml that will detect what has changed and trigger tests. when all our test pass successfully we will build a docker image and then push it to the registry.

shippable.yml :

language: node_js

services:
  - selenium

build:
  ci:
    - ./detectchangedfolders.sh

this yaml l will run the below shell script, which will detect changes in folders (i.e., services in our repository and run tests). also note that we are specifying all the services required in shippable.yml .

detectchangedfolders.sh :

#!/bin/bash -e
export ignore_files=$(ls -p | grep -v /)

detect_changed_folders() {
 echo "detecting changes for this build"
 folders=`git diff --name-only $shippable_commit_range | sort -u | awk 'begin {fs="/"} {print $1}' | uniq`
 export changed_components=$folders
}

run_tests() {
 for component in $changed_components
 do
 echo "----------------------------------------------"
 echo "$component has changed"
 if ! [[ " ${ignore_files[@]} " =~ "$component" ]]; then
 echo "-------------------running tests for $component---------------------"
 cd "$component"
 ./run.sh "$component"
 cd ..
 else
 echo "ignoring this change"
 fi
 done
}

if [ "$is_pull_request" != true ]; then
 detect_changed_folders
 run_tests
else
 echo "skipping because it's a pr"
fi

1. enable

enable the project on shippable. now, when navigating to the pipelines tab, you can view your project on spog.

on every build triggered, the changed folders are detected and an image is built and pushed to the registry. in our example, the run.sh file that we have in every folder is responsible for triggering tests, building an image, and pushing it to the registry.

2. deploy

now that our images are built, we will use pipelines to deploy these images. we will begin our resources and jobs in shippable.resources.yml and shippable.jobs.yml , respectively.

here is how we will define resources and jobs for our pipeline:

resources:
 - name: api-img
 type: image
 integration: dockerhub
 pointer:
 sourcename: "chetantarale/sampleapp"
 seed:
 versionname: master.3

 - name: www-img
 type: image
 integration: dockerhub
 pointer:
 sourcename: "chetantarale/sampleapp"
 seed:
 versionname: master.3

 - name: ddc-cluster
 type: cluster
 integration: ddc-cluster

 - name: www-options #required
 type: dockeroptions #required
 version:
 portmappings: #optional
 - "51000:51000"

 - name: api-options #required
 type: dockeroptions #required
 version:
 portmappings: #optional
 - "50000:50000"

 - name: dockerhub
 type: integration
 integration: "dockerhub"

www-img and api-img are the two images that will be built when the ci job runs. these images are the outs from the ci job.

dockerhub is a resource of type integration that points to the registry where we will push our image. ddc-cluster is an integration of type cluster where we will deploy our app. www-options and api-options are the options that can be appended to a docker image to be pushed.

we will define the following jobs in our job's yaml:

• a runci job that builds our images.
• two manifest jobs to generate a manifest.
• two deploy jobs that deploy our app.

jobs:
 - name: monorepo_sample_runci
 type: runci
 steps:
 - out: api-img
 - out: www-img
 on_success:
 - script: echo 'on success !!!!!'
 on_failure:
 - script: echo 'failed job .... :('

 - name: www-man
 type: manifest
 steps:
 - in: www-options
 - in: www-img
 force: true

 - name: www-deploy
 type: deploy
 steps:
 - in: www-man
 - in: ddc-cluster

 - name: api-man
 type: manifest
 steps:
 - in: api-options
 - in: api-img
 force: true

 - name: api-deploy
 type: deploy
 steps:
 - in: api-man
 - in: ddc-cluster

now, push these files and add this repo as syncrepo from shippable ui . shippable will read the jobs and resource's yamls and create your pipeline.

we have successfully set up our pipeline. for any changes to a project within in the monorepo, the ci for that corresponding project is triggered and a docker image is built and deployed.