DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Last call! Secure your stack and shape the future! Help dev teams across the globe navigate their software supply chain security challenges.

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Releasing software shouldn't be stressful or risky. Learn how to leverage progressive delivery techniques to ensure safer deployments.

Avoid machine learning mistakes and boost model performance! Discover key ML patterns, anti-patterns, data strategies, and more.

Related

  • Did We Build The Right Thing? That's What UAT Is About.
  • 5 Simple Tips to Keep Dockerized Apps Secure
  • Taking KubeMQ Build & Deploy for a Test Drive: My Thoughts and Impressions
  • Deploying a Simple Golang Web App on Heroku

Trending

  • The Human Side of Logs: What Unstructured Data Is Trying to Tell You
  • Unlocking the Potential of Apache Iceberg: A Comprehensive Analysis
  • The Cypress Edge: Next-Level Testing Strategies for React Developers
  • Cookies Revisited: A Networking Solution for Third-Party Cookies
  1. DZone
  2. Software Design and Architecture
  3. Microservices
  4. Build, Test, and Deploy Apps Independently From a Monorepo

Build, Test, and Deploy Apps Independently From a Monorepo

Set up your pipeline so that when changes occur to a project in the monorepo, the CI for that corresponding project is triggered and a Docker image is built and deployed.

By 
Chetan Tarale user avatar
Chetan Tarale
·
Mar. 29, 17 · Tutorial
Likes (2)
Comment
Save
Tweet
Share
15.8K Views

Join the DZone community and get the full member experience.

Join For Free

in our previous blog posts , we've shared our thoughts and experiences on our approach with monorepo. we received a few questions after that blog on how ci and deploys go with the monorepo.

in this article, we will learn how to run ci and build and deploy applications independently from a monorepo. on each pr and commit, we will run tests on the service that has changed build a docker image from it and push it to a registry. this image can then be deployed to a cluster on any supported container service . we will use shippable for this scenario.

setup

our monorepo has two projects. both of them are simple node js apps. www is a front-end app and will run selenium tests. api is the backend app, which we will test using superagent and chai. the initial setup can be a bit tricky and depends on how projects in your repository are structured. detecting what has changed in the repository is important. we will call a shell script from shippable.yml that will detect what has changed and trigger tests. when all our test pass successfully we will build a docker image and then push it to the registry.

shippable.yml :

language: node_js

services:
  - selenium

build:
  ci:
    - ./detectchangedfolders.sh

this yaml l will run the below shell script, which will detect changes in folders (i.e., services in our repository and run tests). also note that we are specifying all the services required in shippable.yml .

detectchangedfolders.sh :

#!/bin/bash -e
export ignore_files=$(ls -p | grep -v /)

detect_changed_folders() {
 echo "detecting changes for this build"
 folders=`git diff --name-only $shippable_commit_range | sort -u | awk 'begin {fs="/"} {print $1}' | uniq`
 export changed_components=$folders
}

run_tests() {
 for component in $changed_components
 do
 echo "----------------------------------------------"
 echo "$component has changed"
 if ! [[ " ${ignore_files[@]} " =~ "$component" ]]; then
 echo "-------------------running tests for $component---------------------"
 cd "$component"
 ./run.sh "$component"
 cd ..
 else
 echo "ignoring this change"
 fi
 done
}

if [ "$is_pull_request" != true ]; then
 detect_changed_folders
 run_tests
else
 echo "skipping because it's a pr"
fi

1. enable

enable the project on shippable. now, when navigating to the pipelines tab, you can view your project on spog.

selection_006.png

on every build triggered, the changed folders are detected and an image is built and pushed to the registry. in our example, the run.sh file that we have in every folder is responsible for triggering tests, building an image, and pushing it to the registry.

2. deploy

now that our images are built, we will use pipelines to deploy these images. we will begin our resources and jobs in shippable.resources.yml and shippable.jobs.yml , respectively.

here is how we will define resources and jobs for our pipeline:

resources:
 - name: api-img
 type: image
 integration: dockerhub
 pointer:
 sourcename: "chetantarale/sampleapp"
 seed:
 versionname: master.3

 - name: www-img
 type: image
 integration: dockerhub
 pointer:
 sourcename: "chetantarale/sampleapp"
 seed:
 versionname: master.3

 - name: ddc-cluster
 type: cluster
 integration: ddc-cluster

 - name: www-options #required
 type: dockeroptions #required
 version:
 portmappings: #optional
 - "51000:51000"

 - name: api-options #required
 type: dockeroptions #required
 version:
 portmappings: #optional
 - "50000:50000"

 - name: dockerhub
 type: integration
 integration: "dockerhub"

www-img and api-img are the two images that will be built when the ci job runs. these images are the outs from the ci job.

dockerhub is a resource of type integration that points to the registry where we will push our image. ddc-cluster is an integration of type cluster where we will deploy our app. www-options and api-options are the options that can be appended to a docker image to be pushed.

we will define the following jobs in our job's yaml:

  • a runci job that builds our images.
  • two manifest jobs to generate a manifest.
  • two deploy jobs that deploy our app.
jobs:
 - name: monorepo_sample_runci
 type: runci
 steps:
 - out: api-img
 - out: www-img
 on_success:
 - script: echo 'on success !!!!!'
 on_failure:
 - script: echo 'failed job .... :('

 - name: www-man
 type: manifest
 steps:
 - in: www-options
 - in: www-img
 force: true

 - name: www-deploy
 type: deploy
 steps:
 - in: www-man
 - in: ddc-cluster

 - name: api-man
 type: manifest
 steps:
 - in: api-options
 - in: api-img
 force: true

 - name: api-deploy
 type: deploy
 steps:
 - in: api-man
 - in: ddc-cluster

now, push these files and add this repo as syncrepo from shippable ui . shippable will read the jobs and resource's yamls and create your pipeline.

selection_007.png

selection_010.png

we have successfully set up our pipeline. for any changes to a project within in the monorepo, the ci for that corresponding project is triggered and a docker image is built and deployed.

app Testing Monorepo Build (game engine) Docker (software) career

Published at DZone with permission of Chetan Tarale, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • Did We Build The Right Thing? That's What UAT Is About.
  • 5 Simple Tips to Keep Dockerized Apps Secure
  • Taking KubeMQ Build & Deploy for a Test Drive: My Thoughts and Impressions
  • Deploying a Simple Golang Web App on Heroku

Partner Resources

×

Comments

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends: