Creating a Microservices Build Pipeline Using goPaddle
Creating a Microservices Build Pipeline Using goPaddle
Using the goPaddle tool to create microservice designs built for Continuous Delivery.
Join the DZone community and get the full member experience.Join For Free
goPaddle is a DevOps platform that helps to build, deploy, and scale microservices. The key benefit of goPaddle is to design once and deploy it anywhere. A design in goPaddle is a logical grouping of microservices called components. Each component contains a microservice artifact and its deployment specification. A microservice design can be deployed on a container cluster like Kubernetes for web scale deployments and easy service lifecycle management.
In this article, we will see how to create a build pipeline using a micr0services design created using goPaddle. To get started, the goPaddle hosted platform can be used to create a microservice design. Using this design, we can build a release pipeline using the goPaddle Jenkins plugin. Here is a quick view of the steps involved in creating a pipeline:
1. Register a source control account in goPaddle portal
2. Create a microservice design in the goPaddle portal
3. Register / Create a Kubernetes Cluster in the goPaddle portal
4. Set up Jenkins environment
5. Install and configure goPaddle plugin
6. Create a design deployment job
7. Create a design test job
8. Create a build pipeline
1. Register a Source Control Account
Continuous Delivery pipelines can be built using a design in which one or more components are built from a private source control repository. In order to use a private source control repository, we must first register the source control repository with goPaddle.
Login to the goPaddle portal and navigate to the profile settings.
Click on "Source Control", select the GitHub icon, and authenticate access to the GitHub account. Now the GitHub account is registered with goPaddle. This account can be used to authenticate any private repositories while creating a microservice design component.
Fork the repository https://github.com/gopaddledemo/springboot.git in to your GitHub account. Use this private repository to build a sample design.
2. Create a Microservice Design
There are different options through which one can create a goPadde design, such as importing an existing Kubernetes template, importing artifacts from a source control repository, importing a docker image from Docker hub, or creating a fresh new microservice by defining its service endpoints.
In this example, let's create a design from the private GitHub repository that was forked in step 1. In the goPaddle portal, navigate to the Designs section in the top navigation banner and click on NEW DESIGN.
From the components panel, select "GitHub" as the repository. This would place a microservice component in the design.
Under PROPERTY section fill out the general property of the component.
Provide the Url of the GitHub private repository. Select the branch/tag and the platform to build and authenticate GitHub access in order to enable Continuous Delivery at the time of deployment. In this example, consider the repository Url as https://github.com/<your git account>/springboot.git and the platform as "Java 1.7".
Under SCRIPT section, provide the build script as mvn package which builds and packages the microservice using maven and the start script as java -jar springboot-jersey-1.0-SNAPSHOT.jar to run the service as a Java application.
Under NETWORK property, specify the port as 8080. This exposes the microservice on port 8080 to the outside world.
Save the component.
Now click "Add Edge" and link the www component to the GitHub component by clicking on www and dragging a line to GitHub. The design will look like this:
Rename the design and publish it.
Now the design is ready for deployment.
3. Register/Create a Kubernetes Cluster:
To deploy the design, either register an external Kuberenetes cluster or create a cluster in Google Container Engine. goPaddle allows Single-Sign-On using a Google cloud account, so that Kubernetes clusters can be created seemlessly using Google Container Engine (GKE). From the goPaddle portal, click on the user profile and navigate to the settings page. In the setting page, under the Kubernetes section, select "Create Kubernetes". Enter the cluster specification and the cluster would get created in your Google Cloud account. You can choose to create multiple clusters, one-by-one for Dev, QA, Stage, and Production environments.
Once the goPaddle design and the Kubernetes environments are ready, a build pipeline can be created using the Jenkins plugin.
4. Setup Jenkins Environment
Launch an Ubuntu 14.0 LTS instance in Amazon or Google Cloud with the init script as below. This installs Java Jre, Jenkins, and the required plugins. Ensure that the firewall ports are open in the Cloud Console, in order to access the Jenkins service on port 8080.
echo "188.8.131.52" >> /etc/resolv.conf curl https://gist.githubusercontent.com/trovsankar/6504e5f69f069925118339375981fe66/raw/e00cf01d27a13bc983f6d610cfe8db24203c81d4/Jenkins-demo.sh > jenkininstall bash jenkininstall
5. Install and Configure goPaddle Plugin
Download the goPaddle Jenkins plugin from here. Install the plugin by navigating to this url in the browser http://<jenkins-vm-ip>:8080/pluginManager/advanced and upload the plugin hpi file under the "Upload Plugin" section.
In order to authenticate access to the goPaddle platform, navigate to the url http://<jenkins-vm-ip>:8080/configure and under goPaddle section, configure your Google Service EmailId and the P12 file.
You can obtain the Google Service Email Id from the Google cloud console ccount under https://console.developers.google.com/permissions/serviceaccounts?project=<project-id>
If a fresh new service email ID needs to be created, then in the Google Cloud Console, navigate to the Credentials section and create a Service account key.
Select p12 as the key type and store the p12 file on your computer.
Every time a new service account key is created, the goPaddle portal has to be updated to authneticate access using the newly generated key. From the goPaddle portal, click on the user profile and navigate to the setting page. Under settings page, in the General section, update the Google Service accounts.
Now the service account email ID and the p12 file can be used in the goPaddle plugin configuration.
6. Create a Design Deployment Job
From the Jenkins portal, create a freestyle Jenkins job and name it "goPaddle-deploy-job".
Select the option, "This build is parameterized" and provide the Name and Value of the String Parameter as cdelivery.
Under the build configuration section, find the option to build the project using goPaddle.
Type the design name that was created in step 2 and the Kubernetes cluster where the design needs to be deployed. Select the "Continuous Delivery" option and save the job.
Once this job is executed, it exports the environment variable accessUrl, which can be used in a dependent job. The accessUrl has the IP and the port number on which the application can be accessed. In this example, the jersey application that gets deployed will be accessible at <accessUrl>/rest/hello . Let us reuse this accessUrl in a QA job to test the service end point.
7. Create a Design Test Job
Create a QA job and name it "goPaddle-QA-job". Under Advanced Project Options, add a build step to execute the shell script below:
# Parent job returns the environment variable "$accessUrl" once the design is launched using the goPaddle plugin echo "Access URL is -> $accessUrl" if [ -f "demoQA-ng-v1.2.jar" ]; then echo "File already founded" else echo "Downloading Jar file....." #wget https://storage.googleapis.com/gopaddle/demoQA.jar wget https://storage.googleapis.com/gopaddle-jenkins-plugin/DemoQA/demoQA-ng-v1.2.jar fi sleep 2m # Pod restarting intervel time java -jar -DUrl=$accessUrl/rest/hello demoQA-ng-v1.2.jar
This script downloads a jar file which contains the test script and executes the test script along with the accessUrl parameter. Once the test is complete, it places the test output in the selected html report folder.
As a post build opertation, select the directory where the HTML reports need to be placed.
8. Create a Build Pipeline
Once the jobs are ready, you can create a pipeline by linking the Jobs and then creating a pipeline view in Jenkins. Edit the "goPaddle-deploy-job" job and add a post-build action to trigger a parameterized build.
Add a new string parameter as below and select the QA job as the post build trigger.
Create a pipeline view, with "goPaddle-deploy-job" as the initial job.
Now a continuous build pipeline is ready for microservices deployments.
Advantages of Using goPaddle CI/CD Pipeline
By enabling Continuous Delivery for a design, any component upgrades within the design will result in rebuilding and redeploying the service in the target environment.
Jenkins plugin gives the flexibility of reusing the designs developed in the goPaddle portal while leveraging the familiarity of building CI/CD pipelines using Jenkins.
Different jobs can be created targeting differen Kubernetes clusters thus a design can be reused across Dev, QA, Stage and Production environments.
For more information, get in touch with us at firstname.lastname@example.org.
Opinions expressed by DZone contributors are their own.