Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Building a Fully Automated CI/CD Process for API Development With WSO2 API Manager

DZone's Guide to

Building a Fully Automated CI/CD Process for API Development With WSO2 API Manager

Automation can be applied to your CI/C pipeline for API development. Learn how to set this up with the WSO2 API Manager.

· DevOps Zone ·
Free Resource

The need for DevOps innovation has never been greater. Get the results from over 100 business value assessments in this whitepaper, Digital Darwinism: Driving Digital Transformation, to see the positive impact of DevOps first hand.

It is the age of automation where everyone tries to minimize manual steps while building software. API management is no exception. The term "CI/CD," which expands to "Continous Integration/ Continous Deployment," is the more technical term for this automation process. According to this definition, it contains a 2-step process for automating your software development and delivery.

  • Continuous Integration — This means that whenever you make a change to your source code, it needs to be well tested before integrating into the existing stable codebase.
  • Continuous Deployment — This means that once the integration tests have passed, the deployment of the new piece of code to the relevant environments (staging, prod) needs to happen automatically as part of the build process.

When it comes to the development of enterprise software, enterprise architects design the software in such a manner that entire software system breaks down into several layers (layered architecture). The back end services, which implement the business logic, are developed with a general-purpose programming language and the help of existing frameworks. Implementing a CI/CD process on top of this kind of approach is quite straightforward since it is easier to write unit tests using the same programming language, and source code can be managed in a GitHub repository. With a tool like TravisCI, you can easily integrate the integration and deployment process together when there is a change in the source code.

WSO2 API Manager is a full lifecycle API management platform which comes with a componentized architecture to make life easier for API designers, API consumers, and API owners. WSO2 API Manager comes with a dedicated web application called API Publisher for designing and developing the APIs. Even though this is a pretty useful tool for manually building APIs through a proper governance workflow, applying a CI/CD process for this is not possible. But WSO2 API Manager is designed in such a manner that it exposes a set of REST APIs at the product core runtime level which can be used to build a proper CI/CD pipeline for API development.

This article is hugely inspired by the work done by @Manjula Rathnayake of the WSO2 team, where he has published a similar concept and source code on GitHub. I will be reusing his code to explain the process in a more granular and API Manager-focused manner.

At a higher level, we are going to build a CI/CD pipeline to deploy an API to the WSO2 API Manager, which connects to a given backend URL. In this scenario, the source code of the API definition is stored in a GitHub repository and there is a TravisCI build job configured to the master branch of that repository so that when there is a change occur in the GitHub repo, it will automatically deploy the API to the staging environment after testing in the development environment. This process is explained in the above GitHub repository and I'm using the diagram which was used there.

If you want to see how things working first, you can follow the steps mentioned here.

I'm going to explain what is actually happening under the hood so that anyone can use the scripts which are implemented here and change them according to their requirements. Once you clone the above GitHub repository, you will find there are 6 main directories under the root directory of the repository. The contents of those directories are explained below.

  • BackendServiceImpl: This is the source code of the backend service which is going to be exposed through the API created in WSO2 API Manager. This has been implemented using the WSO2 MSF4J framework. It contains a build script to build the flat jar for the microservice.
  • DeployBackendService: This directory contains the shell script to deploy the backend service to the relevant environment based on the flag set when running this script. This can be used to deploy the backend service to either a dev environment (tenant) or staging environment based on the passed value.
  • DeployAPI: This directory contains important information about the API definition and its metadata required for WSO2 API Manager. The API definition is included in the swagger.json file in a standard format. Users can define their API with this file. This information is not enough to create an API within the WSO2 API Manager. It requires a set of metadata about the API (e.g. endpoint URL, throttling, security, caching, mediation, etc.) to be passed in when creating an API. These metadata and the requests which need to be sent to the API Manager runtime are defined within a Postman collection file with the name "WSO2_API_PUBLISHER.postman_collection.json." This is the step where we connect with the API Manager runtime and use the product-level APIs to create the API definition within WSO2 API Manager. It contains the following requests in the given order.
  • Dynamic Client Registration request: First we need to register a dynamic client to create the API in the runtime. This is sent to the following URL
{{gatewayURL}}/client-registration/register

{{gatewayURL}}/token

{{gatewayURL}}/token

{{gatewayURL}}/api/am/publisher/apis?query=name:{{apiName}}

{{gatewayURL}}/api/am/publisher/apis

{{gatewayURL}}/api/am/publisher/apis/{{apiId}}/swagger

{{gatewayURL}}/token

{{gatewayURL}}/api/am/publisher/apis/change-lifecycle?action=Publish&apiId={{apiId}}

The above steps are sequentially configured in the Postman collection which will be executed by the "newman" tool. Once this script is executed, the API will be created in the relevant tenant and will be in the "published" state.

  • TestAPI: This directory includes a Postman collection which is used to test the deployed API. Before publishing into the upper environments, we need to make sure that tests are properly passing in the lower environments. We are executing the following steps within this Postman collection.
  • Dynamic Client Registration request: First we need to register a dynamic client to subscribe to the API in the runtime. This is sent to the following URL
{{gatewayURL}}/client-registration/register

{{gatewayURL}}/token

{{gatewayURL}}/api/am/store/applications?query=testingApp

{{gatewayURL}}/api/am/store/applications

{{gatewayURL}}/api/am/store/apis?query=name:{{apiName}}

{{gatewayURL}}/api/am/store/subscriptions

{{gatewayURL}}/api/am/store/applications/{{applicationId}}

{{gatewayURL}}/api/am/store/applications/generate-keys?applicationId={{applicationId}}

{{gatewayURL}}/token

{{gatewayURL}}/t/{{tenantDomain}}/hello/1.0.0/

Once the above steps are executed with the Postman collection, we move onto the next step of the execution flow.

  • DevEnvironment: This directory contains the environment variables related to the development environment. The file "Development.postman_environment.json" is passed in as an input parameter to Newman when executing the Postman collection stored in the "DeployAPI" section. The relevant URLs and tenant domain information are extracted from this file during the deployment. The contents within the "backendService.properites" are used when deploying the backend service within the bash script of "DeployBackendService" directory.
  • StagingEnvironment: This directory contains the same content as the previous "DevEnvironment" directory, which is related to the staging environment.

Last but not least, we have the Travis YAML descriptor file, which defines the execution flow of the build and deploy pipeline. The sequence is as follows:

  1. Build the backend service implementation
  2. Deploy the backend service to integration cloud "Development" tenant (environment)
  3. Deploy the API to the WSO2 API cloud "Development" tenant
  4. Test the API deployed at "Development" tenant
  5. Deploy the backend service to integration cloud "Staging" tenant
  6. Deploy the API to the WSO2 API cloud "Staging" tenant
  7. Test the API deployed at "Staging" tenant

With TravisCI, you can configure the above script to be executed when there is a change in the GitHub repository. You can follow the steps there to see this in action.

In this post, I have explained the steps to fully automate the deployment of APIs with WSO2 API Manager. You can tweak the steps mentioned in this post and define your own CI/CD pipeline. On top of this automation, if you have more than one node, you can have proper file sharing and database sharing mechanisms to deploy the APIs across all the nodes in the cluster.

Interested in Kubernetes but unsure where to start? Check out this whitepaper, A Roundup of Managed Kubernetes Platforms from Codeship by Cloudbees, for an overview and comparison of Kubernetes platforms. 

Topics:
devops ,tutorial ,api ,ci/cd

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}