Using Shipa to Automate Kubernetes Object Creation

If you’re like most developers, you don’t have a ton of spare time (and perhaps also not much desire) to learn how to create and maintain Kubernetes objects. Where developers are required to manage Kubernetes, slower application delivery is often the result. In these scenarios, platform and DevSecOps teams must also work to support developers’ Kubernetes efforts, and cope with ever-greater chances that deployed applications will suffer from misconfigured objects. Platform teams are usually tasked with creating an extra platform layer as well, adding to costs and making maintenance and scaling more difficult (while also increasing the potential for failure).

Take a scenario in which you deploy a single 3-tier application. This requires maintaining a dozen Kubernetes objects, creating a container image of your application, and optimizing the container OS for storage, security, compliance, and more. Enterprise IT must carefully consider the time it takes for developers to deploy and operate the application, scale clusters, and create Kubernetes objects and YAML files. They must also factor in the tools they use for security monitoring, the frequency with which they must write Ansible and Terraform scripts, and whether they have time to focus on cloud infrastructure APIs.

Facing the challenges of development while engaging with Kubernetes myself, I’ve found the “landing pad for cloud native applications” offered by Shipa and Oracle Kubernetes Engine (OKE) pretty inviting.

Shipa is relatively new (the technology was released in October 2020, and the open source deployment engine, Ketch, was released in November). It’s designed to make it simpler for developers to run their code on Kubernetes in production. As a continuous operation platform, Shipa abstracts underlying cloud and Kubernetes infrastructure so that developers can forget about servers and focus on application deployment. Shipa introduces the concept of landing pads called “pools,” representing the destination where a CI/CD pipeline is to deliver application code. When code arrives, these pools create the necessary application objects across multiple OKE clusters. They also run security scans, attach policies, manage resource limits, produce application metrics, etc. The Shipa API has a scheduler algorithm that intelligently distributes applications across OKE nodes.

In this way, Shipa offers a simplified solution for delivering cloud-native applications across heterogeneous clusters and cloud environments, supporting developers, and accelerating application delivery. Specifically, teams can hook Shipa pools into their CI/CD pipelines and deploy application code instead of objects, relying on the pool itself to create all objects required by the application across the OKE clusters. Shipa also includes a Ship CLI tool that assists developers in managing their applications. Overall, Shipa enables more focus on application development and faster delivery by taking infrastructure concerns off the table, no matter what environment or environments are used.

How to Connect OKE Clusters to Your Shipa Instance

To begin using Shipa, first make a pool to connect to the OKE cluster. To create a pool, use the following YAML code (also available on GitHub):

xxxxxxxxxx

    ShipaPool: oke

    Resources:

      General:

        Setup:

          Default: false

          Public: false

          Provisioner: kubernetes

          Force: false

          KubeNamespace: ""

        Plan:

          Name: k8s

        Security:

          Disable-Scan: false

          Scan-Platform-Layers: false

          Ignore-Components:

          - systemd

          Ignore-CVES:

          - CVE-2019-18276

          - CVE-2016-2781

          - CVE-2019-9169

        Access:

          Append:

          - admin

        Services:

          Append:

          - mongo-service

        Volumes:

          Name: 

            Definition: 

            - Name: volume-x

            - Platform: csi-provider-x

            - Option1: capacity=10Gi

            - Option2: access-modes=ReadWriteOnce

        AppQuota:

          Limit: "8"

      ShipaNode:

        AutoScale: null

Now create the pool using this command: $ shipa pool-add k8s-pool-sample.yaml

Next, confirm that the pool has been created, either through the dashboard or with this command: $ shipa pool-list

With the pool available, connect existing Kubernetes clusters to Shipa using the following process. You’ll need a Kubernetes API URL, a Kubernetes CA Certificate, and service token.

API URL

Shipa uses this URL to access the Kubernetes API. Kubernetes exposes several APIs: use the base URL they share in common, for example https://kubernetes.example.com, not https://kubernetes.example.com/api/v1.

You can get the Kubernetes API URL by using this command: kubectl cluster-info | grep 'Kubernetes master' | awk '/http/ {print $NF}'

CA Certificate

Shipa requires a valid Kubernetes certificate to authenticate access the cluster. A certificate is created by default, the contents of which we can access with the following command. That certificate content should then be saved to a .crt file:

kubectl get secret $(kubectl get secret | grep default-token | awk '{print $1}') -o jsonpath='{.data.ca\.crt}' | base64 --decode > ca.crt

Token

Shipa uses service tokens, scoped to a specific namespace, to authenticate Kubernetes. This token must belong to a service account holding cluster-admin privileges. Create the necessary service account by following these steps:

Create a file named shipa-admin-service-account.yaml, containing the following.

xxxxxxxxxx

apiVersion: v1

kind: ServiceAccount

metadata:

  name: shipa-admin

  namespace: kube-system

---

apiVersion: rbac.authorization.k8s.io/v1beta1

kind: ClusterRoleBinding

metadata:

  name: shipa-admin

roleRef:

  apiGroup: rbac.authorization.k8s.io

  kind: ClusterRole

  name: cluster-admin

subjects:

- kind: ServiceAccount

  name: shipa-admin

  namespace: kube-system

Next, apply the service account and cluster role binding to the cluster.

kubectl apply -f shipa-admin-service-account.yaml

Now retrieve the token for the shipa-admin service account.

kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep shipa-admin | awk '{print $1}')

From the output, copy the <authentication_token> value.

Putting it Together

With these three pieces of information available, add your existing Kubernetes cluster to Shipa:Shell with this command:

$ shipa cluster-add <cluster-name> --addr https://<API URL> --cacert <file-name>.crt --pool oke --token=<authentication_token>

Now that you’ve connected one OKE cluster, repeat the process to add more clusters, or begin deploying applications.

Application Deployment

Shipa supports deploying applications via these four methods:

1) CI/CD

Shipa includes the shipa-ci utility, capable of enabling all major CI/CD platforms to deploy applications to Shipa.

2) Git

Shipa platforms are available to enable Git deployments. These platforms also usefully track differences among deployments and offer code version control. 

3) App-deploy

This method also utilizes Shipa platforms, and offers the full benefits of automated deployments.

4) Docker image

These deployments utilize Docker images in a registry, making certain that the same image runs in development and production. 

Example

Here I’ll deploy a basic Python Flask app with Shipa. To begin, clone the following Python sample app. The app prints a “Hello World” string.

$ git clone https://github.com/shipa-corp/python-sample

$ cd python-sample

$ shipa app-create blog python -t admin -o oke-psarkar

$ shipa app-list

$ shipa app-deploy -a blog

You'll now see something like:

You’ve successfully deployed an application to OKE, with no required knowledge of the Docker, Kubernetes, and ingress constructs involved. Now if you log into the Shipa Instance, you’ll see the app under Applications.

Here you can view specific information on the app, such as Metadata, Resources, Security, and more.