Hands-On GitOps With OneDev and Kubernetes

Set Up Kubernetes Cluster

First, we need a Kubernetes cluster. You may use an existing one or set up a new one. For this tutorial, we chose to use GKE (Google Kubernetes Engine).

Just follow the quick start to create a cluster. To save your money, the default pool with only one node is sufficient for our testing. For node image type, use the default Container-Optimized OS; for machine type, please select one with at least 8G mem. After creating the cluster, go ahead to configure kubectl to connect to the cluster following this guide.

To verify that things are working, run below command in your terminal to make sure it is successful:

xxxxxxxxxx

$ kubectl cluster-info

Install OneDev Into Kubernetes Cluster

Now that we have a working cluster, let’s deploy OneDev into the cluster to manage code of our demo project

1. Download OneDev k8s resources, extract it, and run the below commands to deploy OneDev. Note that OneDev by default will apply two persistent volumes to store data. one is of 100G to store git repositories, and another is of 10G to store database files. This setting can be adjusted via file k8s-resources/production/disk-settings.yaml if desired. 
   Shell

    

   
   

   
   

   
   

   
   

   xxxxxxxxxx

   1

   
   

   
   

    

   1

   $ cd k8s-resources/production

   2

   $ kubectl apply -k .

   
   

   
   

   
   

2. After deployment, run the below command to show external IP of the OneDev service (you may need to wait a while for the external IP to be assigned):
   Shell

    

   
   

   
   

   
   

   
   

   xxxxxxxxxx

   1

   
   

   
   

    

   1

   $ kubectl get services -n onedev

   
   

   
   

   
   

3. Open the URL http:// with your browser to set up OneDev. Wait a while and try again if OneDev is not ready.

Add a Demo Project Into OneDev

OneDev should be up and running. Let’s set up a demo project now:

1. From OneDev projects page, add a project named gitops-demo.
2. From the terminal, create a React project, and push to OneDev by running the below commands. Note that you need to have a node.js environment. Refer to the React documentationif you are not familiar with it.
   Shell

    

   
   

   
   

   
   

   
   

   xxxxxxxxxx

   1

   
   

   
   

    

   1

   $ npx create-react-app gitops-demo 

   2

   $ cd gitops-demo

   3

   $ git remote add origin http:///gitops-demo

   4

   $ git push --set-upstream origin master 

   5

   (use OneDev admin account created previously for authentication)

   
   

   
   

   
   

3. Refresh files page of the demo project in OneDev, and click the “add build spec” link as below:
4. From the build spec edit page, switch to the Edit Source tab without adding any job and replace the source with below content. Don’t worry about the syntax. In most cases, you only need to work with the GUI editor.
   Plain Text

    

   
   

   
   

   
   

   
   

   xxxxxxxxxx

   1

   29

   
   

   
   

    

   1

   version: 1

   2

   jobs:

   3

   - name: CI

   4

     image: node:10.16-alpine

   5

     commands:

   6

     - set -e

   7

     - ''

   8

     - apk add --update jq

   9

     - buildVersion=`jq -r '.version' package.json`

   10

     - echo "##onedev[SetBuildVersion '$buildVersion']"

   11

     - ''

   12

     - npm install -g yarn

   13

     - yarn install

   14

     - ''

   15

     - export CI=true

   16

     - yarn test

   17

     triggers:

   18

     - !BranchUpdateTrigger {}

   19

     retrieveSource: true

   20

     cloneCredential: !DefaultCredential {}

   21

     cpuRequirement: 250m

   22

     memoryRequirement: 128m

   23

     retryCondition: never

   24

     maxRetries: 3

   25

     retryDelay: 30

   26

     caches:

   27

     - key: npm-cache

   28

       path: /root/.npm

   29

     timeout: 3600

   
   

   
   

   
   

5. Save and commit the change. Our CI build should be running now as demonstrated below:
   

Set Up GitOps for the Demo Project

We now have a demo project with CI ability. Let’s improve it to be able to deploy to Kubernetes:

1. To deploy to Kubernetes, we need to build and publish Docker image of the demo project. We will use the official Docker hub registry here. Please visit https://hub.docker.com, login with your account, and create a public repository, say gitops-demo (OneDev also works with private repositories; we created a public repository here for simplicity).

2. To be able to publish a Docker image to your repository, OneDev needs to know credential of your Docker hub account. To do it, add a job secret in the demo project, give it a name, (for instance, dockerhub-password), and input the Docker hub password as a value of the secret.
   

3. To deploy Docker image of the demo project into Kubernetes, we need to give extra permissions to OneDev builds. To do this, create a file, say gitops-demo-role.yaml, in your terminal with the below content:
   Plain Text

    

   
   

   
   

   
   

   
   

   xxxxxxxxxx

   1

   11

   
   

   
   

    

   1

   kind: ClusterRole

   2

   apiVersion: rbac.authorization.k8s.io/v1

   3

   metadata:

   4

     name: gitops-demo

   5

   rules:

   6

   - apiGroups: [""]

   7

     resources: ["services"]

   8

     verbs: ["get", "create"]

   9

   - apiGroups: ["apps"]

   10

     resources: ["deployments"]

   11

     verbs: ["get", "patch", "create"]

   
   

   
   
   Then, run the below command to add the role into Kubernetes:
   Shell

    

   
   

   
   

   
   

   
   

   xxxxxxxxxx

   1

   
   

   
   

    

   1

   $ kubectl apply -f gitops-demo-role.yaml

   
   

   
   
   After doing the above, switch to the job executors page in OneDev, delete the default auto-discover job executor, add a new Kubernetes executor, give it a name, and specify property Cluster Role as gitops-demo, leave all other fields untouched, and save the executor.
   
4. Since we need to build a Docker image of our demo project, let’s add a Dockerfile in root of the project with below content:
   Dockerfile

    

   
   

   
   

   
   

   
   

   xxxxxxxxxx

   1

   
   

   
   

    

   1

   FROM nginx:1.19.5

   2

   COPY build /usr/share/nginx/html

   3

   EXPOSE 80

   
   

   
   

   
   

5. We deploy the demo project via Kubernetes resources, so continue to add a file k8s.yaml in root of the project with below content and replace all occurrences of with name of your Docker hub account:
   Plain Text

    

   
   

   
   

   
   

   
   

   xxxxxxxxxx

   1

   39

   
   

   
   

    

   1

   apiVersion: apps/v1

   2

   kind: Deployment

   3

   metadata:

   4

     name: gitops-demo

   5

     labels:

   6

       tier: gitops-demo

   7

   spec:

   8

     selector:

   9

       matchLabels:

   10

         tier: gitops-demo

   11

     strategy:

   12

       type: Recreate

   13

     template:

   14

       metadata:

   15

         name: gitops-demo

   16

         labels:

   17

           tier: gitops-demo

   18

       spec:

   19

         containers:

   20

         - name: gitops-demo

   21

           image: /gitops-demo:imageTag

   22

           ports:

   23

           - containerPort: 80

   24

   ---

   25

   apiVersion: v1

   26

   kind: Service

   27

   metadata:

   28

     name: gitops-demo

   29

     labels:

   30

       tier: gitops-demo

   31

   spec:

   32

     type: LoadBalancer

   33

     ports:

   34

     - name: http

   35

       port: 80

   36

       targetPort: 80

   37

       protocol: TCP

   38

     selector:

   39

       tier: gitops-demo

   
   

   
   

   
   

6. Edit build spec in OneDev (click the file .onedev-buildspec.yaml from the root of the project and then edit it), replace property image with docker:19.03.5, replace property commandswith the below content, and replace all occurrences of with name of your Docker hub account.
   Shell

    

   
   

   
   

   
   

   
   

   xxxxxxxxxx

   1

   16

   
   

   
   

    

   1

   set -e

   2

   apk add --update npm jq curl

   3

   buildVersion=`jq -r '.version' package.json`

   4

   echo "##onedev[SetBuildVersion '$buildVersion']"

   5

   npm install -g yarn

   6

   yarn install

   7

   export CI=true

   8

   yarn test

   9

   yarn build

   10

   docker build -t /gitops-demo:@commit_hash@ .

   11

   docker login -u  -p @secrets:dockerhub-password@

   12

   docker push /gitops-demo:@commit_hash@

   13

   curl -o /usr/local/bin/kubectl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl"

   14

   chmod +x /usr/local/bin/kubectl

   15

   sed -i "s/imageTag/@commit_hash@/g" k8s.yaml

   16

   kubectl apply -f k8s.yaml -n default

   
   

   
   

   
   

7. Save and commit the change. OneDev will start to build and deploy the demo project. After build is successful, run the below command to show the external IP address of the published demo service:
   Shell

    

   
   

   
   

   
   

   
   

   xxxxxxxxxx

   1

   
   

   
   

    

   1

   $ kubectl get service gitops-demo

   
   

   
   

   
   

   Open the URL http:// with your browser to show the deployed demo app.
8. Congrats! You’ve successfully set up GitOps for the demo project. Whenever there is a new commit pushed to master branch, the CI job will be triggered to test, build, and redeploy the project. If you want to revert to the previous deployment, just run git revert master and push the change.

Multiple Deployment Environments

In our setup above, we deploy the demo project directly into the default namespace, which is not good. In real world, we may need to create multiple namespaces to cater deployment of multiple environments. Let’s create two namespaces, one for test and one for production:

xxxxxxxxxx

$ kubectl create namespace test

$ kubectl create namespace production

This tells OneDev to deploy our demo project into production namespace if built against the master branch and deploy into the namespace identified by branch name otherwise. After committing the change, OneDev should be deploying the demo project into production namespace.

Now let’s create the test branch to track deployments for the test environment. After creating the branch, OneDev will start to deploy the demo project into test namespace.

To get Kubernetes printing external ip address of deployed demo service in specified namespaces, run below command:

xxxxxxxxxx

kubectl get service -n <namespace>

Now, from the commits page, we have a clear view of deployment status in different environments and we can control deployments in different environments by pushing/merging into corresponding branches.

Thanks for reading!