{{announcement.body}}
{{announcement.title}}

Building Cloud-Native Applications with Kubebuilder and Kind

DZone 's Guide to

Building Cloud-Native Applications with Kubebuilder and Kind

In this article, we take a look at installing and creating cloud-native projects using Kubebuilder and Kind.

· Cloud Zone ·
Free Resource

Introduction

In this article, we will explore how to use Kubebuilder and Kind to create a local test cluster and an operator. Following that operation, we will then deploy that operator in the cluster and test it. All of the code is included below to port-forward to private endpoints the Kubernetes way. Also, if you want to learn more about the idea and the project check out the Forward operator page here.

Essentially, what the code does is to create an alpine/socat pod. You can specify the host, port, and protocol and it will make a tunnel for you, so then you can use port-forward or a service or ingress or whatever to expose things that are in another private subnet.

While this might not sound like a good idea at first, it does have some specific and essential use cases. Check your security constraints, though, before doing any of this—in a normal scenario it should be safe. In terms of use cases, this project is useful for testing or for reaching a database while doing some debugging or testing. The tools used in this project are what makes it really interesting as this is for building cloud-native applications, since it native to Kubernetes, and that’s what we will explore here.

While Kind is not actually a requirement, I used that for testing and really liked it, it’s much faster and simpler than Minikube.

Also, if you are interested in how I came up with the idea to make this operator, check out this GitHub issue here.

Prerequisites

Create the Project

In this step, we need to create the Kubebuilder project, so in an empty folder we run:

Shell
 




x
14


 
1
$ go mod init techsquad.rocks
2
go: creating new go.mod: module techsquad.rocks
3
 
          
4
$ kubebuilder init --domain techsquad.rocks
5
go get sigs.k8s.io/controller-runtime@v0.4.0
6
go mod tidy
7
Running make...
8
make
9
/home/kainlite/Webs/go/bin/controller-gen object:headerFile=./hack/boilerplate.go.txt paths="./..."
10
go fmt ./...
11
go vet ./...
12
go build -o bin/manager main.go
13
Next: Define a resource with:
14
$ kubebuilder create api


Create the API

Next let’s create an API, something for us to use to run our controller.

Shell
 




xxxxxxxxxx
1
13


 
1
$ kubebuilder create api --group forward --version v1beta1 --kind Map
2
Create Resource [y/n]
3
y
4
Create Controller [y/n]
5
y
6
Writing scaffold for you to edit...
7
api/v1beta1/map_types.go
8
controllers/map_controller.go
9
Running make...
10
/home/kainlite/Webs/go/bin/controller-gen object:headerFile=./hack/boilerplate.go.txt paths="./..."
11
go fmt ./...
12
go vet ./...
13
go build -o bin/manager main.go


Up until this point here, we have only created a boilerplate/basic or empty project with defaults. If you were to test it now, it will work, but it won’t do anything interesting. However, it covers a lot of ground and we should be grateful that such a tool exists.

Add Our Code to the Mix

First, we will add our code to api/v1beta1/map_types.go, which will add our fields to our type.

Go
 




xxxxxxxxxx
1
170


 
1
/*
2
 
          
3
Licensed under the Apache License, Version 2.0 (the "License");
4
you may not use this file except in compliance with the License.
5
You may obtain a copy of the License at
6
 
          
7
    http://www.apache.org/licenses/LICENSE-2.0
8
 
          
9
Unless required by applicable law or agreed to in writing, software
10
distributed under the License is distributed on an "AS IS" BASIS,
11
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12
See the License for the specific language governing permissions and
13
limitations under the License.
14
*/
15
 
          
16
package controllers
17
 
          
18
import (
19
    "context"
20
    "fmt"
21
    "strconv"
22
    "strings"
23
 
          
24
    "github.com/go-logr/logr"
25
    "k8s.io/apimachinery/pkg/api/errors"
26
    "k8s.io/apimachinery/pkg/runtime"
27
    "k8s.io/apimachinery/pkg/types"
28
    ctrl "sigs.k8s.io/controller-runtime"
29
    "sigs.k8s.io/controller-runtime/pkg/client"
30
    "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
31
    "sigs.k8s.io/controller-runtime/pkg/reconcile"
32
 
          
33
    corev1 "k8s.io/api/core/v1"
34
    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
35
 
          
36
    forwardv1beta1 "github.com/kainlite/forward/api/v1beta1"
37
)
38
 
          
39
// +kubebuilder:rbac:groups=maps.forward.techsquad.rocks,resources=pods,verbs=get;list;watch;create;update;patch;delete
40
// +kubebuilder:rbac:groups=map.forward.techsquad.rocks,resources=pods,verbs=get;list;watch;create;update;patch;delete
41
// +kubebuilder:rbac:groups=forward.techsquad.rocks,resources=maps,verbs=get;list;watch;create;update;patch;delete
42
// +kubebuilder:rbac:groups=forward.techsquad.rocks,resources=pods/status,verbs=get;update;patch
43
// +kubebuilder:rbac:groups="",resources=pods,verbs=get;list;watch;create;update;patch;delete
44
 
          
45
// MapReconciler reconciles a Map object
46
type MapReconciler struct {
47
    client.Client
48
    Log    logr.Logger
49
    Scheme *runtime.Scheme
50
}
51
 
          
52
func newPodForCR(cr *forwardv1beta1.Map) *corev1.Pod {
53
    labels := map[string]string{
54
        "app": cr.Name,
55
    }
56
    var command string
57
    if strings.EqualFold(cr.Spec.Protocol, "tcp") {
58
        command = fmt.Sprintf("socat -d -d tcp-listen:%s,fork,reuseaddr tcp-connect:%s:%s", strconv.Itoa(cr.Spec.Port), cr.Spec.Host, strconv.Itoa(cr.Spec.Port))
59
    } else if strings.EqualFold(cr.Spec.Protocol, "udp") {
60
        command = fmt.Sprintf("socat -d -d UDP4-RECVFROM:%s,fork,reuseaddr UDP4-SENDTO:%s:%s", strconv.Itoa(cr.Spec.Port), cr.Spec.Host, strconv.Itoa(cr.Spec.Port))
61
    } else {
62
        // TODO: Create a proper error here if the protocol doesn't match or is unsupported
63
        command = fmt.Sprintf("socat -V")
64
    }
65
 
          
66
    return &corev1.Pod{
67
        ObjectMeta: metav1.ObjectMeta{
68
            Name:      "forward-" + cr.Name + "-pod",
69
            Namespace: cr.Namespace,
70
            Labels:    labels,
71
        },
72
        Spec: corev1.PodSpec{
73
            Containers: []corev1.Container{
74
                {
75
                    Name:    "map",
76
                    Image:   "alpine/socat",
77
                    Command: strings.Split(command, " "),
78
                },
79
            },
80
            RestartPolicy: corev1.RestartPolicyOnFailure,
81
        },
82
    }
83
}
84
 
          
85
func (r *MapReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) {
86
    reqLogger := r.Log.WithValues("namespace", req.Namespace, "MapForward", req.Name)
87
    reqLogger.Info("=== Reconciling Forward Map")
88
    // Fetch the Map instance
89
    instance := &forwardv1beta1.Map{}
90
    err := r.Get(context.TODO(), req.NamespacedName, instance)
91
    if err != nil {
92
        if errors.IsNotFound(err) {
93
            // Request object not found, could have been deleted after
94
            // reconcile request—return and don't requeue:
95
            return reconcile.Result{}, nil
96
        }
97
        // Error reading the object—requeue the request:
98
        return reconcile.Result{}, err
99
    }
100
 
          
101
    // If no phase set, default to pending (the initial phase):
102
    if instance.Status.Phase == "" || instance.Status.Phase == "PENDING" {
103
        instance.Status.Phase = forwardv1beta1.PhaseRunning
104
    }
105
 
          
106
    // Now let's make the main case distinction: implementing
107
    // the state diagram PENDING -> RUNNING or PENDING -> FAILED
108
    switch instance.Status.Phase {
109
    case forwardv1beta1.PhasePending:
110
        reqLogger.Info("Phase: PENDING")
111
        reqLogger.Info("Waiting to forward", "Host", instance.Spec.Host, "Port", instance.Spec.Port)
112
        instance.Status.Phase = forwardv1beta1.PhaseRunning
113
    case forwardv1beta1.PhaseRunning:
114
        reqLogger.Info("Phase: RUNNING")
115
        pod := newPodForCR(instance)
116
        // Set Map instance as the owner and controller
117
        err := controllerutil.SetControllerReference(instance, pod, r.Scheme)
118
        if err != nil {
119
            // requeue with error
120
            return reconcile.Result{}, err
121
        }
122
        found := &corev1.Pod{}
123
        nsName := types.NamespacedName{Name: pod.Name, Namespace: pod.Namespace}
124
        err = r.Get(context.TODO(), nsName, found)
125
        // Try to see if the pod already exists and if not
126
        // (which we expect) then create a one-shot pod as per spec:
127
        if err != nil && errors.IsNotFound(err) {
128
            err = r.Create(context.TODO(), pod)
129
            if err != nil {
130
                // requeue with error
131
                return reconcile.Result{}, err
132
            }
133
            reqLogger.Info("Pod launched", "name", pod.Name)
134
        } else if err != nil {
135
            // requeue with error
136
            return reconcile.Result{}, err
137
        } else if found.Status.Phase == corev1.PodFailed ||
138
            found.Status.Phase == corev1.PodSucceeded {
139
            reqLogger.Info("Container terminated", "reason",
140
                found.Status.Reason, "message", found.Status.Message)
141
            instance.Status.Phase = forwardv1beta1.PhaseFailed
142
        } else {
143
            // Don't requeue because it will happen automatically when the
144
            // pod status changes.
145
            return reconcile.Result{}, nil
146
        }
147
    case forwardv1beta1.PhaseFailed:
148
        reqLogger.Info("Phase: Failed, check that the host and port are reachable from the cluster and that there are no networks policies preventing this access or firewall rules...")
149
        return reconcile.Result{}, nil
150
    default:
151
        reqLogger.Info("NOP")
152
        return reconcile.Result{}, nil
153
    }
154
 
          
155
    // Update the At instance, setting the status to the respective phase:
156
    err = r.Status().Update(context.TODO(), instance)
157
    if err != nil {
158
        return reconcile.Result{}, err
159
    }
160
 
          
161
    // Don't requeue. We should be reconcile because either the pod
162
    // or the CR changes.
163
    return reconcile.Result{}, nil
164
}
165
 
          
166
func (r *MapReconciler) SetupWithManager(mgr ctrl.Manager) error {
167
    return ctrl.NewControllerManagedBy(mgr).
168
        For(&forwardv1beta1.Map{}).
169
        Complete(r)
170
}


Essentially, we just edited the MapSpec and the MapStatus structure.

Now, we need to add the code to our controller in controllers/map_controller.go

In this controller, we have now added two functions: one to create a pod and the other to modify the entire Reconcile function (this one takes care of checking the status and make the transitions—in other words, it makes a controller work like a controller. Also, have you noticed how the Kubebuilder annotations generates the RBAC config for us? Pretty handy, right?

Starting the Cluster with Kind

Next, we will use Kind to create a local cluster to test:

Shell
 




xxxxxxxxxx
1
12


 
1
$ kind create cluster --name test-cluster-1
2
Creating cluster "test-cluster-1" ...
3
 ✓ Ensuring node image (kindest/node:v1.16.3) �� 
4
 ✓ Preparing nodes �� 
5
 ✓ Writing configuration �� 
6
 ✓ Starting control-plane ��️ 
7
 ✓ Installing CNI �� 
8
 ✓ Installing StorageClass �� 
9
Set kubectl context to "kind-test-cluster-1"
10
You can now use your cluster with:
11
 
          
12
kubectl cluster-info --context kind-test-cluster-1


Could it really be that easy!? Well, yes, it is!

Running Our Operator Locally

For testing, you can run your operator locally like this:

Shell
 




xxxxxxxxxx
1
12


 
1
$ make run
2
/home/kainlite/Webs/go/bin/controller-gen object:headerFile=./hack/boilerplate.go.txt paths="./..."
3
go fmt ./...
4
go vet ./...
5
/home/kainlite/Webs/go/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
6
go run ./main.go
7
2020-01-17T21:00:14.465-0300    INFO    controller-runtime.metrics      metrics server is starting to listen    {"addr": ":8080"}
8
2020-01-17T21:00:14.466-0300    INFO    setup   starting manager
9
2020-01-17T21:00:14.466-0300    INFO    controller-runtime.manager      starting metrics server {"path": "/metrics"}
10
2020-01-17T21:00:14.566-0300    INFO    controller-runtime.controller   Starting EventSource    {"controller": "map", "source": "kind source: /, Kind="}
11
2020-01-17T21:00:14.667-0300    INFO    controller-runtime.controller   Starting Controller     {"controller": "map"}
12
2020-01-17T21:00:14.767-0300    INFO    controller-runtime.controller   Starting workers        {"controller": "map", "worker count": 1}


Testing It

First, we spin up a pod, and launch nc -l -p 8000.

Shell
 




xxxxxxxxxx
1
23


 
1
$ kubectl run -it --rm --restart=Never alpine --image=alpine sh
2
If you don't see a command prompt, try pressing enter.
3
 
          
4
# ifconfig
5
eth0      Link encap:Ethernet  HWaddr E6:49:53:CA:3D:89  
6
          inet addr:10.244.0.8  Bcast:10.244.0.255  Mask:255.255.255.0
7
          inet6 addr: fe80::e449:53ff:feca:3d89/64 Scope:Link
8
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
9
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
10
          TX packets:9 errors:0 dropped:0 overruns:0 carrier:0
11
          collisions:0 txqueuelen:0 
12
          RX bytes:0 (0.0 B)  TX bytes:698 (698.0 B)
13
 
          
14
lo        Link encap:Local Loopback  
15
          inet addr:127.0.0.1  Mask:255.0.0.0
16
          inet6 addr: ::1/128 Scope:Host
17
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
18
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
19
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
20
          collisions:0 txqueuelen:1000 
21
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
22
/ # nc -l -p 8000
23
test


Now, we edit our manifest and apply it—checking that everything is in place. Then we do the port-forward and launch another nc localhost 8000 to test if everything went well. First, the manifest:

YAML
 




xxxxxxxxxx
1
10


 
1
$ cat config/samples/forward_v1beta1_map.yaml 
2
apiVersion: forward.techsquad.rocks/v1beta1
3
kind: Map
4
metadata:
5
  name: mapsample
6
  namespace: default
7
spec:
8
  host: 10.244.0.8
9
  port: 8000
10
  protocol: tcp


Then port-forward and test the code.

Java
 




xxxxxxxxxx
1
15


 
1
$ kubectl apply -f config/samples/forward_v1beta1_map.yaml
2
map.forward.techsquad.rocks/mapsample configured
3
 
          
4
# Logs in the controller
5
2020-01-17T23:38:27.650Z        INFO    controllers.Map === Reconciling Forward Map     {"namespace": "default", "MapForward": "mapsample"}
6
2020-01-17T23:38:27.691Z        INFO    controllers.Map Phase: RUNNING  {"namespace": "default", "MapForward": "mapsample"}
7
2020-01-17T23:38:27.698Z        DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "map", "request": "default/mapsample"}
8
 
          
9
$ kubectl port-forward forward-mapsample-pod 8000:8000                                                                                                                                                                       
10
Forwarding from 127.0.0.1:8000 -> 8000                                                                                                                                                                                                                                           
11
Handling connection for 8000                                               
12
 
          
13
# In another terminal or tab or split
14
$ nc localhost 8000
15
test


Making It Publicly Ready

Here, we just build and push the Docker image to Docker Hub or to your favorite public registry.

Shell
 




xxxxxxxxxx
1
26


 
1
$ make docker-build docker-push IMG=kainlite/forward:0.0.1
2
/home/kainlite/Webs/go/bin/controller-gen object:headerFile=./hack/boilerplate.go.txt paths="./..."
3
go fmt ./...
4
go vet ./...
5
/home/kainlite/Webs/go/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
6
go test ./... -coverprofile cover.out
7
?       github.com/kainlite/forward     [no test files]
8
?       github.com/kainlite/forward/api/v1beta1 [no test files]
9
ok      github.com/kainlite/forward/controllers 6.720s  coverage: 0.0% of statements
10
docker build . -t kainlite/forward:0.0.1
11
Sending build context to Docker daemon  45.02MB
12
Step 1/14 : FROM golang:1.13 as builder
13
1.13: Pulling from library/golang
14
8f0fdd3eaac0: Pull complete
15
...
16
...
17
...
18
 ---> 4dab137d22a1
19
Successfully built 4dab137d22a1
20
Successfully tagged kainlite/forward:0.0.1
21
docker push kainlite/forward:0.0.1
22
The push refers to repository [docker.io/kainlite/forward]
23
50a214d52a70: Pushed 
24
84ff92691f90: Pushed 
25
0d1435bd79e4: Pushed 
26
0.0.1: digest: sha256:b4479e4721aa9ec9e92d35ac7ad5c4c0898986d9d2c9559c4085d4c98d2e4ae3 size: 945


Then you can install it with make deploy IMG=kainlite/forward:0.0.1 and uninstall it with make uninstall.

Closing Notes

Be sure to check out the Kubebuilder book if you want to learn more and also read the Kind docs. If you enjoyed this blog, please follow me on Twitter or GitHub!

If you spot any errors or have any suggestions, please send me a message so that I can update the code.

Also, you can check the source code and changes in the generated code and the sources here.

This blog was originally published here.

License: MIT License

Topics:
cloud, cloud native, cloud native and kubernetes, cloud native applications, kind, kubernetes, minikube, pod

Published at DZone with permission of Gabriel Garrido . See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}