Install Spinnaker With Halyard on Kubernetes

This article will walk you through the steps that can be used to install and setup a Spinnaker instance on Kubernetes that’s behind a corporate proxy. We will use Halyard on docker to manage our Spinnaker deployment.

For a super quick installation, you can use Spinnaker’s Helm chart.

Prerequisites

Make sure to take care of these prerequisites before installing Spinnaker:

• Docker 17.x with proxies configured (click here for OL setup)
• A Kubernetes cluster (click here for OL setup)
• Helm with RBAC enabled (click here for generic setup)

Install Halyard on Docker

Halyard is used to install and manage a Spinnaker deployment. In fact, all production grade Spinnaker deployments require Halyard in order to properly configure and maintain Spinnaker. Let’s use Docker to install Halyard.

Create a docker volume or create a host directory to hold the persistent data used by Halyard. For the purposes of this article, let’s create a host directory and grant users full access:

mkdir halyard && chmod 747 halyard

Halyard needs to interact with your Kubernetes cluster. So we pass the $KUBECONFIG file to it. One way would be to mount a host directory into the container that has your Kubernetes cluster details. Let’s create the directory “k8s” and copy the $KUBECONFIG file and make it visible to the user inside the Halyard container.

mkdir k8s && cp $KUBECONFIG k8s/config && chmod 755 k8s/config

Time to download and run Halyard docker image:

docker run -p 8084:8084 -p 9000:9000 \
    --name halyard -d \
    -v /sandbox/halyard:/home/spinnaker/.hal \
    -v /sandbox/k8s:/home/spinnaker/k8s \
    -e http_proxy=http://<proxy_host>:<proxy_port> \
    -e https_proxy=https://<proxy_host>:<proxy_port> \    
    -e JAVA_OPTS="-Dhttps.proxyHost=<proxy_host> -Dhttps.proxyPort=<proxy_port>" \
    -e KUBECONFIG=/home/spinnaker/k8s/config \
    gcr.io/spinnaker-marketplace/halyard:stable

Make sure to replace the “<proxy_host>” and “<proxy_port>” with your corporate proxy values.

Login to the “halyard” container to test the connection to your Kubernetes cluster:

docker exec -it halyard bashkubectl get pods -n spinnaker

Optionally, if you want command completion run the following inside the halyard container:

source <(hal --print-bash-completion)

Set Provider to “Kubernetes”

In Spinnaker terms, to deploy applications we use integrations to specific cloud platforms. We have to configure Halyard and set the cloud provider to Kubernetes v2 (manifest based) since we want to deploy Spinnaker onto a Kubernetes cluster:

hal config provider kubernetes enable

Next, we create an account. In Spinnaker, an account is a named credential Spinnaker uses to authenticate against an integration provider — Kubernetes in our case:

hal config provider kubernetes account add <my_account> \
    --provider-version v2 \
    --context $(kubectl config current-context)

Make sure to replace “<my_account>” with an account name of your choice. Save the account name in an environment variable $ACCOUNT. Next, we need to enable Halyard to use artifacts:

hal config features edit --artifacts true

Set Deployment Type to “Distributed”

Halyard supports multiple types of Spinnaker deployments. Let’s tell Halyard that we need a distributed deployment of Spinnaker:

hal config deploy edit --type distributed --account-name $ACCOUNT

Set Persistent Store to “Minio”

Spinnaker needs a persistent store to save the continuous delivery pipelines and other configurations. Halyard let’s you choose from multiple storage providers. For the purposes of this article, we will use “Minio.”

Let’s use Helm to install a simple instance of Minio. Run the command from outside the Halyard docker container on a node that has access to your Kubernetes cluster and Helm:

helm install --namespace spinnaker \
             --name minio \
             --set accessKey=<access_key> \
             --set secretKey=<secret_key> \
             stable/minio

Make sure to replace “<access_key>” and “<secret_key>” with values of your choosing.

If you are using a local k8s cluster with no real persistent volume support, you can pass “persistence.enabled=false” as a set to the previous Helm command. As the flag suggests, if Minio goes down, you will lose your changes.

According to the Spinnaker docs, Minio does not support versioning objects. So let’s disable versioning under Halyard configuration. Back in the Halyard docker container run these commands:

mkdir ~/.hal/default/profiles && \
  touch ~/.hal/default/profiles/front50-local.yml

Add the following to the front50-local.yml file:

spinnaker.s3.versioning: false

Now run the following command to configure the storage provider:

echo $MINIO_SECRET_KEY | \
     hal config storage s3 edit --endpoint http://minio:9000 \
         --access-key-id $MINIO_ACCESS_KEY \
         --secret-access-key

Make sure to set the $MINIO_ACCESS_KEY and $MINIO_SECRET_KEY environment variables to the <access_key> and <secret_key> values that you used when you installed Minio.

Finally, let’s enable the s3 storage provider:

hal config storage edit --type s3

Set Version to “Latest”

You have to select a specific version of Spinnaker and configure Halyard so it knows which version to deploy. You can view the available versions by running this command:

hal version list

Pick the latest version number from the list (or any other version that you want to deploy) and update Halyard:

hal config version edit --version <version>

Deploy Spinnaker

At this point, Halyard should have all the information that it needs to deploy a Spinnaker instance. Let’s go ahead and deploy Spinnaker by running this command:

hal deploy apply

Note that first time deployments might take a while.

Make Spinnaker Reachable

We need to expose the Spinnaker UI and Gateway services in order to interact with the Spinnaker dashboard and start creating pipelines. When we deployed Spinnaker using Halyard, a number of Kubernetes services get created in the “spinnaker” namespace. These services are by default exposed within the cluster (type is “ClusterIP”). Let’s change the service type of the services fronting the UI and API servers of Spinnaker to “NodePort” to make them available to end users outside the Kubernetes cluster.

Edit the “spin-deck” service by running the following command:

kubectl edit svc spin-deck -n spinnaker

Change the type to “NodePort” and optionally specify the port on which you want the service exposed. Here’s a snapshot of the service definition:

...
spec:
  type: NodePort
  ports:
  - port: 9000
    protocol: TCP
    targetPort: 9000
    nodePort: 30900
  selector:
    app: spin
    cluster: spin-deck
  sessionAffinity: None  
status:
...

Next, edit the “spin-gate” service by running the following command:

kubectl edit svc spin-gate -n spinnaker

Change the type to “NodePort” and optionally specify the port on which you want the API gateway service exposed.

Note that Kubernetes services can be exposed in multiple ways. If you want to expose Spinnaker onto the public internet, you can use a LoadBalancer or an Ingress with https turned on. You should configure authentication to lock down access to unauthorized users.

Save the node’s hostname or its IP address that will be used to access Spinnaker in an environment variable $SPIN_HOST. Using Halyard, configure the UI and API servers to receive incoming requests:

hal config security ui edit \
    --override-base-url "http://$SPIN_HOST:30900"
hal config security api edit \
    --override-base-url "http://$SPIN_HOST:30808"

Redeploy Spinnaker so it picks up the configuration changes:

hal deploy apply

You can access the Spinnaker UI at “http://$SPIN_HOST:30900”

Create a “Hello-World” Application

Let’s take Spinnaker for a spin (pun intended). Using Spinnaker’s UI, let’s create a “hello-world” application. Use the “Actions” drop-down and click “Create Application”:

Once the application is created, navigate to “Pipelines” tab and click “Configure a new pipeline”:

Now add a new stage to the pipeline to create a manifest based deployment:

Under the “Manifest Configuration”, add the following as the manifest source text:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: hello-world
  name: hello-world
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hello-world
  template:
    metadata:
      labels:
        app: hello-world
    spec:
      containers:
        - image: '<docker_repository>:5000/helloworld:v1'
          name: hello-world
          ports:
            - containerPort: 80

Replace the “<docker_repository>” with the name of your internal docker registry that is made available to your Kubernetes cluster.

Let’s take a quick side tour to create a “helloworld” docker image. We will create a “nginx” based image that hosts an “index.html” file containing:

<h1>Hello World</h1>

We will then create the corresponding “Dockerfile” in the same directory that holds the “index.html” file from the previous step:

FROM nginx:alpine
COPY . /usr/share/nginx/html

Next, we build the docker image by running the following command:

docker build -t <docker_repository>:5000/helloworld:v1 .

Make sure to replace the “<docker_repository>” with the name of your internal docker registry that is made available to your Kubernetes cluster. Push the docker image to the “<docker_repository>” to make it available to the Kubernetes cluster.

docker push <docker_repository>:5000/helloworld:v1

Back in the Spinnaker UI, let’s manually run the “hello-world” pipeline. After a successful execution you can drill down into the pipeline instance details:

To quickly test our hello-world app, we can create a manifest based “LoadBalancer” in the Spinnaker UI. Click the “+” icon:

Add the following service definition to create the load balancer:

kind: Service
apiVersion: v1
metadata:
  name: hello-world
spec:
  type: NodePort
  selector:
    app: hello-world
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
    nodePort: 31080

Once Spinnaker provisions the load balancer, hit the hello-world app’s URL at “http://$SPIN_HOST:31080” in your browser. Voila! There you have it, “Hello World” is rendered.

Conclusion

Spinnaker is a multi-cloud continuous delivery platform for releasing software with high velocity. We used Halyard to install Spinnaker on a Kubernetes cluster and deployed a simple hello-world pipeline. Of course, we barely scratched the surface in terms of what Spinnaker offers. Head over to the guides to learn more about Spinnaker.