Containers

Node.js has been a favorite among serious programmers for the last five years running. The JavaScript Runtime Environment for Maximum Throughput is a free and open-source program that aims to improve the performance of JavaScript across several platforms. Because of its event-driven, non-blocking I/O approach, Node.js is small in size and quick in processing requests, making it an excellent choice for data-intensive, real-time, and distributed applications. Developers are increasingly turning to node.js application optimization services; thus, it's important to streamline the process of designing and releasing cross-platform applications. So, let's get into the context of the article. Suggestions for Containerizing and Optimizing Node Apps Here are listed seven ways of containerizing your node.js application, so let's have a look at them in brief. 1. Use a Specific Base Image Tag Instead of "Version:Latest" Useful tags that define version information, intended destination (prod or test, for example), stability, or other relevant information for distributing your application across environments should always be included when creating Docker images. Outside of the development environment, you shouldn't depend on the most recent tag that Docker automatically downloads. The usage of the most recent version of a program might result in strange or even harmful effects. Suppose you're constantly updating to the most recent version of an image. In that case, eventually, one of those updates is certain to include a brand-new build or untested code that will cause your app to stop functioning as intended. Take this example Dockerfile that targets that node: # Create image based on the official Node image from dockerhub FROM node:lts-buster # Create app directory WORKDIR /usr/src/app # Copy dependency definitions COPY package.json ./package.json COPY package-lock.json ./package-lock.json # Install dependencies #RUN npm set progress=false \ # && npm config set depth 0 \ # && npm i install RUN npm ci # Get all the code needed to run the app COPY . . # Expose the port the app runs in EXPOSE 3000 # Serve the app CMD ["npm", "start"] Instead of using node:latest, you should use the lts-buster Docker image. Considering that lts-buster is a static picture, this method may be preferable. 2. Use a Multi-Stage Build One single Docker base image may be used throughout several stages of a build, including compilation, packaging, and unit testing. However, the actual code that executes the program is stored in a different image. As the finished image won't have any development or debugging tools, it'll be more secure and take up less space. In addition, if you use Docker's multi-stage build process, you can be certain that your builds will be both efficient and repeatable. You can create multiple stages within a Dockerfile to control how you build that image. You can containerize your Node application using a multi-layer approach. Different parts of the application, like code, assets, and even snapshot dependencies, may be located in each of the many layers that make up the program. What if we wish to create an independent image of our application? To see an example Dockerfile of this in action, please check the following: FROM NODE:LTS-BUSTER-SLIM AS DEVELOPMENT WORKDIR /USR/SRC/APP COPY PACKAGE.JSON ./PACKAGE.JSON COPY PACKAGE-LOCK.JSON ./PACKAGE-LOCK.JSON RUN NPM CI COPY . . EXPOSE 3000 CMD [ "NPM", "RUN", "DEV" ] FROM DEVELOPMENT AS DEV-ENVS RUN <<EOF APT-GET UPDATE APT-GET INSTALL -Y --NO-INSTALL-RECOMMENDS GIT EOF # INSTALL DOCKER TOOLS (CLI, BUILDX, COMPOSE) COPY --FROM=GLOURSDOCKER/DOCKER / / CMD [ "NPM", "RUN", "DEV" ] We first add an AS development label to the node:lts-buster-slim statement. This lets us refer to this build stage in other build stages. Next, we add a new development stage labeled dev-envs. We'll use this stage to run our development. Now, let's rebuild our image and run our development. To execute just the development build stage, we'll use the same docker build command as before, but this time we'll use the —target development parameter. docker build -t node-docker --target dev-envs 3. Fix Security Vulnerabilities in Your Node Image In order to create modern services, programmers often use preexisting third-party software. However, it's important to be cautious when integrating third-party software into your project since it may present security holes. Using verified image sources and maintaining vigilant container monitoring are both useful security measures. Docker Desktop will notify you to do security checks on the newly created node:lts-buster-slim Docker image. Let's have a look at our Node.js app with the help of the Snyk Plugin for Docker Desktop. Begin by setting up Docker Desktop 4.8.0+ on your Mac, Windows, or Linux PC. Next, choose the Allow Docker Extensions checkbox under Settings > Extensions. After that, you can search for Snyk in the Extensions Marketplace by selecting the "Add Extensions" option on the left sidebar. Put in the Snyk and log onto the network: lts-buster-slim Type "Node Docker Official Image" into the "Choose image name" box. In order to begin scanning, you will need to log in to Docker Hub. If you don't have an account, don't fret; making one is easy, quick, and completely free. With Docker Desktop, the outcome of a scan looks like this: During this scan, Snyk discovered 70 vulnerabilities of varied severity. After you've identified them, you may start fixing them to improve your reputation. Not just that. Using the docker scan command on your Dockerfile will execute a vulnerability scan: 4. Leverage HEALTHCHECK The HEALTHCHECK directive instructs Docker on how to check the health of a container. For example, this may be used to determine whether or not a web server is in an endless loop and unable to accept new connections, even while the server process is still active. # syntax=docker/dockerfile:1.4 FROM node:lts-buster-slim AS development # Create app directory WORKDIR /usr/src/app COPY package.json ./package.json COPY package-lock.json ./package-lock.json RUN npm ci COPY . . EXPOSE 3000 CMD [ "npm", "run", "dev" ] FROM development as dev-envs RUN <<EOF apt-get update apt-get install -y --no-install-recommends git EOF RUN <<EOF useradd -s /bin/bash -m vscode groupadd docker usermod -aG docker vscode EOF HEALTHCHECK CMD curl --fail http://localhost:3000 || exit 1 # install Docker tools (cli, buildx, compose) COPY --from=gloursdocker/docker / / CMD [ "npm", "run", "dev" ] In the production stage, applications are often managed by an orchestrator such as Kubernetes or a service fabric. HEALTHCHECK allows you to inform the orchestrator about the health of your containers, which may be used for configuration-based management. Here's a case in point: BACKEND: CONTAINER_NAME: BACKEND RESTART: ALWAYS BUILD: BACKEND VOLUMES: - ./BACKEND:/USR/SRC/APP - /USR/SRC/APP/NODE_MODULES DEPENDS_ON: - MONGO NETWORKS: - EXPRESS-MONGO - REACT-EXPRESS EXPOSE: - 3000 HEALTHCHECK: TEST: ["CMD", "CURL", "-F", "HTTP://LOCALHOST:3000"] INTERVAL: 1M30S TIMEOUT: 10S RETRIES: 3 START_PERIOD: 40S 5. Use .dockerignore We suggest creating a.dockerignore file in the same folder as your Dockerfile to improve build times. This guide requires a single line in your.dockerignore file: NODE_MODULES The node modules directory, which includes Maven's output, is not included in the Docker build context, thanks to this line. There are numerous advantages to having a well-organized.dockerignore file, but for the time being, this simple file will suffice. Next, I'll describe the built environment and why it's so important. Docker images may be created using the Docker build command by combining a Dockerfile and a "context." In this setting, everything you do applies to the directory structure or URL you just gave me. Any of these files may be used in the construction process. Meanwhile, the node developer operates in the compilation context. A directory on Mac, Windows, or Linux. Everything required to run the program may be found in this folder, including the source code, settings, libraries, and plugins. If you provide a.dockerignore file, we may use it to skip over certain parts of your project while creating your new image: code, configuration files, libraries, plugins, etc. For example, if you want to keep the node modules directory out of your build, you may do so by adding the following to your.dockerignore file. Backend Frontend 6. Run as a Non-Root User for Security Purposes It is safer to run apps with the user's permission since this helps reduce vulnerabilities. Even with Docker containers. Docker containers and their contents automatically get root access to the host system. That's why it's recommended to never run Docker containers as the root user. This may be accomplished by including certain USER directives in your Dockerfile. When executing the image and for any future RUN, CMD, or ENTRYPOINT instructions, the USER command specifies the desired user name (or UID) and, optionally, the user group (or GID): FROM NODE:LTS-BUSTER AS DEVELOPMENT WORKDIR /USR/SRC/APP COPY PACKAGE.JSON ./PACKAGE.JSON COPY PACKAGE-LOCK.JSON ./PACKAGE-LOCK.JSON RUN NPM CI COPY . . EXPOSE 3000 CMD ["NPM", "START"] FROM DEVELOPMENT AS DEV-ENVS RUN <<EOF APT-GET UPDATE APT-GET INSTALL -Y --NO-INSTALL-RECOMMENDS GIT EOF RUN <<EOF USERADD -S /BIN/BASH -M VSCODE GROUPADD DOCKER USERMOD -AG DOCKER VSCODE EOF # INSTALL DOCKER TOOLS (CLI, BUILDX, COMPOSE) COPY --FROM=GLOURSDOCKER/DOCKER / / CMD [ "NPM", "START" ] 7. Explore Graceful Shutdown Options for Node Temporary storage spaces created in Docker for Node. They are easy to prevent, destroy, and then either replace or repurpose. It is possible to kill containers by giving the process the SIGTERM signal. In order to make the most of this brief window of opportunity, your app must be able to process incoming requests and free up any associated resources without delay. Node.js, on the other hand, is crucial for a successful shutdown of your app since it takes and passes signals like SIGINT and SIGTERM from the OS. Because of Node.js, your app may select how to respond to the signals it receives. If you don't program for them or use a module that does, your app won't terminate properly. However, it will continue to operate normally until Docker or Kubernetes terminates it due to a timeout. If you're unable to modify your application's code, you may still use the docker run —init or tini init option inside your Dockerfile. It is recommended, however, that you provide code to manage appropriate signal handling for graceful shutdowns. Conclusion In this tutorial, we covered a wide range of topics related to Docker image optimization, from constructing a solid Dockerfile to using Snyk to check for vulnerabilities. It's not difficult to make better Node.js applications. If you master some basic skills, you'll be in good condition.

In the past few years, there has been a growing number of organizations and developers joining the Docker journey. Containerization simplifies the software development process because it eliminates dealing with dependencies and working with specific hardware. Nonetheless, the biggest advantage of using containers is down to the portability they offer. But, it can be quite confusing how to run a container on the cloud. You could certainly deploy these containers to servers on your cloud provider using Infrastructure as a Service (IaaS). However, this approach will only take you back to the issue we mentioned previously, which is, you’d have to maintain these servers when there’s a better way to do that. Table of Contents How To Run a Docker Container on the Cloud Using a Container Registry Using Container-as-a-Service Why Should I Use CaaS? What Are the Best CaaS Solutions? AWS ECS AWS Lambda AWS App Runner Azure Container Instances Google Cloud Run Conclusion How To Run a Docker Container on the Cloud Using a Container Registry You are probably reading this if your container runs locally but are wondering how to run it on the cloud. In this case, the next step to take to bring it to the cloud is to select a container registry that will act as a centralized location to store your containers. Essentially, you will need to push your container to this registry, whether public or private, so your image can be distributed from there. Using Container-as-a-Service Containers-as-a-Service (CaaS) is a concept that allows companies to directly run their container on the cloud using any given provider of choice. With CaaS, the infrastructure required to run containers such as orchestration tools, e.g, Docker Swarm, Kubernetes, OpenStack, etc., as well as cluster management software are non-existent for the user. As a side note, CaaS joins the already established cloud service models such as Infrastructure-as-a-Service (IaaS), Platform-as-a-Service (PaaS), and Software-as-a-Service (SaaS). Why Should I Use CaaS? Some of the advantages of using Container-as-a-Service are: Cost reduction: it eliminates the time, effort, and money spent on maintaining secure infrastructure to run your container. Flexibility: you can easily move from cloud to cloud or even back to your on-prem infrastructure, freeing you from vendor lock-in. Speed: Since the underlying infrastructure abstracts from it, you can deploy your container quicker. Overall, CaaS will not only simplify the running process of a software application but also improve overall security around it as most CaaS solutions offer vulnerability scans. Furthermore, you don’t have to worry about managing the hardware that will run your container. What Are the Best CaaS Solutions? When choosing a CaaS solution, some of the key considerations include: Can it operate multi-container applications? What networks and storage functions are available? Which file format does it support? How is storage achieved? Which billing model does it use? Amazon Elastic Container Service (Amazon ECS) Amazon ECS is a scalable container orchestration platform by AWS designed to run, stop, and manage containers in a cluster environment by using task definition. Essentially, task definition is where you define: The container to use. How many containers to run. How your containers are linked. What resources your containers use. Note: AWS ECS also supports mounting EFS volumes. With that in mind, you have two ways of using ECS: By using EC2 Instances. By using Fargate. ECS With EC2 In this case, containers will be deployed to EC2 Instances (VMs) created for the cluster. The merits include: Full control over the type of EC2 instance used. Your container is used for machine learning and is GPU-oriented, meaning you can choose to run on an EC2 instance that is optimized for this usage. Reduce your cost by using Spot instances, which can reduce your cost by up to 90%. On the other hand, the only demerit is that: You are responsible for patching, managing network security, and the scalability associated with these instances. Pricing: When it comes to cost, you are charged for the EC2 instances run within your ECS cluster and VPC networking. ECS With Fargate AWS Fargate was launched in 2017, and with this model, you don’t have to be worried about managing EC2 Instances. AWS Fargate directly manages the underlying servers required to run your container by pre-configuring a cluster for you. You will just need to add your workload to it. The advantages include: No infrastructure to manage. AWS deals with availability and scalability of your container application. Fargate Spot, based on similar principles as the Spot instances, AWS mentions a cost reduction of up to 70%. In contrast, the downside is: Only one networking mode is currently supported (awsvpc), which might limit you with the network layers in some specific scenarios you might try to achieve. A recent report by Datadog, mentions that, in 2021, 32% of AWS container environments were using AWS Fargate. This trend confirms that companies are switching gradually to serverless environments. Pricing: Fargate’s pricing is based on a “pay as you go” model. There are no upfront costs and you only pay for the compute and memory resources consumed. Here’s a pricing example for the region US West (Oregon): $0.04048 per vCPU per hour. $0.004445 per gigabyte per hour. The table below will help you better understand the terminology used with ECS/Fargate and Kubernetes: Infrastructure Layer Component ECS Fargate Kubernetes Workload Deployment UnitDesired StateAccess Endpoint TaskServiceALB PodDeploymentIngress Control Plane API EndpointSchedulerControllerState Management Frontend ServiceCapacity ManagerCluster ManagerState DB Kube-apiserverKube-schedulerKube-controlleretcd Data Plane Guest OSAgentContainer RuntimeNetwork Amazon Linux 2Fargate AgentDockerENI/VPC Linux/WindowsKubeletContainerdCN/Kubeproxy AWS Lambda A serverless service by AWS whereby you bring your code, whether it is Java, Go, C#, Python, Powershell, Node.js or Ruby, and Amazon will run it into a callable function that complies with their language’s Lambda interface. Lambda functions are mostly called by connecting them to AWS API Gateway, which exposes the functions as REST API calls. You might be wondering why we are even mentioning AWS Lambda at this point as there’s no link with Docker or container images. According to AWS, in December 2020, AWS Lambda began supporting running container images up to 10GB in size. Using Lambda to run a Docker container on the cloud gives you: Scalability: Lambda will automatically create new instances of your function to meet demand as it can scale up to 500 new instances every minute. However, you may have to contend with: Reduced portability: Since Lambda is AWS’ proprietary serverless tech, you will need to significantly adjust your function to move to another cloud provider. Slow scalability: When we mentioned how Lambda can spin up new instances, we weren’t talking about its speed. A cold start for your function will require time and has a hard impact on Java and .NET applications. Can’t run long-running tasks: Lambda functions can only run up to 15 minutes. Pricing: You are charged by the number of requests for their functions and by the duration (time spent to execute the function). Pricing will also vary depending on the amount of memory you allocate to your function. Nonetheless, Lambda offers a free tier that works even if you use your annual AWS Free Tier term, which offers 400,000 GB-seconds of compute time every month. AWS App Runner Launched in May 2021, AWS App Runner facilitates bringing a web application to the cloud without worrying about scaling or the infrastructure associated with it. Essentially, it simply runs Amazon ECS with Fargate to execute your container but you don’t need to set up or configure anything related to Fargate to get going. It can run in build mode, which pulls code from your GitHub repository and builds the application at any commits you might push to your main branch. Alternatively, it can run in container mode, where you will connect your container registry (only AWS ECR is supported) and point to your image. If you want to see what AWS has planned for App Runner, they outline everything you need to know with their detailed roadmap. The core advantage of AWS App Runner when it comes to running a Docker container on the cloud is that: It is easy to configure and provides a simple way to get a web application to run in the cloud. On the other hand, the disadvantages include: Build mode only supports Python and Node.js runtimes. Can’t scale down to 0, you need to pay for at least one instance. Build mode has no integration with AWS CodeCommit or other Source Control Management, meaning you will be forced to use GitHub. App cannot communicate with private VPC: More details here. Pricing: You are billed for what you use. For example, a minimal instance (1vCPU, 2GB) will cost $0.078 per hour or around $56.00 per month, plus a little extra for automatic build and deployment, if it is always running: $0.064 per vCPU per hour. $0.007 per gigabyte per hour. Automatic deployment: $1 per application per month. Build Fee: $0.005/build-minute. Detailed pricing information is available on their website. Azure Container Instances (ACI) Microsoft was a late entrant in the CaaS market since Azure Container Instances was announced in July 2017. It offers: Support for persistent storage by mounting Azure file share to the container. Co-scheduled groups, Azure supports the scheduling of multi-container groups that share a host machine, local network, or storage. Container is in your virtual network and can communicate with other resources in that network. Full control over the instance that runs your container. Adding GPU compute power is not a problem with ACI. The only downside associated with it is that it: Only supports calling Docker containers from a registry. Pricing: Billing is per hour of vCPU, Memory, GPU, and OS used. Using a container that requires a GPU or Windows will be more expensive. $0.04660 per vCPU per hour. $0.0051 per gigabyte per hour. Google Cloud Run Google Cloud Run, GCP’s CaaS solution, became available in November 2019. Similar to the other options of running Docker containers in the cloud listed above, this service is built on the Knative platform based on Kubernetes. Similar to AWS App Runner, you can choose to point to a container registry or repository that contains your application code. Benefits: Use of secrets from Google Secret Manager. Deployment from source code supports Go, Python, Java, Node.js, Ruby, and more. Support traffic splitting between revisions. Disadvantage: Not directly related to Cloud Run but the only disadvantage is connected to GCP as a whole, whereby there is a limited number of regions compared to Azure or AWS, for instance. Pricing: Anyone could try Cloud Run for free with the $300 credit that GCP offers to their new customers. After that, you’ll be billed once you go over the free tier. The free monthly quotas for Google Cloud Run are as follows: CPU: The first 180,000 vCPU-seconds. Memory: The first 360,000 GB-seconds. Requests: The first 2 million requests. Networking: The first 1 GB egress traffic (platform-wide). Once you bypass these limits; however, you’ll need to pay for your usage. The costs for the paid tier of Google Cloud Run are: CPU: $0.00144 per vCPU per minute. Memory: $0.00015 per GB per minute. Requests: $0.40 per 1 million requests. Networking: $0.085 per GB delivered. Conclusion Cloud providers are always innovating to fulfill the needs of customers by continually bringing new services. A minor concern is that the delivery of more services and features makes it even more confusing for developers and organizations. Although there may be slight differences in AWS, Azure, and Google Cloud offerings, it is evident that they all share a common goal. They are all seeking to simplify running Docker containers on the cloud orchestration while maintaining the flexibility required to support a wide range of developer use cases.

Introduction The purpose of this article is to provide guidelines for those who are interested in the details of setting up the Kubernetes cluster, understanding internetworking of pods, and other aspects related to the Kubernetes cluster. This article provided the details for setting up Kubernetes Cluster on Ubuntu. The main topics are the following: Overview Kubernetes Architecture Container Network Interface with Calico Detailed Procedures for Setting up Kubernetes Cluster Troubleshooting Procedures For the Kubernetes setup, I use three nodes cluster in which we have one master node and two worker nodes. The following are the specific types for the Kubernetes cluster Container Runtime: containerd Network Policy and CNI: Calico Operating System: Ubuntu Architecture Overview The following diagram illustrates the components in Kubernetes clusters. Kubernetes nodes fall into two categories: Control Plane and Worker Nodes. All the applications are running on the worker nodes as containers inside the pod. Click here for more details. A few key points worth noting: The API server is the brain of the cluster. Virtually all the communication and administration are carried out by this component The communication between worker nodes and the control plane is through kubelets and API-server. Kubernetes Component Architecture For CNI/network policy, I use Calio. There are many CNI providers. The following table is a key summary of open-source CNI providers: Flannel Calico Cilium Weavenet Canal Mode of Deployment DaemonSet DaemonSet DaemonSet DaemonSet DaemonSet Encapsulation and Routing VxLAN IPinIP,BGP,eBPF VxLAN,eBPF VxLAN VxLAN Support for Network Policies Yes Yes Yes Yes Yes Datastore used Etcd Etcd Etcd No Etcd Encryption Yes Yes Yes Yes No Ingress Support No Yes Yes Yes Yes Enterprise Support Yes Yes No Yes No Three big cloud providers, Azure, GCP, and AWS have the following CNI: Microsft has Azure CNI Google GKE uses kubenet CNI which is on top of Calico AWS uses Amazon VPN CNI Kubernetes Pod Networking With Calico Setup Details In this section, I will describe the details for setting up the Kubernetes cluster. Pre-Setup Setup Kubeetc autocomplete on the Master node: Add the following to the ~/.bashrc on the master node: source <(kubectl completion bash) # set up autocomplete in bash into the current shell, bash-completion package should be installed first. echo "source <(kubectl completion bash)" >> ~/.bashrc # add autocomplete permanently to your bash shell. alias k=kubectl complete -o default -F __start_kubectl k Visit the Kubectl Cheat Sheet Common Commands On All Nodes Change Hostname sudo hostnamectl set-hostname "k8smaster.ggl.com" sudo hostnamectl set-hostname "k8sworker1.ggl.com" sudo hostnamectl set-hostname "k8sworker2.ggl.com" Add Entries to /Etc/Hosts 172.31.105.189 k8smaster.ggl.com k8smaster 172.31.104.148 k8sworker1.ggl.com k8sworker1 172.31.100.4 k8sworker1.ggl.com k8sworker2 Disable Swap sudo swapoff -a sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab cat /etc/fstab Note: The reason for disabling the memory swapping (swapoff) is for stability and performance considerations. This is the required step in any Kubernetes setup (AKS, GKE, EKS). Load Kernel Modules sudo tee /etc/modules-load.d/containerd.conf <<EOF overlay br_netfilter EOF sudo modprobe overlay sudo modprobe br_netfilter lsmod | egrep "overlay|br_net" Note: Kubernetes uses an overlay kernel module for the file systems. Linux kernel br_netfilter is for forwarding IP4 traffic and letting iptable see the bridged traffic. Set Kernel Parameters sudo tee /etc/sysctl.d/kubernetes.conf <<EOF net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1 EOF sudo sysctl --system Enable Docker Repository sudo apt install -y curl gnupg2 software-properties-common apt-transport-https ca-certificates sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmour -o /etc/apt/trusted.gpg.d/docker.gpg sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" Install and Enable Containered Runtime sudo apt update sudo apt install -y containerd.io containerd config default | sudo tee /etc/containerd/config.toml >/dev/null 2>&1 sudo sed -i 's/SystemdCgroup \= false/SystemdCgroup \= true/g' /etc/containerd/config.toml cat /etc/containerd/config.toml sudo systemctl restart containerd.service sudo systemctl enable containerd sudo systemctl status containerd Add Kubernetes Repository curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main" sudo apt update Install kubeadm kubelet kubectl sudo apt install -y kubelet kubeadm kubectl After executing the above commands on all the nodes (master and worker nodes), we will need to perform the following: Initialize the Kubernetes clusters with kubeadm init command Have worker nodes join the cluster Control Plane Setup Commands Initialize Kubernetes Cluster sudo kubeadm init --pod-network-cidr=192.168.0.0/16 --control-plane-endpoint=k8smaster.ggl.com The following is the output from the kubeadm init command: To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config Alternatively, if you are the root user, you can run: export KUBECONFIG=/etc/kubernetes/admin.conf You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ You can now join any number of control-plane nodes by copying certificate authorities and service account keys on each node and then running the following as root: kubeadm join k8smaster.ggl.com:6443 --token 7sx7ky.z54u1q0pexh5vh25 \ --discovery-token-ca-cert-hash sha256:6cce1257cfdbd54c981ad64e2a553711e276afc402a52e3899a4725470902686 \ --control-plane Then you can join any number of worker nodes by running the following on each as root: kubeadm join k8smaster.ggl.com:6443 --token 7sx7ky.z54u1q0pexh5vh25 \ --discovery-token-ca-cert-hash sha256:6cce1257cfdbd54c981ad64e2a553711e276afc402a52e3899a4725470902686 Note: the joining commands for master nodes and worker node are different with only option --control-plane. You may perform kubectl get nodes to check node, but you will see the node is in the NOT ready state. This is because we have not installed the CNI. Install Calico Container Network Interface (CNI) kubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.25.0/manifests/calico.yaml Check Cluster Information $ kubectl get nodes NAME STATUS ROLES AGE VERSION k8smaster.ggl.com Ready control-plane 6m22s v1.26.1 Note: At this moment, we have not joined the worker node yet. Thus we have only one node is in the ready state. Open Kubernetes Ports $ sudo lsof -iTCP -nP | egrep LISTEN systemd-r 515 systemd-resolve 14u IPv4 15918 0t0 TCP 127.0.0.53:53 (LISTEN) sshd 935 root 3u IPv4 17128 0t0 TCP *:22 (LISTEN) sshd 935 root 4u IPv6 17130 0t0 TCP *:22 (LISTEN) node 1464 root 18u IPv4 20071 0t0 TCP *:31297 (LISTEN) container 3743 root 13u IPv4 35540 0t0 TCP 127.0.0.1:45019 (LISTEN) kube-sche 6398 root 7u IPv4 47451 0t0 TCP 127.0.0.1:10259 (LISTEN) kube-cont 6425 root 7u IPv4 47423 0t0 TCP 127.0.0.1:10257 (LISTEN) kube-apis 6446 root 7u IPv6 48257 0t0 TCP *:6443 (LISTEN) etcd 6471 root 7u IPv4 47402 0t0 TCP 172.31.105.189:2380 (LISTEN) etcd 6471 root 8u IPv4 47406 0t0 TCP 127.0.0.1:2379 (LISTEN) etcd 6471 root 9u IPv4 47407 0t0 TCP 172.31.105.189:2379 (LISTEN) etcd 6471 root 14u IPv4 48266 0t0 TCP 127.0.0.1:2381 (LISTEN) kubelet 6549 root 23u IPv6 47676 0t0 TCP *:10250 (LISTEN) kubelet 6549 root 26u IPv4 47683 0t0 TCP 127.0.0.1:10248 (LISTEN) kube-prox 6662 root 14u IPv4 49323 0t0 TCP 127.0.0.1:10249 (LISTEN) kube-prox 6662 root 15u IPv6 49330 0t0 TCP *:10256 (LISTEN) calico-no 7377 root 10u IPv4 55531 0t0 TCP 127.0.0.1:9099 (LISTEN) Open Ports For the Pods Communication sudo ufw status verbose sudo ufw allow 6443/tcp sudo ufw allow 10250/tcp sudo ufw allow 2379:2380/tcp This is a very important step to open all the necessary ports. Otherwise, we will not be able to join the worker node. In the other Linux environments, we need to disable the firewall in order for the work node to join the cluster. Print Commands for Worker Node To Join the Cluster sudo kubeadm token create --print-join-command Worker Node Commands Join The Kubernetes Cluster sudo kubeadm join k8smaster.example.net:6443 \ --token ufkijl.ukrhpo372w6eoung \ --discovery-token-ca-cert-hash sha256:e6b04ca3f6f4258b027d22a5de4284d03d543331b81cae93ec4c982ab94c342f Open Ports On Worker Node sudo ufw status sudo ufw allow 10250 sudo ufw allow 30000:32767/tcp sudo ufw status Verify The Kubernetes Setup Create NGINX Deployment kubectl create deployment nginx-app --image=nginx --replicas=2 kubectl get deployments.apps nginx-app kubectl expose deployment nginx-app --type=NodePort --port 80 kubectl get svc nginx-app kubectl describe service nginx-app $ k get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx-app-5d47bf8b9-ps6q2 1/1 Running 2 (169m ago) 23h 192.168.159.4 k8sworker1.ggl.com <none> <none> nginx-app-5d47bf8b9-xbdzz 1/1 Running 2 (169m ago) 23h 192.168.186.196 k8sworker2.ggl.com <none> <none> $ kubectl describe service nginx-app Name: nginx-app Namespace: default Labels: app=nginx-app Annotations: <none> Selector: app=nginx-app Type: NodePort IP Family Policy: SingleStack IP Families: IPv4 IP: 10.105.76.203 IPs: 10.105.76.203 Port: <unset> 80/TCP TargetPort: 80/TCP NodePort: <unset> 32592/TCP Endpoints: 192.168.159.4:80,192.168.186.196:80 Session Affinity: None External Traffic Policy: Cluster Events: <none> Note: the nginx service is run on the port 32592. $ curl http://k8sworker1:32592 <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style> html { color-scheme: light dark; } body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html> This means everything is working. Troubleshooting Techniques During setup, in general, everything works pretty smoothly. However, sometimes, there may be firewalls blocking access to the kube-API service. In this case, run the following command: $ sudo lsof -iTCP | egrep LISTEN Check which port the kube-API is listening. In my case, it looks like the following: kube-apis 1346 root 7u IPv6 19750 0t0 TCP *:6443 (LISTEN) Thus, we need to verify the port is open by the following command from the worker node: $ telnet k8smaster 6443 Trying 172.31.105.189... Connected to k8smaster.ggl.com. Escape character is '^]'. This means it is working. If it hangs, it means the port is not open To troubleshoot network issues, the following Linux tools are very useful: lsof nmap netstat telnet ping

Many engineers get lost in Azure’s services for running containers. Starting with the basic Azure container services, the cloud provider quickly developed other options like AKS and Azure Container Apps to give users even more options of where and how to run containers. A high number of container-as-a-service solutions Azure offers address the needs of enterprises looking to move legacy monolithic applications to the cloud and containers, decreasing application deployment and management complexity at the same time. But Azure’s offering spans more than that. You can find plenty of easily accessible information about Azure service offerings for running containers. But there aren’t many direct comparisons in terms of pricing. At the end of the article, we added a table of prices normalized based on compute resources. That way, you can compare these services like apples to apples. But first, let’s get through the high-level functionality of each service. List of Azure Compute Services Where You Can Run Containers Azure’s offer is so broad that it’s easy to quickly get lost in all of the available options. For compute, Azure prepared a handy breakdown of all the hosting models for the resources your application runs on. (1) To run a container on Azure, you can choose from the following services: Azure Virtual Machines — deploy and manage VMs within an Azure virtual network. This option should be used as a last resort if there’re no other means to reach the desired customization level. This will require handling additional complexity and work around managing a virtual machine on your own. Azure App Service — a handy managed service for hosting all kinds of web apps, mobile app backends, RESTful APIs, or business automation processes. Azure Functions — this is a managed function as a service (FaaS) service, a popular choice for serverless architectures. Azure Kubernetes Service — AKS is a managed Kubernetes service for running containerized applications with advanced and highly customizable orchestration functionality. Azure Red Hat OpenShift — yet another flavor of Kubernetes in Azure. Red Hat OpenShift offers additional security, and observability features out of the box but at the expense of increased price. Azure Container Apps — this managed service was developed on top of Kubernetes to help devs simplify the deployment of containerized applications without worrying about orchestration or infrastructure. Azure Container Instances — an efficient but pretty basic service for running containers in Azure. You don’t have to dabble in virtual machine provisioning or adopt a higher-level service. Azure Spring Apps — a managed service created and optimized for hosting Java’s Spring Boot apps. Azure Service Fabric — a platform for distributed systems that can run in many environments, be it Azure or on-premises. It was developed as an Azure custom orchestration platform way before Kubernetes got mainstream. Azure Batch — a managed service for large-scale parallel and high-performance computing (HPC) applications. As the name implies, it is mostly suited for scheduled jobs. Overview of Azure Containers Hosting Options Virtual Machine (i.e., Running Containers Standalone) You can adopt a DIY approach and provision an Azure VM, install a container runtime on it, and run your containers there. But you’ll have to deal with all the responsibilities around the configuration, management, and maintenance. Going for a managed service just makes more sense — unless you need full control over the underlying operating system or you’re planning to customize things in a way that’s not supported by any of Azure’s PaaS services. Running a virtual machine on your own should be used as the last tool in the box. Azure App Service Container hosting from App Service mainly supports web apps or APIs. If that’s your case and you can make it work without complex orchestration, this may be a good option. You don’t need to deal with as much management and get many features in exchange. But if your workload is anything else than a web app or API, you’re better off using a different service. App Service doesn’t let you expose your service on ports or traffic different from HTTP(S). The service is also a bad choice if you expect large-scale bursting. Price-wise App Service is only a little more expensive than standalone VMs. But its basic and standard plans support only Linux containers. If you’re planning to run Windows, you’ll need the Premium V3 plan, which can get pretty expensive. Azure Functions If you have function-based or short-lived serverless workloads and want to use containers to develop and deploy them, Azure Functions is a great pick. You’ll get all the features like vNet integrations, triggers, and bindings. However, if your application doesn’t really fit into the serverless universe, there’s no point in choosing Azure Functions. What about the price of Azure Functions? Custom containers often end up requiring a pricier premium or always-on consumption tiers that are bound to increase your expenses. It’s the priciest solution when comparing resources cost per runtime. But you only get charged for only what you run. Great cost optimization when running infrequent requests, but get extremely expensive when business grows. This behavior caught many people off guard Azure Kubernetes Service (AKS) This is the fully managed Kubernetes service from Azure, where you can still customize the machines where your workloads run. A great pick for containers that need comprehensive orchestration or use advanced Kubernetes features. However, you still need to know your way around K8s and how deploying applications to this service works. So, if you aim to run a few containers, AKS will bring too much complexity and overhead. Pick an Azure PaaS offering instead, like Azure Container Apps. What about AKS pricing? The solution costs as much as the VMs where your containers run — but note that some space is reserved for Kubelet running on every node. Azure Red Hat Openshift (ARO) ARO gives you a bunch of extra management and integration services on top of Kubernetes. Plus, it is fully integrated into the RedHat/IBM ecosystem. If you could use them, great — ARO is your choice. If you don’t, stick to a service like Azure Kubernetes Service. This is a pricey option because you need to count in the costs for the master, additional infrastructure nodes, and licenses. Still, the cost of this service is justified for enterprises who look at the additional benefits: Managed observability stack out of the box. It runs on the aforementioned infrastructure nodes. Which are automatically upscaled if/when the cluster grows. Advanced security and user management. It’s a more strict environment compared to Kubernetes defaults to allow everything at the start. Cluster management is abstracted by an additional OpenShift orchestration layer. It prevents direct access to Kubernetes API, hence lowering the possibilities of what could be customized there. It’s a pro and con at the same time, as you can do less and break less too. Enterprise-grade security – you get top compliance, operations, and security with an SLA of 99.95% and plenty of certifications (PCI DSS, ISO 27001, SOC 2 Type II, and more). Azure Container Apps (ACA) This service was designed to offer the same benefits as AKS but with less complexity to deal with. If you need an orchestrator and ACA has the features you need, go for it. You’ll remove a lot of management and maintenance tasks from your schedule. This should be the first option to look at if Kubernetes complexity is not required. The service is also a good choice for applications that scale dynamically, as it supports HTTP and KEDA scaling, as well as scaling to zero. If you want more control or require a feature that ACA doesn’t have — AKS is your next bet. What about the costs? ACA charges you per container runtime instead of hourly pricing of VMs where your container runs. As you can see in the table below, it can get pricey, but you’d be actually using all these resources, whereas, in Kubernetes, your cluster might run half-empty if not configured properly. Azure Container Instances (ACI) This solution works well if you just need to run a container and not much else. ACI is also helpful for running many instances of the same container or dynamically scaling up the number of containers in use. It makes sense to pick ACI over other services if you have a job-based workload that runs for a limited time — or an on-demand service. Note that ACI isn’t an orchestrator. If your containers need a bit more complicated orchestration or you’d like to have more control over the underlying hardware, ACI isn’t a good choice. The same is true for scenarios like private container-to-container networking. Another limitation is container size. If you need containers that are larger than 4 cores and 16 GB RAM, ACI can’t help you. Another issue relates to large container images — if you have them, expect a slow start time – that’s because ACU needs to download the container image every time it’s run. What about the pricing? As you can see in the table below, ACI is slightly more expensive than an analogous VM. Spring Apps A fully managed service for Spring developers, which helps them to manage the lifecycle of Spring Boot applications. Choosing this service would be a good choice for migrating a Spring project from self-hosted infrastructure to the cloud since it wouldn’t require many additional engineering expenses to make applications cloud-ready. Spring Apps includes monitoring and diagnostics, configuration management, service discovery, CI/CD integration, and blue-green deployments. The service also comes with its joint engineering, operation, and integration from Microsoft and VMware. Service Fabric This orchestrator had one advantage – it offered the Actor framework built into the platform. It is a custom Azure orchestration framework that was built ten years ago. If you don’t know what it is and why you should use it. You can achieve the same with AKS and DAPR. Consider choosing it only if you require specific functionality that’s only available there. Or if your project is already built into this platform. Otherwise, Kubernetes, with the help of AKS or ACA, is a better alternative. Azure Batch If you’re after a service that meets your specific requirements around job processing, these will be more important than what you need as a container hosting platform. Using containers with Batch improves the management of development and deployment. Pick it if you have a job-based batch workload and are looking to use containers for enhancing your development and deployment process. Workloads that don’t match the job-based approach better run elsewhere. How To Pick the Right Azure Containers Service Consider these aspects when reviewing the services Azure offers for containers: Your knowledge of and skills around container orchestrators like Kubernetes — are you ready to take on configuration tasks? Do you know the Kubernetes ecosystem well? If you don’t, Azure still offers a bunch of services you can use to run your application using this orchestrator. Your familiarity with the service offering from Azure — some Azure services are more complex, while others do the job for you. Having clarity around the service offering, together with its pros and cons, is a basic condition. The application you’re looking to containerize — Azure offers a bunch of services that clearly target the legacy modernization use case. The good news is that you can carry this process out gradually instead of migrating everything instantly (and facing a bunch of issues). The money you’re willing to spend — if you go through all of those services, the pricing will turn out to be the deciding factor. Azure makes moving applications to the cloud and containers easy, but you’ll always encounter some extra fees. Azure provides a handy decision tree that helps to understand your options: Source (2) The diagram refers to two migration strategies: Lift and shift — where you don’t make any code changes or redesign your services for the cloud; just move it there. Cloud optimized — you refactor an application to fully benefit from the cloud-native capabilities of Azure. I strongly recommend checking out Azure’s materials if you need more information on the hosting models, networking, DevOps issues, scalability, availability, security, and other criteria.3 There’s one more aspect that’s worth considering here: pricing. Pricing Comparison for Azure Containers Services To compare the pricing of these services, let’s normalize the price of each service. I chose a pretty old general-purpose family: The dv3-series. It’s the most common machine family throughout the Azure ecosystem and is suggested as the default instance for many services. Let’s use the Standard_D4_v3 instance as an example. We’ll also use the East US region and Linux OS. This instance has 4 vCPUs and 16GiB of RAM and costs $0.192 per hour. Selecting a larger instance doesn’t give any price advantage per given resource. Note: Normalizing Standard_D4_v3 to separate CPU and RAM usage per month requires a specific formula. We’re working on a detailed article on how it’s calculated and will share our results pretty soon. Considering that the article is not read by robots, we’ll use pricing units per month, so it’s easier to comprehend for humans. Service Name Normalized 1 vCPU / month Normalized 1 GiB RAM / month Resources Overhead Final1 vCPU / month cost Final1 GiB Ram / month cost Virtual Machine $22.81 $3.06 – $22.81 $3.06 Azure App Service $36.83 $4.93 – $36.83 $4.93 Azure Functions $126.29 $8.98 – $126.29 $8.98 Azure Kubernetes Service $22.81 $3.06 ¹ Kubelet reserved CPU 2%Mem 16%Optional $73 for control plane 99.95% SLA $23.27 $3.55 Azure Red Hat Openshift (ARO) $22.81 $3.06 Red Hat Licensing ~+89%¹ Kubelet reserved CPU 2%Mem 16%² Master and Infra nodes. At least 6x Standard_D8s_v3 (+$1681.92/month) $43.98 $6.71 Azure Container Apps $63.07 $7.89 – $63.07 $7.89 Azure Container Instances $29.57 $3.25 – $29.57 $3.25 Azure Spring Apps $55.78 $6.28 – $55.78 $6.28 Azure Service Fabric $29.57 $3.25 – $29.57 $3.25 Azure Batch $22.81 $3.06 – $22.81 $3.06 This table considers costs only related to compute resources. ¹ For a 4 CPU and 16 GiB node. The percentage is smaller if you use larger nodes. ² ARO requires running control-plane and infrastructure nodes – three of both, so six nodes in total. Their size depends on the cluster size. These nodes are upscaled vertically (ARO control plane requirements).4 Summary There are plenty of offerings to choose from when thinking about where to run your containers in Azure. A wide range of options covers many different use cases. But at the same time, it’s not that easy to select which one is best suited for your use case now, let alone predict if your choice will work in the future. Azure developed and released the above selection of services throughout the last decade. More legacy services like Service Fabric have technological disadvantages compared to Kubernetes orchestration, but they can’t be decomposed because of existing users and systems. So one has to be careful when choosing where to build a green field system to not select outdated technologies. Lastly, as shown in the pricing table, there are big differences in how these services are priced. Choosing the Functions service might look cheap at first as you pay only for how long the application runs, but you might go broke once your application takes off and you achieve business success, but at the same time are locked into an unreasonable cloud deal. At the same time, choosing Kubernetes as a starting point for a simple application would create a lot of engineering and cost overhead, which would obviously be better spent on creating value for the product you’re developing. At the end of the day, you’re the one responsible for choosing the right service because you’re the only person out there who fully understands the current and possible future states of your application. Wherever You Run Azure Containers, Don’t Forget To Secure Them Take a look around these Azure container services to pick the ones matching your needs and level of complexity. And no matter where you end up running them, don’t forget about security. If you happen to pick Azure Kubernetes Service, here’s a handy guide to help you achieve airtight AKS security for your cluster.

Linux containers are a powerful solution for: Software standardization Acceleration of development and testing Effective resource management throughout the whole lifecycle of an application Here, we will provide a step-by-step guide to building Linux containers with applications intended for Cloud deployment. As an example, we will use BellSoft’s open-source Alpaquita Stream containers hosted on the Docker Hub Container Image Library. To complete the guide, you must have the docker daemon installed and running. We will explain how to pull a selected container image from the repository and run it with Java, Python, GCC-based applications, and native images. You can use your own app or create a sample project, as shown below. Linux Containers for Java Applications Pull a Docker Image To create a container, run: $ docker image pull docker bellsoft/<repository>:<image_tag> Start the container with: $ docker run -it --rm bellsoft/<repository>:<image_tag> Alternatively, utilize a combined command: $ docker container run --rm -it bellsoft/<repository>:<image_tag> Suppose you have chosen a Liberica Runtime Container, which includes musl-based Alpaquita Stream (a glibc-based option is also available) and Liberica JDK Lite 11. The command will be as follows: $ docker container run --rm -it bellsoft/liberica-runtime-container:jdk-11.0.17-musl Note that pulling an image is not obligatory but recommended if you don’t want to pull an image every time you repeat the build process. Write a Dockerfile and Run the App By using a JRE image instead of a full JDK to run a Java application, you will reduce the container size by approx. 50%. So, in the Dockerfile for our application, we will specify two images — one for building an app and another for running it. We will use a standard Spring Petclinic project as a sample. Write the following Dockerfile: Dockerfile FROM bellsoft/liberica-runtime-container:jdk-17-stream-musl as builder WORKDIR /home/myapp RUN apk add git RUN git clone https://github.com/spring-projects/spring-petclinic.git RUN cd spring-petclinic && ./mvnw package FROM bellsoft/liberica-runtime-container:jre-17-stream-musl WORKDIR /home/myapp COPY --from=builder /home/myapp/spring-petclinic/target . CMD ["java", "-jar", "spring-petclinic-3.0.0-SNAPSHOT.jar"] Build the app image: $ docker build --progress plain -t javaappimage . Now, run the image: docker run -p 8080:8080 javaappimage Done! Open the browser to access the Petclinic application. Linux Containers for Native Image Pull a Docker Image We provide images with Liberica Native Image Kit (NIK), an open-source GraalVM-based utility for converting JVM applications into native executables. Developers who work with native images or consider integrating the technology into their project can pull the following image: $ docker container run --rm -it bellsoft/liberica-native-image-kit-container:jdk-11-nik-21.3.3-stream-musl Or any image you deem appropriate depending on the JDK and NIK version and libc implementation (BellSoft’s optimized musl or glibc). Create a Native Image First, let’s write a simple application to be converted into a native executable. Alternatively, use your own project. Java public class Example { public static void main(String[] args) { String str = "Native Image is awesome"; String reversed = reverseString(str); System.out.println("The reversed string is: " + reversed); } public static String reverseString(String str) { if (str.isEmpty()) return str; return reverseString(str.substring(1)) + str.charAt(0); } } The Dockerfile must contain the following information: Dockerfile FROM bellsoft/liberica-native-image-kit-container:jdk-11-nik-21.3.3-stream-musl WORKDIR /home/myapp COPY Example.java /home/myapp RUN javac Example.java RUN native-image Example FROM bellsoft/alpaquita-linux-base:stream-musl WORKDIR /home/myapp COPY --from=0 /home/myapp/example. CMD ["./example"] Go to the application directory and run: $ docker build. Below is the process of building the image: Dockerfile [+] Building 370.9s (14/14) FINISHED => [internal] load build definition from Dockerfile 0.1s => => transferring dockerfile: 357B 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/bellsoft/alpaquita-linux-base:stream-musl 0.0s => [internal] load metadata for docker.io/bellsoft/liberica-native-image-kit-container:jdk-11-nik-21.3.3-stream-musl 4.8s => [internal] load build context 0.1s => => transferring context: 454B 0.0s => [stage-1 1/3] FROM docker.io/bellsoft/alpaquita-linux-base:stream-musl 0.1s => [stage-0 1/5] FROM docker.io/bellsoft/liberica-native-image-kit-container:jdk-11-nik-21.3.3-stream-musl@sha256:3c5a09abb2559cf0 293.4s => => resolve docker.io/bellsoft/liberica-native-image-kit-container:jdk-11-nik-21.3.3-stream-musl@sha256:3c5a09abb2559cf04e7527b81b 0.0s => => sha256:3c5a09abb2559cf04e7527b81bfa123ca52592ef2bce1512d837fbd0336440e9 951B / 951B 0.0s => => sha256:79998423fd609bb42881f8eb6721aa22451cdb46363b4fc5f49fc50390759138 2.72kB / 2.72kB 0.0s => => sha256:4f4293381c77f4d905cfa2034b4fe3adff4515508d08b0093a8db16728ba4879 3.34MB / 3.34MB 7.1s => => sha256:c3263c13b6f5c1ba5a386dc02047af54fdeefc8a92538ffc973915cdb0de9881 532.42kB / 532.42kB 7.0s => => sha256:f6c678d36153293dc6b01a73696b4ef60a9ecb44e62da72fd96b31e679f1ed2d 323.95MB / 323.95MB 284.4s => => extracting sha256:4f4293381c77f4d905cfa2034b4fe3adff4515508d08b0093a8db16728ba4879 0.1s => => extracting sha256:c3263c13b6f5c1ba5a386dc02047af54fdeefc8a92538ffc973915cdb0de9881 0.0s => => extracting sha256:f6c678d36153293dc6b01a73696b4ef60a9ecb44e62da72fd96b31e679f1ed2d 8.8s => [stage-1 2/3] WORKDIR /home/myapp 0.0s => [stage-0 2/5] WORKDIR /home/myapp 0.5s => [stage-0 3/5] COPY Example.java /home/myapp 0.0s => [stage-0 4/5] RUN javac Example.java 1.0s => [stage-0 5/5] RUN native-image Example 70.6s => [stage-1 3/3] COPY --from=0 /home/myapp/example . 0.1s => exporting to image 0.1s => => exporting layers 0.1s => => writing image sha256:f03df73515790f3ffe210fc139d5a0752c088fd242f571e19d111377f7703c6a 0.0s Verify the image was created. Dockerfile $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE <none> <none> f03df7351579 About a minute ago 19MB Tag the newly built image with a meaningful name. Dockerfile $ docker tag f03df7351579 bellsoft-nik:example-musl $ docker run -it --rm f03df7351579 The reversed string is: emosewa si egamI evitaN Linux Containers for Python BellSoft provides Alpaquita Linux images with Python 3.10 and basic Python utilities (pip, setuptools, wheel). Two libc options (BellSoft’s musl perf and glibc) are available. First, let’s write a simple flask application: Dockerfile $ cat requirements.txt flask markup jinja2 $ cat test.py from flask import Flask app = Flask(__name__) @app.route('/') def hello_world(): return 'Hello, Docker!' Now, write a Dockerfile with the following contents: Dockerfile FROM bellsoft/alpaquita-linux-python:3.10-stream-musl WORKDIR /myapp COPY requirements.txt . RUN pip3 install -r requirements.txt COPY test.py . ENV PATH=/root/.local:$PATH CMD ["python3", "-m" , "flask", "--app", "./test.py", "run", "--host=0.0.0.0"] Finally, build a Docker image with this Dockerfile: $ docker build -t flaskimage . The process of building an image will start: Dockerfile [+] Building 1.3s (10/10) FINISHED => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 38B 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/bellsoft/alpaquita-linux-python:3.10-stream-musl 1.2s => [internal] load build context 0.0s => => transferring context: 63B 0.0s => [1/5] FROM docker.io/bellsoft/alpaquita-linux-python:3.10-stream-musl@sha256:2d5656810f19f028b5992cbfcdd3e833cfb1839b5b6078b5a2a1 0.0s => CACHED [2/5] WORKDIR /myapp 0.0s => CACHED [3/5] COPY requirements.txt . 0.0s => CACHED [4/5] RUN pip3 install -r requirements.txt 0.0s => CACHED [5/5] COPY test.py . 0.0s => exporting to image 0.0s => => exporting layers 0.0s => => writing image sha256:9f9335b2d7339a656e1b860e0d0d293c528e06aa21d0be41ee46b67fbb9e9f12 0.0s => => naming to docker.io/library/flaskimage 0.0s You can now run your Python application in Docker: Dockerfile $ docker run -d -p 6000:5000 flaskimage 630ae5583186bf0a5d99c97bca68b41401cf6b4b1d22f770fa4f757d23dc240e $ curl http://127.0.0.1:6000 Hello, Docker! Linux Containers for GCC BellSoft provides Alpaquita Linux images with the GCC compiler version 12.2, tools, and libraries for development in C/C++. Two libc options (BellSoft’s musl perf and glibc) are available. We will use a C++ example to show that our GCC image could be used for building both C++ and C projects. Our GCC images include the following packages that are not obligatory but could be useful for building: C++ autoconf automake bash binutils bzip2 bzip2-dev curl curl-dev diffutils file findutils fortify-headers g++ gawk gcc gdb git grep hexdump jpeg-dev krb5-dev libevent-dev libffi-dev libjpeg-turbo-dev libpng-dev libtool libxml2-dev libxslt-dev make ncurses-dev openssl-dev patch pkgconf readline-dev sed subversion unzip xz xz-dev yaml-dev zlib-dev First, let’s write a simple C++ program: C++ $ cat hello.cpp #include <iostream> using namespace std; int main() { cout << "Hello World" << endl; return 0; } In the Dockerfile, specify an Alpaquita image for building an app, and a base image, which will be used for working with the container further on: Dockerfile FROM bellsoft/alpaquita-linux-gcc:12.2-stream-musl as builder WORKDIR /home/myapp COPY hello.cpp . RUN g++ hello.cpp -o hello -static FROM bellsoft/alpaquita-linux-base:stream-musl WORKDIR /home/myapp COPY --from=builder /home/myapp/hello . CMD ["./hello"] Now, build the Docker container image: Dockerfile $ docker build -t cppappimage . Run it with the following command: $ docker run --rm cppappimage Hello World Conclusion In this article, we learned how to construct Linux containers with Java, Python, C/C++ applications, and Native Image. We used Alpaquita Stream as a base Linux distribution. As you can see, containerization with Alpaquita is fast and effortless, and in the end, you get a performant microcontainer that can be used as is or further configured for specific purposes.

Docker is a containerization technology that allows developers to package and deploy applications in lightweight, portable containers. These containers are isolated from the host operating system, which makes them portable across different environments and eliminates the “works on my machine” problem. Docker is a popular platform for creating and managing containerized applications; however, several alternatives for Docker can also be used for this purpose. Podman, Kubernetes, Openshift, LXD, Docker Swarm, BuidKit, and Mesos are some of the popular Docker alternatives available in the market today. In this article, we’ll discuss the following three Docker hub alternatives: Podman, Containerd, and LXD. So, let’s Begin! Podman Developed by RedHat, Podman is a daemonless, open-source, Linux-native container engine that is considered one of the best alternatives for Docker. Podman is used to build, run, and manage Linux OCI containers and container images. A container engine is an all-in-one software that is responsible for creating, running, and managing containers. A container engine provides an API or command-line interface for interacting with the containers, allowing developers to create, start, stop, and manage containers. Examples of container engines include Docker, Podman, and CRI-O. Podman uses the lib pod library, which provides a higher-level API for managing pods and containers. It also provides built-in support for rootless containers and improved security features. Advantages of Podman Easy to use: Podman has a simple and intuitive command-line interface that is similar to Docker’s command-line interface, making it easy for users who are already familiar with Docker to start using Podman. Compatible with Kubernetes: Podman can be used in confluence with Kubernetes, which means it can be used to run containers on a cluster adn locally. Support for multiple container formats: Podman supports both OCI and Docker container formats, which means it can run containers created using either format. Support for Cgroups v2: Podman supports Cgroups v2, which is a new version of the Linux kernel’s control group (cgroup) mechanism that provides more fine-grained control over resource allocation. Network support namespaces: Podman supports network namespaces, which allows you to use different network configurations for different containers. Differences Between Podman and Docker Docker and Podman are container engines, but there are some key differences between the two. Docker and Docker hub alternatives, such as Podman, are widely used and supported in the industry, and it depends on the specific use case and requirements of which one to use. Here are some of the key differences between Docker and Podman: Daemonless: Podman does not require a daemon to run containers, whereas Docker uses a daemon to manage containers. This means that Podman can run containers directly without the need for an additional service running in the background. Rootless: Podman can run containers without needing root access, whereas Docker requires root access to manage the container daemon. This makes Podman more secure, as it limits the potential attack surface. Image storage: Podman stores images in the local file system, whereas Docker uses a centralized image registry. This means that, with Podman, it is not necessary to have an internet connection to use local images. Networking: Docker uses its own networking stack, while Podman uses the host’s networking stack. CLI: Both have similar command line interfaces, so it’s easy to switch between them. Overall, Docker and Podman are powerful tools for containerization. For these two, and any other Docker alternatives, the ultimate choice between them often comes down to personal preference and specific use case requirements. Containerd Next on the list of Docker alternatives is Containerd. Containerd is a high-level, lightweight container runtime that provides a consistent and stable interface for running containers. Designed to be used as a daemon process that runs on a host system, it manages the lifecycle of containers by starting and stopping them, as well as providing other features, such as image management and storage. Containerd is also designed to work with other container orchestration tools, such as Kubernetes, to manage the scaling and scheduling of containers in a cluster. Advantages of Containerd Lightweight: Containerd is designed to be lightweight and fast, which means it has a small footprint and uses minimal resources. This makes it well-suited for use in high-performance and resource-constrained environments. Consistency: Containerd provides a consistent and stable interface for running containers, which makes it easier to manage and orchestrate them at scale. Flexibility: Containerd can be used with a variety of container orchestration tools, such as Kubernetes and Docker Swarm, which allows for greater flexibility in terms of how containers are managed and scaled. Plugins: Containerd has a modular design and support for plugins, which allows for easy customization and extension of its functionality. Security: Containerd provides a secure and isolated environment for running containers, and it has built-in support for image signing and validation. Support: Containerd is an open-source project with a large and active community, which means it has a wide range of support and resources available. Differences Between Containerd and Docker Containerd and Docker are container runtimes, but they have some key differences. Let’s take a look at these: Design: Containerd is designed to be a lightweight and minimal container runtime, while Docker is a more fully-featured container platform that includes additional components such as a built-in container registry and a management API. Functionality: Containerd focuses on providing a stable and consistent interface for running containers, while Docker provides a more comprehensive set of features such as image management and orchestration. Deployment: Containerd is intended to be used as a daemon process that runs on a host system, while Docker is typically deployed as a standalone service. Architecture: Containerd has a modular architecture that is designed to work with other container orchestration tools, while Docker has its own built-in orchestration features. Support: Containerd is an open-source project that is backed by a large and active community, while Docker is a commercial product that is supported by the company behind it. Plugins: Containerd has a pluggable architecture, which means it can be extended or customized using plugins, while Docker does not have a similar feature. Security: Containerd provides a secure and isolated environment for running containers and has built-in support for image signing and validation, while Docker does not have this feature by default. LXD Now, we’ll discuss one of the most commonly used alternatives of Docker in the list of Docker hub alternatives. LXD (Linux Containers Daemon) is a container hypervisor for Linux. It allows multiple isolated Linux systems (containers) to run on a single host, providing a lightweight alternative to virtual machines. LXD uses Linux kernel features, such as control groups and namespaces, to provide isolation, while also providing a simple and user-friendly command-line interface for managing containers. LXD is designed to work with existing Linux distributions and tools and supports a wide range of container images and formats, including Docker. It also provides advanced features like live migration, storage management, and network management. Developed and maintained by Canonicals, LXD is one of the well-known Docker hub alternatives and is the default container hypervisor for Ubuntu 20.04 and later versions. Advantages of LXD There are several advantages to using LXD as a container hypervisor. LXD is one of the most known Docker desktop alternatives available in the industry today. Take a look at the advantages of LXD below: Lightweight and fast: LXD uses Linux kernel features, such as control groups and namespaces, which are more lightweight and efficient than traditional virtualization methods. This results in faster startup times and lower resource overhead for containers. Easy to use: LXD provides a simple and user-friendly command-line interface for managing containers, making it easy to create, start, stop, and manage containers. Compatible with existing Linux distributions and tools: LXD is designed to work with existing Linux distributions and tools, and supports a wide range of container images and formats, including Docker. Advanced features: LXD provides advanced features, such as live migration, storage management, and network management, which allows you to move running containers between hosts without interruption, manage storage resources, and network interfaces within containers. Security: LXD uses AppArmor and Seccomp to provide additional security to the containers. Networking: LXD provides easy-to-use networking features to manage the container’s network interfaces, assign IP addresses, and create virtual networks. Scalability: LXD can run thousands of containers on a single host, making it highly scalable for large-scale deployments. High availability: LXD supports clustering features with HAproxy, allowing the creation of a highly available environment with automatic failover. Differences Between LXD and Docker LXD and Docker are containerization technologies, but they have some key differences. The decision of choosing Docker desktop alternatives should be made based on the use case and business requirements. Use case: LXD is a container hypervisor, while Docker is a container runtime. This means that LXD provides an additional layer of abstraction, allowing multiple isolated Linux systems (containers) to run on a single host, while Docker focuses on running individual containers. Containerization: LXD provides a more complete, system-level containerization experience, while Docker is more focused on application-level containerization. Design: LXD is designed to work with existing Linux distributions and tools and supports a wide range of container images and formats, including Docker. Docker, on the other hand, is focused on its own container format and ecosystem. Security Integration: LXD uses AppArmor and Seccomp to provide additional security to the containers, while Docker uses namespaces and control groups to isolate containers. Networking: LXD provides easy-to-use Networking features to manage the container's network interfaces and assign IP addresses, and create virtual networks, while Docker uses virtual networks based on IP addresses and network interfaces provided by the host. Overall, while Docker and LXD are powerful containerization technologies, they are designed to solve different problems and have different use cases. Depending on the use case, these alternatives of Docker can be used. How To Choose the Best Docker Alternative When choosing a Docker alternative, it is important to consider the following factors: Compatibility: Make sure the alternative is compatible with your existing infrastructure and technologies. Features: Evaluate the features offered by the alternative and see if they align with your needs. Support: Consider the level of support offered by the alternative and its community. Performance: Consider the performance of the alternative in terms of resource usage and scalability. Security: Evaluate the security features offered by the alternative and see if they meet your requirements. Cost: Consider the cost of using the alternative and compare it to other options. Conclusion So these were some of the popular alternatives for Docker. Each of these Docker alternatives has its own strengths and weaknesses, so it's important to analyze the pros and cons of each and study your business requirements before choosing any of these alternatives of Docker.

SaaS applications are the new normal nowadays, and software providers are looking to transform their applications into a Software As a Service application. For this, the only solution is to build a multi-tenant architecture SaaS application. Have you ever wondered how Slack, Salesforce, AWS (Amazon Web Services), and Zendesk can serve multiple organizations? Does each one have its unique and custom cloud software per customer? For example, have you ever noticed that, on Slack, you have your own URL “yourcompanyname.slack.com?” Most people think that, in the background, they created a particular environment for each organization—application or codebase—and believe that Slack customers have their own server/app environment. If this is you, you might have assumed they have a repeatable process to run thousands of apps across all their customers. Well, no. The real solution is a multi-tenant architecture on AWS for a SaaS application. Let’s start with this impressive fact: 70% of all web apps are considered SaaS applications according to IDC Research. So, if you know about SaaS architecture and multi-tenant, you are probably covering 70% of the web app architecture landscape that would be available in the future. “70% of all web apps are SaaS, but only a few of them are multi-tenant.” This research is intended to showcase an overview of the strategies, challenges, and constraints that DevOps and software developers are likely to face when architecting a SaaS multi-tenant application.There are two concepts that are important for us to understand before starting: The next points are what we will explore in a multi-tenant architecture for your SaaS application, and my contributions will be: What Is Multi-Tenant Architecture? First of all, you need to understand what single tenant and multi-tenant architecture is: Single-tenant architecture (siloed model): is a single architecture per organization where the application has its own infrastructure, hardware, and software ecosystem. Let’s say you have ten organizations; in this case, you would need to create ten standalone environments, and your SaaS application, or company, will function as a single tenant architecture. Additionally, it implies more costs, maintenance, and a level of difficulty to update across the environments. Multi-tenant architecture: is an ecosystem or model where a single environment can serve multiple tenants utilizing a scalable, available, and resilient architecture. The underlying infrastructure is completely shared, logically isolated, and with fully centralized services. The multi-tenant architecture evolves according to the organization or subdomain (organization.saas.com) that is logged into the SaaS application; and is totally transparent to the end-user. Bear in mind that in this paper, we will discuss two multi-tenant architecture models, one for the application layer and one for the database layer. Multi-Tenant Benefits The adoption of a multi-tenant architecture approach will bring extensive valuable benefits for your SaaS application. Let’s go through the next contributions: A reduction of server infrastructure costs utilizing a multi-tenant architecture strategy: Instead of creating a SaaS environment per customer, you include one application environment for all your customers. This enables your AWS hosting costs to be dramatically reduced from hundreds of servers to a single one. One single source of trust: Let’s say again you have a customer using your SaaS. Imagine how many code repositories you would have per customer. At least 3-4 branches per customer, which would be a lot of overhead and misaligned code releases. Even worse, visualize the process of deploying your code to the entire farm of tenants; it is extremely complicated. This is unviable and time-consuming. With a multi-tenant SaaS architecture, you avoid this type of conflict, where you’ll have one codebase (source of trust), and a code repository with a few branches (dev/test/prod). By following the below practice—with a single command (one-click-deployment)—you will quickly perform the deployment process in a few seconds. Cost reductions of development and time-to-market: Cost reduction considers a sequence of decisions to make, such as having a single codebase, a SaaS platform environment, a multi-tenant database architecture, a centralized storage, APIs, and following The Twelve-Factor Methodology. All of them will allow you to reduce development labor costs, time-to-market, and operational efficiencies. SaaS Technology Stack for an Architecture on AWS To build a multi-tenant architecture, you need to integrate the correct AWS web stack, including OS, language, libraries, and services to AWS technologies. This is just the first step towards creating a next-generation multi-tenant architecture. Even though we will surface a few other multi-tenant architecture best practices, this article will be primarily oriented to this AWS SaaS web stack. Let’s dive into our SaaS Technology Stack on AWS: Programming Language It doesn’t really matter which language platform you select. What is vital is that your application can scale, utilize multi-tenant architecture best practices, cloud-native principles, and a well-known language by the open-source community. The latest trends to build SaaS applications are Python + React + AWS. Another “variant” is Node.js + React + AWS, but in the end, the common denominators are always AWS and React. If you are a financial company, ML or AI, with complex algorithms or backend work, I’ll say you should go for Python. On the other hand, if you are using modern technologies like real-time chats, mini feeds, streaming, etc. then go for Node.js. There is a market in the banking sector that is leveraging Java, but that’s for established enterprises. Any new SaaS application better goes with the mentioned web stack. Again, this is just what I’ve noticed as a trend, and what the community is demanding. Note: This data comes from a survey we performed a few months ago for financial services and SaaS companies. Ideal Languages Cloud Provider As a team of DevOps experts, I’ve noticed a cloud variation in the last two years, and which corresponds to these percentages: 70% of our DevOps implementations are based on AWS, 25% with Azure, and 5% go to GCP and digital ocean. Each year the trend is similar, with the exception that Azure is gradually growing with the years. Those are not only my words, but also ideas supported by multiple DevOps partners. So, I strongly recommend deploying your SaaS application under AWS. It has a number of benefits; every day there is a new service available for you, and a new feature that facilitates your development and deployment. Totally recommended to deploy your SaaS on AWS. Microservices If you are planning to leverage the cloud, you must leverage cloud-native principles. One of these principles is to incorporate microservices with Docker. Make sure your SaaS application is under microservices, which brings multiple benefits, including flexibility and standardization, easier to troubleshoot, problems isolation, and portability. Just like the cloud, Docker and microservices have transformed the IT ecosystem and will stay for a long while. Container Orchestration Platform This is a complicated and abstract decision; there are three options in AWS to manage, orchestrate, and create a microservice cluster environment: Amazon ECS: It is the natural Amazon container orchestration system in the AWS ecosystem. (Highly recommended for startups, small SaaS, and medium SaaS). Amazon Fargate: Almost serverless and price and management is per task. Minimal operational effort vs. ECS. There are some studies conducted by our DevOps team; in terms of performance. Fargate can be slower than ECS, so for this particular case, I would recommend Amazon ECS, instead of Fargate. Another thought is that if your team is pure developers and not planning to hire a DevOps engineer, perhaps Fargate is the way to go. Amazon EKS: It is a managed service that makes Kubernetes on AWS easy to manage. Use Amazon EKS instead of deploying a Kubernetes cluster on an EC2 instance, set up the Kubernetes networking, and worker nodes. (Recommended for large SaaS apps and a sophisticated DevOps and web development Team). Database The inherent database will be PostgreSQL with Amazon RDS. However, I strongly recommend that if you have a senior development team, and are projecting a high-traffic for your SaaS application—or even hundreds of tenants—you’d better architect your database with MongoDB. In addition to this, utilize the best practices that will be mentioned below about multi-tenant database. In this case, I would go for Amazon RDS with PostgreSQL or DynamoDB (MongoDB). “If you are projecting a high-traffic for your SaaS application, you’d better architect your database with MongoDB.” GraphQL or Amazon AppSync GraphQL is a query language and an alternative to a RESTful API for your database services. This new and modern ecosystem is adopted as a middleman among the client and the database server. It allows you to retrieve database data faster, mitigate the over-fetching in databases, retrieve the accurate data needed from the GraphQL schema, and maintaining the speed of development by iterating more quickly than a RESTful service. Adopting a monolithic backend application into a multi-tenant microservice architecture is the perfect time to leverage GraphQL or AppSync. Hence, when transforming your SaaS application, don’t forget to include GraphQL! Note: I didn’t include this service in the AWS SaaS architecture diagram, because it is implemented in multiple ways, and it would require an in-depth explanation on this topic. Automation You need a mechanism to trigger or launch new tenants/organizations and attach it to your multi-tenant SaaS architecture. Let’s say you have a new client that just subscribed to your SaaS application, how do you include this new organization inside your environment, database, and business logic? You need an automated process to launch new tenants; this is called Infrastructure as Code (IaC). This script/procedure should live within a git/bitbucket repository, one of the fundamental DevOps principles. A strong argument to leverage automation and IaC is that you need a mechanism to automate your SaaS application for your code deployments. In the same lines, automate the provisioning of new infrastructure for your Dev/Test environments. Infrastructure as Code and Automation Tools It doesn’t matter which Infrastructure as Code tool to use, they are both useful (Terraform and CloudFormation); they do the same job, and are highly known by the DevOps community. I don’t have a winner, they are both good. Terraform (from Hashicorp): A popular cloud-agnostic tool. Used widely for all DevOps communities. It is easier to find DevOps with this skill. Amazon CloudFormation: It is easier to integrate with Amazon Web Services, AWS built-in Automation tool. Whenever there is a new Amazon technology just released, the compatibility with AWS and CloudFormation is released sooner than Terraform. Trust on an AWS CloudFormation expert to automate and release in a secure manner. Message Queue System (MQS) The common MQS are Amazon SQS, RabbitMQ, or Celery. What I suggest here is to utilize the service that requires you less operation, in this case, is Amazon SQS. There are multiple times you need asynchronous communication. From delaying or scheduling a task, to increasing reliability and persistence with critical web transactions, decoupling your monolithic or micro-service application, and, most importantly: using a queue system to communicate event-driven serverless applications (Amazon Lambda functions). Caching System AWS ElastiCache is a caching and data storage system that is fully scalable, available, and managed. It aims to improve the application performance of distributed cache data and in-memory data structure stores. It’s an in-memory key-value store for Memcached and Redis engines. With a few clicks, you can run this AWS component entirely self-managed. It is essential to include a caching system for your SaaS application. Cloud Storage System Amazon S3 and Amazon CloudFront CDN for your static content. All static content, including images, media and HTML, will be hosted on Amazon S3—the cloud system with infinite storage and elasticity. In front of Amazon S3 we will include AWS CloudFront, integrating this pair of elements is vital, in order to cache the entire static content and reduce bandwidth costs. SaaS Web Stack: Multi-Tenant SaaS Architecture Example on AWS Types of Multi-Tenant SaaS Architectures One of the most important questions among the multi-tenant adoption would be which multi-tenant architecture suits better for your SaaS application on AWS. We will explore the two layers needed to enable your application to act as a real SaaS platform since it is paramount to decide which multi-tenant architecture you’ll incorporate in your SaaS platfrom, the application, and database layer. These two types of multi-tenant architectures are: The application layer multi-tenancy. The database layer multi-tenancy. The Application Layer Multi-Tenancy The application layer is an architectural design that enables hosting for tenants and is primarily delivered for Software as a Service applications (SaaS apps). In this first model, the application layer is commonly shared among multiple customers. Monolithic Architecture for SaaS If you haven’t seen this article before—or if you have already developed and architected your own SaaS application—I’m sure you have fallen into this approach. The monolithic components include EC2 instances in the web tier, app tier, and Amazon RDS with MySQL for your database. The monolithic architecture is not a bad approach, with the exception that you are wasting resources massively in the mentioned tiers. At least around 50% and 70% of your CPU/RAM usage is wasted due to the nature of the monolithic (cloud) architecture. Monolithic Architecture Diagram Microservices Architecture for SaaS With Containers and Amazon ECS Microservices are a recommended type of architecture since they provide a balance between modernization and maximum use of available cloud resources (EC2 instances and compute units). As well as it introduces a decomposed system with more granular services (microservices). We won’t touch on much about the microservice benefits since it’s widely expressed in the community. However, I’ll recommend you to utilize the formula of multi-tenant architecture + AWS Services + microservices + Amazon ECS as the container orchestrator; they can be the perfect match. Mainly, consider that Amazon ECS gives fewer efforts to configure your cluster and more NoOps for your DevOps team. “By 2022, 90% of all new apps will feature microservices architectures that improve the ability to design, debug, update, and leverage third-party code; 35% of all production apps will be cloud-native.” —Source: Forbes, 2019 With a talented team, the best multi-tenant SaaS architecture approach would be this use case scenario. Along the same lines, it covers the SaaS software and architecture’s main attributes, including agility, innovation, repeatability, reduced cycle time, cost efficiency, and manageability. The Perfect Match Multi-tenant architecture + AWS Services + microservices + Amazon ECS (as the container orchestrator). Microservices Architecture Diagram Kubernetes Architecture for SaaS With Amazon EKS You may be wondering: what about Kubernetes or Amazon EKS? Well, Kubernetes is another alternative of microservice architecture that adds an extra layer of complexity in the SaaS equation. However, you can overcome this complexity by leveraging Amazon EKS, Amazon Elastic Container Service for Kubernetes; the managed Kubernetes service from Amazon, which is a de facto service by the Kubernetes community. What highlights of this component from the rest of the architectures is that it provides the use of namespaces. This attribute aids to isolate every tenant and its own environment within the corresponding Kubernetes cluster. In this sense, you don’t have to create different clusters per each tenant (you could, but, to satisfy a different approach). By using ResourceQuota, you can limit the resources used per namespace and avoid creating noise to the other tenants. Another point to consider is that if you would like to isolate your namespaces, you need to include Kubernetes network policies because, by default, the networking is open and can communicate across namespaces and containers. Here is a comparison of Amazon ECS vs Kubernetes. If you have a SaaS enterprise, I’ll recommend better to control your microservice via Amazon EKS or Kubernetes since it allows you to have more granular changes. So, what would a Kubernetes multi-tenant architecture look like? Here is a simple Kubernetes multi-tenant architecture and siloed by its respective namespaces. Kubernetes Architecture Diagram A simple multi-tenant architecture with Kubernetes and siloed by Kubernetes namespaces. Serverless Architecture for SaaS on AWS The dream of any AWS architect is to create a multi-tenant SaaS architecture with a serverless approach. That’s a dream that can come true as a DevOps or SaaS architect, but it especially adds a fair amount of complexity as a tradeoff. Additionally, it requires a reasonable amount of collaboration time with your dev team, extensive changes of code application, and a transformative serverless mindset. Given that, in a few years, it will be the ultimate solution, and it all depends on the talent, capabilities, and use case. A Serverless SaaS architecture enables applications to obtain more agility, resilience, and fewer development efforts, a truly NoOps ecosystem. At a high-level, what are the new parts of this next-generation serverless SaaS architecture? Every call becomes an isolated tenant call, either going to a logical service (Lambda function) or going to the database data coming from the Amazon API Gateway as an entry point in the serverless SaaS application. Now that you have decoupled every logical service, the authentication and authorization module needs to be handled by a third-party service like Amazon Cognito, which will be the one in charge to identify the tenant, user, tier, IAM tenant role, and bring back an STS token with these aspects. Particularly, the API Gateway will route all the tenant functions to the correct Lambda functions matching the STS Token. Here is a diagram of a multi tenant architecture example for AWS SaaS applications that are using serverless. Serverless Architecture Diagram The Database Layer Multi-Tenancy The multi-tenancy concept comes with different architecture layers. We have already advocated the multi-tenancy application layer and its variants. Now, it is time to explore multi-tenancy in the database layer, which is another aspect to discover. Paradoxically, the easiest and cost-effective multi-tenant database architecture is the pure and real database multi-tenancy. The database layer is right the opposite of the previous model, the application layer. Over here, the DB layer is kept in common among tenants, and the application layer is isolated. As a next step, you need to evaluate what multi-tenant database architecture to pursue with tables, schemas, or siloed databases. When choosing your database architecture, there are multiple criterias to assess: Scalability: Number of tenants, storage per-tenant, workload. Tenant isolation Database costs: Per tenant costs. Development complexity: Changes in schemas, queries, etc. Operational complexity: Database clustering, update tenant data, database administration, and maintenance. Single Database: A Table Per Tenant A table per tenant single database refers to a pure database multi-tenancy and pooled model. This database architecture is the common and default solution by DevOps or software architects. It is very cost-effective when having a small startup or a few dozen organizations. It consists of leveraging a table per each organization within a database schema. There are specific trade-offs for this architecture, including the sacrifice of data isolation, noise among tenants, and performance degradation—meaning one tenant can overuse compute and ram resources from another. Lastly, every table name has its own tenantID, which is very straightforward to design and architect. In regard to data isolation and compliance, let’s say that one of your developers makes a mistake and brings the wrong tenant information to your customer. Imagine the data breach—please ensure never to expose information from more than one tenant. That’s why compliant SaaS applications, this architecture model is not recommended, however, is used widely because of its cost-effectiveness. Alternative Single-Tenant Database Architecture A shared table in a single schema in a single schema in a single database. Perfect for DynamoDB. (We didn’t cover this approach—FYI). Single Database: A Schema Per Tenant A schema per tenant single database, also known as the bridge model, is a multi-tenant database approach, which is still very cost-effective and more secure than the pure tenancy (DB pooled model), since you are with a single database, with the exception of the database schema isolation per tenant. If you are concerned about data partitioning, this solution is slightly better than the previous one (a table per tenant). Similarly, it is simple to manage across multiple schemas in your application code configuration. One important distinction to notice is that with more than 100 schemas or tenants within a database, it can provoke a lag in your database performance. Hence, it is recommended to split the database into two (add the second database as a replica). However, the best database tool for this approach is PostgreSQL, which supports multiple schemas without much complexity. Lastly, this strategy, of a schema per tenant, shares resources, compute, and storage across all its tenants. As a result, it provokes noisy tenants that utilize more resources than expected. Database Server Per Tenant Also call the siloed model, where you need a database instance per customer. Expensive, but the best for isolation and security compliance. This technique is significantly more costly than the rest of multi-tenant database architectures, but it complies with security regulations; the best for performance, scalability, and data isolation. This pattern uses one database server per tenant, meaning if the SaaS app has 100 tenants, therefore there will be 100 database servers, which is extremely costly. When PCI, HIPAA, or SOC2 is needed, it is vital to utilize a database siloed model, or at least find a workaround with the correct IAM roles and the best container orchestration—either Kubernetes or Amazon ECS namespaces—a VPC per tenant and encryption everywhere. Multi-Tenant Database Architecture Tools Amazon RDS with PostgreSQL (best option). DynamoDB (a great option for a single-tenant database with a single shared table). Amazon RDS with MySQL. GraphQL, as described previously, use it in front of any of these databases to increase speed on data retrieval, speed on development, and alternative to RESTful API, which helps relieve requests from the backed servers to the client. Application Code Changes Once you have selected your multi-tenant strategy in every layer, let’s start considering what is needed to change in the code level, in terms of code changes. If you have decided to adopt Django (from Python) for your SaaS application, you need a few tweak changes to align your current application with your multi-tenant architecture from the database and application layer. Fortunately, web application languages and frameworks are able to capture the URL or subdomain that is coming from the request. The ability to obtain this information (subdomain) at runtime is critical to handling dynamic subdomains for your multi-tenant architecture. We won’t cover in-depth what lines of codes we need to include in your Django application—or in any other framework—but, at least, I’ll let you know what items should be considered in this section. Python Django Multi-Tenancy in a Nutshell Add an app called tenant.py: a class for tenantAwareModel with multiple pool classes. How to identify tenants: you need to give each tenant a subdomain; to do so, you need to modify a few DNS changes, Nginx/Apache tweaks, and add a utility method (utils.py). Now, whenever you have a request, you can use this method to get the tenant. Determine how to extract the tenant utilizing the host header: (subdomain). Admin isolation Note: Previous code suggestions could change depending on the architecture. Wildcard DNS Subdomain: URL-Based SaaS Platform Basically, every organization must have its own subdomain, and they are quite useful for identifying organizations. Per tenant, it is a unique dedicated space, environment, and custom application (at least logically); for example, “org1.saas.com,” “org2.saas.com,” and so on. This URL structure will dynamically provision your SaaS multi-tenant application, and this DNS change will facilitate the identification, authentication, and authorization of every tenant. However, another workaround is called path-based per tenant, which is not recommended, for example, “app.saas.com/org1/…,“ “app.saas.com/org2\…,” and so on. So, the following is required in this particular section: A wildcard record should be in place in your DNS management records. This wildcard subdomain redirects all routes to your multi-tenant architecture (either to the load balancer, application server, or cluster end-point). Similarly, a CNAME record labeled (*) pointing to your “app.saas.com” or “saas.com/login.” An asterisk (*) means a wildcard to your app domain. As a final step, another (A) record pointing your “app.saas.com“ domain to your Amazon ECS cluster, ALB, or IP. DNS Records Entries “*.saas.com” CNAME “app.saas.com” “app.saas.com” A 1.2.3.4 OR “app.saas.com” A (alias) balancer.us-east-1.elb.amazonaws.co Note: An (A) Alias record is when you are utilizing an ALB/ELB (Load Balancer) from AWS. Web Server Setup With NGINX Configuration Let’s move down to your web server, specifically Nginx. In this stage, you will need to configure your Nginx.conf and server blocks (virtual hosts). Set up a wildcard vhost for your Nginx web server. Make sure it is an alias (ServerAlias) and a catch-all wildcard site. You don’t have to create a subdomain VirtualHost in Nginx per tenant; instead, you need to set up a single wildcard VirtualHost for all your tenants. Naturally, the wildcard pattern will match your subdomains and route accordingly to the correct and unique patch of your SaaS app document root. SSL Certificates Just don’t forget to deal with the certificates under your tenant subdomains. You would need to add them either in the Cloudfront CDN, load balancer, or in your web server. Note: This solution can be accomplished using the Apache webserver. Follow the 12-factor Methodology Framework Following the 12-factor methodology represents the pure DevOps and cloud-native principles, including immutable infrastructure, dev/test and prod parity with Docker, CI/CD principles, stateless SaaS application, and more. Multi-Tenant SaaS Architecture Best Practices How is your SaaS platform going to scale? The multi tenant SaaS architecture best practices are: Amazon AutoScaling, either with EC2 instances or microservices. Database replication with Amazon RDS, Amazon Aurora, or DynamoDB. Application load balancer. Including a CloudFront CDN for your static content. Amazon S3 for all your static/media content. Caching system including Redis/Memcached or its equivalent in the AWS cloud—Amazon ElastiCache. Multi-availability zone set up for redundancy and availability. Code Deployments With CI/CD Another crucial aspect to consider is how to deploy your code releases across tenants and your multiple environments (dev, test, and prod). You will need a Continuous Integration and Continuous Delivery (CI/CD) process to streamline your code releases across all environments and tenants. If you follow-up on my previous best practices, it won’t be difficult. Tools to embrace CI/CD CI/CD tools: Jenkins, CircleCi, or AWS Code pipelines (along with Codebuild and CodeDeploy). My advice: If you want a sophisticated DevOps team and a widely known tool, go for Jenkins; otherwise, go for CircleCI. If you want to keep leveraging AWS technologies exclusively, then go for AWS Code pipelines. But, if you’re looking for compliance, banks, or regulated environments, go for Gitlab. DevOps Automation: Automate Your New Tenant Creation Process How are you creating new tenants per each subscription? Identify the process of launching new tenants into your SaaS environment. You need to trigger a script to launch or attach the new multi-tenant environment to your existing multi-tenant architecture, meaning to automate the setup of new tenants. Consider that it can be after your customer gets registered in your onboarding page, or you need to trigger the script manually. Automation Tools Terraform (Recommended) Amazon CloudFormation (Trust on an AWS CloudFormation certified team) Ansible. Note: Ensure you utilize Infrastructure as Code principles in this aspect. Siloed Compute and Siloed Storage How will your architecture be isolated from other tenants? You just need to identify the next: every layer of the SaaS application needs to be isolated. The customer workflow is touching multiple layers, pages, backend, networking, front-end, storage, and more bits, so… How is your isolation strategy? Take in mind the next aspect: IAM Roles per function or microservices. Amazon S3 security policies. VPC isolation. Amazon ECS/Kubernetes namespace isolation. Database isolation (tenant per table/schema/silo database). Tenant Compute Capacity Have you considered how many SaaS tenants it can support per environment? Just think, you have 99 tenants, compute/database load is almost to the limits, do you have a ready environment to support the new tenants? What about the databases? You have a particular customer that wants an isolated tenant environment for its SaaS application. How would you support an extra tenant environment that is separated from the rest of the multi-tenant architecture? Would you do it? What are the implications? Just consider a scenario for this aspect. Tenant Clean-Up What are you doing with the tenants that are idle or not used anymore? Perhaps a clean-up process for any tenant that has been inactive for a prolonged period, or removes unused resources and tenants by hand, but you need a process or automation script. Final Thoughts Multi-tenant architecture and SaaS applications under AWS. What a topic that we just discovered! Now, you understand the whole multi-tenant SaaS architecture cycle from end-to-end, including server configuration, code, and what architecture pursues per every IT layer. As you can notice, there is no global solution for this ecosystem. There are multiple variants per each IT layer, either all fully multi-tenant, partially tenant, or just silo tenants. It falls more on what you need, budget, complexity, and the expertise of your DevOps team. I strongly recommend going for microservices (ECS/EKS), partially multi-tenant SaaS in the app, and database layer. As well, include cloud-native principles, and finally, adopt the multi-tenant architecture best practices and considerations described in this article. That being said, brainstorm your AWS SaaS architecture firstly by thinking on how to gain agility, cost-efficiency, IT labor costs, and leveraging a nearshore collaboration model, which adds another layer of cost-savings. In regard, automation with Terraform and CloudFormation is our best choice. Even better, most of our AWS/DevOps projects are following PCI, HIPAA, and SOC2 regulations. If you are a fintech, healthcare, or SaaS company, well, you know this type of requirement should be included in your processes.

In earlier days, it was easy to deduct and debug a problem in monolithic applications because there was only one service running in the back end and front end. Now, we are moving toward microservices architecture, where applications are divided into multiple independently deployable services. These services have their own goal and logic to serve. In this kind of application architecture, it becomes difficult to observe how one service depends on or affects other services. To make the system observable, some logs, metrics, or traces must be emitted from the code, and this data must be sent to an observability back end. This is where OpenTelemetry and Jaeger come into the picture. In this article, we will see how to monitor application trace data (Traces and Spans) with the help of OpenTelemetry and Jaeger. A trace is used to observe the requests as they propagate through the services in a distributed system. Spans are a basic unit of the trace; they represent a single event within the trace, and a trace can have one or multiple spans. A span consists of log messages, time-related data, and other attributes to provide information about the operation it tracks. We will use the distributed tracing method to observe requests moving across microservices, generating data about the request and making it available for analysis. The produced data will have a record of the flow of requests in our microservices, and it will help us understand our application's performance. OpenTelemetry Telemetry is the collection and transmission of data using agents and protocols from the source in observability. The telemetry data includes logs, metrics, and traces, which help us understand what is happening in our application. OpenTelemetry (also known as OTel) is an open source framework comprising a collection of tools, APIs, and SDKs. OpenTelemetry makes generating, instrumenting, collecting, and exporting telemetry data easy. The data collected from OpenTelemetry is vendor-agnostic and can be exported in many formats. OpenTelemetry is formed after merging two projects OpenCensus and OpenTracing. Instrumenting The process of adding observability code to your application is known as instrumentation. Instrumentation helps make our application observable, meaning the code must produce some metrics, traces, and logs. OpenTelemetry provides two ways to instrument our code: Manual instrumentation Auto instrumentation 1. Manual Instrumentation The user needs to add an OpenTelemetry code to the application. The manual instrumentation provides more options for customization in spans and traces. Languages supported for manual instrumentations are C++, .NET, Go, Java, Python, and so on. 2. Automatic Instrumentation This is the easiest way of instrumentation as it requires no code changes and no need to recompile the application. It uses an intelligent agent that gets attached to an application, reads its activity, and extracts the traces. Automatic instrumentation supports Java, NodeJS, Python, and so on. Difference Between Manual and Automatic Instrumentation Both manual and automatic instrumentation have advantages and disadvantages that you might consider while writing your code. A few of them are listed below: Manual Instrumentation Automatic Instrumentation Code changes are required. Code changes are not required. It supports maximum programming languages. Currently, .Net, Java, NodeJS, and Python are supported. It consumes a lot of time as code changes are required. Easy to implement as we do not need to touch the code. Provide more options for the customization of spans and traces. As you have more control over the telemetry data generated by your application. Fewer options for customization. Possibilities of error are high as manual changes are required. No error possibilities. As we don't have to touch our application code. To make the instrumentation process hassle-free, use automatic instrumentation, as it does not require any modification in the code and reduces the possibility of errors. Automatic instrumentation is done by an agent which reads your application's telemetry data, so no manual changes are required. For the scope of this post, we will see how you can use automatic instrumentation in a Kubernetes-based microservices environment. Jaeger Jaeger is a distributed tracing tool initially built by Uber and released as open-source in 2015. Jaeger is also a Cloud Native Computing Foundation graduate project and was influenced by Dapper and OpenZipkin. It is used for monitoring and troubleshooting microservices-based distributed systems. The Jaeger components which we have used for this blog are: Jaeger Collector Jaeger Query Jaeger UI / Console Storage Backend Jaeger Collector: The Jaeger distributed tracing system includes the Jaeger collector. It is in charge of gathering and keeping the information. After receiving spans, the collector adds them to a processing queue. Collectors need a persistent storage backend, hence Jaeger also provides a pluggable span storage mechanism. Jaeger Query: This is a service used to get traces out of storage. The web-based user interface for the Jaeger distributed tracing system is called Jaeger Query. It provides various features and tools to help you understand the performance and behavior of your distributed application and enables you to search, filter, and visualise the data gathered by Jaeger. Jaeger UI/Console: Jaeger UI lets you view and analyze traces generated by your application. Storage Back End: This is used to store the traces generated by an application for the long term. In this article, we are going to use Elasticsearch to store the traces. What Is the Need for Integrating OpenTelemetry With Jaeger? OpenTelemetry and Jaeger are the tools that help us in setting the observability in microservices-based distributed systems, but they are intended to address different issues. OpenTelemetry provides an instrumentation layer for the application, which helps us generate, collect and export the telemetry data for analysis. In contrast, Jaeger is used to store and visualize telemetry data. OpenTelemetry can only generate and collect the data. It does not have a UI for the visualization. So we need to integrate Jaeger with OpenTelemetry as it has a storage backend and a web UI for the visualization of the telemetry data. With the help of Jaeger UI, we can quickly troubleshoot microservices-based distributed systems. Note: OpenTelemetry can generate logs, metrics, and traces. Jaeger does not support logs and metrics. Now you have an idea about OpenTelemetry and Jaeger. Let's see how we can integrate them with each other to visualize the traces and spans generated by our application. Implementing OpenTelemetry Auto-Instrumentation We will integrate OpenTelemetry with Jaeger, where OpenTelemetry will act as an instrumentation layer for our application, and Jaeger will act as the back-end analysis tool to visualize the trace data. Jaeger will get the telemetry data from the OpenTelemetry agent. It will store the data in the storage backend, from where we will query the stored data and visualize it in the Jaeger UI. Prerequisites for this article are: The target Kubernetes cluster is up and running. You have access to run the kubectl command against the Kubernetes cluster to deploy resources. Cert manager is installed and running. You can install it from the website cert-manager.io if it is not installed. We assume that you have all the prerequisites and now you are ready for the installation. The files we have used for this post are available in this GitHub repo. Installation The installation part contains three steps: Elasticsearch installation Jaeger installation OpenTelemetry installation Elasticsearch By default, Jaeger uses in-memory storage to store spans, which is not a recommended approach for the production environment. There are various tools available to use as a storage back end in Jaeger; you can read about them in the official documentation of Jaeger span storage back end. In this article, we will use Elasticsearch as a storage back end. You can deploy Elasticsearch in your Kubernetes cluster using the Elasticsearch Helm chart. While deploying Elasticsearch, ensure you have enabled the password-based authentication and deploy that Elasticsearch in observability namespaces. Elasticsearch is deployed in our Kubernetes cluster, and you can see the output by running the following command. Shell $ kubectl get all -n observability NAME READY STATUS RESTARTS AGE pod/elasticsearch-0 1/1 Running 0 17m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/elasticsearch ClusterIP None <none> 9200/TCP,9300/TCP 17m NAME READY AGE statefulset.apps/elasticsearch 1/1 17m Jaeger Installation We are going to use Jaeger to visualize the trace data. Let's deploy the Jaeger Operator on our cluster. Before proceeding with the installation, we will deploy a ConfigMap in the observability namespace. In this ConfigMap, we will pass the username and password of the Elasticsearch which we have deployed in the previous step. Replace the credentials based on your setup. YAML kubectl -n observability apply -f - <<EOF apiVersion: v1 kind: ConfigMap metadata: name: jaeger-configuration labels: app: jaeger app.kubernetes.io/name: jaeger data: span-storage-type: elasticsearch collector: | es: server-urls: http://elasticsearch:9200 username: elastic password: changeme collector: zipkin: http-port: 9411 query: | es: server-urls: http://elasticsearch:9200 username: elastic password: changeme agent: | collector: host-port: "jaeger-collector:14267" EOF If you are going to deploy Jaeger in another namespace and you have changed the Jaeger collector service name, then you need to change the values of the host-port value under the agent collector. Jaeger Operator The Jaeger Operator is a Kubernetes operator for deploying and managing Jaeger, an open source, distributed tracing system. It works by automating the deployment, scaling, and management of Jaeger components on a Kubernetes cluster. The Jaeger Operator uses custom resources and custom controllers to extend the Kubernetes API with Jaeger-specific functionality. It manages the creation, update, and deletion of Jaeger components, such as the Jaeger collector, query, and agent components. When a Jaeger instance is created, the Jaeger Operator deploys the necessary components and sets up the required services and configurations. We are going to deploy the Jaeger Operator in the observability namespace. Use the below-mentioned command to deploy the operator. Shell $ kubectl create -f https://github.com/jaegertracing/jaeger-operator/releases/download/v1.38.0/jaeger-operator.yaml -n observability We are using the latest version of Jaeger, which is 1.38.0 at the time of writing this article. By default, the Jaeger script is provided for cluster-wide mode. Suppose you want to watch only a particular namespace. In that case, you need to change the ClusterRole to Role and ClusterBindingRole to RoleBinding in the operator manifest and set the WATCH_NAMESPACE env variable on the Jaeger Operator deployment. To verify whether Jaeger is deployed successfully or not, run the following command: Shell $ kubectl get all -n observability NAME READY STATUS RESTARTS AGE pod/elasticsearch-0 1/1 Running 0 17m pod/jaeger-operator-5597f99c79-hd9pw 2/2 Running 0 11m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/elasticsearch ClusterIP None <none> 9200/TCP,9300/TCP 17m service/jaeger-operator-metrics ClusterIP 172.20.220.212 <none> 8443/TCP 11m service/jaeger-operator-webhook-service ClusterIP 172.20.224.23 <none> 443/TCP 11m NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/jaeger-operator 1/1 1 1 11m NAME DESIRED CURRENT READY AGE replicaset.apps/jaeger-operator-5597f99c79 1 1 1 11m NAME READY AGE statefulset.apps/elasticsearch 1/1 17m As we can see in the above output, our Jaeger Operator is deployed successfully, and all of its pods are up and running; this means Jaeger Operator is ready to install the Jaeger instances (CRs). The Jaeger instance will contain Jaeger components (Query, Collector, Agent); later, we will use these components to query OpenTelemetry metrics. Jaeger Instance A Jaeger Instance is a deployment of the Jaeger distributed tracing system. It is used to collect and store trace data from microservices or distributed applications, and provide a UI to visualize and analyze the trace data. To deploy the Jaeger instance, use the following command. Shell $ kubectl apply -f https://raw.githubusercontent.com/infracloudio/Opentelemertrywithjaeger/master/jaeger-production-template.yaml To verify the status of the Jaeger instance, run the following command: Shell $ kubectl get all -n observability NAME READY STATUS RESTARTS AGE pod/elasticsearch-0 1/1 Running 0 17m pod/jaeger-agent-27fcp 1/1 Running 0 14s pod/jaeger-agent-6lvp2 1/1 Running 0 15s pod/jaeger-collector-69d7cd5df9-t6nz9 1/1 Running 0 19s pod/jaeger-operator-5597f99c79-hd9pw 2/2 Running 0 11m pod/jaeger-query-6c975459b6-8xlwc 1/1 Running 0 16s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/elasticsearch ClusterIP None <none> 9200/TCP,9300/TCP 17m service/jaeger-collector ClusterIP 172.20.24.132 <none> 14267/TCP,14268/TCP,9411/TCP,14250/TCP 19s service/jaeger-operator-metrics ClusterIP 172.20.220.212 <none> 8443/TCP 11m service/jaeger-operator-webhook-service ClusterIP 172.20.224.23 <none> 443/TCP 11m service/jaeger-query LoadBalancer 172.20.74.114 a567a8de8fd5149409c7edeb54bd39ef-365075103.us-west-2.elb.amazonaws.com 80:32406/TCP 16s service/zipkin ClusterIP 172.20.61.72 <none> 9411/TCP 18s NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE daemonset.apps/jaeger-agent 2 2 2 2 2 <none> 16s NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/jaeger-collector 1/1 1 1 21s deployment.apps/jaeger-operator 1/1 1 1 11m deployment.apps/jaeger-query 1/1 1 1 18s NAME DESIRED CURRENT READY AGE replicaset.apps/jaeger-collector-69d7cd5df9 1 1 1 21s replicaset.apps/jaeger-operator-5597f99c79 1 1 1 11m replicaset.apps/jaeger-query-6c975459b6 1 1 1 18s NAME READY AGE statefulset.apps/elasticsearch 1/1 17m As we can see in the above screenshot, our Jaeger instance is up and running. OpenTelemetry To install the OpenTelemetry, we need to install the OpenTelemetry Operator. The OpenTelemetry Operator uses custom resources and custom controllers to extend the Kubernetes API with OpenTelemetry-specific functionality, making it easier to deploy and manage the OpenTelemetry observability stack in a Kubernetes environment. The operator manages two things: Collectors: It offers a vendor-agnostic implementation of how to receive, process, and export telemetry data. Auto-instrumentation of the workload using OpenTelemetry instrumentation libraries. It does not require the end-user to modify the application source code. OpenTelemetry Operator To implement the auto-instrumentation, we need to deploy the OpenTelemetry operator on our Kubernetes cluster. To deploy the k8s operator for OpenTelemetry, follow the K8s operator documentation. You can verify the deployment of the OpenTelemetry operator by running the below-mentioned command: Shell $ kubectl get all -n opentelemetry-operator-system NAME READY STATUS RESTARTS AGE pod/opentelemetry-operator-controller-manager-7f479c786d-zzfd8 2/2 Running 0 30s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/opentelemetry-operator-controller-manager-metrics-service ClusterIP 172.20.70.244 <none> 8443/TCP 32s service/opentelemetry-operator-webhook-service ClusterIP 172.20.150.120 <none> 443/TCP 31s NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/opentelemetry-operator-controller-manager 1/1 1 1 31s NAME DESIRED CURRENT READY AGE replicaset.apps/opentelemetry-operator-controller-manager-7f479c786d 1 1 1 31s As we can see in the above output, the opentelemetry-operator-controller-manager deployment is running in the opentelemetry-operator-system namespace. OpenTelemetry Collector The OpenTelemetry facilitates the collection of telemetry data via the OpenTelemetry Collector. Collector offers a vendor-agnostic implementation on how to receive, process, and export the telemetry data. The collector is made up of the following components: Receivers: It manages how to get data into the collector. Processors: It manages the processing of data. Exporters: Responsible for sending the received data. We also need to export the telemetry data to the Jaeger instance. Use the following manifest to deploy the collector. YAML kubectl apply -f - <<EOF apiVersion: opentelemetry.io/v1alpha1 kind: OpenTelemetryCollector metadata: name: otel spec: config: | receivers: otlp: protocols: grpc: http: processors: exporters: logging: jaeger: endpoint: "jaeger-collector.observability.svc.cluster.local:14250" tls: insecure: true service: pipelines: traces: receivers: [otlp] processors: [] exporters: [logging, jaeger] EOF In the above code, the Jaeger endpoint is the address of the Jaeger service which is running inside the observability namespace. We need to deploy this manifest in the same namespace where our application is deployed, so that it can fetch the traces from the application and export them to Jaeger. To verify the deployment of the collector, run the following command. Shell $ kubectl get deploy otel-collector NAME READY UP-TO-DATE AVAILABLE AGE otel-collector 1/1 1 1 41s OpenTelemetry Auto-Instrumentation Injection The above-deployed operator can inject and configure the auto-instrumentation libraries of OpenTelemetry into an application's codebase as it runs. To enable the auto-instrumentation on our cluster, we need to configure an instrumentation resource with the configuration for the SDK and instrumentation. Use the below-given manifest to create the auto-instrumentation. YAML kubectl apply -f - <<EOF apiVersion: opentelemetry.io/v1alpha1 kind: Instrumentation metadata: name: my-instrumentation spec: exporter: endpoint: http://otel-collector:4317 propagators: - tracecontext - baggage - b3 sampler: type: parentbased_traceidratio argument: "0.25" EOF In the above manifest, we have used three things: exporter, propagator, and sampler. Exporter: Used to send data to OpenTelemetry collector at the specified endpoint. In our scenario, it is "http://otel-collector:4317". Propagators: Carry traces, context, and baggage data between distributed tracing systems. They have three propagation mechanisms: tracecontext: This refers to the W3C Trace Context specification, which defines a standard way to propagate trace context information between services. baggage: This refers to the OpenTelemetry baggage mechanism, which allows for the propagation of arbitrary key-value pairs along with the trace context information. b3: This refers to the B3 header format, which is a popular trace context propagation format used by the Zipkin tracing system. Sampler: Uses a "parent-based trace ID ratio" strategy with a sample rate of 0.25 (25%). This means that when tracing a request, if any of its parent requests has already been sampled (with a probability of 0.25), then this request will also be sampled, otherwise it will not be traced. To verify that our custom resource is created or not, we can use the below-mentioned command. Shell $ kubectl get otelinst NAME AGE ENDPOINT SAMPLER SAMPLER ARG my-instrumentation 6s http://otel-collector:4317 parentbased_traceidratio 0.25 This means our custom resource is created successfully. We are using the OpenTelemetry auto-instrumented method, so we don’t need to write instrumentation code in our application. All we need to do is, add an annotation in the pod of our application for auto-instrumentation. As we are going to demo a Java application, the annotation that we will have to use here is: Shell instrumentation.opentelemetry.io/inject-java: "true" Note: The annotation can be added to a namespace as well so that all pods within that namespace will get instrumentation, or by adding the annotation to individual PodSpec objects, available as part of Deployment, Statefulset, and other resources. Below is an example of how your manifest will look after adding the annotations. In the below example, we are using annotation for a Java application. YAML apiVersion: apps/v1 kind: Deployment metadata: name: demo-sagar spec: replicas: 1 selector: matchLabels: app: demo-sagar template: metadata: labels: app: demo-sagar annotations: instrumentation.opentelemetry.io/inject-java: "true" instrumentation.opentelemetry.io/container-names: "spring" spec: containers: - name: spring image: sagar27/petclinic-demo ports: - containerPort: 8080 We have added instrumentation “inject-java” and “container-name” under annotations. If you have multiple container pods, you can add them in the same “container-names” annotation, separated by a comma. For example, “container-name1,container-name-2,container-name-3” etc. After adding the annotations, deploy your application and access it on the browser. Here in our scenario, we are using port-forward to access the application. Shell $ kubectl port-forward service/demo-sagar 8080:8080 To generate traces, either you can navigate through all the pages of this website or you can use the following Bash script: Shell while true; do curl http://localhost:8080/ curl http://localhost:8080/owners/find curl http://localhost:8080/owners?lastName= curl http://localhost:8080/vets.html curl http://localhost:8080/oups curl http://localhost:8080/oups sleep 0.01 done The above-given script will make a curl request to all the pages of the website, and we will see the traces of the request on the Jaeger UI. We are making curl requests to https://localhost:8080 because we use the port-forwarding technique to access the application. You can make changes in the Bash script according to your scenario. Now let’s access the Jaeger UI, as our service jaeger-query uses service type LoadBalancer, we can access the Jaeger UI on the browser by using the load balancer domain/IP. Paste the load balancer domain/IP on the browser and you will see the Jaeger UI there. We have to select our app from the service list and it will show us the traces it generates. In the above screenshot, we have selected our app name “demo-sagar” under the services option and its traces are visible on Jaeger UI. We can further click on the traces to get more details about it. Summary In this article, we have gone through how you can easily instrument your application using the OpenTelemetry auto-instrumentation method. We also learned how this telemetric data could be exported to the Elasticsearch backend and visualized it using Jaeger. Integrating OpenTelemetry with Jaeger will help you in monitoring and troubleshooting. It also helps perform root cause analysis of any bug/issues in your microservice-based distributed systems, performance/latency optimization, service dependency analysis, and so on. We hope you found this post informative and engaging. References OpenTelemetry Jaeger Tracing

Liquibase is an open-source database-independent library for tracking, managing, and applying database changes. It allows developers to version control their database changes and easily roll them out in a controlled and predictable manner. Kubernetes, on the other hand, is an open-source container orchestration system that automates the deployment, scaling, and management of containerized applications. When deploying a containerized application on Kubernetes, it is common to use an init container to perform any necessary setup tasks before the main application container starts. This can include tasks such as installing dependencies, configuring environment variables, or running database migrations. In this article, we will show you how to set up an init container in Kubernetes to run Liquibase migrations on your database. Liquibase Setup To use Liquibase in an init container, we first need to create a Docker image that contains the Liquibase tool and any necessary dependencies, such as a JDBC driver for the database. The following Dockerfile can be used to create an image that includes Liquibase and the MySQL JDBC driver: Dockerfile FROM openjdk:8-jdk-alpine RUN apk add --update bash curl ENV LIQUIBASE_VERSION=4.1.1 RUN curl -L https://github.com/liquibase/liquibase/releases/download/v${LIQUIBASE_VERSION}/liquibase-${LIQUIBASE_VERSION}.tar.gz \ | tar -xz -C /opt \ && ln -s /opt/liquibase-${LIQUIBASE_VERSION}/liquibase /usr/local/bin/liquibase RUN curl -L https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.22/mysql-connector-java-8.0.22.jar \ -o /opt/mysql-connector-java-8.0.22.jar Before we begin, make sure that you have the following prerequisites: A Kubernetes cluster set up and running A database running in a separate container or on a separate host A Liquibase project set up for your application Here are the steps you need to follow: Create a new Kubernetes deployment for your application. In the deployment definition, specify the init container and the main application container. YAML apiVersion: apps/v1 kind: Deployment metadata: name: my-app spec: selector: matchLabels: app: my-app template: metadata: labels: app: my-app spec: initContainers: - name: liquibase image: liquibase/liquibase:latest env: - name: LIQUIBASE_URL value: jdbc:postgresql://my-database:5432/my-database - name: LIQUIBASE_USERNAME value: my-user - name: LIQUIBASE_PASSWORD value: my-password command: ["liquibase", "update"] volumeMounts: - name: liquibase-config mountPath: /liquibase/ - name: my-app image: my-app:latest ports: - containerPort: 8080 env: - name: DATABASE_URL value: jdbc:postgresql://my-database:5432/my-database - name: DATABASE_USERNAME value: my-user - name: DATABASE_PASSWORD value: my-password volumes: - name: liquibase-config configMap: name: liquibase-config Create a configMap to store the Liquibase configuration files. YAML apiVersion: v1 kind: ConfigMap metadata: name: liquibase-config data: liquibase.properties: | changeLogFile: /liquibase/changelog.xml driver: org.postgresql.Driver classpath: /liquibase/postgresql-42.2.18.jar Run the deployment on your Kubernetes cluster. Shell kubectl apply -f my-app-deployment.yaml Validation In Liquibase, you can use Liquibase changelog table to validate whether your DB migrations are successful. In Liquibase, a changelog is a collection of changesets that describes the changes that need to be made to the database. Each changeset is a single, atomic change to the database, such as adding a new table, modifying an existing column, or running a SQL script. When Liquibase runs, it keeps track of which changesets have been executed in the database by storing information in a special table called the DATABASECHANGELOG table. This table contains a record for each changeset that has been executed, including the change's unique ID, author, and execution date. By default, the DATABASECHANGELOG table is created in the same schema as the rest of the database, but its name and schema can be customized. Here is an example of the DATABASECHANGELOG table structure: SQL ID | VARCHAR(255) | NOT NULL AUTHOR | VARCHAR(255) | NOT NULL FILENAME | VARCHAR(255) | NOT NULL DATEEXECUTED | TIMESTAMP | NOT NULL ORDEREXECUTED | INT | NOT NULL EXECTYPE | VARCHAR(10) | NOT NULL MD5SUM | VARCHAR(35) | DESCRIPTION | VARCHAR(255) | COMMENTS | VARCHAR(255) | TAG | VARCHAR(255) | LIQUIBASE | VARCHAR(20) | You can query the DATABASECHANGELOG table to see which changesets have been executed, and in what order. Additionally, you can also use the Liquibase command-line tool to generate reports on the current state of the database, including a list of all executed changesets and any pending changes that have not yet been applied. Liquibase Rollback In Liquibase, it is possible to revert a specific changeset that has already been applied to the database. This can be useful in cases where a mistake was made in a previous change or if you want to undo a change in a development or testing environment. To revert a changeset in Liquibase, you can use the "rollback" command and specify the ID of the changeset that you want to revert. The rollback command will undo the changes made by the specified changeset and update the DATABASECHANGELOG table to reflect the change. Here is an example of how to revert a changeset with ID "12345" using the Liquibase command-line tool: Shell liquibase rollback 12345 In Kubernetes, to revert a changeset, you will have to create a new deployment with the rollback command and apply it to the cluster. Here is an example of how to revert a changeset with ID "12345" in Kubernetes: YAML apiVersion: apps/v1 kind: Deployment metadata: name: my-app-rollback spec: selector: matchLabels: app: my-app-rollback template: metadata: labels: app: my-app-rollback spec: initContainers: - name: liquibase-rollback image: liquibase/liquibase:latest env: - name: LIQUIBASE_URL value: jdbc:postgresql://my-database:5432/my-database - name: LIQUIBASE_USERNAME value: my-user - name: LIQUIBASE_PASSWORD value: my-password command: ["liquibase", "rollback", "12345"] volumeMounts: - name: liquibase-config mountPath: /liquibase/ volumes: - name: liquibase-config configMap: name: liquibase-config It's worth noting that depending on the changes made by the changeset, reverting it might not be straightforward and can have an impact on other parts of the database, for example, if the changeset creates a table, reverting it will drop the table and all the data inside it. it's important to test the rollback in a development environment before applying it to production. In conclusion, using an init container in Kubernetes to run Liquibase database migrations is a convenient and efficient way to manage and version database changes. It allows developers to track and roll back changes easily and ensures that the database is in the correct state before the main application starts.

Security is one of the key challenges in Kubernetes because of its configuration complexity and vulnerability. Managed container services like Google Kubernetes Engine (GKE) provide many protection features but don’t take all related responsibilities off your plate. Read on to learn more about GKE security and best practices to secure your cluster. Basic Overview of GKE Security GKE protects your workload in many layers, which include your container image, its runtime, the cluster network, and access to the cluster API server. That’s why Google recommends a layered approach to protecting your clusters and workloads. Enabling the right level of flexibility and security for your organization to deploy and maintain workloads may require different tradeoffs, as some settings may be too constraining. The most critical aspects of GKE security involve the following: Authentication and authorization; Control plane security, including components and configuration; Node security; Network security. These elements are also reflected in CIS Benchmarks, which help to structure work around security configurations for Kubernetes. Why Are CIS Benchmarks Crucial for GKE Security? Handling K8s security configuration isn’t exactly a walk in the park. Red Hat 2022 State of Kubernetes and Container Security found that almost one in four serious issues were vulnerabilities that could be remediated. Nearly 70% of incidents happened due to misconfigurations. Since its release by the Center of Internet Security (CIS), Benchmarks have become globally recognized best practices for implementing and managing cybersecurity mechanisms. The CIS Kubernetes Benchmark involves recommendations for K8s configuration that support a strong security posture. Written for the open-source Kubernetes distribution, it intends to be as universally applicable as possible. CIS GKE Benchmarking in Practice With a managed service like GKE, not all items on the CIS Benchmark are under your control. That’s why there are recommendations that you cannot audit or modify directly on your own. These involve: The control plane; The Kubernetes distribution; The nodes’ operating system. However, you still have to take care of upgrading the nodes that run your workloads and, of course, the workloads themselves. You need to audit and remediate any recommendations to these components. You could do it manually or use a tool that handles CIS benchmarking. With CAST AI’s container security module, for example, you can get an overview of benchmark discrepancies within minutes of connecting your cluster. The platform also prioritizes the issues it identifies, so you know which items require remediation first. When scanning your cluster, you also check it against the industry’s best practices, so you can better assess your overall security posture and plan further GKE hardening. Top 10 strategies to Ensure GKE Security 1. Apply the Principle of Least Privilege This basic security tenet refers to granting a user account only the privileges that are essential to perform the intended function. It comes in CIS GKE Benchmark 6.2.1: Prefer not running GKE clusters using the Compute Engine default service account. By default, your nodes get access to the Compute Engine service account. Its broad access makes it useful to multiple applications, but it also has more permissions than necessary to run your GKE cluster. That’s why you must create and use a minimally privileged service account instead of the default one – and follow suit in other contexts, too. 2. Use RBAC to Strengthen Authentication and Authorization GKE supports multiple options for managing access to your clusters with role-based access control (RBAC). RBAC enables more granular access to Kubernetes resources at cluster and namespace levels, but it also lets you create detailed permission policies. CIS GKE Benchmark 6.8.4 underscores the need to give preference to RBAC over the legacy Attribute Based Access Control (ABAC). Another CIS GKE Benchmark (6.8.3) recommends using groups to manage users as it simplifies controlling identities and permissions. It also removes the need to update the RBAC configuration whenever users are added or removed from the group. 3. Enhance Your Control Plane’s Security Under the Shared Responsibility Model, Google manages the GKE control plane components for you. However, you remain responsible for securing your nodes, containers, and pods. By default, the Kubernetes API server uses a public IP address. You can protect it by using authorized networks and private clusters, which enable you to assign a private IP address. You can also enhance your control plane’s security by doing a regular credential rotation. When you initiate the process, the TLS certificates and cluster certificate authority are rotated automatically. 4. Upgrade Your GKE Infrastructure Regularly Kubernetes frequently releases new security features and patches, so keeping your K8s up-to-date is one of the simplest ways to improve your security posture. GKE patches and upgrades the control planes for you automatically. Node auto-upgrade also automatically upgrades nodes in your cluster. CIS GKE Benchmark 6.5.3 recommends keeping that setting on. If for any reason, you need to disable the auto-upgrade, Google advises performing upgrades monthly and following the GKE security bulletins for critical patches. 5. Protect Node Metadata CIS GKE Benchmarks 6.4.1 and 6.4.2 refer to two critical factors compromising your node security, which is still your responsibility. The v0.1 and v1beta1 Compute Engine metadata server endpoints were deprecated and shut down in 2020 as they didn’t enforce metadata query headers. Some attacks against Kubernetes rely on access to the VM’s metadata server to extract credentials. You can prevent those attacks with Workload identity or Metadata Concealment. 6. Disable the Kubernetes Dashboard Some years back, the world was electrified by the news of attackers gaining access to Tesla’s cloud resources and using them to mine cryptocurrency. The vector of attack, in that case, was a Kubernetes dashboard, which was exposed to the public with no authentication or elevated privileges. Complying with CIS GKE Benchmark 6.10.1 is recommended if you want to avoid following Tesla’s plight. This standard clearly outlines that you should disable Kubernetes web UI when running on GKE. By default, GKE 1.10 and later disable the K8s dashboard. You can also use the following code: gcloud container clusters update CLUSTER_NAME \ --update-addons=KubernetesDashboard=DISABLED 7. Follow the NSA-CISA Framework CIS Kubernetes Benchmark gives you a strong foundation for building a secure operating environment. But if you want to go further, make space for NSA-CISA Kubernetes Hardening Guidance in your security procedures. The NSA-CISA report outlines vulnerabilities within a Kubernetes ecosystem and recommends best practices for configuring your cluster for security. It presents recommendations on vulnerability scanning, identifying misconfigurations, log auditing, and authentication, helping you to ensure that you appropriately address common security challenges. 8. Improve Your Network Security Most workloads running in GKE need to communicate with other services running inside and outside the cluster. However, you can control the traffic allowed to flow through your clusters. First, you can use network policies to limit pod-to-pod communication. By default, all cluster pods can be reached over the network via their pod IP address. You can lock down the connection in a namespace by defining traffic flowing through your pods and stopping it for those that don’t match the configured labels. Second, you can balance your Kubernetes pods with a network load balancer. To do so, you create a LoadBalancer service matching your pod’s labels. You will have an external-facing IP mapping to ports on your Kubernetes Pods, and you’ll be able to filter authorized traffic at the node level with kube-proxy. 9. Secure Pod Access to Google Cloud Resources Your containers and pods might need access to other resources in Google Cloud. There are three ways to do this: with Workload Identity, Node Service Account, and Service Account JSON Key. The simplest and most secure option to access Google Cloud resources is by using Workload Identity. This method allows your pods running on GKE to get permissions on the Google Cloud service account. You should use application-specific Google Cloud service accounts to provide credentials so that applications have the minimal necessary permissions that you can revoke in case of a compromise. 10. Get a GKE-Configured Secret Manager CIS GKE Benchmark 6.3.1. recommends encrypting Kubernetes Secrets using keys managed in Cloud KMS. Google Kubernetes Engine gives you several options for secret management. You can use Kubernetes secrets natively in GKE, but you can also protect these at an application layer with a key you manage and application-layer secret encryption. There are also secrets managers like Hashicorp Vault, which provide a consistent, production-ready way to manage secrets in GKE. Make sure you check your options out and pick an optimal solution. Assess GKE Security Within Minutes The Kubernetes ecosystem keeps growing, but so are its security configuration challenges. If you want to stay on top of GKE container security, you need to be able to identify potential threats and track them efficiently. Kubernetes security reports let you scan your GKE cluster against CIS benchmark, NSA-CISA framework, and other container security best practices to identify vulnerabilities, spot misconfigurations, and prioritize them. It only takes a few minutes to get a complete overview of your cluster’s security posture.