Kubernetes Ecosystem: Key Tools and Tech From KubeCon 2017
Kubernetes Ecosystem: Key Tools and Tech From KubeCon 2017
Couldn't get to KubeCon 2017? Let's see what's new in container orchestration and what the future of microservices looks like.
Join the DZone community and get the full member experience.Join For Free
See why enterprise app developers love Cloud Foundry. Download the 2018 User Survey for a snapshot of Cloud Foundry users’ deployments and productivity.
Attending the KubeCon 2017 conference in Austin (Dec 6-8) helped me get a deeper insight into Kubernetes and understand that Kubernetes is something bigger than I thought. This article is meant to be quick read-through for any audience — from managers to developers — of the overall, growing Kubernetes ecosystem.
First I want to share few key details to clear any misinformation out there.
Is Google Using Kubernetes Internally?
I am impressed with the architecture of Kubernetes and the forethought of keeping future growth in mind. From the Kubernetes blog: “Many of the developers at Google working on Kubernetes were formerly developers on the Borg project. We've incorporated the best ideas from Borg in Kubernetes, and have tried to address some pain points that users identified with Borg over the years”… you can read more on this from: http://blog.kubernetes.io/2015/04/borg-predecessor-to-kubernetes.html
I also spoke to many speakers and people involved with Kubernetes, both inside and outside Google. It's loud and clear that Google uses Kubernetes internally on some projects, but you can't imagine changing some of existing critical products anytime soon. So rest assured that even if there is something better that shows in up future, Kubernetes can be easily customized — and the best part is that you can mix and match custom components with existing components as needed.
What Is a Pod?
When we directly work with abstraction provided by RedHat OpenShift, it might give the impression that a Pod is just a container for running a Docker image. A Pod is the smallest item that Kubernetes can control. It can run one or more containers. Please see this nice link for more information: https://kubernetes.io/docs/concepts/workloads/pods/pod/
What Is a Sidecar?
Sidecar is the term used for running an additional container in the same Pod that runs the primary container. Its uses are to support secondary functionality like logging, service proxies, or various service metrics.
What Is a Service Mesh?
The term service mesh is often used to describe the network of microservices that make up such applications and the interactions between them. We will focus more on this later when we cover a few key products. But if you are interested, you can visit this link to learn more about service meshes: https://istio.io/docs/concepts/what-is-istio/overview.html
OCI (Open Container Initiative) and CRI-O
Container technology is coming from different vendors, and not all containers are implemented the same way. But if all of them don’t follow some standard, it becomes difficult to use them the same way. Kubernetes has a nicely decoupled architecture that plays well with any container as long as it follows the OCI. Please look at the CRI-O implementation link for more information: https://github.com/kubernetes-incubator/cri-o. You can read more on the OCI at: http://www.itprotoday.com/cloud-data-center/using-containers-look-oci-seal-approval
What Is SPIFFE?
SPIFFE stands for Secure Production Identity Framework For Everyone. From what I heard and read, it is a standard way a service identity can be defined and shared across services globally — meaning across cloud providers. Read more on this at: https://spiffe.io/spiffe/
The Kubernetes Ecosystem: A Few Key Projects and Tools
Kubernetes has a wealth of open source and vendor-supporting tools and products. I was hoping to see a lot more visibility in terms of automated testing, but this year's KubeCon focus seemed to be traceability. You could definitely see that from vendor booths and the flood of products around logging, tracing, and metrics. I want to focus on some key open source projects and tools out there that you don't want to miss. Vendor tool coverage, however, is not something I am planning to do.
If you count the number of times you heard a specific word at KubeCon, then after Kubernetes, it was Istio. If you write a lot of microservice code, you are used to dealing with a lot of common scenarios to make our code robust: retrying when the connection is not available, handling network errors, providing metrics, etc. There is certainly a bit more overhead when we deal with code, rather than actually focusing on just what we need for microservices. Istio is the answer to simplify things in this area. See more on this at: https://istio.io/docs/concepts/what-is-istio/overview.html
From what I heard from many people at the conference, this is going to be a very useful product, so I wanted to mention this first in the list.
We all saw the underlying components of what makes a container and how it is faster than creating VMs. But what if there was an even better and faster way to create containers? That is the focus of Kata Containers.
Kata Containers combine technology from Intel Clear Containers and Hyper runV — the best of both worlds together. It was really nice attending the session on this. The part I liked best is that you can run existing Docker images in Kata Containers. And remember, Kubernetes works with any OCI-based containers — it does not matter if it is Docker or Kata.
Helm Charts: A Package Manager for Kubernetes
You might have used Docker Compose to deploy multiple Docker containers. With Kubernetes, as the number of services and pods grow, it becomes a bit more complex to directly manage configuration. Helm allows users to easily templatize their Kubernetes manifests and provide a set of configuration parameters that allow users to customize their deployment. We can use the repository of templates to create a configuration for similar deployments. More on this at: http://blog.kubernetes.io/2016/10/helm-charts-making-it-simple-to-package-and-deploy-apps-on-kubernetes.html
OpenTracing is about consistent, expressive, vendor-neutral APIs for popular platforms. OpenTracing makes it easy for developers to add (or switch) tracing implementations. Providing metrics and tracing is key to quickly troubleshoot a production issue or proactively monitor before you get into one. OpenTracing addresses this space. You can read more at: http://opentracing.io/documentation/. There is also a very good article on DZone on this.
I am sure most of you are already familiar with this tool. Prometheus is an open-source system monitoring and alerting toolkit. There are many projects and frameworks (like Tectonic) that use Prometheus. Read more on this at: https://prometheus.io/docs/introduction/overview/.
OPA: Open Policy Agent
In almost every project we work on, kick-off, authorization, and entitlement are the big story. Interestingly, almost every project I worked has its own custom implementation. I only wish I knew about this OPA implementation earlier and was really impressed with all the details, which you can find here: http://www.openpolicyagent.org/.
Netflix built a lightweight tool around this with unit tests inside so that we can avoid manual mistakes and keep the policies live and accurate with future changes. From what I understood, the UI is not something Netflix is ready to share as open source, so this leaves a really nice opportunity for any of you to build a UI and contribute to this open source project.
Metaparticle: An Amazing Keynote from Brendon
I was really impressed with Keynote from Bendon on Metaparticle: https://github.com/brendandburns/metaparticle. Brendan Burns is a software engineer at Microsoft Azure and co-founder of the Kubernetes project. That probably lets you know what you can expect from him. It was an amazing keynote that you must watch — and you can find the video here: http://events.linuxfoundation.org/events/kubecon-and-cloudnativecon-north-america/program/live-video-stream.
What Is Metaparticle, by the Way?
In my opinion, Microsoft is known to make everything simple for developers. At its core, think of Metaparticle as a declarative language plugged right into your microservice code. All you do and how your services will be deployed are taken care without any additional complexities, like dealing YAML files, etc. Those of you who have worked with WCF from Microsoft might see a slight similarity, but it is very interesting. More than that, the way Brendon delivered this keynote is something all developers must watch to see his dedication and insight in this area.
OpenShift and Techtonic Frameworks Based on Kubernetes
If you are currently using or planning to use Red Hat OpenShift or CoreOS Techtonic, it is best that you spend some time to get a good hold of Kubernetes. Although these frameworks are created so that you don’t need to know much about Kubernetes, it helps to have this insight. At least from my experience and what I have used so far, my preference is to stay as close to Kubernetes as possible. More on this and building CI/CD pipelines will be in my next article.
It is almost impossible to do justice in one article to cover a three-day conference with a lot of wonderful stuff. I strongly encourage all of you to attend next KubeCon (2018).
ELK vs EFK: Fluentd
I attended a few sessions that used fluentd and see the benefit of the EFK stack over the ELK stack. More on fluentd can be seen at: https://www.fluentd.org/
CSNF and CSNF projects: https://www.cncf.io/
Kubernetes Deconstructed: A must read: https://schd.ws/hosted_files/kccncna17/2e/Carson-Anderson_Kubernetes-Desconstructed.pdf
Opinions expressed by DZone contributors are their own.