OSCON 2018: Distributed Apps and Composability

The 20th anniversary of the O'Reilly Open Source Convention (OSCON) brought a few surprises with it. The dominance of enterprise speakers and enterprise-driven open-source projects was one of the most surprising.

In 1999, enterprises were nowhere to be found at OSCON. But there were plenty of individuals talking about their open-source projects that were just getting off the ground. Twenty years later, open source has won over the enterprise and is now an integral component of enterprise business. As a result, the majority of open source projects being discussed in the sessions and on the floor were all enterprise-backed. And all of the open source projects focused specifically on solving enterprise problems like managing large-scale distributed applications.

As enterprises have migrated into the cloud, distributed applications have become more and more common despite the fact that managing them hasn’t gotten easier. Cue the ascendancy of Docker and Kubernetes. It was hard to find a session, talk to a vendor or just have a discussion that didn’t focus on or at least mention support for Docker and Kubernetes — they were literally everywhere at OSCON this year.

Even though Kubernetes is still in its infancy, the modern, structured way it approaches container orchestration is quickly becoming key to deploying and managing distributed applications. Unfortunately for newcomers, the steep learning curve associated with configuring Kubernetes can be intimidating. Enter ksonnet from Heptio, who worked with Box, Microsoft and Bitnami to replace the slog of editing “walls of YAML code” with a new mechanism that allows you to write Kubernetes configuration as code. ksonnet simplifies Kubernetes configuration and eases the learning curve, with its support for things like loops, variables and functions.

Building on top of Kubernetes, the Istio project from IBM, Google, and Lyft provides a common way to manage, monitor and secure traffic between container-based microservices. Istio layers on top of Kubernetes, allowing you to work directly with the services (rather than nodes) from which distributed applications are composed. Istio provides control and insight into key service-level capabilities like rate limits, access control, authentication, and so on.

Other ideas to help manage distributed applications included an architectural pattern called “sidecar” that can be applied to both Kubernetes deployments and Istio implementations. The sidecar pattern allows you to attach functionality like logging or security (e.g., TLS between nodes) to your distributed application without needing to write code or deploy new infrastructure resources (i.e., the sidecar will utilize the application’s existing resources). And when you kill the application, the sidecar goes with it.

It’s not just applications that are benefiting from services that apply to deployments as a whole rather than adding a discrete feature/function. The Linux Kit session from Docker focused on allowing you to compose OS-level services in order to build a completely functional Linux system. With Linux Kit, instead of issuing a traditional apt-get command you can just import a Perl or PHP or NGINX image, installing new functionality in an immutable container. Now, if you screw up your NGINX configuration, you can just kill it and reinstall it. Or you can just swap out an older version with a newer one. With apt-get, the resource becomes intertwined with Linux, so there’s no easy way to update or uninstall it.

The trends seem clear:

• Open source is becoming more and more enterprise led
• Distributed applications composed from container-based services have become the norm
• The Kubernetes bandwagon continues to gain momentum, dominating alternatives like Docker Swarm

Agree? Disagree? Let us know in the comments exactly where you might plot yourselves on this journey to the implementation of distributed services and composable applications.