Spring Cloud Kubernetes For Hybrid Microservices Architecture
Spring Cloud Kubernetes For Hybrid Microservices Architecture
The only problem with starting applications outside Kubernetes is that there is no auto-configured registration mechanism — find out how to fix it!
Join the DZone community and get the full member experience.Join For Free
You might use Spring Cloud Kubernetes to build applications running both inside and outside the Kubernetes cluster. The only problem with starting applications outside Kubernetes is that there is no auto-configured registration mechanism.
Spring Cloud Kubernetes delegates registration to the platform, what is an obvious behavior if you are deploying your application internally using Kubernetes objects. With the external application, the situation is different. You should guarantee registration by yourself on the application side.
You may also like: Quick Guide to Microservices With Kubernetes, Spring Boot 2.0, and Docker
This article is an explanation of motivation to add auto-registration mechanisms to Spring Cloud Kubernetes project only for external applications. Let's consider the architecture where some microservices are running outside the Kubernetes cluster and some others are running inside it. There can be many explanations for such a situation. The most obvious explanation seems to be a migration of your microservices from older infrastructure to Kubernetes.
Assuming it is still in progress, you have some microservices already moved to the cluster, while some others still running on the older infrastructure. Moreover, you can decide to start some kind of experimental cluster with only a few of your applications, until you have more experience with using Kubernetes on production. I think it is not a very rare case.
Of course, there are different approaches to that issue. For example, you may maintain two independent microservices-based architectures, with different discovery registry and configuration sources. But you can also connect external microservices through Kubernetes API with the cluster to load configuration from
Secret, and register them there allow inter-service communication with Spring Cloud Kubernetes Ribbon.
The sample application source code is available on GitHub under branch hybrid in the sample-spring-microservices-Kubernetes repository: https://github.com/piomin/sample-spring-microservices-kubernetes/tree/hybrid.
We move one of the sample microservices employee-service, described in the mentioned article, outside the Kubernetes cluster. Now, the applications which are communicating with employee-service need to use the addresses outside the cluster.
Also, they should be able to handle a port number dynamically generated on the application during startup (
server.port=0). Our applications are still distributed across different namespaces, so it is important to enabling the multi-namespaces discovery features — also described in my previous article. The application employee-service is connecting to MongoDB, which is still deployed on Kubernetes. In that case, the integration is performed via Kubernetes Service. The following picture illustrates our current architecture.
The situation with distributed configuration is clear. We don't have to implement any additional code to be able to use it externally. Just, before starting client application we have to set the environment variable
KUBERNETES_NAMESPACE. Since we set it to external we first need to create such a namespace.
Then we may apply some property sources to that namespace. The configuration is consisting of Kubernetes
Secret. We store there Mongo location, credentials, and some other properties. Here's our
The port number is taken from
Service, which is deployed as
Then, we are creating resources inside
bootstrap.yml file we need to set the address of Kubernetes API server and property responsible for trusting server's cert. We should also enable using Secret as a property source, which is disabled by default for Spring Cloud Kubernetes Config.
External Registration Implementation
The situation with service discovery is much more complicated. Since Spring Cloud Kubernetes delegates discovery to the platform, what is perfectly right for internal applications, the lack of auto-configured registration is a problem for external applications. That's why I decided to implement a module for Spring Cloud Kubernetes auto-configured registration for external application.
Currently, it is available inside our sample repository as the spring-cloud-Kubernetes-discovery-ext module. It is implemented according to the Spring Cloud Discovery registration pattern. Let's begin with dependencies. We just need to include spring-cloud-starter-Kubernetes, which contains core and discovery modules.
Here's our registration object. It implements
Registration interface from Spring Cloud Commons, which defines some basic getters. We should provide the hostname, port, serviceId, etc.
We have some additional configuration properties in comparison to Spring Cloud Kubernetes Discovery. They are available under the same prefix
There is also a class that should extend abstract
AbstractAutoServiceRegistration. It is responsible for managing the registration process. First, it enables the registration mechanism only if the application is running outside Kubernetes.
PodUtils bean defined in Spring Cloud Kubernetes Core for that. It also implements a method for building registration objects. The port may be generated dynamically on startup. The rest of the process is performed inside the abstract subclass.
The process should be initialized just after the application startup. To catch startup event we prepare bean that implements
SmartApplicationListener interface. The listener method calls bean
KubernetesAutoServiceRegistration to prepare the registration object and start the process.
Here's the auto-configuration for all previously described beans.
Finally, we may proceed to the most important step — an integration with Kubernetes API. Spring Cloud Kubernetes uses Fabric Kubernetes Client for communication with master API. The
KubernetesClient bean is already auto-configured, so we may inject it.
deregister methods are implemented in class
KubernetesServiceRegistry that implements
ServiceRegistry interface. Discovery in Kubernetes is configured via Endpoint API.
Endpoint contains a list of
EndpointSubset that stores a list of registered IPs inside
EndpointAddress and a list of listening ports inside
EndpointPort. Here's the implementation of register and deregister methods.
The auto-configuration beans are registered in
Now, we may include already created library to any Spring Cloud application running outside Kubernetes, for example to the employee-service. We are using it together with Spring Cloud Kubernetes.
The registration is still disabled since we won't set properly
Sometimes it might be used to set the static IP address in configuration, in case you would have multiple network interfaces.
192.168.99.1 as a static IP address, I'm able to easily perform some tests with Minikube node, which is running on VM available under 192.168.99.100.
Let's start employee-service locally. As you see on the screen below it has successfully load configuration from Kubernetes and connected with MongoDB running on the cluster.
After startup, the application has registered itself in Kubernetes.We can view details of
kubectl describe endpointscommand as shown below.
Finally, we can perform some test calls, for example via gateway-service running on Minikube.
Since Spring Cloud Kubernetes does not allow discovery across all namespaces for Ribbon clients, we should override Ribbon configuration using
DiscoveryClient as shown below.
There are some limitations related to discovery with Kubernetes. For example, there is not a build-in heartbeat mechanism, so we should take care of removing application endpoints on shutdown. Also, I'm not considering security aspects related to allowing discovery across all namespaces and allowing access to API for external applications.
I'm assuming you have guaranteed the required level of security when building your Kubernetes cluster, especially if you decide to allow external access to the API. API is still just API and we may use it. This article shows an example of a use case, which may be useful for you.
If you compare it with my previous article about Spring Cloud Kubernetes you see that with small configuration changes you can move application outside cluster without adding any new components for discovery or a distributed configuration.
Published at DZone with permission of Piotr Mińkowski , DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.