Developing A Spring Boot Application for Kubernetes Cluster: A Tutorial [Part 4]
In the final part of this series, we take the final steps and learn how to deploy each of our constructed Spring Boot layers to Kubernetes.
Join the DZone community and get the full member experience.
Join For FreeThis is the final installment of this four part series. Check out part 1, part 2, and part 3 here.
Service Deployment Into Kubernetes
So far we have created a Kubernetes cluster in Amazon EC2 environment. We have also developed our sample application consisting of service and web layers and deployed its components into a private repository in Docker Hub. In the remainder of this article, we will discuss how to retrieve the application from Docker Hub and deploy it into the cluster. Throughout this section, all commands should be executed as root in Kubernetes master node.
Secret Key
The first step is to create a secret key to access Docker Hub. Execute
kubectl create secret docker-registry regcred --docker-server=https://index.docker.io/v1/ --docker-username=<username> --docker-password=<password> --docker-email=<email>
where username, password and email are the username, password and email associated with the Docker Hub private repository user. Note that value of --docker-server parameter is the repository server for Docker Hub. Now, you can execute
kubectl get secret regcred --output=yaml
to describe in YAML format the newly created key. The output will be similar to below.
apiVersion: v1
data:
.dockerconfigjson: eyJhdXRocyI6eyJodHRwczov...
metadata:
creationTimestamp: 2018-08-01T19:27:46Z
name: regcred
namespace: default
resourceVersion: "4648"
selfLink: /api/v1/namespaces/default/secrets/regcred
uid: f7e846b0-95c0-11e8-8ade-066280576724
type: kubernetes.io/dockerconfigjson
Node Labels
Referring to the output of kubectl get nodes executed previously, we consider worker nodes ip-172-31-16-16 and ip-172-31-42-220 as Web-1 and Web-2, and ip-172-31-33-22 and ip-172-31-35-232 as Service-1 and Service-2, respectively. We will create a label named servicetype and set its value to webservice in Web-1, Web-2 and zipcodeservice in Service-1, Service-2.
Execute those four commands in sequence:
kubectl label nodes ip-172-31-33-22 servicetype=zipcodeservice
kubectl label nodes ip-172-31-35-232 servicetype=zipcodeservice
kubectl label nodes ip-172-31-16-16 servicetype=webservice
kubectl label nodes ip-172-31-42-220 servicetype=webservice
To verify that labels have been assigned execute
kubectl get nodes --show-labels
In the response, you should see the newly assigned labels under LABELS column as shown below.
NAME STATUS ROLES AGE VERSION LABELS
ip-172-31-16-16 Ready <none> 23h v1.11.1 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=ip-172-31-16-16,servicetype=webservice
ip-172-31-22-14 Ready master 1d v1.11.1 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=ip-172-31-22-14,node-role.kubernetes.io/master=
ip-172-31-33-22 Ready <none> 23h v1.11.1 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=ip-172-31-33-22,servicetype=zipcodeservice
ip-172-31-35-232 Ready <none> 23h v1.11.1 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=ip-172-31-35-232,servicetype=zipcodeservice
ip-172-31-42-220 Ready <none> 23h v1.11.1 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=ip-172-31-42-220,servicetype=webservice
Deploying the Service Layer
Create a file named zip-service-deployment.yaml with the following content.
apiVersion: apps/v1
kind: Deployment
metadata:
name: zip-service-deployment
namespace: default
labels:
app: zip-service
spec:
replicas: 2
selector:
matchLabels:
app: zip-service
template:
metadata:
labels:
app: zip-service
spec:
containers:
- name: zip-service
image: konuratdocker/spark-examples:zipcode-service
command: ["java"]
args: ["-jar","/app.jar","8085"]
ports:
- containerPort: 8085
nodeSelector:
servicetype: zipcodeservice
imagePullSecrets:
- name: regcred
dnsConfig:
nameservers:
- 8.8.8.8
Highlights:
- Name of the service layer application is zip-service, value of app parameter.
- The "konuratdocker/spark-examples:zipcode-service", value of the image parameter, is constructed from the values of
repository
,tag
, andimageTag
elements in pom.xml for the service layer. - The value of name parameter under
imagePullSecrets
isregcred
, the secret key we had created to access the private repository in Docker Hub. - In Kubernetes environment, we would like to run the service application at port 8085 and therefore we override the port number 2223 in Dockerfile. (Of course, we could have kept the same port — we just wanted to illustrate how to override the value previously set in Dockerfile.)
- The value 8.8.8.8 is IP of the DNS server for our application to get the IP address associated with zipcodeapi.com.
- Observe the replicas: 2 and servicetype: zipcodeservice for nodeSelector. We create two replicas of the service and those will be deployed in the two nodes with label
zipcodeservice
.
Execute
kubectl create -f ./zip-service-deployment.yaml
You should see a response similar to this.
deployment.apps/zip-service-deployment created
After waiting a few minutes, execute
kubectl get pods -o wide
In response, you should see something like:
NAME READY STATUS RESTARTS AGE IP NODE
zip-service-deployment-6546457848-crwdp 1/1 Running 0 20s 10.244.3.2 ip-172-31-33-22
zip-service-deployment-6546457848-tx7tv 1/1 Running 0 20s 10.244.4.2 ip-172-31-35-232
In each of ip-172-31-33-22 (Service-1) and ip-172-31-35-232 (Service-2), a pod has been created in which an instance of the service layer application is running inside a container.
Let us inspect the application log. Pass name of pod to kubectl logs. For example,
kubectl logs zip-service-deployment-6546457848-crwdp
would display:
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.0.3.RELEASE)
1: INFO ZipcodeServer - No active profile set, falling back to default profiles: default
1: INFO Http11NioProtocol - Initializing ProtocolHandler ["http-nio-8085"]
1: INFO StandardService - Starting service [Tomcat]
1: INFO StandardEngine - Starting Servlet Engine: Apache Tomcat/8.5.31
1: INFO AprLifecycleListener - The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: [/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server:/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64:/usr/lib/jvm/java-1.8-openjdk/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib]
1: INFO [/] - Initializing Spring embedded WebApplicationContext
1: INFO ServiceController - ServiceController initiated
1: INFO ServiceConfiguration$$EnhancerBySpringCGLIB$$f5e038b8 - ServiceConfiguration initialized
1: INFO Http11NioProtocol - Starting ProtocolHandler ["http-nio-8085"]
1: INFO NioSelectorPool - Using a shared selector for servlet write/read
1: INFO ZipcodeServer - Started ZipcodeServer in 8.225 seconds (JVM running for 9.674)
1: INFO [/] - Initializing Spring FrameworkServlet 'dispatcherServlet'
Observe the line printed by Tomcat's HTTP NIO connector, ["http-nio-8085"], the port number coincides with the port specified in zip-service-deployment.yaml.)
The next step is to create a service to access the newly created deployment. Create file named zipcode-service.yaml:
apiVersion: v1
kind: Service
metadata:
name: zipcode-service
namespace: default
spec:
ports:
- port: 8085
targetPort: 8085
name: http
protocol: TCP
selector:
app: zip-service
type: ClusterIP
Highlights:
- Name of the service is zipcode-service, in default namespace.
- The service port number is 8085, which coincides with the port in zip-service-deployment.yaml file.
- Value of app under selector is zip-service, name of the service layer application in zip-service-deployment.yaml.
- The type of service is ClusterIP (https://kubernetes.io/docs/concepts/services-networking/service/), which will expose the service via a cluster-internal IP. The service IP address will be accessible only from inside the cluster. The IP could be resolved by the nodes inside the cluster using the kube-dns service we had discussed earlier.
Now execute
kubectl create -f ./zipcode-service.yaml
You should see a response like this:
service/zipcode-service created
The domain name of the newly created service will be zipcode-service.default.svc.cluster.local where zipcode-service.default is constructed from metadata section of zipcode-service.yaml. If you execute
kubectl get services -o wide
response should look like below:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 23h <none>
zipcode-service ClusterIP 10.97.71.233 <none> 8085/TCP 30s app=zip-service
The newly created service has been assigned a ClusterIP of 10.97.71.233. In addition, if we execute
dig zipcode-service.default.svc.cluster.local @10.96.0.10
the answer section in response will include
...
zipcode-service.default.svc.cluster.local. 5 INA 10.97.71.233
...
where IP corresponds to the ClusterIP and zipcode-service.default.svc.cluster.local is domain name of the service. Note that 10.96.0.10 is IP of the kube-dns service we had previously talked about.
We can test the service layer as follows. In master node, edit /etc/resolv.conf temporarily to insert 10.96.0.10 as the first line. Hence, the file should read:
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
10.96.0.10
...
Then, execute
curl http://zipcode-service.default.svc.cluster.local:8085/zipcodeservice/info/33301
The response will be as follows:
{"zip_code":"33301","lat":26.121317,"lng":-80.128146,"city":"Fort Lauderdale","state":"FL","timezone":{"timezone_identifier":"America\/New_York","timezone_abbr":"EDT","utc_offset_sec":-14400,"is_dst":"T"},"acceptable_city_names":[{"city":"Ft Lauderdale","state":"FL"}]}
Similarly,
curl http://zipcode-service.default.svc.cluster.local:8085/zipcodeservice/nearby/33301/5
should display:
{"zip_codes":[{"zip_code":"33004","distance":4.428,"city":"Dania","state":"FL"},{"zip_code":"33315","distance":2.821,"city":"Fort Lauderdale","state":"FL"},{"zip_code":"33312","distance":4.052,"city":"Fort Lauderdale","state":"FL"},...]}
High Availability
The service named zipcode-service is an access point to the service layer running in two distinct nodes. In EC2 console, stop one of the instances corresponding to those nodes, e.g. Service-1, as shown below.
With only Service-2 node running, if you execute
curl http://zipcode-service.default.svc.cluster.local:8085/zipcodeservice/info/33301
and
curl http://zipcode-service.default.svc.cluster.local:8085/zipcodeservice/nearby/33301/5
you would still get the same responses from the service as before. Then, start instance Service-1 and stop Service-2. Execute curl, you will continue getting the same response.
Deploying the Web Layer
Create a file named zip-web-deployment.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: zip-web-deployment
namespace: default
labels:
app: zip-web
spec:
replicas: 2
selector:
matchLabels:
app: zip-web
template:
metadata:
labels:
app: zip-web
spec:
containers:
- name: zip-web
image: konuratdocker/spark-examples:web-service
command: ["java"]
args: ["-jar","/app.jar","zipcode-service.default.svc.cluster.local:8085","3334"]
ports:
- containerPort: 3334
nodeSelector:
servicetype: webservice
imagePullSecrets:
- name: regcred
dnsConfig:
nameservers:
- 10.96.0.10
Highlights:
- Name of the web layer application is zip-web, value of app parameter.
- The "konuratdocker/spark-examples:web-service", the value of image parameter, is constructed from the values of
repository
,tag
andimageTag
elements in pom.xml for the web layer. - The value of name parameter under imagePullSecrets is regcred, the secret key we had created to access the private repository in Docker Hub.
- In Kubernetes environment, we would like to run the service application at port 3334 and therefore we override the port number in Dockerfile, 3333. The service layer will be accessed via domain name zipcode-service.default.svc.cluster.local at port 8085.
- The value 10.96.0.10 is IP of the DNS server for our application to get the IP address associated with the service. That IP corresponds to kube-dns service.
- Observe the replicas: 2 and servicetype: webservice for nodeSelector. We create two replicas of the service and those will be deployed in the two nodes with label webservice.
Now executing
kubectl create -f ./zip-web-deployment.yaml
would yield
deployment.apps/zip-web-deployment created
To verify that the web layer has been deployed, we execute
kubectl get pods -o wide
The response should read like this:
NAME READY STATUS RESTARTS AGE IP NODE
...
zip-web-deployment-85cd564498-9c86s 1/1 Running 0 2m 10.244.2.2 ip-172-31-42-220
zip-web-deployment-85cd564498-wzfqx 1/1 Running 0 2m 10.244.1.2 ip-172-31-16-16
In each of ip-172-31-16-16 (Web-1) and ip-172-31-42-220 (Web-2), a pod has been created in which an instance of the web layer application is running inside a container. Let us inspect the log file generated when the application inside one of those pods is started. For example, if we execute
kubectl logs zip-web-deployment-85cd564498-9c86s
the following will be displayed:
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.0.1.RELEASE)
1: INFO WebServer - No active profile set, falling back to default profiles: default
1: INFO Http11NioProtocol - Initializing ProtocolHandler ["http-nio-3334"]
1: INFO StandardService - Starting service [Tomcat]
1: INFO StandardEngine - Starting Servlet Engine: Apache Tomcat/8.5.29
1: INFO AprLifecycleListener - The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: [/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server:/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64:/usr/lib/jvm/java-1.8-openjdk/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib]
1: INFO [/] - Initializing Spring embedded WebApplicationContext
1: INFO WebController - WebController initiated
1: INFO Http11NioProtocol - Starting ProtocolHandler ["http-nio-3334"]
1: INFO NioSelectorPool - Using a shared selector for servlet write/read
1: INFO WebServer - Started WebServer in 8.574 seconds (JVM running for 10.092)
1: INFO [/] - Initializing Spring FrameworkServlet 'dispatcherServlet'
(Observe the line printed by Tomcat's HTTP NIO connector, ["http-nio-3334"], the port number specified in zip-web-deployment.yaml.)
Next, create a file named zip-web-service.yaml:
apiVersion: v1
kind: Service
metadata:
name: zip-web-service
namespace: default
spec:
ports:
- port: 3334
targetPort: 3334
nodePort: 30000
name: http
protocol: TCP
selector:
app: zip-web
type: NodePort
Highlights:
- Name of the service is zip-web-service, in default namespace.
- Port number 3334 coincides with the port in zip-web-deployment.yaml.
- Type of service is NodePort (https://kubernetes.io/docs/concepts/services-networking/service/), which will allow a node outside the cluster to access web service at port 30000 (nodePort) using individual IP addresses of the nodes where the service is deployed. The advantage of NodePort is that it allows access from outside the cluster. (With service type being NodePort, the service still gets a ClusterIP and a request to node-specific IP address to access the service from outside the cluster is routed to the particular ClusterIP.)
- Finally, value of app under selector is zip-web, name of deployment in zip-web-deployment.yaml. Execute
kubectl create -f ./zip-web-service.yaml
You should see a response similar to below.
service/zip-web-service created
At this point, we have created the two services needed. Execute "kubectl get services". You should see:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
...
zip-web-service NodePort 10.110.145.20 <none> 3334:30000/TCP 14s
zipcode-service ClusterIP 10.97.71.233 <none> 8085/TCP 25m
Let us test the web layer with internal IP addresses of the individual nodes for Web-1 and Web-2. Execute
kubectl get nodes -o wide
The response should look like this:
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
ip-172-31-16-16 Ready <none> 1h v1.11.1 172.31.16.16 <none> Ubuntu 16.04.4 LTS 4.4.0-1061-aws docker://17.3.2
ip-172-31-22-14 Ready master 1d v1.11.1 172.31.22.14 <none> Ubuntu 16.04.5 LTS 4.4.0-1061-aws docker://17.3.2
ip-172-31-33-22 Ready <none> 1h v1.11.1 172.31.33.22 <none> Ubuntu 16.04.4 LTS 4.4.0-1061-aws docker://17.3.2
ip-172-31-35-232 Ready <none> 1h v1.11.1 172.31.35.232 <none> Ubuntu 16.04.4 LTS 4.4.0-1061-aws docker://17.3.2
ip-172-31-42-220 Ready <none> 1h v1.11.1 172.31.42.220 <none> Ubuntu 16.04.4 LTS 4.4.0-1061-aws docker://17.3.2
Observe that the internal IPs for ip-172-31-16-16 (Web-1) and ip-172-31-42-220 (Web-2) are 172.31.16.16 and 172.31.42.220, respectively.
Execute
curl http://172.31.16.16:30000/zip/getZipcodeInfo/33301
In response, you should see:
<html><body><p>Zipcode Information:<p>zip: 33301, latitude: 26.121317, longitude: -80.128146, city: Fort Lauderdale, state: FL<p>Timezone: America/New_York (EDT)<p>Acceptable City Names:<p>Ft Lauderdale, FL</body></html>
Similarly,
curl http://172.31.16.16:30000/zip/getNearbyZipcodes/33301/5
should yield:
<html><body><p>Zip codes:<br><p>zip_code=33004, distance=4.428 miles, city=Dania, state=FL<p>zip_code=33315, distance=2.821 miles, city=Fort Lauderdale, state=FL...</body></html>
If you replace '172.31.16.16' with '172.31.42.220', you would get the same responses.
Connect to the outside node, i.e. the test node outside the cluster. Execute
curl http://172.31.16.16:30000/zip/getZipcodeInfo/33301
and
curl http://172.31.42.220:30000/zip/getZipcodeInfo/33301
Those should provide the same responses as we obtained in the master node. The same applies for
curl http://172.31.16.16:30000/zip/getNearbyZipcodes/33301/5
and
curl http://172.31.42.220:30000/getNearbyZipcodes/33301/5
Conclusions
In this tutorial, we discussed deploying a Spring Boot application into a Kubernetes cluster in Amazon EC2 environment. The Spring application consisted of weakly coupled web and service layers each running in their own Docker container. We described how to locally test web and service layers and then how to push them to a private repository in Docker Hub using com.spotify Maven plug-in as part of Maven build. (The files related to the application can be obtained from Git Hub.)
We provided detailed steps to create a single master Kubernetes cluster in Amazon EC2 environment using kubeadm. Then, we discussed how to deploy the individual web and service layers of the application into the cluster. We illustrated how to override during deployment the ENTRYPOINT instruction originally set in the Dockerfile, e.g. to change the listening port.
We deployed the web and service layers into separate nodes. For that purpose, we first assigned labels to individual nodes to give them either web or service layer designation. Of course, it is also possible to deploy each of the web and service layers into every node. However, typically service layer applications will need to access additional sources, e.g. databases and it may be wise to separately deploy web layer, directly accessed by end users, to avoid potential security risks. That is why we chose the segregated approach described in the tutorial.
Because service layer is accessed only within the cluster (by the web layer), its service type was set as ClusterIP, which ensures that a distinct IP address is assigned to the service, resolvable inside the cluster via a domain name. A request to the service is routed to one of the back-end nodes where service is running to ensure a level of high availability. The NodePort service type used for the web layer did not have the same type built-in high availability because the consumers have to access a web layer node using its node-specific IP address. In a real production deployment this limitation could be addressed by employing a load balancer in front of the web layer.
Here, a minimum set of required ports are listed while installing a Kubernetes cluster via kubeadm. Depending on the particular network plug-in and the hardware/network infrastructure additional ports may need to be opened. For simplicity, we allowed all TCP and UDP ports in every node. However, in a real application a more restrictive policy should be adhered to.
Flannel, as we used in this tutorial, does not provide network policy. In a real cluster environment you will probably need an additional network policy plug-in, such as Calico. Also see the discussion here.
The single master Kubernetes cluster discussed in this article has the obvious drawback of having only one master, i.e. a single point of failure. For creating highly available clusters with multiple masters see here.
Opinions expressed by DZone contributors are their own.
Comments