Demystifying Kuma Service Mesh

If we look carefully at the above outputs, we can see that the first time the request was served successfully, but the second time server returned an error with status code 423. This means that the rate limit is working as expected. This is how you can enable rate-limiting using Kuma.

Circuit Breaker

Instead of additional traffic, Kuma inspects existing live service traffic that is being exchanged between data plane proxies and marks unhealthy proxies based on detectors. Kuma makes use of detectors for deviation detection. Detectors include the conditions based on which Kuma decides when to open or close a circuit breaker.

In this policy, 5 types of detectors can all be enabled at a time. Once any one of them is triggered because of deviations, no additional traffic can reach that service avoiding cascade failure. The duration is decided based on the value of baseEjectionTime, which defaults to 30 seconds. The default circuit breaker policy is enabled by Kuma when you inject the sidecar.

You can view the existing circuit breakers using the following command:

$ kumactl get circuit-breakers -o json

For production deployments, it is advised to enable circuit breakers with detectors. We are going to enable a total error detector. Total errors help to identify the count of consecutive 5xx and locally generated errors. The default value is 5 for it.

Use the following yaml file to enable total errors and gateway errors detector with default values for the front-end service.

apiVersion: kuma.io/v1alpha1
kind: CircuitBreaker
mesh: default
metadata:
name: circuit-breaker-example
spec:
sources:
- match:
kuma.io/service: "*"
destinations:
- match:
kuma.io/service: front-end_sock-shop_svc_80
conf:
interval: 5s
baseEjectionTime: 30s
maxEjectionPercent: 20
splitExternalAndLocalErrors: false
thresholds:
maxConnections: 2
maxPendingRequests: 2
maxRequests: 2
maxRetries: 2
detectors:
totalErrors:
consecutive: 2

Once the detectors are enabled, any deviation will trigger the detectors. We have set the value for the maxConnections as 2, so if more than 2 parallel connection requests hit the frontend service, it will trip the circuit breaker. Also, for the total errors, we have set the consecutive as 2. So any 2 consecutive 5xx errors will trip the circuit breaker. Once the circuit breaker is tripped, it marks the front-end DPP as unhealthy and will eject it for 30 seconds based on the baseEjectionTime value. Thus the service will be unavailable. Since the circuit breaker policy does not send additional traffic to check DPP health, it will make use of the live traffic. In every 5s interval it will check for the policy again. This is how Kuma avoids cascade failure. CircuitBreaker resources list can help you understand the details of each field.

Apply the configuration:

$ kubectl apply -f circuit-breaker.yaml
circuitbreaker.kuma.io/circuit-breaker-example created

Circuit breaker policy is successfully enabled with detectors in Kuma. Let us see the circuit breaker in action by tripping it. Here we have tripped the circuit breaker by sending 10 connection requests at a time to the service. Since the Kuma Gateway provides full support for the policy, let us hit the gateway IP.

$ seq 1 10 | xargs -n1 -P10 curl -I "http://192.168.49.2:30526"
HTTP/1.1 200 OK
x-powered-by: Express
accept-ranges: bytes
cache-control: public, max-age=0
last-modified: Tue, 21 Mar 2017 11:31:47 GMT
etag: W/"21f0–15af0a320b8"
content-type: text/html; charset=UTF-8
content-length: 8688
date: Wed, 28 Jun 2023 11:02:07 GMT
x-envoy-upstream-service-time: 1
server: Kuma Gateway
HTTP/1.1 200 OK
x-powered-by: Express
accept-ranges: bytes
cache-control: public, max-age=0
last-modified: Tue, 21 Mar 2017 11:31:47 GMT
etag: W/"21f0–15af0a320b8"
content-type: text/html; charset=UTF-8
content-length: 8688
date: Wed, 28 Jun 2023 11:02:07 GMT
x-envoy-upstream-service-time: 1
server: Kuma Gateway
HTTP/1.1 503 Service Unavailable
x-envoy-overloaded: true
content-length: 81
content-type: text/plain
vary: Accept-Encoding
date: Wed, 28 Jun 2023 11:02:07 GMT
server: Kuma Gateway

If we look at the output above, we can see first two requests were served, and then we received 503 service-unavailable responses. This is how you can enable circuit breakers in Kuma and avoid cascade failure. The values of the errors can be adjusted as per your application’s requirements. You can refer to the policy field details of all the detectors and their default values and can set them accordingly.

Traffic Route

With the traffic route policy, you can enable L4 traffic routing using Kuma. This makes it easier to implement blue/green deployments and canary releases. This policy is enabled by default with a load balancer. The policy is applied to all the data planes in the mesh. You can view the policy manifest using the following command.

$ kumactl get traffic-routes -o json

In the default traffic route policy, Kuma uses a tag assigned to the service to apply the traffic route to all services. The routing is done in the round-robin way mentioned in the loadBalancer.

For L4 traffic split over two versions of the application, you can update the conf with weight and the destination for the matching source. Kuma also supports L7 traffic split, moderation, and rerouting. For more information on the fields to use, you can check the TrafficRoute resources list.

Fault Injection

Only setting up the guardrails is not enough unless we test the microservices against resilience. Fault injection allows us to mimic errors or faults in the infrastructure. This policy is available only for L7 HTTP traffic, so we need to ensure that the destination for which we need to inject fault should have http, http2, or grpc protocol support. You can check the rate-limit section to ensure HTTP protocol support.

Kuma provides 3 HTTP faults that can be imitated:

• Abort: When abort is injected, it returns a percentage of requests with only a pre-defined status code. So those are the two things you need to set.
  1. status code: HTTP status code that will be sent as a response to any request to the destination.
  2. percentage: of requests on which abort will be injected. Values are allowed from 0–100.
• ResponseBandwidth limit: With this fault injected, a limit is set to the speed of responding to the request for a percentage of requests. Here are the two things you need to set:
  1. limit: speed of responding in gbps, mbps, kbps, or bps.
  2. percentage: of requests on which responseBandwidth fault will be injected. Values are allowed from 0–100.
• Delay: With delay injected, the response will be delayed for the defined duration for a percentage of requests. Here are the two things you need to set.
  1. value: duration for which response will be delayed
  2. percentage: of requests on which delay will be injected. Values are allowed from 0–100.

Use the following yaml to enable fault injection policy with abort fault:

apiVersion: kuma.io/v1alpha1
kind: FaultInjection
mesh: default
metadata:
name: fi1
spec:
sources:
- match:
kuma.io/service: '*'
kuma.io/protocol: http
destinations:
- match:
kuma.io/service: front-end_sock-shop_svc_80
kuma.io/protocol: http
conf:
abort:
httpStatus: 500
percentage: 100

Here we have injected abort fault for all the requests by setting the percentage to 100 in case the request is sent to the front-end service from any source over HTTP protocol. The response status code will be 500.

Let us go and make a request to front-end service before enabling the fault injection policy to see what response we are getting.

$ kubectl -n sock-shop exec front-end-7d89d49d6b-8kpsp -it - sh
/usr/src/app $ wget http://front-end
Connecting to front-end (10.102.50.169:80)
wget: can't open 'index.html': Read-only file system

So it can connect successfully.

Let us now enable the policy.

$ kubectl apply -f fi.yaml
faultinjection.kuma.io/fi1 created

Verify with kumactl:

$ kumactl get fault-injections
MESH NAME AGE
default fi1 3m

Abort fault injection in action:

$ kubectl -n sock-shop exec front-end-7d89d49d6b-8kpsp -it - sh
/usr/src/app $ wget http://front-end
Connecting to front-end (10.102.50.169:80)
wget: server returned error: HTTP/1.1 500 Internal Server Error

We can see in the above output that the response is 500. It is the same as what we have set in the configuration.

This is how you can inject more errors in your microservice application and test it against resiliency.

These are just a few policies that we have enabled and shown from a beginner’s perspective. For the production-grade setup, you can configure more policies like fault injection, retry, timeout, etc., using Kuma. From version 2.0, Kuma has started beta support for `targetref` policies. For each source-destination policy, there is an equivalent `targetref` policy. The major reason to initiate the transition is the challenges with existing S&D, which makes it complex to implement.

With policies enabled, the application can be more robust without actually making any changes to the application codebase. Let us now understand how we can have monitored over a Kuma service mesh.

Observability Features of Kuma: Monitoring Service Mesh

Kuma provides a built-in observability stack that helps with monitoring the service mesh. It comes with a package that can be used to enable components like metrics, logs, and traces using kumactl. Use the following command.

$ kumactl install observability | kubectl apply -f -

On successful execution, Kuma creates a namespace mesh-observability

$ kubectl get ns | grep observability
mesh-observability Active 19s

The namespace spins up the following pods, which can be used for monitoring traffic.

$ kubectl get svc -n mesh-observability
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
grafana ClusterIP 10.100.220.39 <none> 80/TCP 9m
jaeger-agent ClusterIP None <none> 5775/UDP,6831/UDP,6832/UDP,5778/TCP 9m
jaeger-collector ClusterIP 10.104.6.217 <none> 14267/TCP,14268/TCP,9411/TCP,4317/TCP,4318/TCP 9m
jaeger-query ClusterIP 10.111.72.152 <none> 80/TCP 9m
loki ClusterIP 10.96.109.104 <none> 3100/TCP 9m
loki-headless ClusterIP None <none> 3100/TCP 9m
prometheus-kube-state-metrics ClusterIP None <none> 80/TCP,81/TCP 9m
prometheus-server ClusterIP 10.103.244.232 <none> 80/TCP 9m
zipkin ClusterIP None <none> 9411/TCP 9m

Kuma provides integration with Prometheus, Grafana, Loki, and Jaeger. In this blog, we will be talking about traffic metrics and traffic trace. You can read more about the traffic logs policy supported by Kuma.

Integration With Prometheus and Grafana

Kuma provides full integration with Prometheus by exposing metrics in Prometheus format and an API to monitor every proxy in the mesh and the application. The collected metrics can be queried and displayed using Grafana. Kuma does a basic installation and setup of Grafana with the `observability` package that we installed. We need to expose the metrics to enable telemetry.

Use the following yaml to expose metrics from DPP to enable the collection of metrics for all our service mesh traffic:

apiVersion: kuma.io/v1alpha1
kind: Mesh
metadata:
name: default
spec:
metrics:
enabledBackend: prometheus-1
backends:
- name: prometheus-1
type: prometheus

Apply the configuration:

$ kubectl apply -f prom.yaml
mesh.kuma.io/default configured

Verify Prometheus dashboard using prometheus-server ClusterIP. Click on Status and select targets to view the endpoints. In the kuma-dataplanes, you can view the status of each data plane.

Verify the Grafana dashboard using grafana ClusterIP. Enter the username as admin and password as admin. Configure the Data Sources with Prometheus type. Click on Save and test. Grafana identifies the data source and fetches the metrics. Kuma provides data sources and dashboards as Grafana extensions. We have talked about some of the extensions below:

• Kuma Dataplane: helps in investigating a single data plane in the mesh. Select Kuma Dataplane in the Dashboards, and in the dropdown, select the dataplane you want to view. We have selected the frontend data plane.
• Kuma CP: This helps to view control plane statistics like the actual status of the service mesh, performance of the service mesh, etc. Select Kuma CP in the Dashboards to view.

In the above graph, we can see the status of the control plane and data plane communication in terms of aggregated discovery services, health discovery services, etc.

Similarly, you can view the aggregated statistics like the number of requests and error rates using Kuma Mesh and aggregated statistics of Kuma Service to Service.

Integration With Jaeger

Kuma also installs and does Jaeger’s basic setup with the observability package. Using Grafana we can visualize the traces from Jaeger. In Grafana, add Data Sources with Jaeger type and get started with creating dashboards.

Here we have selected the Jaeger data source and plotted a graph to visualize traces of the response time of each /api/service.

Similarly, we can also enable the metrics collection for the application and monitor the application traffic. With this, Kuma makes it easier to enable observability and monitor the service mesh. We agree that this deployment and setup are not production ready, but it helps in getting started. We can build on top easily once the basic setup is ready to use. Let us understand some more such benefits of Kuma over other service mesh.

Benefits of Kuma

In this blog, we have seen that with the help of easy-to-understand Kuma documentation, we have been able to set up Kuma, deploy an application, enforce policies that strengthen the network, and enable observability. Service mesh does not have to be complex since scalability is directly proportional to operational cost. Too many moving parts mean a lack of clarity on how high the operational cost could be. Kuma ships most of the features via policies, provides a built-in stack for observability, and much more; it becomes easy to implement these features since it does not have too many moving parts.

For organizations that are still transitioning from bare metals to Kubernetes, the infrastructure could be a hybrid of both. Now with Kuma, the service mesh could still be implemented for the complete infrastructure. Also, for the different teams, multiple individual meshes can be configured with a single control plane. Not all these infrastructure components need to be geographically close. With Kuma’s multi-zone support, you can configure service mesh on multiple clouds and regions. The network can be improved by implementing easy-to-setup policies. With other service mesh, it is not so easy to achieve all this using a single solution.

Conclusion

Service mesh helps in strengthening the infrastructure without making any changes to the application. This also helps an organization shape better in terms of developer and platform engineer responsibilities. With the service mesh, there is no denying that resource utilization increased, but there is such service mesh that is easy to deploy, optimize and manage.

Kuma’s easy to setup installer and kumactl CLI tool helps to deploy service mesh with policies like a circuit breaker, rate limit, etc., that help avoid issues in the runtime and prevent downtimes. Kuma provides observability over service mesh traffic and the application traffic that helps in reduced time to resolve production issues.

We have seen in the blog post how easy it is to get started with Kuma using an e-commerce application. In the case of production-grade deployment, it could be challenging to monitor traffic and gain fine control of the application, which can be made easy with the right implementation of policies in Kuma.