Istio Ambient Mesh Performance Test and Benchmarking

Istio is the most popular service mesh, but the DevOps and SREs community constantly complain about its performance. Istio Ambient is a sidecar-less approach by the Istio committee (majorly driven by SOLO.io) to improve performance. Since there are many promotions about Ambient mesh being production-ready, many of our prospects and enterprises are generally eager to try or migrate to Ambient mesh. 

Architecturally, the Istio Ambient mesh is a great design that improves performance. But whether it performs quickly is still a question. We have tried Istio Ambient Mesh and observed the performance countless times between January 2024 and July 2024, and we have yet to see any significant performance gains. 

Below is the lab setup on which we ran our experiments.

Lab Setup to Load Test Istio Ambient Mesh

1. Load testing tool: Fortio
2. Application configuration: Bookinfo Application
3. Total requests fired: 1000 queries/second (QPS), 10 connections, and for 30 seconds
4. Cluster configuration: Azure (AKS) clusters with 3 nodes
5. Node configuration: 2 VCPU and 7GB memory for each node
6. CNI used: Kube CNI and Cilium (We did not use Flannel because it was not working well with AKS.)

Note:

1. We have kept all the applications and Fortio in different nodes.
2. We have exposed the Rating microservice and NOT Details service to handle external traffic. Because the Details microservice is written in Ruby, it is unfit for handling higher QPS. We sent a load of 100 QPS and 1000 QPS to the Details service without Istio, and the p99 latency for 100 QPS is around 6 ms, but it goes up to 50 ms for 1000 QPS. 

Performance Test on Istio Ambient Mesh With Kube CNI and Cilium

We have carried out the performance or load test for the following cases:

1. Kube CNI 
2. Kube CNI + Istio sidecar (mTLS enabled)
3. Kube CNI + Istio Ambient mesh (mTLS enabled)
4. Cilium CNI
5. Cilium CNI + Istio sidecar (mTLS enabled)
6. Cilium CNI + Istio Ambient mesh (mTLS enabled)

Although we have tested the load for each case multiple times, we have attached only one screenshot to showcase the standard deviation of P99 latency in each case. Please refer to the load test results in the next section. 

Load Test Results for Kube CNI Without Istio

• Observed (Median) P99 latency: 1.12ms

Figure 1: Kube CNI + Without Istio

Load Test of Kube CNI and Istio Sidecar (mTLS Enabled)

• Observed (Median) P99 latency: 4.72 ms

Figure 2: Kube CNI + With Istio Sidecar (mtLS enabled)

Load Test of Kube CNI and Istio Ambient Mesh (mTLS Enabled)

• Observed (Median) P99 latency: 3.6 ms

Figure 3: Kube CNI + With Istio Ambient (mtLS enabled)

Load Test of Cilium CNI Without Istio

• Observed (Median) P99 latency: 4.5 ms

Figure 4: Cilium CNI + Without Istio

Load Test of Cilium CNI and Istio Sidecar (mTLS Enabled)

• Observed (Median) P99 latency: 8.8 ms

Figure 5: Cilium CNI + With Istio Sidecar

Load Test of Cilium CNI and Istio Ambient Mesh (mTLS Enabled)

• Observed (Median) P99 latency: 6.8 ms

Figure 6: Cilium CNI + With Istio Ambient

Final Load Test Results and Benchmarking of Rating Service With and Without Istio

Here are the benchmarking results for the p99 latency of the Rating service with and without Istio (sidecar and Ambient mesh). 

Conclusion

Three items are concluded from this extensive load test of Istio Ambient Mesh:

1. The performance of Istio Ambient mesh will never give you thunderbolt improvements over latency when compared with plain Kube CNI. Note that using Ztunnel for encryption still involves network hops, which will increase the latency. Yes, it is better than Istio sidecar architecture. 
2. Regardless of the CNI used, the performance (p99 latency) of the Istio Ambient Mesh is 20% better than that of the Istio sidecar. 
3. Combining Cilium and Istio (sidecar or Ambient) produces undesirable results. If you are looking for performance improvements, you should avoid this mix.