DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Testing, Deployment, and Maintenance
  3. Deployment
  4. Fighting Service Latency in Microservices With Kubernetes

Fighting Service Latency in Microservices With Kubernetes

Sometimes when moving to a microservices-based architecture, latency creeps in. What can you do to fight it and keep it down? Look at some suggestions using Kubernetes.

Bilgin Ibryam user avatar by
Bilgin Ibryam
·
May. 19, 17 · Tutorial
Like (5)
Save
Tweet
Share
11.95K Views

Join the DZone community and get the full member experience.

Join For Free

CPU and network speed have increased significantly in the last decade, as well as memory and disk sizes. But still one of the possible side effects of moving from a monolithic architecture to microservices is the increase in the service latency. Here are few quick ideas on how to fight it using Kubernetes.

It Is Not the Network

In recent years, networks transitioned to using protocols that are more efficient and moved from 1 GBit to 10 GBit and even to a 25 GBit limit. Applications send much smaller payloads with less verbose data formats. With all that in mind, the chances are the bottleneck in a distributed application is not in the network interactions, but somewhere else like the database. We can safely ignore the rest of this article and go back to tuning the storage system.

Kubernetes Scheduler and Service Affinity

Co-located Pods in the same Region/Zone/Rack

Co-located pods in the same region/zone/rack.

If two services (deployed as Pods in the Kubernetes world) are going to interact a lot, the first approach to reduce the network latency would be to ask politely the scheduler to place the pods as close as possible using the node affinity feature. How close depends on our high-availability requirements (covered by anti-affinity), but it can be co-located in the same region, availability zone, or rack, or even on the same host.

Run Services in the Same Pod

Containers/Service co-located in the same Pod

Containers/service co-located in the same pod.

The deployment unit in Kubernetes (pod) allows a service to be independently updated, deployed, and scaled. But if performance is a higher priority, we could put two services in the same pod, as long as that is a deliberate decision. Both services would still be independently developed, tested, and released as containers, but they would share the same runtime lifecycle in the same deployment unit. That would allow the services to talk to each other over localhost rather than using the service layer, or use the file system, or use some other high performant IPC mechanism on the shared host, or shared memory.

Run Services in the Same Process

Co-located Services in the same Process

Co-located services in the same process.

If co-locating two services on the same host are not good enough, we could have a hybrid between microservices and monolith by sharing the same process for multiple services. That means we are back to a monolith, but we could still use some of the principles of microservices, allow development time independence, and make a compromise in favor of performance on rare occasions.

We could develop and release two services independently by two different teams, but place them in the same container and share the runtime.

For example, in the Java world that would be placing two.jar files in the same Tomcat, WildFly, or Karaf server. At runtime, the services can find each other and interact using a public static field that is accessible from any application in the same JVM. This same approach is used in Apache Camel direct component, which allows synchronous in-memory interaction of Camel routes from different.jar files by sharing the same JVM.

Other Areas to Explore

If none of the above approaches seems like a good idea, maybe you are exploring in the wrong direction. It might be better to explore whether using some alternative approaches such using a cache, data compression, HTTP/2, or something else might help for the overall application performance.

microservice Kubernetes

Published at DZone with permission of Bilgin Ibryam, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Top Authentication Trends to Watch Out for in 2023
  • How To Generate Code Coverage Report Using JaCoCo-Maven Plugin
  • Writing a Modern HTTP(S) Tunnel in Rust
  • Why Does DevOps Recommend Shift-Left Testing Principles?

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: