DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Last call! Secure your stack and shape the future! Help dev teams across the globe navigate their software supply chain security challenges.

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Releasing software shouldn't be stressful or risky. Learn how to leverage progressive delivery techniques to ensure safer deployments.

Avoid machine learning mistakes and boost model performance! Discover key ML patterns, anti-patterns, data strategies, and more.

Related

  • Auto-Scaling a Spring Boot Native App With Nomad
  • 3 Reasons for the Mounting Demand for Smart Cloud-Native Application Development
  • Monoliths to Microservices: Untangling Your Spaghetti
  • Tips for Managing Multi-Cluster Kubernetes Deployment With High Efficiencies

Trending

  • Cookies Revisited: A Networking Solution for Third-Party Cookies
  • Start Coding With Google Cloud Workstations
  • Automating Data Pipelines: Generating PySpark and SQL Jobs With LLMs in Cloudera
  • Measuring the Impact of AI on Software Engineering Productivity
  1. DZone
  2. Software Design and Architecture
  3. Microservices
  4. Microservices Architecture: Introduction to Auto Scaling

Microservices Architecture: Introduction to Auto Scaling

In this article, we look at auto scaling and the important parts of implementing auto scaling in a microservices architecture.

By 
Ranga Karanam user avatar
Ranga Karanam
DZone Core CORE ·
Jun. 05, 19 · Tutorial
Likes (17)
Comment
Save
Tweet
Share
36.6K Views

Join the DZone community and get the full member experience.

Join For Free

In this article, we focus our attention on dynamic scaling, aslo known as auto scaling, and why we need applications that can auto scale.

You Will Learn

  • What auto or dynamic scaling is.
  • Why dynamic scaling is important in a microservices context.
  • How you can implement dynamic scaling in the coud.

Cloud and Microservices Terminology

This is the fifth article in a series of six articles on terminology used with cloud and microservices. The previous four can be found here:

  1. Microservices Architecture: What Is Service Discovery?
  2. Microservices Architecture: Centralized Configuration and Config Server
  3. Microservices Architecture: Introduction to API Gateways 
  4. Microservices Architecture: THe Importance of Centralized Logging

The Load on Applications Varies

The load on your applications vary depending on time of the day, the day of the month or the month of the year.

Take for instance, amazon.com. It has very high loads during Thanksgiving, up to 20 times the normal load. However, during the major sports events such as the Super Bowl or a FIFA World Cup, the traffic could be considerably less - because every body is busy watching the event.

How can you setup infrastructure for applications to manage varying loads?

It is quite possible that the infrastructure needs to handle 10x the normal load.

If you have on-premise infrastructure, you need a large infrastructure in place to handle peak load.

During periods with less load, a lot of infrastructure would be sitting idle.

Cloud to the Rescue

That's where cloud comes into the picture. With cloud, you can request more resources when the load is high and give them back to the cloud when you have less load.

This is called Scale Out (create more instances as the load increases) and Scale In (reduces instances as the load goes down)

How do you build applications that are cloud enabled, i.e. applications that work well in the cloud?

That's where a microservices architecture comes into the picture.

Introducing Auto Scaling

Building your application using microservices enables you to increase the number of microservice instances during high load, and reduce them during times with less load.

Consider the following example of a CurrencyConversionService:

Basic Microservice ArchitectureBasic Microservice Architecture

The CurrencyConversionService talks to the ForexService. The ForexService is concerned with calculating how many INR can result from 1 USD, or how many INR can result from 1 EUR.

The CurrencyConversionService takes a bag of currencies and amounts and produces the total amount in a currency of your choice. For example, it will tell the total worth in INR of 10 EUR and 25 USD.

The ForexService might also be consumed from a number of other microservices.

Scaling Infrastructure to Match Load

The load on the ForexService might be different from the load on the CurrencyConversionService. You might need to have a different number of instances of the CurrencyConversionService and ForexService. For example, there may be two instances of the CurrencyConversionService, and five instances of the ForexService:

Basic Microservice ArchitectureBasic Microservice Architecture

At a later point in time, the load on the CurrencyConversionService could be low, needing just two instances. On the other hand, a much higher load on the ForexService could need 50 instances. The requests coming in from the two instances of CurrencyConversionService are distributed across the 50 instances of the ForexService.

That, in essence, is the requirement for auto scaling — a dynamically changing number of microservice instances, and evenly distributing the load across them.

Implementing Auto Scaling

There are a few important concepts involved in implementing auto scaling. The following sections discuss them in some detail.

Naming Server

Naming servers enable something called location transparency. Every microservice registers with the naming service. Any microservice that needs to talk to another microservice will ask the naming server for its location.

Whenever a new instance of CurrencyConversionService or ForexService comes up, it registers with the naming server. Basic Microservice Architecture Auto Scaling

When CurrencyConversionService wants to talk to ForexService, it asks the naming server for available instances.

Implementing Location Transparency

CurrencyConversionService knows that there are five instances of the ForexService.

How does it distribute the load among all these instances?

That's where a load balancer comes into the picture.

A popular client side load balancing framework is Ribbon. Basic Microservice Architecture

Let's look at a diagram to understand whats happening:

Load balancing framework

As soon as any instance of CurrencyConversionService or ForexService comes up, it registers itself with the naming server. If the CCSInstance2 wants to know the URL of ForexService instances, it again talks to the naming server. The naming server responds with a list of all instances of the ForexService — FSInstance1 and FSinstance2 — and their corresponding URLs.

The Ribbon load balancer does a round-robin among the ForexService instances to balance out the load among the instances.

Ribbon offers wide variety of load balancing algorithms to choose from.

When to Increase and Decrease Microservices Instances

There is one question we did not really talk about.

How do we know when to increase or decrease the number of instances of a microservices?

That is where application monitoring and container (Docker) management (using Kubernetes) comes into the picture.

Auto scaling microservices

An application needs to be monitored to find out how much load it has. For this, the application has to expose metrics for us to track the load.

You can containerize each microservice using Docker and create an image.

Kubernetes has the capability to manage containers. Kubernetes can be configured to auto scale based on the load. Kubernetes can identify the application instances, monitor their loads, and automatically scale up and down.

Summary

In this article, we talked about auto scaling. We looked at important parts of implementing auto scaling — naming server, load balancer, containers (Docker), and container orchestration (Kubernetes).

microservice Scaling (geometry) Architecture application Kubernetes Load balancing (computing) Cloud

Published at DZone with permission of Ranga Karanam, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • Auto-Scaling a Spring Boot Native App With Nomad
  • 3 Reasons for the Mounting Demand for Smart Cloud-Native Application Development
  • Monoliths to Microservices: Untangling Your Spaghetti
  • Tips for Managing Multi-Cluster Kubernetes Deployment With High Efficiencies

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!