DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Because the DevOps movement has redefined engineering responsibilities, SREs now have to become stewards of observability strategy.

Apache Cassandra combines the benefits of major NoSQL databases to support data management needs not covered by traditional RDBMS vendors.

The software you build is only as secure as the code that powers it. Learn how malicious code creeps into your software supply chain.

Generative AI has transformed nearly every industry. How can you leverage GenAI to improve your productivity and efficiency?

Related

  • Distributed Tracing System (Spring Cloud Sleuth + OpenZipkin)
  • How To Build Web Service Using Spring Boot 2.x
  • Build Reactive REST APIs With Spring WebFlux
  • Node.js Http Module to Consume Spring RESTful Web Application

Trending

  • Can You Run a MariaDB Cluster on a $150 Kubernetes Lab? I Gave It a Shot
  • Building a Real-Time Audio Transcription System With OpenAI’s Realtime API
  • AI Speaks for the World... But Whose Humanity Does It Learn From?
  • The Evolution of Scalable and Resilient Container Infrastructure
  1. DZone
  2. Coding
  3. Frameworks
  4. Spring Webflux: EventLoop vs Thread Per Request Model

Spring Webflux: EventLoop vs Thread Per Request Model

In this article, we take a look at Spring Webflux to understand how we can use reactive programming to increase application performance.

By 
Kaushal Singh user avatar
Kaushal Singh
·
Updated Apr. 16, 20 · Tutorial
Likes (23)
Comment
Save
Tweet
Share
53.3K Views

Join the DZone community and get the full member experience.

Join For Free

Spring Webflux Introduction

Spring Webflux has been introduced as part of Spring 5, and with this, it started to support Reactive Programming. It uses an asynchronous programming model. The use of reactive programming makes applications highly scalable to support high request load over a period of time.

What Problem It Solved Compared With Spring MVC

Spring MVC uses a Synchronous Programming model, where each request has been mapped to a thread and which is responsible to take the response back to the request socket. Cases where applications make some network calls like, fetch data from a database, fetch a response from another application, file read/write etc., a request thread has to wait to get the desired response. 

The request thread here is blocked, and there is no CPU utilization in this period. That’s why this model uses a large thread pool for request processing. This could be fine for low request rate applications, but for applications with high request rates, this can ultimately make the application slow or unresponsive. This certainly affects businesses in today’s market.

Spring MVC with Servlet container

For such applications, reactive programming can be used effectively. Here, a request thread sends the required input data to the network and doesn’t wait for a response. Rather, it assigns a callback function, which will get executed once this blocking task is completed. This way, the request thread makes itself available to handle another request. 

If used in a proper manner, not a single thread can be blocked, and threads can utilize the CPU efficiently. Proper manner here means, for this model, it is required for every thread to behave reactively. Hence, database drivers, inter-service communication, web server, etc. should use reactive threads as well. 

Embedded Server

Spring Webflux by default has Netty as an embedded server. Apart from that, it is also supported on Tomcat, Jetty, Undertow, and other Servlet 3.1+ containers. The thing to note here is that Netty and Undertow are non-servlet runtimes and Tomcat and Jetty are well-known servlet containers.

Before the introduction of Spring Webflux, Spring MVC used to have Tomcat as its default embedded server, which uses the Thread Per Request Model. Now, with Webflux, Netty has been preferred as a default, which uses the Event Loop Model. Although Webflux is also supported on Tomcat, it's only with versions that implement Servlet 3.1 or later APIs. 

Note: Servlet 3 introduces the feature for asynchronous programming, and Servlet 3.1 introduced additionally asynchronous I/O allowing everything asynchronous. 

EventLoop

EventLoop is a Non-Blocking I/O Thread (NIO), which runs continuously and takes new requests from a range of socket channels. If there are multiple EventLoops, then each EventLoop is assigned to a group of Socket Channels, and all EventLoops are managed under an EventLoopGroup.

The following diagram shows how EventLoops work: 

Event Loop Working

Here, EventLoop is shown to explain its usage on the server-side. But, it works the same way when it's on the client-side, and it sends requests to another server i.e. for I/O requests.

  1. All requests are received on a unique socket, associated with a channel, known as SocketChannel. 
  2. There is always a single EventLoop thread associated with a range of SocketChannels. So, all requests to that Sockets/SocketChannels are handed over to the same EventLoop.
  3. Requests on the EventLoop go through a Channel Pipeline, where a number of Inbound channel handlers or WebFilters are configured for the required processing.
  4. After this, EventLoop executes the application-specific code. 
  5. On its completion, EventLoop again goes through a number of Outbound Channel handlers for configured processing. 
  6. In the end, EventLoop handed back the response to the same SocketChannel/Socket.
  7. Repeat Step 1 to Step 6 in a loop.

That’s a simple use case that will certainly minimize the resource usage due to single thread usage, but this is not going to solve the real problem. 

What happens if an application blocks the EventLoop for a long time due to any of the following reasons?

  • High CPU intensive work.
  • Database operations like read/write etc.
  • File read/write.
  • Any call to another application over the network. 

In these cases, EventLoop will get blocked at Step 4, and we could be in the same situation where our application could become slow or unresponsive very soon. Creating additional EventLoop is not the solution. As already mentioned, a range of sockets are bound to a single EventLoop. Hence, a block in any EventLoop doesn’t allow other EventLoop to work on its behalf for already bounded sockets.  

Note: Multiple EventLoops should be created only if an application is running on a multi-CPU platform, as EventLoops must be kept running, and a single CPU can’t keep multiple EventLoops running simultaneously. By default, the application starts with as many EventLoops as it has CPU cores.

Now, that’s where the real power of NIO EventLoop comes into the picture with superb simplicity behind it. A user application should delegate the request to another thread and return the result asynchronously via a callback to unblock the EventLoop for new request handling. So, the updated working steps for EventLoop would be like the following:

  1. Step 1 to Step 3 as above.
  2. EventLoop delegates the request to a new Worker Thread.
    1. Worker Thread perform these long tasks.
    2. After completion, it writes the response to a task and adds that in ScheduledTaskQueue.
  3. EventLoop poll tasks in task queue ScheduledTaskQueue
    1. If there are any, perform Step 5 to Step 6 via a task’s Runnable run method.
    2. Otherwise, it continues to poll new requests at SocketChannel.

Fully Reactive approach with Webclient

Advantage:

  1. Lightweight request processing thread.
  2. Optimum utilization of hardware resources.
  3. Single EventLoop can be shared between an HTTP client and request processing.
  4. A single thread can handle requests over many sockets (i.e. from different clients).
  5. This model provides support for backpressure handling in case of infinite stream response.

Thread Per Request Model (Implementing Servlet 3.1+)

The Thread per Request model is in practice since the introduction of synchronous Servlet programming. This model was adopted by Servlet Containers to handle incoming requests. As request handling is synchronous, this model requires many threads to handle requests. Hence, this increases resource utilization. There was no consideration of network I/O, which can block these threads. Hence, for scalability, generally, resources need to scale-up. This increases the system cost. 

Going reactive, using Spring Webflux gives the option to use Servlet Containers as well, if they implement Servlet 3.1+ APIs. Tomcat is the most commonly used Servlet Container, and it supports reactive programming as well.

When you choose Spring Webflux on Tomcat, the application starts with a certain number of request processing threads under the thread pool (e.g. 10). Requests from sockets are assigned to a thread in this pool. The thing to note here is that there is no permanent binding of a socket to Thread.

If any request thread gets blocked for an I/O call, requests can still be handled by other threads in the thread pool. But, more such requests can block all request threads. Until now, it is the same as we used to have in synchronous processing. But, in a reactive approach, it allows, a user to delegate requests to another pool of Worker Thread, as discussed with EventLoop. That way, request processing threads become available and the application remains responsive. 

Reactive Thread Per Request Model with Webclient

The following are the steps executed while handling a request in this model:

  1. All requests are received on a unique socket, associated with a channel known as SocketChannel. 
  2. The request is assigned to a Thread from the Thread Pool.
  3. The request on the Thread goes through a certain handler (like filter, servlet) for necessary pre-processing.
  4. The request thread can delegate requests to a Worker thread or Reactive Web client, while executing any blocking code in a Controller.
  5. On its completion, the Worker Thread or Web client (EventLoop) will be responsible to give back response to concerned Socket.

Again, by making applications fully reactive by choosing reactive clients (like Reactive Webclient) and Reactive DB drivers, minimum threads can provide effective scalability to the system. 

Advantages:

  1. Supports and allows the use of Servlet APIs.
  2. If any request thread blocks, it only blocks a single client socket, not a range of sockets as in EventLoop.
  3. Optimum utilization of hardware resources

Performance Comparison

For this comparison, a sample Reactive Spring Boot Application using Spring Webflux has been created with a deliberate delay of 100ms. The request has been delegated to another Worker Thread to unblock the request processing thread. 

The following configuration has been used for this comparison:

  • Single EventLoop configured for Netty Performance using property reactor.netty.ioWorkerCount=1
  • Single Request processing thread configured for Tomcat Performance using property server.tomcat.max-threads=1
  • AWS EC2 instance with configuration of t2.micro (1GB RAM, 1CPU).
  • Jmeter test script with 100 users and ramp-up of 1 second executed for 10 minutes.

CPU Utilization Comparison

Max CPU utilization

Tomcat Thread Per Request Model: 37.6%

Netty EventLoop: 34.6% 

Result : Tomcat shows 8.67052% increase in CPU utilization.

Performance Result

Tomcat 90 percentile response time: 114ms

Response time from Tomcat

 

Netty EventLoop 90 percentile response time: 109ms

Netty EventLoop response time

 

Result: Tomcat shows 4.58716% increase in response time.

Requests Spring Framework application

Opinions expressed by DZone contributors are their own.

Related

  • Distributed Tracing System (Spring Cloud Sleuth + OpenZipkin)
  • How To Build Web Service Using Spring Boot 2.x
  • Build Reactive REST APIs With Spring WebFlux
  • Node.js Http Module to Consume Spring RESTful Web Application

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!