Understanding Spring Reactive: Client-to-Server Communication

The purpose of this blog series is to take readers through the journey of Java servlet and Spring from thread per the request model to the single thread model in Spring Reactive (Spring Webflux to be more precise).

When we talk about a Servlet/Spring application, the flow of data takes place in two phases:

i) Client (e.g browser) to ServletContainer (e.g. Tomcat) and vice-versa
ii) ServletContainer to Servlet/Spring and vice-versa.

Client-to-Server Communication

In HTTP 1.1, all connections between the browser and the server are considered persistent unless declared otherwise. Persistence, in this context, means to use a single TCP connection to send and receive multiple HTTP requests/responses, as opposed to opening a new connection for every single request/response pair.

Until Tomcat 6, which is the implementation of Java EE 5 (and hence servlet 2.5), the default HTTP connector is asynchronous blocking and follows a one thread per connection model. This means that, in order to serve 100 concurrent users, it requires 100 active threads. We end up wasting resources (the thread), because connections may not be used heavily, but just enough to avoid a timeout.

In order to configure tomcat to use the non-blocking NIO connector instead of the default asynchronous blocking BIO, one simply needs to change the value of the protocol attribute of the connector tag in the server.xml from HTTP/1.1 to org.apache.coyote.http11.Http11NioProtocol.

One Thread Per Connection Model

Opposed to this is the relatively new NIO or non-blocking connector (based on J2SE 1.4). This connector has a couple of poller threads (usually depends upon the number of cores) used to keep the connection alive for all connected users while worker threads are called whenever data (a new HTTP request) is available. Threads can be allocated to connections only when requests are being processed. When a connection is idle between requests, the thread can be recycled and the connection is placed in a centralized NIO select set to detect new requests without consuming a separate thread. This model, called a thread per request, potentially allows web servers to handle a growing number of user connections with a fixed number of threads. This model leads to a much better sharing of resources (threads) and a larger number of concurrent users can be served from the same server. Java NIO API can be referenced for a better understanding of Channel, Selectors, and other related stuff.

From Tomcat 8.x, NIO is the default connector, and BIO support is dropped altogether from 8.5x.

NIO implementation (One Thread Per Request)

So, with the introduction of NIO connector, client-to-server communication was non-blocking, but the server-to-servlet connection was still blocking, which effectively meant that a thread would be blocked for every request leading to the One Thread per Request Model. So, as Servlet containers have evolved, there was a need for non-blocking support in Servlet API. Let's explain this with the following table.

In the above table, I wrote simulated blocking for NIO connectors, this means a blocking API (server-to-servlet and vice-versa) wrapped around the underlying non-blocking NIO API (browser-to-server and vice versa).

In the next article, we will be focusing on server-to-servlet/Spring communication. Stay tuned!