Understanding Spring Reactive: Servlet 3.1/Spring MVC Non-Blocking IO

Servlet 3.0 was released as part of Java EE 6 and made huge changes focused at ease-of-use. The idea was to leverage the latest language features, such as annotations and generics, and modernize how Servlets can be written. One of the major changes was Async Servlets. The web.xml was also made as optional as possible. Servlet 3.1, released as part of Java EE 7, was an incremental release, focusing on a couple of key features like non-blocking IO.

Non-blocking I/O with Servlet 3.0 (Async Servlets as discussed in the previous article) allowed asynchronous request processing, but only the traditional I/O was permitted, which is blocking. This can restrict that scalability of your applications. Non-blocking I/O allows you to build scalable applications.

Let’s discuss what I mean by the above. We have learned, in the previous article, that in the case of an async servlet, we must use non-blocking code. So, let’s modify our earlier MyServlet code and replace runnable logic as below:

Let’s revisit the code snippet that we discussed in the previous article:

@WebServlet(name="myServlet", urlPatterns={"/asyncprocess"}, asyncSupported=true)
public class MyServlet extends HttpServlet {
    public void doGet(HttpServletRequest request, HttpServletResponse response) {
OutputStream out = response.getOutputStream();
            AsyncContext aCtx = request.startAsync(request, response); 
            doAsyncREST(request).thenAccept(json -> {
out.write(json);  ---> BLOCKING!
ctx.complete();
          });
}

In the above code, the container request thread is released and work is done in another thread. So, for the Async Servlet to actually work as expected, we have the following requirements:

•  doAsyncREST() must use the Async library to call REST and return CompletableFuture.This is possible using AsyncHttpClient, which we have already used in the previous article.
•  thenAccept() should use Async libraries.

But, in Servlet 3.0, IO was traditional for blocking, and hence, the thread calling out.write() will block.

Let’s say that we have to write a large JSON file back to the client. As we are using the NIO connector, OutputStream will first write to buffers and those buffers need to be emptied by clients, using the selector/channel mechanism of NIO. Now, if clients are on a slow network, then out.write() will have to wait until buffers are empty again, as  InputStream/OutputStream is blocking.

The above problem of blocking was removed by the Servlet 3.1 release by introducing Async IO.

Servlet 3.1 Async IO

Let’s discuss this with the help of the code snippet shown below:

 void doGet(request, response) {
        ServletOutputStream out = response.getOutputStream();
        AsyncContext ctx = request.startAsync();
        out.setWriteListener(new WriteListener() {
            void onWritePossible() {
                while (out.isReady()) {
                    byte[] buffer = readFromSomeSource();
                    if (buffer != null)
                        out.write(buffer); ---> Async Write!
                    else{
                        ctx.complete(); break;
                    }
                  }
                }
            });
        }

In the above code, we are making use of the Write/Read Listener, which were introduced in 3.1. WriteListener is an interface that has an onWritePossible() method, which gets called by the Servlet Container. ServletOutputStreamt.isReady()is used to check if it is possible to write in NIO channel buffers. In case it returns false, then it schedules a call on the Servlet container for the  onWritePossible() method, and at some later point, onWritePossible() is called on another thread. So, in this way, out.write() never blocks for the slow client to empty the channel buffers.

Non-Blocking IO in Spring?

To use this feature of non-blocking IO in the Spring application, we would need Spring 5, which has Java EE 7 as its baseline version. So, our earlier example, which is also mentioned below, will execute in full non-blocking mode if we run this code on Spring 5 MVC, Tomcat 8.5+:

@GetMapping(value = "/asyncNonBlockingRequestProcessing")
    public CompletableFuture<String> asyncNonBlockingRequestProcessing(){
            ListenableFuture<String> listenableFuture = getRequest.execute(new AsyncCompletionHandler<String>() {
                @Override
                public String onCompleted(Response response) throws Exception {
                    logger.debug("Async Non Blocking Request processing completed");
                    return "Async Non blocking...";
                 }
            });
            return listenableFuture.toCompletableFuture();
    }

By now, we have discussed how Servlet has evolved, along with Spring, to provide complete non-blocking support. This means that we can scale our application with a smaller number of threads. In our next article, we will be discussing the Spring Reactive stack (i.e. Spring Webflux). One might think that, if Spring MVC is capable of handling request in a non-blocking way, then why use Spring Webflux because it was released as a separate stack?

Stay tuned! We will answer this question in our next article.