Eliran David

hey Porter. agree. Today I am more familiar with WebFlux that allow you easily to make all your server I/O operations non blocking. Following your comment, using WebFlux, the number of threads is fixed (comparing to the traditional approach) so I will suggest to use that (when choosing between Java options) if your application has mainly I/O operations and should scale. See my new post that relate to that :)
https://medium.com/nmc-techblog/shifting-from-java-to-node-js-duel-to-the-death-b052fd7ac1d1

hey Marc. Today I am more familiar with WebFlux, see my new post :)
https://medium.com/nmc-techblog/shifting-from-java-to-node-js-duel-to-the-death-b052fd7ac1d1

that's a good point GG GG. Tomcat can handle more than 600 because it maintain a queue of requests with no available threads, that's why we see more than 600 requests that got 200 OK. some of those requests in the queue got timeout because they were too much time in the queue waiting for a thread to serve them.
Today I am more familiar with WebFlux that allow you easily to make all your server I/O operations non blocking so I will suggest to use that (when choosing between Java options) if your application has mainly I/O operations and should scale. See my new post for more details:
https://medium.com/nmc-techblog/shifting-from-java-to-node-js-duel-to-the-death-b052fd7ac1d1

hey Adrian, the goal of making an endpoint async is to release the tomcat thread to do other stuff, but in the code examples I reviewed all other (TCP) operations were blocking, like calling third party service. Therefore we still have the problem of threads that are waiting and become a bottleneck. Today I am more familiar with WebFlux that allow you easily to make all your server I/O operations non blocking so I will suggest to use that (when choosing between Java options) if your application has mainly I/O operations and should scale. See my new post for more details:
https://medium.com/nmc-techblog/shifting-from-java-to-node-js-duel-to-the-death-b052fd7ac1d1

I think you can find the answer in my post. see the Callable example. Callable does not give threads as many as you need. When using Callable as return value, Spring will use its default spring thread pool (can be changed via spring configuration), which probably has more threads than ForkJoinPool. Log the thread ids to see it. Anyway, we would like to avoid the approach of "as many threads as we need" because that we will lead to CPU wasting too much time on context switch. You need to tune the number of threads, by monitoring, according to your use case.

Thank you Renier. Monitor the threads, perhaps you will find there a good explanation. The only thing I can think of is that your async code use more resources (assuming both are doing the same thing). Here are the Gatling simulations (scala):
https://github.com/elirandav/springAsyncServlet/tree/master/src/main/resources

Thank you Normandes. I don't think that return async will make the service scale better because in both approaches (async or not async), you need to decide on the number of pool's threads. The difference is that in not async you align only the threads number in tomcat properties and with the async approach you need to align tomcat properties and the pool that is used to run the async requests.

Hi Gopal, the maxThreads limits only the pool of tomact workers, not the java processes' threads. tomcat has more threads except the workers, like "Acceptor", "BlockPoller", etc. Also, the process has threads related to Garbage collector and monitoring (JMX). If you use IntelliJ for example, you can see the list of all threads in the Debugger window while your application is running.

Thank you Ammar! That's was disturbing. Now make sense :)

Thank you Robert. Currently, I understand the advantages of reactive approach only in theory but never had a full reactive service (and tested its performance in scale). It is my next subject for a blog post (to understand it better). In case you already know a post that includes a performance test (Spring Boot service with embedded tomcat) for reactive vs non-reactive, then a link will be helpful.

Thanks Marc. Currently, I understand the advantages of reactive approach only in theory but never had a full reactive service (and tested its performance in scale). It is my next subject for a blog post (to understand it better). In case you already know a post that includes a performance test (Spring Boot service with embedded tomcat) for reactive vs non-reactive, then a link will be helpful.

Thank you Tal.
Actually Callable and DeferredResult are completely different things except that they both triggers asynchronous request processing in Spring MVC when one of them is the return value of the controller. Callable is an interface that is part of java.util, and it is an improvement for the Runnable interface (should be implemented by any class whose instances are intended to be executed by a thread). Callable allows to return a value, while Runnable isn't. DeferredResult is a class designed by Spring to allow more options (that I will describe) for asynchronous request processing in SpringMVC, and this class just holds the result (as implied by its name) while your Callable implementation holds the async code. So it means you can use both in your controller, run your async code with Callable and set the result in DeferredResult, which will be the controller return value. So what do you get by using DeferredResult as the return value instead of Callable? DeferredResult has built-in callbacks like onError, onTimeout, and onCompletion. It makes error handling very easy.In addition, as it is just the result container, you can choose any thread (or thread pool) to run on your async code. With Callable, you don't have this choice.