Design Flexible and Scalable Online Task Processing Systems in a Java EE 6 Web Container
by
·
Join the DZone community and get the full member experience.
Join For FreeIt was painful for web application architects and designers to handle long running online tasks in web containers until servlet asynchronous processing (Servlet 3.0 specification) was introduced. This article will introduce a solution to leverage this technical edge to create a flexible task processing engine without losing the scalability of the web container.
Long running tasks such as a long web service call or a long running query used to block the current thread on the pending request for a long time. Therefore it could significantly downgrade tje web container’s throughput and reduce the scalability of the system. There are some other solutions as discussed in the attached resources, but that solution needs the EJB container’s help; therefore, in the case of a web container like Tomcat, it is not applicable.
My solution is based on Servlet specification 3.0 standards, part of Java EE 6 release. The following diagram illustrates this architecture.
Architecture Diagram
The Task Engine mainly consists of a Thread Pool. In order to improve task processing capacity, a thread pool is created to handle multiple tasks concurrently, and the thread pool also allows having a Blocking Queue to hold tasks waiting for processing. This Task Engine actually delegates its task processing to this thread pool, and provides concurrent task submissions initiated from web clients.
Task Engine Class Diagram
Design Considerations
1. Usage of ServletContext listener. This is the right place to hook up this multithreaded task engine to the web container, since it is loosely coupled with the servlet to carry out recycling on attributes and allocated thread pool objects; and make it available in the application scope. Besides, with the annotation @WebListener, we could make it pluggable to any servlet.
2. Separation of task processing logic from others. The Task interface is an abstract layer above concrete processing in every thread in the thread pool. The methods beforeRun() and afterRun() in this interface are designed to put servlet related logic like request/response handling over there while pure task processing is designated to the inherited method Run(), therefore extensibility and reusability is achieved to some extend.
3. Communication to Client. With Ajax and asynchronous processing support, users won’t be blocked from further interaction on web pages after submits a long running task, and this request handling in the servlet container won’t be blocked until the requested resource become available any more. Servlet container is able to recycle this request thread to serve other requests and later when necessary this request could be resumed to push back data got from task processing process to the client.
When the asyncSupported attribute is set to true for a servlet, the response object is not committed on method exit. Calling startAsync() returns an AsyncContext object that caches the request/response object pair. The AsyncContext object is then served as a task context to initialize a task object and put into the application-scoped task engine waiting for processing. The original request thread is then recycled. When task processing ends or task is rejected by task engine, we could either call AsyncContext.getResponse().getWriter().print(...), and then complete() to commit the response, or calling dispatch()to direct the flow to another JSP page as a result.
4. Policy for dealing with overload considerations. In most cases the usage of unbounded queue in thread Pool is enough for web server. New tasks will queue up if there is no idle thread in the thread pool, which could lead to smooth out transient burst of tasks efficiently. But for some cases for example, where there is a significant request peak time in your system everyday or the back end task processing is experiencing of slowing down for quite a while, this unbounded queue may keep growing, which may exhaust resources and kill system finally. Then the bounded queue is coming to rescue. The task engine with a bounded queue could be designed to reject any new task after it is full, which makes system work quite stably. In addition, the most important thing to make this strategy work is that with the help of AsynContext and Ajax the client could be acknowledged immediately about this rejection by telling that system is experiencing high volume now; please resubmit your task later. This is acceptable in most cases from client perspective since they are allowed to continue working on other pages meanwhile.
There are four policy implementations provided for thread pool totally in Java 6 when a bounded work queue fills up: AbortPolicy, CallerRunsPolicy, DiscardPolicy, and DiscardOldestPolicy. The default policy abort, causes execute to throw the unchecked Rejected-ExecutionException. Our system is taking this policy, and then catches this exception and implement our own overflow handling here. Please refer to the following state diagram for details. The CallerRunsPolicy allows the thread that invokes execute itself runs the task. This provides a simple feedback control mechanism that will slow down the rate that new tasks are submitted. The discard policy silently discards the newly submitted task if it cannot be queued for execution; the discard-oldest policy discards the task that would otherwise be executed next and tries to resubmit the new task.
State Diagram for Task Processing Based on Abort Policy
Sample Code Snippets (Adopting Abort Policy):
@WebListener( )
public class TaskEngineListener implements ServletContextListener {
public void contextInitialized(ServletContextEvent sce) {
logger.info("Start Task Engine...");
TaskEngine taskengine= new TaskEngine();
sce.getServletContext().setAttribute("TaskEngine", taskengine);
}
public void contextDestroyed(ServletContextEvent sce) {
logger.info("Try to shut down Task Engine");
ServletContext ctx= sce.getServletContext();
TaskEngine taskengine=(TaskEngine)ctx.getAttribute("TaskEngine");
if(taskengine!=null){
taskengine.shutdown();
}
ctx.removeAttribute("TaskEngine");
}
}
@WebServlet (asyncSupported = true)
public class AsyncServlet extends HttpServlet {
…
@Override
protected void doPost(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
response.setContentType("text/html;charset=UTF-8");
response.setHeader("Cache-Control", "private");
response.setHeader("Pragma", "no-cache");
final AsyncContext ac = request.startAsync(request, response);
ac.setTimeout(10 * 60 * 1000);
Task task= new AsyncServletTask(ac);
TaskEngine taskengine = (TaskEngine) request.getServletContext().getAttribute("TaskEngine");
if (!(taskengine.addTask(task)))
{ PrintWriter out = null;
try {
String name = ac.getRequest().getParameter("name");
out = ac.getResponse().getWriter();
out.println("Task: " + name + " is declined, Please submit later");
out.close();
} catch (IOException ex) {
logger.log(Level.SEVERE, "Task Rejection Exception",ex);
} finally {
ac.complete();
}
}
}
}
public class TaskEngine {
private final static int POOLMAX=4;
private final static int TASKQUEUELENGTH=3;
private final ExecutorService jobExecutor
= new TaskEngineThreadPool(POOLMAX, POOLMAX,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>(TASKQUEUELENGTH), new ThreadPoolExecutor.AbortPolicy());
public TaskEngine(){
}
public synchronized boolean addTask(Task task){
boolean done=true;
try{
jobExecutor.execute(task);
}catch(RejectedExecutionException ex){
done=false;
}
return done;
}
...
}
public class AsyncServletTask implements Task {
private static final Logger logger = Logger.getLogger(AsyncServletTask.class.getName());
private AsyncContext ac=null;
private String taskName=null;
public AsyncServletTask(AsyncContext in){
ac=in;
}
@Override
public void run() {
//handle task processing here
}
@Override
public void beforeRun() {
this.taskName=ac.getRequest().getParameter("name");
}
@Override
public void afterRun() {
try {
PrintWriter out = ac.getResponse().getWriter();
//logger.info("Get Task:"+taskName);
out.println("Servlet AsyncServlet "+taskName+" response");
out.close();
} catch(IOException ex) {
logger.log(Level.SEVERE,"AfterRun() exception",ex);
}
ac.complete();
}
}
In Conclusion
This article introduces a solution design to handle online task processing under web container with consideration of system scalability and extensibility. The advantages of this solution are:
- Make servlet container manage its threads more efficiently when handling long running tasks.
- Lightweight task engine implementation here allows a centralized and customized control over long running tasks, which improves system efficiency overall
- Provide flexible processing acknowledgement to the client and therefore make user page more responsive.
Disadvantages:
- Thread Pool tuning is suggested.
About the author
Kui Zhang is a Sun Microsystems Certified Enterprise Architect working in an International IT consulting company. He has architected, designed and developed many J2EE/JEE applications on Websphere and Weblogic
Resources:
- http://java.sun.com/javase/6/docs/api/java/util/concurrent/ThreadPoolExecutor.html
- http://blogs.sun.com/enterprisetechtips/entry/asynchronous_support_in_servlet_3
- http://www.javaranch.com/journal/2004/03/AsynchronousProcessingFromServlets.html
- http://www.javaworld.com/javaworld/jw-02-2009/jw-02-servlet3.html
- Java Concurrency in Practice Author: Brian Goetz
Opinions expressed by DZone contributors are their own.
Trending
-
DevOps Midwest: A Community Event Full of DevSecOps Best Practices
-
How To Manage Vulnerabilities in Modern Cloud-Native Applications
-
How Web3 Is Driving Social and Financial Empowerment
-
Building A Log Analytics Solution 10 Times More Cost-Effective Than Elasticsearch
Comments