Displaying Progress of Spring Application Startup in the Browser

When you restart your enterprise application, what do your clients see when they open the web browser?

1. They see nothing, the server doesn't respond yet so the web browser displays ERR_CONNECTION_REFUSED.
2. The web proxy (if any) in front of your application notices that it's down and display "friendly" error message
3. The website takes forever to load—it accepted a socket connection and HTTP request but waits for response until the application actually boots up.
   
4. Your application is scaled out so that other nodes quickly pick up requests and no one notices (and the session is replicated anyway).
   
5. ... Or the application is so fast to start that no one notices any disruption. (Hey, a plain Spring Boot Hello world app takes less than 3 seconds from hitting java -jar ... [Enter] to start serving requests.) BTW check out SPR-8767: Parallel bean initialization during startup.
   

It's definitely better to be in situation 4 and 5, but in this article we'll cover more robust handling of situations 1 and 3.

A typical Spring Boot application starts a web container (e.g. Tomcat) at the very end, when all beans are loaded (situation 1). This is a very reasonable default as it prevents clients from reaching our endpoints until they are fully configured. However, this means we cannot differentiate between application that start up for several seconds and applications that are down. So the idea is to have an application that shows some meaningful startup page while it loads, similar to a web proxy showing "Service unavailable". However, since such a startup page is part of our application, it can potentially have greater insight into startup progress. We want to start Tomcat earlier in the initialization lifecycle, but serve a special purpose startup page after Spring fully bootstraps. This special page should intercept every possible request - thus it sounds like a servlet filter.

Starting Tomcat Eagerly and Early

In Spring Boot servlet a container is initialized via EmbeddedServletContainerFactory that creates an instance ofEmbeddedServletContainer. We have an opportunity to intercept this process usingEmbeddedServletContainerCustomizer. The container is created early in the application lifecycle, but it's startedmuch later, when whole context is done. So I thought I will simply call start() in my own customizer and that's it.

Unfortunately, ConfigurableEmbeddedServletContainer doesn't expose such an API, so I had to decorateEmbeddedServletContainerFactory like this:

class ProgressBeanPostProcessor implements BeanPostProcessor {

    //...

    @Override
    public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
        if (bean instanceof EmbeddedServletContainerFactory) {
            return wrap((EmbeddedServletContainerFactory) bean);
        } else {
            return bean;
        }
    }

    private EmbeddedServletContainerFactory wrap(EmbeddedServletContainerFactory factory) {
        return new EmbeddedServletContainerFactory() {
            @Override
            public EmbeddedServletContainer getEmbeddedServletContainer(ServletContextInitializer... initializers) {
                final EmbeddedServletContainer container = factory.getEmbeddedServletContainer(initializers);
                log.debug("Eagerly starting {}", container);
                container.start();
                return container;
            }
        };
    }
}

You might think that BeanPostProcessor is overkill, but it will become very useful later on. What we do here is that if we encounter EmbeddedServletContainerFactory being requested from an application context, we return a decorator that eagerly starts Tomcat. This leaves us with rather unstable setup, where Tomcat accepts connections to not yet initialized context. So let's put a servlet filter intercepting all requests until context is done.

Intercepting Requests During Startup

I started simply by adding FilterRegistrationBean to Spring context, hoping it would intercept incoming request untils context is started. This was fruitless: I had to wait a long second until the filter was registered and ready, therefore from the user perspective the application was hanging. Later on I even tried registering filter directly in Tomcat using the servlet API (javax.servlet.ServletContext.addFilter()) but apparently the whole DispatcherServlet had to be bootstrapped beforehand. Remember all I wanted was extremely fast feedback from the application that it's about to initialize.

So I ended up with Tomcat's proprietary API: org.apache.catalina.Valve. Valve is similar to servlet filter, but it's part of Tomcat's architecture. Tomcat bundles multiple valves on its own to handle various container features like SSL, session clustering and X-Forwarded-For handling. Also Logback Access uses this API so I'm not feeling that guilty. The Valve looks like this:

package com.nurkiewicz.progress;

import org.apache.catalina.connector.Request;
import org.apache.catalina.connector.Response;
import org.apache.catalina.valves.ValveBase;
import org.apache.tomcat.util.http.fileupload.IOUtils;

import javax.servlet.ServletException;
import java.io.IOException;
import java.io.InputStream;

public class ProgressValve extends ValveBase {

    @Override
    public void invoke(Request request, Response response) throws IOException, ServletException {
        try (InputStream loadingHtml = getClass().getResourceAsStream("loading.html")) {
            IOUtils.copy(loadingHtml, response.getOutputStream());
        }
    }
}

Valves typically delegate to the next valve in a chain, but this time we simply return a static loading.html page for every single request. Registering such a valve is surprisingly simple, Spring Boot has an API for that!

if (factory instanceof TomcatEmbeddedServletContainerFactory) {
    ((TomcatEmbeddedServletContainerFactory) factory).addContextValves(new ProgressValve());
}

A custom valve turned out to be a great idea, it starts immediately with Tomcat and is fairly easy to use. However, you might have noticed that we never gave up serving loading.html, even after our application started. That's bad. There are multiple ways Spring context can signal initialization, e.g. with ApplicationListener<ContextRefreshedEvent>:

@Component
class Listener implements ApplicationListener<ContextRefreshedEvent> {

    private static final CompletableFuture<ContextRefreshedEvent> promise = new CompletableFuture<>();

    public static CompletableFuture<ContextRefreshedEvent> initialization() {
        return promise;
    }

    public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException {
        return bean;
    }

    @Override
    public void onApplicationEvent(ContextRefreshedEvent event) {
        promise.complete(event);
    }

}

I know what you think, "static"? But inside Valve I don't want to touch the Spring context at all, as it might introduce blocking or even deadlock if I ask for some bean at the wrong point in time from a random thread. When we complete promise,Valve deregisters itself:

public class ProgressValve extends ValveBase {

    public ProgressValve() {
        Listener
                .initialization()
                .thenRun(this::removeMyself);
    }

    private void removeMyself() {
        getContainer().getPipeline().removeValve(this);
    }

    //...

}

This is a surprisingly clean solution: when Valve is no longer needed, rather than paying the cost on every single request, we simply remove it from the processing pipeline. I'm not going to demonstrate how and why it works, let's move directly to the target solution.

Monitoring Progress

Monitoring progress of Spring application context startup is surprisingly simple. Also, I'm amazed how "hackable" Spring is, as opposed to API- and spec-driven frameworks like EJB or JSF. In Spring I can simply implementBeanPostProcessor to be notified about each and every bean being created and initialized (full source code):

package com.nurkiewicz.progress;

import org.springframework.beans.BeansException;
import org.springframework.beans.factory.config.BeanPostProcessor;
import org.springframework.context.ApplicationListener;
import org.springframework.context.event.ContextRefreshedEvent;
import rx.Observable;
import rx.subjects.ReplaySubject;
import rx.subjects.Subject;

class ProgressBeanPostProcessor implements BeanPostProcessor, ApplicationListener<ContextRefreshedEvent> {

    private static final Subject<String, String> beans = ReplaySubject.create();

    public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException {
        return bean;
    }

    @Override
    public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
        beans.onNext(beanName);
        return bean;
    }

    @Override
    public void onApplicationEvent(ContextRefreshedEvent event) {
        beans.onCompleted();
    }

    static Observable<String> observe() {
        return beans;
    }
}

Every time a new bean is initialized, I publish its name into RxJava's observable. When the whole application is initialized, I complete Observable. This Observable can later be consumed by anyone, e.g. our custom ProgressValve (full source code):

public class ProgressValve extends ValveBase {

    public ProgressValve() {
        super(true);
        ProgressBeanPostProcessor.observe().subscribe(
                beanName -> log.trace("Bean found: {}", beanName),
                t -> log.error("Failed", t),
                this::removeMyself);
    }

    @Override
    public void invoke(Request request, Response response) throws IOException, ServletException {
        switch (request.getRequestURI()) {
            case "/init.stream":
                final AsyncContext asyncContext = request.startAsync();
                streamProgress(asyncContext);
                break;
            case "/health":
            case "/info":
                response.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR);
                break;
            default:
                sendHtml(response, "loading.html");
        }
    }

    //...

}

ProgressValve is now way more complex, and we're not done yet. It can handle multiple different requests, for example I intentionally return 503 on /health and /info Actuator endpoints so that the application appears as if it was down during startup. All other requests except init.stream show the familiar loading.html. /init.stream is special. It's a server-sent events endpoint that will push messages every time a new bean is initialized (sorry for a wall of code):

private void streamProgress(AsyncContext asyncContext) throws IOException {
    final ServletResponse resp = asyncContext.getResponse();
    resp.setContentType("text/event-stream");
    resp.setCharacterEncoding("UTF-8");
    resp.flushBuffer();
    final Subscription subscription = ProgressBeanPostProcessor.observe()
            .map(beanName -> "data: " + beanName)
            .subscribeOn(Schedulers.io())
            .subscribe(
                    event -> stream(event, asyncContext.getResponse()),
                    e -> log.error("Error in observe()", e),
                    () -> complete(asyncContext)
            );
    unsubscribeOnDisconnect(asyncContext, subscription);
}

private void complete(AsyncContext asyncContext) {
    stream("event: complete\ndata:", asyncContext.getResponse());
    asyncContext.complete();
}

private void unsubscribeOnDisconnect(AsyncContext asyncContext, final Subscription subscription) {
    asyncContext.addListener(new AsyncListener() {
        @Override
        public void onComplete(AsyncEvent event) throws IOException {
            subscription.unsubscribe();
        }

        @Override
        public void onTimeout(AsyncEvent event) throws IOException {
            subscription.unsubscribe();
        }

        @Override
        public void onError(AsyncEvent event) throws IOException {
            subscription.unsubscribe();
        }

        @Override
        public void onStartAsync(AsyncEvent event) throws IOException {}
    });
}

private void stream(String event, ServletResponse response) {
    try {
        final PrintWriter writer = response.getWriter();
        writer.println(event);
        writer.println();
        writer.flush();
    } catch (IOException e) {
        log.warn("Failed to stream", e);
    }
}

This means we can track the progress of Spring's application context startup using simple HTTP interface (!):

$ curl -v localhost:8090/init.stream
> GET /init.stream HTTP/1.1
> User-Agent: curl/7.35.0
> Host: localhost:8090
> Accept: */*

< HTTP/1.1 200 OK
< Content-Type: text/event-stream;charset=UTF-8
< Transfer-Encoding: chunked

data: org.springframework.boot.autoconfigure.web.EmbeddedServletContainerAutoConfiguration$EmbeddedTomcat

data: org.springframework.boot.autoconfigure.websocket.WebSocketAutoConfiguration$TomcatWebSocketConfiguration

data: websocketContainerCustomizer

data: org.springframework.boot.autoconfigure.web.ServerPropertiesAutoConfiguration

data: toStringFriendlyJsonNodeToStringConverter

data: org.hibernate.validator.internal.constraintvalidators.bv.NotNullValidator

data: serverProperties

data: org.springframework.boot.autoconfigure.web.ErrorMvcAutoConfiguration

...

data: beanNameViewResolver

data: basicErrorController

data: org.springframework.boot.autoconfigure.orm.jpa.JpaBaseConfiguration$JpaWebConfiguration$JpaWebMvcConfiguration

This endpoint will stream in real-time (see also: Server-sent events with RxJava and SseEmitter) every single bean name being initialized. Having such an amazing tool we'll build more robust (reactive - there, I said it) loading.html page.

Fancy Progress Front-end

First we need to identify which Spring beans represent which subsystems, high-level components (or maybe even bounded contexts) in our system. I encoded this inside HTML using data-bean custom attribute:

<h2 data-bean="websocketContainerCustomizer" class="waiting">
    Web socket support
</h2>

<h2 data-bean="messageConverters" class="waiting">
    Spring MVC
</h2>

<h2 data-bean="metricFilter" class="waiting">
    Metrics
</h2>

<h2 data-bean="endpointMBeanExporter" class="waiting">
    Actuator
</h2>

<h2 data-bean="mongoTemplate" class="waiting">
    MongoDB
</h2>

<h2 data-bean="dataSource" class="waiting">
    Database
</h2>

<h2 data-bean="entityManagerFactory" class="waiting">
    Hibernate
</h2>

CSS class="waiting" means that a given module is not yet initialized, i.e. the given bean hasn't yet appeared in the SSE stream. Initially all components are in "waiting" state. I then subscribe to init.stream and changed the CSS class to reflect the module state changes:

var source = new EventSource('init.stream');
source.addEventListener('message', function (e) {
    var h2 = document.querySelector('h2[data-bean="' + e.data + '"]');
    if(h2) {
        h2.className = 'done';
    }
});

Simple, huh? Apparently one can write front-end without jQuery in pure JavaScript. When all beans are loaded,Observable is completed on the server side and SSE emits event: complete. Let's handle that:

source.addEventListener('complete', function (e) {
    window.location.reload();
});

Because front-end is notified on application context startup, we can simply reload the current page. At that point in time, ourProgressValve will have already deregistered itself, so reloading will open the true application, not the loading.html placeholder. Our job is done. Additionally, I count how many beans started and knowing how many beans are in total (I hardcoded it in JavaScript, forgive me), I can calculate startup progress as a percentage. A picture is worth a thousand words; let this screencast show you the result we achieved:

Subsequent modules are starting up nicely and we no longer look at a browser error. Progress measured in percentage makes the whole startup progress feels very smooth. Last but not least, when the application starts, we are automatically redirected. Hope you enjoyed this proof-of-concept. The whole working sample application is available on GitHub.