Migrating Spring Java Applications to Azure App Service (Part 2 - Logging and Monitoring)

As we demonstrated in Part 1 of the series, running on the cloud is not only for cool new applications following twelve-factor principles and coded to be cloud-native. In the first article, we demonstrated how to migrate a legacy Java Spring application to run on Azure App Service and how to address handling JNDI, credentials, and externalizing the configuration. In this article, we will show to enable logging, and monitoring to use Azure native capabilities. The full application example is available on Github.

Application Insights

Application Insights is an extensible Application Performance Management (APM) service helping to monitor web applications. It will automatically detect dependencies such as HTPP or JDBC calls and includes powerful analytics tools to help you diagnose performance issues. For more details, refer to Microsoft's Getting Started Guide.

• Enable App Insights in Azure App Service and get the instrumentation keys.

• To instrument Spring application add the Application Insights SDK to the list of dependencies:

 <dependency>
     <groupId>com.microsoft.azure</groupId>
     <artifactId>applicationinsights-web</artifactId>
     <!-- or applicationinsights-core for bare API -->
     <version>[2.1,)</version>
  </dependency>

• Add the SDK configuration file ApplicationInsights.xml to application resources, as shown in Github example application.

It will configure Telemetry Modules and Performance Counters that will be forwarded tothe  Azure Monitoring system.

• Define ApplicationInsights filter in web.xml   if the application is web-based.

   <filter>
       <filter-name>ApplicationInsightsWebFilter</filter-name>
       <filter-class>
           com.microsoft.applicationinsights.web.internal.WebRequestTrackingFilter
       </filter-class>
   </filter>
   <filter-mapping>
       <filter-name>ApplicationInsightsWebFilter</filter-name>
       <url-pattern>/*</url-pattern>
   </filter-mapping>
     <!-- This listener handles shutting down the TelemetryClient when an application/servlet is undeployed. -->
    <listener>
      <listener-class>com.microsoft.applicationinsights.web.internal.ApplicationInsightsServletContextListener</listener-class>
    </listener>

If Spring MVC is used in the web app, make sure to enable factory methods in the *-servlet.xml  file, in our example  mvx-dispatcher-servlet.xml :

  <context:component-scan base-package="tutorial, com.microsoft.applicationinsights.web.spring"/>

   <!-- This factory method causes ApplicationInsights.xml to get loaded -->
    <bean name="appinsightTelemetryConfiguration"
                 class="com.microsoft.applicationinsights.TelemetryConfiguration"
                 factory-method="getActive">
    </bean>

    <mvc:interceptors>
        <mvc:interceptor>
            <mvc:mapping path="/**"/>
            <bean class="com.microsoft.applicationinsights.web.spring.RequestNameHandlerInterceptorAdapter" />
        </mvc:interceptor>
    </mvc:interceptors>

Deploy the application using Maven as described in Part 1.

mvn clean package
mvn azure-webapp:deploy

Application Insights Views

Now you should be able to see the Live Metrics stream, showing the Request duration and Failures. 

JMX Performance Counters  (and any custom ones created with trackMetric) will be available in the Metrics view. Especially useful metrics such as Heap Memory that help identify memory leaks.

The Performance view will show pages and controllers that take a long time:

Dependency Tracking in Application Insights

To get deeper insights into the application without code changes, you could configure Application Insights Java Agent that will get data about calls such as HTTP, JDBC, and Redis and get information about execptions that are handled by code and hard to detect otherwise. For more details on Agent see here.

Installing Agent in our example is rather simple:

• Create a directory for agent appinsights  and download the same version of the agent as an SDK you have used in steps above.  Agent download is availble on Github. 

• Add file AI-Agent.xml   in the same directory that describes the Agent settings:

<ApplicationInsightsAgent>
    <Instrumentation>
        <!-- Collect remote dependency data -->
        <BuiltIn enabled="true">
            <!-- Disable Redis or alter threshold call duration above which arguments are sent.
                Defaults: enabled, 10000 ms -->
            <Jedis enabled="true" thresholdInMS="1000"/>
            <!-- Set SQL query duration above which query plan is reported (MySQL, PostgreSQL). Default is 10000 ms. -->
            <MaxStatementQueryLimitInMS>1000</MaxStatementQueryLimitInMS>
        </BuiltIn>

        <!-- Collect data about caught exceptions
             and method execution times -->
        <Class name="tutorial.UsersController">
            <Method name="createUser"
                    reportCaughtExceptions="true"
                    reportExecutionTime="true"
            />
        </Class>
    </Instrumentation>
</ApplicationInsightsAgent>

By default, our example's SQL calls will be detected and shown in dependency views. But we also want to see how long the call to method createUser takes and we are defining it in lines 14-19 as a dependency.

•  For the last step, we need to define JavaAgent  as part of the Java process bootstrapping:

 <appSettings>
   <property>
      <name>JAVA_OPTS</name>
      <value>-javaagent:D:\home\site\wwwroot\appinsights\applicationinsights-agent-2.1.2.jar -DSQLDB_URL=jdbc:sqlserver://<dbsrv>.database.windows.net:1433;database=<db>;encrypt=true;trustServerCertificate=false;hostNameInCertificate=*.database.windows.net;loginTimeout=30;</value>
  </property>              
</appSettings>

And we make sure to upload contents of the agent directory to App Service by modifying pom.xml  

 <resource>
    <!-- Where your artifacts are stored -->
    <directory>${project.basedir}/appinsights</directory>
    <!-- Relative path to /site/wwwroot/ -->
    <targetPath>appinsights</targetPath>
    <includes>
       <include>*</include>
     </includes>
 </resource>

Deploy the app and verify that Java agent directory was uploaded in App Service.

Java Agent Dependency Views

As a result, we can see in the Application Map the SQL dependency that was automatically tracked by agent, and the dependency we explicitly specified in  AI-Agent.xml <Class> definition. It helps to see the map and average times spent in dependencies. 

To further drill down into transactions, click on Investigate Performance button (or the Performance view), and drill down to see End-to-End transaction details. You will see the time taken by dependencies in the call:

These agent views will help to diagnose where the application spends most of the time.

Logging

Our example Spring application uses the Log4j logging framework to log required data and traces. 

<dependency>
     <groupId>org.apache.logging.log4j</groupId>
     <artifactId>log4j-api</artifactId>
     <version>2.11.1</version>
</dependency>
<dependency>
     <groupId>org.apache.logging.log4j</groupId>
     <artifactId>log4j-core</artifactId>
     <version>2.11.1</version>
</dependency>
 <dependency>
     <groupId>com.microsoft.azure</groupId>
     <artifactId>applicationinsights-logging-log4j2</artifactId>
     <version>[2.1,)</version>
  </dependency>

• It's common practice to forward logs to a central logging system (like ELK, Splunk, etc.). Azure Application Insights provides the ability to forward logs to Azure's central logging system, Log Analytics, with configurable retention periods and the ability to further filter and direct logs to other systems. A common example is to forward logs that are important for security review to SIEM systems.

• First, add AppInsights logging library as shown on lines 12-16.

• Define AppInsights appender that will forward logs to Azure Log Analytics:

log4j.appender.file=org.apache.log4j.RollingFileAppender
log4j.appender.file.File=D:\\home\\LogFiles\\logs.txt

log4j.appender.aiAppender=com.microsoft.applicationinsights.log4j.v2.ApplicationInsightsAppender

log4j.logger.tutorial=DEBUG, file, aiAppender
log4j.logger.com.microsoft=DEBUG, file, aiAppender

For more details on logging configuration refer to these Microsoft documents. As a result you could see all log statements in a Search view for the Log Analytics:

Summary

In this article, we have demonstrated how to take a legacy Java Spring application and enable monitoring with Azure's native performance monitor, Application Insights. We have also shown how to get the application logs forwarded to Azure's central logging system. The full code for this article if available on Github.