Integrate Log4j Nagios Appender Into Apache Karaf / ServiceMix

This is perhaps a post for a rather narrow audience.

As the ServiceMix/Karaf users know, the Pax Logging framework used in Karaf integrates nicely with Apache Log4J out of the box. In fact the Karaf logging configuration file located in etc/org.ops4j.pax.logging.cfg uses Log4J configuration syntax.

Log4J integration comes out of the box in Karad, the Pax Logging Service OSGi bundle includes the core Log4J classes, so that the default Log4J appenders are supported. 

Now there are loads of additional appenders for Log4J available (a non-exhaustive list is here). One such appender is the Log4J Nagios appender, which pushes logging messages to Nagios (via an NSCA server).

I have never used Nagios before but had the task to get this Nagios appender working in Fuse ESB Enterprise 7.0.2. Fuse ESB is based on Apache ServiceMix, so the outlined solution applies to ServiceMix 4.x and Karaf as well.

As things were not dead-simple I take the time to document my solution so that it can hopefully save someone else's time.

To install and setup Nagios, I fired up a Ubuntu VM and installed the nagios and nsca packages using the Ubuntu package manager. With regards to configuring Nagios I followed this article and this document explaining the NSCA server setup. Although I had no previous knowledge on Nagios, I got it setup and running within an hour thanks to the referenced articles.

Turning back to ServiceMix/Karaf. The Nagios Log4J appender comes as a plain jar file (not OSGi enabled).  A possible Nagios Log4j configuration is given at the end of this post.

The problem is that logging is performed by the Pax Logging Service in Karaf. So how do you tell the pax-logging-service system bundle that it should also load the Nagios Log4J appender from a different jar file deployed into Karaf?
There is probably other ways to resolve this but I found it easiest to use the OSGi fragment bundle concept.

In OSGi there is the concept of a fragment bundle. From the OSGi Wiki:
"A Bundle fragment, or simply a fragment, is a bundle whose contents are made available to another bundle (the fragment host). Importantly, fragments share the classloader of their parent bundle."

Its important to note that fragments use the classloader of their parent or host bundle. 
By using a fragment bundle you can extend the classes that can be loaded by the host bundle, without having to modify the OSGi Import-Package list.

With respect to my use case this means: By making the Nagios Lo4J jar file a fragment bundle of the Pax Logging Service bundle, the Pax Logging Service bundle will be able to load the Nagios Log4J appender classes and send logging statements to the Nagios NSCA server.

The Nagios Log4J jar does not contain any OSGi metadata, so I had to manually add these. I extracted the jar file and modified META-INF/MANIFEST.MF to contain these headers.

Manifest-Version: 1.0
Ant-Version: Apache Ant 1.8.2
Created-By: 1.6.0_13-b03 (Sun Microsystems Inc.)
Bundle-Name: log4j-nagios
Bundle-SymbolicName: org.apache.log4j.nagios
Bundle-Version: 2.0.0
Bundle-ManifestVersion: 2
Fragment-Host: org.ops4j.pax.logging.pax-logging-service
Export-Package: org.apache.log4j.nagios;version="2.0.0"

Notice the Fragment-Host header, it sets the host to the pax-logging-service OSGi bundle.
Further there is no need to define an Import-Package list as all required Log4J classes will be made available by the host bundle.

I then rebuild the jar file and named it log4j-nagios-appender-2.0.0.osgi.jar.
If you don't want to run these steps manually yourself, you can download the OSGi enabled jar file using the above link.

Deploying this new jar is easy. The perhaps simplest form is to start with a fresh container (having no or an empty data/ folder).
Assuming etc/org.ops4j.pax.logging.cfg already configures for Nagios logging (see example config below) you can simply copy log4j-nagios-appender-2.0.0.osgi.jar to the ServiceMix deploy/ folder and startup ServiceMix.

It may raise the following exception on the first startup

java.lang.ClassNotFoundException: org.apache.log4j.nagios.NagiosAppender not found 

by org.ops4j.pax.logging.pax-logging-service [3]

But you can ignore that. Because the pax-logging-service bundle was started before the fragment Nagios Log4J bundl, Pax Logging is not able to load the Nagios appender right at startup. However when the Nagios Log4J fragment bundle attaches to the pax-logging-service, the Nagios appender classes will get loaded and logging via that appender will start. Messages will get pushed to the NSCA server.
On subsequent restarts of Karaf the bundles are already wired together (i.e. the pax-logging-service knows there is a fragment bundle), so this exception will not be raised anymore.

Hope this helps.

Example org.ops4j.pax.logging.cfg configuration using Nagios appender:

################################################################################
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#
################################################################################

# Root logger
log4j.rootLogger=INFO, out, osgi:* , NAGIOS
log4j.throwableRenderer=org.apache.log4j.OsgiThrowableRenderer

# CONSOLE appender not used by default
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{ISO8601} | %-5.5p | %-16.16t | %-32.32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n

# File appender
log4j.appender.out=org.apache.log4j.RollingFileAppender
log4j.appender.out.layout=org.apache.log4j.PatternLayout
log4j.appender.out.layout.ConversionPattern=%d{ISO8601} | %-5.5p | %-16.16t | %-32.32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
log4j.appender.out.file=${karaf.data}/log/karaf.log
log4j.appender.out.append=true
log4j.appender.out.maxFileSize=1MB
log4j.appender.out.maxBackupIndex=10

# Sift appender
log4j.appender.sift=org.apache.log4j.sift.MDCSiftingAppender
log4j.appender.sift.key=bundle.name
log4j.appender.sift.default=karaf
log4j.appender.sift.appender=org.apache.log4j.FileAppender
log4j.appender.sift.appender.layout=org.apache.log4j.PatternLayout
log4j.appender.sift.appender.layout.ConversionPattern=%d{ISO8601} | %-5.5p | %-16.16t | %-32.32c{1} | %m%n
log4j.appender.sift.appender.file=${karaf.data}/log/$\\{bundle.name\\}.log
log4j.appender.sift.appender.append=true


# Nagios Log4J configuration
# ------------------------------------------------------------
# set the appender for Nagios
log4j.appender.NAGIOS=org.apache.log4j.nagios.NagiosAppender

# Nagios configurations
log4j.appender.NAGIOS.Host=192.168.178.44
log4j.appender.NAGIOS.Port=5667
log4j.appender.NAGIOS.ServiceNameDefault=FuseESB
log4j.appender.NAGIOS.MDCCanonicalHostNameKey=nagios_canonical_hostname


# It may be required to set a Nagios config file if non-default
# data encryption algorithms are used.
log4j.appender.NAGIOS.ConfigFile=/opt/fuse/SMX/fuse-esb-7.0.2.fuse-097/send_nsca.cfg

# mapping warning levels.
log4j.appender.NAGIOS.Log4j_Level_INFO=NAGIOS_OK
log4j.appender.NAGIOS.Log4j_Level_WARN=NAGIOS_WARN
log4j.appender.NAGIOS.Log4j_Level_ERROR=NAGIOS_CRITICAL
log4j.appender.NAGIOS.Log4j_Level_FATAL=NAGIOS_CRITICAL

# set the layout for appender Nagios
log4j.appender.NAGIOS.layout=org.apache.log4j.PatternLayout
log4j.appender.NAGIOS.layout.conversionPattern=server: %X{nagios_canonical_hostname}: %m%n