From log4j to XpoLog

DZone 's Guide to

From log4j to XpoLog

Learn how to make the best use of those log4j logs using XpoLog.

· Performance Zone ·
Free Resource

XpoLog V6 is here and already taking on an Exabyte-sized storm of logs as I write this. In this series of posts I will cover some of the ways you can use and benefit from its new features and enhancements. I will concentrate mainly on how to get the maximum from your log4j event logs.

Once your log4j logs have been transferred to and properly defined in XpoLog Center, you can troubleshoot your java application by running Analytic Search on your log4j data, measure your application performance, create your own Apps or use XpoLog’s Apps for better monitoring, and create dashboards, charts, slide-shows, and make use of other visualization gadgets for maximum analysis.

You can download XpoLog for free here if you want to follow as you go along.

This post will show you how to transfer your log events to XpoLog using log4j. There are two ways of doing this. The first method is to allow XpoLog direct access to your files. The other method is by defining a SysLog appender and sending your events and messages to XpoLog. XpoLog supports both methods.

Allowing XpoLog access to your files (Pull)

Assuming you are already using log4j to write your log events to files, to allow for XpoLog to perform analytics on your log data, you need to give XpoLog access to these files. Define the name, pattern, and data pattern so that XpoLog can read these files, collect and index the data, and start analyzing.

Using Direct Access (Local or Remote)

XpoLog can access a local log file, i.e. a log file that resides on the same server as XpoLog. XpoLog can also access a log file on a remote server to which it has been provided direct access, as long as XpoLog is provided with the UNC path (\\hostname\dirname) to the log files on the remote server.

Using SSH (Secured Shell)

XpoLog can access log files on remote servers over SSH agent-less, provided that XpoLog has an account with a username and password or private/public key for connecting to the SSH server where the log files are situated.

Note that XpoLog requires Read permissions for any log it reads, regardless of the source of the log file.

To allow for XpoLog to pull (data from) the files, define the logger and give XpoLog access to the remote server where the logger is defined; then add the log to XpoLog.

For example:

#Logger definition




#Appender data for mylog






log4j.appender.mylog.layout.ConversionPattern=[%d] [%t] [%p] [%c] [%l] %m%n

Adding a Log to XpoLog:

  1. Inside XpoLog Center, go to Manager > Administration > Add Log. The Add Log screen opens.
  2. Give the log a name and a parent folder, and select an AppTag (Tag to Application(s)) from the drop-down list or create a new AppTag. You can select and create any number of AppTags for the same log. You do not have to tag the log at all, but in my forthcoming posts you will see how useful these AppTags can be. If you cannot wait for my next post, have a look at our “spoiler” here.
  3. Select the log type to be Local  and give a path (the screen capture below shows the example given)   Image title
  4. Click Next to view the sample text from the log, the conversion pattern in the Pattern Editor  field (toggle between Manual and Wizard to see how XpoLog reads the pattern) and the log records analysis results, or click Save.

Sending Your log4j Log Events to XpoLog (PUSH)

To send log events and log messages to XpoLog through SysLog, define a SysLog appender that uses the XpoLog server as the SysLog host. From inside XpoLog Center, define a TCP or a UDP SysLog Listener account and make sure the port (usually 1468 for TCP or 514 for UDP) is open on XpoLog’s machine. We recommend using TCP.

Defining a TCP SysLog Listener Account:

  1. Inside XpoLog Center, go to Manager > Administration > Listeners. A Listeners accounts console opens and presents all the configured listeners available.

  2. Click Syslog TCP. The Syslog TCP Account window opens.Image title

  3. Add a descriptive name for the Listener account, click Advanced Settings and continue. Note that for General Information > Enabled you make sure the account is infact enabled.

  4. Click Save. The data received from the Syslog listener account will be placed under the configured parent folder you selected.

Configuring log4j:

Now all you need to do is to make sure the SysLog events from your java application are sent to XpoLog. Configure log4j to use a SysLog appender. Here is an example configuration:

log4j.rootLogger=INFO, SYSLOG

log4j.appender.SYSLOG.layout.conversionPattern=%d{ISO8601} %-5p [%t] %c{2} %x - %m%n

After Your Logs Reach XpoLog

Once your log events have been pushed to or pulled by XpoLog, XpoLog can start collecting, parsing, monitoring, and analyzing all your log data. XpoLog V6 has enhanced its Analytic Search, added over 20 new visualization gadgets to its Apps, and also gives you the opportunity to create your own Apps and Dashboards, making performance monitoring, analysis, and visualization naturally fast and easy.

In my next post, I will cover how properly define and edit your java log patterns, thus paving the way for receiving the highest possible value from the XpoLog analysis. Stay tuned, or go directly to our hands-on-guide.

bigdata, devops, java, log analysis, log4j, performance

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}