DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
View Events Video Library
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Integrating PostgreSQL Databases with ANF: Join this workshop to learn how to create a PostgreSQL server using Instaclustr’s managed service

Mobile Database Essentials: Assess data needs, storage requirements, and more when leveraging databases for cloud and edge applications.

Monitoring and Observability for LLMs: Datadog and Google Cloud discuss how to achieve optimal AI model performance.

Automated Testing: The latest on architecture, TDD, and the benefits of AI and low-code tools.

Related

  • The Generic Way To Convert Between Java and PostgreSQL Enums
  • Testcontainers With Kotlin and Spring Data R2DBC
  • Accepting Crypto Payments in a Classic Commerce App
  • Migrating from Sakila-MySQL to Couchbase - Part 3: Stored Procedures

Trending

  • Java Parallel GC Tuning
  • Agile Metrics and KPIs in Action
  • How To Verify Database Connection From a Spring Boot Application
  • Best Plugins For JetBrains IDEs
  1. DZone
  2. Data Engineering
  3. Databases
  4. Log2j - How to Convert a Log File in POJOs Using Annotation

Log2j - How to Convert a Log File in POJOs Using Annotation

Decebal Suiu user avatar by
Decebal Suiu
·
Jul. 03, 13 · Interview
Like (0)
Save
Tweet
Share
7.69K Views

Join the DZone community and get the full member experience.

Join For Free
Recently I had a job to create some statistics on interactions between our visitors and our landing page. When a visitor downloads our product, requires a license or buys the product, a new line is inserted in the log file of the web server.

All I have is a big log file with a lot of information.

For example DownloadServlet for each download request writes in the container's log a line in the format
[webapp 2008/10/06 16:12:16] - 192.168.12.124, /download/next-reports-setup-1.7-jre.exe
What I want is to see some statistics about downloads. 

My idea is to convert the log file in a collection of business objects (Download in my case) and to make some query using josql  or cqengine  and extract the required information. I can also insert all downloads in a database table and use sql to extract the required information. 

With Log2j  I can convert a log's lines in java objects with a single line:
new Log2j()
    .map(Download.class)
    .addEntityHandler(new DownloadHandler())
    .parse(new InputStreamReader(input));
What is Log2j?

Log2j is an open source (Apache license) tiny (around 15KB) log file to POJOs converter, with zero dependencies and a quick learning curve. No XML, only Java.

My first step is to create a simple POJO class with the name Download having some properties.
@RegexEntity(pattern = "PATTERN")
public class Download {

    // [webapp 2008/10/06 16:12:16] - 192.168.12.124, /download/next-report-setup-1.7-jre.exe, f13dfc7fe609480297a0b15d611676b4    

    public static final String PATTERN = "\\[webapp\\s"
        + "(20[0-1][0-9]/\\d{2}/\\d{2}\\s\\d{2}:\\d{2}:\\d{2})" // date
        + "\\]\\s-\\s<\\$>\\s"
        + "(\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3})" // ip
        + ",\\s/download/"
        + "([^,]*)" // file 
        + ".*";

    @RegexField(group = 1, converter = MyDateConverter.class)
    private Date date;

    @RegexField(group = 3)
    private String ip;

    @RegexField(group = 4)
    private String file;

    // getters and setters

}
I added annotation @RegexEntity(pattern = "PATTERN") to class Download. The @RegexEntity annotation may have a parameter named pattern which represents the constant name that returns the regex string. 

The default value is PATTERN (in Download class I added pattern value for transparency). 


This annotation is informing Log2j that all log's lines that respect the pattern will be transformed in Download objects.

The second step is to specify how Log2j will transform text fragments from log's lines in Download's properties. 


For this purpose I used @RegexField annotation. This annotation takes a mandatory parameter named group that represents the group index from the java's regex Matcher object and an optional parameter named converter that will be used by Log2j to convert the text fragment in property's value.
The third step is to create a DownloadHandler for handling Download objects.
public class DownloadHandler implements EntityHandler<Download> {

    private int count;

    @Override
    public void beforeFirstEntity() {
        count = 0;
    }

    @Override
    public void handleEntity(Download entity) {
        count++;
        // only display the entity 
        System.out.println(entity);    
    }

    @Override
    public void afterLastEntity() {
        System.out.println("Handled " + count + " Download entities");
    }
}

The methods beforeFirstEntity() and afterLastEntity() are callback methods. 

Log2j will call these methods one time at the start/end of log parsing. 

For example you can start a database transaction, clear an entity table in beforeFirstEntity() and commit the database transaction in afterLastEntity().

In the example above my DownloadHandler init count variable to zero in beforeFirstEntity(), prints all download objects to System.out in handleEntity() and prints a count with download entities in afterLastEntity().

Another important concept in Log2j is Converter.


A Converter is used by Log2j to transform a text fragment into a POJO property's value. Log2j comes with builtin converters for all primitive values (Boolean, Byte, Short, ...). 

Log2j also allows you to register new general converters (for each field's type) or you can register a converter only for a particular field.
public class MyDateConverter implements Converter<Date> {

    @Override
    public Date convert(String text) {
        try {
            return new SimpleDateFormat("yyyy/MM/dd").parse(text);
        } catch (ParseException e) {
            e.printStackTrace();
        }

        return null;
    }

}
For example MyDateConverter is used in Download class by Log2j to transform the text fragment in Date object. If you want to use MyDateConverter for all POJO fields with type Date you can do it with:
new Log2j()
    ...
    registerConverter(new MyDateConverter();
In my next article about Log2j I will talk about validations and how you can interrogate the collection of POJOs.
Annotation Convert (command) Database Download

Opinions expressed by DZone contributors are their own.

Related

  • The Generic Way To Convert Between Java and PostgreSQL Enums
  • Testcontainers With Kotlin and Spring Data R2DBC
  • Accepting Crypto Payments in a Classic Commerce App
  • Migrating from Sakila-MySQL to Couchbase - Part 3: Stored Procedures

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends: