DZone
Performance Zone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
  • Refcardz
  • Trend Reports
  • Webinars
  • Zones
  • |
    • Agile
    • AI
    • Big Data
    • Cloud
    • Database
    • DevOps
    • Integration
    • IoT
    • Java
    • Microservices
    • Open Source
    • Performance
    • Security
    • Web Dev
DZone > Performance Zone > What If Log Entries Had Structure?

What If Log Entries Had Structure?

Developers often use logging mechanisms, but often log messages are unstructured. Therefore, it's necessary to extract data from the logs. Here's a breakdown of strutured logging: what it is, and a simple use case.

Siva Prasad Rao Janapati user avatar by
Siva Prasad Rao Janapati
·
Mar. 09, 16 · Performance Zone · Analysis
Like (8)
Save
Tweet
4.94K Views

Join the DZone community and get the full member experience.

Join For Free

                                Image title

As developers, many of us come across with the logging mechanism where we use logging frameworks like apache log4j, commons logging, slf4j etc…We knew that log means “a stream of messages generated from running application.” The way we put the log statements in the code that should be understandable by humans. That is, the log messages which we get are “unstructured.” We have to use grep, awk, or scripts with regular expressions to get the intended information out of the log files. Most of the times we will include the log messages to understand the exceptions/errors from the system. But, there are more use cases where we can get more information from the logs. The use cases include user behavior, security auditing, analytics, and application monitoring (which will enable the user to get alerts of weird application behavior). There is another scenario where a human cannot go to each server instance and see the logs if we have 1000’s of server instances in the cluster. We should have a centralized log system where a machine can process the logs and provide us the insights.

To achieve the above-said functionality from the logs, we have to feed the log messages to a machine where the machine will understand the log messages and generates insights to the user. To make a machine understand the log messages we need to follow “structured logging.” For example, the unstructured logging will look like below.



log.info("User " + userName + " is logged in from ” + browser + “browser”);

The above log message is the concatenated string with the required variables. The output will be like “User xyz is logged in from Firefox browser.” It’s totally unstructured. Understanding it and providing insights by using a machine is trivial. The same unstructured log message can be converted into structured like below.



{“User logged in”,”username”:”xyz”,”browser”:”Firefox”}

In the above message “User logged in” is an event and the data is paired with a key value. The machine can segregate the events, keys and values and then generate the report.

For further reading on “Structured Logging” follow the below links:

http://dev.splunk.com/view/logging-best-practices/SP-CAAADP6

http://www.fluentd.org/architecture

Machine application Insight (email client) Event Log4j cluster Data (computing)

Published at DZone with permission of Siva Prasad Rao Janapati, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Which JVM Version Is the Fastest?
  • Debugging Deadlocks and Race Conditions
  • Portfolio Architecture Examples: Retail Collection
  • Why I'm Choosing Pulumi Over Terraform

Comments

Performance Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • MVB Program
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends:

DZone.com is powered by 

AnswerHub logo