DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Data Engineering
  3. Databases
  4. How to Set Logging Content for Tools like Splunk

How to Set Logging Content for Tools like Splunk

Mark O'Neill user avatar by
Mark O'Neill
·
Sep. 12, 12 · Interview
Like (0)
Save
Tweet
Share
5.33K Views

Join the DZone community and get the full member experience.

Join For Free
The Vordel API Server allows you to set the data which gets logged into its various log destinations: log files, database, syslog, etc. Very often, it's useful to log in a particular format for log-crunching tools such as Splunk to process. Here is how to do this:

In the "Settings" section of the API Server configuration in Policy Studio, there is an "audit log" tab. In the screenshot below, I'm setting the format of what gets put into the text logs. You can see that I've chosen it to be:

${timestamp},${id},'${text}'



 

So what does "${timestamp},${id},'${text}'" mean? It means that I'm logging the timestamp, the message ID (which is autogenerated for each message), and the text of each logging filter. I've chosen to separate them with a comma. So the next question is, what goes into ${text} ?

The answer can be seen if you look at any filter in a policy in Policy Studio, and click "Next". This is where you configure what gets logged. What gets logged is what goes into that ${text} value. In the example below, I have a filter which is logging the following upon the "Success" condition:
 
${authentication.subject.id} accessed the API
 
 
If you're doing authentication in your policy, then the value of ${authentication.subject.id} is automatically populated by the authenticated client's ID. Note that I am overriding the default logging settings (where only the "Failure" case is logged). I want the info in the "Success" case to go into the log. This is what I want to go into the ${text} part of ${timestamp},${id},'${text}'



 
The log files for the API Server v7 are written out to \groups\group-n\instance-m\logs\audit  , where n and m refer to the group which the API Server is in, and the API Server itself.

So now, when I look at the log file being written out, I see this:
 
# ProductName=apiserver1 7.0.0-2012-07-07 (Windows)
# CurrentDate=Tue, 11 Sep 2012 17:58:16 -0500
# CurrentDateUTC=1347400696
# TZ=Eastern Daylight Time

09.11.2012 22:59:11,884,Id-0599f187504fb42f112c0c5d,'JoeDeveloper accessed the API'
09.11.2012 22:59:11,916,Id-58e1fd41504fb42f06110000,'JaneDeveloper accessed the API'
09.11.2012 22:59:12,070,Id-ab634e7c504fb430152c0c5d,'JoeDeveloper accessed the API'
 
 
This is simple for a tool like Splunk to process.

While we're looking at the logging settings, let's look at the database logging. You can see in the screenshot below that I'm logging to the database:
 




The same content, ${authentication.subject.id} accessed the API , gets logged to the database. This can then be found in the Analytics of the API Server, in the Audit Trail. In the screenshot below, you can see the view of the Audit Trail, showing the entries. This is the same information as is written into the text log. In fact, if I also choose to write to Syslog (let's say) then the same information is written there too.




Clicking in on any of these entries, I can see the message ID, and the filter which wrote that log text. If many filters in my policy wrote out log data for the same message, then this is where I see the path through the policy, filter by filter.
 


I can also search the Audit Log here. Let's say I am only interested in the API accesses by "Joe Developer". Here in the screenshot below I construct a search looking for lines which contain "Joe":
 
 


So in summary, there is great flexibility in the Vordel API Server for both the format (e.g. comma-separated) and content (e.g. insert Client IDs) of logged data.
 
 
 
API

Published at DZone with permission of Mark O'Neill, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • The Data Leakage Nightmare in AI
  • Java Development Trends 2023
  • PostgreSQL: Bulk Loading Data With Node.js and Sequelize
  • Using JSON Web Encryption (JWE)

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: