5 Ways to Change Log Levels at Runtime
In this article, we'll walk through a critical aspect of logging—logging. Logs are important for any application debugging, so understanding them is important.
Join the DZone community and get the full member experience.Join For Free
Logs are very important for any application for debugging. Logs are the first and, many times, the last tool available to get it to the root cause of any issue.
In this blog, I am not going to discuss what to log and how to log effectively. But I will walk through the critical aspect of logging which is log levels. In every environment whether it is production/QA/Dev each one has particular log levels configured. In the production environment, log levels are strictly managed to make sure log file size is minimum.
Every typical day is not the same, bugs are unavoidable in software development. Very often a situation occurs where the current logs which are getting logged at a particular level are insufficient. So, need to arise to bump up the log levels.
In a poorly designed application where log files are inbuild (either in the jar or image), it all together needs new build to merely change the log levels. In decently design application log configuration is externalized. But still, to merely change the log levels it needs an application restart.
In terms of scaling, the challenge is to change log levels in each instance. So to avoid all pitfalls following is needed:
- Dynamically change the log level at runtime without application restart.
- Propagation of log level changes across the application instances.
Logging frameworks have undergone lots of changes in the past 5 years and we are going to leverage that. As everyone is moving towards microservice architecture we are inevitably going to adhere that and will use Spring Boot, sl4j, and Logback as our application stack.
The following are the options to achieve that. Usage is not only restricted to microservice architecture it can be applied in other styles of architecture too provided we are using the sl4j and Logback.
Option 1: Using the Spring Boot Admin
This is the best and most recommended option if you are using Spring Boot as everything comes out of the box. In this option, your application needs to enable the spring actuator and get it registered with Spring Boot Admin.
The user will use the spring boot admin UI to change the log levels.
Merely select the logger and change the log level in just click of a button. As highlighted in the figure it will change the log level in all running instances you no need to perform anything also.
Option 2: Scan the logback.xml
You can configure Logback to periodic scan and update the logging configuration via logback.xml
If you alter the log levels in the logback.xml file then changes will get effect max after 15 seconds. To propagate the log level changes across all container instance at once follow the below architecture:
Over here you externalize the logback.xml file and make it available to the container via NFS mount point. Pass the log file path to the container via JVM argument: ” -Dlogging.config=/logpath/logback.xml”.
Option -3 Access logback.xml via URL
In this option you need to expose the logback.xml via URL with the help of nginx/apache/HAProxy etc. and the application will consume the logback.xml via JVM arg: “-Dlogging.config = http://host:9999/logback.xml” Woefully, the auto-scan doesn’t work if you expose the logback.xml via URL. So to achieve that we need some polling mechanism.
Log level changes propagation:
Option 4: Creating API To Change the Log Levels
Another way is to change the log level programmatically by exposing the API.
To propagate the log level changes across the instances we need to follow one of the methods below:
We need a custom utility that will expose the API to change the log level. The utility will fetch the instance/s IP address and execute the API call on each running APP instance.
Option 5: Pub / Sub
In this option, the application instance needs to subscribe to the topic where log level changes will get published.
As per me, this is termed over-engineering if you are doing for just log level changes. But this can be very effective in microservice architecture if you want to change the log levels of all applications in one shot.
There are other ways to expose and load the log configuration via FTP, socket, etc. but the approach is the same. Which approach to take is utterly dependent on the deployment topology, application architecture style, and need.
The above represent the approaches but securing the log configuration and auditing the changes are out of the scope of the blog. I am assuming you will take care of your respective environment.
Hope you enjoy the post. Feel free to click the like button.
Entire code you can get here.
Published at DZone with permission of Milind Deobhankar. See the original article here.
Opinions expressed by DZone contributors are their own.