Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Python Logging: How to Log to Multiple Locations

DZone's Guide to

Python Logging: How to Log to Multiple Locations

· DevOps Zone
Free Resource

The Nexus Suite is uniquely architected for a DevOps native world and creates value early in the development pipeline, provides precise contextual controls at every phase, and accelerates DevOps innovation with automation you can trust. Read how in this ebook.

Today I decided to figure out how to make Python log to a file and the console simultaneously. Most of the time, I just want to log to a file, but occasionally I want to be able to see stuff on the console too to help with debugging. I found this ancient example in the Python documentation and ended up using it to mock up the following script:

import logging
 
#----------------------------------------------------------------------
def log(path, multipleLocs=False):
    """
    Log to multiple locations if multipleLocs is True
    """
    fmt_str = '%(asctime)s - %(name)s - %(message)s'
    formatter = logging.Formatter(fmt_str)
 
    logging.basicConfig(filename=path, level=logging.INFO,
                        format=fmt_str)
 
    if multipleLocs:
        console = logging.StreamHandler()
        console.setLevel(logging.INFO)
        console.setFormatter(formatter)
 
        logging.getLogger("").addHandler(console)
 
    logging.info("This is an informational message")
    try:
        1 / 0
    except ZeroDivisionError:
        logging.exception("You can't do that!")
 
    logging.critical("THIS IS A SHOW STOPPER!!!")
 
if __name__ == "__main__":
    log("sample.log") # log only to file
    log("sample2.log", multipleLocs=True) # log to file AND console!

As you can see, when you pass True to the second argument, the script will create an instance ofStreamHandler() which you can then configure and add to the current logger via the following call:

logging.getLogger("").addHandler(console)

This works great on Linux, but on Windows 7 the sample2.log wasn’t getting created, so I had to modify the if statement as follows:

if multipleLocs:
    console = logging.StreamHandler()
    console.setLevel(logging.INFO)
    console.setFormatter(formatter)
 
    fhandler = logging.FileHandler(path)
    fhandler.setFormatter(formatter)
 
    logging.getLogger("").addHandler(console)
    logging.getLogger("").addHandler(fhandler)

Now I should note that this causes a rather odd bug in that Python is somehow keeping track of the file names I write to across calls to my log function such that when I tell it to write to sample2.log, it write to it PLUS the original sample.log. To get around this, we have to create a logger instance with a unique name each time we call the script. Here’s an updated example that works correctly:

import logging
import os
 
#----------------------------------------------------------------------
def log(path, multipleLocs=False):
    """
    Log to multiple locations if multipleLocs is True
    """
    fname = os.path.splitext(path)[0]
    logger = logging.getLogger("Test_logger_%s" % fname)
    logger.setLevel(logging.INFO)
    fh = logging.FileHandler(path)
    formatter = logging.Formatter('%(asctime)s - %(name)s - %(message)s')
    fh.setFormatter(formatter)
    logger.addHandler(fh)
 
    if multipleLocs:
        console = logging.StreamHandler()
        console.setLevel(logging.INFO)
        console.setFormatter(formatter)
        logger.addHandler(console)
 
    logger.info("This is an informational message")
    try:
        1 / 0
    except ZeroDivisionError:
        logger.exception("You can't do that!")
 
    logger.critical("THIS IS A SHOW STOPPER!!!")
 
if __name__ == "__main__":
    log("sample.log") # log only to file
    log("sample2.log", multipleLocs=True) # log to file AND console!

You will note that this time we base the logger name on the file name of the log. The logging module is pretty slick and lots of fun to play around with. I hope you found that as interesting as I did.

The DevOps Zone is brought to you in partnership with Sonatype Nexus.  See how the Nexus platform infuses precise open source component intelligence into the DevOps pipeline early, everywhere, and at scale. Read how in this ebook

Topics:

Published at DZone with permission of Mike Driscoll, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}