Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Python Logging: How to Log to Multiple Locations

DZone's Guide to

Python Logging: How to Log to Multiple Locations

· DevOps Zone ·
Free Resource

DevOps involves integrating development, testing, deployment and release cycles into a collaborative process. Learn more about the 4 steps to an effective DevSecOps infrastructure.

Today I decided to figure out how to make Python log to a file and the console simultaneously. Most of the time, I just want to log to a file, but occasionally I want to be able to see stuff on the console too to help with debugging. I found this ancient example in the Python documentation and ended up using it to mock up the following script:

import logging
 
#----------------------------------------------------------------------
def log(path, multipleLocs=False):
    """
    Log to multiple locations if multipleLocs is True
    """
    fmt_str = '%(asctime)s - %(name)s - %(message)s'
    formatter = logging.Formatter(fmt_str)
 
    logging.basicConfig(filename=path, level=logging.INFO,
                        format=fmt_str)
 
    if multipleLocs:
        console = logging.StreamHandler()
        console.setLevel(logging.INFO)
        console.setFormatter(formatter)
 
        logging.getLogger("").addHandler(console)
 
    logging.info("This is an informational message")
    try:
        1 / 0
    except ZeroDivisionError:
        logging.exception("You can't do that!")
 
    logging.critical("THIS IS A SHOW STOPPER!!!")
 
if __name__ == "__main__":
    log("sample.log") # log only to file
    log("sample2.log", multipleLocs=True) # log to file AND console!

As you can see, when you pass True to the second argument, the script will create an instance ofStreamHandler() which you can then configure and add to the current logger via the following call:

logging.getLogger("").addHandler(console)

This works great on Linux, but on Windows 7 the sample2.log wasn’t getting created, so I had to modify the if statement as follows:

if multipleLocs:
    console = logging.StreamHandler()
    console.setLevel(logging.INFO)
    console.setFormatter(formatter)
 
    fhandler = logging.FileHandler(path)
    fhandler.setFormatter(formatter)
 
    logging.getLogger("").addHandler(console)
    logging.getLogger("").addHandler(fhandler)

Now I should note that this causes a rather odd bug in that Python is somehow keeping track of the file names I write to across calls to my log function such that when I tell it to write to sample2.log, it write to it PLUS the original sample.log. To get around this, we have to create a logger instance with a unique name each time we call the script. Here’s an updated example that works correctly:

import logging
import os
 
#----------------------------------------------------------------------
def log(path, multipleLocs=False):
    """
    Log to multiple locations if multipleLocs is True
    """
    fname = os.path.splitext(path)[0]
    logger = logging.getLogger("Test_logger_%s" % fname)
    logger.setLevel(logging.INFO)
    fh = logging.FileHandler(path)
    formatter = logging.Formatter('%(asctime)s - %(name)s - %(message)s')
    fh.setFormatter(formatter)
    logger.addHandler(fh)
 
    if multipleLocs:
        console = logging.StreamHandler()
        console.setLevel(logging.INFO)
        console.setFormatter(formatter)
        logger.addHandler(console)
 
    logger.info("This is an informational message")
    try:
        1 / 0
    except ZeroDivisionError:
        logger.exception("You can't do that!")
 
    logger.critical("THIS IS A SHOW STOPPER!!!")
 
if __name__ == "__main__":
    log("sample.log") # log only to file
    log("sample2.log", multipleLocs=True) # log to file AND console!

You will note that this time we base the logger name on the file name of the log. The logging module is pretty slick and lots of fun to play around with. I hope you found that as interesting as I did.

Read the 4-part DevOps testing eBook to learn how to detect problems earlier in your DevOps testing processes.

Topics:

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}