Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Python Logging: How to Log to Multiple Locations

DZone's Guide to

Python Logging: How to Log to Multiple Locations

· DevOps Zone
Free Resource

Download “The DevOps Journey - From Waterfall to Continuous Delivery” to learn learn about the importance of integrating automated testing into the DevOps workflow, brought to you in partnership with Sauce Labs.

Today I decided to figure out how to make Python log to a file and the console simultaneously. Most of the time, I just want to log to a file, but occasionally I want to be able to see stuff on the console too to help with debugging. I found this ancient example in the Python documentation and ended up using it to mock up the following script:

import logging
 
#----------------------------------------------------------------------
def log(path, multipleLocs=False):
    """
    Log to multiple locations if multipleLocs is True
    """
    fmt_str = '%(asctime)s - %(name)s - %(message)s'
    formatter = logging.Formatter(fmt_str)
 
    logging.basicConfig(filename=path, level=logging.INFO,
                        format=fmt_str)
 
    if multipleLocs:
        console = logging.StreamHandler()
        console.setLevel(logging.INFO)
        console.setFormatter(formatter)
 
        logging.getLogger("").addHandler(console)
 
    logging.info("This is an informational message")
    try:
        1 / 0
    except ZeroDivisionError:
        logging.exception("You can't do that!")
 
    logging.critical("THIS IS A SHOW STOPPER!!!")
 
if __name__ == "__main__":
    log("sample.log") # log only to file
    log("sample2.log", multipleLocs=True) # log to file AND console!

As you can see, when you pass True to the second argument, the script will create an instance ofStreamHandler() which you can then configure and add to the current logger via the following call:

logging.getLogger("").addHandler(console)

This works great on Linux, but on Windows 7 the sample2.log wasn’t getting created, so I had to modify the if statement as follows:

if multipleLocs:
    console = logging.StreamHandler()
    console.setLevel(logging.INFO)
    console.setFormatter(formatter)
 
    fhandler = logging.FileHandler(path)
    fhandler.setFormatter(formatter)
 
    logging.getLogger("").addHandler(console)
    logging.getLogger("").addHandler(fhandler)

Now I should note that this causes a rather odd bug in that Python is somehow keeping track of the file names I write to across calls to my log function such that when I tell it to write to sample2.log, it write to it PLUS the original sample.log. To get around this, we have to create a logger instance with a unique name each time we call the script. Here’s an updated example that works correctly:

import logging
import os
 
#----------------------------------------------------------------------
def log(path, multipleLocs=False):
    """
    Log to multiple locations if multipleLocs is True
    """
    fname = os.path.splitext(path)[0]
    logger = logging.getLogger("Test_logger_%s" % fname)
    logger.setLevel(logging.INFO)
    fh = logging.FileHandler(path)
    formatter = logging.Formatter('%(asctime)s - %(name)s - %(message)s')
    fh.setFormatter(formatter)
    logger.addHandler(fh)
 
    if multipleLocs:
        console = logging.StreamHandler()
        console.setLevel(logging.INFO)
        console.setFormatter(formatter)
        logger.addHandler(console)
 
    logger.info("This is an informational message")
    try:
        1 / 0
    except ZeroDivisionError:
        logger.exception("You can't do that!")
 
    logger.critical("THIS IS A SHOW STOPPER!!!")
 
if __name__ == "__main__":
    log("sample.log") # log only to file
    log("sample2.log", multipleLocs=True) # log to file AND console!

You will note that this time we base the logger name on the file name of the log. The logging module is pretty slick and lots of fun to play around with. I hope you found that as interesting as I did.

Discover how to optimize your DevOps workflows with our cloud-based automated testing infrastructure, brought to you in partnership with Sauce Labs

Topics:

Published at DZone with permission of Mike Driscoll, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}