Over a million developers have joined DZone.

A Deferred Logging File Handler for Django

· DevOps Zone

The DevOps zone is brought to you in partnership with Sonatype Nexus. The Nexus suite helps scale your DevOps delivery with continuous component intelligence integrated into development tools, including Eclipse, IntelliJ, Jenkins, Bamboo, SonarQube and more. Schedule a demo today

At Tangent we handle environment-specific configuration of Django projects using the method outlined by David Cramer. This involves distinguishing between core settings (which we keep in core/default.py) and environment specific settings (eg core/stage.py, core/test.py). The standard settings.pymodule imports all defaults and then uses a enviromental shell variable to determine which environment settings module to import.

A problem

One tricky issue with this arrangement is logging to file. Ideally, we want to define a single LOGGING dict in the default settings but have file logging use an environment-specific folder. For example, logging to file in the test environment goes to /var/log/project/test/ while stage goes to a file in/var/log/project/stage.

One solution

This can be solved by using a string template for the filename argument to each FileHandler in the LOGGING setting:

# conf/default.py

LOGGING = {
    'version': 1,
    ...
    'handlers': {
        'error_file': {
            'level': 'INFO',
            'class': 'logging.FileHandler',
            'filename': '{log_root}errors.log',
            'formatter': 'verbose'
        },
    },
    'loggers': {
        'django.request': {
            'handlers': ['error_file'],
            'level': 'ERROR',
            'propagate': False,
        },
    }
}

then importing the default LOGGING dict into your environment-specific settings and formatting each filename with the correct path:

# conf/test.py

from conf.default import LOGGING

LOG_ROOT = '/var/log/project/test/'
for handler in LOGGING['handlers'].values():
    if handler['class'] == 'logging.FileHandler':
        handler['filename'] = handler['filename'].format(log_root=LOG_ROOT)

This works but is rather clunky. For instance, the default LOGGING setting (without an environmental override) will lead to an error .

Another solution

Another, possibly more elegant, solution is to use a specialisd logging handler that defers evaluation of the filepath until it tries to log a record.

from logging import FileHandler as BaseFileHandler
import os
class DeferredFileHandler(BaseFileHandler):

    def __init__(self, filename, *args, **kwargs):
        self.filename = filename
        kwargs['delay'] = True
        BaseFileHandler.__init__(self, "/dev/null", *args, **kwargs)

    def _open(self):
        # We import settings here to avoid a circular reference as this module
        # will be imported when settings.py is executed.
        from django.conf import settings
        self.baseFilename = os.path.join(settings.LOG_ROOT, self.filename)
        return BaseFileHandler._open(self)

Now, all we need to do is use the new handler in our LOGGING dict:

# conf/default.py

LOGGING = {
    'version': 1,
    ...
    'handlers': {
        'error_file': {
            'level': 'INFO',
            'class': 'deferred_filelogger.DeferredFilehandler',
            'filename': 'errors.log',
            'formatter': 'verbose'
        },
    },
    'loggers': {
        'django.request': {
            'handlers': ['error_file'],
            'level': 'ERROR',
            'propagate': False,
        },
    }
}

and specify a LOG_ROOT setting for each environment:

# conf/test.py

LOG_ROOT = '/var/log/project/test/'

Such a logger is part of django-oscar, but I've packaged it up separately so it can be used in non-Oscar projects. The package is called django-deferred-filelogger and can be installed from PyPI using:

$ pip install django-deferred-filelogger

The DevOps zone is brought to you in partnership with Sonatype Nexus. Use the Nexus Suite to automate your software supply chain and ensure you're using the highest quality open source components at every step of the development lifecycle. Get Nexus today

Topics:

Published at DZone with permission of David Winterbottom, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

SEE AN EXAMPLE
Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.
Subscribe

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}