Python logging and rotating files

I have a python program that is writing to a log file that is being rotated by Linux's logrotate command. When this happens I need to signal my program to stop writing to the old file and start writing to the new one. I can handle the signal but how do I tell python to write to the new file?

I am opening the file like this:

logging.basicConfig(format='%(asctime)s:%(filename)s:%(levelname)s:%(message)s',filename=log_file, level=logging.INFO) 

and writing to it like this:

logging.log(level,"%s" % (msg)) 

The logging modules look very powerful but also overwhelming. Thanks.

4 Answers

Don't use logging.basicConfig, use WatchedFileHandler. Here's how to use it.

import time import logging import logging.handlers def log_setup(): log_handler = logging.handlers.WatchedFileHandler('my.log') formatter = logging.Formatter( '%(asctime)s program_name [%(process)d]: %(message)s', '%b %d %H:%M:%S') formatter.converter = time.gmtime # if you want UTC time log_handler.setFormatter(formatter) logger = logging.getLogger() logger.addHandler(log_handler) logger.setLevel(logging.DEBUG) log_setup() logging.info('Hello, World!') import os os.rename('my.log', 'my.log-old') logging.info('Hello, New World!') 

You may want to look at WatchedFileHandler to implement this, or as an alternative, implement log rotation with RotatingFileHandler, both of which are in the logging.handlers module.

from logging import handlers handler = handlers.TimedRotatingFileHandler(filename, when=LOG_ROTATE) handler.setFormatter(logging.Formatter(log_format, datefmt="%d-%m-%Y %H:%M:%S")) #LOG_ROTATE = midnight #set your log format 

This should help you in handling rotating log

Since rotation is already being done by logrotate, in your signal handler you should just call logging.basicConfig(...) again and that should reopen the log file.