PageRenderTime 261ms CodeModel.GetById 156ms app.highlight 4ms RepoModel.GetById 73ms app.codeStats 23ms

/docs/topics/logging.txt

https://code.google.com/p/mango-py/
Plain Text | 543 lines | 411 code | 132 blank | 0 comment | 0 complexity | a92f803255c65eba4ee6d7b94e0a098f MD5 | raw file
  1=======
  2Logging
  3=======
  4
  5.. versionadded:: 1.3
  6
  7.. module:: django.utils.log
  8   :synopsis: Logging tools for Django applications
  9
 10A quick logging primer
 11======================
 12
 13Django uses Python's builtin logging module to perform system logging.
 14The usage of the logging module is discussed in detail in `Python's
 15own documentation`_. However, if you've never used Python's logging
 16framework (or even if you have), here's a quick primer.
 17
 18.. _Python's own documentation: http://docs.python.org/library/logging.html
 19
 20The cast of players
 21-------------------
 22
 23A Python logging configuration consists of four parts:
 24
 25    * :ref:`topic-logging-parts-loggers`
 26    * :ref:`topic-logging-parts-handlers`
 27    * :ref:`topic-logging-parts-filters`
 28    * :ref:`topic-logging-parts-formatters`
 29
 30.. _topic-logging-parts-loggers:
 31
 32Loggers
 33~~~~~~~
 34
 35A logger is the entry point into the logging system. Each logger is
 36a named bucket to which messages can be written for processing.
 37
 38A logger is configured to have a *log level*. This log level describes
 39the severity of the messages that the logger will handle. Python
 40defines the following log levels:
 41
 42    * ``DEBUG``: Low level system information for debugging purposes
 43
 44    * ``INFO``: General system information
 45
 46    * ``WARNING``: Information describing a minor problem that has
 47      occurred.
 48
 49    * ``ERROR``: Information describing a major problem that has
 50      occurred.
 51
 52    * ``CRITICAL``: Information describing a critical problem that has
 53      occurred.
 54
 55Each message that is written to the logger is a *Log Record*. Each log
 56record also has a *log level* indicating the severity of that specific
 57message. A log record can also contain useful metadata that describes
 58the event that is being logged. This can include details such as a
 59stack trace or an error code.
 60
 61When a message is given to the logger, the log level of the message is
 62compared to the log level of the logger. If the log level of the
 63message meets or exceeds the log level of the logger itself, the
 64message will undergo further processing. If it doesn't, the message
 65will be ignored.
 66
 67Once a logger has determined that a message needs to be processed,
 68it is passed to a *Handler*.
 69
 70.. _topic-logging-parts-handlers:
 71
 72Handlers
 73~~~~~~~~
 74
 75The handler is the engine that determines what happens to each message
 76in a logger. It describes a particular logging behavior, such as
 77writing a message to the screen, to a file, or to a network socket.
 78
 79Like loggers, handlers also have a log level. If the log level of a
 80log record doesn't meet or exceed the level of the handler, the
 81handler will ignore the message.
 82
 83A logger can have multiple handlers, and each handler can have a
 84different log level. In this way, it is possible to provide different
 85forms of notification depending on the importance of a message. For
 86example, you could install one handler that forwards ``ERROR`` and
 87``CRITICAL`` messages to a paging service, while a second handler
 88logs all messages (including ``ERROR`` and ``CRITICAL`` messages) to a
 89file for later analysis.
 90
 91.. _topic-logging-parts-filters:
 92
 93Filters
 94~~~~~~~
 95
 96A filter is used to provide additional control over which log records
 97are passed from logger to handler.
 98
 99By default, any log message that meets log level requirements will be
100handled. However, by installing a filter, you can place additional
101criteria on the logging process. For example, you could install a
102filter that only allows ``ERROR`` messages from a particular source to
103be emitted.
104
105Filters can also be used to modify the logging record prior to being
106emitted. For example, you could write a filter that downgrades
107``ERROR`` log records to ``WARNING`` records if a particular set of
108criteria are met.
109
110Filters can be installed on loggers or on handlers; multiple filters
111can be used in a chain to perform multiple filtering actions.
112
113.. _topic-logging-parts-formatters:
114
115Formatters
116~~~~~~~~~~
117
118Ultimately, a log record needs to be rendered as text. Formatters
119describe the exact format of that text. A formatter usually consists
120of a Python formatting string; however, you can also write custom
121formatters to implement specific formatting behavior.
122
123Using logging
124=============
125
126Once you have configured your loggers, handlers, filters and
127formatters, you need to place logging calls into your code. Using the
128logging framework is very simple. Here's an example::
129
130    # import the logging library
131    import logging
132
133    # Get an instance of a logger
134    logger = logging.getLogger(__name__)
135
136    def my_view(request, arg1, arg):
137        ...
138        if bad_mojo:
139            # Log an error message
140            logger.error('Something went wrong!')
141
142And that's it! Every time the ``bad_mojo`` condition is activated, an
143error log record will be written.
144
145Naming loggers
146--------------
147
148The call to :meth:`logging.getLogger()` obtains (creating, if
149necessary) an instance of a logger. The logger instance is identified
150by a name. This name is used to identify the logger for configuration
151purposes.
152
153By convention, the logger name is usually ``__name__``, the name of
154the python module that contains the logger. This allows you to filter
155and handle logging calls on a per-module basis. However, if you have
156some other way of organizing your logging messages, you can provide
157any dot-separated name to identify your logger::
158
159    # Get an instance of a specific named logger
160    logger = logging.getLogger('project.interesting.stuff')
161
162The dotted paths of logger names define a hierarchy. The
163``project.interesting`` logger is considered to be a parent of the
164``project.interesting.stuff`` logger; the ``project`` logger
165is a parent of the ``project.interesting`` logger.
166
167Why is the hierarchy important? Well, because loggers can be set to
168*propagate* their logging calls to their parents. In this way, you can
169define a single set of handlers at the root of a logger tree, and
170capture all logging calls in the subtree of loggers. A logging handler
171defined in the ``project`` namespace will catch all logging messages
172issued on the ``project.interesting`` and
173``project.interesting.stuff`` loggers.
174
175This propagation can be controlled on a per-logger basis. If
176you don't want a particular logger to propagate to it's parents, you
177can turn off this behavior.
178
179Making logging calls
180--------------------
181
182The logger instance contains an entry method for each of the default
183log levels:
184
185    * ``logger.critical()``
186    * ``logger.error()``
187    * ``logger.warning()``
188    * ``logger.info()``
189    * ``logger.debug()``
190
191There are two other logging calls available:
192
193    * ``logger.log()``: Manually emits a logging message with a
194      specific log level.
195
196    * ``logger.exception()``: Creates an ``ERROR`` level logging
197      message wrapping the current exception stack frame.
198
199Configuring logging
200===================
201
202Of course, it isn't enough to just put logging calls into your code.
203You also need to configure the loggers, handlers, filters and
204formatters to ensure that logging output is output in a useful way.
205
206Python's logging library provides several techniques to configure
207logging, ranging from a programmatic interface to configuration files.
208By default, Django uses the `dictConfig format`_.
209
210.. note::
211    ``logging.dictConfig`` is a builtin library in Python 2.7. In
212    order to make this library available for users of earlier Python
213    versions, Django includes a copy as part of ``django.utils.log``.
214    If you have Python 2.7, the system native library will be used; if
215    you have Python 2.6 or earlier, Django's copy will be used.
216
217In order to configure logging, you use :setting:`LOGGING` to define a
218dictionary of logging settings. These settings describes the loggers,
219handlers, filters and formatters that you want in your logging setup,
220and the log levels and other properties that you want those components
221to have.
222
223Logging is configured immediately after settings have been loaded.
224Since the loading of settings is one of the first things that Django
225does, you can be certain that loggers are always ready for use in your
226project code.
227
228.. _dictConfig format: http://docs.python.org/library/logging.config.html#configuration-dictionary-schema
229
230.. _a third-party library: http://bitbucket.org/vinay.sajip/dictconfig
231
232An example
233----------
234
235The full documentation for `dictConfig format`_ is the best source of
236information about logging configuration dictionaries. However, to give
237you a taste of what is possible, here is an example of a fairly
238complex logging setup, configured using :meth:`logging.dictConfig`::
239
240    LOGGING = {
241        'version': 1,
242        'disable_existing_loggers': True,
243        'formatters': {
244            'verbose': {
245                'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
246            },
247            'simple': {
248                'format': '%(levelname)s %(message)s'
249            },
250        },
251        'filters': {
252            'special': {
253                '()': 'project.logging.SpecialFilter',
254                'foo': 'bar',
255            }
256        },
257        'handlers': {
258            'null': {
259                'level':'DEBUG',
260                'class':'django.utils.log.NullHandler',
261            },
262            'console':{
263                'level':'DEBUG',
264                'class':'logging.StreamHandler',
265                'formatter': 'simple'
266            },
267            'mail_admins': {
268                'level': 'ERROR',
269                'class': 'django.utils.log.AdminEmailHandler',
270                'filters': ['special']
271            }
272        },
273        'loggers': {
274            'django': {
275                'handlers':['null'],
276                'propagate': True,
277                'level':'INFO',
278            },
279            'django.request': {
280                'handlers': ['mail_admins'],
281                'level': 'ERROR',
282                'propagate': False,
283            },
284            'myproject.custom': {
285                'handlers': ['console', 'mail_admins'],
286                'level': 'INFO',
287                'filters': ['special']
288            }
289        }
290    }
291
292This logging configuration does the following things:
293
294    * Identifies the configuration as being in 'dictConfig version 1'
295      format. At present, this is the only dictConfig format version.
296
297    * Disables all existing logging configurations.
298
299    * Defines two formatters:
300
301        * ``simple``, that just outputs the log level name (e.g.,
302          ``DEBUG``) and the log message.
303
304          The `format` string is a normal Python formatting string
305          describing the details that are to be output on each logging
306          line. The full list of detail that can be output can be
307          found in the `formatter documentation`_.
308
309        * ``verbose``, that outputs the log level name, the log
310          message, plus the time, process, thread and module that
311          generate the log message.
312
313
314    * Defines one filter -- :class:`project.logging.SpecialFilter`,
315      using the alias ``special``. If this filter required additional
316      arguments at time of construction, they can be provided as
317      additional keys in the filter configuration dictionary. In this
318      case, the argument ``foo`` will be given a value of ``bar`` when
319      instantiating the :class:`SpecialFilter`.
320
321    * Defines three handlers:
322
323        * ``null``, a NullHandler, which will pass any ``DEBUG`` or
324          higher message to ``/dev/null``.
325
326        * ``console``, a StreamHandler, which will print any ``DEBUG``
327          message to stdout. This handler uses the `simple` output
328          format.
329
330        * ``mail_admins``, an AdminEmailHandler, which will e-mail any
331          ``ERROR`` level message to the site admins. This handler uses
332          the ``special`` filter.
333
334    * Configures three loggers:
335
336        * ``django``, which passes all messages at ``INFO`` or higher
337          to the ``null`` handler.
338
339        * ``django.request``, which passes all ``ERROR`` messages to
340          the ``mail_admins`` handler. In addition, this logger is
341          marked to *not* propagate messages. This means that log
342          messages written to ``django.request`` will not be handled
343          by the ``django`` logger.
344
345        * ``myproject.custom``, which passes all messages at ``INFO``
346          or higher that also pass the ``special`` filter to two
347          handlers -- the ``console``, and ``mail_admins``. This
348          means that all ``INFO`` level messages (or higher) will be
349          printed to the console; ``ERROR`` and ``CRITICAL``
350          messages will also be output via e-mail.
351
352.. admonition:: Custom handlers and circular imports
353
354    If your ``settings.py`` specifies a custom handler class and the file
355    defining that class also imports ``settings.py`` a circular import will
356    occur.
357
358    For example, if ``settings.py`` contains the following config for
359    :setting:`LOGGING`::
360
361        LOGGING = {
362          'version': 1,
363          'handlers': {
364            'custom_handler': {
365              'level': 'INFO',
366              'class': 'myproject.logconfig.MyHandler',
367            }
368          }
369        }
370
371    and ``myproject/logconfig.py`` has the following line before the
372    ``MyHandler`` definition::
373
374        from django.conf import settings
375
376    then the ``dictconfig`` module will raise an exception like the following::
377
378        ValueError: Unable to configure handler 'custom_handler':
379        Unable to configure handler 'custom_handler':
380        'module' object has no attribute 'logconfig'
381
382.. _formatter documentation: http://docs.python.org/library/logging.html#formatter-objects
383
384Custom logging configuration
385----------------------------
386
387If you don't want to use Python's dictConfig format to configure your
388logger, you can specify your own configuration scheme.
389
390The :setting:`LOGGING_CONFIG` setting defines the callable that will
391be used to configure Django's loggers. By default, it points at
392Python's :meth:`logging.dictConfig()` method. However, if you want to
393use a different configuration process, you can use any other callable
394that takes a single argument. The contents of :setting:`LOGGING` will
395be provided as the value of that argument when logging is configured.
396
397Disabling logging configuration
398-------------------------------
399
400If you don't want to configure logging at all (or you want to manually
401configure logging using your own approach), you can set
402:setting:`LOGGING_CONFIG` to ``None``. This will disable the
403configuration process.
404
405.. note::
406    Setting :setting:`LOGGING_CONFIG` to ``None`` only means that the
407    configuration process is disabled, not logging itself. If you
408    disable the configuration process, Django will still make logging
409    calls, falling back to whatever default logging behavior is
410    defined.
411
412Django's logging extensions
413===========================
414
415Django provides a number of utilities to handle the unique
416requirements of logging in Web server environment.
417
418Loggers
419-------
420
421Django provides three built-in loggers.
422
423``django``
424~~~~~~~~~~
425
426``django`` is the catch-all logger. No messages are posted directly to
427this logger.
428
429``django.request``
430~~~~~~~~~~~~~~~~~~
431
432Log messages related to the handling of requests. 5XX responses are
433raised as ``ERROR`` messages; 4XX responses are raised as ``WARNING``
434messages.
435
436Messages to this logger have the following extra context:
437
438    * ``status_code``: The HTTP response code associated with the
439      request.
440
441    * ``request``: The request object that generated the logging
442      message.
443
444.. note::
445    Due to a limitation in the logging library, this extra
446    context is not available if you are using Python 2.4.
447
448``django.db.backends``
449~~~~~~~~~~~~~~~~~~~~~~
450
451Messages relating to the interaction of code with the database.
452For example, every SQL statement executed by a request is logged
453at the ``DEBUG`` level to this logger.
454
455Messages to this logger have the following extra context:
456
457    * ``duration``: The time taken to execute the SQL statement.
458    * ``sql``: The SQL statement that was executed.
459    * ``params``: The parameters that were used in the SQL call.
460
461For performance reasons, SQL logging is only enabled when
462``settings.DEBUG`` is set to ``True``, regardless of the logging
463level or handlers that are installed.
464
465.. note::
466    Due to a limitation in the logging library, this extra
467    context is not available if you are using Python 2.4.
468
469Handlers
470--------
471
472Django provides one log handler in addition to those provided by the
473Python logging module.
474
475.. class:: AdminEmailHandler([include_html=False])
476
477    This handler sends an e-mail to the site admins for each log
478    message it receives.
479
480    If the log record contains a ``request`` attribute, the full details
481    of the request will be included in the e-mail.
482
483    If the log record contains stack trace information, that stack
484    trace will be included in the e-mail.
485
486    The ``include_html`` argument of ``AdminEmailHandler`` is used to
487    control whether the traceback e-mail includes an HTML attachment
488    containing the full content of the debug Web page that would have been
489    produced if :setting:`DEBUG` were ``True``. To set this value in your
490    configuration, include it in the handler definition for
491    ``django.utils.log.AdminEmailHandler``, like this::
492
493        'handlers': {
494            'mail_admins': {
495                'level': 'ERROR',
496                'class': 'django.utils.log.AdminEmailHandler',
497                'include_html': True,
498            }
499        },
500
501    Note that this HTML version of the e-mail contains a full traceback,
502    with names and values of local variables at each level of the stack, plus
503    the values of your Django settings. This information is potentially very
504    sensitive, and you may not want to send it over e-mail. Consider using
505    something such as `django-sentry`_ to get the best of both worlds -- the
506    rich information of full tracebacks plus the security of *not* sending the
507    information over e-mail.
508
509.. _django-sentry: http://pypi.python.org/pypi/django-sentry
510
511
512Filters
513-------
514
515Django provides one log filter in addition to those provided by the
516Python logging module.
517
518.. class:: CallbackFilter(callback)
519
520   .. versionadded:: 1.4
521
522   This filter accepts a callback function (which should accept a single
523   argument, the record to be logged), and calls it for each record that passes
524   through the filter. Handling of that record will not proceed if the callback
525   returns False.
526
527   This filter is used as follows in the default :setting:`LOGGING`
528   configuration to ensure that the :class:`AdminEmailHandler` only sends error
529   emails to admins when :setting:`DEBUG` is `False`::
530
531       'filters': {
532            'require_debug_false': {
533                '()': 'django.utils.log.CallbackFilter',
534                'callback': lambda r: not DEBUG
535            }
536        },
537        'handlers': {
538            'mail_admins': {
539                'level': 'ERROR',
540                'filters': ['require_debug_false'],
541                'class': 'django.utils.log.AdminEmailHandler'
542            }
543        },