Debian Codegraph @

docs/topics/logging.txt

raw
1 .. _logging-explanation:
2
3 =======
4 Logging
5 =======
6
7 .. seealso::
8
9 * :ref:`logging-how-to`
10 * :ref:`Django logging reference <logging-ref>`
11
12 Python programmers will often use ``print()`` in their code as a quick and
13 convenient debugging tool. Using the logging framework is only a little more
14 effort than that, but it's much more elegant and flexible. As well as being
15 useful for debugging, logging can also provide you with more - and better
16 structured - information about the state and health of your application.
17
18 Overview
19 ========
20
21 Django uses and extends Python's builtin :mod:`logging` module to perform
22 system logging. This module is discussed in detail in Python's own
23 documentation; this section provides a quick overview.
24
25 The cast of players
26 -------------------
27
28 A Python logging configuration consists of four parts:
29
30 * :ref:`topic-logging-parts-loggers`
31 * :ref:`topic-logging-parts-handlers`
32 * :ref:`topic-logging-parts-filters`
33 * :ref:`topic-logging-parts-formatters`
34
35 .. _topic-logging-parts-loggers:
36
37 Loggers
38 ~~~~~~~
39
40 A *logger* is the entry point into the logging system. Each logger is a named
41 bucket to which messages can be written for processing.
42
43 A logger is configured to have a *log level*. This log level describes
44 the severity of the messages that the logger will handle. Python
45 defines the following log levels:
46
47 * ``DEBUG``: Low level system information for debugging purposes
48
49 * ``INFO``: General system information
50
51 * ``WARNING``: Information describing a minor problem that has
52 occurred.
53
54 * ``ERROR``: Information describing a major problem that has
55 occurred.
56
57 * ``CRITICAL``: Information describing a critical problem that has
58 occurred.
59
60 Each message that is written to the logger is a *Log Record*. Each log
61 record also has a *log level* indicating the severity of that specific
62 message. A log record can also contain useful metadata that describes
63 the event that is being logged. This can include details such as a
64 stack trace or an error code.
65
66 When a message is given to the logger, the log level of the message is
67 compared to the log level of the logger. If the log level of the
68 message meets or exceeds the log level of the logger itself, the
69 message will undergo further processing. If it doesn't, the message
70 will be ignored.
71
72 Once a logger has determined that a message needs to be processed,
73 it is passed to a *Handler*.
74
75 .. _topic-logging-parts-handlers:
76
77 Handlers
78 ~~~~~~~~
79
80 The *handler* is the engine that determines what happens to each message
81 in a logger. It describes a particular logging behavior, such as
82 writing a message to the screen, to a file, or to a network socket.
83
84 Like loggers, handlers also have a log level. If the log level of a
85 log record doesn't meet or exceed the level of the handler, the
86 handler will ignore the message.
87
88 A logger can have multiple handlers, and each handler can have a
89 different log level. In this way, it is possible to provide different
90 forms of notification depending on the importance of a message. For
91 example, you could install one handler that forwards ``ERROR`` and
92 ``CRITICAL`` messages to a paging service, while a second handler
93 logs all messages (including ``ERROR`` and ``CRITICAL`` messages) to a
94 file for later analysis.
95
96 .. _topic-logging-parts-filters:
97
98 Filters
99 ~~~~~~~
100
101 A *filter* is used to provide additional control over which log records
102 are passed from logger to handler.
103
104 By default, any log message that meets log level requirements will be
105 handled. However, by installing a filter, you can place additional
106 criteria on the logging process. For example, you could install a
107 filter that only allows ``ERROR`` messages from a particular source to
108 be emitted.
109
110 Filters can also be used to modify the logging record prior to being
111 emitted. For example, you could write a filter that downgrades
112 ``ERROR`` log records to ``WARNING`` records if a particular set of
113 criteria are met.
114
115 Filters can be installed on loggers or on handlers; multiple filters
116 can be used in a chain to perform multiple filtering actions.
117
118 .. _topic-logging-parts-formatters:
119
120 Formatters
121 ~~~~~~~~~~
122
123 Ultimately, a log record needs to be rendered as text. *Formatters*
124 describe the exact format of that text. A formatter usually consists
125 of a Python formatting string containing
126 :ref:`LogRecord attributes <python:logrecord-attributes>`; however,
127 you can also write custom formatters to implement specific formatting behavior.
128
129 .. _logging-security-implications:
130
131 Security implications
132 =====================
133
134 The logging system handles potentially sensitive information. For example, the
135 log record may contain information about a web request or a stack trace, while
136 some of the data you collect in your own loggers may also have security
137 implications. You need to be sure you know:
138
139 * what information is collected
140 * where it will subsequently be stored
141 * how it will be transferred
142 * who might have access to it.
143
144 To help control the collection of sensitive information, you can explicitly
145 designate certain sensitive information to be filtered out of error reports --
146 read more about how to :ref:`filter error reports <filtering-error-reports>`.
147
148 ``AdminEmailHandler``
149 ---------------------
150
151 The built-in :class:`~django.utils.log.AdminEmailHandler` deserves a mention in
152 the context of security. If its ``include_html`` option is enabled, the email
153 message it sends will contain a full traceback, with names and values of local
154 variables at each level of the stack, plus the values of your Django settings
155 (in other words, the same level of detail that is exposed in a web page when
156 :setting:`DEBUG` is ``True``).
157
158 It's generally not considered a good idea to send such potentially sensitive
159 information over email. Consider instead using one of the many third-party
160 services to which detailed logs can be sent to get the best of multiple worlds
161 -- the rich information of full tracebacks, clear management of who is notified
162 and has access to the information, and so on.
163
164 .. _configuring-logging:
165
166 Configuring logging
167 ===================
168
169 Python's logging library provides several techniques to configure
170 logging, ranging from a programmatic interface to configuration files.
171 By default, Django uses the :ref:`dictConfig format
172 <logging-config-dictschema>`.
173
174 In order to configure logging, you use :setting:`LOGGING` to define a
175 dictionary of logging settings. These settings describe the loggers,
176 handlers, filters and formatters that you want in your logging setup,
177 and the log levels and other properties that you want those components
178 to have.
179
180 By default, the :setting:`LOGGING` setting is merged with :ref:`Django's
181 default logging configuration <default-logging-configuration>` using the
182 following scheme.
183
184 If the ``disable_existing_loggers`` key in the :setting:`LOGGING` dictConfig is
185 set to ``True`` (which is the ``dictConfig`` default if the key is missing)
186 then all loggers from the default configuration will be disabled. Disabled
187 loggers are not the same as removed; the logger will still exist, but will
188 silently discard anything logged to it, not even propagating entries to a
189 parent logger. Thus you should be very careful using
190 ``'disable_existing_loggers': True``; it's probably not what you want. Instead,
191 you can set ``disable_existing_loggers`` to ``False`` and redefine some or all
192 of the default loggers; or you can set :setting:`LOGGING_CONFIG` to ``None``
193 and :ref:`handle logging config yourself <disabling-logging-configuration>`.
194
195 Logging is configured as part of the general Django ``setup()`` function.
196 Therefore, you can be certain that loggers are always ready for use in your
197 project code.
198
199 Examples
200 --------
201
202 The full documentation for :ref:`dictConfig format <logging-config-dictschema>`
203 is the best source of information about logging configuration dictionaries.
204 However, to give you a taste of what is possible, here are several examples.
205
206 To begin, here's a small configuration that will allow you to output all log
207 messages to the console:
208
209 .. code-block:: python
210 :caption: ``settings.py``
211
212 import os
213
214 LOGGING = {
215 "version": 1,
216 "disable_existing_loggers": False,
217 "handlers": {
218 "console": {
219 "class": "logging.StreamHandler",
220 },
221 },
222 "root": {
223 "handlers": ["console"],
224 "level": "WARNING",
225 },
226 }
227
228 This configures the parent ``root`` logger to send messages with the
229 ``WARNING`` level and higher to the console handler. By adjusting the level to
230 ``INFO`` or ``DEBUG`` you can display more messages. This may be useful during
231 development.
232
233 Next we can add more fine-grained logging. Here's an example of how to make the
234 logging system print more messages from just the :ref:`django-logger` named
235 logger:
236
237 .. code-block:: python
238 :caption: ``settings.py``
239
240 import os
241
242 LOGGING = {
243 "version": 1,
244 "disable_existing_loggers": False,
245 "handlers": {
246 "console": {
247 "class": "logging.StreamHandler",
248 },
249 },
250 "root": {
251 "handlers": ["console"],
252 "level": "WARNING",
253 },
254 "loggers": {
255 "django": {
256 "handlers": ["console"],
257 "level": os.getenv("DJANGO_LOG_LEVEL", "INFO"),
258 "propagate": False,
259 },
260 },
261 }
262
263 By default, this config sends messages from the ``django`` logger of level
264 ``INFO`` or higher to the console. This is the same level as Django's default
265 logging config, except that the default config only displays log records when
266 ``DEBUG=True``. Django does not log many such ``INFO`` level messages. With
267 this config, however, you can also set the environment variable
268 ``DJANGO_LOG_LEVEL=DEBUG`` to see all of Django's debug logging which is very
269 verbose as it includes all database queries.
270
271 You don't have to log to the console. Here's a configuration which writes all
272 logging from the :ref:`django-logger` named logger to a local file:
273
274 .. code-block:: python
275 :caption: ``settings.py``
276
277 LOGGING = {
278 "version": 1,
279 "disable_existing_loggers": False,
280 "handlers": {
281 "file": {
282 "level": "DEBUG",
283 "class": "logging.FileHandler",
284 "filename": "/path/to/django/debug.log",
285 },
286 },
287 "loggers": {
288 "django": {
289 "handlers": ["file"],
290 "level": "DEBUG",
291 "propagate": True,
292 },
293 },
294 }
295
296 If you use this example, be sure to change the ``'filename'`` path to a
297 location that's writable by the user that's running the Django application.
298
299 Finally, here's an example of a fairly complex logging setup:
300
301 .. code-block:: python
302 :caption: ``settings.py``
303
304 LOGGING = {
305 "version": 1,
306 "disable_existing_loggers": False,
307 "formatters": {
308 "verbose": {
309 "format": "{levelname} {asctime} {module} {process:d} {thread:d} {message}",
310 "style": "{",
311 },
312 "simple": {
313 "format": "{levelname} {message}",
314 "style": "{",
315 },
316 },
317 "filters": {
318 "special": {
319 "()": "project.logging.SpecialFilter",
320 "foo": "bar",
321 },
322 "require_debug_true": {
323 "()": "django.utils.log.RequireDebugTrue",
324 },
325 },
326 "handlers": {
327 "console": {
328 "level": "INFO",
329 "filters": ["require_debug_true"],
330 "class": "logging.StreamHandler",
331 "formatter": "simple",
332 },
333 "mail_admins": {
334 "level": "ERROR",
335 "class": "django.utils.log.AdminEmailHandler",
336 "filters": ["special"],
337 },
338 },
339 "loggers": {
340 "django": {
341 "handlers": ["console"],
342 "propagate": True,
343 },
344 "django.request": {
345 "handlers": ["mail_admins"],
346 "level": "ERROR",
347 "propagate": False,
348 },
349 "myproject.custom": {
350 "handlers": ["console", "mail_admins"],
351 "level": "INFO",
352 "filters": ["special"],
353 },
354 },
355 }
356
357 This logging configuration does the following things:
358
359 * Identifies the configuration as being in 'dictConfig version 1'
360 format. At present, this is the only dictConfig format version.
361
362 * Defines two formatters:
363
364 * ``simple``, that outputs the log level name (e.g., ``DEBUG``) and the log
365 message.
366
367 The ``format`` string is a normal Python formatting string
368 describing the details that are to be output on each logging
369 line. The full list of detail that can be output can be
370 found in :ref:`formatter-objects`.
371
372 * ``verbose``, that outputs the log level name, the log
373 message, plus the time, process, thread and module that
374 generate the log message.
375
376 * Defines two filters:
377
378 * ``project.logging.SpecialFilter``, using the alias ``special``. If this
379 filter required additional arguments, they can be provided as additional
380 keys in the filter configuration dictionary. In this case, the argument
381 ``foo`` will be given a value of ``bar`` when instantiating
382 ``SpecialFilter``.
383
384 * ``django.utils.log.RequireDebugTrue``, which passes on records when
385 :setting:`DEBUG` is ``True``.
386
387 * Defines two handlers:
388
389 * ``console``, a :class:`~logging.StreamHandler`, which prints any ``INFO``
390 (or higher) message to ``sys.stderr``. This handler uses the ``simple``
391 output format.
392
393 * ``mail_admins``, an :class:`~django.utils.log.AdminEmailHandler`, which
394 emails any ``ERROR`` (or higher) message to the site :setting:`ADMINS`.
395 This handler uses the ``special`` filter.
396
397 * Configures three loggers:
398
399 * ``django``, which passes all messages to the ``console`` handler.
400
401 * ``django.request``, which passes all ``ERROR`` messages to
402 the ``mail_admins`` handler. In addition, this logger is
403 marked to *not* propagate messages. This means that log
404 messages written to ``django.request`` will not be handled
405 by the ``django`` logger.
406
407 * ``myproject.custom``, which passes all messages at ``INFO``
408 or higher that also pass the ``special`` filter to two
409 handlers -- the ``console``, and ``mail_admins``. This
410 means that all ``INFO`` level messages (or higher) will be
411 printed to the console; ``ERROR`` and ``CRITICAL``
412 messages will also be output via email.
413
414 Custom logging configuration
415 ----------------------------
416
417 If you don't want to use Python's dictConfig format to configure your
418 logger, you can specify your own configuration scheme.
419
420 The :setting:`LOGGING_CONFIG` setting defines the callable that will
421 be used to configure Django's loggers. By default, it points at
422 Python's :func:`logging.config.dictConfig()` function. However, if you want to
423 use a different configuration process, you can use any other callable
424 that takes a single argument. The contents of :setting:`LOGGING` will
425 be provided as the value of that argument when logging is configured.
426
427 .. _disabling-logging-configuration:
428
429 Disabling logging configuration
430 -------------------------------
431
432 If you don't want to configure logging at all (or you want to manually
433 configure logging using your own approach), you can set
434 :setting:`LOGGING_CONFIG` to ``None``. This will disable the
435 configuration process for :ref:`Django's default logging
436 <default-logging-configuration>`.
437
438 Setting :setting:`LOGGING_CONFIG` to ``None`` only means that the automatic
439 configuration process is disabled, not logging itself. If you disable the
440 configuration process, Django will still make logging calls, falling back to
441 whatever default logging behavior is defined.
442
443 Here's an example that disables Django's logging configuration and then
444 manually configures logging:
445
446 .. code-block:: python
447 :caption: ``settings.py``
448
449 LOGGING_CONFIG = None
450
451 import logging.config
452
453 logging.config.dictConfig(...)
454
455 Note that the default configuration process only calls
456 :setting:`LOGGING_CONFIG` once settings are fully-loaded. In contrast, manually
457 configuring the logging in your settings file will load your logging config
458 immediately. As such, your logging config must appear *after* any settings on
459 which it depends.