| 1 |
.. _logging-explanation: |
| 2 |
|
| 3 |
======= |
| 4 |
Logging |
| 5 |
======= |
| 6 |
|
| 7 |
.. seealso:: |
| 8 |
|
| 9 |
* :ref:`logging-how-to` |
| 10 |
* :ref:`Django logging reference <logging-ref>` |
| 11 |
|
| 12 |
Python programmers will often use ``print()`` in their code as a quick and |
| 13 |
convenient debugging tool. Using the logging framework is only a little more |
| 14 |
effort than that, but it's much more elegant and flexible. As well as being |
| 15 |
useful for debugging, logging can also provide you with more - and better |
| 16 |
structured - information about the state and health of your application. |
| 17 |
|
| 18 |
Overview |
| 19 |
======== |
| 20 |
|
| 21 |
Django uses and extends Python's builtin :mod:`logging` module to perform |
| 22 |
system logging. This module is discussed in detail in Python's own |
| 23 |
documentation; this section provides a quick overview. |
| 24 |
|
| 25 |
The cast of players |
| 26 |
------------------- |
| 27 |
|
| 28 |
A Python logging configuration consists of four parts: |
| 29 |
|
| 30 |
* :ref:`topic-logging-parts-loggers` |
| 31 |
* :ref:`topic-logging-parts-handlers` |
| 32 |
* :ref:`topic-logging-parts-filters` |
| 33 |
* :ref:`topic-logging-parts-formatters` |
| 34 |
|
| 35 |
.. _topic-logging-parts-loggers: |
| 36 |
|
| 37 |
Loggers |
| 38 |
~~~~~~~ |
| 39 |
|
| 40 |
A *logger* is the entry point into the logging system. Each logger is a named |
| 41 |
bucket to which messages can be written for processing. |
| 42 |
|
| 43 |
A logger is configured to have a *log level*. This log level describes |
| 44 |
the severity of the messages that the logger will handle. Python |
| 45 |
defines the following log levels: |
| 46 |
|
| 47 |
* ``DEBUG``: Low level system information for debugging purposes |
| 48 |
|
| 49 |
* ``INFO``: General system information |
| 50 |
|
| 51 |
* ``WARNING``: Information describing a minor problem that has |
| 52 |
occurred. |
| 53 |
|
| 54 |
* ``ERROR``: Information describing a major problem that has |
| 55 |
occurred. |
| 56 |
|
| 57 |
* ``CRITICAL``: Information describing a critical problem that has |
| 58 |
occurred. |
| 59 |
|
| 60 |
Each message that is written to the logger is a *Log Record*. Each log |
| 61 |
record also has a *log level* indicating the severity of that specific |
| 62 |
message. A log record can also contain useful metadata that describes |
| 63 |
the event that is being logged. This can include details such as a |
| 64 |
stack trace or an error code. |
| 65 |
|
| 66 |
When a message is given to the logger, the log level of the message is |
| 67 |
compared to the log level of the logger. If the log level of the |
| 68 |
message meets or exceeds the log level of the logger itself, the |
| 69 |
message will undergo further processing. If it doesn't, the message |
| 70 |
will be ignored. |
| 71 |
|
| 72 |
Once a logger has determined that a message needs to be processed, |
| 73 |
it is passed to a *Handler*. |
| 74 |
|
| 75 |
.. _topic-logging-parts-handlers: |
| 76 |
|
| 77 |
Handlers |
| 78 |
~~~~~~~~ |
| 79 |
|
| 80 |
The *handler* is the engine that determines what happens to each message |
| 81 |
in a logger. It describes a particular logging behavior, such as |
| 82 |
writing a message to the screen, to a file, or to a network socket. |
| 83 |
|
| 84 |
Like loggers, handlers also have a log level. If the log level of a |
| 85 |
log record doesn't meet or exceed the level of the handler, the |
| 86 |
handler will ignore the message. |
| 87 |
|
| 88 |
A logger can have multiple handlers, and each handler can have a |
| 89 |
different log level. In this way, it is possible to provide different |
| 90 |
forms of notification depending on the importance of a message. For |
| 91 |
example, you could install one handler that forwards ``ERROR`` and |
| 92 |
``CRITICAL`` messages to a paging service, while a second handler |
| 93 |
logs all messages (including ``ERROR`` and ``CRITICAL`` messages) to a |
| 94 |
file for later analysis. |
| 95 |
|
| 96 |
.. _topic-logging-parts-filters: |
| 97 |
|
| 98 |
Filters |
| 99 |
~~~~~~~ |
| 100 |
|
| 101 |
A *filter* is used to provide additional control over which log records |
| 102 |
are passed from logger to handler. |
| 103 |
|
| 104 |
By default, any log message that meets log level requirements will be |
| 105 |
handled. However, by installing a filter, you can place additional |
| 106 |
criteria on the logging process. For example, you could install a |
| 107 |
filter that only allows ``ERROR`` messages from a particular source to |
| 108 |
be emitted. |
| 109 |
|
| 110 |
Filters can also be used to modify the logging record prior to being |
| 111 |
emitted. For example, you could write a filter that downgrades |
| 112 |
``ERROR`` log records to ``WARNING`` records if a particular set of |
| 113 |
criteria are met. |
| 114 |
|
| 115 |
Filters can be installed on loggers or on handlers; multiple filters |
| 116 |
can be used in a chain to perform multiple filtering actions. |
| 117 |
|
| 118 |
.. _topic-logging-parts-formatters: |
| 119 |
|
| 120 |
Formatters |
| 121 |
~~~~~~~~~~ |
| 122 |
|
| 123 |
Ultimately, a log record needs to be rendered as text. *Formatters* |
| 124 |
describe the exact format of that text. A formatter usually consists |
| 125 |
of a Python formatting string containing |
| 126 |
:ref:`LogRecord attributes <python:logrecord-attributes>`; however, |
| 127 |
you can also write custom formatters to implement specific formatting behavior. |
| 128 |
|
| 129 |
.. _logging-security-implications: |
| 130 |
|
| 131 |
Security implications |
| 132 |
===================== |
| 133 |
|
| 134 |
The logging system handles potentially sensitive information. For example, the |
| 135 |
log record may contain information about a web request or a stack trace, while |
| 136 |
some of the data you collect in your own loggers may also have security |
| 137 |
implications. You need to be sure you know: |
| 138 |
|
| 139 |
* what information is collected |
| 140 |
* where it will subsequently be stored |
| 141 |
* how it will be transferred |
| 142 |
* who might have access to it. |
| 143 |
|
| 144 |
To help control the collection of sensitive information, you can explicitly |
| 145 |
designate certain sensitive information to be filtered out of error reports -- |
| 146 |
read more about how to :ref:`filter error reports <filtering-error-reports>`. |
| 147 |
|
| 148 |
``AdminEmailHandler`` |
| 149 |
--------------------- |
| 150 |
|
| 151 |
The built-in :class:`~django.utils.log.AdminEmailHandler` deserves a mention in |
| 152 |
the context of security. If its ``include_html`` option is enabled, the email |
| 153 |
message it sends will contain a full traceback, with names and values of local |
| 154 |
variables at each level of the stack, plus the values of your Django settings |
| 155 |
(in other words, the same level of detail that is exposed in a web page when |
| 156 |
:setting:`DEBUG` is ``True``). |
| 157 |
|
| 158 |
It's generally not considered a good idea to send such potentially sensitive |
| 159 |
information over email. Consider instead using one of the many third-party |
| 160 |
services to which detailed logs can be sent to get the best of multiple worlds |
| 161 |
-- the rich information of full tracebacks, clear management of who is notified |
| 162 |
and has access to the information, and so on. |
| 163 |
|
| 164 |
.. _configuring-logging: |
| 165 |
|
| 166 |
Configuring logging |
| 167 |
=================== |
| 168 |
|
| 169 |
Python's logging library provides several techniques to configure |
| 170 |
logging, ranging from a programmatic interface to configuration files. |
| 171 |
By default, Django uses the :ref:`dictConfig format |
| 172 |
<logging-config-dictschema>`. |
| 173 |
|
| 174 |
In order to configure logging, you use :setting:`LOGGING` to define a |
| 175 |
dictionary of logging settings. These settings describe the loggers, |
| 176 |
handlers, filters and formatters that you want in your logging setup, |
| 177 |
and the log levels and other properties that you want those components |
| 178 |
to have. |
| 179 |
|
| 180 |
By default, the :setting:`LOGGING` setting is merged with :ref:`Django's |
| 181 |
default logging configuration <default-logging-configuration>` using the |
| 182 |
following scheme. |
| 183 |
|
| 184 |
If the ``disable_existing_loggers`` key in the :setting:`LOGGING` dictConfig is |
| 185 |
set to ``True`` (which is the ``dictConfig`` default if the key is missing) |
| 186 |
then all loggers from the default configuration will be disabled. Disabled |
| 187 |
loggers are not the same as removed; the logger will still exist, but will |
| 188 |
silently discard anything logged to it, not even propagating entries to a |
| 189 |
parent logger. Thus you should be very careful using |
| 190 |
``'disable_existing_loggers': True``; it's probably not what you want. Instead, |
| 191 |
you can set ``disable_existing_loggers`` to ``False`` and redefine some or all |
| 192 |
of the default loggers; or you can set :setting:`LOGGING_CONFIG` to ``None`` |
| 193 |
and :ref:`handle logging config yourself <disabling-logging-configuration>`. |
| 194 |
|
| 195 |
Logging is configured as part of the general Django ``setup()`` function. |
| 196 |
Therefore, you can be certain that loggers are always ready for use in your |
| 197 |
project code. |
| 198 |
|
| 199 |
Examples |
| 200 |
-------- |
| 201 |
|
| 202 |
The full documentation for :ref:`dictConfig format <logging-config-dictschema>` |
| 203 |
is the best source of information about logging configuration dictionaries. |
| 204 |
However, to give you a taste of what is possible, here are several examples. |
| 205 |
|
| 206 |
To begin, here's a small configuration that will allow you to output all log |
| 207 |
messages to the console: |
| 208 |
|
| 209 |
.. code-block:: python |
| 210 |
:caption: ``settings.py`` |
| 211 |
|
| 212 |
import os |
| 213 |
|
| 214 |
LOGGING = { |
| 215 |
"version": 1, |
| 216 |
"disable_existing_loggers": False, |
| 217 |
"handlers": { |
| 218 |
"console": { |
| 219 |
"class": "logging.StreamHandler", |
| 220 |
}, |
| 221 |
}, |
| 222 |
"root": { |
| 223 |
"handlers": ["console"], |
| 224 |
"level": "WARNING", |
| 225 |
}, |
| 226 |
} |
| 227 |
|
| 228 |
This configures the parent ``root`` logger to send messages with the |
| 229 |
``WARNING`` level and higher to the console handler. By adjusting the level to |
| 230 |
``INFO`` or ``DEBUG`` you can display more messages. This may be useful during |
| 231 |
development. |
| 232 |
|
| 233 |
Next we can add more fine-grained logging. Here's an example of how to make the |
| 234 |
logging system print more messages from just the :ref:`django-logger` named |
| 235 |
logger: |
| 236 |
|
| 237 |
.. code-block:: python |
| 238 |
:caption: ``settings.py`` |
| 239 |
|
| 240 |
import os |
| 241 |
|
| 242 |
LOGGING = { |
| 243 |
"version": 1, |
| 244 |
"disable_existing_loggers": False, |
| 245 |
"handlers": { |
| 246 |
"console": { |
| 247 |
"class": "logging.StreamHandler", |
| 248 |
}, |
| 249 |
}, |
| 250 |
"root": { |
| 251 |
"handlers": ["console"], |
| 252 |
"level": "WARNING", |
| 253 |
}, |
| 254 |
"loggers": { |
| 255 |
"django": { |
| 256 |
"handlers": ["console"], |
| 257 |
"level": os.getenv("DJANGO_LOG_LEVEL", "INFO"), |
| 258 |
"propagate": False, |
| 259 |
}, |
| 260 |
}, |
| 261 |
} |
| 262 |
|
| 263 |
By default, this config sends messages from the ``django`` logger of level |
| 264 |
``INFO`` or higher to the console. This is the same level as Django's default |
| 265 |
logging config, except that the default config only displays log records when |
| 266 |
``DEBUG=True``. Django does not log many such ``INFO`` level messages. With |
| 267 |
this config, however, you can also set the environment variable |
| 268 |
``DJANGO_LOG_LEVEL=DEBUG`` to see all of Django's debug logging which is very |
| 269 |
verbose as it includes all database queries. |
| 270 |
|
| 271 |
You don't have to log to the console. Here's a configuration which writes all |
| 272 |
logging from the :ref:`django-logger` named logger to a local file: |
| 273 |
|
| 274 |
.. code-block:: python |
| 275 |
:caption: ``settings.py`` |
| 276 |
|
| 277 |
LOGGING = { |
| 278 |
"version": 1, |
| 279 |
"disable_existing_loggers": False, |
| 280 |
"handlers": { |
| 281 |
"file": { |
| 282 |
"level": "DEBUG", |
| 283 |
"class": "logging.FileHandler", |
| 284 |
"filename": "/path/to/django/debug.log", |
| 285 |
}, |
| 286 |
}, |
| 287 |
"loggers": { |
| 288 |
"django": { |
| 289 |
"handlers": ["file"], |
| 290 |
"level": "DEBUG", |
| 291 |
"propagate": True, |
| 292 |
}, |
| 293 |
}, |
| 294 |
} |
| 295 |
|
| 296 |
If you use this example, be sure to change the ``'filename'`` path to a |
| 297 |
location that's writable by the user that's running the Django application. |
| 298 |
|
| 299 |
Finally, here's an example of a fairly complex logging setup: |
| 300 |
|
| 301 |
.. code-block:: python |
| 302 |
:caption: ``settings.py`` |
| 303 |
|
| 304 |
LOGGING = { |
| 305 |
"version": 1, |
| 306 |
"disable_existing_loggers": False, |
| 307 |
"formatters": { |
| 308 |
"verbose": { |
| 309 |
"format": "{levelname} {asctime} {module} {process:d} {thread:d} {message}", |
| 310 |
"style": "{", |
| 311 |
}, |
| 312 |
"simple": { |
| 313 |
"format": "{levelname} {message}", |
| 314 |
"style": "{", |
| 315 |
}, |
| 316 |
}, |
| 317 |
"filters": { |
| 318 |
"special": { |
| 319 |
"()": "project.logging.SpecialFilter", |
| 320 |
"foo": "bar", |
| 321 |
}, |
| 322 |
"require_debug_true": { |
| 323 |
"()": "django.utils.log.RequireDebugTrue", |
| 324 |
}, |
| 325 |
}, |
| 326 |
"handlers": { |
| 327 |
"console": { |
| 328 |
"level": "INFO", |
| 329 |
"filters": ["require_debug_true"], |
| 330 |
"class": "logging.StreamHandler", |
| 331 |
"formatter": "simple", |
| 332 |
}, |
| 333 |
"mail_admins": { |
| 334 |
"level": "ERROR", |
| 335 |
"class": "django.utils.log.AdminEmailHandler", |
| 336 |
"filters": ["special"], |
| 337 |
}, |
| 338 |
}, |
| 339 |
"loggers": { |
| 340 |
"django": { |
| 341 |
"handlers": ["console"], |
| 342 |
"propagate": True, |
| 343 |
}, |
| 344 |
"django.request": { |
| 345 |
"handlers": ["mail_admins"], |
| 346 |
"level": "ERROR", |
| 347 |
"propagate": False, |
| 348 |
}, |
| 349 |
"myproject.custom": { |
| 350 |
"handlers": ["console", "mail_admins"], |
| 351 |
"level": "INFO", |
| 352 |
"filters": ["special"], |
| 353 |
}, |
| 354 |
}, |
| 355 |
} |
| 356 |
|
| 357 |
This logging configuration does the following things: |
| 358 |
|
| 359 |
* Identifies the configuration as being in 'dictConfig version 1' |
| 360 |
format. At present, this is the only dictConfig format version. |
| 361 |
|
| 362 |
* Defines two formatters: |
| 363 |
|
| 364 |
* ``simple``, that outputs the log level name (e.g., ``DEBUG``) and the log |
| 365 |
message. |
| 366 |
|
| 367 |
The ``format`` string is a normal Python formatting string |
| 368 |
describing the details that are to be output on each logging |
| 369 |
line. The full list of detail that can be output can be |
| 370 |
found in :ref:`formatter-objects`. |
| 371 |
|
| 372 |
* ``verbose``, that outputs the log level name, the log |
| 373 |
message, plus the time, process, thread and module that |
| 374 |
generate the log message. |
| 375 |
|
| 376 |
* Defines two filters: |
| 377 |
|
| 378 |
* ``project.logging.SpecialFilter``, using the alias ``special``. If this |
| 379 |
filter required additional arguments, they can be provided as additional |
| 380 |
keys in the filter configuration dictionary. In this case, the argument |
| 381 |
``foo`` will be given a value of ``bar`` when instantiating |
| 382 |
``SpecialFilter``. |
| 383 |
|
| 384 |
* ``django.utils.log.RequireDebugTrue``, which passes on records when |
| 385 |
:setting:`DEBUG` is ``True``. |
| 386 |
|
| 387 |
* Defines two handlers: |
| 388 |
|
| 389 |
* ``console``, a :class:`~logging.StreamHandler`, which prints any ``INFO`` |
| 390 |
(or higher) message to ``sys.stderr``. This handler uses the ``simple`` |
| 391 |
output format. |
| 392 |
|
| 393 |
* ``mail_admins``, an :class:`~django.utils.log.AdminEmailHandler`, which |
| 394 |
emails any ``ERROR`` (or higher) message to the site :setting:`ADMINS`. |
| 395 |
This handler uses the ``special`` filter. |
| 396 |
|
| 397 |
* Configures three loggers: |
| 398 |
|
| 399 |
* ``django``, which passes all messages to the ``console`` handler. |
| 400 |
|
| 401 |
* ``django.request``, which passes all ``ERROR`` messages to |
| 402 |
the ``mail_admins`` handler. In addition, this logger is |
| 403 |
marked to *not* propagate messages. This means that log |
| 404 |
messages written to ``django.request`` will not be handled |
| 405 |
by the ``django`` logger. |
| 406 |
|
| 407 |
* ``myproject.custom``, which passes all messages at ``INFO`` |
| 408 |
or higher that also pass the ``special`` filter to two |
| 409 |
handlers -- the ``console``, and ``mail_admins``. This |
| 410 |
means that all ``INFO`` level messages (or higher) will be |
| 411 |
printed to the console; ``ERROR`` and ``CRITICAL`` |
| 412 |
messages will also be output via email. |
| 413 |
|
| 414 |
Custom logging configuration |
| 415 |
---------------------------- |
| 416 |
|
| 417 |
If you don't want to use Python's dictConfig format to configure your |
| 418 |
logger, you can specify your own configuration scheme. |
| 419 |
|
| 420 |
The :setting:`LOGGING_CONFIG` setting defines the callable that will |
| 421 |
be used to configure Django's loggers. By default, it points at |
| 422 |
Python's :func:`logging.config.dictConfig()` function. However, if you want to |
| 423 |
use a different configuration process, you can use any other callable |
| 424 |
that takes a single argument. The contents of :setting:`LOGGING` will |
| 425 |
be provided as the value of that argument when logging is configured. |
| 426 |
|
| 427 |
.. _disabling-logging-configuration: |
| 428 |
|
| 429 |
Disabling logging configuration |
| 430 |
------------------------------- |
| 431 |
|
| 432 |
If you don't want to configure logging at all (or you want to manually |
| 433 |
configure logging using your own approach), you can set |
| 434 |
:setting:`LOGGING_CONFIG` to ``None``. This will disable the |
| 435 |
configuration process for :ref:`Django's default logging |
| 436 |
<default-logging-configuration>`. |
| 437 |
|
| 438 |
Setting :setting:`LOGGING_CONFIG` to ``None`` only means that the automatic |
| 439 |
configuration process is disabled, not logging itself. If you disable the |
| 440 |
configuration process, Django will still make logging calls, falling back to |
| 441 |
whatever default logging behavior is defined. |
| 442 |
|
| 443 |
Here's an example that disables Django's logging configuration and then |
| 444 |
manually configures logging: |
| 445 |
|
| 446 |
.. code-block:: python |
| 447 |
:caption: ``settings.py`` |
| 448 |
|
| 449 |
LOGGING_CONFIG = None |
| 450 |
|
| 451 |
import logging.config |
| 452 |
|
| 453 |
logging.config.dictConfig(...) |
| 454 |
|
| 455 |
Note that the default configuration process only calls |
| 456 |
:setting:`LOGGING_CONFIG` once settings are fully-loaded. In contrast, manually |
| 457 |
configuring the logging in your settings file will load your logging config |
| 458 |
immediately. As such, your logging config must appear *after* any settings on |
| 459 |
which it depends. |