Debian Codegraph @

Doc/library/logging.config.rst

raw
1 :mod:`!logging.config` --- Logging configuration
2 ================================================
3
4 .. module:: logging.config
5 :synopsis: Configuration of the logging module.
6
7 .. moduleauthor:: Vinay Sajip <[email protected]>
8 .. sectionauthor:: Vinay Sajip <[email protected]>
9
10 **Source code:** :source:`Lib/logging/config.py`
11
12 .. sidebar:: Important
13
14 This page contains only reference information. For tutorials,
15 please see
16
17 * :ref:`Basic Tutorial <logging-basic-tutorial>`
18 * :ref:`Advanced Tutorial <logging-advanced-tutorial>`
19 * :ref:`Logging Cookbook <logging-cookbook>`
20
21 --------------
22
23 This section describes the API for configuring the logging module.
24
25 .. _logging-config-api:
26
27 Configuration functions
28 ^^^^^^^^^^^^^^^^^^^^^^^
29
30 The following functions configure the logging module. They are located in the
31 :mod:`!logging.config` module. Their use is optional --- you can configure the
32 logging module using these functions or by making calls to the main API (defined
33 in :mod:`logging` itself) and defining handlers which are declared either in
34 :mod:`logging` or :mod:`logging.handlers`.
35
36 .. function:: dictConfig(config)
37
38 Takes the logging configuration from a dictionary. The contents of
39 this dictionary are described in :ref:`logging-config-dictschema`
40 below.
41
42 If an error is encountered during configuration, this function will
43 raise a :exc:`ValueError`, :exc:`TypeError`, :exc:`AttributeError`
44 or :exc:`ImportError` with a suitably descriptive message. The
45 following is a (possibly incomplete) list of conditions which will
46 raise an error:
47
48 * A ``level`` which is not a string or which is a string not
49 corresponding to an actual logging level.
50 * A ``propagate`` value which is not a boolean.
51 * An id which does not have a corresponding destination.
52 * A non-existent handler id found during an incremental call.
53 * An invalid logger name.
54 * Inability to resolve to an internal or external object.
55
56 Parsing is performed by the :class:`DictConfigurator` class, whose
57 constructor is passed the dictionary used for configuration, and
58 has a :meth:`configure` method. The :mod:`!logging.config` module
59 has a callable attribute :attr:`dictConfigClass`
60 which is initially set to :class:`DictConfigurator`.
61 You can replace the value of :attr:`dictConfigClass` with a
62 suitable implementation of your own.
63
64 :func:`dictConfig` calls :attr:`dictConfigClass` passing
65 the specified dictionary, and then calls the :meth:`configure` method on
66 the returned object to put the configuration into effect::
67
68 def dictConfig(config):
69 dictConfigClass(config).configure()
70
71 For example, a subclass of :class:`DictConfigurator` could call
72 ``DictConfigurator.__init__()`` in its own :meth:`__init__`, then
73 set up custom prefixes which would be usable in the subsequent
74 :meth:`configure` call. :attr:`dictConfigClass` would be bound to
75 this new subclass, and then :func:`dictConfig` could be called exactly as
76 in the default, uncustomized state.
77
78 .. versionadded:: 3.2
79
80 .. function:: fileConfig(fname, defaults=None, disable_existing_loggers=True, encoding=None)
81
82 Reads the logging configuration from a :mod:`configparser`\-format file. The
83 format of the file should be as described in
84 :ref:`logging-config-fileformat`.
85 This function can be called several times from an application, allowing an
86 end user to select from various pre-canned configurations (if the developer
87 provides a mechanism to present the choices and load the chosen
88 configuration).
89
90 It will raise :exc:`FileNotFoundError` if the file
91 doesn't exist and :exc:`RuntimeError` if the file is invalid or
92 empty.
93
94 :param fname: A filename, or a file-like object, or an instance derived
95 from :class:`~configparser.RawConfigParser`. If a
96 :class:`!RawConfigParser`-derived instance is passed, it is used as
97 is. Otherwise, a :class:`~configparser.ConfigParser` is
98 instantiated, and the configuration read by it from the
99 object passed in ``fname``. If that has a :meth:`readline`
100 method, it is assumed to be a file-like object and read using
101 :meth:`~configparser.ConfigParser.read_file`; otherwise,
102 it is assumed to be a filename and passed to
103 :meth:`~configparser.ConfigParser.read`.
104
105
106 :param defaults: Defaults to be passed to the :class:`!ConfigParser` can be specified
107 in this argument.
108
109 :param disable_existing_loggers: If specified as ``False``, loggers which
110 exist when this call is made are left
111 enabled. The default is ``True`` because this
112 enables old behaviour in a
113 backward-compatible way. This behaviour is to
114 disable any existing non-root loggers unless
115 they or their ancestors are explicitly named
116 in the logging configuration.
117
118 :param encoding: The encoding used to open file when *fname* is filename.
119
120 .. versionchanged:: 3.4
121 An instance of a subclass of :class:`~configparser.RawConfigParser` is
122 now accepted as a value for ``fname``. This facilitates:
123
124 * Use of a configuration file where logging configuration is just part
125 of the overall application configuration.
126 * Use of a configuration read from a file, and then modified by the using
127 application (e.g. based on command-line parameters or other aspects
128 of the runtime environment) before being passed to ``fileConfig``.
129
130 .. versionchanged:: 3.10
131 Added the *encoding* parameter.
132
133 .. versionchanged:: 3.12
134 An exception will be thrown if the provided file
135 doesn't exist or is invalid or empty.
136
137 .. function:: listen(port=DEFAULT_LOGGING_CONFIG_PORT, verify=None)
138
139 Starts up a socket server on the specified port, and listens for new
140 configurations. If no port is specified, the module's default
141 :const:`DEFAULT_LOGGING_CONFIG_PORT` is used. Logging configurations will be
142 sent as a file suitable for processing by :func:`dictConfig` or
143 :func:`fileConfig`. Returns a :class:`~threading.Thread` instance on which
144 you can call :meth:`~threading.Thread.start` to start the server, and which
145 you can :meth:`~threading.Thread.join` when appropriate. To stop the server,
146 call :func:`stopListening`.
147
148 The ``verify`` argument, if specified, should be a callable which should
149 verify whether bytes received across the socket are valid and should be
150 processed. This could be done by encrypting and/or signing what is sent
151 across the socket, such that the ``verify`` callable can perform
152 signature verification and/or decryption. The ``verify`` callable is called
153 with a single argument - the bytes received across the socket - and should
154 return the bytes to be processed, or ``None`` to indicate that the bytes should
155 be discarded. The returned bytes could be the same as the passed in bytes
156 (e.g. when only verification is done), or they could be completely different
157 (perhaps if decryption were performed).
158
159 To send a configuration to the socket, read in the configuration file and
160 send it to the socket as a sequence of bytes preceded by a four-byte length
161 string packed in binary using ``struct.pack('>L', n)``.
162
163 .. _logging-eval-security:
164
165 .. note::
166
167 Because portions of the configuration are passed through
168 :func:`eval`, use of this function may open its users to a security risk.
169 While the function only binds to a socket on ``localhost``, and so does
170 not accept connections from remote machines, there are scenarios where
171 untrusted code could be run under the account of the process which calls
172 :func:`listen`. Specifically, if the process calling :func:`listen` runs
173 on a multi-user machine where users cannot trust each other, then a
174 malicious user could arrange to run essentially arbitrary code in a
175 victim user's process, simply by connecting to the victim's
176 :func:`listen` socket and sending a configuration which runs whatever
177 code the attacker wants to have executed in the victim's process. This is
178 especially easy to do if the default port is used, but not hard even if a
179 different port is used. To avoid the risk of this happening, use the
180 ``verify`` argument to :func:`listen` to prevent unrecognised
181 configurations from being applied.
182
183 .. versionchanged:: 3.4
184 The ``verify`` argument was added.
185
186 .. note::
187
188 If you want to send configurations to the listener which don't
189 disable existing loggers, you will need to use a JSON format for
190 the configuration, which will use :func:`dictConfig` for configuration.
191 This method allows you to specify ``disable_existing_loggers`` as
192 ``False`` in the configuration you send.
193
194
195 .. function:: stopListening()
196
197 Stops the listening server which was created with a call to :func:`listen`.
198 This is typically called before calling :meth:`join` on the return value from
199 :func:`listen`.
200
201
202 Security considerations
203 ^^^^^^^^^^^^^^^^^^^^^^^
204
205 The logging configuration functionality tries to offer convenience, and in part this
206 is done by offering the ability to convert text in configuration files into Python
207 objects used in logging configuration - for example, as described in
208 :ref:`logging-config-dict-userdef`. However, these same mechanisms (importing
209 callables from user-defined modules and calling them with parameters from the
210 configuration) could be used to invoke any code you like, and for this reason you
211 should treat configuration files from untrusted sources with *extreme caution* and
212 satisfy yourself that nothing bad can happen if you load them, before actually loading
213 them.
214
215
216 .. _logging-config-dictschema:
217
218 Configuration dictionary schema
219 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
220
221 Describing a logging configuration requires listing the various
222 objects to create and the connections between them; for example, you
223 may create a handler named 'console' and then say that the logger
224 named 'startup' will send its messages to the 'console' handler.
225 These objects aren't limited to those provided by the :mod:`logging`
226 module because you might write your own formatter or handler class.
227 The parameters to these classes may also need to include external
228 objects such as ``sys.stderr``. The syntax for describing these
229 objects and connections is defined in :ref:`logging-config-dict-connections`
230 below.
231
232 Dictionary Schema Details
233 """""""""""""""""""""""""
234
235 The dictionary passed to :func:`dictConfig` must contain the following
236 keys:
237
238 * *version* - to be set to an integer value representing the schema
239 version. The only valid value at present is 1, but having this key
240 allows the schema to evolve while still preserving backwards
241 compatibility.
242
243 All other keys are optional, but if present they will be interpreted
244 as described below. In all cases below where a 'configuring dict' is
245 mentioned, it will be checked for the special ``'()'`` key to see if a
246 custom instantiation is required. If so, the mechanism described in
247 :ref:`logging-config-dict-userdef` below is used to create an instance;
248 otherwise, the context is used to determine what to instantiate.
249
250 .. _logging-config-dictschema-formatters:
251
252 * *formatters* - the corresponding value will be a dict in which each
253 key is a formatter id and each value is a dict describing how to
254 configure the corresponding :class:`~logging.Formatter` instance.
255
256 The configuring dict is searched for the following optional keys
257 which correspond to the arguments passed to create a
258 :class:`~logging.Formatter` object:
259
260 * ``format``
261 * ``datefmt``
262 * ``style``
263 * ``validate`` (since version >=3.8)
264 * ``defaults`` (since version >=3.12)
265
266 An optional ``class`` key indicates the name of the formatter's
267 class (as a dotted module and class name). The instantiation
268 arguments are as for :class:`~logging.Formatter`, thus this key is
269 most useful for instantiating a customised subclass of
270 :class:`~logging.Formatter`. For example, the alternative class
271 might present exception tracebacks in an expanded or condensed
272 format. If your formatter requires different or extra configuration
273 keys, you should use :ref:`logging-config-dict-userdef`.
274
275 * *filters* - the corresponding value will be a dict in which each key
276 is a filter id and each value is a dict describing how to configure
277 the corresponding Filter instance.
278
279 The configuring dict is searched for the key ``name`` (defaulting to the
280 empty string) and this is used to construct a :class:`logging.Filter`
281 instance.
282
283 * *handlers* - the corresponding value will be a dict in which each
284 key is a handler id and each value is a dict describing how to
285 configure the corresponding Handler instance.
286
287 The configuring dict is searched for the following keys:
288
289 * ``class`` (mandatory). This is the fully qualified name of the
290 handler class.
291
292 * ``level`` (optional). The level of the handler.
293
294 * ``formatter`` (optional). The id of the formatter for this
295 handler.
296
297 * ``filters`` (optional). A list of ids of the filters for this
298 handler.
299
300 .. versionchanged:: 3.11
301 ``filters`` can take filter instances in addition to ids.
302
303 All *other* keys are passed through as keyword arguments to the
304 handler's constructor. For example, given the snippet:
305
306 .. code-block:: yaml
307
308 handlers:
309 console:
310 class : logging.StreamHandler
311 formatter: brief
312 level : INFO
313 filters: [allow_foo]
314 stream : ext://sys.stdout
315 file:
316 class : logging.handlers.RotatingFileHandler
317 formatter: precise
318 filename: logconfig.log
319 maxBytes: 1024
320 backupCount: 3
321
322 the handler with id ``console`` is instantiated as a
323 :class:`logging.StreamHandler`, using ``sys.stdout`` as the underlying
324 stream. The handler with id ``file`` is instantiated as a
325 :class:`logging.handlers.RotatingFileHandler` with the keyword arguments
326 ``filename='logconfig.log', maxBytes=1024, backupCount=3``.
327
328 * *loggers* - the corresponding value will be a dict in which each key
329 is a logger name and each value is a dict describing how to
330 configure the corresponding Logger instance.
331
332 The configuring dict is searched for the following keys:
333
334 * ``level`` (optional). The level of the logger.
335
336 * ``propagate`` (optional). The propagation setting of the logger.
337
338 * ``filters`` (optional). A list of ids of the filters for this
339 logger.
340
341 .. versionchanged:: 3.11
342 ``filters`` can take filter instances in addition to ids.
343
344 * ``handlers`` (optional). A list of ids of the handlers for this
345 logger.
346
347 The specified loggers will be configured according to the level,
348 propagation, filters and handlers specified.
349
350 * *root* - this will be the configuration for the root logger.
351 Processing of the configuration will be as for any logger, except
352 that the ``propagate`` setting will not be applicable.
353
354 * *incremental* - whether the configuration is to be interpreted as
355 incremental to the existing configuration. This value defaults to
356 ``False``, which means that the specified configuration replaces the
357 existing configuration with the same semantics as used by the
358 existing :func:`fileConfig` API.
359
360 If the specified value is ``True``, the configuration is processed
361 as described in the section on :ref:`logging-config-dict-incremental`.
362
363 * *disable_existing_loggers* - whether any existing non-root loggers are
364 to be disabled. This setting mirrors the parameter of the same name in
365 :func:`fileConfig`. If absent, this parameter defaults to ``True``.
366 This value is ignored if *incremental* is ``True``.
367
368 .. _logging-config-dict-incremental:
369
370 Incremental Configuration
371 """""""""""""""""""""""""
372
373 It is difficult to provide complete flexibility for incremental
374 configuration. For example, because objects such as filters
375 and formatters are anonymous, once a configuration is set up, it is
376 not possible to refer to such anonymous objects when augmenting a
377 configuration.
378
379 Furthermore, there is not a compelling case for arbitrarily altering
380 the object graph of loggers, handlers, filters, formatters at
381 run-time, once a configuration is set up; the verbosity of loggers and
382 handlers can be controlled just by setting levels (and, in the case of
383 loggers, propagation flags). Changing the object graph arbitrarily in
384 a safe way is problematic in a multi-threaded environment; while not
385 impossible, the benefits are not worth the complexity it adds to the
386 implementation.
387
388 Thus, when the ``incremental`` key of a configuration dict is present
389 and is ``True``, the system will completely ignore any ``formatters`` and
390 ``filters`` entries, and process only the ``level``
391 settings in the ``handlers`` entries, and the ``level`` and
392 ``propagate`` settings in the ``loggers`` and ``root`` entries.
393
394 Using a value in the configuration dict lets configurations to be sent
395 over the wire as pickled dicts to a socket listener. Thus, the logging
396 verbosity of a long-running application can be altered over time with
397 no need to stop and restart the application.
398
399 .. _logging-config-dict-connections:
400
401 Object connections
402 """"""""""""""""""
403
404 The schema describes a set of logging objects - loggers,
405 handlers, formatters, filters - which are connected to each other in
406 an object graph. Thus, the schema needs to represent connections
407 between the objects. For example, say that, once configured, a
408 particular logger has attached to it a particular handler. For the
409 purposes of this discussion, we can say that the logger represents the
410 source, and the handler the destination, of a connection between the
411 two. Of course in the configured objects this is represented by the
412 logger holding a reference to the handler. In the configuration dict,
413 this is done by giving each destination object an id which identifies
414 it unambiguously, and then using the id in the source object's
415 configuration to indicate that a connection exists between the source
416 and the destination object with that id.
417
418 So, for example, consider the following YAML snippet:
419
420 .. code-block:: yaml
421
422 formatters:
423 brief:
424 # configuration for formatter with id 'brief' goes here
425 precise:
426 # configuration for formatter with id 'precise' goes here
427 handlers:
428 h1: #This is an id
429 # configuration of handler with id 'h1' goes here
430 formatter: brief
431 h2: #This is another id
432 # configuration of handler with id 'h2' goes here
433 formatter: precise
434 loggers:
435 foo.bar.baz:
436 # other configuration for logger 'foo.bar.baz'
437 handlers: [h1, h2]
438
439 (Note: YAML used here because it's a little more readable than the
440 equivalent Python source form for the dictionary.)
441
442 The ids for loggers are the logger names which would be used
443 programmatically to obtain a reference to those loggers, e.g.
444 ``foo.bar.baz``. The ids for Formatters and Filters can be any string
445 value (such as ``brief``, ``precise`` above) and they are transient,
446 in that they are only meaningful for processing the configuration
447 dictionary and used to determine connections between objects, and are
448 not persisted anywhere when the configuration call is complete.
449
450 The above snippet indicates that logger named ``foo.bar.baz`` should
451 have two handlers attached to it, which are described by the handler
452 ids ``h1`` and ``h2``. The formatter for ``h1`` is that described by id
453 ``brief``, and the formatter for ``h2`` is that described by id
454 ``precise``.
455
456
457 .. _logging-config-dict-userdef:
458
459 User-defined objects
460 """"""""""""""""""""
461
462 The schema supports user-defined objects for handlers, filters and
463 formatters. (Loggers do not need to have different types for
464 different instances, so there is no support in this configuration
465 schema for user-defined logger classes.)
466
467 Objects to be configured are described by dictionaries
468 which detail their configuration. In some places, the logging system
469 will be able to infer from the context how an object is to be
470 instantiated, but when a user-defined object is to be instantiated,
471 the system will not know how to do this. In order to provide complete
472 flexibility for user-defined object instantiation, the user needs
473 to provide a 'factory' - a callable which is called with a
474 configuration dictionary and which returns the instantiated object.
475 This is signalled by an absolute import path to the factory being
476 made available under the special key ``'()'``. Here's a concrete
477 example:
478
479 .. code-block:: yaml
480
481 formatters:
482 brief:
483 format: '%(message)s'
484 default:
485 format: '%(asctime)s %(levelname)-8s %(name)-15s %(message)s'
486 datefmt: '%Y-%m-%d %H:%M:%S'
487 custom:
488 (): my.package.customFormatterFactory
489 bar: baz
490 spam: 99.9
491 answer: 42
492
493 The above YAML snippet defines three formatters. The first, with id
494 ``brief``, is a standard :class:`logging.Formatter` instance with the
495 specified format string. The second, with id ``default``, has a
496 longer format and also defines the time format explicitly, and will
497 result in a :class:`logging.Formatter` initialized with those two format
498 strings. Shown in Python source form, the ``brief`` and ``default``
499 formatters have configuration sub-dictionaries::
500
501 {
502 'format' : '%(message)s'
503 }
504
505 and::
506
507 {
508 'format' : '%(asctime)s %(levelname)-8s %(name)-15s %(message)s',
509 'datefmt' : '%Y-%m-%d %H:%M:%S'
510 }
511
512 respectively, and as these dictionaries do not contain the special key
513 ``'()'``, the instantiation is inferred from the context: as a result,
514 standard :class:`logging.Formatter` instances are created. The
515 configuration sub-dictionary for the third formatter, with id
516 ``custom``, is::
517
518 {
519 '()' : 'my.package.customFormatterFactory',
520 'bar' : 'baz',
521 'spam' : 99.9,
522 'answer' : 42
523 }
524
525 and this contains the special key ``'()'``, which means that
526 user-defined instantiation is wanted. In this case, the specified
527 factory callable will be used. If it is an actual callable it will be
528 used directly - otherwise, if you specify a string (as in the example)
529 the actual callable will be located using normal import mechanisms.
530 The callable will be called with the **remaining** items in the
531 configuration sub-dictionary as keyword arguments. In the above
532 example, the formatter with id ``custom`` will be assumed to be
533 returned by the call::
534
535 my.package.customFormatterFactory(bar='baz', spam=99.9, answer=42)
536
537 .. warning:: The values for keys such as ``bar``, ``spam`` and ``answer`` in
538 the above example should not be configuration dictionaries or references such
539 as ``cfg://foo`` or ``ext://bar``, because they will not be processed by the
540 configuration machinery, but passed to the callable as-is.
541
542 The key ``'()'`` has been used as the special key because it is not a
543 valid keyword parameter name, and so will not clash with the names of
544 the keyword arguments used in the call. The ``'()'`` also serves as a
545 mnemonic that the corresponding value is a callable.
546
547 .. versionchanged:: 3.11
548 The ``filters`` member of ``handlers`` and ``loggers`` can take
549 filter instances in addition to ids.
550
551 You can also specify a special key ``'.'`` whose value is a
552 mapping of attribute names to values. If found, the specified attributes will
553 be set on the user-defined object before it is returned. Thus, with the
554 following configuration::
555
556 {
557 '()' : 'my.package.customFormatterFactory',
558 'bar' : 'baz',
559 'spam' : 99.9,
560 'answer' : 42,
561 '.' {
562 'foo': 'bar',
563 'baz': 'bozz'
564 }
565 }
566
567 the returned formatter will have attribute ``foo`` set to ``'bar'`` and
568 attribute ``baz`` set to ``'bozz'``.
569
570 .. warning:: The values for attributes such as ``foo`` and ``baz`` in
571 the above example should not be configuration dictionaries or references such
572 as ``cfg://foo`` or ``ext://bar``, because they will not be processed by the
573 configuration machinery, but set as attribute values as-is.
574
575
576 .. _handler-config-dict-order:
577
578 Handler configuration order
579 """""""""""""""""""""""""""
580
581 Handlers are configured in alphabetical order of their keys, and a configured
582 handler replaces the configuration dictionary in (a working copy of) the
583 ``handlers`` dictionary in the schema. If you use a construct such as
584 ``cfg://handlers.foo``, then initially ``handlers['foo']`` points to the
585 configuration dictionary for the handler named ``foo``, and later (once that
586 handler has been configured) it points to the configured handler instance.
587 Thus, ``cfg://handlers.foo`` could resolve to either a dictionary or a handler
588 instance. In general, it is wise to name handlers in a way such that dependent
589 handlers are configured *after* any handlers they depend on; that allows
590 something like ``cfg://handlers.foo`` to be used in configuring a handler that
591 depends on handler ``foo``. If that dependent handler were named ``bar``,
592 problems would result, because the configuration of ``bar`` would be attempted
593 before that of ``foo``, and ``foo`` would not yet have been configured.
594 However, if the dependent handler were named ``foobar``, it would be configured
595 after ``foo``, with the result that ``cfg://handlers.foo`` would resolve to
596 configured handler ``foo``, and not its configuration dictionary.
597
598
599 .. _logging-config-dict-externalobj:
600
601 Access to external objects
602 """"""""""""""""""""""""""
603
604 There are times where a configuration needs to refer to objects
605 external to the configuration, for example ``sys.stderr``. If the
606 configuration dict is constructed using Python code, this is
607 straightforward, but a problem arises when the configuration is
608 provided via a text file (e.g. JSON, YAML). In a text file, there is
609 no standard way to distinguish ``sys.stderr`` from the literal string
610 ``'sys.stderr'``. To facilitate this distinction, the configuration
611 system looks for certain special prefixes in string values and
612 treat them specially. For example, if the literal string
613 ``'ext://sys.stderr'`` is provided as a value in the configuration,
614 then the ``ext://`` will be stripped off and the remainder of the
615 value processed using normal import mechanisms.
616
617 The handling of such prefixes is done in a way analogous to protocol
618 handling: there is a generic mechanism to look for prefixes which
619 match the regular expression ``^(?P<prefix>[a-z]+)://(?P<suffix>.*)$``
620 whereby, if the ``prefix`` is recognised, the ``suffix`` is processed
621 in a prefix-dependent manner and the result of the processing replaces
622 the string value. If the prefix is not recognised, then the string
623 value will be left as-is.
624
625
626 .. _logging-config-dict-internalobj:
627
628 Access to internal objects
629 """"""""""""""""""""""""""
630
631 As well as external objects, there is sometimes also a need to refer
632 to objects in the configuration. This will be done implicitly by the
633 configuration system for things that it knows about. For example, the
634 string value ``'DEBUG'`` for a ``level`` in a logger or handler will
635 automatically be converted to the value ``logging.DEBUG``, and the
636 ``handlers``, ``filters`` and ``formatter`` entries will take an
637 object id and resolve to the appropriate destination object.
638
639 However, a more generic mechanism is needed for user-defined
640 objects which are not known to the :mod:`logging` module. For
641 example, consider :class:`logging.handlers.MemoryHandler`, which takes
642 a ``target`` argument which is another handler to delegate to. Since
643 the system already knows about this class, then in the configuration,
644 the given ``target`` just needs to be the object id of the relevant
645 target handler, and the system will resolve to the handler from the
646 id. If, however, a user defines a ``my.package.MyHandler`` which has
647 an ``alternate`` handler, the configuration system would not know that
648 the ``alternate`` referred to a handler. To cater for this, a generic
649 resolution system allows the user to specify:
650
651 .. code-block:: yaml
652
653 handlers:
654 file:
655 # configuration of file handler goes here
656
657 custom:
658 (): my.package.MyHandler
659 alternate: cfg://handlers.file
660
661 The literal string ``'cfg://handlers.file'`` will be resolved in an
662 analogous way to strings with the ``ext://`` prefix, but looking
663 in the configuration itself rather than the import namespace. The
664 mechanism allows access by dot or by index, in a similar way to
665 that provided by ``str.format``. Thus, given the following snippet:
666
667 .. code-block:: yaml
668
669 handlers:
670 email:
671 class: logging.handlers.SMTPHandler
672 mailhost: localhost
673 fromaddr: [email protected]
674 toaddrs:
675 - [email protected]
676 - [email protected]
677 subject: Houston, we have a problem.
678
679 in the configuration, the string ``'cfg://handlers'`` would resolve to
680 the dict with key ``handlers``, the string ``'cfg://handlers.email``
681 would resolve to the dict with key ``email`` in the ``handlers`` dict,
682 and so on. The string ``'cfg://handlers.email.toaddrs[1]`` would
683 resolve to ``'[email protected]'`` and the string
684 ``'cfg://handlers.email.toaddrs[0]'`` would resolve to the value
685 ``'[email protected]'``. The ``subject`` value could be accessed
686 using either ``'cfg://handlers.email.subject'`` or, equivalently,
687 ``'cfg://handlers.email[subject]'``. The latter form only needs to be
688 used if the key contains spaces or non-alphanumeric characters. Please note
689 that the characters ``[`` and ``]`` are not allowed in the keys. If an
690 index value consists only of decimal digits, access will be attempted
691 using the corresponding integer value, falling back to the string
692 value if needed.
693
694 Given a string ``cfg://handlers.myhandler.mykey.123``, this will
695 resolve to ``config_dict['handlers']['myhandler']['mykey']['123']``.
696 If the string is specified as ``cfg://handlers.myhandler.mykey[123]``,
697 the system will attempt to retrieve the value from
698 ``config_dict['handlers']['myhandler']['mykey'][123]``, and fall back
699 to ``config_dict['handlers']['myhandler']['mykey']['123']`` if that
700 fails.
701
702
703 .. _logging-import-resolution:
704
705 Import resolution and custom importers
706 """"""""""""""""""""""""""""""""""""""
707
708 Import resolution, by default, uses the builtin :func:`__import__` function
709 to do its importing. You may want to replace this with your own importing
710 mechanism: if so, you can replace the :attr:`importer` attribute of the
711 :class:`DictConfigurator` or its superclass, the
712 :class:`BaseConfigurator` class. However, you need to be
713 careful because of the way functions are accessed from classes via
714 descriptors. If you are using a Python callable to do your imports, and you
715 want to define it at class level rather than instance level, you need to wrap
716 it with :func:`staticmethod`. For example::
717
718 from importlib import import_module
719 from logging.config import BaseConfigurator
720
721 BaseConfigurator.importer = staticmethod(import_module)
722
723 You don't need to wrap with :func:`staticmethod` if you're setting the import
724 callable on a configurator *instance*.
725
726 .. _configure-queue:
727
728 Configuring QueueHandler and QueueListener
729 """"""""""""""""""""""""""""""""""""""""""
730
731 If you want to configure a :class:`~logging.handlers.QueueHandler`, noting that this
732 is normally used in conjunction with a :class:`~logging.handlers.QueueListener`, you
733 can configure both together. After the configuration, the ``QueueListener`` instance
734 will be available as the :attr:`~logging.handlers.QueueHandler.listener` attribute of
735 the created handler, and that in turn will be available to you using
736 :func:`~logging.getHandlerByName` and passing the name you have used for the
737 ``QueueHandler`` in your configuration. The dictionary schema for configuring the pair
738 is shown in the example YAML snippet below.
739
740 .. code-block:: yaml
741
742 handlers:
743 qhand:
744 class: logging.handlers.QueueHandler
745 queue: my.module.queue_factory
746 listener: my.package.CustomListener
747 handlers:
748 - hand_name_1
749 - hand_name_2
750 ...
751
752 The ``queue`` and ``listener`` keys are optional.
753
754 If the ``queue`` key is present, the corresponding value can be one of the following:
755
756 * An object implementing the :meth:`Queue.put_nowait <queue.Queue.put_nowait>`
757 and :meth:`Queue.get <queue.Queue.get>` public API. For instance, this may be
758 an actual instance of :class:`queue.Queue` or a subclass thereof, or a proxy
759 obtained by :meth:`multiprocessing.managers.SyncManager.Queue`.
760
761 This is of course only possible if you are constructing or modifying
762 the configuration dictionary in code.
763
764 * A string that resolves to a callable which, when called with no arguments, returns
765 the queue instance to use. That callable could be a :class:`queue.Queue` subclass
766 or a function which returns a suitable queue instance,
767 such as ``my.module.queue_factory()``.
768
769 * A dict with a ``'()'`` key which is constructed in the usual way as discussed in
770 :ref:`logging-config-dict-userdef`. The result of this construction should be a
771 :class:`queue.Queue` instance.
772
773 If the ``queue`` key is absent, a standard unbounded :class:`queue.Queue` instance is
774 created and used.
775
776 If the ``listener`` key is present, the corresponding value can be one of the following:
777
778 * A subclass of :class:`logging.handlers.QueueListener`. This is of course only
779 possible if you are constructing or modifying the configuration dictionary in
780 code.
781
782 * A string which resolves to a class which is a subclass of ``QueueListener``, such as
783 ``'my.package.CustomListener'``.
784
785 * A dict with a ``'()'`` key which is constructed in the usual way as discussed in
786 :ref:`logging-config-dict-userdef`. The result of this construction should be a
787 callable with the same signature as the ``QueueListener`` initializer.
788
789 If the ``listener`` key is absent, :class:`logging.handlers.QueueListener` is used.
790
791 The values under the ``handlers`` key are the names of other handlers in the
792 configuration (not shown in the above snippet) which will be passed to the queue
793 listener.
794
795 Any custom queue handler and listener classes will need to be defined with the same
796 initialization signatures as :class:`~logging.handlers.QueueHandler` and
797 :class:`~logging.handlers.QueueListener`.
798
799 .. versionadded:: 3.12
800
801 .. _logging-config-fileformat:
802
803 Configuration file format
804 ^^^^^^^^^^^^^^^^^^^^^^^^^
805
806 The configuration file format understood by :func:`fileConfig` is based on
807 :mod:`configparser` functionality. The file must contain sections called
808 ``[loggers]``, ``[handlers]`` and ``[formatters]`` which identify by name the
809 entities of each type which are defined in the file. For each such entity, there
810 is a separate section which identifies how that entity is configured. Thus, for
811 a logger named ``log01`` in the ``[loggers]`` section, the relevant
812 configuration details are held in a section ``[logger_log01]``. Similarly, a
813 handler called ``hand01`` in the ``[handlers]`` section will have its
814 configuration held in a section called ``[handler_hand01]``, while a formatter
815 called ``form01`` in the ``[formatters]`` section will have its configuration
816 specified in a section called ``[formatter_form01]``. The root logger
817 configuration must be specified in a section called ``[logger_root]``.
818
819 .. note::
820
821 The :func:`fileConfig` API is older than the :func:`dictConfig` API and does
822 not provide functionality to cover certain aspects of logging. For example,
823 you cannot configure :class:`~logging.Filter` objects, which provide for
824 filtering of messages beyond simple integer levels, using :func:`fileConfig`.
825 If you need to have instances of :class:`~logging.Filter` in your logging
826 configuration, you will need to use :func:`dictConfig`. Note that future
827 enhancements to configuration functionality will be added to
828 :func:`dictConfig`, so it's worth considering transitioning to this newer
829 API when it's convenient to do so.
830
831 Examples of these sections in the file are given below.
832
833 .. code-block:: ini
834
835 [loggers]
836 keys=root,log02,log03,log04,log05,log06,log07
837
838 [handlers]
839 keys=hand01,hand02,hand03,hand04,hand05,hand06,hand07,hand08,hand09
840
841 [formatters]
842 keys=form01,form02,form03,form04,form05,form06,form07,form08,form09
843
844 The root logger must specify a level and a list of handlers. An example of a
845 root logger section is given below.
846
847 .. code-block:: ini
848
849 [logger_root]
850 level=NOTSET
851 handlers=hand01
852
853 The ``level`` entry can be one of ``DEBUG, INFO, WARNING, ERROR, CRITICAL`` or
854 ``NOTSET``. For the root logger only, ``NOTSET`` means that all messages will be
855 logged. Level values are :ref:`evaluated <func-eval>` in the context of the ``logging``
856 package's namespace.
857
858 The ``handlers`` entry is a comma-separated list of handler names, which must
859 appear in the ``[handlers]`` section. These names must appear in the
860 ``[handlers]`` section and have corresponding sections in the configuration
861 file.
862
863 For loggers other than the root logger, some additional information is required.
864 This is illustrated by the following example.
865
866 .. code-block:: ini
867
868 [logger_parser]
869 level=DEBUG
870 handlers=hand01
871 propagate=1
872 qualname=compiler.parser
873
874 The ``level`` and ``handlers`` entries are interpreted as for the root logger,
875 except that if a non-root logger's level is specified as ``NOTSET``, the system
876 consults loggers higher up the hierarchy to determine the effective level of the
877 logger. The ``propagate`` entry is set to 1 to indicate that messages must
878 propagate to handlers higher up the logger hierarchy from this logger, or 0 to
879 indicate that messages are **not** propagated to handlers up the hierarchy. The
880 ``qualname`` entry is the hierarchical channel name of the logger, that is to
881 say the name used by the application to get the logger.
882
883 Sections which specify handler configuration are exemplified by the following.
884
885 .. code-block:: ini
886
887 [handler_hand01]
888 class=StreamHandler
889 level=NOTSET
890 formatter=form01
891 args=(sys.stdout,)
892
893 The ``class`` entry indicates the handler's class (as determined by :func:`eval`
894 in the ``logging`` package's namespace). The ``level`` is interpreted as for
895 loggers, and ``NOTSET`` is taken to mean 'log everything'.
896
897 The ``formatter`` entry indicates the key name of the formatter for this
898 handler. If blank, a default formatter (``logging._defaultFormatter``) is used.
899 If a name is specified, it must appear in the ``[formatters]`` section and have
900 a corresponding section in the configuration file.
901
902 The ``args`` entry, when :ref:`evaluated <func-eval>` in the context of the ``logging``
903 package's namespace, is the list of arguments to the constructor for the handler
904 class. Refer to the constructors for the relevant handlers, or to the examples
905 below, to see how typical entries are constructed. If not provided, it defaults
906 to ``()``.
907
908 The optional ``kwargs`` entry, when :ref:`evaluated <func-eval>` in the context of the
909 ``logging`` package's namespace, is the keyword argument dict to the constructor
910 for the handler class. If not provided, it defaults to ``{}``.
911
912 .. code-block:: ini
913
914 [handler_hand02]
915 class=FileHandler
916 level=DEBUG
917 formatter=form02
918 args=('python.log', 'w')
919
920 [handler_hand03]
921 class=handlers.SocketHandler
922 level=INFO
923 formatter=form03
924 args=('localhost', handlers.DEFAULT_TCP_LOGGING_PORT)
925
926 [handler_hand04]
927 class=handlers.DatagramHandler
928 level=WARN
929 formatter=form04
930 args=('localhost', handlers.DEFAULT_UDP_LOGGING_PORT)
931
932 [handler_hand05]
933 class=handlers.SysLogHandler
934 level=ERROR
935 formatter=form05
936 args=(('localhost', handlers.SYSLOG_UDP_PORT), handlers.SysLogHandler.LOG_USER)
937
938 [handler_hand06]
939 class=handlers.NTEventLogHandler
940 level=CRITICAL
941 formatter=form06
942 args=('Python Application', '', 'Application')
943
944 [handler_hand07]
945 class=handlers.SMTPHandler
946 level=WARN
947 formatter=form07
948 args=('localhost', 'from@abc', ['user1@abc', 'user2@xyz'], 'Logger Subject')
949 kwargs={'timeout': 10.0}
950
951 [handler_hand08]
952 class=handlers.MemoryHandler
953 level=NOTSET
954 formatter=form08
955 target=
956 args=(10, ERROR)
957
958 [handler_hand09]
959 class=handlers.HTTPHandler
960 level=NOTSET
961 formatter=form09
962 args=('localhost:9022', '/log', 'GET')
963 kwargs={'secure': True}
964
965 Sections which specify formatter configuration are typified by the following.
966
967 .. code-block:: ini
968
969 [formatter_form01]
970 format=F1 %(asctime)s %(levelname)s %(message)s %(customfield)s
971 datefmt=
972 style=%
973 validate=True
974 defaults={'customfield': 'defaultvalue'}
975 class=logging.Formatter
976
977 The arguments for the formatter configuration are the same as the keys
978 in the dictionary schema :ref:`formatters section
979 <logging-config-dictschema-formatters>`.
980
981 The ``defaults`` entry, when :ref:`evaluated <func-eval>` in the context of
982 the ``logging`` package's namespace, is a dictionary of default values for
983 custom formatting fields. If not provided, it defaults to ``None``.
984
985
986 .. note::
987
988 Due to the use of :func:`eval` as described above, there are
989 potential security risks which result from using the :func:`listen` to send
990 and receive configurations via sockets. The risks are limited to where
991 multiple users with no mutual trust run code on the same machine; see the
992 :func:`listen` documentation for more information.
993
994 .. seealso::
995
996 Module :mod:`logging`
997 API reference for the logging module.
998
999 Module :mod:`logging.handlers`
1000 Useful handlers included with the logging module.