Debian Codegraph @

Doc/library/contextlib.rst

raw
1 :mod:`!contextlib` --- Utilities for :keyword:`!with`\ -statement contexts
2 ==========================================================================
3
4 .. module:: contextlib
5 :synopsis: Utilities for with-statement contexts.
6
7 **Source code:** :source:`Lib/contextlib.py`
8
9 --------------
10
11 This module provides utilities for common tasks involving the :keyword:`with`
12 statement. For more information see also :ref:`typecontextmanager` and
13 :ref:`context-managers`.
14
15
16 Utilities
17 ---------
18
19 Functions and classes provided:
20
21 .. class:: AbstractContextManager
22
23 An :term:`abstract base class` for classes that implement
24 :meth:`~object.__enter__` and :meth:`~object.__exit__`. A default
25 implementation for :meth:`~object.__enter__` is provided which returns
26 ``self`` while :meth:`~object.__exit__` is an abstract method which by default
27 returns ``None``. See also the definition of :ref:`typecontextmanager`.
28
29 .. versionadded:: 3.6
30
31
32 .. class:: AbstractAsyncContextManager
33
34 An :term:`abstract base class` for classes that implement
35 :meth:`~object.__aenter__` and :meth:`~object.__aexit__`. A default
36 implementation for :meth:`~object.__aenter__` is provided which returns
37 ``self`` while :meth:`~object.__aexit__` is an abstract method which by default
38 returns ``None``. See also the definition of
39 :ref:`async-context-managers`.
40
41 .. versionadded:: 3.7
42
43
44 .. decorator:: contextmanager
45
46 This function is a :term:`decorator` that can be used to define a factory
47 function for :keyword:`with` statement context managers, without needing to
48 create a class or separate :meth:`~object.__enter__` and :meth:`~object.__exit__` methods.
49
50 While many objects natively support use in with statements, sometimes a
51 resource needs to be managed that isn't a context manager in its own right,
52 and doesn't implement a ``close()`` method for use with ``contextlib.closing``.
53
54 An abstract example would be the following to ensure correct resource
55 management::
56
57 from contextlib import contextmanager
58
59 @contextmanager
60 def managed_resource(*args, **kwds):
61 # Code to acquire resource, e.g.:
62 resource = acquire_resource(*args, **kwds)
63 try:
64 yield resource
65 finally:
66 # Code to release resource, e.g.:
67 release_resource(resource)
68
69 The function can then be used like this::
70
71 >>> with managed_resource(timeout=3600) as resource:
72 ... # Resource is released at the end of this block,
73 ... # even if code in the block raises an exception
74
75 The function being decorated must return a :term:`generator`-iterator when
76 called. This iterator must yield exactly one value, which will be bound to
77 the targets in the :keyword:`with` statement's :keyword:`!as` clause, if any.
78
79 At the point where the generator yields, the block nested in the :keyword:`with`
80 statement is executed. The generator is then resumed after the block is exited.
81 If an unhandled exception occurs in the block, it is reraised inside the
82 generator at the point where the yield occurred. Thus, you can use a
83 :keyword:`try`...\ :keyword:`except`...\ :keyword:`finally` statement to trap
84 the error (if any), or ensure that some cleanup takes place. If an exception is
85 trapped merely in order to log it or to perform some action (rather than to
86 suppress it entirely), the generator must reraise that exception. Otherwise the
87 generator context manager will indicate to the :keyword:`!with` statement that
88 the exception has been handled, and execution will resume with the statement
89 immediately following the :keyword:`!with` statement.
90
91 :func:`contextmanager` uses :class:`ContextDecorator` so the context managers
92 it creates can be used as decorators as well as in :keyword:`with` statements.
93 When used as a decorator, a new generator instance is implicitly created on
94 each function call (this allows the otherwise "one-shot" context managers
95 created by :func:`contextmanager` to meet the requirement that context
96 managers support multiple invocations in order to be used as decorators).
97
98 .. versionchanged:: 3.2
99 Use of :class:`ContextDecorator`.
100
101
102 .. decorator:: asynccontextmanager
103
104 Similar to :func:`~contextlib.contextmanager`, but creates an
105 :ref:`asynchronous context manager <async-context-managers>`.
106
107 This function is a :term:`decorator` that can be used to define a factory
108 function for :keyword:`async with` statement asynchronous context managers,
109 without needing to create a class or separate :meth:`~object.__aenter__` and
110 :meth:`~object.__aexit__` methods. It must be applied to an :term:`asynchronous
111 generator` function.
112
113 A simple example::
114
115 from contextlib import asynccontextmanager
116
117 @asynccontextmanager
118 async def get_connection():
119 conn = await acquire_db_connection()
120 try:
121 yield conn
122 finally:
123 await release_db_connection(conn)
124
125 async def get_all_users():
126 async with get_connection() as conn:
127 return conn.query('SELECT ...')
128
129 .. versionadded:: 3.7
130
131 Context managers defined with :func:`asynccontextmanager` can be used
132 either as decorators or with :keyword:`async with` statements::
133
134 import time
135 from contextlib import asynccontextmanager
136
137 @asynccontextmanager
138 async def timeit():
139 now = time.monotonic()
140 try:
141 yield
142 finally:
143 print(f'it took {time.monotonic() - now}s to run')
144
145 @timeit()
146 async def main():
147 # ... async code ...
148
149 When used as a decorator, a new generator instance is implicitly created on
150 each function call. This allows the otherwise "one-shot" context managers
151 created by :func:`asynccontextmanager` to meet the requirement that context
152 managers support multiple invocations in order to be used as decorators.
153
154 .. versionchanged:: 3.10
155 Async context managers created with :func:`asynccontextmanager` can
156 be used as decorators.
157
158
159 .. function:: closing(thing)
160
161 Return a context manager that closes *thing* upon completion of the block. This
162 is basically equivalent to::
163
164 from contextlib import contextmanager
165
166 @contextmanager
167 def closing(thing):
168 try:
169 yield thing
170 finally:
171 thing.close()
172
173 And lets you write code like this::
174
175 from contextlib import closing
176 from urllib.request import urlopen
177
178 with closing(urlopen('https://www.python.org')) as page:
179 for line in page:
180 print(line)
181
182 without needing to explicitly close ``page``. Even if an error occurs,
183 ``page.close()`` will be called when the :keyword:`with` block is exited.
184
185 .. note::
186
187 Most types managing resources support the :term:`context manager` protocol,
188 which closes *thing* on leaving the :keyword:`with` statement.
189 As such, :func:`!closing` is most useful for third party types that don't
190 support context managers.
191 This example is purely for illustration purposes,
192 as :func:`~urllib.request.urlopen` would normally be used in a context manager.
193
194 .. function:: aclosing(thing)
195
196 Return an async context manager that calls the ``aclose()`` method of *thing*
197 upon completion of the block. This is basically equivalent to::
198
199 from contextlib import asynccontextmanager
200
201 @asynccontextmanager
202 async def aclosing(thing):
203 try:
204 yield thing
205 finally:
206 await thing.aclose()
207
208 Significantly, ``aclosing()`` supports deterministic cleanup of async
209 generators when they happen to exit early by :keyword:`break` or an
210 exception. For example::
211
212 from contextlib import aclosing
213
214 async with aclosing(my_generator()) as values:
215 async for value in values:
216 if value == 42:
217 break
218
219 This pattern ensures that the generator's async exit code is executed in
220 the same context as its iterations (so that exceptions and context
221 variables work as expected, and the exit code isn't run after the
222 lifetime of some task it depends on).
223
224 .. versionadded:: 3.10
225
226
227 .. _simplifying-support-for-single-optional-context-managers:
228
229 .. function:: nullcontext(enter_result=None)
230
231 Return a context manager that returns *enter_result* from :meth:`~object.__enter__`, but
232 otherwise does nothing. It is intended to be used as a stand-in for an
233 optional context manager, for example::
234
235 def myfunction(arg, ignore_exceptions=False):
236 if ignore_exceptions:
237 # Use suppress to ignore all exceptions.
238 cm = contextlib.suppress(Exception)
239 else:
240 # Do not ignore any exceptions, cm has no effect.
241 cm = contextlib.nullcontext()
242 with cm:
243 # Do something
244
245 An example using *enter_result*::
246
247 def process_file(file_or_path):
248 if isinstance(file_or_path, str):
249 # If string, open file
250 cm = open(file_or_path)
251 else:
252 # Caller is responsible for closing file
253 cm = nullcontext(file_or_path)
254
255 with cm as file:
256 # Perform processing on the file
257
258 It can also be used as a stand-in for
259 :ref:`asynchronous context managers <async-context-managers>`::
260
261 async def send_http(session=None):
262 if not session:
263 # If no http session, create it with aiohttp
264 cm = aiohttp.ClientSession()
265 else:
266 # Caller is responsible for closing the session
267 cm = nullcontext(session)
268
269 async with cm as session:
270 # Send http requests with session
271
272 .. versionadded:: 3.7
273
274 .. versionchanged:: 3.10
275 :term:`asynchronous context manager` support was added.
276
277
278
279 .. function:: suppress(*exceptions)
280
281 Return a context manager that suppresses any of the specified exceptions
282 if they occur in the body of a :keyword:`!with` statement and then
283 resumes execution with the first statement following the end of the
284 :keyword:`!with` statement.
285
286 As with any other mechanism that completely suppresses exceptions, this
287 context manager should be used only to cover very specific errors where
288 silently continuing with program execution is known to be the right
289 thing to do.
290
291 For example::
292
293 from contextlib import suppress
294
295 with suppress(FileNotFoundError):
296 os.remove('somefile.tmp')
297
298 with suppress(FileNotFoundError):
299 os.remove('someotherfile.tmp')
300
301 This code is equivalent to::
302
303 try:
304 os.remove('somefile.tmp')
305 except FileNotFoundError:
306 pass
307
308 try:
309 os.remove('someotherfile.tmp')
310 except FileNotFoundError:
311 pass
312
313 This context manager is :ref:`reentrant <reentrant-cms>`.
314
315 If the code within the :keyword:`!with` block raises a
316 :exc:`BaseExceptionGroup`, suppressed exceptions are removed from the
317 group. Any exceptions of the group which are not suppressed are re-raised in
318 a new group which is created using the original group's :meth:`~BaseExceptionGroup.derive`
319 method.
320
321 .. versionadded:: 3.4
322
323 .. versionchanged:: 3.12
324 ``suppress`` now supports suppressing exceptions raised as
325 part of a :exc:`BaseExceptionGroup`.
326
327 .. function:: redirect_stdout(new_target)
328
329 Context manager for temporarily redirecting :data:`sys.stdout` to
330 another file or file-like object.
331
332 This tool adds flexibility to existing functions or classes whose output
333 is hardwired to stdout.
334
335 For example, the output of :func:`help` normally is sent to *sys.stdout*.
336 You can capture that output in a string by redirecting the output to an
337 :class:`io.StringIO` object. The replacement stream is returned from the
338 :meth:`~object.__enter__` method and so is available as the target of the
339 :keyword:`with` statement::
340
341 with redirect_stdout(io.StringIO()) as f:
342 help(pow)
343 s = f.getvalue()
344
345 To send the output of :func:`help` to a file on disk, redirect the output
346 to a regular file::
347
348 with open('help.txt', 'w') as f:
349 with redirect_stdout(f):
350 help(pow)
351
352 To send the output of :func:`help` to *sys.stderr*::
353
354 with redirect_stdout(sys.stderr):
355 help(pow)
356
357 Note that the global side effect on :data:`sys.stdout` means that this
358 context manager is not suitable for use in library code and most threaded
359 applications. It also has no effect on the output of subprocesses.
360 However, it is still a useful approach for many utility scripts.
361
362 This context manager is :ref:`reentrant <reentrant-cms>`.
363
364 .. versionadded:: 3.4
365
366
367 .. function:: redirect_stderr(new_target)
368
369 Similar to :func:`~contextlib.redirect_stdout` but redirecting
370 :data:`sys.stderr` to another file or file-like object.
371
372 This context manager is :ref:`reentrant <reentrant-cms>`.
373
374 .. versionadded:: 3.5
375
376
377 .. function:: chdir(path)
378
379 Non parallel-safe context manager to change the current working directory.
380 As this changes a global state, the working directory, it is not suitable
381 for use in most threaded or async contexts. It is also not suitable for most
382 non-linear code execution, like generators, where the program execution is
383 temporarily relinquished -- unless explicitly desired, you should not yield
384 when this context manager is active.
385
386 This is a simple wrapper around :func:`~os.chdir`, it changes the current
387 working directory upon entering and restores the old one on exit.
388
389 This context manager is :ref:`reentrant <reentrant-cms>`.
390
391 .. versionadded:: 3.11
392
393
394 .. class:: ContextDecorator()
395
396 A base class that enables a context manager to also be used as a decorator.
397
398 Context managers inheriting from ``ContextDecorator`` have to implement
399 :meth:`~object.__enter__` and :meth:`~object.__exit__` as normal.
400 ``__exit__`` retains its optional
401 exception handling even when used as a decorator.
402
403 ``ContextDecorator`` is used by :func:`contextmanager`, so you get this
404 functionality automatically.
405
406 Example of ``ContextDecorator``::
407
408 from contextlib import ContextDecorator
409
410 class mycontext(ContextDecorator):
411 def __enter__(self):
412 print('Starting')
413 return self
414
415 def __exit__(self, *exc):
416 print('Finishing')
417 return False
418
419 The class can then be used like this::
420
421 >>> @mycontext()
422 ... def function():
423 ... print('The bit in the middle')
424 ...
425 >>> function()
426 Starting
427 The bit in the middle
428 Finishing
429
430 >>> with mycontext():
431 ... print('The bit in the middle')
432 ...
433 Starting
434 The bit in the middle
435 Finishing
436
437 This change is just syntactic sugar for any construct of the following form::
438
439 def f():
440 with cm():
441 # Do stuff
442
443 ``ContextDecorator`` lets you instead write::
444
445 @cm()
446 def f():
447 # Do stuff
448
449 It makes it clear that the ``cm`` applies to the whole function, rather than
450 just a piece of it (and saving an indentation level is nice, too).
451
452 Existing context managers that already have a base class can be extended by
453 using ``ContextDecorator`` as a mixin class::
454
455 from contextlib import ContextDecorator
456
457 class mycontext(ContextBaseClass, ContextDecorator):
458 def __enter__(self):
459 return self
460
461 def __exit__(self, *exc):
462 return False
463
464 .. note::
465 As the decorated function must be able to be called multiple times, the
466 underlying context manager must support use in multiple :keyword:`with`
467 statements. If this is not the case, then the original construct with the
468 explicit :keyword:`!with` statement inside the function should be used.
469
470 .. versionadded:: 3.2
471
472
473 .. class:: AsyncContextDecorator
474
475 Similar to :class:`ContextDecorator` but only for asynchronous functions.
476
477 Example of ``AsyncContextDecorator``::
478
479 from asyncio import run
480 from contextlib import AsyncContextDecorator
481
482 class mycontext(AsyncContextDecorator):
483 async def __aenter__(self):
484 print('Starting')
485 return self
486
487 async def __aexit__(self, *exc):
488 print('Finishing')
489 return False
490
491 The class can then be used like this::
492
493 >>> @mycontext()
494 ... async def function():
495 ... print('The bit in the middle')
496 ...
497 >>> run(function())
498 Starting
499 The bit in the middle
500 Finishing
501
502 >>> async def function():
503 ... async with mycontext():
504 ... print('The bit in the middle')
505 ...
506 >>> run(function())
507 Starting
508 The bit in the middle
509 Finishing
510
511 .. versionadded:: 3.10
512
513
514 .. class:: ExitStack()
515
516 A context manager that is designed to make it easy to programmatically
517 combine other context managers and cleanup functions, especially those
518 that are optional or otherwise driven by input data.
519
520 For example, a set of files may easily be handled in a single with
521 statement as follows::
522
523 with ExitStack() as stack:
524 files = [stack.enter_context(open(fname)) for fname in filenames]
525 # All opened files will automatically be closed at the end of
526 # the with statement, even if attempts to open files later
527 # in the list raise an exception
528
529 The :meth:`~object.__enter__` method returns the :class:`ExitStack` instance, and
530 performs no additional operations.
531
532 Each instance maintains a stack of registered callbacks that are called in
533 reverse order when the instance is closed (either explicitly or implicitly
534 at the end of a :keyword:`with` statement). Note that callbacks are *not*
535 invoked implicitly when the context stack instance is garbage collected.
536
537 This stack model is used so that context managers that acquire their
538 resources in their ``__init__`` method (such as file objects) can be
539 handled correctly.
540
541 Since registered callbacks are invoked in the reverse order of
542 registration, this ends up behaving as if multiple nested :keyword:`with`
543 statements had been used with the registered set of callbacks. This even
544 extends to exception handling - if an inner callback suppresses or replaces
545 an exception, then outer callbacks will be passed arguments based on that
546 updated state.
547
548 This is a relatively low level API that takes care of the details of
549 correctly unwinding the stack of exit callbacks. It provides a suitable
550 foundation for higher level context managers that manipulate the exit
551 stack in application specific ways.
552
553 .. versionadded:: 3.3
554
555 .. method:: enter_context(cm)
556
557 Enters a new context manager and adds its :meth:`~object.__exit__` method to
558 the callback stack. The return value is the result of the context
559 manager's own :meth:`~object.__enter__` method.
560
561 These context managers may suppress exceptions just as they normally
562 would if used directly as part of a :keyword:`with` statement.
563
564 .. versionchanged:: 3.11
565 Raises :exc:`TypeError` instead of :exc:`AttributeError` if *cm*
566 is not a context manager.
567
568 .. method:: push(exit)
569
570 Adds a context manager's :meth:`~object.__exit__` method to the callback stack.
571
572 As ``__enter__`` is *not* invoked, this method can be used to cover
573 part of an :meth:`~object.__enter__` implementation with a context manager's own
574 :meth:`~object.__exit__` method.
575
576 If passed an object that is not a context manager, this method assumes
577 it is a callback with the same signature as a context manager's
578 :meth:`~object.__exit__` method and adds it directly to the callback stack.
579
580 By returning true values, these callbacks can suppress exceptions the
581 same way context manager :meth:`~object.__exit__` methods can.
582
583 The passed in object is returned from the function, allowing this
584 method to be used as a function decorator.
585
586 .. method:: callback(callback, /, *args, **kwds)
587
588 Accepts an arbitrary callback function and arguments and adds it to
589 the callback stack.
590
591 Unlike the other methods, callbacks added this way cannot suppress
592 exceptions (as they are never passed the exception details).
593
594 The passed in callback is returned from the function, allowing this
595 method to be used as a function decorator.
596
597 .. method:: pop_all()
598
599 Transfers the callback stack to a fresh :class:`ExitStack` instance
600 and returns it. No callbacks are invoked by this operation - instead,
601 they will now be invoked when the new stack is closed (either
602 explicitly or implicitly at the end of a :keyword:`with` statement).
603
604 For example, a group of files can be opened as an "all or nothing"
605 operation as follows::
606
607 with ExitStack() as stack:
608 files = [stack.enter_context(open(fname)) for fname in filenames]
609 # Hold onto the close method, but don't call it yet.
610 close_files = stack.pop_all().close
611 # If opening any file fails, all previously opened files will be
612 # closed automatically. If all files are opened successfully,
613 # they will remain open even after the with statement ends.
614 # close_files() can then be invoked explicitly to close them all.
615
616 .. method:: close()
617
618 Immediately unwinds the callback stack, invoking callbacks in the
619 reverse order of registration. For any context managers and exit
620 callbacks registered, the arguments passed in will indicate that no
621 exception occurred.
622
623 .. class:: AsyncExitStack()
624
625 An :ref:`asynchronous context manager <async-context-managers>`, similar
626 to :class:`ExitStack`, that supports combining both synchronous and
627 asynchronous context managers, as well as having coroutines for
628 cleanup logic.
629
630 The :meth:`~ExitStack.close` method is not implemented; :meth:`aclose` must be used
631 instead.
632
633 .. method:: enter_async_context(cm)
634 :async:
635
636 Similar to :meth:`ExitStack.enter_context` but expects an asynchronous context
637 manager.
638
639 .. versionchanged:: 3.11
640 Raises :exc:`TypeError` instead of :exc:`AttributeError` if *cm*
641 is not an asynchronous context manager.
642
643 .. method:: push_async_exit(exit)
644
645 Similar to :meth:`ExitStack.push` but expects either an asynchronous context manager
646 or a coroutine function.
647
648 .. method:: push_async_callback(callback, /, *args, **kwds)
649
650 Similar to :meth:`ExitStack.callback` but expects a coroutine function.
651
652 .. method:: aclose()
653 :async:
654
655 Similar to :meth:`ExitStack.close` but properly handles awaitables.
656
657 Continuing the example for :func:`asynccontextmanager`::
658
659 async with AsyncExitStack() as stack:
660 connections = [await stack.enter_async_context(get_connection())
661 for i in range(5)]
662 # All opened connections will automatically be released at the end of
663 # the async with statement, even if attempts to open a connection
664 # later in the list raise an exception.
665
666 .. versionadded:: 3.7
667
668 Examples and Recipes
669 --------------------
670
671 This section describes some examples and recipes for making effective use of
672 the tools provided by :mod:`!contextlib`.
673
674
675 Supporting a variable number of context managers
676 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
677
678 The primary use case for :class:`ExitStack` is the one given in the class
679 documentation: supporting a variable number of context managers and other
680 cleanup operations in a single :keyword:`with` statement. The variability
681 may come from the number of context managers needed being driven by user
682 input (such as opening a user specified collection of files), or from
683 some of the context managers being optional::
684
685 with ExitStack() as stack:
686 for resource in resources:
687 stack.enter_context(resource)
688 if need_special_resource():
689 special = acquire_special_resource()
690 stack.callback(release_special_resource, special)
691 # Perform operations that use the acquired resources
692
693 As shown, :class:`ExitStack` also makes it quite easy to use :keyword:`with`
694 statements to manage arbitrary resources that don't natively support the
695 context management protocol.
696
697
698 Catching exceptions from ``__enter__`` methods
699 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
700
701 It is occasionally desirable to catch exceptions from an :meth:`~object.__enter__`
702 method implementation, *without* inadvertently catching exceptions from
703 the :keyword:`with` statement body or the context manager's :meth:`~object.__exit__`
704 method. By using :class:`ExitStack` the steps in the context management
705 protocol can be separated slightly in order to allow this::
706
707 stack = ExitStack()
708 try:
709 x = stack.enter_context(cm)
710 except Exception:
711 # handle __enter__ exception
712 else:
713 with stack:
714 # Handle normal case
715
716 Actually needing to do this is likely to indicate that the underlying API
717 should be providing a direct resource management interface for use with
718 :keyword:`try`/:keyword:`except`/:keyword:`finally` statements, but not
719 all APIs are well designed in that regard. When a context manager is the
720 only resource management API provided, then :class:`ExitStack` can make it
721 easier to handle various situations that can't be handled directly in a
722 :keyword:`with` statement.
723
724
725 Cleaning up in an ``__enter__`` implementation
726 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
727
728 As noted in the documentation of :meth:`ExitStack.push`, this
729 method can be useful in cleaning up an already allocated resource if later
730 steps in the :meth:`~object.__enter__` implementation fail.
731
732 Here's an example of doing this for a context manager that accepts resource
733 acquisition and release functions, along with an optional validation function,
734 and maps them to the context management protocol::
735
736 from contextlib import contextmanager, AbstractContextManager, ExitStack
737
738 class ResourceManager(AbstractContextManager):
739
740 def __init__(self, acquire_resource, release_resource, check_resource_ok=None):
741 self.acquire_resource = acquire_resource
742 self.release_resource = release_resource
743 if check_resource_ok is None:
744 def check_resource_ok(resource):
745 return True
746 self.check_resource_ok = check_resource_ok
747
748 @contextmanager
749 def _cleanup_on_error(self):
750 with ExitStack() as stack:
751 stack.push(self)
752 yield
753 # The validation check passed and didn't raise an exception
754 # Accordingly, we want to keep the resource, and pass it
755 # back to our caller
756 stack.pop_all()
757
758 def __enter__(self):
759 resource = self.acquire_resource()
760 with self._cleanup_on_error():
761 if not self.check_resource_ok(resource):
762 msg = "Failed validation for {!r}"
763 raise RuntimeError(msg.format(resource))
764 return resource
765
766 def __exit__(self, *exc_details):
767 # We don't need to duplicate any of our resource release logic
768 self.release_resource()
769
770
771 Replacing any use of ``try-finally`` and flag variables
772 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
773
774 A pattern you will sometimes see is a ``try-finally`` statement with a flag
775 variable to indicate whether or not the body of the ``finally`` clause should
776 be executed. In its simplest form (that can't already be handled just by
777 using an ``except`` clause instead), it looks something like this::
778
779 cleanup_needed = True
780 try:
781 result = perform_operation()
782 if result:
783 cleanup_needed = False
784 finally:
785 if cleanup_needed:
786 cleanup_resources()
787
788 As with any ``try`` statement based code, this can cause problems for
789 development and review, because the setup code and the cleanup code can end
790 up being separated by arbitrarily long sections of code.
791
792 :class:`ExitStack` makes it possible to instead register a callback for
793 execution at the end of a ``with`` statement, and then later decide to skip
794 executing that callback::
795
796 from contextlib import ExitStack
797
798 with ExitStack() as stack:
799 stack.callback(cleanup_resources)
800 result = perform_operation()
801 if result:
802 stack.pop_all()
803
804 This allows the intended cleanup behaviour to be made explicit up front,
805 rather than requiring a separate flag variable.
806
807 If a particular application uses this pattern a lot, it can be simplified
808 even further by means of a small helper class::
809
810 from contextlib import ExitStack
811
812 class Callback(ExitStack):
813 def __init__(self, callback, /, *args, **kwds):
814 super().__init__()
815 self.callback(callback, *args, **kwds)
816
817 def cancel(self):
818 self.pop_all()
819
820 with Callback(cleanup_resources) as cb:
821 result = perform_operation()
822 if result:
823 cb.cancel()
824
825 If the resource cleanup isn't already neatly bundled into a standalone
826 function, then it is still possible to use the decorator form of
827 :meth:`ExitStack.callback` to declare the resource cleanup in
828 advance::
829
830 from contextlib import ExitStack
831
832 with ExitStack() as stack:
833 @stack.callback
834 def cleanup_resources():
835 ...
836 result = perform_operation()
837 if result:
838 stack.pop_all()
839
840 Due to the way the decorator protocol works, a callback function
841 declared this way cannot take any parameters. Instead, any resources to
842 be released must be accessed as closure variables.
843
844
845 Using a context manager as a function decorator
846 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
847
848 :class:`ContextDecorator` makes it possible to use a context manager in
849 both an ordinary ``with`` statement and also as a function decorator.
850
851 For example, it is sometimes useful to wrap functions or groups of statements
852 with a logger that can track the time of entry and time of exit. Rather than
853 writing both a function decorator and a context manager for the task,
854 inheriting from :class:`ContextDecorator` provides both capabilities in a
855 single definition::
856
857 from contextlib import ContextDecorator
858 import logging
859
860 logging.basicConfig(level=logging.INFO)
861
862 class track_entry_and_exit(ContextDecorator):
863 def __init__(self, name):
864 self.name = name
865
866 def __enter__(self):
867 logging.info('Entering: %s', self.name)
868
869 def __exit__(self, exc_type, exc, exc_tb):
870 logging.info('Exiting: %s', self.name)
871
872 Instances of this class can be used as both a context manager::
873
874 with track_entry_and_exit('widget loader'):
875 print('Some time consuming activity goes here')
876 load_widget()
877
878 And also as a function decorator::
879
880 @track_entry_and_exit('widget loader')
881 def activity():
882 print('Some time consuming activity goes here')
883 load_widget()
884
885 Note that there is one additional limitation when using context managers
886 as function decorators: there's no way to access the return value of
887 :meth:`~object.__enter__`. If that value is needed, then it is still necessary to use
888 an explicit ``with`` statement.
889
890 .. seealso::
891
892 :pep:`343` - The "with" statement
893 The specification, background, and examples for the Python :keyword:`with`
894 statement.
895
896 .. _single-use-reusable-and-reentrant-cms:
897
898 Single use, reusable and reentrant context managers
899 ---------------------------------------------------
900
901 Most context managers are written in a way that means they can only be
902 used effectively in a :keyword:`with` statement once. These single use
903 context managers must be created afresh each time they're used -
904 attempting to use them a second time will trigger an exception or
905 otherwise not work correctly.
906
907 This common limitation means that it is generally advisable to create
908 context managers directly in the header of the :keyword:`with` statement
909 where they are used (as shown in all of the usage examples above).
910
911 Files are an example of effectively single use context managers, since
912 the first :keyword:`with` statement will close the file, preventing any
913 further IO operations using that file object.
914
915 Context managers created using :func:`contextmanager` are also single use
916 context managers, and will complain about the underlying generator failing
917 to yield if an attempt is made to use them a second time::
918
919 >>> from contextlib import contextmanager
920 >>> @contextmanager
921 ... def singleuse():
922 ... print("Before")
923 ... yield
924 ... print("After")
925 ...
926 >>> cm = singleuse()
927 >>> with cm:
928 ... pass
929 ...
930 Before
931 After
932 >>> with cm:
933 ... pass
934 ...
935 Traceback (most recent call last):
936 ...
937 RuntimeError: generator didn't yield
938
939
940 .. _reentrant-cms:
941
942 Reentrant context managers
943 ^^^^^^^^^^^^^^^^^^^^^^^^^^
944
945 More sophisticated context managers may be "reentrant". These context
946 managers can not only be used in multiple :keyword:`with` statements,
947 but may also be used *inside* a :keyword:`!with` statement that is already
948 using the same context manager.
949
950 :class:`threading.RLock` is an example of a reentrant context manager, as are
951 :func:`suppress`, :func:`redirect_stdout`, and :func:`chdir`. Here's a very
952 simple example of reentrant use::
953
954 >>> from contextlib import redirect_stdout
955 >>> from io import StringIO
956 >>> stream = StringIO()
957 >>> write_to_stream = redirect_stdout(stream)
958 >>> with write_to_stream:
959 ... print("This is written to the stream rather than stdout")
960 ... with write_to_stream:
961 ... print("This is also written to the stream")
962 ...
963 >>> print("This is written directly to stdout")
964 This is written directly to stdout
965 >>> print(stream.getvalue())
966 This is written to the stream rather than stdout
967 This is also written to the stream
968
969 Real world examples of reentrancy are more likely to involve multiple
970 functions calling each other and hence be far more complicated than this
971 example.
972
973 Note also that being reentrant is *not* the same thing as being thread safe.
974 :func:`redirect_stdout`, for example, is definitely not thread safe, as it
975 makes a global modification to the system state by binding :data:`sys.stdout`
976 to a different stream.
977
978
979 .. _reusable-cms:
980
981 Reusable context managers
982 ^^^^^^^^^^^^^^^^^^^^^^^^^
983
984 Distinct from both single use and reentrant context managers are "reusable"
985 context managers (or, to be completely explicit, "reusable, but not
986 reentrant" context managers, since reentrant context managers are also
987 reusable). These context managers support being used multiple times, but
988 will fail (or otherwise not work correctly) if the specific context manager
989 instance has already been used in a containing with statement.
990
991 :class:`threading.Lock` is an example of a reusable, but not reentrant,
992 context manager (for a reentrant lock, it is necessary to use
993 :class:`threading.RLock` instead).
994
995 Another example of a reusable, but not reentrant, context manager is
996 :class:`ExitStack`, as it invokes *all* currently registered callbacks
997 when leaving any with statement, regardless of where those callbacks
998 were added::
999
1000 >>> from contextlib import ExitStack
1001 >>> stack = ExitStack()
1002 >>> with stack:
1003 ... stack.callback(print, "Callback: from first context")
1004 ... print("Leaving first context")
1005 ...
1006 Leaving first context
1007 Callback: from first context
1008 >>> with stack:
1009 ... stack.callback(print, "Callback: from second context")
1010 ... print("Leaving second context")
1011 ...
1012 Leaving second context
1013 Callback: from second context
1014 >>> with stack:
1015 ... stack.callback(print, "Callback: from outer context")
1016 ... with stack:
1017 ... stack.callback(print, "Callback: from inner context")
1018 ... print("Leaving inner context")
1019 ... print("Leaving outer context")
1020 ...
1021 Leaving inner context
1022 Callback: from inner context
1023 Callback: from outer context
1024 Leaving outer context
1025
1026 As the output from the example shows, reusing a single stack object across
1027 multiple with statements works correctly, but attempting to nest them
1028 will cause the stack to be cleared at the end of the innermost with
1029 statement, which is unlikely to be desirable behaviour.
1030
1031 Using separate :class:`ExitStack` instances instead of reusing a single
1032 instance avoids that problem::
1033
1034 >>> from contextlib import ExitStack
1035 >>> with ExitStack() as outer_stack:
1036 ... outer_stack.callback(print, "Callback: from outer context")
1037 ... with ExitStack() as inner_stack:
1038 ... inner_stack.callback(print, "Callback: from inner context")
1039 ... print("Leaving inner context")
1040 ... print("Leaving outer context")
1041 ...
1042 Leaving inner context
1043 Callback: from inner context
1044 Leaving outer context
1045 Callback: from outer context