| 1 |
:mod:`!contextlib` --- Utilities for :keyword:`!with`\ -statement contexts |
| 2 |
========================================================================== |
| 3 |
|
| 4 |
.. module:: contextlib |
| 5 |
:synopsis: Utilities for with-statement contexts. |
| 6 |
|
| 7 |
**Source code:** :source:`Lib/contextlib.py` |
| 8 |
|
| 9 |
-------------- |
| 10 |
|
| 11 |
This module provides utilities for common tasks involving the :keyword:`with` |
| 12 |
statement. For more information see also :ref:`typecontextmanager` and |
| 13 |
:ref:`context-managers`. |
| 14 |
|
| 15 |
|
| 16 |
Utilities |
| 17 |
--------- |
| 18 |
|
| 19 |
Functions and classes provided: |
| 20 |
|
| 21 |
.. class:: AbstractContextManager |
| 22 |
|
| 23 |
An :term:`abstract base class` for classes that implement |
| 24 |
:meth:`~object.__enter__` and :meth:`~object.__exit__`. A default |
| 25 |
implementation for :meth:`~object.__enter__` is provided which returns |
| 26 |
``self`` while :meth:`~object.__exit__` is an abstract method which by default |
| 27 |
returns ``None``. See also the definition of :ref:`typecontextmanager`. |
| 28 |
|
| 29 |
.. versionadded:: 3.6 |
| 30 |
|
| 31 |
|
| 32 |
.. class:: AbstractAsyncContextManager |
| 33 |
|
| 34 |
An :term:`abstract base class` for classes that implement |
| 35 |
:meth:`~object.__aenter__` and :meth:`~object.__aexit__`. A default |
| 36 |
implementation for :meth:`~object.__aenter__` is provided which returns |
| 37 |
``self`` while :meth:`~object.__aexit__` is an abstract method which by default |
| 38 |
returns ``None``. See also the definition of |
| 39 |
:ref:`async-context-managers`. |
| 40 |
|
| 41 |
.. versionadded:: 3.7 |
| 42 |
|
| 43 |
|
| 44 |
.. decorator:: contextmanager |
| 45 |
|
| 46 |
This function is a :term:`decorator` that can be used to define a factory |
| 47 |
function for :keyword:`with` statement context managers, without needing to |
| 48 |
create a class or separate :meth:`~object.__enter__` and :meth:`~object.__exit__` methods. |
| 49 |
|
| 50 |
While many objects natively support use in with statements, sometimes a |
| 51 |
resource needs to be managed that isn't a context manager in its own right, |
| 52 |
and doesn't implement a ``close()`` method for use with ``contextlib.closing``. |
| 53 |
|
| 54 |
An abstract example would be the following to ensure correct resource |
| 55 |
management:: |
| 56 |
|
| 57 |
from contextlib import contextmanager |
| 58 |
|
| 59 |
@contextmanager |
| 60 |
def managed_resource(*args, **kwds): |
| 61 |
# Code to acquire resource, e.g.: |
| 62 |
resource = acquire_resource(*args, **kwds) |
| 63 |
try: |
| 64 |
yield resource |
| 65 |
finally: |
| 66 |
# Code to release resource, e.g.: |
| 67 |
release_resource(resource) |
| 68 |
|
| 69 |
The function can then be used like this:: |
| 70 |
|
| 71 |
>>> with managed_resource(timeout=3600) as resource: |
| 72 |
... # Resource is released at the end of this block, |
| 73 |
... # even if code in the block raises an exception |
| 74 |
|
| 75 |
The function being decorated must return a :term:`generator`-iterator when |
| 76 |
called. This iterator must yield exactly one value, which will be bound to |
| 77 |
the targets in the :keyword:`with` statement's :keyword:`!as` clause, if any. |
| 78 |
|
| 79 |
At the point where the generator yields, the block nested in the :keyword:`with` |
| 80 |
statement is executed. The generator is then resumed after the block is exited. |
| 81 |
If an unhandled exception occurs in the block, it is reraised inside the |
| 82 |
generator at the point where the yield occurred. Thus, you can use a |
| 83 |
:keyword:`try`...\ :keyword:`except`...\ :keyword:`finally` statement to trap |
| 84 |
the error (if any), or ensure that some cleanup takes place. If an exception is |
| 85 |
trapped merely in order to log it or to perform some action (rather than to |
| 86 |
suppress it entirely), the generator must reraise that exception. Otherwise the |
| 87 |
generator context manager will indicate to the :keyword:`!with` statement that |
| 88 |
the exception has been handled, and execution will resume with the statement |
| 89 |
immediately following the :keyword:`!with` statement. |
| 90 |
|
| 91 |
:func:`contextmanager` uses :class:`ContextDecorator` so the context managers |
| 92 |
it creates can be used as decorators as well as in :keyword:`with` statements. |
| 93 |
When used as a decorator, a new generator instance is implicitly created on |
| 94 |
each function call (this allows the otherwise "one-shot" context managers |
| 95 |
created by :func:`contextmanager` to meet the requirement that context |
| 96 |
managers support multiple invocations in order to be used as decorators). |
| 97 |
|
| 98 |
.. versionchanged:: 3.2 |
| 99 |
Use of :class:`ContextDecorator`. |
| 100 |
|
| 101 |
|
| 102 |
.. decorator:: asynccontextmanager |
| 103 |
|
| 104 |
Similar to :func:`~contextlib.contextmanager`, but creates an |
| 105 |
:ref:`asynchronous context manager <async-context-managers>`. |
| 106 |
|
| 107 |
This function is a :term:`decorator` that can be used to define a factory |
| 108 |
function for :keyword:`async with` statement asynchronous context managers, |
| 109 |
without needing to create a class or separate :meth:`~object.__aenter__` and |
| 110 |
:meth:`~object.__aexit__` methods. It must be applied to an :term:`asynchronous |
| 111 |
generator` function. |
| 112 |
|
| 113 |
A simple example:: |
| 114 |
|
| 115 |
from contextlib import asynccontextmanager |
| 116 |
|
| 117 |
@asynccontextmanager |
| 118 |
async def get_connection(): |
| 119 |
conn = await acquire_db_connection() |
| 120 |
try: |
| 121 |
yield conn |
| 122 |
finally: |
| 123 |
await release_db_connection(conn) |
| 124 |
|
| 125 |
async def get_all_users(): |
| 126 |
async with get_connection() as conn: |
| 127 |
return conn.query('SELECT ...') |
| 128 |
|
| 129 |
.. versionadded:: 3.7 |
| 130 |
|
| 131 |
Context managers defined with :func:`asynccontextmanager` can be used |
| 132 |
either as decorators or with :keyword:`async with` statements:: |
| 133 |
|
| 134 |
import time |
| 135 |
from contextlib import asynccontextmanager |
| 136 |
|
| 137 |
@asynccontextmanager |
| 138 |
async def timeit(): |
| 139 |
now = time.monotonic() |
| 140 |
try: |
| 141 |
yield |
| 142 |
finally: |
| 143 |
print(f'it took {time.monotonic() - now}s to run') |
| 144 |
|
| 145 |
@timeit() |
| 146 |
async def main(): |
| 147 |
# ... async code ... |
| 148 |
|
| 149 |
When used as a decorator, a new generator instance is implicitly created on |
| 150 |
each function call. This allows the otherwise "one-shot" context managers |
| 151 |
created by :func:`asynccontextmanager` to meet the requirement that context |
| 152 |
managers support multiple invocations in order to be used as decorators. |
| 153 |
|
| 154 |
.. versionchanged:: 3.10 |
| 155 |
Async context managers created with :func:`asynccontextmanager` can |
| 156 |
be used as decorators. |
| 157 |
|
| 158 |
|
| 159 |
.. function:: closing(thing) |
| 160 |
|
| 161 |
Return a context manager that closes *thing* upon completion of the block. This |
| 162 |
is basically equivalent to:: |
| 163 |
|
| 164 |
from contextlib import contextmanager |
| 165 |
|
| 166 |
@contextmanager |
| 167 |
def closing(thing): |
| 168 |
try: |
| 169 |
yield thing |
| 170 |
finally: |
| 171 |
thing.close() |
| 172 |
|
| 173 |
And lets you write code like this:: |
| 174 |
|
| 175 |
from contextlib import closing |
| 176 |
from urllib.request import urlopen |
| 177 |
|
| 178 |
with closing(urlopen('https://www.python.org')) as page: |
| 179 |
for line in page: |
| 180 |
print(line) |
| 181 |
|
| 182 |
without needing to explicitly close ``page``. Even if an error occurs, |
| 183 |
``page.close()`` will be called when the :keyword:`with` block is exited. |
| 184 |
|
| 185 |
.. note:: |
| 186 |
|
| 187 |
Most types managing resources support the :term:`context manager` protocol, |
| 188 |
which closes *thing* on leaving the :keyword:`with` statement. |
| 189 |
As such, :func:`!closing` is most useful for third party types that don't |
| 190 |
support context managers. |
| 191 |
This example is purely for illustration purposes, |
| 192 |
as :func:`~urllib.request.urlopen` would normally be used in a context manager. |
| 193 |
|
| 194 |
.. function:: aclosing(thing) |
| 195 |
|
| 196 |
Return an async context manager that calls the ``aclose()`` method of *thing* |
| 197 |
upon completion of the block. This is basically equivalent to:: |
| 198 |
|
| 199 |
from contextlib import asynccontextmanager |
| 200 |
|
| 201 |
@asynccontextmanager |
| 202 |
async def aclosing(thing): |
| 203 |
try: |
| 204 |
yield thing |
| 205 |
finally: |
| 206 |
await thing.aclose() |
| 207 |
|
| 208 |
Significantly, ``aclosing()`` supports deterministic cleanup of async |
| 209 |
generators when they happen to exit early by :keyword:`break` or an |
| 210 |
exception. For example:: |
| 211 |
|
| 212 |
from contextlib import aclosing |
| 213 |
|
| 214 |
async with aclosing(my_generator()) as values: |
| 215 |
async for value in values: |
| 216 |
if value == 42: |
| 217 |
break |
| 218 |
|
| 219 |
This pattern ensures that the generator's async exit code is executed in |
| 220 |
the same context as its iterations (so that exceptions and context |
| 221 |
variables work as expected, and the exit code isn't run after the |
| 222 |
lifetime of some task it depends on). |
| 223 |
|
| 224 |
.. versionadded:: 3.10 |
| 225 |
|
| 226 |
|
| 227 |
.. _simplifying-support-for-single-optional-context-managers: |
| 228 |
|
| 229 |
.. function:: nullcontext(enter_result=None) |
| 230 |
|
| 231 |
Return a context manager that returns *enter_result* from :meth:`~object.__enter__`, but |
| 232 |
otherwise does nothing. It is intended to be used as a stand-in for an |
| 233 |
optional context manager, for example:: |
| 234 |
|
| 235 |
def myfunction(arg, ignore_exceptions=False): |
| 236 |
if ignore_exceptions: |
| 237 |
# Use suppress to ignore all exceptions. |
| 238 |
cm = contextlib.suppress(Exception) |
| 239 |
else: |
| 240 |
# Do not ignore any exceptions, cm has no effect. |
| 241 |
cm = contextlib.nullcontext() |
| 242 |
with cm: |
| 243 |
# Do something |
| 244 |
|
| 245 |
An example using *enter_result*:: |
| 246 |
|
| 247 |
def process_file(file_or_path): |
| 248 |
if isinstance(file_or_path, str): |
| 249 |
# If string, open file |
| 250 |
cm = open(file_or_path) |
| 251 |
else: |
| 252 |
# Caller is responsible for closing file |
| 253 |
cm = nullcontext(file_or_path) |
| 254 |
|
| 255 |
with cm as file: |
| 256 |
# Perform processing on the file |
| 257 |
|
| 258 |
It can also be used as a stand-in for |
| 259 |
:ref:`asynchronous context managers <async-context-managers>`:: |
| 260 |
|
| 261 |
async def send_http(session=None): |
| 262 |
if not session: |
| 263 |
# If no http session, create it with aiohttp |
| 264 |
cm = aiohttp.ClientSession() |
| 265 |
else: |
| 266 |
# Caller is responsible for closing the session |
| 267 |
cm = nullcontext(session) |
| 268 |
|
| 269 |
async with cm as session: |
| 270 |
# Send http requests with session |
| 271 |
|
| 272 |
.. versionadded:: 3.7 |
| 273 |
|
| 274 |
.. versionchanged:: 3.10 |
| 275 |
:term:`asynchronous context manager` support was added. |
| 276 |
|
| 277 |
|
| 278 |
|
| 279 |
.. function:: suppress(*exceptions) |
| 280 |
|
| 281 |
Return a context manager that suppresses any of the specified exceptions |
| 282 |
if they occur in the body of a :keyword:`!with` statement and then |
| 283 |
resumes execution with the first statement following the end of the |
| 284 |
:keyword:`!with` statement. |
| 285 |
|
| 286 |
As with any other mechanism that completely suppresses exceptions, this |
| 287 |
context manager should be used only to cover very specific errors where |
| 288 |
silently continuing with program execution is known to be the right |
| 289 |
thing to do. |
| 290 |
|
| 291 |
For example:: |
| 292 |
|
| 293 |
from contextlib import suppress |
| 294 |
|
| 295 |
with suppress(FileNotFoundError): |
| 296 |
os.remove('somefile.tmp') |
| 297 |
|
| 298 |
with suppress(FileNotFoundError): |
| 299 |
os.remove('someotherfile.tmp') |
| 300 |
|
| 301 |
This code is equivalent to:: |
| 302 |
|
| 303 |
try: |
| 304 |
os.remove('somefile.tmp') |
| 305 |
except FileNotFoundError: |
| 306 |
pass |
| 307 |
|
| 308 |
try: |
| 309 |
os.remove('someotherfile.tmp') |
| 310 |
except FileNotFoundError: |
| 311 |
pass |
| 312 |
|
| 313 |
This context manager is :ref:`reentrant <reentrant-cms>`. |
| 314 |
|
| 315 |
If the code within the :keyword:`!with` block raises a |
| 316 |
:exc:`BaseExceptionGroup`, suppressed exceptions are removed from the |
| 317 |
group. Any exceptions of the group which are not suppressed are re-raised in |
| 318 |
a new group which is created using the original group's :meth:`~BaseExceptionGroup.derive` |
| 319 |
method. |
| 320 |
|
| 321 |
.. versionadded:: 3.4 |
| 322 |
|
| 323 |
.. versionchanged:: 3.12 |
| 324 |
``suppress`` now supports suppressing exceptions raised as |
| 325 |
part of a :exc:`BaseExceptionGroup`. |
| 326 |
|
| 327 |
.. function:: redirect_stdout(new_target) |
| 328 |
|
| 329 |
Context manager for temporarily redirecting :data:`sys.stdout` to |
| 330 |
another file or file-like object. |
| 331 |
|
| 332 |
This tool adds flexibility to existing functions or classes whose output |
| 333 |
is hardwired to stdout. |
| 334 |
|
| 335 |
For example, the output of :func:`help` normally is sent to *sys.stdout*. |
| 336 |
You can capture that output in a string by redirecting the output to an |
| 337 |
:class:`io.StringIO` object. The replacement stream is returned from the |
| 338 |
:meth:`~object.__enter__` method and so is available as the target of the |
| 339 |
:keyword:`with` statement:: |
| 340 |
|
| 341 |
with redirect_stdout(io.StringIO()) as f: |
| 342 |
help(pow) |
| 343 |
s = f.getvalue() |
| 344 |
|
| 345 |
To send the output of :func:`help` to a file on disk, redirect the output |
| 346 |
to a regular file:: |
| 347 |
|
| 348 |
with open('help.txt', 'w') as f: |
| 349 |
with redirect_stdout(f): |
| 350 |
help(pow) |
| 351 |
|
| 352 |
To send the output of :func:`help` to *sys.stderr*:: |
| 353 |
|
| 354 |
with redirect_stdout(sys.stderr): |
| 355 |
help(pow) |
| 356 |
|
| 357 |
Note that the global side effect on :data:`sys.stdout` means that this |
| 358 |
context manager is not suitable for use in library code and most threaded |
| 359 |
applications. It also has no effect on the output of subprocesses. |
| 360 |
However, it is still a useful approach for many utility scripts. |
| 361 |
|
| 362 |
This context manager is :ref:`reentrant <reentrant-cms>`. |
| 363 |
|
| 364 |
.. versionadded:: 3.4 |
| 365 |
|
| 366 |
|
| 367 |
.. function:: redirect_stderr(new_target) |
| 368 |
|
| 369 |
Similar to :func:`~contextlib.redirect_stdout` but redirecting |
| 370 |
:data:`sys.stderr` to another file or file-like object. |
| 371 |
|
| 372 |
This context manager is :ref:`reentrant <reentrant-cms>`. |
| 373 |
|
| 374 |
.. versionadded:: 3.5 |
| 375 |
|
| 376 |
|
| 377 |
.. function:: chdir(path) |
| 378 |
|
| 379 |
Non parallel-safe context manager to change the current working directory. |
| 380 |
As this changes a global state, the working directory, it is not suitable |
| 381 |
for use in most threaded or async contexts. It is also not suitable for most |
| 382 |
non-linear code execution, like generators, where the program execution is |
| 383 |
temporarily relinquished -- unless explicitly desired, you should not yield |
| 384 |
when this context manager is active. |
| 385 |
|
| 386 |
This is a simple wrapper around :func:`~os.chdir`, it changes the current |
| 387 |
working directory upon entering and restores the old one on exit. |
| 388 |
|
| 389 |
This context manager is :ref:`reentrant <reentrant-cms>`. |
| 390 |
|
| 391 |
.. versionadded:: 3.11 |
| 392 |
|
| 393 |
|
| 394 |
.. class:: ContextDecorator() |
| 395 |
|
| 396 |
A base class that enables a context manager to also be used as a decorator. |
| 397 |
|
| 398 |
Context managers inheriting from ``ContextDecorator`` have to implement |
| 399 |
:meth:`~object.__enter__` and :meth:`~object.__exit__` as normal. |
| 400 |
``__exit__`` retains its optional |
| 401 |
exception handling even when used as a decorator. |
| 402 |
|
| 403 |
``ContextDecorator`` is used by :func:`contextmanager`, so you get this |
| 404 |
functionality automatically. |
| 405 |
|
| 406 |
Example of ``ContextDecorator``:: |
| 407 |
|
| 408 |
from contextlib import ContextDecorator |
| 409 |
|
| 410 |
class mycontext(ContextDecorator): |
| 411 |
def __enter__(self): |
| 412 |
print('Starting') |
| 413 |
return self |
| 414 |
|
| 415 |
def __exit__(self, *exc): |
| 416 |
print('Finishing') |
| 417 |
return False |
| 418 |
|
| 419 |
The class can then be used like this:: |
| 420 |
|
| 421 |
>>> @mycontext() |
| 422 |
... def function(): |
| 423 |
... print('The bit in the middle') |
| 424 |
... |
| 425 |
>>> function() |
| 426 |
Starting |
| 427 |
The bit in the middle |
| 428 |
Finishing |
| 429 |
|
| 430 |
>>> with mycontext(): |
| 431 |
... print('The bit in the middle') |
| 432 |
... |
| 433 |
Starting |
| 434 |
The bit in the middle |
| 435 |
Finishing |
| 436 |
|
| 437 |
This change is just syntactic sugar for any construct of the following form:: |
| 438 |
|
| 439 |
def f(): |
| 440 |
with cm(): |
| 441 |
# Do stuff |
| 442 |
|
| 443 |
``ContextDecorator`` lets you instead write:: |
| 444 |
|
| 445 |
@cm() |
| 446 |
def f(): |
| 447 |
# Do stuff |
| 448 |
|
| 449 |
It makes it clear that the ``cm`` applies to the whole function, rather than |
| 450 |
just a piece of it (and saving an indentation level is nice, too). |
| 451 |
|
| 452 |
Existing context managers that already have a base class can be extended by |
| 453 |
using ``ContextDecorator`` as a mixin class:: |
| 454 |
|
| 455 |
from contextlib import ContextDecorator |
| 456 |
|
| 457 |
class mycontext(ContextBaseClass, ContextDecorator): |
| 458 |
def __enter__(self): |
| 459 |
return self |
| 460 |
|
| 461 |
def __exit__(self, *exc): |
| 462 |
return False |
| 463 |
|
| 464 |
.. note:: |
| 465 |
As the decorated function must be able to be called multiple times, the |
| 466 |
underlying context manager must support use in multiple :keyword:`with` |
| 467 |
statements. If this is not the case, then the original construct with the |
| 468 |
explicit :keyword:`!with` statement inside the function should be used. |
| 469 |
|
| 470 |
.. versionadded:: 3.2 |
| 471 |
|
| 472 |
|
| 473 |
.. class:: AsyncContextDecorator |
| 474 |
|
| 475 |
Similar to :class:`ContextDecorator` but only for asynchronous functions. |
| 476 |
|
| 477 |
Example of ``AsyncContextDecorator``:: |
| 478 |
|
| 479 |
from asyncio import run |
| 480 |
from contextlib import AsyncContextDecorator |
| 481 |
|
| 482 |
class mycontext(AsyncContextDecorator): |
| 483 |
async def __aenter__(self): |
| 484 |
print('Starting') |
| 485 |
return self |
| 486 |
|
| 487 |
async def __aexit__(self, *exc): |
| 488 |
print('Finishing') |
| 489 |
return False |
| 490 |
|
| 491 |
The class can then be used like this:: |
| 492 |
|
| 493 |
>>> @mycontext() |
| 494 |
... async def function(): |
| 495 |
... print('The bit in the middle') |
| 496 |
... |
| 497 |
>>> run(function()) |
| 498 |
Starting |
| 499 |
The bit in the middle |
| 500 |
Finishing |
| 501 |
|
| 502 |
>>> async def function(): |
| 503 |
... async with mycontext(): |
| 504 |
... print('The bit in the middle') |
| 505 |
... |
| 506 |
>>> run(function()) |
| 507 |
Starting |
| 508 |
The bit in the middle |
| 509 |
Finishing |
| 510 |
|
| 511 |
.. versionadded:: 3.10 |
| 512 |
|
| 513 |
|
| 514 |
.. class:: ExitStack() |
| 515 |
|
| 516 |
A context manager that is designed to make it easy to programmatically |
| 517 |
combine other context managers and cleanup functions, especially those |
| 518 |
that are optional or otherwise driven by input data. |
| 519 |
|
| 520 |
For example, a set of files may easily be handled in a single with |
| 521 |
statement as follows:: |
| 522 |
|
| 523 |
with ExitStack() as stack: |
| 524 |
files = [stack.enter_context(open(fname)) for fname in filenames] |
| 525 |
# All opened files will automatically be closed at the end of |
| 526 |
# the with statement, even if attempts to open files later |
| 527 |
# in the list raise an exception |
| 528 |
|
| 529 |
The :meth:`~object.__enter__` method returns the :class:`ExitStack` instance, and |
| 530 |
performs no additional operations. |
| 531 |
|
| 532 |
Each instance maintains a stack of registered callbacks that are called in |
| 533 |
reverse order when the instance is closed (either explicitly or implicitly |
| 534 |
at the end of a :keyword:`with` statement). Note that callbacks are *not* |
| 535 |
invoked implicitly when the context stack instance is garbage collected. |
| 536 |
|
| 537 |
This stack model is used so that context managers that acquire their |
| 538 |
resources in their ``__init__`` method (such as file objects) can be |
| 539 |
handled correctly. |
| 540 |
|
| 541 |
Since registered callbacks are invoked in the reverse order of |
| 542 |
registration, this ends up behaving as if multiple nested :keyword:`with` |
| 543 |
statements had been used with the registered set of callbacks. This even |
| 544 |
extends to exception handling - if an inner callback suppresses or replaces |
| 545 |
an exception, then outer callbacks will be passed arguments based on that |
| 546 |
updated state. |
| 547 |
|
| 548 |
This is a relatively low level API that takes care of the details of |
| 549 |
correctly unwinding the stack of exit callbacks. It provides a suitable |
| 550 |
foundation for higher level context managers that manipulate the exit |
| 551 |
stack in application specific ways. |
| 552 |
|
| 553 |
.. versionadded:: 3.3 |
| 554 |
|
| 555 |
.. method:: enter_context(cm) |
| 556 |
|
| 557 |
Enters a new context manager and adds its :meth:`~object.__exit__` method to |
| 558 |
the callback stack. The return value is the result of the context |
| 559 |
manager's own :meth:`~object.__enter__` method. |
| 560 |
|
| 561 |
These context managers may suppress exceptions just as they normally |
| 562 |
would if used directly as part of a :keyword:`with` statement. |
| 563 |
|
| 564 |
.. versionchanged:: 3.11 |
| 565 |
Raises :exc:`TypeError` instead of :exc:`AttributeError` if *cm* |
| 566 |
is not a context manager. |
| 567 |
|
| 568 |
.. method:: push(exit) |
| 569 |
|
| 570 |
Adds a context manager's :meth:`~object.__exit__` method to the callback stack. |
| 571 |
|
| 572 |
As ``__enter__`` is *not* invoked, this method can be used to cover |
| 573 |
part of an :meth:`~object.__enter__` implementation with a context manager's own |
| 574 |
:meth:`~object.__exit__` method. |
| 575 |
|
| 576 |
If passed an object that is not a context manager, this method assumes |
| 577 |
it is a callback with the same signature as a context manager's |
| 578 |
:meth:`~object.__exit__` method and adds it directly to the callback stack. |
| 579 |
|
| 580 |
By returning true values, these callbacks can suppress exceptions the |
| 581 |
same way context manager :meth:`~object.__exit__` methods can. |
| 582 |
|
| 583 |
The passed in object is returned from the function, allowing this |
| 584 |
method to be used as a function decorator. |
| 585 |
|
| 586 |
.. method:: callback(callback, /, *args, **kwds) |
| 587 |
|
| 588 |
Accepts an arbitrary callback function and arguments and adds it to |
| 589 |
the callback stack. |
| 590 |
|
| 591 |
Unlike the other methods, callbacks added this way cannot suppress |
| 592 |
exceptions (as they are never passed the exception details). |
| 593 |
|
| 594 |
The passed in callback is returned from the function, allowing this |
| 595 |
method to be used as a function decorator. |
| 596 |
|
| 597 |
.. method:: pop_all() |
| 598 |
|
| 599 |
Transfers the callback stack to a fresh :class:`ExitStack` instance |
| 600 |
and returns it. No callbacks are invoked by this operation - instead, |
| 601 |
they will now be invoked when the new stack is closed (either |
| 602 |
explicitly or implicitly at the end of a :keyword:`with` statement). |
| 603 |
|
| 604 |
For example, a group of files can be opened as an "all or nothing" |
| 605 |
operation as follows:: |
| 606 |
|
| 607 |
with ExitStack() as stack: |
| 608 |
files = [stack.enter_context(open(fname)) for fname in filenames] |
| 609 |
# Hold onto the close method, but don't call it yet. |
| 610 |
close_files = stack.pop_all().close |
| 611 |
# If opening any file fails, all previously opened files will be |
| 612 |
# closed automatically. If all files are opened successfully, |
| 613 |
# they will remain open even after the with statement ends. |
| 614 |
# close_files() can then be invoked explicitly to close them all. |
| 615 |
|
| 616 |
.. method:: close() |
| 617 |
|
| 618 |
Immediately unwinds the callback stack, invoking callbacks in the |
| 619 |
reverse order of registration. For any context managers and exit |
| 620 |
callbacks registered, the arguments passed in will indicate that no |
| 621 |
exception occurred. |
| 622 |
|
| 623 |
.. class:: AsyncExitStack() |
| 624 |
|
| 625 |
An :ref:`asynchronous context manager <async-context-managers>`, similar |
| 626 |
to :class:`ExitStack`, that supports combining both synchronous and |
| 627 |
asynchronous context managers, as well as having coroutines for |
| 628 |
cleanup logic. |
| 629 |
|
| 630 |
The :meth:`~ExitStack.close` method is not implemented; :meth:`aclose` must be used |
| 631 |
instead. |
| 632 |
|
| 633 |
.. method:: enter_async_context(cm) |
| 634 |
:async: |
| 635 |
|
| 636 |
Similar to :meth:`ExitStack.enter_context` but expects an asynchronous context |
| 637 |
manager. |
| 638 |
|
| 639 |
.. versionchanged:: 3.11 |
| 640 |
Raises :exc:`TypeError` instead of :exc:`AttributeError` if *cm* |
| 641 |
is not an asynchronous context manager. |
| 642 |
|
| 643 |
.. method:: push_async_exit(exit) |
| 644 |
|
| 645 |
Similar to :meth:`ExitStack.push` but expects either an asynchronous context manager |
| 646 |
or a coroutine function. |
| 647 |
|
| 648 |
.. method:: push_async_callback(callback, /, *args, **kwds) |
| 649 |
|
| 650 |
Similar to :meth:`ExitStack.callback` but expects a coroutine function. |
| 651 |
|
| 652 |
.. method:: aclose() |
| 653 |
:async: |
| 654 |
|
| 655 |
Similar to :meth:`ExitStack.close` but properly handles awaitables. |
| 656 |
|
| 657 |
Continuing the example for :func:`asynccontextmanager`:: |
| 658 |
|
| 659 |
async with AsyncExitStack() as stack: |
| 660 |
connections = [await stack.enter_async_context(get_connection()) |
| 661 |
for i in range(5)] |
| 662 |
# All opened connections will automatically be released at the end of |
| 663 |
# the async with statement, even if attempts to open a connection |
| 664 |
# later in the list raise an exception. |
| 665 |
|
| 666 |
.. versionadded:: 3.7 |
| 667 |
|
| 668 |
Examples and Recipes |
| 669 |
-------------------- |
| 670 |
|
| 671 |
This section describes some examples and recipes for making effective use of |
| 672 |
the tools provided by :mod:`!contextlib`. |
| 673 |
|
| 674 |
|
| 675 |
Supporting a variable number of context managers |
| 676 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 677 |
|
| 678 |
The primary use case for :class:`ExitStack` is the one given in the class |
| 679 |
documentation: supporting a variable number of context managers and other |
| 680 |
cleanup operations in a single :keyword:`with` statement. The variability |
| 681 |
may come from the number of context managers needed being driven by user |
| 682 |
input (such as opening a user specified collection of files), or from |
| 683 |
some of the context managers being optional:: |
| 684 |
|
| 685 |
with ExitStack() as stack: |
| 686 |
for resource in resources: |
| 687 |
stack.enter_context(resource) |
| 688 |
if need_special_resource(): |
| 689 |
special = acquire_special_resource() |
| 690 |
stack.callback(release_special_resource, special) |
| 691 |
# Perform operations that use the acquired resources |
| 692 |
|
| 693 |
As shown, :class:`ExitStack` also makes it quite easy to use :keyword:`with` |
| 694 |
statements to manage arbitrary resources that don't natively support the |
| 695 |
context management protocol. |
| 696 |
|
| 697 |
|
| 698 |
Catching exceptions from ``__enter__`` methods |
| 699 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 700 |
|
| 701 |
It is occasionally desirable to catch exceptions from an :meth:`~object.__enter__` |
| 702 |
method implementation, *without* inadvertently catching exceptions from |
| 703 |
the :keyword:`with` statement body or the context manager's :meth:`~object.__exit__` |
| 704 |
method. By using :class:`ExitStack` the steps in the context management |
| 705 |
protocol can be separated slightly in order to allow this:: |
| 706 |
|
| 707 |
stack = ExitStack() |
| 708 |
try: |
| 709 |
x = stack.enter_context(cm) |
| 710 |
except Exception: |
| 711 |
# handle __enter__ exception |
| 712 |
else: |
| 713 |
with stack: |
| 714 |
# Handle normal case |
| 715 |
|
| 716 |
Actually needing to do this is likely to indicate that the underlying API |
| 717 |
should be providing a direct resource management interface for use with |
| 718 |
:keyword:`try`/:keyword:`except`/:keyword:`finally` statements, but not |
| 719 |
all APIs are well designed in that regard. When a context manager is the |
| 720 |
only resource management API provided, then :class:`ExitStack` can make it |
| 721 |
easier to handle various situations that can't be handled directly in a |
| 722 |
:keyword:`with` statement. |
| 723 |
|
| 724 |
|
| 725 |
Cleaning up in an ``__enter__`` implementation |
| 726 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 727 |
|
| 728 |
As noted in the documentation of :meth:`ExitStack.push`, this |
| 729 |
method can be useful in cleaning up an already allocated resource if later |
| 730 |
steps in the :meth:`~object.__enter__` implementation fail. |
| 731 |
|
| 732 |
Here's an example of doing this for a context manager that accepts resource |
| 733 |
acquisition and release functions, along with an optional validation function, |
| 734 |
and maps them to the context management protocol:: |
| 735 |
|
| 736 |
from contextlib import contextmanager, AbstractContextManager, ExitStack |
| 737 |
|
| 738 |
class ResourceManager(AbstractContextManager): |
| 739 |
|
| 740 |
def __init__(self, acquire_resource, release_resource, check_resource_ok=None): |
| 741 |
self.acquire_resource = acquire_resource |
| 742 |
self.release_resource = release_resource |
| 743 |
if check_resource_ok is None: |
| 744 |
def check_resource_ok(resource): |
| 745 |
return True |
| 746 |
self.check_resource_ok = check_resource_ok |
| 747 |
|
| 748 |
@contextmanager |
| 749 |
def _cleanup_on_error(self): |
| 750 |
with ExitStack() as stack: |
| 751 |
stack.push(self) |
| 752 |
yield |
| 753 |
# The validation check passed and didn't raise an exception |
| 754 |
# Accordingly, we want to keep the resource, and pass it |
| 755 |
# back to our caller |
| 756 |
stack.pop_all() |
| 757 |
|
| 758 |
def __enter__(self): |
| 759 |
resource = self.acquire_resource() |
| 760 |
with self._cleanup_on_error(): |
| 761 |
if not self.check_resource_ok(resource): |
| 762 |
msg = "Failed validation for {!r}" |
| 763 |
raise RuntimeError(msg.format(resource)) |
| 764 |
return resource |
| 765 |
|
| 766 |
def __exit__(self, *exc_details): |
| 767 |
# We don't need to duplicate any of our resource release logic |
| 768 |
self.release_resource() |
| 769 |
|
| 770 |
|
| 771 |
Replacing any use of ``try-finally`` and flag variables |
| 772 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 773 |
|
| 774 |
A pattern you will sometimes see is a ``try-finally`` statement with a flag |
| 775 |
variable to indicate whether or not the body of the ``finally`` clause should |
| 776 |
be executed. In its simplest form (that can't already be handled just by |
| 777 |
using an ``except`` clause instead), it looks something like this:: |
| 778 |
|
| 779 |
cleanup_needed = True |
| 780 |
try: |
| 781 |
result = perform_operation() |
| 782 |
if result: |
| 783 |
cleanup_needed = False |
| 784 |
finally: |
| 785 |
if cleanup_needed: |
| 786 |
cleanup_resources() |
| 787 |
|
| 788 |
As with any ``try`` statement based code, this can cause problems for |
| 789 |
development and review, because the setup code and the cleanup code can end |
| 790 |
up being separated by arbitrarily long sections of code. |
| 791 |
|
| 792 |
:class:`ExitStack` makes it possible to instead register a callback for |
| 793 |
execution at the end of a ``with`` statement, and then later decide to skip |
| 794 |
executing that callback:: |
| 795 |
|
| 796 |
from contextlib import ExitStack |
| 797 |
|
| 798 |
with ExitStack() as stack: |
| 799 |
stack.callback(cleanup_resources) |
| 800 |
result = perform_operation() |
| 801 |
if result: |
| 802 |
stack.pop_all() |
| 803 |
|
| 804 |
This allows the intended cleanup behaviour to be made explicit up front, |
| 805 |
rather than requiring a separate flag variable. |
| 806 |
|
| 807 |
If a particular application uses this pattern a lot, it can be simplified |
| 808 |
even further by means of a small helper class:: |
| 809 |
|
| 810 |
from contextlib import ExitStack |
| 811 |
|
| 812 |
class Callback(ExitStack): |
| 813 |
def __init__(self, callback, /, *args, **kwds): |
| 814 |
super().__init__() |
| 815 |
self.callback(callback, *args, **kwds) |
| 816 |
|
| 817 |
def cancel(self): |
| 818 |
self.pop_all() |
| 819 |
|
| 820 |
with Callback(cleanup_resources) as cb: |
| 821 |
result = perform_operation() |
| 822 |
if result: |
| 823 |
cb.cancel() |
| 824 |
|
| 825 |
If the resource cleanup isn't already neatly bundled into a standalone |
| 826 |
function, then it is still possible to use the decorator form of |
| 827 |
:meth:`ExitStack.callback` to declare the resource cleanup in |
| 828 |
advance:: |
| 829 |
|
| 830 |
from contextlib import ExitStack |
| 831 |
|
| 832 |
with ExitStack() as stack: |
| 833 |
@stack.callback |
| 834 |
def cleanup_resources(): |
| 835 |
... |
| 836 |
result = perform_operation() |
| 837 |
if result: |
| 838 |
stack.pop_all() |
| 839 |
|
| 840 |
Due to the way the decorator protocol works, a callback function |
| 841 |
declared this way cannot take any parameters. Instead, any resources to |
| 842 |
be released must be accessed as closure variables. |
| 843 |
|
| 844 |
|
| 845 |
Using a context manager as a function decorator |
| 846 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 847 |
|
| 848 |
:class:`ContextDecorator` makes it possible to use a context manager in |
| 849 |
both an ordinary ``with`` statement and also as a function decorator. |
| 850 |
|
| 851 |
For example, it is sometimes useful to wrap functions or groups of statements |
| 852 |
with a logger that can track the time of entry and time of exit. Rather than |
| 853 |
writing both a function decorator and a context manager for the task, |
| 854 |
inheriting from :class:`ContextDecorator` provides both capabilities in a |
| 855 |
single definition:: |
| 856 |
|
| 857 |
from contextlib import ContextDecorator |
| 858 |
import logging |
| 859 |
|
| 860 |
logging.basicConfig(level=logging.INFO) |
| 861 |
|
| 862 |
class track_entry_and_exit(ContextDecorator): |
| 863 |
def __init__(self, name): |
| 864 |
self.name = name |
| 865 |
|
| 866 |
def __enter__(self): |
| 867 |
logging.info('Entering: %s', self.name) |
| 868 |
|
| 869 |
def __exit__(self, exc_type, exc, exc_tb): |
| 870 |
logging.info('Exiting: %s', self.name) |
| 871 |
|
| 872 |
Instances of this class can be used as both a context manager:: |
| 873 |
|
| 874 |
with track_entry_and_exit('widget loader'): |
| 875 |
print('Some time consuming activity goes here') |
| 876 |
load_widget() |
| 877 |
|
| 878 |
And also as a function decorator:: |
| 879 |
|
| 880 |
@track_entry_and_exit('widget loader') |
| 881 |
def activity(): |
| 882 |
print('Some time consuming activity goes here') |
| 883 |
load_widget() |
| 884 |
|
| 885 |
Note that there is one additional limitation when using context managers |
| 886 |
as function decorators: there's no way to access the return value of |
| 887 |
:meth:`~object.__enter__`. If that value is needed, then it is still necessary to use |
| 888 |
an explicit ``with`` statement. |
| 889 |
|
| 890 |
.. seealso:: |
| 891 |
|
| 892 |
:pep:`343` - The "with" statement |
| 893 |
The specification, background, and examples for the Python :keyword:`with` |
| 894 |
statement. |
| 895 |
|
| 896 |
.. _single-use-reusable-and-reentrant-cms: |
| 897 |
|
| 898 |
Single use, reusable and reentrant context managers |
| 899 |
--------------------------------------------------- |
| 900 |
|
| 901 |
Most context managers are written in a way that means they can only be |
| 902 |
used effectively in a :keyword:`with` statement once. These single use |
| 903 |
context managers must be created afresh each time they're used - |
| 904 |
attempting to use them a second time will trigger an exception or |
| 905 |
otherwise not work correctly. |
| 906 |
|
| 907 |
This common limitation means that it is generally advisable to create |
| 908 |
context managers directly in the header of the :keyword:`with` statement |
| 909 |
where they are used (as shown in all of the usage examples above). |
| 910 |
|
| 911 |
Files are an example of effectively single use context managers, since |
| 912 |
the first :keyword:`with` statement will close the file, preventing any |
| 913 |
further IO operations using that file object. |
| 914 |
|
| 915 |
Context managers created using :func:`contextmanager` are also single use |
| 916 |
context managers, and will complain about the underlying generator failing |
| 917 |
to yield if an attempt is made to use them a second time:: |
| 918 |
|
| 919 |
>>> from contextlib import contextmanager |
| 920 |
>>> @contextmanager |
| 921 |
... def singleuse(): |
| 922 |
... print("Before") |
| 923 |
... yield |
| 924 |
... print("After") |
| 925 |
... |
| 926 |
>>> cm = singleuse() |
| 927 |
>>> with cm: |
| 928 |
... pass |
| 929 |
... |
| 930 |
Before |
| 931 |
After |
| 932 |
>>> with cm: |
| 933 |
... pass |
| 934 |
... |
| 935 |
Traceback (most recent call last): |
| 936 |
... |
| 937 |
RuntimeError: generator didn't yield |
| 938 |
|
| 939 |
|
| 940 |
.. _reentrant-cms: |
| 941 |
|
| 942 |
Reentrant context managers |
| 943 |
^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 944 |
|
| 945 |
More sophisticated context managers may be "reentrant". These context |
| 946 |
managers can not only be used in multiple :keyword:`with` statements, |
| 947 |
but may also be used *inside* a :keyword:`!with` statement that is already |
| 948 |
using the same context manager. |
| 949 |
|
| 950 |
:class:`threading.RLock` is an example of a reentrant context manager, as are |
| 951 |
:func:`suppress`, :func:`redirect_stdout`, and :func:`chdir`. Here's a very |
| 952 |
simple example of reentrant use:: |
| 953 |
|
| 954 |
>>> from contextlib import redirect_stdout |
| 955 |
>>> from io import StringIO |
| 956 |
>>> stream = StringIO() |
| 957 |
>>> write_to_stream = redirect_stdout(stream) |
| 958 |
>>> with write_to_stream: |
| 959 |
... print("This is written to the stream rather than stdout") |
| 960 |
... with write_to_stream: |
| 961 |
... print("This is also written to the stream") |
| 962 |
... |
| 963 |
>>> print("This is written directly to stdout") |
| 964 |
This is written directly to stdout |
| 965 |
>>> print(stream.getvalue()) |
| 966 |
This is written to the stream rather than stdout |
| 967 |
This is also written to the stream |
| 968 |
|
| 969 |
Real world examples of reentrancy are more likely to involve multiple |
| 970 |
functions calling each other and hence be far more complicated than this |
| 971 |
example. |
| 972 |
|
| 973 |
Note also that being reentrant is *not* the same thing as being thread safe. |
| 974 |
:func:`redirect_stdout`, for example, is definitely not thread safe, as it |
| 975 |
makes a global modification to the system state by binding :data:`sys.stdout` |
| 976 |
to a different stream. |
| 977 |
|
| 978 |
|
| 979 |
.. _reusable-cms: |
| 980 |
|
| 981 |
Reusable context managers |
| 982 |
^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 983 |
|
| 984 |
Distinct from both single use and reentrant context managers are "reusable" |
| 985 |
context managers (or, to be completely explicit, "reusable, but not |
| 986 |
reentrant" context managers, since reentrant context managers are also |
| 987 |
reusable). These context managers support being used multiple times, but |
| 988 |
will fail (or otherwise not work correctly) if the specific context manager |
| 989 |
instance has already been used in a containing with statement. |
| 990 |
|
| 991 |
:class:`threading.Lock` is an example of a reusable, but not reentrant, |
| 992 |
context manager (for a reentrant lock, it is necessary to use |
| 993 |
:class:`threading.RLock` instead). |
| 994 |
|
| 995 |
Another example of a reusable, but not reentrant, context manager is |
| 996 |
:class:`ExitStack`, as it invokes *all* currently registered callbacks |
| 997 |
when leaving any with statement, regardless of where those callbacks |
| 998 |
were added:: |
| 999 |
|
| 1000 |
>>> from contextlib import ExitStack |
| 1001 |
>>> stack = ExitStack() |
| 1002 |
>>> with stack: |
| 1003 |
... stack.callback(print, "Callback: from first context") |
| 1004 |
... print("Leaving first context") |
| 1005 |
... |
| 1006 |
Leaving first context |
| 1007 |
Callback: from first context |
| 1008 |
>>> with stack: |
| 1009 |
... stack.callback(print, "Callback: from second context") |
| 1010 |
... print("Leaving second context") |
| 1011 |
... |
| 1012 |
Leaving second context |
| 1013 |
Callback: from second context |
| 1014 |
>>> with stack: |
| 1015 |
... stack.callback(print, "Callback: from outer context") |
| 1016 |
... with stack: |
| 1017 |
... stack.callback(print, "Callback: from inner context") |
| 1018 |
... print("Leaving inner context") |
| 1019 |
... print("Leaving outer context") |
| 1020 |
... |
| 1021 |
Leaving inner context |
| 1022 |
Callback: from inner context |
| 1023 |
Callback: from outer context |
| 1024 |
Leaving outer context |
| 1025 |
|
| 1026 |
As the output from the example shows, reusing a single stack object across |
| 1027 |
multiple with statements works correctly, but attempting to nest them |
| 1028 |
will cause the stack to be cleared at the end of the innermost with |
| 1029 |
statement, which is unlikely to be desirable behaviour. |
| 1030 |
|
| 1031 |
Using separate :class:`ExitStack` instances instead of reusing a single |
| 1032 |
instance avoids that problem:: |
| 1033 |
|
| 1034 |
>>> from contextlib import ExitStack |
| 1035 |
>>> with ExitStack() as outer_stack: |
| 1036 |
... outer_stack.callback(print, "Callback: from outer context") |
| 1037 |
... with ExitStack() as inner_stack: |
| 1038 |
... inner_stack.callback(print, "Callback: from inner context") |
| 1039 |
... print("Leaving inner context") |
| 1040 |
... print("Leaving outer context") |
| 1041 |
... |
| 1042 |
Leaving inner context |
| 1043 |
Callback: from inner context |
| 1044 |
Leaving outer context |
| 1045 |
Callback: from outer context |