| 1 |
.. currentmodule:: asyncio |
| 2 |
|
| 3 |
|
| 4 |
==================== |
| 5 |
Coroutines and tasks |
| 6 |
==================== |
| 7 |
|
| 8 |
This section outlines high-level asyncio APIs to work with coroutines |
| 9 |
and Tasks. |
| 10 |
|
| 11 |
.. contents:: |
| 12 |
:depth: 1 |
| 13 |
:local: |
| 14 |
|
| 15 |
|
| 16 |
.. _coroutine: |
| 17 |
|
| 18 |
Coroutines |
| 19 |
========== |
| 20 |
|
| 21 |
**Source code:** :source:`Lib/asyncio/coroutines.py` |
| 22 |
|
| 23 |
---------------------------------------------------- |
| 24 |
|
| 25 |
:term:`Coroutines <coroutine>` declared with the async/await syntax is the |
| 26 |
preferred way of writing asyncio applications. For example, the following |
| 27 |
snippet of code prints "hello", waits 1 second, |
| 28 |
and then prints "world":: |
| 29 |
|
| 30 |
>>> import asyncio |
| 31 |
|
| 32 |
>>> async def main(): |
| 33 |
... print('hello') |
| 34 |
... await asyncio.sleep(1) |
| 35 |
... print('world') |
| 36 |
|
| 37 |
>>> asyncio.run(main()) |
| 38 |
hello |
| 39 |
world |
| 40 |
|
| 41 |
Note that simply calling a coroutine will not schedule it to |
| 42 |
be executed:: |
| 43 |
|
| 44 |
>>> main() |
| 45 |
<coroutine object main at 0x1053bb7c8> |
| 46 |
|
| 47 |
To actually run a coroutine, asyncio provides the following mechanisms: |
| 48 |
|
| 49 |
* The :func:`asyncio.run` function to run the top-level |
| 50 |
entry point "main()" function (see the above example.) |
| 51 |
|
| 52 |
* Awaiting on a coroutine. The following snippet of code will |
| 53 |
print "hello" after waiting for 1 second, and then print "world" |
| 54 |
after waiting for *another* 2 seconds:: |
| 55 |
|
| 56 |
import asyncio |
| 57 |
import time |
| 58 |
|
| 59 |
async def say_after(delay, what): |
| 60 |
await asyncio.sleep(delay) |
| 61 |
print(what) |
| 62 |
|
| 63 |
async def main(): |
| 64 |
print(f"started at {time.strftime('%X')}") |
| 65 |
|
| 66 |
await say_after(1, 'hello') |
| 67 |
await say_after(2, 'world') |
| 68 |
|
| 69 |
print(f"finished at {time.strftime('%X')}") |
| 70 |
|
| 71 |
asyncio.run(main()) |
| 72 |
|
| 73 |
Expected output:: |
| 74 |
|
| 75 |
started at 17:13:52 |
| 76 |
hello |
| 77 |
world |
| 78 |
finished at 17:13:55 |
| 79 |
|
| 80 |
* The :func:`asyncio.create_task` function to run coroutines |
| 81 |
concurrently as asyncio :class:`Tasks <Task>`. |
| 82 |
|
| 83 |
Let's modify the above example and run two ``say_after`` coroutines |
| 84 |
*concurrently*:: |
| 85 |
|
| 86 |
async def main(): |
| 87 |
task1 = asyncio.create_task( |
| 88 |
say_after(1, 'hello')) |
| 89 |
|
| 90 |
task2 = asyncio.create_task( |
| 91 |
say_after(2, 'world')) |
| 92 |
|
| 93 |
print(f"started at {time.strftime('%X')}") |
| 94 |
|
| 95 |
# Wait until both tasks are completed (should take |
| 96 |
# around 2 seconds.) |
| 97 |
await task1 |
| 98 |
await task2 |
| 99 |
|
| 100 |
print(f"finished at {time.strftime('%X')}") |
| 101 |
|
| 102 |
Note that expected output now shows that the snippet runs |
| 103 |
1 second faster than before:: |
| 104 |
|
| 105 |
started at 17:14:32 |
| 106 |
hello |
| 107 |
world |
| 108 |
finished at 17:14:34 |
| 109 |
|
| 110 |
* The :class:`asyncio.TaskGroup` class provides a more modern |
| 111 |
alternative to :func:`create_task`. |
| 112 |
Using this API, the last example becomes:: |
| 113 |
|
| 114 |
async def main(): |
| 115 |
async with asyncio.TaskGroup() as tg: |
| 116 |
task1 = tg.create_task( |
| 117 |
say_after(1, 'hello')) |
| 118 |
|
| 119 |
task2 = tg.create_task( |
| 120 |
say_after(2, 'world')) |
| 121 |
|
| 122 |
print(f"started at {time.strftime('%X')}") |
| 123 |
|
| 124 |
# The await is implicit when the context manager exits. |
| 125 |
|
| 126 |
print(f"finished at {time.strftime('%X')}") |
| 127 |
|
| 128 |
The timing and output should be the same as for the previous version. |
| 129 |
|
| 130 |
.. versionadded:: 3.11 |
| 131 |
:class:`asyncio.TaskGroup`. |
| 132 |
|
| 133 |
|
| 134 |
.. _asyncio-awaitables: |
| 135 |
|
| 136 |
Awaitables |
| 137 |
========== |
| 138 |
|
| 139 |
We say that an object is an **awaitable** object if it can be used |
| 140 |
in an :keyword:`await` expression. Many asyncio APIs are designed to |
| 141 |
accept awaitables. |
| 142 |
|
| 143 |
There are three main types of *awaitable* objects: |
| 144 |
**coroutines**, **Tasks**, and **Futures**. |
| 145 |
|
| 146 |
|
| 147 |
.. rubric:: Coroutines |
| 148 |
|
| 149 |
Python coroutines are *awaitables* and therefore can be awaited from |
| 150 |
other coroutines:: |
| 151 |
|
| 152 |
import asyncio |
| 153 |
|
| 154 |
async def nested(): |
| 155 |
return 42 |
| 156 |
|
| 157 |
async def main(): |
| 158 |
# Nothing happens if we just call "nested()". |
| 159 |
# A coroutine object is created but not awaited, |
| 160 |
# so it *won't run at all*. |
| 161 |
nested() # will raise a "RuntimeWarning". |
| 162 |
|
| 163 |
# Let's do it differently now and await it: |
| 164 |
print(await nested()) # will print "42". |
| 165 |
|
| 166 |
asyncio.run(main()) |
| 167 |
|
| 168 |
.. important:: |
| 169 |
|
| 170 |
In this documentation the term "coroutine" can be used for |
| 171 |
two closely related concepts: |
| 172 |
|
| 173 |
* a *coroutine function*: an :keyword:`async def` function; |
| 174 |
|
| 175 |
* a *coroutine object*: an object returned by calling a |
| 176 |
*coroutine function*. |
| 177 |
|
| 178 |
|
| 179 |
.. rubric:: Tasks |
| 180 |
|
| 181 |
*Tasks* are used to schedule coroutines *concurrently*. |
| 182 |
|
| 183 |
When a coroutine is wrapped into a *Task* with functions like |
| 184 |
:func:`asyncio.create_task` the coroutine is automatically |
| 185 |
scheduled to run soon:: |
| 186 |
|
| 187 |
import asyncio |
| 188 |
|
| 189 |
async def nested(): |
| 190 |
return 42 |
| 191 |
|
| 192 |
async def main(): |
| 193 |
# Schedule nested() to run soon concurrently |
| 194 |
# with "main()". |
| 195 |
task = asyncio.create_task(nested()) |
| 196 |
|
| 197 |
# "task" can now be used to cancel "nested()", or |
| 198 |
# can simply be awaited to wait until it is complete: |
| 199 |
await task |
| 200 |
|
| 201 |
asyncio.run(main()) |
| 202 |
|
| 203 |
|
| 204 |
.. rubric:: Futures |
| 205 |
|
| 206 |
A :class:`Future` is a special **low-level** awaitable object that |
| 207 |
represents an **eventual result** of an asynchronous operation. |
| 208 |
|
| 209 |
When a Future object is *awaited* it means that the coroutine will |
| 210 |
wait until the Future is resolved in some other place. |
| 211 |
|
| 212 |
Future objects in asyncio are needed to allow callback-based code |
| 213 |
to be used with async/await. |
| 214 |
|
| 215 |
Normally **there is no need** to create Future objects at the |
| 216 |
application level code. |
| 217 |
|
| 218 |
Future objects, sometimes exposed by libraries and some asyncio |
| 219 |
APIs, can be awaited:: |
| 220 |
|
| 221 |
async def main(): |
| 222 |
await function_that_returns_a_future_object() |
| 223 |
|
| 224 |
# this is also valid: |
| 225 |
await asyncio.gather( |
| 226 |
function_that_returns_a_future_object(), |
| 227 |
some_python_coroutine() |
| 228 |
) |
| 229 |
|
| 230 |
A good example of a low-level function that returns a Future object |
| 231 |
is :meth:`loop.run_in_executor`. |
| 232 |
|
| 233 |
|
| 234 |
Creating tasks |
| 235 |
============== |
| 236 |
|
| 237 |
**Source code:** :source:`Lib/asyncio/tasks.py` |
| 238 |
|
| 239 |
----------------------------------------------- |
| 240 |
|
| 241 |
.. function:: create_task(coro, *, name=None, context=None, eager_start=None, **kwargs) |
| 242 |
|
| 243 |
Wrap the *coro* :ref:`coroutine <coroutine>` into a :class:`Task` |
| 244 |
and schedule its execution. Return the Task object. |
| 245 |
|
| 246 |
The full function signature is largely the same as that of the |
| 247 |
:class:`Task` constructor (or factory) - all of the keyword arguments to |
| 248 |
this function are passed through to that interface. |
| 249 |
|
| 250 |
An optional keyword-only *context* argument allows specifying a |
| 251 |
custom :class:`contextvars.Context` for the *coro* to run in. |
| 252 |
The current context copy is created when no *context* is provided. |
| 253 |
|
| 254 |
An optional keyword-only *eager_start* argument allows specifying |
| 255 |
if the task should execute eagerly during the call to create_task, |
| 256 |
or be scheduled later. If *eager_start* is not passed the mode set |
| 257 |
by :meth:`loop.set_task_factory` will be used. |
| 258 |
|
| 259 |
The task is executed in the loop returned by :func:`get_running_loop`, |
| 260 |
:exc:`RuntimeError` is raised if there is no running loop in |
| 261 |
current thread. |
| 262 |
|
| 263 |
.. note:: |
| 264 |
|
| 265 |
:meth:`asyncio.TaskGroup.create_task` is a new alternative |
| 266 |
leveraging structural concurrency; it allows for waiting |
| 267 |
for a group of related tasks with strong safety guarantees. |
| 268 |
|
| 269 |
.. important:: |
| 270 |
|
| 271 |
Save a reference to the result of this function, to avoid |
| 272 |
a task disappearing mid-execution. The event loop only keeps |
| 273 |
weak references to tasks. A task that isn't referenced elsewhere |
| 274 |
may get garbage collected at any time, even before it's done. |
| 275 |
For reliable "fire-and-forget" background tasks, gather them in |
| 276 |
a collection:: |
| 277 |
|
| 278 |
background_tasks = set() |
| 279 |
|
| 280 |
for i in range(10): |
| 281 |
task = asyncio.create_task(some_coro(param=i)) |
| 282 |
|
| 283 |
# Add task to the set. This creates a strong reference. |
| 284 |
background_tasks.add(task) |
| 285 |
|
| 286 |
# To prevent keeping references to finished tasks forever, |
| 287 |
# make each task remove its own reference from the set after |
| 288 |
# completion: |
| 289 |
task.add_done_callback(background_tasks.discard) |
| 290 |
|
| 291 |
.. versionadded:: 3.7 |
| 292 |
|
| 293 |
.. versionchanged:: 3.8 |
| 294 |
Added the *name* parameter. |
| 295 |
|
| 296 |
.. versionchanged:: 3.11 |
| 297 |
Added the *context* parameter. |
| 298 |
|
| 299 |
.. versionchanged:: 3.14 |
| 300 |
Added the *eager_start* parameter by passing on all *kwargs*. |
| 301 |
|
| 302 |
|
| 303 |
Task cancellation |
| 304 |
================= |
| 305 |
|
| 306 |
Tasks can easily and safely be cancelled. |
| 307 |
When a task is cancelled, :exc:`asyncio.CancelledError` will be raised |
| 308 |
in the task at the next opportunity. |
| 309 |
|
| 310 |
It is recommended that coroutines use ``try/finally`` blocks to robustly |
| 311 |
perform clean-up logic. In case :exc:`asyncio.CancelledError` |
| 312 |
is explicitly caught, it should generally be propagated when |
| 313 |
clean-up is complete. :exc:`asyncio.CancelledError` directly subclasses |
| 314 |
:exc:`BaseException` so most code will not need to be aware of it. |
| 315 |
|
| 316 |
The asyncio components that enable structured concurrency, like |
| 317 |
:class:`asyncio.TaskGroup` and :func:`asyncio.timeout`, |
| 318 |
are implemented using cancellation internally and might misbehave if |
| 319 |
a coroutine swallows :exc:`asyncio.CancelledError`. Similarly, user code |
| 320 |
should not generally call :meth:`uncancel <asyncio.Task.uncancel>`. |
| 321 |
However, in cases when suppressing :exc:`asyncio.CancelledError` is |
| 322 |
truly desired, it is necessary to also call ``uncancel()`` to completely |
| 323 |
remove the cancellation state. |
| 324 |
|
| 325 |
.. _taskgroups: |
| 326 |
|
| 327 |
Task groups |
| 328 |
=========== |
| 329 |
|
| 330 |
Task groups combine a task creation API with a convenient |
| 331 |
and reliable way to wait for all tasks in the group to finish. |
| 332 |
|
| 333 |
.. class:: TaskGroup() |
| 334 |
|
| 335 |
An :ref:`asynchronous context manager <async-context-managers>` |
| 336 |
holding a group of tasks. |
| 337 |
Tasks can be added to the group using :meth:`create_task`. |
| 338 |
All tasks are awaited when the context manager exits. |
| 339 |
|
| 340 |
.. versionadded:: 3.11 |
| 341 |
|
| 342 |
.. method:: create_task(coro, *, name=None, context=None, eager_start=None, **kwargs) |
| 343 |
|
| 344 |
Create a task in this task group. |
| 345 |
The signature matches that of :func:`asyncio.create_task`. |
| 346 |
If the task group is inactive (e.g. not yet entered, |
| 347 |
already finished, or in the process of shutting down), |
| 348 |
we will close the given ``coro``. |
| 349 |
|
| 350 |
.. versionchanged:: 3.13 |
| 351 |
|
| 352 |
Close the given coroutine if the task group is not active. |
| 353 |
|
| 354 |
.. versionchanged:: 3.14 |
| 355 |
|
| 356 |
Passes on all *kwargs* to :meth:`loop.create_task` |
| 357 |
|
| 358 |
Example:: |
| 359 |
|
| 360 |
async def main(): |
| 361 |
async with asyncio.TaskGroup() as tg: |
| 362 |
task1 = tg.create_task(some_coro(...)) |
| 363 |
task2 = tg.create_task(another_coro(...)) |
| 364 |
print(f"Both tasks have completed now: {task1.result()}, {task2.result()}") |
| 365 |
|
| 366 |
The ``async with`` statement will wait for all tasks in the group to finish. |
| 367 |
While waiting, new tasks may still be added to the group |
| 368 |
(for example, by passing ``tg`` into one of the coroutines |
| 369 |
and calling ``tg.create_task()`` in that coroutine). |
| 370 |
Once the last task has finished and the ``async with`` block is exited, |
| 371 |
no new tasks may be added to the group. |
| 372 |
|
| 373 |
The first time any of the tasks belonging to the group fails |
| 374 |
with an exception other than :exc:`asyncio.CancelledError`, |
| 375 |
the remaining tasks in the group are cancelled. |
| 376 |
No further tasks can then be added to the group. |
| 377 |
At this point, if the body of the ``async with`` statement is still active |
| 378 |
(i.e., :meth:`~object.__aexit__` hasn't been called yet), |
| 379 |
the task directly containing the ``async with`` statement is also cancelled. |
| 380 |
The resulting :exc:`asyncio.CancelledError` will interrupt an ``await``, |
| 381 |
but it will not bubble out of the containing ``async with`` statement. |
| 382 |
|
| 383 |
Once all tasks have finished, if any tasks have failed |
| 384 |
with an exception other than :exc:`asyncio.CancelledError`, |
| 385 |
those exceptions are combined in an |
| 386 |
:exc:`ExceptionGroup` or :exc:`BaseExceptionGroup` |
| 387 |
(as appropriate; see their documentation) |
| 388 |
which is then raised. |
| 389 |
|
| 390 |
Two base exceptions are treated specially: |
| 391 |
If any task fails with :exc:`KeyboardInterrupt` or :exc:`SystemExit`, |
| 392 |
the task group still cancels the remaining tasks and waits for them, |
| 393 |
but then the initial :exc:`KeyboardInterrupt` or :exc:`SystemExit` |
| 394 |
is re-raised instead of :exc:`ExceptionGroup` or :exc:`BaseExceptionGroup`. |
| 395 |
|
| 396 |
If the body of the ``async with`` statement exits with an exception |
| 397 |
(so :meth:`~object.__aexit__` is called with an exception set), |
| 398 |
this is treated the same as if one of the tasks failed: |
| 399 |
the remaining tasks are cancelled and then waited for, |
| 400 |
and non-cancellation exceptions are grouped into an |
| 401 |
exception group and raised. |
| 402 |
The exception passed into :meth:`~object.__aexit__`, |
| 403 |
unless it is :exc:`asyncio.CancelledError`, |
| 404 |
is also included in the exception group. |
| 405 |
The same special case is made for |
| 406 |
:exc:`KeyboardInterrupt` and :exc:`SystemExit` as in the previous paragraph. |
| 407 |
|
| 408 |
Task groups are careful not to mix up the internal cancellation used to |
| 409 |
"wake up" their :meth:`~object.__aexit__` with cancellation requests |
| 410 |
for the task in which they are running made by other parties. |
| 411 |
In particular, when one task group is syntactically nested in another, |
| 412 |
and both experience an exception in one of their child tasks simultaneously, |
| 413 |
the inner task group will process its exceptions, and then the outer task group |
| 414 |
will receive another cancellation and process its own exceptions. |
| 415 |
|
| 416 |
In the case where a task group is cancelled externally and also must |
| 417 |
raise an :exc:`ExceptionGroup`, it will call the parent task's |
| 418 |
:meth:`~asyncio.Task.cancel` method. This ensures that a |
| 419 |
:exc:`asyncio.CancelledError` will be raised at the next |
| 420 |
:keyword:`await`, so the cancellation is not lost. |
| 421 |
|
| 422 |
Task groups preserve the cancellation count |
| 423 |
reported by :meth:`asyncio.Task.cancelling`. |
| 424 |
|
| 425 |
.. versionchanged:: 3.13 |
| 426 |
|
| 427 |
Improved handling of simultaneous internal and external cancellations |
| 428 |
and correct preservation of cancellation counts. |
| 429 |
|
| 430 |
Terminating a task group |
| 431 |
------------------------ |
| 432 |
|
| 433 |
While terminating a task group is not natively supported by the standard |
| 434 |
library, termination can be achieved by adding an exception-raising task |
| 435 |
to the task group and ignoring the raised exception: |
| 436 |
|
| 437 |
.. code-block:: python |
| 438 |
|
| 439 |
import asyncio |
| 440 |
from asyncio import TaskGroup |
| 441 |
|
| 442 |
class TerminateTaskGroup(Exception): |
| 443 |
"""Exception raised to terminate a task group.""" |
| 444 |
|
| 445 |
async def force_terminate_task_group(): |
| 446 |
"""Used to force termination of a task group.""" |
| 447 |
raise TerminateTaskGroup() |
| 448 |
|
| 449 |
async def job(task_id, sleep_time): |
| 450 |
print(f'Task {task_id}: start') |
| 451 |
await asyncio.sleep(sleep_time) |
| 452 |
print(f'Task {task_id}: done') |
| 453 |
|
| 454 |
async def main(): |
| 455 |
try: |
| 456 |
async with TaskGroup() as group: |
| 457 |
# spawn some tasks |
| 458 |
group.create_task(job(1, 0.5)) |
| 459 |
group.create_task(job(2, 1.5)) |
| 460 |
# sleep for 1 second |
| 461 |
await asyncio.sleep(1) |
| 462 |
# add an exception-raising task to force the group to terminate |
| 463 |
group.create_task(force_terminate_task_group()) |
| 464 |
except* TerminateTaskGroup: |
| 465 |
pass |
| 466 |
|
| 467 |
asyncio.run(main()) |
| 468 |
|
| 469 |
Expected output: |
| 470 |
|
| 471 |
.. code-block:: text |
| 472 |
|
| 473 |
Task 1: start |
| 474 |
Task 2: start |
| 475 |
Task 1: done |
| 476 |
|
| 477 |
Sleeping |
| 478 |
======== |
| 479 |
|
| 480 |
.. function:: sleep(delay, result=None) |
| 481 |
:async: |
| 482 |
|
| 483 |
Block for *delay* seconds. |
| 484 |
|
| 485 |
If *result* is provided, it is returned to the caller |
| 486 |
when the coroutine completes. |
| 487 |
|
| 488 |
``sleep()`` always suspends the current task, allowing other tasks |
| 489 |
to run. |
| 490 |
|
| 491 |
Setting the delay to 0 provides an optimized path to allow other |
| 492 |
tasks to run. This can be used by long-running functions to avoid |
| 493 |
blocking the event loop for the full duration of the function call. |
| 494 |
|
| 495 |
.. _asyncio_example_sleep: |
| 496 |
|
| 497 |
Example of coroutine displaying the current date every second |
| 498 |
for 5 seconds:: |
| 499 |
|
| 500 |
import asyncio |
| 501 |
import datetime as dt |
| 502 |
|
| 503 |
async def display_date(): |
| 504 |
loop = asyncio.get_running_loop() |
| 505 |
end_time = loop.time() + 5.0 |
| 506 |
while True: |
| 507 |
print(dt.datetime.now()) |
| 508 |
if (loop.time() + 1.0) >= end_time: |
| 509 |
break |
| 510 |
await asyncio.sleep(1) |
| 511 |
|
| 512 |
asyncio.run(display_date()) |
| 513 |
|
| 514 |
|
| 515 |
.. versionchanged:: 3.10 |
| 516 |
Removed the *loop* parameter. |
| 517 |
|
| 518 |
.. versionchanged:: 3.13 |
| 519 |
Raises :exc:`ValueError` if *delay* is :data:`~math.nan`. |
| 520 |
|
| 521 |
|
| 522 |
Running tasks concurrently |
| 523 |
========================== |
| 524 |
|
| 525 |
.. awaitablefunction:: gather(*aws, return_exceptions=False) |
| 526 |
|
| 527 |
Run :ref:`awaitable objects <asyncio-awaitables>` in the *aws* |
| 528 |
sequence *concurrently*. |
| 529 |
|
| 530 |
If any awaitable in *aws* is a coroutine, it is automatically |
| 531 |
scheduled as a Task. |
| 532 |
|
| 533 |
If all awaitables are completed successfully, the result is an |
| 534 |
aggregate list of returned values. The order of result values |
| 535 |
corresponds to the order of awaitables in *aws*. |
| 536 |
|
| 537 |
If *return_exceptions* is ``False`` (default), the first |
| 538 |
raised exception is immediately propagated to the task that |
| 539 |
awaits on ``gather()``. Other awaitables in the *aws* sequence |
| 540 |
**won't be cancelled** and will continue to run. |
| 541 |
|
| 542 |
If *return_exceptions* is ``True``, exceptions are treated the |
| 543 |
same as successful results, and aggregated in the result list. |
| 544 |
|
| 545 |
If ``gather()`` is *cancelled*, all submitted awaitables |
| 546 |
(that have not completed yet) are also *cancelled*. |
| 547 |
|
| 548 |
If any Task or Future from the *aws* sequence is *cancelled*, it is |
| 549 |
treated as if it raised :exc:`CancelledError` -- the ``gather()`` |
| 550 |
call is **not** cancelled in this case. This is to prevent the |
| 551 |
cancellation of one submitted Task/Future to cause other |
| 552 |
Tasks/Futures to be cancelled. |
| 553 |
|
| 554 |
.. note:: |
| 555 |
A new alternative to create and run tasks concurrently and |
| 556 |
wait for their completion is :class:`asyncio.TaskGroup`. *TaskGroup* |
| 557 |
provides stronger safety guarantees than *gather* for scheduling a nesting of subtasks: |
| 558 |
if a task (or a subtask, a task scheduled by a task) |
| 559 |
raises an exception, *TaskGroup* will, while *gather* will not, |
| 560 |
cancel the remaining scheduled tasks. |
| 561 |
|
| 562 |
.. _asyncio_example_gather: |
| 563 |
|
| 564 |
Example:: |
| 565 |
|
| 566 |
import asyncio |
| 567 |
|
| 568 |
async def factorial(name, number): |
| 569 |
f = 1 |
| 570 |
for i in range(2, number + 1): |
| 571 |
print(f"Task {name}: Compute factorial({number}), currently i={i}...") |
| 572 |
await asyncio.sleep(1) |
| 573 |
f *= i |
| 574 |
print(f"Task {name}: factorial({number}) = {f}") |
| 575 |
return f |
| 576 |
|
| 577 |
async def main(): |
| 578 |
# Schedule three calls *concurrently*: |
| 579 |
L = await asyncio.gather( |
| 580 |
factorial("A", 2), |
| 581 |
factorial("B", 3), |
| 582 |
factorial("C", 4), |
| 583 |
) |
| 584 |
print(L) |
| 585 |
|
| 586 |
asyncio.run(main()) |
| 587 |
|
| 588 |
# Expected output: |
| 589 |
# |
| 590 |
# Task A: Compute factorial(2), currently i=2... |
| 591 |
# Task B: Compute factorial(3), currently i=2... |
| 592 |
# Task C: Compute factorial(4), currently i=2... |
| 593 |
# Task A: factorial(2) = 2 |
| 594 |
# Task B: Compute factorial(3), currently i=3... |
| 595 |
# Task C: Compute factorial(4), currently i=3... |
| 596 |
# Task B: factorial(3) = 6 |
| 597 |
# Task C: Compute factorial(4), currently i=4... |
| 598 |
# Task C: factorial(4) = 24 |
| 599 |
# [2, 6, 24] |
| 600 |
|
| 601 |
.. note:: |
| 602 |
If *return_exceptions* is false, cancelling gather() after it |
| 603 |
has been marked done won't cancel any submitted awaitables. |
| 604 |
For instance, gather can be marked done after propagating an |
| 605 |
exception to the caller, therefore, calling ``gather.cancel()`` |
| 606 |
after catching an exception (raised by one of the awaitables) from |
| 607 |
gather won't cancel any other awaitables. |
| 608 |
|
| 609 |
.. versionchanged:: 3.7 |
| 610 |
If the *gather* itself is cancelled, the cancellation is |
| 611 |
propagated regardless of *return_exceptions*. |
| 612 |
|
| 613 |
.. versionchanged:: 3.10 |
| 614 |
Removed the *loop* parameter. |
| 615 |
|
| 616 |
.. deprecated:: 3.10 |
| 617 |
Deprecation warning is emitted if no positional arguments are provided |
| 618 |
or not all positional arguments are Future-like objects |
| 619 |
and there is no running event loop. |
| 620 |
|
| 621 |
|
| 622 |
.. _eager-task-factory: |
| 623 |
|
| 624 |
Eager task factory |
| 625 |
================== |
| 626 |
|
| 627 |
.. function:: eager_task_factory(loop, coro, *, name=None, context=None) |
| 628 |
|
| 629 |
A task factory for eager task execution. |
| 630 |
|
| 631 |
When using this factory (via :meth:`loop.set_task_factory(asyncio.eager_task_factory) <loop.set_task_factory>`), |
| 632 |
coroutines begin execution synchronously during :class:`Task` construction. |
| 633 |
Tasks are only scheduled on the event loop if they block. |
| 634 |
This can be a performance improvement as the overhead of loop scheduling |
| 635 |
is avoided for coroutines that complete synchronously. |
| 636 |
|
| 637 |
A common example where this is beneficial is coroutines which employ |
| 638 |
caching or memoization to avoid actual I/O when possible. |
| 639 |
|
| 640 |
.. note:: |
| 641 |
|
| 642 |
Immediate execution of the coroutine is a semantic change. |
| 643 |
If the coroutine returns or raises, the task is never scheduled |
| 644 |
to the event loop. If the coroutine execution blocks, the task is |
| 645 |
scheduled to the event loop. This change may introduce behavior |
| 646 |
changes to existing applications. For example, |
| 647 |
the application's task execution order is likely to change. |
| 648 |
|
| 649 |
.. versionadded:: 3.12 |
| 650 |
|
| 651 |
.. function:: create_eager_task_factory(custom_task_constructor) |
| 652 |
|
| 653 |
Create an eager task factory, similar to :func:`eager_task_factory`, |
| 654 |
using the provided *custom_task_constructor* when creating a new task instead |
| 655 |
of the default :class:`Task`. |
| 656 |
|
| 657 |
*custom_task_constructor* must be a *callable* with the signature matching |
| 658 |
the signature of :class:`Task.__init__ <Task>`. |
| 659 |
The callable must return a :class:`asyncio.Task`-compatible object. |
| 660 |
|
| 661 |
This function returns a *callable* intended to be used as a task factory of an |
| 662 |
event loop via :meth:`loop.set_task_factory(factory) <loop.set_task_factory>`). |
| 663 |
|
| 664 |
.. versionadded:: 3.12 |
| 665 |
|
| 666 |
|
| 667 |
Shielding from cancellation |
| 668 |
=========================== |
| 669 |
|
| 670 |
.. awaitablefunction:: shield(aw) |
| 671 |
|
| 672 |
Protect an :ref:`awaitable object <asyncio-awaitables>` |
| 673 |
from being :meth:`cancelled <Task.cancel>`. |
| 674 |
|
| 675 |
If *aw* is a coroutine it is automatically scheduled as a Task. |
| 676 |
|
| 677 |
The statement:: |
| 678 |
|
| 679 |
task = asyncio.create_task(something()) |
| 680 |
res = await shield(task) |
| 681 |
|
| 682 |
is equivalent to:: |
| 683 |
|
| 684 |
res = await something() |
| 685 |
|
| 686 |
*except* that if the coroutine containing it is cancelled, the |
| 687 |
Task running in ``something()`` is not cancelled. From the point |
| 688 |
of view of ``something()``, the cancellation did not happen. |
| 689 |
Although its caller is still cancelled, so the "await" expression |
| 690 |
still raises a :exc:`CancelledError`. |
| 691 |
|
| 692 |
If ``something()`` is cancelled by other means (i.e. from within |
| 693 |
itself) that would also cancel ``shield()``. |
| 694 |
|
| 695 |
If it is desired to completely ignore cancellation (not recommended) |
| 696 |
the ``shield()`` function should be combined with a try/except |
| 697 |
clause, as follows:: |
| 698 |
|
| 699 |
task = asyncio.create_task(something()) |
| 700 |
try: |
| 701 |
res = await shield(task) |
| 702 |
except CancelledError: |
| 703 |
res = None |
| 704 |
|
| 705 |
.. important:: |
| 706 |
|
| 707 |
Save a reference to tasks passed to this function, to avoid |
| 708 |
a task disappearing mid-execution. The event loop only keeps |
| 709 |
weak references to tasks. A task that isn't referenced elsewhere |
| 710 |
may get garbage collected at any time, even before it's done. |
| 711 |
|
| 712 |
.. versionchanged:: 3.10 |
| 713 |
Removed the *loop* parameter. |
| 714 |
|
| 715 |
.. deprecated:: 3.10 |
| 716 |
Deprecation warning is emitted if *aw* is not Future-like object |
| 717 |
and there is no running event loop. |
| 718 |
|
| 719 |
|
| 720 |
Timeouts |
| 721 |
======== |
| 722 |
|
| 723 |
.. function:: timeout(delay) |
| 724 |
|
| 725 |
Return an :ref:`asynchronous context manager <async-context-managers>` |
| 726 |
that can be used to limit the amount of time spent waiting on |
| 727 |
something. |
| 728 |
|
| 729 |
*delay* can either be ``None``, or a float/int number of |
| 730 |
seconds to wait. If *delay* is ``None``, no time limit will |
| 731 |
be applied; this can be useful if the delay is unknown when |
| 732 |
the context manager is created. |
| 733 |
|
| 734 |
In either case, the context manager can be rescheduled after |
| 735 |
creation using :meth:`Timeout.reschedule`. |
| 736 |
|
| 737 |
Example:: |
| 738 |
|
| 739 |
async def main(): |
| 740 |
async with asyncio.timeout(10): |
| 741 |
await long_running_task() |
| 742 |
|
| 743 |
If ``long_running_task`` takes more than 10 seconds to complete, |
| 744 |
the context manager will cancel the current task and handle |
| 745 |
the resulting :exc:`asyncio.CancelledError` internally, transforming it |
| 746 |
into a :exc:`TimeoutError` which can be caught and handled. |
| 747 |
|
| 748 |
.. note:: |
| 749 |
|
| 750 |
The :func:`asyncio.timeout` context manager is what transforms |
| 751 |
the :exc:`asyncio.CancelledError` into a :exc:`TimeoutError`, |
| 752 |
which means the :exc:`TimeoutError` can only be caught |
| 753 |
*outside* of the context manager. |
| 754 |
|
| 755 |
Example of catching :exc:`TimeoutError`:: |
| 756 |
|
| 757 |
async def main(): |
| 758 |
try: |
| 759 |
async with asyncio.timeout(10): |
| 760 |
await long_running_task() |
| 761 |
except TimeoutError: |
| 762 |
print("The long operation timed out, but we've handled it.") |
| 763 |
|
| 764 |
print("This statement will run regardless.") |
| 765 |
|
| 766 |
The context manager produced by :func:`asyncio.timeout` can be |
| 767 |
rescheduled to a different deadline and inspected. |
| 768 |
|
| 769 |
.. class:: Timeout(when) |
| 770 |
|
| 771 |
An :ref:`asynchronous context manager <async-context-managers>` |
| 772 |
for cancelling overdue coroutines. |
| 773 |
|
| 774 |
Prefer using :func:`asyncio.timeout` or :func:`asyncio.timeout_at` |
| 775 |
rather than instantiating :class:`!Timeout` directly. |
| 776 |
|
| 777 |
``when`` should be an absolute time at which the context should time out, |
| 778 |
as measured by the event loop's clock: |
| 779 |
|
| 780 |
- If ``when`` is ``None``, the timeout will never trigger. |
| 781 |
- If ``when < loop.time()``, the timeout will trigger on the next |
| 782 |
iteration of the event loop. |
| 783 |
|
| 784 |
.. method:: when() -> float | None |
| 785 |
|
| 786 |
Return the current deadline, or ``None`` if the current |
| 787 |
deadline is not set. |
| 788 |
|
| 789 |
.. method:: reschedule(when: float | None) |
| 790 |
|
| 791 |
Reschedule the timeout. |
| 792 |
|
| 793 |
.. method:: expired() -> bool |
| 794 |
|
| 795 |
Return whether the context manager has exceeded its deadline |
| 796 |
(expired). |
| 797 |
|
| 798 |
Example:: |
| 799 |
|
| 800 |
async def main(): |
| 801 |
try: |
| 802 |
# We do not know the timeout when starting, so we pass ``None``. |
| 803 |
async with asyncio.timeout(None) as cm: |
| 804 |
# We know the timeout now, so we reschedule it. |
| 805 |
new_deadline = get_running_loop().time() + 10 |
| 806 |
cm.reschedule(new_deadline) |
| 807 |
|
| 808 |
await long_running_task() |
| 809 |
except TimeoutError: |
| 810 |
pass |
| 811 |
|
| 812 |
if cm.expired(): |
| 813 |
print("Looks like we haven't finished on time.") |
| 814 |
|
| 815 |
Timeout context managers can be safely nested. |
| 816 |
|
| 817 |
.. versionadded:: 3.11 |
| 818 |
|
| 819 |
.. function:: timeout_at(when) |
| 820 |
|
| 821 |
Similar to :func:`asyncio.timeout`, except *when* is the absolute time |
| 822 |
to stop waiting, or ``None``. |
| 823 |
|
| 824 |
Example:: |
| 825 |
|
| 826 |
async def main(): |
| 827 |
loop = get_running_loop() |
| 828 |
deadline = loop.time() + 20 |
| 829 |
try: |
| 830 |
async with asyncio.timeout_at(deadline): |
| 831 |
await long_running_task() |
| 832 |
except TimeoutError: |
| 833 |
print("The long operation timed out, but we've handled it.") |
| 834 |
|
| 835 |
print("This statement will run regardless.") |
| 836 |
|
| 837 |
.. versionadded:: 3.11 |
| 838 |
|
| 839 |
.. function:: wait_for(aw, timeout) |
| 840 |
:async: |
| 841 |
|
| 842 |
Wait for the *aw* :ref:`awaitable <asyncio-awaitables>` |
| 843 |
to complete with a timeout. |
| 844 |
|
| 845 |
If *aw* is a coroutine it is automatically scheduled as a Task. |
| 846 |
|
| 847 |
*timeout* can either be ``None`` or a float or int number of seconds |
| 848 |
to wait for. If *timeout* is ``None``, block until the future |
| 849 |
completes. |
| 850 |
|
| 851 |
If a timeout occurs, it cancels the task and raises |
| 852 |
:exc:`TimeoutError`. |
| 853 |
|
| 854 |
To avoid the task :meth:`cancellation <Task.cancel>`, |
| 855 |
wrap it in :func:`shield`. |
| 856 |
|
| 857 |
The function will wait until the future is actually cancelled, |
| 858 |
so the total wait time may exceed the *timeout*. If an exception |
| 859 |
happens during cancellation, it is propagated. |
| 860 |
|
| 861 |
If the wait is cancelled, the future *aw* is also cancelled. |
| 862 |
|
| 863 |
.. _asyncio_example_waitfor: |
| 864 |
|
| 865 |
Example:: |
| 866 |
|
| 867 |
async def eternity(): |
| 868 |
# Sleep for one hour |
| 869 |
await asyncio.sleep(3600) |
| 870 |
print('yay!') |
| 871 |
|
| 872 |
async def main(): |
| 873 |
# Wait for at most 1 second |
| 874 |
try: |
| 875 |
await asyncio.wait_for(eternity(), timeout=1.0) |
| 876 |
except TimeoutError: |
| 877 |
print('timeout!') |
| 878 |
|
| 879 |
asyncio.run(main()) |
| 880 |
|
| 881 |
# Expected output: |
| 882 |
# |
| 883 |
# timeout! |
| 884 |
|
| 885 |
.. versionchanged:: 3.7 |
| 886 |
When *aw* is cancelled due to a timeout, ``wait_for`` waits |
| 887 |
for *aw* to be cancelled. Previously, it raised |
| 888 |
:exc:`TimeoutError` immediately. |
| 889 |
|
| 890 |
.. versionchanged:: 3.10 |
| 891 |
Removed the *loop* parameter. |
| 892 |
|
| 893 |
.. versionchanged:: 3.11 |
| 894 |
Raises :exc:`TimeoutError` instead of :exc:`asyncio.TimeoutError`. |
| 895 |
|
| 896 |
|
| 897 |
Waiting primitives |
| 898 |
================== |
| 899 |
|
| 900 |
.. function:: wait(aws, *, timeout=None, return_when=ALL_COMPLETED) |
| 901 |
:async: |
| 902 |
|
| 903 |
Run :class:`~asyncio.Future` and :class:`~asyncio.Task` instances in the *aws* |
| 904 |
iterable concurrently and block until the condition specified |
| 905 |
by *return_when*. |
| 906 |
|
| 907 |
The *aws* iterable must not be empty. |
| 908 |
|
| 909 |
Returns two sets of Tasks/Futures: ``(done, pending)``. |
| 910 |
|
| 911 |
Usage:: |
| 912 |
|
| 913 |
done, pending = await asyncio.wait(aws) |
| 914 |
|
| 915 |
*timeout* (a float or int), if specified, can be used to control |
| 916 |
the maximum number of seconds to wait before returning. |
| 917 |
|
| 918 |
Note that this function does not raise :exc:`TimeoutError`. |
| 919 |
Futures or Tasks that aren't done when the timeout occurs are simply |
| 920 |
returned in the second set. |
| 921 |
|
| 922 |
*return_when* indicates when this function should return. It must |
| 923 |
be one of the following constants: |
| 924 |
|
| 925 |
.. list-table:: |
| 926 |
:header-rows: 1 |
| 927 |
|
| 928 |
* - Constant |
| 929 |
- Description |
| 930 |
|
| 931 |
* - .. data:: FIRST_COMPLETED |
| 932 |
- The function will return when any future finishes or is cancelled. |
| 933 |
|
| 934 |
* - .. data:: FIRST_EXCEPTION |
| 935 |
- The function will return when any future finishes by raising an |
| 936 |
exception. If no future raises an exception |
| 937 |
then it is equivalent to :const:`ALL_COMPLETED`. |
| 938 |
|
| 939 |
* - .. data:: ALL_COMPLETED |
| 940 |
- The function will return when all futures finish or are cancelled. |
| 941 |
|
| 942 |
Unlike :func:`~asyncio.wait_for`, ``wait()`` does not cancel the |
| 943 |
futures when a timeout occurs. |
| 944 |
|
| 945 |
.. versionchanged:: 3.10 |
| 946 |
Removed the *loop* parameter. |
| 947 |
|
| 948 |
.. versionchanged:: 3.11 |
| 949 |
Passing coroutine objects to ``wait()`` directly is forbidden. |
| 950 |
|
| 951 |
.. versionchanged:: 3.12 |
| 952 |
Added support for generators yielding tasks. |
| 953 |
|
| 954 |
|
| 955 |
.. function:: as_completed(aws, *, timeout=None) |
| 956 |
|
| 957 |
Run :ref:`awaitable objects <asyncio-awaitables>` in the *aws* iterable |
| 958 |
concurrently. The returned object can be iterated to obtain the results |
| 959 |
of the awaitables as they finish. |
| 960 |
|
| 961 |
The object returned by ``as_completed()`` can be iterated as an |
| 962 |
:term:`asynchronous iterator` or a plain :term:`iterator`. When asynchronous |
| 963 |
iteration is used, the originally-supplied awaitables are yielded if they |
| 964 |
are tasks or futures. This makes it easy to correlate previously-scheduled |
| 965 |
tasks with their results. Example:: |
| 966 |
|
| 967 |
ipv4_connect = create_task(open_connection("127.0.0.1", 80)) |
| 968 |
ipv6_connect = create_task(open_connection("::1", 80)) |
| 969 |
tasks = [ipv4_connect, ipv6_connect] |
| 970 |
|
| 971 |
async for earliest_connect in as_completed(tasks): |
| 972 |
# earliest_connect is done. The result can be obtained by |
| 973 |
# awaiting it or calling earliest_connect.result() |
| 974 |
reader, writer = await earliest_connect |
| 975 |
|
| 976 |
if earliest_connect is ipv6_connect: |
| 977 |
print("IPv6 connection established.") |
| 978 |
else: |
| 979 |
print("IPv4 connection established.") |
| 980 |
|
| 981 |
During asynchronous iteration, implicitly-created tasks will be yielded for |
| 982 |
supplied awaitables that aren't tasks or futures. |
| 983 |
|
| 984 |
When used as a plain iterator, each iteration yields a new coroutine that |
| 985 |
returns the result or raises the exception of the next completed awaitable. |
| 986 |
This pattern is compatible with Python versions older than 3.13:: |
| 987 |
|
| 988 |
ipv4_connect = create_task(open_connection("127.0.0.1", 80)) |
| 989 |
ipv6_connect = create_task(open_connection("::1", 80)) |
| 990 |
tasks = [ipv4_connect, ipv6_connect] |
| 991 |
|
| 992 |
for next_connect in as_completed(tasks): |
| 993 |
# next_connect is not one of the original task objects. It must be |
| 994 |
# awaited to obtain the result value or raise the exception of the |
| 995 |
# awaitable that finishes next. |
| 996 |
reader, writer = await next_connect |
| 997 |
|
| 998 |
A :exc:`TimeoutError` is raised if the timeout occurs before all awaitables |
| 999 |
are done. This is raised by the ``async for`` loop during asynchronous |
| 1000 |
iteration or by the coroutines yielded during plain iteration. |
| 1001 |
|
| 1002 |
.. versionchanged:: 3.10 |
| 1003 |
Removed the *loop* parameter. |
| 1004 |
|
| 1005 |
.. deprecated:: 3.10 |
| 1006 |
Deprecation warning is emitted if not all awaitable objects in the *aws* |
| 1007 |
iterable are Future-like objects and there is no running event loop. |
| 1008 |
|
| 1009 |
.. versionchanged:: 3.12 |
| 1010 |
Added support for generators yielding tasks. |
| 1011 |
|
| 1012 |
.. versionchanged:: 3.13 |
| 1013 |
The result can now be used as either an :term:`asynchronous iterator` |
| 1014 |
or as a plain :term:`iterator` (previously it was only a plain iterator). |
| 1015 |
|
| 1016 |
|
| 1017 |
Running in threads |
| 1018 |
================== |
| 1019 |
|
| 1020 |
.. function:: to_thread(func, /, *args, **kwargs) |
| 1021 |
:async: |
| 1022 |
|
| 1023 |
Asynchronously run function *func* in a separate thread. |
| 1024 |
|
| 1025 |
Any \*args and \*\*kwargs supplied for this function are directly passed |
| 1026 |
to *func*. Also, the current :class:`contextvars.Context` is propagated, |
| 1027 |
allowing context variables from the event loop thread to be accessed in the |
| 1028 |
separate thread. |
| 1029 |
|
| 1030 |
Return a coroutine that can be awaited to get the eventual result of *func*. |
| 1031 |
|
| 1032 |
This coroutine function is primarily intended to be used for executing |
| 1033 |
IO-bound functions/methods that would otherwise block the event loop if |
| 1034 |
they were run in the main thread. For example:: |
| 1035 |
|
| 1036 |
def blocking_io(): |
| 1037 |
print(f"start blocking_io at {time.strftime('%X')}") |
| 1038 |
# Note that time.sleep() can be replaced with any blocking |
| 1039 |
# IO-bound operation, such as file operations. |
| 1040 |
time.sleep(1) |
| 1041 |
print(f"blocking_io complete at {time.strftime('%X')}") |
| 1042 |
|
| 1043 |
async def main(): |
| 1044 |
print(f"started main at {time.strftime('%X')}") |
| 1045 |
|
| 1046 |
await asyncio.gather( |
| 1047 |
asyncio.to_thread(blocking_io), |
| 1048 |
asyncio.sleep(1)) |
| 1049 |
|
| 1050 |
print(f"finished main at {time.strftime('%X')}") |
| 1051 |
|
| 1052 |
|
| 1053 |
asyncio.run(main()) |
| 1054 |
|
| 1055 |
# Expected output: |
| 1056 |
# |
| 1057 |
# started main at 19:50:53 |
| 1058 |
# start blocking_io at 19:50:53 |
| 1059 |
# blocking_io complete at 19:50:54 |
| 1060 |
# finished main at 19:50:54 |
| 1061 |
|
| 1062 |
Directly calling ``blocking_io()`` in any coroutine would block the event loop |
| 1063 |
for its duration, resulting in an additional 1 second of run time. Instead, |
| 1064 |
by using ``asyncio.to_thread()``, we can run it in a separate thread without |
| 1065 |
blocking the event loop. |
| 1066 |
|
| 1067 |
.. note:: |
| 1068 |
|
| 1069 |
Due to the :term:`GIL`, ``asyncio.to_thread()`` can typically only be used |
| 1070 |
to make IO-bound functions non-blocking. However, for extension modules |
| 1071 |
that release the GIL or alternative Python implementations that don't |
| 1072 |
have one, ``asyncio.to_thread()`` can also be used for CPU-bound functions. |
| 1073 |
|
| 1074 |
.. versionadded:: 3.9 |
| 1075 |
|
| 1076 |
|
| 1077 |
Scheduling from other threads |
| 1078 |
============================= |
| 1079 |
|
| 1080 |
.. function:: run_coroutine_threadsafe(coro, loop) |
| 1081 |
|
| 1082 |
Submit a coroutine to the given event loop. Thread-safe. |
| 1083 |
|
| 1084 |
Return a :class:`concurrent.futures.Future` to wait for the result |
| 1085 |
from another OS thread. |
| 1086 |
|
| 1087 |
This function is meant to be called from a different OS thread |
| 1088 |
than the one where the event loop is running. Example:: |
| 1089 |
|
| 1090 |
def in_thread(loop: asyncio.AbstractEventLoop) -> None: |
| 1091 |
# Run some blocking IO |
| 1092 |
pathlib.Path("example.txt").write_text("hello world", encoding="utf8") |
| 1093 |
|
| 1094 |
# Create a coroutine |
| 1095 |
coro = asyncio.sleep(1, result=3) |
| 1096 |
|
| 1097 |
# Submit the coroutine to a given loop |
| 1098 |
future = asyncio.run_coroutine_threadsafe(coro, loop) |
| 1099 |
|
| 1100 |
# Wait for the result with an optional timeout argument |
| 1101 |
assert future.result(timeout=2) == 3 |
| 1102 |
|
| 1103 |
async def amain() -> None: |
| 1104 |
# Get the running loop |
| 1105 |
loop = asyncio.get_running_loop() |
| 1106 |
|
| 1107 |
# Run something in a thread |
| 1108 |
await asyncio.to_thread(in_thread, loop) |
| 1109 |
|
| 1110 |
It's also possible to run the other way around. Example:: |
| 1111 |
|
| 1112 |
@contextlib.contextmanager |
| 1113 |
def loop_in_thread() -> Generator[asyncio.AbstractEventLoop]: |
| 1114 |
loop_fut = concurrent.futures.Future[asyncio.AbstractEventLoop]() |
| 1115 |
stop_event = asyncio.Event() |
| 1116 |
|
| 1117 |
async def main() -> None: |
| 1118 |
loop_fut.set_result(asyncio.get_running_loop()) |
| 1119 |
await stop_event.wait() |
| 1120 |
|
| 1121 |
with concurrent.futures.ThreadPoolExecutor(1) as tpe: |
| 1122 |
complete_fut = tpe.submit(asyncio.run, main()) |
| 1123 |
for fut in concurrent.futures.as_completed((loop_fut, complete_fut)): |
| 1124 |
if fut is loop_fut: |
| 1125 |
loop = loop_fut.result() |
| 1126 |
try: |
| 1127 |
yield loop |
| 1128 |
finally: |
| 1129 |
loop.call_soon_threadsafe(stop_event.set) |
| 1130 |
else: |
| 1131 |
fut.result() |
| 1132 |
|
| 1133 |
# Create a loop in another thread |
| 1134 |
with loop_in_thread() as loop: |
| 1135 |
# Create a coroutine |
| 1136 |
coro = asyncio.sleep(1, result=3) |
| 1137 |
|
| 1138 |
# Submit the coroutine to a given loop |
| 1139 |
future = asyncio.run_coroutine_threadsafe(coro, loop) |
| 1140 |
|
| 1141 |
# Wait for the result with an optional timeout argument |
| 1142 |
assert future.result(timeout=2) == 3 |
| 1143 |
|
| 1144 |
If an exception is raised in the coroutine, the returned Future |
| 1145 |
will be notified. It can also be used to cancel the task in |
| 1146 |
the event loop:: |
| 1147 |
|
| 1148 |
try: |
| 1149 |
result = future.result(timeout) |
| 1150 |
except TimeoutError: |
| 1151 |
print('The coroutine took too long, cancelling the task...') |
| 1152 |
future.cancel() |
| 1153 |
except Exception as exc: |
| 1154 |
print(f'The coroutine raised an exception: {exc!r}') |
| 1155 |
else: |
| 1156 |
print(f'The coroutine returned: {result!r}') |
| 1157 |
|
| 1158 |
See the :ref:`concurrency and multithreading <asyncio-multithreading>` |
| 1159 |
section of the documentation. |
| 1160 |
|
| 1161 |
Unlike other asyncio functions this function requires the *loop* |
| 1162 |
argument to be passed explicitly. |
| 1163 |
|
| 1164 |
.. versionadded:: 3.5.1 |
| 1165 |
|
| 1166 |
|
| 1167 |
Introspection |
| 1168 |
============= |
| 1169 |
|
| 1170 |
|
| 1171 |
.. function:: current_task(loop=None) |
| 1172 |
|
| 1173 |
Return the currently running :class:`Task` instance, or ``None`` if |
| 1174 |
no task is running. |
| 1175 |
|
| 1176 |
If *loop* is ``None`` :func:`get_running_loop` is used to get |
| 1177 |
the current loop. |
| 1178 |
|
| 1179 |
.. versionadded:: 3.7 |
| 1180 |
|
| 1181 |
|
| 1182 |
.. function:: all_tasks(loop=None) |
| 1183 |
|
| 1184 |
Return a set of not yet finished :class:`Task` objects run by |
| 1185 |
the loop. |
| 1186 |
|
| 1187 |
If *loop* is ``None``, :func:`get_running_loop` is used for getting |
| 1188 |
current loop. |
| 1189 |
|
| 1190 |
.. versionadded:: 3.7 |
| 1191 |
|
| 1192 |
|
| 1193 |
.. function:: iscoroutine(obj) |
| 1194 |
|
| 1195 |
Return ``True`` if *obj* is a coroutine object. |
| 1196 |
|
| 1197 |
.. versionadded:: 3.4 |
| 1198 |
|
| 1199 |
.. _asyncio-task-obj: |
| 1200 |
|
| 1201 |
Task object |
| 1202 |
=========== |
| 1203 |
|
| 1204 |
.. class:: Task(coro, *, loop=None, name=None, context=None, eager_start=False) |
| 1205 |
|
| 1206 |
A :class:`Future-like <Future>` object that runs a Python |
| 1207 |
:ref:`coroutine <coroutine>`. Not thread-safe. |
| 1208 |
|
| 1209 |
Tasks are used to run coroutines in event loops. |
| 1210 |
If a coroutine awaits on a Future, the Task suspends |
| 1211 |
the execution of the coroutine and waits for the completion |
| 1212 |
of the Future. When the Future is *done*, the execution of |
| 1213 |
the wrapped coroutine resumes. |
| 1214 |
|
| 1215 |
Event loops use cooperative scheduling: an event loop runs |
| 1216 |
one Task at a time. While a Task awaits for the completion of a |
| 1217 |
Future, the event loop runs other Tasks, callbacks, or performs |
| 1218 |
IO operations. |
| 1219 |
|
| 1220 |
Use the high-level :func:`asyncio.create_task` function to create |
| 1221 |
Tasks, or the low-level :meth:`loop.create_task` or |
| 1222 |
:func:`ensure_future` functions. Manual instantiation of Tasks |
| 1223 |
is discouraged. |
| 1224 |
|
| 1225 |
To cancel a running Task use the :meth:`cancel` method. Calling it |
| 1226 |
will cause the Task to throw a :exc:`CancelledError` exception into |
| 1227 |
the wrapped coroutine. If a coroutine is awaiting on a Future |
| 1228 |
object during cancellation, the Future object will be cancelled. |
| 1229 |
|
| 1230 |
:meth:`cancelled` can be used to check if the Task was cancelled. |
| 1231 |
The method returns ``True`` if the wrapped coroutine did not |
| 1232 |
suppress the :exc:`CancelledError` exception and was actually |
| 1233 |
cancelled. |
| 1234 |
|
| 1235 |
:class:`asyncio.Task` inherits from :class:`Future` all of its |
| 1236 |
APIs except :meth:`Future.set_result` and |
| 1237 |
:meth:`Future.set_exception`. |
| 1238 |
|
| 1239 |
An optional keyword-only *context* argument allows specifying a |
| 1240 |
custom :class:`contextvars.Context` for the *coro* to run in. |
| 1241 |
If no *context* is provided, the Task copies the current context |
| 1242 |
and later runs its coroutine in the copied context. |
| 1243 |
|
| 1244 |
An optional keyword-only *eager_start* argument allows eagerly starting |
| 1245 |
the execution of the :class:`asyncio.Task` at task creation time. |
| 1246 |
If set to ``True`` and the event loop is running, the task will start |
| 1247 |
executing the coroutine immediately, until the first time the coroutine |
| 1248 |
blocks. If the coroutine returns or raises without blocking, the task |
| 1249 |
will be finished eagerly and will skip scheduling to the event loop. |
| 1250 |
|
| 1251 |
.. versionchanged:: 3.7 |
| 1252 |
Added support for the :mod:`contextvars` module. |
| 1253 |
|
| 1254 |
.. versionchanged:: 3.8 |
| 1255 |
Added the *name* parameter. |
| 1256 |
|
| 1257 |
.. deprecated:: 3.10 |
| 1258 |
Deprecation warning is emitted if *loop* is not specified |
| 1259 |
and there is no running event loop. |
| 1260 |
|
| 1261 |
.. versionchanged:: 3.11 |
| 1262 |
Added the *context* parameter. |
| 1263 |
|
| 1264 |
.. versionchanged:: 3.12 |
| 1265 |
Added the *eager_start* parameter. |
| 1266 |
|
| 1267 |
.. method:: done() |
| 1268 |
|
| 1269 |
Return ``True`` if the Task is *done*. |
| 1270 |
|
| 1271 |
A Task is *done* when the wrapped coroutine either returned |
| 1272 |
a value, raised an exception, or the Task was cancelled. |
| 1273 |
|
| 1274 |
.. method:: result() |
| 1275 |
|
| 1276 |
Return the result of the Task. |
| 1277 |
|
| 1278 |
If the Task is *done*, the result of the wrapped coroutine |
| 1279 |
is returned (or if the coroutine raised an exception, that |
| 1280 |
exception is re-raised.) |
| 1281 |
|
| 1282 |
If the Task has been *cancelled*, this method raises |
| 1283 |
a :exc:`CancelledError` exception. |
| 1284 |
|
| 1285 |
If the Task's result isn't yet available, this method raises |
| 1286 |
an :exc:`InvalidStateError` exception. |
| 1287 |
|
| 1288 |
.. method:: exception() |
| 1289 |
|
| 1290 |
Return the exception of the Task. |
| 1291 |
|
| 1292 |
If the wrapped coroutine raised an exception that exception |
| 1293 |
is returned. If the wrapped coroutine returned normally |
| 1294 |
this method returns ``None``. |
| 1295 |
|
| 1296 |
If the Task has been *cancelled*, this method raises a |
| 1297 |
:exc:`CancelledError` exception. |
| 1298 |
|
| 1299 |
If the Task isn't *done* yet, this method raises an |
| 1300 |
:exc:`InvalidStateError` exception. |
| 1301 |
|
| 1302 |
.. method:: add_done_callback(callback, *, context=None) |
| 1303 |
|
| 1304 |
Add a callback to be run when the Task is *done*. |
| 1305 |
|
| 1306 |
This method should only be used in low-level callback-based code. |
| 1307 |
|
| 1308 |
See the documentation of :meth:`Future.add_done_callback` |
| 1309 |
for more details. |
| 1310 |
|
| 1311 |
.. method:: remove_done_callback(callback) |
| 1312 |
|
| 1313 |
Remove *callback* from the callbacks list. |
| 1314 |
|
| 1315 |
This method should only be used in low-level callback-based code. |
| 1316 |
|
| 1317 |
See the documentation of :meth:`Future.remove_done_callback` |
| 1318 |
for more details. |
| 1319 |
|
| 1320 |
.. method:: get_stack(*, limit=None) |
| 1321 |
|
| 1322 |
Return the list of stack frames for this Task. |
| 1323 |
|
| 1324 |
If the wrapped coroutine is not done, this returns the stack |
| 1325 |
where it is suspended. If the coroutine has completed |
| 1326 |
successfully or was cancelled, this returns an empty list. |
| 1327 |
If the coroutine was terminated by an exception, this returns |
| 1328 |
the list of traceback frames. |
| 1329 |
|
| 1330 |
The frames are always ordered from oldest to newest. |
| 1331 |
|
| 1332 |
Only one stack frame is returned for a suspended coroutine. |
| 1333 |
|
| 1334 |
The optional *limit* argument sets the maximum number of frames |
| 1335 |
to return; by default all available frames are returned. |
| 1336 |
The ordering of the returned list differs depending on whether |
| 1337 |
a stack or a traceback is returned: the newest frames of a |
| 1338 |
stack are returned, but the oldest frames of a traceback are |
| 1339 |
returned. (This matches the behavior of the traceback module.) |
| 1340 |
|
| 1341 |
.. method:: print_stack(*, limit=None, file=None) |
| 1342 |
|
| 1343 |
Print the stack or traceback for this Task. |
| 1344 |
|
| 1345 |
This produces output similar to that of the traceback module |
| 1346 |
for the frames retrieved by :meth:`get_stack`. |
| 1347 |
|
| 1348 |
The *limit* argument is passed to :meth:`get_stack` directly. |
| 1349 |
|
| 1350 |
The *file* argument is an I/O stream to which the output |
| 1351 |
is written; by default output is written to :data:`sys.stdout`. |
| 1352 |
|
| 1353 |
.. method:: get_coro() |
| 1354 |
|
| 1355 |
Return the coroutine object wrapped by the :class:`Task`. |
| 1356 |
|
| 1357 |
.. note:: |
| 1358 |
|
| 1359 |
This will return ``None`` for Tasks which have already |
| 1360 |
completed eagerly. See the :ref:`Eager Task Factory <eager-task-factory>`. |
| 1361 |
|
| 1362 |
.. versionadded:: 3.8 |
| 1363 |
|
| 1364 |
.. versionchanged:: 3.12 |
| 1365 |
|
| 1366 |
Newly added eager task execution means result may be ``None``. |
| 1367 |
|
| 1368 |
.. method:: get_context() |
| 1369 |
|
| 1370 |
Return the :class:`contextvars.Context` object |
| 1371 |
associated with the task. |
| 1372 |
|
| 1373 |
.. versionadded:: 3.12 |
| 1374 |
|
| 1375 |
.. method:: get_name() |
| 1376 |
|
| 1377 |
Return the name of the Task. |
| 1378 |
|
| 1379 |
If no name has been explicitly assigned to the Task, the default |
| 1380 |
asyncio Task implementation generates a default name during |
| 1381 |
instantiation. |
| 1382 |
|
| 1383 |
.. versionadded:: 3.8 |
| 1384 |
|
| 1385 |
.. method:: set_name(value) |
| 1386 |
|
| 1387 |
Set the name of the Task. |
| 1388 |
|
| 1389 |
The *value* argument can be any object, which is then |
| 1390 |
converted to a string. |
| 1391 |
|
| 1392 |
In the default Task implementation, the name will be visible |
| 1393 |
in the :func:`repr` output of a task object. |
| 1394 |
|
| 1395 |
.. versionadded:: 3.8 |
| 1396 |
|
| 1397 |
.. method:: cancel(msg=None) |
| 1398 |
|
| 1399 |
Request the Task to be cancelled. |
| 1400 |
|
| 1401 |
If the Task is already *done* or *cancelled*, return ``False``, |
| 1402 |
otherwise, return ``True``. |
| 1403 |
|
| 1404 |
The method arranges for a :exc:`CancelledError` exception to be thrown |
| 1405 |
into the wrapped coroutine on the next cycle of the event loop. |
| 1406 |
|
| 1407 |
The coroutine then has a chance to clean up or even deny the |
| 1408 |
request by suppressing the exception with a :keyword:`try` ... |
| 1409 |
... ``except CancelledError`` ... :keyword:`finally` block. |
| 1410 |
Therefore, unlike :meth:`Future.cancel`, :meth:`Task.cancel` does |
| 1411 |
not guarantee that the Task will be cancelled, although |
| 1412 |
suppressing cancellation completely is not common and is actively |
| 1413 |
discouraged. Should the coroutine nevertheless decide to suppress |
| 1414 |
the cancellation, it needs to call :meth:`Task.uncancel` in addition |
| 1415 |
to catching the exception. |
| 1416 |
|
| 1417 |
.. versionchanged:: 3.9 |
| 1418 |
Added the *msg* parameter. |
| 1419 |
|
| 1420 |
.. versionchanged:: 3.11 |
| 1421 |
The ``msg`` parameter is propagated from cancelled task to its awaiter. |
| 1422 |
|
| 1423 |
.. _asyncio_example_task_cancel: |
| 1424 |
|
| 1425 |
The following example illustrates how coroutines can intercept |
| 1426 |
the cancellation request:: |
| 1427 |
|
| 1428 |
async def cancel_me(): |
| 1429 |
print('cancel_me(): before sleep') |
| 1430 |
|
| 1431 |
try: |
| 1432 |
# Wait for 1 hour |
| 1433 |
await asyncio.sleep(3600) |
| 1434 |
except asyncio.CancelledError: |
| 1435 |
print('cancel_me(): cancel sleep') |
| 1436 |
raise |
| 1437 |
finally: |
| 1438 |
print('cancel_me(): after sleep') |
| 1439 |
|
| 1440 |
async def main(): |
| 1441 |
# Create a "cancel_me" Task |
| 1442 |
task = asyncio.create_task(cancel_me()) |
| 1443 |
|
| 1444 |
# Wait for 1 second |
| 1445 |
await asyncio.sleep(1) |
| 1446 |
|
| 1447 |
task.cancel() |
| 1448 |
try: |
| 1449 |
await task |
| 1450 |
except asyncio.CancelledError: |
| 1451 |
print("main(): cancel_me is cancelled now") |
| 1452 |
|
| 1453 |
asyncio.run(main()) |
| 1454 |
|
| 1455 |
# Expected output: |
| 1456 |
# |
| 1457 |
# cancel_me(): before sleep |
| 1458 |
# cancel_me(): cancel sleep |
| 1459 |
# cancel_me(): after sleep |
| 1460 |
# main(): cancel_me is cancelled now |
| 1461 |
|
| 1462 |
.. method:: cancelled() |
| 1463 |
|
| 1464 |
Return ``True`` if the Task is *cancelled*. |
| 1465 |
|
| 1466 |
The Task is *cancelled* when the cancellation was requested with |
| 1467 |
:meth:`cancel` and the wrapped coroutine propagated the |
| 1468 |
:exc:`CancelledError` exception thrown into it. |
| 1469 |
|
| 1470 |
.. method:: uncancel() |
| 1471 |
|
| 1472 |
Decrement the count of cancellation requests to this Task. |
| 1473 |
|
| 1474 |
Returns the remaining number of cancellation requests. |
| 1475 |
|
| 1476 |
Note that once execution of a cancelled task completed, further |
| 1477 |
calls to :meth:`uncancel` are ineffective. |
| 1478 |
|
| 1479 |
.. versionadded:: 3.11 |
| 1480 |
|
| 1481 |
This method is used by asyncio's internals and isn't expected to be |
| 1482 |
used by end-user code. In particular, if a Task gets successfully |
| 1483 |
uncancelled, this allows for elements of structured concurrency like |
| 1484 |
:ref:`taskgroups` and :func:`asyncio.timeout` to continue running, |
| 1485 |
isolating cancellation to the respective structured block. |
| 1486 |
For example:: |
| 1487 |
|
| 1488 |
async def make_request_with_timeout(): |
| 1489 |
try: |
| 1490 |
async with asyncio.timeout(1): |
| 1491 |
# Structured block affected by the timeout: |
| 1492 |
await make_request() |
| 1493 |
await make_another_request() |
| 1494 |
except TimeoutError: |
| 1495 |
log("There was a timeout") |
| 1496 |
# Outer code not affected by the timeout: |
| 1497 |
await unrelated_code() |
| 1498 |
|
| 1499 |
While the block with ``make_request()`` and ``make_another_request()`` |
| 1500 |
might get cancelled due to the timeout, ``unrelated_code()`` should |
| 1501 |
continue running even in case of the timeout. This is implemented |
| 1502 |
with :meth:`uncancel`. :class:`TaskGroup` context managers use |
| 1503 |
:func:`uncancel` in a similar fashion. |
| 1504 |
|
| 1505 |
If end-user code is, for some reason, suppressing cancellation by |
| 1506 |
catching :exc:`CancelledError`, it needs to call this method to remove |
| 1507 |
the cancellation state. |
| 1508 |
|
| 1509 |
When this method decrements the cancellation count to zero, |
| 1510 |
the method checks if a previous :meth:`cancel` call had arranged |
| 1511 |
for :exc:`CancelledError` to be thrown into the task. |
| 1512 |
If it hasn't been thrown yet, that arrangement will be |
| 1513 |
rescinded (by resetting the internal ``_must_cancel`` flag). |
| 1514 |
|
| 1515 |
.. versionchanged:: 3.13 |
| 1516 |
Changed to rescind pending cancellation requests upon reaching zero. |
| 1517 |
|
| 1518 |
.. method:: cancelling() |
| 1519 |
|
| 1520 |
Return the number of pending cancellation requests to this Task, i.e., |
| 1521 |
the number of calls to :meth:`cancel` less the number of |
| 1522 |
:meth:`uncancel` calls. |
| 1523 |
|
| 1524 |
Note that if this number is greater than zero but the Task is |
| 1525 |
still executing, :meth:`cancelled` will still return ``False``. |
| 1526 |
This is because this number can be lowered by calling :meth:`uncancel`, |
| 1527 |
which can lead to the task not being cancelled after all if the |
| 1528 |
cancellation requests go down to zero. |
| 1529 |
|
| 1530 |
This method is used by asyncio's internals and isn't expected to be |
| 1531 |
used by end-user code. See :meth:`uncancel` for more details. |
| 1532 |
|
| 1533 |
.. versionadded:: 3.11 |