| 1 |
.. _a-conceptual-overview-of-asyncio: |
| 2 |
|
| 3 |
**************************************** |
| 4 |
A Conceptual Overview of :mod:`!asyncio` |
| 5 |
**************************************** |
| 6 |
|
| 7 |
This :ref:`HOWTO <how-tos>` article seeks to help you build a sturdy mental |
| 8 |
model of how :mod:`asyncio` fundamentally works, helping you understand the |
| 9 |
how and why behind the recommended patterns. |
| 10 |
|
| 11 |
You might be curious about some key :mod:`!asyncio` concepts. |
| 12 |
By the end of this article, you'll be able to comfortably answer these questions: |
| 13 |
|
| 14 |
- What's happening behind the scenes when an object is awaited? |
| 15 |
- How does :mod:`!asyncio` differentiate between a task which doesn't need |
| 16 |
CPU time (such as a network request or file read) as opposed to a task that |
| 17 |
does (such as computing n-factorial)? |
| 18 |
- How to write an asynchronous variant of an operation, such as |
| 19 |
an async sleep or database request. |
| 20 |
|
| 21 |
.. seealso:: |
| 22 |
|
| 23 |
* The `guide <https://github.com/anordin95/a-conceptual-overview-of-asyncio/ |
| 24 |
tree/main>`_ that inspired this HOWTO article, by Alexander Nordin. |
| 25 |
* This in-depth `YouTube tutorial series <https://www.youtube.com/ |
| 26 |
watch?v=Xbl7XjFYsN4&list=PLhNSoGM2ik6SIkVGXWBwerucXjgP1rHmB>`_ on |
| 27 |
``asyncio`` created by Python core team member, Ćukasz Langa. |
| 28 |
* `500 Lines or Less: A Web Crawler With asyncio Coroutines <https:// |
| 29 |
aosabook.org/en/500L/a-web-crawler-with-asyncio-coroutines.html>`_ by A. |
| 30 |
Jesse Jiryu Davis and Guido van Rossum. |
| 31 |
|
| 32 |
-------------------------------------------- |
| 33 |
A conceptual overview part 1: the high-level |
| 34 |
-------------------------------------------- |
| 35 |
|
| 36 |
In part 1, we'll cover the main, high-level building blocks of :mod:`!asyncio`: |
| 37 |
the event loop, coroutine functions, coroutine objects, tasks, and ``await``. |
| 38 |
|
| 39 |
========== |
| 40 |
Event Loop |
| 41 |
========== |
| 42 |
|
| 43 |
Everything in :mod:`!asyncio` happens relative to the event loop. |
| 44 |
It's the star of the show. |
| 45 |
It's like an orchestra conductor. |
| 46 |
It's behind the scenes managing resources. |
| 47 |
Some power is explicitly granted to it, but a lot of its ability to get things |
| 48 |
done comes from the respect and cooperation of its worker bees. |
| 49 |
|
| 50 |
In more technical terms, the event loop contains a collection of jobs to be run. |
| 51 |
Some jobs are added directly by you, and some indirectly by :mod:`!asyncio`. |
| 52 |
The event loop takes a job from its backlog of work and invokes it (or "gives |
| 53 |
it control"), similar to calling a function, and then that job runs. |
| 54 |
Once it pauses or completes, it returns control to the event loop. |
| 55 |
The event loop will then select another job from its pool and invoke it. |
| 56 |
You can *roughly* think of the collection of jobs as a queue: jobs are added and |
| 57 |
then processed one at a time, generally (but not always) in order. |
| 58 |
This process repeats indefinitely, with the event loop cycling endlessly |
| 59 |
onwards. |
| 60 |
If there are no more jobs pending execution, the event loop is smart enough to |
| 61 |
rest and avoid needlessly wasting CPU cycles, and will come back when there's |
| 62 |
more work to be done. |
| 63 |
|
| 64 |
Effective execution relies on jobs sharing well and cooperating; a greedy job |
| 65 |
could hog control and leave the other jobs to starve, rendering the overall |
| 66 |
event loop approach rather useless. |
| 67 |
|
| 68 |
:: |
| 69 |
|
| 70 |
import asyncio |
| 71 |
|
| 72 |
# This creates an event loop and indefinitely cycles through |
| 73 |
# its collection of jobs. |
| 74 |
event_loop = asyncio.new_event_loop() |
| 75 |
event_loop.run_forever() |
| 76 |
|
| 77 |
===================================== |
| 78 |
Asynchronous functions and coroutines |
| 79 |
===================================== |
| 80 |
|
| 81 |
This is a basic, boring Python function:: |
| 82 |
|
| 83 |
def hello_printer(): |
| 84 |
print( |
| 85 |
"Hi, I am a lowly, simple printer, though I have all I " |
| 86 |
"need in life -- \nfresh paper and my dearly beloved octopus " |
| 87 |
"partner in crime." |
| 88 |
) |
| 89 |
|
| 90 |
Calling a regular function invokes its logic or body:: |
| 91 |
|
| 92 |
>>> hello_printer() |
| 93 |
Hi, I am a lowly, simple printer, though I have all I need in life -- |
| 94 |
fresh paper and my dearly beloved octopus partner in crime. |
| 95 |
|
| 96 |
The :ref:`async def <async def>`, as opposed to just a plain ``def``, makes |
| 97 |
this an asynchronous function (or "coroutine function"). |
| 98 |
Calling it creates and returns a :ref:`coroutine <coroutine>` object. |
| 99 |
|
| 100 |
:: |
| 101 |
|
| 102 |
async def loudmouth_penguin(magic_number: int): |
| 103 |
print( |
| 104 |
"I am a super special talking penguin. Far cooler than that printer. " |
| 105 |
f"By the way, my lucky number is: {magic_number}." |
| 106 |
) |
| 107 |
|
| 108 |
Calling the async function, ``loudmouth_penguin``, does not execute the print statement; |
| 109 |
instead, it creates a coroutine object:: |
| 110 |
|
| 111 |
>>> loudmouth_penguin(magic_number=3) |
| 112 |
<coroutine object loudmouth_penguin at 0x104ed2740> |
| 113 |
|
| 114 |
The terms "coroutine function" and "coroutine object" are often conflated |
| 115 |
as coroutine. |
| 116 |
That can be confusing! |
| 117 |
In this article, coroutine specifically refers to a coroutine object, or more |
| 118 |
precisely, an instance of :data:`types.CoroutineType` (native coroutine). |
| 119 |
Note that coroutines can also exist as instances of |
| 120 |
:class:`collections.abc.Coroutine` -- a distinction that matters for type |
| 121 |
checking. |
| 122 |
|
| 123 |
A coroutine represents the function's body or logic. |
| 124 |
A coroutine has to be explicitly started; again, merely creating the coroutine |
| 125 |
does not start it. |
| 126 |
Notably, the coroutine can be paused and resumed at various points within the |
| 127 |
function's body. |
| 128 |
That pausing and resuming ability is what allows for asynchronous behavior! |
| 129 |
|
| 130 |
Coroutines and coroutine functions were built by leveraging the functionality |
| 131 |
of :term:`generators <generator iterator>` and |
| 132 |
:term:`generator functions <generator>`. |
| 133 |
Recall, a generator function is a function that :keyword:`yield`\s, like this |
| 134 |
one:: |
| 135 |
|
| 136 |
def get_random_number(): |
| 137 |
# This would be a bad random number generator! |
| 138 |
print("Hi") |
| 139 |
yield 1 |
| 140 |
print("Hello") |
| 141 |
yield 7 |
| 142 |
print("Howdy") |
| 143 |
yield 4 |
| 144 |
... |
| 145 |
|
| 146 |
Similar to a coroutine function, calling a generator function does not run it. |
| 147 |
Instead, it creates a generator object:: |
| 148 |
|
| 149 |
>>> get_random_number() |
| 150 |
<generator object get_random_number at 0x1048671c0> |
| 151 |
|
| 152 |
You can proceed to the next ``yield`` of a generator by using the |
| 153 |
built-in function :func:`next`. |
| 154 |
In other words, the generator runs, then pauses. |
| 155 |
For example:: |
| 156 |
|
| 157 |
>>> generator = get_random_number() |
| 158 |
>>> next(generator) |
| 159 |
Hi |
| 160 |
1 |
| 161 |
>>> next(generator) |
| 162 |
Hello |
| 163 |
7 |
| 164 |
|
| 165 |
===== |
| 166 |
Tasks |
| 167 |
===== |
| 168 |
|
| 169 |
Roughly speaking, :ref:`tasks <asyncio-task-obj>` are coroutines (not coroutine |
| 170 |
functions) tied to an event loop. |
| 171 |
A task also maintains a list of callback functions whose importance will become |
| 172 |
clear in a moment when we discuss :keyword:`await`. |
| 173 |
The recommended way to create tasks is via :func:`asyncio.create_task`. |
| 174 |
|
| 175 |
Creating a task automatically schedules it for execution (by adding a |
| 176 |
callback to run it in the event loop's to-do list, that is, collection of jobs). |
| 177 |
|
| 178 |
:mod:`!asyncio` automatically associates tasks with the event loop for you. |
| 179 |
This automatic association was purposely designed into :mod:`!asyncio` for |
| 180 |
the sake of simplicity. |
| 181 |
Without it, you'd have to keep track of the event loop object and pass it to |
| 182 |
any coroutine function that wants to create tasks, adding redundant clutter |
| 183 |
to your code. |
| 184 |
|
| 185 |
:: |
| 186 |
|
| 187 |
coroutine = loudmouth_penguin(magic_number=5) |
| 188 |
# This creates a Task object and schedules its execution via the event loop. |
| 189 |
task = asyncio.create_task(coroutine) |
| 190 |
|
| 191 |
Earlier, we manually created the event loop and set it to run forever. |
| 192 |
In practice, it's recommended to use (and common to see) :func:`asyncio.run`, |
| 193 |
which takes care of managing the event loop and ensuring the provided |
| 194 |
coroutine finishes before advancing. |
| 195 |
For example, many async programs follow this setup:: |
| 196 |
|
| 197 |
import asyncio |
| 198 |
|
| 199 |
async def main(): |
| 200 |
# Perform all sorts of wacky, wild asynchronous things... |
| 201 |
... |
| 202 |
|
| 203 |
if __name__ == "__main__": |
| 204 |
asyncio.run(main()) |
| 205 |
# The program will not reach the following print statement until the |
| 206 |
# coroutine main() finishes. |
| 207 |
print("coroutine main() is done!") |
| 208 |
|
| 209 |
It's important to be aware that the task itself is not added to the event loop, |
| 210 |
only a callback to the task is. |
| 211 |
This matters if the task object you created is garbage collected before it's |
| 212 |
called by the event loop. |
| 213 |
For example, consider this program: |
| 214 |
|
| 215 |
.. code-block:: |
| 216 |
:linenos: |
| 217 |
|
| 218 |
async def hello(): |
| 219 |
print("hello!") |
| 220 |
|
| 221 |
async def main(): |
| 222 |
asyncio.create_task(hello()) |
| 223 |
# Other asynchronous instructions which run for a while |
| 224 |
# and cede control to the event loop... |
| 225 |
... |
| 226 |
|
| 227 |
asyncio.run(main()) |
| 228 |
|
| 229 |
Because there's no reference to the task object created on line 5, it *might* |
| 230 |
be garbage collected before the event loop invokes it. |
| 231 |
Later instructions in the coroutine ``main()`` hand control back to the event |
| 232 |
loop so it can invoke other jobs. |
| 233 |
When the event loop eventually tries to run the task, it might fail and |
| 234 |
discover the task object does not exist! |
| 235 |
This can also happen even if a coroutine keeps a reference to a task but |
| 236 |
completes before that task finishes. |
| 237 |
When the coroutine exits, local variables go out of scope and may be subject |
| 238 |
to garbage collection. |
| 239 |
In practice, ``asyncio`` and Python's garbage collector work pretty hard to |
| 240 |
ensure this sort of thing doesn't happen. |
| 241 |
But that's no reason to be reckless! |
| 242 |
|
| 243 |
===== |
| 244 |
await |
| 245 |
===== |
| 246 |
|
| 247 |
:keyword:`await` is a Python keyword that's commonly used in one of two |
| 248 |
different ways:: |
| 249 |
|
| 250 |
await task |
| 251 |
await coroutine |
| 252 |
|
| 253 |
In a crucial way, the behavior of ``await`` depends on the type of object |
| 254 |
being awaited. |
| 255 |
|
| 256 |
Awaiting a task will cede control from the current task or coroutine to |
| 257 |
the event loop. |
| 258 |
In the process of relinquishing control, a few important things happen. |
| 259 |
We'll use the following code example to illustrate:: |
| 260 |
|
| 261 |
async def plant_a_tree(): |
| 262 |
dig_the_hole_task = asyncio.create_task(dig_the_hole()) |
| 263 |
await dig_the_hole_task |
| 264 |
|
| 265 |
# Other instructions associated with planting a tree. |
| 266 |
... |
| 267 |
|
| 268 |
In this example, imagine the event loop has passed control to the start of the |
| 269 |
coroutine ``plant_a_tree()``. |
| 270 |
As seen above, the coroutine creates a task and then awaits it. |
| 271 |
The ``await dig_the_hole_task`` instruction adds a callback (which will resume |
| 272 |
``plant_a_tree()``) to the ``dig_the_hole_task`` object's list of callbacks. |
| 273 |
And then, the instruction cedes control to the event loop. |
| 274 |
Some time later, the event loop will pass control to ``dig_the_hole_task`` |
| 275 |
and the task will finish whatever it needs to do. |
| 276 |
Once the task finishes, it will add its various callbacks to the event loop, |
| 277 |
in this case, a call to resume ``plant_a_tree()``. |
| 278 |
|
| 279 |
Generally speaking, when the awaited task finishes (``dig_the_hole_task``), |
| 280 |
the original task or coroutine (``plant_a_tree()``) is added back to the event |
| 281 |
loop's to-do list to be resumed. |
| 282 |
|
| 283 |
This is a basic, yet reliable mental model. |
| 284 |
In practice, the control handoffs are slightly more complex, but not by much. |
| 285 |
In part 2, we'll walk through the details that make this possible. |
| 286 |
|
| 287 |
**Unlike tasks, awaiting a coroutine does not hand control back to the event |
| 288 |
loop!** |
| 289 |
Wrapping a coroutine in a task first, then awaiting that would cede |
| 290 |
control. |
| 291 |
The behavior of ``await coroutine`` is effectively the same as invoking a |
| 292 |
regular, synchronous Python function. |
| 293 |
Consider this program:: |
| 294 |
|
| 295 |
import asyncio |
| 296 |
|
| 297 |
async def coro_a(): |
| 298 |
print("I am coro_a(). Hi!") |
| 299 |
|
| 300 |
async def coro_b(): |
| 301 |
print("I am coro_b(). I sure hope no one hogs the event loop...") |
| 302 |
|
| 303 |
async def main(): |
| 304 |
task_b = asyncio.create_task(coro_b()) |
| 305 |
num_repeats = 3 |
| 306 |
for _ in range(num_repeats): |
| 307 |
await coro_a() |
| 308 |
await task_b |
| 309 |
|
| 310 |
asyncio.run(main()) |
| 311 |
|
| 312 |
The first statement in the coroutine ``main()`` creates ``task_b`` and schedules |
| 313 |
it for execution via the event loop. |
| 314 |
Then, ``coro_a()`` is repeatedly awaited. Control never cedes to the |
| 315 |
event loop, which is why we see the output of all three ``coro_a()`` |
| 316 |
invocations before ``coro_b()``'s output: |
| 317 |
|
| 318 |
.. code-block:: none |
| 319 |
|
| 320 |
I am coro_a(). Hi! |
| 321 |
I am coro_a(). Hi! |
| 322 |
I am coro_a(). Hi! |
| 323 |
I am coro_b(). I sure hope no one hogs the event loop... |
| 324 |
|
| 325 |
If we change ``await coro_a()`` to ``await asyncio.create_task(coro_a())``, the |
| 326 |
behavior changes. |
| 327 |
The coroutine ``main()`` cedes control to the event loop with that statement. |
| 328 |
The event loop then proceeds through its backlog of work, calling ``task_b`` |
| 329 |
and then the task which wraps ``coro_a()`` before resuming the coroutine |
| 330 |
``main()``. |
| 331 |
|
| 332 |
.. code-block:: none |
| 333 |
|
| 334 |
I am coro_b(). I sure hope no one hogs the event loop... |
| 335 |
I am coro_a(). Hi! |
| 336 |
I am coro_a(). Hi! |
| 337 |
I am coro_a(). Hi! |
| 338 |
|
| 339 |
This behavior of ``await coroutine`` can trip a lot of people up! |
| 340 |
That example highlights how using only ``await coroutine`` could |
| 341 |
unintentionally hog control from other tasks and effectively stall the event |
| 342 |
loop. |
| 343 |
:func:`asyncio.run` can help you detect such occurrences via the |
| 344 |
``debug=True`` flag, which enables |
| 345 |
:ref:`debug mode <asyncio-debug-mode>`. |
| 346 |
Among other things, it will log any coroutines that monopolize execution for |
| 347 |
100ms or longer. |
| 348 |
|
| 349 |
The design intentionally trades off some conceptual clarity around usage of |
| 350 |
``await`` for improved performance. |
| 351 |
Each time a task is awaited, control needs to be passed all the way up the |
| 352 |
call stack to the event loop. |
| 353 |
That might sound minor, but in a large program with many ``await`` statements and a deep |
| 354 |
call stack, that overhead can add up to a meaningful performance drag. |
| 355 |
|
| 356 |
------------------------------------------------ |
| 357 |
A conceptual overview part 2: the nuts and bolts |
| 358 |
------------------------------------------------ |
| 359 |
|
| 360 |
Part 2 goes into detail on the mechanisms :mod:`!asyncio` uses to manage |
| 361 |
control flow. |
| 362 |
This is where the magic happens. |
| 363 |
You'll come away from this section knowing what ``await`` does behind the scenes |
| 364 |
and how to make your own asynchronous operators. |
| 365 |
|
| 366 |
================================ |
| 367 |
The inner workings of coroutines |
| 368 |
================================ |
| 369 |
|
| 370 |
:mod:`!asyncio` leverages four components to pass around control. |
| 371 |
|
| 372 |
:meth:`coroutine.send(arg) <generator.send>` is the method used to start or |
| 373 |
resume a coroutine. |
| 374 |
If the coroutine was paused and is now being resumed, the argument ``arg`` |
| 375 |
will be sent in as the return value of the ``yield`` statement which originally |
| 376 |
paused it. |
| 377 |
If the coroutine is being used for the first time (as opposed to being resumed), |
| 378 |
``arg`` must be ``None``. |
| 379 |
|
| 380 |
.. code-block:: |
| 381 |
:linenos: |
| 382 |
|
| 383 |
class Rock: |
| 384 |
def __await__(self): |
| 385 |
value_sent_in = yield 7 |
| 386 |
print(f"Rock.__await__ resuming with value: {value_sent_in}.") |
| 387 |
return value_sent_in |
| 388 |
|
| 389 |
async def main(): |
| 390 |
print("Beginning coroutine main().") |
| 391 |
rock = Rock() |
| 392 |
print("Awaiting rock...") |
| 393 |
value_from_rock = await rock |
| 394 |
print(f"Coroutine received value: {value_from_rock} from rock.") |
| 395 |
return 23 |
| 396 |
|
| 397 |
coroutine = main() |
| 398 |
intermediate_result = coroutine.send(None) |
| 399 |
print(f"Coroutine paused and returned intermediate value: {intermediate_result}.") |
| 400 |
|
| 401 |
print(f"Resuming coroutine and sending in value: 42.") |
| 402 |
try: |
| 403 |
coroutine.send(42) |
| 404 |
except StopIteration as e: |
| 405 |
returned_value = e.value |
| 406 |
print(f"Coroutine main() finished and provided value: {returned_value}.") |
| 407 |
|
| 408 |
:ref:`yield <yieldexpr>`, as usual, pauses execution and returns control |
| 409 |
to the caller. |
| 410 |
In the example above, the ``yield``, on line 3, is called by |
| 411 |
``... = await rock`` on line 11. |
| 412 |
More broadly speaking, ``await`` calls the :meth:`~object.__await__` method of |
| 413 |
the given object. |
| 414 |
``await`` also does one more very special thing: it propagates (or "passes |
| 415 |
along") any ``yield``\ s it receives up the call chain. |
| 416 |
In this case, that's back to ``... = coroutine.send(None)`` on line 16. |
| 417 |
|
| 418 |
The coroutine is resumed via the ``coroutine.send(42)`` call on line 21. |
| 419 |
The coroutine picks back up from where it ``yield``\ ed (or paused) on line 3 |
| 420 |
and executes the remaining statements in its body. |
| 421 |
When a coroutine finishes, it raises a :exc:`StopIteration` exception with the |
| 422 |
return value attached in the :attr:`~StopIteration.value` attribute. |
| 423 |
|
| 424 |
That snippet produces this output: |
| 425 |
|
| 426 |
.. code-block:: none |
| 427 |
|
| 428 |
Beginning coroutine main(). |
| 429 |
Awaiting rock... |
| 430 |
Coroutine paused and returned intermediate value: 7. |
| 431 |
Resuming coroutine and sending in value: 42. |
| 432 |
Rock.__await__ resuming with value: 42. |
| 433 |
Coroutine received value: 42 from rock. |
| 434 |
Coroutine main() finished and provided value: 23. |
| 435 |
|
| 436 |
It's worth pausing for a moment here and making sure you followed the various |
| 437 |
ways that control flow and values were passed. A lot of important ideas were |
| 438 |
covered and it's worth ensuring your understanding is firm. |
| 439 |
|
| 440 |
The only way to yield (or effectively cede control) from a coroutine is to |
| 441 |
``await`` an object that ``yield``\ s in its ``__await__`` method. |
| 442 |
That might sound odd to you. You might be thinking: |
| 443 |
|
| 444 |
1. What about a ``yield`` directly within the coroutine function? The |
| 445 |
coroutine function becomes an |
| 446 |
:ref:`async generator function <asynchronous-generator-functions>`, a |
| 447 |
different beast entirely. |
| 448 |
|
| 449 |
2. What about a :ref:`yield from <yieldexpr>` within the coroutine function to a (plain) |
| 450 |
generator? |
| 451 |
That causes the error: ``SyntaxError: yield from not allowed in a coroutine.`` |
| 452 |
This was intentionally designed for the sake of simplicity -- mandating only |
| 453 |
one way of using coroutines. |
| 454 |
Initially ``yield`` was barred as well, but was re-accepted to allow for |
| 455 |
async generators. |
| 456 |
Despite that, ``yield from`` and ``await`` effectively do the same thing. |
| 457 |
|
| 458 |
======= |
| 459 |
Futures |
| 460 |
======= |
| 461 |
|
| 462 |
A :ref:`future <asyncio-future-obj>` is an object meant to represent a |
| 463 |
computation's status and result. |
| 464 |
The term is a nod to the idea of something still to come or not yet happened, |
| 465 |
and the object is a way to keep an eye on that something. |
| 466 |
|
| 467 |
A future has a few important attributes. One is its state, which can be either |
| 468 |
"pending", "cancelled", or "done". |
| 469 |
Another is its result, which is set when the state transitions to done. |
| 470 |
Unlike a coroutine, a future does not represent the actual computation to be |
| 471 |
done; instead, it represents the status and result of that computation, kind of |
| 472 |
like a status light (red, yellow, or green) or indicator. |
| 473 |
|
| 474 |
:class:`asyncio.Task` subclasses :class:`asyncio.Future` in order to gain |
| 475 |
these various capabilities. |
| 476 |
The prior section said tasks store a list of callbacks, which wasn't entirely |
| 477 |
accurate. |
| 478 |
It's actually the ``Future`` class that implements this logic, which ``Task`` |
| 479 |
inherits. |
| 480 |
|
| 481 |
Futures may also be used directly (not via tasks). |
| 482 |
Tasks mark themselves as done when their coroutine is complete. |
| 483 |
Futures are much more versatile and will be marked as done when you say so. |
| 484 |
In this way, they're the flexible interface for you to make your own conditions |
| 485 |
for waiting and resuming. |
| 486 |
|
| 487 |
======================== |
| 488 |
A homemade asyncio.sleep |
| 489 |
======================== |
| 490 |
|
| 491 |
We'll go through an example of how you could leverage a future to create your |
| 492 |
own variant of asynchronous sleep (``async_sleep``) which mimics |
| 493 |
:func:`asyncio.sleep`. |
| 494 |
|
| 495 |
This snippet registers a few tasks with the event loop and then awaits the task |
| 496 |
created by ``asyncio.create_task``, which wraps the ``async_sleep(3)`` coroutine. |
| 497 |
We want that task to finish only after three seconds have elapsed, but without |
| 498 |
preventing other tasks from running. |
| 499 |
|
| 500 |
:: |
| 501 |
|
| 502 |
async def other_work(): |
| 503 |
print("I like work. Work work.") |
| 504 |
|
| 505 |
async def main(): |
| 506 |
# Add a few other tasks to the event loop, so there's something |
| 507 |
# to do while asynchronously sleeping. |
| 508 |
work_tasks = [ |
| 509 |
asyncio.create_task(other_work()), |
| 510 |
asyncio.create_task(other_work()), |
| 511 |
asyncio.create_task(other_work()) |
| 512 |
] |
| 513 |
print( |
| 514 |
"Beginning asynchronous sleep at time: " |
| 515 |
f"{datetime.datetime.now().strftime("%H:%M:%S")}." |
| 516 |
) |
| 517 |
await asyncio.create_task(async_sleep(3)) |
| 518 |
print( |
| 519 |
"Done asynchronous sleep at time: " |
| 520 |
f"{datetime.datetime.now().strftime("%H:%M:%S")}." |
| 521 |
) |
| 522 |
# asyncio.gather effectively awaits each task in the collection. |
| 523 |
await asyncio.gather(*work_tasks) |
| 524 |
|
| 525 |
|
| 526 |
Below, we use a future to enable custom control over when that task will be |
| 527 |
marked as done. |
| 528 |
If :meth:`future.set_result() <asyncio.Future.set_result>` (the method |
| 529 |
responsible for marking that future as done) is never called, then this task |
| 530 |
will never finish. |
| 531 |
We've also enlisted the help of another task, which we'll see in a moment, that |
| 532 |
will monitor how much time has elapsed and, accordingly, call |
| 533 |
``future.set_result()``. |
| 534 |
|
| 535 |
:: |
| 536 |
|
| 537 |
async def async_sleep(seconds: float): |
| 538 |
future = asyncio.Future() |
| 539 |
time_to_wake = time.time() + seconds |
| 540 |
# Add the watcher-task to the event loop. |
| 541 |
watcher_task = asyncio.create_task(_sleep_watcher(future, time_to_wake)) |
| 542 |
# Block until the future is marked as done. |
| 543 |
await future |
| 544 |
|
| 545 |
Below, we use a rather bare ``YieldToEventLoop()`` object to ``yield`` |
| 546 |
from its ``__await__`` method, ceding control to the event loop. |
| 547 |
This is effectively the same as calling ``asyncio.sleep(0)``, but this approach |
| 548 |
offers more clarity, not to mention it's somewhat cheating to use |
| 549 |
``asyncio.sleep`` when showcasing how to implement it! |
| 550 |
|
| 551 |
As usual, the event loop cycles through its tasks, giving them control |
| 552 |
and receiving control back when they pause or finish. |
| 553 |
The ``watcher_task``, which runs the coroutine ``_sleep_watcher(...)``, will |
| 554 |
be invoked once per full cycle of the event loop. |
| 555 |
On each resumption, it'll check the time and if not enough has elapsed, then |
| 556 |
it'll pause once again and hand control back to the event loop. |
| 557 |
Once enough time has elapsed, ``_sleep_watcher(...)`` |
| 558 |
marks the future as done and completes by exiting its |
| 559 |
infinite ``while`` loop. |
| 560 |
Given this helper task is only invoked once per cycle of the event loop, |
| 561 |
you'd be correct to note that this asynchronous sleep will sleep *at least* |
| 562 |
three seconds, rather than exactly three seconds. |
| 563 |
Note this is also true of ``asyncio.sleep``. |
| 564 |
|
| 565 |
:: |
| 566 |
|
| 567 |
class YieldToEventLoop: |
| 568 |
def __await__(self): |
| 569 |
yield |
| 570 |
|
| 571 |
async def _sleep_watcher(future, time_to_wake): |
| 572 |
while True: |
| 573 |
if time.time() >= time_to_wake: |
| 574 |
# This marks the future as done. |
| 575 |
future.set_result(None) |
| 576 |
break |
| 577 |
else: |
| 578 |
await YieldToEventLoop() |
| 579 |
|
| 580 |
Here is the full program's output: |
| 581 |
|
| 582 |
.. code-block:: none |
| 583 |
|
| 584 |
$ python custom-async-sleep.py |
| 585 |
Beginning asynchronous sleep at time: 14:52:22. |
| 586 |
I like work. Work work. |
| 587 |
I like work. Work work. |
| 588 |
I like work. Work work. |
| 589 |
Done asynchronous sleep at time: 14:52:25. |
| 590 |
|
| 591 |
You might feel this implementation of asynchronous sleep was unnecessarily |
| 592 |
convoluted. |
| 593 |
And, well, it was. |
| 594 |
The example was meant to showcase the versatility of futures with a simple |
| 595 |
example that could be mimicked for more complex needs. |
| 596 |
For reference, you could implement it without futures, like so:: |
| 597 |
|
| 598 |
async def simpler_async_sleep(seconds): |
| 599 |
time_to_wake = time.time() + seconds |
| 600 |
while True: |
| 601 |
if time.time() >= time_to_wake: |
| 602 |
return |
| 603 |
else: |
| 604 |
await YieldToEventLoop() |
| 605 |
|
| 606 |
But that's all for now. Hopefully you're ready to more confidently dive into |
| 607 |
some async programming or check out advanced topics in the |
| 608 |
:mod:`rest of the documentation <asyncio>`. |