Debian Codegraph @

Doc/howto/a-conceptual-overview-of-asyncio.rst

raw
1 .. _a-conceptual-overview-of-asyncio:
2
3 ****************************************
4 A Conceptual Overview of :mod:`!asyncio`
5 ****************************************
6
7 This :ref:`HOWTO <how-tos>` article seeks to help you build a sturdy mental
8 model of how :mod:`asyncio` fundamentally works, helping you understand the
9 how and why behind the recommended patterns.
10
11 You might be curious about some key :mod:`!asyncio` concepts.
12 By the end of this article, you'll be able to comfortably answer these questions:
13
14 - What's happening behind the scenes when an object is awaited?
15 - How does :mod:`!asyncio` differentiate between a task which doesn't need
16 CPU time (such as a network request or file read) as opposed to a task that
17 does (such as computing n-factorial)?
18 - How to write an asynchronous variant of an operation, such as
19 an async sleep or database request.
20
21 .. seealso::
22
23 * The `guide <https://github.com/anordin95/a-conceptual-overview-of-asyncio/
24 tree/main>`_ that inspired this HOWTO article, by Alexander Nordin.
25 * This in-depth `YouTube tutorial series <https://www.youtube.com/
26 watch?v=Xbl7XjFYsN4&list=PLhNSoGM2ik6SIkVGXWBwerucXjgP1rHmB>`_ on
27 ``asyncio`` created by Python core team member, Ɓukasz Langa.
28 * `500 Lines or Less: A Web Crawler With asyncio Coroutines <https://
29 aosabook.org/en/500L/a-web-crawler-with-asyncio-coroutines.html>`_ by A.
30 Jesse Jiryu Davis and Guido van Rossum.
31
32 --------------------------------------------
33 A conceptual overview part 1: the high-level
34 --------------------------------------------
35
36 In part 1, we'll cover the main, high-level building blocks of :mod:`!asyncio`:
37 the event loop, coroutine functions, coroutine objects, tasks, and ``await``.
38
39 ==========
40 Event Loop
41 ==========
42
43 Everything in :mod:`!asyncio` happens relative to the event loop.
44 It's the star of the show.
45 It's like an orchestra conductor.
46 It's behind the scenes managing resources.
47 Some power is explicitly granted to it, but a lot of its ability to get things
48 done comes from the respect and cooperation of its worker bees.
49
50 In more technical terms, the event loop contains a collection of jobs to be run.
51 Some jobs are added directly by you, and some indirectly by :mod:`!asyncio`.
52 The event loop takes a job from its backlog of work and invokes it (or "gives
53 it control"), similar to calling a function, and then that job runs.
54 Once it pauses or completes, it returns control to the event loop.
55 The event loop will then select another job from its pool and invoke it.
56 You can *roughly* think of the collection of jobs as a queue: jobs are added and
57 then processed one at a time, generally (but not always) in order.
58 This process repeats indefinitely, with the event loop cycling endlessly
59 onwards.
60 If there are no more jobs pending execution, the event loop is smart enough to
61 rest and avoid needlessly wasting CPU cycles, and will come back when there's
62 more work to be done.
63
64 Effective execution relies on jobs sharing well and cooperating; a greedy job
65 could hog control and leave the other jobs to starve, rendering the overall
66 event loop approach rather useless.
67
68 ::
69
70 import asyncio
71
72 # This creates an event loop and indefinitely cycles through
73 # its collection of jobs.
74 event_loop = asyncio.new_event_loop()
75 event_loop.run_forever()
76
77 =====================================
78 Asynchronous functions and coroutines
79 =====================================
80
81 This is a basic, boring Python function::
82
83 def hello_printer():
84 print(
85 "Hi, I am a lowly, simple printer, though I have all I "
86 "need in life -- \nfresh paper and my dearly beloved octopus "
87 "partner in crime."
88 )
89
90 Calling a regular function invokes its logic or body::
91
92 >>> hello_printer()
93 Hi, I am a lowly, simple printer, though I have all I need in life --
94 fresh paper and my dearly beloved octopus partner in crime.
95
96 The :ref:`async def <async def>`, as opposed to just a plain ``def``, makes
97 this an asynchronous function (or "coroutine function").
98 Calling it creates and returns a :ref:`coroutine <coroutine>` object.
99
100 ::
101
102 async def loudmouth_penguin(magic_number: int):
103 print(
104 "I am a super special talking penguin. Far cooler than that printer. "
105 f"By the way, my lucky number is: {magic_number}."
106 )
107
108 Calling the async function, ``loudmouth_penguin``, does not execute the print statement;
109 instead, it creates a coroutine object::
110
111 >>> loudmouth_penguin(magic_number=3)
112 <coroutine object loudmouth_penguin at 0x104ed2740>
113
114 The terms "coroutine function" and "coroutine object" are often conflated
115 as coroutine.
116 That can be confusing!
117 In this article, coroutine specifically refers to a coroutine object, or more
118 precisely, an instance of :data:`types.CoroutineType` (native coroutine).
119 Note that coroutines can also exist as instances of
120 :class:`collections.abc.Coroutine` -- a distinction that matters for type
121 checking.
122
123 A coroutine represents the function's body or logic.
124 A coroutine has to be explicitly started; again, merely creating the coroutine
125 does not start it.
126 Notably, the coroutine can be paused and resumed at various points within the
127 function's body.
128 That pausing and resuming ability is what allows for asynchronous behavior!
129
130 Coroutines and coroutine functions were built by leveraging the functionality
131 of :term:`generators <generator iterator>` and
132 :term:`generator functions <generator>`.
133 Recall, a generator function is a function that :keyword:`yield`\s, like this
134 one::
135
136 def get_random_number():
137 # This would be a bad random number generator!
138 print("Hi")
139 yield 1
140 print("Hello")
141 yield 7
142 print("Howdy")
143 yield 4
144 ...
145
146 Similar to a coroutine function, calling a generator function does not run it.
147 Instead, it creates a generator object::
148
149 >>> get_random_number()
150 <generator object get_random_number at 0x1048671c0>
151
152 You can proceed to the next ``yield`` of a generator by using the
153 built-in function :func:`next`.
154 In other words, the generator runs, then pauses.
155 For example::
156
157 >>> generator = get_random_number()
158 >>> next(generator)
159 Hi
160 1
161 >>> next(generator)
162 Hello
163 7
164
165 =====
166 Tasks
167 =====
168
169 Roughly speaking, :ref:`tasks <asyncio-task-obj>` are coroutines (not coroutine
170 functions) tied to an event loop.
171 A task also maintains a list of callback functions whose importance will become
172 clear in a moment when we discuss :keyword:`await`.
173 The recommended way to create tasks is via :func:`asyncio.create_task`.
174
175 Creating a task automatically schedules it for execution (by adding a
176 callback to run it in the event loop's to-do list, that is, collection of jobs).
177
178 :mod:`!asyncio` automatically associates tasks with the event loop for you.
179 This automatic association was purposely designed into :mod:`!asyncio` for
180 the sake of simplicity.
181 Without it, you'd have to keep track of the event loop object and pass it to
182 any coroutine function that wants to create tasks, adding redundant clutter
183 to your code.
184
185 ::
186
187 coroutine = loudmouth_penguin(magic_number=5)
188 # This creates a Task object and schedules its execution via the event loop.
189 task = asyncio.create_task(coroutine)
190
191 Earlier, we manually created the event loop and set it to run forever.
192 In practice, it's recommended to use (and common to see) :func:`asyncio.run`,
193 which takes care of managing the event loop and ensuring the provided
194 coroutine finishes before advancing.
195 For example, many async programs follow this setup::
196
197 import asyncio
198
199 async def main():
200 # Perform all sorts of wacky, wild asynchronous things...
201 ...
202
203 if __name__ == "__main__":
204 asyncio.run(main())
205 # The program will not reach the following print statement until the
206 # coroutine main() finishes.
207 print("coroutine main() is done!")
208
209 It's important to be aware that the task itself is not added to the event loop,
210 only a callback to the task is.
211 This matters if the task object you created is garbage collected before it's
212 called by the event loop.
213 For example, consider this program:
214
215 .. code-block::
216 :linenos:
217
218 async def hello():
219 print("hello!")
220
221 async def main():
222 asyncio.create_task(hello())
223 # Other asynchronous instructions which run for a while
224 # and cede control to the event loop...
225 ...
226
227 asyncio.run(main())
228
229 Because there's no reference to the task object created on line 5, it *might*
230 be garbage collected before the event loop invokes it.
231 Later instructions in the coroutine ``main()`` hand control back to the event
232 loop so it can invoke other jobs.
233 When the event loop eventually tries to run the task, it might fail and
234 discover the task object does not exist!
235 This can also happen even if a coroutine keeps a reference to a task but
236 completes before that task finishes.
237 When the coroutine exits, local variables go out of scope and may be subject
238 to garbage collection.
239 In practice, ``asyncio`` and Python's garbage collector work pretty hard to
240 ensure this sort of thing doesn't happen.
241 But that's no reason to be reckless!
242
243 =====
244 await
245 =====
246
247 :keyword:`await` is a Python keyword that's commonly used in one of two
248 different ways::
249
250 await task
251 await coroutine
252
253 In a crucial way, the behavior of ``await`` depends on the type of object
254 being awaited.
255
256 Awaiting a task will cede control from the current task or coroutine to
257 the event loop.
258 In the process of relinquishing control, a few important things happen.
259 We'll use the following code example to illustrate::
260
261 async def plant_a_tree():
262 dig_the_hole_task = asyncio.create_task(dig_the_hole())
263 await dig_the_hole_task
264
265 # Other instructions associated with planting a tree.
266 ...
267
268 In this example, imagine the event loop has passed control to the start of the
269 coroutine ``plant_a_tree()``.
270 As seen above, the coroutine creates a task and then awaits it.
271 The ``await dig_the_hole_task`` instruction adds a callback (which will resume
272 ``plant_a_tree()``) to the ``dig_the_hole_task`` object's list of callbacks.
273 And then, the instruction cedes control to the event loop.
274 Some time later, the event loop will pass control to ``dig_the_hole_task``
275 and the task will finish whatever it needs to do.
276 Once the task finishes, it will add its various callbacks to the event loop,
277 in this case, a call to resume ``plant_a_tree()``.
278
279 Generally speaking, when the awaited task finishes (``dig_the_hole_task``),
280 the original task or coroutine (``plant_a_tree()``) is added back to the event
281 loop's to-do list to be resumed.
282
283 This is a basic, yet reliable mental model.
284 In practice, the control handoffs are slightly more complex, but not by much.
285 In part 2, we'll walk through the details that make this possible.
286
287 **Unlike tasks, awaiting a coroutine does not hand control back to the event
288 loop!**
289 Wrapping a coroutine in a task first, then awaiting that would cede
290 control.
291 The behavior of ``await coroutine`` is effectively the same as invoking a
292 regular, synchronous Python function.
293 Consider this program::
294
295 import asyncio
296
297 async def coro_a():
298 print("I am coro_a(). Hi!")
299
300 async def coro_b():
301 print("I am coro_b(). I sure hope no one hogs the event loop...")
302
303 async def main():
304 task_b = asyncio.create_task(coro_b())
305 num_repeats = 3
306 for _ in range(num_repeats):
307 await coro_a()
308 await task_b
309
310 asyncio.run(main())
311
312 The first statement in the coroutine ``main()`` creates ``task_b`` and schedules
313 it for execution via the event loop.
314 Then, ``coro_a()`` is repeatedly awaited. Control never cedes to the
315 event loop, which is why we see the output of all three ``coro_a()``
316 invocations before ``coro_b()``'s output:
317
318 .. code-block:: none
319
320 I am coro_a(). Hi!
321 I am coro_a(). Hi!
322 I am coro_a(). Hi!
323 I am coro_b(). I sure hope no one hogs the event loop...
324
325 If we change ``await coro_a()`` to ``await asyncio.create_task(coro_a())``, the
326 behavior changes.
327 The coroutine ``main()`` cedes control to the event loop with that statement.
328 The event loop then proceeds through its backlog of work, calling ``task_b``
329 and then the task which wraps ``coro_a()`` before resuming the coroutine
330 ``main()``.
331
332 .. code-block:: none
333
334 I am coro_b(). I sure hope no one hogs the event loop...
335 I am coro_a(). Hi!
336 I am coro_a(). Hi!
337 I am coro_a(). Hi!
338
339 This behavior of ``await coroutine`` can trip a lot of people up!
340 That example highlights how using only ``await coroutine`` could
341 unintentionally hog control from other tasks and effectively stall the event
342 loop.
343 :func:`asyncio.run` can help you detect such occurrences via the
344 ``debug=True`` flag, which enables
345 :ref:`debug mode <asyncio-debug-mode>`.
346 Among other things, it will log any coroutines that monopolize execution for
347 100ms or longer.
348
349 The design intentionally trades off some conceptual clarity around usage of
350 ``await`` for improved performance.
351 Each time a task is awaited, control needs to be passed all the way up the
352 call stack to the event loop.
353 That might sound minor, but in a large program with many ``await`` statements and a deep
354 call stack, that overhead can add up to a meaningful performance drag.
355
356 ------------------------------------------------
357 A conceptual overview part 2: the nuts and bolts
358 ------------------------------------------------
359
360 Part 2 goes into detail on the mechanisms :mod:`!asyncio` uses to manage
361 control flow.
362 This is where the magic happens.
363 You'll come away from this section knowing what ``await`` does behind the scenes
364 and how to make your own asynchronous operators.
365
366 ================================
367 The inner workings of coroutines
368 ================================
369
370 :mod:`!asyncio` leverages four components to pass around control.
371
372 :meth:`coroutine.send(arg) <generator.send>` is the method used to start or
373 resume a coroutine.
374 If the coroutine was paused and is now being resumed, the argument ``arg``
375 will be sent in as the return value of the ``yield`` statement which originally
376 paused it.
377 If the coroutine is being used for the first time (as opposed to being resumed),
378 ``arg`` must be ``None``.
379
380 .. code-block::
381 :linenos:
382
383 class Rock:
384 def __await__(self):
385 value_sent_in = yield 7
386 print(f"Rock.__await__ resuming with value: {value_sent_in}.")
387 return value_sent_in
388
389 async def main():
390 print("Beginning coroutine main().")
391 rock = Rock()
392 print("Awaiting rock...")
393 value_from_rock = await rock
394 print(f"Coroutine received value: {value_from_rock} from rock.")
395 return 23
396
397 coroutine = main()
398 intermediate_result = coroutine.send(None)
399 print(f"Coroutine paused and returned intermediate value: {intermediate_result}.")
400
401 print(f"Resuming coroutine and sending in value: 42.")
402 try:
403 coroutine.send(42)
404 except StopIteration as e:
405 returned_value = e.value
406 print(f"Coroutine main() finished and provided value: {returned_value}.")
407
408 :ref:`yield <yieldexpr>`, as usual, pauses execution and returns control
409 to the caller.
410 In the example above, the ``yield``, on line 3, is called by
411 ``... = await rock`` on line 11.
412 More broadly speaking, ``await`` calls the :meth:`~object.__await__` method of
413 the given object.
414 ``await`` also does one more very special thing: it propagates (or "passes
415 along") any ``yield``\ s it receives up the call chain.
416 In this case, that's back to ``... = coroutine.send(None)`` on line 16.
417
418 The coroutine is resumed via the ``coroutine.send(42)`` call on line 21.
419 The coroutine picks back up from where it ``yield``\ ed (or paused) on line 3
420 and executes the remaining statements in its body.
421 When a coroutine finishes, it raises a :exc:`StopIteration` exception with the
422 return value attached in the :attr:`~StopIteration.value` attribute.
423
424 That snippet produces this output:
425
426 .. code-block:: none
427
428 Beginning coroutine main().
429 Awaiting rock...
430 Coroutine paused and returned intermediate value: 7.
431 Resuming coroutine and sending in value: 42.
432 Rock.__await__ resuming with value: 42.
433 Coroutine received value: 42 from rock.
434 Coroutine main() finished and provided value: 23.
435
436 It's worth pausing for a moment here and making sure you followed the various
437 ways that control flow and values were passed. A lot of important ideas were
438 covered and it's worth ensuring your understanding is firm.
439
440 The only way to yield (or effectively cede control) from a coroutine is to
441 ``await`` an object that ``yield``\ s in its ``__await__`` method.
442 That might sound odd to you. You might be thinking:
443
444 1. What about a ``yield`` directly within the coroutine function? The
445 coroutine function becomes an
446 :ref:`async generator function <asynchronous-generator-functions>`, a
447 different beast entirely.
448
449 2. What about a :ref:`yield from <yieldexpr>` within the coroutine function to a (plain)
450 generator?
451 That causes the error: ``SyntaxError: yield from not allowed in a coroutine.``
452 This was intentionally designed for the sake of simplicity -- mandating only
453 one way of using coroutines.
454 Initially ``yield`` was barred as well, but was re-accepted to allow for
455 async generators.
456 Despite that, ``yield from`` and ``await`` effectively do the same thing.
457
458 =======
459 Futures
460 =======
461
462 A :ref:`future <asyncio-future-obj>` is an object meant to represent a
463 computation's status and result.
464 The term is a nod to the idea of something still to come or not yet happened,
465 and the object is a way to keep an eye on that something.
466
467 A future has a few important attributes. One is its state, which can be either
468 "pending", "cancelled", or "done".
469 Another is its result, which is set when the state transitions to done.
470 Unlike a coroutine, a future does not represent the actual computation to be
471 done; instead, it represents the status and result of that computation, kind of
472 like a status light (red, yellow, or green) or indicator.
473
474 :class:`asyncio.Task` subclasses :class:`asyncio.Future` in order to gain
475 these various capabilities.
476 The prior section said tasks store a list of callbacks, which wasn't entirely
477 accurate.
478 It's actually the ``Future`` class that implements this logic, which ``Task``
479 inherits.
480
481 Futures may also be used directly (not via tasks).
482 Tasks mark themselves as done when their coroutine is complete.
483 Futures are much more versatile and will be marked as done when you say so.
484 In this way, they're the flexible interface for you to make your own conditions
485 for waiting and resuming.
486
487 ========================
488 A homemade asyncio.sleep
489 ========================
490
491 We'll go through an example of how you could leverage a future to create your
492 own variant of asynchronous sleep (``async_sleep``) which mimics
493 :func:`asyncio.sleep`.
494
495 This snippet registers a few tasks with the event loop and then awaits the task
496 created by ``asyncio.create_task``, which wraps the ``async_sleep(3)`` coroutine.
497 We want that task to finish only after three seconds have elapsed, but without
498 preventing other tasks from running.
499
500 ::
501
502 async def other_work():
503 print("I like work. Work work.")
504
505 async def main():
506 # Add a few other tasks to the event loop, so there's something
507 # to do while asynchronously sleeping.
508 work_tasks = [
509 asyncio.create_task(other_work()),
510 asyncio.create_task(other_work()),
511 asyncio.create_task(other_work())
512 ]
513 print(
514 "Beginning asynchronous sleep at time: "
515 f"{datetime.datetime.now().strftime("%H:%M:%S")}."
516 )
517 await asyncio.create_task(async_sleep(3))
518 print(
519 "Done asynchronous sleep at time: "
520 f"{datetime.datetime.now().strftime("%H:%M:%S")}."
521 )
522 # asyncio.gather effectively awaits each task in the collection.
523 await asyncio.gather(*work_tasks)
524
525
526 Below, we use a future to enable custom control over when that task will be
527 marked as done.
528 If :meth:`future.set_result() <asyncio.Future.set_result>` (the method
529 responsible for marking that future as done) is never called, then this task
530 will never finish.
531 We've also enlisted the help of another task, which we'll see in a moment, that
532 will monitor how much time has elapsed and, accordingly, call
533 ``future.set_result()``.
534
535 ::
536
537 async def async_sleep(seconds: float):
538 future = asyncio.Future()
539 time_to_wake = time.time() + seconds
540 # Add the watcher-task to the event loop.
541 watcher_task = asyncio.create_task(_sleep_watcher(future, time_to_wake))
542 # Block until the future is marked as done.
543 await future
544
545 Below, we use a rather bare ``YieldToEventLoop()`` object to ``yield``
546 from its ``__await__`` method, ceding control to the event loop.
547 This is effectively the same as calling ``asyncio.sleep(0)``, but this approach
548 offers more clarity, not to mention it's somewhat cheating to use
549 ``asyncio.sleep`` when showcasing how to implement it!
550
551 As usual, the event loop cycles through its tasks, giving them control
552 and receiving control back when they pause or finish.
553 The ``watcher_task``, which runs the coroutine ``_sleep_watcher(...)``, will
554 be invoked once per full cycle of the event loop.
555 On each resumption, it'll check the time and if not enough has elapsed, then
556 it'll pause once again and hand control back to the event loop.
557 Once enough time has elapsed, ``_sleep_watcher(...)``
558 marks the future as done and completes by exiting its
559 infinite ``while`` loop.
560 Given this helper task is only invoked once per cycle of the event loop,
561 you'd be correct to note that this asynchronous sleep will sleep *at least*
562 three seconds, rather than exactly three seconds.
563 Note this is also true of ``asyncio.sleep``.
564
565 ::
566
567 class YieldToEventLoop:
568 def __await__(self):
569 yield
570
571 async def _sleep_watcher(future, time_to_wake):
572 while True:
573 if time.time() >= time_to_wake:
574 # This marks the future as done.
575 future.set_result(None)
576 break
577 else:
578 await YieldToEventLoop()
579
580 Here is the full program's output:
581
582 .. code-block:: none
583
584 $ python custom-async-sleep.py
585 Beginning asynchronous sleep at time: 14:52:22.
586 I like work. Work work.
587 I like work. Work work.
588 I like work. Work work.
589 Done asynchronous sleep at time: 14:52:25.
590
591 You might feel this implementation of asynchronous sleep was unnecessarily
592 convoluted.
593 And, well, it was.
594 The example was meant to showcase the versatility of futures with a simple
595 example that could be mimicked for more complex needs.
596 For reference, you could implement it without futures, like so::
597
598 async def simpler_async_sleep(seconds):
599 time_to_wake = time.time() + seconds
600 while True:
601 if time.time() >= time_to_wake:
602 return
603 else:
604 await YieldToEventLoop()
605
606 But that's all for now. Hopefully you're ready to more confidently dive into
607 some async programming or check out advanced topics in the
608 :mod:`rest of the documentation <asyncio>`.