Debian Codegraph @

Doc/library/concurrent.futures.rst

raw
1 :mod:`!concurrent.futures` --- Launching parallel tasks
2 =======================================================
3
4 .. module:: concurrent.futures
5 :synopsis: Execute computations concurrently using threads or processes.
6
7 .. versionadded:: 3.2
8
9 **Source code:** :source:`Lib/concurrent/futures/thread.py`,
10 :source:`Lib/concurrent/futures/process.py`,
11 and :source:`Lib/concurrent/futures/interpreter.py`
12
13 --------------
14
15 The :mod:`!concurrent.futures` module provides a high-level interface for
16 asynchronously executing callables.
17
18 The asynchronous execution can be performed with threads, using
19 :class:`ThreadPoolExecutor` or :class:`InterpreterPoolExecutor`,
20 or separate processes, using :class:`ProcessPoolExecutor`.
21 Each implements the same interface, which is defined
22 by the abstract :class:`Executor` class.
23
24 :class:`concurrent.futures.Future` must not be confused with
25 :class:`asyncio.Future`, which is designed for use with :mod:`asyncio`
26 tasks and coroutines. See the :doc:`asyncio's Future <asyncio-future>`
27 documentation for a detailed comparison of the two.
28
29 .. include:: ../includes/wasm-notavail.rst
30
31 Executor Objects
32 ----------------
33
34 .. class:: Executor
35
36 An abstract class that provides methods to execute calls asynchronously. It
37 should not be used directly, but through its concrete subclasses.
38
39 .. method:: submit(fn, /, *args, **kwargs)
40
41 Schedules the callable, *fn*, to be executed as ``fn(*args, **kwargs)``
42 and returns a :class:`Future` object representing the execution of the
43 callable. ::
44
45 with ThreadPoolExecutor(max_workers=1) as executor:
46 future = executor.submit(pow, 323, 1235)
47 print(future.result())
48
49 .. method:: map(fn, *iterables, timeout=None, chunksize=1, buffersize=None)
50
51 Similar to :func:`map(fn, *iterables) <map>` except:
52
53 * The *iterables* are collected immediately rather than lazily, unless a
54 *buffersize* is specified to limit the number of submitted tasks whose
55 results have not yet been yielded. If the buffer is full, iteration over
56 the *iterables* pauses until a result is yielded from the buffer.
57
58 * *fn* is executed asynchronously and several calls to
59 *fn* may be made concurrently.
60
61 The returned iterator raises a :exc:`TimeoutError`
62 if :meth:`~iterator.__next__` is called and the result isn't available
63 after *timeout* seconds from the original call to :meth:`Executor.map`.
64 *timeout* can be an int or a float. If *timeout* is not specified or
65 ``None``, there is no limit to the wait time.
66
67 If a *fn* call raises an exception, then that exception will be
68 raised when its value is retrieved from the iterator.
69
70 When using :class:`ProcessPoolExecutor`, this method chops *iterables*
71 into a number of chunks which it submits to the pool as separate
72 tasks. The (approximate) size of these chunks can be specified by
73 setting *chunksize* to a positive integer. For very long iterables,
74 using a large value for *chunksize* can significantly improve
75 performance compared to the default size of 1. With
76 :class:`ThreadPoolExecutor` and :class:`InterpreterPoolExecutor`,
77 *chunksize* has no effect.
78
79 .. versionchanged:: 3.5
80 Added the *chunksize* parameter.
81
82 .. versionchanged:: 3.14
83 Added the *buffersize* parameter.
84
85 .. method:: shutdown(wait=True, *, cancel_futures=False)
86
87 Signal the executor that it should free any resources that it is using
88 when the currently pending futures are done executing. Calls to
89 :meth:`Executor.submit` and :meth:`Executor.map` made after shutdown will
90 raise :exc:`RuntimeError`.
91
92 If *wait* is ``True`` then this method will not return until all the
93 pending futures are done executing and the resources associated with the
94 executor have been freed. If *wait* is ``False`` then this method will
95 return immediately and the resources associated with the executor will be
96 freed when all pending futures are done executing. Regardless of the
97 value of *wait*, the entire Python program will not exit until all
98 pending futures are done executing.
99
100 If *cancel_futures* is ``True``, this method will cancel all pending
101 futures that the executor has not started running. Any futures that
102 are completed or running won't be cancelled, regardless of the value
103 of *cancel_futures*.
104
105 If both *cancel_futures* and *wait* are ``True``, all futures that the
106 executor has started running will be completed prior to this method
107 returning. The remaining futures are cancelled.
108
109 You can avoid having to call this method explicitly if you use the executor
110 as a :term:`context manager` via the :keyword:`with` statement, which
111 will shutdown the :class:`Executor` (waiting as if :meth:`Executor.shutdown`
112 were called with *wait* set to ``True``)::
113
114 import shutil
115 with ThreadPoolExecutor(max_workers=4) as e:
116 e.submit(shutil.copy, 'src1.txt', 'dest1.txt')
117 e.submit(shutil.copy, 'src2.txt', 'dest2.txt')
118 e.submit(shutil.copy, 'src3.txt', 'dest3.txt')
119 e.submit(shutil.copy, 'src4.txt', 'dest4.txt')
120
121 .. versionchanged:: 3.9
122 Added *cancel_futures*.
123
124
125 ThreadPoolExecutor
126 ------------------
127
128 :class:`ThreadPoolExecutor` is an :class:`Executor` subclass that uses a pool of
129 threads to execute calls asynchronously.
130
131 Deadlocks can occur when the callable associated with a :class:`Future` waits on
132 the results of another :class:`Future`. For example::
133
134 import time
135 def wait_on_b():
136 time.sleep(5)
137 print(b.result()) # b will never complete because it is waiting on a.
138 return 5
139
140 def wait_on_a():
141 time.sleep(5)
142 print(a.result()) # a will never complete because it is waiting on b.
143 return 6
144
145
146 executor = ThreadPoolExecutor(max_workers=2)
147 a = executor.submit(wait_on_b)
148 b = executor.submit(wait_on_a)
149
150 And::
151
152 def wait_on_future():
153 f = executor.submit(pow, 5, 2)
154 # This will never complete because there is only one worker thread and
155 # it is executing this function.
156 print(f.result())
157
158 executor = ThreadPoolExecutor(max_workers=1)
159 future = executor.submit(wait_on_future)
160 # Note: calling future.result() would also cause a deadlock because
161 # the single worker thread is already waiting for wait_on_future().
162
163
164 .. class:: ThreadPoolExecutor(max_workers=None, thread_name_prefix='', initializer=None, initargs=())
165
166 An :class:`Executor` subclass that uses a pool of at most *max_workers*
167 threads to execute calls asynchronously.
168
169 All threads enqueued to ``ThreadPoolExecutor`` will be joined before the
170 interpreter can exit. Note that the exit handler which does this is
171 executed *before* any exit handlers added using ``atexit``. This means
172 exceptions in the main thread must be caught and handled in order to
173 signal threads to exit gracefully. For this reason, it is recommended
174 that ``ThreadPoolExecutor`` not be used for long-running tasks.
175
176 *initializer* is an optional callable that is called at the start of
177 each worker thread; *initargs* is a tuple of arguments passed to the
178 initializer. Should *initializer* raise an exception, all currently
179 pending jobs will raise a :exc:`~concurrent.futures.thread.BrokenThreadPool`,
180 as well as any attempt to submit more jobs to the pool.
181
182 .. versionchanged:: 3.5
183 If *max_workers* is ``None`` or
184 not given, it will default to the number of processors on the machine,
185 multiplied by ``5``, assuming that :class:`ThreadPoolExecutor` is often
186 used to overlap I/O instead of CPU work and the number of workers
187 should be higher than the number of workers
188 for :class:`ProcessPoolExecutor`.
189
190 .. versionchanged:: 3.6
191 Added the *thread_name_prefix* parameter to allow users to
192 control the :class:`threading.Thread` names for worker threads created by
193 the pool for easier debugging.
194
195 .. versionchanged:: 3.7
196 Added the *initializer* and *initargs* arguments.
197
198 .. versionchanged:: 3.8
199 Default value of *max_workers* is changed to ``min(32, os.cpu_count() + 4)``.
200 This default value preserves at least 5 workers for I/O bound tasks.
201 It utilizes at most 32 CPU cores for CPU bound tasks which release the GIL.
202 And it avoids using very large resources implicitly on many-core machines.
203
204 ThreadPoolExecutor now reuses idle worker threads before starting
205 *max_workers* worker threads too.
206
207 .. versionchanged:: 3.13
208 Default value of *max_workers* is changed to
209 ``min(32, (os.process_cpu_count() or 1) + 4)``.
210
211
212 .. _threadpoolexecutor-example:
213
214 ThreadPoolExecutor Example
215 ~~~~~~~~~~~~~~~~~~~~~~~~~~
216 ::
217
218 import concurrent.futures
219 import urllib.request
220
221 URLS = ['http://www.foxnews.com/',
222 'http://www.cnn.com/',
223 'http://europe.wsj.com/',
224 'http://www.bbc.co.uk/',
225 'http://nonexistent-subdomain.python.org/']
226
227 # Retrieve a single page and report the URL and contents
228 def load_url(url, timeout):
229 with urllib.request.urlopen(url, timeout=timeout) as conn:
230 return conn.read()
231
232 # We can use a with statement to ensure threads are cleaned up promptly
233 with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
234 # Start the load operations and mark each future with its URL
235 future_to_url = {executor.submit(load_url, url, 60): url for url in URLS}
236 for future in concurrent.futures.as_completed(future_to_url):
237 url = future_to_url[future]
238 try:
239 data = future.result()
240 except Exception as exc:
241 print('%r generated an exception: %s' % (url, exc))
242 else:
243 print('%r page is %d bytes' % (url, len(data)))
244
245
246 InterpreterPoolExecutor
247 -----------------------
248
249 .. versionadded:: 3.14
250
251 The :class:`InterpreterPoolExecutor` class uses a pool of interpreters
252 to execute calls asynchronously. It is a :class:`ThreadPoolExecutor`
253 subclass, which means each worker is running in its own thread.
254 The difference here is that each worker has its own interpreter,
255 and runs each task using that interpreter.
256
257 The biggest benefit to using interpreters instead of only threads
258 is true multi-core parallelism. Each interpreter has its own
259 :term:`Global Interpreter Lock <global interpreter lock>`, so code
260 running in one interpreter can run on one CPU core, while code in
261 another interpreter runs unblocked on a different core.
262
263 The tradeoff is that writing concurrent code for use with multiple
264 interpreters can take extra effort. However, this is because it
265 forces you to be deliberate about how and when interpreters interact,
266 and to be explicit about what data is shared between interpreters.
267 This results in several benefits that help balance the extra effort,
268 including true multi-core parallelism, For example, code written
269 this way can make it easier to reason about concurrency. Another
270 major benefit is that you don't have to deal with several of the
271 big pain points of using threads, like race conditions.
272
273 Each worker's interpreter is isolated from all the other interpreters.
274 "Isolated" means each interpreter has its own runtime state and
275 operates completely independently. For example, if you redirect
276 :data:`sys.stdout` in one interpreter, it will not be automatically
277 redirected to any other interpreter. If you import a module in one
278 interpreter, it is not automatically imported in any other. You
279 would need to import the module separately in interpreter where
280 you need it. In fact, each module imported in an interpreter is
281 a completely separate object from the same module in a different
282 interpreter, including :mod:`sys`, :mod:`builtins`,
283 and even ``__main__``.
284
285 Isolation means a mutable object, or other data, cannot be used
286 by more than one interpreter at the same time. That effectively means
287 interpreters cannot actually share such objects or data. Instead,
288 each interpreter must have its own copy, and you will have to
289 synchronize any changes between the copies manually. Immutable
290 objects and data, like the builtin singletons, strings, and tuples
291 of immutable objects, don't have these limitations.
292
293 Communicating and synchronizing between interpreters is most effectively
294 done using dedicated tools, like those proposed in :pep:`734`. One less
295 efficient alternative is to serialize with :mod:`pickle` and then send
296 the bytes over a shared :mod:`socket <socket>` or
297 :func:`pipe <os.pipe>`.
298
299 .. class:: InterpreterPoolExecutor(max_workers=None, thread_name_prefix='', initializer=None, initargs=())
300
301 A :class:`ThreadPoolExecutor` subclass that executes calls asynchronously
302 using a pool of at most *max_workers* threads. Each thread runs
303 tasks in its own interpreter. The worker interpreters are isolated
304 from each other, which means each has its own runtime state and that
305 they can't share any mutable objects or other data. Each interpreter
306 has its own :term:`Global Interpreter Lock <global interpreter lock>`,
307 which means code run with this executor has true multi-core parallelism.
308
309 The optional *initializer* and *initargs* arguments have the same
310 meaning as for :class:`!ThreadPoolExecutor`: the initializer is run
311 when each worker is created, though in this case it is run in
312 the worker's interpreter. The executor serializes the *initializer*
313 and *initargs* using :mod:`pickle` when sending them to the worker's
314 interpreter.
315
316 .. note::
317 The executor may replace uncaught exceptions from *initializer*
318 with :class:`~concurrent.interpreters.ExecutionFailed`.
319
320 Other caveats from parent :class:`ThreadPoolExecutor` apply here.
321
322 :meth:`~Executor.submit` and :meth:`~Executor.map` work like normal,
323 except the worker serializes the callable and arguments using
324 :mod:`pickle` when sending them to its interpreter. The worker
325 likewise serializes the return value when sending it back.
326
327 When a worker's current task raises an uncaught exception, the worker
328 always tries to preserve the exception as-is. If that is successful
329 then it also sets the ``__cause__`` to a corresponding
330 :class:`~concurrent.interpreters.ExecutionFailed`
331 instance, which contains a summary of the original exception.
332 In the uncommon case that the worker is not able to preserve the
333 original as-is then it directly preserves the corresponding
334 :class:`~concurrent.interpreters.ExecutionFailed`
335 instance instead.
336
337
338 ProcessPoolExecutor
339 -------------------
340
341 The :class:`ProcessPoolExecutor` class is an :class:`Executor` subclass that
342 uses a pool of processes to execute calls asynchronously.
343 :class:`ProcessPoolExecutor` uses the :mod:`multiprocessing` module, which
344 allows it to side-step the :term:`Global Interpreter Lock
345 <global interpreter lock>` but also means that
346 only picklable objects can be executed and returned.
347
348 The ``__main__`` module must be importable by worker subprocesses. This means
349 that :class:`ProcessPoolExecutor` will not work in the interactive interpreter.
350
351 Calling :class:`Executor` or :class:`Future` methods from a callable submitted
352 to a :class:`ProcessPoolExecutor` will result in deadlock.
353
354 Note that the restrictions on functions and arguments needing to picklable as
355 per :class:`multiprocessing.Process` apply when using :meth:`~Executor.submit`
356 and :meth:`~Executor.map` on a :class:`ProcessPoolExecutor`. A function defined
357 in a REPL or a lambda should not be expected to work.
358
359 .. class:: ProcessPoolExecutor(max_workers=None, mp_context=None, initializer=None, initargs=(), max_tasks_per_child=None)
360
361 An :class:`Executor` subclass that executes calls asynchronously using a pool
362 of at most *max_workers* processes. If *max_workers* is ``None`` or not
363 given, it will default to :func:`os.process_cpu_count`.
364 If *max_workers* is less than or equal to ``0``, then a :exc:`ValueError`
365 will be raised.
366 On Windows, *max_workers* must be less than or equal to ``61``. If it is not
367 then :exc:`ValueError` will be raised. If *max_workers* is ``None``, then
368 the default chosen will be at most ``61``, even if more processors are
369 available.
370 *mp_context* can be a :mod:`multiprocessing` context or ``None``. It will be
371 used to launch the workers. If *mp_context* is ``None`` or not given, the
372 default :mod:`multiprocessing` context is used.
373 See :ref:`multiprocessing-start-methods`.
374
375 *initializer* is an optional callable that is called at the start of
376 each worker process; *initargs* is a tuple of arguments passed to the
377 initializer. Should *initializer* raise an exception, all currently
378 pending jobs will raise a :exc:`~concurrent.futures.process.BrokenProcessPool`,
379 as well as any attempt to submit more jobs to the pool.
380
381 *max_tasks_per_child* is an optional argument that specifies the maximum
382 number of tasks a single process can execute before it will exit and be
383 replaced with a fresh worker process. By default *max_tasks_per_child* is
384 ``None`` which means worker processes will live as long as the pool. When
385 a max is specified, the "spawn" multiprocessing start method will be used by
386 default in absence of a *mp_context* parameter. This feature is incompatible
387 with the "fork" start method.
388
389 .. note::
390 Bugs have been reported when using the *max_tasks_per_child* feature that
391 can result in the :class:`ProcessPoolExecutor` hanging in some
392 circumstances. Follow its eventual resolution in :gh:`115634`.
393
394 .. versionchanged:: 3.3
395 When one of the worker processes terminates abruptly, a
396 :exc:`~concurrent.futures.process.BrokenProcessPool` error is now raised.
397 Previously, behaviour
398 was undefined but operations on the executor or its futures would often
399 freeze or deadlock.
400
401 .. versionchanged:: 3.7
402 The *mp_context* argument was added to allow users to control the
403 start_method for worker processes created by the pool.
404
405 Added the *initializer* and *initargs* arguments.
406
407 .. versionchanged:: 3.11
408 The *max_tasks_per_child* argument was added to allow users to
409 control the lifetime of workers in the pool.
410
411 .. versionchanged:: 3.12
412 On POSIX systems, if your application has multiple threads and the
413 :mod:`multiprocessing` context uses the ``"fork"`` start method:
414 The :func:`os.fork` function called internally to spawn workers may raise a
415 :exc:`DeprecationWarning`. Pass a *mp_context* configured to use a
416 different start method. See the :func:`os.fork` documentation for
417 further explanation.
418
419 .. versionchanged:: 3.13
420 *max_workers* uses :func:`os.process_cpu_count` by default, instead of
421 :func:`os.cpu_count`.
422
423 .. versionchanged:: 3.14
424 The default process start method (see
425 :ref:`multiprocessing-start-methods`) changed away from *fork*. If you
426 require the *fork* start method for :class:`ProcessPoolExecutor` you must
427 explicitly pass ``mp_context=multiprocessing.get_context("fork")``.
428
429 .. method:: terminate_workers()
430
431 Attempt to terminate all living worker processes immediately by calling
432 :meth:`Process.terminate <multiprocessing.Process.terminate>` on each of them.
433 Internally, it will also call :meth:`Executor.shutdown` to ensure that all
434 other resources associated with the executor are freed.
435
436 After calling this method the caller should no longer submit tasks to the
437 executor.
438
439 .. versionadded:: 3.14
440
441 .. method:: kill_workers()
442
443 Attempt to kill all living worker processes immediately by calling
444 :meth:`Process.kill <multiprocessing.Process.kill>` on each of them.
445 Internally, it will also call :meth:`Executor.shutdown` to ensure that all
446 other resources associated with the executor are freed.
447
448 After calling this method the caller should no longer submit tasks to the
449 executor.
450
451 .. versionadded:: 3.14
452
453 .. _processpoolexecutor-example:
454
455 ProcessPoolExecutor Example
456 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
457 ::
458
459 import concurrent.futures
460 import math
461
462 PRIMES = [
463 112272535095293,
464 112582705942171,
465 112272535095293,
466 115280095190773,
467 115797848077099,
468 1099726899285419]
469
470 def is_prime(n):
471 if n < 2:
472 return False
473 if n == 2:
474 return True
475 if n % 2 == 0:
476 return False
477
478 sqrt_n = int(math.floor(math.sqrt(n)))
479 for i in range(3, sqrt_n + 1, 2):
480 if n % i == 0:
481 return False
482 return True
483
484 def main():
485 with concurrent.futures.ProcessPoolExecutor() as executor:
486 for number, prime in zip(PRIMES, executor.map(is_prime, PRIMES)):
487 print('%d is prime: %s' % (number, prime))
488
489 if __name__ == '__main__':
490 main()
491
492
493 Future Objects
494 --------------
495
496 The :class:`Future` class encapsulates the asynchronous execution of a callable.
497 :class:`Future` instances are created by :meth:`Executor.submit`.
498
499 .. class:: Future
500
501 Encapsulates the asynchronous execution of a callable. :class:`Future`
502 instances are created by :meth:`Executor.submit` and should not be created
503 directly except for testing.
504
505 .. method:: cancel()
506
507 Attempt to cancel the call. If the call is currently being executed or
508 finished running and cannot be cancelled then the method will return
509 ``False``, otherwise the call will be cancelled and the method will
510 return ``True``.
511
512 .. method:: cancelled()
513
514 Return ``True`` if the call was successfully cancelled.
515
516 .. method:: running()
517
518 Return ``True`` if the call is currently being executed and cannot be
519 cancelled.
520
521 .. method:: done()
522
523 Return ``True`` if the call was successfully cancelled or finished
524 running.
525
526 .. method:: result(timeout=None)
527
528 Return the value returned by the call. If the call hasn't yet completed
529 then this method will wait up to *timeout* seconds. If the call hasn't
530 completed in *timeout* seconds, then a
531 :exc:`TimeoutError` will be raised. *timeout* can be
532 an int or float. If *timeout* is not specified or ``None``, there is no
533 limit to the wait time.
534
535 If the future is cancelled before completing then :exc:`.CancelledError`
536 will be raised.
537
538 If the call raised an exception, this method will raise the same exception.
539
540 .. method:: exception(timeout=None)
541
542 Return the exception raised by the call. If the call hasn't yet
543 completed then this method will wait up to *timeout* seconds. If the
544 call hasn't completed in *timeout* seconds, then a
545 :exc:`TimeoutError` will be raised. *timeout* can be
546 an int or float. If *timeout* is not specified or ``None``, there is no
547 limit to the wait time.
548
549 If the future is cancelled before completing then :exc:`.CancelledError`
550 will be raised.
551
552 If the call completed without raising, ``None`` is returned.
553
554 .. method:: add_done_callback(fn)
555
556 Attaches the callable *fn* to the future. *fn* will be called, with the
557 future as its only argument, when the future is cancelled or finishes
558 running.
559
560 Added callables are called in the order that they were added and are
561 always called in a thread belonging to the process that added them. If
562 the callable raises an :exc:`Exception` subclass, it will be logged and
563 ignored. If the callable raises a :exc:`BaseException` subclass, the
564 behavior is undefined.
565
566 If the future has already completed or been cancelled, *fn* will be
567 called immediately.
568
569 The following :class:`Future` methods are meant for use in unit tests and
570 :class:`Executor` implementations.
571
572 .. method:: set_running_or_notify_cancel()
573
574 This method should only be called by :class:`Executor` implementations
575 before executing the work associated with the :class:`Future` and by unit
576 tests.
577
578 If the method returns ``False`` then the :class:`Future` was cancelled,
579 i.e. :meth:`Future.cancel` was called and returned ``True``. Any threads
580 waiting on the :class:`Future` completing (i.e. through
581 :func:`as_completed` or :func:`wait`) will be woken up.
582
583 If the method returns ``True`` then the :class:`Future` was not cancelled
584 and has been put in the running state, i.e. calls to
585 :meth:`Future.running` will return ``True``.
586
587 This method can only be called once and cannot be called after
588 :meth:`Future.set_result` or :meth:`Future.set_exception` have been
589 called.
590
591 .. method:: set_result(result)
592
593 Sets the result of the work associated with the :class:`Future` to
594 *result*.
595
596 This method should only be used by :class:`Executor` implementations and
597 unit tests.
598
599 .. versionchanged:: 3.8
600 This method raises
601 :exc:`concurrent.futures.InvalidStateError` if the :class:`Future` is
602 already done.
603
604 .. method:: set_exception(exception)
605
606 Sets the result of the work associated with the :class:`Future` to the
607 :class:`Exception` *exception*.
608
609 This method should only be used by :class:`Executor` implementations and
610 unit tests.
611
612 .. versionchanged:: 3.8
613 This method raises
614 :exc:`concurrent.futures.InvalidStateError` if the :class:`Future` is
615 already done.
616
617 Module Functions
618 ----------------
619
620 .. function:: wait(fs, timeout=None, return_when=ALL_COMPLETED)
621
622 Wait for the :class:`Future` instances (possibly created by different
623 :class:`Executor` instances) given by *fs* to complete. Duplicate futures
624 given to *fs* are removed and will be returned only once. Returns a named
625 2-tuple of sets. The first set, named ``done``, contains the futures that
626 completed (finished or cancelled futures) before the wait completed. The
627 second set, named ``not_done``, contains the futures that did not complete
628 (pending or running futures).
629
630 *timeout* can be used to control the maximum number of seconds to wait before
631 returning. *timeout* can be an int or float. If *timeout* is not specified
632 or ``None``, there is no limit to the wait time.
633
634 *return_when* indicates when this function should return. It must be one of
635 the following constants:
636
637 .. list-table::
638 :header-rows: 1
639
640 * - Constant
641 - Description
642
643 * - .. data:: FIRST_COMPLETED
644 - The function will return when any future finishes or is cancelled.
645
646 * - .. data:: FIRST_EXCEPTION
647 - The function will return when any future finishes by raising an
648 exception. If no future raises an exception
649 then it is equivalent to :const:`ALL_COMPLETED`.
650
651 * - .. data:: ALL_COMPLETED
652 - The function will return when all futures finish or are cancelled.
653
654 .. function:: as_completed(fs, timeout=None)
655
656 Returns an iterator over the :class:`Future` instances (possibly created by
657 different :class:`Executor` instances) given by *fs* that yields futures as
658 they complete (finished or cancelled futures). Any futures given by *fs* that
659 are duplicated will be returned once. Any futures that completed before
660 :func:`as_completed` is called will be yielded first. The returned iterator
661 raises a :exc:`TimeoutError` if :meth:`~iterator.__next__`
662 is called and the result isn't available after *timeout* seconds from the
663 original call to :func:`as_completed`. *timeout* can be an int or float. If
664 *timeout* is not specified or ``None``, there is no limit to the wait time.
665
666
667 .. seealso::
668
669 :pep:`3148` -- futures - execute computations asynchronously
670 The proposal which described this feature for inclusion in the Python
671 standard library.
672
673
674 Exception classes
675 -----------------
676
677 .. currentmodule:: concurrent.futures
678
679 .. exception:: CancelledError
680
681 Raised when a future is cancelled.
682
683 .. exception:: TimeoutError
684
685 A deprecated alias of :exc:`TimeoutError`,
686 raised when a future operation exceeds the given timeout.
687
688 .. versionchanged:: 3.11
689
690 This class was made an alias of :exc:`TimeoutError`.
691
692
693 .. exception:: BrokenExecutor
694
695 Derived from :exc:`RuntimeError`, this exception class is raised
696 when an executor is broken for some reason, and cannot be used
697 to submit or execute new tasks.
698
699 .. versionadded:: 3.7
700
701 .. exception:: InvalidStateError
702
703 Raised when an operation is performed on a future that is not allowed
704 in the current state.
705
706 .. versionadded:: 3.8
707
708 .. currentmodule:: concurrent.futures.thread
709
710 .. exception:: BrokenThreadPool
711
712 Derived from :exc:`~concurrent.futures.BrokenExecutor`, this exception
713 class is raised when one of the workers
714 of a :class:`~concurrent.futures.ThreadPoolExecutor`
715 has failed initializing.
716
717 .. versionadded:: 3.7
718
719 .. currentmodule:: concurrent.futures.interpreter
720
721 .. exception:: BrokenInterpreterPool
722
723 Derived from :exc:`~concurrent.futures.thread.BrokenThreadPool`,
724 this exception class is raised when one of the workers
725 of a :class:`~concurrent.futures.InterpreterPoolExecutor`
726 has failed initializing.
727
728 .. versionadded:: 3.14
729
730 .. currentmodule:: concurrent.futures.process
731
732 .. exception:: BrokenProcessPool
733
734 Derived from :exc:`~concurrent.futures.BrokenExecutor` (formerly
735 :exc:`RuntimeError`), this exception class is raised when one of the
736 workers of a :class:`~concurrent.futures.ProcessPoolExecutor`
737 has terminated in a non-clean
738 fashion (for example, if it was killed from the outside).
739
740 .. versionadded:: 3.3