| 1 |
:mod:`!functools` --- Higher-order functions and operations on callable objects |
| 2 |
=============================================================================== |
| 3 |
|
| 4 |
.. module:: functools |
| 5 |
:synopsis: Higher-order functions and operations on callable objects. |
| 6 |
|
| 7 |
.. moduleauthor:: Peter Harris <[email protected]> |
| 8 |
.. moduleauthor:: Raymond Hettinger <[email protected]> |
| 9 |
.. moduleauthor:: Nick Coghlan <[email protected]> |
| 10 |
.. moduleauthor:: Ćukasz Langa <[email protected]> |
| 11 |
.. moduleauthor:: Pablo Galindo <[email protected]> |
| 12 |
.. sectionauthor:: Peter Harris <[email protected]> |
| 13 |
|
| 14 |
**Source code:** :source:`Lib/functools.py` |
| 15 |
|
| 16 |
.. testsetup:: default |
| 17 |
|
| 18 |
import functools |
| 19 |
from functools import * |
| 20 |
|
| 21 |
-------------- |
| 22 |
|
| 23 |
The :mod:`!functools` module is for higher-order functions: functions that act on |
| 24 |
or return other functions. In general, any callable object can be treated as a |
| 25 |
function for the purposes of this module. |
| 26 |
|
| 27 |
The :mod:`!functools` module defines the following functions: |
| 28 |
|
| 29 |
.. decorator:: cache(user_function) |
| 30 |
|
| 31 |
Simple lightweight unbounded function cache. Sometimes called |
| 32 |
`"memoize" <https://en.wikipedia.org/wiki/Memoization>`_. |
| 33 |
|
| 34 |
Returns the same as ``lru_cache(maxsize=None)``, creating a thin |
| 35 |
wrapper around a dictionary lookup for the function arguments. Because it |
| 36 |
never needs to evict old values, this is smaller and faster than |
| 37 |
:func:`lru_cache` with a size limit. |
| 38 |
|
| 39 |
For example:: |
| 40 |
|
| 41 |
@cache |
| 42 |
def factorial(n): |
| 43 |
return n * factorial(n-1) if n else 1 |
| 44 |
|
| 45 |
>>> factorial(10) # no previously cached result, makes 11 recursive calls |
| 46 |
3628800 |
| 47 |
>>> factorial(5) # no new calls, just returns the cached result |
| 48 |
120 |
| 49 |
>>> factorial(12) # two new recursive calls, factorial(10) is cached |
| 50 |
479001600 |
| 51 |
|
| 52 |
The cache is threadsafe so that the wrapped function can be used in |
| 53 |
multiple threads. This means that the underlying data structure will |
| 54 |
remain coherent during concurrent updates. |
| 55 |
|
| 56 |
It is possible for the wrapped function to be called more than once if |
| 57 |
another thread makes an additional call before the initial call has been |
| 58 |
completed and cached. |
| 59 |
|
| 60 |
.. versionadded:: 3.9 |
| 61 |
|
| 62 |
|
| 63 |
.. decorator:: cached_property(func) |
| 64 |
|
| 65 |
Transform a method of a class into a property whose value is computed once |
| 66 |
and then cached as a normal attribute for the life of the instance. Similar |
| 67 |
to :func:`property`, with the addition of caching. Useful for expensive |
| 68 |
computed properties of instances that are otherwise effectively immutable. |
| 69 |
|
| 70 |
Example:: |
| 71 |
|
| 72 |
class DataSet: |
| 73 |
|
| 74 |
def __init__(self, sequence_of_numbers): |
| 75 |
self._data = tuple(sequence_of_numbers) |
| 76 |
|
| 77 |
@cached_property |
| 78 |
def stdev(self): |
| 79 |
return statistics.stdev(self._data) |
| 80 |
|
| 81 |
The mechanics of :func:`cached_property` are somewhat different from |
| 82 |
:func:`property`. A regular property blocks attribute writes unless a |
| 83 |
setter is defined. In contrast, a *cached_property* allows writes. |
| 84 |
|
| 85 |
The *cached_property* decorator only runs on lookups and only when an |
| 86 |
attribute of the same name doesn't exist. When it does run, the |
| 87 |
*cached_property* writes to the attribute with the same name. Subsequent |
| 88 |
attribute reads and writes take precedence over the *cached_property* |
| 89 |
method and it works like a normal attribute. |
| 90 |
|
| 91 |
The cached value can be cleared by deleting the attribute. This |
| 92 |
allows the *cached_property* method to run again. |
| 93 |
|
| 94 |
The *cached_property* does not prevent a possible race condition in |
| 95 |
multi-threaded usage. The getter function could run more than once on the |
| 96 |
same instance, with the latest run setting the cached value. If the cached |
| 97 |
property is idempotent or otherwise not harmful to run more than once on an |
| 98 |
instance, this is fine. If synchronization is needed, implement the necessary |
| 99 |
locking inside the decorated getter function or around the cached property |
| 100 |
access. |
| 101 |
|
| 102 |
Note, this decorator interferes with the operation of :pep:`412` |
| 103 |
key-sharing dictionaries. This means that instance dictionaries |
| 104 |
can take more space than usual. |
| 105 |
|
| 106 |
Also, this decorator requires that the ``__dict__`` attribute on each instance |
| 107 |
be a mutable mapping. This means it will not work with some types, such as |
| 108 |
metaclasses (since the ``__dict__`` attributes on type instances are |
| 109 |
read-only proxies for the class namespace), and those that specify |
| 110 |
``__slots__`` without including ``__dict__`` as one of the defined slots |
| 111 |
(as such classes don't provide a ``__dict__`` attribute at all). |
| 112 |
|
| 113 |
If a mutable mapping is not available or if space-efficient key sharing is |
| 114 |
desired, an effect similar to :func:`cached_property` can also be achieved by |
| 115 |
stacking :func:`property` on top of :func:`lru_cache`. See |
| 116 |
:ref:`faq-cache-method-calls` for more details on how this differs from :func:`cached_property`. |
| 117 |
|
| 118 |
.. versionadded:: 3.8 |
| 119 |
|
| 120 |
.. versionchanged:: 3.12 |
| 121 |
Prior to Python 3.12, ``cached_property`` included an undocumented lock to |
| 122 |
ensure that in multi-threaded usage the getter function was guaranteed to |
| 123 |
run only once per instance. However, the lock was per-property, not |
| 124 |
per-instance, which could result in unacceptably high lock contention. In |
| 125 |
Python 3.12+ this locking is removed. |
| 126 |
|
| 127 |
|
| 128 |
.. function:: cmp_to_key(func) |
| 129 |
|
| 130 |
Transform an old-style comparison function to a :term:`key function`. Used |
| 131 |
with tools that accept key functions (such as :func:`sorted`, :func:`min`, |
| 132 |
:func:`max`, :func:`heapq.nlargest`, :func:`heapq.nsmallest`, |
| 133 |
:func:`itertools.groupby`). This function is primarily used as a transition |
| 134 |
tool for programs being converted from Python 2 which supported the use of |
| 135 |
comparison functions. |
| 136 |
|
| 137 |
A comparison function is any callable that accepts two arguments, compares them, |
| 138 |
and returns a negative number for less-than, zero for equality, or a positive |
| 139 |
number for greater-than. A key function is a callable that accepts one |
| 140 |
argument and returns another value to be used as the sort key. |
| 141 |
|
| 142 |
Example:: |
| 143 |
|
| 144 |
sorted(iterable, key=cmp_to_key(locale.strcoll)) # locale-aware sort order |
| 145 |
|
| 146 |
For sorting examples and a brief sorting tutorial, see :ref:`sortinghowto`. |
| 147 |
|
| 148 |
.. versionadded:: 3.2 |
| 149 |
|
| 150 |
|
| 151 |
.. decorator:: lru_cache(user_function) |
| 152 |
lru_cache(maxsize=128, typed=False) |
| 153 |
|
| 154 |
Decorator to wrap a function with a memoizing callable that saves up to the |
| 155 |
*maxsize* most recent calls. It can save time when an expensive or I/O bound |
| 156 |
function is periodically called with the same arguments. |
| 157 |
|
| 158 |
The cache is threadsafe so that the wrapped function can be used in |
| 159 |
multiple threads. This means that the underlying data structure will |
| 160 |
remain coherent during concurrent updates. |
| 161 |
|
| 162 |
It is possible for the wrapped function to be called more than once if |
| 163 |
another thread makes an additional call before the initial call has been |
| 164 |
completed and cached. |
| 165 |
|
| 166 |
Since a dictionary is used to cache results, the positional and keyword |
| 167 |
arguments to the function must be :term:`hashable`. |
| 168 |
|
| 169 |
Distinct argument patterns may be considered to be distinct calls with |
| 170 |
separate cache entries. For example, ``f(a=1, b=2)`` and ``f(b=2, a=1)`` |
| 171 |
differ in their keyword argument order and may have two separate cache |
| 172 |
entries. |
| 173 |
|
| 174 |
If *user_function* is specified, it must be a callable. This allows the |
| 175 |
*lru_cache* decorator to be applied directly to a user function, leaving |
| 176 |
the *maxsize* at its default value of 128:: |
| 177 |
|
| 178 |
@lru_cache |
| 179 |
def count_vowels(sentence): |
| 180 |
return sum(sentence.count(vowel) for vowel in 'AEIOUaeiou') |
| 181 |
|
| 182 |
If *maxsize* is set to ``None``, the LRU feature is disabled and the cache can |
| 183 |
grow without bound. |
| 184 |
|
| 185 |
If *typed* is set to true, function arguments of different types will be |
| 186 |
cached separately. If *typed* is false, the implementation will usually |
| 187 |
regard them as equivalent calls and only cache a single result. (Some |
| 188 |
types such as *str* and *int* may be cached separately even when *typed* |
| 189 |
is false.) |
| 190 |
|
| 191 |
Note, type specificity applies only to the function's immediate arguments |
| 192 |
rather than their contents. The scalar arguments, ``Decimal(42)`` and |
| 193 |
``Fraction(42)`` are treated as distinct calls with distinct results. |
| 194 |
In contrast, the tuple arguments ``('answer', Decimal(42))`` and |
| 195 |
``('answer', Fraction(42))`` are treated as equivalent. |
| 196 |
|
| 197 |
The wrapped function is instrumented with a :func:`!cache_parameters` |
| 198 |
function that returns a new :class:`dict` showing the values for *maxsize* |
| 199 |
and *typed*. This is for information purposes only. Mutating the values |
| 200 |
has no effect. |
| 201 |
|
| 202 |
.. method:: lru_cache.cache_info() |
| 203 |
:no-typesetting: |
| 204 |
|
| 205 |
To help measure the effectiveness of the cache and tune the *maxsize* |
| 206 |
parameter, the wrapped function is instrumented with a :func:`!cache_info` |
| 207 |
function that returns a :term:`named tuple` showing *hits*, *misses*, |
| 208 |
*maxsize* and *currsize*. |
| 209 |
|
| 210 |
.. method:: lru_cache.cache_clear() |
| 211 |
:no-typesetting: |
| 212 |
|
| 213 |
The decorator also provides a :func:`!cache_clear` function for clearing or |
| 214 |
invalidating the cache. |
| 215 |
|
| 216 |
The original underlying function is accessible through the |
| 217 |
:attr:`__wrapped__` attribute. This is useful for introspection, for |
| 218 |
bypassing the cache, or for rewrapping the function with a different cache. |
| 219 |
|
| 220 |
The cache keeps references to the arguments and return values until they age |
| 221 |
out of the cache or until the cache is cleared. |
| 222 |
|
| 223 |
If a method is cached, the ``self`` instance argument is included in the |
| 224 |
cache. See :ref:`faq-cache-method-calls` |
| 225 |
|
| 226 |
An `LRU (least recently used) cache |
| 227 |
<https://en.wikipedia.org/wiki/Cache_replacement_policies#Least_Recently_Used_(LRU)>`_ |
| 228 |
works best when the most recent calls are the best predictors of upcoming |
| 229 |
calls (for example, the most popular articles on a news server tend to |
| 230 |
change each day). The cache's size limit assures that the cache does not |
| 231 |
grow without bound on long-running processes such as web servers. |
| 232 |
|
| 233 |
In general, the LRU cache should only be used when you want to reuse |
| 234 |
previously computed values. Accordingly, it doesn't make sense to cache |
| 235 |
functions with side-effects, functions that need to create |
| 236 |
distinct mutable objects on each call (such as generators and async functions), |
| 237 |
or impure functions such as time() or random(). |
| 238 |
|
| 239 |
Example of an LRU cache for static web content:: |
| 240 |
|
| 241 |
@lru_cache(maxsize=32) |
| 242 |
def get_pep(num): |
| 243 |
'Retrieve text of a Python Enhancement Proposal' |
| 244 |
resource = f'https://peps.python.org/pep-{num:04d}' |
| 245 |
try: |
| 246 |
with urllib.request.urlopen(resource) as s: |
| 247 |
return s.read() |
| 248 |
except urllib.error.HTTPError: |
| 249 |
return 'Not Found' |
| 250 |
|
| 251 |
>>> for n in 8, 290, 308, 320, 8, 218, 320, 279, 289, 320, 9991: |
| 252 |
... pep = get_pep(n) |
| 253 |
... print(n, len(pep)) |
| 254 |
|
| 255 |
>>> get_pep.cache_info() |
| 256 |
CacheInfo(hits=3, misses=8, maxsize=32, currsize=8) |
| 257 |
|
| 258 |
Example of efficiently computing |
| 259 |
`Fibonacci numbers <https://en.wikipedia.org/wiki/Fibonacci_number>`_ |
| 260 |
using a cache to implement a |
| 261 |
`dynamic programming <https://en.wikipedia.org/wiki/Dynamic_programming>`_ |
| 262 |
technique:: |
| 263 |
|
| 264 |
@lru_cache(maxsize=None) |
| 265 |
def fib(n): |
| 266 |
if n < 2: |
| 267 |
return n |
| 268 |
return fib(n-1) + fib(n-2) |
| 269 |
|
| 270 |
>>> [fib(n) for n in range(16)] |
| 271 |
[0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610] |
| 272 |
|
| 273 |
>>> fib.cache_info() |
| 274 |
CacheInfo(hits=28, misses=16, maxsize=None, currsize=16) |
| 275 |
|
| 276 |
.. versionadded:: 3.2 |
| 277 |
|
| 278 |
.. versionchanged:: 3.3 |
| 279 |
Added the *typed* option. |
| 280 |
|
| 281 |
.. versionchanged:: 3.8 |
| 282 |
Added the *user_function* option. |
| 283 |
|
| 284 |
.. versionchanged:: 3.9 |
| 285 |
Added the function :func:`!cache_parameters` |
| 286 |
|
| 287 |
.. decorator:: total_ordering |
| 288 |
|
| 289 |
Given a class defining one or more rich comparison ordering methods, this |
| 290 |
class decorator supplies the rest. This simplifies the effort involved |
| 291 |
in specifying all of the possible rich comparison operations: |
| 292 |
|
| 293 |
The class must define one of :meth:`~object.__lt__`, :meth:`~object.__le__`, |
| 294 |
:meth:`~object.__gt__`, or :meth:`~object.__ge__`. |
| 295 |
In addition, the class should supply an :meth:`~object.__eq__` method. |
| 296 |
|
| 297 |
For example:: |
| 298 |
|
| 299 |
@total_ordering |
| 300 |
class Student: |
| 301 |
def _is_valid_operand(self, other): |
| 302 |
return (hasattr(other, "lastname") and |
| 303 |
hasattr(other, "firstname")) |
| 304 |
def __eq__(self, other): |
| 305 |
if not self._is_valid_operand(other): |
| 306 |
return NotImplemented |
| 307 |
return ((self.lastname.lower(), self.firstname.lower()) == |
| 308 |
(other.lastname.lower(), other.firstname.lower())) |
| 309 |
def __lt__(self, other): |
| 310 |
if not self._is_valid_operand(other): |
| 311 |
return NotImplemented |
| 312 |
return ((self.lastname.lower(), self.firstname.lower()) < |
| 313 |
(other.lastname.lower(), other.firstname.lower())) |
| 314 |
|
| 315 |
.. note:: |
| 316 |
|
| 317 |
While this decorator makes it easy to create well behaved totally |
| 318 |
ordered types, it *does* come at the cost of slower execution and |
| 319 |
more complex stack traces for the derived comparison methods. If |
| 320 |
performance benchmarking indicates this is a bottleneck for a given |
| 321 |
application, implementing all six rich comparison methods instead is |
| 322 |
likely to provide an easy speed boost. |
| 323 |
|
| 324 |
.. note:: |
| 325 |
|
| 326 |
This decorator makes no attempt to override methods that have been |
| 327 |
declared in the class *or its superclasses*. Meaning that if a |
| 328 |
superclass defines a comparison operator, *total_ordering* will not |
| 329 |
implement it again, even if the original method is abstract. |
| 330 |
|
| 331 |
.. versionadded:: 3.2 |
| 332 |
|
| 333 |
.. versionchanged:: 3.4 |
| 334 |
Returning ``NotImplemented`` from the underlying comparison function for |
| 335 |
unrecognised types is now supported. |
| 336 |
|
| 337 |
.. data:: Placeholder |
| 338 |
|
| 339 |
A singleton object used as a sentinel to reserve a place |
| 340 |
for positional arguments when calling :func:`partial` |
| 341 |
and :func:`partialmethod`. |
| 342 |
|
| 343 |
.. versionadded:: 3.14 |
| 344 |
|
| 345 |
.. function:: partial(func, /, *args, **keywords) |
| 346 |
|
| 347 |
Return a new :ref:`partial object<partial-objects>` which when called |
| 348 |
will behave like *func* called with the positional arguments *args* |
| 349 |
and keyword arguments *keywords*. If more arguments are supplied to the |
| 350 |
call, they are appended to *args*. If additional keyword arguments are |
| 351 |
supplied, they extend and override *keywords*. |
| 352 |
Roughly equivalent to:: |
| 353 |
|
| 354 |
def partial(func, /, *args, **keywords): |
| 355 |
def newfunc(*more_args, **more_keywords): |
| 356 |
return func(*args, *more_args, **(keywords | more_keywords)) |
| 357 |
newfunc.func = func |
| 358 |
newfunc.args = args |
| 359 |
newfunc.keywords = keywords |
| 360 |
return newfunc |
| 361 |
|
| 362 |
The :func:`!partial` function is used for partial function application which "freezes" |
| 363 |
some portion of a function's arguments and/or keywords resulting in a new object |
| 364 |
with a simplified signature. For example, :func:`partial` can be used to create |
| 365 |
a callable that behaves like the :func:`int` function where the *base* argument |
| 366 |
defaults to ``2``: |
| 367 |
|
| 368 |
.. doctest:: |
| 369 |
|
| 370 |
>>> basetwo = partial(int, base=2) |
| 371 |
>>> basetwo.__doc__ = 'Convert base 2 string to an int.' |
| 372 |
>>> basetwo('10010') |
| 373 |
18 |
| 374 |
|
| 375 |
If :data:`Placeholder` sentinels are present in *args*, they will be filled first |
| 376 |
when :func:`!partial` is called. This makes it possible to pre-fill any positional |
| 377 |
argument with a call to :func:`!partial`; without :data:`!Placeholder`, |
| 378 |
only the chosen number of leading positional arguments can be pre-filled. |
| 379 |
|
| 380 |
If any :data:`!Placeholder` sentinels are present, all must be filled at call time: |
| 381 |
|
| 382 |
.. doctest:: |
| 383 |
|
| 384 |
>>> say_to_world = partial(print, Placeholder, Placeholder, "world!") |
| 385 |
>>> say_to_world('Hello', 'dear') |
| 386 |
Hello dear world! |
| 387 |
|
| 388 |
Calling ``say_to_world('Hello')`` raises a :exc:`TypeError`, because |
| 389 |
only one positional argument is provided, but there are two placeholders |
| 390 |
that must be filled in. |
| 391 |
|
| 392 |
If :func:`!partial` is applied to an existing :func:`!partial` object, |
| 393 |
:data:`!Placeholder` sentinels of the input object are filled in with |
| 394 |
new positional arguments. |
| 395 |
A placeholder can be retained by inserting a new |
| 396 |
:data:`!Placeholder` sentinel to the place held by a previous :data:`!Placeholder`: |
| 397 |
|
| 398 |
.. doctest:: |
| 399 |
|
| 400 |
>>> from functools import partial, Placeholder as _ |
| 401 |
>>> remove = partial(str.replace, _, _, '') |
| 402 |
>>> message = 'Hello, dear dear world!' |
| 403 |
>>> remove(message, ' dear') |
| 404 |
'Hello, world!' |
| 405 |
>>> remove_dear = partial(remove, _, ' dear') |
| 406 |
>>> remove_dear(message) |
| 407 |
'Hello, world!' |
| 408 |
>>> remove_first_dear = partial(remove_dear, _, 1) |
| 409 |
>>> remove_first_dear(message) |
| 410 |
'Hello, dear world!' |
| 411 |
|
| 412 |
:data:`!Placeholder` cannot be passed to :func:`!partial` as a keyword argument. |
| 413 |
|
| 414 |
.. versionchanged:: 3.14 |
| 415 |
Added support for :data:`Placeholder` in positional arguments. |
| 416 |
|
| 417 |
.. class:: partialmethod(func, /, *args, **keywords) |
| 418 |
|
| 419 |
Return a new :class:`partialmethod` descriptor which behaves |
| 420 |
like :class:`partial` except that it is designed to be used as a method |
| 421 |
definition rather than being directly callable. |
| 422 |
|
| 423 |
*func* must be a :term:`descriptor` or a callable (objects which are both, |
| 424 |
like normal functions, are handled as descriptors). |
| 425 |
|
| 426 |
When *func* is a descriptor (such as a normal Python function, |
| 427 |
:func:`classmethod`, :func:`staticmethod`, :func:`~abc.abstractmethod` or |
| 428 |
another instance of :class:`partialmethod`), calls to ``__get__`` are |
| 429 |
delegated to the underlying descriptor, and an appropriate |
| 430 |
:ref:`partial object<partial-objects>` returned as the result. |
| 431 |
|
| 432 |
When *func* is a non-descriptor callable, an appropriate bound method is |
| 433 |
created dynamically. This behaves like a normal Python function when |
| 434 |
used as a method: the *self* argument will be inserted as the first |
| 435 |
positional argument, even before the *args* and *keywords* supplied to |
| 436 |
the :class:`partialmethod` constructor. |
| 437 |
|
| 438 |
Example:: |
| 439 |
|
| 440 |
>>> class Cell: |
| 441 |
... def __init__(self): |
| 442 |
... self._alive = False |
| 443 |
... @property |
| 444 |
... def alive(self): |
| 445 |
... return self._alive |
| 446 |
... def set_state(self, state): |
| 447 |
... self._alive = bool(state) |
| 448 |
... set_alive = partialmethod(set_state, True) |
| 449 |
... set_dead = partialmethod(set_state, False) |
| 450 |
... |
| 451 |
>>> c = Cell() |
| 452 |
>>> c.alive |
| 453 |
False |
| 454 |
>>> c.set_alive() |
| 455 |
>>> c.alive |
| 456 |
True |
| 457 |
|
| 458 |
.. versionadded:: 3.4 |
| 459 |
|
| 460 |
|
| 461 |
.. function:: reduce(function, iterable, /[, initial]) |
| 462 |
|
| 463 |
Apply *function* of two arguments cumulatively to the items of *iterable*, from |
| 464 |
left to right, so as to reduce the iterable to a single value. For example, |
| 465 |
``reduce(lambda x, y: x+y, [1, 2, 3, 4, 5])`` calculates ``((((1+2)+3)+4)+5)``. |
| 466 |
The left argument, *x*, is the accumulated value and the right argument, *y*, is |
| 467 |
the update value from the *iterable*. If the optional *initial* is present, |
| 468 |
it is placed before the items of the iterable in the calculation, and serves as |
| 469 |
a default when the iterable is empty. If *initial* is not given and |
| 470 |
*iterable* contains only one item, the first item is returned. |
| 471 |
|
| 472 |
Roughly equivalent to:: |
| 473 |
|
| 474 |
initial_missing = object() |
| 475 |
|
| 476 |
def reduce(function, iterable, /, initial=initial_missing): |
| 477 |
it = iter(iterable) |
| 478 |
if initial is initial_missing: |
| 479 |
value = next(it) |
| 480 |
else: |
| 481 |
value = initial |
| 482 |
for element in it: |
| 483 |
value = function(value, element) |
| 484 |
return value |
| 485 |
|
| 486 |
See :func:`itertools.accumulate` for an iterator that yields all intermediate |
| 487 |
values. |
| 488 |
|
| 489 |
.. versionchanged:: 3.14 |
| 490 |
*initial* is now supported as a keyword argument. |
| 491 |
|
| 492 |
.. decorator:: singledispatch |
| 493 |
|
| 494 |
Transform a function into a :term:`single-dispatch <single |
| 495 |
dispatch>` :term:`generic function`. |
| 496 |
|
| 497 |
To define a generic function, decorate it with the ``@singledispatch`` |
| 498 |
decorator. When defining a function using ``@singledispatch``, note that the |
| 499 |
dispatch happens on the type of the first argument:: |
| 500 |
|
| 501 |
>>> from functools import singledispatch |
| 502 |
>>> @singledispatch |
| 503 |
... def fun(arg, verbose=False): |
| 504 |
... if verbose: |
| 505 |
... print("Let me just say,", end=" ") |
| 506 |
... print(arg) |
| 507 |
|
| 508 |
.. method:: singledispatch.register() |
| 509 |
:no-typesetting: |
| 510 |
|
| 511 |
To add overloaded implementations to the function, use the :func:`!register` |
| 512 |
attribute of the generic function, which can be used as a decorator. For |
| 513 |
functions annotated with types, the decorator will infer the type of the |
| 514 |
first argument automatically:: |
| 515 |
|
| 516 |
>>> @fun.register |
| 517 |
... def _(arg: int, verbose=False): |
| 518 |
... if verbose: |
| 519 |
... print("Strength in numbers, eh?", end=" ") |
| 520 |
... print(arg) |
| 521 |
... |
| 522 |
>>> @fun.register |
| 523 |
... def _(arg: list, verbose=False): |
| 524 |
... if verbose: |
| 525 |
... print("Enumerate this:") |
| 526 |
... for i, elem in enumerate(arg): |
| 527 |
... print(i, elem) |
| 528 |
|
| 529 |
:class:`typing.Union` can also be used:: |
| 530 |
|
| 531 |
>>> @fun.register |
| 532 |
... def _(arg: int | float, verbose=False): |
| 533 |
... if verbose: |
| 534 |
... print("Strength in numbers, eh?", end=" ") |
| 535 |
... print(arg) |
| 536 |
... |
| 537 |
>>> from typing import Union |
| 538 |
>>> @fun.register |
| 539 |
... def _(arg: Union[list, set], verbose=False): |
| 540 |
... if verbose: |
| 541 |
... print("Enumerate this:") |
| 542 |
... for i, elem in enumerate(arg): |
| 543 |
... print(i, elem) |
| 544 |
... |
| 545 |
|
| 546 |
For code which doesn't use type annotations, the appropriate type |
| 547 |
argument can be passed explicitly to the decorator itself:: |
| 548 |
|
| 549 |
>>> @fun.register(complex) |
| 550 |
... def _(arg, verbose=False): |
| 551 |
... if verbose: |
| 552 |
... print("Better than complicated.", end=" ") |
| 553 |
... print(arg.real, arg.imag) |
| 554 |
... |
| 555 |
|
| 556 |
For code that dispatches on a collections type (e.g., ``list``), but wants |
| 557 |
to typehint the items of the collection (e.g., ``list[int]``), the |
| 558 |
dispatch type should be passed explicitly to the decorator itself with the |
| 559 |
typehint going into the function definition:: |
| 560 |
|
| 561 |
>>> @fun.register(list) |
| 562 |
... def _(arg: list[int], verbose=False): |
| 563 |
... if verbose: |
| 564 |
... print("Enumerate this:") |
| 565 |
... for i, elem in enumerate(arg): |
| 566 |
... print(i, elem) |
| 567 |
|
| 568 |
.. note:: |
| 569 |
|
| 570 |
At runtime the function will dispatch on an instance of a list regardless |
| 571 |
of the type contained within the list i.e. ``[1,2,3]`` will be |
| 572 |
dispatched the same as ``["foo", "bar", "baz"]``. The annotation |
| 573 |
provided in this example is for static type checkers only and has no |
| 574 |
runtime impact. |
| 575 |
|
| 576 |
To enable registering :term:`lambdas<lambda>` and pre-existing functions, |
| 577 |
the :func:`~singledispatch.register` attribute can also be used in a functional form:: |
| 578 |
|
| 579 |
>>> def nothing(arg, verbose=False): |
| 580 |
... print("Nothing.") |
| 581 |
... |
| 582 |
>>> fun.register(type(None), nothing) |
| 583 |
|
| 584 |
The :func:`~singledispatch.register` attribute returns the undecorated function. This |
| 585 |
enables decorator stacking, :mod:`pickling<pickle>`, and the creation |
| 586 |
of unit tests for each variant independently:: |
| 587 |
|
| 588 |
>>> @fun.register(float) |
| 589 |
... @fun.register(Decimal) |
| 590 |
... def fun_num(arg, verbose=False): |
| 591 |
... if verbose: |
| 592 |
... print("Half of your number:", end=" ") |
| 593 |
... print(arg / 2) |
| 594 |
... |
| 595 |
>>> fun_num is fun |
| 596 |
False |
| 597 |
|
| 598 |
When called, the generic function dispatches on the type of the first |
| 599 |
argument:: |
| 600 |
|
| 601 |
>>> fun("Hello, world.") |
| 602 |
Hello, world. |
| 603 |
>>> fun("test.", verbose=True) |
| 604 |
Let me just say, test. |
| 605 |
>>> fun(42, verbose=True) |
| 606 |
Strength in numbers, eh? 42 |
| 607 |
>>> fun(['spam', 'spam', 'eggs', 'spam'], verbose=True) |
| 608 |
Enumerate this: |
| 609 |
0 spam |
| 610 |
1 spam |
| 611 |
2 eggs |
| 612 |
3 spam |
| 613 |
>>> fun(None) |
| 614 |
Nothing. |
| 615 |
>>> fun(1.23) |
| 616 |
0.615 |
| 617 |
|
| 618 |
Where there is no registered implementation for a specific type, its |
| 619 |
method resolution order is used to find a more generic implementation. |
| 620 |
The original function decorated with ``@singledispatch`` is registered |
| 621 |
for the base :class:`object` type, which means it is used if no better |
| 622 |
implementation is found. |
| 623 |
|
| 624 |
If an implementation is registered to an :term:`abstract base class`, |
| 625 |
virtual subclasses of the base class will be dispatched to that |
| 626 |
implementation:: |
| 627 |
|
| 628 |
>>> from collections.abc import Mapping |
| 629 |
>>> @fun.register |
| 630 |
... def _(arg: Mapping, verbose=False): |
| 631 |
... if verbose: |
| 632 |
... print("Keys & Values") |
| 633 |
... for key, value in arg.items(): |
| 634 |
... print(key, "=>", value) |
| 635 |
... |
| 636 |
>>> fun({"a": "b"}) |
| 637 |
a => b |
| 638 |
|
| 639 |
To check which implementation the generic function will choose for |
| 640 |
a given type, use the ``dispatch()`` attribute:: |
| 641 |
|
| 642 |
>>> fun.dispatch(float) |
| 643 |
<function fun_num at 0x1035a2840> |
| 644 |
>>> fun.dispatch(dict) # note: default implementation |
| 645 |
<function fun at 0x103fe0000> |
| 646 |
|
| 647 |
To access all registered implementations, use the read-only ``registry`` |
| 648 |
attribute:: |
| 649 |
|
| 650 |
>>> fun.registry.keys() |
| 651 |
dict_keys([<class 'NoneType'>, <class 'int'>, <class 'object'>, |
| 652 |
<class 'decimal.Decimal'>, <class 'list'>, |
| 653 |
<class 'float'>]) |
| 654 |
>>> fun.registry[float] |
| 655 |
<function fun_num at 0x1035a2840> |
| 656 |
>>> fun.registry[object] |
| 657 |
<function fun at 0x103fe0000> |
| 658 |
|
| 659 |
.. versionadded:: 3.4 |
| 660 |
|
| 661 |
.. versionchanged:: 3.7 |
| 662 |
The :func:`~singledispatch.register` attribute now supports using type annotations. |
| 663 |
|
| 664 |
.. versionchanged:: 3.11 |
| 665 |
The :func:`~singledispatch.register` attribute now supports |
| 666 |
:class:`typing.Union` as a type annotation. |
| 667 |
|
| 668 |
|
| 669 |
.. class:: singledispatchmethod(func) |
| 670 |
|
| 671 |
Transform a method into a :term:`single-dispatch <single |
| 672 |
dispatch>` :term:`generic function`. |
| 673 |
|
| 674 |
To define a generic method, decorate it with the ``@singledispatchmethod`` |
| 675 |
decorator. When defining a method using ``@singledispatchmethod``, note |
| 676 |
that the dispatch happens on the type of the first non-*self* or non-*cls* |
| 677 |
argument:: |
| 678 |
|
| 679 |
class Negator: |
| 680 |
@singledispatchmethod |
| 681 |
def neg(self, arg): |
| 682 |
raise NotImplementedError("Cannot negate a") |
| 683 |
|
| 684 |
@neg.register |
| 685 |
def _(self, arg: int): |
| 686 |
return -arg |
| 687 |
|
| 688 |
@neg.register |
| 689 |
def _(self, arg: bool): |
| 690 |
return not arg |
| 691 |
|
| 692 |
``@singledispatchmethod`` supports nesting with other decorators such as |
| 693 |
:deco:`classmethod`. Note that to allow for |
| 694 |
``dispatcher.register``, ``singledispatchmethod`` must be the *outer most* |
| 695 |
decorator. Here is the ``Negator`` class with the ``neg`` methods bound to |
| 696 |
the class, rather than an instance of the class:: |
| 697 |
|
| 698 |
class Negator: |
| 699 |
@singledispatchmethod |
| 700 |
@classmethod |
| 701 |
def neg(cls, arg): |
| 702 |
raise NotImplementedError("Cannot negate a") |
| 703 |
|
| 704 |
@neg.register |
| 705 |
@classmethod |
| 706 |
def _(cls, arg: int): |
| 707 |
return -arg |
| 708 |
|
| 709 |
@neg.register |
| 710 |
@classmethod |
| 711 |
def _(cls, arg: bool): |
| 712 |
return not arg |
| 713 |
|
| 714 |
The same pattern can be used for other similar decorators: |
| 715 |
:deco:`staticmethod`, :deco:`~abc.abstractmethod`, and others. |
| 716 |
|
| 717 |
.. versionadded:: 3.8 |
| 718 |
|
| 719 |
|
| 720 |
.. function:: update_wrapper(wrapper, wrapped, assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES) |
| 721 |
|
| 722 |
Update a *wrapper* function to look like the *wrapped* function. The optional |
| 723 |
arguments are tuples to specify which attributes of the original function are |
| 724 |
assigned directly to the matching attributes on the wrapper function and which |
| 725 |
attributes of the wrapper function are updated with the corresponding attributes |
| 726 |
from the original function. The default values for these arguments are the |
| 727 |
module level constants ``WRAPPER_ASSIGNMENTS`` (which assigns to the wrapper |
| 728 |
function's :attr:`~function.__module__`, :attr:`~function.__name__`, |
| 729 |
:attr:`~function.__qualname__`, :attr:`~function.__annotations__`, |
| 730 |
:attr:`~function.__type_params__`, and :attr:`~function.__doc__`, the |
| 731 |
documentation string) and ``WRAPPER_UPDATES`` (which updates the wrapper |
| 732 |
function's :attr:`~function.__dict__`, i.e. the instance dictionary). |
| 733 |
|
| 734 |
To allow access to the original function for introspection and other purposes |
| 735 |
(e.g. bypassing a caching decorator such as :func:`lru_cache`), this function |
| 736 |
automatically adds a ``__wrapped__`` attribute to the wrapper that refers to |
| 737 |
the function being wrapped. |
| 738 |
|
| 739 |
The main intended use for this function is in :term:`decorator` functions which |
| 740 |
wrap the decorated function and return the wrapper. If the wrapper function is |
| 741 |
not updated, the metadata of the returned function will reflect the wrapper |
| 742 |
definition rather than the original function definition, which is typically less |
| 743 |
than helpful. |
| 744 |
|
| 745 |
:func:`update_wrapper` may be used with callables other than functions. Any |
| 746 |
attributes named in *assigned* or *updated* that are missing from the object |
| 747 |
being wrapped are ignored (i.e. this function will not attempt to set them |
| 748 |
on the wrapper function). :exc:`AttributeError` is still raised if the |
| 749 |
wrapper function itself is missing any attributes named in *updated*. |
| 750 |
|
| 751 |
.. versionchanged:: 3.2 |
| 752 |
The ``__wrapped__`` attribute is now automatically added. |
| 753 |
The :attr:`~function.__annotations__` attribute is now copied by default. |
| 754 |
Missing attributes no longer trigger an :exc:`AttributeError`. |
| 755 |
|
| 756 |
.. versionchanged:: 3.4 |
| 757 |
The ``__wrapped__`` attribute now always refers to the wrapped |
| 758 |
function, even if that function defined a ``__wrapped__`` attribute. |
| 759 |
(see :issue:`17482`) |
| 760 |
|
| 761 |
.. versionchanged:: 3.12 |
| 762 |
The :attr:`~function.__type_params__` attribute is now copied by default. |
| 763 |
|
| 764 |
|
| 765 |
.. decorator:: wraps(wrapped, assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES) |
| 766 |
|
| 767 |
This is a convenience function for invoking :func:`update_wrapper` as a |
| 768 |
function decorator when defining a wrapper function. It is equivalent to |
| 769 |
``partial(update_wrapper, wrapped=wrapped, assigned=assigned, updated=updated)``. |
| 770 |
For example:: |
| 771 |
|
| 772 |
>>> from functools import wraps |
| 773 |
>>> def my_decorator(f): |
| 774 |
... @wraps(f) |
| 775 |
... def wrapper(*args, **kwds): |
| 776 |
... print('Calling decorated function') |
| 777 |
... return f(*args, **kwds) |
| 778 |
... return wrapper |
| 779 |
... |
| 780 |
>>> @my_decorator |
| 781 |
... def example(): |
| 782 |
... """Docstring""" |
| 783 |
... print('Called example function') |
| 784 |
... |
| 785 |
>>> example() |
| 786 |
Calling decorated function |
| 787 |
Called example function |
| 788 |
>>> example.__name__ |
| 789 |
'example' |
| 790 |
>>> example.__doc__ |
| 791 |
'Docstring' |
| 792 |
|
| 793 |
Without the use of this decorator factory, the name of the example function |
| 794 |
would have been ``'wrapper'``, and the docstring of the original :func:`!example` |
| 795 |
would have been lost. |
| 796 |
|
| 797 |
|
| 798 |
.. _partial-objects: |
| 799 |
|
| 800 |
:class:`partial` Objects |
| 801 |
------------------------ |
| 802 |
|
| 803 |
:class:`partial` objects are callable objects created by :func:`partial`. They |
| 804 |
have three read-only attributes: |
| 805 |
|
| 806 |
|
| 807 |
.. attribute:: partial.func |
| 808 |
|
| 809 |
A callable object or function. Calls to the :class:`partial` object will be |
| 810 |
forwarded to :attr:`func` with new arguments and keywords. |
| 811 |
|
| 812 |
|
| 813 |
.. attribute:: partial.args |
| 814 |
|
| 815 |
The leftmost positional arguments that will be prepended to the positional |
| 816 |
arguments provided to a :class:`partial` object call. |
| 817 |
|
| 818 |
|
| 819 |
.. attribute:: partial.keywords |
| 820 |
|
| 821 |
The keyword arguments that will be supplied when the :class:`partial` object is |
| 822 |
called. |
| 823 |
|
| 824 |
:class:`partial` objects are like :ref:`function objects <user-defined-funcs>` in that they are |
| 825 |
callable, weak referenceable, and can have attributes. There are some important |
| 826 |
differences. For instance, the :attr:`~definition.__name__` and :attr:`~definition.__doc__` attributes |
| 827 |
are not created automatically. |