Debian Codegraph @

Doc/library/collections.abc.rst

raw
1 :mod:`!collections.abc` --- Abstract Base Classes for Containers
2 ================================================================
3
4 .. module:: collections.abc
5 :synopsis: Abstract base classes for containers
6
7 .. moduleauthor:: Raymond Hettinger <python at rcn.com>
8 .. sectionauthor:: Raymond Hettinger <python at rcn.com>
9
10 .. versionadded:: 3.3
11 Formerly, this module was part of the :mod:`collections` module.
12
13 **Source code:** :source:`Lib/_collections_abc.py`
14
15 .. testsetup:: *
16
17 from collections.abc import *
18 import itertools
19 __name__ = '<doctest>'
20
21 --------------
22
23 This module provides :term:`abstract base classes <abstract base class>` that
24 can be used to test whether a class provides a particular interface; for
25 example, whether it is :term:`hashable` or whether it is a :term:`mapping`.
26
27 An :func:`issubclass` or :func:`isinstance` test for an interface works in one
28 of three ways.
29
30 1) A newly written class can inherit directly from one of the
31 abstract base classes. The class must supply the required abstract
32 methods. The remaining mixin methods come from inheritance and can be
33 overridden if desired. Other methods may be added as needed:
34
35 .. testcode::
36
37 class C(Sequence): # Direct inheritance
38 def __init__(self): ... # Extra method not required by the ABC
39 def __getitem__(self, index): ... # Required abstract method
40 def __len__(self): ... # Required abstract method
41 def count(self, value): ... # Optionally override a mixin method
42
43 .. doctest::
44
45 >>> issubclass(C, Sequence)
46 True
47 >>> isinstance(C(), Sequence)
48 True
49
50 2) Existing classes and built-in classes can be registered as "virtual
51 subclasses" of the ABCs. Those classes should define the full API
52 including all of the abstract methods and all of the mixin methods.
53 This lets users rely on :func:`issubclass` or :func:`isinstance` tests
54 to determine whether the full interface is supported. The exception to
55 this rule is for methods that are automatically inferred from the rest
56 of the API:
57
58 .. testcode::
59
60 class D: # No inheritance
61 def __init__(self): ... # Extra method not required by the ABC
62 def __getitem__(self, index): ... # Abstract method
63 def __len__(self): ... # Abstract method
64 def count(self, value): ... # Mixin method
65 def index(self, value): ... # Mixin method
66
67 Sequence.register(D) # Register instead of inherit
68
69 .. doctest::
70
71 >>> issubclass(D, Sequence)
72 True
73 >>> isinstance(D(), Sequence)
74 True
75
76 In this example, class :class:`!D` does not need to define
77 ``__contains__``, ``__iter__``, and ``__reversed__`` because the
78 :ref:`in-operator <comparisons>`, the :term:`iteration <iterable>`
79 logic, and the :func:`reversed` function automatically fall back to
80 using ``__getitem__`` and ``__len__``.
81
82 3) Some simple interfaces are directly recognizable by the presence of
83 the required methods (unless those methods have been set to :const:`None`):
84
85 .. testcode::
86
87 class E:
88 def __iter__(self): ...
89 def __next__(self): ...
90
91 .. doctest::
92
93 >>> issubclass(E, Iterable)
94 True
95 >>> isinstance(E(), Iterable)
96 True
97
98 Complex interfaces do not support this last technique because an
99 interface is more than just the presence of method names. Interfaces
100 specify semantics and relationships between methods that cannot be
101 inferred solely from the presence of specific method names. For
102 example, knowing that a class supplies ``__getitem__``, ``__len__``, and
103 ``__iter__`` is insufficient for distinguishing a :class:`Sequence` from
104 a :class:`Mapping`.
105
106 .. versionadded:: 3.9
107 These abstract classes now support ``[]``. See :ref:`types-genericalias`
108 and :pep:`585`.
109
110 .. _collections-abstract-base-classes:
111
112 Collections Abstract Base Classes
113 ---------------------------------
114
115 The collections module offers the following :term:`ABCs <abstract base class>`:
116
117 .. tabularcolumns:: |l|L|L|L|
118
119 ============================== ====================== ======================= ====================================================
120 ABC Inherits from Abstract Methods Mixin Methods
121 ============================== ====================== ======================= ====================================================
122 :class:`Container` [1]_ ``__contains__``
123 :class:`Hashable` [1]_ ``__hash__``
124 :class:`Iterable` [1]_ [2]_ ``__iter__``
125 :class:`Iterator` [1]_ :class:`Iterable` ``__next__`` ``__iter__``
126 :class:`Reversible` [1]_ :class:`Iterable` ``__reversed__``
127 :class:`Generator` [1]_ :class:`Iterator` ``send``, ``throw`` ``close``, ``__iter__``, ``__next__``
128 :class:`Sized` [1]_ ``__len__``
129 :class:`Callable` [1]_ ``__call__``
130 :class:`Collection` [1]_ :class:`Sized`, ``__contains__``,
131 :class:`Iterable`, ``__iter__``,
132 :class:`Container` ``__len__``
133
134 :class:`Sequence` :class:`Reversible`, ``__getitem__``, ``__contains__``, ``__iter__``, ``__reversed__``,
135 :class:`Collection` ``__len__`` ``index``, and ``count``
136
137 :class:`MutableSequence` :class:`Sequence` ``__getitem__``, Inherited :class:`Sequence` methods and
138 ``__setitem__``, ``append``, ``clear``, ``reverse``, ``extend``,
139 ``__delitem__``, ``pop``, ``remove``, and ``__iadd__``
140 ``__len__``,
141 ``insert``
142
143 :class:`ByteString` :class:`Sequence` ``__getitem__``, Inherited :class:`Sequence` methods
144 ``__len__``
145
146 :class:`Set` :class:`Collection` ``__contains__``, ``__le__``, ``__lt__``, ``__eq__``, ``__ne__``,
147 ``__iter__``, ``__gt__``, ``__ge__``, ``__and__``, ``__or__``,
148 ``__len__`` ``__sub__``, ``__rsub__``, ``__xor__``, ``__rxor__``
149 and ``isdisjoint``
150
151 :class:`MutableSet` :class:`Set` ``__contains__``, Inherited :class:`Set` methods and
152 ``__iter__``, ``clear``, ``pop``, ``remove``, ``__ior__``,
153 ``__len__``, ``__iand__``, ``__ixor__``, and ``__isub__``
154 ``add``,
155 ``discard``
156
157 :class:`Mapping` :class:`Collection` ``__getitem__``, ``__contains__``, ``keys``, ``items``, ``values``,
158 ``__iter__``, ``get``, ``__eq__``, and ``__ne__``
159 ``__len__``
160
161 :class:`MutableMapping` :class:`Mapping` ``__getitem__``, Inherited :class:`Mapping` methods and
162 ``__setitem__``, ``pop``, ``popitem``, ``clear``, ``update``,
163 ``__delitem__``, and ``setdefault``
164 ``__iter__``,
165 ``__len__``
166
167
168 :class:`MappingView` :class:`Sized` ``__init__``, ``__len__`` and ``__repr__``
169 :class:`ItemsView` :class:`MappingView`, ``__contains__``,
170 :class:`Set` ``__iter__``
171 :class:`KeysView` :class:`MappingView`, ``__contains__``,
172 :class:`Set` ``__iter__``
173 :class:`ValuesView` :class:`MappingView`, ``__contains__``, ``__iter__``
174 :class:`Collection`
175 :class:`Awaitable` [1]_ ``__await__``
176 :class:`Coroutine` [1]_ :class:`Awaitable` ``send``, ``throw`` ``close``
177 :class:`AsyncIterable` [1]_ ``__aiter__``
178 :class:`AsyncIterator` [1]_ :class:`AsyncIterable` ``__anext__`` ``__aiter__``
179 :class:`AsyncGenerator` [1]_ :class:`AsyncIterator` ``asend``, ``athrow`` ``aclose``, ``__aiter__``, ``__anext__``
180 :class:`Buffer` [1]_ ``__buffer__``
181 ============================== ====================== ======================= ====================================================
182
183
184 .. rubric:: Footnotes
185
186 .. [1] These ABCs override :meth:`~abc.ABCMeta.__subclasshook__` to support
187 testing an interface by verifying the required methods are present
188 and have not been set to :const:`None`. This only works for simple
189 interfaces. More complex interfaces require registration or direct
190 subclassing.
191
192 .. [2] Checking ``isinstance(obj, Iterable)`` detects classes that are
193 registered as :class:`Iterable` or that have an :meth:`~container.__iter__`
194 method, but it does not detect classes that iterate with the
195 :meth:`~object.__getitem__` method. The only reliable way to determine
196 whether an object is :term:`iterable` is to call ``iter(obj)``.
197
198
199 Collections Abstract Base Classes -- Detailed Descriptions
200 ----------------------------------------------------------
201
202
203 .. class:: Container
204
205 ABC for classes that provide the :meth:`~object.__contains__` method.
206
207 .. class:: Hashable
208
209 ABC for classes that provide the :meth:`~object.__hash__` method.
210
211 .. class:: Sized
212
213 ABC for classes that provide the :meth:`~object.__len__` method.
214
215 .. class:: Callable
216
217 ABC for classes that provide the :meth:`~object.__call__` method.
218
219 See :ref:`annotating-callables` for details on how to use
220 :class:`!Callable` in type annotations.
221
222 .. class:: Iterable
223
224 ABC for classes that provide the :meth:`~container.__iter__` method.
225
226 Checking ``isinstance(obj, Iterable)`` detects classes that are registered
227 as :class:`Iterable` or that have an :meth:`~container.__iter__` method,
228 but it does
229 not detect classes that iterate with the :meth:`~object.__getitem__` method.
230 The only reliable way to determine whether an object is :term:`iterable`
231 is to call ``iter(obj)``.
232
233 .. class:: Collection
234
235 ABC for sized iterable container classes.
236
237 .. versionadded:: 3.6
238
239 .. class:: Iterator
240
241 ABC for classes that provide the :meth:`~iterator.__iter__` and
242 :meth:`~iterator.__next__` methods. See also the definition of
243 :term:`iterator`.
244
245 .. class:: Reversible
246
247 ABC for iterable classes that also provide the :meth:`~object.__reversed__`
248 method.
249
250 .. versionadded:: 3.6
251
252 .. class:: Generator
253
254 ABC for :term:`generator` classes that implement the protocol defined in
255 :pep:`342` that extends :term:`iterators <iterator>` with the
256 :meth:`~generator.send`,
257 :meth:`~generator.throw` and :meth:`~generator.close` methods.
258
259 See :ref:`annotating-generators-and-coroutines`
260 for details on using :class:`!Generator` in type annotations.
261
262 .. versionadded:: 3.5
263
264 .. class:: Sequence
265 MutableSequence
266 ByteString
267
268 ABCs for read-only and mutable :term:`sequences <sequence>`.
269
270 Implementation note: Some of the mixin methods, such as
271 :meth:`~container.__iter__`, :meth:`~object.__reversed__`,
272 and :meth:`~sequence.index` make repeated calls to the underlying
273 :meth:`~object.__getitem__` method.
274 Consequently, if :meth:`~object.__getitem__` is implemented with constant
275 access speed, the mixin methods will have linear performance;
276 however, if the underlying method is linear (as it would be with a
277 linked list), the mixins will have quadratic performance and will
278 likely need to be overridden.
279
280 .. method:: index(value, start=0, stop=None)
281
282 Return first index of *value*.
283
284 Raises :exc:`ValueError` if the value is not present.
285
286 Supporting the *start* and *stop* arguments is optional, but recommended.
287
288 .. versionchanged:: 3.5
289 The :meth:`~sequence.index` method gained support for
290 the *stop* and *start* arguments.
291
292 .. deprecated-removed:: 3.12 3.17
293 The :class:`ByteString` ABC has been deprecated.
294
295 Use ``isinstance(obj, collections.abc.Buffer)`` to test if ``obj``
296 implements the :ref:`buffer protocol <bufferobjects>` at runtime. For use
297 in type annotations, either use :class:`Buffer` or a union that
298 explicitly specifies the types your code supports (e.g.,
299 ``bytes | bytearray | memoryview``).
300
301 :class:`!ByteString` was originally intended to be an abstract class that
302 would serve as a supertype of both :class:`bytes` and :class:`bytearray`.
303 However, since the ABC never had any methods, knowing that an object was
304 an instance of :class:`!ByteString` never actually told you anything
305 useful about the object. Other common buffer types such as
306 :class:`memoryview` were also never understood as subtypes of
307 :class:`!ByteString` (either at runtime or by static type checkers).
308
309 See :pep:`PEP 688 <688#current-options>` for more details.
310
311 .. class:: Set
312 MutableSet
313
314 ABCs for read-only and mutable :ref:`sets <types-set>`.
315
316 .. class:: Mapping
317 MutableMapping
318
319 ABCs for read-only and mutable :term:`mappings <mapping>`.
320
321 .. class:: MappingView
322 ItemsView
323 KeysView
324 ValuesView
325
326 ABCs for mapping, items, keys, and values :term:`views <dictionary view>`.
327
328 .. class:: Awaitable
329
330 ABC for :term:`awaitable` objects, which can be used in :keyword:`await`
331 expressions. Custom implementations must provide the
332 :meth:`~object.__await__` method.
333
334 :term:`Coroutine <coroutine>` objects and instances of the
335 :class:`~collections.abc.Coroutine` ABC are all instances of this ABC.
336
337 .. note::
338 In CPython, generator-based coroutines (:term:`generators <generator>`
339 decorated with :deco:`types.coroutine`) are
340 *awaitables*, even though they do not have an :meth:`~object.__await__` method.
341 Using ``isinstance(gencoro, Awaitable)`` for them will return ``False``.
342 Use :func:`inspect.isawaitable` to detect them.
343
344 .. versionadded:: 3.5
345
346 .. class:: Coroutine
347
348 ABC for :term:`coroutine` compatible classes. These implement the
349 following methods, defined in :ref:`coroutine-objects`:
350 :meth:`~coroutine.send`, :meth:`~coroutine.throw`, and
351 :meth:`~coroutine.close`. Custom implementations must also implement
352 :meth:`~object.__await__`. All :class:`Coroutine` instances are also
353 instances of :class:`Awaitable`.
354
355 .. note::
356 In CPython, generator-based coroutines (:term:`generators <generator>`
357 decorated with :deco:`types.coroutine`) are
358 *awaitables*, even though they do not have an :meth:`~object.__await__` method.
359 Using ``isinstance(gencoro, Coroutine)`` for them will return ``False``.
360 Use :func:`inspect.isawaitable` to detect them.
361
362 See :ref:`annotating-generators-and-coroutines`
363 for details on using :class:`!Coroutine` in type annotations.
364 The variance and order of type parameters correspond to those of
365 :class:`Generator`.
366
367 .. versionadded:: 3.5
368
369 .. class:: AsyncIterable
370
371 ABC for classes that provide an ``__aiter__`` method. See also the
372 definition of :term:`asynchronous iterable`.
373
374 .. versionadded:: 3.5
375
376 .. class:: AsyncIterator
377
378 ABC for classes that provide ``__aiter__`` and ``__anext__``
379 methods. See also the definition of :term:`asynchronous iterator`.
380
381 .. versionadded:: 3.5
382
383 .. class:: AsyncGenerator
384
385 ABC for :term:`asynchronous generator` classes that implement the protocol
386 defined in :pep:`525` and :pep:`492`.
387
388 See :ref:`annotating-generators-and-coroutines`
389 for details on using :class:`!AsyncGenerator` in type annotations.
390
391 .. versionadded:: 3.6
392
393 .. class:: Buffer
394
395 ABC for classes that provide the :meth:`~object.__buffer__` method,
396 implementing the :ref:`buffer protocol <bufferobjects>`. See :pep:`688`.
397
398 .. versionadded:: 3.12
399
400 Examples and Recipes
401 --------------------
402
403 ABCs allow us to ask classes or instances if they provide
404 particular functionality, for example::
405
406 size = None
407 if isinstance(myvar, collections.abc.Sized):
408 size = len(myvar)
409
410 Several of the ABCs are also useful as mixins that make it easier to develop
411 classes supporting container APIs. For example, to write a class supporting
412 the full :class:`Set` API, it is only necessary to supply the three underlying
413 abstract methods: :meth:`~object.__contains__`, :meth:`~container.__iter__`, and
414 :meth:`~object.__len__`. The ABC supplies the remaining methods such as
415 :meth:`!__and__` and :meth:`~frozenset.isdisjoint`::
416
417 class ListBasedSet(collections.abc.Set):
418 ''' Alternate set implementation favoring space over speed
419 and not requiring the set elements to be hashable. '''
420 def __init__(self, iterable):
421 self.elements = lst = []
422 for value in iterable:
423 if value not in lst:
424 lst.append(value)
425
426 def __iter__(self):
427 return iter(self.elements)
428
429 def __contains__(self, value):
430 return value in self.elements
431
432 def __len__(self):
433 return len(self.elements)
434
435 s1 = ListBasedSet('abcdef')
436 s2 = ListBasedSet('defghi')
437 overlap = s1 & s2 # The __and__() method is supported automatically
438
439 Notes on using :class:`Set` and :class:`MutableSet` as a mixin:
440
441 (1)
442 Since some set operations create new sets, the default mixin methods need
443 a way to create new instances from an :term:`iterable`. The class constructor is
444 assumed to have a signature in the form ``ClassName(iterable)``.
445 That assumption is factored-out to an internal :class:`classmethod` called
446 :meth:`!_from_iterable` which calls ``cls(iterable)`` to produce a new set.
447 If the :class:`Set` mixin is being used in a class with a different
448 constructor signature, you will need to override :meth:`!_from_iterable`
449 with a classmethod or regular method that can construct new instances from
450 an iterable argument.
451
452 (2)
453 To override the comparisons (presumably for speed, as the
454 semantics are fixed), redefine :meth:`~object.__le__` and
455 :meth:`~object.__ge__`,
456 then the other operations will automatically follow suit.
457
458 (3)
459 The :class:`Set` mixin provides a :meth:`!_hash` method to compute a hash value
460 for the set; however, :meth:`~object.__hash__` is not defined because not all sets
461 are :term:`hashable` or immutable. To add set hashability using mixins,
462 inherit from both :class:`Set` and :class:`Hashable`, then define
463 ``__hash__ = Set._hash``.
464
465 .. seealso::
466
467 * `OrderedSet recipe <https://code.activestate.com/recipes/576694/>`_ for an
468 example built on :class:`MutableSet`.
469
470 * For more about ABCs, see the :mod:`abc` module and :pep:`3119`.