| 1 |
:mod:`!collections.abc` --- Abstract Base Classes for Containers |
| 2 |
================================================================ |
| 3 |
|
| 4 |
.. module:: collections.abc |
| 5 |
:synopsis: Abstract base classes for containers |
| 6 |
|
| 7 |
.. moduleauthor:: Raymond Hettinger <python at rcn.com> |
| 8 |
.. sectionauthor:: Raymond Hettinger <python at rcn.com> |
| 9 |
|
| 10 |
.. versionadded:: 3.3 |
| 11 |
Formerly, this module was part of the :mod:`collections` module. |
| 12 |
|
| 13 |
**Source code:** :source:`Lib/_collections_abc.py` |
| 14 |
|
| 15 |
.. testsetup:: * |
| 16 |
|
| 17 |
from collections.abc import * |
| 18 |
import itertools |
| 19 |
__name__ = '<doctest>' |
| 20 |
|
| 21 |
-------------- |
| 22 |
|
| 23 |
This module provides :term:`abstract base classes <abstract base class>` that |
| 24 |
can be used to test whether a class provides a particular interface; for |
| 25 |
example, whether it is :term:`hashable` or whether it is a :term:`mapping`. |
| 26 |
|
| 27 |
An :func:`issubclass` or :func:`isinstance` test for an interface works in one |
| 28 |
of three ways. |
| 29 |
|
| 30 |
1) A newly written class can inherit directly from one of the |
| 31 |
abstract base classes. The class must supply the required abstract |
| 32 |
methods. The remaining mixin methods come from inheritance and can be |
| 33 |
overridden if desired. Other methods may be added as needed: |
| 34 |
|
| 35 |
.. testcode:: |
| 36 |
|
| 37 |
class C(Sequence): # Direct inheritance |
| 38 |
def __init__(self): ... # Extra method not required by the ABC |
| 39 |
def __getitem__(self, index): ... # Required abstract method |
| 40 |
def __len__(self): ... # Required abstract method |
| 41 |
def count(self, value): ... # Optionally override a mixin method |
| 42 |
|
| 43 |
.. doctest:: |
| 44 |
|
| 45 |
>>> issubclass(C, Sequence) |
| 46 |
True |
| 47 |
>>> isinstance(C(), Sequence) |
| 48 |
True |
| 49 |
|
| 50 |
2) Existing classes and built-in classes can be registered as "virtual |
| 51 |
subclasses" of the ABCs. Those classes should define the full API |
| 52 |
including all of the abstract methods and all of the mixin methods. |
| 53 |
This lets users rely on :func:`issubclass` or :func:`isinstance` tests |
| 54 |
to determine whether the full interface is supported. The exception to |
| 55 |
this rule is for methods that are automatically inferred from the rest |
| 56 |
of the API: |
| 57 |
|
| 58 |
.. testcode:: |
| 59 |
|
| 60 |
class D: # No inheritance |
| 61 |
def __init__(self): ... # Extra method not required by the ABC |
| 62 |
def __getitem__(self, index): ... # Abstract method |
| 63 |
def __len__(self): ... # Abstract method |
| 64 |
def count(self, value): ... # Mixin method |
| 65 |
def index(self, value): ... # Mixin method |
| 66 |
|
| 67 |
Sequence.register(D) # Register instead of inherit |
| 68 |
|
| 69 |
.. doctest:: |
| 70 |
|
| 71 |
>>> issubclass(D, Sequence) |
| 72 |
True |
| 73 |
>>> isinstance(D(), Sequence) |
| 74 |
True |
| 75 |
|
| 76 |
In this example, class :class:`!D` does not need to define |
| 77 |
``__contains__``, ``__iter__``, and ``__reversed__`` because the |
| 78 |
:ref:`in-operator <comparisons>`, the :term:`iteration <iterable>` |
| 79 |
logic, and the :func:`reversed` function automatically fall back to |
| 80 |
using ``__getitem__`` and ``__len__``. |
| 81 |
|
| 82 |
3) Some simple interfaces are directly recognizable by the presence of |
| 83 |
the required methods (unless those methods have been set to :const:`None`): |
| 84 |
|
| 85 |
.. testcode:: |
| 86 |
|
| 87 |
class E: |
| 88 |
def __iter__(self): ... |
| 89 |
def __next__(self): ... |
| 90 |
|
| 91 |
.. doctest:: |
| 92 |
|
| 93 |
>>> issubclass(E, Iterable) |
| 94 |
True |
| 95 |
>>> isinstance(E(), Iterable) |
| 96 |
True |
| 97 |
|
| 98 |
Complex interfaces do not support this last technique because an |
| 99 |
interface is more than just the presence of method names. Interfaces |
| 100 |
specify semantics and relationships between methods that cannot be |
| 101 |
inferred solely from the presence of specific method names. For |
| 102 |
example, knowing that a class supplies ``__getitem__``, ``__len__``, and |
| 103 |
``__iter__`` is insufficient for distinguishing a :class:`Sequence` from |
| 104 |
a :class:`Mapping`. |
| 105 |
|
| 106 |
.. versionadded:: 3.9 |
| 107 |
These abstract classes now support ``[]``. See :ref:`types-genericalias` |
| 108 |
and :pep:`585`. |
| 109 |
|
| 110 |
.. _collections-abstract-base-classes: |
| 111 |
|
| 112 |
Collections Abstract Base Classes |
| 113 |
--------------------------------- |
| 114 |
|
| 115 |
The collections module offers the following :term:`ABCs <abstract base class>`: |
| 116 |
|
| 117 |
.. tabularcolumns:: |l|L|L|L| |
| 118 |
|
| 119 |
============================== ====================== ======================= ==================================================== |
| 120 |
ABC Inherits from Abstract Methods Mixin Methods |
| 121 |
============================== ====================== ======================= ==================================================== |
| 122 |
:class:`Container` [1]_ ``__contains__`` |
| 123 |
:class:`Hashable` [1]_ ``__hash__`` |
| 124 |
:class:`Iterable` [1]_ [2]_ ``__iter__`` |
| 125 |
:class:`Iterator` [1]_ :class:`Iterable` ``__next__`` ``__iter__`` |
| 126 |
:class:`Reversible` [1]_ :class:`Iterable` ``__reversed__`` |
| 127 |
:class:`Generator` [1]_ :class:`Iterator` ``send``, ``throw`` ``close``, ``__iter__``, ``__next__`` |
| 128 |
:class:`Sized` [1]_ ``__len__`` |
| 129 |
:class:`Callable` [1]_ ``__call__`` |
| 130 |
:class:`Collection` [1]_ :class:`Sized`, ``__contains__``, |
| 131 |
:class:`Iterable`, ``__iter__``, |
| 132 |
:class:`Container` ``__len__`` |
| 133 |
|
| 134 |
:class:`Sequence` :class:`Reversible`, ``__getitem__``, ``__contains__``, ``__iter__``, ``__reversed__``, |
| 135 |
:class:`Collection` ``__len__`` ``index``, and ``count`` |
| 136 |
|
| 137 |
:class:`MutableSequence` :class:`Sequence` ``__getitem__``, Inherited :class:`Sequence` methods and |
| 138 |
``__setitem__``, ``append``, ``clear``, ``reverse``, ``extend``, |
| 139 |
``__delitem__``, ``pop``, ``remove``, and ``__iadd__`` |
| 140 |
``__len__``, |
| 141 |
``insert`` |
| 142 |
|
| 143 |
:class:`ByteString` :class:`Sequence` ``__getitem__``, Inherited :class:`Sequence` methods |
| 144 |
``__len__`` |
| 145 |
|
| 146 |
:class:`Set` :class:`Collection` ``__contains__``, ``__le__``, ``__lt__``, ``__eq__``, ``__ne__``, |
| 147 |
``__iter__``, ``__gt__``, ``__ge__``, ``__and__``, ``__or__``, |
| 148 |
``__len__`` ``__sub__``, ``__rsub__``, ``__xor__``, ``__rxor__`` |
| 149 |
and ``isdisjoint`` |
| 150 |
|
| 151 |
:class:`MutableSet` :class:`Set` ``__contains__``, Inherited :class:`Set` methods and |
| 152 |
``__iter__``, ``clear``, ``pop``, ``remove``, ``__ior__``, |
| 153 |
``__len__``, ``__iand__``, ``__ixor__``, and ``__isub__`` |
| 154 |
``add``, |
| 155 |
``discard`` |
| 156 |
|
| 157 |
:class:`Mapping` :class:`Collection` ``__getitem__``, ``__contains__``, ``keys``, ``items``, ``values``, |
| 158 |
``__iter__``, ``get``, ``__eq__``, and ``__ne__`` |
| 159 |
``__len__`` |
| 160 |
|
| 161 |
:class:`MutableMapping` :class:`Mapping` ``__getitem__``, Inherited :class:`Mapping` methods and |
| 162 |
``__setitem__``, ``pop``, ``popitem``, ``clear``, ``update``, |
| 163 |
``__delitem__``, and ``setdefault`` |
| 164 |
``__iter__``, |
| 165 |
``__len__`` |
| 166 |
|
| 167 |
|
| 168 |
:class:`MappingView` :class:`Sized` ``__init__``, ``__len__`` and ``__repr__`` |
| 169 |
:class:`ItemsView` :class:`MappingView`, ``__contains__``, |
| 170 |
:class:`Set` ``__iter__`` |
| 171 |
:class:`KeysView` :class:`MappingView`, ``__contains__``, |
| 172 |
:class:`Set` ``__iter__`` |
| 173 |
:class:`ValuesView` :class:`MappingView`, ``__contains__``, ``__iter__`` |
| 174 |
:class:`Collection` |
| 175 |
:class:`Awaitable` [1]_ ``__await__`` |
| 176 |
:class:`Coroutine` [1]_ :class:`Awaitable` ``send``, ``throw`` ``close`` |
| 177 |
:class:`AsyncIterable` [1]_ ``__aiter__`` |
| 178 |
:class:`AsyncIterator` [1]_ :class:`AsyncIterable` ``__anext__`` ``__aiter__`` |
| 179 |
:class:`AsyncGenerator` [1]_ :class:`AsyncIterator` ``asend``, ``athrow`` ``aclose``, ``__aiter__``, ``__anext__`` |
| 180 |
:class:`Buffer` [1]_ ``__buffer__`` |
| 181 |
============================== ====================== ======================= ==================================================== |
| 182 |
|
| 183 |
|
| 184 |
.. rubric:: Footnotes |
| 185 |
|
| 186 |
.. [1] These ABCs override :meth:`~abc.ABCMeta.__subclasshook__` to support |
| 187 |
testing an interface by verifying the required methods are present |
| 188 |
and have not been set to :const:`None`. This only works for simple |
| 189 |
interfaces. More complex interfaces require registration or direct |
| 190 |
subclassing. |
| 191 |
|
| 192 |
.. [2] Checking ``isinstance(obj, Iterable)`` detects classes that are |
| 193 |
registered as :class:`Iterable` or that have an :meth:`~container.__iter__` |
| 194 |
method, but it does not detect classes that iterate with the |
| 195 |
:meth:`~object.__getitem__` method. The only reliable way to determine |
| 196 |
whether an object is :term:`iterable` is to call ``iter(obj)``. |
| 197 |
|
| 198 |
|
| 199 |
Collections Abstract Base Classes -- Detailed Descriptions |
| 200 |
---------------------------------------------------------- |
| 201 |
|
| 202 |
|
| 203 |
.. class:: Container |
| 204 |
|
| 205 |
ABC for classes that provide the :meth:`~object.__contains__` method. |
| 206 |
|
| 207 |
.. class:: Hashable |
| 208 |
|
| 209 |
ABC for classes that provide the :meth:`~object.__hash__` method. |
| 210 |
|
| 211 |
.. class:: Sized |
| 212 |
|
| 213 |
ABC for classes that provide the :meth:`~object.__len__` method. |
| 214 |
|
| 215 |
.. class:: Callable |
| 216 |
|
| 217 |
ABC for classes that provide the :meth:`~object.__call__` method. |
| 218 |
|
| 219 |
See :ref:`annotating-callables` for details on how to use |
| 220 |
:class:`!Callable` in type annotations. |
| 221 |
|
| 222 |
.. class:: Iterable |
| 223 |
|
| 224 |
ABC for classes that provide the :meth:`~container.__iter__` method. |
| 225 |
|
| 226 |
Checking ``isinstance(obj, Iterable)`` detects classes that are registered |
| 227 |
as :class:`Iterable` or that have an :meth:`~container.__iter__` method, |
| 228 |
but it does |
| 229 |
not detect classes that iterate with the :meth:`~object.__getitem__` method. |
| 230 |
The only reliable way to determine whether an object is :term:`iterable` |
| 231 |
is to call ``iter(obj)``. |
| 232 |
|
| 233 |
.. class:: Collection |
| 234 |
|
| 235 |
ABC for sized iterable container classes. |
| 236 |
|
| 237 |
.. versionadded:: 3.6 |
| 238 |
|
| 239 |
.. class:: Iterator |
| 240 |
|
| 241 |
ABC for classes that provide the :meth:`~iterator.__iter__` and |
| 242 |
:meth:`~iterator.__next__` methods. See also the definition of |
| 243 |
:term:`iterator`. |
| 244 |
|
| 245 |
.. class:: Reversible |
| 246 |
|
| 247 |
ABC for iterable classes that also provide the :meth:`~object.__reversed__` |
| 248 |
method. |
| 249 |
|
| 250 |
.. versionadded:: 3.6 |
| 251 |
|
| 252 |
.. class:: Generator |
| 253 |
|
| 254 |
ABC for :term:`generator` classes that implement the protocol defined in |
| 255 |
:pep:`342` that extends :term:`iterators <iterator>` with the |
| 256 |
:meth:`~generator.send`, |
| 257 |
:meth:`~generator.throw` and :meth:`~generator.close` methods. |
| 258 |
|
| 259 |
See :ref:`annotating-generators-and-coroutines` |
| 260 |
for details on using :class:`!Generator` in type annotations. |
| 261 |
|
| 262 |
.. versionadded:: 3.5 |
| 263 |
|
| 264 |
.. class:: Sequence |
| 265 |
MutableSequence |
| 266 |
ByteString |
| 267 |
|
| 268 |
ABCs for read-only and mutable :term:`sequences <sequence>`. |
| 269 |
|
| 270 |
Implementation note: Some of the mixin methods, such as |
| 271 |
:meth:`~container.__iter__`, :meth:`~object.__reversed__`, |
| 272 |
and :meth:`~sequence.index` make repeated calls to the underlying |
| 273 |
:meth:`~object.__getitem__` method. |
| 274 |
Consequently, if :meth:`~object.__getitem__` is implemented with constant |
| 275 |
access speed, the mixin methods will have linear performance; |
| 276 |
however, if the underlying method is linear (as it would be with a |
| 277 |
linked list), the mixins will have quadratic performance and will |
| 278 |
likely need to be overridden. |
| 279 |
|
| 280 |
.. method:: index(value, start=0, stop=None) |
| 281 |
|
| 282 |
Return first index of *value*. |
| 283 |
|
| 284 |
Raises :exc:`ValueError` if the value is not present. |
| 285 |
|
| 286 |
Supporting the *start* and *stop* arguments is optional, but recommended. |
| 287 |
|
| 288 |
.. versionchanged:: 3.5 |
| 289 |
The :meth:`~sequence.index` method gained support for |
| 290 |
the *stop* and *start* arguments. |
| 291 |
|
| 292 |
.. deprecated-removed:: 3.12 3.17 |
| 293 |
The :class:`ByteString` ABC has been deprecated. |
| 294 |
|
| 295 |
Use ``isinstance(obj, collections.abc.Buffer)`` to test if ``obj`` |
| 296 |
implements the :ref:`buffer protocol <bufferobjects>` at runtime. For use |
| 297 |
in type annotations, either use :class:`Buffer` or a union that |
| 298 |
explicitly specifies the types your code supports (e.g., |
| 299 |
``bytes | bytearray | memoryview``). |
| 300 |
|
| 301 |
:class:`!ByteString` was originally intended to be an abstract class that |
| 302 |
would serve as a supertype of both :class:`bytes` and :class:`bytearray`. |
| 303 |
However, since the ABC never had any methods, knowing that an object was |
| 304 |
an instance of :class:`!ByteString` never actually told you anything |
| 305 |
useful about the object. Other common buffer types such as |
| 306 |
:class:`memoryview` were also never understood as subtypes of |
| 307 |
:class:`!ByteString` (either at runtime or by static type checkers). |
| 308 |
|
| 309 |
See :pep:`PEP 688 <688#current-options>` for more details. |
| 310 |
|
| 311 |
.. class:: Set |
| 312 |
MutableSet |
| 313 |
|
| 314 |
ABCs for read-only and mutable :ref:`sets <types-set>`. |
| 315 |
|
| 316 |
.. class:: Mapping |
| 317 |
MutableMapping |
| 318 |
|
| 319 |
ABCs for read-only and mutable :term:`mappings <mapping>`. |
| 320 |
|
| 321 |
.. class:: MappingView |
| 322 |
ItemsView |
| 323 |
KeysView |
| 324 |
ValuesView |
| 325 |
|
| 326 |
ABCs for mapping, items, keys, and values :term:`views <dictionary view>`. |
| 327 |
|
| 328 |
.. class:: Awaitable |
| 329 |
|
| 330 |
ABC for :term:`awaitable` objects, which can be used in :keyword:`await` |
| 331 |
expressions. Custom implementations must provide the |
| 332 |
:meth:`~object.__await__` method. |
| 333 |
|
| 334 |
:term:`Coroutine <coroutine>` objects and instances of the |
| 335 |
:class:`~collections.abc.Coroutine` ABC are all instances of this ABC. |
| 336 |
|
| 337 |
.. note:: |
| 338 |
In CPython, generator-based coroutines (:term:`generators <generator>` |
| 339 |
decorated with :deco:`types.coroutine`) are |
| 340 |
*awaitables*, even though they do not have an :meth:`~object.__await__` method. |
| 341 |
Using ``isinstance(gencoro, Awaitable)`` for them will return ``False``. |
| 342 |
Use :func:`inspect.isawaitable` to detect them. |
| 343 |
|
| 344 |
.. versionadded:: 3.5 |
| 345 |
|
| 346 |
.. class:: Coroutine |
| 347 |
|
| 348 |
ABC for :term:`coroutine` compatible classes. These implement the |
| 349 |
following methods, defined in :ref:`coroutine-objects`: |
| 350 |
:meth:`~coroutine.send`, :meth:`~coroutine.throw`, and |
| 351 |
:meth:`~coroutine.close`. Custom implementations must also implement |
| 352 |
:meth:`~object.__await__`. All :class:`Coroutine` instances are also |
| 353 |
instances of :class:`Awaitable`. |
| 354 |
|
| 355 |
.. note:: |
| 356 |
In CPython, generator-based coroutines (:term:`generators <generator>` |
| 357 |
decorated with :deco:`types.coroutine`) are |
| 358 |
*awaitables*, even though they do not have an :meth:`~object.__await__` method. |
| 359 |
Using ``isinstance(gencoro, Coroutine)`` for them will return ``False``. |
| 360 |
Use :func:`inspect.isawaitable` to detect them. |
| 361 |
|
| 362 |
See :ref:`annotating-generators-and-coroutines` |
| 363 |
for details on using :class:`!Coroutine` in type annotations. |
| 364 |
The variance and order of type parameters correspond to those of |
| 365 |
:class:`Generator`. |
| 366 |
|
| 367 |
.. versionadded:: 3.5 |
| 368 |
|
| 369 |
.. class:: AsyncIterable |
| 370 |
|
| 371 |
ABC for classes that provide an ``__aiter__`` method. See also the |
| 372 |
definition of :term:`asynchronous iterable`. |
| 373 |
|
| 374 |
.. versionadded:: 3.5 |
| 375 |
|
| 376 |
.. class:: AsyncIterator |
| 377 |
|
| 378 |
ABC for classes that provide ``__aiter__`` and ``__anext__`` |
| 379 |
methods. See also the definition of :term:`asynchronous iterator`. |
| 380 |
|
| 381 |
.. versionadded:: 3.5 |
| 382 |
|
| 383 |
.. class:: AsyncGenerator |
| 384 |
|
| 385 |
ABC for :term:`asynchronous generator` classes that implement the protocol |
| 386 |
defined in :pep:`525` and :pep:`492`. |
| 387 |
|
| 388 |
See :ref:`annotating-generators-and-coroutines` |
| 389 |
for details on using :class:`!AsyncGenerator` in type annotations. |
| 390 |
|
| 391 |
.. versionadded:: 3.6 |
| 392 |
|
| 393 |
.. class:: Buffer |
| 394 |
|
| 395 |
ABC for classes that provide the :meth:`~object.__buffer__` method, |
| 396 |
implementing the :ref:`buffer protocol <bufferobjects>`. See :pep:`688`. |
| 397 |
|
| 398 |
.. versionadded:: 3.12 |
| 399 |
|
| 400 |
Examples and Recipes |
| 401 |
-------------------- |
| 402 |
|
| 403 |
ABCs allow us to ask classes or instances if they provide |
| 404 |
particular functionality, for example:: |
| 405 |
|
| 406 |
size = None |
| 407 |
if isinstance(myvar, collections.abc.Sized): |
| 408 |
size = len(myvar) |
| 409 |
|
| 410 |
Several of the ABCs are also useful as mixins that make it easier to develop |
| 411 |
classes supporting container APIs. For example, to write a class supporting |
| 412 |
the full :class:`Set` API, it is only necessary to supply the three underlying |
| 413 |
abstract methods: :meth:`~object.__contains__`, :meth:`~container.__iter__`, and |
| 414 |
:meth:`~object.__len__`. The ABC supplies the remaining methods such as |
| 415 |
:meth:`!__and__` and :meth:`~frozenset.isdisjoint`:: |
| 416 |
|
| 417 |
class ListBasedSet(collections.abc.Set): |
| 418 |
''' Alternate set implementation favoring space over speed |
| 419 |
and not requiring the set elements to be hashable. ''' |
| 420 |
def __init__(self, iterable): |
| 421 |
self.elements = lst = [] |
| 422 |
for value in iterable: |
| 423 |
if value not in lst: |
| 424 |
lst.append(value) |
| 425 |
|
| 426 |
def __iter__(self): |
| 427 |
return iter(self.elements) |
| 428 |
|
| 429 |
def __contains__(self, value): |
| 430 |
return value in self.elements |
| 431 |
|
| 432 |
def __len__(self): |
| 433 |
return len(self.elements) |
| 434 |
|
| 435 |
s1 = ListBasedSet('abcdef') |
| 436 |
s2 = ListBasedSet('defghi') |
| 437 |
overlap = s1 & s2 # The __and__() method is supported automatically |
| 438 |
|
| 439 |
Notes on using :class:`Set` and :class:`MutableSet` as a mixin: |
| 440 |
|
| 441 |
(1) |
| 442 |
Since some set operations create new sets, the default mixin methods need |
| 443 |
a way to create new instances from an :term:`iterable`. The class constructor is |
| 444 |
assumed to have a signature in the form ``ClassName(iterable)``. |
| 445 |
That assumption is factored-out to an internal :class:`classmethod` called |
| 446 |
:meth:`!_from_iterable` which calls ``cls(iterable)`` to produce a new set. |
| 447 |
If the :class:`Set` mixin is being used in a class with a different |
| 448 |
constructor signature, you will need to override :meth:`!_from_iterable` |
| 449 |
with a classmethod or regular method that can construct new instances from |
| 450 |
an iterable argument. |
| 451 |
|
| 452 |
(2) |
| 453 |
To override the comparisons (presumably for speed, as the |
| 454 |
semantics are fixed), redefine :meth:`~object.__le__` and |
| 455 |
:meth:`~object.__ge__`, |
| 456 |
then the other operations will automatically follow suit. |
| 457 |
|
| 458 |
(3) |
| 459 |
The :class:`Set` mixin provides a :meth:`!_hash` method to compute a hash value |
| 460 |
for the set; however, :meth:`~object.__hash__` is not defined because not all sets |
| 461 |
are :term:`hashable` or immutable. To add set hashability using mixins, |
| 462 |
inherit from both :class:`Set` and :class:`Hashable`, then define |
| 463 |
``__hash__ = Set._hash``. |
| 464 |
|
| 465 |
.. seealso:: |
| 466 |
|
| 467 |
* `OrderedSet recipe <https://code.activestate.com/recipes/576694/>`_ for an |
| 468 |
example built on :class:`MutableSet`. |
| 469 |
|
| 470 |
* For more about ABCs, see the :mod:`abc` module and :pep:`3119`. |