Debian Codegraph @

Doc/library/importlib.metadata.rst

raw
1 .. _using:
2
3 ========================================================
4 :mod:`!importlib.metadata` -- Accessing package metadata
5 ========================================================
6
7 .. module:: importlib.metadata
8 :synopsis: Accessing package metadata
9
10 .. versionadded:: 3.8
11 .. versionchanged:: 3.10
12 ``importlib.metadata`` is no longer provisional.
13
14 **Source code:** :source:`Lib/importlib/metadata/__init__.py`
15
16 ``importlib.metadata`` is a library that provides access to
17 the metadata of an installed `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_,
18 such as its entry points
19 or its top-level names (`Import Package <https://packaging.python.org/en/latest/glossary/#term-Import-Package>`_\s, modules, if any).
20 Built in part on Python's import system, this library
21 provides the entry point and metadata APIs that were previously
22 exposed by the now-removed ``pkg_resources`` package. Along with
23 :mod:`importlib.resources`, it supersedes ``pkg_resources``.
24
25 ``importlib.metadata`` operates on third-party *distribution packages*
26 installed into Python's ``site-packages`` directory via tools such as
27 :pypi:`pip`.
28 Specifically, it works with distributions with discoverable
29 ``dist-info`` or ``egg-info`` directories,
30 and metadata defined by the `Core metadata specifications <https://packaging.python.org/en/latest/specifications/core-metadata/#core-metadata>`_.
31
32 .. important::
33
34 These are *not* necessarily equivalent to or correspond 1:1 with
35 the top-level *import package* names
36 that can be imported inside Python code.
37 One *distribution package* can contain multiple *import packages*
38 (and single modules),
39 and one top-level *import package*
40 may map to multiple *distribution packages*
41 if it is a namespace package.
42 You can use :ref:`packages_distributions() <package-distributions>`
43 to get a mapping between them.
44
45 By default, distribution metadata can live on the file system
46 or in zip archives on
47 :data:`sys.path`. Through an extension mechanism, the metadata can live almost
48 anywhere.
49
50
51 .. seealso::
52
53 https://importlib-metadata.readthedocs.io/
54 The documentation for ``importlib_metadata``, which supplies a
55 backport of ``importlib.metadata``.
56 This includes an `API reference
57 <https://importlib-metadata.readthedocs.io/en/latest/api.html>`__
58 for this module's classes and functions,
59 as well as a `migration guide
60 <https://importlib-metadata.readthedocs.io/en/latest/migration.html>`__
61 for existing users of ``pkg_resources``.
62
63
64 Overview
65 ========
66
67 Let's say you wanted to get the version string for a
68 `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_ you've installed
69 using ``pip``. We start by creating a virtual environment and installing
70 something into it:
71
72 .. code-block:: shell-session
73
74 $ python -m venv example
75 $ source example/bin/activate
76 (example) $ python -m pip install wheel
77
78 You can get the version string for ``wheel`` by running the following:
79
80 .. code-block:: pycon
81
82 (example) $ python
83 >>> from importlib.metadata import version # doctest: +SKIP
84 >>> version('wheel') # doctest: +SKIP
85 '0.32.3'
86
87 You can also get a collection of entry points selectable by properties of the EntryPoint (typically 'group' or 'name'), such as
88 ``console_scripts``, ``distutils.commands`` and others. Each group contains a
89 collection of :ref:`EntryPoint <entry-points>` objects.
90
91 You can get the :ref:`metadata for a distribution <metadata>`::
92
93 >>> list(metadata('wheel')) # doctest: +SKIP
94 ['Metadata-Version', 'Name', 'Version', 'Summary', 'Home-page', 'Author', 'Author-email', 'Maintainer', 'Maintainer-email', 'License', 'Project-URL', 'Project-URL', 'Project-URL', 'Keywords', 'Platform', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Requires-Python', 'Provides-Extra', 'Requires-Dist', 'Requires-Dist']
95
96 You can also get a :ref:`distribution's version number <version>`, list its
97 :ref:`constituent files <files>`, and get a list of the distribution's
98 :ref:`requirements`.
99
100
101 .. exception:: PackageNotFoundError
102
103 Subclass of :class:`ModuleNotFoundError` raised by several functions in this
104 module when queried for a distribution package which is not installed in the
105 current Python environment.
106
107
108 Functional API
109 ==============
110
111 This package provides the following functionality via its public API.
112
113
114 .. _entry-points:
115
116 Entry points
117 ------------
118
119 .. function:: entry_points(**select_params)
120
121 Returns a :class:`EntryPoints` instance describing entry points for the
122 current environment. Any given keyword parameters are passed to the
123 :meth:`!select` method for comparison to the attributes of
124 the individual entry point definitions.
125
126 Note: it is not currently possible to query for entry points based on
127 their :attr:`!EntryPoint.dist` attribute (as different :class:`!Distribution`
128 instances do not currently compare equal, even if they have the same attributes)
129
130 .. class:: EntryPoints
131
132 Details of a collection of installed entry points.
133
134 Also provides a ``.groups`` attribute that reports all identified entry
135 point groups, and a ``.names`` attribute that reports all identified entry
136 point names.
137
138 .. class:: EntryPoint
139
140 Details of an installed entry point.
141
142 Each :class:`!EntryPoint` instance has ``.name``, ``.group``, and ``.value``
143 attributes and a ``.load()`` method to resolve the value. There are also
144 ``.module``, ``.attr``, and ``.extras`` attributes for getting the
145 components of the ``.value`` attribute, and ``.dist`` for obtaining
146 information regarding the distribution package that provides the entry point.
147
148 Query all entry points::
149
150 >>> eps = entry_points() # doctest: +SKIP
151
152 The :func:`!entry_points` function returns a :class:`!EntryPoints` object,
153 a collection of all :class:`!EntryPoint` objects with ``names`` and ``groups``
154 attributes for convenience::
155
156 >>> sorted(eps.groups) # doctest: +SKIP
157 ['console_scripts', 'distutils.commands', 'distutils.setup_keywords', 'egg_info.writers', 'setuptools.installation']
158
159 :class:`!EntryPoints` has a :meth:`!select` method to select entry points
160 matching specific properties. Select entry points in the
161 ``console_scripts`` group::
162
163 >>> scripts = eps.select(group='console_scripts') # doctest: +SKIP
164
165 Equivalently, since :func:`!entry_points` passes keyword arguments
166 through to select::
167
168 >>> scripts = entry_points(group='console_scripts') # doctest: +SKIP
169
170 Pick out a specific script named "wheel" (found in the wheel project)::
171
172 >>> 'wheel' in scripts.names # doctest: +SKIP
173 True
174 >>> wheel = scripts['wheel'] # doctest: +SKIP
175
176 Equivalently, query for that entry point during selection::
177
178 >>> (wheel,) = entry_points(group='console_scripts', name='wheel') # doctest: +SKIP
179 >>> (wheel,) = entry_points().select(group='console_scripts', name='wheel') # doctest: +SKIP
180
181 Inspect the resolved entry point::
182
183 >>> wheel # doctest: +SKIP
184 EntryPoint(name='wheel', value='wheel.cli:main', group='console_scripts')
185 >>> wheel.module # doctest: +SKIP
186 'wheel.cli'
187 >>> wheel.attr # doctest: +SKIP
188 'main'
189 >>> wheel.extras # doctest: +SKIP
190 []
191 >>> main = wheel.load() # doctest: +SKIP
192 >>> main # doctest: +SKIP
193 <function main at 0x103528488>
194
195 The ``group`` and ``name`` are arbitrary values defined by the package author
196 and usually a client will wish to resolve all entry points for a particular
197 group. Read `the setuptools docs
198 <https://setuptools.pypa.io/en/latest/userguide/entry_point.html>`_
199 for more information on entry points, their definition, and usage.
200
201 .. versionchanged:: 3.12
202 The "selectable" entry points were introduced in ``importlib_metadata``
203 3.6 and Python 3.10. Prior to those changes, ``entry_points`` accepted
204 no parameters and always returned a dictionary of entry points, keyed
205 by group. With ``importlib_metadata`` 5.0 and Python 3.12,
206 ``entry_points`` always returns an ``EntryPoints`` object. See
207 :pypi:`backports.entry_points_selectable`
208 for compatibility options.
209
210 .. versionchanged:: 3.13
211 ``EntryPoint`` objects no longer present a tuple-like interface
212 (:meth:`~object.__getitem__`).
213
214 .. _metadata:
215
216 Distribution metadata
217 ---------------------
218
219 .. function:: metadata(distribution_name)
220
221 Return the distribution metadata corresponding to the named
222 distribution package as a :class:`PackageMetadata` instance.
223
224 Raises :exc:`PackageNotFoundError` if the named distribution
225 package is not installed in the current Python environment.
226
227 .. class:: PackageMetadata
228
229 A concrete implementation of the
230 `PackageMetadata protocol <https://importlib-metadata.readthedocs.io/en/latest/api.html#importlib_metadata.PackageMetadata>`_.
231
232 In addition to providing the defined protocol methods and attributes, subscripting
233 the instance is equivalent to calling the :meth:`!get` method.
234
235 Every `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_
236 includes some metadata, which you can extract using the :func:`!metadata` function::
237
238 >>> wheel_metadata = metadata('wheel') # doctest: +SKIP
239
240 The keys of the returned data structure name the metadata keywords, and
241 the values are returned unparsed from the distribution metadata::
242
243 >>> wheel_metadata['Requires-Python'] # doctest: +SKIP
244 '>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*'
245
246 :class:`PackageMetadata` also presents a :attr:`!json` attribute that returns
247 all the metadata in a JSON-compatible form per :PEP:`566`::
248
249 >>> wheel_metadata.json['requires_python']
250 '>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*'
251
252 The full set of available metadata is not described here.
253 See the PyPA `Core metadata specification <https://packaging.python.org/en/latest/specifications/core-metadata/#core-metadata>`_ for additional details.
254
255 .. versionchanged:: 3.10
256 The ``Description`` is now included in the metadata when presented
257 through the payload. Line continuation characters have been removed.
258
259 The ``json`` attribute was added.
260
261
262 .. _version:
263
264 Distribution versions
265 ---------------------
266
267 .. function:: version(distribution_name)
268
269 Return the installed distribution package
270 `version <https://packaging.python.org/en/latest/specifications/core-metadata/#version>`__
271 for the named distribution package.
272
273 Raises :exc:`PackageNotFoundError` if the named distribution
274 package is not installed in the current Python environment.
275
276 The :func:`!version` function is the quickest way to get a
277 `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_'s version
278 number, as a string::
279
280 >>> version('wheel') # doctest: +SKIP
281 '0.32.3'
282
283
284 .. _files:
285
286 Distribution files
287 ------------------
288
289 .. function:: files(distribution_name)
290
291 Return the full set of files contained within the named
292 distribution package.
293
294 Raises :exc:`PackageNotFoundError` if the named distribution
295 package is not installed in the current Python environment.
296
297 Returns :const:`None` if the distribution is found but the installation
298 database records reporting the files associated with the distribution package
299 are missing.
300
301 .. class:: PackagePath
302
303 A :class:`pathlib.PurePath` derived object with additional ``dist``,
304 ``size``, and ``hash`` properties corresponding to the distribution
305 package's installation metadata for that file.
306
307 The :func:`!files` function takes a
308 `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_
309 name and returns all of the files installed by this distribution. Each file is reported
310 as a :class:`PackagePath` instance. For example::
311
312 >>> util = [p for p in files('wheel') if 'util.py' in str(p)][0] # doctest: +SKIP
313 >>> util # doctest: +SKIP
314 PackagePath('wheel/util.py')
315 >>> util.size # doctest: +SKIP
316 859
317 >>> util.dist # doctest: +SKIP
318 <importlib.metadata._hooks.PathDistribution object at 0x101e0cef0>
319 >>> util.hash # doctest: +SKIP
320 <FileHash mode: sha256 value: bYkw5oMccfazVCoYQwKkkemoVyMAFoR34mmKBx8R1NI>
321
322 Once you have the file, you can also read its contents::
323
324 >>> print(util.read_text()) # doctest: +SKIP
325 import base64
326 import sys
327 ...
328 def as_bytes(s):
329 if isinstance(s, text_type):
330 return s.encode('utf-8')
331 return s
332
333 You can also use the :meth:`!locate` method to get the absolute
334 path to the file::
335
336 >>> util.locate() # doctest: +SKIP
337 PosixPath('/home/gustav/example/lib/site-packages/wheel/util.py')
338
339 In the case where the metadata file listing files
340 (``RECORD`` or ``SOURCES.txt``) is missing, :func:`!files` will
341 return :const:`None`. The caller may wish to wrap calls to
342 :func:`!files` in `always_iterable
343 <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.always_iterable>`_
344 or otherwise guard against this condition if the target
345 distribution is not known to have the metadata present.
346
347 .. _requirements:
348
349 Distribution requirements
350 -------------------------
351
352 .. function:: requires(distribution_name)
353
354 Return the declared dependency specifiers for the named
355 distribution package.
356
357 Raises :exc:`PackageNotFoundError` if the named distribution
358 package is not installed in the current Python environment.
359
360 To get the full set of requirements for a `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_,
361 use the :func:`!requires`
362 function::
363
364 >>> requires('wheel') # doctest: +SKIP
365 ["pytest (>=3.0.0) ; extra == 'test'", "pytest-cov ; extra == 'test'"]
366
367
368 .. _package-distributions:
369 .. _import-distribution-package-mapping:
370
371 Mapping import to distribution packages
372 ---------------------------------------
373
374 .. function:: packages_distributions()
375
376 Return a mapping from the top level module and import package
377 names found via :data:`sys.meta_path` to the names of the distribution
378 packages (if any) that provide the corresponding files.
379
380 To allow for namespace packages (which may have members provided by
381 multiple distribution packages), each top level import name maps to a
382 list of distribution names rather than mapping directly to a single name.
383
384 A convenience method to resolve the `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_
385 name (or names, in the case of a namespace package)
386 that provide each importable top-level
387 Python module or `Import Package <https://packaging.python.org/en/latest/glossary/#term-Import-Package>`_::
388
389 >>> packages_distributions()
390 {'importlib_metadata': ['importlib-metadata'], 'yaml': ['PyYAML'], 'jaraco': ['jaraco.classes', 'jaraco.functools'], ...}
391
392 Some editable installs, `do not supply top-level names
393 <https://github.com/pypa/packaging-problems/issues/609>`_, and thus this
394 function is not reliable with such installs.
395
396 .. versionadded:: 3.10
397
398 .. _distributions:
399
400 Distributions
401 =============
402
403 .. function:: distribution(distribution_name)
404
405 Return a :class:`Distribution` instance describing the named
406 distribution package.
407
408 Raises :exc:`PackageNotFoundError` if the named distribution
409 package is not installed in the current Python environment.
410
411 .. class:: Distribution
412
413 Details of an installed distribution package.
414
415 Note: different :class:`!Distribution` instances do not currently compare
416 equal, even if they relate to the same installed distribution and
417 accordingly have the same attributes.
418
419 While the module level API described above is the most common and convenient usage,
420 you can get all of that information from the :class:`!Distribution` class.
421 :class:`!Distribution` is an abstract object that represents the metadata for
422 a Python `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_.
423 You can get the concrete :class:`!Distribution` subclass instance for an installed
424 distribution package by calling the :func:`distribution` function::
425
426 >>> from importlib.metadata import distribution # doctest: +SKIP
427 >>> dist = distribution('wheel') # doctest: +SKIP
428 >>> type(dist) # doctest: +SKIP
429 <class 'importlib.metadata.PathDistribution'>
430
431 Thus, an alternative way to get the version number is through the
432 :class:`!Distribution` instance::
433
434 >>> dist.version # doctest: +SKIP
435 '0.32.3'
436
437 There are all kinds of additional metadata available on :class:`!Distribution`
438 instances::
439
440 >>> dist.metadata['Requires-Python'] # doctest: +SKIP
441 '>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*'
442 >>> dist.metadata['License'] # doctest: +SKIP
443 'MIT'
444
445 For editable packages, an ``origin`` property may present :pep:`610`
446 metadata::
447
448 >>> dist.origin.url
449 'file:///path/to/wheel-0.32.3.editable-py3-none-any.whl'
450
451 The full set of available metadata is not described here.
452 See the PyPA `Core metadata specification <https://packaging.python.org/en/latest/specifications/core-metadata/#core-metadata>`_ for additional details.
453
454 .. versionadded:: 3.13
455 The ``.origin`` property was added.
456
457 Distribution Discovery
458 ======================
459
460 By default, this package provides built-in support for discovery of metadata
461 for file system and zip file `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_\s.
462 This metadata finder search defaults to ``sys.path``, but varies slightly in how it interprets those values from how other import machinery does. In particular:
463
464 - ``importlib.metadata`` does not honor :class:`bytes` objects on ``sys.path``.
465 - ``importlib.metadata`` will incidentally honor :py:class:`pathlib.Path` objects on ``sys.path`` even though such values will be ignored for imports.
466
467
468 Implementing Custom Providers
469 =============================
470
471 ``importlib.metadata`` address two API surfaces, one for *consumers*
472 and another for *providers*. Most users are consumers, consuming
473 metadata provided by the packages. There are other use-cases, however,
474 where users wish to expose metadata through some other mechanism,
475 such as alongside a custom importer. Such a use case calls for a
476 *custom provider*.
477
478 Because `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_ metadata
479 is not available through :data:`sys.path` searches, or
480 package loaders directly,
481 the metadata for a distribution is found through import
482 system :ref:`finders <finders-and-loaders>`. To find a distribution package's metadata,
483 ``importlib.metadata`` queries the list of :term:`meta path finders <meta path finder>` on
484 :data:`sys.meta_path`.
485
486 The implementation has hooks integrated into the ``PathFinder``,
487 serving metadata for distribution packages found on the file system.
488
489 The abstract class :py:class:`importlib.abc.MetaPathFinder` defines the
490 interface expected of finders by Python's import system.
491 ``importlib.metadata`` extends this protocol by looking for an optional
492 ``find_distributions`` callable on the finders from
493 :data:`sys.meta_path` and presents this extended interface as the
494 ``DistributionFinder`` abstract base class, which defines this abstract
495 method::
496
497 @abc.abstractmethod
498 def find_distributions(context=DistributionFinder.Context()) -> Iterable[Distribution]:
499 """Return an iterable of all Distribution instances capable of
500 loading the metadata for packages for the indicated ``context``.
501 """
502
503 The ``DistributionFinder.Context`` object provides ``.path`` and ``.name``
504 properties indicating the path to search and name to match and may
505 supply other relevant context sought by the consumer.
506
507 In practice, to support finding distribution package
508 metadata in locations other than the file system, subclass
509 ``Distribution`` and implement the abstract methods. Then from
510 a custom finder, return instances of this derived ``Distribution`` in the
511 ``find_distributions()`` method.
512
513 Example
514 -------
515
516 Imagine a custom finder that loads Python modules from a database::
517
518 class DatabaseImporter(importlib.abc.MetaPathFinder):
519 def __init__(self, db):
520 self.db = db
521
522 def find_spec(self, fullname, target=None) -> ModuleSpec:
523 return self.db.spec_from_name(fullname)
524
525 sys.meta_path.append(DatabaseImporter(connect_db(...)))
526
527 That importer now presumably provides importable modules from a
528 database, but it provides no metadata or entry points. For this
529 custom importer to provide metadata, it would also need to implement
530 ``DistributionFinder``::
531
532 from importlib.metadata import DistributionFinder
533
534 class DatabaseImporter(DistributionFinder):
535 ...
536
537 def find_distributions(self, context=DistributionFinder.Context()):
538 query = dict(name=context.name) if context.name else {}
539 for dist_record in self.db.query_distributions(query):
540 yield DatabaseDistribution(dist_record)
541
542 In this way, ``query_distributions`` would return records for
543 each distribution served by the database matching the query. For
544 example, if ``requests-1.0`` is in the database, ``find_distributions``
545 would yield a ``DatabaseDistribution`` for ``Context(name='requests')``
546 or ``Context(name=None)``.
547
548 For the sake of simplicity, this example ignores ``context.path``\. The
549 ``path`` attribute defaults to ``sys.path`` and is the set of import paths to
550 be considered in the search. A ``DatabaseImporter`` could potentially function
551 without any concern for a search path. Assuming the importer does no
552 partitioning, the "path" would be irrelevant. In order to illustrate the
553 purpose of ``path``, the example would need to illustrate a more complex
554 ``DatabaseImporter`` whose behavior varied depending on
555 ``sys.path``/``PYTHONPATH``. In that case, the ``find_distributions`` should
556 honor the ``context.path`` and only yield ``Distribution``\ s pertinent to that
557 path.
558
559 ``DatabaseDistribution``, then, would look something like::
560
561 class DatabaseDistribution(importlib.metadata.Distribution):
562 def __init__(self, record):
563 self.record = record
564
565 def read_text(self, filename):
566 """
567 Read a file like "METADATA" for the current distribution.
568 """
569 if filename == "METADATA":
570 return f"""Name: {self.record.name}
571 Version: {self.record.version}
572 """
573 if filename == "entry_points.txt":
574 return "\n".join(
575 f"""[{ep.group}]\n{ep.name}={ep.value}"""
576 for ep in self.record.entry_points)
577
578 def locate_file(self, path):
579 raise RuntimeError("This distribution has no file system")
580
581 This basic implementation should provide metadata and entry points for
582 packages served by the ``DatabaseImporter``, assuming that the
583 ``record`` supplies suitable ``.name``, ``.version``, and
584 ``.entry_points`` attributes.
585
586 The ``DatabaseDistribution`` may also provide other metadata files, like
587 ``RECORD`` (required for ``Distribution.files``) or override the
588 implementation of ``Distribution.files``. See the source for more inspiration.