| 1 |
|
| 2 |
.. _importsystem: |
| 3 |
|
| 4 |
***************** |
| 5 |
The import system |
| 6 |
***************** |
| 7 |
|
| 8 |
.. index:: single: import machinery |
| 9 |
|
| 10 |
Python code in one :term:`module` gains access to the code in another module |
| 11 |
by the process of :term:`importing` it. The :keyword:`import` statement is |
| 12 |
the most common way of invoking the import machinery, but it is not the only |
| 13 |
way. Functions such as :func:`importlib.import_module` and built-in |
| 14 |
:func:`__import__` can also be used to invoke the import machinery. |
| 15 |
|
| 16 |
The :keyword:`import` statement combines two operations; it searches for the |
| 17 |
named module, then it binds the results of that search to a name in the local |
| 18 |
scope. The search operation of the :keyword:`!import` statement is defined as |
| 19 |
a call to the :func:`__import__` function, with the appropriate arguments. |
| 20 |
The return value of :func:`__import__` is used to perform the name |
| 21 |
binding operation of the :keyword:`!import` statement. See the |
| 22 |
:keyword:`!import` statement for the exact details of that name binding |
| 23 |
operation. |
| 24 |
|
| 25 |
A direct call to :func:`__import__` performs only the module search and, if |
| 26 |
found, the module creation operation. While certain side-effects may occur, |
| 27 |
such as the importing of parent packages, and the updating of various caches |
| 28 |
(including :data:`sys.modules`), only the :keyword:`import` statement performs |
| 29 |
a name binding operation. |
| 30 |
|
| 31 |
When an :keyword:`import` statement is executed, the standard builtin |
| 32 |
:func:`__import__` function is called. Other mechanisms for invoking the |
| 33 |
import system (such as :func:`importlib.import_module`) may choose to bypass |
| 34 |
:func:`__import__` and use their own solutions to implement import semantics. |
| 35 |
|
| 36 |
When a module is first imported, Python searches for the module and if found, |
| 37 |
it creates a module object [#fnmo]_, initializing it. If the named module |
| 38 |
cannot be found, a :exc:`ModuleNotFoundError` is raised. Python implements various |
| 39 |
strategies to search for the named module when the import machinery is |
| 40 |
invoked. These strategies can be modified and extended by using various hooks |
| 41 |
described in the sections below. |
| 42 |
|
| 43 |
.. versionchanged:: 3.3 |
| 44 |
The import system has been updated to fully implement the second phase |
| 45 |
of :pep:`302`. There is no longer any implicit import machinery - the full |
| 46 |
import system is exposed through :data:`sys.meta_path`. In addition, |
| 47 |
native namespace package support has been implemented (see :pep:`420`). |
| 48 |
|
| 49 |
|
| 50 |
:mod:`importlib` |
| 51 |
================ |
| 52 |
|
| 53 |
The :mod:`importlib` module provides a rich API for interacting with the |
| 54 |
import system. For example :func:`importlib.import_module` provides a |
| 55 |
recommended, simpler API than built-in :func:`__import__` for invoking the |
| 56 |
import machinery. Refer to the :mod:`importlib` library documentation for |
| 57 |
additional detail. |
| 58 |
|
| 59 |
|
| 60 |
|
| 61 |
Packages |
| 62 |
======== |
| 63 |
|
| 64 |
.. index:: |
| 65 |
single: package |
| 66 |
|
| 67 |
Python has only one type of module object, and all modules are of this type, |
| 68 |
regardless of whether the module is implemented in Python, C, or something |
| 69 |
else. To help organize modules and provide a naming hierarchy, Python has a |
| 70 |
concept of :term:`packages <package>`. |
| 71 |
|
| 72 |
You can think of packages as the directories on a file system and modules as |
| 73 |
files within directories, but don't take this analogy too literally since |
| 74 |
packages and modules need not originate from the file system. For the |
| 75 |
purposes of this documentation, we'll use this convenient analogy of |
| 76 |
directories and files. Like file system directories, packages are organized |
| 77 |
hierarchically, and packages may themselves contain subpackages, as well as |
| 78 |
regular modules. |
| 79 |
|
| 80 |
It's important to keep in mind that all packages are modules, but not all |
| 81 |
modules are packages. Or put another way, packages are just a special kind of |
| 82 |
module. Specifically, any module that contains a ``__path__`` attribute is |
| 83 |
considered a package. |
| 84 |
|
| 85 |
All modules have a name. Subpackage names are separated from their parent |
| 86 |
package name by a dot, akin to Python's standard attribute access syntax. Thus |
| 87 |
you might have a package called :mod:`email`, which in turn has a subpackage |
| 88 |
called :mod:`email.mime` and a module within that subpackage called |
| 89 |
:mod:`email.mime.text`. |
| 90 |
|
| 91 |
|
| 92 |
Regular packages |
| 93 |
---------------- |
| 94 |
|
| 95 |
.. index:: |
| 96 |
pair: package; regular |
| 97 |
|
| 98 |
Python defines two types of packages, :term:`regular packages <regular |
| 99 |
package>` and :term:`namespace packages <namespace package>`. Regular |
| 100 |
packages are traditional packages as they existed in Python 3.2 and earlier. |
| 101 |
A regular package is typically implemented as a directory containing an |
| 102 |
``__init__.py`` file. When a regular package is imported, this |
| 103 |
``__init__.py`` file is implicitly executed, and the objects it defines are |
| 104 |
bound to names in the package's namespace. The ``__init__.py`` file can |
| 105 |
contain the same Python code that any other module can contain, and Python |
| 106 |
will add some additional attributes to the module when it is imported. |
| 107 |
|
| 108 |
For example, the following file system layout defines a top level ``parent`` |
| 109 |
package with three subpackages:: |
| 110 |
|
| 111 |
parent/ |
| 112 |
__init__.py |
| 113 |
one/ |
| 114 |
__init__.py |
| 115 |
two/ |
| 116 |
__init__.py |
| 117 |
three/ |
| 118 |
__init__.py |
| 119 |
|
| 120 |
Importing ``parent.one`` will implicitly execute ``parent/__init__.py`` and |
| 121 |
``parent/one/__init__.py``. Subsequent imports of ``parent.two`` or |
| 122 |
``parent.three`` will execute ``parent/two/__init__.py`` and |
| 123 |
``parent/three/__init__.py`` respectively. |
| 124 |
|
| 125 |
A subdirectory inside a regular package that does not contain an |
| 126 |
``__init__.py`` file is treated as an implicit |
| 127 |
:ref:`namespace package <reference-namespace-package>` (a "namespace |
| 128 |
subpackage") rooted in that parent. See :pep:`420` for the underlying |
| 129 |
specification. |
| 130 |
|
| 131 |
|
| 132 |
.. _reference-namespace-package: |
| 133 |
|
| 134 |
Namespace packages |
| 135 |
------------------ |
| 136 |
|
| 137 |
.. index:: |
| 138 |
pair: package; namespace |
| 139 |
pair: package; portion |
| 140 |
|
| 141 |
A namespace package is a composite of various :term:`portions <portion>`, |
| 142 |
where each portion contributes a subpackage to the parent package. Portions |
| 143 |
may reside in different locations on the file system. Portions may also be |
| 144 |
found in zip files, on the network, or anywhere else that Python searches |
| 145 |
during import. Namespace packages may or may not correspond directly to |
| 146 |
objects on the file system; they may be virtual modules that have no concrete |
| 147 |
representation. |
| 148 |
|
| 149 |
Namespace packages do not use an ordinary list for their ``__path__`` |
| 150 |
attribute. They instead use a custom iterable type which will automatically |
| 151 |
perform a new search for package portions on the next import attempt within |
| 152 |
that package if the path of their parent package (or :data:`sys.path` for a |
| 153 |
top level package) changes. |
| 154 |
|
| 155 |
With namespace packages, there is no ``parent/__init__.py`` file. In fact, |
| 156 |
there may be multiple ``parent`` directories found during import search, where |
| 157 |
each one is provided by a different portion. Thus ``parent/one`` may not be |
| 158 |
physically located next to ``parent/two``. In this case, Python will create a |
| 159 |
namespace package for the top-level ``parent`` package whenever it or one of |
| 160 |
its subpackages is imported. |
| 161 |
|
| 162 |
Namespace packages may also be nested inside a regular package. When the |
| 163 |
import system searches a regular package's ``__path__`` and encounters a |
| 164 |
subdirectory that does not contain an ``__init__.py`` file, that |
| 165 |
subdirectory becomes a :term:`portion` contributing to a namespace |
| 166 |
subpackage of the enclosing regular package. |
| 167 |
|
| 168 |
See also :pep:`420` for the namespace package specification. |
| 169 |
|
| 170 |
|
| 171 |
Searching |
| 172 |
========= |
| 173 |
|
| 174 |
To begin the search, Python needs the :term:`fully qualified <qualified name>` |
| 175 |
name of the module (or package, but for the purposes of this discussion, the |
| 176 |
difference is immaterial) being imported. This name may come from various |
| 177 |
arguments to the :keyword:`import` statement, or from the parameters to the |
| 178 |
:func:`importlib.import_module` or :func:`__import__` functions. |
| 179 |
|
| 180 |
This name will be used in various phases of the import search, and it may be |
| 181 |
the dotted path to a submodule, e.g. ``foo.bar.baz``. In this case, Python |
| 182 |
first tries to import ``foo``, then ``foo.bar``, and finally ``foo.bar.baz``. |
| 183 |
If any of the intermediate imports fail, a :exc:`ModuleNotFoundError` is raised. |
| 184 |
|
| 185 |
|
| 186 |
The module cache |
| 187 |
---------------- |
| 188 |
|
| 189 |
.. index:: |
| 190 |
single: sys.modules |
| 191 |
|
| 192 |
The first place checked during import search is :data:`sys.modules`. This |
| 193 |
mapping serves as a cache of all modules that have been previously imported, |
| 194 |
including the intermediate paths. So if ``foo.bar.baz`` was previously |
| 195 |
imported, :data:`sys.modules` will contain entries for ``foo``, ``foo.bar``, |
| 196 |
and ``foo.bar.baz``. Each key will have as its value the corresponding module |
| 197 |
object. |
| 198 |
|
| 199 |
During import, the module name is looked up in :data:`sys.modules` and if |
| 200 |
present, the associated value is the module satisfying the import, and the |
| 201 |
process completes. However, if the value is ``None``, then a |
| 202 |
:exc:`ModuleNotFoundError` is raised. If the module name is missing, Python will |
| 203 |
continue searching for the module. |
| 204 |
|
| 205 |
:data:`sys.modules` is writable. Deleting a key may not destroy the |
| 206 |
associated module (as other modules may hold references to it), |
| 207 |
but it will invalidate the cache entry for the named module, causing |
| 208 |
Python to search anew for the named module upon its next |
| 209 |
import. The key can also be assigned to ``None``, forcing the next import |
| 210 |
of the module to result in a :exc:`ModuleNotFoundError`. |
| 211 |
|
| 212 |
Beware though, as if you keep a reference to the module object, |
| 213 |
invalidate its cache entry in :data:`sys.modules`, and then re-import the |
| 214 |
named module, the two module objects will *not* be the same. By contrast, |
| 215 |
:func:`importlib.reload` will reuse the *same* module object, and simply |
| 216 |
reinitialise the module contents by rerunning the module's code. |
| 217 |
|
| 218 |
|
| 219 |
.. _finders-and-loaders: |
| 220 |
|
| 221 |
Finders and loaders |
| 222 |
------------------- |
| 223 |
|
| 224 |
.. index:: |
| 225 |
single: finder |
| 226 |
single: loader |
| 227 |
single: module spec |
| 228 |
|
| 229 |
If the named module is not found in :data:`sys.modules`, then Python's import |
| 230 |
protocol is invoked to find and load the module. This protocol consists of |
| 231 |
two conceptual objects, :term:`finders <finder>` and :term:`loaders <loader>`. |
| 232 |
A finder's job is to determine whether it can find the named module using |
| 233 |
whatever strategy it knows about. Objects that implement both of these |
| 234 |
interfaces are referred to as :term:`importers <importer>` - they return |
| 235 |
themselves when they find that they can load the requested module. |
| 236 |
|
| 237 |
Python includes a number of default finders and importers. The first one |
| 238 |
knows how to locate built-in modules, and the second knows how to locate |
| 239 |
frozen modules. A third default finder searches an :term:`import path` |
| 240 |
for modules. The :term:`import path` is a list of locations that may |
| 241 |
name file system paths or zip files. It can also be extended to search |
| 242 |
for any locatable resource, such as those identified by URLs. |
| 243 |
|
| 244 |
The import machinery is extensible, so new finders can be added to extend the |
| 245 |
range and scope of module searching. |
| 246 |
|
| 247 |
Finders do not actually load modules. If they can find the named module, they |
| 248 |
return a :dfn:`module spec`, an encapsulation of the module's import-related |
| 249 |
information, which the import machinery then uses when loading the module. |
| 250 |
|
| 251 |
The following sections describe the protocol for finders and loaders in more |
| 252 |
detail, including how you can create and register new ones to extend the |
| 253 |
import machinery. |
| 254 |
|
| 255 |
.. versionchanged:: 3.4 |
| 256 |
In previous versions of Python, finders returned :term:`loaders <loader>` |
| 257 |
directly, whereas now they return module specs which *contain* loaders. |
| 258 |
Loaders are still used during import but have fewer responsibilities. |
| 259 |
|
| 260 |
Import hooks |
| 261 |
------------ |
| 262 |
|
| 263 |
.. index:: |
| 264 |
single: import hooks |
| 265 |
single: meta hooks |
| 266 |
single: path hooks |
| 267 |
pair: hooks; import |
| 268 |
pair: hooks; meta |
| 269 |
pair: hooks; path |
| 270 |
|
| 271 |
The import machinery is designed to be extensible; the primary mechanism for |
| 272 |
this are the *import hooks*. There are two types of import hooks: *meta |
| 273 |
hooks* and *import path hooks*. |
| 274 |
|
| 275 |
Meta hooks are called at the start of import processing, before any other |
| 276 |
import processing has occurred, other than :data:`sys.modules` cache look up. |
| 277 |
This allows meta hooks to override :data:`sys.path` processing, frozen |
| 278 |
modules, or even built-in modules. Meta hooks are registered by adding new |
| 279 |
finder objects to :data:`sys.meta_path`, as described below. |
| 280 |
|
| 281 |
Import path hooks are called as part of :data:`sys.path` (or |
| 282 |
``package.__path__``) processing, at the point where their associated path |
| 283 |
item is encountered. Import path hooks are registered by adding new callables |
| 284 |
to :data:`sys.path_hooks` as described below. |
| 285 |
|
| 286 |
|
| 287 |
The meta path |
| 288 |
------------- |
| 289 |
|
| 290 |
.. index:: |
| 291 |
single: sys.meta_path |
| 292 |
pair: finder; find_spec |
| 293 |
|
| 294 |
When the named module is not found in :data:`sys.modules`, Python next |
| 295 |
searches :data:`sys.meta_path`, which contains a list of meta path finder |
| 296 |
objects. These finders are queried in order to see if they know how to handle |
| 297 |
the named module. Meta path finders must implement a method called |
| 298 |
:meth:`~importlib.abc.MetaPathFinder.find_spec` which takes three arguments: |
| 299 |
a name, an import path, and (optionally) a target module. The meta path |
| 300 |
finder can use any strategy it wants to determine whether it can handle |
| 301 |
the named module or not. |
| 302 |
|
| 303 |
If the meta path finder knows how to handle the named module, it returns a |
| 304 |
spec object. If it cannot handle the named module, it returns ``None``. If |
| 305 |
:data:`sys.meta_path` processing reaches the end of its list without returning |
| 306 |
a spec, then a :exc:`ModuleNotFoundError` is raised. Any other exceptions |
| 307 |
raised are simply propagated up, aborting the import process. |
| 308 |
|
| 309 |
The :meth:`~importlib.abc.MetaPathFinder.find_spec` method of meta path |
| 310 |
finders is called with two or three arguments. The first is the fully |
| 311 |
qualified name of the module being imported, for example ``foo.bar.baz``. |
| 312 |
The second argument is the path entries to use for the module search. For |
| 313 |
top-level modules, the second argument is ``None``, but for submodules or |
| 314 |
subpackages, the second argument is the value of the parent package's |
| 315 |
``__path__`` attribute. If the appropriate ``__path__`` attribute cannot |
| 316 |
be accessed, a :exc:`ModuleNotFoundError` is raised. The third argument |
| 317 |
is an existing module object that will be the target of loading later. |
| 318 |
The import system passes in a target module only during reload. |
| 319 |
|
| 320 |
The meta path may be traversed multiple times for a single import request. |
| 321 |
For example, assuming none of the modules involved has already been cached, |
| 322 |
importing ``foo.bar.baz`` will first perform a top level import, calling |
| 323 |
``mpf.find_spec("foo", None, None)`` on each meta path finder (``mpf``). After |
| 324 |
``foo`` has been imported, ``foo.bar`` will be imported by traversing the |
| 325 |
meta path a second time, calling |
| 326 |
``mpf.find_spec("foo.bar", foo.__path__, None)``. Once ``foo.bar`` has been |
| 327 |
imported, the final traversal will call |
| 328 |
``mpf.find_spec("foo.bar.baz", foo.bar.__path__, None)``. |
| 329 |
|
| 330 |
Some meta path finders only support top level imports. These importers will |
| 331 |
always return ``None`` when anything other than ``None`` is passed as the |
| 332 |
second argument. |
| 333 |
|
| 334 |
Python's default :data:`sys.meta_path` has three meta path finders, one that |
| 335 |
knows how to import built-in modules, one that knows how to import frozen |
| 336 |
modules, and one that knows how to import modules from an :term:`import path` |
| 337 |
(i.e. the :term:`path based finder`). |
| 338 |
|
| 339 |
.. versionchanged:: 3.4 |
| 340 |
The :meth:`~importlib.abc.MetaPathFinder.find_spec` method of meta path |
| 341 |
finders replaced :meth:`!find_module`, which |
| 342 |
is now deprecated. While it will continue to work without change, the |
| 343 |
import machinery will try it only if the finder does not implement |
| 344 |
:meth:`~importlib.abc.MetaPathFinder.find_spec`. |
| 345 |
|
| 346 |
.. versionchanged:: 3.10 |
| 347 |
Use of :meth:`!find_module` by the import system |
| 348 |
now raises :exc:`ImportWarning`. |
| 349 |
|
| 350 |
.. versionchanged:: 3.12 |
| 351 |
:meth:`!find_module` has been removed. |
| 352 |
Use :meth:`~importlib.abc.MetaPathFinder.find_spec` instead. |
| 353 |
|
| 354 |
|
| 355 |
Loading |
| 356 |
======= |
| 357 |
|
| 358 |
If and when a module spec is found, the import machinery will use it (and |
| 359 |
the loader it contains) when loading the module. Here is an approximation |
| 360 |
of what happens during the loading portion of import:: |
| 361 |
|
| 362 |
module = None |
| 363 |
if spec.loader is not None and hasattr(spec.loader, 'create_module'): |
| 364 |
# It is assumed 'exec_module' will also be defined on the loader. |
| 365 |
module = spec.loader.create_module(spec) |
| 366 |
if module is None: |
| 367 |
module = ModuleType(spec.name) |
| 368 |
# The import-related module attributes get set here: |
| 369 |
_init_module_attrs(spec, module) |
| 370 |
|
| 371 |
if spec.loader is None: |
| 372 |
# unsupported |
| 373 |
raise ImportError |
| 374 |
if spec.origin is None and spec.submodule_search_locations is not None: |
| 375 |
# namespace package |
| 376 |
sys.modules[spec.name] = module |
| 377 |
elif not hasattr(spec.loader, 'exec_module'): |
| 378 |
module = spec.loader.load_module(spec.name) |
| 379 |
else: |
| 380 |
sys.modules[spec.name] = module |
| 381 |
try: |
| 382 |
spec.loader.exec_module(module) |
| 383 |
except BaseException: |
| 384 |
try: |
| 385 |
del sys.modules[spec.name] |
| 386 |
except KeyError: |
| 387 |
pass |
| 388 |
raise |
| 389 |
return sys.modules[spec.name] |
| 390 |
|
| 391 |
Note the following details: |
| 392 |
|
| 393 |
* If there is an existing module object with the given name in |
| 394 |
:data:`sys.modules`, import will have already returned it. |
| 395 |
|
| 396 |
* The module will exist in :data:`sys.modules` before the loader |
| 397 |
executes the module code. This is crucial because the module code may |
| 398 |
(directly or indirectly) import itself; adding it to :data:`sys.modules` |
| 399 |
beforehand prevents unbounded recursion in the worst case and multiple |
| 400 |
loading in the best. |
| 401 |
|
| 402 |
* If loading fails, the failing module -- and only the failing module -- |
| 403 |
gets removed from :data:`sys.modules`. Any module already in the |
| 404 |
:data:`sys.modules` cache, and any module that was successfully loaded |
| 405 |
as a side-effect, must remain in the cache. This contrasts with |
| 406 |
reloading where even the failing module is left in :data:`sys.modules`. |
| 407 |
|
| 408 |
* After the module is created but before execution, the import machinery |
| 409 |
sets the import-related module attributes ("_init_module_attrs" in |
| 410 |
the pseudo-code example above), as summarized in a |
| 411 |
:ref:`later section <import-mod-attrs>`. |
| 412 |
|
| 413 |
* Module execution is the key moment of loading in which the module's |
| 414 |
namespace gets populated. Execution is entirely delegated to the |
| 415 |
loader, which gets to decide what gets populated and how. |
| 416 |
|
| 417 |
* The module created during loading and passed to exec_module() may |
| 418 |
not be the one returned at the end of import [#fnlo]_. |
| 419 |
|
| 420 |
.. versionchanged:: 3.4 |
| 421 |
The import system has taken over the boilerplate responsibilities of |
| 422 |
loaders. These were previously performed by the |
| 423 |
:meth:`importlib.abc.Loader.load_module` method. |
| 424 |
|
| 425 |
Loaders |
| 426 |
------- |
| 427 |
|
| 428 |
Module loaders provide the critical function of loading: module execution. |
| 429 |
The import machinery calls the :meth:`importlib.abc.Loader.exec_module` |
| 430 |
method with a single argument, the module object to execute. Any value |
| 431 |
returned from :meth:`~importlib.abc.Loader.exec_module` is ignored. |
| 432 |
|
| 433 |
Loaders must satisfy the following requirements: |
| 434 |
|
| 435 |
* If the module is a Python module (as opposed to a built-in module or a |
| 436 |
dynamically loaded extension), the loader should execute the module's code |
| 437 |
in the module's global name space (``module.__dict__``). |
| 438 |
|
| 439 |
* If the loader cannot execute the module, it should raise an |
| 440 |
:exc:`ImportError`, although any other exception raised during |
| 441 |
:meth:`~importlib.abc.Loader.exec_module` will be propagated. |
| 442 |
|
| 443 |
In many cases, the finder and loader can be the same object; in such cases the |
| 444 |
:meth:`~importlib.abc.MetaPathFinder.find_spec` method would just return a |
| 445 |
spec with the loader set to ``self``. |
| 446 |
|
| 447 |
Module loaders may opt in to creating the module object during loading |
| 448 |
by implementing a :meth:`~importlib.abc.Loader.create_module` method. |
| 449 |
It takes one argument, the module spec, and returns the new module object |
| 450 |
to use during loading. ``create_module()`` does not need to set any attributes |
| 451 |
on the module object. If the method returns ``None``, the |
| 452 |
import machinery will create the new module itself. |
| 453 |
|
| 454 |
.. versionadded:: 3.4 |
| 455 |
The :meth:`~importlib.abc.Loader.create_module` method of loaders. |
| 456 |
|
| 457 |
.. versionchanged:: 3.4 |
| 458 |
The :meth:`~importlib.abc.Loader.load_module` method was replaced by |
| 459 |
:meth:`~importlib.abc.Loader.exec_module` and the import |
| 460 |
machinery assumed all the boilerplate responsibilities of loading. |
| 461 |
|
| 462 |
For compatibility with existing loaders, the import machinery will use |
| 463 |
the ``load_module()`` method of loaders if it exists and the loader does |
| 464 |
not also implement ``exec_module()``. However, ``load_module()`` has been |
| 465 |
deprecated and loaders should implement ``exec_module()`` instead. |
| 466 |
|
| 467 |
The ``load_module()`` method must implement all the boilerplate loading |
| 468 |
functionality described above in addition to executing the module. All |
| 469 |
the same constraints apply, with some additional clarification: |
| 470 |
|
| 471 |
* If there is an existing module object with the given name in |
| 472 |
:data:`sys.modules`, the loader must use that existing module. |
| 473 |
(Otherwise, :func:`importlib.reload` will not work correctly.) If the |
| 474 |
named module does not exist in :data:`sys.modules`, the loader |
| 475 |
must create a new module object and add it to :data:`sys.modules`. |
| 476 |
|
| 477 |
* The module *must* exist in :data:`sys.modules` before the loader |
| 478 |
executes the module code, to prevent unbounded recursion or multiple |
| 479 |
loading. |
| 480 |
|
| 481 |
* If loading fails, the loader must remove any modules it has inserted |
| 482 |
into :data:`sys.modules`, but it must remove **only** the failing |
| 483 |
module(s), and only if the loader itself has loaded the module(s) |
| 484 |
explicitly. |
| 485 |
|
| 486 |
.. versionchanged:: 3.5 |
| 487 |
A :exc:`DeprecationWarning` is raised when ``exec_module()`` is defined but |
| 488 |
``create_module()`` is not. |
| 489 |
|
| 490 |
.. versionchanged:: 3.6 |
| 491 |
An :exc:`ImportError` is raised when ``exec_module()`` is defined but |
| 492 |
``create_module()`` is not. |
| 493 |
|
| 494 |
.. versionchanged:: 3.10 |
| 495 |
Use of ``load_module()`` will raise :exc:`ImportWarning`. |
| 496 |
|
| 497 |
Submodules |
| 498 |
---------- |
| 499 |
|
| 500 |
When a submodule is loaded using any mechanism (e.g. ``importlib`` APIs, the |
| 501 |
``import`` or ``import-from`` statements, or built-in ``__import__()``) a |
| 502 |
binding is placed in the parent module's namespace to the submodule object. |
| 503 |
For example, if package ``spam`` has a submodule ``foo``, after importing |
| 504 |
``spam.foo``, ``spam`` will have an attribute ``foo`` which is bound to the |
| 505 |
submodule. Let's say you have the following directory structure:: |
| 506 |
|
| 507 |
spam/ |
| 508 |
__init__.py |
| 509 |
foo.py |
| 510 |
|
| 511 |
and ``spam/__init__.py`` has the following line in it:: |
| 512 |
|
| 513 |
from .foo import Foo |
| 514 |
|
| 515 |
then executing the following puts name bindings for ``foo`` and ``Foo`` in the |
| 516 |
``spam`` module:: |
| 517 |
|
| 518 |
>>> import spam |
| 519 |
>>> spam.foo |
| 520 |
<module 'spam.foo' from '/tmp/imports/spam/foo.py'> |
| 521 |
>>> spam.Foo |
| 522 |
<class 'spam.foo.Foo'> |
| 523 |
|
| 524 |
Given Python's familiar name binding rules this might seem surprising, but |
| 525 |
it's actually a fundamental feature of the import system. The invariant |
| 526 |
holding is that if you have ``sys.modules['spam']`` and |
| 527 |
``sys.modules['spam.foo']`` (as you would after the above import), the latter |
| 528 |
must appear as the ``foo`` attribute of the former. |
| 529 |
|
| 530 |
.. _module-specs: |
| 531 |
|
| 532 |
Module specs |
| 533 |
------------ |
| 534 |
|
| 535 |
The import machinery uses a variety of information about each module |
| 536 |
during import, especially before loading. Most of the information is |
| 537 |
common to all modules. The purpose of a module's spec is to encapsulate |
| 538 |
this import-related information on a per-module basis. |
| 539 |
|
| 540 |
Using a spec during import allows state to be transferred between import |
| 541 |
system components, e.g. between the finder that creates the module spec |
| 542 |
and the loader that executes it. Most importantly, it allows the |
| 543 |
import machinery to perform the boilerplate operations of loading, |
| 544 |
whereas without a module spec the loader had that responsibility. |
| 545 |
|
| 546 |
The module's spec is exposed as :attr:`module.__spec__`. Setting |
| 547 |
:attr:`!__spec__` appropriately applies equally to |
| 548 |
:ref:`modules initialized during interpreter startup <programs>`. |
| 549 |
The one exception is ``__main__``, where :attr:`!__spec__` is |
| 550 |
:ref:`set to None in some cases <main_spec>`. |
| 551 |
|
| 552 |
See :class:`~importlib.machinery.ModuleSpec` for details on the contents of |
| 553 |
the module spec. |
| 554 |
|
| 555 |
.. versionadded:: 3.4 |
| 556 |
|
| 557 |
.. _package-path-rules: |
| 558 |
|
| 559 |
__path__ attributes on modules |
| 560 |
------------------------------ |
| 561 |
|
| 562 |
The :attr:`~module.__path__` attribute should be a (possibly empty) |
| 563 |
:term:`sequence` of strings enumerating the locations where the package's |
| 564 |
submodules will be found. By definition, if a module has a :attr:`!__path__` |
| 565 |
attribute, it is a :term:`package`. |
| 566 |
|
| 567 |
A package's :attr:`~module.__path__` attribute is used during imports of its |
| 568 |
subpackages. |
| 569 |
Within the import machinery, it functions much the same as :data:`sys.path`, |
| 570 |
i.e. providing a list of locations to search for modules during import. |
| 571 |
However, :attr:`!__path__` is typically much more constrained than |
| 572 |
:data:`!sys.path`. |
| 573 |
|
| 574 |
The same rules used for :data:`sys.path` also apply to a package's |
| 575 |
:attr:`!__path__`. :data:`sys.path_hooks` (described below) are |
| 576 |
consulted when traversing a package's :attr:`!__path__`. |
| 577 |
|
| 578 |
A package's ``__init__.py`` file may set or alter the package's |
| 579 |
:attr:`~module.__path__` |
| 580 |
attribute, and this was typically the way namespace packages were implemented |
| 581 |
prior to :pep:`420`. With the adoption of :pep:`420`, namespace packages no |
| 582 |
longer need to supply ``__init__.py`` files containing only :attr:`!__path__` |
| 583 |
manipulation code; the import machinery automatically sets :attr:`!__path__` |
| 584 |
correctly for the namespace package. |
| 585 |
|
| 586 |
Module reprs |
| 587 |
------------ |
| 588 |
|
| 589 |
By default, all modules have a usable repr, however depending on the |
| 590 |
attributes set above, and in the module's spec, you can more explicitly |
| 591 |
control the repr of module objects. |
| 592 |
|
| 593 |
If the module has a spec (``__spec__``), the import machinery will try |
| 594 |
to generate a repr from it. If that fails or there is no spec, the import |
| 595 |
system will craft a default repr using whatever information is available |
| 596 |
on the module. It will try to use the ``module.__name__``, |
| 597 |
``module.__file__``, and ``module.__loader__`` as input into the repr, |
| 598 |
with defaults for whatever information is missing. |
| 599 |
|
| 600 |
Here are the exact rules used: |
| 601 |
|
| 602 |
* If the module has a ``__spec__`` attribute, the information in the spec |
| 603 |
is used to generate the repr. The "name", "loader", "origin", and |
| 604 |
"has_location" attributes are consulted. |
| 605 |
|
| 606 |
* If the module has a ``__file__`` attribute, this is used as part of the |
| 607 |
module's repr. |
| 608 |
|
| 609 |
* If the module has no ``__file__`` but does have a ``__loader__`` that is not |
| 610 |
``None``, then the loader's repr is used as part of the module's repr. |
| 611 |
|
| 612 |
* Otherwise, just use the module's ``__name__`` in the repr. |
| 613 |
|
| 614 |
.. versionchanged:: 3.12 |
| 615 |
Use of :meth:`!module_repr`, having been deprecated since Python 3.4, was |
| 616 |
removed in Python 3.12 and is no longer called during the resolution of a |
| 617 |
module's repr. |
| 618 |
|
| 619 |
.. _pyc-invalidation: |
| 620 |
|
| 621 |
Cached bytecode invalidation |
| 622 |
---------------------------- |
| 623 |
|
| 624 |
Before Python loads cached bytecode from a ``.pyc`` file, it checks whether the |
| 625 |
cache is up-to-date with the source ``.py`` file. By default, Python does this |
| 626 |
by storing the source's last-modified timestamp and size in the cache file when |
| 627 |
writing it. At runtime, the import system then validates the cache file by |
| 628 |
checking the stored metadata in the cache file against the source's |
| 629 |
metadata. |
| 630 |
|
| 631 |
Python also supports "hash-based" cache files, which store a hash of the source |
| 632 |
file's contents rather than its metadata. There are two variants of hash-based |
| 633 |
``.pyc`` files: checked and unchecked. For checked hash-based ``.pyc`` files, |
| 634 |
Python validates the cache file by hashing the source file and comparing the |
| 635 |
resulting hash with the hash in the cache file. If a checked hash-based cache |
| 636 |
file is found to be invalid, Python regenerates it and writes a new checked |
| 637 |
hash-based cache file. For unchecked hash-based ``.pyc`` files, Python simply |
| 638 |
assumes the cache file is valid if it exists. Hash-based ``.pyc`` files |
| 639 |
validation behavior may be overridden with the :option:`--check-hash-based-pycs` |
| 640 |
flag. |
| 641 |
|
| 642 |
.. versionchanged:: 3.7 |
| 643 |
Added hash-based ``.pyc`` files. Previously, Python only supported |
| 644 |
timestamp-based invalidation of bytecode caches. |
| 645 |
|
| 646 |
|
| 647 |
The Path Based Finder |
| 648 |
===================== |
| 649 |
|
| 650 |
.. index:: |
| 651 |
single: path based finder |
| 652 |
|
| 653 |
As mentioned previously, Python comes with several default meta path finders. |
| 654 |
One of these, called the :term:`path based finder` |
| 655 |
(:class:`~importlib.machinery.PathFinder`), searches an :term:`import path`, |
| 656 |
which contains a list of :term:`path entries <path entry>`. Each path |
| 657 |
entry names a location to search for modules. |
| 658 |
|
| 659 |
The path based finder itself doesn't know how to import anything. Instead, it |
| 660 |
traverses the individual path entries, associating each of them with a |
| 661 |
path entry finder that knows how to handle that particular kind of path. |
| 662 |
|
| 663 |
The default set of path entry finders implement all the semantics for finding |
| 664 |
modules on the file system, handling special file types such as Python source |
| 665 |
code (``.py`` files), Python byte code (``.pyc`` files) and |
| 666 |
shared libraries (e.g. ``.so`` files). When supported by the :mod:`zipimport` |
| 667 |
module in the standard library, the default path entry finders also handle |
| 668 |
loading all of these file types (other than shared libraries) from zipfiles. |
| 669 |
|
| 670 |
Path entries need not be limited to file system locations. They can refer to |
| 671 |
URLs, database queries, or any other location that can be specified as a |
| 672 |
string. |
| 673 |
|
| 674 |
The path based finder provides additional hooks and protocols so that you |
| 675 |
can extend and customize the types of searchable path entries. For example, |
| 676 |
if you wanted to support path entries as network URLs, you could write a hook |
| 677 |
that implements HTTP semantics to find modules on the web. This hook (a |
| 678 |
callable) would return a :term:`path entry finder` supporting the protocol |
| 679 |
described below, which was then used to get a loader for the module from the |
| 680 |
web. |
| 681 |
|
| 682 |
A word of warning: this section and the previous both use the term *finder*, |
| 683 |
distinguishing between them by using the terms :term:`meta path finder` and |
| 684 |
:term:`path entry finder`. These two types of finders are very similar, |
| 685 |
support similar protocols, and function in similar ways during the import |
| 686 |
process, but it's important to keep in mind that they are subtly different. |
| 687 |
In particular, meta path finders operate at the beginning of the import |
| 688 |
process, as keyed off the :data:`sys.meta_path` traversal. |
| 689 |
|
| 690 |
By contrast, path entry finders are in a sense an implementation detail |
| 691 |
of the path based finder, and in fact, if the path based finder were to be |
| 692 |
removed from :data:`sys.meta_path`, none of the path entry finder semantics |
| 693 |
would be invoked. |
| 694 |
|
| 695 |
|
| 696 |
Path entry finders |
| 697 |
------------------ |
| 698 |
|
| 699 |
.. index:: |
| 700 |
single: sys.path |
| 701 |
single: sys.path_hooks |
| 702 |
single: sys.path_importer_cache |
| 703 |
single: PYTHONPATH |
| 704 |
|
| 705 |
The :term:`path based finder` is responsible for finding and loading |
| 706 |
Python modules and packages whose location is specified with a string |
| 707 |
:term:`path entry`. Most path entries name locations in the file system, |
| 708 |
but they need not be limited to this. |
| 709 |
|
| 710 |
As a meta path finder, the :term:`path based finder` implements the |
| 711 |
:meth:`~importlib.abc.MetaPathFinder.find_spec` protocol previously |
| 712 |
described, however it exposes additional hooks that can be used to |
| 713 |
customize how modules are found and loaded from the :term:`import path`. |
| 714 |
|
| 715 |
Three variables are used by the :term:`path based finder`, :data:`sys.path`, |
| 716 |
:data:`sys.path_hooks` and :data:`sys.path_importer_cache`. The ``__path__`` |
| 717 |
attributes on package objects are also used. These provide additional ways |
| 718 |
that the import machinery can be customized. |
| 719 |
|
| 720 |
:data:`sys.path` contains a list of strings providing search locations for |
| 721 |
modules and packages. It is initialized from the :envvar:`PYTHONPATH` |
| 722 |
environment variable and various other installation- and |
| 723 |
implementation-specific defaults. Entries in :data:`sys.path` can name |
| 724 |
directories on the file system, zip files, and potentially other "locations" |
| 725 |
(see the :mod:`site` module) that should be searched for modules, such as |
| 726 |
URLs, or database queries. Only strings should be present on |
| 727 |
:data:`sys.path`; all other data types are ignored. |
| 728 |
|
| 729 |
The :term:`path based finder` is a :term:`meta path finder`, so the import |
| 730 |
machinery begins the :term:`import path` search by calling the path |
| 731 |
based finder's :meth:`~importlib.machinery.PathFinder.find_spec` method as |
| 732 |
described previously. When the ``path`` argument to |
| 733 |
:meth:`~importlib.machinery.PathFinder.find_spec` is given, it will be a |
| 734 |
list of string paths to traverse - typically a package's ``__path__`` |
| 735 |
attribute for an import within that package. If the ``path`` argument is |
| 736 |
``None``, this indicates a top level import and :data:`sys.path` is used. |
| 737 |
|
| 738 |
The path based finder iterates over every entry in the search path, and |
| 739 |
for each of these, looks for an appropriate :term:`path entry finder` |
| 740 |
(:class:`~importlib.abc.PathEntryFinder`) for the |
| 741 |
path entry. Because this can be an expensive operation (e.g. there may be |
| 742 |
``stat()`` call overheads for this search), the path based finder maintains |
| 743 |
a cache mapping path entries to path entry finders. This cache is maintained |
| 744 |
in :data:`sys.path_importer_cache` (despite the name, this cache actually |
| 745 |
stores finder objects rather than being limited to :term:`importer` objects). |
| 746 |
In this way, the expensive search for a particular :term:`path entry` |
| 747 |
location's :term:`path entry finder` need only be done once. User code is |
| 748 |
free to remove cache entries from :data:`sys.path_importer_cache` forcing |
| 749 |
the path based finder to perform the path entry search again. |
| 750 |
|
| 751 |
If the path entry is not present in the cache, the path based finder iterates |
| 752 |
over every callable in :data:`sys.path_hooks`. Each of the :term:`path entry |
| 753 |
hooks <path entry hook>` in this list is called with a single argument, the |
| 754 |
path entry to be searched. This callable may either return a :term:`path |
| 755 |
entry finder` that can handle the path entry, or it may raise |
| 756 |
:exc:`ImportError`. An :exc:`ImportError` is used by the path based finder to |
| 757 |
signal that the hook cannot find a :term:`path entry finder` |
| 758 |
for that :term:`path entry`. The |
| 759 |
exception is ignored and :term:`import path` iteration continues. The hook |
| 760 |
should expect either a string or bytes object; the encoding of bytes objects |
| 761 |
is up to the hook (e.g. it may be a file system encoding, UTF-8, or something |
| 762 |
else), and if the hook cannot decode the argument, it should raise |
| 763 |
:exc:`ImportError`. |
| 764 |
|
| 765 |
If :data:`sys.path_hooks` iteration ends with no :term:`path entry finder` |
| 766 |
being returned, then the path based finder's |
| 767 |
:meth:`~importlib.machinery.PathFinder.find_spec` method will store ``None`` |
| 768 |
in :data:`sys.path_importer_cache` (to indicate that there is no finder for |
| 769 |
this path entry) and return ``None``, indicating that this |
| 770 |
:term:`meta path finder` could not find the module. |
| 771 |
|
| 772 |
If a :term:`path entry finder` *is* returned by one of the :term:`path entry |
| 773 |
hook` callables on :data:`sys.path_hooks`, then the following protocol is used |
| 774 |
to ask the finder for a module spec, which is then used when loading the |
| 775 |
module. |
| 776 |
|
| 777 |
The current working directory -- denoted by an empty string -- is handled |
| 778 |
slightly differently from other entries on :data:`sys.path`. First, if the |
| 779 |
current working directory cannot be determined or is found not to exist, no |
| 780 |
value is stored in :data:`sys.path_importer_cache`. Second, the value for the |
| 781 |
current working directory is looked up fresh for each module lookup. Third, |
| 782 |
the path used for :data:`sys.path_importer_cache` and returned by |
| 783 |
:meth:`importlib.machinery.PathFinder.find_spec` will be the actual current |
| 784 |
working directory and not the empty string. |
| 785 |
|
| 786 |
Path entry finder protocol |
| 787 |
-------------------------- |
| 788 |
|
| 789 |
In order to support imports of modules and initialized packages and also to |
| 790 |
contribute portions to namespace packages, path entry finders must implement |
| 791 |
the :meth:`~importlib.abc.PathEntryFinder.find_spec` method. |
| 792 |
|
| 793 |
:meth:`~importlib.abc.PathEntryFinder.find_spec` takes two arguments: the |
| 794 |
fully qualified name of the module being imported, and the (optional) target |
| 795 |
module. ``find_spec()`` returns a fully populated spec for the module. |
| 796 |
This spec will always have "loader" set (with one exception). |
| 797 |
|
| 798 |
To indicate to the import machinery that the spec represents a namespace |
| 799 |
:term:`portion`, the path entry finder sets ``submodule_search_locations`` to |
| 800 |
a list containing the portion. |
| 801 |
|
| 802 |
.. versionchanged:: 3.4 |
| 803 |
:meth:`~importlib.abc.PathEntryFinder.find_spec` replaced |
| 804 |
:meth:`!find_loader` and |
| 805 |
:meth:`!find_module`, both of which |
| 806 |
are now deprecated, but will be used if ``find_spec()`` is not defined. |
| 807 |
|
| 808 |
Older path entry finders may implement one of these two deprecated methods |
| 809 |
instead of ``find_spec()``. The methods are still respected for the |
| 810 |
sake of backward compatibility. However, if ``find_spec()`` is |
| 811 |
implemented on the path entry finder, the legacy methods are ignored. |
| 812 |
|
| 813 |
:meth:`!find_loader` takes one argument, the |
| 814 |
fully qualified name of the module being imported. ``find_loader()`` |
| 815 |
returns a 2-tuple where the first item is the loader and the second item |
| 816 |
is a namespace :term:`portion`. |
| 817 |
|
| 818 |
For backwards compatibility with other implementations of the import |
| 819 |
protocol, many path entry finders also support the same, |
| 820 |
traditional ``find_module()`` method that meta path finders support. |
| 821 |
However path entry finder ``find_module()`` methods are never called |
| 822 |
with a ``path`` argument (they are expected to record the appropriate |
| 823 |
path information from the initial call to the path hook). |
| 824 |
|
| 825 |
The ``find_module()`` method on path entry finders is deprecated, |
| 826 |
as it does not allow the path entry finder to contribute portions to |
| 827 |
namespace packages. If both ``find_loader()`` and ``find_module()`` |
| 828 |
exist on a path entry finder, the import system will always call |
| 829 |
``find_loader()`` in preference to ``find_module()``. |
| 830 |
|
| 831 |
.. versionchanged:: 3.10 |
| 832 |
Calls to :meth:`!find_module` and |
| 833 |
:meth:`!find_loader` by the import |
| 834 |
system will raise :exc:`ImportWarning`. |
| 835 |
|
| 836 |
.. versionchanged:: 3.12 |
| 837 |
``find_module()`` and ``find_loader()`` have been removed. |
| 838 |
|
| 839 |
|
| 840 |
Replacing the standard import system |
| 841 |
==================================== |
| 842 |
|
| 843 |
The most reliable mechanism for replacing the entire import system is to |
| 844 |
delete the default contents of :data:`sys.meta_path`, replacing them |
| 845 |
entirely with a custom meta path hook. |
| 846 |
|
| 847 |
If it is acceptable to only alter the behaviour of import statements |
| 848 |
without affecting other APIs that access the import system, then replacing |
| 849 |
the builtin :func:`__import__` function may be sufficient. |
| 850 |
|
| 851 |
To selectively prevent the import of some modules from a hook early on the |
| 852 |
meta path (rather than disabling the standard import system entirely), |
| 853 |
it is sufficient to raise :exc:`ModuleNotFoundError` directly from |
| 854 |
:meth:`~importlib.abc.MetaPathFinder.find_spec` instead of returning |
| 855 |
``None``. The latter indicates that the meta path search should continue, |
| 856 |
while raising an exception terminates it immediately. |
| 857 |
|
| 858 |
.. _relativeimports: |
| 859 |
|
| 860 |
Package Relative Imports |
| 861 |
======================== |
| 862 |
|
| 863 |
Relative imports use leading dots. A single leading dot indicates a relative |
| 864 |
import, starting with the current package. Two or more leading dots indicate a |
| 865 |
relative import to the parent(s) of the current package, one level per dot |
| 866 |
after the first. For example, given the following package layout:: |
| 867 |
|
| 868 |
package/ |
| 869 |
__init__.py |
| 870 |
subpackage1/ |
| 871 |
__init__.py |
| 872 |
moduleX.py |
| 873 |
moduleY.py |
| 874 |
subpackage2/ |
| 875 |
__init__.py |
| 876 |
moduleZ.py |
| 877 |
moduleA.py |
| 878 |
|
| 879 |
In either ``subpackage1/moduleX.py`` or ``subpackage1/__init__.py``, |
| 880 |
the following are valid relative imports:: |
| 881 |
|
| 882 |
from .moduleY import spam |
| 883 |
from .moduleY import spam as ham |
| 884 |
from . import moduleY |
| 885 |
from ..subpackage1 import moduleY |
| 886 |
from ..subpackage2.moduleZ import eggs |
| 887 |
from ..moduleA import foo |
| 888 |
|
| 889 |
Absolute imports may use either the ``import <>`` or ``from <> import <>`` |
| 890 |
syntax, but relative imports may only use the second form; the reason |
| 891 |
for this is that:: |
| 892 |
|
| 893 |
import XXX.YYY.ZZZ |
| 894 |
|
| 895 |
should expose ``XXX.YYY.ZZZ`` as a usable expression, but .moduleY is |
| 896 |
not a valid expression. |
| 897 |
|
| 898 |
|
| 899 |
.. _import-dunder-main: |
| 900 |
|
| 901 |
Special considerations for __main__ |
| 902 |
=================================== |
| 903 |
|
| 904 |
The :mod:`__main__` module is a special case relative to Python's import |
| 905 |
system. As noted :ref:`elsewhere <programs>`, the ``__main__`` module |
| 906 |
is directly initialized at interpreter startup, much like :mod:`sys` and |
| 907 |
:mod:`builtins`. However, unlike those two, it doesn't strictly |
| 908 |
qualify as a built-in module. This is because the manner in which |
| 909 |
``__main__`` is initialized depends on the flags and other options with |
| 910 |
which the interpreter is invoked. |
| 911 |
|
| 912 |
.. _main_spec: |
| 913 |
|
| 914 |
__main__.__spec__ |
| 915 |
----------------- |
| 916 |
|
| 917 |
Depending on how :mod:`__main__` is initialized, ``__main__.__spec__`` |
| 918 |
gets set appropriately or to ``None``. |
| 919 |
|
| 920 |
When Python is started with the :option:`-m` option, ``__spec__`` is set |
| 921 |
to the module spec of the corresponding module or package. ``__spec__`` is |
| 922 |
also populated when the ``__main__`` module is loaded as part of executing a |
| 923 |
directory, zipfile or other :data:`sys.path` entry. |
| 924 |
|
| 925 |
In :ref:`the remaining cases <using-on-interface-options>` |
| 926 |
``__main__.__spec__`` is set to ``None``, as the code used to populate the |
| 927 |
:mod:`__main__` does not correspond directly with an importable module: |
| 928 |
|
| 929 |
- interactive prompt |
| 930 |
- :option:`-c` option |
| 931 |
- running from stdin |
| 932 |
- running directly from a source or bytecode file |
| 933 |
|
| 934 |
Note that ``__main__.__spec__`` is always ``None`` in the last case, |
| 935 |
*even if* the file could technically be imported directly as a module |
| 936 |
instead. Use the :option:`-m` switch if valid module metadata is desired |
| 937 |
in :mod:`__main__`. |
| 938 |
|
| 939 |
Note also that even when ``__main__`` corresponds with an importable module |
| 940 |
and ``__main__.__spec__`` is set accordingly, they're still considered |
| 941 |
*distinct* modules. This is due to the fact that blocks guarded by |
| 942 |
``if __name__ == "__main__":`` checks only execute when the module is used |
| 943 |
to populate the ``__main__`` namespace, and not during normal import. |
| 944 |
|
| 945 |
|
| 946 |
References |
| 947 |
========== |
| 948 |
|
| 949 |
The import machinery has evolved considerably since Python's early days. The |
| 950 |
original `specification for packages |
| 951 |
<https://www.python.org/doc/essays/packages/>`_ is still available to read, |
| 952 |
although some details have changed since the writing of that document. |
| 953 |
|
| 954 |
The original specification for :data:`sys.meta_path` was :pep:`302`, with |
| 955 |
subsequent extension in :pep:`420`. |
| 956 |
|
| 957 |
:pep:`420` introduced :term:`namespace packages <namespace package>` for |
| 958 |
Python 3.3. :pep:`420` also introduced the :meth:`!find_loader` protocol as an |
| 959 |
alternative to :meth:`!find_module`. |
| 960 |
|
| 961 |
:pep:`366` describes the addition of the ``__package__`` attribute for |
| 962 |
explicit relative imports in main modules. |
| 963 |
|
| 964 |
:pep:`328` introduced absolute and explicit relative imports and initially |
| 965 |
proposed ``__name__`` for semantics :pep:`366` would eventually specify for |
| 966 |
``__package__``. |
| 967 |
|
| 968 |
:pep:`338` defines executing modules as scripts. |
| 969 |
|
| 970 |
:pep:`451` adds the encapsulation of per-module import state in spec |
| 971 |
objects. It also off-loads most of the boilerplate responsibilities of |
| 972 |
loaders back onto the import machinery. These changes allow the |
| 973 |
deprecation of several APIs in the import system and also addition of new |
| 974 |
methods to finders and loaders. |
| 975 |
|
| 976 |
.. rubric:: Footnotes |
| 977 |
|
| 978 |
.. [#fnmo] See :class:`types.ModuleType`. |
| 979 |
|
| 980 |
.. [#fnlo] The importlib implementation avoids using the return value |
| 981 |
directly. Instead, it gets the module object by looking the module name up |
| 982 |
in :data:`sys.modules`. The indirect effect of this is that an imported |
| 983 |
module may replace itself in :data:`sys.modules`. This is |
| 984 |
implementation-specific behavior that is not guaranteed to work in other |
| 985 |
Python implementations. |