# Import Hook [maturin_import_hook](https://pypi.org/project/maturin-import-hook/) provides a mechanism to automatically rebuild Maturin projects when they are imported by Python scripts. This reduces friction when developing mixed Python/Rust codebases because edits made to Rust components take effect automatically like edits to Python components do. Automatic rebuilding eliminates the possibility of Python code using outdated rust components, which often leads to confusing behaviour. The import hook can be configured to activate automatically as the interpreter starts or manually in individual scripts. Only Maturin packages installed editable mode (`maturin develop` or `pip install -e`) are considered by the import hook. When the build is up to date the overhead of the import hook is minimal. The hook has some additional features such as importing stand-alone `.rs` files. See [Features](#features) for a complete list. ## Installation Install the package with: ```shell $ pip install maturin_import_hook ``` Then install it into the current virtual environment with: ```shell $ python -m maturin_import_hook site install ``` This will activate the hook automatically as the interpreter starts. This command only has to be run once per virtual environment. To use `site install`, ensure you have write access to `site-packages`. Using a [virtual environment](https://docs.python.org/3/library/venv.html) is recommended rather than installing into the system interpreter. The managed site installation can be removed with: ```shell $ python -m maturin_import_hook site uninstall ``` To manually activate the import hook only for specific scripts instead of installing into the virtual environment see [Manual Activation](#manual-activation). ## Basic Configuration The site installation can be customized. For example, to build in release mode: ```shell $ python -m maturin_import_hook site install --args="--release" ``` The `--args` option accepts `maturin develop` arguments. For more options see `--help`. The site installation can also be edited manually. Use `python -m maturin_import_hook site info` to locate the `sitecustomize.py` file. See [Install Arguments](#install-arguments) for a list of arguments to `install()`. To avoid unnecessary rebuilds caused by irrelevant file modifications, create empty `.maturin_hook_ignore` files to manually ignore directories. Several directories and file types such as `target/` and `*.py` are ignored by default. If more customization is needed, see [Custom File Searching](#custom-file-searching). ## CLI The package provides a CLI for managing the build cache and site installation. For details, run: ```shell python -m maturin_import_hook --help ``` * `site (info | install | uninstall)` * Manage import hook installation in [`sitecustomize.py`](#sitecustomize) of the active environment. * `install` options: * `--force`: Whether to overwrite any existing managed import hook installation. * `--(no-)-project-importer`: Whether to enable the project importer. * `--(no-)-rs-file-importer`: Whether to enable the rs file importer. * `--(no-)-detect-uv`: Whether to automatically detect and use the `--uv` flag. * `--args`: The arguments to pass to `maturin`. * `--user`: whether to install into `usercustomize.py` instead of `sitecustomize.py`. * `cache (info | clear)` * Get info about the size and location of the build cache. * Manage the build cache of the active environment. * `version` * Show version info of the import hook and associated tools. Useful for providing information to bug reports. ## Features The import hook supports: * All the binding types and project layouts supported by Maturin. * All [CPython](https://www.python.org/) and [PyPy](https://pypy.org/) versions supported by Maturin. * Windows, Linux and MacOS. * Interactive environments such as [IPython](https://ipython.org/), [Jupyter](https://jupyter.org/) and [Python REPL](https://docs.python.org/3/tutorial/interpreter.html#interactive-mode). * Importing multiple Maturin projects in the same script. * Importing stand-alone `.rs` files that use `PyO3` bindings (see [Import Rust File](#import-rust-file)). * `importlib.reload()` (currently unsupported on Windows). * Detecting edits to path dependencies at any depth. * Multiple environments with different interpreters using isolated build caches. * Concurrent imports/builds from multiple scripts (useful for tools like [`pytest-xdist`](https://pypi.org/project/pytest-xdist/)) * Extensibility (see [Advanced Usage](#advanced-usage)). ## Manual Activation To activate the import hook in a single Python script, call `install()` at the top of the script: ```python import maturin_import_hook maturin_import_hook.install() # Must come first. Not active for imports above. import my_rust_package # An editable-installed Maturin project. import foo.my_rust_script # `foo/my_rust_script.rs` defines a pyo3 module. ``` The Maturin project importer and the Rust file importer can be used separately: ```python from maturin_import_hook import project_importer project_importer.install() from maturin_import_hook import rust_file_importer rust_file_importer.install() ``` The import hook can be configured to control its behaviour (see [Install Arguments](#install-arguments)). The import hook is intended for use in development environments and not for production environments, so any calls to `install()` should ideally be removed (or disabled, see [Environment Variables](#environment-variables)) before reaching production. This is another reason why the site installation is convenient. The hook remains active across multiple Python modules so a single `install()` at the top of the main module is sometimes sufficient. This can cause problems however, when the main module is not imported or not imported first, such as in tests. `install()` can be called in each module but be aware that each call replaces the previous settings. Tip: If you want to use non-defaults across multiple modules you can create a function like: ```python import maturin_import_hook _HOOK_INSTALLED = False def install_maturin_hook() -> None: global _HOOK_INSTALLED if not _HOOK_INSTALLED: maturin_import_hook.install( ... # your custom configuration here ) _HOOK_INSTALLED = True ``` Then call `install_maturin_hook()` at the top of each module. This will ensure the custom options are used everywhere without any code duplication. ## Import Rust File When a `.rs` file is imported, a Maturin project with [pyo3](https://pyo3.rs/) bindings will be created for it in the Maturin build cache and the resulting library will be built and loaded. For example, `my_extension.rs`: ```rust use pyo3::prelude::*; #[pyfunction] fn double(x: usize) -> usize { x * 2 } #[pymodule] fn my_extension(m: &Bound<'_, PyModule>) -> PyResult<()> { m.add_function(wrap_pyfunction!(double, m)?) } ``` The `#[pymodule]` must have the same name as the filename. The version of `pyo3` is determined by `maturin new --bindings pyo3`. ## Environment Variables The import hook can be disabled by setting `MATURIN_IMPORT_HOOK_ENABLED=0`. This can be used to disable the import hook in production while leaving calls to `install()` in place. Build files will be stored in an appropriate place for the current system but can be overridden by setting `MATURIN_BUILD_DIR`. These files can be deleted without causing any issues (unless a build is in progress). The precedence for storing build files is: * `MATURIN_BUILD_DIR` * (Each environment will store its cache in a subdirectory of the given path). * `/maturin_build_cache` * `/maturin_build_cache` * e.g. `~/.cache/maturin_build_cache` on POSIX. See the location being used with the CLI: `python -m maturin_import_hook cache info` ## Logging By default, the `maturin_import_hook` logger does not propagate to the root logger. This is so that `INFO` level messages are shown without having to configure logging (`INFO` level is normally not visible). The import hook also has extensive `DEBUG` level logging that would clutter application logs. So by not propagating, `DEBUG` messages from the import hook are not shown even if the root logger has `DEBUG` level visible. If you prefer, `maturin_import_hook.reset_logger()` can be called to undo the default configuration and propagate the messages as normal. When debugging issues with the import hook, you should first call `reset_logger()` then configure the root logger to show `DEBUG` messages. You can also run with the environment variable `RUST_LOG=maturin=debug` to get more information from Maturin. ```python import logging logging.basicConfig(format='%(asctime)s %(name)s [%(levelname)s] %(message)s', level=logging.DEBUG) import maturin_import_hook maturin_import_hook.reset_logger() maturin_import_hook.install() ``` ## Background ### Import Hook An [import hook](https://docs.python.org/3/reference/import.html#import-hooks) is a class added to [sys.meta_path](https://docs.python.org/3/library/sys.html#sys.meta_path) that is invoked by the Python interpreter whenever it encounters an `import` statement (or import via other means). `find_spec()` is called for each hook until one returns a `ModuleSpec` indicating that it found the module. If a hook cannot handle a particular import it returns `None`. Import hooks can be used to create modules on demand (e.g. the `.rs` importer) or trigger side-effects with imports (e.g. the project importer). [MaturinProjectImporter](https://github.com/PyO3/maturin-import-hook/blob/main/src/maturin_import_hook/project_importer.py): 1. Detects when an import corresponds to an editable installed Maturin project. 2. Determines if the current build of the project is up to date. 3. Rebuilds the project if necessary. [MaturinRustFileImporter](https://github.com/PyO3/maturin-import-hook/blob/main/src/maturin_import_hook/rust_file_importer.py): 1. Searches `sys.path` for an `.rs` file matching the import (e.g. for `import foo.bar`, the hook will look for `foo/bar.rs` at each search path). 2. Creates a temporary Maturin project for the `.rs` file or reuses the project if it already exists. 3. Rebuilds the project if necessary. The above steps are a simplification as supporting `importlib.reload()` requires more complex logic. See [reloading.md](https://github.com/PyO3/maturin-import-hook/blob/main/docs/reloading.md) for more details. ### Sitecustomize If a [sitecustomize.py](https://docs.python.org/3/library/site.html) file exists in the `site-packages` directory of a Python installation it is loaded automatically by the Python interpreter as it starts unless the `-S` flag is passed to it. This makes `sitecustomize.py` a convenient place to activate the import hook. ## Advanced Usage ### Install Arguments The arguments to `install()` are ([source](https://github.com/PyO3/maturin-import-hook/blob/main/src/maturin_import_hook/__init__.py)): ```python """ enable_project_importer: enable the hook for automatically rebuilding editable installed maturin projects enable_rs_file_importer: enable the hook for importing .rs files as though they were regular python modules enable_reloading: enable workarounds to allow the extension modules to be reloaded with `importlib.reload()` settings: settings corresponding to flags passed to maturin. build_dir: where to put the compiled artifacts. defaults to `$MATURIN_BUILD_DIR`, `sys.exec_prefix / 'maturin_build_cache'` or `$HOME/.cache/maturin_build_cache/` in order of preference force_rebuild: whether to always rebuild and skip checking whether anything has changed lock_timeout_seconds: a lock is required to prevent projects from being built concurrently. If the lock is not released before this timeout is reached the import hook stops waiting and aborts. A value of None means that the import hook will wait for the lock indefinitely. show_warnings: whether to show compilation warnings file_searcher: an object used to find source and installed project files that are used to determine whether a project has changed and needs to be rebuilt enable_automatic_install: whether to install detected packages using the import hook even if they are not already installed into the virtual environment or are installed in non-editable mode. """ ``` ### Subclassing The import hook classes can be subclassed to further customize to specific use cases. For example settings can be configured per-project or loaded from configuration files. ```python import sys from pathlib import Path from maturin_import_hook.settings import MaturinSettings from maturin_import_hook.project_importer import MaturinProjectImporter class CustomImporter(MaturinProjectImporter): def get_settings(self, module_path: str, source_path: Path) -> MaturinSettings: return MaturinSettings( release=True, strip=True, # ... ) sys.meta_path.insert(0, CustomImporter()) ``` ### Custom File Searching The `file_searcher` argument to `install()` can be used to customize which files should be included/excluded when deciding whether a package needs to be rebuilt: ```python from collections.abc import Iterator from pathlib import Path from maturin_import_hook.project_importer import ProjectFileSearcher, install class CustomFileSearcher(ProjectFileSearcher): def get_source_paths( self, project_dir: Path, all_path_dependencies: list[Path], installed_package_root: Path, ) -> Iterator[Path]: ... def get_installation_paths(self, installed_package_root: Path) -> Iterator[Path]: ... install(file_searcher=CustomFileSearcher()) ``` See `maturin_import_hook.project.DefaultProjectFileSearcher` for the default list of include/exclude criteria. The class variables of `DefaultProjectFileSearcher` can be edited **before** calling `install()` to make simple customizations.