| 1 |
# Distribution |
| 2 |
|
| 3 |
## Build Wheels |
| 4 |
|
| 5 |
When building Rust binary or library, it's possible to depend on libraries and symbols only available on the build machine. |
| 6 |
To ensure the wheels are portable, native python modules on linux must only dynamically link a set of libraries and symbols called manylinux and musllinux. |
| 7 |
The pypa offers special docker images to ensure compliance with the [manylinux rules](https://peps.python.org/pep-0599/#the-manylinux2014-policy). |
| 8 |
If you want to publish linux wheels on pypi, **you need to use a manylinux docker image or [build with zig](#use-zig)**. |
| 9 |
|
| 10 |
The Rust compiler since version 1.64 [requires at least glibc 2.17](https://blog.rust-lang.org/2022/08/01/Increasing-glibc-kernel-requirements.html), so you need to use at least manylinux2014. |
| 11 |
For publishing, we recommend enforcing the same manylinux version as the image with the manylinux flag, e.g. use `--manylinux 2014` if you are building in `quay.io/pypa/manylinux2014_x86_64`. |
| 12 |
The [PyO3/maturin-action](https://github.com/PyO3/maturin-action) github action already takes care of this if you set e.g. `manylinux: 2014`. |
| 13 |
|
| 14 |
If you are publishing to PyPI, you can use `--compatibility pypi` to allow only builds for targets that are accepted by PyPI, and reject builds for unsupported operating systems and architectures. |
| 15 |
|
| 16 |
maturin can check the generated library for manylinux compliance (an auditwheel reimplementation) and gives the wheel the proper platform tag. |
| 17 |
|
| 18 |
- If your system's glibc is too new, it will assign the `linux` tag. |
| 19 |
- If you link other shared libraries, maturin will try to bundle them within the wheel, note that this requires [patchelf](https://github.com/NixOS/patchelf), |
| 20 |
which can be installed along with maturin from PyPI: `pip install maturin[patchelf]`. |
| 21 |
|
| 22 |
You can also manually disable those checks and directly use native linux target with `--manylinux off`. |
| 23 |
|
| 24 |
The [pyo3/maturin](https://ghcr.io/pyo3/maturin) image is based on the manylinux2014 image, and passes arguments to the `maturin` binary. You can use it like this: |
| 25 |
|
| 26 |
``` |
| 27 |
docker run --rm -v $(pwd):/io ghcr.io/pyo3/maturin build --release # or other maturin arguments |
| 28 |
``` |
| 29 |
|
| 30 |
Note that this image is very basic and only contains python, maturin and stable Rust. If you need additional tools, you can run commands inside the manylinux container. |
| 31 |
See [konstin/complex-manylinux-maturin-docker](https://github.com/konstin/complex-manylinux-maturin-docker) for a small educational example |
| 32 |
or [nanoporetech/fast-ctc-decode](https://github.com/nanoporetech/fast-ctc-decode/blob/b226ea0f2b2f4f474eff47349703d57d2ea4801b/.github/workflows/publish.yml) for a real world setup. |
| 33 |
|
| 34 |
``` |
| 35 |
Usage: maturin build [OPTIONS] [ARGS]... |
| 36 |
|
| 37 |
Arguments: |
| 38 |
[ARGS]... |
| 39 |
Rustc flags |
| 40 |
|
| 41 |
Options: |
| 42 |
--strip |
| 43 |
Strip the library for minimum file size |
| 44 |
|
| 45 |
--sdist |
| 46 |
Build a source distribution |
| 47 |
|
| 48 |
--compatibility [<compatibility>...] |
| 49 |
Control platform tags. Use `pypi` to ensure PyPI compatibility, or specify platform-specific |
| 50 |
tags like `manylinux2014`, `musllinux_1_2`, or `linux`. |
| 51 |
|
| 52 |
The default is the lowest compatible `manylinux` tag, or plain `linux` if nothing matched |
| 53 |
|
| 54 |
-i, --interpreter [<INTERPRETER>...] |
| 55 |
The python versions to build wheels for, given as the executables of interpreters such as `python3.9` or `/usr/bin/python3.8` |
| 56 |
|
| 57 |
-f, --find-interpreter |
| 58 |
Find interpreters from the host machine |
| 59 |
|
| 60 |
-b, --bindings <BINDINGS> |
| 61 |
Which kind of bindings to use |
| 62 |
|
| 63 |
[possible values: pyo3, pyo3-ffi, cffi, uniffi, bin] |
| 64 |
|
| 65 |
-o, --out <OUT> |
| 66 |
The directory to store the built wheels in. Defaults to a new "wheels" directory in the project's target directory |
| 67 |
|
| 68 |
--auditwheel <AUDITWHEEL> |
| 69 |
Audit wheel for manylinux compliance |
| 70 |
|
| 71 |
Possible values: |
| 72 |
- repair: Audit and repair wheel for manylinux compliance |
| 73 |
- check: Check wheel for manylinux compliance, but do not repair |
| 74 |
- skip: Don't check for manylinux compliance |
| 75 |
|
| 76 |
--zig |
| 77 |
For manylinux targets, use zig to ensure compliance for the chosen manylinux version |
| 78 |
|
| 79 |
Default to manylinux2014/manylinux_2_17 if you do not specify an `--compatibility` |
| 80 |
|
| 81 |
Make sure you installed zig with `pip install maturin[zig]` |
| 82 |
|
| 83 |
-q, --quiet |
| 84 |
Do not print cargo log messages |
| 85 |
|
| 86 |
--ignore-rust-version |
| 87 |
Ignore `rust-version` specification in packages |
| 88 |
|
| 89 |
-v, --verbose... |
| 90 |
Use verbose output (-vv very verbose/build.rs output) |
| 91 |
|
| 92 |
--color <WHEN> |
| 93 |
Coloring: auto, always, never |
| 94 |
|
| 95 |
--config <KEY=VALUE> |
| 96 |
Override a configuration value (unstable) |
| 97 |
|
| 98 |
-Z <FLAG> |
| 99 |
Unstable (nightly-only) flags to Cargo, see 'cargo -Z help' for details |
| 100 |
|
| 101 |
--future-incompat-report |
| 102 |
Outputs a future incompatibility report at the end of the build (unstable) |
| 103 |
|
| 104 |
--compression-method <COMPRESSION_METHOD> |
| 105 |
Zip compresson method to use |
| 106 |
|
| 107 |
[default: deflated] |
| 108 |
|
| 109 |
Possible values: |
| 110 |
- deflated: Deflate compression |
| 111 |
- stored: No compression |
| 112 |
- zstd: Zstandard compression |
| 113 |
|
| 114 |
-h, --help |
| 115 |
Print help (see a summary with '-h') |
| 116 |
|
| 117 |
Compilation Options: |
| 118 |
-r, --release |
| 119 |
Build artifacts in release mode, with optimizations |
| 120 |
|
| 121 |
-j, --jobs <N> |
| 122 |
Number of parallel jobs, defaults to # of CPUs |
| 123 |
|
| 124 |
--profile <PROFILE-NAME> |
| 125 |
Build artifacts with the specified Cargo profile |
| 126 |
|
| 127 |
--target <TRIPLE> |
| 128 |
Build for the target triple |
| 129 |
|
| 130 |
[env: CARGO_BUILD_TARGET=] |
| 131 |
|
| 132 |
--target-dir <DIRECTORY> |
| 133 |
Directory for all generated artifacts |
| 134 |
|
| 135 |
--timings=<FMTS> |
| 136 |
Timing output formats (unstable) (comma separated): html, json |
| 137 |
|
| 138 |
Feature Selection: |
| 139 |
-F, --features <FEATURES> |
| 140 |
Space or comma separated list of features to activate |
| 141 |
|
| 142 |
--all-features |
| 143 |
Activate all available features |
| 144 |
|
| 145 |
--no-default-features |
| 146 |
Do not activate the `default` feature |
| 147 |
|
| 148 |
Manifest Options: |
| 149 |
-m, --manifest-path <PATH> |
| 150 |
Path to Cargo.toml |
| 151 |
|
| 152 |
--frozen |
| 153 |
Require Cargo.lock and cache are up to date |
| 154 |
|
| 155 |
--locked |
| 156 |
Require Cargo.lock is up to date |
| 157 |
|
| 158 |
--offline |
| 159 |
Run without accessing the network |
| 160 |
``` |
| 161 |
|
| 162 |
## Source Distribution |
| 163 |
|
| 164 |
Maturin supports building through `pyproject.toml`. To use it, create a `pyproject.toml` next to your `Cargo.toml` with the following content: |
| 165 |
|
| 166 |
```toml |
| 167 |
[build-system] |
| 168 |
requires = ["maturin>=1.0,<2.0"] |
| 169 |
build-backend = "maturin" |
| 170 |
``` |
| 171 |
|
| 172 |
If a `pyproject.toml` with a `[build-system]` entry is present, maturin can build a source distribution of your package when `--sdist` is specified. |
| 173 |
The source distribution will contain the same files as `cargo package`. To only build a source distribution, use the `maturin sdist` command. |
| 174 |
|
| 175 |
You can then e.g. install your package with `pip install .`. With `pip install . -v` you can see the output of cargo and maturin. |
| 176 |
|
| 177 |
You can use the options `compatibility`, `skip-auditwheel`, `bindings`, `strip` and common Cargo build options such as `features` under `[tool.maturin]` the same way you would when running maturin directly. |
| 178 |
The `bindings` key is required for cffi and bin projects as those can't be automatically detected. Currently, all builds are in release mode (see [this thread](https://discuss.python.org/t/pep-517-debug-vs-release-builds/1924) for details). |
| 179 |
|
| 180 |
For a non-manylinux build with cffi bindings you could use the following: |
| 181 |
|
| 182 |
```toml |
| 183 |
[build-system] |
| 184 |
requires = ["maturin>=1.0,<2.0"] |
| 185 |
build-backend = "maturin" |
| 186 |
|
| 187 |
[tool.maturin] |
| 188 |
bindings = "cffi" |
| 189 |
compatibility = "linux" |
| 190 |
``` |
| 191 |
|
| 192 |
`manylinux` option is also accepted as an alias of `compatibility` for backward compatibility with old version of maturin. |
| 193 |
|
| 194 |
To include arbitrary files in the sdist for use during compilation specify `include` as an array of `path` globs with `format` set to `sdist`: |
| 195 |
|
| 196 |
```toml |
| 197 |
[tool.maturin] |
| 198 |
include = [{ path = "path/**/*", format = "sdist" }] |
| 199 |
``` |
| 200 |
|
| 201 |
### Cross Compiling |
| 202 |
|
| 203 |
Maturin has decent cross compilation support for `pyo3` and `bin` bindings, |
| 204 |
other kind of bindings may work but aren't tested regularly. |
| 205 |
|
| 206 |
#### Cross-compile to Linux/macOS |
| 207 |
|
| 208 |
##### Use Docker |
| 209 |
|
| 210 |
For manylinux support the [manylinux-cross](https://github.com/rust-cross/manylinux-cross) docker images can be used. |
| 211 |
And [maturin-action](https://github.com/PyO3/maturin-action) makes it easy to do cross compilation on GitHub Actions. |
| 212 |
|
| 213 |
##### Use Zig |
| 214 |
|
| 215 |
Since v0.12.7 maturin added support for linking with [`zig cc`](https://andrewkelley.me/post/zig-cc-powerful-drop-in-replacement-gcc-clang.html), |
| 216 |
compile for Linux works and is regularly tested on CI, other platforms may also work but aren't tested regularly. |
| 217 |
|
| 218 |
You can install zig following the [official documentation](https://ziglang.org/download), or install it from PyPI via `pip install ziglang`. |
| 219 |
Then pass `--zig` to maturin `build` or `publish` commands to use it, for example |
| 220 |
|
| 221 |
```bash |
| 222 |
maturin build --release --target aarch64-unknown-linux-gnu --zig |
| 223 |
``` |
| 224 |
|
| 225 |
#### Cross-compile to Windows |
| 226 |
|
| 227 |
Pyo3 0.16.5 added an experimental feature `generate-import-lib` enables the user to cross compile |
| 228 |
extension modules for Windows targets without setting the `PYO3_CROSS_LIB_DIR` environment variable |
| 229 |
or providing any Windows Python library files. |
| 230 |
|
| 231 |
```toml |
| 232 |
[dependencies] |
| 233 |
pyo3 = { version = "0.25.0", features = ["extension-module", "generate-import-lib"] } |
| 234 |
``` |
| 235 |
|
| 236 |
It uses an external [`python3-dll-a`](https://docs.rs/python3-dll-a/latest/python3_dll_a/) crate to |
| 237 |
generate import libraries for the Python DLL for MinGW-w64 and MSVC compile targets. |
| 238 |
Note: MSVC targets require LLVM binutils or MSVC build tools to be available on the host system. |
| 239 |
More specifically, `python3-dll-a` requires `llvm-dlltool` or `lib.exe` executable to be present in `PATH` when targeting `*-pc-windows-msvc`. |
| 240 |
|
| 241 |
maturin integrates [`cargo-xwin`](https://github.com/messense/cargo-xwin) to enable MSVC targets cross compilation support, |
| 242 |
it will download and unpack the Microsoft CRT headers and import libraries, and Windows SDK headers and import libraries |
| 243 |
needed for compiling and linking automatically. |
| 244 |
|
| 245 |
**By using this to cross compiling to Windows MSVC targets you are consented to accept the license at [https://go.microsoft.com/fwlink/?LinkId=2086102](https://go.microsoft.com/fwlink/?LinkId=2086102)**. |
| 246 |
(Building on Windows natively does not apply.) |
| 247 |
|
| 248 |
## GitHub Actions |
| 249 |
|
| 250 |
If your project uses GitHub Actions, you can use the `maturin generate-ci` command to generate a GitHub Actions workflow file. |
| 251 |
|
| 252 |
```bash |
| 253 |
mkdir -p .github/workflows |
| 254 |
maturin generate-ci github > .github/workflows/CI.yml |
| 255 |
``` |
| 256 |
|
| 257 |
There are some options to customize the generated workflow file: |
| 258 |
|
| 259 |
``` |
| 260 |
Generate CI configuration |
| 261 |
|
| 262 |
Usage: maturin generate-ci [OPTIONS] <CI> |
| 263 |
|
| 264 |
Arguments: |
| 265 |
<CI> |
| 266 |
CI provider |
| 267 |
|
| 268 |
Possible values: |
| 269 |
- github: GitHub |
| 270 |
|
| 271 |
Options: |
| 272 |
-m, --manifest-path <PATH> |
| 273 |
Path to Cargo.toml |
| 274 |
|
| 275 |
-v, --verbose... |
| 276 |
Use verbose output. |
| 277 |
|
| 278 |
* Default: Show build information and `cargo build` output. * `-v`: Use `cargo build -v`. |
| 279 |
* `-vv`: Show debug logging and use `cargo build -vv`. * `-vvv`: Show trace logging. |
| 280 |
|
| 281 |
You can configure fine-grained logging using the `RUST_LOG` environment variable. |
| 282 |
(<https://docs.rs/tracing-subscriber/latest/tracing_subscriber/filter/struct.EnvFilter.html#directives>) |
| 283 |
|
| 284 |
-o, --output <PATH> |
| 285 |
Output path |
| 286 |
|
| 287 |
[default: -] |
| 288 |
|
| 289 |
--platform <platform>... |
| 290 |
Platform support |
| 291 |
|
| 292 |
[default: linux musllinux windows macos] |
| 293 |
|
| 294 |
Possible values: |
| 295 |
- all: All |
| 296 |
- manylinux: Manylinux |
| 297 |
- musllinux: Musllinux |
| 298 |
- windows: Windows |
| 299 |
- macos: macOS |
| 300 |
- emscripten: Emscripten |
| 301 |
|
| 302 |
--pytest |
| 303 |
Enable pytest |
| 304 |
|
| 305 |
--zig |
| 306 |
Use zig to do cross compilation |
| 307 |
|
| 308 |
--skip-attestation |
| 309 |
Skip artifact attestation |
| 310 |
|
| 311 |
-h, --help |
| 312 |
Print help (see a summary with '-h') |
| 313 |
``` |
| 314 |
|
| 315 |
### Using PyPI's trusted publishing |
| 316 |
|
| 317 |
By default, the workflow provided by `generate-ci` will publish the release artifacts to PyPI using API token authentication. However, maturin also supports [trusted publishing (OpenID Connect)](https://docs.pypi.org/trusted-publishers/). |
| 318 |
|
| 319 |
To enable it, modify the `release` action in the generated GitHub workflow file: |
| 320 |
|
| 321 |
- remove `MATURIN_PYPI_TOKEN` from the `env` section to make maturin use trusted publishing |
| 322 |
- add `id-token: write` to the action's `permissions` (see [Configuring OpenID Connect in PyPI](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-pypi) from GitHub's documentation). |
| 323 |
- if `Environment name: pypi` was set in PyPI, add `environment: pypi` |
| 324 |
|
| 325 |
Make sure to follow the steps listed in [PyPI's documentation](https://docs.pypi.org/trusted-publishers/adding-a-publisher/) to set up your GitHub repository as a trusted publisher in the PyPI project settings before attempting to run the workflow. |