| 1 |
########## |
| 2 |
Concepts |
| 3 |
########## |
| 4 |
|
| 5 |
********** |
| 6 |
Overview |
| 7 |
********** |
| 8 |
|
| 9 |
tox is an environment orchestrator. Use it to define how to set up and execute various tools on your projects. The tool |
| 10 |
can set up environments for and invoke: |
| 11 |
|
| 12 |
- test runners (such as :pypi:`pytest`), |
| 13 |
- linters (e.g., :pypi:`ruff`), |
| 14 |
- formatters (for example :pypi:`black` or :pypi:`isort`), |
| 15 |
- documentation generators (e.g., :pypi:`Sphinx`), |
| 16 |
- build and publishing tools (e.g., :pypi:`build` with :pypi:`twine`), |
| 17 |
- ... |
| 18 |
|
| 19 |
For a step-by-step introduction, see the :doc:`tutorial/getting-started` tutorial. For practical recipes, see |
| 20 |
:doc:`how-to/usage`. |
| 21 |
|
| 22 |
***************** |
| 23 |
System overview |
| 24 |
***************** |
| 25 |
|
| 26 |
Below is a graphical representation of the tox lifecycle: |
| 27 |
|
| 28 |
.. mermaid:: |
| 29 |
|
| 30 |
flowchart TD |
| 31 |
start(( )) --> config[Load configuration] |
| 32 |
|
| 33 |
config --> envloop |
| 34 |
|
| 35 |
subgraph envloop [for each selected environment] |
| 36 |
direction TB |
| 37 |
create[Create environment] |
| 38 |
create --> depmode{pylock set?} |
| 39 |
depmode -- yes --> pylock[Install from pylock.toml] |
| 40 |
depmode -- no --> deps[Install deps + dependency groups] |
| 41 |
pylock --> pkg{package project?} |
| 42 |
deps --> pkg |
| 43 |
pkg -- yes --> build[Build and install package] |
| 44 |
pkg -- no --> extra |
| 45 |
build --> extra[Run extra_setup_commands] |
| 46 |
extra --> cmds |
| 47 |
cmds[Run commands] |
| 48 |
end |
| 49 |
|
| 50 |
cmds --> report[Report results] |
| 51 |
report --> done(( )) |
| 52 |
|
| 53 |
classDef configStyle fill:#dbeafe,stroke:#3b82f6,stroke-width:2px,color:#1e3a5f |
| 54 |
classDef envStyle fill:#dcfce7,stroke:#22c55e,stroke-width:2px,color:#14532d |
| 55 |
classDef pkgStyle fill:#ffedd5,stroke:#f97316,stroke-width:2px,color:#7c2d12 |
| 56 |
classDef setupStyle fill:#fde68a,stroke:#fbbf24,stroke-width:2px,color:#78350f |
| 57 |
classDef cmdStyle fill:#ede9fe,stroke:#8b5cf6,stroke-width:2px,color:#3b0764 |
| 58 |
classDef reportStyle fill:#ccfbf1,stroke:#14b8a6,stroke-width:2px,color:#134e4a |
| 59 |
classDef decisionStyle fill:#fef9c3,stroke:#eab308,stroke-width:2px,color:#713f12 |
| 60 |
classDef pylockStyle fill:#e0e7ff,stroke:#6366f1,stroke-width:2px,color:#312e81 |
| 61 |
|
| 62 |
class config configStyle |
| 63 |
class create,deps envStyle |
| 64 |
class build pkgStyle |
| 65 |
class extra setupStyle |
| 66 |
class cmds cmdStyle |
| 67 |
class report reportStyle |
| 68 |
class depmode,pkg decisionStyle |
| 69 |
class pylock pylockStyle |
| 70 |
|
| 71 |
The primary tox states are: |
| 72 |
|
| 73 |
1. **Configuration:** load tox configuration files (such as ``tox.toml``, ``tox.ini``, ``pyproject.toml`` and |
| 74 |
``toxfile.py``) and merge it with options from the command line plus the operating system environment variables. |
| 75 |
Configuration is loaded lazily -- individual environment settings are only read when that environment is used. |
| 76 |
2. **Environment**: for each selected tox environment (e.g. ``3.13``, ``lint``) do: |
| 77 |
|
| 78 |
1. **Creation**: create a fresh environment; by default :pypi:`virtualenv` is used, but configurable via |
| 79 |
:ref:`runner`. For ``virtualenv`` tox will use the `virtualenv discovery logic |
| 80 |
<https://virtualenv.pypa.io/en/latest/user_guide.html#python-discovery>`_ where the python specification is |
| 81 |
defined by the tox environments :ref:`base_python` (if not set will try to extract it from the environment name, |
| 82 |
then fall back to :ref:`default_base_python`, and finally to the Python running tox). The specification can |
| 83 |
include a CPU architecture suffix (e.g. ``cpython3.12-64-arm64``) to constrain discovery to a specific ISA — the |
| 84 |
architecture is derived from :func:`python:sysconfig.get_platform` and validated after discovery. This is created |
| 85 |
at first run only to be reused at subsequent runs. If certain aspects of the project change (python version, |
| 86 |
dependencies removed, etc.), a re-creation of the environment is automatically triggered. To force the recreation |
| 87 |
tox can be invoked with the :ref:`recreate` flag (``-r``). When recreation occurs, any :ref:`recreate_commands` |
| 88 |
run inside the old environment before its directory is removed -- this lets tools like pre-commit clean their |
| 89 |
external caches. Failures in these commands are logged as warnings but never block the recreation. |
| 90 |
2. **Install dependencies** (optional): install the environment dependencies. When :ref:`pylock` is set, tox installs |
| 91 |
locked dependencies from the :PEP:`751` lock file (filtered by extras, dependency groups, and platform markers). |
| 92 |
Otherwise, it installs :ref:`deps` and :ref:`dependency_groups`. By default ``pip`` is used to install packages, |
| 93 |
however one can customize this via ``install_command``. |
| 94 |
3. **Packaging** (optional): create a distribution of the current project (see :ref:`packaging` below). |
| 95 |
|
| 96 |
Steps 2 and 3 can be selectively skipped with CLI flags: |
| 97 |
|
| 98 |
- ``--skip-pkg-install`` skips step 3 only (packaging and package installation), while still installing dependencies. |
| 99 |
- ``--skip-env-install`` skips both steps 2 and 3 entirely, reusing the environment as-is. This is useful when |
| 100 |
working offline or when the environment is already fully set up from a previous run. See :ref:`skip-env-install` |
| 101 |
for practical usage. |
| 102 |
|
| 103 |
4. **Extra setup commands** (optional): run the :ref:`extra_setup_commands` specified. These execute after all |
| 104 |
installations complete but before test commands, and run during the ``--notest`` phase. |
| 105 |
5. **Commands**: run the specified commands in the specified order. Whenever the exit code of any of them is not |
| 106 |
zero, stop and mark the environment failed. When you start a command with a dash character, the exit code will be |
| 107 |
ignored. If :ref:`commands_retry` is set, failed commands are retried up to the configured number of times before |
| 108 |
being treated as a failure. |
| 109 |
|
| 110 |
3. **Report** print out a report of outcomes for each tox environment: |
| 111 |
|
| 112 |
.. code-block:: bash |
| 113 |
|
| 114 |
____________________ summary ____________________ |
| 115 |
3.13: commands succeeded |
| 116 |
ERROR: 3.12: commands failed |
| 117 |
|
| 118 |
Only if all environments ran successfully tox will return exit code ``0`` (success). In this case you'll also see the |
| 119 |
message ``congratulations :)``. |
| 120 |
|
| 121 |
Environment variable handling |
| 122 |
============================= |
| 123 |
|
| 124 |
tox takes care of environment variable isolation. By default, it removes system environment variables not specified via |
| 125 |
:ref:`pass_env` and alters the ``PATH`` variable so that commands resolve within the current active tox environment. |
| 126 |
External commands need to be explicitly allowed via :ref:`allowlist_externals`. |
| 127 |
|
| 128 |
**Evaluation order**: tox composes the environment for command execution in this order: |
| 129 |
|
| 130 |
1. **pass_env** -- glob patterns are matched against the host ``os.environ``. Matching variables are passed through. |
| 131 |
2. **disallow_pass_env** -- exclusion patterns are applied to remove specific variables after ``pass_env`` expansion. |
| 132 |
3. **PATH** -- tox prepends the virtual environment's binary directory to ``PATH``, so commands resolve inside the |
| 133 |
virtualenv first. |
| 134 |
4. **set_env** -- values defined here are applied last and can override anything from the previous steps, including |
| 135 |
``PATH``. |
| 136 |
5. **Injected variables** -- tox adds ``TOX_ENV_NAME``, ``TOX_WORK_DIR``, ``TOX_ENV_DIR``, ``VIRTUAL_ENV``, |
| 137 |
``PIP_USER=0``, and ``PYTHONIOENCODING=utf-8``. These cannot be overridden. |
| 138 |
|
| 139 |
**PATH behavior**: because tox prepends the virtualenv ``bin/`` directory to ``PATH`` at step 3, commands like |
| 140 |
``python`` and ``pip`` resolve to the virtualenv versions. If you override ``PATH`` in ``set_env``, be aware that this |
| 141 |
replaces the composed ``PATH`` entirely -- you should include ``{env_bin_dir}`` in your custom value to preserve |
| 142 |
virtualenv resolution: |
| 143 |
|
| 144 |
.. tab:: TOML |
| 145 |
|
| 146 |
.. code-block:: toml |
| 147 |
|
| 148 |
[env_run_base] |
| 149 |
set_env.PATH = "{env_bin_dir}{:}/usr/local/bin{:}{env:PATH}" |
| 150 |
|
| 151 |
.. tab:: INI |
| 152 |
|
| 153 |
.. code-block:: ini |
| 154 |
|
| 155 |
[testenv] |
| 156 |
set_env = |
| 157 |
PATH = {env_bin_dir}{:}/usr/local/bin{:}{env:PATH} |
| 158 |
|
| 159 |
**Conditional environment variables**: in INI, ``set_env`` supports PEP 508-style markers separated by ``;`` to |
| 160 |
conditionally set variables based on platform: |
| 161 |
|
| 162 |
.. code-block:: ini |
| 163 |
|
| 164 |
[testenv] |
| 165 |
set_env = |
| 166 |
COVERAGE_FILE = {work_dir}/.coverage.{env_name} |
| 167 |
LDFLAGS = -L/usr/local/lib ; sys_platform == "darwin" |
| 168 |
|
| 169 |
.. _conditional-values-explained: |
| 170 |
|
| 171 |
Conditional value evaluation |
| 172 |
============================ |
| 173 |
|
| 174 |
.. versionadded:: 4.40 |
| 175 |
|
| 176 |
.. versionchanged:: 4.50 Added ``factor['NAME']``/``env['VAR']`` subscript syntax and ``env_name`` variable. |
| 177 |
|
| 178 |
TOML configurations support ``replace = "if"`` to conditionally select values at configuration load time. The |
| 179 |
``condition`` field accepts expressions that are parsed using Python's ``ast`` module and evaluated against the host |
| 180 |
``os.environ``. |
| 181 |
|
| 182 |
The expression language supports: |
| 183 |
|
| 184 |
- ``env.VAR`` -- resolves to the value of the environment variable ``VAR``, or empty string if unset. An empty string is |
| 185 |
falsy, any non-empty string is truthy. |
| 186 |
- ``env['VAR']`` -- same as ``env.VAR``, useful when the variable name needs to be in a string. |
| 187 |
- ``factor.NAME`` -- resolves to ``True`` if ``NAME`` is a factor in the environment name or platform, ``False`` |
| 188 |
otherwise. |
| 189 |
- ``factor['NAME']`` -- same as ``factor.NAME``, for names that aren't valid Python identifiers. Use this for version |
| 190 |
numbers like ``factor['3.14']`` since ``factor.3.14`` would be a syntax error. |
| 191 |
- ``env_name`` -- the full environment name as a string (e.g., ``test-3.14``). Use for exact name matching. |
| 192 |
- ``'literal'`` -- a string literal for comparison. |
| 193 |
- ``==``, ``!=`` -- string equality and inequality. |
| 194 |
- ``and``, ``or``, ``not`` -- boolean combinators with standard Python precedence. |
| 195 |
|
| 196 |
Conditions are evaluated before ``set_env`` is applied. The ``env.VAR`` lookup reads directly from ``os.environ``, not |
| 197 |
from the tox ``set_env`` configuration. This avoids circular dependencies -- a ``set_env`` value can use ``replace = |
| 198 |
"if"`` to check a host variable without triggering a recursive load. |
| 199 |
|
| 200 |
Both ``then`` and ``else`` values are processed through the normal TOML replacement pipeline, so they can contain nested |
| 201 |
substitutions like ``{env_name}`` or ``{ replace = "env", ... }``. Only the selected branch is evaluated. |
| 202 |
|
| 203 |
For syntax details and examples, see :ref:`conditional-value-reference`. For practical recipes, see |
| 204 |
:ref:`howto_conditional_values`. |
| 205 |
|
| 206 |
Dependency change detection |
| 207 |
=========================== |
| 208 |
|
| 209 |
tox discovers package dependency changes (via :PEP:`621` or :PEP:`517` |
| 210 |
``prepare_metadata_for_build_wheel``/``build_wheel`` metadata). When new dependencies are added they are installed on |
| 211 |
the next run. When a dependency is removed the entire environment is automatically recreated. This also works for |
| 212 |
``requirements`` files within :ref:`deps`. In most cases you should never need to use the ``--recreate`` flag -- tox |
| 213 |
detects changes and applies them automatically. |
| 214 |
|
| 215 |
.. _pylock-explanation: |
| 216 |
|
| 217 |
Lock file installation (PEP 751) |
| 218 |
================================ |
| 219 |
|
| 220 |
.. versionadded:: 4.44 |
| 221 |
|
| 222 |
The :ref:`pylock` setting installs dependencies from a :PEP:`751` lock file (``pylock.toml``). It is mutually exclusive |
| 223 |
with :ref:`deps` — a lock file already contains all transitive dependencies with exact versions, so mixing both sources |
| 224 |
would create conflicts. Lock files differ from ``deps`` in that every dependency is already resolved — the file contains |
| 225 |
exact versions, markers, and artifact URLs for every package. |
| 226 |
|
| 227 |
**Extras and dependency groups:** lock files can declare ``extras`` and ``dependency-groups`` at the top level, with |
| 228 |
per-package markers like ``'docs' in extras`` or ``'dev' in dependency_groups``. Use the existing :ref:`extras` and |
| 229 |
:ref:`dependency_groups` settings to select which groups to include — tox evaluates these markers together with platform |
| 230 |
markers (``sys_platform``, ``python_version``, etc.) against the **target** Python interpreter (not the host running |
| 231 |
tox) to filter packages at install time. |
| 232 |
|
| 233 |
**How it works today:** pip does not yet support installing from ``pylock.toml`` directly. tox parses the lock file |
| 234 |
using the ``packaging.pylock`` module, evaluates markers to filter packages, transpiles matching packages to a temporary |
| 235 |
requirements file (``{env_dir}/pylock.txt``), and passes it to pip with ``--no-deps``. The ``--no-deps`` flag prevents |
| 236 |
pip from re-resolving transitive dependencies, ensuring the exact versions from the lock file are installed. |
| 237 |
|
| 238 |
**Plugin support:** the ``Pylock`` object is passed through the ``tox_on_install`` plugin hook. Installer plugins can |
| 239 |
inspect it via ``isinstance(arguments, Pylock)`` and handle lock files natively (e.g. ``uv pip install --pylock``) |
| 240 |
instead of relying on the transpile path. |
| 241 |
|
| 242 |
**Future:** when pip gains native ``pylock.toml`` support, the transpile step will be replaced with a direct pip |
| 243 |
invocation. No configuration changes will be needed. |
| 244 |
|
| 245 |
**Change detection** works the same as for ``deps``: tox caches the resolved requirements. When packages are added, only |
| 246 |
the new ones are installed. When packages are removed, the environment is recreated. |
| 247 |
|
| 248 |
.. _pep723-explanation: |
| 249 |
|
| 250 |
Inline script metadata (PEP 723) |
| 251 |
================================ |
| 252 |
|
| 253 |
.. versionadded:: 4.52 |
| 254 |
|
| 255 |
:PEP:`723` lets Python scripts declare their dependencies and required Python version directly in comments at the top of |
| 256 |
the file: |
| 257 |
|
| 258 |
.. code-block:: python |
| 259 |
|
| 260 |
# /// script |
| 261 |
# requires-python = ">=3.12" |
| 262 |
# dependencies = ["requests>=2.31", "rich"] |
| 263 |
# /// |
| 264 |
|
| 265 |
import requests |
| 266 |
from rich import print |
| 267 |
|
| 268 |
print(requests.get("https://httpbin.org/get").json()) |
| 269 |
|
| 270 |
The ``virtualenv-pep-723`` runner reads this metadata so that a tox environment needs only a ``runner`` and ``script`` |
| 271 |
key — no ``deps`` or ``base_python`` duplication. Both ``requires-python`` and ``dependencies`` are optional per the PEP |
| 272 |
723 spec; omitting ``requires-python`` uses the host Python, and omitting ``dependencies`` installs nothing. |
| 273 |
|
| 274 |
**How it differs from the default runner:** the ``virtualenv-pep-723`` runner skips ``PythonRun`` in the class |
| 275 |
hierarchy. Config keys like ``deps``, ``extras``, ``dependency_groups``, and ``pylock`` are structurally absent — they |
| 276 |
do not exist for this runner. Packaging is unconditionally disabled since scripts are not packages. Setting |
| 277 |
``base_python`` explicitly is rejected to prevent conflicts with the script's ``requires-python`` specifier. |
| 278 |
|
| 279 |
**Python version resolution:** the runner uses the normal Python discovery (env name factors, ``.python-version`` files, |
| 280 |
or the host Python running tox). When ``requires-python`` is present, tox validates that the discovered Python satisfies |
| 281 |
the constraint and fails with a clear error if it does not. This works correctly with free-threaded interpreters and |
| 282 |
avoids forcing an unnecessarily old Python version. |
| 283 |
|
| 284 |
**Plugin support:** the ``Pep723Mixin`` class at ``tox.tox_env.python.pep723`` contains all the PEP 723 logic |
| 285 |
independent of the venv backend. Third-party plugins (e.g. tox-uv) can compose it with their own venv implementation |
| 286 |
without duplicating code. |
| 287 |
|
| 288 |
Open-ended range bounds |
| 289 |
======================= |
| 290 |
|
| 291 |
Both INI and TOML support generative environment lists with open-ended ranges. INI uses curly-brace syntax |
| 292 |
(``3.{10-}``), while TOML uses range dicts that appear directly in ``env_list`` (``{ prefix = "3.", start = 10 }``) -- |
| 293 |
no ``product`` wrapper is needed for a single axis. Instead of probing the system for available interpreters (which |
| 294 |
would be slow and environment-dependent), tox tracks the `supported CPython versions |
| 295 |
<https://devguide.python.org/versions/>`_ via two constants: |
| 296 |
|
| 297 |
- ``LATEST_PYTHON_MINOR_MIN`` -- the oldest supported CPython minor version (currently **10**, for Python 3.10) |
| 298 |
- ``LATEST_PYTHON_MINOR_MAX`` -- the latest supported CPython minor version (currently **14**, for Python 3.14) |
| 299 |
|
| 300 |
These values are updated with each tox release. A right-open range ``{10-}`` uses ``LATEST_PYTHON_MINOR_MAX`` as its |
| 301 |
upper bound; a left-open range ``{-13}`` uses ``LATEST_PYTHON_MINOR_MIN`` as its lower bound. |
| 302 |
|
| 303 |
This design is deterministic and fast -- the expansion happens at configuration load time with no I/O -- while keeping |
| 304 |
environment lists future-proof across tox upgrades. Environments for interpreters not installed on the system are |
| 305 |
naturally skipped by the :ref:`skip_missing_interpreters` setting. |
| 306 |
|
| 307 |
Why TOML composes with structured dicts instead of a string DSL |
| 308 |
--------------------------------------------------------------- |
| 309 |
|
| 310 |
INI's ``py3{10-14}-django{42,50}`` expression is a compact mini-language that parses curly-brace groups and numeric |
| 311 |
ranges out of the string itself. TOML deliberately does not do this. String items in ``env_list`` are always literal |
| 312 |
environment names; composition happens exclusively through structured dicts (range, labeled, and ``product``). |
| 313 |
|
| 314 |
This keeps the contract between configuration and parser minimal: TOML already handles quoting, escaping, and types, so |
| 315 |
there is no second grammar to learn and no ambiguity between "env name that happens to contain braces" and |
| 316 |
"expand-this-pattern". It also keeps the loader trivial — there is no string expansion pass, no recursive brace |
| 317 |
tokeniser, and no interaction with tox's other ``{...}`` substitutions (``{env_name}``, ``{posargs}``, |
| 318 |
``{factor:label}``). New matrix ergonomics are added by introducing a new dict shape, not by growing the DSL inside |
| 319 |
strings. |
| 320 |
|
| 321 |
Environment templates (``env_base``) |
| 322 |
==================================== |
| 323 |
|
| 324 |
tox provides three levels for configuring environments: |
| 325 |
|
| 326 |
1. **``env_run_base``** -- global defaults inherited by all run environments. Use this when every environment shares the |
| 327 |
same setting (e.g. a common ``deps`` or ``commands``). |
| 328 |
2. **``env_base.{name}``** -- named templates that generate multiple environments from factor combinations. Use this |
| 329 |
when a group of environments shares configuration but differs from other groups. The ``factors`` key defines which |
| 330 |
environments to generate via Cartesian product; all other keys in the section become the template's defaults. |
| 331 |
3. **``env.{name}``** -- explicit per-environment configuration. Use this for one-off environments (like ``lint``) or to |
| 332 |
override specific settings in a generated environment. |
| 333 |
|
| 334 |
The inheritance chain resolves bottom-up: ``env.{name}`` > ``env_base.{template}`` > ``env_run_base``. A setting in |
| 335 |
``env.{name}`` shadows the same setting in ``env_base``, which in turn shadows ``env_run_base``. |
| 336 |
|
| 337 |
Templates themselves are not runnable environments -- they exist only to define shared configuration. Only the generated |
| 338 |
environments (template name + factor suffix) appear in ``tox list`` and can be run. |
| 339 |
|
| 340 |
For the configuration reference, see :ref:`env-base-templates`. For practical recipes, see :ref:`howto_env_base_matrix`. |
| 341 |
|
| 342 |
*************** |
| 343 |
Main features |
| 344 |
*************** |
| 345 |
|
| 346 |
- **automation of tedious Python related test activities** |
| 347 |
- **test your Python package against many interpreter and dependency configurations** |
| 348 |
|
| 349 |
- automatic customizable (re)creation of :pypi:`virtualenv` test environments |
| 350 |
- installs your project into each virtual environment |
| 351 |
- test-tool agnostic: runs pytest, unittest, or any other test runner |
| 352 |
|
| 353 |
- **plugin system** to modify tox execution with simple hooks |
| 354 |
- uses :pypi:`pip` and :pypi:`virtualenv` by default; plugins can replace either |
| 355 |
- **cross-Python compatible**: tox requires CPython 3.10 and higher, but it can create environments for older versions. |
| 356 |
Special configuration might be required: :ref:`eol-version-support`. |
| 357 |
- **cross-platform**: Windows, macOS and Unix style environments |
| 358 |
- **full interoperability with devpi**: is integrated with and is used for testing in the :pypi:`devpi` system, a |
| 359 |
versatile PyPI index server and release managing tool |
| 360 |
- **concise reporting** about tool invocations and configuration errors |
| 361 |
- supports using different / multiple PyPI index servers |
| 362 |
|
| 363 |
.. _packaging: |
| 364 |
|
| 365 |
*********** |
| 366 |
Packaging |
| 367 |
*********** |
| 368 |
|
| 369 |
tox builds projects in a :PEP:`518` compatible virtual environment and communicates with the build backend according to |
| 370 |
the interface defined in :PEP:`517` and :PEP:`660`. To define package build dependencies and specify the build backend |
| 371 |
to use, create a ``pyproject.toml`` at the root of the project. For example to use hatch: |
| 372 |
|
| 373 |
.. code-block:: toml |
| 374 |
|
| 375 |
[build-system] |
| 376 |
build-backend = "hatchling.build" |
| 377 |
requires = ["hatchling>=0.22", "hatch-vcs>=0.2"] |
| 378 |
|
| 379 |
Package modes |
| 380 |
============= |
| 381 |
|
| 382 |
The :ref:`package` configuration controls how the project is packaged: |
| 383 |
|
| 384 |
.. mermaid:: |
| 385 |
|
| 386 |
flowchart TD |
| 387 |
pkg{package setting} |
| 388 |
pkg -- sdist --> sdist[sdist — default] |
| 389 |
pkg -- wheel --> wheel[wheel — faster install] |
| 390 |
pkg -- sdist-wheel --> sdistwheel[sdist-wheel — build wheel from sdist] |
| 391 |
pkg -- editable --> editable[editable — PEP 660] |
| 392 |
pkg -- editable-legacy --> legacy[editable-legacy — pip -e] |
| 393 |
pkg -- skip --> skip[skip — no packaging] |
| 394 |
pkg -- external --> external[external — provided] |
| 395 |
|
| 396 |
sdist --> env_pkg[build env: .pkg] |
| 397 |
wheel --> env_wheel[build env: .pkg-cpython313] |
| 398 |
sdistwheel --> env_pkg[sdist in: .pkg] |
| 399 |
sdistwheel --> env_wheel2[wheel in: .pkg-cpython313] |
| 400 |
editable --> env_wheel |
| 401 |
legacy --> env_pkg |
| 402 |
skip --> no_env[no build env] |
| 403 |
external --> no_env |
| 404 |
|
| 405 |
classDef decisionStyle fill:#fef9c3,stroke:#eab308,stroke-width:2px,color:#713f12 |
| 406 |
classDef pkgStyle fill:#ffedd5,stroke:#f97316,stroke-width:2px,color:#7c2d12 |
| 407 |
classDef envStyle fill:#dcfce7,stroke:#22c55e,stroke-width:2px,color:#14532d |
| 408 |
classDef skipStyle fill:#f3f4f6,stroke:#9ca3af,stroke-width:2px,color:#374151 |
| 409 |
|
| 410 |
class pkg decisionStyle |
| 411 |
class sdist,wheel,sdistwheel,editable,legacy,external pkgStyle |
| 412 |
class env_pkg,env_wheel,env_wheel2 envStyle |
| 413 |
class skip,no_env skipStyle |
| 414 |
|
| 415 |
- ``sdist`` (default): builds a source distribution |
| 416 |
- ``wheel``: builds a wheel (much faster to install) |
| 417 |
- ``sdist-wheel``: builds a source distribution first, then builds a wheel from that sdist (validates sdist |
| 418 |
completeness) |
| 419 |
- ``editable``: builds an editable wheel as defined by :PEP:`660` |
| 420 |
- ``editable-legacy``: invokes pip with ``-e`` (fallback when the backend doesn't support PEP 660) |
| 421 |
- ``skip``: skips packaging entirely (useful for tools like linters that don't need the project installed) |
| 422 |
- ``external``: uses an externally provided package |
| 423 |
|
| 424 |
Build environments |
| 425 |
================== |
| 426 |
|
| 427 |
tox uses a virtual environment for building, whose name depends on the artifact type: |
| 428 |
|
| 429 |
- For source distributions: the :ref:`package_env` (default ``.pkg``) |
| 430 |
- For wheels: the :ref:`wheel_build_env` (default ``.pkg-<impl><version>``, e.g. ``.pkg-cpython313``) |
| 431 |
- For sdist-wheel: uses two environments — the :ref:`package_env` for building the sdist, and the :ref:`wheel_build_env` |
| 432 |
(default ``.pkg-<impl><version>``) for building the wheel from the extracted sdist |
| 433 |
|
| 434 |
For pure Python projects (no C extensions), set :ref:`wheel_build_env` to the same value as :ref:`package_env`. This way |
| 435 |
the wheel is built once and reused for all tox environments: |
| 436 |
|
| 437 |
.. tab:: TOML |
| 438 |
|
| 439 |
.. code-block:: toml |
| 440 |
|
| 441 |
[env_run_base] |
| 442 |
package = "wheel" |
| 443 |
wheel_build_env = ".pkg" |
| 444 |
|
| 445 |
.. tab:: INI |
| 446 |
|
| 447 |
.. code-block:: ini |
| 448 |
|
| 449 |
[testenv] |
| 450 |
package = wheel |
| 451 |
wheel_build_env = .pkg |
| 452 |
|
| 453 |
Packaging environment configuration |
| 454 |
=================================== |
| 455 |
|
| 456 |
Packaging environments do **not** inherit settings from ``env_run_base`` (TOML) or ``[testenv]`` (INI). Instead, they |
| 457 |
inherit from the ``env_pkg_base`` table (TOML) or ``[pkgenv]`` section (INI). This prevents test settings from |
| 458 |
conflicting with packaging settings. |
| 459 |
|
| 460 |
.. tab:: TOML |
| 461 |
|
| 462 |
.. code-block:: toml |
| 463 |
|
| 464 |
[env_pkg_base] |
| 465 |
pass_env = ["PKG_CONFIG", "PKG_CONFIG_PATH"] |
| 466 |
|
| 467 |
.. tab:: INI |
| 468 |
|
| 469 |
.. code-block:: ini |
| 470 |
|
| 471 |
[pkgenv] |
| 472 |
pass_env = |
| 473 |
PKG_CONFIG |
| 474 |
PKG_CONFIG_PATH |
| 475 |
|
| 476 |
To configure a specific packaging environment, use the standard environment syntax (e.g. ``[testenv:.pkg]`` in INI or |
| 477 |
``env.".pkg"`` in TOML). |
| 478 |
|
| 479 |
.. _auto-provisioning: |
| 480 |
|
| 481 |
******************* |
| 482 |
Auto-provisioning |
| 483 |
******************* |
| 484 |
|
| 485 |
When the installed tox version does not satisfy either the :ref:`requires` or the :ref:`min_version`, tox automatically |
| 486 |
creates a virtual environment under :ref:`provision_tox_env` name that satisfies those constraints and delegates all |
| 487 |
calls to this meta environment. |
| 488 |
|
| 489 |
.. mermaid:: |
| 490 |
|
| 491 |
flowchart TD |
| 492 |
invoke[tox invoked] --> check{requires and min_version satisfied?} |
| 493 |
check -- yes --> run[run normally] |
| 494 |
check -- no --> create[create .tox/.tox virtualenv] |
| 495 |
create --> install[install tox + required plugins] |
| 496 |
install --> delegate[re-invoke tox inside .tox/.tox] |
| 497 |
delegate --> run |
| 498 |
|
| 499 |
classDef decisionStyle fill:#fef9c3,stroke:#eab308,stroke-width:2px,color:#713f12 |
| 500 |
classDef envStyle fill:#dcfce7,stroke:#22c55e,stroke-width:2px,color:#14532d |
| 501 |
classDef cmdStyle fill:#ede9fe,stroke:#8b5cf6,stroke-width:2px,color:#3b0764 |
| 502 |
|
| 503 |
class check decisionStyle |
| 504 |
class create,install envStyle |
| 505 |
class invoke,run,delegate cmdStyle |
| 506 |
|
| 507 |
For example given: |
| 508 |
|
| 509 |
.. tab:: TOML |
| 510 |
|
| 511 |
.. code-block:: toml |
| 512 |
|
| 513 |
requires = ["tox>=4", "tox-uv>=1"] |
| 514 |
|
| 515 |
.. tab:: INI |
| 516 |
|
| 517 |
.. code-block:: ini |
| 518 |
|
| 519 |
[tox] |
| 520 |
requires = |
| 521 |
tox>=4 |
| 522 |
tox-uv>=1 |
| 523 |
|
| 524 |
tox will automatically ensure that both the minimum version and requires constraints are satisfied, by creating a |
| 525 |
virtual environment under ``.tox``, and then installing into it ``tox>=4`` and ``tox-uv>=1``. Afterwards all tox |
| 526 |
invocations are forwarded to the tox installed inside ``.tox/.tox`` (referred to as the provisioned tox). |
| 527 |
|
| 528 |
This allows tox to automatically set up itself with all its plugins for the current project. If the host tox satisfies |
| 529 |
the constraints no provisioning is done (to avoid setup cost and indirection). |
| 530 |
|
| 531 |
.. note:: |
| 532 |
|
| 533 |
The provisioning environment (``.tox`` by default) does **not** inherit settings from ``env_run_base`` (TOML) or |
| 534 |
``[testenv]`` (INI). It must be explicitly configured if you need to customize it (e.g. ``env.".tox"`` in TOML or |
| 535 |
``[testenv:.tox]`` in INI). |
| 536 |
|
| 537 |
.. _parallel_mode: |
| 538 |
|
| 539 |
*************** |
| 540 |
Parallel mode |
| 541 |
*************** |
| 542 |
|
| 543 |
tox allows running environments in parallel mode via the ``parallel`` sub-command: |
| 544 |
|
| 545 |
- After the packaging phase completes tox runs environments in parallel processes (multi-thread based). |
| 546 |
- The ``--parallel`` flag takes an argument specifying the degree of parallelization, defaulting to ``auto``: |
| 547 |
|
| 548 |
- ``all`` to run all invoked environments in parallel, |
| 549 |
- ``auto`` to limit it to CPU count, |
| 550 |
- or pass an integer to set that limit. |
| 551 |
|
| 552 |
- Parallel mode displays a progress spinner while running environments in parallel, and reports outcome as soon as |
| 553 |
environments complete. To run without the spinner, use ``--parallel-no-spinner``. |
| 554 |
- Parallel mode by default shows output only of failed environments and ones marked as :ref:`parallel_show_output` ``= |
| 555 |
true``. |
| 556 |
- Environments can declare dependencies on other environments via :ref:`depends`. tox re-orders the environment list to |
| 557 |
satisfy these dependencies (also for sequential runs). In parallel mode, tox only schedules an environment once all of |
| 558 |
its dependencies have finished (independent of their outcome). |
| 559 |
|
| 560 |
``depends`` supports glob patterns (``*``, ``?``, ``[seq]``) using :py:mod:`fnmatch`, so instead of listing each |
| 561 |
environment explicitly you can write ``depends = 3.*`` to match all environments starting with ``3.``. |
| 562 |
|
| 563 |
.. warning:: |
| 564 |
|
| 565 |
``depends`` does not pull in dependencies into the run target. For example, if you select ``3.13,3.12,coverage`` |
| 566 |
via ``-e`` tox will only run those three (even if ``coverage`` may specify as ``depends`` other targets too -- |
| 567 |
such as ``3.13, 3.12, 3.11``). |
| 568 |
|
| 569 |
- ``--parallel-live`` / ``-o`` shows live output of stdout and stderr, and turns off the spinner. |
| 570 |
- Parallel evaluation disables standard input. Use non-parallel invocation if you need standard input. |
| 571 |
|
| 572 |
Example final output: |
| 573 |
|
| 574 |
.. code-block:: bash |
| 575 |
|
| 576 |
$ tox -e 3.13,3.12,coverage -p all |
| 577 |
✔ OK 3.12 in 9.533 seconds |
| 578 |
✔ OK 3.13 in 9.96 seconds |
| 579 |
✔ OK coverage in 2.0 seconds |
| 580 |
___________________________ summary ______________________________________________________ |
| 581 |
3.13: commands succeeded |
| 582 |
3.12: commands succeeded |
| 583 |
coverage: commands succeeded |
| 584 |
congratulations :) |
| 585 |
|
| 586 |
Example progress bar, showing a rotating spinner, the number of environments running and their list (limited up to 120 |
| 587 |
characters): |
| 588 |
|
| 589 |
.. code-block:: bash |
| 590 |
|
| 591 |
⠹ [2] 3.13 | 3.12 |
| 592 |
|
| 593 |
**************** |
| 594 |
Fail-fast mode |
| 595 |
**************** |
| 596 |
|
| 597 |
When running multiple environments, tox normally runs all of them even if an early environment fails. The |
| 598 |
``--fail-fast`` flag stops execution after the first environment failure, saving time in CI pipelines and development |
| 599 |
workflows. |
| 600 |
|
| 601 |
.. code-block:: bash |
| 602 |
|
| 603 |
tox run -e 3.13,3.12,3.11 --fail-fast |
| 604 |
|
| 605 |
You can also enable it per-environment via the ``fail_fast`` configuration: |
| 606 |
|
| 607 |
.. tab:: TOML |
| 608 |
|
| 609 |
.. code-block:: toml |
| 610 |
|
| 611 |
[env.critical] |
| 612 |
fail_fast = true |
| 613 |
commands = [["pytest", "tests/critical"]] |
| 614 |
|
| 615 |
.. tab:: INI |
| 616 |
|
| 617 |
.. code-block:: ini |
| 618 |
|
| 619 |
[testenv:critical] |
| 620 |
fail_fast = true |
| 621 |
commands = pytest tests/critical |
| 622 |
|
| 623 |
The fail-fast behavior: |
| 624 |
|
| 625 |
- Works in both sequential and parallel execution modes. |
| 626 |
- In parallel mode, environments already running when a failure occurs will continue to completion. Only environments |
| 627 |
not yet queued will be skipped. |
| 628 |
- Respects :ref:`ignore_outcome` -- environments with ``ignore_outcome = true`` will not trigger fail-fast even if they |
| 629 |
fail. |
| 630 |
- Respects environment dependencies defined via :ref:`depends` -- dependent environments will not run if a dependency |
| 631 |
fails with fail-fast enabled. |
| 632 |
- Environments not yet started are skipped with exit code -2 and marked as ``SKIP`` in the output. |
| 633 |
- The overall tox exit code will be the exit code of the first failed environment. |
| 634 |
|
| 635 |
*************************** |
| 636 |
Configuration inheritance |
| 637 |
*************************** |
| 638 |
|
| 639 |
Every tox environment has its own configuration section. If a value is not defined in the environment-specific section, |
| 640 |
it falls back to the base section (:ref:`base` configuration). For ``tox.toml`` this is the ``env_run_base`` table, for |
| 641 |
``tox.ini`` this is ``[testenv]``. If the base section also lacks the value, the configuration default is used. |
| 642 |
|
| 643 |
.. tab:: TOML |
| 644 |
|
| 645 |
.. code-block:: toml |
| 646 |
|
| 647 |
[env_run_base] |
| 648 |
commands = [["pytest", "tests"]] |
| 649 |
|
| 650 |
[env.test] |
| 651 |
description = "run the test suite with pytest" |
| 652 |
|
| 653 |
.. tab:: INI |
| 654 |
|
| 655 |
.. code-block:: ini |
| 656 |
|
| 657 |
[testenv] |
| 658 |
commands = pytest tests |
| 659 |
|
| 660 |
[testenv:test] |
| 661 |
description = run the test suite with pytest |
| 662 |
|
| 663 |
Here ``test`` inherits ``commands`` from the base because it is not specified in ``[env.test]``. |
| 664 |
|
| 665 |
.. _virtualenv-version-pinning: |
| 666 |
|
| 667 |
**************************** |
| 668 |
Virtualenv version pinning |
| 669 |
**************************** |
| 670 |
|
| 671 |
tox creates isolated environments using :pypi:`virtualenv`, which it imports as a library. This works well when the |
| 672 |
installed virtualenv supports all target Python versions, but breaks down at the edges: older virtualenv releases |
| 673 |
(pre-20.22) are required for Python 3.6 support, while the latest virtualenv is needed for Python 3.15+. Since tox can |
| 674 |
only import one virtualenv version per process, projects that need both old and new Pythons in a single ``tox.toml`` hit |
| 675 |
a wall. |
| 676 |
|
| 677 |
The :ref:`virtualenv_spec` setting resolves this by decoupling the virtualenv used for environment creation from the one |
| 678 |
tox imports. When set, tox: |
| 679 |
|
| 680 |
1. Creates a bootstrap venv (using the stdlib ``venv`` module) in ``.tox/.virtualenv-bootstrap/``. |
| 681 |
2. Installs the specified virtualenv version into that bootstrap venv via pip. |
| 682 |
3. Runs the bootstrapped virtualenv as a subprocess instead of calling ``session_via_cli()`` from the imported library. |
| 683 |
|
| 684 |
The bootstrap is content-addressed by a hash of the spec string, so different specs get separate cached environments. A |
| 685 |
file lock protects against concurrent bootstrap creation (relevant in parallel mode). Once bootstrapped, subsequent runs |
| 686 |
skip directly to step 3. |
| 687 |
|
| 688 |
When ``virtualenv_spec`` is empty (the default), tox uses the imported virtualenv with zero overhead -- the subprocess |
| 689 |
path only activates when explicitly configured. The spec is included in the environment cache key, so changing it |
| 690 |
triggers automatic recreation. |
| 691 |
|
| 692 |
This design mirrors tox's own auto-provisioning mechanism (``requires`` / ``min_version``), where tox bootstraps itself |
| 693 |
into a separate environment when the running installation doesn't meet the declared requirements. |
| 694 |
|
| 695 |
******************* |
| 696 |
Known limitations |
| 697 |
******************* |
| 698 |
|
| 699 |
Interactive terminal programs |
| 700 |
============================= |
| 701 |
|
| 702 |
Programs that require advanced terminal control — such as IPython, debuggers with rich UIs, or any tool built on |
| 703 |
`prompt_toolkit <https://python-prompt-toolkit.readthedocs.io/>`__ — need direct terminal access to work correctly. |
| 704 |
|
| 705 |
By default, tox captures subprocess output by routing ``stdout`` and ``stderr`` through pseudo-terminal (PTY) pairs. |
| 706 |
This is necessary for logging, result reporting, and colorized output. However, the subprocess's ``stdin`` remains |
| 707 |
connected to the real terminal. This means ``stdin`` and ``stdout`` are on *different* terminal devices. |
| 708 |
|
| 709 |
Libraries like ``prompt_toolkit`` assume all streams share the same terminal. They set raw mode on ``stdin`` (to read |
| 710 |
individual keystrokes) while writing VT100 escape sequences to ``stdout`` (for cursor positioning, screen clearing, |
| 711 |
etc.). When ``stdout`` goes through tox's capture buffer instead of directly to the terminal, escape sequences are |
| 712 |
delayed and the synchronous terminal control these libraries depend on breaks. |
| 713 |
|
| 714 |
Solution: |
| 715 |
|
| 716 |
Use the ``--no-capture`` (or ``-i``) flag to disable output capture and give the subprocess direct terminal access: |
| 717 |
|
| 718 |
.. code-block:: bash |
| 719 |
|
| 720 |
# Run IPython with full terminal support |
| 721 |
tox run -e 3.13 -i -- ipython |
| 722 |
|
| 723 |
# Run debugger interactively |
| 724 |
tox run -e 3.13 -i -- python -m pdb script.py |
| 725 |
|
| 726 |
This flag is mutually exclusive with ``--result-json`` and parallel mode. See :ref:`run-interactive-programs` for |
| 727 |
details. |
| 728 |
|
| 729 |
Alternative workarounds if you cannot use ``--no-capture``: |
| 730 |
|
| 731 |
- For IPython, pass ``--simple-prompt`` to disable ``prompt_toolkit``'s advanced terminal features. |
| 732 |
- For other tools, look for a "dumb terminal" or "no-color" mode that avoids VT100 escape sequences. |
| 733 |
|
| 734 |
Debian/Ubuntu without ``python3-venv`` |
| 735 |
====================================== |
| 736 |
|
| 737 |
On Debian and Ubuntu, the system Python is split into multiple packages. The ``python3`` package does not include the |
| 738 |
:mod:`venv` module or :mod:`ensurepip` — these are in the separate ``python3-venv`` package. |
| 739 |
|
| 740 |
tox itself is **not affected** because it uses :pypi:`virtualenv` (which bundles its own bootstrap mechanism) rather |
| 741 |
than stdlib :mod:`venv`. However, tools that tox *runs as commands* inside environments may use stdlib :mod:`venv` |
| 742 |
internally and fail. A common example is :pypi:`build` (``pyproject-build``), which creates an isolated build |
| 743 |
environment using :mod:`venv` when a recent ``pip`` is available. |
| 744 |
|
| 745 |
If you see errors like: |
| 746 |
|
| 747 |
.. code-block:: text |
| 748 |
|
| 749 |
The virtual environment was not created successfully because ensurepip is not available. |
| 750 |
On Debian/Ubuntu systems, you need to install the python3-venv package. |
| 751 |
|
| 752 |
this is the tool inside the tox environment hitting the missing system package, not tox itself. |
| 753 |
|
| 754 |
**Solutions:** |
| 755 |
|
| 756 |
- Install the system venv package: ``apt install python3-venv`` (or ``python3.X-venv`` for a specific version). |
| 757 |
- Use :pypi:`tox-uv`, which replaces both the environment creation and package installation with :pypi:`uv`, avoiding |
| 758 |
the stdlib :mod:`venv` dependency entirely. |
| 759 |
|
| 760 |
Misplaced configuration keys |
| 761 |
============================ |
| 762 |
|
| 763 |
tox configuration is split into two sections: **core** (``[tox]`` / top-level TOML) and **environment** (``[testenv]`` / |
| 764 |
``env_run_base`` in TOML). Options placed in the wrong section are silently ignored because tox cannot distinguish a |
| 765 |
misplaced option from a plugin-defined key that isn't loaded yet (e.g. during provisioning). |
| 766 |
|
| 767 |
To detect misplaced keys: |
| 768 |
|
| 769 |
- Run ``tox run -v`` — unused keys are printed as warnings before the final report. |
| 770 |
- Run ``tox config`` — unused keys appear as ``# !!! unused:`` comments per section. |
| 771 |
- Run ``tox config --format json`` or ``--format toml`` — unused keys appear in an ``"unused"`` array per section. |
| 772 |
|
| 773 |
For example, putting ``ignore_base_python_conflict`` in ``[testenv]`` instead of ``[tox]``: |
| 774 |
|
| 775 |
.. tab:: INI |
| 776 |
|
| 777 |
.. code-block:: text |
| 778 |
|
| 779 |
[testenv:py] |
| 780 |
... |
| 781 |
# !!! unused: ignore_base_python_conflict |
| 782 |
|
| 783 |
.. tab:: JSON |
| 784 |
|
| 785 |
.. code-block:: json |
| 786 |
|
| 787 |
{ |
| 788 |
"env": { |
| 789 |
"py": { |
| 790 |
"unused": ["ignore_base_python_conflict"] |
| 791 |
} |
| 792 |
} |
| 793 |
} |
| 794 |
|
| 795 |
.. tab:: TOML |
| 796 |
|
| 797 |
.. code-block:: toml |
| 798 |
|
| 799 |
[env.py] |
| 800 |
unused = ["ignore_base_python_conflict"] |
| 801 |
|
| 802 |
See the :ref:`Core <conf-core>` and :ref:`tox environment <conf-testenv>` reference sections for which options belong |
| 803 |
where. |
| 804 |
|
| 805 |
****************** |
| 806 |
Related projects |
| 807 |
****************** |
| 808 |
|
| 809 |
tox has influenced several other projects in the Python test automation space. If tox doesn't quite fit your needs or |
| 810 |
you want to do more research, we recommend taking a look at these projects: |
| 811 |
|
| 812 |
- `nox <https://nox.thea.codes/en/stable/>`__ is a project similar in spirit to tox but different in approach. The |
| 813 |
primary key difference is that it uses Python scripts instead of a configuration file. It might be useful if you find |
| 814 |
tox configuration too limiting but aren't looking to move to something as general-purpose as ``Invoke`` or ``make``. |
| 815 |
- `Invoke <https://www.pyinvoke.org/>`__ is a general-purpose task execution library, similar to Make. Invoke is far |
| 816 |
more general-purpose than tox but it does not contain the Python testing-specific features that tox specializes in. |