| 1 |
Reference |
| 2 |
========= |
| 3 |
|
| 4 |
Trio mode |
| 5 |
--------- |
| 6 |
|
| 7 |
Most users will want to enable "Trio mode". Without Trio mode: |
| 8 |
|
| 9 |
* Pytest-trio only handles tests that have been decorated with |
| 10 |
``@pytest.mark.trio`` |
| 11 |
* Pytest-trio only handles fixtures if they're async *and* used by a |
| 12 |
test that's decorated with ``@pytest.mark.trio``, or if they're |
| 13 |
decorated with ``@pytest_trio.trio_fixture`` (instead of |
| 14 |
``@pytest.fixture``). |
| 15 |
|
| 16 |
When Trio mode is enabled, two extra things happen: |
| 17 |
|
| 18 |
* Async tests automatically have the ``trio`` mark added, so you don't |
| 19 |
have to do it yourself. |
| 20 |
* Async fixtures using ``@pytest.fixture`` automatically get converted |
| 21 |
to Trio fixtures. (The main effect of this is that it helps you |
| 22 |
catch mistakes like using an async fixture with a non-async |
| 23 |
test.) |
| 24 |
|
| 25 |
There are two ways to enable Trio mode. |
| 26 |
|
| 27 |
The first option is to **use a pytest configuration file**. The exact |
| 28 |
rules for how pytest finds configuration files are `a bit complicated |
| 29 |
<https://docs.pytest.org/en/latest/customize.html>`__, but you want to |
| 30 |
end up with something like: |
| 31 |
|
| 32 |
.. code-block:: ini |
| 33 |
|
| 34 |
# pytest.ini |
| 35 |
[pytest] |
| 36 |
trio_mode = true |
| 37 |
|
| 38 |
The second option is **use a conftest.py file**. Inside your tests |
| 39 |
directory, create a file called ``conftest.py``, with the following |
| 40 |
contents:: |
| 41 |
|
| 42 |
# conftest.py |
| 43 |
from pytest_trio.enable_trio_mode import * |
| 44 |
|
| 45 |
This does exactly the same thing as setting ``trio_mode = true`` in |
| 46 |
``pytest.ini``, except for two things: |
| 47 |
|
| 48 |
* Some people like to ship their tests as part of their library, so |
| 49 |
they (or their users) can test the final installed software by |
| 50 |
running ``pytest --pyargs PACKAGENAME``. In this mode, |
| 51 |
``pytest.ini`` files don't work, but ``conftest.py`` files do. |
| 52 |
|
| 53 |
* Enabling Trio mode in ``pytest.ini`` always enables it globally for |
| 54 |
your entire testsuite. Enabling it in ``conftest.py`` only enables |
| 55 |
it for test files that are in the same directory as the |
| 56 |
``conftest.py``, or its subdirectories. |
| 57 |
|
| 58 |
If you have software that uses multiple async libraries, then you can |
| 59 |
use ``conftest.py`` to enable Trio mode for just the part of your |
| 60 |
testsuite that uses Trio; or, if you need even finer-grained control, |
| 61 |
you can leave Trio mode disabled and use ``@pytest.mark.trio`` |
| 62 |
explicitly on all your Trio tests. |
| 63 |
|
| 64 |
|
| 65 |
Trio fixtures |
| 66 |
------------- |
| 67 |
|
| 68 |
Normally, pytest runs fixture code before starting the test, and |
| 69 |
teardown code afterwards. For technical reasons, we can't wrap this |
| 70 |
whole process in :func:`trio.run` – only the test itself. As a |
| 71 |
workaround, pytest-trio introduces the concept of a "Trio fixture", |
| 72 |
which acts like a normal fixture for most purposes, but actually does |
| 73 |
the setup and teardown inside the test's call to :func:`trio.run`. |
| 74 |
|
| 75 |
The following fixtures are treated as Trio fixtures: |
| 76 |
|
| 77 |
* Any function decorated with ``@pytest_trio.trio_fixture``. |
| 78 |
* Any async function decorated with ``@pytest.fixture``, *if* |
| 79 |
Trio mode is enabled *or* this fixture is being requested by a Trio |
| 80 |
test. |
| 81 |
* Any fixture which depends on a Trio fixture. |
| 82 |
|
| 83 |
The most notable difference between regular fixtures and Trio fixtures |
| 84 |
is that regular fixtures can't use Trio APIs, but Trio fixtures can. |
| 85 |
Most of the time you don't need to worry about this, because you |
| 86 |
normally only call Trio APIs from async functions, and when Trio mode |
| 87 |
is enabled, all async fixtures are automatically Trio fixtures. |
| 88 |
However, if for some reason you do want to use Trio APIs from a |
| 89 |
synchronous fixture, then you'll have to use |
| 90 |
``@pytest_trio.trio_fixture``:: |
| 91 |
|
| 92 |
# This fixture is not very useful |
| 93 |
# But it is an example where @pytest.fixture doesn't work |
| 94 |
@pytest_trio.trio_fixture |
| 95 |
def trio_time(): |
| 96 |
return trio.current_time() |
| 97 |
|
| 98 |
Only Trio tests can use Trio fixtures. If you have a regular |
| 99 |
(synchronous) test that tries to use a Trio fixture, then that's an |
| 100 |
error. |
| 101 |
|
| 102 |
And finally, regular fixtures can be `scoped to the test, class, |
| 103 |
module, or session |
| 104 |
<https://docs.pytest.org/en/latest/fixture.html#scope-sharing-a-fixture-instance-across-tests-in-a-class-module-or-session>`__, |
| 105 |
but Trio fixtures **must be test scoped**. Class, module, and session |
| 106 |
scope are not supported. |
| 107 |
|
| 108 |
|
| 109 |
.. _cancel-yield: |
| 110 |
|
| 111 |
An important note about ``yield`` fixtures |
| 112 |
------------------------------------------ |
| 113 |
|
| 114 |
Like any pytest fixture, Trio fixtures can contain both setup and |
| 115 |
teardown code separated by a ``yield``:: |
| 116 |
|
| 117 |
@pytest.fixture |
| 118 |
async def my_fixture(): |
| 119 |
... setup code ... |
| 120 |
yield |
| 121 |
... teardown code ... |
| 122 |
|
| 123 |
When pytest-trio executes this fixture, it creates a new task, and |
| 124 |
runs the setup code until it reaches the ``yield``. Then the fixture's |
| 125 |
task goes to sleep. Once the test has finished, the fixture task wakes |
| 126 |
up again and resumes at the ``yield``, so it can execute the teardown |
| 127 |
code. |
| 128 |
|
| 129 |
So the ``yield`` in a fixture is sort of like calling ``await |
| 130 |
wait_for_test_to_finish()``. And in Trio, any ``await``\-able |
| 131 |
operation can be cancelled. For example, we could put a timeout on the |
| 132 |
``yield``:: |
| 133 |
|
| 134 |
@pytest.fixture |
| 135 |
async def my_fixture(): |
| 136 |
... setup code ... |
| 137 |
with trio.move_on_after(5): |
| 138 |
yield # this yield gets cancelled after 5 seconds |
| 139 |
... teardown code ... |
| 140 |
|
| 141 |
Now if the test takes more than 5 seconds to execute, this fixture |
| 142 |
will cancel the ``yield``. |
| 143 |
|
| 144 |
That's kind of a strange thing to do, but there's another version of |
| 145 |
this that's extremely common. Suppose your fixture spawns a background |
| 146 |
task, and then the background task raises an exception. Whenever a |
| 147 |
background task raises an exception, it automatically cancels |
| 148 |
everything inside the nursery's scope – which includes our ``yield``:: |
| 149 |
|
| 150 |
@pytest.fixture |
| 151 |
async def my_fixture(nursery): |
| 152 |
nursery.start_soon(function_that_raises_exception) |
| 153 |
yield # this yield gets cancelled after the background task crashes |
| 154 |
... teardown code ... |
| 155 |
|
| 156 |
If you use fixtures with background tasks, you'll probably end up |
| 157 |
cancelling one of these ``yield``\s sooner or later. So what happens |
| 158 |
if the ``yield`` gets cancelled? |
| 159 |
|
| 160 |
First, pytest-trio assumes that something has gone wrong and there's |
| 161 |
no point in continuing the test. If the top-level test function is |
| 162 |
running, then it cancels it. |
| 163 |
|
| 164 |
Then, pytest-trio waits for the test function to finish, and |
| 165 |
then begins tearing down fixtures as normal. |
| 166 |
|
| 167 |
During this teardown process, it will eventually reach the fixture |
| 168 |
that cancelled its ``yield``. This fixture gets resumed to execute its |
| 169 |
teardown logic, but with a special twist: since the ``yield`` was |
| 170 |
cancelled, the ``yield`` raises :exc:`trio.Cancelled`. |
| 171 |
|
| 172 |
Now, here's the punchline: this means that in our examples above, the |
| 173 |
teardown code might not be executed at all! **This is different from |
| 174 |
how pytest fixtures normally work.** Normally, the ``yield`` in a |
| 175 |
pytest fixture never raises an exception, so you can be certain that |
| 176 |
any code you put after it will execute as normal. But if you have a |
| 177 |
fixture with background tasks, and they crash, then your ``yield`` |
| 178 |
might raise an exception, and Python will skip executing the code |
| 179 |
after the ``yield``. |
| 180 |
|
| 181 |
In our experience, most fixtures are fine with this, and it prevents |
| 182 |
some `weird problems |
| 183 |
<https://github.com/python-trio/pytest-trio/issues/75>`__ that can |
| 184 |
happen otherwise. But it's something to be aware of. |
| 185 |
|
| 186 |
If you have a fixture where the ``yield`` might be cancelled but you |
| 187 |
still need to run teardown code, then you can use a ``finally`` |
| 188 |
block:: |
| 189 |
|
| 190 |
@pytest.fixture |
| 191 |
async def my_fixture(nursery): |
| 192 |
nursery.start_soon(function_that_crashes) |
| 193 |
try: |
| 194 |
# This yield could be cancelled... |
| 195 |
yield |
| 196 |
finally: |
| 197 |
# But this code will run anyway |
| 198 |
... teardown code ... |
| 199 |
|
| 200 |
(But, watch out: the teardown code is still running in a cancelled |
| 201 |
context, so if it has any ``await``\s it could raise |
| 202 |
:exc:`trio.Cancelled` again.) |
| 203 |
|
| 204 |
Or if you use ``with`` to handle teardown, then you don't have to |
| 205 |
worry about this because ``with`` blocks always perform cleanup even |
| 206 |
if there's an exception:: |
| 207 |
|
| 208 |
@pytest.fixture |
| 209 |
async def my_fixture(nursery): |
| 210 |
with get_obj_that_must_be_torn_down() as obj: |
| 211 |
nursery.start_soon(function_that_crashes, obj) |
| 212 |
# This could raise trio.Cancelled... |
| 213 |
# ...but that's OK, the 'with' block will still tear down 'obj' |
| 214 |
yield obj |
| 215 |
|
| 216 |
|
| 217 |
Concurrent setup/teardown |
| 218 |
------------------------- |
| 219 |
|
| 220 |
If your test uses multiple fixtures, then for speed, pytest-trio will |
| 221 |
try to run their setup and teardown code concurrently whenever this is |
| 222 |
possible while respecting the fixture dependencies. |
| 223 |
|
| 224 |
Here's an example, where a test depends on ``fix_b`` and ``fix_c``, |
| 225 |
and these both depend on ``fix_a``:: |
| 226 |
|
| 227 |
@trio_fixture |
| 228 |
def fix_a(): |
| 229 |
... |
| 230 |
|
| 231 |
@trio_fixture |
| 232 |
def fix_b(fix_a): |
| 233 |
... |
| 234 |
|
| 235 |
@trio_fixture |
| 236 |
def fix_c(fix_a): |
| 237 |
... |
| 238 |
|
| 239 |
@pytest.mark.trio |
| 240 |
async def test_example(fix_b, fix_c): |
| 241 |
... |
| 242 |
|
| 243 |
When running ``test_example``, pytest-trio will perform the following |
| 244 |
sequence of actions: |
| 245 |
|
| 246 |
1. Set up ``fix_a`` |
| 247 |
2. Set up ``fix_b`` and ``fix_c``, concurrently. |
| 248 |
3. Run the test. |
| 249 |
4. Tear down ``fix_b`` and ``fix_c``, concurrently. |
| 250 |
5. Tear down ``fix_a``. |
| 251 |
|
| 252 |
We're `seeking feedback |
| 253 |
<https://github.com/python-trio/pytest-trio/issues/57>`__ on whether |
| 254 |
this feature's benefits outweigh its negatives. |
| 255 |
|
| 256 |
|
| 257 |
Handling of ContextVars |
| 258 |
----------------------- |
| 259 |
|
| 260 |
The :mod:`contextvars` module lets you create |
| 261 |
:class:`~contextvars.ContextVar` objects to represent task-local |
| 262 |
variables. Normally, in Trio, each task gets its own |
| 263 |
:class:`~contextvars.Context`, so that changes to |
| 264 |
:class:`~contextvars.ContextVar` objects are only visible inside the |
| 265 |
task that performs them. But pytest-trio overrides this, and for each |
| 266 |
test it uses a single :class:`~contextvars.Context` which is shared by |
| 267 |
all fixtures and the test function itself. |
| 268 |
|
| 269 |
The benefit of this is that you can set |
| 270 |
:class:`~contextvars.ContextVar` values inside a fixture, and your |
| 271 |
settings will be visible in dependent fixtures and the test itself. |
| 272 |
For example, `trio-asyncio <https://trio-asyncio.readthedocs.io/>`__ |
| 273 |
uses a :class:`~contextvars.ContextVar` to hold the current asyncio |
| 274 |
loop object, so this lets you open a loop inside a fixture and then |
| 275 |
use it inside other fixtures or the test itself. |
| 276 |
|
| 277 |
The downside is that if two fixtures are run concurrently (see |
| 278 |
previous section), and both mutate the same |
| 279 |
:class:`~contextvars.ContextVar`, then there will be a race condition |
| 280 |
and the the final value will be unpredictable. If you make one fixture |
| 281 |
depend on the other, then this will force an ordering and make the |
| 282 |
final value predictable again. |
| 283 |
|
| 284 |
|
| 285 |
Built-in fixtures |
| 286 |
----------------- |
| 287 |
|
| 288 |
These fixtures are automatically available to any code using |
| 289 |
pytest-trio. |
| 290 |
|
| 291 |
.. data:: autojump_clock |
| 292 |
|
| 293 |
A :class:`trio.testing.MockClock`, configured with ``rate=0, |
| 294 |
autojump_threshold=0``. |
| 295 |
|
| 296 |
.. data:: mock_clock |
| 297 |
|
| 298 |
A :class:`trio.testing.MockClock`, with its default configuration |
| 299 |
(``rate=0, autojump_threshold=inf``). |
| 300 |
|
| 301 |
What makes these particularly useful is that whenever pytest-trio runs |
| 302 |
a test, it checks the fixtures to see if one of them is a |
| 303 |
:class:`trio.abc.Clock` object. If so, it passes that object to |
| 304 |
:func:`trio.run`. So if your test requests one of these fixtures, it |
| 305 |
automatically uses that clock. |
| 306 |
|
| 307 |
If you implement your own :class:`~trio.abc.Clock`, and implement a |
| 308 |
fixture that returns it, then it will work the same way. |
| 309 |
|
| 310 |
Of course, like any pytest fixture, you also get the actual object |
| 311 |
available. For example, you can call |
| 312 |
:meth:`~trio.testing.MockClock.jump`:: |
| 313 |
|
| 314 |
async def test_time_travel(mock_clock): |
| 315 |
assert trio.current_time() == 0 |
| 316 |
mock_clock.jump(10) |
| 317 |
assert trio.current_time() == 10 |
| 318 |
|
| 319 |
.. data:: nursery |
| 320 |
|
| 321 |
A nursery created and managed by pytest-trio itself, which |
| 322 |
surrounds the test/fixture that requested it, and is automatically |
| 323 |
cancelled after the test/fixture completes. Basically, these are |
| 324 |
equivalent:: |
| 325 |
|
| 326 |
# Boring way |
| 327 |
async def test_with_background_task(): |
| 328 |
async with trio.open_nursery() as nursery: |
| 329 |
try: |
| 330 |
... |
| 331 |
finally: |
| 332 |
nursery.cancel_scope.cancel() |
| 333 |
|
| 334 |
# Fancy way |
| 335 |
async def test_with_background_task(nursery): |
| 336 |
... |
| 337 |
|
| 338 |
For a fixture, the cancellation always happens after the fixture |
| 339 |
completes its teardown phase. (Or if it doesn't have a teardown |
| 340 |
phase, then the cancellation happens after the teardown phase |
| 341 |
*would* have happened.) |
| 342 |
|
| 343 |
This fixture is even more magical than most pytest fixtures, |
| 344 |
because if it gets requested several times within the same test, |
| 345 |
then it creates multiple nurseries, one for each fixture/test that |
| 346 |
requested it. |
| 347 |
|
| 348 |
See :ref:`server-fixture-example` for an example of how this can be |
| 349 |
used. |
| 350 |
|
| 351 |
|
| 352 |
Integration with the Hypothesis library |
| 353 |
--------------------------------------- |
| 354 |
|
| 355 |
There isn't too much to say here, since the obvious thing just works:: |
| 356 |
|
| 357 |
from hypothesis import given |
| 358 |
import hypothesis.strategies as st |
| 359 |
|
| 360 |
@given(st.binary()) |
| 361 |
async def test_trio_and_hypothesis(data): |
| 362 |
... |
| 363 |
|
| 364 |
Under the hood, this requires some coordination between Hypothesis and |
| 365 |
pytest-trio. Hypothesis runs your test multiple times with different |
| 366 |
examples of random data. For each example, pytest-trio calls |
| 367 |
:func:`trio.run` again (so you get a fresh clean Trio environment), |
| 368 |
sets up any Trio fixtures, runs the actual test, and then tears down |
| 369 |
any Trio fixtures. Notice that this is a bit different than regular |
| 370 |
pytest fixtures, which are `instantiated once and then re-used for all |
| 371 |
<https://github.com/pytest-dev/pytest/issues/916>`__. Most of the time |
| 372 |
this shouldn't matter (and `is probably what you want anyway |
| 373 |
<https://github.com/HypothesisWorks/hypothesis/issues/377>`__), but in |
| 374 |
some unusual cases it could surprise you. And this only applies to |
| 375 |
Trio fixtures – if a Trio test uses a mix of regular fixtures and Trio |
| 376 |
fixtures, then the regular fixtures will be reused, while the Trio |
| 377 |
fixtures will be repeatedly reinstantiated. |
| 378 |
|
| 379 |
Also, pytest-trio only handles ``@given``\-based tests. If you want to |
| 380 |
write `stateful tests |
| 381 |
<https://hypothesis.readthedocs.io/en/latest/stateful.html>`__ for |
| 382 |
Trio-based libraries, then check out `hypothesis-trio |
| 383 |
<https://github.com/python-trio/hypothesis-trio>`__. |
| 384 |
|
| 385 |
|
| 386 |
.. _trio-run-config: |
| 387 |
|
| 388 |
Using alternative Trio runners |
| 389 |
------------------------------ |
| 390 |
|
| 391 |
If you are working with a library that provides integration with Trio, |
| 392 |
such as via :ref:`guest mode <trio:guest-mode>`, it can be used with |
| 393 |
pytest-trio as well. Setting ``trio_run`` in the pytest configuration |
| 394 |
makes your choice the global default for both tests explicitly marked |
| 395 |
with ``@pytest.mark.trio`` and those automatically marked by Trio mode. |
| 396 |
``trio_run`` presently supports ``trio`` and ``qtrio``. |
| 397 |
|
| 398 |
.. code-block:: ini |
| 399 |
|
| 400 |
# pytest.ini |
| 401 |
[pytest] |
| 402 |
trio_mode = true |
| 403 |
trio_run = qtrio |
| 404 |
|
| 405 |
.. code-block:: python |
| 406 |
|
| 407 |
import pytest |
| 408 |
|
| 409 |
@pytest.mark.trio |
| 410 |
async def test(): |
| 411 |
assert True |
| 412 |
|
| 413 |
If you want more granular control or need to use a specific function, |
| 414 |
it can be passed directly to the marker. |
| 415 |
|
| 416 |
.. code-block:: python |
| 417 |
|
| 418 |
import pytest |
| 419 |
|
| 420 |
@pytest.mark.trio(run=qtrio.run) |
| 421 |
async def test(): |
| 422 |
assert True |