| 1 |
======================= |
| 2 |
Advanced testing topics |
| 3 |
======================= |
| 4 |
|
| 5 |
The request factory |
| 6 |
=================== |
| 7 |
|
| 8 |
.. currentmodule:: django.test |
| 9 |
|
| 10 |
.. class:: RequestFactory |
| 11 |
|
| 12 |
The :class:`~django.test.RequestFactory` shares the same API as |
| 13 |
the test client. However, instead of behaving like a browser, the |
| 14 |
RequestFactory provides a way to generate a request instance that can |
| 15 |
be used as the first argument to any view. This means you can test a |
| 16 |
view function the same way as you would test any other function -- as |
| 17 |
a black box, with exactly known inputs, testing for specific outputs. |
| 18 |
|
| 19 |
The API for the :class:`~django.test.RequestFactory` is a slightly |
| 20 |
restricted subset of the test client API: |
| 21 |
|
| 22 |
* It only has access to the HTTP methods :meth:`~Client.get()`, |
| 23 |
:meth:`~Client.post()`, :meth:`~Client.put()`, |
| 24 |
:meth:`~Client.delete()`, :meth:`~Client.head()`, |
| 25 |
:meth:`~Client.options()`, and :meth:`~Client.trace()`. |
| 26 |
|
| 27 |
* These methods accept all the same arguments *except* for |
| 28 |
``follow``. Since this is just a factory for producing |
| 29 |
requests, it's up to you to handle the response. |
| 30 |
|
| 31 |
* It does not support middleware. Session and authentication |
| 32 |
attributes must be supplied by the test itself if required |
| 33 |
for the view to function properly. |
| 34 |
|
| 35 |
.. versionchanged:: 5.1 |
| 36 |
|
| 37 |
The ``query_params`` parameter was added. |
| 38 |
|
| 39 |
Example |
| 40 |
------- |
| 41 |
|
| 42 |
The following is a unit test using the request factory:: |
| 43 |
|
| 44 |
from django.contrib.auth.models import AnonymousUser, User |
| 45 |
from django.test import RequestFactory, TestCase |
| 46 |
|
| 47 |
from .views import MyView, my_view |
| 48 |
|
| 49 |
|
| 50 |
class SimpleTest(TestCase): |
| 51 |
def setUp(self): |
| 52 |
# Every test needs access to the request factory. |
| 53 |
self.factory = RequestFactory() |
| 54 |
self.user = User.objects.create_user( |
| 55 |
username="jacob", email="jacob@…", password="top_secret" |
| 56 |
) |
| 57 |
|
| 58 |
def test_details(self): |
| 59 |
# Create an instance of a GET request. |
| 60 |
request = self.factory.get("/customer/details") |
| 61 |
|
| 62 |
# Recall that middleware are not supported. You can simulate a |
| 63 |
# logged-in user by setting request.user manually. |
| 64 |
request.user = self.user |
| 65 |
|
| 66 |
# Or you can simulate an anonymous user by setting request.user to |
| 67 |
# an AnonymousUser instance. |
| 68 |
request.user = AnonymousUser() |
| 69 |
|
| 70 |
# Test my_view() as if it were deployed at /customer/details |
| 71 |
response = my_view(request) |
| 72 |
# Use this syntax for class-based views. |
| 73 |
response = MyView.as_view()(request) |
| 74 |
self.assertEqual(response.status_code, 200) |
| 75 |
|
| 76 |
AsyncRequestFactory |
| 77 |
------------------- |
| 78 |
|
| 79 |
.. class:: AsyncRequestFactory |
| 80 |
|
| 81 |
``RequestFactory`` creates WSGI-like requests. If you want to create ASGI-like |
| 82 |
requests, including having a correct ASGI ``scope``, you can instead use |
| 83 |
``django.test.AsyncRequestFactory``. |
| 84 |
|
| 85 |
This class is directly API-compatible with ``RequestFactory``, with the only |
| 86 |
difference being that it returns ``ASGIRequest`` instances rather than |
| 87 |
``WSGIRequest`` instances. All of its methods are still synchronous callables. |
| 88 |
|
| 89 |
Arbitrary keyword arguments in ``defaults`` are added directly into the ASGI |
| 90 |
scope. |
| 91 |
|
| 92 |
.. versionchanged:: 5.1 |
| 93 |
|
| 94 |
The ``query_params`` parameter was added. |
| 95 |
|
| 96 |
Testing class-based views |
| 97 |
========================= |
| 98 |
|
| 99 |
In order to test class-based views outside of the request/response cycle you |
| 100 |
must ensure that they are configured correctly, by calling |
| 101 |
:meth:`~django.views.generic.base.View.setup` after instantiation. |
| 102 |
|
| 103 |
For example, assuming the following class-based view: |
| 104 |
|
| 105 |
.. code-block:: python |
| 106 |
:caption: ``views.py`` |
| 107 |
|
| 108 |
from django.views.generic import TemplateView |
| 109 |
|
| 110 |
|
| 111 |
class HomeView(TemplateView): |
| 112 |
template_name = "myapp/home.html" |
| 113 |
|
| 114 |
def get_context_data(self, **kwargs): |
| 115 |
kwargs["environment"] = "Production" |
| 116 |
return super().get_context_data(**kwargs) |
| 117 |
|
| 118 |
You may directly test the ``get_context_data()`` method by first instantiating |
| 119 |
the view, then passing a ``request`` to ``setup()``, before proceeding with |
| 120 |
your test's code: |
| 121 |
|
| 122 |
.. code-block:: python |
| 123 |
:caption: ``tests.py`` |
| 124 |
|
| 125 |
from django.test import RequestFactory, TestCase |
| 126 |
from .views import HomeView |
| 127 |
|
| 128 |
|
| 129 |
class HomePageTest(TestCase): |
| 130 |
def test_environment_set_in_context(self): |
| 131 |
request = RequestFactory().get("/") |
| 132 |
view = HomeView() |
| 133 |
view.setup(request) |
| 134 |
|
| 135 |
context = view.get_context_data() |
| 136 |
self.assertIn("environment", context) |
| 137 |
|
| 138 |
.. _topics-testing-advanced-multiple-hosts: |
| 139 |
|
| 140 |
Tests and multiple host names |
| 141 |
============================= |
| 142 |
|
| 143 |
The :setting:`ALLOWED_HOSTS` setting is validated when running tests. This |
| 144 |
allows the test client to differentiate between internal and external URLs. |
| 145 |
|
| 146 |
Projects that support multitenancy or otherwise alter business logic based on |
| 147 |
the request's host and use custom host names in tests must include those hosts |
| 148 |
in :setting:`ALLOWED_HOSTS`. |
| 149 |
|
| 150 |
The first option to do so is to add the hosts to your settings file. For |
| 151 |
example, the test suite for docs.djangoproject.com includes the following:: |
| 152 |
|
| 153 |
from django.test import TestCase |
| 154 |
|
| 155 |
|
| 156 |
class SearchFormTestCase(TestCase): |
| 157 |
def test_empty_get(self): |
| 158 |
response = self.client.get( |
| 159 |
"/en/dev/search/", |
| 160 |
headers={"host": "docs.djangoproject.dev:8000"}, |
| 161 |
) |
| 162 |
self.assertEqual(response.status_code, 200) |
| 163 |
|
| 164 |
and the settings file includes a list of the domains supported by the project:: |
| 165 |
|
| 166 |
ALLOWED_HOSTS = ["www.djangoproject.dev", "docs.djangoproject.dev", ...] |
| 167 |
|
| 168 |
Another option is to add the required hosts to :setting:`ALLOWED_HOSTS` using |
| 169 |
:meth:`~django.test.override_settings()` or |
| 170 |
:attr:`~django.test.SimpleTestCase.modify_settings()`. This option may be |
| 171 |
preferable in standalone apps that can't package their own settings file or |
| 172 |
for projects where the list of domains is not static (e.g., subdomains for |
| 173 |
multitenancy). For example, you could write a test for the domain |
| 174 |
``http://otherserver/`` as follows:: |
| 175 |
|
| 176 |
from django.test import TestCase, override_settings |
| 177 |
|
| 178 |
|
| 179 |
class MultiDomainTestCase(TestCase): |
| 180 |
@override_settings(ALLOWED_HOSTS=["otherserver"]) |
| 181 |
def test_other_domain(self): |
| 182 |
response = self.client.get("http://otherserver/foo/bar/") |
| 183 |
|
| 184 |
Disabling :setting:`ALLOWED_HOSTS` checking (``ALLOWED_HOSTS = ['*']``) when |
| 185 |
running tests prevents the test client from raising a helpful error message if |
| 186 |
you follow a redirect to an external URL. |
| 187 |
|
| 188 |
.. _topics-testing-advanced-multidb: |
| 189 |
|
| 190 |
Tests and multiple databases |
| 191 |
============================ |
| 192 |
|
| 193 |
.. _topics-testing-primaryreplica: |
| 194 |
|
| 195 |
Testing primary/replica configurations |
| 196 |
-------------------------------------- |
| 197 |
|
| 198 |
If you're testing a multiple database configuration with primary/replica |
| 199 |
(referred to as master/slave by some databases) replication, this strategy of |
| 200 |
creating test databases poses a problem. |
| 201 |
When the test databases are created, there won't be any replication, |
| 202 |
and as a result, data created on the primary won't be seen on the |
| 203 |
replica. |
| 204 |
|
| 205 |
To compensate for this, Django allows you to define that a database is |
| 206 |
a *test mirror*. Consider the following (simplified) example database |
| 207 |
configuration:: |
| 208 |
|
| 209 |
DATABASES = { |
| 210 |
"default": { |
| 211 |
"ENGINE": "django.db.backends.mysql", |
| 212 |
"NAME": "myproject", |
| 213 |
"HOST": "dbprimary", |
| 214 |
# ... plus some other settings |
| 215 |
}, |
| 216 |
"replica": { |
| 217 |
"ENGINE": "django.db.backends.mysql", |
| 218 |
"NAME": "myproject", |
| 219 |
"HOST": "dbreplica", |
| 220 |
"TEST": { |
| 221 |
"MIRROR": "default", |
| 222 |
}, |
| 223 |
# ... plus some other settings |
| 224 |
}, |
| 225 |
} |
| 226 |
|
| 227 |
In this setup, we have two database servers: ``dbprimary``, described |
| 228 |
by the database alias ``default``, and ``dbreplica`` described by the |
| 229 |
alias ``replica``. As you might expect, ``dbreplica`` has been configured |
| 230 |
by the database administrator as a read replica of ``dbprimary``, so in |
| 231 |
normal activity, any write to ``default`` will appear on ``replica``. |
| 232 |
|
| 233 |
If Django created two independent test databases, this would break any |
| 234 |
tests that expected replication to occur. However, the ``replica`` |
| 235 |
database has been configured as a test mirror (using the |
| 236 |
:setting:`MIRROR <TEST_MIRROR>` test setting), indicating that under |
| 237 |
testing, ``replica`` should be treated as a mirror of ``default``. |
| 238 |
|
| 239 |
When the test environment is configured, a test version of ``replica`` |
| 240 |
will *not* be created. Instead the connection to ``replica`` |
| 241 |
will be redirected to point at ``default``. As a result, writes to |
| 242 |
``default`` will appear on ``replica`` -- but because they are actually |
| 243 |
the same database, not because there is data replication between the |
| 244 |
two databases. As this depends on transactions, the tests must use |
| 245 |
:class:`~django.test.TransactionTestCase` instead of |
| 246 |
:class:`~django.test.TestCase`. |
| 247 |
|
| 248 |
.. _topics-testing-creation-dependencies: |
| 249 |
|
| 250 |
Controlling creation order for test databases |
| 251 |
--------------------------------------------- |
| 252 |
|
| 253 |
By default, Django will assume all databases depend on the ``default`` |
| 254 |
database and therefore always create the ``default`` database first. |
| 255 |
However, no guarantees are made on the creation order of any other |
| 256 |
databases in your test setup. |
| 257 |
|
| 258 |
If your database configuration requires a specific creation order, you |
| 259 |
can specify the dependencies that exist using the :setting:`DEPENDENCIES |
| 260 |
<TEST_DEPENDENCIES>` test setting. Consider the following (simplified) |
| 261 |
example database configuration:: |
| 262 |
|
| 263 |
DATABASES = { |
| 264 |
"default": { |
| 265 |
# ... db settings |
| 266 |
"TEST": { |
| 267 |
"DEPENDENCIES": ["diamonds"], |
| 268 |
}, |
| 269 |
}, |
| 270 |
"diamonds": { |
| 271 |
# ... db settings |
| 272 |
"TEST": { |
| 273 |
"DEPENDENCIES": [], |
| 274 |
}, |
| 275 |
}, |
| 276 |
"clubs": { |
| 277 |
# ... db settings |
| 278 |
"TEST": { |
| 279 |
"DEPENDENCIES": ["diamonds"], |
| 280 |
}, |
| 281 |
}, |
| 282 |
"spades": { |
| 283 |
# ... db settings |
| 284 |
"TEST": { |
| 285 |
"DEPENDENCIES": ["diamonds", "hearts"], |
| 286 |
}, |
| 287 |
}, |
| 288 |
"hearts": { |
| 289 |
# ... db settings |
| 290 |
"TEST": { |
| 291 |
"DEPENDENCIES": ["diamonds", "clubs"], |
| 292 |
}, |
| 293 |
}, |
| 294 |
} |
| 295 |
|
| 296 |
Under this configuration, the ``diamonds`` database will be created first, |
| 297 |
as it is the only database alias without dependencies. The ``default`` and |
| 298 |
``clubs`` alias will be created next (although the order of creation of this |
| 299 |
pair is not guaranteed), then ``hearts``, and finally ``spades``. |
| 300 |
|
| 301 |
If there are any circular dependencies in the :setting:`DEPENDENCIES |
| 302 |
<TEST_DEPENDENCIES>` definition, an |
| 303 |
:exc:`~django.core.exceptions.ImproperlyConfigured` exception will be raised. |
| 304 |
|
| 305 |
Advanced features of ``TransactionTestCase`` |
| 306 |
============================================ |
| 307 |
|
| 308 |
.. attribute:: TransactionTestCase.available_apps |
| 309 |
|
| 310 |
.. warning:: |
| 311 |
|
| 312 |
This attribute is a private API. It may be changed or removed without |
| 313 |
a deprecation period in the future, for instance to accommodate changes |
| 314 |
in application loading. |
| 315 |
|
| 316 |
It's used to optimize Django's own test suite, which contains hundreds |
| 317 |
of models but no relations between models in different applications. |
| 318 |
|
| 319 |
By default, ``available_apps`` is set to ``None``. After each test, Django |
| 320 |
calls :djadmin:`flush` to reset the database state. This empties all tables |
| 321 |
and emits the :data:`~django.db.models.signals.post_migrate` signal, which |
| 322 |
recreates one content type and four permissions for each model. This |
| 323 |
operation gets expensive proportionally to the number of models. |
| 324 |
|
| 325 |
Setting ``available_apps`` to a list of applications instructs Django to |
| 326 |
behave as if only the models from these applications were available. The |
| 327 |
behavior of ``TransactionTestCase`` changes as follows: |
| 328 |
|
| 329 |
- :data:`~django.db.models.signals.post_migrate` is fired before each |
| 330 |
test to create the content types and permissions for each model in |
| 331 |
available apps, in case they're missing. |
| 332 |
- After each test, Django empties only tables corresponding to models in |
| 333 |
available apps. However, at the database level, truncation may cascade to |
| 334 |
related models in unavailable apps. Furthermore |
| 335 |
:data:`~django.db.models.signals.post_migrate` isn't fired; it will be |
| 336 |
fired by the next ``TransactionTestCase``, after the correct set of |
| 337 |
applications is selected. |
| 338 |
|
| 339 |
Since the database isn't fully flushed, if a test creates instances of |
| 340 |
models not included in ``available_apps``, they will leak and they may |
| 341 |
cause unrelated tests to fail. Be careful with tests that use sessions; |
| 342 |
the default session engine stores them in the database. |
| 343 |
|
| 344 |
Since :data:`~django.db.models.signals.post_migrate` isn't emitted after |
| 345 |
flushing the database, its state after a ``TransactionTestCase`` isn't the |
| 346 |
same as after a ``TestCase``: it's missing the rows created by listeners |
| 347 |
to :data:`~django.db.models.signals.post_migrate`. Considering the |
| 348 |
:ref:`order in which tests are executed <order-of-tests>`, this isn't an |
| 349 |
issue, provided either all ``TransactionTestCase`` in a given test suite |
| 350 |
declare ``available_apps``, or none of them. |
| 351 |
|
| 352 |
``available_apps`` is mandatory in Django's own test suite. |
| 353 |
|
| 354 |
.. attribute:: TransactionTestCase.reset_sequences |
| 355 |
|
| 356 |
Setting ``reset_sequences = True`` on a ``TransactionTestCase`` will make |
| 357 |
sure sequences are always reset before the test run:: |
| 358 |
|
| 359 |
class TestsThatDependsOnPrimaryKeySequences(TransactionTestCase): |
| 360 |
reset_sequences = True |
| 361 |
|
| 362 |
def test_animal_pk(self): |
| 363 |
lion = Animal.objects.create(name="lion", sound="roar") |
| 364 |
# lion.pk is guaranteed to always be 1 |
| 365 |
self.assertEqual(lion.pk, 1) |
| 366 |
|
| 367 |
Unless you are explicitly testing primary keys sequence numbers, it is |
| 368 |
recommended that you do not hard code primary key values in tests. |
| 369 |
|
| 370 |
Using ``reset_sequences = True`` will slow down the test, since the primary |
| 371 |
key reset is a relatively expensive database operation. |
| 372 |
|
| 373 |
.. _topics-testing-enforce-run-sequentially: |
| 374 |
|
| 375 |
Enforce running test classes sequentially |
| 376 |
========================================= |
| 377 |
|
| 378 |
If you have test classes that cannot be run in parallel (e.g. because they |
| 379 |
share a common resource), you can use ``django.test.testcases.SerializeMixin`` |
| 380 |
to run them sequentially. This mixin uses a filesystem ``lockfile``. |
| 381 |
|
| 382 |
For example, you can use ``__file__`` to determine that all test classes in the |
| 383 |
same file that inherit from ``SerializeMixin`` will run sequentially:: |
| 384 |
|
| 385 |
import os |
| 386 |
|
| 387 |
from django.test import TestCase |
| 388 |
from django.test.testcases import SerializeMixin |
| 389 |
|
| 390 |
|
| 391 |
class ImageTestCaseMixin(SerializeMixin): |
| 392 |
lockfile = __file__ |
| 393 |
|
| 394 |
def setUp(self): |
| 395 |
self.filename = os.path.join(temp_storage_dir, "my_file.png") |
| 396 |
self.file = create_file(self.filename) |
| 397 |
|
| 398 |
|
| 399 |
class RemoveImageTests(ImageTestCaseMixin, TestCase): |
| 400 |
def test_remove_image(self): |
| 401 |
os.remove(self.filename) |
| 402 |
self.assertFalse(os.path.exists(self.filename)) |
| 403 |
|
| 404 |
|
| 405 |
class ResizeImageTests(ImageTestCaseMixin, TestCase): |
| 406 |
def test_resize_image(self): |
| 407 |
resize_image(self.file, (48, 48)) |
| 408 |
self.assertEqual(get_image_size(self.file), (48, 48)) |
| 409 |
|
| 410 |
.. _testing-reusable-applications: |
| 411 |
|
| 412 |
Using the Django test runner to test reusable applications |
| 413 |
========================================================== |
| 414 |
|
| 415 |
If you are writing a :doc:`reusable application </intro/reusable-apps>` |
| 416 |
you may want to use the Django test runner to run your own test suite |
| 417 |
and thus benefit from the Django testing infrastructure. |
| 418 |
|
| 419 |
A common practice is a *tests* directory next to the application code, with the |
| 420 |
following structure: |
| 421 |
|
| 422 |
.. code-block:: text |
| 423 |
|
| 424 |
runtests.py |
| 425 |
polls/ |
| 426 |
__init__.py |
| 427 |
models.py |
| 428 |
... |
| 429 |
tests/ |
| 430 |
__init__.py |
| 431 |
models.py |
| 432 |
test_settings.py |
| 433 |
tests.py |
| 434 |
|
| 435 |
Let's take a look inside a couple of those files: |
| 436 |
|
| 437 |
.. code-block:: python |
| 438 |
:caption: ``runtests.py`` |
| 439 |
|
| 440 |
#!/usr/bin/env python |
| 441 |
import os |
| 442 |
import sys |
| 443 |
|
| 444 |
import django |
| 445 |
from django.conf import settings |
| 446 |
from django.test.utils import get_runner |
| 447 |
|
| 448 |
if __name__ == "__main__": |
| 449 |
os.environ["DJANGO_SETTINGS_MODULE"] = "tests.test_settings" |
| 450 |
django.setup() |
| 451 |
TestRunner = get_runner(settings) |
| 452 |
test_runner = TestRunner() |
| 453 |
failures = test_runner.run_tests(["tests"]) |
| 454 |
sys.exit(bool(failures)) |
| 455 |
|
| 456 |
|
| 457 |
This is the script that you invoke to run the test suite. It sets up the |
| 458 |
Django environment, creates the test database and runs the tests. |
| 459 |
|
| 460 |
For the sake of clarity, this example contains only the bare minimum |
| 461 |
necessary to use the Django test runner. You may want to add |
| 462 |
command-line options for controlling verbosity, passing in specific test |
| 463 |
labels to run, etc. |
| 464 |
|
| 465 |
.. code-block:: python |
| 466 |
:caption: ``tests/test_settings.py`` |
| 467 |
|
| 468 |
SECRET_KEY = "fake-key" |
| 469 |
INSTALLED_APPS = [ |
| 470 |
"tests", |
| 471 |
] |
| 472 |
|
| 473 |
This file contains the :doc:`Django settings </topics/settings>` |
| 474 |
required to run your app's tests. |
| 475 |
|
| 476 |
Again, this is a minimal example; your tests may require additional |
| 477 |
settings to run. |
| 478 |
|
| 479 |
Since the *tests* package is included in :setting:`INSTALLED_APPS` when |
| 480 |
running your tests, you can define test-only models in its ``models.py`` |
| 481 |
file. |
| 482 |
|
| 483 |
|
| 484 |
.. _other-testing-frameworks: |
| 485 |
|
| 486 |
Using different testing frameworks |
| 487 |
================================== |
| 488 |
|
| 489 |
Clearly, :mod:`unittest` is not the only Python testing framework. While Django |
| 490 |
doesn't provide explicit support for alternative frameworks, it does provide a |
| 491 |
way to invoke tests constructed for an alternative framework as if they were |
| 492 |
normal Django tests. |
| 493 |
|
| 494 |
When you run ``./manage.py test``, Django looks at the :setting:`TEST_RUNNER` |
| 495 |
setting to determine what to do. By default, :setting:`TEST_RUNNER` points to |
| 496 |
``'django.test.runner.DiscoverRunner'``. This class defines the default Django |
| 497 |
testing behavior. This behavior involves: |
| 498 |
|
| 499 |
#. Performing global pre-test setup. |
| 500 |
|
| 501 |
#. Looking for tests in any file below the current directory whose name matches |
| 502 |
the pattern ``test*.py``. |
| 503 |
|
| 504 |
#. Creating the test databases. |
| 505 |
|
| 506 |
#. Running ``migrate`` to install models and initial data into the test |
| 507 |
databases. |
| 508 |
|
| 509 |
#. Running the :doc:`system checks </topics/checks>`. |
| 510 |
|
| 511 |
#. Running the tests that were found. |
| 512 |
|
| 513 |
#. Destroying the test databases. |
| 514 |
|
| 515 |
#. Performing global post-test teardown. |
| 516 |
|
| 517 |
If you define your own test runner class and point :setting:`TEST_RUNNER` at |
| 518 |
that class, Django will execute your test runner whenever you run |
| 519 |
``./manage.py test``. In this way, it is possible to use any test framework |
| 520 |
that can be executed from Python code, or to modify the Django test execution |
| 521 |
process to satisfy whatever testing requirements you may have. |
| 522 |
|
| 523 |
.. _topics-testing-test_runner: |
| 524 |
|
| 525 |
Defining a test runner |
| 526 |
---------------------- |
| 527 |
|
| 528 |
.. currentmodule:: django.test.runner |
| 529 |
|
| 530 |
A test runner is a class defining a ``run_tests()`` method. Django ships |
| 531 |
with a ``DiscoverRunner`` class that defines the default Django testing |
| 532 |
behavior. This class defines the ``run_tests()`` entry point, plus a |
| 533 |
selection of other methods that are used by ``run_tests()`` to set up, execute |
| 534 |
and tear down the test suite. |
| 535 |
|
| 536 |
.. class:: DiscoverRunner(pattern='test*.py', top_level=None, verbosity=1, interactive=True, failfast=False, keepdb=False, reverse=False, debug_mode=False, debug_sql=False, parallel=0, tags=None, exclude_tags=None, test_name_patterns=None, pdb=False, buffer=False, enable_faulthandler=True, timing=True, shuffle=False, logger=None, durations=None, **kwargs) |
| 537 |
|
| 538 |
``DiscoverRunner`` will search for tests in any file matching ``pattern``. |
| 539 |
|
| 540 |
``top_level`` can be used to specify the directory containing your |
| 541 |
top-level Python modules. Usually Django can figure this out automatically, |
| 542 |
so it's not necessary to specify this option. If specified, it should |
| 543 |
generally be the directory containing your ``manage.py`` file. |
| 544 |
|
| 545 |
``verbosity`` determines the amount of notification and debug information |
| 546 |
that will be printed to the console; ``0`` is no output, ``1`` is normal |
| 547 |
output, and ``2`` is verbose output. |
| 548 |
|
| 549 |
If ``interactive`` is ``True``, the test suite has permission to ask the |
| 550 |
user for instructions when the test suite is executed. An example of this |
| 551 |
behavior would be asking for permission to delete an existing test |
| 552 |
database. If ``interactive`` is ``False``, the test suite must be able to |
| 553 |
run without any manual intervention. |
| 554 |
|
| 555 |
If ``failfast`` is ``True``, the test suite will stop running after the |
| 556 |
first test failure is detected. |
| 557 |
|
| 558 |
If ``keepdb`` is ``True``, the test suite will use the existing database, |
| 559 |
or create one if necessary. If ``False``, a new database will be created, |
| 560 |
prompting the user to remove the existing one, if present. |
| 561 |
|
| 562 |
If ``reverse`` is ``True``, test cases will be executed in the opposite |
| 563 |
order. This could be useful to debug tests that aren't properly isolated |
| 564 |
and have side effects. :ref:`Grouping by test class <order-of-tests>` is |
| 565 |
preserved when using this option. This option can be used in conjunction |
| 566 |
with ``--shuffle`` to reverse the order for a particular random seed. |
| 567 |
|
| 568 |
``debug_mode`` specifies what the :setting:`DEBUG` setting should be |
| 569 |
set to prior to running tests. |
| 570 |
|
| 571 |
``parallel`` specifies the number of processes. If ``parallel`` is greater |
| 572 |
than ``1``, the test suite will run in ``parallel`` processes. If there are |
| 573 |
fewer test case classes than configured processes, Django will reduce the |
| 574 |
number of processes accordingly. Each process gets its own database. This |
| 575 |
option requires the third-party ``tblib`` package to display tracebacks |
| 576 |
correctly. |
| 577 |
|
| 578 |
``tags`` can be used to specify a set of :ref:`tags for filtering tests |
| 579 |
<topics-tagging-tests>`. May be combined with ``exclude_tags``. |
| 580 |
|
| 581 |
``exclude_tags`` can be used to specify a set of |
| 582 |
:ref:`tags for excluding tests <topics-tagging-tests>`. May be combined |
| 583 |
with ``tags``. |
| 584 |
|
| 585 |
If ``debug_sql`` is ``True``, failing test cases will output SQL queries |
| 586 |
logged to the :ref:`django.db.backends logger <django-db-logger>` as well |
| 587 |
as the traceback. If ``verbosity`` is ``2``, then queries in all tests are |
| 588 |
output. |
| 589 |
|
| 590 |
``test_name_patterns`` can be used to specify a set of patterns for |
| 591 |
filtering test methods and classes by their names. |
| 592 |
|
| 593 |
If ``pdb`` is ``True``, a debugger (``pdb`` or ``ipdb``) will be spawned at |
| 594 |
each test error or failure. |
| 595 |
|
| 596 |
If ``buffer`` is ``True``, outputs from passing tests will be discarded. |
| 597 |
|
| 598 |
If ``enable_faulthandler`` is ``True``, :py:mod:`faulthandler` will be |
| 599 |
enabled. |
| 600 |
|
| 601 |
If ``timing`` is ``True``, test timings, including database setup and total |
| 602 |
run time, will be shown. |
| 603 |
|
| 604 |
If ``shuffle`` is an integer, test cases will be shuffled in a random order |
| 605 |
prior to execution, using the integer as a random seed. If ``shuffle`` is |
| 606 |
``None``, the seed will be generated randomly. In both cases, the seed will |
| 607 |
be logged and set to ``self.shuffle_seed`` prior to running tests. This |
| 608 |
option can be used to help detect tests that aren't properly isolated. |
| 609 |
:ref:`Grouping by test class <order-of-tests>` is preserved when using this |
| 610 |
option. |
| 611 |
|
| 612 |
``logger`` can be used to pass a Python :py:ref:`Logger object <logger>`. |
| 613 |
If provided, the logger will be used to log messages instead of printing to |
| 614 |
the console. The logger object will respect its logging level rather than |
| 615 |
the ``verbosity``. |
| 616 |
|
| 617 |
``durations`` will show a list of the N slowest test cases. Setting this |
| 618 |
option to ``0`` will result in the duration for all tests being shown. |
| 619 |
Requires Python 3.12+. |
| 620 |
|
| 621 |
Django may, from time to time, extend the capabilities of the test runner |
| 622 |
by adding new arguments. The ``**kwargs`` declaration allows for this |
| 623 |
expansion. If you subclass ``DiscoverRunner`` or write your own test |
| 624 |
runner, ensure it accepts ``**kwargs``. |
| 625 |
|
| 626 |
Your test runner may also define additional command-line options. |
| 627 |
Create or override an ``add_arguments(cls, parser)`` class method and add |
| 628 |
custom arguments by calling ``parser.add_argument()`` inside the method, so |
| 629 |
that the :djadmin:`test` command will be able to use those arguments. |
| 630 |
|
| 631 |
Attributes |
| 632 |
~~~~~~~~~~ |
| 633 |
|
| 634 |
.. attribute:: DiscoverRunner.test_suite |
| 635 |
|
| 636 |
The class used to build the test suite. By default it is set to |
| 637 |
``unittest.TestSuite``. This can be overridden if you wish to implement |
| 638 |
different logic for collecting tests. |
| 639 |
|
| 640 |
.. attribute:: DiscoverRunner.test_runner |
| 641 |
|
| 642 |
This is the class of the low-level test runner which is used to execute |
| 643 |
the individual tests and format the results. By default it is set to |
| 644 |
``unittest.TextTestRunner``. Despite the unfortunate similarity in |
| 645 |
naming conventions, this is not the same type of class as |
| 646 |
``DiscoverRunner``, which covers a broader set of responsibilities. You |
| 647 |
can override this attribute to modify the way tests are run and reported. |
| 648 |
|
| 649 |
.. attribute:: DiscoverRunner.test_loader |
| 650 |
|
| 651 |
This is the class that loads tests, whether from TestCases or modules or |
| 652 |
otherwise and bundles them into test suites for the runner to execute. |
| 653 |
By default it is set to ``unittest.defaultTestLoader``. You can override |
| 654 |
this attribute if your tests are going to be loaded in unusual ways. |
| 655 |
|
| 656 |
Methods |
| 657 |
~~~~~~~ |
| 658 |
|
| 659 |
.. method:: DiscoverRunner.run_tests(test_labels, **kwargs) |
| 660 |
|
| 661 |
Run the test suite. |
| 662 |
|
| 663 |
``test_labels`` allows you to specify which tests to run and supports |
| 664 |
several formats (see :meth:`DiscoverRunner.build_suite` for a list of |
| 665 |
supported formats). |
| 666 |
|
| 667 |
This method should return the number of tests that failed. |
| 668 |
|
| 669 |
.. classmethod:: DiscoverRunner.add_arguments(parser) |
| 670 |
|
| 671 |
Override this class method to add custom arguments accepted by the |
| 672 |
:djadmin:`test` management command. See |
| 673 |
:py:meth:`argparse.ArgumentParser.add_argument()` for details about adding |
| 674 |
arguments to a parser. |
| 675 |
|
| 676 |
.. method:: DiscoverRunner.setup_test_environment(**kwargs) |
| 677 |
|
| 678 |
Sets up the test environment by calling |
| 679 |
:func:`~django.test.utils.setup_test_environment` and setting |
| 680 |
:setting:`DEBUG` to ``self.debug_mode`` (defaults to ``False``). |
| 681 |
|
| 682 |
.. method:: DiscoverRunner.build_suite(test_labels=None, **kwargs) |
| 683 |
|
| 684 |
Constructs a test suite that matches the test labels provided. |
| 685 |
|
| 686 |
``test_labels`` is a list of strings describing the tests to be run. A test |
| 687 |
label can take one of four forms: |
| 688 |
|
| 689 |
* ``path.to.test_module.TestCase.test_method`` -- Run a single test method |
| 690 |
in a test case class. |
| 691 |
* ``path.to.test_module.TestCase`` -- Run all the test methods in a test |
| 692 |
case. |
| 693 |
* ``path.to.module`` -- Search for and run all tests in the named Python |
| 694 |
package or module. |
| 695 |
* ``path/to/directory`` -- Search for and run all tests below the named |
| 696 |
directory. |
| 697 |
|
| 698 |
If ``test_labels`` has a value of ``None``, the test runner will search for |
| 699 |
tests in all files below the current directory whose names match its |
| 700 |
``pattern`` (see above). |
| 701 |
|
| 702 |
Returns a ``TestSuite`` instance ready to be run. |
| 703 |
|
| 704 |
.. method:: DiscoverRunner.setup_databases(**kwargs) |
| 705 |
|
| 706 |
Creates the test databases by calling |
| 707 |
:func:`~django.test.utils.setup_databases`. |
| 708 |
|
| 709 |
.. method:: DiscoverRunner.run_checks(databases) |
| 710 |
|
| 711 |
Runs the :doc:`system checks </topics/checks>` on the test ``databases``. |
| 712 |
|
| 713 |
.. method:: DiscoverRunner.run_suite(suite, **kwargs) |
| 714 |
|
| 715 |
Runs the test suite. |
| 716 |
|
| 717 |
Returns the result produced by the running the test suite. |
| 718 |
|
| 719 |
.. method:: DiscoverRunner.get_test_runner_kwargs() |
| 720 |
|
| 721 |
Returns the keyword arguments to instantiate the |
| 722 |
``DiscoverRunner.test_runner`` with. |
| 723 |
|
| 724 |
.. method:: DiscoverRunner.teardown_databases(old_config, **kwargs) |
| 725 |
|
| 726 |
Destroys the test databases, restoring pre-test conditions by calling |
| 727 |
:func:`~django.test.utils.teardown_databases`. |
| 728 |
|
| 729 |
.. method:: DiscoverRunner.teardown_test_environment(**kwargs) |
| 730 |
|
| 731 |
Restores the pre-test environment. |
| 732 |
|
| 733 |
.. method:: DiscoverRunner.suite_result(suite, result, **kwargs) |
| 734 |
|
| 735 |
Computes and returns a return code based on a test suite, and the result |
| 736 |
from that test suite. |
| 737 |
|
| 738 |
.. method:: DiscoverRunner.log(msg, level=None) |
| 739 |
|
| 740 |
If a ``logger`` is set, logs the message at the given integer |
| 741 |
`logging level`_ (e.g. ``logging.DEBUG``, ``logging.INFO``, or |
| 742 |
``logging.WARNING``). Otherwise, the message is printed to the console, |
| 743 |
respecting the current ``verbosity``. For example, no message will be |
| 744 |
printed if the ``verbosity`` is 0, ``INFO`` and above will be printed if |
| 745 |
the ``verbosity`` is at least 1, and ``DEBUG`` will be printed if it is at |
| 746 |
least 2. The ``level`` defaults to ``logging.INFO``. |
| 747 |
|
| 748 |
.. _`logging level`: https://docs.python.org/3/library/logging.html#levels |
| 749 |
|
| 750 |
Testing utilities |
| 751 |
----------------- |
| 752 |
|
| 753 |
``django.test.utils`` |
| 754 |
~~~~~~~~~~~~~~~~~~~~~ |
| 755 |
|
| 756 |
.. module:: django.test.utils |
| 757 |
:synopsis: Helpers to write custom test runners. |
| 758 |
|
| 759 |
To assist in the creation of your own test runner, Django provides a number of |
| 760 |
utility methods in the ``django.test.utils`` module. |
| 761 |
|
| 762 |
.. function:: setup_test_environment(debug=None) |
| 763 |
|
| 764 |
Performs global pre-test setup, such as installing instrumentation for the |
| 765 |
template rendering system and setting up the dummy email outbox. |
| 766 |
|
| 767 |
If ``debug`` isn't ``None``, the :setting:`DEBUG` setting is updated to its |
| 768 |
value. |
| 769 |
|
| 770 |
.. function:: teardown_test_environment() |
| 771 |
|
| 772 |
Performs global post-test teardown, such as removing instrumentation from |
| 773 |
the template system and restoring normal email services. |
| 774 |
|
| 775 |
.. function:: setup_databases(verbosity, interactive, *, time_keeper=None, keepdb=False, debug_sql=False, parallel=0, aliases=None, serialized_aliases=None, **kwargs) |
| 776 |
|
| 777 |
Creates the test databases. |
| 778 |
|
| 779 |
Returns a data structure that provides enough detail to undo the changes |
| 780 |
that have been made. This data will be provided to the |
| 781 |
:func:`teardown_databases` function at the conclusion of testing. |
| 782 |
|
| 783 |
The ``aliases`` argument determines which :setting:`DATABASES` aliases test |
| 784 |
databases should be set up for. If it's not provided, it defaults to all of |
| 785 |
:setting:`DATABASES` aliases. |
| 786 |
|
| 787 |
The ``serialized_aliases`` argument determines what subset of ``aliases`` |
| 788 |
test databases should have their state serialized to allow usage of the |
| 789 |
:ref:`serialized_rollback <test-case-serialized-rollback>` feature. If |
| 790 |
it's not provided, it defaults to ``aliases``. |
| 791 |
|
| 792 |
.. function:: teardown_databases(old_config, parallel=0, keepdb=False) |
| 793 |
|
| 794 |
Destroys the test databases, restoring pre-test conditions. |
| 795 |
|
| 796 |
``old_config`` is a data structure defining the changes in the database |
| 797 |
configuration that need to be reversed. It's the return value of the |
| 798 |
:meth:`setup_databases` method. |
| 799 |
|
| 800 |
``django.db.connection.creation`` |
| 801 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 802 |
|
| 803 |
.. currentmodule:: django.db.connection.creation |
| 804 |
|
| 805 |
The creation module of the database backend also provides some utilities that |
| 806 |
can be useful during testing. |
| 807 |
|
| 808 |
.. function:: create_test_db(verbosity=1, autoclobber=False, serialize=True, keepdb=False) |
| 809 |
|
| 810 |
Creates a new test database and runs ``migrate`` against it. |
| 811 |
|
| 812 |
``verbosity`` has the same behavior as in ``run_tests()``. |
| 813 |
|
| 814 |
``autoclobber`` describes the behavior that will occur if a |
| 815 |
database with the same name as the test database is discovered: |
| 816 |
|
| 817 |
* If ``autoclobber`` is ``False``, the user will be asked to |
| 818 |
approve destroying the existing database. ``sys.exit`` is |
| 819 |
called if the user does not approve. |
| 820 |
|
| 821 |
* If ``autoclobber`` is ``True``, the database will be destroyed |
| 822 |
without consulting the user. |
| 823 |
|
| 824 |
``serialize`` determines if Django serializes the database into an |
| 825 |
in-memory JSON string before running tests (used to restore the database |
| 826 |
state between tests if you don't have transactions). You can set this to |
| 827 |
``False`` to speed up creation time if you don't have any test classes |
| 828 |
with :ref:`serialized_rollback=True <test-case-serialized-rollback>`. |
| 829 |
|
| 830 |
``keepdb`` determines if the test run should use an existing |
| 831 |
database, or create a new one. If ``True``, the existing |
| 832 |
database will be used, or created if not present. If ``False``, |
| 833 |
a new database will be created, prompting the user to remove |
| 834 |
the existing one, if present. |
| 835 |
|
| 836 |
Returns the name of the test database that it created. |
| 837 |
|
| 838 |
``create_test_db()`` has the side effect of modifying the value of |
| 839 |
:setting:`NAME` in :setting:`DATABASES` to match the name of the test |
| 840 |
database. |
| 841 |
|
| 842 |
.. function:: destroy_test_db(old_database_name, verbosity=1, keepdb=False) |
| 843 |
|
| 844 |
Destroys the database whose name is the value of :setting:`NAME` in |
| 845 |
:setting:`DATABASES`, and sets :setting:`NAME` to the value of |
| 846 |
``old_database_name``. |
| 847 |
|
| 848 |
The ``verbosity`` argument has the same behavior as for |
| 849 |
:class:`~django.test.runner.DiscoverRunner`. |
| 850 |
|
| 851 |
If the ``keepdb`` argument is ``True``, then the connection to the |
| 852 |
database will be closed, but the database will not be destroyed. |
| 853 |
|
| 854 |
.. function:: serialize_db_to_string() |
| 855 |
|
| 856 |
Serializes the database into an in-memory JSON string that can be used to |
| 857 |
restore the database state between tests if the backend doesn't support |
| 858 |
transactions or if your suite contains test classes with |
| 859 |
:ref:`serialized_rollback=True <test-case-serialized-rollback>` enabled. |
| 860 |
|
| 861 |
This function should only be called once all test databases have been |
| 862 |
created as the serialization process could result in queries against |
| 863 |
non-test databases depending on your |
| 864 |
:ref:`routing configuration <topics-db-multi-db-routing>`. |
| 865 |
|
| 866 |
.. _topics-testing-code-coverage: |
| 867 |
|
| 868 |
Integration with ``coverage.py`` |
| 869 |
================================ |
| 870 |
|
| 871 |
Code coverage describes how much source code has been tested. It shows which |
| 872 |
parts of your code are being exercised by tests and which are not. It's an |
| 873 |
important part of testing applications, so it's strongly recommended to check |
| 874 |
the coverage of your tests. |
| 875 |
|
| 876 |
Django can be easily integrated with `coverage.py`_, a tool for measuring code |
| 877 |
coverage of Python programs. First, install :pypi:`coverage`. Next, run the |
| 878 |
following from your project folder containing ``manage.py``: |
| 879 |
|
| 880 |
.. code-block:: shell |
| 881 |
|
| 882 |
coverage run --source='.' manage.py test myapp |
| 883 |
|
| 884 |
This runs your tests and collects coverage data of the executed files in your |
| 885 |
project. You can see a report of this data by typing following command: |
| 886 |
|
| 887 |
.. code-block:: shell |
| 888 |
|
| 889 |
coverage report |
| 890 |
|
| 891 |
Note that some Django code was executed while running tests, but it is not |
| 892 |
listed here because of the ``source`` flag passed to the previous command. |
| 893 |
|
| 894 |
For more options like annotated HTML listings detailing missed lines, see the |
| 895 |
`coverage.py`_ docs. |
| 896 |
|
| 897 |
.. _coverage.py: https://coverage.readthedocs.io/ |