Debian Codegraph @

docs/topics/testing/advanced.txt

raw
1 =======================
2 Advanced testing topics
3 =======================
4
5 The request factory
6 ===================
7
8 .. currentmodule:: django.test
9
10 .. class:: RequestFactory
11
12 The :class:`~django.test.RequestFactory` shares the same API as
13 the test client. However, instead of behaving like a browser, the
14 RequestFactory provides a way to generate a request instance that can
15 be used as the first argument to any view. This means you can test a
16 view function the same way as you would test any other function -- as
17 a black box, with exactly known inputs, testing for specific outputs.
18
19 The API for the :class:`~django.test.RequestFactory` is a slightly
20 restricted subset of the test client API:
21
22 * It only has access to the HTTP methods :meth:`~Client.get()`,
23 :meth:`~Client.post()`, :meth:`~Client.put()`,
24 :meth:`~Client.delete()`, :meth:`~Client.head()`,
25 :meth:`~Client.options()`, and :meth:`~Client.trace()`.
26
27 * These methods accept all the same arguments *except* for
28 ``follow``. Since this is just a factory for producing
29 requests, it's up to you to handle the response.
30
31 * It does not support middleware. Session and authentication
32 attributes must be supplied by the test itself if required
33 for the view to function properly.
34
35 .. versionchanged:: 5.1
36
37 The ``query_params`` parameter was added.
38
39 Example
40 -------
41
42 The following is a unit test using the request factory::
43
44 from django.contrib.auth.models import AnonymousUser, User
45 from django.test import RequestFactory, TestCase
46
47 from .views import MyView, my_view
48
49
50 class SimpleTest(TestCase):
51 def setUp(self):
52 # Every test needs access to the request factory.
53 self.factory = RequestFactory()
54 self.user = User.objects.create_user(
55 username="jacob", email="jacob@…", password="top_secret"
56 )
57
58 def test_details(self):
59 # Create an instance of a GET request.
60 request = self.factory.get("/customer/details")
61
62 # Recall that middleware are not supported. You can simulate a
63 # logged-in user by setting request.user manually.
64 request.user = self.user
65
66 # Or you can simulate an anonymous user by setting request.user to
67 # an AnonymousUser instance.
68 request.user = AnonymousUser()
69
70 # Test my_view() as if it were deployed at /customer/details
71 response = my_view(request)
72 # Use this syntax for class-based views.
73 response = MyView.as_view()(request)
74 self.assertEqual(response.status_code, 200)
75
76 AsyncRequestFactory
77 -------------------
78
79 .. class:: AsyncRequestFactory
80
81 ``RequestFactory`` creates WSGI-like requests. If you want to create ASGI-like
82 requests, including having a correct ASGI ``scope``, you can instead use
83 ``django.test.AsyncRequestFactory``.
84
85 This class is directly API-compatible with ``RequestFactory``, with the only
86 difference being that it returns ``ASGIRequest`` instances rather than
87 ``WSGIRequest`` instances. All of its methods are still synchronous callables.
88
89 Arbitrary keyword arguments in ``defaults`` are added directly into the ASGI
90 scope.
91
92 .. versionchanged:: 5.1
93
94 The ``query_params`` parameter was added.
95
96 Testing class-based views
97 =========================
98
99 In order to test class-based views outside of the request/response cycle you
100 must ensure that they are configured correctly, by calling
101 :meth:`~django.views.generic.base.View.setup` after instantiation.
102
103 For example, assuming the following class-based view:
104
105 .. code-block:: python
106 :caption: ``views.py``
107
108 from django.views.generic import TemplateView
109
110
111 class HomeView(TemplateView):
112 template_name = "myapp/home.html"
113
114 def get_context_data(self, **kwargs):
115 kwargs["environment"] = "Production"
116 return super().get_context_data(**kwargs)
117
118 You may directly test the ``get_context_data()`` method by first instantiating
119 the view, then passing a ``request`` to ``setup()``, before proceeding with
120 your test's code:
121
122 .. code-block:: python
123 :caption: ``tests.py``
124
125 from django.test import RequestFactory, TestCase
126 from .views import HomeView
127
128
129 class HomePageTest(TestCase):
130 def test_environment_set_in_context(self):
131 request = RequestFactory().get("/")
132 view = HomeView()
133 view.setup(request)
134
135 context = view.get_context_data()
136 self.assertIn("environment", context)
137
138 .. _topics-testing-advanced-multiple-hosts:
139
140 Tests and multiple host names
141 =============================
142
143 The :setting:`ALLOWED_HOSTS` setting is validated when running tests. This
144 allows the test client to differentiate between internal and external URLs.
145
146 Projects that support multitenancy or otherwise alter business logic based on
147 the request's host and use custom host names in tests must include those hosts
148 in :setting:`ALLOWED_HOSTS`.
149
150 The first option to do so is to add the hosts to your settings file. For
151 example, the test suite for docs.djangoproject.com includes the following::
152
153 from django.test import TestCase
154
155
156 class SearchFormTestCase(TestCase):
157 def test_empty_get(self):
158 response = self.client.get(
159 "/en/dev/search/",
160 headers={"host": "docs.djangoproject.dev:8000"},
161 )
162 self.assertEqual(response.status_code, 200)
163
164 and the settings file includes a list of the domains supported by the project::
165
166 ALLOWED_HOSTS = ["www.djangoproject.dev", "docs.djangoproject.dev", ...]
167
168 Another option is to add the required hosts to :setting:`ALLOWED_HOSTS` using
169 :meth:`~django.test.override_settings()` or
170 :attr:`~django.test.SimpleTestCase.modify_settings()`. This option may be
171 preferable in standalone apps that can't package their own settings file or
172 for projects where the list of domains is not static (e.g., subdomains for
173 multitenancy). For example, you could write a test for the domain
174 ``http://otherserver/`` as follows::
175
176 from django.test import TestCase, override_settings
177
178
179 class MultiDomainTestCase(TestCase):
180 @override_settings(ALLOWED_HOSTS=["otherserver"])
181 def test_other_domain(self):
182 response = self.client.get("http://otherserver/foo/bar/")
183
184 Disabling :setting:`ALLOWED_HOSTS` checking (``ALLOWED_HOSTS = ['*']``) when
185 running tests prevents the test client from raising a helpful error message if
186 you follow a redirect to an external URL.
187
188 .. _topics-testing-advanced-multidb:
189
190 Tests and multiple databases
191 ============================
192
193 .. _topics-testing-primaryreplica:
194
195 Testing primary/replica configurations
196 --------------------------------------
197
198 If you're testing a multiple database configuration with primary/replica
199 (referred to as master/slave by some databases) replication, this strategy of
200 creating test databases poses a problem.
201 When the test databases are created, there won't be any replication,
202 and as a result, data created on the primary won't be seen on the
203 replica.
204
205 To compensate for this, Django allows you to define that a database is
206 a *test mirror*. Consider the following (simplified) example database
207 configuration::
208
209 DATABASES = {
210 "default": {
211 "ENGINE": "django.db.backends.mysql",
212 "NAME": "myproject",
213 "HOST": "dbprimary",
214 # ... plus some other settings
215 },
216 "replica": {
217 "ENGINE": "django.db.backends.mysql",
218 "NAME": "myproject",
219 "HOST": "dbreplica",
220 "TEST": {
221 "MIRROR": "default",
222 },
223 # ... plus some other settings
224 },
225 }
226
227 In this setup, we have two database servers: ``dbprimary``, described
228 by the database alias ``default``, and ``dbreplica`` described by the
229 alias ``replica``. As you might expect, ``dbreplica`` has been configured
230 by the database administrator as a read replica of ``dbprimary``, so in
231 normal activity, any write to ``default`` will appear on ``replica``.
232
233 If Django created two independent test databases, this would break any
234 tests that expected replication to occur. However, the ``replica``
235 database has been configured as a test mirror (using the
236 :setting:`MIRROR <TEST_MIRROR>` test setting), indicating that under
237 testing, ``replica`` should be treated as a mirror of ``default``.
238
239 When the test environment is configured, a test version of ``replica``
240 will *not* be created. Instead the connection to ``replica``
241 will be redirected to point at ``default``. As a result, writes to
242 ``default`` will appear on ``replica`` -- but because they are actually
243 the same database, not because there is data replication between the
244 two databases. As this depends on transactions, the tests must use
245 :class:`~django.test.TransactionTestCase` instead of
246 :class:`~django.test.TestCase`.
247
248 .. _topics-testing-creation-dependencies:
249
250 Controlling creation order for test databases
251 ---------------------------------------------
252
253 By default, Django will assume all databases depend on the ``default``
254 database and therefore always create the ``default`` database first.
255 However, no guarantees are made on the creation order of any other
256 databases in your test setup.
257
258 If your database configuration requires a specific creation order, you
259 can specify the dependencies that exist using the :setting:`DEPENDENCIES
260 <TEST_DEPENDENCIES>` test setting. Consider the following (simplified)
261 example database configuration::
262
263 DATABASES = {
264 "default": {
265 # ... db settings
266 "TEST": {
267 "DEPENDENCIES": ["diamonds"],
268 },
269 },
270 "diamonds": {
271 # ... db settings
272 "TEST": {
273 "DEPENDENCIES": [],
274 },
275 },
276 "clubs": {
277 # ... db settings
278 "TEST": {
279 "DEPENDENCIES": ["diamonds"],
280 },
281 },
282 "spades": {
283 # ... db settings
284 "TEST": {
285 "DEPENDENCIES": ["diamonds", "hearts"],
286 },
287 },
288 "hearts": {
289 # ... db settings
290 "TEST": {
291 "DEPENDENCIES": ["diamonds", "clubs"],
292 },
293 },
294 }
295
296 Under this configuration, the ``diamonds`` database will be created first,
297 as it is the only database alias without dependencies. The ``default`` and
298 ``clubs`` alias will be created next (although the order of creation of this
299 pair is not guaranteed), then ``hearts``, and finally ``spades``.
300
301 If there are any circular dependencies in the :setting:`DEPENDENCIES
302 <TEST_DEPENDENCIES>` definition, an
303 :exc:`~django.core.exceptions.ImproperlyConfigured` exception will be raised.
304
305 Advanced features of ``TransactionTestCase``
306 ============================================
307
308 .. attribute:: TransactionTestCase.available_apps
309
310 .. warning::
311
312 This attribute is a private API. It may be changed or removed without
313 a deprecation period in the future, for instance to accommodate changes
314 in application loading.
315
316 It's used to optimize Django's own test suite, which contains hundreds
317 of models but no relations between models in different applications.
318
319 By default, ``available_apps`` is set to ``None``. After each test, Django
320 calls :djadmin:`flush` to reset the database state. This empties all tables
321 and emits the :data:`~django.db.models.signals.post_migrate` signal, which
322 recreates one content type and four permissions for each model. This
323 operation gets expensive proportionally to the number of models.
324
325 Setting ``available_apps`` to a list of applications instructs Django to
326 behave as if only the models from these applications were available. The
327 behavior of ``TransactionTestCase`` changes as follows:
328
329 - :data:`~django.db.models.signals.post_migrate` is fired before each
330 test to create the content types and permissions for each model in
331 available apps, in case they're missing.
332 - After each test, Django empties only tables corresponding to models in
333 available apps. However, at the database level, truncation may cascade to
334 related models in unavailable apps. Furthermore
335 :data:`~django.db.models.signals.post_migrate` isn't fired; it will be
336 fired by the next ``TransactionTestCase``, after the correct set of
337 applications is selected.
338
339 Since the database isn't fully flushed, if a test creates instances of
340 models not included in ``available_apps``, they will leak and they may
341 cause unrelated tests to fail. Be careful with tests that use sessions;
342 the default session engine stores them in the database.
343
344 Since :data:`~django.db.models.signals.post_migrate` isn't emitted after
345 flushing the database, its state after a ``TransactionTestCase`` isn't the
346 same as after a ``TestCase``: it's missing the rows created by listeners
347 to :data:`~django.db.models.signals.post_migrate`. Considering the
348 :ref:`order in which tests are executed <order-of-tests>`, this isn't an
349 issue, provided either all ``TransactionTestCase`` in a given test suite
350 declare ``available_apps``, or none of them.
351
352 ``available_apps`` is mandatory in Django's own test suite.
353
354 .. attribute:: TransactionTestCase.reset_sequences
355
356 Setting ``reset_sequences = True`` on a ``TransactionTestCase`` will make
357 sure sequences are always reset before the test run::
358
359 class TestsThatDependsOnPrimaryKeySequences(TransactionTestCase):
360 reset_sequences = True
361
362 def test_animal_pk(self):
363 lion = Animal.objects.create(name="lion", sound="roar")
364 # lion.pk is guaranteed to always be 1
365 self.assertEqual(lion.pk, 1)
366
367 Unless you are explicitly testing primary keys sequence numbers, it is
368 recommended that you do not hard code primary key values in tests.
369
370 Using ``reset_sequences = True`` will slow down the test, since the primary
371 key reset is a relatively expensive database operation.
372
373 .. _topics-testing-enforce-run-sequentially:
374
375 Enforce running test classes sequentially
376 =========================================
377
378 If you have test classes that cannot be run in parallel (e.g. because they
379 share a common resource), you can use ``django.test.testcases.SerializeMixin``
380 to run them sequentially. This mixin uses a filesystem ``lockfile``.
381
382 For example, you can use ``__file__`` to determine that all test classes in the
383 same file that inherit from ``SerializeMixin`` will run sequentially::
384
385 import os
386
387 from django.test import TestCase
388 from django.test.testcases import SerializeMixin
389
390
391 class ImageTestCaseMixin(SerializeMixin):
392 lockfile = __file__
393
394 def setUp(self):
395 self.filename = os.path.join(temp_storage_dir, "my_file.png")
396 self.file = create_file(self.filename)
397
398
399 class RemoveImageTests(ImageTestCaseMixin, TestCase):
400 def test_remove_image(self):
401 os.remove(self.filename)
402 self.assertFalse(os.path.exists(self.filename))
403
404
405 class ResizeImageTests(ImageTestCaseMixin, TestCase):
406 def test_resize_image(self):
407 resize_image(self.file, (48, 48))
408 self.assertEqual(get_image_size(self.file), (48, 48))
409
410 .. _testing-reusable-applications:
411
412 Using the Django test runner to test reusable applications
413 ==========================================================
414
415 If you are writing a :doc:`reusable application </intro/reusable-apps>`
416 you may want to use the Django test runner to run your own test suite
417 and thus benefit from the Django testing infrastructure.
418
419 A common practice is a *tests* directory next to the application code, with the
420 following structure:
421
422 .. code-block:: text
423
424 runtests.py
425 polls/
426 __init__.py
427 models.py
428 ...
429 tests/
430 __init__.py
431 models.py
432 test_settings.py
433 tests.py
434
435 Let's take a look inside a couple of those files:
436
437 .. code-block:: python
438 :caption: ``runtests.py``
439
440 #!/usr/bin/env python
441 import os
442 import sys
443
444 import django
445 from django.conf import settings
446 from django.test.utils import get_runner
447
448 if __name__ == "__main__":
449 os.environ["DJANGO_SETTINGS_MODULE"] = "tests.test_settings"
450 django.setup()
451 TestRunner = get_runner(settings)
452 test_runner = TestRunner()
453 failures = test_runner.run_tests(["tests"])
454 sys.exit(bool(failures))
455
456
457 This is the script that you invoke to run the test suite. It sets up the
458 Django environment, creates the test database and runs the tests.
459
460 For the sake of clarity, this example contains only the bare minimum
461 necessary to use the Django test runner. You may want to add
462 command-line options for controlling verbosity, passing in specific test
463 labels to run, etc.
464
465 .. code-block:: python
466 :caption: ``tests/test_settings.py``
467
468 SECRET_KEY = "fake-key"
469 INSTALLED_APPS = [
470 "tests",
471 ]
472
473 This file contains the :doc:`Django settings </topics/settings>`
474 required to run your app's tests.
475
476 Again, this is a minimal example; your tests may require additional
477 settings to run.
478
479 Since the *tests* package is included in :setting:`INSTALLED_APPS` when
480 running your tests, you can define test-only models in its ``models.py``
481 file.
482
483
484 .. _other-testing-frameworks:
485
486 Using different testing frameworks
487 ==================================
488
489 Clearly, :mod:`unittest` is not the only Python testing framework. While Django
490 doesn't provide explicit support for alternative frameworks, it does provide a
491 way to invoke tests constructed for an alternative framework as if they were
492 normal Django tests.
493
494 When you run ``./manage.py test``, Django looks at the :setting:`TEST_RUNNER`
495 setting to determine what to do. By default, :setting:`TEST_RUNNER` points to
496 ``'django.test.runner.DiscoverRunner'``. This class defines the default Django
497 testing behavior. This behavior involves:
498
499 #. Performing global pre-test setup.
500
501 #. Looking for tests in any file below the current directory whose name matches
502 the pattern ``test*.py``.
503
504 #. Creating the test databases.
505
506 #. Running ``migrate`` to install models and initial data into the test
507 databases.
508
509 #. Running the :doc:`system checks </topics/checks>`.
510
511 #. Running the tests that were found.
512
513 #. Destroying the test databases.
514
515 #. Performing global post-test teardown.
516
517 If you define your own test runner class and point :setting:`TEST_RUNNER` at
518 that class, Django will execute your test runner whenever you run
519 ``./manage.py test``. In this way, it is possible to use any test framework
520 that can be executed from Python code, or to modify the Django test execution
521 process to satisfy whatever testing requirements you may have.
522
523 .. _topics-testing-test_runner:
524
525 Defining a test runner
526 ----------------------
527
528 .. currentmodule:: django.test.runner
529
530 A test runner is a class defining a ``run_tests()`` method. Django ships
531 with a ``DiscoverRunner`` class that defines the default Django testing
532 behavior. This class defines the ``run_tests()`` entry point, plus a
533 selection of other methods that are used by ``run_tests()`` to set up, execute
534 and tear down the test suite.
535
536 .. class:: DiscoverRunner(pattern='test*.py', top_level=None, verbosity=1, interactive=True, failfast=False, keepdb=False, reverse=False, debug_mode=False, debug_sql=False, parallel=0, tags=None, exclude_tags=None, test_name_patterns=None, pdb=False, buffer=False, enable_faulthandler=True, timing=True, shuffle=False, logger=None, durations=None, **kwargs)
537
538 ``DiscoverRunner`` will search for tests in any file matching ``pattern``.
539
540 ``top_level`` can be used to specify the directory containing your
541 top-level Python modules. Usually Django can figure this out automatically,
542 so it's not necessary to specify this option. If specified, it should
543 generally be the directory containing your ``manage.py`` file.
544
545 ``verbosity`` determines the amount of notification and debug information
546 that will be printed to the console; ``0`` is no output, ``1`` is normal
547 output, and ``2`` is verbose output.
548
549 If ``interactive`` is ``True``, the test suite has permission to ask the
550 user for instructions when the test suite is executed. An example of this
551 behavior would be asking for permission to delete an existing test
552 database. If ``interactive`` is ``False``, the test suite must be able to
553 run without any manual intervention.
554
555 If ``failfast`` is ``True``, the test suite will stop running after the
556 first test failure is detected.
557
558 If ``keepdb`` is ``True``, the test suite will use the existing database,
559 or create one if necessary. If ``False``, a new database will be created,
560 prompting the user to remove the existing one, if present.
561
562 If ``reverse`` is ``True``, test cases will be executed in the opposite
563 order. This could be useful to debug tests that aren't properly isolated
564 and have side effects. :ref:`Grouping by test class <order-of-tests>` is
565 preserved when using this option. This option can be used in conjunction
566 with ``--shuffle`` to reverse the order for a particular random seed.
567
568 ``debug_mode`` specifies what the :setting:`DEBUG` setting should be
569 set to prior to running tests.
570
571 ``parallel`` specifies the number of processes. If ``parallel`` is greater
572 than ``1``, the test suite will run in ``parallel`` processes. If there are
573 fewer test case classes than configured processes, Django will reduce the
574 number of processes accordingly. Each process gets its own database. This
575 option requires the third-party ``tblib`` package to display tracebacks
576 correctly.
577
578 ``tags`` can be used to specify a set of :ref:`tags for filtering tests
579 <topics-tagging-tests>`. May be combined with ``exclude_tags``.
580
581 ``exclude_tags`` can be used to specify a set of
582 :ref:`tags for excluding tests <topics-tagging-tests>`. May be combined
583 with ``tags``.
584
585 If ``debug_sql`` is ``True``, failing test cases will output SQL queries
586 logged to the :ref:`django.db.backends logger <django-db-logger>` as well
587 as the traceback. If ``verbosity`` is ``2``, then queries in all tests are
588 output.
589
590 ``test_name_patterns`` can be used to specify a set of patterns for
591 filtering test methods and classes by their names.
592
593 If ``pdb`` is ``True``, a debugger (``pdb`` or ``ipdb``) will be spawned at
594 each test error or failure.
595
596 If ``buffer`` is ``True``, outputs from passing tests will be discarded.
597
598 If ``enable_faulthandler`` is ``True``, :py:mod:`faulthandler` will be
599 enabled.
600
601 If ``timing`` is ``True``, test timings, including database setup and total
602 run time, will be shown.
603
604 If ``shuffle`` is an integer, test cases will be shuffled in a random order
605 prior to execution, using the integer as a random seed. If ``shuffle`` is
606 ``None``, the seed will be generated randomly. In both cases, the seed will
607 be logged and set to ``self.shuffle_seed`` prior to running tests. This
608 option can be used to help detect tests that aren't properly isolated.
609 :ref:`Grouping by test class <order-of-tests>` is preserved when using this
610 option.
611
612 ``logger`` can be used to pass a Python :py:ref:`Logger object <logger>`.
613 If provided, the logger will be used to log messages instead of printing to
614 the console. The logger object will respect its logging level rather than
615 the ``verbosity``.
616
617 ``durations`` will show a list of the N slowest test cases. Setting this
618 option to ``0`` will result in the duration for all tests being shown.
619 Requires Python 3.12+.
620
621 Django may, from time to time, extend the capabilities of the test runner
622 by adding new arguments. The ``**kwargs`` declaration allows for this
623 expansion. If you subclass ``DiscoverRunner`` or write your own test
624 runner, ensure it accepts ``**kwargs``.
625
626 Your test runner may also define additional command-line options.
627 Create or override an ``add_arguments(cls, parser)`` class method and add
628 custom arguments by calling ``parser.add_argument()`` inside the method, so
629 that the :djadmin:`test` command will be able to use those arguments.
630
631 Attributes
632 ~~~~~~~~~~
633
634 .. attribute:: DiscoverRunner.test_suite
635
636 The class used to build the test suite. By default it is set to
637 ``unittest.TestSuite``. This can be overridden if you wish to implement
638 different logic for collecting tests.
639
640 .. attribute:: DiscoverRunner.test_runner
641
642 This is the class of the low-level test runner which is used to execute
643 the individual tests and format the results. By default it is set to
644 ``unittest.TextTestRunner``. Despite the unfortunate similarity in
645 naming conventions, this is not the same type of class as
646 ``DiscoverRunner``, which covers a broader set of responsibilities. You
647 can override this attribute to modify the way tests are run and reported.
648
649 .. attribute:: DiscoverRunner.test_loader
650
651 This is the class that loads tests, whether from TestCases or modules or
652 otherwise and bundles them into test suites for the runner to execute.
653 By default it is set to ``unittest.defaultTestLoader``. You can override
654 this attribute if your tests are going to be loaded in unusual ways.
655
656 Methods
657 ~~~~~~~
658
659 .. method:: DiscoverRunner.run_tests(test_labels, **kwargs)
660
661 Run the test suite.
662
663 ``test_labels`` allows you to specify which tests to run and supports
664 several formats (see :meth:`DiscoverRunner.build_suite` for a list of
665 supported formats).
666
667 This method should return the number of tests that failed.
668
669 .. classmethod:: DiscoverRunner.add_arguments(parser)
670
671 Override this class method to add custom arguments accepted by the
672 :djadmin:`test` management command. See
673 :py:meth:`argparse.ArgumentParser.add_argument()` for details about adding
674 arguments to a parser.
675
676 .. method:: DiscoverRunner.setup_test_environment(**kwargs)
677
678 Sets up the test environment by calling
679 :func:`~django.test.utils.setup_test_environment` and setting
680 :setting:`DEBUG` to ``self.debug_mode`` (defaults to ``False``).
681
682 .. method:: DiscoverRunner.build_suite(test_labels=None, **kwargs)
683
684 Constructs a test suite that matches the test labels provided.
685
686 ``test_labels`` is a list of strings describing the tests to be run. A test
687 label can take one of four forms:
688
689 * ``path.to.test_module.TestCase.test_method`` -- Run a single test method
690 in a test case class.
691 * ``path.to.test_module.TestCase`` -- Run all the test methods in a test
692 case.
693 * ``path.to.module`` -- Search for and run all tests in the named Python
694 package or module.
695 * ``path/to/directory`` -- Search for and run all tests below the named
696 directory.
697
698 If ``test_labels`` has a value of ``None``, the test runner will search for
699 tests in all files below the current directory whose names match its
700 ``pattern`` (see above).
701
702 Returns a ``TestSuite`` instance ready to be run.
703
704 .. method:: DiscoverRunner.setup_databases(**kwargs)
705
706 Creates the test databases by calling
707 :func:`~django.test.utils.setup_databases`.
708
709 .. method:: DiscoverRunner.run_checks(databases)
710
711 Runs the :doc:`system checks </topics/checks>` on the test ``databases``.
712
713 .. method:: DiscoverRunner.run_suite(suite, **kwargs)
714
715 Runs the test suite.
716
717 Returns the result produced by the running the test suite.
718
719 .. method:: DiscoverRunner.get_test_runner_kwargs()
720
721 Returns the keyword arguments to instantiate the
722 ``DiscoverRunner.test_runner`` with.
723
724 .. method:: DiscoverRunner.teardown_databases(old_config, **kwargs)
725
726 Destroys the test databases, restoring pre-test conditions by calling
727 :func:`~django.test.utils.teardown_databases`.
728
729 .. method:: DiscoverRunner.teardown_test_environment(**kwargs)
730
731 Restores the pre-test environment.
732
733 .. method:: DiscoverRunner.suite_result(suite, result, **kwargs)
734
735 Computes and returns a return code based on a test suite, and the result
736 from that test suite.
737
738 .. method:: DiscoverRunner.log(msg, level=None)
739
740 If a ``logger`` is set, logs the message at the given integer
741 `logging level`_ (e.g. ``logging.DEBUG``, ``logging.INFO``, or
742 ``logging.WARNING``). Otherwise, the message is printed to the console,
743 respecting the current ``verbosity``. For example, no message will be
744 printed if the ``verbosity`` is 0, ``INFO`` and above will be printed if
745 the ``verbosity`` is at least 1, and ``DEBUG`` will be printed if it is at
746 least 2. The ``level`` defaults to ``logging.INFO``.
747
748 .. _`logging level`: https://docs.python.org/3/library/logging.html#levels
749
750 Testing utilities
751 -----------------
752
753 ``django.test.utils``
754 ~~~~~~~~~~~~~~~~~~~~~
755
756 .. module:: django.test.utils
757 :synopsis: Helpers to write custom test runners.
758
759 To assist in the creation of your own test runner, Django provides a number of
760 utility methods in the ``django.test.utils`` module.
761
762 .. function:: setup_test_environment(debug=None)
763
764 Performs global pre-test setup, such as installing instrumentation for the
765 template rendering system and setting up the dummy email outbox.
766
767 If ``debug`` isn't ``None``, the :setting:`DEBUG` setting is updated to its
768 value.
769
770 .. function:: teardown_test_environment()
771
772 Performs global post-test teardown, such as removing instrumentation from
773 the template system and restoring normal email services.
774
775 .. function:: setup_databases(verbosity, interactive, *, time_keeper=None, keepdb=False, debug_sql=False, parallel=0, aliases=None, serialized_aliases=None, **kwargs)
776
777 Creates the test databases.
778
779 Returns a data structure that provides enough detail to undo the changes
780 that have been made. This data will be provided to the
781 :func:`teardown_databases` function at the conclusion of testing.
782
783 The ``aliases`` argument determines which :setting:`DATABASES` aliases test
784 databases should be set up for. If it's not provided, it defaults to all of
785 :setting:`DATABASES` aliases.
786
787 The ``serialized_aliases`` argument determines what subset of ``aliases``
788 test databases should have their state serialized to allow usage of the
789 :ref:`serialized_rollback <test-case-serialized-rollback>` feature. If
790 it's not provided, it defaults to ``aliases``.
791
792 .. function:: teardown_databases(old_config, parallel=0, keepdb=False)
793
794 Destroys the test databases, restoring pre-test conditions.
795
796 ``old_config`` is a data structure defining the changes in the database
797 configuration that need to be reversed. It's the return value of the
798 :meth:`setup_databases` method.
799
800 ``django.db.connection.creation``
801 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
802
803 .. currentmodule:: django.db.connection.creation
804
805 The creation module of the database backend also provides some utilities that
806 can be useful during testing.
807
808 .. function:: create_test_db(verbosity=1, autoclobber=False, serialize=True, keepdb=False)
809
810 Creates a new test database and runs ``migrate`` against it.
811
812 ``verbosity`` has the same behavior as in ``run_tests()``.
813
814 ``autoclobber`` describes the behavior that will occur if a
815 database with the same name as the test database is discovered:
816
817 * If ``autoclobber`` is ``False``, the user will be asked to
818 approve destroying the existing database. ``sys.exit`` is
819 called if the user does not approve.
820
821 * If ``autoclobber`` is ``True``, the database will be destroyed
822 without consulting the user.
823
824 ``serialize`` determines if Django serializes the database into an
825 in-memory JSON string before running tests (used to restore the database
826 state between tests if you don't have transactions). You can set this to
827 ``False`` to speed up creation time if you don't have any test classes
828 with :ref:`serialized_rollback=True <test-case-serialized-rollback>`.
829
830 ``keepdb`` determines if the test run should use an existing
831 database, or create a new one. If ``True``, the existing
832 database will be used, or created if not present. If ``False``,
833 a new database will be created, prompting the user to remove
834 the existing one, if present.
835
836 Returns the name of the test database that it created.
837
838 ``create_test_db()`` has the side effect of modifying the value of
839 :setting:`NAME` in :setting:`DATABASES` to match the name of the test
840 database.
841
842 .. function:: destroy_test_db(old_database_name, verbosity=1, keepdb=False)
843
844 Destroys the database whose name is the value of :setting:`NAME` in
845 :setting:`DATABASES`, and sets :setting:`NAME` to the value of
846 ``old_database_name``.
847
848 The ``verbosity`` argument has the same behavior as for
849 :class:`~django.test.runner.DiscoverRunner`.
850
851 If the ``keepdb`` argument is ``True``, then the connection to the
852 database will be closed, but the database will not be destroyed.
853
854 .. function:: serialize_db_to_string()
855
856 Serializes the database into an in-memory JSON string that can be used to
857 restore the database state between tests if the backend doesn't support
858 transactions or if your suite contains test classes with
859 :ref:`serialized_rollback=True <test-case-serialized-rollback>` enabled.
860
861 This function should only be called once all test databases have been
862 created as the serialization process could result in queries against
863 non-test databases depending on your
864 :ref:`routing configuration <topics-db-multi-db-routing>`.
865
866 .. _topics-testing-code-coverage:
867
868 Integration with ``coverage.py``
869 ================================
870
871 Code coverage describes how much source code has been tested. It shows which
872 parts of your code are being exercised by tests and which are not. It's an
873 important part of testing applications, so it's strongly recommended to check
874 the coverage of your tests.
875
876 Django can be easily integrated with `coverage.py`_, a tool for measuring code
877 coverage of Python programs. First, install :pypi:`coverage`. Next, run the
878 following from your project folder containing ``manage.py``:
879
880 .. code-block:: shell
881
882 coverage run --source='.' manage.py test myapp
883
884 This runs your tests and collects coverage data of the executed files in your
885 project. You can see a report of this data by typing following command:
886
887 .. code-block:: shell
888
889 coverage report
890
891 Note that some Django code was executed while running tests, but it is not
892 listed here because of the ``source`` flag passed to the previous command.
893
894 For more options like annotated HTML listings detailing missed lines, see the
895 `coverage.py`_ docs.
896
897 .. _coverage.py: https://coverage.readthedocs.io/