Debian Codegraph @

docs/topics/testing/overview.txt

raw
1 =========================
2 Writing and running tests
3 =========================
4
5 .. module:: django.test
6 :synopsis: Testing tools for Django applications.
7
8 .. seealso::
9
10 The :doc:`testing tutorial </intro/tutorial05>`, the :doc:`testing tools
11 reference </topics/testing/tools>`, and the :doc:`advanced testing topics
12 </topics/testing/advanced>`.
13
14 This document is split into two primary sections. First, we explain how to write
15 tests with Django. Then, we explain how to run them.
16
17 Writing tests
18 =============
19
20 Django's unit tests use a Python standard library module: :mod:`unittest`. This
21 module defines tests using a class-based approach.
22
23 Here is an example which subclasses from :class:`django.test.TestCase`,
24 which is a subclass of :class:`unittest.TestCase` that runs each test inside a
25 transaction to provide isolation::
26
27 from django.test import TestCase
28 from myapp.models import Animal
29
30
31 class AnimalTestCase(TestCase):
32 def setUp(self):
33 Animal.objects.create(name="lion", sound="roar")
34 Animal.objects.create(name="cat", sound="meow")
35
36 def test_animals_can_speak(self):
37 """Animals that can speak are correctly identified"""
38 lion = Animal.objects.get(name="lion")
39 cat = Animal.objects.get(name="cat")
40 self.assertEqual(lion.speak(), 'The lion says "roar"')
41 self.assertEqual(cat.speak(), 'The cat says "meow"')
42
43 When you :ref:`run your tests <running-tests>`, the default behavior of the
44 test utility is to find all the test case classes (that is, subclasses of
45 :class:`unittest.TestCase`) in any file whose name begins with ``test``,
46 automatically build a test suite out of those test case classes, and run that
47 suite.
48
49 For more details about :mod:`unittest`, see the Python documentation.
50
51 .. admonition:: Where should the tests live?
52
53 The default :djadmin:`startapp` template creates a ``tests.py`` file in the
54 new application. This might be fine if you only have a few tests, but as
55 your test suite grows you'll likely want to restructure it into a tests
56 package so you can split your tests into different submodules such as
57 ``test_models.py``, ``test_views.py``, ``test_forms.py``, etc. Feel free to
58 pick whatever organizational scheme you like.
59
60 See also :ref:`testing-reusable-applications`.
61
62 .. warning::
63
64 If your tests rely on database access such as creating or querying models,
65 be sure to create your test classes as subclasses of
66 :class:`django.test.TestCase` rather than :class:`unittest.TestCase`.
67
68 Using :class:`unittest.TestCase` avoids the cost of running each test in a
69 transaction and flushing the database, but if your tests interact with
70 the database their behavior will vary based on the order that the test
71 runner executes them. This can lead to unit tests that pass when run in
72 isolation but fail when run in a suite.
73
74 .. _running-tests:
75
76 Running tests
77 =============
78
79 Once you've written tests, run them using the :djadmin:`test` command of
80 your project's ``manage.py`` utility:
81
82 .. code-block:: shell
83
84 $ ./manage.py test
85
86 Test discovery is based on the unittest module's :py:ref:`built-in test
87 discovery <unittest-test-discovery>`. By default, this will discover tests in
88 any file named ``test*.py`` under the current working directory.
89
90 You can specify particular tests to run by supplying any number of "test
91 labels" to ``./manage.py test``. Each test label can be a full Python dotted
92 path to a package, module, ``TestCase`` subclass, or test method. For instance:
93
94 .. code-block:: shell
95
96 # Run all the tests in the animals.tests module
97 $ ./manage.py test animals.tests
98
99 # Run all the tests found within the 'animals' package
100 $ ./manage.py test animals
101
102 # Run just one test case class
103 $ ./manage.py test animals.tests.AnimalTestCase
104
105 # Run just one test method
106 $ ./manage.py test animals.tests.AnimalTestCase.test_animals_can_speak
107
108 You can also provide a path to a directory to discover tests below that
109 directory:
110
111 .. code-block:: shell
112
113 $ ./manage.py test animals/
114
115 You can specify a custom filename pattern match using the ``-p`` (or
116 ``--pattern``) option, if your test files are named differently from the
117 ``test*.py`` pattern:
118
119 .. code-block:: shell
120
121 $ ./manage.py test --pattern="tests_*.py"
122
123 If you press ``Ctrl-C`` while the tests are running, the test runner will
124 wait for the currently running test to complete and then exit gracefully.
125 During a graceful exit the test runner will output details of any test
126 failures, report on how many tests were run and how many errors and failures
127 were encountered, and destroy any test databases as usual. Thus pressing
128 ``Ctrl-C`` can be very useful if you forget to pass the :option:`--failfast
129 <test --failfast>` option, notice that some tests are unexpectedly failing and
130 want to get details on the failures without waiting for the full test run to
131 complete.
132
133 If you do not want to wait for the currently running test to finish, you
134 can press ``Ctrl-C`` a second time and the test run will halt immediately,
135 but not gracefully. No details of the tests run before the interruption will
136 be reported, and any test databases created by the run will not be destroyed.
137
138 .. admonition:: Test with warnings enabled
139
140 It's a good idea to run your tests with Python warnings enabled:
141 ``python -Wa manage.py test``. The ``-Wa`` flag tells Python to
142 display deprecation warnings. Django, like many other Python libraries,
143 uses these warnings to flag when features are going away. It also might
144 flag areas in your code that aren't strictly wrong but could benefit
145 from a better implementation.
146
147
148 .. _the-test-database:
149
150 The test database
151 -----------------
152
153 Tests that require a database (namely, model tests) will not use your "real"
154 (production) database. Separate, blank databases are created for the tests.
155
156 Regardless of whether the tests pass or fail, the test databases are destroyed
157 when all the tests have been executed.
158
159 You can prevent the test databases from being destroyed by using the
160 :option:`test --keepdb` option. This will preserve the test database between
161 runs. If the database does not exist, it will first be created. Any migrations
162 will also be applied in order to keep it up to date.
163
164 As described in the previous section, if a test run is forcefully interrupted,
165 the test database may not be destroyed. On the next run, you'll be asked
166 whether you want to reuse or destroy the database. Use the :option:`test
167 --noinput` option to suppress that prompt and automatically destroy the
168 database. This can be useful when running tests on a continuous integration
169 server where tests may be interrupted by a timeout, for example.
170
171 The default test database names are created by prepending ``test_`` to the
172 value of each :setting:`NAME` in :setting:`DATABASES`. When using SQLite, the
173 tests will use an in-memory database by default (i.e., the database will be
174 created in memory, bypassing the filesystem entirely!). The :setting:`TEST
175 <DATABASE-TEST>` dictionary in :setting:`DATABASES` offers a number of settings
176 to configure your test database. For example, if you want to use a different
177 database name, specify :setting:`NAME <TEST_NAME>` in the :setting:`TEST
178 <DATABASE-TEST>` dictionary for any given database in :setting:`DATABASES`.
179
180 On PostgreSQL, :setting:`USER` will also need read access to the built-in
181 ``postgres`` database.
182
183 Aside from using a separate database, the test runner will otherwise
184 use all of the same database settings you have in your settings file:
185 :setting:`ENGINE <DATABASE-ENGINE>`, :setting:`USER`, :setting:`HOST`, etc. The
186 test database is created by the user specified by :setting:`USER`, so you'll
187 need to make sure that the given user account has sufficient privileges to
188 create a new database on the system.
189
190 For fine-grained control over the character encoding of your test
191 database, use the :setting:`CHARSET <TEST_CHARSET>` TEST option. If you're using
192 MySQL, you can also use the :setting:`COLLATION <TEST_COLLATION>` option to
193 control the particular collation used by the test database. See the
194 :doc:`settings documentation </ref/settings>` for details of these
195 and other advanced settings.
196
197 If using an SQLite in-memory database with SQLite, `shared cache
198 <https://www.sqlite.org/sharedcache.html>`_ is enabled, so you can write tests
199 with ability to share the database between threads.
200
201 .. admonition:: Finding data from your production database when running tests?
202
203 If your code attempts to access the database when its modules are compiled,
204 this will occur *before* the test database is set up, with potentially
205 unexpected results. For example, if you have a database query in
206 module-level code and a real database exists, production data could pollute
207 your tests. *It is a bad idea to have such import-time database queries in
208 your code* anyway - rewrite your code so that it doesn't do this.
209
210 This also applies to customized implementations of
211 :meth:`~django.apps.AppConfig.ready()`.
212
213 .. seealso::
214
215 The :ref:`advanced multi-db testing topics <topics-testing-advanced-multidb>`.
216
217 .. _order-of-tests:
218
219 Order in which tests are executed
220 ---------------------------------
221
222 In order to guarantee that all ``TestCase`` code starts with a clean database,
223 the Django test runner reorders tests in the following way:
224
225 * All :class:`~django.test.TestCase` subclasses are run first.
226
227 * Then, all other Django-based tests (test case classes based on
228 :class:`~django.test.SimpleTestCase`, including
229 :class:`~django.test.TransactionTestCase`) are run with no particular
230 ordering guaranteed nor enforced among them.
231
232 * Then any other :class:`unittest.TestCase` tests (including doctests) that may
233 alter the database without restoring it to its original state are run.
234
235 .. note::
236
237 The new ordering of tests may reveal unexpected dependencies on test case
238 ordering. This is the case with doctests that relied on state left in the
239 database by a given :class:`~django.test.TransactionTestCase` test, they
240 must be updated to be able to run independently.
241
242 .. note::
243
244 Failures detected when loading tests are ordered before all of the above
245 for quicker feedback. This includes things like test modules that couldn't
246 be found or that couldn't be loaded due to syntax errors.
247
248 You may randomize and/or reverse the execution order inside groups using the
249 :option:`test --shuffle` and :option:`--reverse <test --reverse>` options. This
250 can help with ensuring your tests are independent from each other.
251
252 .. _test-case-serialized-rollback:
253
254 Rollback emulation
255 ------------------
256
257 Any initial data loaded in migrations will only be available in ``TestCase``
258 tests and not in ``TransactionTestCase`` tests, and additionally only on
259 backends where transactions are supported (the most important exception being
260 MyISAM). This is also true for tests which rely on ``TransactionTestCase``
261 such as :class:`LiveServerTestCase` and
262 :class:`~django.contrib.staticfiles.testing.StaticLiveServerTestCase`.
263
264 Django can reload that data for you on a per-testcase basis by
265 setting the ``serialized_rollback`` option to ``True`` in the body of the
266 ``TestCase`` or ``TransactionTestCase``, but note that this will slow down
267 that test suite by approximately 3x.
268
269 Third-party apps or those developing against MyISAM will need to set this;
270 in general, however, you should be developing your own projects against a
271 transactional database and be using ``TestCase`` for most tests, and thus
272 not need this setting.
273
274 The initial serialization is usually very quick, but if you wish to exclude
275 some apps from this process (and speed up test runs slightly), you may add
276 those apps to :setting:`TEST_NON_SERIALIZED_APPS`.
277
278 To prevent serialized data from being loaded twice, setting
279 ``serialized_rollback=True`` disables the
280 :data:`~django.db.models.signals.post_migrate` signal when flushing the test
281 database.
282
283 .. versionchanged:: 5.2
284
285 For :class:`TransactionTestCase`, serialized migration data is made
286 available during ``setUpClass()``.
287
288 Other test conditions
289 ---------------------
290
291 Regardless of the value of the :setting:`DEBUG` setting in your configuration
292 file, all Django tests run with :setting:`DEBUG`\=False. This is to ensure that
293 the observed output of your code matches what will be seen in a production
294 setting.
295
296 Caches are not cleared after each test, and running ``manage.py test fooapp``
297 can insert data from the tests into the cache of a live system if you run your
298 tests in production because, unlike databases, a separate "test cache" is not
299 used. This behavior :ticket:`may change <11505>` in the future.
300
301 Understanding the test output
302 -----------------------------
303
304 When you run your tests, you'll see a number of messages as the test runner
305 prepares itself. You can control the level of detail of these messages with the
306 ``verbosity`` option on the command line:
307
308 .. code-block:: shell
309
310 Creating test database...
311 Creating table myapp_animal
312 Creating table myapp_mineral
313
314 This tells you that the test runner is creating a test database, as described
315 in the previous section.
316
317 Once the test database has been created, Django will run your tests.
318 If everything goes well, you'll see something like this:
319
320 .. code-block:: shell
321
322 ----------------------------------------------------------------------
323 Ran 22 tests in 0.221s
324
325 OK
326
327 If there are test failures, however, you'll see full details about which tests
328 failed:
329
330 .. code-block:: shell
331
332 ======================================================================
333 FAIL: test_was_published_recently_with_future_poll (polls.tests.PollMethodTests)
334 ----------------------------------------------------------------------
335 Traceback (most recent call last):
336 File "/dev/mysite/polls/tests.py", line 16, in test_was_published_recently_with_future_poll
337 self.assertIs(future_poll.was_published_recently(), False)
338 AssertionError: True is not False
339
340 ----------------------------------------------------------------------
341 Ran 1 test in 0.003s
342
343 FAILED (failures=1)
344
345 A full explanation of this error output is beyond the scope of this document,
346 but it's pretty intuitive. You can consult the documentation of Python's
347 :mod:`unittest` library for details.
348
349 Note that the return code for the test-runner script is 1 for any number of
350 failed tests (whether the failure was caused by an error, a failed assertion,
351 or an unexpected success). If all the tests pass, the return code is 0. This
352 feature is useful if you're using the test-runner script in a shell script and
353 need to test for success or failure at that level.
354
355 .. _speeding-up-tests-auth-hashers:
356
357 Speeding up the tests
358 ---------------------
359
360 Running tests in parallel
361 ~~~~~~~~~~~~~~~~~~~~~~~~~
362
363 As long as your tests are properly isolated, you can run them in parallel to
364 gain a speed up on multi-core hardware. See :option:`test --parallel`.
365
366 Password hashing
367 ~~~~~~~~~~~~~~~~
368
369 The default password hasher is rather slow by design. If you're authenticating
370 many users in your tests, you may want to use a custom settings file and set
371 the :setting:`PASSWORD_HASHERS` setting to a faster hashing algorithm::
372
373 PASSWORD_HASHERS = [
374 "django.contrib.auth.hashers.MD5PasswordHasher",
375 ]
376
377 Don't forget to also include in :setting:`PASSWORD_HASHERS` any hashing
378 algorithm used in fixtures, if any.
379
380 Preserving the test database
381 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
382
383 The :option:`test --keepdb` option preserves the test database between test
384 runs. It skips the create and destroy actions which can greatly decrease the
385 time to run tests.
386
387 Avoiding disk access for media files
388 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
389
390 The :class:`~django.core.files.storage.InMemoryStorage` is a convenient way to
391 prevent disk access for media files. All data is kept in memory, then it gets
392 discarded after tests run.