Debian Codegraph @

Doc/library/unittest.rst

raw
1 :mod:`!unittest` --- Unit testing framework
2 ===========================================
3
4 .. module:: unittest
5 :synopsis: Unit testing framework for Python.
6
7 .. moduleauthor:: Steve Purcell <[email protected]>
8 .. sectionauthor:: Steve Purcell <[email protected]>
9 .. sectionauthor:: Fred L. Drake, Jr. <[email protected]>
10 .. sectionauthor:: Raymond Hettinger <[email protected]>
11
12 **Source code:** :source:`Lib/unittest/__init__.py`
13
14 --------------
15
16 (If you are already familiar with the basic concepts of testing, you might want
17 to skip to :ref:`the list of assert methods <assert-methods>`.)
18
19 The :mod:`unittest` unit testing framework was originally inspired by JUnit
20 and has a similar flavor as major unit testing frameworks in other
21 languages. It supports test automation, sharing of setup and shutdown code
22 for tests, aggregation of tests into collections, and independence of the
23 tests from the reporting framework.
24
25 To achieve this, :mod:`unittest` supports some important concepts in an
26 object-oriented way:
27
28 test fixture
29 A :dfn:`test fixture` represents the preparation needed to perform one or more
30 tests, and any associated cleanup actions. This may involve, for example,
31 creating temporary or proxy databases, directories, or starting a server
32 process.
33
34 test case
35 A :dfn:`test case` is the individual unit of testing. It checks for a specific
36 response to a particular set of inputs. :mod:`unittest` provides a base class,
37 :class:`TestCase`, which may be used to create new test cases.
38
39 test suite
40 A :dfn:`test suite` is a collection of test cases, test suites, or both. It is
41 used to aggregate tests that should be executed together.
42
43 test runner
44 A :dfn:`test runner` is a component which orchestrates the execution of tests
45 and provides the outcome to the user. The runner may use a graphical interface,
46 a textual interface, or return a special value to indicate the results of
47 executing the tests.
48
49
50 .. seealso::
51
52 Module :mod:`doctest`
53 Another test-support module with a very different flavor.
54
55 `Simple Smalltalk Testing: With Patterns <https://web.archive.org/web/20150315073817/http://www.xprogramming.com/testfram.htm>`_
56 Kent Beck's original paper on testing frameworks using the pattern shared
57 by :mod:`unittest`.
58
59 `pytest <https://docs.pytest.org/>`_
60 Third-party unittest framework with a lighter-weight syntax for writing
61 tests. For example, ``assert func(10) == 42``.
62
63 `The Python Testing Tools Taxonomy <https://wiki.python.org/moin/PythonTestingToolsTaxonomy>`_
64 An extensive list of Python testing tools including functional testing
65 frameworks and mock object libraries.
66
67 `Testing in Python Mailing List <http://lists.idyll.org/listinfo/testing-in-python>`_
68 A special-interest-group for discussion of testing, and testing tools,
69 in Python.
70
71 The script :file:`Tools/unittestgui/unittestgui.py` in the Python source distribution is
72 a GUI tool for test discovery and execution. This is intended largely for ease of use
73 for those new to unit testing. For production environments it is
74 recommended that tests be driven by a continuous integration system such as
75 `Buildbot <https://buildbot.net/>`_, `Jenkins <https://www.jenkins.io/>`_,
76 `GitHub Actions <https://github.com/features/actions>`_, or
77 `AppVeyor <https://www.appveyor.com/>`_.
78
79
80 .. _unittest-minimal-example:
81
82 Basic example
83 -------------
84
85 The :mod:`unittest` module provides a rich set of tools for constructing and
86 running tests. This section demonstrates that a small subset of the tools
87 suffice to meet the needs of most users.
88
89 Here is a short script to test three string methods::
90
91 import unittest
92
93 class TestStringMethods(unittest.TestCase):
94
95 def test_upper(self):
96 self.assertEqual('foo'.upper(), 'FOO')
97
98 def test_isupper(self):
99 self.assertTrue('FOO'.isupper())
100 self.assertFalse('Foo'.isupper())
101
102 def test_split(self):
103 s = 'hello world'
104 self.assertEqual(s.split(), ['hello', 'world'])
105 # check that s.split fails when the separator is not a string
106 with self.assertRaises(TypeError):
107 s.split(2)
108
109 if __name__ == '__main__':
110 unittest.main()
111
112
113 A test case is created by subclassing :class:`unittest.TestCase`. The three
114 individual tests are defined with methods whose names start with the letters
115 ``test``. This naming convention informs the test runner about which methods
116 represent tests.
117
118 The crux of each test is a call to :meth:`~TestCase.assertEqual` to check for an
119 expected result; :meth:`~TestCase.assertTrue` or :meth:`~TestCase.assertFalse`
120 to verify a condition; or :meth:`~TestCase.assertRaises` to verify that a
121 specific exception gets raised. These methods are used instead of the
122 :keyword:`assert` statement so the test runner can accumulate all test results
123 and produce a report.
124
125 The :meth:`~TestCase.setUp` and :meth:`~TestCase.tearDown` methods allow you
126 to define instructions that will be executed before and after each test method.
127 They are covered in more detail in the section :ref:`organizing-tests`.
128
129 The final block shows a simple way to run the tests. :func:`unittest.main`
130 provides a command-line interface to the test script. When run from the command
131 line, the above script produces an output that looks like this::
132
133 ...
134 ----------------------------------------------------------------------
135 Ran 3 tests in 0.000s
136
137 OK
138
139 Passing the ``-v`` option to your test script will instruct :func:`unittest.main`
140 to enable a higher level of verbosity, and produce the following output::
141
142 test_isupper (__main__.TestStringMethods.test_isupper) ... ok
143 test_split (__main__.TestStringMethods.test_split) ... ok
144 test_upper (__main__.TestStringMethods.test_upper) ... ok
145
146 ----------------------------------------------------------------------
147 Ran 3 tests in 0.001s
148
149 OK
150
151 The above examples show the most commonly used :mod:`unittest` features which
152 are sufficient to meet many everyday testing needs. The remainder of the
153 documentation explores the full feature set from first principles.
154
155 .. versionchanged:: 3.11
156 The behavior of returning a value from a test method (other than the default
157 ``None`` value), is now deprecated.
158
159
160 .. _unittest-command-line-interface:
161
162 Command-Line Interface
163 ----------------------
164
165 The unittest module can be used from the command line to run tests from
166 modules, classes or even individual test methods::
167
168 python -m unittest test_module1 test_module2
169 python -m unittest test_module.TestClass
170 python -m unittest test_module.TestClass.test_method
171
172 You can pass in a list with any combination of module names, and fully
173 qualified class or method names.
174
175 Test modules can be specified by file path as well::
176
177 python -m unittest tests/test_something.py
178
179 This allows you to use the shell filename completion to specify the test module.
180 The file specified must still be importable as a module. The path is converted
181 to a module name by removing the '.py' and converting path separators into '.'.
182 If you want to execute a test file that isn't importable as a module you should
183 execute the file directly instead.
184
185 You can run tests with more detail (higher verbosity) by passing in the -v flag::
186
187 python -m unittest -v test_module
188
189 When executed without arguments :ref:`unittest-test-discovery` is started::
190
191 python -m unittest
192
193 For a list of all the command-line options::
194
195 python -m unittest -h
196
197 .. versionchanged:: 3.2
198 In earlier versions it was only possible to run individual test methods and
199 not modules or classes.
200
201
202 Command-line options
203 ~~~~~~~~~~~~~~~~~~~~
204
205 :program:`unittest` supports these command-line options:
206
207 .. program:: unittest
208
209 .. option:: -b, --buffer
210
211 The standard output and standard error streams are buffered during the test
212 run. Output during a passing test is discarded. Output is echoed normally
213 on test fail or error and is added to the failure messages.
214
215 .. option:: -c, --catch
216
217 :kbd:`Control-C` during the test run waits for the current test to end and then
218 reports all the results so far. A second :kbd:`Control-C` raises the normal
219 :exc:`KeyboardInterrupt` exception.
220
221 See `Signal Handling`_ for the functions that provide this functionality.
222
223 .. option:: -f, --failfast
224
225 Stop the test run on the first error or failure.
226
227 .. option:: -k
228
229 Only run test methods and classes that match the pattern or substring.
230 This option may be used multiple times, in which case all test cases that
231 match any of the given patterns are included.
232
233 Patterns that contain a wildcard character (``*``) are matched against the
234 test name using :meth:`fnmatch.fnmatchcase`; otherwise simple case-sensitive
235 substring matching is used.
236
237 Patterns are matched against the fully qualified test method name as
238 imported by the test loader.
239
240 For example, ``-k foo`` matches ``foo_tests.SomeTest.test_something``,
241 ``bar_tests.SomeTest.test_foo``, but not ``bar_tests.FooTest.test_something``.
242
243 .. option:: --locals
244
245 Show local variables in tracebacks.
246
247 .. option:: --durations N
248
249 Show the N slowest test cases (N=0 for all).
250
251 .. versionadded:: 3.2
252 The command-line options ``-b``, ``-c`` and ``-f`` were added.
253
254 .. versionadded:: 3.5
255 The command-line option ``--locals``.
256
257 .. versionadded:: 3.7
258 The command-line option ``-k``.
259
260 .. versionadded:: 3.12
261 The command-line option ``--durations``.
262
263 The command line can also be used for test discovery, for running all of the
264 tests in a project or just a subset.
265
266 .. _unittest-test-discovery:
267
268 Test Discovery
269 --------------
270
271 .. versionadded:: 3.2
272
273 Unittest supports simple test discovery. In order to be compatible with test
274 discovery, all of the test files must be :ref:`modules <tut-modules>` or
275 :ref:`packages <tut-packages>` importable from the top-level directory of
276 the project (this means that their filenames must be valid :ref:`identifiers
277 <identifiers>`).
278
279 Test discovery is implemented in :meth:`TestLoader.discover`, but can also be
280 used from the command line. The basic command-line usage is::
281
282 cd project_directory
283 python -m unittest discover
284
285 .. note::
286
287 As a shortcut, ``python -m unittest`` is the equivalent of
288 ``python -m unittest discover``. If you want to pass arguments to test
289 discovery the ``discover`` sub-command must be used explicitly.
290
291 The ``discover`` sub-command has the following options:
292
293 .. program:: unittest discover
294
295 .. option:: -v, --verbose
296
297 Verbose output
298
299 .. option:: -s, --start-directory directory
300
301 Directory to start discovery (``.`` default)
302
303 .. option:: -p, --pattern pattern
304
305 Pattern to match test files (``test*.py`` default)
306
307 .. option:: -t, --top-level-directory directory
308
309 Top level directory of project (defaults to start directory)
310
311 The :option:`-s`, :option:`-p`, and :option:`-t` options can be passed in
312 as positional arguments in that order. The following two command lines
313 are equivalent::
314
315 python -m unittest discover -s project_directory -p "*_test.py"
316 python -m unittest discover project_directory "*_test.py"
317
318 As well as being a path it is possible to pass a package name, for example
319 ``myproject.subpackage.test``, as the start directory. The package name you
320 supply will then be imported and its location on the filesystem will be used
321 as the start directory.
322
323 .. caution::
324
325 Test discovery loads tests by importing them. Once test discovery has found
326 all the test files from the start directory you specify it turns the paths
327 into package names to import. For example :file:`foo/bar/baz.py` will be
328 imported as ``foo.bar.baz``.
329
330 If you have a package installed globally and attempt test discovery on
331 a different copy of the package then the import *could* happen from the
332 wrong place. If this happens test discovery will warn you and exit.
333
334 If you supply the start directory as a package name rather than a
335 path to a directory then discover assumes that whichever location it
336 imports from is the location you intended, so you will not get the
337 warning.
338
339 Test modules and packages can customize test loading and discovery by through
340 the `load_tests protocol`_.
341
342 .. versionchanged:: 3.4
343 Test discovery supports :term:`namespace packages <namespace package>`
344 for the start directory. Note that you need to specify the top level
345 directory too (e.g.
346 ``python -m unittest discover -s root/namespace -t root``).
347
348 .. versionchanged:: 3.11
349 :mod:`unittest` dropped the :term:`namespace packages <namespace package>`
350 support in Python 3.11. It has been broken since Python 3.7. Start directory and
351 subdirectories containing tests must be regular package that have
352 ``__init__.py`` file.
353
354 Directories containing start directory still can be a namespace package.
355 In this case, you need to specify start directory as dotted package name,
356 and target directory explicitly. For example::
357
358 # proj/ <-- current directory
359 # namespace/
360 # mypkg/
361 # __init__.py
362 # test_mypkg.py
363
364 python -m unittest discover -s namespace.mypkg -t .
365
366
367 .. _organizing-tests:
368
369 Organizing test code
370 --------------------
371
372 The basic building blocks of unit testing are :dfn:`test cases` --- single
373 scenarios that must be set up and checked for correctness. In :mod:`unittest`,
374 test cases are represented by :class:`unittest.TestCase` instances.
375 To make your own test cases you must write subclasses of
376 :class:`TestCase` or use :class:`FunctionTestCase`.
377
378 The testing code of a :class:`TestCase` instance should be entirely self
379 contained, such that it can be run either in isolation or in arbitrary
380 combination with any number of other test cases.
381
382 The simplest :class:`TestCase` subclass will simply implement a test method
383 (i.e. a method whose name starts with ``test``) in order to perform specific
384 testing code::
385
386 import unittest
387
388 class DefaultWidgetSizeTestCase(unittest.TestCase):
389 def test_default_widget_size(self):
390 widget = Widget('The widget')
391 self.assertEqual(widget.size(), (50, 50))
392
393 Note that in order to test something, we use one of the :ref:`assert\* methods <assert-methods>`
394 provided by the :class:`TestCase` base class. If the test fails, an
395 exception will be raised with an explanatory message, and :mod:`unittest`
396 will identify the test case as a :dfn:`failure`. Any other exceptions will be
397 treated as :dfn:`errors`.
398
399 Tests can be numerous, and their set-up can be repetitive. Luckily, we
400 can factor out set-up code by implementing a method called
401 :meth:`~TestCase.setUp`, which the testing framework will automatically
402 call for every single test we run::
403
404 import unittest
405
406 class WidgetTestCase(unittest.TestCase):
407 def setUp(self):
408 self.widget = Widget('The widget')
409
410 def test_default_widget_size(self):
411 self.assertEqual(self.widget.size(), (50,50),
412 'incorrect default size')
413
414 def test_widget_resize(self):
415 self.widget.resize(100,150)
416 self.assertEqual(self.widget.size(), (100,150),
417 'wrong size after resize')
418
419 .. note::
420 The order in which the various tests will be run is determined
421 by sorting the test method names with respect to the built-in
422 ordering for strings.
423
424 If the :meth:`~TestCase.setUp` method raises an exception while the test is
425 running, the framework will consider the test to have suffered an error, and
426 the test method will not be executed.
427
428 Similarly, we can provide a :meth:`~TestCase.tearDown` method that tidies up
429 after the test method has been run::
430
431 import unittest
432
433 class WidgetTestCase(unittest.TestCase):
434 def setUp(self):
435 self.widget = Widget('The widget')
436
437 def tearDown(self):
438 self.widget.dispose()
439
440 If :meth:`~TestCase.setUp` succeeded, :meth:`~TestCase.tearDown` will be
441 run whether the test method succeeded or not.
442
443 Such a working environment for the testing code is called a
444 :dfn:`test fixture`. A new TestCase instance is created as a unique
445 test fixture used to execute each individual test method. Thus
446 :meth:`~TestCase.setUp`, :meth:`~TestCase.tearDown`, and :meth:`!TestCase.__init__`
447 will be called once per test.
448
449 It is recommended that you use TestCase implementations to group tests together
450 according to the features they test. :mod:`unittest` provides a mechanism for
451 this: the :dfn:`test suite`, represented by :mod:`unittest`'s
452 :class:`TestSuite` class. In most cases, calling :func:`unittest.main` will do
453 the right thing and collect all the module's test cases for you and execute
454 them.
455
456 However, should you want to customize the building of your test suite,
457 you can do it yourself::
458
459 def suite():
460 suite = unittest.TestSuite()
461 suite.addTest(WidgetTestCase('test_default_widget_size'))
462 suite.addTest(WidgetTestCase('test_widget_resize'))
463 return suite
464
465 if __name__ == '__main__':
466 runner = unittest.TextTestRunner()
467 runner.run(suite())
468
469 You can place the definitions of test cases and test suites in the same modules
470 as the code they are to test (such as :file:`widget.py`), but there are several
471 advantages to placing the test code in a separate module, such as
472 :file:`test_widget.py`:
473
474 * The test module can be run standalone from the command line.
475
476 * The test code can more easily be separated from shipped code.
477
478 * There is less temptation to change test code to fit the code it tests without
479 a good reason.
480
481 * Test code should be modified much less frequently than the code it tests.
482
483 * Tested code can be refactored more easily.
484
485 * Tests for modules written in C must be in separate modules anyway, so why not
486 be consistent?
487
488 * If the testing strategy changes, there is no need to change the source code.
489
490
491 .. _legacy-unit-tests:
492
493 Re-using old test code
494 ----------------------
495
496 Some users will find that they have existing test code that they would like to
497 run from :mod:`unittest`, without converting every old test function to a
498 :class:`TestCase` subclass.
499
500 For this reason, :mod:`unittest` provides a :class:`FunctionTestCase` class.
501 This subclass of :class:`TestCase` can be used to wrap an existing test
502 function. Set-up and tear-down functions can also be provided.
503
504 Given the following test function::
505
506 def testSomething():
507 something = makeSomething()
508 assert something.name is not None
509 # ...
510
511 one can create an equivalent test case instance as follows, with optional
512 set-up and tear-down methods::
513
514 testcase = unittest.FunctionTestCase(testSomething,
515 setUp=makeSomethingDB,
516 tearDown=deleteSomethingDB)
517
518 .. note::
519
520 Even though :class:`FunctionTestCase` can be used to quickly convert an
521 existing test base over to a :mod:`unittest`\ -based system, this approach is
522 not recommended. Taking the time to set up proper :class:`TestCase`
523 subclasses will make future test refactorings infinitely easier.
524
525 In some cases, the existing tests may have been written using the :mod:`doctest`
526 module. If so, :mod:`doctest` provides a :class:`~doctest.DocTestSuite` class that can
527 automatically build :class:`unittest.TestSuite` instances from the existing
528 :mod:`doctest`\ -based tests.
529
530
531 .. _unittest-skipping:
532
533 Skipping tests and expected failures
534 ------------------------------------
535
536 .. versionadded:: 3.1
537
538 Unittest supports skipping individual test methods and even whole classes of
539 tests. In addition, it supports marking a test as an "expected failure," a test
540 that is broken and will fail, but shouldn't be counted as a failure on a
541 :class:`TestResult`.
542
543 Skipping a test is simply a matter of using the :deco:`skip` :term:`decorator`
544 or one of its conditional variants, calling :meth:`TestCase.skipTest` within a
545 :meth:`~TestCase.setUp` or test method, or raising :exc:`SkipTest` directly.
546
547 Basic skipping looks like this::
548
549 class MyTestCase(unittest.TestCase):
550
551 @unittest.skip("demonstrating skipping")
552 def test_nothing(self):
553 self.fail("shouldn't happen")
554
555 @unittest.skipIf(mylib.__version__ < (1, 3),
556 "not supported in this library version")
557 def test_format(self):
558 # Tests that work for only a certain version of the library.
559 pass
560
561 @unittest.skipUnless(sys.platform.startswith("win"), "requires Windows")
562 def test_windows_support(self):
563 # windows specific testing code
564 pass
565
566 def test_maybe_skipped(self):
567 if not external_resource_available():
568 self.skipTest("external resource not available")
569 # test code that depends on the external resource
570 pass
571
572 This is the output of running the example above in verbose mode::
573
574 test_format (__main__.MyTestCase.test_format) ... skipped 'not supported in this library version'
575 test_nothing (__main__.MyTestCase.test_nothing) ... skipped 'demonstrating skipping'
576 test_maybe_skipped (__main__.MyTestCase.test_maybe_skipped) ... skipped 'external resource not available'
577 test_windows_support (__main__.MyTestCase.test_windows_support) ... skipped 'requires Windows'
578
579 ----------------------------------------------------------------------
580 Ran 4 tests in 0.005s
581
582 OK (skipped=4)
583
584 Classes can be skipped just like methods::
585
586 @unittest.skip("showing class skipping")
587 class MySkippedTestCase(unittest.TestCase):
588 def test_not_run(self):
589 pass
590
591 :meth:`TestCase.setUp` can also skip the test. This is useful when a resource
592 that needs to be set up is not available.
593
594 Expected failures use the :deco:`expectedFailure` decorator. ::
595
596 class ExpectedFailureTestCase(unittest.TestCase):
597 @unittest.expectedFailure
598 def test_fail(self):
599 self.assertEqual(1, 0, "broken")
600
601 It's easy to roll your own skipping decorators by making a decorator that calls
602 :func:`skip` on the test when it wants it to be skipped. This decorator skips
603 the test unless the passed object has a certain attribute::
604
605 def skipUnlessHasattr(obj, attr):
606 if hasattr(obj, attr):
607 return lambda func: func
608 return unittest.skip("{!r} doesn't have {!r}".format(obj, attr))
609
610 The following decorators and exception implement test skipping and expected failures:
611
612 .. decorator:: skip(reason)
613
614 Unconditionally skip the decorated test. *reason* should describe why the
615 test is being skipped.
616
617 .. decorator:: skipIf(condition, reason)
618
619 Skip the decorated test if *condition* is true.
620
621 .. decorator:: skipUnless(condition, reason)
622
623 Skip the decorated test unless *condition* is true.
624
625 .. decorator:: expectedFailure
626
627 Mark the test as an expected failure or error. If the test fails or errors
628 in the test function itself (rather than in one of the :dfn:`test fixture`
629 methods) then it will be considered a success. If the test passes, it will
630 be considered a failure.
631
632 .. exception:: SkipTest(reason)
633
634 This exception is raised to skip a test.
635
636 Usually you can use :meth:`TestCase.skipTest` or one of the skipping
637 decorators instead of raising this directly.
638
639 Skipped tests will not have :meth:`~TestCase.setUp` or :meth:`~TestCase.tearDown` run around them.
640 Skipped classes will not have :meth:`~TestCase.setUpClass` or :meth:`~TestCase.tearDownClass` run.
641 Skipped modules will not have :func:`setUpModule` or :func:`tearDownModule` run.
642
643
644 .. _subtests:
645
646 Distinguishing test iterations using subtests
647 ---------------------------------------------
648
649 .. versionadded:: 3.4
650
651 When there are very small differences among your tests, for
652 instance some parameters, unittest allows you to distinguish them inside
653 the body of a test method using the :meth:`~TestCase.subTest` context manager.
654
655 For example, the following test::
656
657 class NumbersTest(unittest.TestCase):
658
659 def test_even(self):
660 """
661 Test that numbers between 0 and 5 are all even.
662 """
663 for i in range(0, 6):
664 with self.subTest(i=i):
665 self.assertEqual(i % 2, 0)
666
667 will produce the following output::
668
669 ======================================================================
670 FAIL: test_even (__main__.NumbersTest.test_even) (i=1)
671 Test that numbers between 0 and 5 are all even.
672 ----------------------------------------------------------------------
673 Traceback (most recent call last):
674 File "subtests.py", line 11, in test_even
675 self.assertEqual(i % 2, 0)
676 ^^^^^^^^^^^^^^^^^^^^^^^^^^
677 AssertionError: 1 != 0
678
679 ======================================================================
680 FAIL: test_even (__main__.NumbersTest.test_even) (i=3)
681 Test that numbers between 0 and 5 are all even.
682 ----------------------------------------------------------------------
683 Traceback (most recent call last):
684 File "subtests.py", line 11, in test_even
685 self.assertEqual(i % 2, 0)
686 ^^^^^^^^^^^^^^^^^^^^^^^^^^
687 AssertionError: 1 != 0
688
689 ======================================================================
690 FAIL: test_even (__main__.NumbersTest.test_even) (i=5)
691 Test that numbers between 0 and 5 are all even.
692 ----------------------------------------------------------------------
693 Traceback (most recent call last):
694 File "subtests.py", line 11, in test_even
695 self.assertEqual(i % 2, 0)
696 ^^^^^^^^^^^^^^^^^^^^^^^^^^
697 AssertionError: 1 != 0
698
699 Without using a subtest, execution would stop after the first failure,
700 and the error would be less easy to diagnose because the value of ``i``
701 wouldn't be displayed::
702
703 ======================================================================
704 FAIL: test_even (__main__.NumbersTest.test_even)
705 ----------------------------------------------------------------------
706 Traceback (most recent call last):
707 File "subtests.py", line 32, in test_even
708 self.assertEqual(i % 2, 0)
709 AssertionError: 1 != 0
710
711
712 .. _unittest-contents:
713
714 Classes and functions
715 ---------------------
716
717 This section describes in depth the API of :mod:`unittest`.
718
719
720 .. _testcase-objects:
721
722 Test cases
723 ~~~~~~~~~~
724
725 .. class:: TestCase(methodName='runTest')
726
727 Instances of the :class:`TestCase` class represent the logical test units
728 in the :mod:`unittest` universe. This class is intended to be used as a base
729 class, with specific tests being implemented by concrete subclasses. This class
730 implements the interface needed by the test runner to allow it to drive the
731 tests, and methods that the test code can use to check for and report various
732 kinds of failure.
733
734 Each instance of :class:`TestCase` will run a single base method: the method
735 named *methodName*.
736 In most uses of :class:`TestCase`, you will neither change
737 the *methodName* nor reimplement the default ``runTest()`` method.
738
739 .. versionchanged:: 3.2
740 :class:`TestCase` can be instantiated successfully without providing a
741 *methodName*. This makes it easier to experiment with :class:`TestCase`
742 from the interactive interpreter.
743
744 :class:`TestCase` instances provide three groups of methods: one group used
745 to run the test, another used by the test implementation to check conditions
746 and report failures, and some inquiry methods allowing information about the
747 test itself to be gathered.
748
749 Methods in the first group (running the test) are:
750
751 .. method:: setUp()
752
753 Method called to prepare the test fixture. This is called immediately
754 before calling the test method; other than :exc:`AssertionError` or :exc:`SkipTest`,
755 any exception raised by this method will be considered an error rather than
756 a test failure. The default implementation does nothing.
757
758
759 .. method:: tearDown()
760
761 Method called immediately after the test method has been called and the
762 result recorded. This is called even if the test method raised an
763 exception, so the implementation in subclasses may need to be particularly
764 careful about checking internal state. Any exception, other than
765 :exc:`AssertionError` or :exc:`SkipTest`, raised by this method will be
766 considered an additional error rather than a test failure (thus increasing
767 the total number of reported errors). This method will only be called if
768 the :meth:`setUp` succeeds, regardless of the outcome of the test method.
769 The default implementation does nothing.
770
771
772 .. method:: setUpClass()
773
774 A class method called before tests in an individual class are run.
775 ``setUpClass`` is called with the class as the only argument
776 and must be decorated as a :deco:`classmethod`::
777
778 @classmethod
779 def setUpClass(cls):
780 ...
781
782 See `Class and Module Fixtures`_ for more details.
783
784 .. versionadded:: 3.2
785
786
787 .. method:: tearDownClass()
788
789 A class method called after tests in an individual class have run.
790 ``tearDownClass`` is called with the class as the only argument
791 and must be decorated as a :deco:`classmethod`::
792
793 @classmethod
794 def tearDownClass(cls):
795 ...
796
797 See `Class and Module Fixtures`_ for more details.
798
799 .. versionadded:: 3.2
800
801
802 .. method:: run(result=None)
803
804 Run the test, collecting the result into the :class:`TestResult` object
805 passed as *result*. If *result* is omitted or ``None``, a temporary
806 result object is created (by calling the :meth:`defaultTestResult`
807 method) and used. The result object is returned to :meth:`run`'s
808 caller.
809
810 The same effect may be had by simply calling the :class:`TestCase`
811 instance.
812
813 .. versionchanged:: 3.3
814 Previous versions of ``run`` did not return the result. Neither did
815 calling an instance.
816
817 .. method:: skipTest(reason)
818
819 Calling this during a test method or :meth:`setUp` skips the current
820 test. See :ref:`unittest-skipping` for more information.
821
822 .. versionadded:: 3.1
823
824
825 .. method:: subTest(msg=None, **params)
826
827 Return a context manager which executes the enclosed code block as a
828 subtest. *msg* and *params* are optional, arbitrary values which are
829 displayed whenever a subtest fails, allowing you to identify them
830 clearly.
831
832 A test case can contain any number of subtest declarations, and
833 they can be arbitrarily nested.
834
835 See :ref:`subtests` for more information.
836
837 .. versionadded:: 3.4
838
839
840 .. method:: debug()
841
842 Run the test without collecting the result. This allows exceptions raised
843 by the test to be propagated to the caller, and can be used to support
844 running tests under a debugger.
845
846 .. _assert-methods:
847
848 The :class:`TestCase` class provides several assert methods to check for and
849 report failures. The following table lists the most commonly used methods
850 (see the tables below for more assert methods):
851
852 +-----------------------------------------+-----------------------------+---------------+
853 | Method | Checks that | New in |
854 +=========================================+=============================+===============+
855 | :meth:`assertEqual(a, b) | ``a == b`` | |
856 | <TestCase.assertEqual>` | | |
857 +-----------------------------------------+-----------------------------+---------------+
858 | :meth:`assertNotEqual(a, b) | ``a != b`` | |
859 | <TestCase.assertNotEqual>` | | |
860 +-----------------------------------------+-----------------------------+---------------+
861 | :meth:`assertTrue(x) | ``bool(x) is True`` | |
862 | <TestCase.assertTrue>` | | |
863 +-----------------------------------------+-----------------------------+---------------+
864 | :meth:`assertFalse(x) | ``bool(x) is False`` | |
865 | <TestCase.assertFalse>` | | |
866 +-----------------------------------------+-----------------------------+---------------+
867 | :meth:`assertIs(a, b) | ``a is b`` | 3.1 |
868 | <TestCase.assertIs>` | | |
869 +-----------------------------------------+-----------------------------+---------------+
870 | :meth:`assertIsNot(a, b) | ``a is not b`` | 3.1 |
871 | <TestCase.assertIsNot>` | | |
872 +-----------------------------------------+-----------------------------+---------------+
873 | :meth:`assertIsNone(x) | ``x is None`` | 3.1 |
874 | <TestCase.assertIsNone>` | | |
875 +-----------------------------------------+-----------------------------+---------------+
876 | :meth:`assertIsNotNone(x) | ``x is not None`` | 3.1 |
877 | <TestCase.assertIsNotNone>` | | |
878 +-----------------------------------------+-----------------------------+---------------+
879 | :meth:`assertIn(a, b) | ``a in b`` | 3.1 |
880 | <TestCase.assertIn>` | | |
881 +-----------------------------------------+-----------------------------+---------------+
882 | :meth:`assertNotIn(a, b) | ``a not in b`` | 3.1 |
883 | <TestCase.assertNotIn>` | | |
884 +-----------------------------------------+-----------------------------+---------------+
885 | :meth:`assertIsInstance(a, b) | ``isinstance(a, b)`` | 3.2 |
886 | <TestCase.assertIsInstance>` | | |
887 +-----------------------------------------+-----------------------------+---------------+
888 | :meth:`assertNotIsInstance(a, b) | ``not isinstance(a, b)`` | 3.2 |
889 | <TestCase.assertNotIsInstance>` | | |
890 +-----------------------------------------+-----------------------------+---------------+
891
892 All the assert methods accept a *msg* argument that, if specified, is used
893 as the error message on failure (see also :data:`longMessage`).
894 Note that the *msg* keyword argument can be passed to :meth:`assertRaises`,
895 :meth:`assertRaisesRegex`, :meth:`assertWarns`, :meth:`assertWarnsRegex`
896 only when they are used as a context manager.
897
898 .. method:: assertEqual(first, second, msg=None)
899
900 Test that *first* and *second* are equal. If the values do not
901 compare equal, the test will fail.
902
903 In addition, if *first* and *second* are the exact same type and one of
904 list, tuple, dict, set, frozenset or str or any type that a subclass
905 registers with :meth:`addTypeEqualityFunc` the type-specific equality
906 function will be called in order to generate a more useful default
907 error message (see also the :ref:`list of type-specific methods
908 <type-specific-methods>`).
909
910 .. versionchanged:: 3.1
911 Added the automatic calling of type-specific equality function.
912
913 .. versionchanged:: 3.2
914 :meth:`assertMultiLineEqual` added as the default type equality
915 function for comparing strings.
916
917
918 .. method:: assertNotEqual(first, second, msg=None)
919
920 Test that *first* and *second* are not equal. If the values do
921 compare equal, the test will fail.
922
923 .. method:: assertTrue(expr, msg=None)
924 assertFalse(expr, msg=None)
925
926 Test that *expr* is true (or false).
927
928 Note that this is equivalent to ``bool(expr) is True`` and not to ``expr
929 is True`` (use ``assertIs(expr, True)`` for the latter). This method
930 should also be avoided when more specific methods are available (e.g.
931 ``assertEqual(a, b)`` instead of ``assertTrue(a == b)``), because they
932 provide a better error message in case of failure.
933
934
935 .. method:: assertIs(first, second, msg=None)
936 assertIsNot(first, second, msg=None)
937
938 Test that *first* and *second* are (or are not) the same object.
939
940 .. versionadded:: 3.1
941
942
943 .. method:: assertIsNone(expr, msg=None)
944 assertIsNotNone(expr, msg=None)
945
946 Test that *expr* is (or is not) ``None``.
947
948 .. versionadded:: 3.1
949
950
951 .. method:: assertIn(member, container, msg=None)
952 assertNotIn(member, container, msg=None)
953
954 Test that *member* is (or is not) in *container*.
955
956 .. versionadded:: 3.1
957
958
959 .. method:: assertIsInstance(obj, cls, msg=None)
960 assertNotIsInstance(obj, cls, msg=None)
961
962 Test that *obj* is (or is not) an instance of *cls* (which can be a
963 class or a tuple of classes, as supported by :func:`isinstance`).
964 To check for the exact type, use :func:`assertIs(type(obj), cls) <assertIs>`.
965
966 .. versionadded:: 3.2
967
968
969
970 It is also possible to check the production of exceptions, warnings, and
971 log messages using the following methods:
972
973 +---------------------------------------------------------+--------------------------------------+------------+
974 | Method | Checks that | New in |
975 +=========================================================+======================================+============+
976 | :meth:`assertRaises(exc, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises *exc* | |
977 | <TestCase.assertRaises>` | | |
978 +---------------------------------------------------------+--------------------------------------+------------+
979 | :meth:`assertRaisesRegex(exc, r, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises *exc* | 3.1 |
980 | <TestCase.assertRaisesRegex>` | and the message matches regex *r* | |
981 +---------------------------------------------------------+--------------------------------------+------------+
982 | :meth:`assertWarns(warn, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises *warn* | 3.2 |
983 | <TestCase.assertWarns>` | | |
984 +---------------------------------------------------------+--------------------------------------+------------+
985 | :meth:`assertWarnsRegex(warn, r, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises *warn* | 3.2 |
986 | <TestCase.assertWarnsRegex>` | and the message matches regex *r* | |
987 +---------------------------------------------------------+--------------------------------------+------------+
988 | :meth:`assertLogs(logger, level) | The ``with`` block logs on *logger* | 3.4 |
989 | <TestCase.assertLogs>` | with minimum *level* | |
990 +---------------------------------------------------------+--------------------------------------+------------+
991 | :meth:`assertNoLogs(logger, level) | The ``with`` block does not log on | 3.10 |
992 | <TestCase.assertNoLogs>` | *logger* with minimum *level* | |
993 +---------------------------------------------------------+--------------------------------------+------------+
994
995 .. method:: assertRaises(exception, callable, *args, **kwds)
996 assertRaises(exception, *, msg=None)
997
998 Test that an exception is raised when *callable* is called with any
999 positional or keyword arguments that are also passed to
1000 :meth:`assertRaises`. The test passes if *exception* is raised, is an
1001 error if another exception is raised, or fails if no exception is raised.
1002 To catch any of a group of exceptions, a tuple containing the exception
1003 classes may be passed as *exception*.
1004
1005 If only the *exception* and possibly the *msg* arguments are given,
1006 return a context manager so that the code under test can be written
1007 inline rather than as a function::
1008
1009 with self.assertRaises(SomeException):
1010 do_something()
1011
1012 When used as a context manager, :meth:`assertRaises` accepts the
1013 additional keyword argument *msg*.
1014
1015 The context manager will store the caught exception object in its
1016 :attr:`!exception` attribute. This can be useful if the intention
1017 is to perform additional checks on the exception raised::
1018
1019 with self.assertRaises(SomeException) as cm:
1020 do_something()
1021
1022 the_exception = cm.exception
1023 self.assertEqual(the_exception.error_code, 3)
1024
1025 .. versionchanged:: 3.1
1026 Added the ability to use :meth:`assertRaises` as a context manager.
1027
1028 .. versionchanged:: 3.2
1029 Added the :attr:`!exception` attribute.
1030
1031 .. versionchanged:: 3.3
1032 Added the *msg* keyword argument when used as a context manager.
1033
1034
1035 .. method:: assertRaisesRegex(exception, regex, callable, *args, **kwds)
1036 assertRaisesRegex(exception, regex, *, msg=None)
1037
1038 Like :meth:`assertRaises` but also tests that *regex* matches
1039 on the string representation of the raised exception. *regex* may be
1040 a regular expression object or a string containing a regular expression
1041 suitable for use by :func:`re.search`. Examples::
1042
1043 self.assertRaisesRegex(ValueError, "invalid literal for.*XYZ'$",
1044 int, 'XYZ')
1045
1046 or::
1047
1048 with self.assertRaisesRegex(ValueError, 'literal'):
1049 int('XYZ')
1050
1051 .. versionadded:: 3.1
1052 Added under the name ``assertRaisesRegexp``.
1053
1054 .. versionchanged:: 3.2
1055 Renamed to :meth:`assertRaisesRegex`.
1056
1057 .. versionchanged:: 3.3
1058 Added the *msg* keyword argument when used as a context manager.
1059
1060
1061 .. method:: assertWarns(warning, callable, *args, **kwds)
1062 assertWarns(warning, *, msg=None)
1063
1064 Test that a warning is triggered when *callable* is called with any
1065 positional or keyword arguments that are also passed to
1066 :meth:`assertWarns`. The test passes if *warning* is triggered and
1067 fails if it isn't. Any exception is an error.
1068 To catch any of a group of warnings, a tuple containing the warning
1069 classes may be passed as *warnings*.
1070
1071 If only the *warning* and possibly the *msg* arguments are given,
1072 return a context manager so that the code under test can be written
1073 inline rather than as a function::
1074
1075 with self.assertWarns(SomeWarning):
1076 do_something()
1077
1078 When used as a context manager, :meth:`assertWarns` accepts the
1079 additional keyword argument *msg*.
1080
1081 The context manager will store the caught warning object in its
1082 :attr:`!warning` attribute, and the source line which triggered the
1083 warnings in the :attr:`!filename` and :attr:`!lineno` attributes.
1084 This can be useful if the intention is to perform additional checks
1085 on the warning caught::
1086
1087 with self.assertWarns(SomeWarning) as cm:
1088 do_something()
1089
1090 self.assertIn('myfile.py', cm.filename)
1091 self.assertEqual(320, cm.lineno)
1092
1093 This method works regardless of the warning filters in place when it
1094 is called.
1095
1096 .. versionadded:: 3.2
1097
1098 .. versionchanged:: 3.3
1099 Added the *msg* keyword argument when used as a context manager.
1100
1101
1102 .. method:: assertWarnsRegex(warning, regex, callable, *args, **kwds)
1103 assertWarnsRegex(warning, regex, *, msg=None)
1104
1105 Like :meth:`assertWarns` but also tests that *regex* matches on the
1106 message of the triggered warning. *regex* may be a regular expression
1107 object or a string containing a regular expression suitable for use
1108 by :func:`re.search`. Example::
1109
1110 self.assertWarnsRegex(DeprecationWarning,
1111 r'legacy_function\(\) is deprecated',
1112 legacy_function, 'XYZ')
1113
1114 or::
1115
1116 with self.assertWarnsRegex(RuntimeWarning, 'unsafe frobnicating'):
1117 frobnicate('/etc/passwd')
1118
1119 .. versionadded:: 3.2
1120
1121 .. versionchanged:: 3.3
1122 Added the *msg* keyword argument when used as a context manager.
1123
1124 .. method:: assertLogs(logger=None, level=None)
1125
1126 A context manager to test that at least one message is logged on
1127 the *logger* or one of its children, with at least the given
1128 *level*.
1129
1130 If given, *logger* should be a :class:`logging.Logger` object or a
1131 :class:`str` giving the name of a logger. The default is the root
1132 logger, which will catch all messages that were not blocked by a
1133 non-propagating descendent logger.
1134
1135 If given, *level* should be either a numeric logging level or
1136 its string equivalent (for example either ``"ERROR"`` or
1137 :const:`logging.ERROR`). The default is :const:`logging.INFO`.
1138
1139 The test passes if at least one message emitted inside the ``with``
1140 block matches the *logger* and *level* conditions, otherwise it fails.
1141
1142 The object returned by the context manager is a recording helper
1143 which keeps tracks of the matching log messages. It has two
1144 attributes:
1145
1146 .. attribute:: records
1147
1148 A list of :class:`logging.LogRecord` objects of the matching
1149 log messages.
1150
1151 .. attribute:: output
1152
1153 A list of :class:`str` objects with the formatted output of
1154 matching messages.
1155
1156 Example::
1157
1158 with self.assertLogs('foo', level='INFO') as cm:
1159 logging.getLogger('foo').info('first message')
1160 logging.getLogger('foo.bar').error('second message')
1161 self.assertEqual(cm.output, ['INFO:foo:first message',
1162 'ERROR:foo.bar:second message'])
1163
1164 .. versionadded:: 3.4
1165
1166 .. method:: assertNoLogs(logger=None, level=None)
1167
1168 A context manager to test that no messages are logged on
1169 the *logger* or one of its children, with at least the given
1170 *level*.
1171
1172 If given, *logger* should be a :class:`logging.Logger` object or a
1173 :class:`str` giving the name of a logger. The default is the root
1174 logger, which will catch all messages.
1175
1176 If given, *level* should be either a numeric logging level or
1177 its string equivalent (for example either ``"ERROR"`` or
1178 :const:`logging.ERROR`). The default is :const:`logging.INFO`.
1179
1180 Unlike :meth:`assertLogs`, nothing will be returned by the context
1181 manager.
1182
1183 .. versionadded:: 3.10
1184
1185 There are also other methods used to perform more specific checks, such as:
1186
1187 +---------------------------------------+--------------------------------+--------------+
1188 | Method | Checks that | New in |
1189 +=======================================+================================+==============+
1190 | :meth:`assertAlmostEqual(a, b) | ``round(a-b, 7) == 0`` | |
1191 | <TestCase.assertAlmostEqual>` | | |
1192 +---------------------------------------+--------------------------------+--------------+
1193 | :meth:`assertNotAlmostEqual(a, b) | ``round(a-b, 7) != 0`` | |
1194 | <TestCase.assertNotAlmostEqual>` | | |
1195 +---------------------------------------+--------------------------------+--------------+
1196 | :meth:`assertGreater(a, b) | ``a > b`` | 3.1 |
1197 | <TestCase.assertGreater>` | | |
1198 +---------------------------------------+--------------------------------+--------------+
1199 | :meth:`assertGreaterEqual(a, b) | ``a >= b`` | 3.1 |
1200 | <TestCase.assertGreaterEqual>` | | |
1201 +---------------------------------------+--------------------------------+--------------+
1202 | :meth:`assertLess(a, b) | ``a < b`` | 3.1 |
1203 | <TestCase.assertLess>` | | |
1204 +---------------------------------------+--------------------------------+--------------+
1205 | :meth:`assertLessEqual(a, b) | ``a <= b`` | 3.1 |
1206 | <TestCase.assertLessEqual>` | | |
1207 +---------------------------------------+--------------------------------+--------------+
1208 | :meth:`assertRegex(s, r) | ``r.search(s)`` | 3.1 |
1209 | <TestCase.assertRegex>` | | |
1210 +---------------------------------------+--------------------------------+--------------+
1211 | :meth:`assertNotRegex(s, r) | ``not r.search(s)`` | 3.2 |
1212 | <TestCase.assertNotRegex>` | | |
1213 +---------------------------------------+--------------------------------+--------------+
1214 | :meth:`assertCountEqual(a, b) | *a* contains the same elements | 3.2 |
1215 | <TestCase.assertCountEqual>` | as *b*, regardless of their | |
1216 | | order. | |
1217 +---------------------------------------+--------------------------------+--------------+
1218
1219
1220 .. method:: assertAlmostEqual(first, second, places=7, msg=None, delta=None)
1221 assertNotAlmostEqual(first, second, places=7, msg=None, delta=None)
1222
1223 Test that *first* and *second* are approximately (or not approximately)
1224 equal by computing the difference, rounding to the given number of
1225 decimal *places* (default 7), and comparing to zero. Note that these
1226 methods round the values to the given number of *decimal places* (i.e.
1227 like the :func:`round` function) and not *significant digits*.
1228
1229 If *delta* is supplied instead of *places* then the difference
1230 between *first* and *second* must be less or equal to (or greater than) *delta*.
1231
1232 Supplying both *delta* and *places* raises a :exc:`TypeError`.
1233
1234 .. versionchanged:: 3.2
1235 :meth:`assertAlmostEqual` automatically considers almost equal objects
1236 that compare equal. :meth:`assertNotAlmostEqual` automatically fails
1237 if the objects compare equal. Added the *delta* keyword argument.
1238
1239
1240 .. method:: assertGreater(first, second, msg=None)
1241 assertGreaterEqual(first, second, msg=None)
1242 assertLess(first, second, msg=None)
1243 assertLessEqual(first, second, msg=None)
1244
1245 Test that *first* is respectively >, >=, < or <= than *second* depending
1246 on the method name. If not, the test will fail::
1247
1248 >>> self.assertGreaterEqual(3, 4)
1249 AssertionError: "3" unexpectedly not greater than or equal to "4"
1250
1251 .. versionadded:: 3.1
1252
1253
1254 .. method:: assertRegex(text, regex, msg=None)
1255 assertNotRegex(text, regex, msg=None)
1256
1257 Test that a *regex* search matches (or does not match) *text*. In case
1258 of failure, the error message will include the pattern and the *text* (or
1259 the pattern and the part of *text* that unexpectedly matched). *regex*
1260 may be a regular expression object or a string containing a regular
1261 expression suitable for use by :func:`re.search`.
1262
1263 .. versionadded:: 3.1
1264 Added under the name ``assertRegexpMatches``.
1265 .. versionchanged:: 3.2
1266 The method ``assertRegexpMatches()`` has been renamed to
1267 :meth:`.assertRegex`.
1268 .. versionadded:: 3.2
1269 :meth:`.assertNotRegex`.
1270
1271
1272 .. method:: assertCountEqual(first, second, msg=None)
1273
1274 Test that sequence *first* contains the same elements as *second*,
1275 regardless of their order. When they don't, an error message listing the
1276 differences between the sequences will be generated.
1277
1278 Duplicate elements are *not* ignored when comparing *first* and
1279 *second*. It verifies whether each element has the same count in both
1280 sequences. Equivalent to:
1281 ``assertEqual(Counter(list(first)), Counter(list(second)))``
1282 but works with sequences of unhashable objects as well.
1283
1284 .. versionadded:: 3.2
1285
1286
1287 .. _type-specific-methods:
1288
1289 The :meth:`assertEqual` method dispatches the equality check for objects of
1290 the same type to different type-specific methods. These methods are already
1291 implemented for most of the built-in types, but it's also possible to
1292 register new methods using :meth:`addTypeEqualityFunc`:
1293
1294 .. method:: addTypeEqualityFunc(typeobj, function)
1295
1296 Registers a type-specific method called by :meth:`assertEqual` to check
1297 if two objects of exactly the same *typeobj* (not subclasses) compare
1298 equal. *function* must take two positional arguments and a third msg=None
1299 keyword argument just as :meth:`assertEqual` does. It must raise
1300 :data:`self.failureException(msg) <failureException>` when inequality
1301 between the first two parameters is detected -- possibly providing useful
1302 information and explaining the inequalities in details in the error
1303 message.
1304
1305 .. versionadded:: 3.1
1306
1307 The list of type-specific methods automatically used by
1308 :meth:`~TestCase.assertEqual` are summarized in the following table. Note
1309 that it's usually not necessary to invoke these methods directly.
1310
1311 +-----------------------------------------+-----------------------------+--------------+
1312 | Method | Used to compare | New in |
1313 +=========================================+=============================+==============+
1314 | :meth:`assertMultiLineEqual(a, b) | strings | 3.1 |
1315 | <TestCase.assertMultiLineEqual>` | | |
1316 +-----------------------------------------+-----------------------------+--------------+
1317 | :meth:`assertSequenceEqual(a, b) | sequences | 3.1 |
1318 | <TestCase.assertSequenceEqual>` | | |
1319 +-----------------------------------------+-----------------------------+--------------+
1320 | :meth:`assertListEqual(a, b) | lists | 3.1 |
1321 | <TestCase.assertListEqual>` | | |
1322 +-----------------------------------------+-----------------------------+--------------+
1323 | :meth:`assertTupleEqual(a, b) | tuples | 3.1 |
1324 | <TestCase.assertTupleEqual>` | | |
1325 +-----------------------------------------+-----------------------------+--------------+
1326 | :meth:`assertSetEqual(a, b) | sets or frozensets | 3.1 |
1327 | <TestCase.assertSetEqual>` | | |
1328 +-----------------------------------------+-----------------------------+--------------+
1329 | :meth:`assertDictEqual(a, b) | dicts | 3.1 |
1330 | <TestCase.assertDictEqual>` | | |
1331 +-----------------------------------------+-----------------------------+--------------+
1332
1333
1334
1335 .. method:: assertMultiLineEqual(first, second, msg=None)
1336
1337 Test that the multiline string *first* is equal to the string *second*.
1338 When not equal a diff of the two strings highlighting the differences
1339 will be included in the error message. This method is used by default
1340 when comparing strings with :meth:`assertEqual`.
1341
1342 .. versionadded:: 3.1
1343
1344
1345 .. method:: assertSequenceEqual(first, second, msg=None, seq_type=None)
1346
1347 Tests that two sequences are equal. If a *seq_type* is supplied, both
1348 *first* and *second* must be instances of *seq_type* or a failure will
1349 be raised. If the sequences are different an error message is
1350 constructed that shows the difference between the two.
1351
1352 This method is not called directly by :meth:`assertEqual`, but
1353 it's used to implement :meth:`assertListEqual` and
1354 :meth:`assertTupleEqual`.
1355
1356 .. versionadded:: 3.1
1357
1358
1359 .. method:: assertListEqual(first, second, msg=None)
1360 assertTupleEqual(first, second, msg=None)
1361
1362 Tests that two lists or tuples are equal. If not, an error message is
1363 constructed that shows only the differences between the two. An error
1364 is also raised if either of the parameters are of the wrong type.
1365 These methods are used by default when comparing lists or tuples with
1366 :meth:`assertEqual`.
1367
1368 .. versionadded:: 3.1
1369
1370
1371 .. method:: assertSetEqual(first, second, msg=None)
1372
1373 Tests that two sets are equal. If not, an error message is constructed
1374 that lists the differences between the sets. This method is used by
1375 default when comparing sets or frozensets with :meth:`assertEqual`.
1376
1377 Fails if either of *first* or *second* does not have a :meth:`~frozenset.difference`
1378 method.
1379
1380 .. versionadded:: 3.1
1381
1382
1383 .. method:: assertDictEqual(first, second, msg=None)
1384
1385 Test that two dictionaries are equal. If not, an error message is
1386 constructed that shows the differences in the dictionaries. This
1387 method will be used by default to compare dictionaries in
1388 calls to :meth:`assertEqual`.
1389
1390 .. versionadded:: 3.1
1391
1392
1393
1394 .. _other-methods-and-attrs:
1395
1396 Finally the :class:`TestCase` provides the following methods and attributes:
1397
1398
1399 .. method:: fail(msg=None)
1400
1401 Signals a test failure unconditionally, with *msg* or ``None`` for
1402 the error message.
1403
1404
1405 .. attribute:: failureException
1406
1407 This class attribute gives the exception raised by the test method. If a
1408 test framework needs to use a specialized exception, possibly to carry
1409 additional information, it must subclass this exception in order to "play
1410 fair" with the framework. The initial value of this attribute is
1411 :exc:`AssertionError`.
1412
1413
1414 .. attribute:: longMessage
1415
1416 This class attribute determines what happens when a custom failure message
1417 is passed as the msg argument to an assertXYY call that fails.
1418 ``True`` is the default value. In this case, the custom message is appended
1419 to the end of the standard failure message.
1420 When set to ``False``, the custom message replaces the standard message.
1421
1422 The class setting can be overridden in individual test methods by assigning
1423 an instance attribute, self.longMessage, to ``True`` or ``False`` before
1424 calling the assert methods.
1425
1426 The class setting gets reset before each test call.
1427
1428 .. versionadded:: 3.1
1429
1430
1431 .. attribute:: maxDiff
1432
1433 This attribute controls the maximum length of diffs output by assert
1434 methods that report diffs on failure. It defaults to 80*8 characters.
1435 Assert methods affected by this attribute are
1436 :meth:`assertSequenceEqual` (including all the sequence comparison
1437 methods that delegate to it), :meth:`assertDictEqual` and
1438 :meth:`assertMultiLineEqual`.
1439
1440 Setting ``maxDiff`` to ``None`` means that there is no maximum length of
1441 diffs.
1442
1443 .. versionadded:: 3.2
1444
1445
1446 Testing frameworks can use the following methods to collect information on
1447 the test:
1448
1449
1450 .. method:: countTestCases()
1451
1452 Return the number of tests represented by this test object. For
1453 :class:`TestCase` instances, this will always be ``1``.
1454
1455
1456 .. method:: defaultTestResult()
1457
1458 Return an instance of the test result class that should be used for this
1459 test case class (if no other result instance is provided to the
1460 :meth:`run` method).
1461
1462 For :class:`TestCase` instances, this will always be an instance of
1463 :class:`TestResult`; subclasses of :class:`TestCase` should override this
1464 as necessary.
1465
1466
1467 .. method:: id()
1468
1469 Return a string identifying the specific test case. This is usually the
1470 full name of the test method, including the module and class name.
1471
1472
1473 .. method:: shortDescription()
1474
1475 Returns a description of the test, or ``None`` if no description
1476 has been provided. The default implementation of this method
1477 returns the first line of the test method's docstring, if available,
1478 or ``None``.
1479
1480 .. versionchanged:: 3.1
1481 In 3.1 this was changed to add the test name to the short description
1482 even in the presence of a docstring. This caused compatibility issues
1483 with unittest extensions and adding the test name was moved to the
1484 :class:`TextTestResult` in Python 3.2.
1485
1486
1487 .. method:: addCleanup(function, /, *args, **kwargs)
1488
1489 Add a function to be called after :meth:`tearDown` to cleanup resources
1490 used during the test. Functions will be called in reverse order to the
1491 order they are added (:abbr:`LIFO (last-in, first-out)`). They
1492 are called with any arguments and keyword arguments passed into
1493 :meth:`addCleanup` when they are added.
1494
1495 If :meth:`setUp` fails, meaning that :meth:`tearDown` is not called,
1496 then any cleanup functions added will still be called.
1497
1498 .. versionadded:: 3.1
1499
1500
1501 .. method:: enterContext(cm)
1502
1503 Enter the supplied :term:`context manager`. If successful, also
1504 add its :meth:`~object.__exit__` method as a cleanup function by
1505 :meth:`addCleanup` and return the result of the
1506 :meth:`~object.__enter__` method.
1507
1508 .. versionadded:: 3.11
1509
1510
1511 .. method:: doCleanups()
1512
1513 This method is called unconditionally after :meth:`tearDown`, or
1514 after :meth:`setUp` if :meth:`setUp` raises an exception.
1515
1516 It is responsible for calling all the cleanup functions added by
1517 :meth:`addCleanup`. If you need cleanup functions to be called
1518 *prior* to :meth:`tearDown` then you can call :meth:`doCleanups`
1519 yourself.
1520
1521 :meth:`doCleanups` pops methods off the stack of cleanup
1522 functions one at a time, so it can be called at any time.
1523
1524 .. versionadded:: 3.1
1525
1526
1527 .. classmethod:: addClassCleanup(function, /, *args, **kwargs)
1528
1529 Add a function to be called after :meth:`tearDownClass` to cleanup
1530 resources used during the test class. Functions will be called in reverse
1531 order to the order they are added (:abbr:`LIFO (last-in, first-out)`).
1532 They are called with any arguments and keyword arguments passed into
1533 :meth:`addClassCleanup` when they are added.
1534
1535 If :meth:`setUpClass` fails, meaning that :meth:`tearDownClass` is not
1536 called, then any cleanup functions added will still be called.
1537
1538 .. versionadded:: 3.8
1539
1540
1541 .. classmethod:: enterClassContext(cm)
1542
1543 Enter the supplied :term:`context manager`. If successful, also
1544 add its :meth:`~object.__exit__` method as a cleanup function by
1545 :meth:`addClassCleanup` and return the result of the
1546 :meth:`~object.__enter__` method.
1547
1548 .. versionadded:: 3.11
1549
1550
1551 .. classmethod:: doClassCleanups()
1552
1553 This method is called unconditionally after :meth:`tearDownClass`, or
1554 after :meth:`setUpClass` if :meth:`setUpClass` raises an exception.
1555
1556 It is responsible for calling all the cleanup functions added by
1557 :meth:`addClassCleanup`. If you need cleanup functions to be called
1558 *prior* to :meth:`tearDownClass` then you can call
1559 :meth:`doClassCleanups` yourself.
1560
1561 :meth:`doClassCleanups` pops methods off the stack of cleanup
1562 functions one at a time, so it can be called at any time.
1563
1564 .. versionadded:: 3.8
1565
1566
1567 .. class:: IsolatedAsyncioTestCase(methodName='runTest')
1568
1569 This class provides an API similar to :class:`TestCase` and also accepts
1570 coroutines as test functions.
1571
1572 .. versionadded:: 3.8
1573
1574 .. attribute:: loop_factory
1575
1576 The *loop_factory* passed to :class:`asyncio.Runner`. Override
1577 in subclasses with :class:`asyncio.EventLoop` to avoid using the
1578 asyncio policy system.
1579
1580 .. versionadded:: 3.13
1581
1582 .. method:: asyncSetUp()
1583 :async:
1584
1585 Method called to prepare the test fixture. This is called after :meth:`TestCase.setUp`.
1586 This is called immediately before calling the test method; other than
1587 :exc:`AssertionError` or :exc:`SkipTest`, any exception raised by this method
1588 will be considered an error rather than a test failure. The default implementation
1589 does nothing.
1590
1591 .. method:: asyncTearDown()
1592 :async:
1593
1594 Method called immediately after the test method has been called and the
1595 result recorded. This is called before :meth:`~TestCase.tearDown`. This is called even if
1596 the test method raised an exception, so the implementation in subclasses may need
1597 to be particularly careful about checking internal state. Any exception, other than
1598 :exc:`AssertionError` or :exc:`SkipTest`, raised by this method will be
1599 considered an additional error rather than a test failure (thus increasing
1600 the total number of reported errors). This method will only be called if
1601 the :meth:`asyncSetUp` succeeds, regardless of the outcome of the test method.
1602 The default implementation does nothing.
1603
1604 .. method:: addAsyncCleanup(function, /, *args, **kwargs)
1605
1606 This method accepts a coroutine that can be used as a cleanup function.
1607
1608 .. method:: enterAsyncContext(cm)
1609 :async:
1610
1611 Enter the supplied :term:`asynchronous context manager`. If successful,
1612 also add its :meth:`~object.__aexit__` method as a cleanup function by
1613 :meth:`addAsyncCleanup` and return the result of the
1614 :meth:`~object.__aenter__` method.
1615
1616 .. versionadded:: 3.11
1617
1618
1619 .. method:: run(result=None)
1620
1621 Sets up a new event loop to run the test, collecting the result into
1622 the :class:`TestResult` object passed as *result*. If *result* is
1623 omitted or ``None``, a temporary result object is created (by calling
1624 the :meth:`~TestCase.defaultTestResult` method) and used. The result object is
1625 returned to :meth:`run`'s caller. At the end of the test all the tasks
1626 in the event loop are cancelled.
1627
1628
1629 An example illustrating the order::
1630
1631 from unittest import IsolatedAsyncioTestCase
1632
1633 events = []
1634
1635
1636 class Test(IsolatedAsyncioTestCase):
1637
1638
1639 def setUp(self):
1640 events.append("setUp")
1641
1642 async def asyncSetUp(self):
1643 self._async_connection = await AsyncConnection()
1644 events.append("asyncSetUp")
1645
1646 async def test_response(self):
1647 events.append("test_response")
1648 response = await self._async_connection.get("https://example.com")
1649 self.assertEqual(response.status_code, 200)
1650 self.addAsyncCleanup(self.on_cleanup)
1651
1652 def tearDown(self):
1653 events.append("tearDown")
1654
1655 async def asyncTearDown(self):
1656 await self._async_connection.close()
1657 events.append("asyncTearDown")
1658
1659 async def on_cleanup(self):
1660 events.append("cleanup")
1661
1662 if __name__ == "__main__":
1663 unittest.main()
1664
1665 After running the test, ``events`` would contain ``["setUp", "asyncSetUp", "test_response", "asyncTearDown", "tearDown", "cleanup"]``.
1666
1667
1668 .. class:: FunctionTestCase(testFunc, setUp=None, tearDown=None, description=None)
1669
1670 This class implements the portion of the :class:`TestCase` interface which
1671 allows the test runner to drive the test, but does not provide the methods
1672 which test code can use to check and report errors. This is used to create
1673 test cases using legacy test code, allowing it to be integrated into a
1674 :mod:`unittest`-based test framework.
1675
1676
1677 .. _testsuite-objects:
1678
1679 Grouping tests
1680 ~~~~~~~~~~~~~~
1681
1682 .. class:: TestSuite(tests=())
1683
1684 This class represents an aggregation of individual test cases and test suites.
1685 The class presents the interface needed by the test runner to allow it to be run
1686 as any other test case. Running a :class:`TestSuite` instance is the same as
1687 iterating over the suite, running each test individually.
1688
1689 If *tests* is given, it must be an iterable of individual test cases or other
1690 test suites that will be used to build the suite initially. Additional methods
1691 are provided to add test cases and suites to the collection later on.
1692
1693 :class:`TestSuite` objects behave much like :class:`TestCase` objects, except
1694 they do not actually implement a test. Instead, they are used to aggregate
1695 tests into groups of tests that should be run together. Some additional
1696 methods are available to add tests to :class:`TestSuite` instances:
1697
1698
1699 .. method:: TestSuite.addTest(test)
1700
1701 Add a :class:`TestCase` or :class:`TestSuite` to the suite.
1702
1703
1704 .. method:: TestSuite.addTests(tests)
1705
1706 Add all the tests from an iterable of :class:`TestCase` and :class:`TestSuite`
1707 instances to this test suite.
1708
1709 This is equivalent to iterating over *tests*, calling :meth:`addTest` for
1710 each element.
1711
1712 :class:`TestSuite` shares the following methods with :class:`TestCase`:
1713
1714
1715 .. method:: run(result)
1716
1717 Run the tests associated with this suite, collecting the result into the
1718 test result object passed as *result*. Note that unlike
1719 :meth:`TestCase.run`, :meth:`TestSuite.run` requires the result object to
1720 be passed in.
1721
1722
1723 .. method:: debug()
1724
1725 Run the tests associated with this suite without collecting the
1726 result. This allows exceptions raised by the test to be propagated to the
1727 caller and can be used to support running tests under a debugger.
1728
1729
1730 .. method:: countTestCases()
1731
1732 Return the number of tests represented by this test object, including all
1733 individual tests and sub-suites.
1734
1735
1736 .. method:: __iter__()
1737
1738 Tests grouped by a :class:`TestSuite` are always accessed by iteration.
1739 Subclasses can lazily provide tests by overriding :meth:`!__iter__`. Note
1740 that this method may be called several times on a single suite (for
1741 example when counting tests or comparing for equality) so the tests
1742 returned by repeated iterations before :meth:`TestSuite.run` must be the
1743 same for each call iteration. After :meth:`TestSuite.run`, callers should
1744 not rely on the tests returned by this method unless the caller uses a
1745 subclass that overrides :meth:`!TestSuite._removeTestAtIndex` to preserve
1746 test references.
1747
1748 .. versionchanged:: 3.2
1749 In earlier versions the :class:`TestSuite` accessed tests directly rather
1750 than through iteration, so overriding :meth:`!__iter__` wasn't sufficient
1751 for providing tests.
1752
1753 .. versionchanged:: 3.4
1754 In earlier versions the :class:`TestSuite` held references to each
1755 :class:`TestCase` after :meth:`TestSuite.run`. Subclasses can restore
1756 that behavior by overriding :meth:`!TestSuite._removeTestAtIndex`.
1757
1758 In the typical usage of a :class:`TestSuite` object, the :meth:`run` method
1759 is invoked by a :class:`!TestRunner` rather than by the end-user test harness.
1760
1761
1762 Loading and running tests
1763 ~~~~~~~~~~~~~~~~~~~~~~~~~
1764
1765 .. class:: TestLoader()
1766
1767 The :class:`TestLoader` class is used to create test suites from classes and
1768 modules. Normally, there is no need to create an instance of this class; the
1769 :mod:`unittest` module provides an instance that can be shared as
1770 :data:`unittest.defaultTestLoader`. Using a subclass or instance, however,
1771 allows customization of some configurable properties.
1772
1773 :class:`TestLoader` objects have the following attributes:
1774
1775
1776 .. attribute:: errors
1777
1778 A list of the non-fatal errors encountered while loading tests. Not reset
1779 by the loader at any point. Fatal errors are signalled by the relevant
1780 method raising an exception to the caller. Non-fatal errors are also
1781 indicated by a synthetic test that will raise the original error when
1782 run.
1783
1784 .. versionadded:: 3.5
1785
1786
1787 :class:`TestLoader` objects have the following methods:
1788
1789
1790 .. method:: loadTestsFromTestCase(testCaseClass)
1791
1792 Return a suite of all test cases contained in the :class:`TestCase`\ -derived
1793 :class:`!testCaseClass`.
1794
1795 A test case instance is created for each method named by
1796 :meth:`getTestCaseNames`. By default these are the method names
1797 beginning with ``test``. If :meth:`getTestCaseNames` returns no
1798 methods, but the :meth:`!runTest` method is implemented, a single test
1799 case is created for that method instead.
1800
1801
1802 .. method:: loadTestsFromModule(module, *, pattern=None)
1803
1804 Return a suite of all test cases contained in the given module. This
1805 method searches *module* for classes derived from :class:`TestCase` and
1806 creates an instance of the class for each test method defined for the
1807 class.
1808
1809 .. note::
1810
1811 While using a hierarchy of :class:`TestCase`\ -derived classes can be
1812 convenient in sharing fixtures and helper functions, defining test
1813 methods on base classes that are not intended to be instantiated
1814 directly does not play well with this method. Doing so, however, can
1815 be useful when the fixtures are different and defined in subclasses.
1816
1817 If a module provides a ``load_tests`` function it will be called to
1818 load the tests. This allows modules to customize test loading.
1819 This is the `load_tests protocol`_. The *pattern* argument is passed as
1820 the third argument to ``load_tests``.
1821
1822 .. versionchanged:: 3.2
1823 Support for ``load_tests`` added.
1824
1825 .. versionchanged:: 3.5
1826 Support for a keyword-only argument *pattern* has been added.
1827
1828 .. versionchanged:: 3.12
1829 The undocumented and unofficial *use_load_tests* parameter has been
1830 removed.
1831
1832
1833 .. method:: loadTestsFromName(name, module=None)
1834
1835 Return a suite of all test cases given a string specifier.
1836
1837 The specifier *name* is a "dotted name" that may resolve either to a
1838 module, a test case class, a test method within a test case class, a
1839 :class:`TestSuite` instance, or a callable object which returns a
1840 :class:`TestCase` or :class:`TestSuite` instance. These checks are
1841 applied in the order listed here; that is, a method on a possible test
1842 case class will be picked up as "a test method within a test case class",
1843 rather than "a callable object".
1844
1845 For example, if you have a module :mod:`!SampleTests` containing a
1846 :class:`TestCase`\ -derived class :class:`!SampleTestCase` with three test
1847 methods (:meth:`!test_one`, :meth:`!test_two`, and :meth:`!test_three`), the
1848 specifier ``'SampleTests.SampleTestCase'`` would cause this method to
1849 return a suite which will run all three test methods. Using the specifier
1850 ``'SampleTests.SampleTestCase.test_two'`` would cause it to return a test
1851 suite which will run only the :meth:`!test_two` test method. The specifier
1852 can refer to modules and packages which have not been imported; they will
1853 be imported as a side-effect.
1854
1855 The method optionally resolves *name* relative to the given *module*.
1856
1857 .. versionchanged:: 3.5
1858 If an :exc:`ImportError` or :exc:`AttributeError` occurs while traversing
1859 *name* then a synthetic test that raises that error when run will be
1860 returned. These errors are included in the errors accumulated by
1861 self.errors.
1862
1863
1864 .. method:: loadTestsFromNames(names, module=None)
1865
1866 Similar to :meth:`loadTestsFromName`, but takes a sequence of names rather
1867 than a single name. The return value is a test suite which supports all
1868 the tests defined for each name.
1869
1870
1871 .. method:: getTestCaseNames(testCaseClass)
1872
1873 Return a sorted sequence of method names found within *testCaseClass*;
1874 this should be a subclass of :class:`TestCase`.
1875
1876
1877 .. method:: discover(start_dir, pattern='test*.py', top_level_dir=None)
1878
1879 Find all the test modules by recursing into subdirectories from the
1880 specified start directory, and return a TestSuite object containing them.
1881 Only test files that match *pattern* will be loaded. (Using shell style
1882 pattern matching.) Only module names that are importable (i.e. are valid
1883 Python identifiers) will be loaded.
1884
1885 All test modules must be importable from the top level of the project. If
1886 the start directory is not the top level directory then *top_level_dir*
1887 must be specified separately.
1888
1889 If importing a module fails, for example due to a syntax error, then
1890 this will be recorded as a single error and discovery will continue. If
1891 the import failure is due to :exc:`SkipTest` being raised, it will be
1892 recorded as a skip instead of an error.
1893
1894 If a package (a directory containing a file named :file:`__init__.py`) is
1895 found, the package will be checked for a ``load_tests`` function. If this
1896 exists then it will be called
1897 ``package.load_tests(loader, tests, pattern)``. Test discovery takes care
1898 to ensure that a package is only checked for tests once during an
1899 invocation, even if the load_tests function itself calls
1900 ``loader.discover``.
1901
1902 If ``load_tests`` exists then discovery does *not* recurse into the
1903 package, ``load_tests`` is responsible for loading all tests in the
1904 package.
1905
1906 The pattern is deliberately not stored as a loader attribute so that
1907 packages can continue discovery themselves.
1908
1909 *top_level_dir* is stored internally, and used as a default to any
1910 nested calls to ``discover()``. That is, if a package's ``load_tests``
1911 calls ``loader.discover()``, it does not need to pass this argument.
1912
1913 *start_dir* can be a dotted module name as well as a directory.
1914
1915 .. versionadded:: 3.2
1916
1917 .. versionchanged:: 3.4
1918 Modules that raise :exc:`SkipTest` on import are recorded as skips,
1919 not errors.
1920
1921 .. versionchanged:: 3.4
1922 *start_dir* can be a :term:`namespace packages <namespace package>`.
1923
1924 .. versionchanged:: 3.4
1925 Paths are sorted before being imported so that execution order is the
1926 same even if the underlying file system's ordering is not dependent
1927 on file name.
1928
1929 .. versionchanged:: 3.5
1930 Found packages are now checked for ``load_tests`` regardless of
1931 whether their path matches *pattern*, because it is impossible for
1932 a package name to match the default pattern.
1933
1934 .. versionchanged:: 3.11
1935 *start_dir* can not be a :term:`namespace packages <namespace package>`.
1936 It has been broken since Python 3.7 and Python 3.11 officially remove it.
1937
1938 .. versionchanged:: 3.13
1939 *top_level_dir* is only stored for the duration of *discover* call.
1940
1941
1942 The following attributes of a :class:`TestLoader` can be configured either by
1943 subclassing or assignment on an instance:
1944
1945
1946 .. attribute:: testMethodPrefix
1947
1948 String giving the prefix of method names which will be interpreted as test
1949 methods. The default value is ``'test'``.
1950
1951 This affects :meth:`getTestCaseNames` and all the ``loadTestsFrom*``
1952 methods.
1953
1954
1955 .. attribute:: sortTestMethodsUsing
1956
1957 Function to be used to compare method names when sorting them in
1958 :meth:`getTestCaseNames` and all the ``loadTestsFrom*`` methods.
1959
1960
1961 .. attribute:: suiteClass
1962
1963 Callable object that constructs a test suite from a list of tests. No
1964 methods on the resulting object are needed. The default value is the
1965 :class:`TestSuite` class.
1966
1967 This affects all the ``loadTestsFrom*`` methods.
1968
1969 .. attribute:: testNamePatterns
1970
1971 List of Unix shell-style wildcard test name patterns that test methods
1972 have to match to be included in test suites (see ``-k`` option).
1973
1974 If this attribute is not ``None`` (the default), all test methods to be
1975 included in test suites must match one of the patterns in this list.
1976 Note that matches are always performed using :meth:`fnmatch.fnmatchcase`,
1977 so unlike patterns passed to the ``-k`` option, simple substring patterns
1978 will have to be converted using ``*`` wildcards.
1979
1980 This affects all the ``loadTestsFrom*`` methods.
1981
1982 .. versionadded:: 3.7
1983
1984
1985 .. class:: TestResult
1986
1987 This class is used to compile information about which tests have succeeded
1988 and which have failed.
1989
1990 A :class:`TestResult` object stores the results of a set of tests. The
1991 :class:`TestCase` and :class:`TestSuite` classes ensure that results are
1992 properly recorded; test authors do not need to worry about recording the
1993 outcome of tests.
1994
1995 Testing frameworks built on top of :mod:`unittest` may want access to the
1996 :class:`TestResult` object generated by running a set of tests for reporting
1997 purposes; a :class:`TestResult` instance is returned by the
1998 :meth:`!TestRunner.run` method for this purpose.
1999
2000 :class:`TestResult` instances have the following attributes that will be of
2001 interest when inspecting the results of running a set of tests:
2002
2003
2004 .. attribute:: errors
2005
2006 A list containing 2-tuples of :class:`TestCase` instances and strings
2007 holding formatted tracebacks. Each tuple represents a test which raised an
2008 unexpected exception.
2009
2010 .. attribute:: failures
2011
2012 A list containing 2-tuples of :class:`TestCase` instances and strings
2013 holding formatted tracebacks. Each tuple represents a test where a failure
2014 was explicitly signalled using the :ref:`assert\* methods <assert-methods>`.
2015
2016 .. attribute:: skipped
2017
2018 A list containing 2-tuples of :class:`TestCase` instances and strings
2019 holding the reason for skipping the test.
2020
2021 .. versionadded:: 3.1
2022
2023 .. attribute:: expectedFailures
2024
2025 A list containing 2-tuples of :class:`TestCase` instances and strings
2026 holding formatted tracebacks. Each tuple represents an expected failure
2027 or error of the test case.
2028
2029 .. attribute:: unexpectedSuccesses
2030
2031 A list containing :class:`TestCase` instances that were marked as expected
2032 failures, but succeeded.
2033
2034 .. attribute:: collectedDurations
2035
2036 A list containing 2-tuples of test case names and floats
2037 representing the elapsed time of each test which was run.
2038
2039 .. versionadded:: 3.12
2040
2041 .. attribute:: shouldStop
2042
2043 Set to ``True`` when the execution of tests should stop by :meth:`stop`.
2044
2045 .. attribute:: testsRun
2046
2047 The total number of tests run so far.
2048
2049 .. attribute:: buffer
2050
2051 If set to true, ``sys.stdout`` and ``sys.stderr`` will be buffered in between
2052 :meth:`startTest` and :meth:`stopTest` being called. Collected output will
2053 only be echoed onto the real ``sys.stdout`` and ``sys.stderr`` if the test
2054 fails or errors. Any output is also attached to the failure / error message.
2055
2056 .. versionadded:: 3.2
2057
2058 .. attribute:: failfast
2059
2060 If set to true :meth:`stop` will be called on the first failure or error,
2061 halting the test run.
2062
2063 .. versionadded:: 3.2
2064
2065 .. attribute:: tb_locals
2066
2067 If set to true then local variables will be shown in tracebacks.
2068
2069 .. versionadded:: 3.5
2070
2071 .. method:: wasSuccessful()
2072
2073 Return ``True`` if all tests run so far have passed, otherwise returns
2074 ``False``.
2075
2076 .. versionchanged:: 3.4
2077 Returns ``False`` if there were any :attr:`unexpectedSuccesses`
2078 from tests marked with the :deco:`expectedFailure` decorator.
2079
2080 .. method:: stop()
2081
2082 This method can be called to signal that the set of tests being run should
2083 be aborted by setting the :attr:`shouldStop` attribute to ``True``.
2084 :class:`!TestRunner` objects should respect this flag and return without
2085 running any additional tests.
2086
2087 For example, this feature is used by the :class:`TextTestRunner` class to
2088 stop the test framework when the user signals an interrupt from the
2089 keyboard. Interactive tools which provide :class:`!TestRunner`
2090 implementations can use this in a similar manner.
2091
2092 The following methods of the :class:`TestResult` class are used to maintain
2093 the internal data structures, and may be extended in subclasses to support
2094 additional reporting requirements. This is particularly useful in building
2095 tools which support interactive reporting while tests are being run.
2096
2097
2098 .. method:: startTest(test)
2099
2100 Called when the test case *test* is about to be run.
2101
2102 .. method:: stopTest(test)
2103
2104 Called after the test case *test* has been executed, regardless of the
2105 outcome.
2106
2107 .. method:: startTestRun()
2108
2109 Called once before any tests are executed.
2110
2111 .. versionadded:: 3.1
2112
2113
2114 .. method:: stopTestRun()
2115
2116 Called once after all tests are executed.
2117
2118 .. versionadded:: 3.1
2119
2120
2121 .. method:: addError(test, err)
2122
2123 Called when the test case *test* raises an unexpected exception. *err* is a
2124 tuple of the form returned by :func:`sys.exc_info`: ``(type, value,
2125 traceback)``.
2126
2127 The default implementation appends a tuple ``(test, formatted_err)`` to
2128 the instance's :attr:`errors` attribute, where *formatted_err* is a
2129 formatted traceback derived from *err*.
2130
2131
2132 .. method:: addFailure(test, err)
2133
2134 Called when the test case *test* signals a failure. *err* is a tuple of
2135 the form returned by :func:`sys.exc_info`: ``(type, value, traceback)``.
2136
2137 The default implementation appends a tuple ``(test, formatted_err)`` to
2138 the instance's :attr:`failures` attribute, where *formatted_err* is a
2139 formatted traceback derived from *err*.
2140
2141
2142 .. method:: addSuccess(test)
2143
2144 Called when the test case *test* succeeds.
2145
2146 The default implementation does nothing.
2147
2148
2149 .. method:: addSkip(test, reason)
2150
2151 Called when the test case *test* is skipped. *reason* is the reason the
2152 test gave for skipping.
2153
2154 The default implementation appends a tuple ``(test, reason)`` to the
2155 instance's :attr:`skipped` attribute.
2156
2157
2158 .. method:: addExpectedFailure(test, err)
2159
2160 Called when the test case *test* fails or errors, but was marked with
2161 the :deco:`expectedFailure` decorator.
2162
2163 The default implementation appends a tuple ``(test, formatted_err)`` to
2164 the instance's :attr:`expectedFailures` attribute, where *formatted_err*
2165 is a formatted traceback derived from *err*.
2166
2167
2168 .. method:: addUnexpectedSuccess(test)
2169
2170 Called when the test case *test* was marked with the
2171 :deco:`expectedFailure` decorator, but succeeded.
2172
2173 The default implementation appends the test to the instance's
2174 :attr:`unexpectedSuccesses` attribute.
2175
2176
2177 .. method:: addSubTest(test, subtest, outcome)
2178
2179 Called when a subtest finishes. *test* is the test case
2180 corresponding to the test method. *subtest* is a custom
2181 :class:`TestCase` instance describing the subtest.
2182
2183 If *outcome* is :const:`None`, the subtest succeeded. Otherwise,
2184 it failed with an exception where *outcome* is a tuple of the form
2185 returned by :func:`sys.exc_info`: ``(type, value, traceback)``.
2186
2187 The default implementation does nothing when the outcome is a
2188 success, and records subtest failures as normal failures.
2189
2190 .. versionadded:: 3.4
2191
2192 .. method:: addDuration(test, elapsed)
2193
2194 Called when the test case finishes. *elapsed* is the time represented in
2195 seconds, and it includes the execution of cleanup functions.
2196
2197 .. versionadded:: 3.12
2198
2199 .. class:: TextTestResult(stream, descriptions, verbosity, *, durations=None)
2200
2201 A concrete implementation of :class:`TestResult` used by the
2202 :class:`TextTestRunner`. Subclasses should accept ``**kwargs`` to ensure
2203 compatibility as the interface changes.
2204
2205 .. versionadded:: 3.2
2206
2207 .. versionchanged:: 3.12
2208 Added the *durations* keyword parameter.
2209
2210 .. data:: defaultTestLoader
2211
2212 Instance of the :class:`TestLoader` class intended to be shared. If no
2213 customization of the :class:`TestLoader` is needed, this instance can be used
2214 instead of repeatedly creating new instances.
2215
2216
2217 .. class:: TextTestRunner(stream=None, descriptions=True, verbosity=1, failfast=False, \
2218 buffer=False, resultclass=None, warnings=None, *, \
2219 tb_locals=False, durations=None)
2220
2221 A basic test runner implementation that outputs results to a stream. If *stream*
2222 is ``None``, the default, :data:`sys.stderr` is used as the output stream. This class
2223 has a few configurable parameters, but is essentially very simple. Graphical
2224 applications which run test suites should provide alternate implementations. Such
2225 implementations should accept ``**kwargs`` as the interface to construct runners
2226 changes when features are added to unittest.
2227
2228 By default this runner shows :exc:`DeprecationWarning`,
2229 :exc:`PendingDeprecationWarning`, :exc:`ResourceWarning` and
2230 :exc:`ImportWarning` even if they are :ref:`ignored by default
2231 <warning-ignored>`. This behavior can
2232 be overridden using Python's :option:`!-Wd` or :option:`!-Wa` options
2233 (see :ref:`Warning control <using-on-warnings>`) and leaving
2234 *warnings* to ``None``.
2235
2236 .. versionchanged:: 3.2
2237 Added the *warnings* parameter.
2238
2239 .. versionchanged:: 3.2
2240 The default stream is set to :data:`sys.stderr` at instantiation time rather
2241 than import time.
2242
2243 .. versionchanged:: 3.5
2244 Added the *tb_locals* parameter.
2245
2246 .. versionchanged:: 3.12
2247 Added the *durations* parameter.
2248
2249 .. method:: _makeResult()
2250
2251 This method returns the instance of ``TestResult`` used by :meth:`run`.
2252 It is not intended to be called directly, but can be overridden in
2253 subclasses to provide a custom ``TestResult``.
2254
2255 ``_makeResult()`` instantiates the class or callable passed in the
2256 ``TextTestRunner`` constructor as the ``resultclass`` argument. It
2257 defaults to :class:`TextTestResult` if no ``resultclass`` is provided.
2258 The result class is instantiated with the following arguments::
2259
2260 stream, descriptions, verbosity
2261
2262 .. method:: run(test)
2263
2264 This method is the main public interface to the ``TextTestRunner``. This
2265 method takes a :class:`TestSuite` or :class:`TestCase` instance. A
2266 :class:`TestResult` is created by calling
2267 :func:`_makeResult` and the test(s) are run and the
2268 results printed to stdout.
2269
2270
2271 .. function:: main(module='__main__', defaultTest=None, argv=None, testRunner=None, \
2272 testLoader=unittest.defaultTestLoader, exit=True, verbosity=1, \
2273 failfast=None, catchbreak=None, buffer=None, warnings=None)
2274
2275 A command-line program that loads a set of tests from *module* and runs them;
2276 this is primarily for making test modules conveniently executable.
2277 The simplest use for this function is to include the following line at the
2278 end of a test script::
2279
2280 if __name__ == '__main__':
2281 unittest.main()
2282
2283 You can run tests with more detailed information by passing in the verbosity
2284 argument::
2285
2286 if __name__ == '__main__':
2287 unittest.main(verbosity=2)
2288
2289 The *defaultTest* argument is either the name of a single test or an
2290 iterable of test names to run if no test names are specified via *argv*. If
2291 not specified or ``None`` and no test names are provided via *argv*, all
2292 tests found in *module* are run.
2293
2294 The *argv* argument can be a list of options passed to the program, with the
2295 first element being the program name. If not specified or ``None``,
2296 the values of :data:`sys.argv` are used.
2297
2298 The *testRunner* argument can either be a test runner class or an already
2299 created instance of it. By default ``main`` calls :func:`sys.exit` with
2300 an exit code indicating success (0) or failure (1) of the tests run.
2301 An exit code of 5 indicates that no tests were run or skipped.
2302
2303 The *testLoader* argument has to be a :class:`TestLoader` instance,
2304 and defaults to :data:`defaultTestLoader`.
2305
2306 ``main`` supports being used from the interactive interpreter by passing in the
2307 argument ``exit=False``. This displays the result on standard output without
2308 calling :func:`sys.exit`::
2309
2310 >>> from unittest import main
2311 >>> main(module='test_module', exit=False)
2312
2313 The *failfast*, *catchbreak* and *buffer* parameters have the same
2314 effect as the same-name `command-line options`_.
2315
2316 The *warnings* argument specifies the :ref:`warning filter <warning-filter>`
2317 that should be used while running the tests. If it's not specified, it will
2318 remain ``None`` if a :option:`!-W` option is passed to :program:`python`
2319 (see :ref:`Warning control <using-on-warnings>`),
2320 otherwise it will be set to ``'default'``.
2321
2322 Calling ``main`` returns an object with the ``result`` attribute that contains
2323 the result of the tests run as a :class:`unittest.TestResult`.
2324
2325 .. versionchanged:: 3.1
2326 The *exit* parameter was added.
2327
2328 .. versionchanged:: 3.2
2329 The *verbosity*, *failfast*, *catchbreak*, *buffer*
2330 and *warnings* parameters were added.
2331
2332 .. versionchanged:: 3.4
2333 The *defaultTest* parameter was changed to also accept an iterable of
2334 test names.
2335
2336
2337 .. _load_tests-protocol:
2338
2339 load_tests Protocol
2340 ###################
2341
2342 .. versionadded:: 3.2
2343
2344 Modules or packages can customize how tests are loaded from them during normal
2345 test runs or test discovery by implementing a function called ``load_tests``.
2346
2347 If a test module defines ``load_tests`` it will be called by
2348 :meth:`TestLoader.loadTestsFromModule` with the following arguments::
2349
2350 load_tests(loader, standard_tests, pattern)
2351
2352 where *pattern* is passed straight through from ``loadTestsFromModule``. It
2353 defaults to ``None``.
2354
2355 It should return a :class:`TestSuite`.
2356
2357 *loader* is the instance of :class:`TestLoader` doing the loading.
2358 *standard_tests* are the tests that would be loaded by default from the
2359 module. It is common for test modules to only want to add or remove tests
2360 from the standard set of tests.
2361 The third argument is used when loading packages as part of test discovery.
2362
2363 A typical ``load_tests`` function that loads tests from a specific set of
2364 :class:`TestCase` classes may look like::
2365
2366 test_cases = (TestCase1, TestCase2, TestCase3)
2367
2368 def load_tests(loader, tests, pattern):
2369 suite = TestSuite()
2370 for test_class in test_cases:
2371 tests = loader.loadTestsFromTestCase(test_class)
2372 suite.addTests(tests)
2373 return suite
2374
2375 If discovery is started in a directory containing a package, either from the
2376 command line or by calling :meth:`TestLoader.discover`, then the package
2377 :file:`__init__.py` will be checked for ``load_tests``. If that function does
2378 not exist, discovery will recurse into the package as though it were just
2379 another directory. Otherwise, discovery of the package's tests will be left up
2380 to ``load_tests`` which is called with the following arguments::
2381
2382 load_tests(loader, standard_tests, pattern)
2383
2384 This should return a :class:`TestSuite` representing all the tests
2385 from the package. (``standard_tests`` will only contain tests
2386 collected from :file:`__init__.py`.)
2387
2388 Because the pattern is passed into ``load_tests`` the package is free to
2389 continue (and potentially modify) test discovery. A 'do nothing'
2390 ``load_tests`` function for a test package would look like::
2391
2392 def load_tests(loader, standard_tests, pattern):
2393 # top level directory cached on loader instance
2394 this_dir = os.path.dirname(__file__)
2395 package_tests = loader.discover(start_dir=this_dir, pattern=pattern)
2396 standard_tests.addTests(package_tests)
2397 return standard_tests
2398
2399 .. versionchanged:: 3.5
2400 Discovery no longer checks package names for matching *pattern* due to the
2401 impossibility of package names matching the default pattern.
2402
2403
2404
2405 Class and Module Fixtures
2406 -------------------------
2407
2408 Class and module level fixtures are implemented in :class:`TestSuite`. When
2409 the test suite encounters a test from a new class then
2410 :meth:`~TestCase.tearDownClass` from the previous class (if there is one)
2411 is called, followed by :meth:`~TestCase.setUpClass` from the new class.
2412
2413 Similarly if a test is from a different module from the previous test then
2414 ``tearDownModule`` from the previous module is run, followed by
2415 ``setUpModule`` from the new module.
2416
2417 After all the tests have run the final ``tearDownClass`` and
2418 ``tearDownModule`` are run.
2419
2420 Note that shared fixtures do not play well with [potential] features like test
2421 parallelization and they break test isolation. They should be used with care.
2422
2423 The default ordering of tests created by the unittest test loaders is to group
2424 all tests from the same modules and classes together. This will lead to
2425 ``setUpClass`` / ``setUpModule`` (etc) being called exactly once per class and
2426 module. If you randomize the order, so that tests from different modules and
2427 classes are adjacent to each other, then these shared fixture functions may be
2428 called multiple times in a single test run.
2429
2430 Shared fixtures are not intended to work with suites with non-standard
2431 ordering. A ``BaseTestSuite`` still exists for frameworks that don't want to
2432 support shared fixtures.
2433
2434 If there are any exceptions raised during one of the shared fixture functions
2435 the test is reported as an error. Because there is no corresponding test
2436 instance an ``_ErrorHolder`` object (that has the same interface as a
2437 :class:`TestCase`) is created to represent the error. If you are just using
2438 the standard unittest test runner then this detail doesn't matter, but if you
2439 are a framework author it may be relevant.
2440
2441
2442 setUpClass and tearDownClass
2443 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2444
2445 These must be implemented as class methods::
2446
2447 import unittest
2448
2449 class Test(unittest.TestCase):
2450 @classmethod
2451 def setUpClass(cls):
2452 cls._connection = createExpensiveConnectionObject()
2453
2454 @classmethod
2455 def tearDownClass(cls):
2456 cls._connection.destroy()
2457
2458 If you want the ``setUpClass`` and ``tearDownClass`` on base classes called
2459 then you must call up to them yourself. The implementations in
2460 :class:`TestCase` are empty.
2461
2462 If an exception is raised during a ``setUpClass`` then the tests in the class
2463 are not run and the ``tearDownClass`` is not run. Skipped classes will not
2464 have ``setUpClass`` or ``tearDownClass`` run. If the exception is a
2465 :exc:`SkipTest` exception then the class will be reported as having been skipped
2466 instead of as an error.
2467
2468
2469 setUpModule and tearDownModule
2470 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2471
2472 .. function:: setUpModule
2473 tearDownModule
2474 :no-typesetting:
2475
2476 These should be implemented as functions::
2477
2478 def setUpModule():
2479 createConnection()
2480
2481 def tearDownModule():
2482 closeConnection()
2483
2484 If an exception is raised in a ``setUpModule`` then none of the tests in the
2485 module will be run and the ``tearDownModule`` will not be run. If the exception is a
2486 :exc:`SkipTest` exception then the module will be reported as having been skipped
2487 instead of as an error.
2488
2489 To add cleanup code that must be run even in the case of an exception, use
2490 ``addModuleCleanup``:
2491
2492
2493 .. function:: addModuleCleanup(function, /, *args, **kwargs)
2494
2495 Add a function to be called after :func:`tearDownModule` to cleanup
2496 resources used during the test class. Functions will be called in reverse
2497 order to the order they are added (:abbr:`LIFO (last-in, first-out)`).
2498 They are called with any arguments and keyword arguments passed into
2499 :meth:`addModuleCleanup` when they are added.
2500
2501 If :meth:`setUpModule` fails, meaning that :func:`tearDownModule` is not
2502 called, then any cleanup functions added will still be called.
2503
2504 .. versionadded:: 3.8
2505
2506
2507 .. function:: enterModuleContext(cm)
2508
2509 Enter the supplied :term:`context manager`. If successful, also
2510 add its :meth:`~object.__exit__` method as a cleanup function by
2511 :func:`addModuleCleanup` and return the result of the
2512 :meth:`~object.__enter__` method.
2513
2514 .. versionadded:: 3.11
2515
2516
2517 .. function:: doModuleCleanups()
2518
2519 This function is called unconditionally after :func:`tearDownModule`, or
2520 after :func:`setUpModule` if :func:`setUpModule` raises an exception.
2521
2522 It is responsible for calling all the cleanup functions added by
2523 :func:`addModuleCleanup`. If you need cleanup functions to be called
2524 *prior* to :func:`tearDownModule` then you can call
2525 :func:`doModuleCleanups` yourself.
2526
2527 :func:`doModuleCleanups` pops methods off the stack of cleanup
2528 functions one at a time, so it can be called at any time.
2529
2530 .. versionadded:: 3.8
2531
2532
2533 Signal Handling
2534 ---------------
2535
2536 .. versionadded:: 3.2
2537
2538 The :option:`-c/--catch <unittest -c>` command-line option to unittest,
2539 along with the ``catchbreak`` parameter to :func:`unittest.main`, provide
2540 more friendly handling of control-C during a test run. With catch break
2541 behavior enabled control-C will allow the currently running test to complete,
2542 and the test run will then end and report all the results so far. A second
2543 control-c will raise a :exc:`KeyboardInterrupt` in the usual way.
2544
2545 The control-c handling signal handler attempts to remain compatible with code or
2546 tests that install their own :const:`signal.SIGINT` handler. If the ``unittest``
2547 handler is called but *isn't* the installed :const:`signal.SIGINT` handler,
2548 i.e. it has been replaced by the system under test and delegated to, then it
2549 calls the default handler. This will normally be the expected behavior by code
2550 that replaces an installed handler and delegates to it. For individual tests
2551 that need ``unittest`` control-c handling disabled the :func:`removeHandler`
2552 decorator can be used.
2553
2554 There are a few utility functions for framework authors to enable control-c
2555 handling functionality within test frameworks.
2556
2557 .. function:: installHandler()
2558
2559 Install the control-c handler. When a :const:`signal.SIGINT` is received
2560 (usually in response to the user pressing control-c) all registered results
2561 have :meth:`~TestResult.stop` called.
2562
2563
2564 .. function:: registerResult(result)
2565
2566 Register a :class:`TestResult` object for control-c handling. Registering a
2567 result stores a weak reference to it, so it doesn't prevent the result from
2568 being garbage collected.
2569
2570 Registering a :class:`TestResult` object has no side-effects if control-c
2571 handling is not enabled, so test frameworks can unconditionally register
2572 all results they create independently of whether or not handling is enabled.
2573
2574
2575 .. function:: removeResult(result)
2576
2577 Remove a registered result. Once a result has been removed then
2578 :meth:`~TestResult.stop` will no longer be called on that result object in
2579 response to a control-c.
2580
2581
2582 .. function:: removeHandler(function=None)
2583
2584 When called without arguments this function removes the control-c handler
2585 if it has been installed. This function can also be used as a test decorator
2586 to temporarily remove the handler while the test is being executed::
2587
2588 @unittest.removeHandler
2589 def test_signal_handling(self):
2590 ...