| 1 |
:mod:`!unittest` --- Unit testing framework |
| 2 |
=========================================== |
| 3 |
|
| 4 |
.. module:: unittest |
| 5 |
:synopsis: Unit testing framework for Python. |
| 6 |
|
| 7 |
.. moduleauthor:: Steve Purcell <[email protected]> |
| 8 |
.. sectionauthor:: Steve Purcell <[email protected]> |
| 9 |
.. sectionauthor:: Fred L. Drake, Jr. <[email protected]> |
| 10 |
.. sectionauthor:: Raymond Hettinger <[email protected]> |
| 11 |
|
| 12 |
**Source code:** :source:`Lib/unittest/__init__.py` |
| 13 |
|
| 14 |
-------------- |
| 15 |
|
| 16 |
(If you are already familiar with the basic concepts of testing, you might want |
| 17 |
to skip to :ref:`the list of assert methods <assert-methods>`.) |
| 18 |
|
| 19 |
The :mod:`!unittest` unit testing framework was originally inspired by JUnit |
| 20 |
and has a similar flavor as major unit testing frameworks in other |
| 21 |
languages. It supports test automation, sharing of setup and shutdown code |
| 22 |
for tests, aggregation of tests into collections, and independence of the |
| 23 |
tests from the reporting framework. |
| 24 |
|
| 25 |
To achieve this, :mod:`!unittest` supports some important concepts in an |
| 26 |
object-oriented way: |
| 27 |
|
| 28 |
test fixture |
| 29 |
A :dfn:`test fixture` represents the preparation needed to perform one or more |
| 30 |
tests, and any associated cleanup actions. This may involve, for example, |
| 31 |
creating temporary or proxy databases, directories, or starting a server |
| 32 |
process. |
| 33 |
|
| 34 |
test case |
| 35 |
A :dfn:`test case` is the individual unit of testing. It checks for a specific |
| 36 |
response to a particular set of inputs. :mod:`!unittest` provides a base class, |
| 37 |
:class:`TestCase`, which may be used to create new test cases. |
| 38 |
|
| 39 |
test suite |
| 40 |
A :dfn:`test suite` is a collection of test cases, test suites, or both. It is |
| 41 |
used to aggregate tests that should be executed together. |
| 42 |
|
| 43 |
test runner |
| 44 |
A :dfn:`test runner` is a component which orchestrates the execution of tests |
| 45 |
and provides the outcome to the user. The runner may use a graphical interface, |
| 46 |
a textual interface, or return a special value to indicate the results of |
| 47 |
executing the tests. |
| 48 |
|
| 49 |
.. seealso:: |
| 50 |
|
| 51 |
Module :mod:`doctest` |
| 52 |
Another test-support module with a very different flavor. |
| 53 |
|
| 54 |
`Simple Smalltalk Testing: With Patterns <https://web.archive.org/web/20150315073817/http://www.xprogramming.com/testfram.htm>`_ |
| 55 |
Kent Beck's original paper on testing frameworks using the pattern shared |
| 56 |
by :mod:`!unittest`. |
| 57 |
|
| 58 |
`pytest <https://docs.pytest.org/>`_ |
| 59 |
Third-party unittest framework with a lighter-weight syntax for writing |
| 60 |
tests. For example, ``assert func(10) == 42``. |
| 61 |
|
| 62 |
`The Python Testing Tools Taxonomy <https://wiki.python.org/moin/PythonTestingToolsTaxonomy>`_ |
| 63 |
An extensive list of Python testing tools including functional testing |
| 64 |
frameworks and mock object libraries. |
| 65 |
|
| 66 |
`Testing in Python Mailing List <http://lists.idyll.org/listinfo/testing-in-python>`_ |
| 67 |
A special-interest-group for discussion of testing, and testing tools, |
| 68 |
in Python. |
| 69 |
|
| 70 |
The script :file:`Tools/unittestgui/unittestgui.py` in the Python source distribution is |
| 71 |
a GUI tool for test discovery and execution. This is intended largely for ease of use |
| 72 |
for those new to unit testing. For production environments it is |
| 73 |
recommended that tests be driven by a continuous integration system such as |
| 74 |
`Buildbot <https://buildbot.net/>`_, `Jenkins <https://www.jenkins.io/>`_, |
| 75 |
`GitHub Actions <https://github.com/features/actions>`_, or |
| 76 |
`AppVeyor <https://www.appveyor.com/>`_. |
| 77 |
|
| 78 |
|
| 79 |
.. _unittest-minimal-example: |
| 80 |
|
| 81 |
Basic example |
| 82 |
------------- |
| 83 |
|
| 84 |
The :mod:`!unittest` module provides a rich set of tools for constructing and |
| 85 |
running tests. This section demonstrates that a small subset of the tools |
| 86 |
suffice to meet the needs of most users. |
| 87 |
|
| 88 |
Here is a short script to test three string methods:: |
| 89 |
|
| 90 |
import unittest |
| 91 |
|
| 92 |
class TestStringMethods(unittest.TestCase): |
| 93 |
|
| 94 |
def test_upper(self): |
| 95 |
self.assertEqual('foo'.upper(), 'FOO') |
| 96 |
|
| 97 |
def test_isupper(self): |
| 98 |
self.assertTrue('FOO'.isupper()) |
| 99 |
self.assertFalse('Foo'.isupper()) |
| 100 |
|
| 101 |
def test_split(self): |
| 102 |
s = 'hello world' |
| 103 |
self.assertEqual(s.split(), ['hello', 'world']) |
| 104 |
# check that s.split fails when the separator is not a string |
| 105 |
with self.assertRaises(TypeError): |
| 106 |
s.split(2) |
| 107 |
|
| 108 |
if __name__ == '__main__': |
| 109 |
unittest.main() |
| 110 |
|
| 111 |
|
| 112 |
A test case is created by subclassing :class:`unittest.TestCase`. The three |
| 113 |
individual tests are defined with methods whose names start with the letters |
| 114 |
``test``. This naming convention informs the test runner about which methods |
| 115 |
represent tests. |
| 116 |
|
| 117 |
The crux of each test is a call to :meth:`~TestCase.assertEqual` to check for an |
| 118 |
expected result; :meth:`~TestCase.assertTrue` or :meth:`~TestCase.assertFalse` |
| 119 |
to verify a condition; or :meth:`~TestCase.assertRaises` to verify that a |
| 120 |
specific exception gets raised. These methods are used instead of the |
| 121 |
:keyword:`assert` statement so the test runner can accumulate all test results |
| 122 |
and produce a report. |
| 123 |
|
| 124 |
The :meth:`~TestCase.setUp` and :meth:`~TestCase.tearDown` methods allow you |
| 125 |
to define instructions that will be executed before and after each test method. |
| 126 |
They are covered in more detail in the section :ref:`organizing-tests`. |
| 127 |
|
| 128 |
The final block shows a simple way to run the tests. :func:`unittest.main` |
| 129 |
provides a command-line interface to the test script. When run from the command |
| 130 |
line, the above script produces an output that looks like this:: |
| 131 |
|
| 132 |
... |
| 133 |
---------------------------------------------------------------------- |
| 134 |
Ran 3 tests in 0.000s |
| 135 |
|
| 136 |
OK |
| 137 |
|
| 138 |
Passing the ``-v`` option to your test script will instruct :func:`unittest.main` |
| 139 |
to enable a higher level of verbosity, and produce the following output:: |
| 140 |
|
| 141 |
test_isupper (__main__.TestStringMethods.test_isupper) ... ok |
| 142 |
test_split (__main__.TestStringMethods.test_split) ... ok |
| 143 |
test_upper (__main__.TestStringMethods.test_upper) ... ok |
| 144 |
|
| 145 |
---------------------------------------------------------------------- |
| 146 |
Ran 3 tests in 0.001s |
| 147 |
|
| 148 |
OK |
| 149 |
|
| 150 |
The above examples show the most commonly used :mod:`!unittest` features which |
| 151 |
are sufficient to meet many everyday testing needs. The remainder of the |
| 152 |
documentation explores the full feature set from first principles. |
| 153 |
|
| 154 |
.. versionchanged:: 3.11 |
| 155 |
The behavior of returning a value from a test method (other than the default |
| 156 |
``None`` value), is now deprecated. |
| 157 |
|
| 158 |
|
| 159 |
.. _unittest-command-line-interface: |
| 160 |
|
| 161 |
Command-Line Interface |
| 162 |
---------------------- |
| 163 |
|
| 164 |
The unittest module can be used from the command line to run tests from |
| 165 |
modules, classes or even individual test methods:: |
| 166 |
|
| 167 |
python -m unittest test_module1 test_module2 |
| 168 |
python -m unittest test_module.TestClass |
| 169 |
python -m unittest test_module.TestClass.test_method |
| 170 |
|
| 171 |
You can pass in a list with any combination of module names, and fully |
| 172 |
qualified class or method names. |
| 173 |
|
| 174 |
Test modules can be specified by file path as well:: |
| 175 |
|
| 176 |
python -m unittest tests/test_something.py |
| 177 |
|
| 178 |
This allows you to use the shell filename completion to specify the test module. |
| 179 |
The file specified must still be importable as a module. The path is converted |
| 180 |
to a module name by removing the '.py' and converting path separators into '.'. |
| 181 |
If you want to execute a test file that isn't importable as a module you should |
| 182 |
execute the file directly instead. |
| 183 |
|
| 184 |
You can run tests with more detail (higher verbosity) by passing in the -v flag:: |
| 185 |
|
| 186 |
python -m unittest -v test_module |
| 187 |
|
| 188 |
When executed without arguments :ref:`unittest-test-discovery` is started:: |
| 189 |
|
| 190 |
python -m unittest |
| 191 |
|
| 192 |
For a list of all the command-line options:: |
| 193 |
|
| 194 |
python -m unittest -h |
| 195 |
|
| 196 |
.. versionchanged:: 3.2 |
| 197 |
In earlier versions it was only possible to run individual test methods and |
| 198 |
not modules or classes. |
| 199 |
|
| 200 |
.. versionadded:: 3.14 |
| 201 |
Output is colorized by default and can be |
| 202 |
:ref:`controlled using environment variables <using-on-controlling-color>`. |
| 203 |
|
| 204 |
Command-line options |
| 205 |
~~~~~~~~~~~~~~~~~~~~ |
| 206 |
|
| 207 |
:program:`unittest` supports these command-line options: |
| 208 |
|
| 209 |
.. program:: unittest |
| 210 |
|
| 211 |
.. option:: -b, --buffer |
| 212 |
|
| 213 |
The standard output and standard error streams are buffered during the test |
| 214 |
run. Output during a passing test is discarded. Output is echoed normally |
| 215 |
on test fail or error and is added to the failure messages. |
| 216 |
|
| 217 |
.. option:: -c, --catch |
| 218 |
|
| 219 |
:kbd:`Control-C` during the test run waits for the current test to end and then |
| 220 |
reports all the results so far. A second :kbd:`Control-C` raises the normal |
| 221 |
:exc:`KeyboardInterrupt` exception. |
| 222 |
|
| 223 |
See `Signal Handling`_ for the functions that provide this functionality. |
| 224 |
|
| 225 |
.. option:: -f, --failfast |
| 226 |
|
| 227 |
Stop the test run on the first error or failure. |
| 228 |
|
| 229 |
.. option:: -k |
| 230 |
|
| 231 |
Only run test methods and classes that match the pattern or substring. |
| 232 |
This option may be used multiple times, in which case all test cases that |
| 233 |
match any of the given patterns are included. |
| 234 |
|
| 235 |
Patterns that contain a wildcard character (``*``) are matched against the |
| 236 |
test name using :meth:`fnmatch.fnmatchcase`; otherwise simple case-sensitive |
| 237 |
substring matching is used. |
| 238 |
|
| 239 |
Patterns are matched against the fully qualified test method name as |
| 240 |
imported by the test loader. |
| 241 |
|
| 242 |
For example, ``-k foo`` matches ``foo_tests.SomeTest.test_something``, |
| 243 |
``bar_tests.SomeTest.test_foo``, but not ``bar_tests.FooTest.test_something``. |
| 244 |
|
| 245 |
.. option:: --locals |
| 246 |
|
| 247 |
Show local variables in tracebacks. |
| 248 |
|
| 249 |
.. option:: --durations N |
| 250 |
|
| 251 |
Show the N slowest test cases (N=0 for all). |
| 252 |
|
| 253 |
.. versionadded:: 3.2 |
| 254 |
The command-line options ``-b``, ``-c`` and ``-f`` were added. |
| 255 |
|
| 256 |
.. versionadded:: 3.5 |
| 257 |
The command-line option ``--locals``. |
| 258 |
|
| 259 |
.. versionadded:: 3.7 |
| 260 |
The command-line option ``-k``. |
| 261 |
|
| 262 |
.. versionadded:: 3.12 |
| 263 |
The command-line option ``--durations``. |
| 264 |
|
| 265 |
The command line can also be used for test discovery, for running all of the |
| 266 |
tests in a project or just a subset. |
| 267 |
|
| 268 |
.. _unittest-test-discovery: |
| 269 |
|
| 270 |
Test Discovery |
| 271 |
-------------- |
| 272 |
|
| 273 |
.. versionadded:: 3.2 |
| 274 |
|
| 275 |
Unittest supports simple test discovery. In order to be compatible with test |
| 276 |
discovery, all of the test files must be :ref:`modules <tut-modules>` or |
| 277 |
:ref:`packages <tut-packages>` importable from the top-level directory of |
| 278 |
the project (this means that their filenames must be valid :ref:`identifiers |
| 279 |
<identifiers>`). |
| 280 |
|
| 281 |
Test discovery is implemented in :meth:`TestLoader.discover`, but can also be |
| 282 |
used from the command line. The basic command-line usage is:: |
| 283 |
|
| 284 |
cd project_directory |
| 285 |
python -m unittest discover |
| 286 |
|
| 287 |
.. note:: |
| 288 |
|
| 289 |
As a shortcut, ``python -m unittest`` is the equivalent of |
| 290 |
``python -m unittest discover``. If you want to pass arguments to test |
| 291 |
discovery the ``discover`` sub-command must be used explicitly. |
| 292 |
|
| 293 |
The ``discover`` sub-command has the following options: |
| 294 |
|
| 295 |
.. program:: unittest discover |
| 296 |
|
| 297 |
.. option:: -v, --verbose |
| 298 |
|
| 299 |
Verbose output |
| 300 |
|
| 301 |
.. option:: -s, --start-directory directory |
| 302 |
|
| 303 |
Directory to start discovery (``.`` default) |
| 304 |
|
| 305 |
.. option:: -p, --pattern pattern |
| 306 |
|
| 307 |
Pattern to match test files (``test*.py`` default) |
| 308 |
|
| 309 |
.. option:: -t, --top-level-directory directory |
| 310 |
|
| 311 |
Top level directory of project (defaults to start directory) |
| 312 |
|
| 313 |
The :option:`-s`, :option:`-p`, and :option:`-t` options can be passed in |
| 314 |
as positional arguments in that order. The following two command lines |
| 315 |
are equivalent:: |
| 316 |
|
| 317 |
python -m unittest discover -s project_directory -p "*_test.py" |
| 318 |
python -m unittest discover project_directory "*_test.py" |
| 319 |
|
| 320 |
As well as being a path it is possible to pass a package name, for example |
| 321 |
``myproject.subpackage.test``, as the start directory. The package name you |
| 322 |
supply will then be imported and its location on the filesystem will be used |
| 323 |
as the start directory. |
| 324 |
|
| 325 |
.. caution:: |
| 326 |
|
| 327 |
Test discovery loads tests by importing them. Once test discovery has found |
| 328 |
all the test files from the start directory you specify it turns the paths |
| 329 |
into package names to import. For example :file:`foo/bar/baz.py` will be |
| 330 |
imported as ``foo.bar.baz``. |
| 331 |
|
| 332 |
If you have a package installed globally and attempt test discovery on |
| 333 |
a different copy of the package then the import *could* happen from the |
| 334 |
wrong place. If this happens test discovery will warn you and exit. |
| 335 |
|
| 336 |
If you supply the start directory as a package name rather than a |
| 337 |
path to a directory then discover assumes that whichever location it |
| 338 |
imports from is the location you intended, so you will not get the |
| 339 |
warning. |
| 340 |
|
| 341 |
Test modules and packages can customize test loading and discovery by through |
| 342 |
the `load_tests protocol`_. |
| 343 |
|
| 344 |
.. versionchanged:: 3.4 |
| 345 |
Test discovery supports :term:`namespace packages <namespace package>`. |
| 346 |
|
| 347 |
.. versionchanged:: 3.11 |
| 348 |
Test discovery dropped the :term:`namespace packages <namespace package>` |
| 349 |
support. It has been broken since Python 3.7. |
| 350 |
Start directory and its subdirectories containing tests must be regular |
| 351 |
package that have ``__init__.py`` file. |
| 352 |
|
| 353 |
If the start directory is the dotted name of the package, the ancestor packages |
| 354 |
can be namespace packages. |
| 355 |
|
| 356 |
.. versionchanged:: 3.14 |
| 357 |
Test discovery supports :term:`namespace package` as start directory again. |
| 358 |
To avoid scanning directories unrelated to Python, |
| 359 |
tests are not searched in subdirectories that do not contain ``__init__.py``. |
| 360 |
|
| 361 |
|
| 362 |
.. _organizing-tests: |
| 363 |
|
| 364 |
Organizing test code |
| 365 |
-------------------- |
| 366 |
|
| 367 |
The basic building blocks of unit testing are :dfn:`test cases` --- single |
| 368 |
scenarios that must be set up and checked for correctness. In :mod:`!unittest`, |
| 369 |
test cases are represented by :class:`unittest.TestCase` instances. |
| 370 |
To make your own test cases you must write subclasses of |
| 371 |
:class:`TestCase` or use :class:`FunctionTestCase`. |
| 372 |
|
| 373 |
The testing code of a :class:`TestCase` instance should be entirely self |
| 374 |
contained, such that it can be run either in isolation or in arbitrary |
| 375 |
combination with any number of other test cases. |
| 376 |
|
| 377 |
The simplest :class:`TestCase` subclass will simply implement a test method |
| 378 |
(i.e. a method whose name starts with ``test``) in order to perform specific |
| 379 |
testing code:: |
| 380 |
|
| 381 |
import unittest |
| 382 |
|
| 383 |
class DefaultWidgetSizeTestCase(unittest.TestCase): |
| 384 |
def test_default_widget_size(self): |
| 385 |
widget = Widget('The widget') |
| 386 |
self.assertEqual(widget.size(), (50, 50)) |
| 387 |
|
| 388 |
Note that in order to test something, we use one of the :ref:`assert\* methods <assert-methods>` |
| 389 |
provided by the :class:`TestCase` base class. If the test fails, an |
| 390 |
exception will be raised with an explanatory message, and :mod:`!unittest` |
| 391 |
will identify the test case as a :dfn:`failure`. Any other exceptions will be |
| 392 |
treated as :dfn:`errors`. |
| 393 |
|
| 394 |
Tests can be numerous, and their set-up can be repetitive. Luckily, we |
| 395 |
can factor out set-up code by implementing a method called |
| 396 |
:meth:`~TestCase.setUp`, which the testing framework will automatically |
| 397 |
call for every single test we run:: |
| 398 |
|
| 399 |
import unittest |
| 400 |
|
| 401 |
class WidgetTestCase(unittest.TestCase): |
| 402 |
def setUp(self): |
| 403 |
self.widget = Widget('The widget') |
| 404 |
|
| 405 |
def test_default_widget_size(self): |
| 406 |
self.assertEqual(self.widget.size(), (50,50), |
| 407 |
'incorrect default size') |
| 408 |
|
| 409 |
def test_widget_resize(self): |
| 410 |
self.widget.resize(100,150) |
| 411 |
self.assertEqual(self.widget.size(), (100,150), |
| 412 |
'wrong size after resize') |
| 413 |
|
| 414 |
.. note:: |
| 415 |
The order in which the various tests will be run is determined |
| 416 |
by sorting the test method names with respect to the built-in |
| 417 |
ordering for strings. |
| 418 |
|
| 419 |
If the :meth:`~TestCase.setUp` method raises an exception while the test is |
| 420 |
running, the framework will consider the test to have suffered an error, and |
| 421 |
the test method will not be executed. |
| 422 |
|
| 423 |
Similarly, we can provide a :meth:`~TestCase.tearDown` method that tidies up |
| 424 |
after the test method has been run:: |
| 425 |
|
| 426 |
import unittest |
| 427 |
|
| 428 |
class WidgetTestCase(unittest.TestCase): |
| 429 |
def setUp(self): |
| 430 |
self.widget = Widget('The widget') |
| 431 |
|
| 432 |
def tearDown(self): |
| 433 |
self.widget.dispose() |
| 434 |
|
| 435 |
If :meth:`~TestCase.setUp` succeeded, :meth:`~TestCase.tearDown` will be |
| 436 |
run whether the test method succeeded or not. |
| 437 |
|
| 438 |
Such a working environment for the testing code is called a |
| 439 |
:dfn:`test fixture`. A new TestCase instance is created as a unique |
| 440 |
test fixture used to execute each individual test method. Thus |
| 441 |
:meth:`~TestCase.setUp`, :meth:`~TestCase.tearDown`, and :meth:`!TestCase.__init__` |
| 442 |
will be called once per test. |
| 443 |
|
| 444 |
It is recommended that you use TestCase implementations to group tests together |
| 445 |
according to the features they test. :mod:`!unittest` provides a mechanism for |
| 446 |
this: the :dfn:`test suite`, represented by :mod:`!unittest`'s |
| 447 |
:class:`TestSuite` class. In most cases, calling :func:`unittest.main` will do |
| 448 |
the right thing and collect all the module's test cases for you and execute |
| 449 |
them. |
| 450 |
|
| 451 |
However, should you want to customize the building of your test suite, |
| 452 |
you can do it yourself:: |
| 453 |
|
| 454 |
def suite(): |
| 455 |
suite = unittest.TestSuite() |
| 456 |
suite.addTest(WidgetTestCase('test_default_widget_size')) |
| 457 |
suite.addTest(WidgetTestCase('test_widget_resize')) |
| 458 |
return suite |
| 459 |
|
| 460 |
if __name__ == '__main__': |
| 461 |
runner = unittest.TextTestRunner() |
| 462 |
runner.run(suite()) |
| 463 |
|
| 464 |
You can place the definitions of test cases and test suites in the same modules |
| 465 |
as the code they are to test (such as :file:`widget.py`), but there are several |
| 466 |
advantages to placing the test code in a separate module, such as |
| 467 |
:file:`test_widget.py`: |
| 468 |
|
| 469 |
* The test module can be run standalone from the command line. |
| 470 |
|
| 471 |
* The test code can more easily be separated from shipped code. |
| 472 |
|
| 473 |
* There is less temptation to change test code to fit the code it tests without |
| 474 |
a good reason. |
| 475 |
|
| 476 |
* Test code should be modified much less frequently than the code it tests. |
| 477 |
|
| 478 |
* Tested code can be refactored more easily. |
| 479 |
|
| 480 |
* Tests for modules written in C must be in separate modules anyway, so why not |
| 481 |
be consistent? |
| 482 |
|
| 483 |
* If the testing strategy changes, there is no need to change the source code. |
| 484 |
|
| 485 |
|
| 486 |
.. _legacy-unit-tests: |
| 487 |
|
| 488 |
Re-using old test code |
| 489 |
---------------------- |
| 490 |
|
| 491 |
Some users will find that they have existing test code that they would like to |
| 492 |
run from :mod:`!unittest`, without converting every old test function to a |
| 493 |
:class:`TestCase` subclass. |
| 494 |
|
| 495 |
For this reason, :mod:`!unittest` provides a :class:`FunctionTestCase` class. |
| 496 |
This subclass of :class:`TestCase` can be used to wrap an existing test |
| 497 |
function. Set-up and tear-down functions can also be provided. |
| 498 |
|
| 499 |
Given the following test function:: |
| 500 |
|
| 501 |
def testSomething(): |
| 502 |
something = makeSomething() |
| 503 |
assert something.name is not None |
| 504 |
# ... |
| 505 |
|
| 506 |
one can create an equivalent test case instance as follows, with optional |
| 507 |
set-up and tear-down methods:: |
| 508 |
|
| 509 |
testcase = unittest.FunctionTestCase(testSomething, |
| 510 |
setUp=makeSomethingDB, |
| 511 |
tearDown=deleteSomethingDB) |
| 512 |
|
| 513 |
.. note:: |
| 514 |
|
| 515 |
Even though :class:`FunctionTestCase` can be used to quickly convert an |
| 516 |
existing test base over to a :mod:`!unittest`\ -based system, this approach is |
| 517 |
not recommended. Taking the time to set up proper :class:`TestCase` |
| 518 |
subclasses will make future test refactorings infinitely easier. |
| 519 |
|
| 520 |
In some cases, the existing tests may have been written using the :mod:`doctest` |
| 521 |
module. If so, :mod:`doctest` provides a :class:`~doctest.DocTestSuite` class that can |
| 522 |
automatically build :class:`unittest.TestSuite` instances from the existing |
| 523 |
:mod:`doctest`\ -based tests. |
| 524 |
|
| 525 |
|
| 526 |
.. _unittest-skipping: |
| 527 |
|
| 528 |
Skipping tests and expected failures |
| 529 |
------------------------------------ |
| 530 |
|
| 531 |
.. versionadded:: 3.1 |
| 532 |
|
| 533 |
Unittest supports skipping individual test methods and even whole classes of |
| 534 |
tests. In addition, it supports marking a test as an "expected failure," a test |
| 535 |
that is broken and will fail, but shouldn't be counted as a failure on a |
| 536 |
:class:`TestResult`. |
| 537 |
|
| 538 |
Skipping a test is simply a matter of using the :func:`skip` :term:`decorator` |
| 539 |
or one of its conditional variants, calling :meth:`TestCase.skipTest` within a |
| 540 |
:meth:`~TestCase.setUp` or test method, or raising :exc:`SkipTest` directly. |
| 541 |
|
| 542 |
Basic skipping looks like this:: |
| 543 |
|
| 544 |
class MyTestCase(unittest.TestCase): |
| 545 |
|
| 546 |
@unittest.skip("demonstrating skipping") |
| 547 |
def test_nothing(self): |
| 548 |
self.fail("shouldn't happen") |
| 549 |
|
| 550 |
@unittest.skipIf(mylib.__version__ < (1, 3), |
| 551 |
"not supported in this library version") |
| 552 |
def test_format(self): |
| 553 |
# Tests that work for only a certain version of the library. |
| 554 |
pass |
| 555 |
|
| 556 |
@unittest.skipUnless(sys.platform.startswith("win"), "requires Windows") |
| 557 |
def test_windows_support(self): |
| 558 |
# windows specific testing code |
| 559 |
pass |
| 560 |
|
| 561 |
def test_maybe_skipped(self): |
| 562 |
if not external_resource_available(): |
| 563 |
self.skipTest("external resource not available") |
| 564 |
# test code that depends on the external resource |
| 565 |
pass |
| 566 |
|
| 567 |
This is the output of running the example above in verbose mode:: |
| 568 |
|
| 569 |
test_format (__main__.MyTestCase.test_format) ... skipped 'not supported in this library version' |
| 570 |
test_nothing (__main__.MyTestCase.test_nothing) ... skipped 'demonstrating skipping' |
| 571 |
test_maybe_skipped (__main__.MyTestCase.test_maybe_skipped) ... skipped 'external resource not available' |
| 572 |
test_windows_support (__main__.MyTestCase.test_windows_support) ... skipped 'requires Windows' |
| 573 |
|
| 574 |
---------------------------------------------------------------------- |
| 575 |
Ran 4 tests in 0.005s |
| 576 |
|
| 577 |
OK (skipped=4) |
| 578 |
|
| 579 |
Classes can be skipped just like methods:: |
| 580 |
|
| 581 |
@unittest.skip("showing class skipping") |
| 582 |
class MySkippedTestCase(unittest.TestCase): |
| 583 |
def test_not_run(self): |
| 584 |
pass |
| 585 |
|
| 586 |
:meth:`TestCase.setUp` can also skip the test. This is useful when a resource |
| 587 |
that needs to be set up is not available. |
| 588 |
|
| 589 |
Expected failures use the :func:`expectedFailure` decorator. :: |
| 590 |
|
| 591 |
class ExpectedFailureTestCase(unittest.TestCase): |
| 592 |
@unittest.expectedFailure |
| 593 |
def test_fail(self): |
| 594 |
self.assertEqual(1, 0, "broken") |
| 595 |
|
| 596 |
It's easy to roll your own skipping decorators by making a decorator that calls |
| 597 |
:func:`skip` on the test when it wants it to be skipped. This decorator skips |
| 598 |
the test unless the passed object has a certain attribute:: |
| 599 |
|
| 600 |
def skipUnlessHasattr(obj, attr): |
| 601 |
if hasattr(obj, attr): |
| 602 |
return lambda func: func |
| 603 |
return unittest.skip("{!r} doesn't have {!r}".format(obj, attr)) |
| 604 |
|
| 605 |
The following decorators and exception implement test skipping and expected failures: |
| 606 |
|
| 607 |
.. decorator:: skip(reason) |
| 608 |
|
| 609 |
Unconditionally skip the decorated test. *reason* should describe why the |
| 610 |
test is being skipped. |
| 611 |
|
| 612 |
.. decorator:: skipIf(condition, reason) |
| 613 |
|
| 614 |
Skip the decorated test if *condition* is true. |
| 615 |
|
| 616 |
.. decorator:: skipUnless(condition, reason) |
| 617 |
|
| 618 |
Skip the decorated test unless *condition* is true. |
| 619 |
|
| 620 |
.. decorator:: expectedFailure |
| 621 |
|
| 622 |
Mark the test as an expected failure or error. If the test fails or errors |
| 623 |
in the test function itself (rather than in one of the :dfn:`test fixture` |
| 624 |
methods) then it will be considered a success. If the test passes, it will |
| 625 |
be considered a failure. |
| 626 |
|
| 627 |
.. exception:: SkipTest(reason) |
| 628 |
|
| 629 |
This exception is raised to skip a test. |
| 630 |
|
| 631 |
Usually you can use :meth:`TestCase.skipTest` or one of the skipping |
| 632 |
decorators instead of raising this directly. |
| 633 |
|
| 634 |
Skipped tests will not have :meth:`~TestCase.setUp` or :meth:`~TestCase.tearDown` run around them. |
| 635 |
Skipped classes will not have :meth:`~TestCase.setUpClass` or :meth:`~TestCase.tearDownClass` run. |
| 636 |
Skipped modules will not have :func:`setUpModule` or :func:`tearDownModule` run. |
| 637 |
|
| 638 |
|
| 639 |
.. _subtests: |
| 640 |
|
| 641 |
Distinguishing test iterations using subtests |
| 642 |
--------------------------------------------- |
| 643 |
|
| 644 |
.. versionadded:: 3.4 |
| 645 |
|
| 646 |
When there are very small differences among your tests, for |
| 647 |
instance some parameters, unittest allows you to distinguish them inside |
| 648 |
the body of a test method using the :meth:`~TestCase.subTest` context manager. |
| 649 |
|
| 650 |
For example, the following test:: |
| 651 |
|
| 652 |
class NumbersTest(unittest.TestCase): |
| 653 |
|
| 654 |
def test_even(self): |
| 655 |
""" |
| 656 |
Test that numbers between 0 and 5 are all even. |
| 657 |
""" |
| 658 |
for i in range(0, 6): |
| 659 |
with self.subTest(i=i): |
| 660 |
self.assertEqual(i % 2, 0) |
| 661 |
|
| 662 |
will produce the following output:: |
| 663 |
|
| 664 |
====================================================================== |
| 665 |
FAIL: test_even (__main__.NumbersTest.test_even) (i=1) |
| 666 |
Test that numbers between 0 and 5 are all even. |
| 667 |
---------------------------------------------------------------------- |
| 668 |
Traceback (most recent call last): |
| 669 |
File "subtests.py", line 11, in test_even |
| 670 |
self.assertEqual(i % 2, 0) |
| 671 |
^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 672 |
AssertionError: 1 != 0 |
| 673 |
|
| 674 |
====================================================================== |
| 675 |
FAIL: test_even (__main__.NumbersTest.test_even) (i=3) |
| 676 |
Test that numbers between 0 and 5 are all even. |
| 677 |
---------------------------------------------------------------------- |
| 678 |
Traceback (most recent call last): |
| 679 |
File "subtests.py", line 11, in test_even |
| 680 |
self.assertEqual(i % 2, 0) |
| 681 |
^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 682 |
AssertionError: 1 != 0 |
| 683 |
|
| 684 |
====================================================================== |
| 685 |
FAIL: test_even (__main__.NumbersTest.test_even) (i=5) |
| 686 |
Test that numbers between 0 and 5 are all even. |
| 687 |
---------------------------------------------------------------------- |
| 688 |
Traceback (most recent call last): |
| 689 |
File "subtests.py", line 11, in test_even |
| 690 |
self.assertEqual(i % 2, 0) |
| 691 |
^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 692 |
AssertionError: 1 != 0 |
| 693 |
|
| 694 |
Without using a subtest, execution would stop after the first failure, |
| 695 |
and the error would be less easy to diagnose because the value of ``i`` |
| 696 |
wouldn't be displayed:: |
| 697 |
|
| 698 |
====================================================================== |
| 699 |
FAIL: test_even (__main__.NumbersTest.test_even) |
| 700 |
---------------------------------------------------------------------- |
| 701 |
Traceback (most recent call last): |
| 702 |
File "subtests.py", line 32, in test_even |
| 703 |
self.assertEqual(i % 2, 0) |
| 704 |
AssertionError: 1 != 0 |
| 705 |
|
| 706 |
|
| 707 |
.. _unittest-contents: |
| 708 |
|
| 709 |
Classes and functions |
| 710 |
--------------------- |
| 711 |
|
| 712 |
This section describes in depth the API of :mod:`!unittest`. |
| 713 |
|
| 714 |
|
| 715 |
.. _testcase-objects: |
| 716 |
|
| 717 |
Test cases |
| 718 |
~~~~~~~~~~ |
| 719 |
|
| 720 |
.. class:: TestCase(methodName='runTest') |
| 721 |
|
| 722 |
Instances of the :class:`TestCase` class represent the logical test units |
| 723 |
in the :mod:`!unittest` universe. This class is intended to be used as a base |
| 724 |
class, with specific tests being implemented by concrete subclasses. This class |
| 725 |
implements the interface needed by the test runner to allow it to drive the |
| 726 |
tests, and methods that the test code can use to check for and report various |
| 727 |
kinds of failure. |
| 728 |
|
| 729 |
Each instance of :class:`TestCase` will run a single base method: the method |
| 730 |
named *methodName*. |
| 731 |
In most uses of :class:`TestCase`, you will neither change |
| 732 |
the *methodName* nor reimplement the default ``runTest()`` method. |
| 733 |
|
| 734 |
.. versionchanged:: 3.2 |
| 735 |
:class:`TestCase` can be instantiated successfully without providing a |
| 736 |
*methodName*. This makes it easier to experiment with :class:`TestCase` |
| 737 |
from the interactive interpreter. |
| 738 |
|
| 739 |
:class:`TestCase` instances provide three groups of methods: one group used |
| 740 |
to run the test, another used by the test implementation to check conditions |
| 741 |
and report failures, and some inquiry methods allowing information about the |
| 742 |
test itself to be gathered. |
| 743 |
|
| 744 |
Methods in the first group (running the test) are: |
| 745 |
|
| 746 |
.. method:: setUp() |
| 747 |
|
| 748 |
Method called to prepare the test fixture. This is called immediately |
| 749 |
before calling the test method; other than :exc:`AssertionError` or :exc:`SkipTest`, |
| 750 |
any exception raised by this method will be considered an error rather than |
| 751 |
a test failure. The default implementation does nothing. |
| 752 |
|
| 753 |
|
| 754 |
.. method:: tearDown() |
| 755 |
|
| 756 |
Method called immediately after the test method has been called and the |
| 757 |
result recorded. This is called even if the test method raised an |
| 758 |
exception, so the implementation in subclasses may need to be particularly |
| 759 |
careful about checking internal state. Any exception, other than |
| 760 |
:exc:`AssertionError` or :exc:`SkipTest`, raised by this method will be |
| 761 |
considered an additional error rather than a test failure (thus increasing |
| 762 |
the total number of reported errors). This method will only be called if |
| 763 |
the :meth:`setUp` succeeds, regardless of the outcome of the test method. |
| 764 |
The default implementation does nothing. |
| 765 |
|
| 766 |
|
| 767 |
.. method:: setUpClass() |
| 768 |
|
| 769 |
A class method called before tests in an individual class are run. |
| 770 |
``setUpClass`` is called with the class as the only argument |
| 771 |
and must be decorated as a :func:`classmethod`:: |
| 772 |
|
| 773 |
@classmethod |
| 774 |
def setUpClass(cls): |
| 775 |
... |
| 776 |
|
| 777 |
See `Class and Module Fixtures`_ for more details. |
| 778 |
|
| 779 |
.. versionadded:: 3.2 |
| 780 |
|
| 781 |
|
| 782 |
.. method:: tearDownClass() |
| 783 |
|
| 784 |
A class method called after tests in an individual class have run. |
| 785 |
``tearDownClass`` is called with the class as the only argument |
| 786 |
and must be decorated as a :meth:`classmethod`:: |
| 787 |
|
| 788 |
@classmethod |
| 789 |
def tearDownClass(cls): |
| 790 |
... |
| 791 |
|
| 792 |
See `Class and Module Fixtures`_ for more details. |
| 793 |
|
| 794 |
.. versionadded:: 3.2 |
| 795 |
|
| 796 |
|
| 797 |
.. method:: run(result=None) |
| 798 |
|
| 799 |
Run the test, collecting the result into the :class:`TestResult` object |
| 800 |
passed as *result*. If *result* is omitted or ``None``, a temporary |
| 801 |
result object is created (by calling the :meth:`defaultTestResult` |
| 802 |
method) and used. The result object is returned to :meth:`run`'s |
| 803 |
caller. |
| 804 |
|
| 805 |
The same effect may be had by simply calling the :class:`TestCase` |
| 806 |
instance. |
| 807 |
|
| 808 |
.. versionchanged:: 3.3 |
| 809 |
Previous versions of ``run`` did not return the result. Neither did |
| 810 |
calling an instance. |
| 811 |
|
| 812 |
.. method:: skipTest(reason) |
| 813 |
|
| 814 |
Calling this during a test method or :meth:`setUp` skips the current |
| 815 |
test. See :ref:`unittest-skipping` for more information. |
| 816 |
|
| 817 |
.. versionadded:: 3.1 |
| 818 |
|
| 819 |
|
| 820 |
.. method:: subTest(msg=None, **params) |
| 821 |
|
| 822 |
Return a context manager which executes the enclosed code block as a |
| 823 |
subtest. *msg* and *params* are optional, arbitrary values which are |
| 824 |
displayed whenever a subtest fails, allowing you to identify them |
| 825 |
clearly. |
| 826 |
|
| 827 |
A test case can contain any number of subtest declarations, and |
| 828 |
they can be arbitrarily nested. |
| 829 |
|
| 830 |
See :ref:`subtests` for more information. |
| 831 |
|
| 832 |
.. versionadded:: 3.4 |
| 833 |
|
| 834 |
|
| 835 |
.. method:: debug() |
| 836 |
|
| 837 |
Run the test without collecting the result. This allows exceptions raised |
| 838 |
by the test to be propagated to the caller, and can be used to support |
| 839 |
running tests under a debugger. |
| 840 |
|
| 841 |
.. _assert-methods: |
| 842 |
|
| 843 |
The :class:`TestCase` class provides several assert methods to check for and |
| 844 |
report failures. The following table lists the most commonly used methods |
| 845 |
(see the tables below for more assert methods): |
| 846 |
|
| 847 |
+-----------------------------------------+-----------------------------+---------------+ |
| 848 |
| Method | Checks that | New in | |
| 849 |
+=========================================+=============================+===============+ |
| 850 |
| :meth:`assertEqual(a, b) | ``a == b`` | | |
| 851 |
| <TestCase.assertEqual>` | | | |
| 852 |
+-----------------------------------------+-----------------------------+---------------+ |
| 853 |
| :meth:`assertNotEqual(a, b) | ``a != b`` | | |
| 854 |
| <TestCase.assertNotEqual>` | | | |
| 855 |
+-----------------------------------------+-----------------------------+---------------+ |
| 856 |
| :meth:`assertTrue(x) | ``bool(x) is True`` | | |
| 857 |
| <TestCase.assertTrue>` | | | |
| 858 |
+-----------------------------------------+-----------------------------+---------------+ |
| 859 |
| :meth:`assertFalse(x) | ``bool(x) is False`` | | |
| 860 |
| <TestCase.assertFalse>` | | | |
| 861 |
+-----------------------------------------+-----------------------------+---------------+ |
| 862 |
| :meth:`assertIs(a, b) | ``a is b`` | 3.1 | |
| 863 |
| <TestCase.assertIs>` | | | |
| 864 |
+-----------------------------------------+-----------------------------+---------------+ |
| 865 |
| :meth:`assertIsNot(a, b) | ``a is not b`` | 3.1 | |
| 866 |
| <TestCase.assertIsNot>` | | | |
| 867 |
+-----------------------------------------+-----------------------------+---------------+ |
| 868 |
| :meth:`assertIsNone(x) | ``x is None`` | 3.1 | |
| 869 |
| <TestCase.assertIsNone>` | | | |
| 870 |
+-----------------------------------------+-----------------------------+---------------+ |
| 871 |
| :meth:`assertIsNotNone(x) | ``x is not None`` | 3.1 | |
| 872 |
| <TestCase.assertIsNotNone>` | | | |
| 873 |
+-----------------------------------------+-----------------------------+---------------+ |
| 874 |
| :meth:`assertIn(a, b) | ``a in b`` | 3.1 | |
| 875 |
| <TestCase.assertIn>` | | | |
| 876 |
+-----------------------------------------+-----------------------------+---------------+ |
| 877 |
| :meth:`assertNotIn(a, b) | ``a not in b`` | 3.1 | |
| 878 |
| <TestCase.assertNotIn>` | | | |
| 879 |
+-----------------------------------------+-----------------------------+---------------+ |
| 880 |
| :meth:`assertIsInstance(a, b) | ``isinstance(a, b)`` | 3.2 | |
| 881 |
| <TestCase.assertIsInstance>` | | | |
| 882 |
+-----------------------------------------+-----------------------------+---------------+ |
| 883 |
| :meth:`assertNotIsInstance(a, b) | ``not isinstance(a, b)`` | 3.2 | |
| 884 |
| <TestCase.assertNotIsInstance>` | | | |
| 885 |
+-----------------------------------------+-----------------------------+---------------+ |
| 886 |
| :meth:`assertIsSubclass(a, b) | ``issubclass(a, b)`` | 3.14 | |
| 887 |
| <TestCase.assertIsSubclass>` | | | |
| 888 |
+-----------------------------------------+-----------------------------+---------------+ |
| 889 |
| :meth:`assertNotIsSubclass(a, b) | ``not issubclass(a, b)`` | 3.14 | |
| 890 |
| <TestCase.assertNotIsSubclass>` | | | |
| 891 |
+-----------------------------------------+-----------------------------+---------------+ |
| 892 |
|
| 893 |
All the assert methods accept a *msg* argument that, if specified, is used |
| 894 |
as the error message on failure (see also :data:`longMessage`). |
| 895 |
Note that the *msg* keyword argument can be passed to :meth:`assertRaises`, |
| 896 |
:meth:`assertRaisesRegex`, :meth:`assertWarns`, :meth:`assertWarnsRegex` |
| 897 |
only when they are used as a context manager. |
| 898 |
|
| 899 |
.. method:: assertEqual(first, second, msg=None) |
| 900 |
|
| 901 |
Test that *first* and *second* are equal. If the values do not |
| 902 |
compare equal, the test will fail. |
| 903 |
|
| 904 |
In addition, if *first* and *second* are the exact same type and one of |
| 905 |
list, tuple, dict, set, frozenset or str or any type that a subclass |
| 906 |
registers with :meth:`addTypeEqualityFunc` the type-specific equality |
| 907 |
function will be called in order to generate a more useful default |
| 908 |
error message (see also the :ref:`list of type-specific methods |
| 909 |
<type-specific-methods>`). |
| 910 |
|
| 911 |
.. versionchanged:: 3.1 |
| 912 |
Added the automatic calling of type-specific equality function. |
| 913 |
|
| 914 |
.. versionchanged:: 3.2 |
| 915 |
:meth:`assertMultiLineEqual` added as the default type equality |
| 916 |
function for comparing strings. |
| 917 |
|
| 918 |
|
| 919 |
.. method:: assertNotEqual(first, second, msg=None) |
| 920 |
|
| 921 |
Test that *first* and *second* are not equal. If the values do |
| 922 |
compare equal, the test will fail. |
| 923 |
|
| 924 |
.. method:: assertTrue(expr, msg=None) |
| 925 |
assertFalse(expr, msg=None) |
| 926 |
|
| 927 |
Test that *expr* is true (or false). |
| 928 |
|
| 929 |
Note that this is equivalent to ``bool(expr) is True`` and not to ``expr |
| 930 |
is True`` (use ``assertIs(expr, True)`` for the latter). This method |
| 931 |
should also be avoided when more specific methods are available (e.g. |
| 932 |
``assertEqual(a, b)`` instead of ``assertTrue(a == b)``), because they |
| 933 |
provide a better error message in case of failure. |
| 934 |
|
| 935 |
|
| 936 |
.. method:: assertIs(first, second, msg=None) |
| 937 |
assertIsNot(first, second, msg=None) |
| 938 |
|
| 939 |
Test that *first* and *second* are (or are not) the same object. |
| 940 |
|
| 941 |
.. versionadded:: 3.1 |
| 942 |
|
| 943 |
|
| 944 |
.. method:: assertIsNone(expr, msg=None) |
| 945 |
assertIsNotNone(expr, msg=None) |
| 946 |
|
| 947 |
Test that *expr* is (or is not) ``None``. |
| 948 |
|
| 949 |
.. versionadded:: 3.1 |
| 950 |
|
| 951 |
|
| 952 |
.. method:: assertIn(member, container, msg=None) |
| 953 |
assertNotIn(member, container, msg=None) |
| 954 |
|
| 955 |
Test that *member* is (or is not) in *container*. |
| 956 |
|
| 957 |
.. versionadded:: 3.1 |
| 958 |
|
| 959 |
|
| 960 |
.. method:: assertIsInstance(obj, cls, msg=None) |
| 961 |
assertNotIsInstance(obj, cls, msg=None) |
| 962 |
|
| 963 |
Test that *obj* is (or is not) an instance of *cls* (which can be a |
| 964 |
class or a tuple of classes, as supported by :func:`isinstance`). |
| 965 |
To check for the exact type, use :func:`assertIs(type(obj), cls) <assertIs>`. |
| 966 |
|
| 967 |
.. versionadded:: 3.2 |
| 968 |
|
| 969 |
|
| 970 |
.. method:: assertIsSubclass(cls, superclass, msg=None) |
| 971 |
assertNotIsSubclass(cls, superclass, msg=None) |
| 972 |
|
| 973 |
Test that *cls* is (or is not) a subclass of *superclass* (which can be a |
| 974 |
class or a tuple of classes, as supported by :func:`issubclass`). |
| 975 |
To check for the exact type, use :func:`assertIs(cls, superclass) <assertIs>`. |
| 976 |
|
| 977 |
.. versionadded:: 3.14 |
| 978 |
|
| 979 |
|
| 980 |
It is also possible to check the production of exceptions, warnings, and |
| 981 |
log messages using the following methods: |
| 982 |
|
| 983 |
+---------------------------------------------------------+--------------------------------------+------------+ |
| 984 |
| Method | Checks that | New in | |
| 985 |
+=========================================================+======================================+============+ |
| 986 |
| :meth:`assertRaises(exc, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises *exc* | | |
| 987 |
| <TestCase.assertRaises>` | | | |
| 988 |
+---------------------------------------------------------+--------------------------------------+------------+ |
| 989 |
| :meth:`assertRaisesRegex(exc, r, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises *exc* | 3.1 | |
| 990 |
| <TestCase.assertRaisesRegex>` | and the message matches regex *r* | | |
| 991 |
+---------------------------------------------------------+--------------------------------------+------------+ |
| 992 |
| :meth:`assertWarns(warn, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises *warn* | 3.2 | |
| 993 |
| <TestCase.assertWarns>` | | | |
| 994 |
+---------------------------------------------------------+--------------------------------------+------------+ |
| 995 |
| :meth:`assertWarnsRegex(warn, r, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises *warn* | 3.2 | |
| 996 |
| <TestCase.assertWarnsRegex>` | and the message matches regex *r* | | |
| 997 |
+---------------------------------------------------------+--------------------------------------+------------+ |
| 998 |
| :meth:`assertLogs(logger, level) | The ``with`` block logs on *logger* | 3.4 | |
| 999 |
| <TestCase.assertLogs>` | with minimum *level* | | |
| 1000 |
+---------------------------------------------------------+--------------------------------------+------------+ |
| 1001 |
| :meth:`assertNoLogs(logger, level) | The ``with`` block does not log on | 3.10 | |
| 1002 |
| <TestCase.assertNoLogs>` | *logger* with minimum *level* | | |
| 1003 |
+---------------------------------------------------------+--------------------------------------+------------+ |
| 1004 |
|
| 1005 |
.. method:: assertRaises(exception, callable, *args, **kwds) |
| 1006 |
assertRaises(exception, *, msg=None) |
| 1007 |
|
| 1008 |
Test that an exception is raised when *callable* is called with any |
| 1009 |
positional or keyword arguments that are also passed to |
| 1010 |
:meth:`assertRaises`. The test passes if *exception* is raised, is an |
| 1011 |
error if another exception is raised, or fails if no exception is raised. |
| 1012 |
To catch any of a group of exceptions, a tuple containing the exception |
| 1013 |
classes may be passed as *exception*. |
| 1014 |
|
| 1015 |
If only the *exception* and possibly the *msg* arguments are given, |
| 1016 |
return a context manager so that the code under test can be written |
| 1017 |
inline rather than as a function:: |
| 1018 |
|
| 1019 |
with self.assertRaises(SomeException): |
| 1020 |
do_something() |
| 1021 |
|
| 1022 |
When used as a context manager, :meth:`assertRaises` accepts the |
| 1023 |
additional keyword argument *msg*. |
| 1024 |
|
| 1025 |
The context manager will store the caught exception object in its |
| 1026 |
:attr:`!exception` attribute. This can be useful if the intention |
| 1027 |
is to perform additional checks on the exception raised:: |
| 1028 |
|
| 1029 |
with self.assertRaises(SomeException) as cm: |
| 1030 |
do_something() |
| 1031 |
|
| 1032 |
the_exception = cm.exception |
| 1033 |
self.assertEqual(the_exception.error_code, 3) |
| 1034 |
|
| 1035 |
.. versionchanged:: 3.1 |
| 1036 |
Added the ability to use :meth:`assertRaises` as a context manager. |
| 1037 |
|
| 1038 |
.. versionchanged:: 3.2 |
| 1039 |
Added the :attr:`!exception` attribute. |
| 1040 |
|
| 1041 |
.. versionchanged:: 3.3 |
| 1042 |
Added the *msg* keyword argument when used as a context manager. |
| 1043 |
|
| 1044 |
|
| 1045 |
.. method:: assertRaisesRegex(exception, regex, callable, *args, **kwds) |
| 1046 |
assertRaisesRegex(exception, regex, *, msg=None) |
| 1047 |
|
| 1048 |
Like :meth:`assertRaises` but also tests that *regex* matches |
| 1049 |
on the string representation of the raised exception. *regex* may be |
| 1050 |
a regular expression object or a string containing a regular expression |
| 1051 |
suitable for use by :func:`re.search`. Examples:: |
| 1052 |
|
| 1053 |
self.assertRaisesRegex(ValueError, "invalid literal for.*XYZ'$", |
| 1054 |
int, 'XYZ') |
| 1055 |
|
| 1056 |
or:: |
| 1057 |
|
| 1058 |
with self.assertRaisesRegex(ValueError, 'literal'): |
| 1059 |
int('XYZ') |
| 1060 |
|
| 1061 |
.. versionadded:: 3.1 |
| 1062 |
Added under the name ``assertRaisesRegexp``. |
| 1063 |
|
| 1064 |
.. versionchanged:: 3.2 |
| 1065 |
Renamed to :meth:`assertRaisesRegex`. |
| 1066 |
|
| 1067 |
.. versionchanged:: 3.3 |
| 1068 |
Added the *msg* keyword argument when used as a context manager. |
| 1069 |
|
| 1070 |
|
| 1071 |
.. method:: assertWarns(warning, callable, *args, **kwds) |
| 1072 |
assertWarns(warning, *, msg=None) |
| 1073 |
|
| 1074 |
Test that a warning is triggered when *callable* is called with any |
| 1075 |
positional or keyword arguments that are also passed to |
| 1076 |
:meth:`assertWarns`. The test passes if *warning* is triggered and |
| 1077 |
fails if it isn't. Any exception is an error. |
| 1078 |
To catch any of a group of warnings, a tuple containing the warning |
| 1079 |
classes may be passed as *warnings*. |
| 1080 |
|
| 1081 |
If only the *warning* and possibly the *msg* arguments are given, |
| 1082 |
return a context manager so that the code under test can be written |
| 1083 |
inline rather than as a function:: |
| 1084 |
|
| 1085 |
with self.assertWarns(SomeWarning): |
| 1086 |
do_something() |
| 1087 |
|
| 1088 |
When used as a context manager, :meth:`assertWarns` accepts the |
| 1089 |
additional keyword argument *msg*. |
| 1090 |
|
| 1091 |
The context manager will store the caught warning object in its |
| 1092 |
:attr:`!warning` attribute, and the source line which triggered the |
| 1093 |
warnings in the :attr:`!filename` and :attr:`!lineno` attributes. |
| 1094 |
This can be useful if the intention is to perform additional checks |
| 1095 |
on the warning caught:: |
| 1096 |
|
| 1097 |
with self.assertWarns(SomeWarning) as cm: |
| 1098 |
do_something() |
| 1099 |
|
| 1100 |
self.assertIn('myfile.py', cm.filename) |
| 1101 |
self.assertEqual(320, cm.lineno) |
| 1102 |
|
| 1103 |
This method works regardless of the warning filters in place when it |
| 1104 |
is called. |
| 1105 |
|
| 1106 |
.. versionadded:: 3.2 |
| 1107 |
|
| 1108 |
.. versionchanged:: 3.3 |
| 1109 |
Added the *msg* keyword argument when used as a context manager. |
| 1110 |
|
| 1111 |
|
| 1112 |
.. method:: assertWarnsRegex(warning, regex, callable, *args, **kwds) |
| 1113 |
assertWarnsRegex(warning, regex, *, msg=None) |
| 1114 |
|
| 1115 |
Like :meth:`assertWarns` but also tests that *regex* matches on the |
| 1116 |
message of the triggered warning. *regex* may be a regular expression |
| 1117 |
object or a string containing a regular expression suitable for use |
| 1118 |
by :func:`re.search`. Example:: |
| 1119 |
|
| 1120 |
self.assertWarnsRegex(DeprecationWarning, |
| 1121 |
r'legacy_function\(\) is deprecated', |
| 1122 |
legacy_function, 'XYZ') |
| 1123 |
|
| 1124 |
or:: |
| 1125 |
|
| 1126 |
with self.assertWarnsRegex(RuntimeWarning, 'unsafe frobnicating'): |
| 1127 |
frobnicate('/etc/passwd') |
| 1128 |
|
| 1129 |
.. versionadded:: 3.2 |
| 1130 |
|
| 1131 |
.. versionchanged:: 3.3 |
| 1132 |
Added the *msg* keyword argument when used as a context manager. |
| 1133 |
|
| 1134 |
.. method:: assertLogs(logger=None, level=None) |
| 1135 |
|
| 1136 |
A context manager to test that at least one message is logged on |
| 1137 |
the *logger* or one of its children, with at least the given |
| 1138 |
*level*. |
| 1139 |
|
| 1140 |
If given, *logger* should be a :class:`logging.Logger` object or a |
| 1141 |
:class:`str` giving the name of a logger. The default is the root |
| 1142 |
logger, which will catch all messages that were not blocked by a |
| 1143 |
non-propagating descendent logger. |
| 1144 |
|
| 1145 |
If given, *level* should be either a numeric logging level or |
| 1146 |
its string equivalent (for example either ``"ERROR"`` or |
| 1147 |
:const:`logging.ERROR`). The default is :const:`logging.INFO`. |
| 1148 |
|
| 1149 |
The test passes if at least one message emitted inside the ``with`` |
| 1150 |
block matches the *logger* and *level* conditions, otherwise it fails. |
| 1151 |
|
| 1152 |
The object returned by the context manager is a recording helper |
| 1153 |
which keeps tracks of the matching log messages. It has two |
| 1154 |
attributes: |
| 1155 |
|
| 1156 |
.. attribute:: records |
| 1157 |
|
| 1158 |
A list of :class:`logging.LogRecord` objects of the matching |
| 1159 |
log messages. |
| 1160 |
|
| 1161 |
.. attribute:: output |
| 1162 |
|
| 1163 |
A list of :class:`str` objects with the formatted output of |
| 1164 |
matching messages. |
| 1165 |
|
| 1166 |
Example:: |
| 1167 |
|
| 1168 |
with self.assertLogs('foo', level='INFO') as cm: |
| 1169 |
logging.getLogger('foo').info('first message') |
| 1170 |
logging.getLogger('foo.bar').error('second message') |
| 1171 |
self.assertEqual(cm.output, ['INFO:foo:first message', |
| 1172 |
'ERROR:foo.bar:second message']) |
| 1173 |
|
| 1174 |
.. versionadded:: 3.4 |
| 1175 |
|
| 1176 |
.. method:: assertNoLogs(logger=None, level=None) |
| 1177 |
|
| 1178 |
A context manager to test that no messages are logged on |
| 1179 |
the *logger* or one of its children, with at least the given |
| 1180 |
*level*. |
| 1181 |
|
| 1182 |
If given, *logger* should be a :class:`logging.Logger` object or a |
| 1183 |
:class:`str` giving the name of a logger. The default is the root |
| 1184 |
logger, which will catch all messages. |
| 1185 |
|
| 1186 |
If given, *level* should be either a numeric logging level or |
| 1187 |
its string equivalent (for example either ``"ERROR"`` or |
| 1188 |
:const:`logging.ERROR`). The default is :const:`logging.INFO`. |
| 1189 |
|
| 1190 |
Unlike :meth:`assertLogs`, nothing will be returned by the context |
| 1191 |
manager. |
| 1192 |
|
| 1193 |
.. versionadded:: 3.10 |
| 1194 |
|
| 1195 |
There are also other methods used to perform more specific checks, such as: |
| 1196 |
|
| 1197 |
+---------------------------------------+--------------------------------+--------------+ |
| 1198 |
| Method | Checks that | New in | |
| 1199 |
+=======================================+================================+==============+ |
| 1200 |
| :meth:`assertAlmostEqual(a, b) | ``round(a-b, 7) == 0`` | | |
| 1201 |
| <TestCase.assertAlmostEqual>` | | | |
| 1202 |
+---------------------------------------+--------------------------------+--------------+ |
| 1203 |
| :meth:`assertNotAlmostEqual(a, b) | ``round(a-b, 7) != 0`` | | |
| 1204 |
| <TestCase.assertNotAlmostEqual>` | | | |
| 1205 |
+---------------------------------------+--------------------------------+--------------+ |
| 1206 |
| :meth:`assertGreater(a, b) | ``a > b`` | 3.1 | |
| 1207 |
| <TestCase.assertGreater>` | | | |
| 1208 |
+---------------------------------------+--------------------------------+--------------+ |
| 1209 |
| :meth:`assertGreaterEqual(a, b) | ``a >= b`` | 3.1 | |
| 1210 |
| <TestCase.assertGreaterEqual>` | | | |
| 1211 |
+---------------------------------------+--------------------------------+--------------+ |
| 1212 |
| :meth:`assertLess(a, b) | ``a < b`` | 3.1 | |
| 1213 |
| <TestCase.assertLess>` | | | |
| 1214 |
+---------------------------------------+--------------------------------+--------------+ |
| 1215 |
| :meth:`assertLessEqual(a, b) | ``a <= b`` | 3.1 | |
| 1216 |
| <TestCase.assertLessEqual>` | | | |
| 1217 |
+---------------------------------------+--------------------------------+--------------+ |
| 1218 |
| :meth:`assertRegex(s, r) | ``r.search(s)`` | 3.1 | |
| 1219 |
| <TestCase.assertRegex>` | | | |
| 1220 |
+---------------------------------------+--------------------------------+--------------+ |
| 1221 |
| :meth:`assertNotRegex(s, r) | ``not r.search(s)`` | 3.2 | |
| 1222 |
| <TestCase.assertNotRegex>` | | | |
| 1223 |
+---------------------------------------+--------------------------------+--------------+ |
| 1224 |
| :meth:`assertCountEqual(a, b) | *a* contains the same elements | 3.2 | |
| 1225 |
| <TestCase.assertCountEqual>` | as *b*, regardless of their | | |
| 1226 |
| | order. | | |
| 1227 |
+---------------------------------------+--------------------------------+--------------+ |
| 1228 |
| :meth:`assertStartsWith(a, b) | ``a.startswith(b)`` | 3.14 | |
| 1229 |
| <TestCase.assertStartsWith>` | | | |
| 1230 |
+---------------------------------------+--------------------------------+--------------+ |
| 1231 |
| :meth:`assertNotStartsWith(a, b) | ``not a.startswith(b)`` | 3.14 | |
| 1232 |
| <TestCase.assertNotStartsWith>` | | | |
| 1233 |
+---------------------------------------+--------------------------------+--------------+ |
| 1234 |
| :meth:`assertEndsWith(a, b) | ``a.endswith(b)`` | 3.14 | |
| 1235 |
| <TestCase.assertEndsWith>` | | | |
| 1236 |
+---------------------------------------+--------------------------------+--------------+ |
| 1237 |
| :meth:`assertNotEndsWith(a, b) | ``not a.endswith(b)`` | 3.14 | |
| 1238 |
| <TestCase.assertNotEndsWith>` | | | |
| 1239 |
+---------------------------------------+--------------------------------+--------------+ |
| 1240 |
| :meth:`assertHasAttr(a, b) | ``hasattr(a, b)`` | 3.14 | |
| 1241 |
| <TestCase.assertHasAttr>` | | | |
| 1242 |
+---------------------------------------+--------------------------------+--------------+ |
| 1243 |
| :meth:`assertNotHasAttr(a, b) | ``not hasattr(a, b)`` | 3.14 | |
| 1244 |
| <TestCase.assertNotHasAttr>` | | | |
| 1245 |
+---------------------------------------+--------------------------------+--------------+ |
| 1246 |
|
| 1247 |
|
| 1248 |
.. method:: assertAlmostEqual(first, second, places=7, msg=None, delta=None) |
| 1249 |
assertNotAlmostEqual(first, second, places=7, msg=None, delta=None) |
| 1250 |
|
| 1251 |
Test that *first* and *second* are approximately (or not approximately) |
| 1252 |
equal by computing the difference, rounding to the given number of |
| 1253 |
decimal *places* (default 7), and comparing to zero. Note that these |
| 1254 |
methods round the values to the given number of *decimal places* (i.e. |
| 1255 |
like the :func:`round` function) and not *significant digits*. |
| 1256 |
|
| 1257 |
If *delta* is supplied instead of *places* then the difference |
| 1258 |
between *first* and *second* must be less or equal to (or greater than) *delta*. |
| 1259 |
|
| 1260 |
Supplying both *delta* and *places* raises a :exc:`TypeError`. |
| 1261 |
|
| 1262 |
.. versionchanged:: 3.2 |
| 1263 |
:meth:`assertAlmostEqual` automatically considers almost equal objects |
| 1264 |
that compare equal. :meth:`assertNotAlmostEqual` automatically fails |
| 1265 |
if the objects compare equal. Added the *delta* keyword argument. |
| 1266 |
|
| 1267 |
|
| 1268 |
.. method:: assertGreater(first, second, msg=None) |
| 1269 |
assertGreaterEqual(first, second, msg=None) |
| 1270 |
assertLess(first, second, msg=None) |
| 1271 |
assertLessEqual(first, second, msg=None) |
| 1272 |
|
| 1273 |
Test that *first* is respectively >, >=, < or <= than *second* depending |
| 1274 |
on the method name. If not, the test will fail:: |
| 1275 |
|
| 1276 |
>>> self.assertGreaterEqual(3, 4) |
| 1277 |
AssertionError: "3" unexpectedly not greater than or equal to "4" |
| 1278 |
|
| 1279 |
.. versionadded:: 3.1 |
| 1280 |
|
| 1281 |
|
| 1282 |
.. method:: assertRegex(text, regex, msg=None) |
| 1283 |
assertNotRegex(text, regex, msg=None) |
| 1284 |
|
| 1285 |
Test that a *regex* search matches (or does not match) *text*. In case |
| 1286 |
of failure, the error message will include the pattern and the *text* (or |
| 1287 |
the pattern and the part of *text* that unexpectedly matched). *regex* |
| 1288 |
may be a regular expression object or a string containing a regular |
| 1289 |
expression suitable for use by :func:`re.search`. |
| 1290 |
|
| 1291 |
.. versionadded:: 3.1 |
| 1292 |
Added under the name ``assertRegexpMatches``. |
| 1293 |
.. versionchanged:: 3.2 |
| 1294 |
The method ``assertRegexpMatches()`` has been renamed to |
| 1295 |
:meth:`.assertRegex`. |
| 1296 |
.. versionadded:: 3.2 |
| 1297 |
:meth:`.assertNotRegex`. |
| 1298 |
|
| 1299 |
|
| 1300 |
.. method:: assertCountEqual(first, second, msg=None) |
| 1301 |
|
| 1302 |
Test that sequence *first* contains the same elements as *second*, |
| 1303 |
regardless of their order. When they don't, an error message listing the |
| 1304 |
differences between the sequences will be generated. |
| 1305 |
|
| 1306 |
Duplicate elements are *not* ignored when comparing *first* and |
| 1307 |
*second*. It verifies whether each element has the same count in both |
| 1308 |
sequences. Equivalent to: |
| 1309 |
``assertEqual(Counter(list(first)), Counter(list(second)))`` |
| 1310 |
but works with sequences of unhashable objects as well. |
| 1311 |
|
| 1312 |
.. versionadded:: 3.2 |
| 1313 |
|
| 1314 |
|
| 1315 |
.. method:: assertStartsWith(s, prefix, msg=None) |
| 1316 |
.. method:: assertNotStartsWith(s, prefix, msg=None) |
| 1317 |
|
| 1318 |
Test that the Unicode or byte string *s* starts (or does not start) |
| 1319 |
with a *prefix*. |
| 1320 |
*prefix* can also be a tuple of strings to try. |
| 1321 |
|
| 1322 |
.. versionadded:: 3.14 |
| 1323 |
|
| 1324 |
|
| 1325 |
.. method:: assertEndsWith(s, suffix, msg=None) |
| 1326 |
.. method:: assertNotEndsWith(s, suffix, msg=None) |
| 1327 |
|
| 1328 |
Test that the Unicode or byte string *s* ends (or does not end) |
| 1329 |
with a *suffix*. |
| 1330 |
*suffix* can also be a tuple of strings to try. |
| 1331 |
|
| 1332 |
.. versionadded:: 3.14 |
| 1333 |
|
| 1334 |
|
| 1335 |
.. method:: assertHasAttr(obj, name, msg=None) |
| 1336 |
.. method:: assertNotHasAttr(obj, name, msg=None) |
| 1337 |
|
| 1338 |
Test that the object *obj* has (or has not) an attribute *name*. |
| 1339 |
|
| 1340 |
.. versionadded:: 3.14 |
| 1341 |
|
| 1342 |
|
| 1343 |
.. _type-specific-methods: |
| 1344 |
|
| 1345 |
The :meth:`assertEqual` method dispatches the equality check for objects of |
| 1346 |
the same type to different type-specific methods. These methods are already |
| 1347 |
implemented for most of the built-in types, but it's also possible to |
| 1348 |
register new methods using :meth:`addTypeEqualityFunc`: |
| 1349 |
|
| 1350 |
.. method:: addTypeEqualityFunc(typeobj, function) |
| 1351 |
|
| 1352 |
Registers a type-specific method called by :meth:`assertEqual` to check |
| 1353 |
if two objects of exactly the same *typeobj* (not subclasses) compare |
| 1354 |
equal. *function* must take two positional arguments and a third msg=None |
| 1355 |
keyword argument just as :meth:`assertEqual` does. It must raise |
| 1356 |
:data:`self.failureException(msg) <failureException>` when inequality |
| 1357 |
between the first two parameters is detected -- possibly providing useful |
| 1358 |
information and explaining the inequalities in details in the error |
| 1359 |
message. |
| 1360 |
|
| 1361 |
.. versionadded:: 3.1 |
| 1362 |
|
| 1363 |
The list of type-specific methods automatically used by |
| 1364 |
:meth:`~TestCase.assertEqual` are summarized in the following table. Note |
| 1365 |
that it's usually not necessary to invoke these methods directly. |
| 1366 |
|
| 1367 |
+-----------------------------------------+-----------------------------+--------------+ |
| 1368 |
| Method | Used to compare | New in | |
| 1369 |
+=========================================+=============================+==============+ |
| 1370 |
| :meth:`assertMultiLineEqual(a, b) | strings | 3.1 | |
| 1371 |
| <TestCase.assertMultiLineEqual>` | | | |
| 1372 |
+-----------------------------------------+-----------------------------+--------------+ |
| 1373 |
| :meth:`assertSequenceEqual(a, b) | sequences | 3.1 | |
| 1374 |
| <TestCase.assertSequenceEqual>` | | | |
| 1375 |
+-----------------------------------------+-----------------------------+--------------+ |
| 1376 |
| :meth:`assertListEqual(a, b) | lists | 3.1 | |
| 1377 |
| <TestCase.assertListEqual>` | | | |
| 1378 |
+-----------------------------------------+-----------------------------+--------------+ |
| 1379 |
| :meth:`assertTupleEqual(a, b) | tuples | 3.1 | |
| 1380 |
| <TestCase.assertTupleEqual>` | | | |
| 1381 |
+-----------------------------------------+-----------------------------+--------------+ |
| 1382 |
| :meth:`assertSetEqual(a, b) | sets or frozensets | 3.1 | |
| 1383 |
| <TestCase.assertSetEqual>` | | | |
| 1384 |
+-----------------------------------------+-----------------------------+--------------+ |
| 1385 |
| :meth:`assertDictEqual(a, b) | dicts | 3.1 | |
| 1386 |
| <TestCase.assertDictEqual>` | | | |
| 1387 |
+-----------------------------------------+-----------------------------+--------------+ |
| 1388 |
|
| 1389 |
|
| 1390 |
|
| 1391 |
.. method:: assertMultiLineEqual(first, second, msg=None) |
| 1392 |
|
| 1393 |
Test that the multiline string *first* is equal to the string *second*. |
| 1394 |
When not equal a diff of the two strings highlighting the differences |
| 1395 |
will be included in the error message. This method is used by default |
| 1396 |
when comparing strings with :meth:`assertEqual`. |
| 1397 |
|
| 1398 |
.. versionadded:: 3.1 |
| 1399 |
|
| 1400 |
|
| 1401 |
.. method:: assertSequenceEqual(first, second, msg=None, seq_type=None) |
| 1402 |
|
| 1403 |
Tests that two sequences are equal. If a *seq_type* is supplied, both |
| 1404 |
*first* and *second* must be instances of *seq_type* or a failure will |
| 1405 |
be raised. If the sequences are different an error message is |
| 1406 |
constructed that shows the difference between the two. |
| 1407 |
|
| 1408 |
This method is not called directly by :meth:`assertEqual`, but |
| 1409 |
it's used to implement :meth:`assertListEqual` and |
| 1410 |
:meth:`assertTupleEqual`. |
| 1411 |
|
| 1412 |
.. versionadded:: 3.1 |
| 1413 |
|
| 1414 |
|
| 1415 |
.. method:: assertListEqual(first, second, msg=None) |
| 1416 |
assertTupleEqual(first, second, msg=None) |
| 1417 |
|
| 1418 |
Tests that two lists or tuples are equal. If not, an error message is |
| 1419 |
constructed that shows only the differences between the two. An error |
| 1420 |
is also raised if either of the parameters are of the wrong type. |
| 1421 |
These methods are used by default when comparing lists or tuples with |
| 1422 |
:meth:`assertEqual`. |
| 1423 |
|
| 1424 |
.. versionadded:: 3.1 |
| 1425 |
|
| 1426 |
|
| 1427 |
.. method:: assertSetEqual(first, second, msg=None) |
| 1428 |
|
| 1429 |
Tests that two sets are equal. If not, an error message is constructed |
| 1430 |
that lists the differences between the sets. This method is used by |
| 1431 |
default when comparing sets or frozensets with :meth:`assertEqual`. |
| 1432 |
|
| 1433 |
Fails if either of *first* or *second* does not have a :meth:`~frozenset.difference` |
| 1434 |
method. |
| 1435 |
|
| 1436 |
.. versionadded:: 3.1 |
| 1437 |
|
| 1438 |
|
| 1439 |
.. method:: assertDictEqual(first, second, msg=None) |
| 1440 |
|
| 1441 |
Test that two dictionaries are equal. If not, an error message is |
| 1442 |
constructed that shows the differences in the dictionaries. This |
| 1443 |
method will be used by default to compare dictionaries in |
| 1444 |
calls to :meth:`assertEqual`. |
| 1445 |
|
| 1446 |
.. versionadded:: 3.1 |
| 1447 |
|
| 1448 |
|
| 1449 |
|
| 1450 |
.. _other-methods-and-attrs: |
| 1451 |
|
| 1452 |
Finally the :class:`TestCase` provides the following methods and attributes: |
| 1453 |
|
| 1454 |
|
| 1455 |
.. method:: fail(msg=None) |
| 1456 |
|
| 1457 |
Signals a test failure unconditionally, with *msg* or ``None`` for |
| 1458 |
the error message. |
| 1459 |
|
| 1460 |
|
| 1461 |
.. attribute:: failureException |
| 1462 |
|
| 1463 |
This class attribute gives the exception raised by the test method. If a |
| 1464 |
test framework needs to use a specialized exception, possibly to carry |
| 1465 |
additional information, it must subclass this exception in order to "play |
| 1466 |
fair" with the framework. The initial value of this attribute is |
| 1467 |
:exc:`AssertionError`. |
| 1468 |
|
| 1469 |
|
| 1470 |
.. attribute:: longMessage |
| 1471 |
|
| 1472 |
This class attribute determines what happens when a custom failure message |
| 1473 |
is passed as the msg argument to an assertXYY call that fails. |
| 1474 |
``True`` is the default value. In this case, the custom message is appended |
| 1475 |
to the end of the standard failure message. |
| 1476 |
When set to ``False``, the custom message replaces the standard message. |
| 1477 |
|
| 1478 |
The class setting can be overridden in individual test methods by assigning |
| 1479 |
an instance attribute, self.longMessage, to ``True`` or ``False`` before |
| 1480 |
calling the assert methods. |
| 1481 |
|
| 1482 |
The class setting gets reset before each test call. |
| 1483 |
|
| 1484 |
.. versionadded:: 3.1 |
| 1485 |
|
| 1486 |
|
| 1487 |
.. attribute:: maxDiff |
| 1488 |
|
| 1489 |
This attribute controls the maximum length of diffs output by assert |
| 1490 |
methods that report diffs on failure. It defaults to 80*8 characters. |
| 1491 |
Assert methods affected by this attribute are |
| 1492 |
:meth:`assertSequenceEqual` (including all the sequence comparison |
| 1493 |
methods that delegate to it), :meth:`assertDictEqual` and |
| 1494 |
:meth:`assertMultiLineEqual`. |
| 1495 |
|
| 1496 |
Setting ``maxDiff`` to ``None`` means that there is no maximum length of |
| 1497 |
diffs. |
| 1498 |
|
| 1499 |
.. versionadded:: 3.2 |
| 1500 |
|
| 1501 |
|
| 1502 |
Testing frameworks can use the following methods to collect information on |
| 1503 |
the test: |
| 1504 |
|
| 1505 |
|
| 1506 |
.. method:: countTestCases() |
| 1507 |
|
| 1508 |
Return the number of tests represented by this test object. For |
| 1509 |
:class:`TestCase` instances, this will always be ``1``. |
| 1510 |
|
| 1511 |
|
| 1512 |
.. method:: defaultTestResult() |
| 1513 |
|
| 1514 |
Return an instance of the test result class that should be used for this |
| 1515 |
test case class (if no other result instance is provided to the |
| 1516 |
:meth:`run` method). |
| 1517 |
|
| 1518 |
For :class:`TestCase` instances, this will always be an instance of |
| 1519 |
:class:`TestResult`; subclasses of :class:`TestCase` should override this |
| 1520 |
as necessary. |
| 1521 |
|
| 1522 |
|
| 1523 |
.. method:: id() |
| 1524 |
|
| 1525 |
Return a string identifying the specific test case. This is usually the |
| 1526 |
full name of the test method, including the module and class name. |
| 1527 |
|
| 1528 |
|
| 1529 |
.. method:: shortDescription() |
| 1530 |
|
| 1531 |
Returns a description of the test, or ``None`` if no description |
| 1532 |
has been provided. The default implementation of this method |
| 1533 |
returns the first line of the test method's docstring, if available, |
| 1534 |
or ``None``. |
| 1535 |
|
| 1536 |
.. versionchanged:: 3.1 |
| 1537 |
In 3.1 this was changed to add the test name to the short description |
| 1538 |
even in the presence of a docstring. This caused compatibility issues |
| 1539 |
with unittest extensions and adding the test name was moved to the |
| 1540 |
:class:`TextTestResult` in Python 3.2. |
| 1541 |
|
| 1542 |
|
| 1543 |
.. method:: addCleanup(function, /, *args, **kwargs) |
| 1544 |
|
| 1545 |
Add a function to be called after :meth:`tearDown` to cleanup resources |
| 1546 |
used during the test. Functions will be called in reverse order to the |
| 1547 |
order they are added (:abbr:`LIFO (last-in, first-out)`). They |
| 1548 |
are called with any arguments and keyword arguments passed into |
| 1549 |
:meth:`addCleanup` when they are added. |
| 1550 |
|
| 1551 |
If :meth:`setUp` fails, meaning that :meth:`tearDown` is not called, |
| 1552 |
then any cleanup functions added will still be called. |
| 1553 |
|
| 1554 |
.. versionadded:: 3.1 |
| 1555 |
|
| 1556 |
|
| 1557 |
.. method:: enterContext(cm) |
| 1558 |
|
| 1559 |
Enter the supplied :term:`context manager`. If successful, also |
| 1560 |
add its :meth:`~object.__exit__` method as a cleanup function by |
| 1561 |
:meth:`addCleanup` and return the result of the |
| 1562 |
:meth:`~object.__enter__` method. |
| 1563 |
|
| 1564 |
.. versionadded:: 3.11 |
| 1565 |
|
| 1566 |
|
| 1567 |
.. method:: doCleanups() |
| 1568 |
|
| 1569 |
This method is called unconditionally after :meth:`tearDown`, or |
| 1570 |
after :meth:`setUp` if :meth:`setUp` raises an exception. |
| 1571 |
|
| 1572 |
It is responsible for calling all the cleanup functions added by |
| 1573 |
:meth:`addCleanup`. If you need cleanup functions to be called |
| 1574 |
*prior* to :meth:`tearDown` then you can call :meth:`doCleanups` |
| 1575 |
yourself. |
| 1576 |
|
| 1577 |
:meth:`doCleanups` pops methods off the stack of cleanup |
| 1578 |
functions one at a time, so it can be called at any time. |
| 1579 |
|
| 1580 |
.. versionadded:: 3.1 |
| 1581 |
|
| 1582 |
|
| 1583 |
.. classmethod:: addClassCleanup(function, /, *args, **kwargs) |
| 1584 |
|
| 1585 |
Add a function to be called after :meth:`tearDownClass` to cleanup |
| 1586 |
resources used during the test class. Functions will be called in reverse |
| 1587 |
order to the order they are added (:abbr:`LIFO (last-in, first-out)`). |
| 1588 |
They are called with any arguments and keyword arguments passed into |
| 1589 |
:meth:`addClassCleanup` when they are added. |
| 1590 |
|
| 1591 |
If :meth:`setUpClass` fails, meaning that :meth:`tearDownClass` is not |
| 1592 |
called, then any cleanup functions added will still be called. |
| 1593 |
|
| 1594 |
.. versionadded:: 3.8 |
| 1595 |
|
| 1596 |
|
| 1597 |
.. classmethod:: enterClassContext(cm) |
| 1598 |
|
| 1599 |
Enter the supplied :term:`context manager`. If successful, also |
| 1600 |
add its :meth:`~object.__exit__` method as a cleanup function by |
| 1601 |
:meth:`addClassCleanup` and return the result of the |
| 1602 |
:meth:`~object.__enter__` method. |
| 1603 |
|
| 1604 |
.. versionadded:: 3.11 |
| 1605 |
|
| 1606 |
|
| 1607 |
.. classmethod:: doClassCleanups() |
| 1608 |
|
| 1609 |
This method is called unconditionally after :meth:`tearDownClass`, or |
| 1610 |
after :meth:`setUpClass` if :meth:`setUpClass` raises an exception. |
| 1611 |
|
| 1612 |
It is responsible for calling all the cleanup functions added by |
| 1613 |
:meth:`addClassCleanup`. If you need cleanup functions to be called |
| 1614 |
*prior* to :meth:`tearDownClass` then you can call |
| 1615 |
:meth:`doClassCleanups` yourself. |
| 1616 |
|
| 1617 |
:meth:`doClassCleanups` pops methods off the stack of cleanup |
| 1618 |
functions one at a time, so it can be called at any time. |
| 1619 |
|
| 1620 |
.. versionadded:: 3.8 |
| 1621 |
|
| 1622 |
|
| 1623 |
.. class:: IsolatedAsyncioTestCase(methodName='runTest') |
| 1624 |
|
| 1625 |
This class provides an API similar to :class:`TestCase` and also accepts |
| 1626 |
coroutines as test functions. |
| 1627 |
|
| 1628 |
.. versionadded:: 3.8 |
| 1629 |
|
| 1630 |
.. attribute:: loop_factory |
| 1631 |
|
| 1632 |
The *loop_factory* passed to :class:`asyncio.Runner`. Override |
| 1633 |
in subclasses with :class:`asyncio.EventLoop` to avoid using the |
| 1634 |
asyncio policy system. |
| 1635 |
|
| 1636 |
.. versionadded:: 3.13 |
| 1637 |
|
| 1638 |
.. method:: asyncSetUp() |
| 1639 |
:async: |
| 1640 |
|
| 1641 |
Method called to prepare the test fixture. This is called after :meth:`TestCase.setUp`. |
| 1642 |
This is called immediately before calling the test method; other than |
| 1643 |
:exc:`AssertionError` or :exc:`SkipTest`, any exception raised by this method |
| 1644 |
will be considered an error rather than a test failure. The default implementation |
| 1645 |
does nothing. |
| 1646 |
|
| 1647 |
.. method:: asyncTearDown() |
| 1648 |
:async: |
| 1649 |
|
| 1650 |
Method called immediately after the test method has been called and the |
| 1651 |
result recorded. This is called before :meth:`~TestCase.tearDown`. This is called even if |
| 1652 |
the test method raised an exception, so the implementation in subclasses may need |
| 1653 |
to be particularly careful about checking internal state. Any exception, other than |
| 1654 |
:exc:`AssertionError` or :exc:`SkipTest`, raised by this method will be |
| 1655 |
considered an additional error rather than a test failure (thus increasing |
| 1656 |
the total number of reported errors). This method will only be called if |
| 1657 |
the :meth:`asyncSetUp` succeeds, regardless of the outcome of the test method. |
| 1658 |
The default implementation does nothing. |
| 1659 |
|
| 1660 |
.. method:: addAsyncCleanup(function, /, *args, **kwargs) |
| 1661 |
|
| 1662 |
This method accepts a coroutine that can be used as a cleanup function. |
| 1663 |
|
| 1664 |
.. method:: enterAsyncContext(cm) |
| 1665 |
:async: |
| 1666 |
|
| 1667 |
Enter the supplied :term:`asynchronous context manager`. If successful, |
| 1668 |
also add its :meth:`~object.__aexit__` method as a cleanup function by |
| 1669 |
:meth:`addAsyncCleanup` and return the result of the |
| 1670 |
:meth:`~object.__aenter__` method. |
| 1671 |
|
| 1672 |
.. versionadded:: 3.11 |
| 1673 |
|
| 1674 |
|
| 1675 |
.. method:: run(result=None) |
| 1676 |
|
| 1677 |
Sets up a new event loop to run the test, collecting the result into |
| 1678 |
the :class:`TestResult` object passed as *result*. If *result* is |
| 1679 |
omitted or ``None``, a temporary result object is created (by calling |
| 1680 |
the :meth:`~TestCase.defaultTestResult` method) and used. The result object is |
| 1681 |
returned to :meth:`run`'s caller. At the end of the test all the tasks |
| 1682 |
in the event loop are cancelled. |
| 1683 |
|
| 1684 |
|
| 1685 |
An example illustrating the order:: |
| 1686 |
|
| 1687 |
from unittest import IsolatedAsyncioTestCase |
| 1688 |
|
| 1689 |
events = [] |
| 1690 |
|
| 1691 |
|
| 1692 |
class Test(IsolatedAsyncioTestCase): |
| 1693 |
|
| 1694 |
|
| 1695 |
def setUp(self): |
| 1696 |
events.append("setUp") |
| 1697 |
|
| 1698 |
async def asyncSetUp(self): |
| 1699 |
self._async_connection = await AsyncConnection() |
| 1700 |
events.append("asyncSetUp") |
| 1701 |
|
| 1702 |
async def test_response(self): |
| 1703 |
events.append("test_response") |
| 1704 |
response = await self._async_connection.get("https://example.com") |
| 1705 |
self.assertEqual(response.status_code, 200) |
| 1706 |
self.addAsyncCleanup(self.on_cleanup) |
| 1707 |
|
| 1708 |
def tearDown(self): |
| 1709 |
events.append("tearDown") |
| 1710 |
|
| 1711 |
async def asyncTearDown(self): |
| 1712 |
await self._async_connection.close() |
| 1713 |
events.append("asyncTearDown") |
| 1714 |
|
| 1715 |
async def on_cleanup(self): |
| 1716 |
events.append("cleanup") |
| 1717 |
|
| 1718 |
if __name__ == "__main__": |
| 1719 |
unittest.main() |
| 1720 |
|
| 1721 |
After running the test, ``events`` would contain ``["setUp", "asyncSetUp", "test_response", "asyncTearDown", "tearDown", "cleanup"]``. |
| 1722 |
|
| 1723 |
|
| 1724 |
.. class:: FunctionTestCase(testFunc, setUp=None, tearDown=None, description=None) |
| 1725 |
|
| 1726 |
This class implements the portion of the :class:`TestCase` interface which |
| 1727 |
allows the test runner to drive the test, but does not provide the methods |
| 1728 |
which test code can use to check and report errors. This is used to create |
| 1729 |
test cases using legacy test code, allowing it to be integrated into a |
| 1730 |
:mod:`!unittest`-based test framework. |
| 1731 |
|
| 1732 |
|
| 1733 |
.. _testsuite-objects: |
| 1734 |
|
| 1735 |
Grouping tests |
| 1736 |
~~~~~~~~~~~~~~ |
| 1737 |
|
| 1738 |
.. class:: TestSuite(tests=()) |
| 1739 |
|
| 1740 |
This class represents an aggregation of individual test cases and test suites. |
| 1741 |
The class presents the interface needed by the test runner to allow it to be run |
| 1742 |
as any other test case. Running a :class:`TestSuite` instance is the same as |
| 1743 |
iterating over the suite, running each test individually. |
| 1744 |
|
| 1745 |
If *tests* is given, it must be an iterable of individual test cases or other |
| 1746 |
test suites that will be used to build the suite initially. Additional methods |
| 1747 |
are provided to add test cases and suites to the collection later on. |
| 1748 |
|
| 1749 |
:class:`TestSuite` objects behave much like :class:`TestCase` objects, except |
| 1750 |
they do not actually implement a test. Instead, they are used to aggregate |
| 1751 |
tests into groups of tests that should be run together. Some additional |
| 1752 |
methods are available to add tests to :class:`TestSuite` instances: |
| 1753 |
|
| 1754 |
|
| 1755 |
.. method:: TestSuite.addTest(test) |
| 1756 |
|
| 1757 |
Add a :class:`TestCase` or :class:`TestSuite` to the suite. |
| 1758 |
|
| 1759 |
|
| 1760 |
.. method:: TestSuite.addTests(tests) |
| 1761 |
|
| 1762 |
Add all the tests from an iterable of :class:`TestCase` and :class:`TestSuite` |
| 1763 |
instances to this test suite. |
| 1764 |
|
| 1765 |
This is equivalent to iterating over *tests*, calling :meth:`addTest` for |
| 1766 |
each element. |
| 1767 |
|
| 1768 |
:class:`TestSuite` shares the following methods with :class:`TestCase`: |
| 1769 |
|
| 1770 |
|
| 1771 |
.. method:: run(result) |
| 1772 |
|
| 1773 |
Run the tests associated with this suite, collecting the result into the |
| 1774 |
test result object passed as *result*. Note that unlike |
| 1775 |
:meth:`TestCase.run`, :meth:`TestSuite.run` requires the result object to |
| 1776 |
be passed in. |
| 1777 |
|
| 1778 |
|
| 1779 |
.. method:: debug() |
| 1780 |
|
| 1781 |
Run the tests associated with this suite without collecting the |
| 1782 |
result. This allows exceptions raised by the test to be propagated to the |
| 1783 |
caller and can be used to support running tests under a debugger. |
| 1784 |
|
| 1785 |
|
| 1786 |
.. method:: countTestCases() |
| 1787 |
|
| 1788 |
Return the number of tests represented by this test object, including all |
| 1789 |
individual tests and sub-suites. |
| 1790 |
|
| 1791 |
|
| 1792 |
.. method:: __iter__() |
| 1793 |
|
| 1794 |
Tests grouped by a :class:`TestSuite` are always accessed by iteration. |
| 1795 |
Subclasses can lazily provide tests by overriding :meth:`!__iter__`. Note |
| 1796 |
that this method may be called several times on a single suite (for |
| 1797 |
example when counting tests or comparing for equality) so the tests |
| 1798 |
returned by repeated iterations before :meth:`TestSuite.run` must be the |
| 1799 |
same for each call iteration. After :meth:`TestSuite.run`, callers should |
| 1800 |
not rely on the tests returned by this method unless the caller uses a |
| 1801 |
subclass that overrides :meth:`!TestSuite._removeTestAtIndex` to preserve |
| 1802 |
test references. |
| 1803 |
|
| 1804 |
.. versionchanged:: 3.2 |
| 1805 |
In earlier versions the :class:`TestSuite` accessed tests directly rather |
| 1806 |
than through iteration, so overriding :meth:`!__iter__` wasn't sufficient |
| 1807 |
for providing tests. |
| 1808 |
|
| 1809 |
.. versionchanged:: 3.4 |
| 1810 |
In earlier versions the :class:`TestSuite` held references to each |
| 1811 |
:class:`TestCase` after :meth:`TestSuite.run`. Subclasses can restore |
| 1812 |
that behavior by overriding :meth:`!TestSuite._removeTestAtIndex`. |
| 1813 |
|
| 1814 |
In the typical usage of a :class:`TestSuite` object, the :meth:`run` method |
| 1815 |
is invoked by a :class:`!TestRunner` rather than by the end-user test harness. |
| 1816 |
|
| 1817 |
|
| 1818 |
Loading and running tests |
| 1819 |
~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 1820 |
|
| 1821 |
.. class:: TestLoader() |
| 1822 |
|
| 1823 |
The :class:`TestLoader` class is used to create test suites from classes and |
| 1824 |
modules. Normally, there is no need to create an instance of this class; the |
| 1825 |
:mod:`!unittest` module provides an instance that can be shared as |
| 1826 |
:data:`unittest.defaultTestLoader`. Using a subclass or instance, however, |
| 1827 |
allows customization of some configurable properties. |
| 1828 |
|
| 1829 |
:class:`TestLoader` objects have the following attributes: |
| 1830 |
|
| 1831 |
|
| 1832 |
.. attribute:: errors |
| 1833 |
|
| 1834 |
A list of the non-fatal errors encountered while loading tests. Not reset |
| 1835 |
by the loader at any point. Fatal errors are signalled by the relevant |
| 1836 |
method raising an exception to the caller. Non-fatal errors are also |
| 1837 |
indicated by a synthetic test that will raise the original error when |
| 1838 |
run. |
| 1839 |
|
| 1840 |
.. versionadded:: 3.5 |
| 1841 |
|
| 1842 |
|
| 1843 |
:class:`TestLoader` objects have the following methods: |
| 1844 |
|
| 1845 |
|
| 1846 |
.. method:: loadTestsFromTestCase(testCaseClass) |
| 1847 |
|
| 1848 |
Return a suite of all test cases contained in the :class:`TestCase`\ -derived |
| 1849 |
:class:`!testCaseClass`. |
| 1850 |
|
| 1851 |
A test case instance is created for each method named by |
| 1852 |
:meth:`getTestCaseNames`. By default these are the method names |
| 1853 |
beginning with ``test``. If :meth:`getTestCaseNames` returns no |
| 1854 |
methods, but the :meth:`!runTest` method is implemented, a single test |
| 1855 |
case is created for that method instead. |
| 1856 |
|
| 1857 |
|
| 1858 |
.. method:: loadTestsFromModule(module, *, pattern=None) |
| 1859 |
|
| 1860 |
Return a suite of all test cases contained in the given module. This |
| 1861 |
method searches *module* for classes derived from :class:`TestCase` and |
| 1862 |
creates an instance of the class for each test method defined for the |
| 1863 |
class. |
| 1864 |
|
| 1865 |
.. note:: |
| 1866 |
|
| 1867 |
While using a hierarchy of :class:`TestCase`\ -derived classes can be |
| 1868 |
convenient in sharing fixtures and helper functions, defining test |
| 1869 |
methods on base classes that are not intended to be instantiated |
| 1870 |
directly does not play well with this method. Doing so, however, can |
| 1871 |
be useful when the fixtures are different and defined in subclasses. |
| 1872 |
|
| 1873 |
If a module provides a ``load_tests`` function it will be called to |
| 1874 |
load the tests. This allows modules to customize test loading. |
| 1875 |
This is the `load_tests protocol`_. The *pattern* argument is passed as |
| 1876 |
the third argument to ``load_tests``. |
| 1877 |
|
| 1878 |
.. versionchanged:: 3.2 |
| 1879 |
Support for ``load_tests`` added. |
| 1880 |
|
| 1881 |
.. versionchanged:: 3.5 |
| 1882 |
Support for a keyword-only argument *pattern* has been added. |
| 1883 |
|
| 1884 |
.. versionchanged:: 3.12 |
| 1885 |
The undocumented and unofficial *use_load_tests* parameter has been |
| 1886 |
removed. |
| 1887 |
|
| 1888 |
|
| 1889 |
.. method:: loadTestsFromName(name, module=None) |
| 1890 |
|
| 1891 |
Return a suite of all test cases given a string specifier. |
| 1892 |
|
| 1893 |
The specifier *name* is a "dotted name" that may resolve either to a |
| 1894 |
module, a test case class, a test method within a test case class, a |
| 1895 |
:class:`TestSuite` instance, or a callable object which returns a |
| 1896 |
:class:`TestCase` or :class:`TestSuite` instance. These checks are |
| 1897 |
applied in the order listed here; that is, a method on a possible test |
| 1898 |
case class will be picked up as "a test method within a test case class", |
| 1899 |
rather than "a callable object". |
| 1900 |
|
| 1901 |
For example, if you have a module :mod:`!SampleTests` containing a |
| 1902 |
:class:`TestCase`\ -derived class :class:`!SampleTestCase` with three test |
| 1903 |
methods (:meth:`!test_one`, :meth:`!test_two`, and :meth:`!test_three`), the |
| 1904 |
specifier ``'SampleTests.SampleTestCase'`` would cause this method to |
| 1905 |
return a suite which will run all three test methods. Using the specifier |
| 1906 |
``'SampleTests.SampleTestCase.test_two'`` would cause it to return a test |
| 1907 |
suite which will run only the :meth:`!test_two` test method. The specifier |
| 1908 |
can refer to modules and packages which have not been imported; they will |
| 1909 |
be imported as a side-effect. |
| 1910 |
|
| 1911 |
The method optionally resolves *name* relative to the given *module*. |
| 1912 |
|
| 1913 |
.. versionchanged:: 3.5 |
| 1914 |
If an :exc:`ImportError` or :exc:`AttributeError` occurs while traversing |
| 1915 |
*name* then a synthetic test that raises that error when run will be |
| 1916 |
returned. These errors are included in the errors accumulated by |
| 1917 |
self.errors. |
| 1918 |
|
| 1919 |
|
| 1920 |
.. method:: loadTestsFromNames(names, module=None) |
| 1921 |
|
| 1922 |
Similar to :meth:`loadTestsFromName`, but takes a sequence of names rather |
| 1923 |
than a single name. The return value is a test suite which supports all |
| 1924 |
the tests defined for each name. |
| 1925 |
|
| 1926 |
|
| 1927 |
.. method:: getTestCaseNames(testCaseClass) |
| 1928 |
|
| 1929 |
Return a sorted sequence of method names found within *testCaseClass*; |
| 1930 |
this should be a subclass of :class:`TestCase`. |
| 1931 |
|
| 1932 |
|
| 1933 |
.. method:: discover(start_dir, pattern='test*.py', top_level_dir=None) |
| 1934 |
|
| 1935 |
Find all the test modules by recursing into subdirectories from the |
| 1936 |
specified start directory, and return a TestSuite object containing them. |
| 1937 |
Only test files that match *pattern* will be loaded. (Using shell style |
| 1938 |
pattern matching.) Only module names that are importable (i.e. are valid |
| 1939 |
Python identifiers) will be loaded. |
| 1940 |
|
| 1941 |
All test modules must be importable from the top level of the project. If |
| 1942 |
the start directory is not the top level directory then *top_level_dir* |
| 1943 |
must be specified separately. |
| 1944 |
|
| 1945 |
If importing a module fails, for example due to a syntax error, then |
| 1946 |
this will be recorded as a single error and discovery will continue. If |
| 1947 |
the import failure is due to :exc:`SkipTest` being raised, it will be |
| 1948 |
recorded as a skip instead of an error. |
| 1949 |
|
| 1950 |
If a package (a directory containing a file named :file:`__init__.py`) is |
| 1951 |
found, the package will be checked for a ``load_tests`` function. If this |
| 1952 |
exists then it will be called |
| 1953 |
``package.load_tests(loader, tests, pattern)``. Test discovery takes care |
| 1954 |
to ensure that a package is only checked for tests once during an |
| 1955 |
invocation, even if the load_tests function itself calls |
| 1956 |
``loader.discover``. |
| 1957 |
|
| 1958 |
If ``load_tests`` exists then discovery does *not* recurse into the |
| 1959 |
package, ``load_tests`` is responsible for loading all tests in the |
| 1960 |
package. |
| 1961 |
|
| 1962 |
The pattern is deliberately not stored as a loader attribute so that |
| 1963 |
packages can continue discovery themselves. |
| 1964 |
|
| 1965 |
*top_level_dir* is stored internally, and used as a default to any |
| 1966 |
nested calls to ``discover()``. That is, if a package's ``load_tests`` |
| 1967 |
calls ``loader.discover()``, it does not need to pass this argument. |
| 1968 |
|
| 1969 |
*start_dir* can be a dotted module name as well as a directory. |
| 1970 |
|
| 1971 |
.. versionadded:: 3.2 |
| 1972 |
|
| 1973 |
.. versionchanged:: 3.4 |
| 1974 |
Modules that raise :exc:`SkipTest` on import are recorded as skips, |
| 1975 |
not errors. |
| 1976 |
|
| 1977 |
*start_dir* can be a :term:`namespace packages <namespace package>`. |
| 1978 |
|
| 1979 |
Paths are sorted before being imported so that execution order is the |
| 1980 |
same even if the underlying file system's ordering is not dependent |
| 1981 |
on file name. |
| 1982 |
|
| 1983 |
.. versionchanged:: 3.5 |
| 1984 |
Found packages are now checked for ``load_tests`` regardless of |
| 1985 |
whether their path matches *pattern*, because it is impossible for |
| 1986 |
a package name to match the default pattern. |
| 1987 |
|
| 1988 |
.. versionchanged:: 3.11 |
| 1989 |
*start_dir* can not be a :term:`namespace packages <namespace package>`. |
| 1990 |
It has been broken since Python 3.7, and Python 3.11 officially removes it. |
| 1991 |
|
| 1992 |
.. versionchanged:: 3.13 |
| 1993 |
*top_level_dir* is only stored for the duration of *discover* call. |
| 1994 |
|
| 1995 |
.. versionchanged:: 3.14 |
| 1996 |
*start_dir* can once again be a :term:`namespace package`. |
| 1997 |
|
| 1998 |
The following attributes of a :class:`TestLoader` can be configured either by |
| 1999 |
subclassing or assignment on an instance: |
| 2000 |
|
| 2001 |
|
| 2002 |
.. attribute:: testMethodPrefix |
| 2003 |
|
| 2004 |
String giving the prefix of method names which will be interpreted as test |
| 2005 |
methods. The default value is ``'test'``. |
| 2006 |
|
| 2007 |
This affects :meth:`getTestCaseNames` and all the ``loadTestsFrom*`` |
| 2008 |
methods. |
| 2009 |
|
| 2010 |
|
| 2011 |
.. attribute:: sortTestMethodsUsing |
| 2012 |
|
| 2013 |
Function to be used to compare method names when sorting them in |
| 2014 |
:meth:`getTestCaseNames` and all the ``loadTestsFrom*`` methods. |
| 2015 |
|
| 2016 |
|
| 2017 |
.. attribute:: suiteClass |
| 2018 |
|
| 2019 |
Callable object that constructs a test suite from a list of tests. No |
| 2020 |
methods on the resulting object are needed. The default value is the |
| 2021 |
:class:`TestSuite` class. |
| 2022 |
|
| 2023 |
This affects all the ``loadTestsFrom*`` methods. |
| 2024 |
|
| 2025 |
.. attribute:: testNamePatterns |
| 2026 |
|
| 2027 |
List of Unix shell-style wildcard test name patterns that test methods |
| 2028 |
have to match to be included in test suites (see ``-k`` option). |
| 2029 |
|
| 2030 |
If this attribute is not ``None`` (the default), all test methods to be |
| 2031 |
included in test suites must match one of the patterns in this list. |
| 2032 |
Note that matches are always performed using :meth:`fnmatch.fnmatchcase`, |
| 2033 |
so unlike patterns passed to the ``-k`` option, simple substring patterns |
| 2034 |
will have to be converted using ``*`` wildcards. |
| 2035 |
|
| 2036 |
This affects all the ``loadTestsFrom*`` methods. |
| 2037 |
|
| 2038 |
.. versionadded:: 3.7 |
| 2039 |
|
| 2040 |
|
| 2041 |
.. class:: TestResult |
| 2042 |
|
| 2043 |
This class is used to compile information about which tests have succeeded |
| 2044 |
and which have failed. |
| 2045 |
|
| 2046 |
A :class:`TestResult` object stores the results of a set of tests. The |
| 2047 |
:class:`TestCase` and :class:`TestSuite` classes ensure that results are |
| 2048 |
properly recorded; test authors do not need to worry about recording the |
| 2049 |
outcome of tests. |
| 2050 |
|
| 2051 |
Testing frameworks built on top of :mod:`!unittest` may want access to the |
| 2052 |
:class:`TestResult` object generated by running a set of tests for reporting |
| 2053 |
purposes; a :class:`TestResult` instance is returned by the |
| 2054 |
:meth:`!TestRunner.run` method for this purpose. |
| 2055 |
|
| 2056 |
:class:`TestResult` instances have the following attributes that will be of |
| 2057 |
interest when inspecting the results of running a set of tests: |
| 2058 |
|
| 2059 |
|
| 2060 |
.. attribute:: errors |
| 2061 |
|
| 2062 |
A list containing 2-tuples of :class:`TestCase` instances and strings |
| 2063 |
holding formatted tracebacks. Each tuple represents a test which raised an |
| 2064 |
unexpected exception. |
| 2065 |
|
| 2066 |
.. attribute:: failures |
| 2067 |
|
| 2068 |
A list containing 2-tuples of :class:`TestCase` instances and strings |
| 2069 |
holding formatted tracebacks. Each tuple represents a test where a failure |
| 2070 |
was explicitly signalled using the :ref:`assert\* methods <assert-methods>`. |
| 2071 |
|
| 2072 |
.. attribute:: skipped |
| 2073 |
|
| 2074 |
A list containing 2-tuples of :class:`TestCase` instances and strings |
| 2075 |
holding the reason for skipping the test. |
| 2076 |
|
| 2077 |
.. versionadded:: 3.1 |
| 2078 |
|
| 2079 |
.. attribute:: expectedFailures |
| 2080 |
|
| 2081 |
A list containing 2-tuples of :class:`TestCase` instances and strings |
| 2082 |
holding formatted tracebacks. Each tuple represents an expected failure |
| 2083 |
or error of the test case. |
| 2084 |
|
| 2085 |
.. attribute:: unexpectedSuccesses |
| 2086 |
|
| 2087 |
A list containing :class:`TestCase` instances that were marked as expected |
| 2088 |
failures, but succeeded. |
| 2089 |
|
| 2090 |
.. attribute:: collectedDurations |
| 2091 |
|
| 2092 |
A list containing 2-tuples of test case names and floats |
| 2093 |
representing the elapsed time of each test which was run. |
| 2094 |
|
| 2095 |
.. versionadded:: 3.12 |
| 2096 |
|
| 2097 |
.. attribute:: shouldStop |
| 2098 |
|
| 2099 |
Set to ``True`` when the execution of tests should stop by :meth:`stop`. |
| 2100 |
|
| 2101 |
.. attribute:: testsRun |
| 2102 |
|
| 2103 |
The total number of tests run so far. |
| 2104 |
|
| 2105 |
.. attribute:: buffer |
| 2106 |
|
| 2107 |
If set to true, ``sys.stdout`` and ``sys.stderr`` will be buffered in between |
| 2108 |
:meth:`startTest` and :meth:`stopTest` being called. Collected output will |
| 2109 |
only be echoed onto the real ``sys.stdout`` and ``sys.stderr`` if the test |
| 2110 |
fails or errors. Any output is also attached to the failure / error message. |
| 2111 |
|
| 2112 |
.. versionadded:: 3.2 |
| 2113 |
|
| 2114 |
.. attribute:: failfast |
| 2115 |
|
| 2116 |
If set to true :meth:`stop` will be called on the first failure or error, |
| 2117 |
halting the test run. |
| 2118 |
|
| 2119 |
.. versionadded:: 3.2 |
| 2120 |
|
| 2121 |
.. attribute:: tb_locals |
| 2122 |
|
| 2123 |
If set to true then local variables will be shown in tracebacks. |
| 2124 |
|
| 2125 |
.. versionadded:: 3.5 |
| 2126 |
|
| 2127 |
.. method:: wasSuccessful() |
| 2128 |
|
| 2129 |
Return ``True`` if all tests run so far have passed, otherwise returns |
| 2130 |
``False``. |
| 2131 |
|
| 2132 |
.. versionchanged:: 3.4 |
| 2133 |
Returns ``False`` if there were any :attr:`unexpectedSuccesses` |
| 2134 |
from tests marked with the :func:`expectedFailure` decorator. |
| 2135 |
|
| 2136 |
.. method:: stop() |
| 2137 |
|
| 2138 |
This method can be called to signal that the set of tests being run should |
| 2139 |
be aborted by setting the :attr:`shouldStop` attribute to ``True``. |
| 2140 |
:class:`!TestRunner` objects should respect this flag and return without |
| 2141 |
running any additional tests. |
| 2142 |
|
| 2143 |
For example, this feature is used by the :class:`TextTestRunner` class to |
| 2144 |
stop the test framework when the user signals an interrupt from the |
| 2145 |
keyboard. Interactive tools which provide :class:`!TestRunner` |
| 2146 |
implementations can use this in a similar manner. |
| 2147 |
|
| 2148 |
The following methods of the :class:`TestResult` class are used to maintain |
| 2149 |
the internal data structures, and may be extended in subclasses to support |
| 2150 |
additional reporting requirements. This is particularly useful in building |
| 2151 |
tools which support interactive reporting while tests are being run. |
| 2152 |
|
| 2153 |
|
| 2154 |
.. method:: startTest(test) |
| 2155 |
|
| 2156 |
Called when the test case *test* is about to be run. |
| 2157 |
|
| 2158 |
.. method:: stopTest(test) |
| 2159 |
|
| 2160 |
Called after the test case *test* has been executed, regardless of the |
| 2161 |
outcome. |
| 2162 |
|
| 2163 |
.. method:: startTestRun() |
| 2164 |
|
| 2165 |
Called once before any tests are executed. |
| 2166 |
|
| 2167 |
.. versionadded:: 3.1 |
| 2168 |
|
| 2169 |
|
| 2170 |
.. method:: stopTestRun() |
| 2171 |
|
| 2172 |
Called once after all tests are executed. |
| 2173 |
|
| 2174 |
.. versionadded:: 3.1 |
| 2175 |
|
| 2176 |
|
| 2177 |
.. method:: addError(test, err) |
| 2178 |
|
| 2179 |
Called when the test case *test* raises an unexpected exception. *err* is a |
| 2180 |
tuple of the form returned by :func:`sys.exc_info`: ``(type, value, |
| 2181 |
traceback)``. |
| 2182 |
|
| 2183 |
The default implementation appends a tuple ``(test, formatted_err)`` to |
| 2184 |
the instance's :attr:`errors` attribute, where *formatted_err* is a |
| 2185 |
formatted traceback derived from *err*. |
| 2186 |
|
| 2187 |
|
| 2188 |
.. method:: addFailure(test, err) |
| 2189 |
|
| 2190 |
Called when the test case *test* signals a failure. *err* is a tuple of |
| 2191 |
the form returned by :func:`sys.exc_info`: ``(type, value, traceback)``. |
| 2192 |
|
| 2193 |
The default implementation appends a tuple ``(test, formatted_err)`` to |
| 2194 |
the instance's :attr:`failures` attribute, where *formatted_err* is a |
| 2195 |
formatted traceback derived from *err*. |
| 2196 |
|
| 2197 |
|
| 2198 |
.. method:: addSuccess(test) |
| 2199 |
|
| 2200 |
Called when the test case *test* succeeds. |
| 2201 |
|
| 2202 |
The default implementation does nothing. |
| 2203 |
|
| 2204 |
|
| 2205 |
.. method:: addSkip(test, reason) |
| 2206 |
|
| 2207 |
Called when the test case *test* is skipped. *reason* is the reason the |
| 2208 |
test gave for skipping. |
| 2209 |
|
| 2210 |
The default implementation appends a tuple ``(test, reason)`` to the |
| 2211 |
instance's :attr:`skipped` attribute. |
| 2212 |
|
| 2213 |
|
| 2214 |
.. method:: addExpectedFailure(test, err) |
| 2215 |
|
| 2216 |
Called when the test case *test* fails or errors, but was marked with |
| 2217 |
the :func:`expectedFailure` decorator. |
| 2218 |
|
| 2219 |
The default implementation appends a tuple ``(test, formatted_err)`` to |
| 2220 |
the instance's :attr:`expectedFailures` attribute, where *formatted_err* |
| 2221 |
is a formatted traceback derived from *err*. |
| 2222 |
|
| 2223 |
|
| 2224 |
.. method:: addUnexpectedSuccess(test) |
| 2225 |
|
| 2226 |
Called when the test case *test* was marked with the |
| 2227 |
:func:`expectedFailure` decorator, but succeeded. |
| 2228 |
|
| 2229 |
The default implementation appends the test to the instance's |
| 2230 |
:attr:`unexpectedSuccesses` attribute. |
| 2231 |
|
| 2232 |
|
| 2233 |
.. method:: addSubTest(test, subtest, outcome) |
| 2234 |
|
| 2235 |
Called when a subtest finishes. *test* is the test case |
| 2236 |
corresponding to the test method. *subtest* is a custom |
| 2237 |
:class:`TestCase` instance describing the subtest. |
| 2238 |
|
| 2239 |
If *outcome* is :const:`None`, the subtest succeeded. Otherwise, |
| 2240 |
it failed with an exception where *outcome* is a tuple of the form |
| 2241 |
returned by :func:`sys.exc_info`: ``(type, value, traceback)``. |
| 2242 |
|
| 2243 |
The default implementation does nothing when the outcome is a |
| 2244 |
success, and records subtest failures as normal failures. |
| 2245 |
|
| 2246 |
.. versionadded:: 3.4 |
| 2247 |
|
| 2248 |
.. method:: addDuration(test, elapsed) |
| 2249 |
|
| 2250 |
Called when the test case finishes. *elapsed* is the time represented in |
| 2251 |
seconds, and it includes the execution of cleanup functions. |
| 2252 |
|
| 2253 |
.. versionadded:: 3.12 |
| 2254 |
|
| 2255 |
.. class:: TextTestResult(stream, descriptions, verbosity, *, durations=None) |
| 2256 |
|
| 2257 |
A concrete implementation of :class:`TestResult` used by the |
| 2258 |
:class:`TextTestRunner`. Subclasses should accept ``**kwargs`` to ensure |
| 2259 |
compatibility as the interface changes. |
| 2260 |
|
| 2261 |
.. versionadded:: 3.2 |
| 2262 |
|
| 2263 |
.. versionchanged:: 3.12 |
| 2264 |
Added the *durations* keyword parameter. |
| 2265 |
|
| 2266 |
.. data:: defaultTestLoader |
| 2267 |
|
| 2268 |
Instance of the :class:`TestLoader` class intended to be shared. If no |
| 2269 |
customization of the :class:`TestLoader` is needed, this instance can be used |
| 2270 |
instead of repeatedly creating new instances. |
| 2271 |
|
| 2272 |
|
| 2273 |
.. class:: TextTestRunner(stream=None, descriptions=True, verbosity=1, failfast=False, \ |
| 2274 |
buffer=False, resultclass=None, warnings=None, *, \ |
| 2275 |
tb_locals=False, durations=None) |
| 2276 |
|
| 2277 |
A basic test runner implementation that outputs results to a stream. If *stream* |
| 2278 |
is ``None``, the default, :data:`sys.stderr` is used as the output stream. This class |
| 2279 |
has a few configurable parameters, but is essentially very simple. Graphical |
| 2280 |
applications which run test suites should provide alternate implementations. Such |
| 2281 |
implementations should accept ``**kwargs`` as the interface to construct runners |
| 2282 |
changes when features are added to unittest. |
| 2283 |
|
| 2284 |
By default this runner shows :exc:`DeprecationWarning`, |
| 2285 |
:exc:`PendingDeprecationWarning`, :exc:`ResourceWarning` and |
| 2286 |
:exc:`ImportWarning` even if they are :ref:`ignored by default |
| 2287 |
<warning-ignored>`. This behavior can |
| 2288 |
be overridden using Python's :option:`!-Wd` or :option:`!-Wa` options |
| 2289 |
(see :ref:`Warning control <using-on-warnings>`) and leaving |
| 2290 |
*warnings* to ``None``. |
| 2291 |
|
| 2292 |
.. versionchanged:: 3.2 |
| 2293 |
Added the *warnings* parameter. |
| 2294 |
|
| 2295 |
.. versionchanged:: 3.2 |
| 2296 |
The default stream is set to :data:`sys.stderr` at instantiation time rather |
| 2297 |
than import time. |
| 2298 |
|
| 2299 |
.. versionchanged:: 3.5 |
| 2300 |
Added the *tb_locals* parameter. |
| 2301 |
|
| 2302 |
.. versionchanged:: 3.12 |
| 2303 |
Added the *durations* parameter. |
| 2304 |
|
| 2305 |
.. method:: _makeResult() |
| 2306 |
|
| 2307 |
This method returns the instance of ``TestResult`` used by :meth:`run`. |
| 2308 |
It is not intended to be called directly, but can be overridden in |
| 2309 |
subclasses to provide a custom ``TestResult``. |
| 2310 |
|
| 2311 |
``_makeResult()`` instantiates the class or callable passed in the |
| 2312 |
``TextTestRunner`` constructor as the ``resultclass`` argument. It |
| 2313 |
defaults to :class:`TextTestResult` if no ``resultclass`` is provided. |
| 2314 |
The result class is instantiated with the following arguments:: |
| 2315 |
|
| 2316 |
stream, descriptions, verbosity |
| 2317 |
|
| 2318 |
.. method:: run(test) |
| 2319 |
|
| 2320 |
This method is the main public interface to the ``TextTestRunner``. This |
| 2321 |
method takes a :class:`TestSuite` or :class:`TestCase` instance. A |
| 2322 |
:class:`TestResult` is created by calling |
| 2323 |
:func:`_makeResult` and the test(s) are run and the |
| 2324 |
results printed to stdout. |
| 2325 |
|
| 2326 |
|
| 2327 |
.. function:: main(module='__main__', defaultTest=None, argv=None, testRunner=None, \ |
| 2328 |
testLoader=unittest.defaultTestLoader, exit=True, verbosity=1, \ |
| 2329 |
failfast=None, catchbreak=None, buffer=None, warnings=None) |
| 2330 |
|
| 2331 |
A command-line program that loads a set of tests from *module* and runs them; |
| 2332 |
this is primarily for making test modules conveniently executable. |
| 2333 |
The simplest use for this function is to include the following line at the |
| 2334 |
end of a test script:: |
| 2335 |
|
| 2336 |
if __name__ == '__main__': |
| 2337 |
unittest.main() |
| 2338 |
|
| 2339 |
You can run tests with more detailed information by passing in the verbosity |
| 2340 |
argument:: |
| 2341 |
|
| 2342 |
if __name__ == '__main__': |
| 2343 |
unittest.main(verbosity=2) |
| 2344 |
|
| 2345 |
The *defaultTest* argument is either the name of a single test or an |
| 2346 |
iterable of test names to run if no test names are specified via *argv*. If |
| 2347 |
not specified or ``None`` and no test names are provided via *argv*, all |
| 2348 |
tests found in *module* are run. |
| 2349 |
|
| 2350 |
The *argv* argument can be a list of options passed to the program, with the |
| 2351 |
first element being the program name. If not specified or ``None``, |
| 2352 |
the values of :data:`sys.argv` are used. |
| 2353 |
|
| 2354 |
The *testRunner* argument can either be a test runner class or an already |
| 2355 |
created instance of it. By default ``main`` calls :func:`sys.exit` with |
| 2356 |
an exit code indicating success (0) or failure (1) of the tests run. |
| 2357 |
An exit code of 5 indicates that no tests were run or skipped. |
| 2358 |
|
| 2359 |
The *testLoader* argument has to be a :class:`TestLoader` instance, |
| 2360 |
and defaults to :data:`defaultTestLoader`. |
| 2361 |
|
| 2362 |
``main`` supports being used from the interactive interpreter by passing in the |
| 2363 |
argument ``exit=False``. This displays the result on standard output without |
| 2364 |
calling :func:`sys.exit`:: |
| 2365 |
|
| 2366 |
>>> from unittest import main |
| 2367 |
>>> main(module='test_module', exit=False) |
| 2368 |
|
| 2369 |
The *failfast*, *catchbreak* and *buffer* parameters have the same |
| 2370 |
effect as the same-name `command-line options`_. |
| 2371 |
|
| 2372 |
The *warnings* argument specifies the :ref:`warning filter <warning-filter>` |
| 2373 |
that should be used while running the tests. If it's not specified, it will |
| 2374 |
remain ``None`` if a :option:`!-W` option is passed to :program:`python` |
| 2375 |
(see :ref:`Warning control <using-on-warnings>`), |
| 2376 |
otherwise it will be set to ``'default'``. |
| 2377 |
|
| 2378 |
Calling ``main`` returns an object with the ``result`` attribute that contains |
| 2379 |
the result of the tests run as a :class:`unittest.TestResult`. |
| 2380 |
|
| 2381 |
.. versionchanged:: 3.1 |
| 2382 |
The *exit* parameter was added. |
| 2383 |
|
| 2384 |
.. versionchanged:: 3.2 |
| 2385 |
The *verbosity*, *failfast*, *catchbreak*, *buffer* |
| 2386 |
and *warnings* parameters were added. |
| 2387 |
|
| 2388 |
.. versionchanged:: 3.4 |
| 2389 |
The *defaultTest* parameter was changed to also accept an iterable of |
| 2390 |
test names. |
| 2391 |
|
| 2392 |
|
| 2393 |
.. _load_tests-protocol: |
| 2394 |
|
| 2395 |
load_tests Protocol |
| 2396 |
################### |
| 2397 |
|
| 2398 |
.. versionadded:: 3.2 |
| 2399 |
|
| 2400 |
Modules or packages can customize how tests are loaded from them during normal |
| 2401 |
test runs or test discovery by implementing a function called ``load_tests``. |
| 2402 |
|
| 2403 |
If a test module defines ``load_tests`` it will be called by |
| 2404 |
:meth:`TestLoader.loadTestsFromModule` with the following arguments:: |
| 2405 |
|
| 2406 |
load_tests(loader, standard_tests, pattern) |
| 2407 |
|
| 2408 |
where *pattern* is passed straight through from ``loadTestsFromModule``. It |
| 2409 |
defaults to ``None``. |
| 2410 |
|
| 2411 |
It should return a :class:`TestSuite`. |
| 2412 |
|
| 2413 |
*loader* is the instance of :class:`TestLoader` doing the loading. |
| 2414 |
*standard_tests* are the tests that would be loaded by default from the |
| 2415 |
module. It is common for test modules to only want to add or remove tests |
| 2416 |
from the standard set of tests. |
| 2417 |
The third argument is used when loading packages as part of test discovery. |
| 2418 |
|
| 2419 |
A typical ``load_tests`` function that loads tests from a specific set of |
| 2420 |
:class:`TestCase` classes may look like:: |
| 2421 |
|
| 2422 |
test_cases = (TestCase1, TestCase2, TestCase3) |
| 2423 |
|
| 2424 |
def load_tests(loader, tests, pattern): |
| 2425 |
suite = TestSuite() |
| 2426 |
for test_class in test_cases: |
| 2427 |
tests = loader.loadTestsFromTestCase(test_class) |
| 2428 |
suite.addTests(tests) |
| 2429 |
return suite |
| 2430 |
|
| 2431 |
If discovery is started in a directory containing a package, either from the |
| 2432 |
command line or by calling :meth:`TestLoader.discover`, then the package |
| 2433 |
:file:`__init__.py` will be checked for ``load_tests``. If that function does |
| 2434 |
not exist, discovery will recurse into the package as though it were just |
| 2435 |
another directory. Otherwise, discovery of the package's tests will be left up |
| 2436 |
to ``load_tests`` which is called with the following arguments:: |
| 2437 |
|
| 2438 |
load_tests(loader, standard_tests, pattern) |
| 2439 |
|
| 2440 |
This should return a :class:`TestSuite` representing all the tests |
| 2441 |
from the package. (``standard_tests`` will only contain tests |
| 2442 |
collected from :file:`__init__.py`.) |
| 2443 |
|
| 2444 |
Because the pattern is passed into ``load_tests`` the package is free to |
| 2445 |
continue (and potentially modify) test discovery. A 'do nothing' |
| 2446 |
``load_tests`` function for a test package would look like:: |
| 2447 |
|
| 2448 |
def load_tests(loader, standard_tests, pattern): |
| 2449 |
# top level directory cached on loader instance |
| 2450 |
this_dir = os.path.dirname(__file__) |
| 2451 |
package_tests = loader.discover(start_dir=this_dir, pattern=pattern) |
| 2452 |
standard_tests.addTests(package_tests) |
| 2453 |
return standard_tests |
| 2454 |
|
| 2455 |
.. versionchanged:: 3.5 |
| 2456 |
Discovery no longer checks package names for matching *pattern* due to the |
| 2457 |
impossibility of package names matching the default pattern. |
| 2458 |
|
| 2459 |
|
| 2460 |
|
| 2461 |
Class and Module Fixtures |
| 2462 |
------------------------- |
| 2463 |
|
| 2464 |
Class and module level fixtures are implemented in :class:`TestSuite`. When |
| 2465 |
the test suite encounters a test from a new class then |
| 2466 |
:meth:`~TestCase.tearDownClass` from the previous class (if there is one) |
| 2467 |
is called, followed by :meth:`~TestCase.setUpClass` from the new class. |
| 2468 |
|
| 2469 |
Similarly if a test is from a different module from the previous test then |
| 2470 |
``tearDownModule`` from the previous module is run, followed by |
| 2471 |
``setUpModule`` from the new module. |
| 2472 |
|
| 2473 |
After all the tests have run the final ``tearDownClass`` and |
| 2474 |
``tearDownModule`` are run. |
| 2475 |
|
| 2476 |
Note that shared fixtures do not play well with [potential] features like test |
| 2477 |
parallelization and they break test isolation. They should be used with care. |
| 2478 |
|
| 2479 |
The default ordering of tests created by the unittest test loaders is to group |
| 2480 |
all tests from the same modules and classes together. This will lead to |
| 2481 |
``setUpClass`` / ``setUpModule`` (etc) being called exactly once per class and |
| 2482 |
module. If you randomize the order, so that tests from different modules and |
| 2483 |
classes are adjacent to each other, then these shared fixture functions may be |
| 2484 |
called multiple times in a single test run. |
| 2485 |
|
| 2486 |
Shared fixtures are not intended to work with suites with non-standard |
| 2487 |
ordering. A ``BaseTestSuite`` still exists for frameworks that don't want to |
| 2488 |
support shared fixtures. |
| 2489 |
|
| 2490 |
If there are any exceptions raised during one of the shared fixture functions |
| 2491 |
the test is reported as an error. Because there is no corresponding test |
| 2492 |
instance an ``_ErrorHolder`` object (that has the same interface as a |
| 2493 |
:class:`TestCase`) is created to represent the error. If you are just using |
| 2494 |
the standard unittest test runner then this detail doesn't matter, but if you |
| 2495 |
are a framework author it may be relevant. |
| 2496 |
|
| 2497 |
|
| 2498 |
setUpClass and tearDownClass |
| 2499 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2500 |
|
| 2501 |
These must be implemented as class methods:: |
| 2502 |
|
| 2503 |
import unittest |
| 2504 |
|
| 2505 |
class Test(unittest.TestCase): |
| 2506 |
@classmethod |
| 2507 |
def setUpClass(cls): |
| 2508 |
cls._connection = createExpensiveConnectionObject() |
| 2509 |
|
| 2510 |
@classmethod |
| 2511 |
def tearDownClass(cls): |
| 2512 |
cls._connection.destroy() |
| 2513 |
|
| 2514 |
If you want the ``setUpClass`` and ``tearDownClass`` on base classes called |
| 2515 |
then you must call up to them yourself. The implementations in |
| 2516 |
:class:`TestCase` are empty. |
| 2517 |
|
| 2518 |
If an exception is raised during a ``setUpClass`` then the tests in the class |
| 2519 |
are not run and the ``tearDownClass`` is not run. Skipped classes will not |
| 2520 |
have ``setUpClass`` or ``tearDownClass`` run. If the exception is a |
| 2521 |
:exc:`SkipTest` exception then the class will be reported as having been skipped |
| 2522 |
instead of as an error. |
| 2523 |
|
| 2524 |
|
| 2525 |
setUpModule and tearDownModule |
| 2526 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2527 |
|
| 2528 |
.. function:: setUpModule |
| 2529 |
tearDownModule |
| 2530 |
:no-typesetting: |
| 2531 |
|
| 2532 |
These should be implemented as functions:: |
| 2533 |
|
| 2534 |
def setUpModule(): |
| 2535 |
createConnection() |
| 2536 |
|
| 2537 |
def tearDownModule(): |
| 2538 |
closeConnection() |
| 2539 |
|
| 2540 |
If an exception is raised in a ``setUpModule`` then none of the tests in the |
| 2541 |
module will be run and the ``tearDownModule`` will not be run. If the exception is a |
| 2542 |
:exc:`SkipTest` exception then the module will be reported as having been skipped |
| 2543 |
instead of as an error. |
| 2544 |
|
| 2545 |
To add cleanup code that must be run even in the case of an exception, use |
| 2546 |
``addModuleCleanup``: |
| 2547 |
|
| 2548 |
|
| 2549 |
.. function:: addModuleCleanup(function, /, *args, **kwargs) |
| 2550 |
|
| 2551 |
Add a function to be called after :func:`tearDownModule` to cleanup |
| 2552 |
resources used during the test class. Functions will be called in reverse |
| 2553 |
order to the order they are added (:abbr:`LIFO (last-in, first-out)`). |
| 2554 |
They are called with any arguments and keyword arguments passed into |
| 2555 |
:meth:`addModuleCleanup` when they are added. |
| 2556 |
|
| 2557 |
If :meth:`setUpModule` fails, meaning that :func:`tearDownModule` is not |
| 2558 |
called, then any cleanup functions added will still be called. |
| 2559 |
|
| 2560 |
.. versionadded:: 3.8 |
| 2561 |
|
| 2562 |
|
| 2563 |
.. function:: enterModuleContext(cm) |
| 2564 |
|
| 2565 |
Enter the supplied :term:`context manager`. If successful, also |
| 2566 |
add its :meth:`~object.__exit__` method as a cleanup function by |
| 2567 |
:func:`addModuleCleanup` and return the result of the |
| 2568 |
:meth:`~object.__enter__` method. |
| 2569 |
|
| 2570 |
.. versionadded:: 3.11 |
| 2571 |
|
| 2572 |
|
| 2573 |
.. function:: doModuleCleanups() |
| 2574 |
|
| 2575 |
This function is called unconditionally after :func:`tearDownModule`, or |
| 2576 |
after :func:`setUpModule` if :func:`setUpModule` raises an exception. |
| 2577 |
|
| 2578 |
It is responsible for calling all the cleanup functions added by |
| 2579 |
:func:`addModuleCleanup`. If you need cleanup functions to be called |
| 2580 |
*prior* to :func:`tearDownModule` then you can call |
| 2581 |
:func:`doModuleCleanups` yourself. |
| 2582 |
|
| 2583 |
:func:`doModuleCleanups` pops methods off the stack of cleanup |
| 2584 |
functions one at a time, so it can be called at any time. |
| 2585 |
|
| 2586 |
.. versionadded:: 3.8 |
| 2587 |
|
| 2588 |
|
| 2589 |
Signal Handling |
| 2590 |
--------------- |
| 2591 |
|
| 2592 |
.. versionadded:: 3.2 |
| 2593 |
|
| 2594 |
The :option:`-c/--catch <unittest -c>` command-line option to unittest, |
| 2595 |
along with the ``catchbreak`` parameter to :func:`unittest.main`, provide |
| 2596 |
more friendly handling of control-C during a test run. With catch break |
| 2597 |
behavior enabled control-C will allow the currently running test to complete, |
| 2598 |
and the test run will then end and report all the results so far. A second |
| 2599 |
control-c will raise a :exc:`KeyboardInterrupt` in the usual way. |
| 2600 |
|
| 2601 |
The control-c handling signal handler attempts to remain compatible with code or |
| 2602 |
tests that install their own :const:`signal.SIGINT` handler. If the ``unittest`` |
| 2603 |
handler is called but *isn't* the installed :const:`signal.SIGINT` handler, |
| 2604 |
i.e. it has been replaced by the system under test and delegated to, then it |
| 2605 |
calls the default handler. This will normally be the expected behavior by code |
| 2606 |
that replaces an installed handler and delegates to it. For individual tests |
| 2607 |
that need ``unittest`` control-c handling disabled the :func:`removeHandler` |
| 2608 |
decorator can be used. |
| 2609 |
|
| 2610 |
There are a few utility functions for framework authors to enable control-c |
| 2611 |
handling functionality within test frameworks. |
| 2612 |
|
| 2613 |
.. function:: installHandler() |
| 2614 |
|
| 2615 |
Install the control-c handler. When a :const:`signal.SIGINT` is received |
| 2616 |
(usually in response to the user pressing control-c) all registered results |
| 2617 |
have :meth:`~TestResult.stop` called. |
| 2618 |
|
| 2619 |
|
| 2620 |
.. function:: registerResult(result) |
| 2621 |
|
| 2622 |
Register a :class:`TestResult` object for control-c handling. Registering a |
| 2623 |
result stores a weak reference to it, so it doesn't prevent the result from |
| 2624 |
being garbage collected. |
| 2625 |
|
| 2626 |
Registering a :class:`TestResult` object has no side-effects if control-c |
| 2627 |
handling is not enabled, so test frameworks can unconditionally register |
| 2628 |
all results they create independently of whether or not handling is enabled. |
| 2629 |
|
| 2630 |
|
| 2631 |
.. function:: removeResult(result) |
| 2632 |
|
| 2633 |
Remove a registered result. Once a result has been removed then |
| 2634 |
:meth:`~TestResult.stop` will no longer be called on that result object in |
| 2635 |
response to a control-c. |
| 2636 |
|
| 2637 |
|
| 2638 |
.. function:: removeHandler(function=None) |
| 2639 |
|
| 2640 |
When called without arguments this function removes the control-c handler |
| 2641 |
if it has been installed. This function can also be used as a test decorator |
| 2642 |
to temporarily remove the handler while the test is being executed:: |
| 2643 |
|
| 2644 |
@unittest.removeHandler |
| 2645 |
def test_signal_handling(self): |
| 2646 |
... |