Debian Codegraph @

Doc/library/devmode.rst

raw
1 .. _devmode:
2
3 Python Development Mode
4 =======================
5
6 .. versionadded:: 3.7
7
8 The Python Development Mode introduces additional runtime checks that are too
9 expensive to be enabled by default. It should not be more verbose than the
10 default if the code is correct; new warnings are only emitted when an issue is
11 detected.
12
13 It can be enabled using the :option:`-X dev <-X>` command line option or by
14 setting the :envvar:`PYTHONDEVMODE` environment variable to ``1``.
15
16 See also :ref:`Python debug build <debug-build>`.
17
18 Effects of the Python Development Mode
19 --------------------------------------
20
21 Enabling the Python Development Mode is similar to the following command, but
22 with additional effects described below::
23
24 PYTHONMALLOC=debug PYTHONASYNCIODEBUG=1 python -W default -X faulthandler
25
26 Effects of the Python Development Mode:
27
28 * Add ``default`` :ref:`warning filter <describing-warning-filters>`. The
29 following warnings are shown:
30
31 * :exc:`DeprecationWarning`
32 * :exc:`ImportWarning`
33 * :exc:`PendingDeprecationWarning`
34 * :exc:`ResourceWarning`
35
36 Normally, the above warnings are filtered by the default :ref:`warning
37 filters <describing-warning-filters>`.
38
39 It behaves as if the :option:`-W default <-W>` command line option is used.
40
41 Use the :option:`-W error <-W>` command line option or set the
42 :envvar:`PYTHONWARNINGS` environment variable to ``error`` to treat warnings
43 as errors.
44
45 * Install debug hooks on memory allocators to check for:
46
47 * Buffer underflow
48 * Buffer overflow
49 * Memory allocator API violation
50 * Unsafe usage of the GIL
51
52 See the :c:func:`PyMem_SetupDebugHooks` C function.
53
54 It behaves as if the :envvar:`PYTHONMALLOC` environment variable is set to
55 ``debug``.
56
57 To enable the Python Development Mode without installing debug hooks on
58 memory allocators, set the :envvar:`PYTHONMALLOC` environment variable to
59 ``default``.
60
61 * Call :func:`faulthandler.enable` at Python startup to install handlers for
62 the :const:`~signal.SIGSEGV`, :const:`~signal.SIGFPE`,
63 :const:`~signal.SIGABRT`, :const:`~signal.SIGBUS` and
64 :const:`~signal.SIGILL` signals to dump the Python traceback on a crash.
65
66 It behaves as if the :option:`-X faulthandler <-X>` command line option is
67 used or if the :envvar:`PYTHONFAULTHANDLER` environment variable is set to
68 ``1``.
69
70 * Enable :ref:`asyncio debug mode <asyncio-debug-mode>`. For example,
71 :mod:`asyncio` checks for coroutines that were not awaited and logs them.
72
73 It behaves as if the :envvar:`PYTHONASYNCIODEBUG` environment variable is set
74 to ``1``.
75
76 * Check the *encoding* and *errors* arguments for string encoding and decoding
77 operations. Examples: :func:`open`, :meth:`str.encode` and
78 :meth:`bytes.decode`.
79
80 By default, for best performance, the *errors* argument is only checked at
81 the first encoding/decoding error and the *encoding* argument is sometimes
82 ignored for empty strings.
83
84 * The :class:`io.IOBase` destructor logs ``close()`` exceptions.
85 * Set the :attr:`~sys.flags.dev_mode` attribute of :data:`sys.flags` to
86 ``True``.
87
88 The Python Development Mode does not enable the :mod:`tracemalloc` module by
89 default, because the overhead cost (to performance and memory) would be too
90 large. Enabling the :mod:`tracemalloc` module provides additional information
91 on the origin of some errors. For example, :exc:`ResourceWarning` logs the
92 traceback where the resource was allocated, and a buffer overflow error logs
93 the traceback where the memory block was allocated.
94
95 The Python Development Mode does not prevent the :option:`-O` command line
96 option from removing :keyword:`assert` statements nor from setting
97 :const:`__debug__` to ``False``.
98
99 The Python Development Mode can only be enabled at the Python startup. Its
100 value can be read from :data:`sys.flags.dev_mode <sys.flags>`.
101
102 .. versionchanged:: 3.8
103 The :class:`io.IOBase` destructor now logs ``close()`` exceptions.
104
105 .. versionchanged:: 3.9
106 The *encoding* and *errors* arguments are now checked for string encoding
107 and decoding operations.
108
109
110 ResourceWarning Example
111 -----------------------
112
113 Example of a script counting the number of lines of the text file specified in
114 the command line::
115
116 import sys
117
118 def main():
119 fp = open(sys.argv[1])
120 nlines = len(fp.readlines())
121 print(nlines)
122 # The file is closed implicitly
123
124 if __name__ == "__main__":
125 main()
126
127 The script does not close the file explicitly. By default, Python does not emit
128 any warning. Example using README.txt, which has 269 lines:
129
130 .. code-block:: shell-session
131
132 $ python script.py README.txt
133 269
134
135 Enabling the Python Development Mode displays a :exc:`ResourceWarning` warning:
136
137 .. code-block:: shell-session
138
139 $ python -X dev script.py README.txt
140 269
141 script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='README.rst' mode='r' encoding='UTF-8'>
142 main()
143 ResourceWarning: Enable tracemalloc to get the object allocation traceback
144
145 In addition, enabling :mod:`tracemalloc` shows the line where the file was
146 opened:
147
148 .. code-block:: shell-session
149
150 $ python -X dev -X tracemalloc=5 script.py README.rst
151 269
152 script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='README.rst' mode='r' encoding='UTF-8'>
153 main()
154 Object allocated at (most recent call last):
155 File "script.py", lineno 10
156 main()
157 File "script.py", lineno 4
158 fp = open(sys.argv[1])
159
160 The fix is to close explicitly the file. Example using a context manager::
161
162 def main():
163 # Close the file explicitly when exiting the with block
164 with open(sys.argv[1]) as fp:
165 nlines = len(fp.readlines())
166 print(nlines)
167
168 Not closing a resource explicitly can leave a resource open for way longer than
169 expected; it can cause severe issues upon exiting Python. It is bad in
170 CPython, but it is even worse in PyPy. Closing resources explicitly makes an
171 application more deterministic and more reliable.
172
173
174 Bad file descriptor error example
175 ---------------------------------
176
177 Script displaying the first line of itself::
178
179 import os
180
181 def main():
182 fp = open(__file__)
183 firstline = fp.readline()
184 print(firstline.rstrip())
185 os.close(fp.fileno())
186 # The file is closed implicitly
187
188 main()
189
190 By default, Python does not emit any warning:
191
192 .. code-block:: shell-session
193
194 $ python script.py
195 import os
196
197 The Python Development Mode shows a :exc:`ResourceWarning` and logs a "Bad file
198 descriptor" error when finalizing the file object:
199
200 .. code-block:: shell-session
201
202 $ python -X dev script.py
203 import os
204 script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='script.py' mode='r' encoding='UTF-8'>
205 main()
206 ResourceWarning: Enable tracemalloc to get the object allocation traceback
207 Exception ignored in: <_io.TextIOWrapper name='script.py' mode='r' encoding='UTF-8'>
208 Traceback (most recent call last):
209 File "script.py", line 10, in <module>
210 main()
211 OSError: [Errno 9] Bad file descriptor
212
213 ``os.close(fp.fileno())`` closes the file descriptor. When the file object
214 finalizer tries to close the file descriptor again, it fails with the ``Bad
215 file descriptor`` error. A file descriptor must be closed only once. In the
216 worst case scenario, closing it twice can lead to a crash (see :issue:`18748`
217 for an example).
218
219 The fix is to remove the ``os.close(fp.fileno())`` line, or open the file with
220 ``closefd=False``.