Debian Codegraph @

Doc/library/compileall.rst

raw
1 :mod:`!compileall` --- Byte-compile Python libraries
2 ====================================================
3
4 .. module:: compileall
5 :synopsis: Tools for byte-compiling all Python source files in a directory tree.
6
7 **Source code:** :source:`Lib/compileall.py`
8
9 --------------
10
11 This module provides some utility functions to support installing Python
12 libraries. These functions compile Python source files in a directory tree.
13 This module can be used to create the cached byte-code files at library
14 installation time, which makes them available for use even by users who don't
15 have write permission to the library directories.
16
17 .. include:: ../includes/wasm-notavail.rst
18
19 .. _compileall-cli:
20
21 Command-line use
22 ----------------
23
24 This module can work as a script (using :program:`python -m compileall`) to
25 compile Python sources.
26
27 .. program:: compileall
28
29 .. option:: directory ...
30 file ...
31
32 Positional arguments are files to compile or directories that contain
33 source files, traversed recursively. If no argument is given, behave as if
34 the command line was :samp:`-l {<directories from sys.path>}`.
35
36 .. option:: -l
37
38 Do not recurse into subdirectories, only compile source code files directly
39 contained in the named or implied directories.
40
41 .. option:: -f
42
43 Force rebuild even if timestamps are up-to-date.
44
45 .. option:: -q
46
47 Do not print the list of files compiled. If passed once, error messages will
48 still be printed. If passed twice (``-qq``), all output is suppressed.
49
50 .. option:: -d destdir
51
52 Directory prepended to the path to each file being compiled. This will
53 appear in compilation time tracebacks, and is also compiled in to the
54 byte-code file, where it will be used in tracebacks and other messages in
55 cases where the source file does not exist at the time the byte-code file is
56 executed.
57
58 .. option:: -s strip_prefix
59
60 Remove the given prefix from paths recorded in the ``.pyc`` files.
61 Paths are made relative to the prefix.
62
63 This option can be used with ``-p`` but not with ``-d``.
64
65 .. option:: -p prepend_prefix
66
67 Prepend the given prefix to paths recorded in the ``.pyc`` files.
68 Use ``-p /`` to make the paths absolute.
69
70 This option can be used with ``-s`` but not with ``-d``.
71
72 .. option:: -x regex
73
74 regex is used to search the full path to each file considered for
75 compilation, and if the regex produces a match, the file is skipped.
76
77 .. option:: -i list
78
79 Read the file ``list`` and add each line that it contains to the list of
80 files and directories to compile. If ``list`` is ``-``, read lines from
81 ``stdin``.
82
83 .. option:: -b
84
85 Write the byte-code files to their legacy locations and names, which may
86 overwrite byte-code files created by another version of Python. The default
87 is to write files to their :pep:`3147` locations and names, which allows
88 byte-code files from multiple versions of Python to coexist.
89
90 .. option:: -r
91
92 Control the maximum recursion level for subdirectories.
93 If this is given, then ``-l`` option will not be taken into account.
94 :program:`python -m compileall <directory> -r 0` is equivalent to
95 :program:`python -m compileall <directory> -l`.
96
97 .. option:: -j N
98
99 Use *N* workers to compile the files within the given directory.
100 If ``0`` is used, then the result of :func:`os.process_cpu_count`
101 will be used.
102
103 .. option:: --invalidation-mode [timestamp|checked-hash|unchecked-hash]
104
105 Control how the generated byte-code files are invalidated at runtime.
106 The ``timestamp`` value, means that ``.pyc`` files with the source timestamp
107 and size embedded will be generated. The ``checked-hash`` and
108 ``unchecked-hash`` values cause hash-based pycs to be generated. Hash-based
109 pycs embed a hash of the source file contents rather than a timestamp. See
110 :ref:`pyc-invalidation` for more information on how Python validates
111 bytecode cache files at runtime.
112 The default is ``timestamp`` if the :envvar:`SOURCE_DATE_EPOCH` environment
113 variable is not set, and ``checked-hash`` if the ``SOURCE_DATE_EPOCH``
114 environment variable is set.
115
116 .. option:: -o level
117
118 Compile with the given optimization level. May be used multiple times
119 to compile for multiple levels at a time (for example,
120 ``compileall -o 1 -o 2``).
121
122 .. option:: -e dir
123
124 Ignore symlinks pointing outside the given directory.
125
126 .. option:: --hardlink-dupes
127
128 If two ``.pyc`` files with different optimization level have
129 the same content, use hard links to consolidate duplicate files.
130
131 .. versionchanged:: 3.2
132 Added the ``-i``, ``-b`` and ``-h`` options.
133
134 .. versionchanged:: 3.5
135 Added the ``-j``, ``-r``, and ``-qq`` options. ``-q`` option
136 was changed to a multilevel value. ``-b`` will always produce a
137 byte-code file ending in ``.pyc``, never ``.pyo``.
138
139 .. versionchanged:: 3.7
140 Added the ``--invalidation-mode`` option.
141
142 .. versionchanged:: 3.9
143 Added the ``-s``, ``-p``, ``-e`` and ``--hardlink-dupes`` options.
144 Raised the default recursion limit from 10 to
145 :py:func:`sys.getrecursionlimit()`.
146 Added the possibility to specify the ``-o`` option multiple times.
147
148
149 There is no command-line option to control the optimization level used by the
150 :func:`compile` function, because the Python interpreter itself already
151 provides the option: :program:`python -O -m compileall`.
152
153 Similarly, the :func:`compile` function respects the :data:`sys.pycache_prefix`
154 setting. The generated bytecode cache will only be useful if :func:`compile` is
155 run with the same :data:`sys.pycache_prefix` (if any) that will be used at
156 runtime.
157
158 Public functions
159 ----------------
160
161 .. function:: compile_dir(dir, maxlevels=sys.getrecursionlimit(), ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, workers=1, invalidation_mode=None, *, stripdir=None, prependdir=None, limit_sl_dest=None, hardlink_dupes=False)
162
163 Recursively descend the directory tree named by *dir*, compiling all :file:`.py`
164 files along the way. Return a true value if all the files compiled successfully,
165 and a false value otherwise.
166
167 The *maxlevels* parameter is used to limit the depth of the recursion; it
168 defaults to ``sys.getrecursionlimit()``.
169
170 If *ddir* is given, it is prepended to the path to each file being compiled
171 for use in compilation time tracebacks, and is also compiled in to the
172 byte-code file, where it will be used in tracebacks and other messages in
173 cases where the source file does not exist at the time the byte-code file is
174 executed.
175
176 If *force* is true, modules are re-compiled even if the timestamps are up to
177 date.
178
179 If *rx* is given, its ``search`` method is called on the complete path to each
180 file considered for compilation, and if it returns a true value, the file
181 is skipped. This can be used to exclude files matching a regular expression,
182 given as a :ref:`re.Pattern <re-objects>` object.
183
184 If *quiet* is ``False`` or ``0`` (the default), the filenames and other
185 information are printed to standard out. Set to ``1``, only errors are
186 printed. Set to ``2``, all output is suppressed.
187
188 If *legacy* is true, byte-code files are written to their legacy locations
189 and names, which may overwrite byte-code files created by another version of
190 Python. The default is to write files to their :pep:`3147` locations and
191 names, which allows byte-code files from multiple versions of Python to
192 coexist.
193
194 *optimize* specifies the optimization level for the compiler. It is passed to
195 the built-in :func:`compile` function. Accepts also a sequence of optimization
196 levels which lead to multiple compilations of one :file:`.py` file in one call.
197
198 The argument *workers* specifies how many workers are used to
199 compile files in parallel. The default is to not use multiple workers.
200 If the platform can't use multiple workers and *workers* argument is given,
201 then sequential compilation will be used as a fallback. If *workers*
202 is 0, the number of cores in the system is used. If *workers* is
203 lower than ``0``, a :exc:`ValueError` will be raised.
204
205 *invalidation_mode* should be a member of the
206 :class:`py_compile.PycInvalidationMode` enum and controls how the generated
207 pycs are invalidated at runtime.
208
209 The *stripdir*, *prependdir* and *limit_sl_dest* arguments correspond to
210 the ``-s``, ``-p`` and ``-e`` options described above.
211 They may be specified as ``str`` or :py:class:`os.PathLike`.
212
213 If *hardlink_dupes* is true and two ``.pyc`` files with different optimization
214 level have the same content, use hard links to consolidate duplicate files.
215
216 .. versionchanged:: 3.2
217 Added the *legacy* and *optimize* parameter.
218
219 .. versionchanged:: 3.5
220 Added the *workers* parameter.
221
222 .. versionchanged:: 3.5
223 *quiet* parameter was changed to a multilevel value.
224
225 .. versionchanged:: 3.5
226 The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files
227 no matter what the value of *optimize* is.
228
229 .. versionchanged:: 3.6
230 Accepts a :term:`path-like object`.
231
232 .. versionchanged:: 3.7
233 The *invalidation_mode* parameter was added.
234
235 .. versionchanged:: 3.7.2
236 The *invalidation_mode* parameter's default value is updated to ``None``.
237
238 .. versionchanged:: 3.8
239 Setting *workers* to 0 now chooses the optimal number of cores.
240
241 .. versionchanged:: 3.9
242 Added *stripdir*, *prependdir*, *limit_sl_dest* and *hardlink_dupes* arguments.
243 Default value of *maxlevels* was changed from ``10`` to ``sys.getrecursionlimit()``
244
245 .. function:: compile_file(fullname, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, invalidation_mode=None, *, stripdir=None, prependdir=None, limit_sl_dest=None, hardlink_dupes=False)
246
247 Compile the file with path *fullname*. Return a true value if the file
248 compiled successfully, and a false value otherwise.
249
250 If *ddir* is given, it is prepended to the path to the file being compiled
251 for use in compilation time tracebacks, and is also compiled in to the
252 byte-code file, where it will be used in tracebacks and other messages in
253 cases where the source file does not exist at the time the byte-code file is
254 executed.
255
256 If *rx* is given, its ``search`` method is passed the full path name to the
257 file being compiled, and if it returns a true value, the file is not
258 compiled and ``True`` is returned. This can be used to exclude files matching
259 a regular expression, given as a :ref:`re.Pattern <re-objects>` object.
260
261 If *quiet* is ``False`` or ``0`` (the default), the filenames and other
262 information are printed to standard out. Set to ``1``, only errors are
263 printed. Set to ``2``, all output is suppressed.
264
265 If *legacy* is true, byte-code files are written to their legacy locations
266 and names, which may overwrite byte-code files created by another version of
267 Python. The default is to write files to their :pep:`3147` locations and
268 names, which allows byte-code files from multiple versions of Python to
269 coexist.
270
271 *optimize* specifies the optimization level for the compiler. It is passed to
272 the built-in :func:`compile` function. Accepts also a sequence of optimization
273 levels which lead to multiple compilations of one :file:`.py` file in one call.
274
275 *invalidation_mode* should be a member of the
276 :class:`py_compile.PycInvalidationMode` enum and controls how the generated
277 pycs are invalidated at runtime.
278
279 The *stripdir*, *prependdir* and *limit_sl_dest* arguments correspond to
280 the ``-s``, ``-p`` and ``-e`` options described above.
281 They may be specified as ``str`` or :py:class:`os.PathLike`.
282
283 If *hardlink_dupes* is true and two ``.pyc`` files with different optimization
284 level have the same content, use hard links to consolidate duplicate files.
285
286 .. versionadded:: 3.2
287
288 .. versionchanged:: 3.5
289 *quiet* parameter was changed to a multilevel value.
290
291 .. versionchanged:: 3.5
292 The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files
293 no matter what the value of *optimize* is.
294
295 .. versionchanged:: 3.7
296 The *invalidation_mode* parameter was added.
297
298 .. versionchanged:: 3.7.2
299 The *invalidation_mode* parameter's default value is updated to ``None``.
300
301 .. versionchanged:: 3.9
302 Added *stripdir*, *prependdir*, *limit_sl_dest* and *hardlink_dupes* arguments.
303
304 .. function:: compile_path(skip_curdir=True, maxlevels=0, force=False, quiet=0, legacy=False, optimize=-1, invalidation_mode=None)
305
306 Byte-compile all the :file:`.py` files found along ``sys.path``. Return a
307 true value if all the files compiled successfully, and a false value otherwise.
308
309 If *skip_curdir* is true (the default), the current directory is not included
310 in the search. All other parameters are passed to the :func:`compile_dir`
311 function. Note that unlike the other compile functions, ``maxlevels``
312 defaults to ``0``.
313
314 .. versionchanged:: 3.2
315 Added the *legacy* and *optimize* parameter.
316
317 .. versionchanged:: 3.5
318 *quiet* parameter was changed to a multilevel value.
319
320 .. versionchanged:: 3.5
321 The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files
322 no matter what the value of *optimize* is.
323
324 .. versionchanged:: 3.7
325 The *invalidation_mode* parameter was added.
326
327 .. versionchanged:: 3.7.2
328 The *invalidation_mode* parameter's default value is updated to ``None``.
329
330 To force a recompile of all the :file:`.py` files in the :file:`Lib/`
331 subdirectory and all its subdirectories::
332
333 import compileall
334
335 compileall.compile_dir('Lib/', force=True)
336
337 # Perform same compilation, excluding files in .svn directories.
338 import re
339 compileall.compile_dir('Lib/', rx=re.compile(r'[/\\][.]svn'), force=True)
340
341 # pathlib.Path objects can also be used.
342 import pathlib
343 compileall.compile_dir(pathlib.Path('Lib/'), force=True)
344
345 .. seealso::
346
347 Module :mod:`py_compile`
348 Byte-compile a single source file.