| 1 |
:mod:`!shutil` --- High-level file operations |
| 2 |
============================================= |
| 3 |
|
| 4 |
.. module:: shutil |
| 5 |
:synopsis: High-level file operations, including copying. |
| 6 |
|
| 7 |
.. sectionauthor:: Fred L. Drake, Jr. <[email protected]> |
| 8 |
.. partly based on the docstrings |
| 9 |
|
| 10 |
**Source code:** :source:`Lib/shutil.py` |
| 11 |
|
| 12 |
.. index:: |
| 13 |
single: file; copying |
| 14 |
single: copying files |
| 15 |
|
| 16 |
-------------- |
| 17 |
|
| 18 |
The :mod:`!shutil` module offers a number of high-level operations on files and |
| 19 |
collections of files. In particular, functions are provided which support file |
| 20 |
copying and removal. For operations on individual files, see also the |
| 21 |
:mod:`os` module. |
| 22 |
|
| 23 |
.. warning:: |
| 24 |
|
| 25 |
Even the higher-level file copying functions (:func:`shutil.copy`, |
| 26 |
:func:`shutil.copy2`) cannot copy all file metadata. |
| 27 |
|
| 28 |
On POSIX platforms, this means that file owner and group are lost as well |
| 29 |
as ACLs. On Mac OS, the resource fork and other metadata are not used. |
| 30 |
This means that resources will be lost and file type and creator codes will |
| 31 |
not be correct. On Windows, file owners, ACLs and alternate data streams |
| 32 |
are not copied. |
| 33 |
|
| 34 |
|
| 35 |
.. _file-operations: |
| 36 |
|
| 37 |
Directory and files operations |
| 38 |
------------------------------ |
| 39 |
|
| 40 |
.. function:: copyfileobj(fsrc, fdst[, length]) |
| 41 |
|
| 42 |
Copy the contents of the :term:`file-like object <file object>` *fsrc* to the file-like object *fdst*. |
| 43 |
The integer *length*, if given, is the buffer size. In particular, a negative |
| 44 |
*length* value means to copy the data without looping over the source data in |
| 45 |
chunks; by default the data is read in chunks to avoid uncontrolled memory |
| 46 |
consumption. Note that if the current file position of the *fsrc* object is not |
| 47 |
0, only the contents from the current file position to the end of the file will |
| 48 |
be copied. |
| 49 |
|
| 50 |
:func:`copyfileobj` will *not* guarantee that the destination stream has |
| 51 |
been flushed on completion of the copy. If you want to read from the |
| 52 |
destination at the completion of the copy operation (for example, reading |
| 53 |
the contents of a temporary file that has been copied from a HTTP stream), |
| 54 |
you must ensure that you have called :func:`~io.IOBase.flush` or |
| 55 |
:func:`~io.IOBase.close` on the file-like object before attempting to read |
| 56 |
the destination file. |
| 57 |
|
| 58 |
.. function:: copyfile(src, dst, *, follow_symlinks=True) |
| 59 |
|
| 60 |
Copy the contents (no metadata) of the file named *src* to a file named |
| 61 |
*dst* and return *dst* in the most efficient way possible. |
| 62 |
*src* and *dst* are :term:`path-like objects <path-like object>` or path names given as strings. |
| 63 |
|
| 64 |
*dst* must be the complete target file name; look at :func:`~shutil.copy` |
| 65 |
for a copy that accepts a target directory path. If *src* and *dst* |
| 66 |
specify the same file, :exc:`SameFileError` is raised. |
| 67 |
|
| 68 |
The destination location must be writable; otherwise, an :exc:`OSError` |
| 69 |
exception will be raised. If *dst* already exists, it will be replaced. |
| 70 |
Special files such as character or block devices and pipes cannot be |
| 71 |
copied with this function. |
| 72 |
|
| 73 |
If *follow_symlinks* is false and *src* is a symbolic link, |
| 74 |
a new symbolic link will be created instead of copying the |
| 75 |
file *src* points to. |
| 76 |
|
| 77 |
.. audit-event:: shutil.copyfile src,dst shutil.copyfile |
| 78 |
|
| 79 |
.. versionchanged:: 3.3 |
| 80 |
:exc:`IOError` used to be raised instead of :exc:`OSError`. |
| 81 |
Added *follow_symlinks* argument. |
| 82 |
Now returns *dst*. |
| 83 |
|
| 84 |
.. versionchanged:: 3.4 |
| 85 |
Raise :exc:`SameFileError` instead of :exc:`Error`. Since the former is |
| 86 |
a subclass of the latter, this change is backward compatible. |
| 87 |
|
| 88 |
.. versionchanged:: 3.8 |
| 89 |
Platform-specific fast-copy syscalls may be used internally in order to |
| 90 |
copy the file more efficiently. See |
| 91 |
:ref:`shutil-platform-dependent-efficient-copy-operations` section. |
| 92 |
|
| 93 |
.. exception:: SpecialFileError |
| 94 |
|
| 95 |
This exception is raised when :func:`copyfile` or :func:`copytree` attempt |
| 96 |
to copy a named pipe. |
| 97 |
|
| 98 |
.. versionadded:: 2.7 |
| 99 |
|
| 100 |
.. exception:: SameFileError |
| 101 |
|
| 102 |
This exception is raised if source and destination in :func:`copyfile` |
| 103 |
are the same file. |
| 104 |
|
| 105 |
.. versionadded:: 3.4 |
| 106 |
|
| 107 |
|
| 108 |
.. function:: copymode(src, dst, *, follow_symlinks=True) |
| 109 |
|
| 110 |
Copy the permission bits from *src* to *dst*. The file contents, owner, and |
| 111 |
group are unaffected. *src* and *dst* are :term:`path-like objects <path-like object>` or path names |
| 112 |
given as strings. |
| 113 |
If *follow_symlinks* is false, and both *src* and *dst* are symbolic links, |
| 114 |
:func:`copymode` will attempt to modify the mode of *dst* itself (rather |
| 115 |
than the file it points to). This functionality is not available on every |
| 116 |
platform; please see :func:`copystat` for more information. If |
| 117 |
:func:`copymode` cannot modify symbolic links on the local platform, and it |
| 118 |
is asked to do so, it will do nothing and return. |
| 119 |
|
| 120 |
.. audit-event:: shutil.copymode src,dst shutil.copymode |
| 121 |
|
| 122 |
.. versionchanged:: 3.3 |
| 123 |
Added *follow_symlinks* argument. |
| 124 |
|
| 125 |
.. function:: copystat(src, dst, *, follow_symlinks=True) |
| 126 |
|
| 127 |
Copy the permission bits, last access time, last modification time, and |
| 128 |
flags from *src* to *dst*. On Linux, :func:`copystat` also copies the |
| 129 |
"extended attributes" where possible. The file contents, owner, and |
| 130 |
group are unaffected. *src* and *dst* are :term:`path-like objects <path-like object>` or path |
| 131 |
names given as strings. |
| 132 |
|
| 133 |
If *follow_symlinks* is false, and *src* and *dst* both |
| 134 |
refer to symbolic links, :func:`copystat` will operate on |
| 135 |
the symbolic links themselves rather than the files the |
| 136 |
symbolic links refer to—reading the information from the |
| 137 |
*src* symbolic link, and writing the information to the |
| 138 |
*dst* symbolic link. |
| 139 |
|
| 140 |
.. note:: |
| 141 |
|
| 142 |
Not all platforms provide the ability to examine and |
| 143 |
modify symbolic links. Python itself can tell you what |
| 144 |
functionality is locally available. |
| 145 |
|
| 146 |
* If ``os.chmod in os.supports_follow_symlinks`` is |
| 147 |
``True``, :func:`copystat` can modify the permission |
| 148 |
bits of a symbolic link. |
| 149 |
|
| 150 |
* If ``os.utime in os.supports_follow_symlinks`` is |
| 151 |
``True``, :func:`copystat` can modify the last access |
| 152 |
and modification times of a symbolic link. |
| 153 |
|
| 154 |
* If ``os.chflags in os.supports_follow_symlinks`` is |
| 155 |
``True``, :func:`copystat` can modify the flags of |
| 156 |
a symbolic link. (``os.chflags`` is not available on |
| 157 |
all platforms.) |
| 158 |
|
| 159 |
On platforms where some or all of this functionality |
| 160 |
is unavailable, when asked to modify a symbolic link, |
| 161 |
:func:`copystat` will copy everything it can. |
| 162 |
:func:`copystat` never returns failure. |
| 163 |
|
| 164 |
Please see :data:`os.supports_follow_symlinks` |
| 165 |
for more information. |
| 166 |
|
| 167 |
.. audit-event:: shutil.copystat src,dst shutil.copystat |
| 168 |
|
| 169 |
.. versionchanged:: 3.3 |
| 170 |
Added *follow_symlinks* argument and support for Linux extended attributes. |
| 171 |
|
| 172 |
.. function:: copy(src, dst, *, follow_symlinks=True) |
| 173 |
|
| 174 |
Copies the file *src* to the file or directory *dst*. *src* and *dst* |
| 175 |
should be :term:`path-like objects <path-like object>` or strings. If |
| 176 |
*dst* specifies a directory, the file will be copied into *dst* using the |
| 177 |
base filename from *src*. If *dst* specifies a file that already exists, |
| 178 |
it will be replaced. Returns the path to the newly created file. |
| 179 |
|
| 180 |
If *follow_symlinks* is false, and *src* is a symbolic link, |
| 181 |
*dst* will be created as a symbolic link. If *follow_symlinks* |
| 182 |
is true and *src* is a symbolic link, *dst* will be a copy of |
| 183 |
the file *src* refers to. |
| 184 |
|
| 185 |
:func:`~shutil.copy` copies the file data and the file's permission |
| 186 |
mode (see :func:`os.chmod`). Other metadata, like the |
| 187 |
file's creation and modification times, is not preserved. |
| 188 |
To preserve all file metadata from the original, use |
| 189 |
:func:`~shutil.copy2` instead. |
| 190 |
|
| 191 |
.. audit-event:: shutil.copyfile src,dst shutil.copy |
| 192 |
|
| 193 |
.. audit-event:: shutil.copymode src,dst shutil.copy |
| 194 |
|
| 195 |
.. versionchanged:: 3.3 |
| 196 |
Added *follow_symlinks* argument. |
| 197 |
Now returns path to the newly created file. |
| 198 |
|
| 199 |
.. versionchanged:: 3.8 |
| 200 |
Platform-specific fast-copy syscalls may be used internally in order to |
| 201 |
copy the file more efficiently. See |
| 202 |
:ref:`shutil-platform-dependent-efficient-copy-operations` section. |
| 203 |
|
| 204 |
.. function:: copy2(src, dst, *, follow_symlinks=True) |
| 205 |
|
| 206 |
Identical to :func:`~shutil.copy` except that :func:`copy2` |
| 207 |
also attempts to preserve file metadata. |
| 208 |
|
| 209 |
When *follow_symlinks* is false, and *src* is a symbolic |
| 210 |
link, :func:`copy2` attempts to copy all metadata from the |
| 211 |
*src* symbolic link to the newly created *dst* symbolic link. |
| 212 |
However, this functionality is not available on all platforms. |
| 213 |
On platforms where some or all of this functionality is |
| 214 |
unavailable, :func:`copy2` will preserve all the metadata |
| 215 |
it can; :func:`copy2` never raises an exception because it |
| 216 |
cannot preserve file metadata. |
| 217 |
|
| 218 |
:func:`copy2` uses :func:`copystat` to copy the file metadata. |
| 219 |
Please see :func:`copystat` for more information |
| 220 |
about platform support for modifying symbolic link metadata. |
| 221 |
|
| 222 |
.. audit-event:: shutil.copyfile src,dst shutil.copy2 |
| 223 |
|
| 224 |
.. audit-event:: shutil.copystat src,dst shutil.copy2 |
| 225 |
|
| 226 |
.. versionchanged:: 3.3 |
| 227 |
Added *follow_symlinks* argument, try to copy extended |
| 228 |
file system attributes too (currently Linux only). |
| 229 |
Now returns path to the newly created file. |
| 230 |
|
| 231 |
.. versionchanged:: 3.8 |
| 232 |
Platform-specific fast-copy syscalls may be used internally in order to |
| 233 |
copy the file more efficiently. See |
| 234 |
:ref:`shutil-platform-dependent-efficient-copy-operations` section. |
| 235 |
|
| 236 |
.. function:: ignore_patterns(*patterns) |
| 237 |
|
| 238 |
This factory function creates a function that can be used as a callable for |
| 239 |
:func:`copytree`\'s *ignore* argument, ignoring files and directories that |
| 240 |
match one of the glob-style *patterns* provided. See the example below. |
| 241 |
|
| 242 |
|
| 243 |
.. function:: copytree(src, dst, symlinks=False, ignore=None, \ |
| 244 |
copy_function=copy2, ignore_dangling_symlinks=False, \ |
| 245 |
dirs_exist_ok=False) |
| 246 |
|
| 247 |
Recursively copy an entire directory tree rooted at *src* to a directory |
| 248 |
named *dst* and return the destination directory. All intermediate |
| 249 |
directories needed to contain *dst* will also be created by default. |
| 250 |
|
| 251 |
Permissions and times of directories are copied with :func:`copystat`, |
| 252 |
individual files are copied using :func:`~shutil.copy2`. |
| 253 |
|
| 254 |
If *symlinks* is true, symbolic links in the source tree are represented as |
| 255 |
symbolic links in the new tree and the metadata of the original links will |
| 256 |
be copied as far as the platform allows; if false or omitted, the contents |
| 257 |
and metadata of the linked files are copied to the new tree. |
| 258 |
|
| 259 |
When *symlinks* is false, if the file pointed to by the symlink doesn't |
| 260 |
exist, an exception will be added in the list of errors raised in |
| 261 |
an :exc:`Error` exception at the end of the copy process. |
| 262 |
You can set the optional *ignore_dangling_symlinks* flag to true if you |
| 263 |
want to silence this exception. Notice that this option has no effect |
| 264 |
on platforms that don't support :func:`os.symlink`. |
| 265 |
|
| 266 |
If *ignore* is given, it must be a callable that will receive as its |
| 267 |
arguments the directory being visited by :func:`copytree`, and a list of its |
| 268 |
contents, as returned by :func:`os.listdir`. Since :func:`copytree` is |
| 269 |
called recursively, the *ignore* callable will be called once for each |
| 270 |
directory that is copied. The callable must return a sequence of directory |
| 271 |
and file names relative to the current directory (i.e. a subset of the items |
| 272 |
in its second argument); these names will then be ignored in the copy |
| 273 |
process. :func:`ignore_patterns` can be used to create such a callable that |
| 274 |
ignores names based on glob-style patterns. |
| 275 |
|
| 276 |
If exception(s) occur, an :exc:`Error` is raised with a list of reasons. |
| 277 |
|
| 278 |
If *copy_function* is given, it must be a callable that will be used to copy |
| 279 |
each file. It will be called with the source path and the destination path |
| 280 |
as arguments. By default, :func:`~shutil.copy2` is used, but any function |
| 281 |
that supports the same signature (like :func:`~shutil.copy`) can be used. |
| 282 |
|
| 283 |
If *dirs_exist_ok* is false (the default) and *dst* already exists, a |
| 284 |
:exc:`FileExistsError` is raised. If *dirs_exist_ok* is true, the copying |
| 285 |
operation will continue if it encounters existing directories, and files |
| 286 |
within the *dst* tree will be overwritten by corresponding files from the |
| 287 |
*src* tree. |
| 288 |
|
| 289 |
.. audit-event:: shutil.copytree src,dst shutil.copytree |
| 290 |
|
| 291 |
.. versionchanged:: 3.2 |
| 292 |
Added the *copy_function* argument to be able to provide a custom copy |
| 293 |
function. |
| 294 |
Added the *ignore_dangling_symlinks* argument to silence dangling symlinks |
| 295 |
errors when *symlinks* is false. |
| 296 |
|
| 297 |
.. versionchanged:: 3.3 |
| 298 |
Copy metadata when *symlinks* is false. |
| 299 |
Now returns *dst*. |
| 300 |
|
| 301 |
.. versionchanged:: 3.8 |
| 302 |
Platform-specific fast-copy syscalls may be used internally in order to |
| 303 |
copy the file more efficiently. See |
| 304 |
:ref:`shutil-platform-dependent-efficient-copy-operations` section. |
| 305 |
|
| 306 |
.. versionchanged:: 3.8 |
| 307 |
Added the *dirs_exist_ok* parameter. |
| 308 |
|
| 309 |
.. function:: rmtree(path, ignore_errors=False, onerror=None, *, onexc=None, dir_fd=None) |
| 310 |
|
| 311 |
.. index:: single: directory; deleting |
| 312 |
|
| 313 |
Delete an entire directory tree; *path* must point to a directory (but not a |
| 314 |
symbolic link to a directory). If *ignore_errors* is true, errors resulting |
| 315 |
from failed removals will be ignored; if false or omitted, such errors are |
| 316 |
handled by calling a handler specified by *onexc* or *onerror* or, if both |
| 317 |
are omitted, exceptions are propagated to the caller. |
| 318 |
|
| 319 |
This function can support :ref:`paths relative to directory descriptors |
| 320 |
<dir_fd>`. |
| 321 |
|
| 322 |
.. note:: |
| 323 |
|
| 324 |
On platforms that support the necessary fd-based functions a symlink |
| 325 |
attack resistant version of :func:`rmtree` is used by default. On other |
| 326 |
platforms, the :func:`rmtree` implementation is susceptible to a symlink |
| 327 |
attack: given proper timing and circumstances, attackers can manipulate |
| 328 |
symlinks on the filesystem to delete files they wouldn't be able to access |
| 329 |
otherwise. Applications can use the :data:`rmtree.avoids_symlink_attacks` |
| 330 |
function attribute to determine which case applies. |
| 331 |
|
| 332 |
If *onexc* is provided, it must be a callable that accepts three parameters: |
| 333 |
*function*, *path*, and *excinfo*. |
| 334 |
|
| 335 |
The first parameter, *function*, is the function which raised the exception; |
| 336 |
it depends on the platform and implementation. The second parameter, |
| 337 |
*path*, will be the path name passed to *function*. The third parameter, |
| 338 |
*excinfo*, is the exception that was raised. Exceptions raised by *onexc* |
| 339 |
will not be caught. |
| 340 |
|
| 341 |
The deprecated *onerror* is similar to *onexc*, except that the third |
| 342 |
parameter it receives is the tuple returned from :func:`sys.exc_info`. |
| 343 |
|
| 344 |
.. seealso:: |
| 345 |
:ref:`shutil-rmtree-example` for an example of handling the removal |
| 346 |
of a directory tree that contains read-only files. |
| 347 |
|
| 348 |
.. audit-event:: shutil.rmtree path,dir_fd shutil.rmtree |
| 349 |
|
| 350 |
.. versionchanged:: 3.3 |
| 351 |
Added a symlink attack resistant version that is used automatically |
| 352 |
if platform supports fd-based functions. |
| 353 |
|
| 354 |
.. versionchanged:: 3.8 |
| 355 |
On Windows, will no longer delete the contents of a directory junction |
| 356 |
before removing the junction. |
| 357 |
|
| 358 |
.. versionchanged:: 3.11 |
| 359 |
Added the *dir_fd* parameter. |
| 360 |
|
| 361 |
.. versionchanged:: 3.12 |
| 362 |
Added the *onexc* parameter, deprecated *onerror*. |
| 363 |
|
| 364 |
.. versionchanged:: 3.13 |
| 365 |
:func:`!rmtree` now ignores :exc:`FileNotFoundError` exceptions for all |
| 366 |
but the top-level path. |
| 367 |
Exceptions other than :exc:`OSError` and subclasses of :exc:`!OSError` |
| 368 |
are now always propagated to the caller. |
| 369 |
|
| 370 |
.. attribute:: rmtree.avoids_symlink_attacks |
| 371 |
|
| 372 |
Indicates whether the current platform and implementation provides a |
| 373 |
symlink attack resistant version of :func:`rmtree`. Currently this is |
| 374 |
only true for platforms supporting fd-based directory access functions. |
| 375 |
|
| 376 |
.. versionadded:: 3.3 |
| 377 |
|
| 378 |
|
| 379 |
.. function:: move(src, dst, copy_function=copy2) |
| 380 |
|
| 381 |
Recursively move a file or directory (*src*) to another location and return |
| 382 |
the destination. |
| 383 |
|
| 384 |
If *dst* is an existing directory or a symlink to a directory, then *src* |
| 385 |
is moved inside that directory. The destination path in that directory must |
| 386 |
not already exist. |
| 387 |
|
| 388 |
If *dst* already exists but is not a directory, it may be overwritten |
| 389 |
depending on :func:`os.rename` semantics. |
| 390 |
|
| 391 |
:func:`os.rename` is preferably used internally when *src* and the destination are on |
| 392 |
the same filesystem. In case :func:`os.rename` fails due to :exc:`OSError` |
| 393 |
(e.g. the user has write permission to the destination file but not to its parent |
| 394 |
directory), this method falls back to using *copy_function*, in which case |
| 395 |
*src* is copied to the destination using *copy_function* and then removed. |
| 396 |
|
| 397 |
In case of symlinks, a new symlink pointing to the target of *src* will be |
| 398 |
created in or as the destination, and *src* will be removed. |
| 399 |
|
| 400 |
If *copy_function* is given, it must be a callable that takes two arguments, |
| 401 |
*src* and the destination, and will be used to copy *src* to the destination |
| 402 |
if :func:`os.rename` cannot be used. If the source is a directory, |
| 403 |
:func:`copytree` is called, passing it the *copy_function*. The |
| 404 |
default *copy_function* is :func:`copy2`. Using :func:`~shutil.copy` as the |
| 405 |
*copy_function* allows the move to succeed when it is not possible to also |
| 406 |
copy the metadata, at the expense of not copying any of the metadata. |
| 407 |
|
| 408 |
.. audit-event:: shutil.move src,dst shutil.move |
| 409 |
|
| 410 |
.. versionchanged:: 3.3 |
| 411 |
Added explicit symlink handling for foreign filesystems, thus adapting |
| 412 |
it to the behavior of GNU's :program:`mv`. |
| 413 |
Now returns *dst*. |
| 414 |
|
| 415 |
.. versionchanged:: 3.5 |
| 416 |
Added the *copy_function* keyword argument. |
| 417 |
|
| 418 |
.. versionchanged:: 3.8 |
| 419 |
Platform-specific fast-copy syscalls may be used internally in order to |
| 420 |
copy the file more efficiently. See |
| 421 |
:ref:`shutil-platform-dependent-efficient-copy-operations` section. |
| 422 |
|
| 423 |
.. versionchanged:: 3.9 |
| 424 |
Accepts a :term:`path-like object` for both *src* and *dst*. |
| 425 |
|
| 426 |
.. function:: disk_usage(path) |
| 427 |
|
| 428 |
Return disk usage statistics about the given path as a :term:`named tuple` |
| 429 |
with the attributes *total*, *used* and *free*, which are the amount of |
| 430 |
total, used and free space, in bytes. *path* may be a file or a |
| 431 |
directory. |
| 432 |
|
| 433 |
.. note:: |
| 434 |
|
| 435 |
On Unix filesystems, *path* must point to a path within a **mounted** |
| 436 |
filesystem partition. On those platforms, CPython doesn't attempt to |
| 437 |
retrieve disk usage information from non-mounted filesystems. |
| 438 |
|
| 439 |
.. versionadded:: 3.3 |
| 440 |
|
| 441 |
.. versionchanged:: 3.8 |
| 442 |
On Windows, *path* can now be a file or directory. |
| 443 |
|
| 444 |
.. availability:: Unix, Windows. |
| 445 |
|
| 446 |
.. function:: chown(path, user=None, group=None, *, dir_fd=None, \ |
| 447 |
follow_symlinks=True) |
| 448 |
|
| 449 |
Change owner *user* and/or *group* of the given *path*. |
| 450 |
|
| 451 |
*user* can be a system user name or a uid; the same applies to *group*. At |
| 452 |
least one argument is required. |
| 453 |
|
| 454 |
See also :func:`os.chown`, the underlying function. |
| 455 |
|
| 456 |
.. audit-event:: shutil.chown path,user,group shutil.chown |
| 457 |
|
| 458 |
.. availability:: Unix. |
| 459 |
|
| 460 |
.. versionadded:: 3.3 |
| 461 |
|
| 462 |
.. versionchanged:: 3.13 |
| 463 |
Added *dir_fd* and *follow_symlinks* parameters. |
| 464 |
|
| 465 |
|
| 466 |
.. function:: which(cmd, mode=os.F_OK | os.X_OK, path=None) |
| 467 |
|
| 468 |
Return the path to an executable which would be run if the given *cmd* was |
| 469 |
called. If no *cmd* would be called, return ``None``. |
| 470 |
|
| 471 |
*mode* is a permission mask passed to :func:`os.access`, by default |
| 472 |
determining if the file exists and is executable. |
| 473 |
|
| 474 |
*path* is a "``PATH`` string" specifying the directories to look in, |
| 475 |
delimited by :data:`os.pathsep`. When no *path* is specified, the |
| 476 |
:envvar:`PATH` environment variable is read from :data:`os.environ`, |
| 477 |
falling back to :data:`os.defpath` if it is not set. |
| 478 |
|
| 479 |
If *cmd* contains a directory component, :func:`!which` only checks the |
| 480 |
specified path directly and does not search the directories listed in |
| 481 |
*path* or in the system's :envvar:`PATH` environment variable. |
| 482 |
|
| 483 |
On Windows, the current directory is prepended to the *path* if *mode* does |
| 484 |
not include ``os.X_OK``. When the *mode* does include ``os.X_OK``, the |
| 485 |
Windows API ``NeedCurrentDirectoryForExePathW`` will be consulted to |
| 486 |
determine if the current directory should be prepended to *path*. To avoid |
| 487 |
consulting the current working directory for executables: set the environment |
| 488 |
variable ``NoDefaultCurrentDirectoryInExePath``. |
| 489 |
|
| 490 |
Also on Windows, the :envvar:`PATHEXT` environment variable is used to |
| 491 |
resolve commands that may not already include an extension. For example, |
| 492 |
if you call ``shutil.which("python")``, :func:`which` will search ``PATHEXT`` |
| 493 |
to know that it should look for ``python.exe`` within the *path* |
| 494 |
directories. For example, on Windows:: |
| 495 |
|
| 496 |
>>> shutil.which("python") |
| 497 |
'C:\\Python33\\python.EXE' |
| 498 |
|
| 499 |
This is also applied when *cmd* is a path that contains a directory |
| 500 |
component:: |
| 501 |
|
| 502 |
>>> shutil.which("C:\\Python33\\python") |
| 503 |
'C:\\Python33\\python.EXE' |
| 504 |
|
| 505 |
.. versionadded:: 3.3 |
| 506 |
|
| 507 |
.. versionchanged:: 3.8 |
| 508 |
The :class:`bytes` type is now accepted. If *cmd* type is |
| 509 |
:class:`bytes`, the result type is also :class:`bytes`. |
| 510 |
|
| 511 |
.. versionchanged:: 3.12 |
| 512 |
On Windows, the current directory is no longer prepended to the search |
| 513 |
path if *mode* includes ``os.X_OK`` and WinAPI |
| 514 |
``NeedCurrentDirectoryForExePathW(cmd)`` is false, else the current |
| 515 |
directory is prepended even if it is already in the search path; |
| 516 |
``PATHEXT`` is used now even when *cmd* includes a directory component |
| 517 |
or ends with an extension that is in ``PATHEXT``; and filenames that |
| 518 |
have no extension can now be found. |
| 519 |
|
| 520 |
.. exception:: Error |
| 521 |
|
| 522 |
This exception collects exceptions that are raised during a multi-file |
| 523 |
operation. For :func:`copytree`, the exception argument is a list of 3-tuples |
| 524 |
(*srcname*, *dstname*, *exception*). |
| 525 |
|
| 526 |
.. _shutil-platform-dependent-efficient-copy-operations: |
| 527 |
|
| 528 |
Platform-dependent efficient copy operations |
| 529 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 530 |
|
| 531 |
Starting from Python 3.8, all functions involving a file copy |
| 532 |
(:func:`copyfile`, :func:`~shutil.copy`, :func:`copy2`, |
| 533 |
:func:`copytree`, and :func:`move`) may use |
| 534 |
platform-specific "fast-copy" syscalls in order to copy the file more |
| 535 |
efficiently (see :issue:`33671`). |
| 536 |
"fast-copy" means that the copying operation occurs within the kernel, avoiding |
| 537 |
the use of userspace buffers in Python as in "``outfd.write(infd.read())``". |
| 538 |
|
| 539 |
On macOS `fcopyfile`_ is used to copy the file content (not metadata). |
| 540 |
|
| 541 |
On Linux :func:`os.copy_file_range` or :func:`os.sendfile` is used. |
| 542 |
|
| 543 |
On Solaris :func:`os.sendfile` is used. |
| 544 |
|
| 545 |
On Windows :func:`shutil.copyfile` uses a bigger default buffer size (1 MiB |
| 546 |
instead of 64 KiB) and a :func:`memoryview`-based variant of |
| 547 |
:func:`shutil.copyfileobj` is used. |
| 548 |
|
| 549 |
If the fast-copy operation fails and no data was written in the destination |
| 550 |
file then shutil will silently fall back to less efficient |
| 551 |
:func:`copyfileobj` function internally. |
| 552 |
|
| 553 |
.. versionchanged:: 3.8 |
| 554 |
|
| 555 |
.. versionchanged:: 3.14 |
| 556 |
Solaris now uses :func:`os.sendfile`. |
| 557 |
|
| 558 |
.. versionchanged:: 3.14 |
| 559 |
Copy-on-write or server-side copy may be used internally via |
| 560 |
:func:`os.copy_file_range` on supported Linux filesystems. |
| 561 |
|
| 562 |
.. _shutil-copytree-example: |
| 563 |
|
| 564 |
copytree example |
| 565 |
~~~~~~~~~~~~~~~~ |
| 566 |
|
| 567 |
An example that uses the :func:`ignore_patterns` helper:: |
| 568 |
|
| 569 |
from shutil import copytree, ignore_patterns |
| 570 |
|
| 571 |
copytree(source, destination, ignore=ignore_patterns('*.pyc', 'tmp*')) |
| 572 |
|
| 573 |
This will copy everything except ``.pyc`` files and files or directories whose |
| 574 |
name starts with ``tmp``. |
| 575 |
|
| 576 |
Another example that uses the *ignore* argument to add a logging call:: |
| 577 |
|
| 578 |
from shutil import copytree |
| 579 |
import logging |
| 580 |
|
| 581 |
def _logpath(path, names): |
| 582 |
logging.info('Working in %s', path) |
| 583 |
return [] # nothing will be ignored |
| 584 |
|
| 585 |
copytree(source, destination, ignore=_logpath) |
| 586 |
|
| 587 |
|
| 588 |
.. _shutil-rmtree-example: |
| 589 |
|
| 590 |
rmtree example |
| 591 |
~~~~~~~~~~~~~~ |
| 592 |
|
| 593 |
This example shows how to remove a directory tree on Windows where some |
| 594 |
of the files have their read-only bit set. It uses the onexc callback |
| 595 |
to clear the readonly bit and reattempt the remove. Any subsequent failure |
| 596 |
will propagate. :: |
| 597 |
|
| 598 |
import os, stat |
| 599 |
import shutil |
| 600 |
|
| 601 |
def remove_readonly(func, path, _): |
| 602 |
"Clear the readonly bit and reattempt the removal" |
| 603 |
os.chmod(path, stat.S_IWRITE) |
| 604 |
func(path) |
| 605 |
|
| 606 |
shutil.rmtree(directory, onexc=remove_readonly) |
| 607 |
|
| 608 |
.. _archiving-operations: |
| 609 |
|
| 610 |
Archiving operations |
| 611 |
-------------------- |
| 612 |
|
| 613 |
.. versionadded:: 3.2 |
| 614 |
|
| 615 |
.. versionchanged:: 3.5 |
| 616 |
Added support for the *xztar* format. |
| 617 |
|
| 618 |
|
| 619 |
High-level utilities to create and read compressed and archived files are also |
| 620 |
provided. They rely on the :mod:`zipfile` and :mod:`tarfile` modules. |
| 621 |
|
| 622 |
.. function:: make_archive(base_name, format, [root_dir, [base_dir, [verbose, [dry_run, [owner, [group, [logger]]]]]]]) |
| 623 |
|
| 624 |
Create an archive file (such as zip or tar) and return its name. |
| 625 |
|
| 626 |
*base_name* is the name of the file to create, including the path, minus |
| 627 |
any format-specific extension. |
| 628 |
|
| 629 |
*format* is the archive format: one of |
| 630 |
"zip" (if the :mod:`zlib` module is available), "tar", "gztar" (if the |
| 631 |
:mod:`zlib` module is available), "bztar" (if the :mod:`bz2` module is |
| 632 |
available), "xztar" (if the :mod:`lzma` module is available), or "zstdtar" |
| 633 |
(if the :mod:`compression.zstd` module is available). |
| 634 |
|
| 635 |
*root_dir* is a directory that will be the root directory of the |
| 636 |
archive, all paths in the archive will be relative to it; for example, |
| 637 |
we typically chdir into *root_dir* before creating the archive. |
| 638 |
|
| 639 |
*base_dir* is the directory where we start archiving from; |
| 640 |
i.e. *base_dir* will be the common prefix of all files and |
| 641 |
directories in the archive. *base_dir* must be given relative |
| 642 |
to *root_dir*. See :ref:`shutil-archiving-example-with-basedir` for how to |
| 643 |
use *base_dir* and *root_dir* together. |
| 644 |
|
| 645 |
*root_dir* and *base_dir* both default to the current directory. |
| 646 |
|
| 647 |
If *dry_run* is true, no archive is created, but the operations that would be |
| 648 |
executed are logged to *logger*. |
| 649 |
|
| 650 |
*owner* and *group* are used when creating a tar archive. By default, |
| 651 |
uses the current owner and group. |
| 652 |
|
| 653 |
*logger* must be an object compatible with :pep:`282`, usually an instance of |
| 654 |
:class:`logging.Logger`. |
| 655 |
|
| 656 |
The *verbose* argument is unused and deprecated. |
| 657 |
|
| 658 |
.. audit-event:: shutil.make_archive base_name,format,root_dir,base_dir shutil.make_archive |
| 659 |
|
| 660 |
.. note:: |
| 661 |
|
| 662 |
This function is not thread-safe when custom archivers registered |
| 663 |
with :func:`register_archive_format` do not support the *root_dir* |
| 664 |
argument. In this case it |
| 665 |
temporarily changes the current working directory of the process |
| 666 |
to *root_dir* to perform archiving. |
| 667 |
|
| 668 |
.. versionchanged:: 3.8 |
| 669 |
The modern pax (POSIX.1-2001) format is now used instead of |
| 670 |
the legacy GNU format for archives created with ``format="tar"``. |
| 671 |
|
| 672 |
.. versionchanged:: 3.10.6 |
| 673 |
This function is now made thread-safe during creation of standard |
| 674 |
``.zip`` and tar archives. |
| 675 |
|
| 676 |
.. function:: get_archive_formats() |
| 677 |
|
| 678 |
Return a list of supported formats for archiving. |
| 679 |
Each element of the returned sequence is a tuple ``(name, description)``. |
| 680 |
|
| 681 |
By default :mod:`!shutil` provides these formats: |
| 682 |
|
| 683 |
- *zip*: ZIP file (if the :mod:`zlib` module is available). |
| 684 |
- *tar*: Uncompressed tar file. Uses POSIX.1-2001 pax format for new archives. |
| 685 |
- *gztar*: gzip'ed tar-file (if the :mod:`zlib` module is available). |
| 686 |
- *bztar*: bzip2'ed tar-file (if the :mod:`bz2` module is available). |
| 687 |
- *xztar*: xz'ed tar-file (if the :mod:`lzma` module is available). |
| 688 |
- *zstdtar*: Zstandard compressed tar-file (if the :mod:`compression.zstd` |
| 689 |
module is available). |
| 690 |
|
| 691 |
You can register new formats or provide your own archiver for any existing |
| 692 |
formats, by using :func:`register_archive_format`. |
| 693 |
|
| 694 |
|
| 695 |
.. function:: register_archive_format(name, function, [extra_args, [description]]) |
| 696 |
|
| 697 |
Register an archiver for the format *name*. |
| 698 |
|
| 699 |
*function* is the callable that will be used to create archives. The callable |
| 700 |
will receive the *base_name* of the file to create, followed by the |
| 701 |
*base_dir* (which defaults to :data:`os.curdir`) to start archiving from. |
| 702 |
Further arguments are passed as keyword arguments: *owner*, *group*, |
| 703 |
*dry_run* and *logger* (as passed in :func:`make_archive`). |
| 704 |
|
| 705 |
If *function* has the custom attribute ``function.supports_root_dir`` set to ``True``, |
| 706 |
the *root_dir* argument is passed as a keyword argument. |
| 707 |
Otherwise the current working directory of the process is temporarily |
| 708 |
changed to *root_dir* before calling *function*. |
| 709 |
In this case :func:`make_archive` is not thread-safe. |
| 710 |
|
| 711 |
If given, *extra_args* is a sequence of ``(name, value)`` pairs that will be |
| 712 |
used as extra keywords arguments when the archiver callable is used. |
| 713 |
|
| 714 |
*description* is used by :func:`get_archive_formats` which returns the |
| 715 |
list of archivers. Defaults to an empty string. |
| 716 |
|
| 717 |
.. versionchanged:: 3.12 |
| 718 |
Added support for functions supporting the *root_dir* argument. |
| 719 |
|
| 720 |
|
| 721 |
.. function:: unregister_archive_format(name) |
| 722 |
|
| 723 |
Remove the archive format *name* from the list of supported formats. |
| 724 |
|
| 725 |
|
| 726 |
.. function:: unpack_archive(filename[, extract_dir[, format[, filter]]]) |
| 727 |
|
| 728 |
Unpack an archive. *filename* is the full path of the archive. |
| 729 |
|
| 730 |
*extract_dir* is the name of the target directory where the archive is |
| 731 |
unpacked. If not provided, the current working directory is used. |
| 732 |
|
| 733 |
*format* is the archive format: one of "zip", "tar", "gztar", "bztar", |
| 734 |
"xztar", or "zstdtar". Or any other format registered with |
| 735 |
:func:`register_unpack_format`. If not provided, :func:`unpack_archive` |
| 736 |
will use the archive file name extension and see if an unpacker was |
| 737 |
registered for that extension. In case none is found, |
| 738 |
a :exc:`ValueError` is raised. |
| 739 |
|
| 740 |
The keyword-only *filter* argument is passed to the underlying unpacking |
| 741 |
function. For zip files, *filter* is not accepted. |
| 742 |
For tar files, it is recommended to use ``'data'`` (default since Python |
| 743 |
3.14), unless using features specific to tar and UNIX-like filesystems. |
| 744 |
(See :ref:`tarfile-extraction-filter` for details.) |
| 745 |
|
| 746 |
.. audit-event:: shutil.unpack_archive filename,extract_dir,format shutil.unpack_archive |
| 747 |
|
| 748 |
.. warning:: |
| 749 |
|
| 750 |
Never extract archives from untrusted sources without prior inspection. |
| 751 |
It is possible that files are created outside of the path specified in |
| 752 |
the *extract_dir* argument, for example, members that have absolute filenames |
| 753 |
or filenames with ".." components. |
| 754 |
|
| 755 |
Since Python 3.14, the defaults for both built-in formats (zip and tar |
| 756 |
files) will prevent the most dangerous of such security issues, |
| 757 |
but will not prevent *all* unintended behavior. |
| 758 |
Read the :ref:`tarfile-further-verification` |
| 759 |
section for tar-specific details. |
| 760 |
|
| 761 |
.. versionchanged:: 3.7 |
| 762 |
Accepts a :term:`path-like object` for *filename* and *extract_dir*. |
| 763 |
|
| 764 |
.. versionchanged:: 3.12 |
| 765 |
Added the *filter* argument. |
| 766 |
|
| 767 |
.. function:: register_unpack_format(name, extensions, function[, extra_args[, description]]) |
| 768 |
|
| 769 |
Registers an unpack format. *name* is the name of the format and |
| 770 |
*extensions* is a list of extensions corresponding to the format, like |
| 771 |
``.zip`` for Zip files. |
| 772 |
|
| 773 |
*function* is the callable that will be used to unpack archives. The |
| 774 |
callable will receive: |
| 775 |
|
| 776 |
- the path of the archive, as a positional argument; |
| 777 |
- the directory the archive must be extracted to, as a positional argument; |
| 778 |
- possibly a *filter* keyword argument, if it was given to |
| 779 |
:func:`unpack_archive`; |
| 780 |
- additional keyword arguments, specified by *extra_args* as a sequence |
| 781 |
of ``(name, value)`` tuples. |
| 782 |
|
| 783 |
*description* can be provided to describe the format, and will be returned |
| 784 |
by the :func:`get_unpack_formats` function. |
| 785 |
|
| 786 |
|
| 787 |
.. function:: unregister_unpack_format(name) |
| 788 |
|
| 789 |
Unregister an unpack format. *name* is the name of the format. |
| 790 |
|
| 791 |
|
| 792 |
.. function:: get_unpack_formats() |
| 793 |
|
| 794 |
Return a list of all registered formats for unpacking. |
| 795 |
Each element of the returned sequence is a tuple |
| 796 |
``(name, extensions, description)``. |
| 797 |
|
| 798 |
By default :mod:`!shutil` provides these formats: |
| 799 |
|
| 800 |
- *zip*: ZIP file (unpacking compressed files works only if the corresponding |
| 801 |
module is available). |
| 802 |
- *tar*: uncompressed tar file. |
| 803 |
- *gztar*: gzip'ed tar-file (if the :mod:`zlib` module is available). |
| 804 |
- *bztar*: bzip2'ed tar-file (if the :mod:`bz2` module is available). |
| 805 |
- *xztar*: xz'ed tar-file (if the :mod:`lzma` module is available). |
| 806 |
- *zstdtar*: Zstandard compressed tar-file (if the :mod:`compression.zstd` |
| 807 |
module is available). |
| 808 |
|
| 809 |
You can register new formats or provide your own unpacker for any existing |
| 810 |
formats, by using :func:`register_unpack_format`. |
| 811 |
|
| 812 |
|
| 813 |
.. _shutil-archiving-example: |
| 814 |
|
| 815 |
Archiving example |
| 816 |
~~~~~~~~~~~~~~~~~ |
| 817 |
|
| 818 |
In this example, we create a gzip'ed tar-file archive containing all files |
| 819 |
found in the :file:`.ssh` directory of the user:: |
| 820 |
|
| 821 |
>>> from shutil import make_archive |
| 822 |
>>> import os |
| 823 |
>>> archive_name = os.path.expanduser(os.path.join('~', 'myarchive')) |
| 824 |
>>> root_dir = os.path.expanduser(os.path.join('~', '.ssh')) |
| 825 |
>>> make_archive(archive_name, 'gztar', root_dir) |
| 826 |
'/Users/tarek/myarchive.tar.gz' |
| 827 |
|
| 828 |
The resulting archive contains: |
| 829 |
|
| 830 |
.. code-block:: shell-session |
| 831 |
|
| 832 |
$ tar -tzvf /Users/tarek/myarchive.tar.gz |
| 833 |
drwx------ tarek/staff 0 2010-02-01 16:23:40 ./ |
| 834 |
-rw-r--r-- tarek/staff 609 2008-06-09 13:26:54 ./authorized_keys |
| 835 |
-rwxr-xr-x tarek/staff 65 2008-06-09 13:26:54 ./config |
| 836 |
-rwx------ tarek/staff 668 2008-06-09 13:26:54 ./id_dsa |
| 837 |
-rwxr-xr-x tarek/staff 609 2008-06-09 13:26:54 ./id_dsa.pub |
| 838 |
-rw------- tarek/staff 1675 2008-06-09 13:26:54 ./id_rsa |
| 839 |
-rw-r--r-- tarek/staff 397 2008-06-09 13:26:54 ./id_rsa.pub |
| 840 |
-rw-r--r-- tarek/staff 37192 2010-02-06 18:23:10 ./known_hosts |
| 841 |
|
| 842 |
|
| 843 |
.. _shutil-archiving-example-with-basedir: |
| 844 |
|
| 845 |
Archiving example with *base_dir* |
| 846 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 847 |
|
| 848 |
In this example, similar to the `one above <shutil-archiving-example_>`_, |
| 849 |
we show how to use :func:`make_archive`, but this time with the usage of |
| 850 |
*base_dir*. We now have the following directory structure: |
| 851 |
|
| 852 |
.. code-block:: shell-session |
| 853 |
|
| 854 |
$ tree tmp |
| 855 |
tmp |
| 856 |
└── root |
| 857 |
└── structure |
| 858 |
├── content |
| 859 |
└── please_add.txt |
| 860 |
└── do_not_add.txt |
| 861 |
|
| 862 |
In the final archive, :file:`please_add.txt` should be included, but |
| 863 |
:file:`do_not_add.txt` should not. Therefore we use the following:: |
| 864 |
|
| 865 |
>>> from shutil import make_archive |
| 866 |
>>> import os |
| 867 |
>>> archive_name = os.path.expanduser(os.path.join('~', 'myarchive')) |
| 868 |
>>> make_archive( |
| 869 |
... archive_name, |
| 870 |
... 'tar', |
| 871 |
... root_dir='tmp/root', |
| 872 |
... base_dir='structure/content', |
| 873 |
... ) |
| 874 |
'/Users/tarek/myarchive.tar' |
| 875 |
|
| 876 |
Listing the files in the resulting archive gives us: |
| 877 |
|
| 878 |
.. code-block:: shell-session |
| 879 |
|
| 880 |
$ python -m tarfile -l /Users/tarek/myarchive.tar |
| 881 |
structure/content/ |
| 882 |
structure/content/please_add.txt |
| 883 |
|
| 884 |
|
| 885 |
Querying the size of the output terminal |
| 886 |
---------------------------------------- |
| 887 |
|
| 888 |
.. function:: get_terminal_size(fallback=(columns, lines)) |
| 889 |
|
| 890 |
Get the size of the terminal window. |
| 891 |
|
| 892 |
For each of the two dimensions, the environment variable, ``COLUMNS`` |
| 893 |
and ``LINES`` respectively, is checked. If the variable is defined and |
| 894 |
the value is a positive integer, it is used. |
| 895 |
|
| 896 |
When ``COLUMNS`` or ``LINES`` is not defined, which is the common case, |
| 897 |
the terminal connected to :data:`sys.__stdout__` is queried |
| 898 |
by invoking :func:`os.get_terminal_size`. |
| 899 |
|
| 900 |
If the terminal size cannot be successfully queried, either because |
| 901 |
the system doesn't support querying, or because we are not |
| 902 |
connected to a terminal, the value given in ``fallback`` parameter |
| 903 |
is used. ``fallback`` defaults to ``(80, 24)`` which is the default |
| 904 |
size used by many terminal emulators. |
| 905 |
|
| 906 |
The value returned is a named tuple of type :class:`os.terminal_size`. |
| 907 |
|
| 908 |
See also: The Single UNIX Specification, Version 2, |
| 909 |
`Other Environment Variables`_. |
| 910 |
|
| 911 |
.. versionadded:: 3.3 |
| 912 |
|
| 913 |
.. versionchanged:: 3.11 |
| 914 |
The ``fallback`` values are also used if :func:`os.get_terminal_size` |
| 915 |
returns zeroes. |
| 916 |
|
| 917 |
.. _`fcopyfile`: |
| 918 |
http://www.manpagez.com/man/3/copyfile/ |
| 919 |
|
| 920 |
.. _`Other Environment Variables`: |
| 921 |
https://pubs.opengroup.org/onlinepubs/7908799/xbd/envvar.html#tag_002_003 |