Debian Codegraph @

Doc/library/tarfile.rst

raw
1 :mod:`!tarfile` --- Read and write tar archive files
2 ====================================================
3
4 .. module:: tarfile
5 :synopsis: Read and write tar-format archive files.
6
7 .. moduleauthor:: Lars Gustäbel <[email protected]>
8 .. sectionauthor:: Lars Gustäbel <[email protected]>
9
10 **Source code:** :source:`Lib/tarfile.py`
11
12 --------------
13
14 The :mod:`!tarfile` module makes it possible to read and write tar
15 archives, including those using gzip, bz2 and lzma compression.
16 Use the :mod:`zipfile` module to read or write :file:`.zip` files, or the
17 higher-level functions in :ref:`shutil <archiving-operations>`.
18
19 Some facts and figures:
20
21 * reads and writes :mod:`gzip`, :mod:`bz2`, :mod:`compression.zstd`, and
22 :mod:`lzma` compressed archives if the respective modules are available.
23
24 ..
25 The following paragraph should be similar to ../includes/optional-module.rst
26
27 If any of these :term:`optional modules <optional module>` are missing from
28 your copy of CPython, look for documentation from your distributor (that is,
29 whoever provided Python to you).
30 If you are the distributor, see :ref:`optional-module-requirements`.
31
32 * read/write support for the POSIX.1-1988 (ustar) format.
33
34 * read/write support for the GNU tar format including *longname* and *longlink*
35 extensions, read-only support for all variants of the *sparse* extension
36 including restoration of sparse files.
37
38 * read/write support for the POSIX.1-2001 (pax) format.
39
40 * handles directories, regular files, hardlinks, symbolic links, fifos,
41 character devices and block devices and is able to acquire and restore file
42 information like timestamp, access permissions and owner.
43
44 .. versionchanged:: 3.3
45 Added support for :mod:`lzma` compression.
46
47 .. versionchanged:: 3.12
48 Archives are extracted using a :ref:`filter <tarfile-extraction-filter>`,
49 which makes it possible to either limit surprising/dangerous features,
50 or to acknowledge that they are expected and the archive is fully trusted.
51
52 .. versionchanged:: 3.14
53 Set the default extraction filter to :func:`data <data_filter>`,
54 which disallows some dangerous features such as links to absolute paths
55 or paths outside of the destination. Previously, the filter strategy
56 was equivalent to :func:`fully_trusted <fully_trusted_filter>`.
57
58 .. versionchanged:: 3.14
59
60 Added support for Zstandard compression using :mod:`compression.zstd`.
61
62 .. function:: open(name=None, mode='r', fileobj=None, bufsize=10240, **kwargs)
63
64 Return a :class:`TarFile` object for the pathname *name*. For detailed
65 information on :class:`TarFile` objects and the keyword arguments that are
66 allowed, see :ref:`tarfile-objects`.
67
68 *mode* has to be a string of the form ``'filemode[:compression]'``, it defaults
69 to ``'r'``. Here is a full list of mode combinations:
70
71 +------------------+---------------------------------------------+
72 | mode | action |
73 +==================+=============================================+
74 | ``'r'`` or | Open for reading with transparent |
75 | ``'r:*'`` | compression (recommended). |
76 +------------------+---------------------------------------------+
77 | ``'r:'`` | Open for reading exclusively without |
78 | | compression. |
79 +------------------+---------------------------------------------+
80 | ``'r:gz'`` | Open for reading with gzip compression. |
81 +------------------+---------------------------------------------+
82 | ``'r:bz2'`` | Open for reading with bzip2 compression. |
83 +------------------+---------------------------------------------+
84 | ``'r:xz'`` | Open for reading with lzma compression. |
85 +------------------+---------------------------------------------+
86 | ``'r:zst'`` | Open for reading with Zstandard compression.|
87 +------------------+---------------------------------------------+
88 | ``'x'`` or | Create a tarfile exclusively without |
89 | ``'x:'`` | compression. |
90 | | Raise a :exc:`FileExistsError` exception |
91 | | if it already exists. |
92 +------------------+---------------------------------------------+
93 | ``'x:gz'`` | Create a tarfile with gzip compression. |
94 | | Raise a :exc:`FileExistsError` exception |
95 | | if it already exists. |
96 +------------------+---------------------------------------------+
97 | ``'x:bz2'`` | Create a tarfile with bzip2 compression. |
98 | | Raise a :exc:`FileExistsError` exception |
99 | | if it already exists. |
100 +------------------+---------------------------------------------+
101 | ``'x:xz'`` | Create a tarfile with lzma compression. |
102 | | Raise a :exc:`FileExistsError` exception |
103 | | if it already exists. |
104 +------------------+---------------------------------------------+
105 | ``'x:zst'`` | Create a tarfile with Zstandard compression.|
106 | | Raise a :exc:`FileExistsError` exception |
107 | | if it already exists. |
108 +------------------+---------------------------------------------+
109 | ``'a'`` or | Open for appending with no compression. The |
110 | ``'a:'`` | file is created if it does not exist. |
111 +------------------+---------------------------------------------+
112 | ``'w'`` or | Open for uncompressed writing. |
113 | ``'w:'`` | |
114 +------------------+---------------------------------------------+
115 | ``'w:gz'`` | Open for gzip compressed writing. |
116 +------------------+---------------------------------------------+
117 | ``'w:bz2'`` | Open for bzip2 compressed writing. |
118 +------------------+---------------------------------------------+
119 | ``'w:xz'`` | Open for lzma compressed writing. |
120 +------------------+---------------------------------------------+
121 | ``'w:zst'`` | Open for Zstandard compressed writing. |
122 +------------------+---------------------------------------------+
123
124 Note that ``'a:gz'``, ``'a:bz2'`` or ``'a:xz'`` is not possible. If *mode*
125 is not suitable to open a certain (compressed) file for reading,
126 :exc:`ReadError` is raised. Use *mode* ``'r'`` to avoid this. If a
127 compression method is not supported, :exc:`CompressionError` is raised.
128
129 If *fileobj* is specified, it is used as an alternative to a :term:`file object`
130 opened in binary mode for *name*. It is supposed to be at position 0.
131
132 For modes ``'w:gz'``, ``'x:gz'``, ``'w|gz'``, ``'w:bz2'``, ``'x:bz2'``,
133 ``'w|bz2'``, :func:`tarfile.open` accepts the keyword argument
134 *compresslevel* (default ``9``) to specify the compression level of the file.
135
136 For modes ``'w:xz'``, ``'x:xz'`` and ``'w|xz'``, :func:`tarfile.open` accepts the
137 keyword argument *preset* to specify the compression level of the file.
138
139 For modes ``'w:zst'``, ``'x:zst'`` and ``'w|zst'``, :func:`tarfile.open`
140 accepts the keyword argument *level* to specify the compression level of
141 the file. The keyword argument *options* may also be passed, providing
142 advanced Zstandard compression parameters described by
143 :class:`~compression.zstd.CompressionParameter`. The keyword argument
144 *zstd_dict* can be passed to provide a :class:`~compression.zstd.ZstdDict`,
145 a Zstandard dictionary used to improve compression of smaller amounts of
146 data.
147
148 For special purposes, there is a second format for *mode*:
149 ``'filemode|[compression]'``. :func:`tarfile.open` will return a :class:`TarFile`
150 object that processes its data as a stream of blocks. No random seeking will
151 be done on the file. If given, *fileobj* may be any object that has a
152 :meth:`~io.RawIOBase.read` or :meth:`~io.RawIOBase.write` method
153 (depending on the *mode*) that works with bytes.
154 *bufsize* specifies the blocksize and defaults to ``20 * 512`` bytes.
155 Use this variant in combination with e.g. ``sys.stdin.buffer``, a socket
156 :term:`file object` or a tape device.
157 However, such a :class:`TarFile` object is limited in that it does
158 not allow random access, see :ref:`tar-examples`. The currently
159 possible modes:
160
161 +-------------+--------------------------------------------+
162 | Mode | Action |
163 +=============+============================================+
164 | ``'r|*'`` | Open a *stream* of tar blocks for reading |
165 | | with transparent compression. |
166 +-------------+--------------------------------------------+
167 | ``'r|'`` | Open a *stream* of uncompressed tar blocks |
168 | | for reading. |
169 +-------------+--------------------------------------------+
170 | ``'r|gz'`` | Open a gzip compressed *stream* for |
171 | | reading. |
172 +-------------+--------------------------------------------+
173 | ``'r|bz2'`` | Open a bzip2 compressed *stream* for |
174 | | reading. |
175 +-------------+--------------------------------------------+
176 | ``'r|xz'`` | Open an lzma compressed *stream* for |
177 | | reading. |
178 +-------------+--------------------------------------------+
179 | ``'r|zst'`` | Open a Zstandard compressed *stream* for |
180 | | reading. |
181 +-------------+--------------------------------------------+
182 | ``'w|'`` | Open an uncompressed *stream* for writing. |
183 +-------------+--------------------------------------------+
184 | ``'w|gz'`` | Open a gzip compressed *stream* for |
185 | | writing. |
186 +-------------+--------------------------------------------+
187 | ``'w|bz2'`` | Open a bzip2 compressed *stream* for |
188 | | writing. |
189 +-------------+--------------------------------------------+
190 | ``'w|xz'`` | Open an lzma compressed *stream* for |
191 | | writing. |
192 +-------------+--------------------------------------------+
193 | ``'w|zst'`` | Open a Zstandard compressed *stream* for |
194 | | writing. |
195 +-------------+--------------------------------------------+
196
197 .. versionchanged:: 3.5
198 The ``'x'`` (exclusive creation) mode was added.
199
200 .. versionchanged:: 3.6
201 The *name* parameter accepts a :term:`path-like object`.
202
203 .. versionchanged:: 3.12
204 The *compresslevel* keyword argument also works for streams.
205
206 .. versionchanged:: 3.14
207 The *preset* keyword argument also works for streams.
208
209
210 .. class:: TarFile
211 :noindex:
212
213 Class for reading and writing tar archives. Do not use this class directly:
214 use :func:`tarfile.open` instead. See :ref:`tarfile-objects`.
215
216
217 .. function:: is_tarfile(name)
218
219 Return :const:`True` if *name* is a tar archive file, that the :mod:`!tarfile`
220 module can read. *name* may be a :class:`str`, file, or file-like object.
221
222 .. versionchanged:: 3.9
223 Support for file and file-like objects.
224
225
226 The :mod:`!tarfile` module defines the following exceptions:
227
228
229 .. exception:: TarError
230
231 Base class for all :mod:`!tarfile` exceptions.
232
233
234 .. exception:: ReadError
235
236 Is raised when a tar archive is opened, that either cannot be handled by the
237 :mod:`!tarfile` module or is somehow invalid.
238
239
240 .. exception:: CompressionError
241
242 Is raised when a compression method is not supported or when the data cannot be
243 decoded properly.
244
245
246 .. exception:: StreamError
247
248 Is raised for the limitations that are typical for stream-like :class:`TarFile`
249 objects.
250
251
252 .. exception:: ExtractError
253
254 Is raised for *non-fatal* errors when using :meth:`TarFile.extract`, but only if
255 :attr:`TarFile.errorlevel`\ ``== 2``.
256
257
258 .. exception:: HeaderError
259
260 Is raised by :meth:`TarInfo.frombuf` if the buffer it gets is invalid.
261
262
263 .. exception:: FilterError
264
265 Base class for members :ref:`refused <tarfile-extraction-refuse>` by
266 filters.
267
268 .. attribute:: tarinfo
269
270 Information about the member that the filter refused to extract,
271 as :ref:`TarInfo <tarinfo-objects>`.
272
273 .. exception:: AbsolutePathError
274
275 Raised to refuse extracting a member with an absolute path.
276
277 .. exception:: OutsideDestinationError
278
279 Raised to refuse extracting a member outside the destination directory.
280
281 .. exception:: SpecialFileError
282
283 Raised to refuse extracting a special file (e.g. a device or pipe).
284
285 .. exception:: AbsoluteLinkError
286
287 Raised to refuse extracting a symbolic link with an absolute path.
288
289 .. exception:: LinkOutsideDestinationError
290
291 Raised to refuse extracting a symbolic link pointing outside the destination
292 directory.
293
294 .. exception:: LinkFallbackError
295
296 Raised to refuse emulating a link (hard or symbolic) by extracting another
297 archive member, when that member would be rejected by the filter location.
298 The exception that was raised to reject the replacement member is available
299 as :attr:`!BaseException.__context__`.
300
301 .. versionadded:: 3.14
302
303
304 The following constants are available at the module level:
305
306 .. data:: ENCODING
307
308 The default character encoding: ``'utf-8'`` on Windows, the value returned by
309 :func:`sys.getfilesystemencoding` otherwise.
310
311 .. data:: REGTYPE
312 AREGTYPE
313
314 A regular file :attr:`~TarInfo.type`.
315
316 .. data:: LNKTYPE
317
318 A link (inside tarfile) :attr:`~TarInfo.type`.
319
320 .. data:: SYMTYPE
321
322 A symbolic link :attr:`~TarInfo.type`.
323
324 .. data:: CHRTYPE
325
326 A character special device :attr:`~TarInfo.type`.
327
328 .. data:: BLKTYPE
329
330 A block special device :attr:`~TarInfo.type`.
331
332 .. data:: DIRTYPE
333
334 A directory :attr:`~TarInfo.type`.
335
336 .. data:: FIFOTYPE
337
338 A FIFO special device :attr:`~TarInfo.type`.
339
340 .. data:: CONTTYPE
341
342 A contiguous file :attr:`~TarInfo.type`.
343
344 .. data:: GNUTYPE_LONGNAME
345
346 A GNU tar longname :attr:`~TarInfo.type`.
347
348 .. data:: GNUTYPE_LONGLINK
349
350 A GNU tar longlink :attr:`~TarInfo.type`.
351
352 .. data:: GNUTYPE_SPARSE
353
354 A GNU tar sparse file :attr:`~TarInfo.type`.
355
356
357 Each of the following constants defines a tar archive format that the
358 :mod:`!tarfile` module is able to create. See section :ref:`tar-formats` for
359 details.
360
361
362 .. data:: USTAR_FORMAT
363
364 POSIX.1-1988 (ustar) format.
365
366
367 .. data:: GNU_FORMAT
368
369 GNU tar format.
370
371
372 .. data:: PAX_FORMAT
373
374 POSIX.1-2001 (pax) format.
375
376
377 .. data:: DEFAULT_FORMAT
378
379 The default format for creating archives. This is currently :const:`PAX_FORMAT`.
380
381 .. versionchanged:: 3.8
382 The default format for new archives was changed to
383 :const:`PAX_FORMAT` from :const:`GNU_FORMAT`.
384
385
386 .. seealso::
387
388 Module :mod:`zipfile`
389 Documentation of the :mod:`zipfile` standard module.
390
391 :ref:`archiving-operations`
392 Documentation of the higher-level archiving facilities provided by the
393 standard :mod:`shutil` module.
394
395 `GNU tar manual, Basic Tar Format <https://www.gnu.org/software/tar/manual/html_node/Standard.html>`_
396 Documentation for tar archive files, including GNU tar extensions.
397
398
399 .. _tarfile-objects:
400
401 TarFile Objects
402 ---------------
403
404 The :class:`TarFile` object provides an interface to a tar archive. A tar
405 archive is a sequence of blocks. An archive member (a stored file) is made up of
406 a header block followed by data blocks. It is possible to store a file in a tar
407 archive several times. Each archive member is represented by a :class:`TarInfo`
408 object, see :ref:`tarinfo-objects` for details.
409
410 A :class:`TarFile` object can be used as a context manager in a :keyword:`with`
411 statement. It will automatically be closed when the block is completed. Please
412 note that in the event of an exception an archive opened for writing will not
413 be finalized; only the internally used file object will be closed. See the
414 :ref:`tar-examples` section for a use case.
415
416 .. versionadded:: 3.2
417 Added support for the context management protocol.
418
419 .. class:: TarFile(name=None, mode='r', fileobj=None, format=DEFAULT_FORMAT, tarinfo=TarInfo, dereference=False, ignore_zeros=False, encoding=ENCODING, errors='surrogateescape', pax_headers=None, debug=0, errorlevel=1, stream=False)
420
421 All following arguments are optional and can be accessed as instance attributes
422 as well.
423
424 *name* is the pathname of the archive. *name* may be a :term:`path-like object`.
425 It can be omitted if *fileobj* is given.
426 In this case, the file object's :attr:`!name` attribute is used if it exists.
427
428 *mode* is either ``'r'`` to read from an existing archive, ``'a'`` to append
429 data to an existing file, ``'w'`` to create a new file overwriting an existing
430 one, or ``'x'`` to create a new file only if it does not already exist.
431
432 If *fileobj* is given, it is used for reading or writing data. If it can be
433 determined, *mode* is overridden by *fileobj*'s mode. *fileobj* will be used
434 from position 0.
435
436 .. note::
437
438 *fileobj* is not closed, when :class:`TarFile` is closed.
439
440 *format* controls the archive format for writing. It must be one of the constants
441 :const:`USTAR_FORMAT`, :const:`GNU_FORMAT` or :const:`PAX_FORMAT` that are
442 defined at module level. When reading, format will be automatically detected, even
443 if different formats are present in a single archive.
444
445 The *tarinfo* argument can be used to replace the default :class:`TarInfo` class
446 with a different one.
447
448 If *dereference* is :const:`False`, add symbolic and hard links to the archive. If it
449 is :const:`True`, add the content of the target files to the archive. This has no
450 effect on systems that do not support symbolic links.
451
452 If *ignore_zeros* is :const:`False`, treat an empty block as the end of the archive.
453 If it is :const:`True`, skip empty (and invalid) blocks and try to get as many members
454 as possible. This is only useful for reading concatenated or damaged archives.
455
456 *debug* can be set from ``0`` (no debug messages) up to ``3`` (all debug
457 messages). The messages are written to ``sys.stderr``.
458
459 *errorlevel* controls how extraction errors are handled,
460 see :attr:`the corresponding attribute <TarFile.errorlevel>`.
461
462 The *encoding* and *errors* arguments define the character encoding to be
463 used for reading or writing the archive and how conversion errors are going
464 to be handled. The default settings will work for most users.
465 See section :ref:`tar-unicode` for in-depth information.
466
467 The *pax_headers* argument is an optional dictionary of strings which
468 will be added as a pax global header if *format* is :const:`PAX_FORMAT`.
469
470 If *stream* is set to :const:`True` then while reading the archive info about files
471 in the archive are not cached, saving memory.
472
473 .. versionchanged:: 3.2
474 Use ``'surrogateescape'`` as the default for the *errors* argument.
475
476 .. versionchanged:: 3.5
477 The ``'x'`` (exclusive creation) mode was added.
478
479 .. versionchanged:: 3.6
480 The *name* parameter accepts a :term:`path-like object`.
481
482 .. versionchanged:: 3.13
483 Add the *stream* parameter.
484
485 .. classmethod:: TarFile.open(...)
486
487 Alternative constructor. The :func:`tarfile.open` function is actually a
488 shortcut to this classmethod.
489
490
491 .. method:: TarFile.getmember(name)
492
493 Return a :class:`TarInfo` object for member *name*. If *name* can not be found
494 in the archive, :exc:`KeyError` is raised.
495
496 .. note::
497
498 If a member occurs more than once in the archive, its last occurrence is assumed
499 to be the most up-to-date version.
500
501
502 .. method:: TarFile.getmembers()
503
504 Return the members of the archive as a list of :class:`TarInfo` objects. The
505 list has the same order as the members in the archive.
506
507
508 .. method:: TarFile.getnames()
509
510 Return the members as a list of their names. It has the same order as the list
511 returned by :meth:`getmembers`.
512
513
514 .. method:: TarFile.list(verbose=True, *, members=None)
515
516 Print a table of contents to ``sys.stdout``. If *verbose* is :const:`False`,
517 only the names of the members are printed. If it is :const:`True`, output
518 similar to that of :program:`ls -l` is produced. If optional *members* is
519 given, it must be a subset of the list returned by :meth:`getmembers`.
520
521 .. versionchanged:: 3.5
522 Added the *members* parameter.
523
524
525 .. method:: TarFile.next()
526
527 Return the next member of the archive as a :class:`TarInfo` object, when
528 :class:`TarFile` is opened for reading. Return :const:`None` if there is no more
529 available.
530
531
532 .. method:: TarFile.extractall(path=".", members=None, *, numeric_owner=False, filter=None)
533
534 Extract all members from the archive to the current working directory or
535 directory *path*. If optional *members* is given, it must be a subset of the
536 list returned by :meth:`getmembers`. Directory information like owner,
537 modification time and permissions are set after all members have been extracted.
538 This is done to work around two problems: A directory's modification time is
539 reset each time a file is created in it. And, if a directory's permissions do
540 not allow writing, extracting files to it will fail.
541
542 If *numeric_owner* is :const:`True`, the uid and gid numbers from the tarfile
543 are used to set the owner/group for the extracted files. Otherwise, the named
544 values from the tarfile are used.
545
546 The *filter* argument specifies how ``members`` are modified or rejected
547 before extraction.
548 See :ref:`tarfile-extraction-filter` for details.
549 It is recommended to set this explicitly only if specific *tar* features
550 are required, or as ``filter='data'`` to support Python versions with a less
551 secure default (3.13 and lower).
552
553 .. warning::
554
555 Never extract archives from untrusted sources without prior inspection.
556
557 Since Python 3.14, the default (:func:`data <data_filter>`) will prevent
558 the most dangerous security issues.
559 However, it will not prevent *all* unintended or insecure behavior.
560 Read the :ref:`tarfile-extraction-filter` section for details.
561
562 .. versionchanged:: 3.5
563 Added the *numeric_owner* parameter.
564
565 .. versionchanged:: 3.6
566 The *path* parameter accepts a :term:`path-like object`.
567
568 .. versionchanged:: 3.12
569 Added the *filter* parameter.
570
571 .. versionchanged:: 3.14
572 The *filter* parameter now defaults to ``'data'``.
573
574
575 .. method:: TarFile.extract(member, path="", set_attrs=True, *, numeric_owner=False, filter=None)
576
577 Extract a member from the archive to the current working directory, using its
578 full name. Its file information is extracted as accurately as possible. *member*
579 may be a filename or a :class:`TarInfo` object. You can specify a different
580 directory using *path*. *path* may be a :term:`path-like object`.
581 File attributes (owner, mtime, mode) are set unless *set_attrs* is false.
582
583 The *numeric_owner* and *filter* arguments are the same as
584 for :meth:`extractall`.
585
586 .. note::
587
588 The :meth:`extract` method does not take care of several extraction issues.
589 In most cases you should consider using the :meth:`extractall` method.
590
591 .. warning::
592
593 Never extract archives from untrusted sources without prior inspection.
594 See the warning for :meth:`extractall` for details.
595
596 .. versionchanged:: 3.2
597 Added the *set_attrs* parameter.
598
599 .. versionchanged:: 3.5
600 Added the *numeric_owner* parameter.
601
602 .. versionchanged:: 3.6
603 The *path* parameter accepts a :term:`path-like object`.
604
605 .. versionchanged:: 3.12
606 Added the *filter* parameter.
607
608
609 .. method:: TarFile.extractfile(member)
610
611 Extract a member from the archive as a file object. *member* may be
612 a filename or a :class:`TarInfo` object. If *member* is a regular file or
613 a link, an :class:`io.BufferedReader` object is returned. For all other
614 existing members, :const:`None` is returned. If *member* does not appear
615 in the archive, :exc:`KeyError` is raised.
616
617 .. versionchanged:: 3.3
618 Return an :class:`io.BufferedReader` object.
619
620 .. versionchanged:: 3.13
621 The returned :class:`io.BufferedReader` object has the :attr:`!mode`
622 attribute which is always equal to ``'rb'``.
623
624 .. attribute:: TarFile.errorlevel
625 :type: int
626
627 If *errorlevel* is ``0``, errors are ignored when using :meth:`TarFile.extract`
628 and :meth:`TarFile.extractall`.
629 Nevertheless, they appear as error messages in the debug output when
630 *debug* is greater than 0.
631 If ``1`` (the default), all *fatal* errors are raised as :exc:`OSError` or
632 :exc:`FilterError` exceptions. If ``2``, all *non-fatal* errors are raised
633 as :exc:`TarError` exceptions as well.
634
635 Some exceptions, e.g. ones caused by wrong argument types or data
636 corruption, are always raised.
637
638 Custom :ref:`extraction filters <tarfile-extraction-filter>`
639 should raise :exc:`FilterError` for *fatal* errors
640 and :exc:`ExtractError` for *non-fatal* ones.
641
642 Note that when an exception is raised, the archive may be partially
643 extracted. It is the user’s responsibility to clean up.
644
645 .. attribute:: TarFile.extraction_filter
646
647 .. versionadded:: 3.12
648
649 The :ref:`extraction filter <tarfile-extraction-filter>` used
650 as a default for the *filter* argument of :meth:`~TarFile.extract`
651 and :meth:`~TarFile.extractall`.
652
653 The attribute may be ``None`` or a callable.
654 String names are not allowed for this attribute, unlike the *filter*
655 argument to :meth:`~TarFile.extract`.
656
657 If ``extraction_filter`` is ``None`` (the default), extraction methods
658 will use the :func:`data <data_filter>` filter by default.
659
660 The attribute may be set on instances or overridden in subclasses.
661 It also is possible to set it on the ``TarFile`` class itself to set a
662 global default, although, since it affects all uses of *tarfile*,
663 it is best practice to only do so in top-level applications or
664 :mod:`site configuration <site>`.
665 To set a global default this way, a filter function needs to be wrapped in
666 :func:`staticmethod` to prevent injection of a ``self`` argument.
667
668 .. versionchanged:: 3.14
669
670 The default filter is set to :func:`data <data_filter>`,
671 which disallows some dangerous features such as links to absolute paths
672 or paths outside of the destination.
673 Previously, the default was equivalent to
674 :func:`fully_trusted <fully_trusted_filter>`.
675
676 .. method:: TarFile.add(name, arcname=None, recursive=True, *, filter=None)
677
678 Add the file *name* to the archive. *name* may be any type of file
679 (directory, fifo, symbolic link, etc.). If given, *arcname* specifies an
680 alternative name for the file in the archive. Directories are added
681 recursively by default. This can be avoided by setting *recursive* to
682 :const:`False`. Recursion adds entries in sorted order.
683 If *filter* is given, it
684 should be a function that takes a :class:`TarInfo` object argument and
685 returns the changed :class:`TarInfo` object. If it instead returns
686 :const:`None` the :class:`TarInfo` object will be excluded from the
687 archive. See :ref:`tar-examples` for an example.
688
689 .. versionchanged:: 3.2
690 Added the *filter* parameter.
691
692 .. versionchanged:: 3.7
693 Recursion adds entries in sorted order.
694
695
696 .. method:: TarFile.addfile(tarinfo, fileobj=None)
697
698 Add the :class:`TarInfo` object *tarinfo* to the archive. If *tarinfo* represents
699 a non zero-size regular file, the *fileobj* argument should be a :term:`binary file`,
700 and ``tarinfo.size`` bytes are read from it and added to the archive. You can
701 create :class:`TarInfo` objects directly, or by using :meth:`gettarinfo`.
702
703 .. versionchanged:: 3.13
704
705 *fileobj* must be given for non-zero-sized regular files.
706
707
708 .. method:: TarFile.gettarinfo(name=None, arcname=None, fileobj=None)
709
710 Create a :class:`TarInfo` object from the result of :func:`os.stat` or
711 equivalent on an existing file. The file is either named by *name*, or
712 specified as a :term:`file object` *fileobj* with a file descriptor.
713 *name* may be a :term:`path-like object`. If
714 given, *arcname* specifies an alternative name for the file in the
715 archive, otherwise, the name is taken from *fileobj*’s
716 :attr:`~io.FileIO.name` attribute, or the *name* argument. The name
717 should be a text string.
718
719 You can modify
720 some of the :class:`TarInfo`’s attributes before you add it using :meth:`addfile`.
721 If the file object is not an ordinary file object positioned at the
722 beginning of the file, attributes such as :attr:`~TarInfo.size` may need
723 modifying. This is the case for objects such as :class:`~gzip.GzipFile`.
724 The :attr:`~TarInfo.name` may also be modified, in which case *arcname*
725 could be a dummy string.
726
727 .. versionchanged:: 3.6
728 The *name* parameter accepts a :term:`path-like object`.
729
730
731 .. method:: TarFile.close()
732
733 Close the :class:`TarFile`. In write mode, two finishing zero blocks are
734 appended to the archive.
735
736
737 .. attribute:: TarFile.pax_headers
738 :type: dict
739
740 A dictionary containing key-value pairs of pax global headers.
741
742
743
744 .. _tarinfo-objects:
745
746 TarInfo Objects
747 ---------------
748
749 A :class:`TarInfo` object represents one member in a :class:`TarFile`. Aside
750 from storing all required attributes of a file (like file type, size, time,
751 permissions, owner etc.), it provides some useful methods to determine its type.
752 It does *not* contain the file's data itself.
753
754 :class:`TarInfo` objects are returned by :class:`TarFile`'s methods
755 :meth:`~TarFile.getmember`, :meth:`~TarFile.getmembers` and
756 :meth:`~TarFile.gettarinfo`.
757
758 Modifying the objects returned by :meth:`~TarFile.getmember` or
759 :meth:`~TarFile.getmembers` will affect all subsequent
760 operations on the archive.
761 For cases where this is unwanted, you can use :mod:`copy.copy() <copy>` or
762 call the :meth:`~TarInfo.replace` method to create a modified copy in one step.
763
764 Several attributes can be set to ``None`` to indicate that a piece of metadata
765 is unused or unknown.
766 Different :class:`TarInfo` methods handle ``None`` differently:
767
768 - The :meth:`~TarFile.extract` or :meth:`~TarFile.extractall` methods will
769 ignore the corresponding metadata, leaving it set to a default.
770 - :meth:`~TarFile.addfile` will fail.
771 - :meth:`~TarFile.list` will print a placeholder string.
772
773 .. class:: TarInfo(name="")
774
775 Create a :class:`TarInfo` object.
776
777
778 .. classmethod:: TarInfo.frombuf(buf, encoding, errors)
779
780 Create and return a :class:`TarInfo` object from string buffer *buf*.
781
782 Raises :exc:`HeaderError` if the buffer is invalid.
783
784
785 .. classmethod:: TarInfo.fromtarfile(tarfile)
786
787 Read the next member from the :class:`TarFile` object *tarfile* and return it as
788 a :class:`TarInfo` object.
789
790
791 .. method:: TarInfo.tobuf(format=DEFAULT_FORMAT, encoding=ENCODING, errors='surrogateescape')
792
793 Create a string buffer from a :class:`TarInfo` object. For information on the
794 arguments see the constructor of the :class:`TarFile` class.
795
796 .. versionchanged:: 3.2
797 Use ``'surrogateescape'`` as the default for the *errors* argument.
798
799
800 A ``TarInfo`` object has the following public data attributes:
801
802
803 .. attribute:: TarInfo.name
804 :type: str
805
806 Name of the archive member.
807
808
809 .. attribute:: TarInfo.size
810 :type: int
811
812 Size in bytes.
813
814
815 .. attribute:: TarInfo.mtime
816 :type: int | float
817
818 Time of last modification in seconds since the :ref:`epoch <epoch>`,
819 as in :attr:`os.stat_result.st_mtime`.
820
821 .. versionchanged:: 3.12
822
823 Can be set to ``None`` for :meth:`~TarFile.extract` and
824 :meth:`~TarFile.extractall`, causing extraction to skip applying this
825 attribute.
826
827 .. attribute:: TarInfo.mode
828 :type: int
829
830 Permission bits, as for :func:`os.chmod`.
831
832 .. versionchanged:: 3.12
833
834 Can be set to ``None`` for :meth:`~TarFile.extract` and
835 :meth:`~TarFile.extractall`, causing extraction to skip applying this
836 attribute.
837
838 .. attribute:: TarInfo.type
839
840 File type. *type* is usually one of these constants: :const:`REGTYPE`,
841 :const:`AREGTYPE`, :const:`LNKTYPE`, :const:`SYMTYPE`, :const:`DIRTYPE`,
842 :const:`FIFOTYPE`, :const:`CONTTYPE`, :const:`CHRTYPE`, :const:`BLKTYPE`,
843 :const:`GNUTYPE_SPARSE`. To determine the type of a :class:`TarInfo` object
844 more conveniently, use the ``is*()`` methods below.
845
846
847 .. attribute:: TarInfo.linkname
848 :type: str
849
850 Name of the target file name, which is only present in :class:`TarInfo` objects
851 of type :const:`LNKTYPE` and :const:`SYMTYPE`.
852
853 For symbolic links (``SYMTYPE``), the *linkname* is relative to the directory
854 that contains the link.
855 For hard links (``LNKTYPE``), the *linkname* is relative to the root of
856 the archive.
857
858
859 .. attribute:: TarInfo.uid
860 :type: int
861
862 User ID of the user who originally stored this member.
863
864 .. versionchanged:: 3.12
865
866 Can be set to ``None`` for :meth:`~TarFile.extract` and
867 :meth:`~TarFile.extractall`, causing extraction to skip applying this
868 attribute.
869
870 .. attribute:: TarInfo.gid
871 :type: int
872
873 Group ID of the user who originally stored this member.
874
875 .. versionchanged:: 3.12
876
877 Can be set to ``None`` for :meth:`~TarFile.extract` and
878 :meth:`~TarFile.extractall`, causing extraction to skip applying this
879 attribute.
880
881 .. attribute:: TarInfo.uname
882 :type: str
883
884 User name.
885
886 .. versionchanged:: 3.12
887
888 Can be set to ``None`` for :meth:`~TarFile.extract` and
889 :meth:`~TarFile.extractall`, causing extraction to skip applying this
890 attribute.
891
892 .. attribute:: TarInfo.gname
893 :type: str
894
895 Group name.
896
897 .. versionchanged:: 3.12
898
899 Can be set to ``None`` for :meth:`~TarFile.extract` and
900 :meth:`~TarFile.extractall`, causing extraction to skip applying this
901 attribute.
902
903 .. attribute:: TarInfo.chksum
904 :type: int
905
906 Header checksum.
907
908
909 .. attribute:: TarInfo.devmajor
910 :type: int
911
912 Device major number.
913
914
915 .. attribute:: TarInfo.devminor
916 :type: int
917
918 Device minor number.
919
920
921 .. attribute:: TarInfo.offset
922 :type: int
923
924 The tar header starts here.
925
926
927 .. attribute:: TarInfo.offset_data
928 :type: int
929
930 The file's data starts here.
931
932
933 .. attribute:: TarInfo.sparse
934
935 Sparse member information.
936
937
938 .. attribute:: TarInfo.pax_headers
939 :type: dict
940
941 A dictionary containing key-value pairs of an associated pax extended header.
942
943 .. method:: TarInfo.replace(name=..., mtime=..., mode=..., linkname=..., \
944 uid=..., gid=..., uname=..., gname=..., \
945 deep=True)
946
947 .. versionadded:: 3.12
948
949 Return a *new* copy of the :class:`!TarInfo` object with the given attributes
950 changed. For example, to return a ``TarInfo`` with the group name set to
951 ``'staff'``, use::
952
953 new_tarinfo = old_tarinfo.replace(gname='staff')
954
955 By default, a deep copy is made.
956 If *deep* is false, the copy is shallow, i.e. ``pax_headers``
957 and any custom attributes are shared with the original ``TarInfo`` object.
958
959 A :class:`TarInfo` object also provides some convenient query methods:
960
961
962 .. method:: TarInfo.isfile()
963
964 Return :const:`True` if the :class:`TarInfo` object is a regular file.
965
966
967 .. method:: TarInfo.isreg()
968
969 Same as :meth:`isfile`.
970
971
972 .. method:: TarInfo.isdir()
973
974 Return :const:`True` if it is a directory.
975
976
977 .. method:: TarInfo.issym()
978
979 Return :const:`True` if it is a symbolic link.
980
981
982 .. method:: TarInfo.islnk()
983
984 Return :const:`True` if it is a hard link.
985
986
987 .. method:: TarInfo.ischr()
988
989 Return :const:`True` if it is a character device.
990
991
992 .. method:: TarInfo.isblk()
993
994 Return :const:`True` if it is a block device.
995
996
997 .. method:: TarInfo.isfifo()
998
999 Return :const:`True` if it is a FIFO.
1000
1001
1002 .. method:: TarInfo.isdev()
1003
1004 Return :const:`True` if it is one of character device, block device or FIFO.
1005
1006
1007 .. _tarfile-extraction-filter:
1008
1009 Extraction filters
1010 ------------------
1011
1012 .. versionadded:: 3.12
1013
1014 The *tar* format is designed to capture all details of a UNIX-like filesystem,
1015 which makes it very powerful.
1016 Unfortunately, the features make it easy to create tar files that have
1017 unintended -- and possibly malicious -- effects when extracted.
1018 For example, extracting a tar file can overwrite arbitrary files in various
1019 ways (e.g. by using absolute paths, ``..`` path components, or symlinks that
1020 affect later members).
1021
1022 In most cases, the full functionality is not needed.
1023 Therefore, *tarfile* supports extraction filters: a mechanism to limit
1024 functionality, and thus mitigate some of the security issues.
1025
1026 .. warning::
1027
1028 None of the available filters blocks *all* dangerous archive features.
1029 Never extract archives from untrusted sources without prior inspection.
1030 See also :ref:`tarfile-further-verification`.
1031
1032 .. seealso::
1033
1034 :pep:`706`
1035 Contains further motivation and rationale behind the design.
1036
1037 The *filter* argument to :meth:`TarFile.extract` or :meth:`~TarFile.extractall`
1038 can be:
1039
1040 * the string ``'fully_trusted'``: Honor all metadata as specified in the
1041 archive.
1042 Should be used if the user trusts the archive completely, or implements
1043 their own complex verification.
1044
1045 * the string ``'tar'``: Honor most *tar*-specific features (i.e. features of
1046 UNIX-like filesystems), but block features that are very likely to be
1047 surprising or malicious. See :func:`tar_filter` for details.
1048
1049 * the string ``'data'``: Ignore or block most features specific to UNIX-like
1050 filesystems. Intended for extracting cross-platform data archives.
1051 See :func:`data_filter` for details.
1052
1053 * ``None`` (default): Use :attr:`TarFile.extraction_filter`.
1054
1055 If that is also ``None`` (the default), the ``'data'`` filter will be used.
1056
1057 .. versionchanged:: 3.14
1058
1059 The default filter is set to :func:`data <data_filter>`.
1060 Previously, the default was equivalent to
1061 :func:`fully_trusted <fully_trusted_filter>`.
1062
1063 * A callable which will be called for each extracted member with a
1064 :ref:`TarInfo <tarinfo-objects>` describing the member and the destination
1065 path to where the archive is extracted (i.e. the same path is used for all
1066 members)::
1067
1068 filter(member: TarInfo, path: str, /) -> TarInfo | None
1069
1070 The callable is called just before each member is extracted, so it can
1071 take the current state of the disk into account.
1072 It can:
1073
1074 - return a :class:`TarInfo` object which will be used instead of the metadata
1075 in the archive, or
1076 - return ``None``, in which case the member will be skipped, or
1077 - raise an exception to abort the operation or skip the member,
1078 depending on :attr:`~TarFile.errorlevel`.
1079 Note that when extraction is aborted, :meth:`~TarFile.extractall` may leave
1080 the archive partially extracted. It does not attempt to clean up.
1081
1082 Default named filters
1083 ~~~~~~~~~~~~~~~~~~~~~
1084
1085 The pre-defined, named filters are available as functions, so they can be
1086 reused in custom filters:
1087
1088 .. function:: fully_trusted_filter(member, path)
1089
1090 Return *member* unchanged.
1091
1092 This implements the ``'fully_trusted'`` filter.
1093
1094 .. function:: tar_filter(member, path)
1095
1096 Implements the ``'tar'`` filter.
1097
1098 - Strip leading slashes (``/`` and :data:`os.sep`) from filenames.
1099 - :ref:`Refuse <tarfile-extraction-refuse>` to extract files with absolute
1100 paths (in case the name is absolute
1101 even after stripping slashes, e.g. ``C:/foo`` on Windows).
1102 This raises :class:`~tarfile.AbsolutePathError`.
1103 - :ref:`Refuse <tarfile-extraction-refuse>` to extract files whose absolute
1104 path (after following symlinks) would end up outside the destination.
1105 This raises :class:`~tarfile.OutsideDestinationError`.
1106 - Clear high mode bits (setuid, setgid, sticky) and group/other write bits
1107 (:const:`~stat.S_IWGRP` | :const:`~stat.S_IWOTH`).
1108
1109 Return the modified ``TarInfo`` member.
1110
1111 .. function:: data_filter(member, path)
1112
1113 Implements the ``'data'`` filter.
1114 In addition to what ``tar_filter`` does:
1115
1116 - Normalize link targets (:attr:`TarInfo.linkname`) using
1117 :func:`os.path.normpath`.
1118 Note that this removes internal ``..`` components, which may change the
1119 meaning of the link if the path in :attr:`!TarInfo.linkname` traverses
1120 symbolic links.
1121
1122 - :ref:`Refuse <tarfile-extraction-refuse>` to extract links (hard or soft)
1123 that link to absolute paths, or ones that link outside the destination.
1124
1125 This raises :class:`~tarfile.AbsoluteLinkError` or
1126 :class:`~tarfile.LinkOutsideDestinationError`.
1127
1128 Note that such files are refused even on platforms that do not support
1129 symbolic links.
1130
1131 - :ref:`Refuse <tarfile-extraction-refuse>` to extract device files
1132 (including pipes).
1133 This raises :class:`~tarfile.SpecialFileError`.
1134
1135 - For regular files, including hard links:
1136
1137 - Set the owner read and write permissions
1138 (:const:`~stat.S_IRUSR` | :const:`~stat.S_IWUSR`).
1139 - Remove the group & other executable permission
1140 (:const:`~stat.S_IXGRP` | :const:`~stat.S_IXOTH`)
1141 if the owner doesn’t have it (:const:`~stat.S_IXUSR`).
1142
1143 - For other files (directories), set ``mode`` to ``None``, so
1144 that extraction methods skip applying permission bits.
1145 - Set user and group info (``uid``, ``gid``, ``uname``, ``gname``)
1146 to ``None``, so that extraction methods skip setting it.
1147
1148 Return the modified ``TarInfo`` member.
1149
1150 Note that this filter does not block *all* dangerous archive features.
1151 See :ref:`tarfile-further-verification` for details.
1152
1153 .. versionchanged:: 3.14
1154
1155 Link targets are now normalized.
1156
1157
1158 .. _tarfile-extraction-refuse:
1159
1160 Filter errors
1161 ~~~~~~~~~~~~~
1162
1163 When a filter refuses to extract a file, it will raise an appropriate exception,
1164 a subclass of :class:`~tarfile.FilterError`.
1165 This will abort the extraction if :attr:`TarFile.errorlevel` is 1 or more.
1166 With ``errorlevel=0`` the error will be logged and the member will be skipped,
1167 but extraction will continue.
1168
1169
1170 .. _tarfile-further-verification:
1171
1172 Hints for further verification
1173 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1174
1175 Even with ``filter='data'``, *tarfile* is not suited for extracting untrusted
1176 files without prior inspection.
1177 Among other issues, the pre-defined filters do not prevent denial-of-service
1178 attacks. Users should do additional checks.
1179
1180 Here is an incomplete list of things to consider:
1181
1182 * Extract to a :func:`new temporary directory <tempfile.mkdtemp>`
1183 to prevent e.g. exploiting pre-existing links, and to make it easier to
1184 clean up after a failed extraction.
1185 * Disallow symbolic links if you do not need the functionality.
1186 * When working with untrusted data, use external (e.g. OS-level) limits on
1187 disk, memory and CPU usage.
1188 * Check filenames against an allow-list of characters
1189 (to filter out control characters, confusables, foreign path separators,
1190 and so on).
1191 * Check that filenames have expected extensions (discouraging files that
1192 execute when you “click on them”, or extension-less files like Windows
1193 special device names).
1194 * Limit the number of extracted files, total size of extracted data,
1195 filename length (including symlink length), and size of individual files.
1196 * Check for files that would be shadowed on case-insensitive filesystems.
1197
1198 Also note that:
1199
1200 * Tar files may contain multiple versions of the same file.
1201 Later ones are expected to overwrite any earlier ones.
1202 This feature is crucial to allow updating tape archives, but can be abused
1203 maliciously.
1204 * *tarfile* does not protect against issues with “live” data,
1205 e.g. an attacker tinkering with the destination (or source) directory while
1206 extraction (or archiving) is in progress.
1207
1208
1209 Supporting older Python versions
1210 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1211
1212 Extraction filters were added to Python 3.12, but may be backported to older
1213 versions as security updates.
1214 To check whether the feature is available, use e.g.
1215 ``hasattr(tarfile, 'data_filter')`` rather than checking the Python version.
1216
1217 The following examples show how to support Python versions with and without
1218 the feature.
1219 Note that setting ``extraction_filter`` will affect any subsequent operations.
1220
1221 * Fully trusted archive::
1222
1223 my_tarfile.extraction_filter = (lambda member, path: member)
1224 my_tarfile.extractall()
1225
1226 * Use the ``'data'`` filter if available, but revert to Python 3.11 behavior
1227 (``'fully_trusted'``) if this feature is not available::
1228
1229 my_tarfile.extraction_filter = getattr(tarfile, 'data_filter',
1230 (lambda member, path: member))
1231 my_tarfile.extractall()
1232
1233 * Use the ``'data'`` filter; *fail* if it is not available::
1234
1235 my_tarfile.extractall(filter=tarfile.data_filter)
1236
1237 or::
1238
1239 my_tarfile.extraction_filter = tarfile.data_filter
1240 my_tarfile.extractall()
1241
1242 * Use the ``'data'`` filter; *warn* if it is not available::
1243
1244 if hasattr(tarfile, 'data_filter'):
1245 my_tarfile.extractall(filter='data')
1246 else:
1247 # remove this when no longer needed
1248 warn_the_user('Extracting may be unsafe; consider updating Python')
1249 my_tarfile.extractall()
1250
1251
1252 Stateful extraction filter example
1253 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1254
1255 While *tarfile*'s extraction methods take a simple *filter* callable,
1256 custom filters may be more complex objects with an internal state.
1257 It may be useful to write these as context managers, to be used like this::
1258
1259 with StatefulFilter() as filter_func:
1260 tar.extractall(path, filter=filter_func)
1261
1262 Such a filter can be written as, for example::
1263
1264 class StatefulFilter:
1265 def __init__(self):
1266 self.file_count = 0
1267
1268 def __enter__(self):
1269 return self
1270
1271 def __call__(self, member, path):
1272 self.file_count += 1
1273 return member
1274
1275 def __exit__(self, *exc_info):
1276 print(f'{self.file_count} files extracted')
1277
1278
1279 .. _tarfile-commandline:
1280 .. program:: tarfile
1281
1282
1283 Command-Line Interface
1284 ----------------------
1285
1286 .. versionadded:: 3.4
1287
1288 The :mod:`!tarfile` module provides a simple command-line interface to interact
1289 with tar archives.
1290
1291 If you want to create a new tar archive, specify its name after the :option:`-c`
1292 option and then list the filename(s) that should be included:
1293
1294 .. code-block:: shell-session
1295
1296 $ python -m tarfile -c monty.tar spam.txt eggs.txt
1297
1298 Passing a directory is also acceptable:
1299
1300 .. code-block:: shell-session
1301
1302 $ python -m tarfile -c monty.tar life-of-brian_1979/
1303
1304 If you want to extract a tar archive into the current directory, use
1305 the :option:`-e` option:
1306
1307 .. code-block:: shell-session
1308
1309 $ python -m tarfile -e monty.tar
1310
1311 You can also extract a tar archive into a different directory by passing the
1312 directory's name:
1313
1314 .. code-block:: shell-session
1315
1316 $ python -m tarfile -e monty.tar other-dir/
1317
1318 For a list of the files in a tar archive, use the :option:`-l` option:
1319
1320 .. code-block:: shell-session
1321
1322 $ python -m tarfile -l monty.tar
1323
1324
1325 Command-line options
1326 ~~~~~~~~~~~~~~~~~~~~
1327
1328 .. option:: -l <tarfile>
1329 --list <tarfile>
1330
1331 List files in a tarfile.
1332
1333 .. option:: -c <tarfile> <source1> ... <sourceN>
1334 --create <tarfile> <source1> ... <sourceN>
1335
1336 Create tarfile from source files.
1337
1338 .. option:: -e <tarfile> [<output_dir>]
1339 --extract <tarfile> [<output_dir>]
1340
1341 Extract tarfile into the current directory if *output_dir* is not specified.
1342
1343 .. option:: -t <tarfile>
1344 --test <tarfile>
1345
1346 Test whether the tarfile is valid or not.
1347
1348 .. option:: -v, --verbose
1349
1350 Verbose output.
1351
1352 .. option:: --filter <filtername>
1353
1354 Specifies the *filter* for ``--extract``.
1355 See :ref:`tarfile-extraction-filter` for details.
1356 Only string names are accepted (that is, ``fully_trusted``, ``tar``,
1357 and ``data``).
1358
1359 .. _tar-examples:
1360
1361 Examples
1362 --------
1363
1364 Reading examples
1365 ~~~~~~~~~~~~~~~~~~~
1366
1367 How to extract an entire tar archive to the current working directory::
1368
1369 import tarfile
1370 tar = tarfile.open("sample.tar.gz")
1371 tar.extractall(filter='data')
1372 tar.close()
1373
1374 How to extract a subset of a tar archive with :meth:`TarFile.extractall` using
1375 a generator function instead of a list::
1376
1377 import os
1378 import tarfile
1379
1380 def py_files(members):
1381 for tarinfo in members:
1382 if os.path.splitext(tarinfo.name)[1] == ".py":
1383 yield tarinfo
1384
1385 tar = tarfile.open("sample.tar.gz")
1386 tar.extractall(members=py_files(tar))
1387 tar.close()
1388
1389 How to read a gzip compressed tar archive and display some member information::
1390
1391 import tarfile
1392 tar = tarfile.open("sample.tar.gz", "r:gz")
1393 for tarinfo in tar:
1394 print(tarinfo.name, "is", tarinfo.size, "bytes in size and is ", end="")
1395 if tarinfo.isreg():
1396 print("a regular file.")
1397 elif tarinfo.isdir():
1398 print("a directory.")
1399 else:
1400 print("something else.")
1401 tar.close()
1402
1403 Writing examples
1404 ~~~~~~~~~~~~~~~~
1405
1406 How to create an uncompressed tar archive from a list of filenames::
1407
1408 import tarfile
1409 tar = tarfile.open("sample.tar", "w")
1410 for name in ["foo", "bar", "quux"]:
1411 tar.add(name)
1412 tar.close()
1413
1414 The same example using the :keyword:`with` statement::
1415
1416 import tarfile
1417 with tarfile.open("sample.tar", "w") as tar:
1418 for name in ["foo", "bar", "quux"]:
1419 tar.add(name)
1420
1421 How to create and write an archive to stdout using
1422 :data:`sys.stdout.buffer <sys.stdout>` in the *fileobj* parameter
1423 in :meth:`TarFile.add`::
1424
1425 import sys
1426 import tarfile
1427 with tarfile.open("sample.tar.gz", "w|gz", fileobj=sys.stdout.buffer) as tar:
1428 for name in ["foo", "bar", "quux"]:
1429 tar.add(name)
1430
1431 How to create an archive and reset the user information using the *filter*
1432 parameter in :meth:`TarFile.add`::
1433
1434 import tarfile
1435 def reset(tarinfo):
1436 tarinfo.uid = tarinfo.gid = 0
1437 tarinfo.uname = tarinfo.gname = "root"
1438 return tarinfo
1439 tar = tarfile.open("sample.tar.gz", "w:gz")
1440 tar.add("foo", filter=reset)
1441 tar.close()
1442
1443
1444 .. _tar-formats:
1445
1446 Supported tar formats
1447 ---------------------
1448
1449 There are three tar formats that can be created with the :mod:`!tarfile` module:
1450
1451 * The POSIX.1-1988 ustar format (:const:`USTAR_FORMAT`). It supports filenames
1452 up to a length of at best 256 characters and linknames up to 100 characters.
1453 The maximum file size is 8 GiB. This is an old and limited but widely
1454 supported format.
1455
1456 * The GNU tar format (:const:`GNU_FORMAT`). It supports long filenames and
1457 linknames, files bigger than 8 GiB and sparse files. It is the de facto
1458 standard on GNU/Linux systems. :mod:`!tarfile` fully supports the GNU tar
1459 extensions for long names, sparse file support is read-only.
1460
1461 * The POSIX.1-2001 pax format (:const:`PAX_FORMAT`). It is the most flexible
1462 format with virtually no limits. It supports long filenames and linknames, large
1463 files and stores pathnames in a portable way. Modern tar implementations,
1464 including GNU tar, bsdtar/libarchive and star, fully support extended *pax*
1465 features; some old or unmaintained libraries may not, but should treat
1466 *pax* archives as if they were in the universally supported *ustar* format.
1467 It is the current default format for new archives.
1468
1469 It extends the existing *ustar* format with extra headers for information
1470 that cannot be stored otherwise. There are two flavours of pax headers:
1471 Extended headers only affect the subsequent file header, global
1472 headers are valid for the complete archive and affect all following files.
1473 All the data in a pax header is encoded in *UTF-8* for portability reasons.
1474
1475 There are some more variants of the tar format which can be read, but not
1476 created:
1477
1478 * The ancient V7 format. This is the first tar format from Unix Seventh Edition,
1479 storing only regular files and directories. Names must not be longer than 100
1480 characters, there is no user/group name information. Some archives have
1481 miscalculated header checksums in case of fields with non-ASCII characters.
1482
1483 * The SunOS tar extended format. This format is a variant of the POSIX.1-2001
1484 pax format, but is not compatible.
1485
1486 .. _tar-unicode:
1487
1488 Unicode issues
1489 --------------
1490
1491 The tar format was originally conceived to make backups on tape drives with the
1492 main focus on preserving file system information. Nowadays tar archives are
1493 commonly used for file distribution and exchanging archives over networks. One
1494 problem of the original format (which is the basis of all other formats) is
1495 that there is no concept of supporting different character encodings. For
1496 example, an ordinary tar archive created on a *UTF-8* system cannot be read
1497 correctly on a *Latin-1* system if it contains non-*ASCII* characters. Textual
1498 metadata (like filenames, linknames, user/group names) will appear damaged.
1499 Unfortunately, there is no way to autodetect the encoding of an archive. The
1500 pax format was designed to solve this problem. It stores non-ASCII metadata
1501 using the universal character encoding *UTF-8*.
1502
1503 The details of character conversion in :mod:`!tarfile` are controlled by the
1504 *encoding* and *errors* keyword arguments of the :class:`TarFile` class.
1505
1506 *encoding* defines the character encoding to use for the metadata in the
1507 archive. The default value is :func:`sys.getfilesystemencoding` or ``'ascii'``
1508 as a fallback. Depending on whether the archive is read or written, the
1509 metadata must be either decoded or encoded. If *encoding* is not set
1510 appropriately, this conversion may fail.
1511
1512 The *errors* argument defines how characters are treated that cannot be
1513 converted. Possible values are listed in section :ref:`error-handlers`.
1514 The default scheme is ``'surrogateescape'`` which Python also uses for its
1515 file system calls, see :ref:`os-filenames`.
1516
1517 For :const:`PAX_FORMAT` archives (the default), *encoding* is generally not needed
1518 because all the metadata is stored using *UTF-8*. *encoding* is only used in
1519 the rare cases when binary pax headers are decoded or when strings with
1520 surrogate characters are stored.