Debian Codegraph @

Doc/library/tarfile.rst

raw
1 :mod:`!tarfile` --- Read and write tar archive files
2 ====================================================
3
4 .. module:: tarfile
5 :synopsis: Read and write tar-format archive files.
6
7 .. moduleauthor:: Lars Gustäbel <[email protected]>
8 .. sectionauthor:: Lars Gustäbel <[email protected]>
9
10 **Source code:** :source:`Lib/tarfile.py`
11
12 --------------
13
14 The :mod:`tarfile` module makes it possible to read and write tar
15 archives, including those using gzip, bz2 and lzma compression.
16 Use the :mod:`zipfile` module to read or write :file:`.zip` files, or the
17 higher-level functions in :ref:`shutil <archiving-operations>`.
18
19 Some facts and figures:
20
21 * reads and writes :mod:`gzip`, :mod:`bz2` and :mod:`lzma` compressed archives
22 if the respective modules are available.
23
24 * read/write support for the POSIX.1-1988 (ustar) format.
25
26 * read/write support for the GNU tar format including *longname* and *longlink*
27 extensions, read-only support for all variants of the *sparse* extension
28 including restoration of sparse files.
29
30 * read/write support for the POSIX.1-2001 (pax) format.
31
32 * handles directories, regular files, hardlinks, symbolic links, fifos,
33 character devices and block devices and is able to acquire and restore file
34 information like timestamp, access permissions and owner.
35
36 .. versionchanged:: 3.3
37 Added support for :mod:`lzma` compression.
38
39 .. versionchanged:: 3.12
40 Archives are extracted using a :ref:`filter <tarfile-extraction-filter>`,
41 which makes it possible to either limit surprising/dangerous features,
42 or to acknowledge that they are expected and the archive is fully trusted.
43 By default, archives are fully trusted, but this default is deprecated
44 and slated to change in Python 3.14.
45
46
47 .. function:: open(name=None, mode='r', fileobj=None, bufsize=10240, **kwargs)
48
49 Return a :class:`TarFile` object for the pathname *name*. For detailed
50 information on :class:`TarFile` objects and the keyword arguments that are
51 allowed, see :ref:`tarfile-objects`.
52
53 *mode* has to be a string of the form ``'filemode[:compression]'``, it defaults
54 to ``'r'``. Here is a full list of mode combinations:
55
56 +------------------+---------------------------------------------+
57 | mode | action |
58 +==================+=============================================+
59 | ``'r'`` or | Open for reading with transparent |
60 | ``'r:*'`` | compression (recommended). |
61 +------------------+---------------------------------------------+
62 | ``'r:'`` | Open for reading exclusively without |
63 | | compression. |
64 +------------------+---------------------------------------------+
65 | ``'r:gz'`` | Open for reading with gzip compression. |
66 +------------------+---------------------------------------------+
67 | ``'r:bz2'`` | Open for reading with bzip2 compression. |
68 +------------------+---------------------------------------------+
69 | ``'r:xz'`` | Open for reading with lzma compression. |
70 +------------------+---------------------------------------------+
71 | ``'x'`` or | Create a tarfile exclusively without |
72 | ``'x:'`` | compression. |
73 | | Raise a :exc:`FileExistsError` exception |
74 | | if it already exists. |
75 +------------------+---------------------------------------------+
76 | ``'x:gz'`` | Create a tarfile with gzip compression. |
77 | | Raise a :exc:`FileExistsError` exception |
78 | | if it already exists. |
79 +------------------+---------------------------------------------+
80 | ``'x:bz2'`` | Create a tarfile with bzip2 compression. |
81 | | Raise a :exc:`FileExistsError` exception |
82 | | if it already exists. |
83 +------------------+---------------------------------------------+
84 | ``'x:xz'`` | Create a tarfile with lzma compression. |
85 | | Raise a :exc:`FileExistsError` exception |
86 | | if it already exists. |
87 +------------------+---------------------------------------------+
88 | ``'a'`` or | Open for appending with no compression. The |
89 | ``'a:'`` | file is created if it does not exist. |
90 +------------------+---------------------------------------------+
91 | ``'w'`` or | Open for uncompressed writing. |
92 | ``'w:'`` | |
93 +------------------+---------------------------------------------+
94 | ``'w:gz'`` | Open for gzip compressed writing. |
95 +------------------+---------------------------------------------+
96 | ``'w:bz2'`` | Open for bzip2 compressed writing. |
97 +------------------+---------------------------------------------+
98 | ``'w:xz'`` | Open for lzma compressed writing. |
99 +------------------+---------------------------------------------+
100
101 Note that ``'a:gz'``, ``'a:bz2'`` or ``'a:xz'`` is not possible. If *mode*
102 is not suitable to open a certain (compressed) file for reading,
103 :exc:`ReadError` is raised. Use *mode* ``'r'`` to avoid this. If a
104 compression method is not supported, :exc:`CompressionError` is raised.
105
106 If *fileobj* is specified, it is used as an alternative to a :term:`file object`
107 opened in binary mode for *name*. It is supposed to be at position 0.
108
109 For modes ``'w:gz'``, ``'x:gz'``, ``'w|gz'``, ``'w:bz2'``, ``'x:bz2'``,
110 ``'w|bz2'``, :func:`tarfile.open` accepts the keyword argument
111 *compresslevel* (default ``9``) to specify the compression level of the file.
112
113 For modes ``'w:xz'`` and ``'x:xz'``, :func:`tarfile.open` accepts the
114 keyword argument *preset* to specify the compression level of the file.
115
116 For special purposes, there is a second format for *mode*:
117 ``'filemode|[compression]'``. :func:`tarfile.open` will return a :class:`TarFile`
118 object that processes its data as a stream of blocks. No random seeking will
119 be done on the file. If given, *fileobj* may be any object that has a
120 :meth:`~io.RawIOBase.read` or :meth:`~io.RawIOBase.write` method
121 (depending on the *mode*) that works with bytes.
122 *bufsize* specifies the blocksize and defaults to ``20 * 512`` bytes.
123 Use this variant in combination with e.g. ``sys.stdin.buffer``, a socket
124 :term:`file object` or a tape device.
125 However, such a :class:`TarFile` object is limited in that it does
126 not allow random access, see :ref:`tar-examples`. The currently
127 possible modes:
128
129 +-------------+--------------------------------------------+
130 | Mode | Action |
131 +=============+============================================+
132 | ``'r|*'`` | Open a *stream* of tar blocks for reading |
133 | | with transparent compression. |
134 +-------------+--------------------------------------------+
135 | ``'r|'`` | Open a *stream* of uncompressed tar blocks |
136 | | for reading. |
137 +-------------+--------------------------------------------+
138 | ``'r|gz'`` | Open a gzip compressed *stream* for |
139 | | reading. |
140 +-------------+--------------------------------------------+
141 | ``'r|bz2'`` | Open a bzip2 compressed *stream* for |
142 | | reading. |
143 +-------------+--------------------------------------------+
144 | ``'r|xz'`` | Open an lzma compressed *stream* for |
145 | | reading. |
146 +-------------+--------------------------------------------+
147 | ``'w|'`` | Open an uncompressed *stream* for writing. |
148 +-------------+--------------------------------------------+
149 | ``'w|gz'`` | Open a gzip compressed *stream* for |
150 | | writing. |
151 +-------------+--------------------------------------------+
152 | ``'w|bz2'`` | Open a bzip2 compressed *stream* for |
153 | | writing. |
154 +-------------+--------------------------------------------+
155 | ``'w|xz'`` | Open an lzma compressed *stream* for |
156 | | writing. |
157 +-------------+--------------------------------------------+
158
159 .. versionchanged:: 3.5
160 The ``'x'`` (exclusive creation) mode was added.
161
162 .. versionchanged:: 3.6
163 The *name* parameter accepts a :term:`path-like object`.
164
165 .. versionchanged:: 3.12
166 The *compresslevel* keyword argument also works for streams.
167
168
169 .. class:: TarFile
170 :noindex:
171
172 Class for reading and writing tar archives. Do not use this class directly:
173 use :func:`tarfile.open` instead. See :ref:`tarfile-objects`.
174
175
176 .. function:: is_tarfile(name)
177
178 Return :const:`True` if *name* is a tar archive file, that the :mod:`tarfile`
179 module can read. *name* may be a :class:`str`, file, or file-like object.
180
181 .. versionchanged:: 3.9
182 Support for file and file-like objects.
183
184
185 The :mod:`tarfile` module defines the following exceptions:
186
187
188 .. exception:: TarError
189
190 Base class for all :mod:`tarfile` exceptions.
191
192
193 .. exception:: ReadError
194
195 Is raised when a tar archive is opened, that either cannot be handled by the
196 :mod:`tarfile` module or is somehow invalid.
197
198
199 .. exception:: CompressionError
200
201 Is raised when a compression method is not supported or when the data cannot be
202 decoded properly.
203
204
205 .. exception:: StreamError
206
207 Is raised for the limitations that are typical for stream-like :class:`TarFile`
208 objects.
209
210
211 .. exception:: ExtractError
212
213 Is raised for *non-fatal* errors when using :meth:`TarFile.extract`, but only if
214 :attr:`TarFile.errorlevel`\ ``== 2``.
215
216
217 .. exception:: HeaderError
218
219 Is raised by :meth:`TarInfo.frombuf` if the buffer it gets is invalid.
220
221
222 .. exception:: FilterError
223
224 Base class for members :ref:`refused <tarfile-extraction-refuse>` by
225 filters.
226
227 .. attribute:: tarinfo
228
229 Information about the member that the filter refused to extract,
230 as :ref:`TarInfo <tarinfo-objects>`.
231
232 .. exception:: AbsolutePathError
233
234 Raised to refuse extracting a member with an absolute path.
235
236 .. exception:: OutsideDestinationError
237
238 Raised to refuse extracting a member outside the destination directory.
239
240 .. exception:: SpecialFileError
241
242 Raised to refuse extracting a special file (e.g. a device or pipe).
243
244 .. exception:: AbsoluteLinkError
245
246 Raised to refuse extracting a symbolic link with an absolute path.
247
248 .. exception:: LinkOutsideDestinationError
249
250 Raised to refuse extracting a symbolic link pointing outside the destination
251 directory.
252
253 .. exception:: LinkFallbackError
254
255 Raised to refuse emulating a link (hard or symbolic) by extracting another
256 archive member, when that member would be rejected by the filter location.
257 The exception that was raised to reject the replacement member is available
258 as :attr:`!BaseException.__context__`.
259
260 .. versionadded:: 3.13.4
261
262
263 The following constants are available at the module level:
264
265 .. data:: ENCODING
266
267 The default character encoding: ``'utf-8'`` on Windows, the value returned by
268 :func:`sys.getfilesystemencoding` otherwise.
269
270 .. data:: REGTYPE
271 AREGTYPE
272
273 A regular file :attr:`~TarInfo.type`.
274
275 .. data:: LNKTYPE
276
277 A link (inside tarfile) :attr:`~TarInfo.type`.
278
279 .. data:: SYMTYPE
280
281 A symbolic link :attr:`~TarInfo.type`.
282
283 .. data:: CHRTYPE
284
285 A character special device :attr:`~TarInfo.type`.
286
287 .. data:: BLKTYPE
288
289 A block special device :attr:`~TarInfo.type`.
290
291 .. data:: DIRTYPE
292
293 A directory :attr:`~TarInfo.type`.
294
295 .. data:: FIFOTYPE
296
297 A FIFO special device :attr:`~TarInfo.type`.
298
299 .. data:: CONTTYPE
300
301 A contiguous file :attr:`~TarInfo.type`.
302
303 .. data:: GNUTYPE_LONGNAME
304
305 A GNU tar longname :attr:`~TarInfo.type`.
306
307 .. data:: GNUTYPE_LONGLINK
308
309 A GNU tar longlink :attr:`~TarInfo.type`.
310
311 .. data:: GNUTYPE_SPARSE
312
313 A GNU tar sparse file :attr:`~TarInfo.type`.
314
315
316 Each of the following constants defines a tar archive format that the
317 :mod:`tarfile` module is able to create. See section :ref:`tar-formats` for
318 details.
319
320
321 .. data:: USTAR_FORMAT
322
323 POSIX.1-1988 (ustar) format.
324
325
326 .. data:: GNU_FORMAT
327
328 GNU tar format.
329
330
331 .. data:: PAX_FORMAT
332
333 POSIX.1-2001 (pax) format.
334
335
336 .. data:: DEFAULT_FORMAT
337
338 The default format for creating archives. This is currently :const:`PAX_FORMAT`.
339
340 .. versionchanged:: 3.8
341 The default format for new archives was changed to
342 :const:`PAX_FORMAT` from :const:`GNU_FORMAT`.
343
344
345 .. seealso::
346
347 Module :mod:`zipfile`
348 Documentation of the :mod:`zipfile` standard module.
349
350 :ref:`archiving-operations`
351 Documentation of the higher-level archiving facilities provided by the
352 standard :mod:`shutil` module.
353
354 `GNU tar manual, Basic Tar Format <https://www.gnu.org/software/tar/manual/html_node/Standard.html>`_
355 Documentation for tar archive files, including GNU tar extensions.
356
357
358 .. _tarfile-objects:
359
360 TarFile Objects
361 ---------------
362
363 The :class:`TarFile` object provides an interface to a tar archive. A tar
364 archive is a sequence of blocks. An archive member (a stored file) is made up of
365 a header block followed by data blocks. It is possible to store a file in a tar
366 archive several times. Each archive member is represented by a :class:`TarInfo`
367 object, see :ref:`tarinfo-objects` for details.
368
369 A :class:`TarFile` object can be used as a context manager in a :keyword:`with`
370 statement. It will automatically be closed when the block is completed. Please
371 note that in the event of an exception an archive opened for writing will not
372 be finalized; only the internally used file object will be closed. See the
373 :ref:`tar-examples` section for a use case.
374
375 .. versionadded:: 3.2
376 Added support for the context management protocol.
377
378 .. class:: TarFile(name=None, mode='r', fileobj=None, format=DEFAULT_FORMAT, tarinfo=TarInfo, dereference=False, ignore_zeros=False, encoding=ENCODING, errors='surrogateescape', pax_headers=None, debug=0, errorlevel=1, stream=False)
379
380 All following arguments are optional and can be accessed as instance attributes
381 as well.
382
383 *name* is the pathname of the archive. *name* may be a :term:`path-like object`.
384 It can be omitted if *fileobj* is given.
385 In this case, the file object's :attr:`!name` attribute is used if it exists.
386
387 *mode* is either ``'r'`` to read from an existing archive, ``'a'`` to append
388 data to an existing file, ``'w'`` to create a new file overwriting an existing
389 one, or ``'x'`` to create a new file only if it does not already exist.
390
391 If *fileobj* is given, it is used for reading or writing data. If it can be
392 determined, *mode* is overridden by *fileobj*'s mode. *fileobj* will be used
393 from position 0.
394
395 .. note::
396
397 *fileobj* is not closed, when :class:`TarFile` is closed.
398
399 *format* controls the archive format for writing. It must be one of the constants
400 :const:`USTAR_FORMAT`, :const:`GNU_FORMAT` or :const:`PAX_FORMAT` that are
401 defined at module level. When reading, format will be automatically detected, even
402 if different formats are present in a single archive.
403
404 The *tarinfo* argument can be used to replace the default :class:`TarInfo` class
405 with a different one.
406
407 If *dereference* is :const:`False`, add symbolic and hard links to the archive. If it
408 is :const:`True`, add the content of the target files to the archive. This has no
409 effect on systems that do not support symbolic links.
410
411 If *ignore_zeros* is :const:`False`, treat an empty block as the end of the archive.
412 If it is :const:`True`, skip empty (and invalid) blocks and try to get as many members
413 as possible. This is only useful for reading concatenated or damaged archives.
414
415 *debug* can be set from ``0`` (no debug messages) up to ``3`` (all debug
416 messages). The messages are written to ``sys.stderr``.
417
418 *errorlevel* controls how extraction errors are handled,
419 see :attr:`the corresponding attribute <TarFile.errorlevel>`.
420
421 The *encoding* and *errors* arguments define the character encoding to be
422 used for reading or writing the archive and how conversion errors are going
423 to be handled. The default settings will work for most users.
424 See section :ref:`tar-unicode` for in-depth information.
425
426 The *pax_headers* argument is an optional dictionary of strings which
427 will be added as a pax global header if *format* is :const:`PAX_FORMAT`.
428
429 If *stream* is set to :const:`True` then while reading the archive info about files
430 in the archive are not cached, saving memory.
431
432 .. versionchanged:: 3.2
433 Use ``'surrogateescape'`` as the default for the *errors* argument.
434
435 .. versionchanged:: 3.5
436 The ``'x'`` (exclusive creation) mode was added.
437
438 .. versionchanged:: 3.6
439 The *name* parameter accepts a :term:`path-like object`.
440
441 .. versionchanged:: 3.13
442 Add the *stream* parameter.
443
444 .. classmethod:: TarFile.open(...)
445
446 Alternative constructor. The :func:`tarfile.open` function is actually a
447 shortcut to this classmethod.
448
449
450 .. method:: TarFile.getmember(name)
451
452 Return a :class:`TarInfo` object for member *name*. If *name* can not be found
453 in the archive, :exc:`KeyError` is raised.
454
455 .. note::
456
457 If a member occurs more than once in the archive, its last occurrence is assumed
458 to be the most up-to-date version.
459
460
461 .. method:: TarFile.getmembers()
462
463 Return the members of the archive as a list of :class:`TarInfo` objects. The
464 list has the same order as the members in the archive.
465
466
467 .. method:: TarFile.getnames()
468
469 Return the members as a list of their names. It has the same order as the list
470 returned by :meth:`getmembers`.
471
472
473 .. method:: TarFile.list(verbose=True, *, members=None)
474
475 Print a table of contents to ``sys.stdout``. If *verbose* is :const:`False`,
476 only the names of the members are printed. If it is :const:`True`, output
477 similar to that of :program:`ls -l` is produced. If optional *members* is
478 given, it must be a subset of the list returned by :meth:`getmembers`.
479
480 .. versionchanged:: 3.5
481 Added the *members* parameter.
482
483
484 .. method:: TarFile.next()
485
486 Return the next member of the archive as a :class:`TarInfo` object, when
487 :class:`TarFile` is opened for reading. Return :const:`None` if there is no more
488 available.
489
490
491 .. method:: TarFile.extractall(path=".", members=None, *, numeric_owner=False, filter=None)
492
493 Extract all members from the archive to the current working directory or
494 directory *path*. If optional *members* is given, it must be a subset of the
495 list returned by :meth:`getmembers`. Directory information like owner,
496 modification time and permissions are set after all members have been extracted.
497 This is done to work around two problems: A directory's modification time is
498 reset each time a file is created in it. And, if a directory's permissions do
499 not allow writing, extracting files to it will fail.
500
501 If *numeric_owner* is :const:`True`, the uid and gid numbers from the tarfile
502 are used to set the owner/group for the extracted files. Otherwise, the named
503 values from the tarfile are used.
504
505 The *filter* argument specifies how ``members`` are modified or rejected
506 before extraction.
507 See :ref:`tarfile-extraction-filter` for details.
508 It is recommended to set this explicitly depending on which *tar* features
509 you need to support.
510
511 .. warning::
512
513 Never extract archives from untrusted sources without prior inspection.
514 It is possible that files are created outside of *path*, e.g. members
515 that have absolute filenames starting with ``"/"`` or filenames with two
516 dots ``".."``.
517
518 Set ``filter='data'`` to prevent the most dangerous security issues,
519 and read the :ref:`tarfile-extraction-filter` section for details.
520
521 .. versionchanged:: 3.5
522 Added the *numeric_owner* parameter.
523
524 .. versionchanged:: 3.6
525 The *path* parameter accepts a :term:`path-like object`.
526
527 .. versionchanged:: 3.12
528 Added the *filter* parameter.
529
530
531 .. method:: TarFile.extract(member, path="", set_attrs=True, *, numeric_owner=False, filter=None)
532
533 Extract a member from the archive to the current working directory, using its
534 full name. Its file information is extracted as accurately as possible. *member*
535 may be a filename or a :class:`TarInfo` object. You can specify a different
536 directory using *path*. *path* may be a :term:`path-like object`.
537 File attributes (owner, mtime, mode) are set unless *set_attrs* is false.
538
539 The *numeric_owner* and *filter* arguments are the same as
540 for :meth:`extractall`.
541
542 .. note::
543
544 The :meth:`extract` method does not take care of several extraction issues.
545 In most cases you should consider using the :meth:`extractall` method.
546
547 .. warning::
548
549 See the warning for :meth:`extractall`.
550
551 Set ``filter='data'`` to prevent the most dangerous security issues,
552 and read the :ref:`tarfile-extraction-filter` section for details.
553
554 .. versionchanged:: 3.2
555 Added the *set_attrs* parameter.
556
557 .. versionchanged:: 3.5
558 Added the *numeric_owner* parameter.
559
560 .. versionchanged:: 3.6
561 The *path* parameter accepts a :term:`path-like object`.
562
563 .. versionchanged:: 3.12
564 Added the *filter* parameter.
565
566
567 .. method:: TarFile.extractfile(member)
568
569 Extract a member from the archive as a file object. *member* may be
570 a filename or a :class:`TarInfo` object. If *member* is a regular file or
571 a link, an :class:`io.BufferedReader` object is returned. For all other
572 existing members, :const:`None` is returned. If *member* does not appear
573 in the archive, :exc:`KeyError` is raised.
574
575 .. versionchanged:: 3.3
576 Return an :class:`io.BufferedReader` object.
577
578 .. versionchanged:: 3.13
579 The returned :class:`io.BufferedReader` object has the :attr:`!mode`
580 attribute which is always equal to ``'rb'``.
581
582 .. attribute:: TarFile.errorlevel
583 :type: int
584
585 If *errorlevel* is ``0``, errors are ignored when using :meth:`TarFile.extract`
586 and :meth:`TarFile.extractall`.
587 Nevertheless, they appear as error messages in the debug output when
588 *debug* is greater than 0.
589 If ``1`` (the default), all *fatal* errors are raised as :exc:`OSError` or
590 :exc:`FilterError` exceptions. If ``2``, all *non-fatal* errors are raised
591 as :exc:`TarError` exceptions as well.
592
593 Some exceptions, e.g. ones caused by wrong argument types or data
594 corruption, are always raised.
595
596 Custom :ref:`extraction filters <tarfile-extraction-filter>`
597 should raise :exc:`FilterError` for *fatal* errors
598 and :exc:`ExtractError` for *non-fatal* ones.
599
600 Note that when an exception is raised, the archive may be partially
601 extracted. It is the user’s responsibility to clean up.
602
603 .. attribute:: TarFile.extraction_filter
604
605 .. versionadded:: 3.12
606
607 The :ref:`extraction filter <tarfile-extraction-filter>` used
608 as a default for the *filter* argument of :meth:`~TarFile.extract`
609 and :meth:`~TarFile.extractall`.
610
611 The attribute may be ``None`` or a callable.
612 String names are not allowed for this attribute, unlike the *filter*
613 argument to :meth:`~TarFile.extract`.
614
615 If ``extraction_filter`` is ``None`` (the default),
616 calling an extraction method without a *filter* argument will raise a
617 ``DeprecationWarning``,
618 and fall back to the :func:`fully_trusted <fully_trusted_filter>` filter,
619 whose dangerous behavior matches previous versions of Python.
620
621 In Python 3.14+, leaving ``extraction_filter=None`` will cause
622 extraction methods to use the :func:`data <data_filter>` filter by default.
623
624 The attribute may be set on instances or overridden in subclasses.
625 It also is possible to set it on the ``TarFile`` class itself to set a
626 global default, although, since it affects all uses of *tarfile*,
627 it is best practice to only do so in top-level applications or
628 :mod:`site configuration <site>`.
629 To set a global default this way, a filter function needs to be wrapped in
630 :deco:`staticmethod` to prevent injection of a ``self`` argument.
631
632 .. method:: TarFile.add(name, arcname=None, recursive=True, *, filter=None)
633
634 Add the file *name* to the archive. *name* may be any type of file
635 (directory, fifo, symbolic link, etc.). If given, *arcname* specifies an
636 alternative name for the file in the archive. Directories are added
637 recursively by default. This can be avoided by setting *recursive* to
638 :const:`False`. Recursion adds entries in sorted order.
639 If *filter* is given, it
640 should be a function that takes a :class:`TarInfo` object argument and
641 returns the changed :class:`TarInfo` object. If it instead returns
642 :const:`None` the :class:`TarInfo` object will be excluded from the
643 archive. See :ref:`tar-examples` for an example.
644
645 .. versionchanged:: 3.2
646 Added the *filter* parameter.
647
648 .. versionchanged:: 3.7
649 Recursion adds entries in sorted order.
650
651
652 .. method:: TarFile.addfile(tarinfo, fileobj=None)
653
654 Add the :class:`TarInfo` object *tarinfo* to the archive. If *tarinfo* represents
655 a non zero-size regular file, the *fileobj* argument should be a :term:`binary file`,
656 and ``tarinfo.size`` bytes are read from it and added to the archive. You can
657 create :class:`TarInfo` objects directly, or by using :meth:`gettarinfo`.
658
659 .. versionchanged:: 3.13
660
661 *fileobj* must be given for non-zero-sized regular files.
662
663
664 .. method:: TarFile.gettarinfo(name=None, arcname=None, fileobj=None)
665
666 Create a :class:`TarInfo` object from the result of :func:`os.stat` or
667 equivalent on an existing file. The file is either named by *name*, or
668 specified as a :term:`file object` *fileobj* with a file descriptor.
669 *name* may be a :term:`path-like object`. If
670 given, *arcname* specifies an alternative name for the file in the
671 archive, otherwise, the name is taken from *fileobj*’s
672 :attr:`~io.FileIO.name` attribute, or the *name* argument. The name
673 should be a text string.
674
675 You can modify
676 some of the :class:`TarInfo`’s attributes before you add it using :meth:`addfile`.
677 If the file object is not an ordinary file object positioned at the
678 beginning of the file, attributes such as :attr:`~TarInfo.size` may need
679 modifying. This is the case for objects such as :class:`~gzip.GzipFile`.
680 The :attr:`~TarInfo.name` may also be modified, in which case *arcname*
681 could be a dummy string.
682
683 .. versionchanged:: 3.6
684 The *name* parameter accepts a :term:`path-like object`.
685
686
687 .. method:: TarFile.close()
688
689 Close the :class:`TarFile`. In write mode, two finishing zero blocks are
690 appended to the archive.
691
692
693 .. attribute:: TarFile.pax_headers
694 :type: dict
695
696 A dictionary containing key-value pairs of pax global headers.
697
698
699
700 .. _tarinfo-objects:
701
702 TarInfo Objects
703 ---------------
704
705 A :class:`TarInfo` object represents one member in a :class:`TarFile`. Aside
706 from storing all required attributes of a file (like file type, size, time,
707 permissions, owner etc.), it provides some useful methods to determine its type.
708 It does *not* contain the file's data itself.
709
710 :class:`TarInfo` objects are returned by :class:`TarFile`'s methods
711 :meth:`~TarFile.getmember`, :meth:`~TarFile.getmembers` and
712 :meth:`~TarFile.gettarinfo`.
713
714 Modifying the objects returned by :meth:`~TarFile.getmember` or
715 :meth:`~TarFile.getmembers` will affect all subsequent
716 operations on the archive.
717 For cases where this is unwanted, you can use :mod:`copy.copy() <copy>` or
718 call the :meth:`~TarInfo.replace` method to create a modified copy in one step.
719
720 Several attributes can be set to ``None`` to indicate that a piece of metadata
721 is unused or unknown.
722 Different :class:`TarInfo` methods handle ``None`` differently:
723
724 - The :meth:`~TarFile.extract` or :meth:`~TarFile.extractall` methods will
725 ignore the corresponding metadata, leaving it set to a default.
726 - :meth:`~TarFile.addfile` will fail.
727 - :meth:`~TarFile.list` will print a placeholder string.
728
729 .. class:: TarInfo(name="")
730
731 Create a :class:`TarInfo` object.
732
733
734 .. classmethod:: TarInfo.frombuf(buf, encoding, errors)
735
736 Create and return a :class:`TarInfo` object from string buffer *buf*.
737
738 Raises :exc:`HeaderError` if the buffer is invalid.
739
740
741 .. classmethod:: TarInfo.fromtarfile(tarfile)
742
743 Read the next member from the :class:`TarFile` object *tarfile* and return it as
744 a :class:`TarInfo` object.
745
746
747 .. method:: TarInfo.tobuf(format=DEFAULT_FORMAT, encoding=ENCODING, errors='surrogateescape')
748
749 Create a string buffer from a :class:`TarInfo` object. For information on the
750 arguments see the constructor of the :class:`TarFile` class.
751
752 .. versionchanged:: 3.2
753 Use ``'surrogateescape'`` as the default for the *errors* argument.
754
755
756 A ``TarInfo`` object has the following public data attributes:
757
758
759 .. attribute:: TarInfo.name
760 :type: str
761
762 Name of the archive member.
763
764
765 .. attribute:: TarInfo.size
766 :type: int
767
768 Size in bytes.
769
770
771 .. attribute:: TarInfo.mtime
772 :type: int | float
773
774 Time of last modification in seconds since the :ref:`epoch <epoch>`,
775 as in :attr:`os.stat_result.st_mtime`.
776
777 .. versionchanged:: 3.12
778
779 Can be set to ``None`` for :meth:`~TarFile.extract` and
780 :meth:`~TarFile.extractall`, causing extraction to skip applying this
781 attribute.
782
783 .. attribute:: TarInfo.mode
784 :type: int
785
786 Permission bits, as for :func:`os.chmod`.
787
788 .. versionchanged:: 3.12
789
790 Can be set to ``None`` for :meth:`~TarFile.extract` and
791 :meth:`~TarFile.extractall`, causing extraction to skip applying this
792 attribute.
793
794 .. attribute:: TarInfo.type
795
796 File type. *type* is usually one of these constants: :const:`REGTYPE`,
797 :const:`AREGTYPE`, :const:`LNKTYPE`, :const:`SYMTYPE`, :const:`DIRTYPE`,
798 :const:`FIFOTYPE`, :const:`CONTTYPE`, :const:`CHRTYPE`, :const:`BLKTYPE`,
799 :const:`GNUTYPE_SPARSE`. To determine the type of a :class:`TarInfo` object
800 more conveniently, use the ``is*()`` methods below.
801
802
803 .. attribute:: TarInfo.linkname
804 :type: str
805
806 Name of the target file name, which is only present in :class:`TarInfo` objects
807 of type :const:`LNKTYPE` and :const:`SYMTYPE`.
808
809 For symbolic links (``SYMTYPE``), the *linkname* is relative to the directory
810 that contains the link.
811 For hard links (``LNKTYPE``), the *linkname* is relative to the root of
812 the archive.
813
814
815 .. attribute:: TarInfo.uid
816 :type: int
817
818 User ID of the user who originally stored this member.
819
820 .. versionchanged:: 3.12
821
822 Can be set to ``None`` for :meth:`~TarFile.extract` and
823 :meth:`~TarFile.extractall`, causing extraction to skip applying this
824 attribute.
825
826 .. attribute:: TarInfo.gid
827 :type: int
828
829 Group ID of the user who originally stored this member.
830
831 .. versionchanged:: 3.12
832
833 Can be set to ``None`` for :meth:`~TarFile.extract` and
834 :meth:`~TarFile.extractall`, causing extraction to skip applying this
835 attribute.
836
837 .. attribute:: TarInfo.uname
838 :type: str
839
840 User name.
841
842 .. versionchanged:: 3.12
843
844 Can be set to ``None`` for :meth:`~TarFile.extract` and
845 :meth:`~TarFile.extractall`, causing extraction to skip applying this
846 attribute.
847
848 .. attribute:: TarInfo.gname
849 :type: str
850
851 Group name.
852
853 .. versionchanged:: 3.12
854
855 Can be set to ``None`` for :meth:`~TarFile.extract` and
856 :meth:`~TarFile.extractall`, causing extraction to skip applying this
857 attribute.
858
859 .. attribute:: TarInfo.chksum
860 :type: int
861
862 Header checksum.
863
864
865 .. attribute:: TarInfo.devmajor
866 :type: int
867
868 Device major number.
869
870
871 .. attribute:: TarInfo.devminor
872 :type: int
873
874 Device minor number.
875
876
877 .. attribute:: TarInfo.offset
878 :type: int
879
880 The tar header starts here.
881
882
883 .. attribute:: TarInfo.offset_data
884 :type: int
885
886 The file's data starts here.
887
888
889 .. attribute:: TarInfo.sparse
890
891 Sparse member information.
892
893
894 .. attribute:: TarInfo.pax_headers
895 :type: dict
896
897 A dictionary containing key-value pairs of an associated pax extended header.
898
899 .. method:: TarInfo.replace(name=..., mtime=..., mode=..., linkname=..., \
900 uid=..., gid=..., uname=..., gname=..., \
901 deep=True)
902
903 .. versionadded:: 3.12
904
905 Return a *new* copy of the :class:`!TarInfo` object with the given attributes
906 changed. For example, to return a ``TarInfo`` with the group name set to
907 ``'staff'``, use::
908
909 new_tarinfo = old_tarinfo.replace(gname='staff')
910
911 By default, a deep copy is made.
912 If *deep* is false, the copy is shallow, i.e. ``pax_headers``
913 and any custom attributes are shared with the original ``TarInfo`` object.
914
915 A :class:`TarInfo` object also provides some convenient query methods:
916
917
918 .. method:: TarInfo.isfile()
919
920 Return :const:`True` if the :class:`TarInfo` object is a regular file.
921
922
923 .. method:: TarInfo.isreg()
924
925 Same as :meth:`isfile`.
926
927
928 .. method:: TarInfo.isdir()
929
930 Return :const:`True` if it is a directory.
931
932
933 .. method:: TarInfo.issym()
934
935 Return :const:`True` if it is a symbolic link.
936
937
938 .. method:: TarInfo.islnk()
939
940 Return :const:`True` if it is a hard link.
941
942
943 .. method:: TarInfo.ischr()
944
945 Return :const:`True` if it is a character device.
946
947
948 .. method:: TarInfo.isblk()
949
950 Return :const:`True` if it is a block device.
951
952
953 .. method:: TarInfo.isfifo()
954
955 Return :const:`True` if it is a FIFO.
956
957
958 .. method:: TarInfo.isdev()
959
960 Return :const:`True` if it is one of character device, block device or FIFO.
961
962
963 .. _tarfile-extraction-filter:
964
965 Extraction filters
966 ------------------
967
968 .. versionadded:: 3.12
969
970 The *tar* format is designed to capture all details of a UNIX-like filesystem,
971 which makes it very powerful.
972 Unfortunately, the features make it easy to create tar files that have
973 unintended -- and possibly malicious -- effects when extracted.
974 For example, extracting a tar file can overwrite arbitrary files in various
975 ways (e.g. by using absolute paths, ``..`` path components, or symlinks that
976 affect later members).
977
978 In most cases, the full functionality is not needed.
979 Therefore, *tarfile* supports extraction filters: a mechanism to limit
980 functionality, and thus mitigate some of the security issues.
981
982 .. seealso::
983
984 :pep:`706`
985 Contains further motivation and rationale behind the design.
986
987 The *filter* argument to :meth:`TarFile.extract` or :meth:`~TarFile.extractall`
988 can be:
989
990 * the string ``'fully_trusted'``: Honor all metadata as specified in the
991 archive.
992 Should be used if the user trusts the archive completely, or implements
993 their own complex verification.
994
995 * the string ``'tar'``: Honor most *tar*-specific features (i.e. features of
996 UNIX-like filesystems), but block features that are very likely to be
997 surprising or malicious. See :func:`tar_filter` for details.
998
999 * the string ``'data'``: Ignore or block most features specific to UNIX-like
1000 filesystems. Intended for extracting cross-platform data archives.
1001 See :func:`data_filter` for details.
1002
1003 * ``None`` (default): Use :attr:`TarFile.extraction_filter`.
1004
1005 If that is also ``None`` (the default), raise a ``DeprecationWarning``,
1006 and fall back to the ``'fully_trusted'`` filter, whose dangerous behavior
1007 matches previous versions of Python.
1008
1009 In Python 3.14, the ``'data'`` filter will become the default instead.
1010 It's possible to switch earlier; see :attr:`TarFile.extraction_filter`.
1011
1012 * A callable which will be called for each extracted member with a
1013 :ref:`TarInfo <tarinfo-objects>` describing the member and the destination
1014 path to where the archive is extracted (i.e. the same path is used for all
1015 members)::
1016
1017 filter(member: TarInfo, path: str, /) -> TarInfo | None
1018
1019 The callable is called just before each member is extracted, so it can
1020 take the current state of the disk into account.
1021 It can:
1022
1023 - return a :class:`TarInfo` object which will be used instead of the metadata
1024 in the archive, or
1025 - return ``None``, in which case the member will be skipped, or
1026 - raise an exception to abort the operation or skip the member,
1027 depending on :attr:`~TarFile.errorlevel`.
1028 Note that when extraction is aborted, :meth:`~TarFile.extractall` may leave
1029 the archive partially extracted. It does not attempt to clean up.
1030
1031 Default named filters
1032 ~~~~~~~~~~~~~~~~~~~~~
1033
1034 The pre-defined, named filters are available as functions, so they can be
1035 reused in custom filters:
1036
1037 .. function:: fully_trusted_filter(member, path)
1038
1039 Return *member* unchanged.
1040
1041 This implements the ``'fully_trusted'`` filter.
1042
1043 .. function:: tar_filter(member, path)
1044
1045 Implements the ``'tar'`` filter.
1046
1047 - Strip leading slashes (``/`` and :data:`os.sep`) from filenames.
1048 - :ref:`Refuse <tarfile-extraction-refuse>` to extract files with absolute
1049 paths (in case the name is absolute
1050 even after stripping slashes, e.g. ``C:/foo`` on Windows).
1051 This raises :class:`~tarfile.AbsolutePathError`.
1052 - :ref:`Refuse <tarfile-extraction-refuse>` to extract files whose absolute
1053 path (after following symlinks) would end up outside the destination.
1054 This raises :class:`~tarfile.OutsideDestinationError`.
1055 - Clear high mode bits (setuid, setgid, sticky) and group/other write bits
1056 (:const:`~stat.S_IWGRP` | :const:`~stat.S_IWOTH`).
1057
1058 Return the modified ``TarInfo`` member.
1059
1060 .. function:: data_filter(member, path)
1061
1062 Implements the ``'data'`` filter.
1063 In addition to what ``tar_filter`` does:
1064
1065 - Normalize link targets (:attr:`TarInfo.linkname`) using
1066 :func:`os.path.normpath`.
1067 Note that this removes internal ``..`` components, which may change the
1068 meaning of the link if the path in :attr:`!TarInfo.linkname` traverses
1069 symbolic links.
1070
1071 - :ref:`Refuse <tarfile-extraction-refuse>` to extract links (hard or soft)
1072 that link to absolute paths, or ones that link outside the destination.
1073
1074 This raises :class:`~tarfile.AbsoluteLinkError` or
1075 :class:`~tarfile.LinkOutsideDestinationError`.
1076
1077 Note that such files are refused even on platforms that do not support
1078 symbolic links.
1079
1080 - :ref:`Refuse <tarfile-extraction-refuse>` to extract device files
1081 (including pipes).
1082 This raises :class:`~tarfile.SpecialFileError`.
1083
1084 - For regular files, including hard links:
1085
1086 - Set the owner read and write permissions
1087 (:const:`~stat.S_IRUSR` | :const:`~stat.S_IWUSR`).
1088 - Remove the group & other executable permission
1089 (:const:`~stat.S_IXGRP` | :const:`~stat.S_IXOTH`)
1090 if the owner doesn’t have it (:const:`~stat.S_IXUSR`).
1091
1092 - For other files (directories), set ``mode`` to ``None``, so
1093 that extraction methods skip applying permission bits.
1094 - Set user and group info (``uid``, ``gid``, ``uname``, ``gname``)
1095 to ``None``, so that extraction methods skip setting it.
1096
1097 Return the modified ``TarInfo`` member.
1098
1099 .. versionchanged:: 3.13.4
1100
1101 Link targets are now normalized.
1102
1103
1104 .. _tarfile-extraction-refuse:
1105
1106 Filter errors
1107 ~~~~~~~~~~~~~
1108
1109 When a filter refuses to extract a file, it will raise an appropriate exception,
1110 a subclass of :class:`~tarfile.FilterError`.
1111 This will abort the extraction if :attr:`TarFile.errorlevel` is 1 or more.
1112 With ``errorlevel=0`` the error will be logged and the member will be skipped,
1113 but extraction will continue.
1114
1115
1116 Hints for further verification
1117 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1118
1119 Even with ``filter='data'``, *tarfile* is not suited for extracting untrusted
1120 files without prior inspection.
1121 Among other issues, the pre-defined filters do not prevent denial-of-service
1122 attacks. Users should do additional checks.
1123
1124 Here is an incomplete list of things to consider:
1125
1126 * Extract to a :func:`new temporary directory <tempfile.mkdtemp>`
1127 to prevent e.g. exploiting pre-existing links, and to make it easier to
1128 clean up after a failed extraction.
1129 * Disallow symbolic links if you do not need the functionality.
1130 * When working with untrusted data, use external (e.g. OS-level) limits on
1131 disk, memory and CPU usage.
1132 * Check filenames against an allow-list of characters
1133 (to filter out control characters, confusables, foreign path separators,
1134 etc.).
1135 * Check that filenames have expected extensions (discouraging files that
1136 execute when you “click on them”, or extension-less files like Windows special device names).
1137 * Limit the number of extracted files, total size of extracted data,
1138 filename length (including symlink length), and size of individual files.
1139 * Check for files that would be shadowed on case-insensitive filesystems.
1140
1141 Also note that:
1142
1143 * Tar files may contain multiple versions of the same file.
1144 Later ones are expected to overwrite any earlier ones.
1145 This feature is crucial to allow updating tape archives, but can be abused
1146 maliciously.
1147 * *tarfile* does not protect against issues with “live” data,
1148 e.g. an attacker tinkering with the destination (or source) directory while
1149 extraction (or archiving) is in progress.
1150
1151
1152 Supporting older Python versions
1153 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1154
1155 Extraction filters were added to Python 3.12, but may be backported to older
1156 versions as security updates.
1157 To check whether the feature is available, use e.g.
1158 ``hasattr(tarfile, 'data_filter')`` rather than checking the Python version.
1159
1160 The following examples show how to support Python versions with and without
1161 the feature.
1162 Note that setting ``extraction_filter`` will affect any subsequent operations.
1163
1164 * Fully trusted archive::
1165
1166 my_tarfile.extraction_filter = (lambda member, path: member)
1167 my_tarfile.extractall()
1168
1169 * Use the ``'data'`` filter if available, but revert to Python 3.11 behavior
1170 (``'fully_trusted'``) if this feature is not available::
1171
1172 my_tarfile.extraction_filter = getattr(tarfile, 'data_filter',
1173 (lambda member, path: member))
1174 my_tarfile.extractall()
1175
1176 * Use the ``'data'`` filter; *fail* if it is not available::
1177
1178 my_tarfile.extractall(filter=tarfile.data_filter)
1179
1180 or::
1181
1182 my_tarfile.extraction_filter = tarfile.data_filter
1183 my_tarfile.extractall()
1184
1185 * Use the ``'data'`` filter; *warn* if it is not available::
1186
1187 if hasattr(tarfile, 'data_filter'):
1188 my_tarfile.extractall(filter='data')
1189 else:
1190 # remove this when no longer needed
1191 warn_the_user('Extracting may be unsafe; consider updating Python')
1192 my_tarfile.extractall()
1193
1194
1195 Stateful extraction filter example
1196 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1197
1198 While *tarfile*'s extraction methods take a simple *filter* callable,
1199 custom filters may be more complex objects with an internal state.
1200 It may be useful to write these as context managers, to be used like this::
1201
1202 with StatefulFilter() as filter_func:
1203 tar.extractall(path, filter=filter_func)
1204
1205 Such a filter can be written as, for example::
1206
1207 class StatefulFilter:
1208 def __init__(self):
1209 self.file_count = 0
1210
1211 def __enter__(self):
1212 return self
1213
1214 def __call__(self, member, path):
1215 self.file_count += 1
1216 return member
1217
1218 def __exit__(self, *exc_info):
1219 print(f'{self.file_count} files extracted')
1220
1221
1222 .. _tarfile-commandline:
1223 .. program:: tarfile
1224
1225
1226 Command-Line Interface
1227 ----------------------
1228
1229 .. versionadded:: 3.4
1230
1231 The :mod:`tarfile` module provides a simple command-line interface to interact
1232 with tar archives.
1233
1234 If you want to create a new tar archive, specify its name after the :option:`-c`
1235 option and then list the filename(s) that should be included:
1236
1237 .. code-block:: shell-session
1238
1239 $ python -m tarfile -c monty.tar spam.txt eggs.txt
1240
1241 Passing a directory is also acceptable:
1242
1243 .. code-block:: shell-session
1244
1245 $ python -m tarfile -c monty.tar life-of-brian_1979/
1246
1247 If you want to extract a tar archive into the current directory, use
1248 the :option:`-e` option:
1249
1250 .. code-block:: shell-session
1251
1252 $ python -m tarfile -e monty.tar
1253
1254 You can also extract a tar archive into a different directory by passing the
1255 directory's name:
1256
1257 .. code-block:: shell-session
1258
1259 $ python -m tarfile -e monty.tar other-dir/
1260
1261 For a list of the files in a tar archive, use the :option:`-l` option:
1262
1263 .. code-block:: shell-session
1264
1265 $ python -m tarfile -l monty.tar
1266
1267
1268 Command-line options
1269 ~~~~~~~~~~~~~~~~~~~~
1270
1271 .. option:: -l <tarfile>
1272 --list <tarfile>
1273
1274 List files in a tarfile.
1275
1276 .. option:: -c <tarfile> <source1> ... <sourceN>
1277 --create <tarfile> <source1> ... <sourceN>
1278
1279 Create tarfile from source files.
1280
1281 .. option:: -e <tarfile> [<output_dir>]
1282 --extract <tarfile> [<output_dir>]
1283
1284 Extract tarfile into the current directory if *output_dir* is not specified.
1285
1286 .. option:: -t <tarfile>
1287 --test <tarfile>
1288
1289 Test whether the tarfile is valid or not.
1290
1291 .. option:: -v, --verbose
1292
1293 Verbose output.
1294
1295 .. option:: --filter <filtername>
1296
1297 Specifies the *filter* for ``--extract``.
1298 See :ref:`tarfile-extraction-filter` for details.
1299 Only string names are accepted (that is, ``fully_trusted``, ``tar``,
1300 and ``data``).
1301
1302 .. _tar-examples:
1303
1304 Examples
1305 --------
1306
1307 Reading examples
1308 ~~~~~~~~~~~~~~~~~~~
1309
1310 How to extract an entire tar archive to the current working directory::
1311
1312 import tarfile
1313 tar = tarfile.open("sample.tar.gz")
1314 tar.extractall(filter='data')
1315 tar.close()
1316
1317 How to extract a subset of a tar archive with :meth:`TarFile.extractall` using
1318 a generator function instead of a list::
1319
1320 import os
1321 import tarfile
1322
1323 def py_files(members):
1324 for tarinfo in members:
1325 if os.path.splitext(tarinfo.name)[1] == ".py":
1326 yield tarinfo
1327
1328 tar = tarfile.open("sample.tar.gz")
1329 tar.extractall(members=py_files(tar))
1330 tar.close()
1331
1332 How to read a gzip compressed tar archive and display some member information::
1333
1334 import tarfile
1335 tar = tarfile.open("sample.tar.gz", "r:gz")
1336 for tarinfo in tar:
1337 print(tarinfo.name, "is", tarinfo.size, "bytes in size and is ", end="")
1338 if tarinfo.isreg():
1339 print("a regular file.")
1340 elif tarinfo.isdir():
1341 print("a directory.")
1342 else:
1343 print("something else.")
1344 tar.close()
1345
1346 Writing examples
1347 ~~~~~~~~~~~~~~~~
1348
1349 How to create an uncompressed tar archive from a list of filenames::
1350
1351 import tarfile
1352 tar = tarfile.open("sample.tar", "w")
1353 for name in ["foo", "bar", "quux"]:
1354 tar.add(name)
1355 tar.close()
1356
1357 The same example using the :keyword:`with` statement::
1358
1359 import tarfile
1360 with tarfile.open("sample.tar", "w") as tar:
1361 for name in ["foo", "bar", "quux"]:
1362 tar.add(name)
1363
1364 How to create and write an archive to stdout using
1365 :data:`sys.stdout.buffer <sys.stdout>` in the *fileobj* parameter
1366 in :meth:`TarFile.add`::
1367
1368 import sys
1369 import tarfile
1370 with tarfile.open("sample.tar.gz", "w|gz", fileobj=sys.stdout.buffer) as tar:
1371 for name in ["foo", "bar", "quux"]:
1372 tar.add(name)
1373
1374 How to create an archive and reset the user information using the *filter*
1375 parameter in :meth:`TarFile.add`::
1376
1377 import tarfile
1378 def reset(tarinfo):
1379 tarinfo.uid = tarinfo.gid = 0
1380 tarinfo.uname = tarinfo.gname = "root"
1381 return tarinfo
1382 tar = tarfile.open("sample.tar.gz", "w:gz")
1383 tar.add("foo", filter=reset)
1384 tar.close()
1385
1386
1387 .. _tar-formats:
1388
1389 Supported tar formats
1390 ---------------------
1391
1392 There are three tar formats that can be created with the :mod:`tarfile` module:
1393
1394 * The POSIX.1-1988 ustar format (:const:`USTAR_FORMAT`). It supports filenames
1395 up to a length of at best 256 characters and linknames up to 100 characters.
1396 The maximum file size is 8 GiB. This is an old and limited but widely
1397 supported format.
1398
1399 * The GNU tar format (:const:`GNU_FORMAT`). It supports long filenames and
1400 linknames, files bigger than 8 GiB and sparse files. It is the de facto
1401 standard on GNU/Linux systems. :mod:`tarfile` fully supports the GNU tar
1402 extensions for long names, sparse file support is read-only.
1403
1404 * The POSIX.1-2001 pax format (:const:`PAX_FORMAT`). It is the most flexible
1405 format with virtually no limits. It supports long filenames and linknames, large
1406 files and stores pathnames in a portable way. Modern tar implementations,
1407 including GNU tar, bsdtar/libarchive and star, fully support extended *pax*
1408 features; some old or unmaintained libraries may not, but should treat
1409 *pax* archives as if they were in the universally supported *ustar* format.
1410 It is the current default format for new archives.
1411
1412 It extends the existing *ustar* format with extra headers for information
1413 that cannot be stored otherwise. There are two flavours of pax headers:
1414 Extended headers only affect the subsequent file header, global
1415 headers are valid for the complete archive and affect all following files.
1416 All the data in a pax header is encoded in *UTF-8* for portability reasons.
1417
1418 There are some more variants of the tar format which can be read, but not
1419 created:
1420
1421 * The ancient V7 format. This is the first tar format from Unix Seventh Edition,
1422 storing only regular files and directories. Names must not be longer than 100
1423 characters, there is no user/group name information. Some archives have
1424 miscalculated header checksums in case of fields with non-ASCII characters.
1425
1426 * The SunOS tar extended format. This format is a variant of the POSIX.1-2001
1427 pax format, but is not compatible.
1428
1429 .. _tar-unicode:
1430
1431 Unicode issues
1432 --------------
1433
1434 The tar format was originally conceived to make backups on tape drives with the
1435 main focus on preserving file system information. Nowadays tar archives are
1436 commonly used for file distribution and exchanging archives over networks. One
1437 problem of the original format (which is the basis of all other formats) is
1438 that there is no concept of supporting different character encodings. For
1439 example, an ordinary tar archive created on a *UTF-8* system cannot be read
1440 correctly on a *Latin-1* system if it contains non-*ASCII* characters. Textual
1441 metadata (like filenames, linknames, user/group names) will appear damaged.
1442 Unfortunately, there is no way to autodetect the encoding of an archive. The
1443 pax format was designed to solve this problem. It stores non-ASCII metadata
1444 using the universal character encoding *UTF-8*.
1445
1446 The details of character conversion in :mod:`tarfile` are controlled by the
1447 *encoding* and *errors* keyword arguments of the :class:`TarFile` class.
1448
1449 *encoding* defines the character encoding to use for the metadata in the
1450 archive. The default value is :func:`sys.getfilesystemencoding` or ``'ascii'``
1451 as a fallback. Depending on whether the archive is read or written, the
1452 metadata must be either decoded or encoded. If *encoding* is not set
1453 appropriately, this conversion may fail.
1454
1455 The *errors* argument defines how characters are treated that cannot be
1456 converted. Possible values are listed in section :ref:`error-handlers`.
1457 The default scheme is ``'surrogateescape'`` which Python also uses for its
1458 file system calls, see :ref:`os-filenames`.
1459
1460 For :const:`PAX_FORMAT` archives (the default), *encoding* is generally not needed
1461 because all the metadata is stored using *UTF-8*. *encoding* is only used in
1462 the rare cases when binary pax headers are decoded or when strings with
1463 surrogate characters are stored.