| 1 |
:mod:`!gzip` --- Support for :program:`gzip` files |
| 2 |
================================================== |
| 3 |
|
| 4 |
.. module:: gzip |
| 5 |
:synopsis: Interfaces for gzip compression and decompression using file objects. |
| 6 |
|
| 7 |
**Source code:** :source:`Lib/gzip.py` |
| 8 |
|
| 9 |
-------------- |
| 10 |
|
| 11 |
This module provides a simple interface to compress and decompress files just |
| 12 |
like the GNU programs :program:`gzip` and :program:`gunzip` would. |
| 13 |
|
| 14 |
.. include:: ../includes/optional-module.rst |
| 15 |
|
| 16 |
The data compression is provided by the :mod:`zlib` module. |
| 17 |
|
| 18 |
The :mod:`!gzip` module provides the :class:`GzipFile` class, as well as the |
| 19 |
:func:`.open`, :func:`compress` and :func:`decompress` convenience functions. |
| 20 |
The :class:`GzipFile` class reads and writes :program:`gzip`\ -format files, |
| 21 |
automatically compressing or decompressing the data so that it looks like an |
| 22 |
ordinary :term:`file object`. |
| 23 |
|
| 24 |
Note that additional file formats which can be decompressed by the |
| 25 |
:program:`gzip` and :program:`gunzip` programs, such as those produced by |
| 26 |
:program:`compress` and :program:`pack`, are not supported by this module. |
| 27 |
|
| 28 |
The module defines the following items: |
| 29 |
|
| 30 |
|
| 31 |
.. function:: open(filename, mode='rb', compresslevel=9, encoding=None, errors=None, newline=None) |
| 32 |
|
| 33 |
Open a gzip-compressed file in binary or text mode, returning a :term:`file |
| 34 |
object`. |
| 35 |
|
| 36 |
The *filename* argument can be an actual filename (a :class:`str` or |
| 37 |
:class:`bytes` object), or an existing file object to read from or write to. |
| 38 |
|
| 39 |
The *mode* argument can be any of ``'r'``, ``'rb'``, ``'a'``, ``'ab'``, |
| 40 |
``'w'``, ``'wb'``, ``'x'`` or ``'xb'`` for binary mode, or ``'rt'``, |
| 41 |
``'at'``, ``'wt'``, or ``'xt'`` for text mode. The default is ``'rb'``. |
| 42 |
|
| 43 |
The *compresslevel* argument is an integer from 0 to 9, as for the |
| 44 |
:class:`GzipFile` constructor. |
| 45 |
|
| 46 |
For binary mode, this function is equivalent to the :class:`GzipFile` |
| 47 |
constructor: ``GzipFile(filename, mode, compresslevel)``. In this case, the |
| 48 |
*encoding*, *errors* and *newline* arguments must not be provided. |
| 49 |
|
| 50 |
For text mode, a :class:`GzipFile` object is created, and wrapped in an |
| 51 |
:class:`io.TextIOWrapper` instance with the specified encoding, error |
| 52 |
handling behavior, and line ending(s). |
| 53 |
|
| 54 |
.. versionchanged:: 3.3 |
| 55 |
Added support for *filename* being a file object, support for text mode, |
| 56 |
and the *encoding*, *errors* and *newline* arguments. |
| 57 |
|
| 58 |
.. versionchanged:: 3.4 |
| 59 |
Added support for the ``'x'``, ``'xb'`` and ``'xt'`` modes. |
| 60 |
|
| 61 |
.. versionchanged:: 3.6 |
| 62 |
Accepts a :term:`path-like object`. |
| 63 |
|
| 64 |
.. exception:: BadGzipFile |
| 65 |
|
| 66 |
An exception raised for invalid gzip files. It inherits from :exc:`OSError`. |
| 67 |
:exc:`EOFError` and :exc:`zlib.error` can also be raised for invalid gzip |
| 68 |
files. |
| 69 |
|
| 70 |
.. versionadded:: 3.8 |
| 71 |
|
| 72 |
.. class:: GzipFile(filename=None, mode=None, compresslevel=9, fileobj=None, mtime=None) |
| 73 |
|
| 74 |
Constructor for the :class:`GzipFile` class, which simulates most of the |
| 75 |
methods of a :term:`file object`, with the exception of the :meth:`~io.IOBase.truncate` |
| 76 |
method. At least one of *fileobj* and *filename* must be given a non-trivial |
| 77 |
value. |
| 78 |
|
| 79 |
The new class instance is based on *fileobj*, which can be a regular file, an |
| 80 |
:class:`io.BytesIO` object, or any other object which simulates a file. It |
| 81 |
defaults to ``None``, in which case *filename* is opened to provide a file |
| 82 |
object. |
| 83 |
|
| 84 |
When *fileobj* is not ``None``, the *filename* argument is only used to be |
| 85 |
included in the :program:`gzip` file header, which may include the original |
| 86 |
filename of the uncompressed file. It defaults to the filename of *fileobj*, if |
| 87 |
discernible; otherwise, it defaults to the empty string, and in this case the |
| 88 |
original filename is not included in the header. |
| 89 |
|
| 90 |
The *mode* argument can be any of ``'r'``, ``'rb'``, ``'a'``, ``'ab'``, ``'w'``, |
| 91 |
``'wb'``, ``'x'``, or ``'xb'``, depending on whether the file will be read or |
| 92 |
written. The default is the mode of *fileobj* if discernible; otherwise, the |
| 93 |
default is ``'rb'``. In future Python releases the mode of *fileobj* will |
| 94 |
not be used. It is better to always specify *mode* for writing. |
| 95 |
|
| 96 |
Note that the file is always opened in binary mode. To open a compressed file |
| 97 |
in text mode, use :func:`.open` (or wrap your :class:`GzipFile` with an |
| 98 |
:class:`io.TextIOWrapper`). |
| 99 |
|
| 100 |
The *compresslevel* argument is an integer from ``0`` to ``9`` controlling |
| 101 |
the level of compression; ``1`` is fastest and produces the least |
| 102 |
compression, and ``9`` is slowest and produces the most compression. ``0`` |
| 103 |
is no compression. The default is ``9``. |
| 104 |
|
| 105 |
The optional *mtime* argument is the timestamp requested by gzip. The time |
| 106 |
is in Unix format, i.e., seconds since 00:00:00 UTC, January 1, 1970. |
| 107 |
If *mtime* is omitted or ``None``, the current time is used. Use *mtime* = 0 |
| 108 |
to generate a compressed stream that does not depend on creation time. |
| 109 |
|
| 110 |
See below for the :attr:`mtime` attribute that is set when decompressing. |
| 111 |
|
| 112 |
Calling a :class:`GzipFile` object's :meth:`!close` method does not close |
| 113 |
*fileobj*, since you might wish to append more material after the compressed |
| 114 |
data. This also allows you to pass an :class:`io.BytesIO` object opened for |
| 115 |
writing as *fileobj*, and retrieve the resulting memory buffer using the |
| 116 |
:class:`io.BytesIO` object's :meth:`~io.BytesIO.getvalue` method. |
| 117 |
|
| 118 |
:class:`GzipFile` supports the :class:`io.BufferedIOBase` interface, |
| 119 |
including iteration and the :keyword:`with` statement. Only the |
| 120 |
:meth:`~io.IOBase.truncate` method isn't implemented. |
| 121 |
|
| 122 |
:class:`GzipFile` also provides the following method and attribute: |
| 123 |
|
| 124 |
.. method:: peek(n) |
| 125 |
|
| 126 |
Read *n* uncompressed bytes without advancing the file position. |
| 127 |
The number of bytes returned may be more or less than requested. |
| 128 |
|
| 129 |
.. note:: While calling :meth:`peek` does not change the file position of |
| 130 |
the :class:`GzipFile`, it may change the position of the underlying |
| 131 |
file object (e.g. if the :class:`GzipFile` was constructed with the |
| 132 |
*fileobj* parameter). |
| 133 |
|
| 134 |
.. versionadded:: 3.2 |
| 135 |
|
| 136 |
.. attribute:: mode |
| 137 |
|
| 138 |
``'rb'`` for reading and ``'wb'`` for writing. |
| 139 |
|
| 140 |
.. versionchanged:: 3.13 |
| 141 |
In previous versions it was an integer ``1`` or ``2``. |
| 142 |
|
| 143 |
.. attribute:: mtime |
| 144 |
|
| 145 |
When decompressing, this attribute is set to the last timestamp in the most |
| 146 |
recently read header. It is an integer, holding the number of seconds |
| 147 |
since the Unix epoch (00:00:00 UTC, January 1, 1970). |
| 148 |
The initial value before reading any headers is ``None``. |
| 149 |
|
| 150 |
.. attribute:: name |
| 151 |
|
| 152 |
The path to the gzip file on disk, as a :class:`str` or :class:`bytes`. |
| 153 |
Equivalent to the output of :func:`os.fspath` on the original input path, |
| 154 |
with no other normalization, resolution or expansion. |
| 155 |
|
| 156 |
.. versionchanged:: 3.1 |
| 157 |
Support for the :keyword:`with` statement was added, along with the |
| 158 |
*mtime* constructor argument and :attr:`mtime` attribute. |
| 159 |
|
| 160 |
.. versionchanged:: 3.2 |
| 161 |
Support for zero-padded and unseekable files was added. |
| 162 |
|
| 163 |
.. versionchanged:: 3.3 |
| 164 |
The :meth:`io.BufferedIOBase.read1` method is now implemented. |
| 165 |
|
| 166 |
.. versionchanged:: 3.4 |
| 167 |
Added support for the ``'x'`` and ``'xb'`` modes. |
| 168 |
|
| 169 |
.. versionchanged:: 3.5 |
| 170 |
Added support for writing arbitrary |
| 171 |
:term:`bytes-like objects <bytes-like object>`. |
| 172 |
The :meth:`~io.BufferedIOBase.read` method now accepts an argument of |
| 173 |
``None``. |
| 174 |
|
| 175 |
.. versionchanged:: 3.6 |
| 176 |
Accepts a :term:`path-like object`. |
| 177 |
|
| 178 |
.. deprecated:: 3.9 |
| 179 |
Opening :class:`GzipFile` for writing without specifying the *mode* |
| 180 |
argument is deprecated. |
| 181 |
|
| 182 |
.. versionchanged:: 3.12 |
| 183 |
Remove the ``filename`` attribute, use the :attr:`~GzipFile.name` |
| 184 |
attribute instead. |
| 185 |
|
| 186 |
|
| 187 |
.. function:: compress(data, compresslevel=9, *, mtime=0) |
| 188 |
|
| 189 |
Compress the *data*, returning a :class:`bytes` object containing |
| 190 |
the compressed data. *compresslevel* and *mtime* have the same meaning as in |
| 191 |
the :class:`GzipFile` constructor above, |
| 192 |
but *mtime* defaults to 0 for reproducible output. |
| 193 |
|
| 194 |
.. versionadded:: 3.2 |
| 195 |
.. versionchanged:: 3.8 |
| 196 |
Added the *mtime* parameter for reproducible output. |
| 197 |
.. versionchanged:: 3.11 |
| 198 |
Speed is improved by compressing all data at once instead of in a |
| 199 |
streamed fashion. Calls with *mtime* set to ``0`` are delegated to |
| 200 |
:func:`zlib.compress` for better speed. In this situation the |
| 201 |
output may contain a gzip header "OS" byte value other than 255 |
| 202 |
"unknown" as supplied by the underlying zlib implementation. |
| 203 |
|
| 204 |
.. versionchanged:: 3.13 |
| 205 |
The gzip header OS byte is guaranteed to be set to 255 when this function |
| 206 |
is used as was the case in 3.10 and earlier. |
| 207 |
.. versionchanged:: 3.14 |
| 208 |
The *mtime* parameter now defaults to 0 for reproducible output. |
| 209 |
For the previous behaviour of using the current time, |
| 210 |
pass ``None`` to *mtime*. |
| 211 |
|
| 212 |
.. function:: decompress(data) |
| 213 |
|
| 214 |
Decompress the *data*, returning a :class:`bytes` object containing the |
| 215 |
uncompressed data. This function is capable of decompressing multi-member |
| 216 |
gzip data (multiple gzip blocks concatenated together). When the data is |
| 217 |
certain to contain only one member the :func:`zlib.decompress` function with |
| 218 |
*wbits* set to 31 is faster. |
| 219 |
|
| 220 |
.. versionadded:: 3.2 |
| 221 |
.. versionchanged:: 3.11 |
| 222 |
Speed is improved by decompressing members at once in memory instead of in |
| 223 |
a streamed fashion. |
| 224 |
|
| 225 |
.. _gzip-usage-examples: |
| 226 |
|
| 227 |
Examples of usage |
| 228 |
----------------- |
| 229 |
|
| 230 |
Example of how to read a compressed file:: |
| 231 |
|
| 232 |
import gzip |
| 233 |
with gzip.open('/home/joe/file.txt.gz', 'rb') as f: |
| 234 |
file_content = f.read() |
| 235 |
|
| 236 |
Example of how to create a compressed GZIP file:: |
| 237 |
|
| 238 |
import gzip |
| 239 |
content = b"Lots of content here" |
| 240 |
with gzip.open('/home/joe/file.txt.gz', 'wb') as f: |
| 241 |
f.write(content) |
| 242 |
|
| 243 |
Example of how to GZIP compress an existing file:: |
| 244 |
|
| 245 |
import gzip |
| 246 |
import shutil |
| 247 |
with open('/home/joe/file.txt', 'rb') as f_in: |
| 248 |
with gzip.open('/home/joe/file.txt.gz', 'wb') as f_out: |
| 249 |
shutil.copyfileobj(f_in, f_out) |
| 250 |
|
| 251 |
Example of how to GZIP compress a binary string:: |
| 252 |
|
| 253 |
import gzip |
| 254 |
s_in = b"Lots of content here" |
| 255 |
s_out = gzip.compress(s_in) |
| 256 |
|
| 257 |
.. seealso:: |
| 258 |
|
| 259 |
Module :mod:`zlib` |
| 260 |
The basic data compression module needed to support the :program:`gzip` file |
| 261 |
format. |
| 262 |
|
| 263 |
In case gzip (de)compression is a bottleneck, the `python-isal`_ |
| 264 |
package speeds up (de)compression with a mostly compatible API. |
| 265 |
|
| 266 |
.. _python-isal: https://github.com/pycompression/python-isal |
| 267 |
|
| 268 |
.. program:: gzip |
| 269 |
|
| 270 |
.. _gzip-cli: |
| 271 |
|
| 272 |
Command-line interface |
| 273 |
---------------------- |
| 274 |
|
| 275 |
The :mod:`!gzip` module provides a simple command line interface to compress or |
| 276 |
decompress files. |
| 277 |
|
| 278 |
Once executed the :mod:`!gzip` module keeps the input file(s). |
| 279 |
|
| 280 |
.. versionchanged:: 3.8 |
| 281 |
|
| 282 |
Add a new command line interface with a usage. |
| 283 |
By default, when you will execute the CLI, the default compression level is 6. |
| 284 |
|
| 285 |
Command-line options |
| 286 |
^^^^^^^^^^^^^^^^^^^^ |
| 287 |
|
| 288 |
.. option:: file |
| 289 |
|
| 290 |
If *file* is not specified, read from :data:`sys.stdin`. |
| 291 |
|
| 292 |
.. option:: --fast |
| 293 |
|
| 294 |
Indicates the fastest compression method (less compression). |
| 295 |
|
| 296 |
.. option:: --best |
| 297 |
|
| 298 |
Indicates the slowest compression method (best compression). |
| 299 |
|
| 300 |
.. option:: -d, --decompress |
| 301 |
|
| 302 |
Decompress the given file. |
| 303 |
|
| 304 |
.. option:: -h, --help |
| 305 |
|
| 306 |
Show the help message. |