Debian Codegraph @

Doc/library/bz2.rst

raw
1 :mod:`!bz2` --- Support for :program:`bzip2` compression
2 ========================================================
3
4 .. module:: bz2
5 :synopsis: Interfaces for bzip2 compression and decompression.
6
7 .. moduleauthor:: Gustavo Niemeyer <[email protected]>
8 .. moduleauthor:: Nadeem Vawda <[email protected]>
9 .. sectionauthor:: Gustavo Niemeyer <[email protected]>
10 .. sectionauthor:: Nadeem Vawda <[email protected]>
11
12 **Source code:** :source:`Lib/bz2.py`
13
14 --------------
15
16 This module provides a comprehensive interface for compressing and
17 decompressing data using the bzip2 compression algorithm.
18
19 The :mod:`!bz2` module contains:
20
21 * The :func:`.open` function and :class:`BZ2File` class for reading and
22 writing compressed files.
23 * The :class:`BZ2Compressor` and :class:`BZ2Decompressor` classes for
24 incremental (de)compression.
25 * The :func:`compress` and :func:`decompress` functions for one-shot
26 (de)compression.
27
28 .. include:: ../includes/optional-module.rst
29
30
31 (De)compression of files
32 ------------------------
33
34 .. function:: open(filename, mode='rb', compresslevel=9, encoding=None, errors=None, newline=None)
35
36 Open a bzip2-compressed file in binary or text mode, returning a :term:`file
37 object`.
38
39 As with the constructor for :class:`BZ2File`, the *filename* argument can be
40 an actual filename (a :class:`str` or :class:`bytes` object), or an existing
41 file object to read from or write to.
42
43 The *mode* argument can be any of ``'r'``, ``'rb'``, ``'w'``, ``'wb'``,
44 ``'x'``, ``'xb'``, ``'a'`` or ``'ab'`` for binary mode, or ``'rt'``,
45 ``'wt'``, ``'xt'``, or ``'at'`` for text mode. The default is ``'rb'``.
46
47 The *compresslevel* argument is an integer from 1 to 9, as for the
48 :class:`BZ2File` constructor.
49
50 For binary mode, this function is equivalent to the :class:`BZ2File`
51 constructor: ``BZ2File(filename, mode, compresslevel=compresslevel)``. In
52 this case, the *encoding*, *errors* and *newline* arguments must not be
53 provided.
54
55 For text mode, a :class:`BZ2File` object is created, and wrapped in an
56 :class:`io.TextIOWrapper` instance with the specified encoding, error
57 handling behavior, and line ending(s).
58
59 .. versionadded:: 3.3
60
61 .. versionchanged:: 3.4
62 The ``'x'`` (exclusive creation) mode was added.
63
64 .. versionchanged:: 3.6
65 Accepts a :term:`path-like object`.
66
67
68 .. class:: BZ2File(filename, mode='r', *, compresslevel=9)
69
70 Open a bzip2-compressed file in binary mode.
71
72 If *filename* is a :class:`str` or :class:`bytes` object, open the named file
73 directly. Otherwise, *filename* should be a :term:`file object`, which will
74 be used to read or write the compressed data.
75
76 The *mode* argument can be either ``'r'`` for reading (default), ``'w'`` for
77 overwriting, ``'x'`` for exclusive creation, or ``'a'`` for appending. These
78 can equivalently be given as ``'rb'``, ``'wb'``, ``'xb'`` and ``'ab'``
79 respectively.
80
81 If *filename* is a file object (rather than an actual file name), a mode of
82 ``'w'`` does not truncate the file, and is instead equivalent to ``'a'``.
83
84 If *mode* is ``'w'`` or ``'a'``, *compresslevel* can be an integer between
85 ``1`` and ``9`` specifying the level of compression: ``1`` produces the
86 least compression, and ``9`` (default) produces the most compression.
87
88 If *mode* is ``'r'``, the input file may be the concatenation of multiple
89 compressed streams.
90
91 :class:`BZ2File` provides all of the members specified by the
92 :class:`io.BufferedIOBase`, except for :meth:`~io.BufferedIOBase.detach`
93 and :meth:`~io.IOBase.truncate`.
94 Iteration and the :keyword:`with` statement are supported.
95
96 :class:`BZ2File` also provides the following methods and attributes:
97
98 .. method:: peek([n])
99
100 Return buffered data without advancing the file position. At least one
101 byte of data will be returned (unless at EOF). The exact number of bytes
102 returned is unspecified.
103
104 .. note:: While calling :meth:`peek` does not change the file position of
105 the :class:`BZ2File`, it may change the position of the underlying file
106 object (e.g. if the :class:`BZ2File` was constructed by passing a file
107 object for *filename*).
108
109 .. versionadded:: 3.3
110
111 .. method:: fileno()
112
113 Return the file descriptor for the underlying file.
114
115 .. versionadded:: 3.3
116
117 .. method:: readable()
118
119 Return whether the file was opened for reading.
120
121 .. versionadded:: 3.3
122
123 .. method:: seekable()
124
125 Return whether the file supports seeking.
126
127 .. versionadded:: 3.3
128
129 .. method:: writable()
130
131 Return whether the file was opened for writing.
132
133 .. versionadded:: 3.3
134
135 .. method:: read1(size=-1)
136
137 Read up to *size* uncompressed bytes, while trying to avoid
138 making multiple reads from the underlying stream. Reads up to a
139 buffer's worth of data if size is negative.
140
141 Returns ``b''`` if the file is at EOF.
142
143 .. versionadded:: 3.3
144
145 .. method:: readinto(b)
146
147 Read bytes into *b*.
148
149 Returns the number of bytes read (0 for EOF).
150
151 .. versionadded:: 3.3
152
153 .. attribute:: mode
154
155 ``'rb'`` for reading and ``'wb'`` for writing.
156
157 .. versionadded:: 3.13
158
159 .. attribute:: name
160
161 The bzip2 file name. Equivalent to the :attr:`~io.FileIO.name`
162 attribute of the underlying :term:`file object`.
163
164 .. versionadded:: 3.13
165
166
167 .. versionchanged:: 3.1
168 Support for the :keyword:`with` statement was added.
169
170 .. versionchanged:: 3.3
171 Support was added for *filename* being a :term:`file object` instead of an
172 actual filename.
173
174 The ``'a'`` (append) mode was added, along with support for reading
175 multi-stream files.
176
177 .. versionchanged:: 3.4
178 The ``'x'`` (exclusive creation) mode was added.
179
180 .. versionchanged:: 3.5
181 The :meth:`~io.BufferedIOBase.read` method now accepts an argument of
182 ``None``.
183
184 .. versionchanged:: 3.6
185 Accepts a :term:`path-like object`.
186
187 .. versionchanged:: 3.9
188 The *buffering* parameter has been removed. It was ignored and deprecated
189 since Python 3.0. Pass an open file object to control how the file is
190 opened.
191
192 The *compresslevel* parameter became keyword-only.
193
194 .. versionchanged:: 3.10
195 This class is thread unsafe in the face of multiple simultaneous
196 readers or writers, just like its equivalent classes in :mod:`gzip` and
197 :mod:`lzma` have always been.
198
199
200 Incremental (de)compression
201 ---------------------------
202
203 .. class:: BZ2Compressor(compresslevel=9)
204
205 Create a new compressor object. This object may be used to compress data
206 incrementally. For one-shot compression, use the :func:`compress` function
207 instead.
208
209 *compresslevel*, if given, must be an integer between ``1`` and ``9``. The
210 default is ``9``.
211
212 .. method:: compress(data)
213
214 Provide data to the compressor object. Returns a chunk of compressed data
215 if possible, or an empty byte string otherwise.
216
217 When you have finished providing data to the compressor, call the
218 :meth:`flush` method to finish the compression process.
219
220
221 .. method:: flush()
222
223 Finish the compression process. Returns the compressed data left in
224 internal buffers.
225
226 The compressor object may not be used after this method has been called.
227
228
229 .. class:: BZ2Decompressor()
230
231 Create a new decompressor object. This object may be used to decompress data
232 incrementally. For one-shot compression, use the :func:`decompress` function
233 instead.
234
235 .. note::
236 This class does not transparently handle inputs containing multiple
237 compressed streams, unlike :func:`decompress` and :class:`BZ2File`. If
238 you need to decompress a multi-stream input with :class:`BZ2Decompressor`,
239 you must use a new decompressor for each stream.
240
241 .. method:: decompress(data, max_length=-1)
242
243 Decompress *data* (a :term:`bytes-like object`), returning
244 uncompressed data as bytes. Some of *data* may be buffered
245 internally, for use in later calls to :meth:`decompress`. The
246 returned data should be concatenated with the output of any
247 previous calls to :meth:`decompress`.
248
249 If *max_length* is nonnegative, returns at most *max_length*
250 bytes of decompressed data. If this limit is reached and further
251 output can be produced, the :attr:`~.needs_input` attribute will
252 be set to ``False``. In this case, the next call to
253 :meth:`~.decompress` may provide *data* as ``b''`` to obtain
254 more of the output.
255
256 If all of the input data was decompressed and returned (either
257 because this was less than *max_length* bytes, or because
258 *max_length* was negative), the :attr:`~.needs_input` attribute
259 will be set to ``True``.
260
261 Attempting to decompress data after the end of stream is reached
262 raises an :exc:`EOFError`. Any data found after the end of the
263 stream is ignored and saved in the :attr:`~.unused_data` attribute.
264
265 .. versionchanged:: 3.5
266 Added the *max_length* parameter.
267
268 .. attribute:: eof
269
270 ``True`` if the end-of-stream marker has been reached.
271
272 .. versionadded:: 3.3
273
274
275 .. attribute:: unused_data
276
277 Data found after the end of the compressed stream.
278
279 If this attribute is accessed before the end of the stream has been
280 reached, its value will be ``b''``.
281
282 .. attribute:: needs_input
283
284 ``False`` if the :meth:`.decompress` method can provide more
285 decompressed data before requiring new uncompressed input.
286
287 .. versionadded:: 3.5
288
289
290 One-shot (de)compression
291 ------------------------
292
293 .. function:: compress(data, compresslevel=9)
294
295 Compress *data*, a :term:`bytes-like object <bytes-like object>`.
296
297 *compresslevel*, if given, must be an integer between ``1`` and ``9``. The
298 default is ``9``.
299
300 For incremental compression, use a :class:`BZ2Compressor` instead.
301
302
303 .. function:: decompress(data)
304
305 Decompress *data*, a :term:`bytes-like object <bytes-like object>`.
306
307 If *data* is the concatenation of multiple compressed streams, decompress
308 all of the streams.
309
310 For incremental decompression, use a :class:`BZ2Decompressor` instead.
311
312 .. versionchanged:: 3.3
313 Support for multi-stream inputs was added.
314
315 .. _bz2-usage-examples:
316
317 Examples of usage
318 -----------------
319
320 Below are some examples of typical usage of the :mod:`!bz2` module.
321
322 Using :func:`compress` and :func:`decompress` to demonstrate round-trip compression:
323
324 >>> import bz2
325 >>> data = b"""\
326 ... Donec rhoncus quis sapien sit amet molestie. Fusce scelerisque vel augue
327 ... nec ullamcorper. Nam rutrum pretium placerat. Aliquam vel tristique lorem,
328 ... sit amet cursus ante. In interdum laoreet mi, sit amet ultrices purus
329 ... pulvinar a. Nam gravida euismod magna, non varius justo tincidunt feugiat.
330 ... Aliquam pharetra lacus non risus vehicula rutrum. Maecenas aliquam leo
331 ... felis. Pellentesque semper nunc sit amet nibh ullamcorper, ac elementum
332 ... dolor luctus. Curabitur lacinia mi ornare consectetur vestibulum."""
333 >>> c = bz2.compress(data)
334 >>> len(data) / len(c) # Data compression ratio
335 1.513595166163142
336 >>> d = bz2.decompress(c)
337 >>> data == d # Check equality to original object after round-trip
338 True
339
340 Using :class:`BZ2Compressor` for incremental compression:
341
342 >>> import bz2
343 >>> def gen_data(chunks=10, chunksize=1000):
344 ... """Yield incremental blocks of chunksize bytes."""
345 ... for _ in range(chunks):
346 ... yield b"z" * chunksize
347 ...
348 >>> comp = bz2.BZ2Compressor()
349 >>> out = b""
350 >>> for chunk in gen_data():
351 ... # Provide data to the compressor object
352 ... out = out + comp.compress(chunk)
353 ...
354 >>> # Finish the compression process. Call this once you have
355 >>> # finished providing data to the compressor.
356 >>> out = out + comp.flush()
357
358 The example above uses a very "nonrandom" stream of data
359 (a stream of ``b"z"`` chunks). Random data tends to compress poorly,
360 while ordered, repetitive data usually yields a high compression ratio.
361
362 Writing and reading a bzip2-compressed file in binary mode:
363
364 >>> import bz2
365 >>> data = b"""\
366 ... Donec rhoncus quis sapien sit amet molestie. Fusce scelerisque vel augue
367 ... nec ullamcorper. Nam rutrum pretium placerat. Aliquam vel tristique lorem,
368 ... sit amet cursus ante. In interdum laoreet mi, sit amet ultrices purus
369 ... pulvinar a. Nam gravida euismod magna, non varius justo tincidunt feugiat.
370 ... Aliquam pharetra lacus non risus vehicula rutrum. Maecenas aliquam leo
371 ... felis. Pellentesque semper nunc sit amet nibh ullamcorper, ac elementum
372 ... dolor luctus. Curabitur lacinia mi ornare consectetur vestibulum."""
373 >>> with bz2.open("myfile.bz2", "wb") as f:
374 ... # Write compressed data to file
375 ... unused = f.write(data)
376 ...
377 >>> with bz2.open("myfile.bz2", "rb") as f:
378 ... # Decompress data from file
379 ... content = f.read()
380 ...
381 >>> content == data # Check equality to original object after round-trip
382 True
383
384 .. testcleanup::
385
386 import os
387 os.remove("myfile.bz2")