Debian Codegraph @

upstream/debian-unstable/man1/bzip2.1

raw
1 .TH bzip2 1
2 .SH NAME
3 bzip2, bunzip2 \- a block-sorting file compressor, v1.0.8
4 .br
5 bzcat \- decompresses files to stdout
6 .br
7 bzip2recover \- recovers data from damaged bzip2 files
8
9 .SH SYNOPSIS
10 .ll +8
11 .B bzip2
12 .RB [ " \-cdfkqstvzVL123456789 " ]
13 [
14 .I "filenames \&..."
15 ]
16 .br
17 .B bzip2
18 .RB [ " \-h|\-\-help " ]
19 .ll -8
20 .br
21 .B bunzip2
22 .RB [ " \-fkvsVL " ]
23 [
24 .I "filenames \&..."
25 ]
26 .br
27 .B bunzip2
28 .RB [ " \-h|\-\-help " ]
29 .br
30 .B bzcat
31 .RB [ " \-s " ]
32 [
33 .I "filenames \&..."
34 ]
35 .br
36 .B bzcat
37 .RB [ " \-h|\-\-help " ]
38 .br
39 .B bzip2recover
40 .I "filename"
41
42 .SH DESCRIPTION
43 .I bzip2
44 compresses files using the Burrows-Wheeler block sorting
45 text compression algorithm, and Huffman coding. Compression is
46 generally considerably better than that achieved by more conventional
47 LZ77/LZ78-based compressors, and approaches the performance of the PPM
48 family of statistical compressors.
49
50 The command-line options are deliberately very similar to
51 those of
52 .I GNU gzip,
53 but they are not identical.
54
55 .I bzip2
56 expects a list of file names to accompany the
57 command-line flags. Each file is replaced by a compressed version of
58 itself, with the name "original_name.bz2".
59 Each compressed file
60 has the same modification date, permissions, and, when possible,
61 ownership as the corresponding original, so that these properties can
62 be correctly restored at decompression time. File name handling is
63 naive in the sense that there is no mechanism for preserving original
64 file names, permissions, ownerships or dates in filesystems which lack
65 these concepts, or have serious file name length restrictions, such as
66 MS-DOS.
67
68 .I bzip2
69 and
70 .I bunzip2
71 will by default not overwrite existing
72 files. If you want this to happen, specify the \-f flag.
73
74 If no file names are specified,
75 .I bzip2
76 compresses from standard
77 input to standard output. In this case,
78 .I bzip2
79 will decline to
80 write compressed output to a terminal, as this would be entirely
81 incomprehensible and therefore pointless.
82
83 .I bunzip2
84 (or
85 .I bzip2 \-d)
86 decompresses all
87 specified files. Files which were not created by
88 .I bzip2
89 will be detected and ignored, and a warning issued.
90 .I bzip2
91 attempts to guess the filename for the decompressed file
92 from that of the compressed file as follows:
93
94 filename.bz2 becomes filename
95 filename.bz becomes filename
96 filename.tbz2 becomes filename.tar
97 filename.tbz becomes filename.tar
98 anyothername becomes anyothername.out
99
100 If the file does not end in one of the recognised endings,
101 .I .bz2,
102 .I .bz,
103 .I .tbz2
104 or
105 .I .tbz,
106 .I bzip2
107 complains that it cannot
108 guess the name of the original file, and uses the original name
109 with
110 .I .out
111 appended.
112
113 As with compression, supplying no
114 filenames causes decompression from
115 standard input to standard output.
116
117 .I bunzip2
118 will correctly decompress a file which is the
119 concatenation of two or more compressed files. The result is the
120 concatenation of the corresponding uncompressed files. Integrity
121 testing (\-t)
122 of concatenated
123 compressed files is also supported.
124
125 You can also compress or decompress files to the standard output by
126 giving the \-c flag. Multiple files may be compressed and
127 decompressed like this. The resulting outputs are fed sequentially to
128 stdout. Compression of multiple files
129 in this manner generates a stream
130 containing multiple compressed file representations. Such a stream
131 can be decompressed correctly only by
132 .I bzip2
133 version 0.9.0 or
134 later. Earlier versions of
135 .I bzip2
136 will stop after decompressing
137 the first file in the stream.
138
139 .I bzcat
140 (or
141 .I bzip2 -dc)
142 decompresses all specified files to
143 the standard output.
144
145 .I bzip2
146 will read arguments from the environment variables
147 .I BZIP2
148 and
149 .I BZIP,
150 in that order, and will process them
151 before any arguments read from the command line. This gives a
152 convenient way to supply default arguments.
153
154 Compression is always performed, even if the compressed
155 file is slightly
156 larger than the original. Files of less than about one hundred bytes
157 tend to get larger, since the compression mechanism has a constant
158 overhead in the region of 50 bytes. Random data (including the output
159 of most file compressors) is coded at about 8.05 bits per byte, giving
160 an expansion of around 0.5%.
161
162 As a self-check for your protection,
163 .I bzip2
164 uses 32-bit CRCs to
165 make sure that the decompressed version of a file is identical to the
166 original. This guards against corruption of the compressed data, and
167 against undetected bugs in
168 .I bzip2
169 (hopefully very unlikely). The
170 chances of data corruption going undetected is microscopic, about one
171 chance in four billion for each file processed. Be aware, though, that
172 the check occurs upon decompression, so it can only tell you that
173 something is wrong. It can't help you
174 recover the original uncompressed
175 data. You can use
176 .I bzip2recover
177 to try to recover data from
178 damaged files.
179
180 Return values: 0 for a normal exit, 1 for environmental problems (file
181 not found, invalid flags, I/O errors, &c), 2 to indicate a corrupt
182 compressed file, 3 for an internal consistency error (eg, bug) which
183 caused
184 .I bzip2
185 to panic.
186
187 .SH OPTIONS
188 .TP
189 .B \-c --stdout
190 Compress or decompress to standard output.
191 .TP
192 .B \-d --decompress
193 Force decompression.
194 .I bzip2,
195 .I bunzip2
196 and
197 .I bzcat
198 are
199 really the same program, and the decision about what actions to take is
200 done on the basis of which name is used. This flag overrides that
201 mechanism, and forces
202 .I bzip2
203 to decompress.
204 .TP
205 .B \-z --compress
206 The complement to \-d: forces compression, regardless of the
207 invocation name.
208 .TP
209 .B \-t --test
210 Check integrity of the specified file(s), but don't decompress them.
211 This really performs a trial decompression and throws away the result.
212 .TP
213 .B \-f --force
214 Force overwrite of output files. Normally,
215 .I bzip2
216 will not overwrite
217 existing output files. Also forces
218 .I bzip2
219 to break hard links
220 to files, which it otherwise wouldn't do.
221
222 bzip2 normally declines to decompress files which don't have the
223 correct magic header bytes. If forced (-f), however, it will pass
224 such files through unmodified. This is how GNU gzip behaves.
225 .TP
226 .B \-k --keep
227 Keep (don't delete) input files during compression
228 or decompression.
229 .TP
230 .B \-s --small
231 Reduce memory usage, for compression, decompression and testing. Files
232 are decompressed and tested using a modified algorithm which only
233 requires 2.5 bytes per block byte. This means any file can be
234 decompressed in 2300\ k of memory, albeit at about half the normal speed.
235
236 During compression, \-s selects a block size of 200\ k, which limits
237 memory use to around the same figure, at the expense of your compression
238 ratio. In short, if your machine is low on memory (8 megabytes or
239 less), use \-s for everything. See MEMORY MANAGEMENT below.
240 .TP
241 .B \-q --quiet
242 Suppress non-essential warning messages. Messages pertaining to
243 I/O errors and other critical events will not be suppressed.
244 .TP
245 .B \-v --verbose
246 Verbose mode -- show the compression ratio for each file processed.
247 Further \-v's increase the verbosity level, spewing out lots of
248 information which is primarily of interest for diagnostic purposes.
249 .TP
250 .B \-h \-\-help
251 Print a help message and exit.
252 .TP
253 .B \-L --license -V --version
254 Display the software version, license terms and conditions.
255 .TP
256 .B \-1 (or \-\-fast) to \-9 (or \-\-best)
257 Set the block size to 100 k, 200 k ... 900 k when compressing. Has no
258 effect when decompressing. See MEMORY MANAGEMENT below.
259 The \-\-fast and \-\-best aliases are primarily for GNU gzip
260 compatibility. In particular, \-\-fast doesn't make things
261 significantly faster.
262 And \-\-best merely selects the default behaviour.
263 .TP
264 .B \--
265 Treats all subsequent arguments as file names, even if they start
266 with a dash. This is so you can handle files with names beginning
267 with a dash, for example: bzip2 \-- \-myfilename.
268 .TP
269 .B \--repetitive-fast --repetitive-best
270 These flags are redundant in versions 0.9.5 and above. They provided
271 some coarse control over the behaviour of the sorting algorithm in
272 earlier versions, which was sometimes useful. 0.9.5 and above have an
273 improved algorithm which renders these flags irrelevant.
274
275 .SH MEMORY MANAGEMENT
276 .I bzip2
277 compresses large files in blocks. The block size affects
278 both the compression ratio achieved, and the amount of memory needed for
279 compression and decompression. The flags \-1 through \-9
280 specify the block size to be 100,000 bytes through 900,000 bytes (the
281 default) respectively. At decompression time, the block size used for
282 compression is read from the header of the compressed file, and
283 .I bunzip2
284 then allocates itself just enough memory to decompress
285 the file. Since block sizes are stored in compressed files, it follows
286 that the flags \-1 to \-9 are irrelevant to and so ignored
287 during decompression.
288
289 Compression and decompression requirements,
290 in bytes, can be estimated as:
291
292 Compression: 400\ k + ( 8 x block size )
293
294 Decompression: 100\ k + ( 4 x block size ), or
295 100\ k + ( 2.5 x block size )
296
297 Larger block sizes give rapidly diminishing marginal returns. Most of
298 the compression comes from the first two or three hundred k of block
299 size, a fact worth bearing in mind when using
300 .I bzip2
301 on small machines.
302 It is also important to appreciate that the decompression memory
303 requirement is set at compression time by the choice of block size.
304
305 For files compressed with the default 900\ k block size,
306 .I bunzip2
307 will require about 3700 kbytes to decompress. To support decompression
308 of any file on a 4 megabyte machine,
309 .I bunzip2
310 has an option to
311 decompress using approximately half this amount of memory, about 2300
312 kbytes. Decompression speed is also halved, so you should use this
313 option only where necessary. The relevant flag is -s.
314
315 In general, try and use the largest block size memory constraints allow,
316 since that maximises the compression achieved. Compression and
317 decompression speed are virtually unaffected by block size.
318
319 Another significant point applies to files which fit in a single block
320 -- that means most files you'd encounter using a large block size. The
321 amount of real memory touched is proportional to the size of the file,
322 since the file is smaller than a block. For example, compressing a file
323 20,000 bytes long with the flag -9 will cause the compressor to
324 allocate around 7600\ k of memory, but only touch 400\ k + 20000 * 8 = 560
325 kbytes of it. Similarly, the decompressor will allocate 3700\ k but only
326 touch 100\ k + 20000 * 4 = 180 kbytes.
327
328 Here is a table which summarises the maximum memory usage for different
329 block sizes. Also recorded is the total compressed size for 14 files of
330 the Calgary Text Compression Corpus totalling 3,141,622 bytes. This
331 column gives some feel for how compression varies with block size.
332 These figures tend to understate the advantage of larger block sizes for
333 larger files, since the Corpus is dominated by smaller files.
334
335 Compress Decompress Decompress Corpus
336 Flag usage usage -s usage Size
337
338 -1 1200k 500k 350k 914704
339 -2 2000k 900k 600k 877703
340 -3 2800k 1300k 850k 860338
341 -4 3600k 1700k 1100k 846899
342 -5 4400k 2100k 1350k 845160
343 -6 5200k 2500k 1600k 838626
344 -7 6100k 2900k 1850k 834096
345 -8 6800k 3300k 2100k 828642
346 -9 7600k 3700k 2350k 828642
347
348 .SH RECOVERING DATA FROM DAMAGED FILES
349 .I bzip2
350 compresses files in blocks, usually 900\ kbytes long. Each
351 block is handled independently. If a media or transmission error causes
352 a multi-block .bz2
353 file to become damaged, it may be possible to
354 recover data from the undamaged blocks in the file.
355
356 The compressed representation of each block is delimited by a 48-bit
357 pattern, which makes it possible to find the block boundaries with
358 reasonable certainty. Each block also carries its own 32-bit CRC, so
359 damaged blocks can be distinguished from undamaged ones.
360
361 .I bzip2recover
362 is a simple program whose purpose is to search for
363 blocks in .bz2 files, and write each block out into its own .bz2
364 file. You can then use
365 .I bzip2
366 \-t
367 to test the
368 integrity of the resulting files, and decompress those which are
369 undamaged.
370
371 .I bzip2recover
372 takes a single argument, the name of the damaged file,
373 and writes a number of files "rec00001file.bz2",
374 "rec00002file.bz2", etc., containing the extracted blocks.
375 The output filenames are designed so that the use of
376 wildcards in subsequent processing -- for example,
377 "bzip2 -dc rec*file.bz2 > recovered_data" -- processes the files in
378 the correct order.
379
380 .I bzip2recover
381 should be of most use dealing with large .bz2
382 files, as these will contain many blocks. It is clearly
383 futile to use it on damaged single-block files, since a
384 damaged block cannot be recovered. If you wish to minimise
385 any potential data loss through media or transmission errors,
386 you might consider compressing with a smaller
387 block size.
388
389 .SH PERFORMANCE NOTES
390 The sorting phase of compression gathers together similar strings in the
391 file. Because of this, files containing very long runs of repeated
392 symbols, like "aabaabaabaab ...\&" (repeated several hundred times) may
393 compress more slowly than normal. Versions 0.9.5 and above fare much
394 better than previous versions in this respect. The ratio between
395 worst-case and average-case compression time is in the region of 10:1.
396 For previous versions, this figure was more like 100:1. You can use the
397 \-vvvv option to monitor progress in great detail, if you want.
398
399 Decompression speed is unaffected by these phenomena.
400
401 .I bzip2
402 usually allocates several megabytes of memory to operate
403 in, and then charges all over it in a fairly random fashion. This means
404 that performance, both for compressing and decompressing, is largely
405 determined by the speed at which your machine can service cache misses.
406 Because of this, small changes to the code to reduce the miss rate have
407 been observed to give disproportionately large performance improvements.
408 I imagine
409 .I bzip2
410 will perform best on machines with very large caches.
411
412 .SH CAVEATS
413 I/O error messages are not as helpful as they could be.
414 .I bzip2
415 tries hard to detect I/O errors and exit cleanly, but the details of
416 what the problem is sometimes seem rather misleading.
417
418 This manual page pertains to version 1.0.8 of
419 .I bzip2.
420 Compressed data created by this version is entirely forwards and
421 backwards compatible with the previous public releases, versions
422 0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1, 1.0.2 and above, but with the following
423 exception: 0.9.0 and above can correctly decompress multiple
424 concatenated compressed files. 0.1pl2 cannot do this; it will stop
425 after decompressing just the first file in the stream.
426
427 .I bzip2recover
428 versions prior to 1.0.2 used 32-bit integers to represent
429 bit positions in compressed files, so they could not handle compressed
430 files more than 512 megabytes long. Versions 1.0.2 and above use
431 64-bit ints on some platforms which support them (GNU supported
432 targets, and Windows). To establish whether or not bzip2recover was
433 built with such a limitation, run it without arguments. In any event
434 you can build yourself an unlimited version if you can recompile it
435 with MaybeUInt64 set to be an unsigned 64-bit integer.
436
437
438
439 .SH AUTHOR
440 Julian Seward, [email protected].
441
442 https://sourceware.org/bzip2/
443
444 The ideas embodied in
445 .I bzip2
446 are due to (at least) the following
447 people: Michael Burrows and David Wheeler (for the block sorting
448 transformation), David Wheeler (again, for the Huffman coder), Peter
449 Fenwick (for the structured coding model in the original
450 .I bzip,
451 and many refinements), and Alistair Moffat, Radford Neal and Ian Witten
452 (for the arithmetic coder in the original
453 .I bzip).
454 I am much
455 indebted for their help, support and advice. See the manual in the
456 source distribution for pointers to sources of documentation. Christian
457 von Roques encouraged me to look for faster sorting algorithms, so as to
458 speed up compression. Bela Lubkin encouraged me to improve the
459 worst-case compression performance.
460 Donna Robinson XMLised the documentation.
461 The bz* scripts are derived from those of GNU gzip.
462 Many people sent patches, helped
463 with portability problems, lent machines, gave advice and were generally
464 helpful.