Debian Codegraph @

upstream/opensuse-tumbleweed/man3/IO::Compress::FAQ.3pm

raw
1 .\" -*- mode: troff; coding: utf-8 -*-
2 .\" Automatically generated by Pod::Man v6.0.2 (Pod::Simple 3.45)
3 .\"
4 .\" Standard preamble:
5 .\" ========================================================================
6 .de Sp \" Vertical space (when we can't use .PP)
7 .if t .sp .5v
8 .if n .sp
9 ..
10 .de Vb \" Begin verbatim text
11 .ft CW
12 .nf
13 .ne \\$1
14 ..
15 .de Ve \" End verbatim text
16 .ft R
17 .fi
18 ..
19 .\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>.
20 .ie n \{\
21 . ds C` ""
22 . ds C' ""
23 'br\}
24 .el\{\
25 . ds C`
26 . ds C'
27 'br\}
28 .\"
29 .\" Escape single quotes in literal strings from groff's Unicode transform.
30 .ie \n(.g .ds Aq \(aq
31 .el .ds Aq '
32 .\"
33 .\" If the F register is >0, we'll generate index entries on stderr for
34 .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
35 .\" entries marked with X<> in POD. Of course, you'll have to process the
36 .\" output yourself in some meaningful fashion.
37 .\"
38 .\" Avoid warning from groff about undefined register 'F'.
39 .de IX
40 ..
41 .nr rF 0
42 .if \n(.g .if rF .nr rF 1
43 .if (\n(rF:(\n(.g==0)) \{\
44 . if \nF \{\
45 . de IX
46 . tm Index:\\$1\t\\n%\t"\\$2"
47 ..
48 . if !\nF==2 \{\
49 . nr % 0
50 . nr F 2
51 . \}
52 . \}
53 .\}
54 .rr rF
55 .\"
56 .\" Required to disable full justification in groff 1.23.0.
57 .if n .ds AD l
58 .\" ========================================================================
59 .\"
60 .IX Title "IO::Compress::FAQ 3pm"
61 .TH IO::Compress::FAQ 3pm 2026-03-09 "perl v5.42.1" "Perl Programmers Reference Guide"
62 .\" For nroff, turn off justification. Always turn off hyphenation; it makes
63 .\" way too many mistakes in technical documents.
64 .if n .ad l
65 .nh
66 .SH NAME
67 IO::Compress::FAQ \-\- Frequently Asked Questions about IO::Compress
68 .SH DESCRIPTION
69 .IX Header "DESCRIPTION"
70 Common questions answered.
71 .SH GENERAL
72 .IX Header "GENERAL"
73 .SS "Compatibility with Unix compress/uncompress."
74 .IX Subsection "Compatibility with Unix compress/uncompress."
75 Although \f(CW\*(C`Compress::Zlib\*(C'\fR has a pair of functions called \f(CW\*(C`compress\*(C'\fR and
76 \&\f(CW\*(C`uncompress\*(C'\fR, they are \fInot\fR related to the Unix programs of the same
77 name. The \f(CW\*(C`Compress::Zlib\*(C'\fR module is not compatible with Unix
78 \&\f(CW\*(C`compress\*(C'\fR.
79 .PP
80 If you have the \f(CW\*(C`uncompress\*(C'\fR program available, you can use this to read
81 compressed files
82 .PP
83 .Vb 4
84 \& open F, "uncompress \-c $filename |";
85 \& while (<F>)
86 \& {
87 \& ...
88 .Ve
89 .PP
90 Alternatively, if you have the \f(CW\*(C`gunzip\*(C'\fR program available, you can use
91 this to read compressed files
92 .PP
93 .Vb 4
94 \& open F, "gunzip \-c $filename |";
95 \& while (<F>)
96 \& {
97 \& ...
98 .Ve
99 .PP
100 and this to write compress files, if you have the \f(CW\*(C`compress\*(C'\fR program
101 available
102 .PP
103 .Vb 4
104 \& open F, "| compress \-c $filename ";
105 \& print F "data";
106 \& ...
107 \& close F ;
108 .Ve
109 .SS "Accessing .tar.Z files"
110 .IX Subsection "Accessing .tar.Z files"
111 The \f(CW\*(C`Archive::Tar\*(C'\fR module can optionally use \f(CW\*(C`Compress::Zlib\*(C'\fR (via the
112 \&\f(CW\*(C`IO::Zlib\*(C'\fR module) to access tar files that have been compressed with
113 \&\f(CW\*(C`gzip\*(C'\fR. Unfortunately tar files compressed with the Unix \f(CW\*(C`compress\*(C'\fR
114 utility cannot be read by \f(CW\*(C`Compress::Zlib\*(C'\fR and so cannot be directly
115 accessed by \f(CW\*(C`Archive::Tar\*(C'\fR.
116 .PP
117 If the \f(CW\*(C`uncompress\*(C'\fR or \f(CW\*(C`gunzip\*(C'\fR programs are available, you can use one
118 of these workarounds to read \f(CW\*(C`.tar.Z\*(C'\fR files from \f(CW\*(C`Archive::Tar\*(C'\fR
119 .PP
120 Firstly with \f(CW\*(C`uncompress\*(C'\fR
121 .PP
122 .Vb 3
123 \& use strict;
124 \& use warnings;
125 \& use Archive::Tar;
126 \&
127 \& open F, "uncompress \-c $filename |";
128 \& my $tar = Archive::Tar\->new(*F);
129 \& ...
130 .Ve
131 .PP
132 and this with \f(CW\*(C`gunzip\*(C'\fR
133 .PP
134 .Vb 3
135 \& use strict;
136 \& use warnings;
137 \& use Archive::Tar;
138 \&
139 \& open F, "gunzip \-c $filename |";
140 \& my $tar = Archive::Tar\->new(*F);
141 \& ...
142 .Ve
143 .PP
144 Similarly, if the \f(CW\*(C`compress\*(C'\fR program is available, you can use this to
145 write a \f(CW\*(C`.tar.Z\*(C'\fR file
146 .PP
147 .Vb 4
148 \& use strict;
149 \& use warnings;
150 \& use Archive::Tar;
151 \& use IO::File;
152 \&
153 \& my $fh = IO::File\->new( "| compress \-c >$filename" );
154 \& my $tar = Archive::Tar\->new();
155 \& ...
156 \& $tar\->write($fh);
157 \& $fh\->close ;
158 .Ve
159 .SS "How do I recompress using a different compression?"
160 .IX Subsection "How do I recompress using a different compression?"
161 This is easier that you might expect if you realise that all the
162 \&\f(CW\*(C`IO::Compress::*\*(C'\fR objects are derived from \f(CW\*(C`IO::File\*(C'\fR and that all the
163 \&\f(CW\*(C`IO::Uncompress::*\*(C'\fR modules can read from an \f(CW\*(C`IO::File\*(C'\fR filehandle.
164 .PP
165 So, for example, say you have a file compressed with gzip that you want to
166 recompress with bzip2. Here is all that is needed to carry out the
167 recompression.
168 .PP
169 .Vb 2
170 \& use IO::Uncompress::Gunzip \*(Aq:all\*(Aq;
171 \& use IO::Compress::Bzip2 \*(Aq:all\*(Aq;
172 \&
173 \& my $gzipFile = "somefile.gz";
174 \& my $bzipFile = "somefile.bz2";
175 \&
176 \& my $gunzip = IO::Uncompress::Gunzip\->new( $gzipFile )
177 \& or die "Cannot gunzip $gzipFile: $GunzipError\en" ;
178 \&
179 \& bzip2 $gunzip => $bzipFile
180 \& or die "Cannot bzip2 to $bzipFile: $Bzip2Error\en" ;
181 .Ve
182 .PP
183 Note, there is a limitation of this technique. Some compression file
184 formats store extra information along with the compressed data payload. For
185 example, gzip can optionally store the original filename and Zip stores a
186 lot of information about the original file. If the original compressed file
187 contains any of this extra information, it will not be transferred to the
188 new compressed file using the technique above.
189 .SH ZIP
190 .IX Header "ZIP"
191 .SS "What Compression Types do IO::Compress::Zip & IO::Uncompress::Unzip support?"
192 .IX Subsection "What Compression Types do IO::Compress::Zip & IO::Uncompress::Unzip support?"
193 The following compression formats are supported by \f(CW\*(C`IO::Compress::Zip\*(C'\fR and
194 \&\f(CW\*(C`IO::Uncompress::Unzip\*(C'\fR
195 .IP \(bu 5
196 Store (method 0)
197 .Sp
198 No compression at all.
199 .IP \(bu 5
200 Deflate (method 8)
201 .Sp
202 This is the default compression used when creating a zip file with
203 \&\f(CW\*(C`IO::Compress::Zip\*(C'\fR.
204 .IP \(bu 5
205 Bzip2 (method 12)
206 .Sp
207 Only supported if the \f(CW\*(C`IO\-Compress\-Bzip2\*(C'\fR module is installed.
208 .IP \(bu 5
209 Lzma (method 14)
210 .Sp
211 Only supported if the \f(CW\*(C`IO\-Compress\-Lzma\*(C'\fR module is installed.
212 .SS "Can I Read/Write Zip files larger the 4 Gig?"
213 .IX Subsection "Can I Read/Write Zip files larger the 4 Gig?"
214 Yes, both the \f(CW\*(C`IO\-Compress\-Zip\*(C'\fR and \f(CW\*(C`IO\-Uncompress\-Unzip\*(C'\fR modules
215 support the zip feature called \fIZip64\fR. That allows them to read/write
216 files/buffers larger than 4Gig.
217 .PP
218 If you are creating a Zip file using the one\-shot interface, and any of the
219 input files is greater than 4Gig, a zip64 complaint zip file will be
220 created.
221 .PP
222 .Vb 1
223 \& zip "really\-large\-file" => "my.zip";
224 .Ve
225 .PP
226 Similarly with the one\-shot interface, if the input is a buffer larger than
227 4 Gig, a zip64 complaint zip file will be created.
228 .PP
229 .Vb 1
230 \& zip \e$really_large_buffer => "my.zip";
231 .Ve
232 .PP
233 The one\-shot interface allows you to force the creation of a zip64 zip file
234 by including the \f(CW\*(C`Zip64\*(C'\fR option.
235 .PP
236 .Vb 1
237 \& zip $filehandle => "my.zip", Zip64 => 1;
238 .Ve
239 .PP
240 If you want to create a zip64 zip file with the OO interface you must
241 specify the \f(CW\*(C`Zip64\*(C'\fR option.
242 .PP
243 .Vb 1
244 \& my $zip = IO::Compress::Zip\->new( "whatever", Zip64 => 1 );
245 .Ve
246 .PP
247 When uncompressing with \f(CW\*(C`IO\-Uncompress\-Unzip\*(C'\fR, it will automatically
248 detect if the zip file is zip64.
249 .PP
250 If you intend to manipulate the Zip64 zip files created with
251 \&\f(CW\*(C`IO\-Compress\-Zip\*(C'\fR using an external zip/unzip, make sure that it supports
252 Zip64.
253 .PP
254 In particular, if you are using Info\-Zip you need to have zip version 3.x
255 or better to update a Zip64 archive and unzip version 6.x to read a zip64
256 archive.
257 .SS "Can I write more that 64K entries is a Zip files?"
258 .IX Subsection "Can I write more that 64K entries is a Zip files?"
259 Yes. Zip64 allows this. See previous question.
260 .SS "Zip Resources"
261 .IX Subsection "Zip Resources"
262 The primary reference for zip files is the "appnote" document available at
263 <http://www.pkware.com/documents/casestudies/APPNOTE.TXT>
264 .PP
265 An alternatively is the Info\-Zip appnote. This is available from
266 <ftp://ftp.info\-zip.org/pub/infozip/doc/>
267 .SH GZIP
268 .IX Header "GZIP"
269 .SS "Gzip Resources"
270 .IX Subsection "Gzip Resources"
271 The primary reference for gzip files is RFC 1952
272 <https://datatracker.ietf.org/doc/html/rfc1952>
273 .PP
274 The primary site for gzip is <http://www.gzip.org>.
275 .SS "Dealing with concatenated gzip files"
276 .IX Subsection "Dealing with concatenated gzip files"
277 If the gunzip program encounters a file containing multiple gzip files
278 concatenated together it will automatically uncompress them all.
279 The example below illustrates this behaviour
280 .PP
281 .Vb 5
282 \& $ echo abc | gzip \-c >x.gz
283 \& $ echo def | gzip \-c >>x.gz
284 \& $ gunzip \-c x.gz
285 \& abc
286 \& def
287 .Ve
288 .PP
289 By default \f(CW\*(C`IO::Uncompress::Gunzip\*(C'\fR will \fInot\fR behave like the gunzip
290 program. It will only uncompress the first gzip data stream in the file, as
291 shown below
292 .PP
293 .Vb 2
294 \& $ perl \-MIO::Uncompress::Gunzip=:all \-e \*(Aqgunzip "x.gz" => \e*STDOUT\*(Aq
295 \& abc
296 .Ve
297 .PP
298 To force \f(CW\*(C`IO::Uncompress::Gunzip\*(C'\fR to uncompress all the gzip data streams,
299 include the \f(CW\*(C`MultiStream\*(C'\fR option, as shown below
300 .PP
301 .Vb 3
302 \& $ perl \-MIO::Uncompress::Gunzip=:all \-e \*(Aqgunzip "x.gz" => \e*STDOUT, MultiStream => 1\*(Aq
303 \& abc
304 \& def
305 .Ve
306 .SS "Reading bgzip files with IO::Uncompress::Gunzip"
307 .IX Subsection "Reading bgzip files with IO::Uncompress::Gunzip"
308 A \f(CW\*(C`bgzip\*(C'\fR file consists of a series of valid gzip\-compliant data streams
309 concatenated together. To read a file created by \f(CW\*(C`bgzip\*(C'\fR with
310 \&\f(CW\*(C`IO::Uncompress::Gunzip\*(C'\fR use the \f(CW\*(C`MultiStream\*(C'\fR option as shown in the
311 previous section.
312 .PP
313 See the section titled "The BGZF compression format" in
314 <http://samtools.github.io/hts\-specs/SAMv1.pdf> for a definition of
315 \&\f(CW\*(C`bgzip\*(C'\fR.
316 .SH ZLIB
317 .IX Header "ZLIB"
318 .SS "Zlib Resources"
319 .IX Subsection "Zlib Resources"
320 The primary site for the \fIzlib\fR compression library is
321 <http://www.zlib.org>.
322 .SH Bzip2
323 .IX Header "Bzip2"
324 .SS "Bzip2 Resources"
325 .IX Subsection "Bzip2 Resources"
326 The primary site for bzip2 is <http://www.bzip.org>.
327 .SS "Dealing with Concatenated bzip2 files"
328 .IX Subsection "Dealing with Concatenated bzip2 files"
329 If the bunzip2 program encounters a file containing multiple bzip2 files
330 concatenated together it will automatically uncompress them all.
331 The example below illustrates this behaviour
332 .PP
333 .Vb 5
334 \& $ echo abc | bzip2 \-c >x.bz2
335 \& $ echo def | bzip2 \-c >>x.bz2
336 \& $ bunzip2 \-c x.bz2
337 \& abc
338 \& def
339 .Ve
340 .PP
341 By default \f(CW\*(C`IO::Uncompress::Bunzip2\*(C'\fR will \fInot\fR behave like the bunzip2
342 program. It will only uncompress the first bunzip2 data stream in the file, as
343 shown below
344 .PP
345 .Vb 2
346 \& $ perl \-MIO::Uncompress::Bunzip2=:all \-e \*(Aqbunzip2 "x.bz2" => \e*STDOUT\*(Aq
347 \& abc
348 .Ve
349 .PP
350 To force \f(CW\*(C`IO::Uncompress::Bunzip2\*(C'\fR to uncompress all the bzip2 data streams,
351 include the \f(CW\*(C`MultiStream\*(C'\fR option, as shown below
352 .PP
353 .Vb 3
354 \& $ perl \-MIO::Uncompress::Bunzip2=:all \-e \*(Aqbunzip2 "x.bz2" => \e*STDOUT, MultiStream => 1\*(Aq
355 \& abc
356 \& def
357 .Ve
358 .SS "Interoperating with Pbzip2"
359 .IX Subsection "Interoperating with Pbzip2"
360 Pbzip2 (<http://compression.ca/pbzip2/>) is a parallel implementation of
361 bzip2. The output from pbzip2 consists of a series of concatenated bzip2
362 data streams.
363 .PP
364 By default \f(CW\*(C`IO::Uncompress::Bzip2\*(C'\fR will only uncompress the first bzip2
365 data stream in a pbzip2 file. To uncompress the complete pbzip2 file you
366 must include the \f(CW\*(C`MultiStream\*(C'\fR option, like this.
367 .PP
368 .Vb 2
369 \& bunzip2 $input => \e$output, MultiStream => 1
370 \& or die "bunzip2 failed: $Bunzip2Error\en";
371 .Ve
372 .SH "HTTP & NETWORK"
373 .IX Header "HTTP & NETWORK"
374 .SS "Apache::GZip Revisited"
375 .IX Subsection "Apache::GZip Revisited"
376 Below is a mod_perl Apache compression module, called \f(CW\*(C`Apache::GZip\*(C'\fR,
377 taken from
378 <http://perl.apache.org/docs/tutorials/tips/mod_perl_tricks/mod_perl_tricks.html#On_the_Fly_Compression>
379 .PP
380 .Vb 2
381 \& package Apache::GZip;
382 \& #File: Apache::GZip.pm
383 \&
384 \& use strict vars;
385 \& use Apache::Constants \*(Aq:common\*(Aq;
386 \& use Compress::Zlib;
387 \& use IO::File;
388 \& use constant GZIP_MAGIC => 0x1f8b;
389 \& use constant OS_MAGIC => 0x03;
390 \&
391 \& sub handler {
392 \& my $r = shift;
393 \& my ($fh,$gz);
394 \& my $file = $r\->filename;
395 \& return DECLINED unless $fh=IO::File\->new($file);
396 \& $r\->header_out(\*(AqContent\-Encoding\*(Aq=>\*(Aqgzip\*(Aq);
397 \& $r\->send_http_header;
398 \& return OK if $r\->header_only;
399 \&
400 \& tie *STDOUT,\*(AqApache::GZip\*(Aq,$r;
401 \& print($_) while <$fh>;
402 \& untie *STDOUT;
403 \& return OK;
404 \& }
405 \&
406 \& sub TIEHANDLE {
407 \& my($class,$r) = @_;
408 \& # initialize a deflation stream
409 \& my $d = deflateInit(\-WindowBits=>\-MAX_WBITS()) || return undef;
410 \&
411 \& # gzip header \-\- don\*(Aqt ask how I found out
412 \& $r\->print(pack("nccVcc",GZIP_MAGIC,Z_DEFLATED,0,time(),0,OS_MAGIC));
413 \&
414 \& return bless { r => $r,
415 \& crc => crc32(undef),
416 \& d => $d,
417 \& l => 0
418 \& },$class;
419 \& }
420 \&
421 \& sub PRINT {
422 \& my $self = shift;
423 \& foreach (@_) {
424 \& # deflate the data
425 \& my $data = $self\->{d}\->deflate($_);
426 \& $self\->{r}\->print($data);
427 \& # keep track of its length and crc
428 \& $self\->{l} += length($_);
429 \& $self\->{crc} = crc32($_,$self\->{crc});
430 \& }
431 \& }
432 \&
433 \& sub DESTROY {
434 \& my $self = shift;
435 \&
436 \& # flush the output buffers
437 \& my $data = $self\->{d}\->flush;
438 \& $self\->{r}\->print($data);
439 \&
440 \& # print the CRC and the total length (uncompressed)
441 \& $self\->{r}\->print(pack("LL",@{$self}{qw/crc l/}));
442 \& }
443 \&
444 \& 1;
445 .Ve
446 .PP
447 Here\*(Aqs the Apache configuration entry you\*(Aqll need to make use of it. Once
448 set it will result in everything in the /compressed directory will be
449 compressed automagically.
450 .PP
451 .Vb 4
452 \& <Location /compressed>
453 \& SetHandler perl\-script
454 \& PerlHandler Apache::GZip
455 \& </Location>
456 .Ve
457 .PP
458 Although at first sight there seems to be quite a lot going on in
459 \&\f(CW\*(C`Apache::GZip\*(C'\fR, you could sum up what the code was doing as follows \-\-
460 read the contents of the file in \f(CW\*(C`$r\->filename\*(C'\fR, compress it and write
461 the compressed data to standard output. That\*(Aqs all.
462 .PP
463 This code has to jump through a few hoops to achieve this because
464 .IP 1. 4
465 The gzip support in \f(CW\*(C`Compress::Zlib\*(C'\fR version 1.x can only work with a real
466 filesystem filehandle. The filehandles used by Apache modules are not
467 associated with the filesystem.
468 .IP 2. 4
469 That means all the gzip support has to be done by hand \- in this case by
470 creating a tied filehandle to deal with creating the gzip header and
471 trailer.
472 .PP
473 \&\f(CW\*(C`IO::Compress::Gzip\*(C'\fR doesn\*(Aqt have that filehandle limitation (this was one
474 of the reasons for writing it in the first place). So if
475 \&\f(CW\*(C`IO::Compress::Gzip\*(C'\fR is used instead of \f(CW\*(C`Compress::Zlib\*(C'\fR the whole tied
476 filehandle code can be removed. Here is the rewritten code.
477 .PP
478 .Vb 1
479 \& package Apache::GZip;
480 \&
481 \& use strict vars;
482 \& use Apache::Constants \*(Aq:common\*(Aq;
483 \& use IO::Compress::Gzip;
484 \& use IO::File;
485 \&
486 \& sub handler {
487 \& my $r = shift;
488 \& my ($fh,$gz);
489 \& my $file = $r\->filename;
490 \& return DECLINED unless $fh=IO::File\->new($file);
491 \& $r\->header_out(\*(AqContent\-Encoding\*(Aq=>\*(Aqgzip\*(Aq);
492 \& $r\->send_http_header;
493 \& return OK if $r\->header_only;
494 \&
495 \& my $gz = IO::Compress::Gzip\->new( \*(Aq\-\*(Aq, Minimal => 1 )
496 \& or return DECLINED ;
497 \&
498 \& print $gz $_ while <$fh>;
499 \&
500 \& return OK;
501 \& }
502 .Ve
503 .PP
504 or even more succinctly, like this, using a one\-shot gzip
505 .PP
506 .Vb 1
507 \& package Apache::GZip;
508 \&
509 \& use strict vars;
510 \& use Apache::Constants \*(Aq:common\*(Aq;
511 \& use IO::Compress::Gzip qw(gzip);
512 \&
513 \& sub handler {
514 \& my $r = shift;
515 \& $r\->header_out(\*(AqContent\-Encoding\*(Aq=>\*(Aqgzip\*(Aq);
516 \& $r\->send_http_header;
517 \& return OK if $r\->header_only;
518 \&
519 \& gzip $r\->filename => \*(Aq\-\*(Aq, Minimal => 1
520 \& or return DECLINED ;
521 \&
522 \& return OK;
523 \& }
524 \&
525 \& 1;
526 .Ve
527 .PP
528 The use of one\-shot \f(CW\*(C`gzip\*(C'\fR above just reads from \f(CW\*(C`$r\->filename\*(C'\fR and
529 writes the compressed data to standard output.
530 .PP
531 Note the use of the \f(CW\*(C`Minimal\*(C'\fR option in the code above. When using gzip
532 for Content\-Encoding you should \fIalways\fR use this option. In the example
533 above it will prevent the filename being included in the gzip header and
534 make the size of the gzip data stream a slight bit smaller.
535 .SS "Compressed files and Net::FTP"
536 .IX Subsection "Compressed files and Net::FTP"
537 The \f(CW\*(C`Net::FTP\*(C'\fR module provides two low\-level methods called \f(CW\*(C`stor\*(C'\fR and
538 \&\f(CW\*(C`retr\*(C'\fR that both return filehandles. These filehandles can used with the
539 \&\f(CW\*(C`IO::Compress/Uncompress\*(C'\fR modules to compress or uncompress files read
540 from or written to an FTP Server on the fly, without having to create a
541 temporary file.
542 .PP
543 Firstly, here is code that uses \f(CW\*(C`retr\*(C'\fR to uncompressed a file as it is
544 read from the FTP Server.
545 .PP
546 .Vb 2
547 \& use Net::FTP;
548 \& use IO::Uncompress::Gunzip qw(:all);
549 \&
550 \& my $ftp = Net::FTP\->new( ... )
551 \&
552 \& my $retr_fh = $ftp\->retr($compressed_filename);
553 \& gunzip $retr_fh => $outFilename, AutoClose => 1
554 \& or die "Cannot uncompress \*(Aq$compressed_file\*(Aq: $GunzipError\en";
555 .Ve
556 .PP
557 and this to compress a file as it is written to the FTP Server
558 .PP
559 .Vb 2
560 \& use Net::FTP;
561 \& use IO::Compress::Gzip qw(:all);
562 \&
563 \& my $stor_fh = $ftp\->stor($filename);
564 \& gzip "filename" => $stor_fh, AutoClose => 1
565 \& or die "Cannot compress \*(Aq$filename\*(Aq: $GzipError\en";
566 .Ve
567 .SH MISC
568 .IX Header "MISC"
569 .ie n .SS "Using ""InputLength"" to uncompress data embedded in a larger file/buffer."
570 .el .SS "Using \f(CWInputLength\fP to uncompress data embedded in a larger file/buffer."
571 .IX Subsection "Using InputLength to uncompress data embedded in a larger file/buffer."
572 A fairly common use\-case is where compressed data is embedded in a larger
573 file/buffer and you want to read both.
574 .PP
575 As an example consider the structure of a zip file. This is a well\-defined
576 file format that mixes both compressed and uncompressed sections of data in
577 a single file.
578 .PP
579 For the purposes of this discussion you can think of a zip file as sequence
580 of compressed data streams, each of which is prefixed by an uncompressed
581 local header. The local header contains information about the compressed
582 data stream, including the name of the compressed file and, in particular,
583 the length of the compressed data stream.
584 .PP
585 To illustrate how to use \f(CW\*(C`InputLength\*(C'\fR here is a script that walks a zip
586 file and prints out how many lines are in each compressed file (if you
587 intend write code to walking through a zip file for real see
588 "Walking through a zip file" in IO::Uncompress::Unzip ). Also, although
589 this example uses the zlib\-based compression, the technique can be used by
590 the other \f(CW\*(C`IO::Uncompress::*\*(C'\fR modules.
591 .PP
592 .Vb 2
593 \& use strict;
594 \& use warnings;
595 \&
596 \& use IO::File;
597 \& use IO::Uncompress::RawInflate qw(:all);
598 \&
599 \& use constant ZIP_LOCAL_HDR_SIG => 0x04034b50;
600 \& use constant ZIP_LOCAL_HDR_LENGTH => 30;
601 \&
602 \& my $file = $ARGV[0] ;
603 \&
604 \& my $fh = IO::File\->new( "<$file" )
605 \& or die "Cannot open \*(Aq$file\*(Aq: $!\en";
606 \&
607 \& while (1)
608 \& {
609 \& my $sig;
610 \& my $buffer;
611 \&
612 \& my $x ;
613 \& ($x = $fh\->read($buffer, ZIP_LOCAL_HDR_LENGTH)) == ZIP_LOCAL_HDR_LENGTH
614 \& or die "Truncated file: $!\en";
615 \&
616 \& my $signature = unpack ("V", substr($buffer, 0, 4));
617 \&
618 \& last unless $signature == ZIP_LOCAL_HDR_SIG;
619 \&
620 \& # Read Local Header
621 \& my $gpFlag = unpack ("v", substr($buffer, 6, 2));
622 \& my $compressedMethod = unpack ("v", substr($buffer, 8, 2));
623 \& my $compressedLength = unpack ("V", substr($buffer, 18, 4));
624 \& my $uncompressedLength = unpack ("V", substr($buffer, 22, 4));
625 \& my $filename_length = unpack ("v", substr($buffer, 26, 2));
626 \& my $extra_length = unpack ("v", substr($buffer, 28, 2));
627 \&
628 \& my $filename ;
629 \& $fh\->read($filename, $filename_length) == $filename_length
630 \& or die "Truncated file\en";
631 \&
632 \& $fh\->read($buffer, $extra_length) == $extra_length
633 \& or die "Truncated file\en";
634 \&
635 \& if ($compressedMethod != 8 && $compressedMethod != 0)
636 \& {
637 \& warn "Skipping file \*(Aq$filename\*(Aq \- not deflated $compressedMethod\en";
638 \& $fh\->read($buffer, $compressedLength) == $compressedLength
639 \& or die "Truncated file\en";
640 \& next;
641 \& }
642 \&
643 \& if ($compressedMethod == 0 && $gpFlag & 8 == 8)
644 \& {
645 \& die "Streamed Stored not supported for \*(Aq$filename\*(Aq\en";
646 \& }
647 \&
648 \& next if $compressedLength == 0;
649 \&
650 \& # Done reading the Local Header
651 \&
652 \& my $inf = IO::Uncompress::RawInflate\->new( $fh,
653 \& Transparent => 1,
654 \& InputLength => $compressedLength )
655 \& or die "Cannot uncompress $file [$filename]: $RawInflateError\en" ;
656 \&
657 \& my $line_count = 0;
658 \&
659 \& while (<$inf>)
660 \& {
661 \& ++ $line_count;
662 \& }
663 \&
664 \& print "$filename: $line_count\en";
665 \& }
666 .Ve
667 .PP
668 The majority of the code above is concerned with reading the zip local
669 header data. The code that I want to focus on is at the bottom.
670 .PP
671 .Vb 1
672 \& while (1) {
673 \&
674 \& # read local zip header data
675 \& # get $filename
676 \& # get $compressedLength
677 \&
678 \& my $inf = IO::Uncompress::RawInflate\->new( $fh,
679 \& Transparent => 1,
680 \& InputLength => $compressedLength )
681 \& or die "Cannot uncompress $file [$filename]: $RawInflateError\en" ;
682 \&
683 \& my $line_count = 0;
684 \&
685 \& while (<$inf>)
686 \& {
687 \& ++ $line_count;
688 \& }
689 \&
690 \& print "$filename: $line_count\en";
691 \& }
692 .Ve
693 .PP
694 The call to \f(CW\*(C`IO::Uncompress::RawInflate\*(C'\fR creates a new filehandle \f(CW$inf\fR
695 that can be used to read from the parent filehandle \f(CW$fh\fR, uncompressing
696 it as it goes. The use of the \f(CW\*(C`InputLength\*(C'\fR option will guarantee that
697 \&\fIat most\fR \f(CW$compressedLength\fR bytes of compressed data will be read from
698 the \f(CW$fh\fR filehandle (The only exception is for an error case like a
699 truncated file or a corrupt data stream).
700 .PP
701 This means that once RawInflate is finished \f(CW$fh\fR will be left at the
702 byte directly after the compressed data stream.
703 .PP
704 Now consider what the code looks like without \f(CW\*(C`InputLength\*(C'\fR
705 .PP
706 .Vb 1
707 \& while (1) {
708 \&
709 \& # read local zip header data
710 \& # get $filename
711 \& # get $compressedLength
712 \&
713 \& # read all the compressed data into $data
714 \& read($fh, $data, $compressedLength);
715 \&
716 \& my $inf = IO::Uncompress::RawInflate\->new( \e$data,
717 \& Transparent => 1 )
718 \& or die "Cannot uncompress $file [$filename]: $RawInflateError\en" ;
719 \&
720 \& my $line_count = 0;
721 \&
722 \& while (<$inf>)
723 \& {
724 \& ++ $line_count;
725 \& }
726 \&
727 \& print "$filename: $line_count\en";
728 \& }
729 .Ve
730 .PP
731 The difference here is the addition of the temporary variable \f(CW$data\fR.
732 This is used to store a copy of the compressed data while it is being
733 uncompressed.
734 .PP
735 If you know that \f(CW$compressedLength\fR isn\*(Aqt that big then using temporary
736 storage won\*(Aqt be a problem. But if \f(CW$compressedLength\fR is very large or
737 you are writing an application that other people will use, and so have no
738 idea how big \f(CW$compressedLength\fR will be, it could be an issue.
739 .PP
740 Using \f(CW\*(C`InputLength\*(C'\fR avoids the use of temporary storage and means the
741 application can cope with large compressed data streams.
742 .PP
743 One final point \-\- obviously \f(CW\*(C`InputLength\*(C'\fR can only be used whenever you
744 know the length of the compressed data beforehand, like here with a zip
745 file.
746 .SH SUPPORT
747 .IX Header "SUPPORT"
748 General feedback/questions/bug reports should be sent to
749 <https://github.com/pmqs//issues> (preferred) or
750 <https://rt.cpan.org/Public/Dist/Display.html?Name=>.
751 .SH "SEE ALSO"
752 .IX Header "SEE ALSO"
753 Compress::Zlib, IO::Compress::Gzip, IO::Uncompress::Gunzip, IO::Compress::Deflate, IO::Uncompress::Inflate, IO::Compress::RawDeflate, IO::Uncompress::RawInflate, IO::Compress::Bzip2, IO::Uncompress::Bunzip2, IO::Compress::Lzma, IO::Uncompress::UnLzma, IO::Compress::Xz, IO::Uncompress::UnXz, IO::Compress::Lzip, IO::Uncompress::UnLzip, IO::Compress::Lzop, IO::Uncompress::UnLzop, IO::Compress::Lzf, IO::Uncompress::UnLzf, IO::Compress::Zstd, IO::Uncompress::UnZstd, IO::Uncompress::AnyInflate, IO::Uncompress::AnyUncompress
754 .PP
755 IO::Compress::FAQ
756 .PP
757 File::GlobMapper, Archive::Zip,
758 Archive::Tar,
759 IO::Zlib
760 .SH AUTHOR
761 .IX Header "AUTHOR"
762 This module was written by Paul Marquess, \f(CW\*(C`[email protected]\*(C'\fR.
763 .SH "MODIFICATION HISTORY"
764 .IX Header "MODIFICATION HISTORY"
765 See the Changes file.
766 .SH "COPYRIGHT AND LICENSE"
767 .IX Header "COPYRIGHT AND LICENSE"
768 Copyright (c) 2005\-2024 Paul Marquess. All rights reserved.
769 .PP
770 This program is free software; you can redistribute it and/or
771 modify it under the same terms as Perl itself.