Debian Codegraph @

upstream/opensuse-tumbleweed/man3/File::Fetch.3pm

raw
1 .\" -*- mode: troff; coding: utf-8 -*-
2 .\" Automatically generated by Pod::Man v6.0.2 (Pod::Simple 3.45)
3 .\"
4 .\" Standard preamble:
5 .\" ========================================================================
6 .de Sp \" Vertical space (when we can't use .PP)
7 .if t .sp .5v
8 .if n .sp
9 ..
10 .de Vb \" Begin verbatim text
11 .ft CW
12 .nf
13 .ne \\$1
14 ..
15 .de Ve \" End verbatim text
16 .ft R
17 .fi
18 ..
19 .\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>.
20 .ie n \{\
21 . ds C` ""
22 . ds C' ""
23 'br\}
24 .el\{\
25 . ds C`
26 . ds C'
27 'br\}
28 .\"
29 .\" Escape single quotes in literal strings from groff's Unicode transform.
30 .ie \n(.g .ds Aq \(aq
31 .el .ds Aq '
32 .\"
33 .\" If the F register is >0, we'll generate index entries on stderr for
34 .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
35 .\" entries marked with X<> in POD. Of course, you'll have to process the
36 .\" output yourself in some meaningful fashion.
37 .\"
38 .\" Avoid warning from groff about undefined register 'F'.
39 .de IX
40 ..
41 .nr rF 0
42 .if \n(.g .if rF .nr rF 1
43 .if (\n(rF:(\n(.g==0)) \{\
44 . if \nF \{\
45 . de IX
46 . tm Index:\\$1\t\\n%\t"\\$2"
47 ..
48 . if !\nF==2 \{\
49 . nr % 0
50 . nr F 2
51 . \}
52 . \}
53 .\}
54 .rr rF
55 .\"
56 .\" Required to disable full justification in groff 1.23.0.
57 .if n .ds AD l
58 .\" ========================================================================
59 .\"
60 .IX Title "File::Fetch 3pm"
61 .TH File::Fetch 3pm 2026-03-09 "perl v5.42.1" "Perl Programmers Reference Guide"
62 .\" For nroff, turn off justification. Always turn off hyphenation; it makes
63 .\" way too many mistakes in technical documents.
64 .if n .ad l
65 .nh
66 .SH NAME
67 File::Fetch \- A generic file fetching mechanism
68 .SH SYNOPSIS
69 .IX Header "SYNOPSIS"
70 .Vb 1
71 \& use File::Fetch;
72 \&
73 \& ### build a File::Fetch object ###
74 \& my $ff = File::Fetch\->new(uri => \*(Aqhttp://some.where.com/dir/a.txt\*(Aq);
75 \&
76 \& ### fetch the uri to cwd() ###
77 \& my $where = $ff\->fetch() or die $ff\->error;
78 \&
79 \& ### fetch the uri to /tmp ###
80 \& my $where = $ff\->fetch( to => \*(Aq/tmp\*(Aq );
81 \&
82 \& ### parsed bits from the uri ###
83 \& $ff\->uri;
84 \& $ff\->scheme;
85 \& $ff\->host;
86 \& $ff\->path;
87 \& $ff\->file;
88 .Ve
89 .SH DESCRIPTION
90 .IX Header "DESCRIPTION"
91 File::Fetch is a generic file fetching mechanism.
92 .PP
93 It allows you to fetch any file pointed to by a \f(CW\*(C`ftp\*(C'\fR, \f(CW\*(C`http\*(C'\fR,
94 \&\f(CW\*(C`file\*(C'\fR, \f(CW\*(C`git\*(C'\fR or \f(CW\*(C`rsync\*(C'\fR uri by a number of different means.
95 .PP
96 See the \f(CW\*(C`HOW IT WORKS\*(C'\fR section further down for details.
97 .SH ACCESSORS
98 .IX Header "ACCESSORS"
99 A \f(CW\*(C`File::Fetch\*(C'\fR object has the following accessors
100 .ie n .IP $ff\->uri 4
101 .el .IP \f(CW$ff\fR\->uri 4
102 .IX Item "$ff->uri"
103 The uri you passed to the constructor
104 .ie n .IP $ff\->scheme 4
105 .el .IP \f(CW$ff\fR\->scheme 4
106 .IX Item "$ff->scheme"
107 The scheme from the uri (like \*(Aqfile\*(Aq, \*(Aqhttp\*(Aq, etc)
108 .ie n .IP $ff\->host 4
109 .el .IP \f(CW$ff\fR\->host 4
110 .IX Item "$ff->host"
111 The hostname in the uri. Will be empty if host was originally
112 \&\*(Aqlocalhost\*(Aq for a \*(Aqfile://\*(Aq url.
113 .ie n .IP $ff\->vol 4
114 .el .IP \f(CW$ff\fR\->vol 4
115 .IX Item "$ff->vol"
116 On operating systems with the concept of a volume the second element
117 of a file:// is considered to the be volume specification for the file.
118 Thus on Win32 this routine returns the volume, on other operating
119 systems this returns nothing.
120 .Sp
121 On Windows this value may be empty if the uri is to a network share, in
122 which case the \*(Aqshare\*(Aq property will be defined. Additionally, volume
123 specifications that use \*(Aq|\*(Aq as \*(Aq:\*(Aq will be converted on read to use \*(Aq:\*(Aq.
124 .Sp
125 On VMS, which has a volume concept, this field will be empty because VMS
126 file specifications are converted to absolute UNIX format and the volume
127 information is transparently included.
128 .ie n .IP $ff\->share 4
129 .el .IP \f(CW$ff\fR\->share 4
130 .IX Item "$ff->share"
131 On systems with the concept of a network share (currently only Windows) returns
132 the sharename from a file://// url. On other operating systems returns empty.
133 .ie n .IP $ff\->path 4
134 .el .IP \f(CW$ff\fR\->path 4
135 .IX Item "$ff->path"
136 The path from the uri, will be at least a single \*(Aq/\*(Aq.
137 .ie n .IP $ff\->file 4
138 .el .IP \f(CW$ff\fR\->file 4
139 .IX Item "$ff->file"
140 The name of the remote file. For the local file name, the
141 result of \f(CW$ff\fR\->output_file will be used.
142 .ie n .IP $ff\->file_default 4
143 .el .IP \f(CW$ff\fR\->file_default 4
144 .IX Item "$ff->file_default"
145 The name of the default local file, that \f(CW$ff\fR\->output_file falls back to if
146 it would otherwise return no filename. For example when fetching a URI like
147 http://www.abc.net.au/ the contents retrieved may be from a remote file called
148 \&\*(Aqindex.html\*(Aq. The default value of this attribute is literally \*(Aqfile_default\*(Aq.
149 .ie n .IP $ff\->output_file 4
150 .el .IP \f(CW$ff\fR\->output_file 4
151 .IX Item "$ff->output_file"
152 The name of the output file. This is the same as \f(CW$ff\fR\->file,
153 but any query parameters are stripped off. For example:
154 .Sp
155 .Vb 1
156 \& http://example.com/index.html?x=y
157 .Ve
158 .Sp
159 would make the output file be \f(CW\*(C`index.html\*(C'\fR rather than
160 \&\f(CW\*(C`index.html?x=y\*(C'\fR.
161 .SH METHODS
162 .IX Header "METHODS"
163 .ie n .SS "$ff = File::Fetch\->new( uri => \*(Aqhttp://some.where.com/dir/file.txt\*(Aq );"
164 .el .SS "\f(CW$ff\fP = File::Fetch\->new( uri => \*(Aqhttp://some.where.com/dir/file.txt\*(Aq );"
165 .IX Subsection "$ff = File::Fetch->new( uri => 'http://some.where.com/dir/file.txt' );"
166 Parses the uri and creates a corresponding File::Fetch::Item object,
167 that is ready to be \f(CW\*(C`fetch\*(C'\fRed and returns it.
168 .PP
169 Returns false on failure.
170 .ie n .SS "$where = $ff\->fetch( [to => /my/output/dir/ | \e$scalar] )"
171 .el .SS "\f(CW$where\fP = \f(CW$ff\fP\->fetch( [to => /my/output/dir/ | \e$scalar] )"
172 .IX Subsection "$where = $ff->fetch( [to => /my/output/dir/ | $scalar] )"
173 Fetches the file you requested and returns the full path to the file.
174 .PP
175 By default it writes to \f(CWcwd()\fR, but you can override that by specifying
176 the \f(CW\*(C`to\*(C'\fR argument:
177 .PP
178 .Vb 2
179 \& ### file fetch to /tmp, full path to the file in $where
180 \& $where = $ff\->fetch( to => \*(Aq/tmp\*(Aq );
181 \&
182 \& ### file slurped into $scalar, full path to the file in $where
183 \& ### file is downloaded to a temp directory and cleaned up at exit time
184 \& $where = $ff\->fetch( to => \e$scalar );
185 .Ve
186 .PP
187 Returns the full path to the downloaded file on success, and false
188 on failure.
189 .ie n .SS $ff\->error([BOOL])
190 .el .SS \f(CW$ff\fP\->error([BOOL])
191 .IX Subsection "$ff->error([BOOL])"
192 Returns the last encountered error as string.
193 Pass it a true value to get the \f(CWCarp::longmess()\fR output instead.
194 .SH "HOW IT WORKS"
195 .IX Header "HOW IT WORKS"
196 File::Fetch is able to fetch a variety of uris, by using several
197 external programs and modules.
198 .PP
199 Below is a mapping of what utilities will be used in what order
200 for what schemes, if available:
201 .PP
202 .Vb 5
203 \& file => LWP, lftp, file
204 \& http => LWP, HTTP::Tiny, wget, curl, lftp, fetch, HTTP::Lite, lynx, iosock
205 \& ftp => LWP, Net::FTP, wget, curl, lftp, fetch, ncftp, ftp
206 \& rsync => rsync
207 \& git => git
208 .Ve
209 .PP
210 If you\*(Aqd like to disable the use of one or more of these utilities
211 and/or modules, see the \f(CW$BLACKLIST\fR variable further down.
212 .PP
213 If a utility or module isn\*(Aqt available, it will be marked in a cache
214 (see the \f(CW$METHOD_FAIL\fR variable further down), so it will not be
215 tried again. The \f(CW\*(C`fetch\*(C'\fR method will only fail when all options are
216 exhausted, and it was not able to retrieve the file.
217 .PP
218 The \f(CW\*(C`fetch\*(C'\fR utility is available on FreeBSD. NetBSD and Dragonfly BSD
219 may also have it from \f(CW\*(C`pkgsrc\*(C'\fR. We only check for \f(CW\*(C`fetch\*(C'\fR on those
220 three platforms.
221 .PP
222 \&\f(CW\*(C`iosock\*(C'\fR is a very limited IO::Socket::INET based mechanism for
223 retrieving \f(CW\*(C`http\*(C'\fR schemed urls. It doesn\*(Aqt follow redirects for instance.
224 .PP
225 \&\f(CW\*(C`git\*(C'\fR only supports \f(CW\*(C`git://\*(C'\fR style urls.
226 .PP
227 A special note about fetching files from an ftp uri:
228 .PP
229 By default, all ftp connections are done in passive mode. To change
230 that, see the \f(CW$FTP_PASSIVE\fR variable further down.
231 .PP
232 Furthermore, ftp uris only support anonymous connections, so no
233 named user/password pair can be passed along.
234 .PP
235 \&\f(CW\*(C`/bin/ftp\*(C'\fR is blacklisted by default; see the \f(CW$BLACKLIST\fR variable
236 further down.
237 .SH "GLOBAL VARIABLES"
238 .IX Header "GLOBAL VARIABLES"
239 The behaviour of File::Fetch can be altered by changing the following
240 global variables:
241 .ie n .SS $File::Fetch::FROM_EMAIL
242 .el .SS \f(CW$File::Fetch::FROM_EMAIL\fP
243 .IX Subsection "$File::Fetch::FROM_EMAIL"
244 This is the email address that will be sent as your anonymous ftp
245 password.
246 .PP
247 Default is \f(CW\*(C`File\[email protected]\*(C'\fR.
248 .ie n .SS $File::Fetch::USER_AGENT
249 .el .SS \f(CW$File::Fetch::USER_AGENT\fP
250 .IX Subsection "$File::Fetch::USER_AGENT"
251 This is the useragent as \f(CW\*(C`LWP\*(C'\fR will report it.
252 .PP
253 Default is \f(CW\*(C`File::Fetch/$VERSION\*(C'\fR.
254 .ie n .SS $File::Fetch::FTP_PASSIVE
255 .el .SS \f(CW$File::Fetch::FTP_PASSIVE\fP
256 .IX Subsection "$File::Fetch::FTP_PASSIVE"
257 This variable controls whether the environment variable \f(CW\*(C`FTP_PASSIVE\*(C'\fR
258 and any passive switches to commandline tools will be set to true.
259 .PP
260 Default value is 1.
261 .PP
262 Note: When \f(CW$FTP_PASSIVE\fR is true, \f(CW\*(C`ncftp\*(C'\fR will not be used to fetch
263 files, since passive mode can only be set interactively for this binary
264 .ie n .SS $File::Fetch::TIMEOUT
265 .el .SS \f(CW$File::Fetch::TIMEOUT\fP
266 .IX Subsection "$File::Fetch::TIMEOUT"
267 When set, controls the network timeout (counted in seconds).
268 .PP
269 Default value is 0.
270 .ie n .SS $File::Fetch::WARN
271 .el .SS \f(CW$File::Fetch::WARN\fP
272 .IX Subsection "$File::Fetch::WARN"
273 This variable controls whether errors encountered internally by
274 \&\f(CW\*(C`File::Fetch\*(C'\fR should be \f(CW\*(C`carp\*(C'\fR\*(Aqd or not.
275 .PP
276 Set to false to silence warnings. Inspect the output of the \f(CWerror()\fR
277 method manually to see what went wrong.
278 .PP
279 Defaults to \f(CW\*(C`true\*(C'\fR.
280 .ie n .SS $File::Fetch::DEBUG
281 .el .SS \f(CW$File::Fetch::DEBUG\fP
282 .IX Subsection "$File::Fetch::DEBUG"
283 This enables debugging output when calling commandline utilities to
284 fetch files.
285 This also enables \f(CW\*(C`Carp::longmess\*(C'\fR errors, instead of the regular
286 \&\f(CW\*(C`carp\*(C'\fR errors.
287 .PP
288 Good for tracking down why things don\*(Aqt work with your particular
289 setup.
290 .PP
291 Default is 0.
292 .ie n .SS $File::Fetch::BLACKLIST
293 .el .SS \f(CW$File::Fetch::BLACKLIST\fP
294 .IX Subsection "$File::Fetch::BLACKLIST"
295 This is an array ref holding blacklisted modules/utilities for fetching
296 files with.
297 .PP
298 To disallow the use of, for example, \f(CW\*(C`LWP\*(C'\fR and \f(CW\*(C`Net::FTP\*(C'\fR, you could
299 set \f(CW$File::Fetch::BLACKLIST\fR to:
300 .PP
301 .Vb 1
302 \& $File::Fetch::BLACKLIST = [qw|lwp netftp|]
303 .Ve
304 .PP
305 The default blacklist is [qw|ftp|], as \f(CW\*(C`/bin/ftp\*(C'\fR is rather unreliable.
306 .PP
307 See the note on \f(CW\*(C`MAPPING\*(C'\fR below.
308 .ie n .SS $File::Fetch::METHOD_FAIL
309 .el .SS \f(CW$File::Fetch::METHOD_FAIL\fP
310 .IX Subsection "$File::Fetch::METHOD_FAIL"
311 This is a hashref registering what modules/utilities were known to fail
312 for fetching files (mostly because they weren\*(Aqt installed).
313 .PP
314 You can reset this cache by assigning an empty hashref to it, or
315 individually remove keys.
316 .PP
317 See the note on \f(CW\*(C`MAPPING\*(C'\fR below.
318 .SH MAPPING
319 .IX Header "MAPPING"
320 Here\*(Aqs a quick mapping for the utilities/modules, and their names for
321 the \f(CW$BLACKLIST\fR, \f(CW$METHOD_FAIL\fR and other internal functions.
322 .PP
323 .Vb 10
324 \& LWP => lwp
325 \& HTTP::Lite => httplite
326 \& HTTP::Tiny => httptiny
327 \& Net::FTP => netftp
328 \& wget => wget
329 \& lynx => lynx
330 \& ncftp => ncftp
331 \& ftp => ftp
332 \& curl => curl
333 \& rsync => rsync
334 \& lftp => lftp
335 \& fetch => fetch
336 \& IO::Socket => iosock
337 .Ve
338 .SH "FREQUENTLY ASKED QUESTIONS"
339 .IX Header "FREQUENTLY ASKED QUESTIONS"
340 .SS "So how do I use a proxy with File::Fetch?"
341 .IX Subsection "So how do I use a proxy with File::Fetch?"
342 \&\f(CW\*(C`File::Fetch\*(C'\fR currently only supports proxies with LWP::UserAgent.
343 You will need to set your environment variables accordingly. For
344 example, to use an ftp proxy:
345 .PP
346 .Vb 1
347 \& $ENV{ftp_proxy} = \*(Aqfoo.com\*(Aq;
348 .Ve
349 .PP
350 Refer to the LWP::UserAgent manpage for more details.
351 .SS "I used \*(Aqlynx\*(Aq to fetch a file, but its contents is all wrong!"
352 .IX Subsection "I used 'lynx' to fetch a file, but its contents is all wrong!"
353 \&\f(CW\*(C`lynx\*(C'\fR can only fetch remote files by dumping its contents to \f(CW\*(C`STDOUT\*(C'\fR,
354 which we in turn capture. If that content is a \*(Aqcustom\*(Aq error file
355 (like, say, a \f(CW\*(C`404 handler\*(C'\fR), you will get that contents instead.
356 .PP
357 Sadly, \f(CW\*(C`lynx\*(C'\fR doesn\*(Aqt support any options to return a different exit
358 code on non\-\f(CW\*(C`200 OK\*(C'\fR status, giving us no way to tell the difference
359 between a \*(Aqsuccessful\*(Aq fetch and a custom error page.
360 .PP
361 Therefor, we recommend to only use \f(CW\*(C`lynx\*(C'\fR as a last resort. This is
362 why it is at the back of our list of methods to try as well.
363 .SS "Files I\*(Aqm trying to fetch have reserved characters or non\-ASCII characters in them. What do I do?"
364 .IX Subsection "Files I'm trying to fetch have reserved characters or non-ASCII characters in them. What do I do?"
365 \&\f(CW\*(C`File::Fetch\*(C'\fR is relatively smart about things. When trying to write
366 a file to disk, it removes the \f(CW\*(C`query parameters\*(C'\fR (see the
367 \&\f(CW\*(C`output_file\*(C'\fR method for details) from the file name before creating
368 it. In most cases this suffices.
369 .PP
370 If you have any other characters you need to escape, please install
371 the \f(CW\*(C`URI::Escape\*(C'\fR module from CPAN, and pre\-encode your URI before
372 passing it to \f(CW\*(C`File::Fetch\*(C'\fR. You can read about the details of URIs
373 and URI encoding here:
374 .PP
375 <https://datatracker.ietf.org/doc/html/rfc2396>
376 .SH TODO
377 .IX Header "TODO"
378 .ie n .IP "Implement $PREFER_BIN" 4
379 .el .IP "Implement \f(CW$PREFER_BIN\fR" 4
380 .IX Item "Implement $PREFER_BIN"
381 To indicate to rather use commandline tools than modules
382 .SH "BUG REPORTS"
383 .IX Header "BUG REPORTS"
384 Please report bugs or other issues to <bug\-file\[email protected]<gt>.
385 .SH AUTHOR
386 .IX Header "AUTHOR"
387 This module by Jos Boumans <[email protected]>.
388 .SH COPYRIGHT
389 .IX Header "COPYRIGHT"
390 This library is free software; you may redistribute and/or modify it
391 under the same terms as Perl itself.