Debian Codegraph @

upstream/debian-unstable/man3/File::Fetch.3perl

raw
1 .\" -*- mode: troff; coding: utf-8 -*-
2 .\" Automatically generated by Pod::Man 5.0102 (Pod::Simple 3.45)
3 .\"
4 .\" Standard preamble:
5 .\" ========================================================================
6 .de Sp \" Vertical space (when we can't use .PP)
7 .if t .sp .5v
8 .if n .sp
9 ..
10 .de Vb \" Begin verbatim text
11 .ft CW
12 .nf
13 .ne \\$1
14 ..
15 .de Ve \" End verbatim text
16 .ft R
17 .fi
18 ..
19 .\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>.
20 .ie n \{\
21 . ds C` ""
22 . ds C' ""
23 'br\}
24 .el\{\
25 . ds C`
26 . ds C'
27 'br\}
28 .\"
29 .\" Escape single quotes in literal strings from groff's Unicode transform.
30 .ie \n(.g .ds Aq \(aq
31 .el .ds Aq '
32 .\"
33 .\" If the F register is >0, we'll generate index entries on stderr for
34 .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
35 .\" entries marked with X<> in POD. Of course, you'll have to process the
36 .\" output yourself in some meaningful fashion.
37 .\"
38 .\" Avoid warning from groff about undefined register 'F'.
39 .de IX
40 ..
41 .nr rF 0
42 .if \n(.g .if rF .nr rF 1
43 .if (\n(rF:(\n(.g==0)) \{\
44 . if \nF \{\
45 . de IX
46 . tm Index:\\$1\t\\n%\t"\\$2"
47 ..
48 . if !\nF==2 \{\
49 . nr % 0
50 . nr F 2
51 . \}
52 . \}
53 .\}
54 .rr rF
55 .\" ========================================================================
56 .\"
57 .IX Title "File::Fetch 3perl"
58 .TH File::Fetch 3perl 2025-11-16 "perl v5.40.1" "Perl Programmers Reference Guide"
59 .\" For nroff, turn off justification. Always turn off hyphenation; it makes
60 .\" way too many mistakes in technical documents.
61 .if n .ad l
62 .nh
63 .SH NAME
64 File::Fetch \- A generic file fetching mechanism
65 .SH SYNOPSIS
66 .IX Header "SYNOPSIS"
67 .Vb 1
68 \& use File::Fetch;
69 \&
70 \& ### build a File::Fetch object ###
71 \& my $ff = File::Fetch\->new(uri => \*(Aqhttp://some.where.com/dir/a.txt\*(Aq);
72 \&
73 \& ### fetch the uri to cwd() ###
74 \& my $where = $ff\->fetch() or die $ff\->error;
75 \&
76 \& ### fetch the uri to /tmp ###
77 \& my $where = $ff\->fetch( to => \*(Aq/tmp\*(Aq );
78 \&
79 \& ### parsed bits from the uri ###
80 \& $ff\->uri;
81 \& $ff\->scheme;
82 \& $ff\->host;
83 \& $ff\->path;
84 \& $ff\->file;
85 .Ve
86 .SH DESCRIPTION
87 .IX Header "DESCRIPTION"
88 File::Fetch is a generic file fetching mechanism.
89 .PP
90 It allows you to fetch any file pointed to by a \f(CW\*(C`ftp\*(C'\fR, \f(CW\*(C`http\*(C'\fR,
91 \&\f(CW\*(C`file\*(C'\fR, \f(CW\*(C`git\*(C'\fR or \f(CW\*(C`rsync\*(C'\fR uri by a number of different means.
92 .PP
93 See the \f(CW\*(C`HOW IT WORKS\*(C'\fR section further down for details.
94 .SH ACCESSORS
95 .IX Header "ACCESSORS"
96 A \f(CW\*(C`File::Fetch\*(C'\fR object has the following accessors
97 .ie n .IP $ff\->uri 4
98 .el .IP \f(CW$ff\fR\->uri 4
99 .IX Item "$ff->uri"
100 The uri you passed to the constructor
101 .ie n .IP $ff\->scheme 4
102 .el .IP \f(CW$ff\fR\->scheme 4
103 .IX Item "$ff->scheme"
104 The scheme from the uri (like 'file', 'http', etc)
105 .ie n .IP $ff\->host 4
106 .el .IP \f(CW$ff\fR\->host 4
107 .IX Item "$ff->host"
108 The hostname in the uri. Will be empty if host was originally
109 \&'localhost' for a 'file://' url.
110 .ie n .IP $ff\->vol 4
111 .el .IP \f(CW$ff\fR\->vol 4
112 .IX Item "$ff->vol"
113 On operating systems with the concept of a volume the second element
114 of a file:// is considered to the be volume specification for the file.
115 Thus on Win32 this routine returns the volume, on other operating
116 systems this returns nothing.
117 .Sp
118 On Windows this value may be empty if the uri is to a network share, in
119 which case the 'share' property will be defined. Additionally, volume
120 specifications that use '|' as ':' will be converted on read to use ':'.
121 .Sp
122 On VMS, which has a volume concept, this field will be empty because VMS
123 file specifications are converted to absolute UNIX format and the volume
124 information is transparently included.
125 .ie n .IP $ff\->share 4
126 .el .IP \f(CW$ff\fR\->share 4
127 .IX Item "$ff->share"
128 On systems with the concept of a network share (currently only Windows) returns
129 the sharename from a file://// url. On other operating systems returns empty.
130 .ie n .IP $ff\->path 4
131 .el .IP \f(CW$ff\fR\->path 4
132 .IX Item "$ff->path"
133 The path from the uri, will be at least a single '/'.
134 .ie n .IP $ff\->file 4
135 .el .IP \f(CW$ff\fR\->file 4
136 .IX Item "$ff->file"
137 The name of the remote file. For the local file name, the
138 result of \f(CW$ff\fR\->output_file will be used.
139 .ie n .IP $ff\->file_default 4
140 .el .IP \f(CW$ff\fR\->file_default 4
141 .IX Item "$ff->file_default"
142 The name of the default local file, that \f(CW$ff\fR\->output_file falls back to if
143 it would otherwise return no filename. For example when fetching a URI like
144 http://www.abc.net.au/ the contents retrieved may be from a remote file called
145 \&'index.html'. The default value of this attribute is literally 'file_default'.
146 .ie n .IP $ff\->output_file 4
147 .el .IP \f(CW$ff\fR\->output_file 4
148 .IX Item "$ff->output_file"
149 The name of the output file. This is the same as \f(CW$ff\fR\->file,
150 but any query parameters are stripped off. For example:
151 .Sp
152 .Vb 1
153 \& http://example.com/index.html?x=y
154 .Ve
155 .Sp
156 would make the output file be \f(CW\*(C`index.html\*(C'\fR rather than
157 \&\f(CW\*(C`index.html?x=y\*(C'\fR.
158 .SH METHODS
159 .IX Header "METHODS"
160 .ie n .SS "$ff = File::Fetch\->new( uri => 'http://some.where.com/dir/file.txt' );"
161 .el .SS "\f(CW$ff\fP = File::Fetch\->new( uri => 'http://some.where.com/dir/file.txt' );"
162 .IX Subsection "$ff = File::Fetch->new( uri => 'http://some.where.com/dir/file.txt' );"
163 Parses the uri and creates a corresponding File::Fetch::Item object,
164 that is ready to be \f(CW\*(C`fetch\*(C'\fRed and returns it.
165 .PP
166 Returns false on failure.
167 .ie n .SS "$where = $ff\->fetch( [to => /my/output/dir/ | \e$scalar] )"
168 .el .SS "\f(CW$where\fP = \f(CW$ff\fP\->fetch( [to => /my/output/dir/ | \e$scalar] )"
169 .IX Subsection "$where = $ff->fetch( [to => /my/output/dir/ | $scalar] )"
170 Fetches the file you requested and returns the full path to the file.
171 .PP
172 By default it writes to \f(CWcwd()\fR, but you can override that by specifying
173 the \f(CW\*(C`to\*(C'\fR argument:
174 .PP
175 .Vb 2
176 \& ### file fetch to /tmp, full path to the file in $where
177 \& $where = $ff\->fetch( to => \*(Aq/tmp\*(Aq );
178 \&
179 \& ### file slurped into $scalar, full path to the file in $where
180 \& ### file is downloaded to a temp directory and cleaned up at exit time
181 \& $where = $ff\->fetch( to => \e$scalar );
182 .Ve
183 .PP
184 Returns the full path to the downloaded file on success, and false
185 on failure.
186 .ie n .SS $ff\->error([BOOL])
187 .el .SS \f(CW$ff\fP\->error([BOOL])
188 .IX Subsection "$ff->error([BOOL])"
189 Returns the last encountered error as string.
190 Pass it a true value to get the \f(CWCarp::longmess()\fR output instead.
191 .SH "HOW IT WORKS"
192 .IX Header "HOW IT WORKS"
193 File::Fetch is able to fetch a variety of uris, by using several
194 external programs and modules.
195 .PP
196 Below is a mapping of what utilities will be used in what order
197 for what schemes, if available:
198 .PP
199 .Vb 5
200 \& file => LWP, lftp, file
201 \& http => LWP, HTTP::Tiny, wget, curl, lftp, fetch, HTTP::Lite, lynx, iosock
202 \& ftp => LWP, Net::FTP, wget, curl, lftp, fetch, ncftp, ftp
203 \& rsync => rsync
204 \& git => git
205 .Ve
206 .PP
207 If you'd like to disable the use of one or more of these utilities
208 and/or modules, see the \f(CW$BLACKLIST\fR variable further down.
209 .PP
210 If a utility or module isn't available, it will be marked in a cache
211 (see the \f(CW$METHOD_FAIL\fR variable further down), so it will not be
212 tried again. The \f(CW\*(C`fetch\*(C'\fR method will only fail when all options are
213 exhausted, and it was not able to retrieve the file.
214 .PP
215 The \f(CW\*(C`fetch\*(C'\fR utility is available on FreeBSD. NetBSD and Dragonfly BSD
216 may also have it from \f(CW\*(C`pkgsrc\*(C'\fR. We only check for \f(CW\*(C`fetch\*(C'\fR on those
217 three platforms.
218 .PP
219 \&\f(CW\*(C`iosock\*(C'\fR is a very limited IO::Socket::INET based mechanism for
220 retrieving \f(CW\*(C`http\*(C'\fR schemed urls. It doesn't follow redirects for instance.
221 .PP
222 \&\f(CW\*(C`git\*(C'\fR only supports \f(CW\*(C`git://\*(C'\fR style urls.
223 .PP
224 A special note about fetching files from an ftp uri:
225 .PP
226 By default, all ftp connections are done in passive mode. To change
227 that, see the \f(CW$FTP_PASSIVE\fR variable further down.
228 .PP
229 Furthermore, ftp uris only support anonymous connections, so no
230 named user/password pair can be passed along.
231 .PP
232 \&\f(CW\*(C`/bin/ftp\*(C'\fR is blacklisted by default; see the \f(CW$BLACKLIST\fR variable
233 further down.
234 .SH "GLOBAL VARIABLES"
235 .IX Header "GLOBAL VARIABLES"
236 The behaviour of File::Fetch can be altered by changing the following
237 global variables:
238 .ie n .SS $File::Fetch::FROM_EMAIL
239 .el .SS \f(CW$File::Fetch::FROM_EMAIL\fP
240 .IX Subsection "$File::Fetch::FROM_EMAIL"
241 This is the email address that will be sent as your anonymous ftp
242 password.
243 .PP
244 Default is \f(CW\*(C`File\[email protected]\*(C'\fR.
245 .ie n .SS $File::Fetch::USER_AGENT
246 .el .SS \f(CW$File::Fetch::USER_AGENT\fP
247 .IX Subsection "$File::Fetch::USER_AGENT"
248 This is the useragent as \f(CW\*(C`LWP\*(C'\fR will report it.
249 .PP
250 Default is \f(CW\*(C`File::Fetch/$VERSION\*(C'\fR.
251 .ie n .SS $File::Fetch::FTP_PASSIVE
252 .el .SS \f(CW$File::Fetch::FTP_PASSIVE\fP
253 .IX Subsection "$File::Fetch::FTP_PASSIVE"
254 This variable controls whether the environment variable \f(CW\*(C`FTP_PASSIVE\*(C'\fR
255 and any passive switches to commandline tools will be set to true.
256 .PP
257 Default value is 1.
258 .PP
259 Note: When \f(CW$FTP_PASSIVE\fR is true, \f(CW\*(C`ncftp\*(C'\fR will not be used to fetch
260 files, since passive mode can only be set interactively for this binary
261 .ie n .SS $File::Fetch::TIMEOUT
262 .el .SS \f(CW$File::Fetch::TIMEOUT\fP
263 .IX Subsection "$File::Fetch::TIMEOUT"
264 When set, controls the network timeout (counted in seconds).
265 .PP
266 Default value is 0.
267 .ie n .SS $File::Fetch::WARN
268 .el .SS \f(CW$File::Fetch::WARN\fP
269 .IX Subsection "$File::Fetch::WARN"
270 This variable controls whether errors encountered internally by
271 \&\f(CW\*(C`File::Fetch\*(C'\fR should be \f(CW\*(C`carp\*(C'\fR'd or not.
272 .PP
273 Set to false to silence warnings. Inspect the output of the \f(CWerror()\fR
274 method manually to see what went wrong.
275 .PP
276 Defaults to \f(CW\*(C`true\*(C'\fR.
277 .ie n .SS $File::Fetch::DEBUG
278 .el .SS \f(CW$File::Fetch::DEBUG\fP
279 .IX Subsection "$File::Fetch::DEBUG"
280 This enables debugging output when calling commandline utilities to
281 fetch files.
282 This also enables \f(CW\*(C`Carp::longmess\*(C'\fR errors, instead of the regular
283 \&\f(CW\*(C`carp\*(C'\fR errors.
284 .PP
285 Good for tracking down why things don't work with your particular
286 setup.
287 .PP
288 Default is 0.
289 .ie n .SS $File::Fetch::BLACKLIST
290 .el .SS \f(CW$File::Fetch::BLACKLIST\fP
291 .IX Subsection "$File::Fetch::BLACKLIST"
292 This is an array ref holding blacklisted modules/utilities for fetching
293 files with.
294 .PP
295 To disallow the use of, for example, \f(CW\*(C`LWP\*(C'\fR and \f(CW\*(C`Net::FTP\*(C'\fR, you could
296 set \f(CW$File::Fetch::BLACKLIST\fR to:
297 .PP
298 .Vb 1
299 \& $File::Fetch::BLACKLIST = [qw|lwp netftp|]
300 .Ve
301 .PP
302 The default blacklist is [qw|ftp|], as \f(CW\*(C`/bin/ftp\*(C'\fR is rather unreliable.
303 .PP
304 See the note on \f(CW\*(C`MAPPING\*(C'\fR below.
305 .ie n .SS $File::Fetch::METHOD_FAIL
306 .el .SS \f(CW$File::Fetch::METHOD_FAIL\fP
307 .IX Subsection "$File::Fetch::METHOD_FAIL"
308 This is a hashref registering what modules/utilities were known to fail
309 for fetching files (mostly because they weren't installed).
310 .PP
311 You can reset this cache by assigning an empty hashref to it, or
312 individually remove keys.
313 .PP
314 See the note on \f(CW\*(C`MAPPING\*(C'\fR below.
315 .SH MAPPING
316 .IX Header "MAPPING"
317 Here's a quick mapping for the utilities/modules, and their names for
318 the \f(CW$BLACKLIST\fR, \f(CW$METHOD_FAIL\fR and other internal functions.
319 .PP
320 .Vb 10
321 \& LWP => lwp
322 \& HTTP::Lite => httplite
323 \& HTTP::Tiny => httptiny
324 \& Net::FTP => netftp
325 \& wget => wget
326 \& lynx => lynx
327 \& ncftp => ncftp
328 \& ftp => ftp
329 \& curl => curl
330 \& rsync => rsync
331 \& lftp => lftp
332 \& fetch => fetch
333 \& IO::Socket => iosock
334 .Ve
335 .SH "FREQUENTLY ASKED QUESTIONS"
336 .IX Header "FREQUENTLY ASKED QUESTIONS"
337 .SS "So how do I use a proxy with File::Fetch?"
338 .IX Subsection "So how do I use a proxy with File::Fetch?"
339 \&\f(CW\*(C`File::Fetch\*(C'\fR currently only supports proxies with LWP::UserAgent.
340 You will need to set your environment variables accordingly. For
341 example, to use an ftp proxy:
342 .PP
343 .Vb 1
344 \& $ENV{ftp_proxy} = \*(Aqfoo.com\*(Aq;
345 .Ve
346 .PP
347 Refer to the LWP::UserAgent manpage for more details.
348 .SS "I used 'lynx' to fetch a file, but its contents is all wrong!"
349 .IX Subsection "I used 'lynx' to fetch a file, but its contents is all wrong!"
350 \&\f(CW\*(C`lynx\*(C'\fR can only fetch remote files by dumping its contents to \f(CW\*(C`STDOUT\*(C'\fR,
351 which we in turn capture. If that content is a 'custom' error file
352 (like, say, a \f(CW\*(C`404 handler\*(C'\fR), you will get that contents instead.
353 .PP
354 Sadly, \f(CW\*(C`lynx\*(C'\fR doesn't support any options to return a different exit
355 code on non\-\f(CW\*(C`200 OK\*(C'\fR status, giving us no way to tell the difference
356 between a 'successful' fetch and a custom error page.
357 .PP
358 Therefor, we recommend to only use \f(CW\*(C`lynx\*(C'\fR as a last resort. This is
359 why it is at the back of our list of methods to try as well.
360 .SS "Files I'm trying to fetch have reserved characters or non-ASCII characters in them. What do I do?"
361 .IX Subsection "Files I'm trying to fetch have reserved characters or non-ASCII characters in them. What do I do?"
362 \&\f(CW\*(C`File::Fetch\*(C'\fR is relatively smart about things. When trying to write
363 a file to disk, it removes the \f(CW\*(C`query parameters\*(C'\fR (see the
364 \&\f(CW\*(C`output_file\*(C'\fR method for details) from the file name before creating
365 it. In most cases this suffices.
366 .PP
367 If you have any other characters you need to escape, please install
368 the \f(CW\*(C`URI::Escape\*(C'\fR module from CPAN, and pre-encode your URI before
369 passing it to \f(CW\*(C`File::Fetch\*(C'\fR. You can read about the details of URIs
370 and URI encoding here:
371 .PP
372 <https://datatracker.ietf.org/doc/html/rfc2396>
373 .SH TODO
374 .IX Header "TODO"
375 .ie n .IP "Implement $PREFER_BIN" 4
376 .el .IP "Implement \f(CW$PREFER_BIN\fR" 4
377 .IX Item "Implement $PREFER_BIN"
378 To indicate to rather use commandline tools than modules
379 .SH "BUG REPORTS"
380 .IX Header "BUG REPORTS"
381 Please report bugs or other issues to <bug\-file\[email protected]<gt>.
382 .SH AUTHOR
383 .IX Header "AUTHOR"
384 This module by Jos Boumans <[email protected]>.
385 .SH COPYRIGHT
386 .IX Header "COPYRIGHT"
387 This library is free software; you may redistribute and/or modify it
388 under the same terms as Perl itself.