Debian Codegraph @

upstream/fedora-44/man1/wget1.1

raw
1 .\" -*- mode: troff; coding: utf-8 -*-
2 .\" Automatically generated by Pod::Man v6.0.2 (Pod::Simple 3.47)
3 .\"
4 .\" Standard preamble:
5 .\" ========================================================================
6 .de Sp \" Vertical space (when we can't use .PP)
7 .if t .sp .5v
8 .if n .sp
9 ..
10 .de Vb \" Begin verbatim text
11 .ft CW
12 .nf
13 .ne \\$1
14 ..
15 .de Ve \" End verbatim text
16 .ft R
17 .fi
18 ..
19 .\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>.
20 .ie n \{\
21 . ds C` ""
22 . ds C' ""
23 'br\}
24 .el\{\
25 . ds C`
26 . ds C'
27 'br\}
28 .\"
29 .\" Escape single quotes in literal strings from groff's Unicode transform.
30 .ie \n(.g .ds Aq \(aq
31 .el .ds Aq '
32 .\"
33 .\" If the F register is >0, we'll generate index entries on stderr for
34 .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
35 .\" entries marked with X<> in POD. Of course, you'll have to process the
36 .\" output yourself in some meaningful fashion.
37 .\"
38 .\" Avoid warning from groff about undefined register 'F'.
39 .de IX
40 ..
41 .nr rF 0
42 .if \n(.g .if rF .nr rF 1
43 .if (\n(rF:(\n(.g==0)) \{\
44 . if \nF \{\
45 . de IX
46 . tm Index:\\$1\t\\n%\t"\\$2"
47 ..
48 . if !\nF==2 \{\
49 . nr % 0
50 . nr F 2
51 . \}
52 . \}
53 .\}
54 .rr rF
55 .\"
56 .\" Required to disable full justification in groff 1.23.0.
57 .if n .ds AD l
58 .\" ========================================================================
59 .\"
60 .IX Title "WGET 1"
61 .TH WGET 1 2025-07-25 "GNU Wget 1.25.0" "GNU Wget"
62 .\" For nroff, turn off justification. Always turn off hyphenation; it makes
63 .\" way too many mistakes in technical documents.
64 .if n .ad l
65 .nh
66 .SH NAME
67 Wget \- The non\-interactive network downloader.
68 .SH SYNOPSIS
69 .IX Header "SYNOPSIS"
70 wget [\fIoption\fR]... [\fIURL\fR]...
71 .SH DESCRIPTION
72 .IX Header "DESCRIPTION"
73 GNU Wget is a free utility for non\-interactive download of files from
74 the Web. It supports HTTP, HTTPS, and FTP protocols, as
75 well as retrieval through HTTP proxies.
76 .PP
77 Wget is non\-interactive, meaning that it can work in the background,
78 while the user is not logged on. This allows you to start a retrieval
79 and disconnect from the system, letting Wget finish the work. By
80 contrast, most of the Web browsers require constant user\*(Aqs presence,
81 which can be a great hindrance when transferring a lot of data.
82 .PP
83 Wget can follow links in HTML, XHTML, and CSS pages, to
84 create local versions of remote web sites, fully recreating the
85 directory structure of the original site. This is sometimes referred to
86 as "recursive downloading." While doing that, Wget respects the Robot
87 Exclusion Standard (\fI/robots.txt\fR). Wget can be instructed to
88 convert the links in downloaded files to point at the local files, for
89 offline viewing.
90 .PP
91 Wget has been designed for robustness over slow or unstable network
92 connections; if a download fails due to a network problem, it will
93 keep retrying until the whole file has been retrieved. If the server
94 supports regetting, it will instruct the server to continue the
95 download from where it left off.
96 .SH OPTIONS
97 .IX Header "OPTIONS"
98 .SS "Option Syntax"
99 .IX Subsection "Option Syntax"
100 Since Wget uses GNU getopt to process command\-line arguments, every
101 option has a long form along with the short one. Long options are
102 more convenient to remember, but take time to type. You may freely
103 mix different option styles, or specify options after the command\-line
104 arguments. Thus you may write:
105 .PP
106 .Vb 1
107 \& wget \-r \-\-tries=10 http://fly.srk.fer.hr/ \-o log
108 .Ve
109 .PP
110 The space between the option accepting an argument and the argument may
111 be omitted. Instead of \fB\-o log\fR you can write \fB\-olog\fR.
112 .PP
113 You may put several options that do not require arguments together,
114 like:
115 .PP
116 .Vb 1
117 \& wget \-drc <URL>
118 .Ve
119 .PP
120 This is completely equivalent to:
121 .PP
122 .Vb 1
123 \& wget \-d \-r \-c <URL>
124 .Ve
125 .PP
126 Since the options can be specified after the arguments, you may
127 terminate them with \fB\-\-\fR. So the following will try to download
128 URL \fB\-x\fR, reporting failure to \fIlog\fR:
129 .PP
130 .Vb 1
131 \& wget \-o log \-\- \-x
132 .Ve
133 .PP
134 The options that accept comma\-separated lists all respect the convention
135 that specifying an empty list clears its value. This can be useful to
136 clear the \fI.wgetrc\fR settings. For instance, if your \fI.wgetrc\fR
137 sets \f(CW\*(C`exclude_directories\*(C'\fR to \fI/cgi\-bin\fR, the following
138 example will first reset it, and then set it to exclude \fI/~nobody\fR
139 and \fI/~somebody\fR. You can also clear the lists in \fI.wgetrc\fR.
140 .PP
141 .Vb 1
142 \& wget \-X "" \-X /~nobody,/~somebody
143 .Ve
144 .PP
145 Most options that do not accept arguments are \fIboolean\fR options,
146 so named because their state can be captured with a yes\-or\-no
147 ("boolean") variable. For example, \fB\-\-follow\-ftp\fR tells Wget
148 to follow FTP links from HTML files and, on the other hand,
149 \&\fB\-\-no\-glob\fR tells it not to perform file globbing on FTP URLs. A
150 boolean option is either \fIaffirmative\fR or \fInegative\fR
151 (beginning with \fB\-\-no\fR). All such options share several
152 properties.
153 .PP
154 Unless stated otherwise, it is assumed that the default behavior is
155 the opposite of what the option accomplishes. For example, the
156 documented existence of \fB\-\-follow\-ftp\fR assumes that the default
157 is to \fInot\fR follow FTP links from HTML pages.
158 .PP
159 Affirmative options can be negated by prepending the \fB\-\-no\-\fR to
160 the option name; negative options can be negated by omitting the
161 \&\fB\-\-no\-\fR prefix. This might seem superfluous\-\-\-if the default for
162 an affirmative option is to not do something, then why provide a way
163 to explicitly turn it off? But the startup file may in fact change
164 the default. For instance, using \f(CW\*(C`follow_ftp = on\*(C'\fR in
165 \&\fI.wgetrc\fR makes Wget \fIfollow\fR FTP links by default, and
166 using \fB\-\-no\-follow\-ftp\fR is the only way to restore the factory
167 default from the command line.
168 .SS "Basic Startup Options"
169 .IX Subsection "Basic Startup Options"
170 .IP \fB\-V\fR 4
171 .IX Item "-V"
172 .PD 0
173 .IP \fB\-\-version\fR 4
174 .IX Item "--version"
175 .PD
176 Display the version of Wget.
177 .IP \fB\-h\fR 4
178 .IX Item "-h"
179 .PD 0
180 .IP \fB\-\-help\fR 4
181 .IX Item "--help"
182 .PD
183 Print a help message describing all of Wget\*(Aqs command\-line options.
184 .IP \fB\-b\fR 4
185 .IX Item "-b"
186 .PD 0
187 .IP \fB\-\-background\fR 4
188 .IX Item "--background"
189 .PD
190 Go to background immediately after startup. If no output file is
191 specified via the \fB\-o\fR, output is redirected to \fIwget\-log\fR.
192 .IP "\fB\-e\fR \fIcommand\fR" 4
193 .IX Item "-e command"
194 .PD 0
195 .IP "\fB\-\-execute\fR \fIcommand\fR" 4
196 .IX Item "--execute command"
197 .PD
198 Execute \fIcommand\fR as if it were a part of \fI.wgetrc\fR. A command thus invoked will be executed
199 \&\fIafter\fR the commands in \fI.wgetrc\fR, thus taking precedence over
200 them. If you need to specify more than one wgetrc command, use multiple
201 instances of \fB\-e\fR.
202 .SS "Logging and Input File Options"
203 .IX Subsection "Logging and Input File Options"
204 .IP "\fB\-o\fR \fIlogfile\fR" 4
205 .IX Item "-o logfile"
206 .PD 0
207 .IP \fB\-\-output\-file=\fR\fIlogfile\fR 4
208 .IX Item "--output-file=logfile"
209 .PD
210 Log all messages to \fIlogfile\fR. The messages are normally reported
211 to standard error.
212 .IP "\fB\-a\fR \fIlogfile\fR" 4
213 .IX Item "-a logfile"
214 .PD 0
215 .IP \fB\-\-append\-output=\fR\fIlogfile\fR 4
216 .IX Item "--append-output=logfile"
217 .PD
218 Append to \fIlogfile\fR. This is the same as \fB\-o\fR, only it appends
219 to \fIlogfile\fR instead of overwriting the old log file. If
220 \&\fIlogfile\fR does not exist, a new file is created.
221 .IP \fB\-d\fR 4
222 .IX Item "-d"
223 .PD 0
224 .IP \fB\-\-debug\fR 4
225 .IX Item "--debug"
226 .PD
227 Turn on debug output, meaning various information important to the
228 developers of Wget if it does not work properly. Your system
229 administrator may have chosen to compile Wget without debug support, in
230 which case \fB\-d\fR will not work. Please note that compiling with
231 debug support is always safe\-\-\-Wget compiled with the debug support will
232 \&\fInot\fR print any debug info unless requested with \fB\-d\fR.
233 .IP \fB\-q\fR 4
234 .IX Item "-q"
235 .PD 0
236 .IP \fB\-\-quiet\fR 4
237 .IX Item "--quiet"
238 .PD
239 Turn off Wget\*(Aqs output.
240 .IP \fB\-v\fR 4
241 .IX Item "-v"
242 .PD 0
243 .IP \fB\-\-verbose\fR 4
244 .IX Item "--verbose"
245 .PD
246 Turn on verbose output, with all the available data. The default output
247 is verbose.
248 .IP \fB\-nv\fR 4
249 .IX Item "-nv"
250 .PD 0
251 .IP \fB\-\-no\-verbose\fR 4
252 .IX Item "--no-verbose"
253 .PD
254 Turn off verbose without being completely quiet (use \fB\-q\fR for
255 that), which means that error messages and basic information still get
256 printed.
257 .IP \fB\-\-report\-speed=\fR\fItype\fR 4
258 .IX Item "--report-speed=type"
259 Output bandwidth as \fItype\fR. The only accepted value is \fBbits\fR.
260 .IP "\fB\-i\fR \fIfile\fR" 4
261 .IX Item "-i file"
262 .PD 0
263 .IP \fB\-\-input\-file=\fR\fIfile\fR 4
264 .IX Item "--input-file=file"
265 .PD
266 Read URLs from a local or external \fIfile\fR. If \fB\-\fR is
267 specified as \fIfile\fR, URLs are read from the standard input.
268 (Use \fB./\-\fR to read from a file literally named \fB\-\fR.)
269 .Sp
270 If this function is used, no URLs need be present on the command
271 line. If there are URLs both on the command line and in an input
272 file, those on the command lines will be the first ones to be
273 retrieved. If \fB\-\-force\-html\fR is not specified, then \fIfile\fR
274 should consist of a series of URLs, one per line.
275 .Sp
276 However, if you specify \fB\-\-force\-html\fR, the document will be
277 regarded as \fBhtml\fR. In that case you may have problems with
278 relative links, which you can solve either by adding \f(CW\*(C`<base
279 href="\fR\f(CIurl\fR\f(CW">\*(C'\fR to the documents or by specifying
280 \&\fB\-\-base=\fR\fIurl\fR on the command line.
281 .Sp
282 If the \fIfile\fR is an external one, the document will be automatically
283 treated as \fBhtml\fR if the Content\-Type matches \fBtext/html\fR.
284 Furthermore, the \fIfile\fR\*(Aqs location will be implicitly used as base
285 href if none was specified.
286 .Sp
287 If the \fIfile\fR is a local file, on systems that support it, it will be
288 opened with the \f(CW\*(C`O_NONBLOCK\*(C'\fR flag to allow non\-blocking reads from the
289 file. \fBWget\fR will attempt to continue reading from the file until
290 EOF is reached or it is closed. This allows one to stream a list of files
291 for retrieval where the list is dynamically generated during the execution
292 of the process. Currently, this feature is not available on Windows platforms.
293 .IP \fB\-\-input\-metalink=\fR\fIfile\fR 4
294 .IX Item "--input-metalink=file"
295 Downloads files covered in local Metalink \fIfile\fR. Metalink version 3
296 and 4 are supported.
297 .IP \fB\-\-keep\-badhash\fR 4
298 .IX Item "--keep-badhash"
299 Keeps downloaded Metalink\*(Aqs files with a bad hash. It appends .badhash
300 to the name of Metalink\*(Aqs files which have a checksum mismatch, except
301 without overwriting existing files.
302 .IP \fB\-\-metalink\-over\-http\fR 4
303 .IX Item "--metalink-over-http"
304 Issues HTTP HEAD request instead of GET and extracts Metalink metadata
305 from response headers. Then it switches to Metalink download.
306 If no valid Metalink metadata is found, it falls back to ordinary HTTP download.
307 Enables \fBContent\-Type: application/metalink4+xml\fR files download/processing.
308 .IP \fB\-\-metalink\-index=\fR\fInumber\fR 4
309 .IX Item "--metalink-index=number"
310 Set the Metalink \fBapplication/metalink4+xml\fR metaurl ordinal
311 NUMBER. From 1 to the total number of "application/metalink4+xml"
312 available. Specify 0 or \fBinf\fR to choose the first good one.
313 Metaurls, such as those from a \fB\-\-metalink\-over\-http\fR, may have
314 been sorted by priority key\*(Aqs value; keep this in mind to choose the
315 right NUMBER.
316 .IP \fB\-\-preferred\-location\fR 4
317 .IX Item "--preferred-location"
318 Set preferred location for Metalink resources. This has effect if multiple
319 resources with same priority are available.
320 .IP \fB\-\-xattr\fR 4
321 .IX Item "--xattr"
322 Enable use of file system\*(Aqs extended attributes to save the
323 original URL and the Referer HTTP header value if used.
324 .Sp
325 Be aware that the URL might contain private information like
326 access tokens or credentials.
327 .IP \fB\-F\fR 4
328 .IX Item "-F"
329 .PD 0
330 .IP \fB\-\-force\-html\fR 4
331 .IX Item "--force-html"
332 .PD
333 When input is read from a file, force it to be treated as an HTML
334 file. This enables you to retrieve relative links from existing
335 HTML files on your local disk, by adding \f(CW\*(C`<base
336 href="\fR\f(CIurl\fR\f(CW">\*(C'\fR to HTML, or using the \fB\-\-base\fR command\-line
337 option.
338 .IP "\fB\-B\fR \fIURL\fR" 4
339 .IX Item "-B URL"
340 .PD 0
341 .IP \fB\-\-base=\fR\fIURL\fR 4
342 .IX Item "--base=URL"
343 .PD
344 Resolves relative links using \fIURL\fR as the point of reference,
345 when reading links from an HTML file specified via the
346 \&\fB\-i\fR/\fB\-\-input\-file\fR option (together with
347 \&\fB\-\-force\-html\fR, or when the input file was fetched remotely from
348 a server describing it as HTML). This is equivalent to the
349 presence of a \f(CW\*(C`BASE\*(C'\fR tag in the HTML input file, with
350 \&\fIURL\fR as the value for the \f(CW\*(C`href\*(C'\fR attribute.
351 .Sp
352 For instance, if you specify \fBhttp://foo/bar/a.html\fR for
353 \&\fIURL\fR, and Wget reads \fB../baz/b.html\fR from the input file, it
354 would be resolved to \fBhttp://foo/baz/b.html\fR.
355 .IP \fB\-\-config=\fR\fIFILE\fR 4
356 .IX Item "--config=FILE"
357 Specify the location of a startup file you wish to use instead of the
358 default one(s). Use \-\-no\-config to disable reading of config files.
359 If both \-\-config and \-\-no\-config are given, \-\-no\-config is ignored.
360 .IP \fB\-\-rejected\-log=\fR\fIlogfile\fR 4
361 .IX Item "--rejected-log=logfile"
362 Logs all URL rejections to \fIlogfile\fR as comma separated values. The values
363 include the reason of rejection, the URL and the parent URL it was found in.
364 .SS "Download Options"
365 .IX Subsection "Download Options"
366 .IP \fB\-\-bind\-address=\fR\fIADDRESS\fR 4
367 .IX Item "--bind-address=ADDRESS"
368 When making client TCP/IP connections, bind to \fIADDRESS\fR on
369 the local machine. \fIADDRESS\fR may be specified as a hostname or IP
370 address. This option can be useful if your machine is bound to multiple
371 IPs.
372 .IP \fB\-\-bind\-dns\-address=\fR\fIADDRESS\fR 4
373 .IX Item "--bind-dns-address=ADDRESS"
374 [libcares only]
375 This address overrides the route for DNS requests. If you ever need to
376 circumvent the standard settings from /etc/resolv.conf, this option together
377 with \fB\-\-dns\-servers\fR is your friend.
378 \&\fIADDRESS\fR must be specified either as IPv4 or IPv6 address.
379 Wget needs to be built with libcares for this option to be available.
380 .IP \fB\-\-dns\-servers=\fR\fIADDRESSES\fR 4
381 .IX Item "--dns-servers=ADDRESSES"
382 [libcares only]
383 The given address(es) override the standard nameserver
384 addresses, e.g. as configured in /etc/resolv.conf.
385 \&\fIADDRESSES\fR may be specified either as IPv4 or IPv6 addresses,
386 comma\-separated.
387 Wget needs to be built with libcares for this option to be available.
388 .IP "\fB\-t\fR \fInumber\fR" 4
389 .IX Item "-t number"
390 .PD 0
391 .IP \fB\-\-tries=\fR\fInumber\fR 4
392 .IX Item "--tries=number"
393 .PD
394 Set number of tries to \fInumber\fR. Specify 0 or \fBinf\fR for
395 infinite retrying. The default is to retry 20 times, with the exception
396 of fatal errors like "connection refused" or "not found" (404),
397 which are not retried.
398 .IP "\fB\-O\fR \fIfile\fR" 4
399 .IX Item "-O file"
400 .PD 0
401 .IP \fB\-\-output\-document=\fR\fIfile\fR 4
402 .IX Item "--output-document=file"
403 .PD
404 The documents will not be written to the appropriate files, but all
405 will be concatenated together and written to \fIfile\fR. If \fB\-\fR
406 is used as \fIfile\fR, documents will be printed to standard output,
407 disabling link conversion. (Use \fB./\-\fR to print to a file
408 literally named \fB\-\fR.)
409 .Sp
410 Use of \fB\-O\fR is \fInot\fR intended to mean simply "use the name
411 \&\fIfile\fR instead of the one in the URL;" rather, it is
412 analogous to shell redirection:
413 \&\fBwget \-O file http://foo\fR is intended to work like
414 \&\fBwget \-O \- http://foo > file\fR; \fIfile\fR will be truncated
415 immediately, and \fIall\fR downloaded content will be written there.
416 .Sp
417 For this reason, \fB\-N\fR (for timestamp\-checking) is not supported
418 in combination with \fB\-O\fR: since \fIfile\fR is always newly
419 created, it will always have a very new timestamp. A warning will be
420 issued if this combination is used.
421 .Sp
422 Similarly, using \fB\-r\fR or \fB\-p\fR with \fB\-O\fR may not work as
423 you expect: Wget won\*(Aqt just download the first file to \fIfile\fR and
424 then download the rest to their normal names: \fIall\fR downloaded
425 content will be placed in \fIfile\fR. This was disabled in version
426 1.11, but has been reinstated (with a warning) in 1.11.2, as there are
427 some cases where this behavior can actually have some use.
428 .Sp
429 A combination with \fB\-nc\fR is only accepted if the given output
430 file does not exist.
431 .Sp
432 Note that a combination with \fB\-k\fR is only permitted when
433 downloading a single document, as in that case it will just convert
434 all relative URIs to external ones; \fB\-k\fR makes no sense for
435 multiple URIs when they\*(Aqre all being downloaded to a single file;
436 \&\fB\-k\fR can be used only when the output is a regular file.
437 .IP \fB\-nc\fR 4
438 .IX Item "-nc"
439 .PD 0
440 .IP \fB\-\-no\-clobber\fR 4
441 .IX Item "--no-clobber"
442 .PD
443 If a file is downloaded more than once in the same directory, Wget\*(Aqs
444 behavior depends on a few options, including \fB\-nc\fR. In certain
445 cases, the local file will be \fIclobbered\fR, or overwritten, upon
446 repeated download. In other cases it will be preserved.
447 .Sp
448 When running Wget without \fB\-N\fR, \fB\-nc\fR, \fB\-r\fR, or
449 \&\fB\-p\fR, downloading the same file in the same directory will result
450 in the original copy of \fIfile\fR being preserved and the second copy
451 being named \fIfile\fR\fB.1\fR. If that file is downloaded yet
452 again, the third copy will be named \fIfile\fR\fB.2\fR, and so on.
453 (This is also the behavior with \fB\-nd\fR, even if \fB\-r\fR or
454 \&\fB\-p\fR are in effect.) When \fB\-nc\fR is specified, this behavior
455 is suppressed, and Wget will refuse to download newer copies of
456 \&\fIfile\fR. Therefore, "\f(CW\*(C`no\-clobber\*(C'\fR" is actually a
457 misnomer in this mode\-\-\-it\*(Aqs not clobbering that\*(Aqs prevented (as the
458 numeric suffixes were already preventing clobbering), but rather the
459 multiple version saving that\*(Aqs prevented.
460 .Sp
461 When running Wget with \fB\-r\fR or \fB\-p\fR, but without \fB\-N\fR,
462 \&\fB\-nd\fR, or \fB\-nc\fR, re\-downloading a file will result in the
463 new copy simply overwriting the old. Adding \fB\-nc\fR will prevent
464 this behavior, instead causing the original version to be preserved
465 and any newer copies on the server to be ignored.
466 .Sp
467 When running Wget with \fB\-N\fR, with or without \fB\-r\fR or
468 \&\fB\-p\fR, the decision as to whether or not to download a newer copy
469 of a file depends on the local and remote timestamp and size of the
470 file. \fB\-nc\fR may not be specified at the
471 same time as \fB\-N\fR.
472 .Sp
473 A combination with \fB\-O\fR/\fB\-\-output\-document\fR is only accepted
474 if the given output file does not exist.
475 .Sp
476 Note that when \fB\-nc\fR is specified, files with the suffixes
477 \&\fB.html\fR or \fB.htm\fR will be loaded from the local disk and
478 parsed as if they had been retrieved from the Web.
479 .IP \fB\-\-backups=\fR\fIbackups\fR 4
480 .IX Item "--backups=backups"
481 Before (over)writing a file, back up an existing file by adding a
482 \&\fB.1\fR suffix (\fB_1\fR on VMS) to the file name. Such backup
483 files are rotated to \fB.2\fR, \fB.3\fR, and so on, up to
484 \&\fIbackups\fR (and lost beyond that).
485 .IP \fB\-\-no\-netrc\fR 4
486 .IX Item "--no-netrc"
487 Do not try to obtain credentials from \fI.netrc\fR file. By default
488 \&\fI.netrc\fR file is searched for credentials in case none have been
489 passed on command line and authentication is required.
490 .IP \fB\-c\fR 4
491 .IX Item "-c"
492 .PD 0
493 .IP \fB\-\-continue\fR 4
494 .IX Item "--continue"
495 .PD
496 Continue getting a partially\-downloaded file. This is useful when you
497 want to finish up a download started by a previous instance of Wget, or
498 by another program. For instance:
499 .Sp
500 .Vb 1
501 \& wget \-c ftp://sunsite.doc.ic.ac.uk/ls\-lR.Z
502 .Ve
503 .Sp
504 If there is a file named \fIls\-lR.Z\fR in the current directory, Wget
505 will assume that it is the first portion of the remote file, and will
506 ask the server to continue the retrieval from an offset equal to the
507 length of the local file.
508 .Sp
509 Note that you don\*(Aqt need to specify this option if you just want the
510 current invocation of Wget to retry downloading a file should the
511 connection be lost midway through. This is the default behavior.
512 \&\fB\-c\fR only affects resumption of downloads started \fIprior\fR to
513 this invocation of Wget, and whose local files are still sitting around.
514 .Sp
515 Without \fB\-c\fR, the previous example would just download the remote
516 file to \fIls\-lR.Z.1\fR, leaving the truncated \fIls\-lR.Z\fR file
517 alone.
518 .Sp
519 If you use \fB\-c\fR on a non\-empty file, and the server does not support
520 continued downloading, Wget will restart the download from scratch and overwrite
521 the existing file entirely.
522 .Sp
523 Beginning with Wget 1.7, if you use \fB\-c\fR on a file which is of
524 equal size as the one on the server, Wget will refuse to download the
525 file and print an explanatory message. The same happens when the file
526 is smaller on the server than locally (presumably because it was changed
527 on the server since your last download attempt)\-\-\-because "continuing"
528 is not meaningful, no download occurs.
529 .Sp
530 On the other side of the coin, while using \fB\-c\fR, any file that\*(Aqs
531 bigger on the server than locally will be considered an incomplete
532 download and only \f(CW\*(C`(length(remote) \- length(local))\*(C'\fR bytes will be
533 downloaded and tacked onto the end of the local file. This behavior can
534 be desirable in certain cases\-\-\-for instance, you can use \fBwget \-c\fR
535 to download just the new portion that\*(Aqs been appended to a data
536 collection or log file.
537 .Sp
538 However, if the file is bigger on the server because it\*(Aqs been
539 \&\fIchanged\fR, as opposed to just \fIappended\fR to, you\*(Aqll end up
540 with a garbled file. Wget has no way of verifying that the local file
541 is really a valid prefix of the remote file. You need to be especially
542 careful of this when using \fB\-c\fR in conjunction with \fB\-r\fR,
543 since every file will be considered as an "incomplete download" candidate.
544 .Sp
545 Another instance where you\*(Aqll get a garbled file if you try to use
546 \&\fB\-c\fR is if you have a lame HTTP proxy that inserts a
547 "transfer interrupted" string into the local file. In the future a
548 "rollback" option may be added to deal with this case.
549 .Sp
550 Note that \fB\-c\fR only works with FTP servers and with HTTP
551 servers that support the \f(CW\*(C`Range\*(C'\fR header.
552 .IP \fB\-\-start\-pos=\fR\fIOFFSET\fR 4
553 .IX Item "--start-pos=OFFSET"
554 Start downloading at zero\-based position \fIOFFSET\fR. Offset may be expressed
555 in bytes, kilobytes with the \`k\*(Aq suffix, or megabytes with the \`m\*(Aq suffix, etc.
556 .Sp
557 \&\fB\-\-start\-pos\fR has higher precedence over \fB\-\-continue\fR. When
558 \&\fB\-\-start\-pos\fR and \fB\-\-continue\fR are both specified, wget will emit a
559 warning then proceed as if \fB\-\-continue\fR was absent.
560 .Sp
561 Server support for continued download is required, otherwise \fB\-\-start\-pos\fR
562 cannot help. See \fB\-c\fR for details.
563 .IP \fB\-\-progress=\fR\fItype\fR 4
564 .IX Item "--progress=type"
565 Select the type of the progress indicator you wish to use. Legal
566 indicators are "dot" and "bar".
567 .Sp
568 The "bar" indicator is used by default. It draws an ASCII progress
569 bar graphics (a.k.a "thermometer" display) indicating the status of
570 retrieval. If the output is not a TTY, the "dot" bar will be used by
571 default.
572 .Sp
573 Use \fB\-\-progress=dot\fR to switch to the "dot" display. It traces
574 the retrieval by printing dots on the screen, each dot representing a
575 fixed amount of downloaded data.
576 .Sp
577 The progress \fItype\fR can also take one or more parameters. The parameters
578 vary based on the \fItype\fR selected. Parameters to \fItype\fR are passed by
579 appending them to the type sperated by a colon (:) like this:
580 \&\fB\-\-progress=\fR\fItype\fR\fB:\fR\fIparameter1\fR\fB:\fR\fIparameter2\fR.
581 .Sp
582 When using the dotted retrieval, you may set the \fIstyle\fR by
583 specifying the type as \fBdot:\fR\fIstyle\fR. Different styles assign
584 different meaning to one dot. With the \f(CW\*(C`default\*(C'\fR style each dot
585 represents 1K, there are ten dots in a cluster and 50 dots in a line.
586 The \f(CW\*(C`binary\*(C'\fR style has a more "computer"\-like orientation\-\-\-8K
587 dots, 16\-dots clusters and 48 dots per line (which makes for 384K
588 lines). The \f(CW\*(C`mega\*(C'\fR style is suitable for downloading large
589 files\-\-\-each dot represents 64K retrieved, there are eight dots in a
590 cluster, and 48 dots on each line (so each line contains 3M).
591 If \f(CW\*(C`mega\*(C'\fR is not enough then you can use the \f(CW\*(C`giga\*(C'\fR
592 style\-\-\-each dot represents 1M retrieved, there are eight dots in a
593 cluster, and 32 dots on each line (so each line contains 32M).
594 .Sp
595 With \fB\-\-progress=bar\fR, there are currently two possible parameters,
596 \&\fIforce\fR and \fInoscroll\fR.
597 .Sp
598 When the output is not a TTY, the progress bar always falls back to "dot",
599 even if \fB\-\-progress=bar\fR was passed to Wget during invocation. This
600 behaviour can be overridden and the "bar" output forced by using the "force"
601 parameter as \fB\-\-progress=bar:force\fR.
602 .Sp
603 By default, the \fBbar\fR style progress bar scroll the name of the file from
604 left to right for the file being downloaded if the filename exceeds the maximum
605 length allotted for its display. In certain cases, such as with
606 \&\fB\-\-progress=bar:force\fR, one may not want the scrolling filename in the
607 progress bar. By passing the "noscroll" parameter, Wget can be forced to
608 display as much of the filename as possible without scrolling through it.
609 .Sp
610 Note that you can set the default style using the \f(CW\*(C`progress\*(C'\fR
611 command in \fI.wgetrc\fR. That setting may be overridden from the
612 command line. For example, to force the bar output without scrolling,
613 use \fB\-\-progress=bar:force:noscroll\fR.
614 .IP \fB\-\-show\-progress\fR 4
615 .IX Item "--show-progress"
616 Force wget to display the progress bar in any verbosity.
617 .Sp
618 By default, wget only displays the progress bar in verbose mode. One may
619 however, want wget to display the progress bar on screen in conjunction with
620 any other verbosity modes like \fB\-\-no\-verbose\fR or \fB\-\-quiet\fR. This
621 is often a desired a property when invoking wget to download several small/large
622 files. In such a case, wget could simply be invoked with this parameter to get
623 a much cleaner output on the screen.
624 .Sp
625 This option will also force the progress bar to be printed to \fIstderr\fR when
626 used alongside the \fB\-\-output\-file\fR option.
627 .IP \fB\-N\fR 4
628 .IX Item "-N"
629 .PD 0
630 .IP \fB\-\-timestamping\fR 4
631 .IX Item "--timestamping"
632 .PD
633 Turn on time\-stamping.
634 .IP \fB\-\-no\-if\-modified\-since\fR 4
635 .IX Item "--no-if-modified-since"
636 Do not send If\-Modified\-Since header in \fB\-N\fR mode. Send preliminary HEAD
637 request instead. This has only effect in \fB\-N\fR mode.
638 .IP \fB\-\-no\-use\-server\-timestamps\fR 4
639 .IX Item "--no-use-server-timestamps"
640 Don\*(Aqt set the local file\*(Aqs timestamp by the one on the server.
641 .Sp
642 By default, when a file is downloaded, its timestamps are set to
643 match those from the remote file. This allows the use of
644 \&\fB\-\-timestamping\fR on subsequent invocations of wget. However, it
645 is sometimes useful to base the local file\*(Aqs timestamp on when it was
646 actually downloaded; for that purpose, the
647 \&\fB\-\-no\-use\-server\-timestamps\fR option has been provided.
648 .IP \fB\-S\fR 4
649 .IX Item "-S"
650 .PD 0
651 .IP \fB\-\-server\-response\fR 4
652 .IX Item "--server-response"
653 .PD
654 Print the headers sent by HTTP servers and responses sent by
655 FTP servers.
656 .IP \fB\-\-spider\fR 4
657 .IX Item "--spider"
658 When invoked with this option, Wget will behave as a Web \fIspider\fR,
659 which means that it will not download the pages, just check that they
660 are there. For example, you can use Wget to check your bookmarks:
661 .Sp
662 .Vb 1
663 \& wget \-\-spider \-\-force\-html \-i bookmarks.html
664 .Ve
665 .Sp
666 This feature needs much more work for Wget to get close to the
667 functionality of real web spiders.
668 .IP "\fB\-T seconds\fR" 4
669 .IX Item "-T seconds"
670 .PD 0
671 .IP \fB\-\-timeout=\fR\fIseconds\fR 4
672 .IX Item "--timeout=seconds"
673 .PD
674 Set the network timeout to \fIseconds\fR seconds. This is equivalent
675 to specifying \fB\-\-dns\-timeout\fR, \fB\-\-connect\-timeout\fR, and
676 \&\fB\-\-read\-timeout\fR, all at the same time.
677 .Sp
678 When interacting with the network, Wget can check for timeout and
679 abort the operation if it takes too long. This prevents anomalies
680 like hanging reads and infinite connects. The only timeout enabled by
681 default is a 900\-second read timeout. Setting a timeout to 0 disables
682 it altogether. Unless you know what you are doing, it is best not to
683 change the default timeout settings.
684 .Sp
685 All timeout\-related options accept decimal values, as well as
686 subsecond values. For example, \fB0.1\fR seconds is a legal (though
687 unwise) choice of timeout. Subsecond timeouts are useful for checking
688 server response times or for testing network latency.
689 .IP \fB\-\-dns\-timeout=\fR\fIseconds\fR 4
690 .IX Item "--dns-timeout=seconds"
691 Set the DNS lookup timeout to \fIseconds\fR seconds. DNS lookups that
692 don\*(Aqt complete within the specified time will fail. By default, there
693 is no timeout on DNS lookups, other than that implemented by system
694 libraries.
695 .IP \fB\-\-connect\-timeout=\fR\fIseconds\fR 4
696 .IX Item "--connect-timeout=seconds"
697 Set the connect timeout to \fIseconds\fR seconds. TCP connections that
698 take longer to establish will be aborted. By default, there is no
699 connect timeout, other than that implemented by system libraries.
700 .IP \fB\-\-read\-timeout=\fR\fIseconds\fR 4
701 .IX Item "--read-timeout=seconds"
702 Set the read (and write) timeout to \fIseconds\fR seconds. The
703 "time" of this timeout refers to \fIidle time\fR: if, at any point in
704 the download, no data is received for more than the specified number
705 of seconds, reading fails and the download is restarted. This option
706 does not directly affect the duration of the entire download.
707 .Sp
708 Of course, the remote server may choose to terminate the connection
709 sooner than this option requires. The default read timeout is 900
710 seconds.
711 .IP \fB\-\-limit\-rate=\fR\fIamount\fR 4
712 .IX Item "--limit-rate=amount"
713 Limit the download speed to \fIamount\fR bytes per second. Amount may
714 be expressed in bytes, kilobytes with the \fBk\fR suffix, or megabytes
715 with the \fBm\fR suffix. For example, \fB\-\-limit\-rate=20k\fR will
716 limit the retrieval rate to 20KB/s. This is useful when, for whatever
717 reason, you don\*(Aqt want Wget to consume the entire available bandwidth.
718 .Sp
719 This option allows the use of decimal numbers, usually in conjunction
720 with power suffixes; for example, \fB\-\-limit\-rate=2.5k\fR is a legal
721 value.
722 .Sp
723 Note that Wget implements the limiting by sleeping the appropriate
724 amount of time after a network read that took less time than specified
725 by the rate. Eventually this strategy causes the TCP transfer to slow
726 down to approximately the specified rate. However, it may take some
727 time for this balance to be achieved, so don\*(Aqt be surprised if limiting
728 the rate doesn\*(Aqt work well with very small files.
729 .IP "\fB\-w\fR \fIseconds\fR" 4
730 .IX Item "-w seconds"
731 .PD 0
732 .IP \fB\-\-wait=\fR\fIseconds\fR 4
733 .IX Item "--wait=seconds"
734 .PD
735 Wait the specified number of seconds between the retrievals. Use of
736 this option is recommended, as it lightens the server load by making the
737 requests less frequent. Instead of in seconds, the time can be
738 specified in minutes using the \f(CW\*(C`m\*(C'\fR suffix, in hours using \f(CW\*(C`h\*(C'\fR
739 suffix, or in days using \f(CW\*(C`d\*(C'\fR suffix.
740 .Sp
741 Specifying a large value for this option is useful if the network or the
742 destination host is down, so that Wget can wait long enough to
743 reasonably expect the network error to be fixed before the retry. The
744 waiting interval specified by this function is influenced by
745 \&\f(CW\*(C`\-\-random\-wait\*(C'\fR, which see.
746 .IP \fB\-\-waitretry=\fR\fIseconds\fR 4
747 .IX Item "--waitretry=seconds"
748 If you don\*(Aqt want Wget to wait between \fIevery\fR retrieval, but only
749 between retries of failed downloads, you can use this option. Wget will
750 use \fIlinear backoff\fR, waiting 1 second after the first failure on a
751 given file, then waiting 2 seconds after the second failure on that
752 file, up to the maximum number of \fIseconds\fR you specify.
753 .Sp
754 By default, Wget will assume a value of 10 seconds.
755 .IP \fB\-\-random\-wait\fR 4
756 .IX Item "--random-wait"
757 Some web sites may perform log analysis to identify retrieval programs
758 such as Wget by looking for statistically significant similarities in
759 the time between requests. This option causes the time between requests
760 to vary between 0.5 and 1.5 * \fIwait\fR seconds, where \fIwait\fR was
761 specified using the \fB\-\-wait\fR option, in order to mask Wget\*(Aqs
762 presence from such analysis.
763 .Sp
764 A 2001 article in a publication devoted to development on a popular
765 consumer platform provided code to perform this analysis on the fly.
766 Its author suggested blocking at the class C address level to ensure
767 automated retrieval programs were blocked despite changing DHCP\-supplied
768 addresses.
769 .Sp
770 The \fB\-\-random\-wait\fR option was inspired by this ill\-advised
771 recommendation to block many unrelated users from a web site due to the
772 actions of one.
773 .IP \fB\-\-no\-proxy\fR 4
774 .IX Item "--no-proxy"
775 Don\*(Aqt use proxies, even if the appropriate \f(CW*_proxy\fR environment
776 variable is defined.
777 .IP "\fB\-Q\fR \fIquota\fR" 4
778 .IX Item "-Q quota"
779 .PD 0
780 .IP \fB\-\-quota=\fR\fIquota\fR 4
781 .IX Item "--quota=quota"
782 .PD
783 Specify download quota for automatic retrievals. The value can be
784 specified in bytes (default), kilobytes (with \fBk\fR suffix), or
785 megabytes (with \fBm\fR suffix).
786 .Sp
787 Note that quota will never affect downloading a single file. So if you
788 specify \fBwget \-Q10k https://example.com/ls\-lR.gz\fR, all of the
789 \&\fIls\-lR.gz\fR will be downloaded. The same goes even when several
790 URLs are specified on the command\-line. The quota is checked only
791 at the end of each downloaded file, so it will never result in a partially
792 downloaded file. Thus you may safely type \fBwget \-Q2m \-i sites\fR\-\-\-download
793 will be aborted after the file that exhausts the quota is completely
794 downloaded.
795 .Sp
796 Setting quota to 0 or to \fBinf\fR unlimits the download quota.
797 .IP \fB\-\-no\-dns\-cache\fR 4
798 .IX Item "--no-dns-cache"
799 Turn off caching of DNS lookups. Normally, Wget remembers the IP
800 addresses it looked up from DNS so it doesn\*(Aqt have to repeatedly
801 contact the DNS server for the same (typically small) set of hosts it
802 retrieves from. This cache exists in memory only; a new Wget run will
803 contact DNS again.
804 .Sp
805 However, it has been reported that in some situations it is not
806 desirable to cache host names, even for the duration of a
807 short\-running application like Wget. With this option Wget issues a
808 new DNS lookup (more precisely, a new call to \f(CW\*(C`gethostbyname\*(C'\fR or
809 \&\f(CW\*(C`getaddrinfo\*(C'\fR) each time it makes a new connection. Please note
810 that this option will \fInot\fR affect caching that might be
811 performed by the resolving library or by an external caching layer,
812 such as NSCD.
813 .Sp
814 If you don\*(Aqt understand exactly what this option does, you probably
815 won\*(Aqt need it.
816 .IP \fB\-\-restrict\-file\-names=\fR\fImodes\fR 4
817 .IX Item "--restrict-file-names=modes"
818 Change which characters found in remote URLs must be escaped during
819 generation of local filenames. Characters that are \fIrestricted\fR
820 by this option are escaped, i.e. replaced with \fR\f(CB%HH\fR\fB\fR, where
821 \&\fBHH\fR is the hexadecimal number that corresponds to the restricted
822 character. This option may also be used to force all alphabetical
823 cases to be either lower\- or uppercase.
824 .Sp
825 By default, Wget escapes the characters that are not valid or safe as
826 part of file names on your operating system, as well as control
827 characters that are typically unprintable. This option is useful for
828 changing these defaults, perhaps because you are downloading to a
829 non\-native partition, or because you want to disable escaping of the
830 control characters, or you want to further restrict characters to only
831 those in the ASCII range of values.
832 .Sp
833 The \fImodes\fR are a comma\-separated set of text values. The
834 acceptable values are \fBunix\fR, \fBwindows\fR, \fBnocontrol\fR,
835 \&\fBascii\fR, \fBlowercase\fR, and \fBuppercase\fR. The values
836 \&\fBunix\fR and \fBwindows\fR are mutually exclusive (one will
837 override the other), as are \fBlowercase\fR and
838 \&\fBuppercase\fR. Those last are special cases, as they do not change
839 the set of characters that would be escaped, but rather force local
840 file paths to be converted either to lower\- or uppercase.
841 .Sp
842 When "unix" is specified, Wget escapes the character \fB/\fR and
843 the control characters in the ranges 0\-\-31 and 128\-\-159. This is the
844 default on Unix\-like operating systems.
845 .Sp
846 When "windows" is given, Wget escapes the characters \fB\e\fR,
847 \&\fB|\fR, \fB/\fR, \fB:\fR, \fB?\fR, \fB"\fR, \fB*\fR, \fB<\fR,
848 \&\fB>\fR, and the control characters in the ranges 0\-\-31 and 128\-\-159.
849 In addition to this, Wget in Windows mode uses \fB+\fR instead of
850 \&\fB:\fR to separate host and port in local file names, and uses
851 \&\fB@\fR instead of \fB?\fR to separate the query portion of the file
852 name from the rest. Therefore, a URL that would be saved as
853 \&\fBwww.xemacs.org:4300/search.pl?input=blah\fR in Unix mode would be
854 saved as \fBwww.xemacs.org+4300/search.pl@input=blah\fR in Windows
855 mode. This mode is the default on Windows.
856 .Sp
857 If you specify \fBnocontrol\fR, then the escaping of the control
858 characters is also switched off. This option may make sense
859 when you are downloading URLs whose names contain UTF\-8 characters, on
860 a system which can save and display filenames in UTF\-8 (some possible
861 byte values used in UTF\-8 byte sequences fall in the range of values
862 designated by Wget as "controls").
863 .Sp
864 The \fBascii\fR mode is used to specify that any bytes whose values
865 are outside the range of ASCII characters (that is, greater than
866 127) shall be escaped. This can be useful when saving filenames
867 whose encoding does not match the one used locally.
868 .IP \fB\-4\fR 4
869 .IX Item "-4"
870 .PD 0
871 .IP \fB\-\-inet4\-only\fR 4
872 .IX Item "--inet4-only"
873 .IP \fB\-6\fR 4
874 .IX Item "-6"
875 .IP \fB\-\-inet6\-only\fR 4
876 .IX Item "--inet6-only"
877 .PD
878 Force connecting to IPv4 or IPv6 addresses. With \fB\-\-inet4\-only\fR
879 or \fB\-4\fR, Wget will only connect to IPv4 hosts, ignoring AAAA
880 records in DNS, and refusing to connect to IPv6 addresses specified in
881 URLs. Conversely, with \fB\-\-inet6\-only\fR or \fB\-6\fR, Wget will
882 only connect to IPv6 hosts and ignore A records and IPv4 addresses.
883 .Sp
884 Neither options should be needed normally. By default, an IPv6\-aware
885 Wget will use the address family specified by the host\*(Aqs DNS record.
886 If the DNS responds with both IPv4 and IPv6 addresses, Wget will try
887 them in sequence until it finds one it can connect to. (Also see
888 \&\f(CW\*(C`\-\-prefer\-family\*(C'\fR option described below.)
889 .Sp
890 These options can be used to deliberately force the use of IPv4 or
891 IPv6 address families on dual family systems, usually to aid debugging
892 or to deal with broken network configuration. Only one of
893 \&\fB\-\-inet6\-only\fR and \fB\-\-inet4\-only\fR may be specified at the
894 same time. Neither option is available in Wget compiled without IPv6
895 support.
896 .IP \fB\-\-prefer\-family=none/IPv4/IPv6\fR 4
897 .IX Item "--prefer-family=none/IPv4/IPv6"
898 When given a choice of several addresses, connect to the addresses
899 with specified address family first. The address order returned by
900 DNS is used without change by default.
901 .Sp
902 This avoids spurious errors and connect attempts when accessing hosts
903 that resolve to both IPv6 and IPv4 addresses from IPv4 networks. For
904 example, \fBwww.kame.net\fR resolves to
905 \&\fB2001:200:0:8002:203:47ff:fea5:3085\fR and to
906 \&\fB203.178.141.194\fR. When the preferred family is \f(CW\*(C`IPv4\*(C'\fR, the
907 IPv4 address is used first; when the preferred family is \f(CW\*(C`IPv6\*(C'\fR,
908 the IPv6 address is used first; if the specified value is \f(CW\*(C`none\*(C'\fR,
909 the address order returned by DNS is used without change.
910 .Sp
911 Unlike \fB\-4\fR and \fB\-6\fR, this option doesn\*(Aqt inhibit access to
912 any address family, it only changes the \fIorder\fR in which the
913 addresses are accessed. Also note that the reordering performed by
914 this option is \fIstable\fR\-\-\-it doesn\*(Aqt affect order of addresses of
915 the same family. That is, the relative order of all IPv4 addresses
916 and of all IPv6 addresses remains intact in all cases.
917 .IP \fB\-\-retry\-connrefused\fR 4
918 .IX Item "--retry-connrefused"
919 Consider "connection refused" a transient error and try again.
920 Normally Wget gives up on a URL when it is unable to connect to the
921 site because failure to connect is taken as a sign that the server is
922 not running at all and that retries would not help. This option is
923 for mirroring unreliable sites whose servers tend to disappear for
924 short periods of time.
925 .IP \fB\-\-user=\fR\fIuser\fR 4
926 .IX Item "--user=user"
927 .PD 0
928 .IP \fB\-\-password=\fR\fIpassword\fR 4
929 .IX Item "--password=password"
930 .PD
931 Specify the username \fIuser\fR and password \fIpassword\fR for both
932 FTP and HTTP file retrieval. These parameters can be overridden
933 using the \fB\-\-ftp\-user\fR and \fB\-\-ftp\-password\fR options for
934 FTP connections and the \fB\-\-http\-user\fR and \fB\-\-http\-password\fR
935 options for HTTP connections.
936 .IP \fB\-\-ask\-password\fR 4
937 .IX Item "--ask-password"
938 Prompt for a password for each connection established. Cannot be specified
939 when \fB\-\-password\fR is being used, because they are mutually exclusive.
940 .IP \fB\-\-use\-askpass=\fR\fIcommand\fR 4
941 .IX Item "--use-askpass=command"
942 Prompt for a user and password using the specified command. If no command is
943 specified then the command in the environment variable WGET_ASKPASS is used.
944 If WGET_ASKPASS is not set then the command in the environment variable
945 SSH_ASKPASS is used.
946 .Sp
947 You can set the default command for use\-askpass in the \fI.wgetrc\fR. That
948 setting may be overridden from the command line.
949 .IP \fB\-\-no\-iri\fR 4
950 .IX Item "--no-iri"
951 Turn off internationalized URI (IRI) support. Use \fB\-\-iri\fR to
952 turn it on. IRI support is activated by default.
953 .Sp
954 You can set the default state of IRI support using the \f(CW\*(C`iri\*(C'\fR
955 command in \fI.wgetrc\fR. That setting may be overridden from the
956 command line.
957 .IP \fB\-\-local\-encoding=\fR\fIencoding\fR 4
958 .IX Item "--local-encoding=encoding"
959 Force Wget to use \fIencoding\fR as the default system encoding. That affects
960 how Wget converts URLs specified as arguments from locale to UTF\-8 for
961 IRI support.
962 .Sp
963 Wget use the function \f(CWnl_langinfo()\fR and then the \f(CW\*(C`CHARSET\*(C'\fR
964 environment variable to get the locale. If it fails, ASCII is used.
965 .Sp
966 You can set the default local encoding using the \f(CW\*(C`local_encoding\*(C'\fR
967 command in \fI.wgetrc\fR. That setting may be overridden from the
968 command line.
969 .IP \fB\-\-remote\-encoding=\fR\fIencoding\fR 4
970 .IX Item "--remote-encoding=encoding"
971 Force Wget to use \fIencoding\fR as the default remote server encoding.
972 That affects how Wget converts URIs found in files from remote encoding
973 to UTF\-8 during a recursive fetch. This options is only useful for
974 IRI support, for the interpretation of non\-ASCII characters.
975 .Sp
976 For HTTP, remote encoding can be found in HTTP \f(CW\*(C`Content\-Type\*(C'\fR
977 header and in HTML \f(CW\*(C`Content\-Type http\-equiv\*(C'\fR meta tag.
978 .Sp
979 You can set the default encoding using the \f(CW\*(C`remoteencoding\*(C'\fR
980 command in \fI.wgetrc\fR. That setting may be overridden from the
981 command line.
982 .IP \fB\-\-unlink\fR 4
983 .IX Item "--unlink"
984 Force Wget to unlink file instead of clobbering existing file. This
985 option is useful for downloading to the directory with hardlinks.
986 .SS "Directory Options"
987 .IX Subsection "Directory Options"
988 .IP \fB\-nd\fR 4
989 .IX Item "-nd"
990 .PD 0
991 .IP \fB\-\-no\-directories\fR 4
992 .IX Item "--no-directories"
993 .PD
994 Do not create a hierarchy of directories when retrieving recursively.
995 With this option turned on, all files will get saved to the current
996 directory, without clobbering (if a name shows up more than once, the
997 filenames will get extensions \fB.n\fR).
998 .IP \fB\-x\fR 4
999 .IX Item "-x"
1000 .PD 0
1001 .IP \fB\-\-force\-directories\fR 4
1002 .IX Item "--force-directories"
1003 .PD
1004 The opposite of \fB\-nd\fR\-\-\-create a hierarchy of directories, even if
1005 one would not have been created otherwise. E.g. \fBwget \-x
1006 http://fly.srk.fer.hr/robots.txt\fR will save the downloaded file to
1007 \&\fIfly.srk.fer.hr/robots.txt\fR.
1008 .IP \fB\-nH\fR 4
1009 .IX Item "-nH"
1010 .PD 0
1011 .IP \fB\-\-no\-host\-directories\fR 4
1012 .IX Item "--no-host-directories"
1013 .PD
1014 Disable generation of host\-prefixed directories. By default, invoking
1015 Wget with \fB\-r http://fly.srk.fer.hr/\fR will create a structure of
1016 directories beginning with \fIfly.srk.fer.hr/\fR. This option disables
1017 such behavior.
1018 .IP \fB\-\-protocol\-directories\fR 4
1019 .IX Item "--protocol-directories"
1020 Use the protocol name as a directory component of local file names. For
1021 example, with this option, \fBwget \-r http://\fR\fIhost\fR will save to
1022 \&\fBhttp/\fR\fIhost\fR\fB/...\fR rather than just to \fIhost\fR\fB/...\fR.
1023 .IP \fB\-\-cut\-dirs=\fR\fInumber\fR 4
1024 .IX Item "--cut-dirs=number"
1025 Ignore \fInumber\fR directory components. This is useful for getting a
1026 fine\-grained control over the directory where recursive retrieval will
1027 be saved.
1028 .Sp
1029 Take, for example, the directory at
1030 \&\fBftp://ftp.xemacs.org/pub/xemacs/\fR. If you retrieve it with
1031 \&\fB\-r\fR, it will be saved locally under
1032 \&\fIftp.xemacs.org/pub/xemacs/\fR. While the \fB\-nH\fR option can
1033 remove the \fIftp.xemacs.org/\fR part, you are still stuck with
1034 \&\fIpub/xemacs\fR. This is where \fB\-\-cut\-dirs\fR comes in handy; it
1035 makes Wget not "see" \fInumber\fR remote directory components. Here
1036 are several examples of how \fB\-\-cut\-dirs\fR option works.
1037 .Sp
1038 .Vb 4
1039 \& No options \-> ftp.xemacs.org/pub/xemacs/
1040 \& \-nH \-> pub/xemacs/
1041 \& \-nH \-\-cut\-dirs=1 \-> xemacs/
1042 \& \-nH \-\-cut\-dirs=2 \-> .
1043 \&
1044 \& \-\-cut\-dirs=1 \-> ftp.xemacs.org/xemacs/
1045 \& ...
1046 .Ve
1047 .Sp
1048 If you just want to get rid of the directory structure, this option is
1049 similar to a combination of \fB\-nd\fR and \fB\-P\fR. However, unlike
1050 \&\fB\-nd\fR, \fB\-\-cut\-dirs\fR does not lose with subdirectories\-\-\-for
1051 instance, with \fB\-nH \-\-cut\-dirs=1\fR, a \fIbeta/\fR subdirectory will
1052 be placed to \fIxemacs/beta\fR, as one would expect.
1053 .IP "\fB\-P\fR \fIprefix\fR" 4
1054 .IX Item "-P prefix"
1055 .PD 0
1056 .IP \fB\-\-directory\-prefix=\fR\fIprefix\fR 4
1057 .IX Item "--directory-prefix=prefix"
1058 .PD
1059 Set directory prefix to \fIprefix\fR. The \fIdirectory prefix\fR is the
1060 directory where all other files and subdirectories will be saved to,
1061 i.e. the top of the retrieval tree. The default is \fB.\fR (the
1062 current directory).
1063 .SS "HTTP Options"
1064 .IX Subsection "HTTP Options"
1065 .IP \fB\-\-default\-page=\fR\fIname\fR 4
1066 .IX Item "--default-page=name"
1067 Use \fIname\fR as the default file name when it isn\*(Aqt known (i.e., for
1068 URLs that end in a slash), instead of \fIindex.html\fR.
1069 .IP \fB\-E\fR 4
1070 .IX Item "-E"
1071 .PD 0
1072 .IP \fB\-\-adjust\-extension\fR 4
1073 .IX Item "--adjust-extension"
1074 .PD
1075 If a file of type \fBapplication/xhtml+xml\fR or \fBtext/html\fR is
1076 downloaded and the URL does not end with the regexp
1077 \&\fB\e.[Hh][Tt][Mm][Ll]?\fR, this option will cause the suffix \fB.html\fR
1078 to be appended to the local filename. This is useful, for instance, when
1079 you\*(Aqre mirroring a remote site that uses \fB.asp\fR pages, but you want
1080 the mirrored pages to be viewable on your stock Apache server. Another
1081 good use for this is when you\*(Aqre downloading CGI\-generated materials. A URL
1082 like \fBhttp://site.com/article.cgi?25\fR will be saved as
1083 \&\fIarticle.cgi?25.html\fR.
1084 .Sp
1085 Note that filenames changed in this way will be re\-downloaded every time
1086 you re\-mirror a site, because Wget can\*(Aqt tell that the local
1087 \&\fIX.html\fR file corresponds to remote URL \fIX\fR (since
1088 it doesn\*(Aqt yet know that the URL produces output of type
1089 \&\fBtext/html\fR or \fBapplication/xhtml+xml\fR.
1090 .Sp
1091 As of version 1.12, Wget will also ensure that any downloaded files of
1092 type \fBtext/css\fR end in the suffix \fB.css\fR, and the option was
1093 renamed from \fB\-\-html\-extension\fR, to better reflect its new
1094 behavior. The old option name is still acceptable, but should now be
1095 considered deprecated.
1096 .Sp
1097 As of version 1.19.2, Wget will also ensure that any downloaded files with
1098 a \f(CW\*(C`Content\-Encoding\*(C'\fR of \fBbr\fR, \fBcompress\fR, \fBdeflate\fR
1099 or \fBgzip\fR end in the suffix \fB.br\fR, \fB.Z\fR, \fB.zlib\fR
1100 and \fB.gz\fR respectively.
1101 .Sp
1102 At some point in the future, this option may well be expanded to
1103 include suffixes for other types of content, including content types
1104 that are not parsed by Wget.
1105 .IP \fB\-\-http\-user=\fR\fIuser\fR 4
1106 .IX Item "--http-user=user"
1107 .PD 0
1108 .IP \fB\-\-http\-password=\fR\fIpassword\fR 4
1109 .IX Item "--http-password=password"
1110 .PD
1111 Specify the username \fIuser\fR and password \fIpassword\fR on an
1112 HTTP server. According to the type of the challenge, Wget will
1113 encode them using either the \f(CW\*(C`basic\*(C'\fR (insecure),
1114 the \f(CW\*(C`digest\*(C'\fR, or the Windows \f(CW\*(C`NTLM\*(C'\fR authentication scheme.
1115 .Sp
1116 Another way to specify username and password is in the URL itself. Either method reveals your password to anyone who
1117 bothers to run \f(CW\*(C`ps\*(C'\fR. To prevent the passwords from being seen,
1118 use the \fB\-\-use\-askpass\fR or store them in \fI.wgetrc\fR or \fI.netrc\fR,
1119 and make sure to protect those files from other users with \f(CW\*(C`chmod\*(C'\fR. If
1120 the passwords are really important, do not leave them lying in those files
1121 either\-\-\-edit the files and delete them after Wget has started the download.
1122 .IP \fB\-\-no\-http\-keep\-alive\fR 4
1123 .IX Item "--no-http-keep-alive"
1124 Turn off the "keep\-alive" feature for HTTP downloads. Normally, Wget
1125 asks the server to keep the connection open so that, when you download
1126 more than one document from the same server, they get transferred over
1127 the same TCP connection. This saves time and at the same time reduces
1128 the load on the server.
1129 .Sp
1130 This option is useful when, for some reason, persistent (keep\-alive)
1131 connections don\*(Aqt work for you, for example due to a server bug or due
1132 to the inability of server\-side scripts to cope with the connections.
1133 .IP \fB\-\-no\-cache\fR 4
1134 .IX Item "--no-cache"
1135 Disable server\-side cache. In this case, Wget will send the remote
1136 server appropriate directives (\fBCache\-Control: no\-cache\fR and
1137 \&\fBPragma: no\-cache\fR) to get the file from the remote service,
1138 rather than returning the cached version. This is especially useful
1139 for retrieving and flushing out\-of\-date documents on proxy servers.
1140 .Sp
1141 Caching is allowed by default.
1142 .IP \fB\-\-no\-cookies\fR 4
1143 .IX Item "--no-cookies"
1144 Disable the use of cookies. Cookies are a mechanism for maintaining
1145 server\-side state. The server sends the client a cookie using the
1146 \&\f(CW\*(C`Set\-Cookie\*(C'\fR header, and the client responds with the same cookie
1147 upon further requests. Since cookies allow the server owners to keep
1148 track of visitors and for sites to exchange this information, some
1149 consider them a breach of privacy. The default is to use cookies;
1150 however, \fIstoring\fR cookies is not on by default.
1151 .IP "\fB\-\-load\-cookies\fR \fIfile\fR" 4
1152 .IX Item "--load-cookies file"
1153 Load cookies from \fIfile\fR before the first HTTP retrieval.
1154 \&\fIfile\fR is a textual file in the format originally used by Netscape\*(Aqs
1155 \&\fIcookies.txt\fR file.
1156 .Sp
1157 You will typically use this option when mirroring sites that require
1158 that you be logged in to access some or all of their content. The login
1159 process typically works by the web server issuing an HTTP cookie
1160 upon receiving and verifying your credentials. The cookie is then
1161 resent by the browser when accessing that part of the site, and so
1162 proves your identity.
1163 .Sp
1164 Mirroring such a site requires Wget to send the same cookies your
1165 browser sends when communicating with the site. This is achieved by
1166 \&\fB\-\-load\-cookies\fR\-\-\-simply point Wget to the location of the
1167 \&\fIcookies.txt\fR file, and it will send the same cookies your browser
1168 would send in the same situation. Different browsers keep textual
1169 cookie files in different locations:
1170 .RS 4
1171 .ie n .IP """Netscape 4.x.""" 4
1172 .el .IP "\f(CWNetscape 4.x.\fR" 4
1173 .IX Item "Netscape 4.x."
1174 The cookies are in \fI~/.netscape/cookies.txt\fR.
1175 .ie n .IP """Mozilla and Netscape 6.x.""" 4
1176 .el .IP "\f(CWMozilla and Netscape 6.x.\fR" 4
1177 .IX Item "Mozilla and Netscape 6.x."
1178 Mozilla\*(Aqs cookie file is also named \fIcookies.txt\fR, located
1179 somewhere under \fI~/.mozilla\fR, in the directory of your profile.
1180 The full path usually ends up looking somewhat like
1181 \&\fI~/.mozilla/default/some\-weird\-string/cookies.txt\fR.
1182 .ie n .IP """Internet Explorer.""" 4
1183 .el .IP "\f(CWInternet Explorer.\fR" 4
1184 .IX Item "Internet Explorer."
1185 You can produce a cookie file Wget can use by using the File menu,
1186 Import and Export, Export Cookies. This has been tested with Internet
1187 Explorer 5; it is not guaranteed to work with earlier versions.
1188 .ie n .IP """Other browsers.""" 4
1189 .el .IP "\f(CWOther browsers.\fR" 4
1190 .IX Item "Other browsers."
1191 If you are using a different browser to create your cookies,
1192 \&\fB\-\-load\-cookies\fR will only work if you can locate or produce a
1193 cookie file in the Netscape format that Wget expects.
1194 .RE
1195 .RS 4
1196 .Sp
1197 If you cannot use \fB\-\-load\-cookies\fR, there might still be an
1198 alternative. If your browser supports a "cookie manager", you can use
1199 it to view the cookies used when accessing the site you\*(Aqre mirroring.
1200 Write down the name and value of the cookie, and manually instruct Wget
1201 to send those cookies, bypassing the "official" cookie support:
1202 .Sp
1203 .Vb 1
1204 \& wget \-\-no\-cookies \-\-header "Cookie: <name>=<value>"
1205 .Ve
1206 .RE
1207 .IP "\fB\-\-save\-cookies\fR \fIfile\fR" 4
1208 .IX Item "--save-cookies file"
1209 Save cookies to \fIfile\fR before exiting. This will not save cookies
1210 that have expired or that have no expiry time (so\-called "session
1211 cookies"), but also see \fB\-\-keep\-session\-cookies\fR.
1212 .IP \fB\-\-keep\-session\-cookies\fR 4
1213 .IX Item "--keep-session-cookies"
1214 When specified, causes \fB\-\-save\-cookies\fR to also save session
1215 cookies. Session cookies are normally not saved because they are
1216 meant to be kept in memory and forgotten when you exit the browser.
1217 Saving them is useful on sites that require you to log in or to visit
1218 the home page before you can access some pages. With this option,
1219 multiple Wget runs are considered a single browser session as far as
1220 the site is concerned.
1221 .Sp
1222 Since the cookie file format does not normally carry session cookies,
1223 Wget marks them with an expiry timestamp of 0. Wget\*(Aqs
1224 \&\fB\-\-load\-cookies\fR recognizes those as session cookies, but it might
1225 confuse other browsers. Also note that cookies so loaded will be
1226 treated as other session cookies, which means that if you want
1227 \&\fB\-\-save\-cookies\fR to preserve them again, you must use
1228 \&\fB\-\-keep\-session\-cookies\fR again.
1229 .IP \fB\-\-ignore\-length\fR 4
1230 .IX Item "--ignore-length"
1231 Unfortunately, some HTTP servers (CGI programs, to be more
1232 precise) send out bogus \f(CW\*(C`Content\-Length\*(C'\fR headers, which makes Wget
1233 go wild, as it thinks not all the document was retrieved. You can spot
1234 this syndrome if Wget retries getting the same document again and again,
1235 each time claiming that the (otherwise normal) connection has closed on
1236 the very same byte.
1237 .Sp
1238 With this option, Wget will ignore the \f(CW\*(C`Content\-Length\*(C'\fR header\-\-\-as
1239 if it never existed.
1240 .IP \fB\-\-header=\fR\fIheader\-line\fR 4
1241 .IX Item "--header=header-line"
1242 Send \fIheader\-line\fR along with the rest of the headers in each
1243 HTTP request. The supplied header is sent as\-is, which means it
1244 must contain name and value separated by colon, and must not contain
1245 newlines.
1246 .Sp
1247 You may define more than one additional header by specifying
1248 \&\fB\-\-header\fR more than once.
1249 .Sp
1250 .Vb 3
1251 \& wget \-\-header=\*(AqAccept\-Charset: iso\-8859\-2\*(Aq \e
1252 \& \-\-header=\*(AqAccept\-Language: hr\*(Aq \e
1253 \& http://fly.srk.fer.hr/
1254 .Ve
1255 .Sp
1256 Specification of an empty string as the header value will clear all
1257 previous user\-defined headers.
1258 .Sp
1259 As of Wget 1.10, this option can be used to override headers otherwise
1260 generated automatically. This example instructs Wget to connect to
1261 localhost, but to specify \fBfoo.bar\fR in the \f(CW\*(C`Host\*(C'\fR header:
1262 .Sp
1263 .Vb 1
1264 \& wget \-\-header="Host: foo.bar" http://localhost/
1265 .Ve
1266 .Sp
1267 In versions of Wget prior to 1.10 such use of \fB\-\-header\fR caused
1268 sending of duplicate headers.
1269 .IP \fB\-\-compression=\fR\fItype\fR 4
1270 .IX Item "--compression=type"
1271 Choose the type of compression to be used. Legal values are
1272 \&\fBauto\fR, \fBgzip\fR and \fBnone\fR.
1273 .Sp
1274 If \fBauto\fR or \fBgzip\fR are specified, Wget asks the server to
1275 compress the file using the gzip compression format. If the server
1276 compresses the file and responds with the \f(CW\*(C`Content\-Encoding\*(C'\fR
1277 header field set appropriately, the file will be decompressed
1278 automatically.
1279 .Sp
1280 If \fBnone\fR is specified, wget will not ask the server to compress
1281 the file and will not decompress any server responses. This is the default.
1282 .Sp
1283 Compression support is currently experimental. In case it is turned on,
1284 please report any bugs to \f(CW\*(C`bug\[email protected]\*(C'\fR.
1285 .IP \fB\-\-max\-redirect=\fR\fInumber\fR 4
1286 .IX Item "--max-redirect=number"
1287 Specifies the maximum number of redirections to follow for a resource.
1288 The default is 20, which is usually far more than necessary. However, on
1289 those occasions where you want to allow more (or fewer), this is the
1290 option to use.
1291 .IP \fB\-\-proxy\-user=\fR\fIuser\fR 4
1292 .IX Item "--proxy-user=user"
1293 .PD 0
1294 .IP \fB\-\-proxy\-password=\fR\fIpassword\fR 4
1295 .IX Item "--proxy-password=password"
1296 .PD
1297 Specify the username \fIuser\fR and password \fIpassword\fR for
1298 authentication on a proxy server. Wget will encode them using the
1299 \&\f(CW\*(C`basic\*(C'\fR authentication scheme.
1300 .Sp
1301 Security considerations similar to those with \fB\-\-http\-password\fR
1302 pertain here as well.
1303 .IP \fB\-\-referer=\fR\fIurl\fR 4
1304 .IX Item "--referer=url"
1305 Include \`Referer: \fIurl\fR\*(Aq header in HTTP request. Useful for
1306 retrieving documents with server\-side processing that assume they are
1307 always being retrieved by interactive web browsers and only come out
1308 properly when Referer is set to one of the pages that point to them.
1309 .IP \fB\-\-save\-headers\fR 4
1310 .IX Item "--save-headers"
1311 Save the headers sent by the HTTP server to the file, preceding the
1312 actual contents, with an empty line as the separator.
1313 .IP "\fB\-U\fR \fIagent\-string\fR" 4
1314 .IX Item "-U agent-string"
1315 .PD 0
1316 .IP \fB\-\-user\-agent=\fR\fIagent\-string\fR 4
1317 .IX Item "--user-agent=agent-string"
1318 .PD
1319 Identify as \fIagent\-string\fR to the HTTP server.
1320 .Sp
1321 The HTTP protocol allows the clients to identify themselves using a
1322 \&\f(CW\*(C`User\-Agent\*(C'\fR header field. This enables distinguishing the
1323 WWW software, usually for statistical purposes or for tracing of
1324 protocol violations. Wget normally identifies as
1325 \&\fBWget/\fR\fIversion\fR, \fIversion\fR being the current version
1326 number of Wget.
1327 .Sp
1328 However, some sites have been known to impose the policy of tailoring
1329 the output according to the \f(CW\*(C`User\-Agent\*(C'\fR\-supplied information.
1330 While this is not such a bad idea in theory, it has been abused by
1331 servers denying information to clients other than (historically)
1332 Netscape or, more frequently, Microsoft Internet Explorer. This
1333 option allows you to change the \f(CW\*(C`User\-Agent\*(C'\fR line issued by Wget.
1334 Use of this option is discouraged, unless you really know what you are
1335 doing.
1336 .Sp
1337 Specifying empty user agent with \fB\-\-user\-agent=""\fR instructs Wget
1338 not to send the \f(CW\*(C`User\-Agent\*(C'\fR header in HTTP requests.
1339 .IP \fB\-\-post\-data=\fR\fIstring\fR 4
1340 .IX Item "--post-data=string"
1341 .PD 0
1342 .IP \fB\-\-post\-file=\fR\fIfile\fR 4
1343 .IX Item "--post-file=file"
1344 .PD
1345 Use POST as the method for all HTTP requests and send the specified
1346 data in the request body. \fB\-\-post\-data\fR sends \fIstring\fR as
1347 data, whereas \fB\-\-post\-file\fR sends the contents of \fIfile\fR.
1348 Other than that, they work in exactly the same way. In particular,
1349 they \fIboth\fR expect content of the form \f(CW\*(C`key1=value1&key2=value2\*(C'\fR,
1350 with percent\-encoding for special characters; the only difference is
1351 that one expects its content as a command\-line parameter and the other
1352 accepts its content from a file. In particular, \fB\-\-post\-file\fR is
1353 \&\fInot\fR for transmitting files as form attachments: those must
1354 appear as \f(CW\*(C`key=value\*(C'\fR data (with appropriate percent\-coding) just
1355 like everything else. Wget does not currently support
1356 \&\f(CW\*(C`multipart/form\-data\*(C'\fR for transmitting POST data; only
1357 \&\f(CW\*(C`application/x\-www\-form\-urlencoded\*(C'\fR. Only one of
1358 \&\fB\-\-post\-data\fR and \fB\-\-post\-file\fR should be specified.
1359 .Sp
1360 Please note that wget does not require the content to be of the form
1361 \&\f(CW\*(C`key1=value1&key2=value2\*(C'\fR, and neither does it test for it. Wget will
1362 simply transmit whatever data is provided to it. Most servers however expect
1363 the POST data to be in the above format when processing HTML Forms.
1364 .Sp
1365 When sending a POST request using the \fB\-\-post\-file\fR option, Wget treats
1366 the file as a binary file and will send every character in the POST request
1367 without stripping trailing newline or formfeed characters. Any other control
1368 characters in the text will also be sent as\-is in the POST request.
1369 .Sp
1370 Please be aware that Wget needs to know the size of the POST data in
1371 advance. Therefore the argument to \f(CW\*(C`\-\-post\-file\*(C'\fR must be a regular
1372 file; specifying a FIFO or something like \fI/dev/stdin\fR won\*(Aqt work.
1373 It\*(Aqs not quite clear how to work around this limitation inherent in
1374 HTTP/1.0. Although HTTP/1.1 introduces \fIchunked\fR transfer that
1375 doesn\*(Aqt require knowing the request length in advance, a client can\*(Aqt
1376 use chunked unless it knows it\*(Aqs talking to an HTTP/1.1 server. And it
1377 can\*(Aqt know that until it receives a response, which in turn requires the
1378 request to have been completed \-\- a chicken\-and\-egg problem.
1379 .Sp
1380 Note: As of version 1.15 if Wget is redirected after the POST request is
1381 completed, its behaviour will depend on the response code returned by the
1382 server. In case of a 301 Moved Permanently, 302 Moved Temporarily or
1383 307 Temporary Redirect, Wget will, in accordance with RFC2616, continue
1384 to send a POST request.
1385 In case a server wants the client to change the Request method upon
1386 redirection, it should send a 303 See Other response code.
1387 .Sp
1388 This example shows how to log in to a server using POST and then proceed to
1389 download the desired pages, presumably only accessible to authorized
1390 users:
1391 .Sp
1392 .Vb 4
1393 \& # Log in to the server. This can be done only once.
1394 \& wget \-\-save\-cookies cookies.txt \e
1395 \& \-\-post\-data \*(Aquser=foo&password=bar\*(Aq \e
1396 \& http://example.com/auth.php
1397 \&
1398 \& # Now grab the page or pages we care about.
1399 \& wget \-\-load\-cookies cookies.txt \e
1400 \& \-p http://example.com/interesting/article.php
1401 .Ve
1402 .Sp
1403 If the server is using session cookies to track user authentication,
1404 the above will not work because \fB\-\-save\-cookies\fR will not save
1405 them (and neither will browsers) and the \fIcookies.txt\fR file will
1406 be empty. In that case use \fB\-\-keep\-session\-cookies\fR along with
1407 \&\fB\-\-save\-cookies\fR to force saving of session cookies.
1408 .IP \fB\-\-method=\fR\fIHTTP\-Method\fR 4
1409 .IX Item "--method=HTTP-Method"
1410 For the purpose of RESTful scripting, Wget allows sending of other HTTP Methods
1411 without the need to explicitly set them using \fB\-\-header=Header\-Line\fR.
1412 Wget will use whatever string is passed to it after \fB\-\-method\fR as the HTTP
1413 Method to the server.
1414 .IP \fB\-\-body\-data=\fR\fIData\-String\fR 4
1415 .IX Item "--body-data=Data-String"
1416 .PD 0
1417 .IP \fB\-\-body\-file=\fR\fIData\-File\fR 4
1418 .IX Item "--body-file=Data-File"
1419 .PD
1420 Must be set when additional data needs to be sent to the server along with the
1421 Method specified using \fB\-\-method\fR. \fB\-\-body\-data\fR sends \fIstring\fR as
1422 data, whereas \fB\-\-body\-file\fR sends the contents of \fIfile\fR. Other than that,
1423 they work in exactly the same way.
1424 .Sp
1425 Currently, \fB\-\-body\-file\fR is \fInot\fR for transmitting files as a whole.
1426 Wget does not currently support \f(CW\*(C`multipart/form\-data\*(C'\fR for transmitting data;
1427 only \f(CW\*(C`application/x\-www\-form\-urlencoded\*(C'\fR. In the future, this may be changed
1428 so that wget sends the \fB\-\-body\-file\fR as a complete file instead of sending its
1429 contents to the server. Please be aware that Wget needs to know the contents of
1430 BODY Data in advance, and hence the argument to \fB\-\-body\-file\fR should be a
1431 regular file. See \fB\-\-post\-file\fR for a more detailed explanation.
1432 Only one of \fB\-\-body\-data\fR and \fB\-\-body\-file\fR should be specified.
1433 .Sp
1434 If Wget is redirected after the request is completed, Wget will
1435 suspend the current method and send a GET request till the redirection
1436 is completed. This is true for all redirection response codes except
1437 307 Temporary Redirect which is used to explicitly specify that the
1438 request method should \fInot\fR change. Another exception is when
1439 the method is set to \f(CW\*(C`POST\*(C'\fR, in which case the redirection rules
1440 specified under \fB\-\-post\-data\fR are followed.
1441 .IP \fB\-\-content\-disposition\fR 4
1442 .IX Item "--content-disposition"
1443 If this is set to on, experimental (not fully\-functional) support for
1444 \&\f(CW\*(C`Content\-Disposition\*(C'\fR headers is enabled. This can currently result in
1445 extra round\-trips to the server for a \f(CW\*(C`HEAD\*(C'\fR request, and is known
1446 to suffer from a few bugs, which is why it is not currently enabled by default.
1447 .Sp
1448 This option is useful for some file\-downloading CGI programs that use
1449 \&\f(CW\*(C`Content\-Disposition\*(C'\fR headers to describe what the name of a
1450 downloaded file should be.
1451 .Sp
1452 When combined with \fB\-\-metalink\-over\-http\fR and \fB\-\-trust\-server\-names\fR,
1453 a \fBContent\-Type: application/metalink4+xml\fR file is named using the
1454 \&\f(CW\*(C`Content\-Disposition\*(C'\fR filename field, if available.
1455 .IP \fB\-\-content\-on\-error\fR 4
1456 .IX Item "--content-on-error"
1457 If this is set to on, wget will not skip the content when the server responds
1458 with a http status code that indicates error.
1459 .IP \fB\-\-trust\-server\-names\fR 4
1460 .IX Item "--trust-server-names"
1461 If this is set, on a redirect, the local file name will be based
1462 on the redirection URL. By default the local file name is based on
1463 the original URL. When doing recursive retrieving this can be helpful
1464 because in many web sites redirected URLs correspond to an underlying
1465 file structure, while link URLs do not.
1466 .IP \fB\-\-auth\-no\-challenge\fR 4
1467 .IX Item "--auth-no-challenge"
1468 If this option is given, Wget will send Basic HTTP authentication
1469 information (plaintext username and password) for all requests, just
1470 like Wget 1.10.2 and prior did by default.
1471 .Sp
1472 Use of this option is not recommended, and is intended only to support
1473 some few obscure servers, which never send HTTP authentication
1474 challenges, but accept unsolicited auth info, say, in addition to
1475 form\-based authentication.
1476 .IP \fB\-\-retry\-on\-host\-error\fR 4
1477 .IX Item "--retry-on-host-error"
1478 Consider host errors, such as "Temporary failure in name resolution",
1479 as non\-fatal, transient errors.
1480 .IP \fB\-\-retry\-on\-http\-error=\fR\fIcode[,code,...]\fR 4
1481 .IX Item "--retry-on-http-error=code[,code,...]"
1482 Consider given HTTP response codes as non\-fatal, transient errors.
1483 Supply a comma\-separated list of 3\-digit HTTP response codes as
1484 argument. Useful to work around special circumstances where retries
1485 are required, but the server responds with an error code normally not
1486 retried by Wget. Such errors might be 503 (Service Unavailable) and
1487 429 (Too Many Requests). Retries enabled by this option are performed
1488 subject to the normal retry timing and retry count limitations of
1489 Wget.
1490 .Sp
1491 Using this option is intended to support special use cases only and is
1492 generally not recommended, as it can force retries even in cases where
1493 the server is actually trying to decrease its load. Please use wisely
1494 and only if you know what you are doing.
1495 .SS "HTTPS (SSL/TLS) Options"
1496 .IX Subsection "HTTPS (SSL/TLS) Options"
1497 To support encrypted HTTP (HTTPS) downloads, Wget must be compiled
1498 with an external SSL library. The current default is GnuTLS.
1499 In addition, Wget also supports HSTS (HTTP Strict Transport Security).
1500 If Wget is compiled without SSL support, none of these options are available.
1501 .IP \fB\-\-secure\-protocol=\fR\fIprotocol\fR 4
1502 .IX Item "--secure-protocol=protocol"
1503 Choose the secure protocol to be used. Legal values are \fBauto\fR,
1504 \&\fBSSLv2\fR, \fBSSLv3\fR, \fBTLSv1\fR, \fBTLSv1_1\fR, \fBTLSv1_2\fR,
1505 \&\fBTLSv1_3\fR and \fBPFS\fR. If \fBauto\fR is used, the SSL library is
1506 given the liberty of choosing the appropriate protocol automatically, which is
1507 achieved by sending a TLSv1 greeting. This is the default.
1508 .Sp
1509 Specifying \fBSSLv2\fR, \fBSSLv3\fR, \fBTLSv1\fR, \fBTLSv1_1\fR,
1510 \&\fBTLSv1_2\fR or \fBTLSv1_3\fR forces the use of the corresponding
1511 protocol. This is useful when talking to old and buggy SSL server
1512 implementations that make it hard for the underlying SSL library to choose
1513 the correct protocol version. Fortunately, such servers are quite rare.
1514 .Sp
1515 Specifying \fBPFS\fR enforces the use of the so\-called Perfect Forward
1516 Security cipher suites. In short, PFS adds security by creating a one\-time
1517 key for each SSL connection. It has a bit more CPU impact on client and server.
1518 We use known to be secure ciphers (e.g. no MD4) and the TLS protocol. This mode
1519 also explicitly excludes non\-PFS key exchange methods, such as RSA.
1520 .IP \fB\-\-https\-only\fR 4
1521 .IX Item "--https-only"
1522 When in recursive mode, only HTTPS links are followed.
1523 .IP \fB\-\-ciphers\fR 4
1524 .IX Item "--ciphers"
1525 Set the cipher list string. Typically this string sets the
1526 cipher suites and other SSL/TLS options that the user wish should be used, in a
1527 set order of preference (GnuTLS calls it \*(Aqpriority string\*(Aq). This string
1528 will be fed verbatim to the SSL/TLS engine (OpenSSL or GnuTLS) and hence
1529 its format and syntax is dependent on that. Wget will not process or manipulate it
1530 in any way. Refer to the OpenSSL or GnuTLS documentation for more information.
1531 .IP \fB\-\-no\-check\-certificate\fR 4
1532 .IX Item "--no-check-certificate"
1533 Don\*(Aqt check the server certificate against the available certificate
1534 authorities. Also don\*(Aqt require the URL host name to match the common
1535 name presented by the certificate.
1536 .Sp
1537 As of Wget 1.10, the default is to verify the server\*(Aqs certificate
1538 against the recognized certificate authorities, breaking the SSL
1539 handshake and aborting the download if the verification fails.
1540 Although this provides more secure downloads, it does break
1541 interoperability with some sites that worked with previous Wget
1542 versions, particularly those using self\-signed, expired, or otherwise
1543 invalid certificates. This option forces an "insecure" mode of
1544 operation that turns the certificate verification errors into warnings
1545 and allows you to proceed.
1546 .Sp
1547 If you encounter "certificate verification" errors or ones saying
1548 that "common name doesn\*(Aqt match requested host name", you can use
1549 this option to bypass the verification and proceed with the download.
1550 \&\fIOnly use this option if you are otherwise convinced of the
1551 site\*(Aqs authenticity, or if you really don\*(Aqt care about the validity of
1552 its certificate.\fR It is almost always a bad idea not to check the
1553 certificates when transmitting confidential or important data.
1554 For self\-signed/internal certificates, you should download the certificate
1555 and verify against that instead of forcing this insecure mode.
1556 If you are really sure of not desiring any certificate verification, you
1557 can specify \-\-check\-certificate=quiet to tell wget to not print any
1558 warning about invalid certificates, albeit in most cases this is the
1559 wrong thing to do.
1560 .IP \fB\-\-certificate=\fR\fIfile\fR 4
1561 .IX Item "--certificate=file"
1562 Use the client certificate stored in \fIfile\fR. This is needed for
1563 servers that are configured to require certificates from the clients
1564 that connect to them. Normally a certificate is not required and this
1565 switch is optional.
1566 .IP \fB\-\-certificate\-type=\fR\fItype\fR 4
1567 .IX Item "--certificate-type=type"
1568 Specify the type of the client certificate. Legal values are
1569 \&\fBPEM\fR (assumed by default) and \fBDER\fR, also known as
1570 \&\fBASN1\fR.
1571 .IP \fB\-\-private\-key=\fR\fIfile\fR 4
1572 .IX Item "--private-key=file"
1573 Read the private key from \fIfile\fR. This allows you to provide the
1574 private key in a file separate from the certificate.
1575 .IP \fB\-\-private\-key\-type=\fR\fItype\fR 4
1576 .IX Item "--private-key-type=type"
1577 Specify the type of the private key. Accepted values are \fBPEM\fR
1578 (the default) and \fBDER\fR.
1579 .IP \fB\-\-ca\-certificate=\fR\fIfile\fR 4
1580 .IX Item "--ca-certificate=file"
1581 Use \fIfile\fR as the file with the bundle of certificate authorities
1582 ("CA") to verify the peers. The certificates must be in PEM format.
1583 .Sp
1584 Without this option Wget looks for CA certificates at the
1585 system\-specified locations, chosen at OpenSSL installation time.
1586 .IP \fB\-\-ca\-directory=\fR\fIdirectory\fR 4
1587 .IX Item "--ca-directory=directory"
1588 Specifies directory containing CA certificates in PEM format. Each
1589 file contains one CA certificate, and the file name is based on a hash
1590 value derived from the certificate. This is achieved by processing a
1591 certificate directory with the \f(CW\*(C`c_rehash\*(C'\fR utility supplied with
1592 OpenSSL. Using \fB\-\-ca\-directory\fR is more efficient than
1593 \&\fB\-\-ca\-certificate\fR when many certificates are installed because
1594 it allows Wget to fetch certificates on demand.
1595 .Sp
1596 Without this option Wget looks for CA certificates at the
1597 system\-specified locations, chosen at OpenSSL installation time.
1598 .IP \fB\-\-crl\-file=\fR\fIfile\fR 4
1599 .IX Item "--crl-file=file"
1600 Specifies a CRL file in \fIfile\fR. This is needed for certificates
1601 that have been revocated by the CAs.
1602 .IP \fB\-\-pinnedpubkey=file/hashes\fR 4
1603 .IX Item "--pinnedpubkey=file/hashes"
1604 Tells wget to use the specified public key file (or hashes) to verify the peer.
1605 This can be a path to a file which contains a single public key in PEM or DER
1606 format, or any number of base64 encoded sha256 hashes preceded by "sha256//"
1607 and separated by ";"
1608 .Sp
1609 When negotiating a TLS or SSL connection, the server sends a certificate
1610 indicating its identity. A public key is extracted from this certificate and if
1611 it does not exactly match the public key(s) provided to this option, wget will
1612 abort the connection before sending or receiving any data.
1613 .IP \fB\-\-random\-file=\fR\fIfile\fR 4
1614 .IX Item "--random-file=file"
1615 [OpenSSL and LibreSSL only]
1616 Use \fIfile\fR as the source of random data for seeding the
1617 pseudo\-random number generator on systems without \fI/dev/urandom\fR.
1618 .Sp
1619 On such systems the SSL library needs an external source of randomness
1620 to initialize. Randomness may be provided by EGD (see
1621 \&\fB\-\-egd\-file\fR below) or read from an external source specified by
1622 the user. If this option is not specified, Wget looks for random data
1623 in \f(CW$RANDFILE\fR or, if that is unset, in \fR\f(CI$HOME\fR\fI/.rnd\fR.
1624 .Sp
1625 If you\*(Aqre getting the "Could not seed OpenSSL PRNG; disabling SSL."
1626 error, you should provide random data using some of the methods
1627 described above.
1628 .IP \fB\-\-egd\-file=\fR\fIfile\fR 4
1629 .IX Item "--egd-file=file"
1630 [OpenSSL only]
1631 Use \fIfile\fR as the EGD socket. EGD stands for \fIEntropy
1632 Gathering Daemon\fR, a user\-space program that collects data from
1633 various unpredictable system sources and makes it available to other
1634 programs that might need it. Encryption software, such as the SSL
1635 library, needs sources of non\-repeating randomness to seed the random
1636 number generator used to produce cryptographically strong keys.
1637 .Sp
1638 OpenSSL allows the user to specify his own source of entropy using the
1639 \&\f(CW\*(C`RAND_FILE\*(C'\fR environment variable. If this variable is unset, or
1640 if the specified file does not produce enough randomness, OpenSSL will
1641 read random data from EGD socket specified using this option.
1642 .Sp
1643 If this option is not specified (and the equivalent startup command is
1644 not used), EGD is never contacted. EGD is not needed on modern Unix
1645 systems that support \fI/dev/urandom\fR.
1646 .IP \fB\-\-no\-hsts\fR 4
1647 .IX Item "--no-hsts"
1648 Wget supports HSTS (HTTP Strict Transport Security, RFC 6797) by default.
1649 Use \fB\-\-no\-hsts\fR to make Wget act as a non\-HSTS\-compliant UA. As a
1650 consequence, Wget would ignore all the \f(CW\*(C`Strict\-Transport\-Security\*(C'\fR
1651 headers, and would not enforce any existing HSTS policy.
1652 .IP \fB\-\-hsts\-file=\fR\fIfile\fR 4
1653 .IX Item "--hsts-file=file"
1654 By default, Wget stores its HSTS database in \fI~/.wget\-hsts\fR.
1655 You can use \fB\-\-hsts\-file\fR to override this. Wget will use
1656 the supplied file as the HSTS database. Such file must conform to the
1657 correct HSTS database format used by Wget. If Wget cannot parse the provided
1658 file, the behaviour is unspecified.
1659 .Sp
1660 The Wget\*(Aqs HSTS database is a plain text file. Each line contains an HSTS entry
1661 (ie. a site that has issued a \f(CW\*(C`Strict\-Transport\-Security\*(C'\fR header and that
1662 therefore has specified a concrete HSTS policy to be applied). Lines starting with
1663 a dash (\f(CW\*(C`#\*(C'\fR) are ignored by Wget. Please note that in spite of this convenient
1664 human\-readability hand\-hacking the HSTS database is generally not a good idea.
1665 .Sp
1666 An HSTS entry line consists of several fields separated by one or more whitespace:
1667 .Sp
1668 \&\f(CW\*(C`<hostname> SP [<port>] SP <include subdomains> SP <created> SP <max\-age>\*(C'\fR
1669 .Sp
1670 The \fIhostname\fR and \fIport\fR fields indicate the hostname and port to which
1671 the given HSTS policy applies. The \fIport\fR field may be zero, and it will, in
1672 most of the cases. That means that the port number will not be taken into account
1673 when deciding whether such HSTS policy should be applied on a given request (only
1674 the hostname will be evaluated). When \fIport\fR is different to zero, both the
1675 target hostname and the port will be evaluated and the HSTS policy will only be applied
1676 if both of them match. This feature has been included for testing/development purposes only.
1677 The Wget testsuite (in \fItestenv/\fR) creates HSTS databases with explicit ports
1678 with the purpose of ensuring Wget\*(Aqs correct behaviour. Applying HSTS policies to ports
1679 other than the default ones is discouraged by RFC 6797 (see Appendix B "Differences
1680 between HSTS Policy and Same\-Origin Policy"). Thus, this functionality should not be used
1681 in production environments and \fIport\fR will typically be zero. The last three fields
1682 do what they are expected to. The field \fIinclude_subdomains\fR can either be \f(CW1\fR
1683 or \f(CW0\fR and it signals whether the subdomains of the target domain should be
1684 part of the given HSTS policy as well. The \fIcreated\fR and \fImax\-age\fR fields
1685 hold the timestamp values of when such entry was created (first seen by Wget) and the
1686 HSTS\-defined value \*(Aqmax\-age\*(Aq, which states how long should that HSTS policy remain active,
1687 measured in seconds elapsed since the timestamp stored in \fIcreated\fR. Once that time
1688 has passed, that HSTS policy will no longer be valid and will eventually be removed
1689 from the database.
1690 .Sp
1691 If you supply your own HSTS database via \fB\-\-hsts\-file\fR, be aware that Wget
1692 may modify the provided file if any change occurs between the HSTS policies
1693 requested by the remote servers and those in the file. When Wget exits,
1694 it effectively updates the HSTS database by rewriting the database file with the new entries.
1695 .Sp
1696 If the supplied file does not exist, Wget will create one. This file will contain the new HSTS
1697 entries. If no HSTS entries were generated (no \f(CW\*(C`Strict\-Transport\-Security\*(C'\fR headers
1698 were sent by any of the servers) then no file will be created, not even an empty one. This
1699 behaviour applies to the default database file (\fI~/.wget\-hsts\fR) as well: it will not be
1700 created until some server enforces an HSTS policy.
1701 .Sp
1702 Care is taken not to override possible changes made by other Wget processes at
1703 the same time over the HSTS database. Before dumping the updated HSTS entries
1704 on the file, Wget will re\-read it and merge the changes.
1705 .Sp
1706 Using a custom HSTS database and/or modifying an existing one is discouraged.
1707 For more information about the potential security threats arose from such practice,
1708 see section 14 "Security Considerations" of RFC 6797, specially section 14.9
1709 "Creative Manipulation of HSTS Policy Store".
1710 .IP \fB\-\-warc\-file=\fR\fIfile\fR 4
1711 .IX Item "--warc-file=file"
1712 Use \fIfile\fR as the destination WARC file.
1713 .IP \fB\-\-warc\-header=\fR\fIstring\fR 4
1714 .IX Item "--warc-header=string"
1715 Use \fIstring\fR into as the warcinfo record.
1716 .IP \fB\-\-warc\-max\-size=\fR\fIsize\fR 4
1717 .IX Item "--warc-max-size=size"
1718 Set the maximum size of the WARC files to \fIsize\fR.
1719 .IP \fB\-\-warc\-cdx\fR 4
1720 .IX Item "--warc-cdx"
1721 Write CDX index files.
1722 .IP \fB\-\-warc\-dedup=\fR\fIfile\fR 4
1723 .IX Item "--warc-dedup=file"
1724 Do not store records listed in this CDX file.
1725 .IP \fB\-\-no\-warc\-compression\fR 4
1726 .IX Item "--no-warc-compression"
1727 Do not compress WARC files with GZIP.
1728 .IP \fB\-\-no\-warc\-digests\fR 4
1729 .IX Item "--no-warc-digests"
1730 Do not calculate SHA1 digests.
1731 .IP \fB\-\-no\-warc\-keep\-log\fR 4
1732 .IX Item "--no-warc-keep-log"
1733 Do not store the log file in a WARC record.
1734 .IP \fB\-\-warc\-tempdir=\fR\fIdir\fR 4
1735 .IX Item "--warc-tempdir=dir"
1736 Specify the location for temporary files created by the WARC writer.
1737 .SS "FTP Options"
1738 .IX Subsection "FTP Options"
1739 .IP \fB\-\-ftp\-user=\fR\fIuser\fR 4
1740 .IX Item "--ftp-user=user"
1741 .PD 0
1742 .IP \fB\-\-ftp\-password=\fR\fIpassword\fR 4
1743 .IX Item "--ftp-password=password"
1744 .PD
1745 Specify the username \fIuser\fR and password \fIpassword\fR on an
1746 FTP server. Without this, or the corresponding startup option,
1747 the password defaults to \fB\-wget@\fR, normally used for anonymous
1748 FTP.
1749 .Sp
1750 Another way to specify username and password is in the URL itself. Either method reveals your password to anyone who
1751 bothers to run \f(CW\*(C`ps\*(C'\fR. To prevent the passwords from being seen,
1752 store them in \fI.wgetrc\fR or \fI.netrc\fR, and make sure to protect
1753 those files from other users with \f(CW\*(C`chmod\*(C'\fR. If the passwords are
1754 really important, do not leave them lying in those files either\-\-\-edit
1755 the files and delete them after Wget has started the download.
1756 .IP \fB\-\-no\-remove\-listing\fR 4
1757 .IX Item "--no-remove-listing"
1758 Don\*(Aqt remove the temporary \fI.listing\fR files generated by FTP
1759 retrievals. Normally, these files contain the raw directory listings
1760 received from FTP servers. Not removing them can be useful for
1761 debugging purposes, or when you want to be able to easily check on the
1762 contents of remote server directories (e.g. to verify that a mirror
1763 you\*(Aqre running is complete).
1764 .Sp
1765 Note that even though Wget writes to a known filename for this file,
1766 this is not a security hole in the scenario of a user making
1767 \&\fI.listing\fR a symbolic link to \fI/etc/passwd\fR or something and
1768 asking \f(CW\*(C`root\*(C'\fR to run Wget in his or her directory. Depending on
1769 the options used, either Wget will refuse to write to \fI.listing\fR,
1770 making the globbing/recursion/time\-stamping operation fail, or the
1771 symbolic link will be deleted and replaced with the actual
1772 \&\fI.listing\fR file, or the listing will be written to a
1773 \&\fI.listing.number\fR file.
1774 .Sp
1775 Even though this situation isn\*(Aqt a problem, though, \f(CW\*(C`root\*(C'\fR should
1776 never run Wget in a non\-trusted user\*(Aqs directory. A user could do
1777 something as simple as linking \fIindex.html\fR to \fI/etc/passwd\fR
1778 and asking \f(CW\*(C`root\*(C'\fR to run Wget with \fB\-N\fR or \fB\-r\fR so the file
1779 will be overwritten.
1780 .IP \fB\-\-no\-glob\fR 4
1781 .IX Item "--no-glob"
1782 Turn off FTP globbing. Globbing refers to the use of shell\-like
1783 special characters (\fIwildcards\fR), like \fB*\fR, \fB?\fR, \fB[\fR
1784 and \fB]\fR to retrieve more than one file from the same directory at
1785 once, like:
1786 .Sp
1787 .Vb 1
1788 \& wget ftp://gnjilux.srk.fer.hr/*.msg
1789 .Ve
1790 .Sp
1791 By default, globbing will be turned on if the URL contains a
1792 globbing character. This option may be used to turn globbing on or off
1793 permanently.
1794 .Sp
1795 You may have to quote the URL to protect it from being expanded by
1796 your shell. Globbing makes Wget look for a directory listing, which is
1797 system\-specific. This is why it currently works only with Unix FTP
1798 servers (and the ones emulating Unix \f(CW\*(C`ls\*(C'\fR output).
1799 .IP \fB\-\-no\-passive\-ftp\fR 4
1800 .IX Item "--no-passive-ftp"
1801 Disable the use of the \fIpassive\fR FTP transfer mode. Passive FTP
1802 mandates that the client connect to the server to establish the data
1803 connection rather than the other way around.
1804 .Sp
1805 If the machine is connected to the Internet directly, both passive and
1806 active FTP should work equally well. Behind most firewall and NAT
1807 configurations passive FTP has a better chance of working. However,
1808 in some rare firewall configurations, active FTP actually works when
1809 passive FTP doesn\*(Aqt. If you suspect this to be the case, use this
1810 option, or set \f(CW\*(C`passive_ftp=off\*(C'\fR in your init file.
1811 .IP \fB\-\-preserve\-permissions\fR 4
1812 .IX Item "--preserve-permissions"
1813 Preserve remote file permissions instead of permissions set by umask.
1814 .IP \fB\-\-retr\-symlinks\fR 4
1815 .IX Item "--retr-symlinks"
1816 By default, when retrieving FTP directories recursively and a symbolic link
1817 is encountered, the symbolic link is traversed and the pointed\-to files are
1818 retrieved. Currently, Wget does not traverse symbolic links to directories to
1819 download them recursively, though this feature may be added in the future.
1820 .Sp
1821 When \fB\-\-retr\-symlinks=no\fR is specified, the linked\-to file is not
1822 downloaded. Instead, a matching symbolic link is created on the local
1823 file system. The pointed\-to file will not be retrieved unless this recursive
1824 retrieval would have encountered it separately and downloaded it anyway. This
1825 option poses a security risk where a malicious FTP Server may cause Wget to
1826 write to files outside of the intended directories through a specially crafted
1827 \&.LISTING file.
1828 .Sp
1829 Note that when retrieving a file (not a directory) because it was
1830 specified on the command\-line, rather than because it was recursed to,
1831 this option has no effect. Symbolic links are always traversed in this
1832 case.
1833 .SS "FTPS Options"
1834 .IX Subsection "FTPS Options"
1835 .IP \fB\-\-ftps\-implicit\fR 4
1836 .IX Item "--ftps-implicit"
1837 This option tells Wget to use FTPS implicitly. Implicit FTPS consists of initializing
1838 SSL/TLS from the very beginning of the control connection. This option does not send
1839 an \f(CW\*(C`AUTH TLS\*(C'\fR command: it assumes the server speaks FTPS and directly starts an
1840 SSL/TLS connection. If the attempt is successful, the session continues just like
1841 regular FTPS (\f(CW\*(C`PBSZ\*(C'\fR and \f(CW\*(C`PROT\*(C'\fR are sent, etc.).
1842 Implicit FTPS is no longer a requirement for FTPS implementations, and thus
1843 many servers may not support it. If \fB\-\-ftps\-implicit\fR is passed and no explicit
1844 port number specified, the default port for implicit FTPS, 990, will be used, instead
1845 of the default port for the "normal" (explicit) FTPS which is the same as that of FTP,
1846 21.
1847 .IP \fB\-\-no\-ftps\-resume\-ssl\fR 4
1848 .IX Item "--no-ftps-resume-ssl"
1849 Do not resume the SSL/TLS session in the data channel. When starting a data connection,
1850 Wget tries to resume the SSL/TLS session previously started in the control connection.
1851 SSL/TLS session resumption avoids performing an entirely new handshake by reusing
1852 the SSL/TLS parameters of a previous session. Typically, the FTPS servers want it that way,
1853 so Wget does this by default. Under rare circumstances however, one might want to
1854 start an entirely new SSL/TLS session in every data connection.
1855 This is what \fB\-\-no\-ftps\-resume\-ssl\fR is for.
1856 .IP \fB\-\-ftps\-clear\-data\-connection\fR 4
1857 .IX Item "--ftps-clear-data-connection"
1858 All the data connections will be in plain text. Only the control connection will be
1859 under SSL/TLS. Wget will send a \f(CW\*(C`PROT C\*(C'\fR command to achieve this, which must be
1860 approved by the server.
1861 .IP \fB\-\-ftps\-fallback\-to\-ftp\fR 4
1862 .IX Item "--ftps-fallback-to-ftp"
1863 Fall back to FTP if FTPS is not supported by the target server. For security reasons,
1864 this option is not asserted by default. The default behaviour is to exit with an error.
1865 If a server does not successfully reply to the initial \f(CW\*(C`AUTH TLS\*(C'\fR command, or in the
1866 case of implicit FTPS, if the initial SSL/TLS connection attempt is rejected, it is
1867 considered that such server does not support FTPS.
1868 .SS "Recursive Retrieval Options"
1869 .IX Subsection "Recursive Retrieval Options"
1870 .IP \fB\-r\fR 4
1871 .IX Item "-r"
1872 .PD 0
1873 .IP \fB\-\-recursive\fR 4
1874 .IX Item "--recursive"
1875 .PD
1876 Turn on recursive retrieving. The default maximum depth is 5.
1877 .IP "\fB\-l\fR \fIdepth\fR" 4
1878 .IX Item "-l depth"
1879 .PD 0
1880 .IP \fB\-\-level=\fR\fIdepth\fR 4
1881 .IX Item "--level=depth"
1882 .PD
1883 Set the maximum number of subdirectories that Wget will recurse into to \fIdepth\fR.
1884 In order to prevent one from accidentally downloading very large websites when using recursion
1885 this is limited to a depth of 5 by default, i.e., it will traverse at most 5 directories deep
1886 starting from the provided URL.
1887 Set \fB\-l 0\fR or \fB\-l inf\fR for infinite recursion depth.
1888 .Sp
1889 .Vb 1
1890 \& wget \-r \-l 0 http://<site>/1.html
1891 .Ve
1892 .Sp
1893 Ideally, one would expect this to download just \fI1.html\fR.
1894 but unfortunately this is not the case, because \fB\-l 0\fR is equivalent to
1895 \&\fB\-l inf\fR\-\-\-that is, infinite recursion. To download a single HTML
1896 page (or a handful of them), specify them all on the command line and leave away \fB\-r\fR
1897 and \fB\-l\fR. To download the essential items to view a single HTML page, see \fBpage requisites\fR.
1898 .IP \fB\-\-delete\-after\fR 4
1899 .IX Item "--delete-after"
1900 This option tells Wget to delete every single file it downloads,
1901 \&\fIafter\fR having done so. It is useful for pre\-fetching popular
1902 pages through a proxy, e.g.:
1903 .Sp
1904 .Vb 1
1905 \& wget \-r \-nd \-\-delete\-after http://whatever.com/~popular/page/
1906 .Ve
1907 .Sp
1908 The \fB\-r\fR option is to retrieve recursively, and \fB\-nd\fR to not
1909 create directories.
1910 .Sp
1911 Note that \fB\-\-delete\-after\fR deletes files on the local machine. It
1912 does not issue the \fBDELE\fR command to remote FTP sites, for
1913 instance. Also note that when \fB\-\-delete\-after\fR is specified,
1914 \&\fB\-\-convert\-links\fR is ignored, so \fB.orig\fR files are simply not
1915 created in the first place.
1916 .IP \fB\-k\fR 4
1917 .IX Item "-k"
1918 .PD 0
1919 .IP \fB\-\-convert\-links\fR 4
1920 .IX Item "--convert-links"
1921 .PD
1922 After the download is complete, convert the links in the document to
1923 make them suitable for local viewing. This affects not only the visible
1924 hyperlinks, but any part of the document that links to external content,
1925 such as embedded images, links to style sheets, hyperlinks to non\-HTML
1926 content, etc.
1927 .Sp
1928 Each link will be changed in one of the two ways:
1929 .RS 4
1930 .IP \(bu 4
1931 The links to files that have been downloaded by Wget will be changed to
1932 refer to the file they point to as a relative link.
1933 .Sp
1934 Example: if the downloaded file \fI/foo/doc.html\fR links to
1935 \&\fI/bar/img.gif\fR, also downloaded, then the link in \fIdoc.html\fR
1936 will be modified to point to \fB../bar/img.gif\fR. This kind of
1937 transformation works reliably for arbitrary combinations of directories.
1938 .IP \(bu 4
1939 The links to files that have not been downloaded by Wget will be changed
1940 to include host name and absolute path of the location they point to.
1941 .Sp
1942 Example: if the downloaded file \fI/foo/doc.html\fR links to
1943 \&\fI/bar/img.gif\fR (or to \fI../bar/img.gif\fR), then the link in
1944 \&\fIdoc.html\fR will be modified to point to
1945 \&\fIhttp://hostname/bar/img.gif\fR.
1946 .RE
1947 .RS 4
1948 .Sp
1949 Because of this, local browsing works reliably: if a linked file was
1950 downloaded, the link will refer to its local name; if it was not
1951 downloaded, the link will refer to its full Internet address rather than
1952 presenting a broken link. The fact that the former links are converted
1953 to relative links ensures that you can move the downloaded hierarchy to
1954 another directory.
1955 .Sp
1956 Note that only at the end of the download can Wget know which links have
1957 been downloaded. Because of that, the work done by \fB\-k\fR will be
1958 performed at the end of all the downloads.
1959 .RE
1960 .IP \fB\-\-convert\-file\-only\fR 4
1961 .IX Item "--convert-file-only"
1962 This option converts only the filename part of the URLs, leaving the rest
1963 of the URLs untouched. This filename part is sometimes referred to as the
1964 "basename", although we avoid that term here in order not to cause confusion.
1965 .Sp
1966 It works particularly well in conjunction with \fB\-\-adjust\-extension\fR, although
1967 this coupling is not enforced. It proves useful to populate Internet caches
1968 with files downloaded from different hosts.
1969 .Sp
1970 Example: if some link points to \fI//foo.com/bar.cgi?xyz\fR with
1971 \&\fB\-\-adjust\-extension\fR asserted and its local destination is intended to be
1972 \&\fI./foo.com/bar.cgi?xyz.css\fR, then the link would be converted to
1973 \&\fI//foo.com/bar.cgi?xyz.css\fR. Note that only the filename part has been
1974 modified. The rest of the URL has been left untouched, including the net path
1975 (\f(CW\*(C`//\*(C'\fR) which would otherwise be processed by Wget and converted to the
1976 effective scheme (ie. \f(CW\*(C`http://\*(C'\fR).
1977 .IP \fB\-K\fR 4
1978 .IX Item "-K"
1979 .PD 0
1980 .IP \fB\-\-backup\-converted\fR 4
1981 .IX Item "--backup-converted"
1982 .PD
1983 When converting a file, back up the original version with a \fB.orig\fR
1984 suffix. Affects the behavior of \fB\-N\fR.
1985 .IP \fB\-m\fR 4
1986 .IX Item "-m"
1987 .PD 0
1988 .IP \fB\-\-mirror\fR 4
1989 .IX Item "--mirror"
1990 .PD
1991 Turn on options suitable for mirroring. This option turns on recursion
1992 and time\-stamping, sets infinite recursion depth and keeps FTP
1993 directory listings. It is currently equivalent to
1994 \&\fB\-r \-N \-l inf \-\-no\-remove\-listing\fR.
1995 .IP \fB\-p\fR 4
1996 .IX Item "-p"
1997 .PD 0
1998 .IP \fB\-\-page\-requisites\fR 4
1999 .IX Item "--page-requisites"
2000 .PD
2001 This option causes Wget to download all the files that are necessary to
2002 properly display a given HTML page. This includes such things as
2003 inlined images, sounds, and referenced stylesheets.
2004 .Sp
2005 Ordinarily, when downloading a single HTML page, any requisite documents
2006 that may be needed to display it properly are not downloaded. Using
2007 \&\fB\-r\fR together with \fB\-l\fR can help, but since Wget does not
2008 ordinarily distinguish between external and inlined documents, one is
2009 generally left with "leaf documents" that are missing their
2010 requisites.
2011 .Sp
2012 For instance, say document \fI1.html\fR contains an \f(CW\*(C`<IMG>\*(C'\fR tag
2013 referencing \fI1.gif\fR and an \f(CW\*(C`<A>\*(C'\fR tag pointing to external
2014 document \fI2.html\fR. Say that \fI2.html\fR is similar but that its
2015 image is \fI2.gif\fR and it links to \fI3.html\fR. Say this
2016 continues up to some arbitrarily high number.
2017 .Sp
2018 If one executes the command:
2019 .Sp
2020 .Vb 1
2021 \& wget \-r \-l 2 http://<site>/1.html
2022 .Ve
2023 .Sp
2024 then \fI1.html\fR, \fI1.gif\fR, \fI2.html\fR, \fI2.gif\fR, and
2025 \&\fI3.html\fR will be downloaded. As you can see, \fI3.html\fR is
2026 without its requisite \fI3.gif\fR because Wget is simply counting the
2027 number of hops (up to 2) away from \fI1.html\fR in order to determine
2028 where to stop the recursion. However, with this command:
2029 .Sp
2030 .Vb 1
2031 \& wget \-r \-l 2 \-p http://<site>/1.html
2032 .Ve
2033 .Sp
2034 all the above files \fIand\fR \fI3.html\fR\*(Aqs requisite \fI3.gif\fR
2035 will be downloaded. Similarly,
2036 .Sp
2037 .Vb 1
2038 \& wget \-r \-l 1 \-p http://<site>/1.html
2039 .Ve
2040 .Sp
2041 will cause \fI1.html\fR, \fI1.gif\fR, \fI2.html\fR, and \fI2.gif\fR
2042 to be downloaded. One might think that:
2043 .Sp
2044 .Vb 1
2045 \& wget \-r \-l 0 \-p http://<site>/1.html
2046 .Ve
2047 .Sp
2048 would download just \fI1.html\fR and \fI1.gif\fR, but unfortunately
2049 this is not the case, because \fB\-l 0\fR is equivalent to
2050 \&\fB\-l inf\fR\-\-\-that is, infinite recursion. To download a single HTML
2051 page (or a handful of them, all specified on the command\-line or in a
2052 \&\fB\-i\fR URL input file) and its (or their) requisites, simply leave off
2053 \&\fB\-r\fR and \fB\-l\fR:
2054 .Sp
2055 .Vb 1
2056 \& wget \-p http://<site>/1.html
2057 .Ve
2058 .Sp
2059 Note that Wget will behave as if \fB\-r\fR had been specified, but only
2060 that single page and its requisites will be downloaded. Links from that
2061 page to external documents will not be followed. Actually, to download
2062 a single page and all its requisites (even if they exist on separate
2063 websites), and make sure the lot displays properly locally, this author
2064 likes to use a few options in addition to \fB\-p\fR:
2065 .Sp
2066 .Vb 1
2067 \& wget \-E \-H \-k \-K \-p http://<site>/<document>
2068 .Ve
2069 .Sp
2070 To finish off this topic, it\*(Aqs worth knowing that Wget\*(Aqs idea of an
2071 external document link is any URL specified in an \f(CW\*(C`<A>\*(C'\fR tag, an
2072 \&\f(CW\*(C`<AREA>\*(C'\fR tag, or a \f(CW\*(C`<LINK>\*(C'\fR tag other than \f(CW\*(C`<LINK
2073 REL="stylesheet">\*(C'\fR.
2074 .IP \fB\-\-strict\-comments\fR 4
2075 .IX Item "--strict-comments"
2076 Turn on strict parsing of HTML comments. The default is to terminate
2077 comments at the first occurrence of \fB\-\->\fR.
2078 .Sp
2079 According to specifications, HTML comments are expressed as SGML
2080 \&\fIdeclarations\fR. Declaration is special markup that begins with
2081 \&\fB<!\fR and ends with \fB>\fR, such as \fB<!DOCTYPE ...>\fR, that
2082 may contain comments between a pair of \fB\-\-\fR delimiters. HTML
2083 comments are "empty declarations", SGML declarations without any
2084 non\-comment text. Therefore, \fB<!\-\-foo\-\->\fR is a valid comment, and
2085 so is \fB<!\-\-one\-\- \-\-two\-\->\fR, but \fB<!\-\-1\-\-2\-\->\fR is not.
2086 .Sp
2087 On the other hand, most HTML writers don\*(Aqt perceive comments as anything
2088 other than text delimited with \fB<!\-\-\fR and \fB\-\->\fR, which is not
2089 quite the same. For example, something like \fB<!\-\-\-\-\-\-\-\-\-\-\-\->\fR
2090 works as a valid comment as long as the number of dashes is a multiple
2091 of four (!). If not, the comment technically lasts until the next
2092 \&\fB\-\-\fR, which may be at the other end of the document. Because of
2093 this, many popular browsers completely ignore the specification and
2094 implement what users have come to expect: comments delimited with
2095 \&\fB<!\-\-\fR and \fB\-\->\fR.
2096 .Sp
2097 Until version 1.9, Wget interpreted comments strictly, which resulted in
2098 missing links in many web pages that displayed fine in browsers, but had
2099 the misfortune of containing non\-compliant comments. Beginning with
2100 version 1.9, Wget has joined the ranks of clients that implements
2101 "naive" comments, terminating each comment at the first occurrence of
2102 \&\fB\-\->\fR.
2103 .Sp
2104 If, for whatever reason, you want strict comment parsing, use this
2105 option to turn it on.
2106 .SS "Recursive Accept/Reject Options"
2107 .IX Subsection "Recursive Accept/Reject Options"
2108 .IP "\fB\-A\fR \fIacclist\fR \fB\-\-accept\fR \fIacclist\fR" 4
2109 .IX Item "-A acclist --accept acclist"
2110 .PD 0
2111 .IP "\fB\-R\fR \fIrejlist\fR \fB\-\-reject\fR \fIrejlist\fR" 4
2112 .IX Item "-R rejlist --reject rejlist"
2113 .PD
2114 Specify comma\-separated lists of file name suffixes or patterns to
2115 accept or reject. Note that if
2116 any of the wildcard characters, \fB*\fR, \fB?\fR, \fB[\fR or
2117 \&\fB]\fR, appear in an element of \fIacclist\fR or \fIrejlist\fR,
2118 it will be treated as a pattern, rather than a suffix.
2119 In this case, you have to enclose the pattern into quotes to prevent
2120 your shell from expanding it, like in \fB\-A "*.mp3"\fR or \fB\-A \*(Aq*.mp3\*(Aq\fR.
2121 .IP "\fB\-\-accept\-regex\fR \fIurlregex\fR" 4
2122 .IX Item "--accept-regex urlregex"
2123 .PD 0
2124 .IP "\fB\-\-reject\-regex\fR \fIurlregex\fR" 4
2125 .IX Item "--reject-regex urlregex"
2126 .PD
2127 Specify a regular expression to accept or reject the complete URL.
2128 .IP "\fB\-\-regex\-type\fR \fIregextype\fR" 4
2129 .IX Item "--regex-type regextype"
2130 Specify the regular expression type. Possible types are \fBposix\fR or
2131 \&\fBpcre\fR. Note that to be able to use \fBpcre\fR type, wget has to be
2132 compiled with libpcre support.
2133 .IP "\fB\-D\fR \fIdomain\-list\fR" 4
2134 .IX Item "-D domain-list"
2135 .PD 0
2136 .IP \fB\-\-domains=\fR\fIdomain\-list\fR 4
2137 .IX Item "--domains=domain-list"
2138 .PD
2139 Set domains to be followed. \fIdomain\-list\fR is a comma\-separated list
2140 of domains. Note that it does \fInot\fR turn on \fB\-H\fR.
2141 .IP "\fB\-\-exclude\-domains\fR \fIdomain\-list\fR" 4
2142 .IX Item "--exclude-domains domain-list"
2143 Specify the domains that are \fInot\fR to be followed.
2144 .IP \fB\-\-follow\-ftp\fR 4
2145 .IX Item "--follow-ftp"
2146 Follow FTP links from HTML documents. Without this option,
2147 Wget will ignore all the FTP links.
2148 .IP \fB\-\-follow\-tags=\fR\fIlist\fR 4
2149 .IX Item "--follow-tags=list"
2150 Wget has an internal table of HTML tag / attribute pairs that it
2151 considers when looking for linked documents during a recursive
2152 retrieval. If a user wants only a subset of those tags to be
2153 considered, however, he or she should be specify such tags in a
2154 comma\-separated \fIlist\fR with this option.
2155 .IP \fB\-\-ignore\-tags=\fR\fIlist\fR 4
2156 .IX Item "--ignore-tags=list"
2157 This is the opposite of the \fB\-\-follow\-tags\fR option. To skip
2158 certain HTML tags when recursively looking for documents to download,
2159 specify them in a comma\-separated \fIlist\fR.
2160 .Sp
2161 In the past, this option was the best bet for downloading a single page
2162 and its requisites, using a command\-line like:
2163 .Sp
2164 .Vb 1
2165 \& wget \-\-ignore\-tags=a,area \-H \-k \-K \-r http://<site>/<document>
2166 .Ve
2167 .Sp
2168 However, the author of this option came across a page with tags like
2169 \&\f(CW\*(C`<LINK REL="home" HREF="/">\*(C'\fR and came to the realization that
2170 specifying tags to ignore was not enough. One can\*(Aqt just tell Wget to
2171 ignore \f(CW\*(C`<LINK>\*(C'\fR, because then stylesheets will not be downloaded.
2172 Now the best bet for downloading a single page and its requisites is the
2173 dedicated \fB\-\-page\-requisites\fR option.
2174 .IP \fB\-\-ignore\-case\fR 4
2175 .IX Item "--ignore-case"
2176 Ignore case when matching files and directories. This influences the
2177 behavior of \-R, \-A, \-I, and \-X options, as well as globbing
2178 implemented when downloading from FTP sites. For example, with this
2179 option, \fB\-A "*.txt"\fR will match \fBfile1.txt\fR, but also
2180 \&\fBfile2.TXT\fR, \fBfile3.TxT\fR, and so on.
2181 The quotes in the example are to prevent the shell from expanding the
2182 pattern.
2183 .IP \fB\-H\fR 4
2184 .IX Item "-H"
2185 .PD 0
2186 .IP \fB\-\-span\-hosts\fR 4
2187 .IX Item "--span-hosts"
2188 .PD
2189 Enable spanning across hosts when doing recursive retrieving.
2190 .IP \fB\-L\fR 4
2191 .IX Item "-L"
2192 .PD 0
2193 .IP \fB\-\-relative\fR 4
2194 .IX Item "--relative"
2195 .PD
2196 Follow relative links only. Useful for retrieving a specific home page
2197 without any distractions, not even those from the same hosts.
2198 .IP "\fB\-I\fR \fIlist\fR" 4
2199 .IX Item "-I list"
2200 .PD 0
2201 .IP \fB\-\-include\-directories=\fR\fIlist\fR 4
2202 .IX Item "--include-directories=list"
2203 .PD
2204 Specify a comma\-separated list of directories you wish to follow when
2205 downloading. Elements
2206 of \fIlist\fR may contain wildcards.
2207 .IP "\fB\-X\fR \fIlist\fR" 4
2208 .IX Item "-X list"
2209 .PD 0
2210 .IP \fB\-\-exclude\-directories=\fR\fIlist\fR 4
2211 .IX Item "--exclude-directories=list"
2212 .PD
2213 Specify a comma\-separated list of directories you wish to exclude from
2214 download. Elements of
2215 \&\fIlist\fR may contain wildcards.
2216 .IP \fB\-np\fR 4
2217 .IX Item "-np"
2218 .PD 0
2219 .IP \fB\-\-no\-parent\fR 4
2220 .IX Item "--no-parent"
2221 .PD
2222 Do not ever ascend to the parent directory when retrieving recursively.
2223 This is a useful option, since it guarantees that only the files
2224 \&\fIbelow\fR a certain hierarchy will be downloaded.
2225 .SH ENVIRONMENT
2226 .IX Header "ENVIRONMENT"
2227 Wget supports proxies for both HTTP and FTP retrievals. The
2228 standard way to specify proxy location, which Wget recognizes, is using
2229 the following environment variables:
2230 .IP \fBhttp_proxy\fR 4
2231 .IX Item "http_proxy"
2232 .PD 0
2233 .IP \fBhttps_proxy\fR 4
2234 .IX Item "https_proxy"
2235 .PD
2236 If set, the \fBhttp_proxy\fR and \fBhttps_proxy\fR variables should
2237 contain the URLs of the proxies for HTTP and HTTPS
2238 connections respectively.
2239 .IP \fBftp_proxy\fR 4
2240 .IX Item "ftp_proxy"
2241 This variable should contain the URL of the proxy for FTP
2242 connections. It is quite common that \fBhttp_proxy\fR and
2243 \&\fBftp_proxy\fR are set to the same URL.
2244 .IP \fBno_proxy\fR 4
2245 .IX Item "no_proxy"
2246 This variable should contain a comma\-separated list of domain extensions
2247 proxy should \fInot\fR be used for. For instance, if the value of
2248 \&\fBno_proxy\fR is \fB.mit.edu\fR, proxy will not be used to retrieve
2249 documents from MIT.
2250 .SH "EXIT STATUS"
2251 .IX Header "EXIT STATUS"
2252 Wget may return one of several error codes if it encounters problems.
2253 .ie n .IP 0 4
2254 .el .IP \f(CW0\fR 4
2255 .IX Item "0"
2256 No problems occurred.
2257 .ie n .IP 1 4
2258 .el .IP \f(CW1\fR 4
2259 .IX Item "1"
2260 Generic error code.
2261 .ie n .IP 2 4
2262 .el .IP \f(CW2\fR 4
2263 .IX Item "2"
2264 Parse error\-\-\-for instance, when parsing command\-line options, the
2265 \&\fB.wgetrc\fR or \fB.netrc\fR...
2266 .ie n .IP 3 4
2267 .el .IP \f(CW3\fR 4
2268 .IX Item "3"
2269 File I/O error.
2270 .ie n .IP 4 4
2271 .el .IP \f(CW4\fR 4
2272 .IX Item "4"
2273 Network failure.
2274 .ie n .IP 5 4
2275 .el .IP \f(CW5\fR 4
2276 .IX Item "5"
2277 SSL verification failure.
2278 .ie n .IP 6 4
2279 .el .IP \f(CW6\fR 4
2280 .IX Item "6"
2281 Username/password authentication failure.
2282 .ie n .IP 7 4
2283 .el .IP \f(CW7\fR 4
2284 .IX Item "7"
2285 Protocol errors.
2286 .ie n .IP 8 4
2287 .el .IP \f(CW8\fR 4
2288 .IX Item "8"
2289 Server issued an error response.
2290 .PP
2291 With the exceptions of 0 and 1, the lower\-numbered exit codes take
2292 precedence over higher\-numbered ones, when multiple types of errors
2293 are encountered.
2294 .PP
2295 In versions of Wget prior to 1.12, Wget\*(Aqs exit status tended to be
2296 unhelpful and inconsistent. Recursive downloads would virtually always
2297 return 0 (success), regardless of any issues encountered, and
2298 non\-recursive fetches only returned the status corresponding to the
2299 most recently\-attempted download.
2300 .SH FILES
2301 .IX Header "FILES"
2302 .IP \fB/usr/local/etc/wgetrc\fR 4
2303 .IX Item "/usr/local/etc/wgetrc"
2304 Default location of the \fIglobal\fR startup file.
2305 .IP \fB.wgetrc\fR 4
2306 .IX Item ".wgetrc"
2307 User startup file.
2308 .SH BUGS
2309 .IX Header "BUGS"
2310 You are welcome to submit bug reports via the GNU Wget bug tracker (see
2311 <\fBhttps://savannah.gnu.org/bugs/?func=additem&group=wget\fR>) or to our
2312 mailing list <\fBbug\[email protected]\fR>.
2313 .PP
2314 Visit <\fBhttps://lists.gnu.org/mailman/listinfo/bug\-wget\fR> to
2315 get more info (how to subscribe, list archives, ...).
2316 .PP
2317 Before actually submitting a bug report, please try to follow a few
2318 simple guidelines.
2319 .IP 1. 4
2320 Please try to ascertain that the behavior you see really is a bug. If
2321 Wget crashes, it\*(Aqs a bug. If Wget does not behave as documented,
2322 it\*(Aqs a bug. If things work strange, but you are not sure about the way
2323 they are supposed to work, it might well be a bug, but you might want to
2324 double\-check the documentation and the mailing lists.
2325 .IP 2. 4
2326 Try to repeat the bug in as simple circumstances as possible. E.g. if
2327 Wget crashes while downloading \fBwget \-rl0 \-kKE \-t5 \-\-no\-proxy
2328 http://example.com \-o /tmp/log\fR, you should try to see if the crash is
2329 repeatable, and if will occur with a simpler set of options. You might
2330 even try to start the download at the page where the crash occurred to
2331 see if that page somehow triggered the crash.
2332 .Sp
2333 Also, while I will probably be interested to know the contents of your
2334 \&\fI.wgetrc\fR file, just dumping it into the debug message is probably
2335 a bad idea. Instead, you should first try to see if the bug repeats
2336 with \fI.wgetrc\fR moved out of the way. Only if it turns out that
2337 \&\fI.wgetrc\fR settings affect the bug, mail me the relevant parts of
2338 the file.
2339 .IP 3. 4
2340 Please start Wget with \fB\-d\fR option and send us the resulting
2341 output (or relevant parts thereof). If Wget was compiled without
2342 debug support, recompile it\-\-\-it is \fImuch\fR easier to trace bugs
2343 with debug support on.
2344 .Sp
2345 Note: please make sure to remove any potentially sensitive information
2346 from the debug log before sending it to the bug address. The
2347 \&\f(CW\*(C`\-d\*(C'\fR won\*(Aqt go out of its way to collect sensitive information,
2348 but the log \fIwill\fR contain a fairly complete transcript of Wget\*(Aqs
2349 communication with the server, which may include passwords and pieces
2350 of downloaded data. Since the bug address is publicly archived, you
2351 may assume that all bug reports are visible to the public.
2352 .IP 4. 4
2353 If Wget has crashed, try to run it in a debugger, e.g. \f(CW\*(C`gdb \`which
2354 wget\` core\*(C'\fR and type \f(CW\*(C`where\*(C'\fR to get the backtrace. This may not
2355 work if the system administrator has disabled core files, but it is
2356 safe to try.
2357 .SH "SEE ALSO"
2358 .IX Header "SEE ALSO"
2359 This is \fBnot\fR the complete manual for GNU Wget.
2360 For more complete information, including more detailed explanations of
2361 some of the options, and a number of commands available
2362 for use with \fI.wgetrc\fR files and the \fB\-e\fR option, see the GNU
2363 Info entry for \fIwget\fR.
2364 .PP
2365 Also see \fBwget2\fR\|(1), the updated version of GNU Wget with even better
2366 support for recursive downloading and modern protocols like HTTP/2.
2367 .SH AUTHOR
2368 .IX Header "AUTHOR"
2369 Originally written by Hrvoje Nikšić <[email protected]>.
2370 Currently maintained by Darshit Shah <[email protected]> and
2371 Tim Rühsen <[email protected]>.
2372 .SH COPYRIGHT
2373 .IX Header "COPYRIGHT"
2374 Copyright (c) 1996\-\-2011, 2015, 2018\-\-2024 Free Software
2375 Foundation, Inc.
2376 .PP
2377 Permission is granted to copy, distribute and/or modify this document
2378 under the terms of the GNU Free Documentation License, Version 1.3 or
2379 any later version published by the Free Software Foundation; with no
2380 Invariant Sections, with no Front\-Cover Texts, and with no Back\-Cover
2381 Texts. A copy of the license is included in the section entitled
2382 "GNU Free Documentation License".