| 1 |
_ _ ____ _ |
| 2 |
___| | | | _ \| | |
| 3 |
/ __| | | | |_) | | |
| 4 |
| (__| |_| | _ <| |___ |
| 5 |
\___|\___/|_| \_\_____| |
| 6 |
|
| 7 |
FAQ |
| 8 |
|
| 9 |
1. Philosophy |
| 10 |
1.1 What is cURL? |
| 11 |
1.2 What is libcurl? |
| 12 |
1.3 What is curl not? |
| 13 |
1.4 When will you make curl do XXXX ? |
| 14 |
1.5 Who makes curl? |
| 15 |
1.6 What do you get for making curl? |
| 16 |
1.7 What about CURL from curl.com? |
| 17 |
1.8 I have a problem who do I mail? |
| 18 |
1.9 Where do I buy commercial support for curl? |
| 19 |
1.10 How many are using curl? |
| 20 |
1.11 Why don't you update ca-bundle.crt |
| 21 |
1.12 I have a problem who can I chat with? |
| 22 |
1.13 curl's ECCN number? |
| 23 |
1.14 How do I submit my patch? |
| 24 |
1.15 How do I port libcurl to my OS? |
| 25 |
|
| 26 |
2. Install Related Problems |
| 27 |
2.1 configure doesn't find OpenSSL even when it is installed |
| 28 |
2.1.1 native linker doesn't find OpenSSL |
| 29 |
2.1.2 only the libssl lib is missing |
| 30 |
2.2 Does curl work/build with other SSL libraries? |
| 31 |
2.3 Where can I find a copy of LIBEAY32.DLL? |
| 32 |
2.4 Does curl support SOCKS (RFC 1928) ? |
| 33 |
|
| 34 |
3. Usage Problems |
| 35 |
3.1 curl: (1) SSL is disabled, https: not supported |
| 36 |
3.2 How do I tell curl to resume a transfer? |
| 37 |
3.3 Why doesn't my posting using -F work? |
| 38 |
3.4 How do I tell curl to run custom FTP commands? |
| 39 |
3.5 How can I disable the Accept: */* header? |
| 40 |
3.6 Does curl support ASP, XML, XHTML or HTML version Y? |
| 41 |
3.7 Can I use curl to delete/rename a file through FTP? |
| 42 |
3.8 How do I tell curl to follow HTTP redirects? |
| 43 |
3.9 How do I use curl in my favorite programming language? |
| 44 |
3.10 What about SOAP, WebDAV, XML-RPC or similar protocols over HTTP? |
| 45 |
3.11 How do I POST with a different Content-Type? |
| 46 |
3.12 Why do FTP-specific features over HTTP proxy fail? |
| 47 |
3.13 Why do my single/double quotes fail? |
| 48 |
3.14 Does curl support Javascript or PAC (automated proxy config)? |
| 49 |
3.15 Can I do recursive fetches with curl? |
| 50 |
3.16 What certificates do I need when I use SSL? |
| 51 |
3.17 How do I list the root dir of an FTP server? |
| 52 |
3.18 Can I use curl to send a POST/PUT and not wait for a response? |
| 53 |
3.19 How do I get HTTP from a host using a specific IP address? |
| 54 |
3.20 How to SFTP from my user's home directory? |
| 55 |
3.21 Protocol xxx not supported or disabled in libcurl |
| 56 |
3.22 curl -X gives me HTTP problems |
| 57 |
|
| 58 |
4. Running Problems |
| 59 |
4.1 Problems connecting to SSL servers. |
| 60 |
4.2 Why do I get problems when I use & or % in the URL? |
| 61 |
4.3 How can I use {, }, [ or ] to specify multiple URLs? |
| 62 |
4.4 Why do I get downloaded data even though the web page doesn't exist? |
| 63 |
4.5 Why do I get return code XXX from a HTTP server? |
| 64 |
4.5.1 "400 Bad Request" |
| 65 |
4.5.2 "401 Unauthorized" |
| 66 |
4.5.3 "403 Forbidden" |
| 67 |
4.5.4 "404 Not Found" |
| 68 |
4.5.5 "405 Method Not Allowed" |
| 69 |
4.5.6 "301 Moved Permanently" |
| 70 |
4.6 Can you tell me what error code 142 means? |
| 71 |
4.7 How do I keep user names and passwords secret in Curl command lines? |
| 72 |
4.8 I found a bug! |
| 73 |
4.9 Curl can't authenticate to the server that requires NTLM? |
| 74 |
4.10 My HTTP request using HEAD, PUT or DELETE doesn't work! |
| 75 |
4.11 Why do my HTTP range requests return the full document? |
| 76 |
4.12 Why do I get "certificate verify failed" ? |
| 77 |
4.13 Why is curl -R on Windows one hour off? |
| 78 |
4.14 Redirects work in browser but not with curl! |
| 79 |
4.15 FTPS doesn't work |
| 80 |
4.16 My HTTP POST or PUT requests are slow! |
| 81 |
4.17 Non-functional connect timeouts on Windows |
| 82 |
4.18 file:// URLs containing drive letters (Windows, NetWare) |
| 83 |
4.19 Why doesn't curl return an error when the network cable is unplugged? |
| 84 |
4.20 curl doesn't return error for HTTP non-200 responses! |
| 85 |
4.21 Why is there a HTTP/1.1 in my HTTP/2 request? |
| 86 |
|
| 87 |
5. libcurl Issues |
| 88 |
5.1 Is libcurl thread-safe? |
| 89 |
5.2 How can I receive all data into a large memory chunk? |
| 90 |
5.3 How do I fetch multiple files with libcurl? |
| 91 |
5.4 Does libcurl do Winsock initing on win32 systems? |
| 92 |
5.5 Does CURLOPT_WRITEDATA and CURLOPT_READDATA work on win32 ? |
| 93 |
5.6 What about Keep-Alive or persistent connections? |
| 94 |
5.7 Link errors when building libcurl on Windows! |
| 95 |
5.8 libcurl.so.X: open failed: No such file or directory |
| 96 |
5.9 How does libcurl resolve host names? |
| 97 |
5.10 How do I prevent libcurl from writing the response to stdout? |
| 98 |
5.11 How do I make libcurl not receive the whole HTTP response? |
| 99 |
5.12 Can I make libcurl fake or hide my real IP address? |
| 100 |
5.13 How do I stop an ongoing transfer? |
| 101 |
5.14 Using C++ non-static functions for callbacks? |
| 102 |
5.15 How do I get an FTP directory listing? |
| 103 |
5.16 I want a different time-out! |
| 104 |
5.17 Can I write a server with libcurl? |
| 105 |
5.18 Does libcurl use threads? |
| 106 |
|
| 107 |
6. License Issues |
| 108 |
6.1 I have a GPL program, can I use the libcurl library? |
| 109 |
6.2 I have a closed-source program, can I use the libcurl library? |
| 110 |
6.3 I have a BSD licensed program, can I use the libcurl library? |
| 111 |
6.4 I have a program that uses LGPL libraries, can I use libcurl? |
| 112 |
6.5 Can I modify curl/libcurl for my program and keep the changes secret? |
| 113 |
6.6 Can you please change the curl/libcurl license to XXXX? |
| 114 |
6.7 What are my obligations when using libcurl in my commercial apps? |
| 115 |
|
| 116 |
7. PHP/CURL Issues |
| 117 |
7.1 What is PHP/CURL? |
| 118 |
7.2 Who wrote PHP/CURL? |
| 119 |
7.3 Can I perform multiple requests using the same handle? |
| 120 |
7.4 Does PHP/CURL have dependencies? |
| 121 |
|
| 122 |
============================================================================== |
| 123 |
|
| 124 |
1. Philosophy |
| 125 |
|
| 126 |
1.1 What is cURL? |
| 127 |
|
| 128 |
cURL is the name of the project. The name is a play on 'Client for URLs', |
| 129 |
originally with URL spelled in uppercase to make it obvious it deals with |
| 130 |
URLs. The fact it can also be pronounced 'see URL' also helped, it works as |
| 131 |
an abbreviation for "Client URL Request Library" or why not the recursive |
| 132 |
version: "Curl URL Request Library". |
| 133 |
|
| 134 |
The cURL project produces two products: |
| 135 |
|
| 136 |
libcurl |
| 137 |
|
| 138 |
A free and easy-to-use client-side URL transfer library, supporting DICT, |
| 139 |
FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, |
| 140 |
POP3S, RTMP, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET and TFTP. |
| 141 |
|
| 142 |
libcurl supports HTTPS certificates, HTTP POST, HTTP PUT, FTP uploading, |
| 143 |
Kerberos, SPNEGO, HTTP form based upload, proxies, cookies, user+password |
| 144 |
authentication, file transfer resume, http proxy tunneling and more! |
| 145 |
|
| 146 |
libcurl is highly portable, it builds and works identically on numerous |
| 147 |
platforms, including Solaris, NetBSD, FreeBSD, OpenBSD, Darwin, HP-UX, |
| 148 |
IRIX, AIX, Tru64, Linux, UnixWare, HURD, Windows, Amiga, OS/2, BeOS, Mac |
| 149 |
OS X, Ultrix, QNX, OpenVMS, RISC OS, Novell NetWare, DOS, Symbian, OSF, |
| 150 |
Android, Minix, IBM TPF and more... |
| 151 |
|
| 152 |
libcurl is free, thread-safe, IPv6 compatible, feature rich, well |
| 153 |
supported and fast. |
| 154 |
|
| 155 |
curl |
| 156 |
|
| 157 |
A command line tool for getting or sending files using URL syntax. |
| 158 |
|
| 159 |
Since curl uses libcurl, curl supports the same wide range of common |
| 160 |
Internet protocols that libcurl does. |
| 161 |
|
| 162 |
We pronounce curl with an initial k sound. It rhymes with words like girl |
| 163 |
and earl. This is a short WAV file to help you: |
| 164 |
|
| 165 |
https://media.merriam-webster.com/soundc11/c/curl0001.wav |
| 166 |
|
| 167 |
There are numerous sub-projects and related projects that also use the word |
| 168 |
curl in the project names in various combinations, but you should take |
| 169 |
notice that this FAQ is directed at the command-line tool named curl (and |
| 170 |
libcurl the library), and may therefore not be valid for other curl-related |
| 171 |
projects. (There is however a small section for the PHP/CURL in this FAQ.) |
| 172 |
|
| 173 |
1.2 What is libcurl? |
| 174 |
|
| 175 |
libcurl is a reliable and portable library which provides you with an easy |
| 176 |
interface to a range of common Internet protocols. |
| 177 |
|
| 178 |
You can use libcurl for free in your application, be it open source, |
| 179 |
commercial or closed-source. |
| 180 |
|
| 181 |
libcurl is most probably the most portable, most powerful and most often |
| 182 |
used C-based multi-platform file transfer library on this planet - be it |
| 183 |
open source or commercial. |
| 184 |
|
| 185 |
1.3 What is curl not? |
| 186 |
|
| 187 |
Curl is not a wget clone. That is a common misconception. Never, during |
| 188 |
curl's development, have we intended curl to replace wget or compete on its |
| 189 |
market. Curl is targeted at single-shot file transfers. |
| 190 |
|
| 191 |
Curl is not a web site mirroring program. If you want to use curl to mirror |
| 192 |
something: fine, go ahead and write a script that wraps around curl to make |
| 193 |
it reality (like curlmirror.pl does). |
| 194 |
|
| 195 |
Curl is not an FTP site mirroring program. Sure, get and send FTP with curl |
| 196 |
but if you want systematic and sequential behavior you should write a |
| 197 |
script (or write a new program that interfaces libcurl) and do it. |
| 198 |
|
| 199 |
Curl is not a PHP tool, even though it works perfectly well when used from |
| 200 |
or with PHP (when using the PHP/CURL module). |
| 201 |
|
| 202 |
Curl is not a program for a single operating system. Curl exists, compiles, |
| 203 |
builds and runs under a wide range of operating systems, including all |
| 204 |
modern Unixes (and a bunch of older ones too), Windows, Amiga, BeOS, OS/2, |
| 205 |
OS X, QNX etc. |
| 206 |
|
| 207 |
1.4 When will you make curl do XXXX ? |
| 208 |
|
| 209 |
We love suggestions of what to change in order to make curl and libcurl |
| 210 |
better. We do however believe in a few rules when it comes to the future of |
| 211 |
curl: |
| 212 |
|
| 213 |
Curl -- the command line tool -- is to remain a non-graphical command line |
| 214 |
tool. If you want GUIs or fancy scripting capabilities, you should look for |
| 215 |
another tool that uses libcurl. |
| 216 |
|
| 217 |
We do not add things to curl that other small and available tools already do |
| 218 |
very well at the side. Curl's output can be piped into another program or |
| 219 |
redirected to another file for the next program to interpret. |
| 220 |
|
| 221 |
We focus on protocol related issues and improvements. If you want to do more |
| 222 |
magic with the supported protocols than curl currently does, chances are good |
| 223 |
we will agree. If you want to add more protocols, we may very well agree. |
| 224 |
|
| 225 |
If you want someone else to do all the work while you wait for us to |
| 226 |
implement it for you, that is not a very friendly attitude. We spend a |
| 227 |
considerable time already on maintaining and developing curl. In order to |
| 228 |
get more out of us, you should consider trading in some of your time and |
| 229 |
effort in return. Simply go to the GitHub repo which resides at |
| 230 |
https://github.com/curl/curl, fork the project, and create pull requests |
| 231 |
with your proposed changes. |
| 232 |
|
| 233 |
If you write the code, chances are better that it will get into curl faster. |
| 234 |
|
| 235 |
1.5 Who makes curl? |
| 236 |
|
| 237 |
curl and libcurl are not made by any single individual. Daniel Stenberg is |
| 238 |
project leader and main developer, but other persons' submissions are |
| 239 |
important and crucial. Anyone can contribute and post their changes and |
| 240 |
improvements and have them inserted in the main sources (of course on the |
| 241 |
condition that developers agree that the fixes are good). |
| 242 |
|
| 243 |
The full list of all contributors is found in the docs/THANKS file. |
| 244 |
|
| 245 |
curl is developed by a community, with Daniel at the wheel. |
| 246 |
|
| 247 |
1.6 What do you get for making curl? |
| 248 |
|
| 249 |
Project cURL is entirely free and open. No person gets paid for developing |
| 250 |
curl full time. We do this voluntarily, mostly in our spare time. |
| 251 |
Occasionally companies pay individual developers to work on curl, but that's |
| 252 |
up to each company and developer. This is not controlled by nor supervised in |
| 253 |
any way by the project. |
| 254 |
|
| 255 |
We still get help from companies. Haxx provides web site, bandwidth, mailing |
| 256 |
lists etc, GitHub hosts the primary git repository and other services like |
| 257 |
the bug tracker at https://github.com/curl/curl. Also again, some companies |
| 258 |
have sponsored certain parts of the development in the past and I hope some |
| 259 |
will continue to do so in the future. |
| 260 |
|
| 261 |
If you want to support our project, consider a donation or a banner-program |
| 262 |
or even better: by helping us with coding, documenting or testing etc. |
| 263 |
|
| 264 |
1.7 What about CURL from curl.com? |
| 265 |
|
| 266 |
During the summer of 2001, curl.com was busy advertising their client-side |
| 267 |
programming language for the web, named CURL. |
| 268 |
|
| 269 |
We are in no way associated with curl.com or their CURL programming |
| 270 |
language. |
| 271 |
|
| 272 |
Our project name curl has been in effective use since 1998. We were not the |
| 273 |
first computer related project to use the name "curl" and do not claim any |
| 274 |
rights to the name. |
| 275 |
|
| 276 |
We recognize that we will be living in parallel with curl.com and wish them |
| 277 |
every success. |
| 278 |
|
| 279 |
1.8 I have a problem whom do I mail? |
| 280 |
|
| 281 |
Please do not mail any single individual unless you really need to. Keep |
| 282 |
curl-related questions on a suitable mailing list. All available mailing |
| 283 |
lists are listed in the MANUAL document and online at |
| 284 |
https://curl.haxx.se/mail/ |
| 285 |
|
| 286 |
Keeping curl-related questions and discussions on mailing lists allows |
| 287 |
others to join in and help, to share their ideas, to contribute their |
| 288 |
suggestions and to spread their wisdom. Keeping discussions on public mailing |
| 289 |
lists also allows for others to learn from this (both current and future |
| 290 |
users thanks to the web based archives of the mailing lists), thus saving us |
| 291 |
from having to repeat ourselves even more. Thanks for respecting this. |
| 292 |
|
| 293 |
If you have found or simply suspect a security problem in curl or libcurl, |
| 294 |
mail curl-security at haxx.se (closed list of receivers, mails are not |
| 295 |
disclosed) and tell. Then we can produce a fix in a timely manner before the |
| 296 |
flaw is announced to the world, thus lessen the impact the problem will have |
| 297 |
on existing users. |
| 298 |
|
| 299 |
1.9 Where do I buy commercial support for curl? |
| 300 |
|
| 301 |
curl is fully open source. It means you can hire any skilled engineer to fix |
| 302 |
your curl-related problems. |
| 303 |
|
| 304 |
We list available alternatives on the curl web site: |
| 305 |
https://curl.haxx.se/support.html |
| 306 |
|
| 307 |
1.10 How many are using curl? |
| 308 |
|
| 309 |
It is impossible to tell. |
| 310 |
|
| 311 |
We don't know how many users that knowingly have installed and use curl. |
| 312 |
|
| 313 |
We don't know how many users that use curl without knowing that they are in |
| 314 |
fact using it. |
| 315 |
|
| 316 |
We don't know how many users that downloaded or installed curl and then |
| 317 |
never use it. |
| 318 |
|
| 319 |
In May 2012 Daniel did a counting game and came up with a number that may |
| 320 |
be completely wrong or somewhat accurate. Over 500 million! |
| 321 |
|
| 322 |
See https://daniel.haxx.se/blog/2012/05/16/300m-users/ |
| 323 |
|
| 324 |
1.11 Why don't you update ca-bundle.crt |
| 325 |
|
| 326 |
The ca cert bundle that used to be shipped with curl was very outdated and |
| 327 |
must be replaced with an up-to-date version by anyone who wants to verify |
| 328 |
peers. It is no longer provided by curl. The last curl release that ever |
| 329 |
shipped a ca cert bundle was curl 7.18.0. |
| 330 |
|
| 331 |
In the cURL project we've decided not to attempt to keep this file updated |
| 332 |
(or even present anymore) since deciding what to add to a ca cert bundle is |
| 333 |
an undertaking we've not been ready to accept, and the one we can get from |
| 334 |
Mozilla is perfectly fine so there's no need to duplicate that work. |
| 335 |
|
| 336 |
Today, with many services performed over HTTPS, every operating system |
| 337 |
should come with a default ca cert bundle that can be deemed somewhat |
| 338 |
trustworthy and that collection (if reasonably updated) should be deemed to |
| 339 |
be a lot better than a private curl version. |
| 340 |
|
| 341 |
If you want the most recent collection of ca certs that Mozilla Firefox |
| 342 |
uses, we recommend that you extract the collection yourself from Mozilla |
| 343 |
Firefox (by running 'make ca-bundle), or by using our online service setup |
| 344 |
for this purpose: https://curl.haxx.se/docs/caextract.html |
| 345 |
|
| 346 |
1.12 I have a problem who can I chat with? |
| 347 |
|
| 348 |
There's a bunch of friendly people hanging out in the #curl channel on the |
| 349 |
IRC network irc.freenode.net. If you're polite and nice, chances are good |
| 350 |
that you can get -- or provide -- help instantly. |
| 351 |
|
| 352 |
1.13 curl's ECCN number? |
| 353 |
|
| 354 |
The US government restricts exports of software that contains or uses |
| 355 |
cryptography. When doing so, the Export Control Classification Number (ECCN) |
| 356 |
is used to identify the level of export control etc. |
| 357 |
|
| 358 |
Apache Software Foundation gives a good explanation of ECCNs at |
| 359 |
https://www.apache.org/dev/crypto.html |
| 360 |
|
| 361 |
We believe curl's number might be ECCN 5D002, another possibility is |
| 362 |
5D992. It seems necessary to write them (the authority that administers ECCN |
| 363 |
numbers), asking to confirm. |
| 364 |
|
| 365 |
Comprehensible explanations of the meaning of such numbers and how to obtain |
| 366 |
them (resp.) are here |
| 367 |
|
| 368 |
https://www.bis.doc.gov/licensing/exportingbasics.htm |
| 369 |
https://www.bis.doc.gov/licensing/do_i_needaneccn.html |
| 370 |
|
| 371 |
An incomprehensible description of the two numbers above is here |
| 372 |
https://www.bis.doc.gov/index.php/documents/new-encryption/1653-ccl5-pt2-3 |
| 373 |
|
| 374 |
1.14 How do I submit my patch? |
| 375 |
|
| 376 |
When you have made a patch or a change of whatever sort, and want to submit |
| 377 |
that to the project, there are a few different ways we prefer: |
| 378 |
|
| 379 |
o send a patch to the curl-library mailing list. We're many subscribers |
| 380 |
there and there are lots of people who can review patches, comment on them |
| 381 |
and "receive" them properly. |
| 382 |
|
| 383 |
o if your patch changes or fixes a bug, you can also opt to submit a bug |
| 384 |
report in the bug tracker and attach your patch there. There are less |
| 385 |
people involved there. |
| 386 |
|
| 387 |
Lots of more details are found in the CONTRIBUTE and INTERNALS docs. |
| 388 |
|
| 389 |
1.15 How do I port libcurl to my OS? |
| 390 |
|
| 391 |
Here's a rough step-by-step: |
| 392 |
|
| 393 |
1. copy a suitable lib/config-*.h file as a start to lib/config-[youros].h |
| 394 |
|
| 395 |
2. edit lib/config-[youros].h to match your OS and setup |
| 396 |
|
| 397 |
3. edit lib/curl_setup.h to include config-[youros].h when your OS is |
| 398 |
detected by the preprocessor, in the style others already exist |
| 399 |
|
| 400 |
4. compile lib/*.c and make them into a library |
| 401 |
|
| 402 |
|
| 403 |
2. Install Related Problems |
| 404 |
|
| 405 |
2.1 configure doesn't find OpenSSL even when it is installed |
| 406 |
|
| 407 |
This may be because of several reasons. |
| 408 |
|
| 409 |
2.1.1 native linker doesn't find openssl |
| 410 |
|
| 411 |
Affected platforms: |
| 412 |
Solaris (native cc compiler) |
| 413 |
HPUX (native cc compiler) |
| 414 |
SGI IRIX (native cc compiler) |
| 415 |
SCO UNIX (native cc compiler) |
| 416 |
|
| 417 |
When configuring curl, I specify --with-ssl. OpenSSL is installed in |
| 418 |
/usr/local/ssl Configure reports SSL in /usr/local/ssl, but fails to find |
| 419 |
CRYPTO_lock in -lcrypto |
| 420 |
|
| 421 |
Cause: The cc for this test places the -L/usr/local/ssl/lib AFTER |
| 422 |
-lcrypto, so ld can't find the library. This is due to a bug in the GNU |
| 423 |
autoconf tool. |
| 424 |
|
| 425 |
Workaround: Specifying "LDFLAGS=-L/usr/local/ssl/lib" in front of |
| 426 |
./configure places the -L/usr/local/ssl/lib early enough in the command |
| 427 |
line to make things work |
| 428 |
|
| 429 |
2.1.2 only the libssl lib is missing |
| 430 |
|
| 431 |
If all include files and the libcrypto lib is present, with only the |
| 432 |
libssl being missing according to configure, this is most likely because |
| 433 |
a few functions are left out from the libssl. |
| 434 |
|
| 435 |
If the function names missing include RSA or RSAREF you can be certain |
| 436 |
that this is because libssl requires the RSA and RSAREF libs to build. |
| 437 |
|
| 438 |
See the INSTALL file section that explains how to add those libs to |
| 439 |
configure. Make sure that you remove the config.cache file before you |
| 440 |
rerun configure with the new flags. |
| 441 |
|
| 442 |
2.2 Does curl work/build with other SSL libraries? |
| 443 |
|
| 444 |
Curl has been written to use a generic SSL function layer internally, and |
| 445 |
that SSL functionality can then be provided by one out of many different SSL |
| 446 |
backends. |
| 447 |
|
| 448 |
curl can be built to use one of the following SSL alternatives: OpenSSL, |
| 449 |
libressl, BoringSSL, GnuTLS, wolfSSL, NSS, mbedTLS, MesaLink, Secure |
| 450 |
Transport (native iOS/OS X), Schannel (native Windows), GSKit (native IBM |
| 451 |
i), or BearSSL. They all have their pros and cons, and we try to maintain a |
| 452 |
comparison of them here: https://curl.haxx.se/docs/ssl-compared.html |
| 453 |
|
| 454 |
2.3 Where can I find a copy of LIBEAY32.DLL? |
| 455 |
|
| 456 |
That is an OpenSSL binary built for Windows. |
| 457 |
|
| 458 |
Curl can be built with OpenSSL to do the SSL stuff. The LIBEAY32.DLL is then |
| 459 |
what curl needs on a windows machine to do https:// etc. Check out the curl |
| 460 |
web site to find accurate and up-to-date pointers to recent OpenSSL DLLs and |
| 461 |
other binary packages. |
| 462 |
|
| 463 |
2.4 Does curl support SOCKS (RFC 1928) ? |
| 464 |
|
| 465 |
Yes, SOCKS 4 and 5 are supported. |
| 466 |
|
| 467 |
|
| 468 |
3. Usage problems |
| 469 |
|
| 470 |
3.1 curl: (1) SSL is disabled, https: not supported |
| 471 |
|
| 472 |
If you get this output when trying to get anything from a https:// server, |
| 473 |
it means that the instance of curl/libcurl that you're using was built |
| 474 |
without support for this protocol. |
| 475 |
|
| 476 |
This could've happened if the configure script that was run at build time |
| 477 |
couldn't find all libs and include files curl requires for SSL to work. If |
| 478 |
the configure script fails to find them, curl is simply built without SSL |
| 479 |
support. |
| 480 |
|
| 481 |
To get the https:// support into a curl that was previously built but that |
| 482 |
reports that https:// is not supported, you should dig through the document |
| 483 |
and logs and check out why the configure script doesn't find the SSL libs |
| 484 |
and/or include files. |
| 485 |
|
| 486 |
Also, check out the other paragraph in this FAQ labeled "configure doesn't |
| 487 |
find OpenSSL even when it is installed". |
| 488 |
|
| 489 |
3.2 How do I tell curl to resume a transfer? |
| 490 |
|
| 491 |
Curl supports resumed transfers both ways on both FTP and HTTP. |
| 492 |
Try the -C option. |
| 493 |
|
| 494 |
3.3 Why doesn't my posting using -F work? |
| 495 |
|
| 496 |
You can't arbitrarily use -F or -d, the choice between -F or -d depends on the |
| 497 |
HTTP operation you need curl to do and what the web server that will receive |
| 498 |
your post expects. |
| 499 |
|
| 500 |
If the form you're trying to submit uses the type 'multipart/form-data', then |
| 501 |
and only then you must use the -F type. In all the most common cases, you |
| 502 |
should use -d which then causes a posting with the type |
| 503 |
'application/x-www-form-urlencoded'. |
| 504 |
|
| 505 |
This is described in some detail in the MANUAL and TheArtOfHttpScripting |
| 506 |
documents, and if you don't understand it the first time, read it again |
| 507 |
before you post questions about this to the mailing list. Also, try reading |
| 508 |
through the mailing list archives for old postings and questions regarding |
| 509 |
this. |
| 510 |
|
| 511 |
3.4 How do I tell curl to run custom FTP commands? |
| 512 |
|
| 513 |
You can tell curl to perform optional commands both before and/or after a |
| 514 |
file transfer. Study the -Q/--quote option. |
| 515 |
|
| 516 |
Since curl is used for file transfers, you don't normally use curl to |
| 517 |
perform FTP commands without transferring anything. Therefore you must |
| 518 |
always specify a URL to transfer to/from even when doing custom FTP |
| 519 |
commands, or use -I which implies the "no body" option sent to libcurl. |
| 520 |
|
| 521 |
3.5 How can I disable the Accept: */* header? |
| 522 |
|
| 523 |
You can change all internally generated headers by adding a replacement with |
| 524 |
the -H/--header option. By adding a header with empty contents you safely |
| 525 |
disable that one. Use -H "Accept:" to disable that specific header. |
| 526 |
|
| 527 |
3.6 Does curl support ASP, XML, XHTML or HTML version Y? |
| 528 |
|
| 529 |
To curl, all contents are alike. It doesn't matter how the page was |
| 530 |
generated. It may be ASP, PHP, Perl, shell-script, SSI or plain HTML |
| 531 |
files. There's no difference to curl and it doesn't even know what kind of |
| 532 |
language that generated the page. |
| 533 |
|
| 534 |
See also item 3.14 regarding javascript. |
| 535 |
|
| 536 |
3.7 Can I use curl to delete/rename a file through FTP? |
| 537 |
|
| 538 |
Yes. You specify custom FTP commands with -Q/--quote. |
| 539 |
|
| 540 |
One example would be to delete a file after you have downloaded it: |
| 541 |
|
| 542 |
curl -O ftp://download.com/coolfile -Q '-DELE coolfile' |
| 543 |
|
| 544 |
or rename a file after upload: |
| 545 |
|
| 546 |
curl -T infile ftp://upload.com/dir/ -Q "-RNFR infile" -Q "-RNTO newname" |
| 547 |
|
| 548 |
3.8 How do I tell curl to follow HTTP redirects? |
| 549 |
|
| 550 |
Curl does not follow so-called redirects by default. The Location: header |
| 551 |
that informs the client about this is only interpreted if you're using the |
| 552 |
-L/--location option. As in: |
| 553 |
|
| 554 |
curl -L http://redirector.com |
| 555 |
|
| 556 |
Not all redirects are HTTP ones, see 4.14 |
| 557 |
|
| 558 |
3.9 How do I use curl in my favorite programming language? |
| 559 |
|
| 560 |
Many programming languages have interfaces/bindings that allow you to use |
| 561 |
curl without having to use the command line tool. If you are fluent in such |
| 562 |
a language, you may prefer to use one of these interfaces instead. |
| 563 |
|
| 564 |
Find out more about which languages that support curl directly, and how to |
| 565 |
install and use them, in the libcurl section of the curl web site: |
| 566 |
https://curl.haxx.se/libcurl/ |
| 567 |
|
| 568 |
All the various bindings to libcurl are made by other projects and people, |
| 569 |
outside of the cURL project. The cURL project itself only produces libcurl |
| 570 |
with its plain C API. If you don't find anywhere else to ask you can ask |
| 571 |
about bindings on the curl-library list too, but be prepared that people on |
| 572 |
that list may not know anything about bindings. |
| 573 |
|
| 574 |
In February 2019, there were interfaces available for the following |
| 575 |
languages: Ada95, Basic, C, C++, Ch, Cocoa, D, Delphi, Dylan, Eiffel, |
| 576 |
Euphoria, Falcon, Ferite, Gambas, glib/GTK+, Go, Guile, Harbour, Haskell, |
| 577 |
Java, Julia, Lisp, Lua, Mono, .NET, node.js, Object-Pascal, OCaml, Pascal, |
| 578 |
Perl, PHP, PostgreSQL, Python, R, Rexx, Ring, RPG, Ruby, Rust, Scheme, |
| 579 |
Scilab, S-Lang, Smalltalk, SP-Forth, SPL, Tcl, Visual Basic, Visual FoxPro, |
| 580 |
Q, wxwidgets, XBLite and Xoho. By the time you read this, additional ones |
| 581 |
may have appeared! |
| 582 |
|
| 583 |
3.10 What about SOAP, WebDAV, XML-RPC or similar protocols over HTTP? |
| 584 |
|
| 585 |
Curl adheres to the HTTP spec, which basically means you can play with *any* |
| 586 |
protocol that is built on top of HTTP. Protocols such as SOAP, WEBDAV and |
| 587 |
XML-RPC are all such ones. You can use -X to set custom requests and -H to |
| 588 |
set custom headers (or replace internally generated ones). |
| 589 |
|
| 590 |
Using libcurl is of course just as good and you'd just use the proper |
| 591 |
library options to do the same. |
| 592 |
|
| 593 |
3.11 How do I POST with a different Content-Type? |
| 594 |
|
| 595 |
You can always replace the internally generated headers with -H/--header. |
| 596 |
To make a simple HTTP POST with text/xml as content-type, do something like: |
| 597 |
|
| 598 |
curl -d "datatopost" -H "Content-Type: text/xml" [URL] |
| 599 |
|
| 600 |
3.12 Why do FTP-specific features over HTTP proxy fail? |
| 601 |
|
| 602 |
Because when you use a HTTP proxy, the protocol spoken on the network will |
| 603 |
be HTTP, even if you specify a FTP URL. This effectively means that you |
| 604 |
normally can't use FTP-specific features such as FTP upload and FTP quote |
| 605 |
etc. |
| 606 |
|
| 607 |
There is one exception to this rule, and that is if you can "tunnel through" |
| 608 |
the given HTTP proxy. Proxy tunneling is enabled with a special option (-p) |
| 609 |
and is generally not available as proxy admins usually disable tunneling to |
| 610 |
ports other than 443 (which is used for HTTPS access through proxies). |
| 611 |
|
| 612 |
3.13 Why do my single/double quotes fail? |
| 613 |
|
| 614 |
To specify a command line option that includes spaces, you might need to |
| 615 |
put the entire option within quotes. Like in: |
| 616 |
|
| 617 |
curl -d " with spaces " url.com |
| 618 |
|
| 619 |
or perhaps |
| 620 |
|
| 621 |
curl -d ' with spaces ' url.com |
| 622 |
|
| 623 |
Exactly what kind of quotes and how to do this is entirely up to the shell |
| 624 |
or command line interpreter that you are using. For most unix shells, you |
| 625 |
can more or less pick either single (') or double (") quotes. For |
| 626 |
Windows/DOS prompts I believe you're forced to use double (") quotes. |
| 627 |
|
| 628 |
Please study the documentation for your particular environment. Examples in |
| 629 |
the curl docs will use a mix of both of these as shown above. You must |
| 630 |
adjust them to work in your environment. |
| 631 |
|
| 632 |
Remember that curl works and runs on more operating systems than most single |
| 633 |
individuals have ever tried. |
| 634 |
|
| 635 |
3.14 Does curl support Javascript or PAC (automated proxy config)? |
| 636 |
|
| 637 |
Many web pages do magic stuff using embedded Javascript. Curl and libcurl |
| 638 |
have no built-in support for that, so it will be treated just like any other |
| 639 |
contents. |
| 640 |
|
| 641 |
.pac files are a netscape invention and are sometimes used by organizations |
| 642 |
to allow them to differentiate which proxies to use. The .pac contents is |
| 643 |
just a Javascript program that gets invoked by the browser and that returns |
| 644 |
the name of the proxy to connect to. Since curl doesn't support Javascript, |
| 645 |
it can't support .pac proxy configuration either. |
| 646 |
|
| 647 |
Some workarounds usually suggested to overcome this Javascript dependency: |
| 648 |
|
| 649 |
Depending on the Javascript complexity, write up a script that translates it |
| 650 |
to another language and execute that. |
| 651 |
|
| 652 |
Read the Javascript code and rewrite the same logic in another language. |
| 653 |
|
| 654 |
Implement a Javascript interpreter, people have successfully used the |
| 655 |
Mozilla Javascript engine in the past. |
| 656 |
|
| 657 |
Ask your admins to stop this, for a static proxy setup or similar. |
| 658 |
|
| 659 |
3.15 Can I do recursive fetches with curl? |
| 660 |
|
| 661 |
No. curl itself has no code that performs recursive operations, such as |
| 662 |
those performed by wget and similar tools. |
| 663 |
|
| 664 |
There exists wrapper scripts with that functionality (for example the |
| 665 |
curlmirror perl script), and you can write programs based on libcurl to do |
| 666 |
it, but the command line tool curl itself cannot. |
| 667 |
|
| 668 |
3.16 What certificates do I need when I use SSL? |
| 669 |
|
| 670 |
There are three different kinds of "certificates" to keep track of when we |
| 671 |
talk about using SSL-based protocols (HTTPS or FTPS) using curl or libcurl. |
| 672 |
|
| 673 |
CLIENT CERTIFICATE |
| 674 |
|
| 675 |
The server you communicate with may require that you can provide this in |
| 676 |
order to prove that you actually are who you claim to be. If the server |
| 677 |
doesn't require this, you don't need a client certificate. |
| 678 |
|
| 679 |
A client certificate is always used together with a private key, and the |
| 680 |
private key has a pass phrase that protects it. |
| 681 |
|
| 682 |
SERVER CERTIFICATE |
| 683 |
|
| 684 |
The server you communicate with has a server certificate. You can and should |
| 685 |
verify this certificate to make sure that you are truly talking to the real |
| 686 |
server and not a server impersonating it. |
| 687 |
|
| 688 |
CERTIFICATE AUTHORITY CERTIFICATE ("CA cert") |
| 689 |
|
| 690 |
You often have several CA certs in a CA cert bundle that can be used to |
| 691 |
verify a server certificate that was signed by one of the authorities in the |
| 692 |
bundle. curl does not come with a CA cert bundle but most curl installs |
| 693 |
provide one. You can also override the default. |
| 694 |
|
| 695 |
The server certificate verification process is made by using a Certificate |
| 696 |
Authority certificate ("CA cert") that was used to sign the server |
| 697 |
certificate. Server certificate verification is enabled by default in curl |
| 698 |
and libcurl and is often the reason for problems as explained in FAQ entry |
| 699 |
4.12 and the SSLCERTS document |
| 700 |
(https://curl.haxx.se/docs/sslcerts.html). Server certificates that are |
| 701 |
"self-signed" or otherwise signed by a CA that you do not have a CA cert |
| 702 |
for, cannot be verified. If the verification during a connect fails, you are |
| 703 |
refused access. You then need to explicitly disable the verification to |
| 704 |
connect to the server. |
| 705 |
|
| 706 |
3.17 How do I list the root dir of an FTP server? |
| 707 |
|
| 708 |
There are two ways. The way defined in the RFC is to use an encoded slash |
| 709 |
in the first path part. List the "/tmp" dir like this: |
| 710 |
|
| 711 |
curl ftp://ftp.sunet.se/%2ftmp/ |
| 712 |
|
| 713 |
or the not-quite-kosher-but-more-readable way, by simply starting the path |
| 714 |
section of the URL with a slash: |
| 715 |
|
| 716 |
curl ftp://ftp.sunet.se//tmp/ |
| 717 |
|
| 718 |
3.18 Can I use curl to send a POST/PUT and not wait for a response? |
| 719 |
|
| 720 |
No. |
| 721 |
|
| 722 |
But you could easily write your own program using libcurl to do such stunts. |
| 723 |
|
| 724 |
3.19 How do I get HTTP from a host using a specific IP address? |
| 725 |
|
| 726 |
For example, you may be trying out a web site installation that isn't yet in |
| 727 |
the DNS. Or you have a site using multiple IP addresses for a given host |
| 728 |
name and you want to address a specific one out of the set. |
| 729 |
|
| 730 |
Set a custom Host: header that identifies the server name you want to reach |
| 731 |
but use the target IP address in the URL: |
| 732 |
|
| 733 |
curl --header "Host: www.example.com" http://127.0.0.1/ |
| 734 |
|
| 735 |
You can also opt to add faked host name entries to curl with the --resolve |
| 736 |
option. That has the added benefit that things like redirects will also work |
| 737 |
properly. The above operation would instead be done as: |
| 738 |
|
| 739 |
curl --resolve www.example.com:80:127.0.0.1 http://www.example.com/ |
| 740 |
|
| 741 |
3.20 How to SFTP from my user's home directory? |
| 742 |
|
| 743 |
Contrary to how FTP works, SFTP and SCP URLs specify the exact directory to |
| 744 |
work with. It means that if you don't specify that you want the user's home |
| 745 |
directory, you get the actual root directory. |
| 746 |
|
| 747 |
To specify a file in your user's home directory, you need to use the correct |
| 748 |
URL syntax which for SFTP might look similar to: |
| 749 |
|
| 750 |
curl -O -u user:password sftp://example.com/~/file.txt |
| 751 |
|
| 752 |
and for SCP it is just a different protocol prefix: |
| 753 |
|
| 754 |
curl -O -u user:password scp://example.com/~/file.txt |
| 755 |
|
| 756 |
3.21 Protocol xxx not supported or disabled in libcurl |
| 757 |
|
| 758 |
When passing on a URL to curl to use, it may respond that the particular |
| 759 |
protocol is not supported or disabled. The particular way this error message |
| 760 |
is phrased is because curl doesn't make a distinction internally of whether |
| 761 |
a particular protocol is not supported (i.e. never got any code added that |
| 762 |
knows how to speak that protocol) or if it was explicitly disabled. curl can |
| 763 |
be built to only support a given set of protocols, and the rest would then |
| 764 |
be disabled or not supported. |
| 765 |
|
| 766 |
Note that this error will also occur if you pass a wrongly spelled protocol |
| 767 |
part as in "htpt://example.com" or as in the less evident case if you prefix |
| 768 |
the protocol part with a space as in " http://example.com/". |
| 769 |
|
| 770 |
3.22 curl -X gives me HTTP problems |
| 771 |
|
| 772 |
In normal circumstances, -X should hardly ever be used. |
| 773 |
|
| 774 |
By default you use curl without explicitly saying which request method to |
| 775 |
use when the URL identifies a HTTP transfer. If you just pass in a URL like |
| 776 |
"curl http://example.com" it will use GET. If you use -d or -F curl will use |
| 777 |
POST, -I will cause a HEAD and -T will make it a PUT. |
| 778 |
|
| 779 |
If for whatever reason you're not happy with these default choices that curl |
| 780 |
does for you, you can override those request methods by specifying -X |
| 781 |
[WHATEVER]. This way you can for example send a DELETE by doing "curl -X |
| 782 |
DELETE [URL]". |
| 783 |
|
| 784 |
It is thus pointless to do "curl -XGET [URL]" as GET would be used |
| 785 |
anyway. In the same vein it is pointless to do "curl -X POST -d data |
| 786 |
[URL]"... But you can make a fun and somewhat rare request that sends a |
| 787 |
request-body in a GET request with something like "curl -X GET -d data |
| 788 |
[URL]" |
| 789 |
|
| 790 |
Note that -X doesn't actually change curl's behavior as it only modifies the |
| 791 |
actual string sent in the request, but that may of course trigger a |
| 792 |
different set of events. |
| 793 |
|
| 794 |
Accordingly, by using -XPOST on a command line that for example would follow |
| 795 |
a 303 redirect, you will effectively prevent curl from behaving |
| 796 |
correctly. Be aware. |
| 797 |
|
| 798 |
|
| 799 |
4. Running Problems |
| 800 |
|
| 801 |
4.1 Problems connecting to SSL servers. |
| 802 |
|
| 803 |
It took a very long time before we could sort out why curl had problems to |
| 804 |
connect to certain SSL servers when using SSLeay or OpenSSL v0.9+. The |
| 805 |
error sometimes showed up similar to: |
| 806 |
|
| 807 |
16570:error:1407D071:SSL routines:SSL2_READ:bad mac decode:s2_pkt.c:233: |
| 808 |
|
| 809 |
It turned out to be because many older SSL servers don't deal with SSLv3 |
| 810 |
requests properly. To correct this problem, tell curl to select SSLv2 from |
| 811 |
the command line (-2/--sslv2). |
| 812 |
|
| 813 |
There have also been examples where the remote server didn't like the SSLv2 |
| 814 |
request and instead you had to force curl to use SSLv3 with -3/--sslv3. |
| 815 |
|
| 816 |
4.2 Why do I get problems when I use & or % in the URL? |
| 817 |
|
| 818 |
In general unix shells, the & symbol is treated specially and when used, it |
| 819 |
runs the specified command in the background. To safely send the & as a part |
| 820 |
of a URL, you should quote the entire URL by using single (') or double (") |
| 821 |
quotes around it. Similar problems can also occur on some shells with other |
| 822 |
characters, including ?*!$~(){}<>\|;`. When in doubt, quote the URL. |
| 823 |
|
| 824 |
An example that would invoke a remote CGI that uses &-symbols could be: |
| 825 |
|
| 826 |
curl 'http://www.altavista.com/cgi-bin/query?text=yes&q=curl' |
| 827 |
|
| 828 |
In Windows, the standard DOS shell treats the percent sign specially and you |
| 829 |
need to use TWO percent signs for each single one you want to use in the |
| 830 |
URL. |
| 831 |
|
| 832 |
If you want a literal percent sign to be part of the data you pass in a POST |
| 833 |
using -d/--data you must encode it as '%25' (which then also needs the |
| 834 |
percent sign doubled on Windows machines). |
| 835 |
|
| 836 |
4.3 How can I use {, }, [ or ] to specify multiple URLs? |
| 837 |
|
| 838 |
Because those letters have a special meaning to the shell, to be used in |
| 839 |
a URL specified to curl you must quote them. |
| 840 |
|
| 841 |
An example that downloads two URLs (sequentially) would be: |
| 842 |
|
| 843 |
curl '{curl,www}.haxx.se' |
| 844 |
|
| 845 |
To be able to use those characters as actual parts of the URL (without using |
| 846 |
them for the curl URL "globbing" system), use the -g/--globoff option: |
| 847 |
|
| 848 |
curl -g 'www.site.com/weirdname[].html' |
| 849 |
|
| 850 |
4.4 Why do I get downloaded data even though the web page doesn't exist? |
| 851 |
|
| 852 |
Curl asks remote servers for the page you specify. If the page doesn't exist |
| 853 |
at the server, the HTTP protocol defines how the server should respond and |
| 854 |
that means that headers and a "page" will be returned. That's simply how |
| 855 |
HTTP works. |
| 856 |
|
| 857 |
By using the --fail option you can tell curl explicitly to not get any data |
| 858 |
if the HTTP return code doesn't say success. |
| 859 |
|
| 860 |
4.5 Why do I get return code XXX from a HTTP server? |
| 861 |
|
| 862 |
RFC2616 clearly explains the return codes. This is a short transcript. Go |
| 863 |
read the RFC for exact details: |
| 864 |
|
| 865 |
4.5.1 "400 Bad Request" |
| 866 |
|
| 867 |
The request could not be understood by the server due to malformed |
| 868 |
syntax. The client SHOULD NOT repeat the request without modifications. |
| 869 |
|
| 870 |
4.5.2 "401 Unauthorized" |
| 871 |
|
| 872 |
The request requires user authentication. |
| 873 |
|
| 874 |
4.5.3 "403 Forbidden" |
| 875 |
|
| 876 |
The server understood the request, but is refusing to fulfill it. |
| 877 |
Authorization will not help and the request SHOULD NOT be repeated. |
| 878 |
|
| 879 |
4.5.4 "404 Not Found" |
| 880 |
|
| 881 |
The server has not found anything matching the Request-URI. No indication |
| 882 |
is given of whether the condition is temporary or permanent. |
| 883 |
|
| 884 |
4.5.5 "405 Method Not Allowed" |
| 885 |
|
| 886 |
The method specified in the Request-Line is not allowed for the resource |
| 887 |
identified by the Request-URI. The response MUST include an Allow header |
| 888 |
containing a list of valid methods for the requested resource. |
| 889 |
|
| 890 |
4.5.6 "301 Moved Permanently" |
| 891 |
|
| 892 |
If you get this return code and an HTML output similar to this: |
| 893 |
|
| 894 |
<H1>Moved Permanently</H1> The document has moved <A |
| 895 |
HREF="http://same_url_now_with_a_trailing_slash/">here</A>. |
| 896 |
|
| 897 |
it might be because you requested a directory URL but without the trailing |
| 898 |
slash. Try the same operation again _with_ the trailing URL, or use the |
| 899 |
-L/--location option to follow the redirection. |
| 900 |
|
| 901 |
4.6 Can you tell me what error code 142 means? |
| 902 |
|
| 903 |
All curl error codes are described at the end of the man page, in the |
| 904 |
section called "EXIT CODES". |
| 905 |
|
| 906 |
Error codes that are larger than the highest documented error code means |
| 907 |
that curl has exited due to a crash. This is a serious error, and we |
| 908 |
appreciate a detailed bug report from you that describes how we could go |
| 909 |
ahead and repeat this! |
| 910 |
|
| 911 |
4.7 How do I keep user names and passwords secret in Curl command lines? |
| 912 |
|
| 913 |
This problem has two sides: |
| 914 |
|
| 915 |
The first part is to avoid having clear-text passwords in the command line |
| 916 |
so that they don't appear in 'ps' outputs and similar. That is easily |
| 917 |
avoided by using the "-K" option to tell curl to read parameters from a file |
| 918 |
or stdin to which you can pass the secret info. curl itself will also |
| 919 |
attempt to "hide" the given password by blanking out the option - this |
| 920 |
doesn't work on all platforms. |
| 921 |
|
| 922 |
To keep the passwords in your account secret from the rest of the world is |
| 923 |
not a task that curl addresses. You could of course encrypt them somehow to |
| 924 |
at least hide them from being read by human eyes, but that is not what |
| 925 |
anyone would call security. |
| 926 |
|
| 927 |
Also note that regular HTTP (using Basic authentication) and FTP passwords |
| 928 |
are sent as cleartext across the network. All it takes for anyone to fetch |
| 929 |
them is to listen on the network. Eavesdropping is very easy. Use more secure |
| 930 |
authentication methods (like Digest, Negotiate or even NTLM) or consider the |
| 931 |
SSL-based alternatives HTTPS and FTPS. |
| 932 |
|
| 933 |
4.8 I found a bug! |
| 934 |
|
| 935 |
It is not a bug if the behavior is documented. Read the docs first. |
| 936 |
Especially check out the KNOWN_BUGS file, it may be a documented bug! |
| 937 |
|
| 938 |
If it is a problem with a binary you've downloaded or a package for your |
| 939 |
particular platform, try contacting the person who built the package/archive |
| 940 |
you have. |
| 941 |
|
| 942 |
If there is a bug, read the BUGS document first. Then report it as described |
| 943 |
in there. |
| 944 |
|
| 945 |
4.9 Curl can't authenticate to the server that requires NTLM? |
| 946 |
|
| 947 |
NTLM support requires OpenSSL, GnuTLS, mbedTLS, NSS, Secure Transport, or |
| 948 |
Microsoft Windows libraries at build-time to provide this functionality. |
| 949 |
|
| 950 |
NTLM is a Microsoft proprietary protocol. Proprietary formats are evil. You |
| 951 |
should not use such ones. |
| 952 |
|
| 953 |
4.10 My HTTP request using HEAD, PUT or DELETE doesn't work! |
| 954 |
|
| 955 |
Many web servers allow or demand that the administrator configures the |
| 956 |
server properly for these requests to work on the web server. |
| 957 |
|
| 958 |
Some servers seem to support HEAD only on certain kinds of URLs. |
| 959 |
|
| 960 |
To fully grasp this, try the documentation for the particular server |
| 961 |
software you're trying to interact with. This is not anything curl can do |
| 962 |
anything about. |
| 963 |
|
| 964 |
4.11 Why do my HTTP range requests return the full document? |
| 965 |
|
| 966 |
Because the range may not be supported by the server, or the server may |
| 967 |
choose to ignore it and return the full document anyway. |
| 968 |
|
| 969 |
4.12 Why do I get "certificate verify failed" ? |
| 970 |
|
| 971 |
You invoke curl 7.10 or later to communicate on a https:// URL and get an |
| 972 |
error back looking something similar to this: |
| 973 |
|
| 974 |
curl: (35) SSL: error:14090086:SSL routines: |
| 975 |
SSL3_GET_SERVER_CERTIFICATE:certificate verify failed |
| 976 |
|
| 977 |
Then it means that curl couldn't verify that the server's certificate was |
| 978 |
good. Curl verifies the certificate using the CA cert bundle that comes with |
| 979 |
the curl installation. |
| 980 |
|
| 981 |
To disable the verification (which makes it act like curl did before 7.10), |
| 982 |
use -k. This does however enable man-in-the-middle attacks. |
| 983 |
|
| 984 |
If you get this failure but are having a CA cert bundle installed and used, |
| 985 |
the server's certificate is not signed by one of the CA's in the bundle. It |
| 986 |
might for example be self-signed. You then correct this problem by obtaining |
| 987 |
a valid CA cert for the server. Or again, decrease the security by disabling |
| 988 |
this check. |
| 989 |
|
| 990 |
Details are also in the SSLCERTS file in the release archives, found online |
| 991 |
here: https://curl.haxx.se/docs/sslcerts.html |
| 992 |
|
| 993 |
4.13 Why is curl -R on Windows one hour off? |
| 994 |
|
| 995 |
Since curl 7.53.0 this issue should be fixed as long as curl was built with |
| 996 |
any modern compiler that allows for a 64-bit curl_off_t type. For older |
| 997 |
compilers or prior curl versions it may set a time that appears one hour off. |
| 998 |
This happens due to a flaw in how Windows stores and uses file modification |
| 999 |
times and it is not easily worked around. For more details read this: |
| 1000 |
https://www.codeproject.com/Articles/1144/Beating-the-Daylight-Savings-Time-bug-and-getting |
| 1001 |
|
| 1002 |
4.14 Redirects work in browser but not with curl! |
| 1003 |
|
| 1004 |
curl supports HTTP redirects well (see item 3.8). Browsers generally support |
| 1005 |
at least two other ways to perform redirects that curl does not: |
| 1006 |
|
| 1007 |
Meta tags. You can write a HTML tag that will cause the browser to redirect |
| 1008 |
to another given URL after a certain time. |
| 1009 |
|
| 1010 |
Javascript. You can write a Javascript program embedded in a HTML page that |
| 1011 |
redirects the browser to another given URL. |
| 1012 |
|
| 1013 |
There is no way to make curl follow these redirects. You must either |
| 1014 |
manually figure out what the page is set to do, or write a script that parses |
| 1015 |
the results and fetches the new URL. |
| 1016 |
|
| 1017 |
4.15 FTPS doesn't work |
| 1018 |
|
| 1019 |
curl supports FTPS (sometimes known as FTP-SSL) both implicit and explicit |
| 1020 |
mode. |
| 1021 |
|
| 1022 |
When a URL is used that starts with FTPS://, curl assumes implicit SSL on |
| 1023 |
the control connection and will therefore immediately connect and try to |
| 1024 |
speak SSL. FTPS:// connections default to port 990. |
| 1025 |
|
| 1026 |
To use explicit FTPS, you use a FTP:// URL and the --ftp-ssl option (or one |
| 1027 |
of its related flavors). This is the most common method, and the one |
| 1028 |
mandated by RFC4217. This kind of connection will then of course use the |
| 1029 |
standard FTP port 21 by default. |
| 1030 |
|
| 1031 |
4.16 My HTTP POST or PUT requests are slow! |
| 1032 |
|
| 1033 |
libcurl makes all POST and PUT requests (except for POST requests with a |
| 1034 |
very tiny request body) use the "Expect: 100-continue" header. This header |
| 1035 |
allows the server to deny the operation early so that libcurl can bail out |
| 1036 |
before having to send any data. This is useful in authentication |
| 1037 |
cases and others. |
| 1038 |
|
| 1039 |
However, many servers don't implement the Expect: stuff properly and if the |
| 1040 |
server doesn't respond (positively) within 1 second libcurl will continue |
| 1041 |
and send off the data anyway. |
| 1042 |
|
| 1043 |
You can disable libcurl's use of the Expect: header the same way you disable |
| 1044 |
any header, using -H / CURLOPT_HTTPHEADER, or by forcing it to use HTTP 1.0. |
| 1045 |
|
| 1046 |
4.17 Non-functional connect timeouts |
| 1047 |
|
| 1048 |
In most Windows setups having a timeout longer than 21 seconds make no |
| 1049 |
difference, as it will only send 3 TCP SYN packets and no more. The second |
| 1050 |
packet sent three seconds after the first and the third six seconds after |
| 1051 |
the second. No more than three packets are sent, no matter how long the |
| 1052 |
timeout is set. |
| 1053 |
|
| 1054 |
See option TcpMaxConnectRetransmissions on this page: |
| 1055 |
https://support.microsoft.com/en-us/kb/175523/en-us |
| 1056 |
|
| 1057 |
Also, even on non-Windows systems there may run a firewall or anti-virus |
| 1058 |
software or similar that accepts the connection but does not actually do |
| 1059 |
anything else. This will make (lib)curl to consider the connection connected |
| 1060 |
and thus the connect timeout won't trigger. |
| 1061 |
|
| 1062 |
4.18 file:// URLs containing drive letters (Windows, NetWare) |
| 1063 |
|
| 1064 |
When using curl to try to download a local file, one might use a URL |
| 1065 |
in this format: |
| 1066 |
|
| 1067 |
file://D:/blah.txt |
| 1068 |
|
| 1069 |
You'll find that even if D:\blah.txt does exist, curl returns a 'file |
| 1070 |
not found' error. |
| 1071 |
|
| 1072 |
According to RFC 1738 (https://www.ietf.org/rfc/rfc1738.txt), |
| 1073 |
file:// URLs must contain a host component, but it is ignored by |
| 1074 |
most implementations. In the above example, 'D:' is treated as the |
| 1075 |
host component, and is taken away. Thus, curl tries to open '/blah.txt'. |
| 1076 |
If your system is installed to drive C:, that will resolve to 'C:\blah.txt', |
| 1077 |
and if that doesn't exist you will get the not found error. |
| 1078 |
|
| 1079 |
To fix this problem, use file:// URLs with *three* leading slashes: |
| 1080 |
|
| 1081 |
file:///D:/blah.txt |
| 1082 |
|
| 1083 |
Alternatively, if it makes more sense, specify 'localhost' as the host |
| 1084 |
component: |
| 1085 |
|
| 1086 |
file://localhost/D:/blah.txt |
| 1087 |
|
| 1088 |
In either case, curl should now be looking for the correct file. |
| 1089 |
|
| 1090 |
4.19 Why doesn't curl return an error when the network cable is unplugged? |
| 1091 |
|
| 1092 |
Unplugging a cable is not an error situation. The TCP/IP protocol stack |
| 1093 |
was designed to be fault tolerant, so even though there may be a physical |
| 1094 |
break somewhere the connection shouldn't be affected, just possibly |
| 1095 |
delayed. Eventually, the physical break will be fixed or the data will be |
| 1096 |
re-routed around the physical problem through another path. |
| 1097 |
|
| 1098 |
In such cases, the TCP/IP stack is responsible for detecting when the |
| 1099 |
network connection is irrevocably lost. Since with some protocols it is |
| 1100 |
perfectly legal for the client to wait indefinitely for data, the stack may |
| 1101 |
never report a problem, and even when it does, it can take up to 20 minutes |
| 1102 |
for it to detect an issue. The curl option --keepalive-time enables |
| 1103 |
keep-alive support in the TCP/IP stack which makes it periodically probe the |
| 1104 |
connection to make sure it is still available to send data. That should |
| 1105 |
reliably detect any TCP/IP network failure. |
| 1106 |
|
| 1107 |
But even that won't detect the network going down before the TCP/IP |
| 1108 |
connection is established (e.g. during a DNS lookup) or using protocols that |
| 1109 |
don't use TCP. To handle those situations, curl offers a number of timeouts |
| 1110 |
on its own. --speed-limit/--speed-time will abort if the data transfer rate |
| 1111 |
falls too low, and --connect-timeout and --max-time can be used to put an |
| 1112 |
overall timeout on the connection phase or the entire transfer. |
| 1113 |
|
| 1114 |
A libcurl-using application running in a known physical environment (e.g. |
| 1115 |
an embedded device with only a single network connection) may want to act |
| 1116 |
immediately if its lone network connection goes down. That can be achieved |
| 1117 |
by having the application monitor the network connection on its own using an |
| 1118 |
OS-specific mechanism, then signaling libcurl to abort (see also item 5.13). |
| 1119 |
|
| 1120 |
4.20 curl doesn't return error for HTTP non-200 responses! |
| 1121 |
|
| 1122 |
Correct. Unless you use -f (--fail). |
| 1123 |
|
| 1124 |
When doing HTTP transfers, curl will perform exactly what you're asking it |
| 1125 |
to do and if successful it will not return an error. You can use curl to |
| 1126 |
test your web server's "file not found" page (that gets 404 back), you can |
| 1127 |
use it to check your authentication protected web pages (that gets a 401 |
| 1128 |
back) and so on. |
| 1129 |
|
| 1130 |
The specific HTTP response code does not constitute a problem or error for |
| 1131 |
curl. It simply sends and delivers HTTP as you asked and if that worked, |
| 1132 |
everything is fine and dandy. The response code is generally providing more |
| 1133 |
higher level error information that curl doesn't care about. The error was |
| 1134 |
not in the HTTP transfer. |
| 1135 |
|
| 1136 |
If you want your command line to treat error codes in the 400 and up range |
| 1137 |
as errors and thus return a non-zero value and possibly show an error |
| 1138 |
message, curl has a dedicated option for that: -f (CURLOPT_FAILONERROR in |
| 1139 |
libcurl speak). |
| 1140 |
|
| 1141 |
You can also use the -w option and the variable %{response_code} to extract |
| 1142 |
the exact response code that was returned in the response. |
| 1143 |
|
| 1144 |
4.21 Why is there a HTTP/1.1 in my HTTP/2 request? |
| 1145 |
|
| 1146 |
If you use verbose to see the HTTP request when you send off a HTTP/2 |
| 1147 |
request, it will still say 1.1. |
| 1148 |
|
| 1149 |
The reason for this is that we first generate the request to send using the |
| 1150 |
old 1.1 style and show that request in the verbose output, and then we |
| 1151 |
convert it over to the binary header-compressed HTTP/2 style. The actual |
| 1152 |
"1.1" part from that request is then not actually used in the transfer. |
| 1153 |
The binary HTTP/2 headers are not human readable. |
| 1154 |
|
| 1155 |
5. libcurl Issues |
| 1156 |
|
| 1157 |
5.1 Is libcurl thread-safe? |
| 1158 |
|
| 1159 |
Yes. |
| 1160 |
|
| 1161 |
We have written the libcurl code specifically adjusted for multi-threaded |
| 1162 |
programs. libcurl will use thread-safe functions instead of non-safe ones if |
| 1163 |
your system has such. Note that you must never share the same handle in |
| 1164 |
multiple threads. |
| 1165 |
|
| 1166 |
There may be some exceptions to thread safety depending on how libcurl was |
| 1167 |
built. Please review the guidelines for thread safety to learn more: |
| 1168 |
https://curl.haxx.se/libcurl/c/threadsafe.html |
| 1169 |
|
| 1170 |
5.2 How can I receive all data into a large memory chunk? |
| 1171 |
|
| 1172 |
[ See also the examples/getinmemory.c source ] |
| 1173 |
|
| 1174 |
You are in full control of the callback function that gets called every time |
| 1175 |
there is data received from the remote server. You can make that callback do |
| 1176 |
whatever you want. You do not have to write the received data to a file. |
| 1177 |
|
| 1178 |
One solution to this problem could be to have a pointer to a struct that you |
| 1179 |
pass to the callback function. You set the pointer using the |
| 1180 |
CURLOPT_WRITEDATA option. Then that pointer will be passed to the callback |
| 1181 |
instead of a FILE * to a file: |
| 1182 |
|
| 1183 |
/* imaginary struct */ |
| 1184 |
struct MemoryStruct { |
| 1185 |
char *memory; |
| 1186 |
size_t size; |
| 1187 |
}; |
| 1188 |
|
| 1189 |
/* imaginary callback function */ |
| 1190 |
size_t |
| 1191 |
WriteMemoryCallback(void *ptr, size_t size, size_t nmemb, void *data) |
| 1192 |
{ |
| 1193 |
size_t realsize = size * nmemb; |
| 1194 |
struct MemoryStruct *mem = (struct MemoryStruct *)data; |
| 1195 |
|
| 1196 |
mem->memory = (char *)realloc(mem->memory, mem->size + realsize + 1); |
| 1197 |
if (mem->memory) { |
| 1198 |
memcpy(&(mem->memory[mem->size]), ptr, realsize); |
| 1199 |
mem->size += realsize; |
| 1200 |
mem->memory[mem->size] = 0; |
| 1201 |
} |
| 1202 |
return realsize; |
| 1203 |
} |
| 1204 |
|
| 1205 |
5.3 How do I fetch multiple files with libcurl? |
| 1206 |
|
| 1207 |
libcurl has excellent support for transferring multiple files. You should |
| 1208 |
just repeatedly set new URLs with curl_easy_setopt() and then transfer it |
| 1209 |
with curl_easy_perform(). The handle you get from curl_easy_init() is not |
| 1210 |
only reusable, but you're even encouraged to reuse it if you can, as that |
| 1211 |
will enable libcurl to use persistent connections. |
| 1212 |
|
| 1213 |
5.4 Does libcurl do Winsock initialization on win32 systems? |
| 1214 |
|
| 1215 |
Yes, if told to in the curl_global_init() call. |
| 1216 |
|
| 1217 |
5.5 Does CURLOPT_WRITEDATA and CURLOPT_READDATA work on win32 ? |
| 1218 |
|
| 1219 |
Yes, but you cannot open a FILE * and pass the pointer to a DLL and have |
| 1220 |
that DLL use the FILE * (as the DLL and the client application cannot access |
| 1221 |
each others' variable memory areas). If you set CURLOPT_WRITEDATA you must |
| 1222 |
also use CURLOPT_WRITEFUNCTION as well to set a function that writes the |
| 1223 |
file, even if that simply writes the data to the specified FILE *. |
| 1224 |
Similarly, if you use CURLOPT_READDATA you must also specify |
| 1225 |
CURLOPT_READFUNCTION. |
| 1226 |
|
| 1227 |
5.6 What about Keep-Alive or persistent connections? |
| 1228 |
|
| 1229 |
curl and libcurl have excellent support for persistent connections when |
| 1230 |
transferring several files from the same server. Curl will attempt to reuse |
| 1231 |
connections for all URLs specified on the same command line/config file, and |
| 1232 |
libcurl will reuse connections for all transfers that are made using the |
| 1233 |
same libcurl handle. |
| 1234 |
|
| 1235 |
When you use the easy interface the connection cache is kept within the easy |
| 1236 |
handle. If you instead use the multi interface, the connection cache will be |
| 1237 |
kept within the multi handle and will be shared among all the easy handles |
| 1238 |
that are used within the same multi handle. |
| 1239 |
|
| 1240 |
5.7 Link errors when building libcurl on Windows! |
| 1241 |
|
| 1242 |
You need to make sure that your project, and all the libraries (both static |
| 1243 |
and dynamic) that it links against, are compiled/linked against the same run |
| 1244 |
time library. |
| 1245 |
|
| 1246 |
This is determined by the /MD, /ML, /MT (and their corresponding /M?d) |
| 1247 |
options to the command line compiler. /MD (linking against MSVCRT dll) seems |
| 1248 |
to be the most commonly used option. |
| 1249 |
|
| 1250 |
When building an application that uses the static libcurl library, you must |
| 1251 |
add -DCURL_STATICLIB to your CFLAGS. Otherwise the linker will look for |
| 1252 |
dynamic import symbols. If you're using Visual Studio, you need to instead |
| 1253 |
add CURL_STATICLIB in the "Preprocessor Definitions" section. |
| 1254 |
|
| 1255 |
If you get linker error like "unknown symbol __imp__curl_easy_init ..." you |
| 1256 |
have linked against the wrong (static) library. If you want to use the |
| 1257 |
libcurl.dll and import lib, you don't need any extra CFLAGS, but use one of |
| 1258 |
the import libraries below. These are the libraries produced by the various |
| 1259 |
lib/Makefile.* files: |
| 1260 |
|
| 1261 |
Target: static lib. import lib for libcurl*.dll. |
| 1262 |
----------------------------------------------------------- |
| 1263 |
MingW: libcurl.a libcurldll.a |
| 1264 |
MSVC (release): libcurl.lib libcurl_imp.lib |
| 1265 |
MSVC (debug): libcurld.lib libcurld_imp.lib |
| 1266 |
Borland: libcurl.lib libcurl_imp.lib |
| 1267 |
|
| 1268 |
5.8 libcurl.so.X: open failed: No such file or directory |
| 1269 |
|
| 1270 |
This is an error message you might get when you try to run a program linked |
| 1271 |
with a shared version of libcurl and your run-time linker (ld.so) couldn't |
| 1272 |
find the shared library named libcurl.so.X. (Where X is the number of the |
| 1273 |
current libcurl ABI, typically 3 or 4). |
| 1274 |
|
| 1275 |
You need to make sure that ld.so finds libcurl.so.X. You can do that |
| 1276 |
multiple ways, and it differs somewhat between different operating systems, |
| 1277 |
but they are usually: |
| 1278 |
|
| 1279 |
* Add an option to the linker command line that specify the hard-coded path |
| 1280 |
the run-time linker should check for the lib (usually -R) |
| 1281 |
|
| 1282 |
* Set an environment variable (LD_LIBRARY_PATH for example) where ld.so |
| 1283 |
should check for libs |
| 1284 |
|
| 1285 |
* Adjust the system's config to check for libs in the directory where you've |
| 1286 |
put the dir (like Linux's /etc/ld.so.conf) |
| 1287 |
|
| 1288 |
'man ld.so' and 'man ld' will tell you more details |
| 1289 |
|
| 1290 |
5.9 How does libcurl resolve host names? |
| 1291 |
|
| 1292 |
libcurl supports a large a number of different name resolve functions. One |
| 1293 |
of them is picked at build-time and will be used unconditionally. Thus, if |
| 1294 |
you want to change name resolver function you must rebuild libcurl and tell |
| 1295 |
it to use a different function. |
| 1296 |
|
| 1297 |
- The non-IPv6 resolver that can use one of four different host name resolve |
| 1298 |
calls (depending on what your system supports): |
| 1299 |
|
| 1300 |
A - gethostbyname() |
| 1301 |
B - gethostbyname_r() with 3 arguments |
| 1302 |
C - gethostbyname_r() with 5 arguments |
| 1303 |
D - gethostbyname_r() with 6 arguments |
| 1304 |
|
| 1305 |
- The IPv6-resolver that uses getaddrinfo() |
| 1306 |
|
| 1307 |
- The c-ares based name resolver that uses the c-ares library for resolves. |
| 1308 |
Using this offers asynchronous name resolves. |
| 1309 |
|
| 1310 |
- The threaded resolver (default option on Windows). It uses: |
| 1311 |
|
| 1312 |
A - gethostbyname() on plain IPv4 hosts |
| 1313 |
B - getaddrinfo() on IPv6 enabled hosts |
| 1314 |
|
| 1315 |
Also note that libcurl never resolves or reverse-lookups addresses given as |
| 1316 |
pure numbers, such as 127.0.0.1 or ::1. |
| 1317 |
|
| 1318 |
5.10 How do I prevent libcurl from writing the response to stdout? |
| 1319 |
|
| 1320 |
libcurl provides a default built-in write function that writes received data |
| 1321 |
to stdout. Set the CURLOPT_WRITEFUNCTION to receive the data, or possibly |
| 1322 |
set CURLOPT_WRITEDATA to a different FILE * handle. |
| 1323 |
|
| 1324 |
5.11 How do I make libcurl not receive the whole HTTP response? |
| 1325 |
|
| 1326 |
You make the write callback (or progress callback) return an error and |
| 1327 |
libcurl will then abort the transfer. |
| 1328 |
|
| 1329 |
5.12 Can I make libcurl fake or hide my real IP address? |
| 1330 |
|
| 1331 |
No. libcurl operates on a higher level. Besides, faking IP address would |
| 1332 |
imply sending IP packets with a made-up source address, and then you normally |
| 1333 |
get a problem with receiving the packet sent back as they would then not be |
| 1334 |
routed to you! |
| 1335 |
|
| 1336 |
If you use a proxy to access remote sites, the sites will not see your local |
| 1337 |
IP address but instead the address of the proxy. |
| 1338 |
|
| 1339 |
Also note that on many networks NATs or other IP-munging techniques are used |
| 1340 |
that makes you see and use a different IP address locally than what the |
| 1341 |
remote server will see you coming from. You may also consider using |
| 1342 |
https://www.torproject.org/ . |
| 1343 |
|
| 1344 |
5.13 How do I stop an ongoing transfer? |
| 1345 |
|
| 1346 |
With the easy interface you make sure to return the correct error code from |
| 1347 |
one of the callbacks, but none of them are instant. There is no function you |
| 1348 |
can call from another thread or similar that will stop it immediately. |
| 1349 |
Instead, you need to make sure that one of the callbacks you use returns an |
| 1350 |
appropriate value that will stop the transfer. Suitable callbacks that you |
| 1351 |
can do this with include the progress callback, the read callback and the |
| 1352 |
write callback. |
| 1353 |
|
| 1354 |
If you're using the multi interface, you can also stop a transfer by |
| 1355 |
removing the particular easy handle from the multi stack at any moment you |
| 1356 |
think the transfer is done or when you wish to abort the transfer. |
| 1357 |
|
| 1358 |
5.14 Using C++ non-static functions for callbacks? |
| 1359 |
|
| 1360 |
libcurl is a C library, it doesn't know anything about C++ member functions. |
| 1361 |
|
| 1362 |
You can overcome this "limitation" with relative ease using a static |
| 1363 |
member function that is passed a pointer to the class: |
| 1364 |
|
| 1365 |
// f is the pointer to your object. |
| 1366 |
static size_t YourClass::func(void *buffer, size_t sz, size_t n, void *f) |
| 1367 |
{ |
| 1368 |
// Call non-static member function. |
| 1369 |
static_cast<YourClass*>(f)->nonStaticFunction(); |
| 1370 |
} |
| 1371 |
|
| 1372 |
// This is how you pass pointer to the static function: |
| 1373 |
curl_easy_setopt(hcurl, CURLOPT_WRITEFUNCTION, YourClass::func); |
| 1374 |
curl_easy_setopt(hcurl, CURLOPT_WRITEDATA, this); |
| 1375 |
|
| 1376 |
5.15 How do I get an FTP directory listing? |
| 1377 |
|
| 1378 |
If you end the FTP URL you request with a slash, libcurl will provide you |
| 1379 |
with a directory listing of that given directory. You can also set |
| 1380 |
CURLOPT_CUSTOMREQUEST to alter what exact listing command libcurl would use |
| 1381 |
to list the files. |
| 1382 |
|
| 1383 |
The follow-up question tends to be how is a program supposed to parse the |
| 1384 |
directory listing. How does it know what's a file and what's a dir and what's |
| 1385 |
a symlink etc. If the FTP server supports the MLSD command then it will |
| 1386 |
return data in a machine-readable format that can be parsed for type. The |
| 1387 |
types are specified by RFC3659 section 7.5.1. If MLSD is not supported then |
| 1388 |
you have to work with what you're given. The LIST output format is entirely |
| 1389 |
at the server's own liking and the NLST output doesn't reveal any types and |
| 1390 |
in many cases doesn't even include all the directory entries. Also, both LIST |
| 1391 |
and NLST tend to hide unix-style hidden files (those that start with a dot) |
| 1392 |
by default so you need to do "LIST -a" or similar to see them. |
| 1393 |
|
| 1394 |
Example - List only directories. |
| 1395 |
ftp.funet.fi supports MLSD and ftp.kernel.org does not: |
| 1396 |
|
| 1397 |
curl -s ftp.funet.fi/pub/ -X MLSD | \ |
| 1398 |
perl -lne 'print if s/(?:^|;)type=dir;[^ ]+ (.+)$/$1/' |
| 1399 |
|
| 1400 |
curl -s ftp.kernel.org/pub/linux/kernel/ | \ |
| 1401 |
perl -lne 'print if s/^d[-rwx]{9}(?: +[^ ]+){7} (.+)$/$1/' |
| 1402 |
|
| 1403 |
If you need to parse LIST output in libcurl one such existing |
| 1404 |
list parser is available at https://cr.yp.to/ftpparse.html Versions of |
| 1405 |
libcurl since 7.21.0 also provide the ability to specify a wildcard to |
| 1406 |
download multiple files from one FTP directory. |
| 1407 |
|
| 1408 |
5.16 I want a different time-out! |
| 1409 |
|
| 1410 |
Time and time again users realize that CURLOPT_TIMEOUT and |
| 1411 |
CURLOPT_CONNECTIMEOUT are not sufficiently advanced or flexible to cover all |
| 1412 |
the various use cases and scenarios applications end up with. |
| 1413 |
|
| 1414 |
libcurl offers many more ways to time-out operations. A common alternative |
| 1415 |
is to use the CURLOPT_LOW_SPEED_LIMIT and CURLOPT_LOW_SPEED_TIME options to |
| 1416 |
specify the lowest possible speed to accept before to consider the transfer |
| 1417 |
timed out. |
| 1418 |
|
| 1419 |
The most flexible way is by writing your own time-out logic and using |
| 1420 |
CURLOPT_XFERINFOFUNCTION (perhaps in combination with other callbacks) and |
| 1421 |
use that to figure out exactly when the right condition is met when the |
| 1422 |
transfer should get stopped. |
| 1423 |
|
| 1424 |
5.17 Can I write a server with libcurl? |
| 1425 |
|
| 1426 |
No. libcurl offers no functions or building blocks to build any kind of |
| 1427 |
internet protocol server. libcurl is only a client-side library. For server |
| 1428 |
libraries, you need to continue your search elsewhere but there exist many |
| 1429 |
good open source ones out there for most protocols you could possibly want a |
| 1430 |
server for. And there are really good stand-alone ones that have been tested |
| 1431 |
and proven for many years. There's no need for you to reinvent them! |
| 1432 |
|
| 1433 |
5.18 Does libcurl use threads? |
| 1434 |
|
| 1435 |
Put simply: no, libcurl will execute in the same thread you call it in. All |
| 1436 |
callbacks will be called in the same thread as the one you call libcurl in. |
| 1437 |
|
| 1438 |
If you want to avoid your thread to be blocked by the libcurl call, you make |
| 1439 |
sure you use the non-blocking API which will do transfers asynchronously - |
| 1440 |
but still in the same single thread. |
| 1441 |
|
| 1442 |
libcurl will potentially internally use threads for name resolving, if it |
| 1443 |
was built to work like that, but in those cases it'll create the child |
| 1444 |
threads by itself and they will only be used and then killed internally by |
| 1445 |
libcurl and never exposed to the outside. |
| 1446 |
|
| 1447 |
6. License Issues |
| 1448 |
|
| 1449 |
Curl and libcurl are released under a MIT/X derivate license. The license is |
| 1450 |
very liberal and should not impose a problem for your project. This section |
| 1451 |
is just a brief summary for the cases we get the most questions. (Parts of |
| 1452 |
this section was much enhanced by Bjorn Reese.) |
| 1453 |
|
| 1454 |
We are not lawyers and this is not legal advice. You should probably consult |
| 1455 |
one if you want true and accurate legal insights without our prejudice. Note |
| 1456 |
especially that this section concerns the libcurl license only; compiling in |
| 1457 |
features of libcurl that depend on other libraries (e.g. OpenSSL) may affect |
| 1458 |
the licensing obligations of your application. |
| 1459 |
|
| 1460 |
6.1 I have a GPL program, can I use the libcurl library? |
| 1461 |
|
| 1462 |
Yes! |
| 1463 |
|
| 1464 |
Since libcurl may be distributed under the MIT/X derivate license, it can be |
| 1465 |
used together with GPL in any software. |
| 1466 |
|
| 1467 |
6.2 I have a closed-source program, can I use the libcurl library? |
| 1468 |
|
| 1469 |
Yes! |
| 1470 |
|
| 1471 |
libcurl does not put any restrictions on the program that uses the library. |
| 1472 |
|
| 1473 |
6.3 I have a BSD licensed program, can I use the libcurl library? |
| 1474 |
|
| 1475 |
Yes! |
| 1476 |
|
| 1477 |
libcurl does not put any restrictions on the program that uses the library. |
| 1478 |
|
| 1479 |
6.4 I have a program that uses LGPL libraries, can I use libcurl? |
| 1480 |
|
| 1481 |
Yes! |
| 1482 |
|
| 1483 |
The LGPL license doesn't clash with other licenses. |
| 1484 |
|
| 1485 |
6.5 Can I modify curl/libcurl for my program and keep the changes secret? |
| 1486 |
|
| 1487 |
Yes! |
| 1488 |
|
| 1489 |
The MIT/X derivate license practically allows you to do almost anything with |
| 1490 |
the sources, on the condition that the copyright texts in the sources are |
| 1491 |
left intact. |
| 1492 |
|
| 1493 |
6.6 Can you please change the curl/libcurl license to XXXX? |
| 1494 |
|
| 1495 |
No. |
| 1496 |
|
| 1497 |
We have carefully picked this license after years of development and |
| 1498 |
discussions and a large amount of people have contributed with source code |
| 1499 |
knowing that this is the license we use. This license puts the restrictions |
| 1500 |
we want on curl/libcurl and it does not spread to other programs or |
| 1501 |
libraries that use it. It should be possible for everyone to use libcurl or |
| 1502 |
curl in their projects, no matter what license they already have in use. |
| 1503 |
|
| 1504 |
6.7 What are my obligations when using libcurl in my commercial apps? |
| 1505 |
|
| 1506 |
Next to none. All you need to adhere to is the MIT-style license (stated in |
| 1507 |
the COPYING file) which basically says you have to include the copyright |
| 1508 |
notice in "all copies" and that you may not use the copyright holder's name |
| 1509 |
when promoting your software. |
| 1510 |
|
| 1511 |
You do not have to release any of your source code. |
| 1512 |
|
| 1513 |
You do not have to reveal or make public any changes to the libcurl source |
| 1514 |
code. |
| 1515 |
|
| 1516 |
You do not have to broadcast to the world that you are using libcurl within |
| 1517 |
your app. |
| 1518 |
|
| 1519 |
All we ask is that you disclose "the copyright notice and this permission |
| 1520 |
notice" somewhere. Most probably like in the documentation or in the section |
| 1521 |
where other third party dependencies already are mentioned and acknowledged. |
| 1522 |
|
| 1523 |
As can be seen here: https://curl.haxx.se/docs/companies.html and elsewhere, |
| 1524 |
more and more companies are discovering the power of libcurl and take |
| 1525 |
advantage of it even in commercial environments. |
| 1526 |
|
| 1527 |
|
| 1528 |
7. PHP/CURL Issues |
| 1529 |
|
| 1530 |
7.1 What is PHP/CURL? |
| 1531 |
|
| 1532 |
The module for PHP that makes it possible for PHP programs to access curl- |
| 1533 |
functions from within PHP. |
| 1534 |
|
| 1535 |
In the cURL project we call this module PHP/CURL to differentiate it from |
| 1536 |
curl the command line tool and libcurl the library. The PHP team however |
| 1537 |
does not refer to it like this (for unknown reasons). They call it plain |
| 1538 |
CURL (often using all caps) or sometimes ext/curl, but both cause much |
| 1539 |
confusion to users which in turn gives us a higher question load. |
| 1540 |
|
| 1541 |
7.2 Who wrote PHP/CURL? |
| 1542 |
|
| 1543 |
PHP/CURL was initially written by Sterling Hughes. |
| 1544 |
|
| 1545 |
7.3 Can I perform multiple requests using the same handle? |
| 1546 |
|
| 1547 |
Yes - at least in PHP version 4.3.8 and later (this has been known to not |
| 1548 |
work in earlier versions, but the exact version when it started to work is |
| 1549 |
unknown to me). |
| 1550 |
|
| 1551 |
After a transfer, you just set new options in the handle and make another |
| 1552 |
transfer. This will make libcurl re-use the same connection if it can. |
| 1553 |
|
| 1554 |
7.4 Does PHP/CURL have dependencies? |
| 1555 |
|
| 1556 |
PHP/CURL is a module that comes with the regular PHP package. It depends on |
| 1557 |
and uses libcurl, so you need to have libcurl installed properly before |
| 1558 |
PHP/CURL can be used. |