| 1 |
.\" ************************************************************************** |
| 2 |
.\" * _ _ ____ _ |
| 3 |
.\" * Project ___| | | | _ \| | |
| 4 |
.\" * / __| | | | |_) | | |
| 5 |
.\" * | (__| |_| | _ <| |___ |
| 6 |
.\" * \___|\___/|_| \_\_____| |
| 7 |
.\" * |
| 8 |
.\" * Copyright (C) 1998 - 2019, Daniel Stenberg, <[email protected]>, et al. |
| 9 |
.\" * |
| 10 |
.\" * This software is licensed as described in the file COPYING, which |
| 11 |
.\" * you should have received as part of this distribution. The terms |
| 12 |
.\" * are also available at https://curl.haxx.se/docs/copyright.html. |
| 13 |
.\" * |
| 14 |
.\" * You may opt to use, copy, modify, merge, publish, distribute and/or sell |
| 15 |
.\" * copies of the Software, and permit persons to whom the Software is |
| 16 |
.\" * furnished to do so, under the terms of the COPYING file. |
| 17 |
.\" * |
| 18 |
.\" * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY |
| 19 |
.\" * KIND, either express or implied. |
| 20 |
.\" * |
| 21 |
.\" ************************************************************************** |
| 22 |
.\" |
| 23 |
.TH libcurl-tutorial 3 "19 Sep 2014" "libcurl" "libcurl programming" |
| 24 |
.SH NAME |
| 25 |
libcurl-tutorial \- libcurl programming tutorial |
| 26 |
.SH "Objective" |
| 27 |
This document attempts to describe the general principles and some basic |
| 28 |
approaches to consider when programming with libcurl. The text will focus |
| 29 |
mainly on the C interface but might apply fairly well on other interfaces as |
| 30 |
well as they usually follow the C one pretty closely. |
| 31 |
|
| 32 |
This document will refer to 'the user' as the person writing the source code |
| 33 |
that uses libcurl. That would probably be you or someone in your position. |
| 34 |
What will be generally referred to as 'the program' will be the collected |
| 35 |
source code that you write that is using libcurl for transfers. The program |
| 36 |
is outside libcurl and libcurl is outside of the program. |
| 37 |
|
| 38 |
To get more details on all options and functions described herein, please |
| 39 |
refer to their respective man pages. |
| 40 |
|
| 41 |
.SH "Building" |
| 42 |
There are many different ways to build C programs. This chapter will assume a |
| 43 |
Unix style build process. If you use a different build system, you can still |
| 44 |
read this to get general information that may apply to your environment as |
| 45 |
well. |
| 46 |
.IP "Compiling the Program" |
| 47 |
Your compiler needs to know where the libcurl headers are located. Therefore |
| 48 |
you must set your compiler's include path to point to the directory where you |
| 49 |
installed them. The 'curl-config'[3] tool can be used to get this information: |
| 50 |
|
| 51 |
$ curl-config --cflags |
| 52 |
|
| 53 |
.IP "Linking the Program with libcurl" |
| 54 |
When having compiled the program, you need to link your object files to create |
| 55 |
a single executable. For that to succeed, you need to link with libcurl and |
| 56 |
possibly also with other libraries that libcurl itself depends on. Like the |
| 57 |
OpenSSL libraries, but even some standard OS libraries may be needed on the |
| 58 |
command line. To figure out which flags to use, once again the 'curl-config' |
| 59 |
tool comes to the rescue: |
| 60 |
|
| 61 |
$ curl-config --libs |
| 62 |
|
| 63 |
.IP "SSL or Not" |
| 64 |
libcurl can be built and customized in many ways. One of the things that |
| 65 |
varies from different libraries and builds is the support for SSL-based |
| 66 |
transfers, like HTTPS and FTPS. If a supported SSL library was detected |
| 67 |
properly at build-time, libcurl will be built with SSL support. To figure out |
| 68 |
if an installed libcurl has been built with SSL support enabled, use |
| 69 |
\&'curl-config' like this: |
| 70 |
|
| 71 |
$ curl-config --feature |
| 72 |
|
| 73 |
And if SSL is supported, the keyword 'SSL' will be written to stdout, |
| 74 |
possibly together with a few other features that could be either on or off on |
| 75 |
for different libcurls. |
| 76 |
|
| 77 |
See also the "Features libcurl Provides" further down. |
| 78 |
.IP "autoconf macro" |
| 79 |
When you write your configure script to detect libcurl and setup variables |
| 80 |
accordingly, we offer a prewritten macro that probably does everything you |
| 81 |
need in this area. See docs/libcurl/libcurl.m4 file - it includes docs on how |
| 82 |
to use it. |
| 83 |
|
| 84 |
.SH "Portable Code in a Portable World" |
| 85 |
The people behind libcurl have put a considerable effort to make libcurl work |
| 86 |
on a large amount of different operating systems and environments. |
| 87 |
|
| 88 |
You program libcurl the same way on all platforms that libcurl runs on. There |
| 89 |
are only very few minor considerations that differ. If you just make sure to |
| 90 |
write your code portable enough, you may very well create yourself a very |
| 91 |
portable program. libcurl shouldn't stop you from that. |
| 92 |
|
| 93 |
.SH "Global Preparation" |
| 94 |
The program must initialize some of the libcurl functionality globally. That |
| 95 |
means it should be done exactly once, no matter how many times you intend to |
| 96 |
use the library. Once for your program's entire life time. This is done using |
| 97 |
|
| 98 |
curl_global_init() |
| 99 |
|
| 100 |
and it takes one parameter which is a bit pattern that tells libcurl what to |
| 101 |
initialize. Using \fICURL_GLOBAL_ALL\fP will make it initialize all known |
| 102 |
internal sub modules, and might be a good default option. The current two bits |
| 103 |
that are specified are: |
| 104 |
.RS |
| 105 |
.IP "CURL_GLOBAL_WIN32" |
| 106 |
which only does anything on Windows machines. When used on |
| 107 |
a Windows machine, it'll make libcurl initialize the win32 socket |
| 108 |
stuff. Without having that initialized properly, your program cannot use |
| 109 |
sockets properly. You should only do this once for each application, so if |
| 110 |
your program already does this or of another library in use does it, you |
| 111 |
should not tell libcurl to do this as well. |
| 112 |
.IP CURL_GLOBAL_SSL |
| 113 |
which only does anything on libcurls compiled and built SSL-enabled. On these |
| 114 |
systems, this will make libcurl initialize the SSL library properly for this |
| 115 |
application. This only needs to be done once for each application so if your |
| 116 |
program or another library already does this, this bit should not be needed. |
| 117 |
.RE |
| 118 |
|
| 119 |
libcurl has a default protection mechanism that detects if |
| 120 |
\fIcurl_global_init(3)\fP hasn't been called by the time |
| 121 |
\fIcurl_easy_perform(3)\fP is called and if that is the case, libcurl runs the |
| 122 |
function itself with a guessed bit pattern. Please note that depending solely |
| 123 |
on this is not considered nice nor very good. |
| 124 |
|
| 125 |
When the program no longer uses libcurl, it should call |
| 126 |
\fIcurl_global_cleanup(3)\fP, which is the opposite of the init call. It will |
| 127 |
then do the reversed operations to cleanup the resources the |
| 128 |
\fIcurl_global_init(3)\fP call initialized. |
| 129 |
|
| 130 |
Repeated calls to \fIcurl_global_init(3)\fP and \fIcurl_global_cleanup(3)\fP |
| 131 |
should be avoided. They should only be called once each. |
| 132 |
|
| 133 |
.SH "Features libcurl Provides" |
| 134 |
It is considered best-practice to determine libcurl features at run-time |
| 135 |
rather than at build-time (if possible of course). By calling |
| 136 |
\fIcurl_version_info(3)\fP and checking out the details of the returned |
| 137 |
struct, your program can figure out exactly what the currently running libcurl |
| 138 |
supports. |
| 139 |
|
| 140 |
.SH "Two Interfaces" |
| 141 |
libcurl first introduced the so called easy interface. All operations in the |
| 142 |
easy interface are prefixed with 'curl_easy'. The easy interface lets you do |
| 143 |
single transfers with a synchronous and blocking function call. |
| 144 |
|
| 145 |
libcurl also offers another interface that allows multiple simultaneous |
| 146 |
transfers in a single thread, the so called multi interface. More about that |
| 147 |
interface is detailed in a separate chapter further down. You still need to |
| 148 |
understand the easy interface first, so please continue reading for better |
| 149 |
understanding. |
| 150 |
.SH "Handle the Easy libcurl" |
| 151 |
To use the easy interface, you must first create yourself an easy handle. You |
| 152 |
need one handle for each easy session you want to perform. Basically, you |
| 153 |
should use one handle for every thread you plan to use for transferring. You |
| 154 |
must never share the same handle in multiple threads. |
| 155 |
|
| 156 |
Get an easy handle with |
| 157 |
|
| 158 |
easyhandle = curl_easy_init(); |
| 159 |
|
| 160 |
It returns an easy handle. Using that you proceed to the next step: setting |
| 161 |
up your preferred actions. A handle is just a logic entity for the upcoming |
| 162 |
transfer or series of transfers. |
| 163 |
|
| 164 |
You set properties and options for this handle using |
| 165 |
\fIcurl_easy_setopt(3)\fP. They control how the subsequent transfer or |
| 166 |
transfers will be made. Options remain set in the handle until set again to |
| 167 |
something different. They are sticky. Multiple requests using the same handle |
| 168 |
will use the same options. |
| 169 |
|
| 170 |
If you at any point would like to blank all previously set options for a |
| 171 |
single easy handle, you can call \fIcurl_easy_reset(3)\fP and you can also |
| 172 |
make a clone of an easy handle (with all its set options) using |
| 173 |
\fIcurl_easy_duphandle(3)\fP. |
| 174 |
|
| 175 |
Many of the options you set in libcurl are "strings", pointers to data |
| 176 |
terminated with a zero byte. When you set strings with |
| 177 |
\fIcurl_easy_setopt(3)\fP, libcurl makes its own copy so that they don't need |
| 178 |
to be kept around in your application after being set[4]. |
| 179 |
|
| 180 |
One of the most basic properties to set in the handle is the URL. You set your |
| 181 |
preferred URL to transfer with \fICURLOPT_URL(3)\fP in a manner similar to: |
| 182 |
|
| 183 |
.nf |
| 184 |
curl_easy_setopt(handle, CURLOPT_URL, "http://domain.com/"); |
| 185 |
.fi |
| 186 |
|
| 187 |
Let's assume for a while that you want to receive data as the URL identifies a |
| 188 |
remote resource you want to get here. Since you write a sort of application |
| 189 |
that needs this transfer, I assume that you would like to get the data passed |
| 190 |
to you directly instead of simply getting it passed to stdout. So, you write |
| 191 |
your own function that matches this prototype: |
| 192 |
|
| 193 |
size_t write_data(void *buffer, size_t size, size_t nmemb, void *userp); |
| 194 |
|
| 195 |
You tell libcurl to pass all data to this function by issuing a function |
| 196 |
similar to this: |
| 197 |
|
| 198 |
curl_easy_setopt(easyhandle, CURLOPT_WRITEFUNCTION, write_data); |
| 199 |
|
| 200 |
You can control what data your callback function gets in the fourth argument |
| 201 |
by setting another property: |
| 202 |
|
| 203 |
curl_easy_setopt(easyhandle, CURLOPT_WRITEDATA, &internal_struct); |
| 204 |
|
| 205 |
Using that property, you can easily pass local data between your application |
| 206 |
and the function that gets invoked by libcurl. libcurl itself won't touch the |
| 207 |
data you pass with \fICURLOPT_WRITEDATA(3)\fP. |
| 208 |
|
| 209 |
libcurl offers its own default internal callback that will take care of the |
| 210 |
data if you don't set the callback with \fICURLOPT_WRITEFUNCTION(3)\fP. It |
| 211 |
will then simply output the received data to stdout. You can have the default |
| 212 |
callback write the data to a different file handle by passing a 'FILE *' to a |
| 213 |
file opened for writing with the \fICURLOPT_WRITEDATA(3)\fP option. |
| 214 |
|
| 215 |
Now, we need to take a step back and have a deep breath. Here's one of those |
| 216 |
rare platform-dependent nitpicks. Did you spot it? On some platforms[2], |
| 217 |
libcurl won't be able to operate on files opened by the program. Thus, if you |
| 218 |
use the default callback and pass in an open file with |
| 219 |
\fICURLOPT_WRITEDATA(3)\fP, it will crash. You should therefore avoid this to |
| 220 |
make your program run fine virtually everywhere. |
| 221 |
|
| 222 |
(\fICURLOPT_WRITEDATA(3)\fP was formerly known as \fICURLOPT_FILE\fP. Both |
| 223 |
names still work and do the same thing). |
| 224 |
|
| 225 |
If you're using libcurl as a win32 DLL, you MUST use the |
| 226 |
\fICURLOPT_WRITEFUNCTION(3)\fP if you set \fICURLOPT_WRITEDATA(3)\fP - or you |
| 227 |
will experience crashes. |
| 228 |
|
| 229 |
There are of course many more options you can set, and we'll get back to a few |
| 230 |
of them later. Let's instead continue to the actual transfer: |
| 231 |
|
| 232 |
success = curl_easy_perform(easyhandle); |
| 233 |
|
| 234 |
\fIcurl_easy_perform(3)\fP will connect to the remote site, do the necessary |
| 235 |
commands and receive the transfer. Whenever it receives data, it calls the |
| 236 |
callback function we previously set. The function may get one byte at a time, |
| 237 |
or it may get many kilobytes at once. libcurl delivers as much as possible as |
| 238 |
often as possible. Your callback function should return the number of bytes it |
| 239 |
\&"took care of". If that is not the exact same amount of bytes that was |
| 240 |
passed to it, libcurl will abort the operation and return with an error code. |
| 241 |
|
| 242 |
When the transfer is complete, the function returns a return code that informs |
| 243 |
you if it succeeded in its mission or not. If a return code isn't enough for |
| 244 |
you, you can use the \fICURLOPT_ERRORBUFFER(3)\fP to point libcurl to a buffer |
| 245 |
of yours where it'll store a human readable error message as well. |
| 246 |
|
| 247 |
If you then want to transfer another file, the handle is ready to be used |
| 248 |
again. Mind you, it is even preferred that you re-use an existing handle if |
| 249 |
you intend to make another transfer. libcurl will then attempt to re-use the |
| 250 |
previous connection. |
| 251 |
|
| 252 |
For some protocols, downloading a file can involve a complicated process of |
| 253 |
logging in, setting the transfer mode, changing the current directory and |
| 254 |
finally transferring the file data. libcurl takes care of all that |
| 255 |
complication for you. Given simply the URL to a file, libcurl will take care |
| 256 |
of all the details needed to get the file moved from one machine to another. |
| 257 |
|
| 258 |
.SH "Multi-threading Issues" |
| 259 |
libcurl is thread safe but there are a few exceptions. Refer to |
| 260 |
\fIlibcurl-thread(3)\fP for more information. |
| 261 |
|
| 262 |
.SH "When It Doesn't Work" |
| 263 |
There will always be times when the transfer fails for some reason. You might |
| 264 |
have set the wrong libcurl option or misunderstood what the libcurl option |
| 265 |
actually does, or the remote server might return non-standard replies that |
| 266 |
confuse the library which then confuses your program. |
| 267 |
|
| 268 |
There's one golden rule when these things occur: set the |
| 269 |
\fICURLOPT_VERBOSE(3)\fP option to 1. It'll cause the library to spew out the |
| 270 |
entire protocol details it sends, some internal info and some received |
| 271 |
protocol data as well (especially when using FTP). If you're using HTTP, |
| 272 |
adding the headers in the received output to study is also a clever way to get |
| 273 |
a better understanding why the server behaves the way it does. Include headers |
| 274 |
in the normal body output with \fICURLOPT_HEADER(3)\fP set 1. |
| 275 |
|
| 276 |
Of course, there are bugs left. We need to know about them to be able to fix |
| 277 |
them, so we're quite dependent on your bug reports! When you do report |
| 278 |
suspected bugs in libcurl, please include as many details as you possibly can: |
| 279 |
a protocol dump that \fICURLOPT_VERBOSE(3)\fP produces, library version, as |
| 280 |
much as possible of your code that uses libcurl, operating system name and |
| 281 |
version, compiler name and version etc. |
| 282 |
|
| 283 |
If \fICURLOPT_VERBOSE(3)\fP is not enough, you increase the level of debug |
| 284 |
data your application receive by using the \fICURLOPT_DEBUGFUNCTION(3)\fP. |
| 285 |
|
| 286 |
Getting some in-depth knowledge about the protocols involved is never wrong, |
| 287 |
and if you're trying to do funny things, you might very well understand |
| 288 |
libcurl and how to use it better if you study the appropriate RFC documents |
| 289 |
at least briefly. |
| 290 |
|
| 291 |
.SH "Upload Data to a Remote Site" |
| 292 |
libcurl tries to keep a protocol independent approach to most transfers, thus |
| 293 |
uploading to a remote FTP site is very similar to uploading data to an HTTP |
| 294 |
server with a PUT request. |
| 295 |
|
| 296 |
Of course, first you either create an easy handle or you re-use one existing |
| 297 |
one. Then you set the URL to operate on just like before. This is the remote |
| 298 |
URL, that we now will upload. |
| 299 |
|
| 300 |
Since we write an application, we most likely want libcurl to get the upload |
| 301 |
data by asking us for it. To make it do that, we set the read callback and |
| 302 |
the custom pointer libcurl will pass to our read callback. The read callback |
| 303 |
should have a prototype similar to: |
| 304 |
|
| 305 |
size_t function(char *bufptr, size_t size, size_t nitems, void *userp); |
| 306 |
|
| 307 |
Where bufptr is the pointer to a buffer we fill in with data to upload and |
| 308 |
size*nitems is the size of the buffer and therefore also the maximum amount |
| 309 |
of data we can return to libcurl in this call. The 'userp' pointer is the |
| 310 |
custom pointer we set to point to a struct of ours to pass private data |
| 311 |
between the application and the callback. |
| 312 |
|
| 313 |
curl_easy_setopt(easyhandle, CURLOPT_READFUNCTION, read_function); |
| 314 |
|
| 315 |
curl_easy_setopt(easyhandle, CURLOPT_READDATA, &filedata); |
| 316 |
|
| 317 |
Tell libcurl that we want to upload: |
| 318 |
|
| 319 |
curl_easy_setopt(easyhandle, CURLOPT_UPLOAD, 1L); |
| 320 |
|
| 321 |
A few protocols won't behave properly when uploads are done without any prior |
| 322 |
knowledge of the expected file size. So, set the upload file size using the |
| 323 |
\fICURLOPT_INFILESIZE_LARGE(3)\fP for all known file sizes like this[1]: |
| 324 |
|
| 325 |
.nf |
| 326 |
/* in this example, file_size must be an curl_off_t variable */ |
| 327 |
curl_easy_setopt(easyhandle, CURLOPT_INFILESIZE_LARGE, file_size); |
| 328 |
.fi |
| 329 |
|
| 330 |
When you call \fIcurl_easy_perform(3)\fP this time, it'll perform all the |
| 331 |
necessary operations and when it has invoked the upload it'll call your |
| 332 |
supplied callback to get the data to upload. The program should return as much |
| 333 |
data as possible in every invoke, as that is likely to make the upload perform |
| 334 |
as fast as possible. The callback should return the number of bytes it wrote |
| 335 |
in the buffer. Returning 0 will signal the end of the upload. |
| 336 |
|
| 337 |
.SH "Passwords" |
| 338 |
Many protocols use or even require that user name and password are provided |
| 339 |
to be able to download or upload the data of your choice. libcurl offers |
| 340 |
several ways to specify them. |
| 341 |
|
| 342 |
Most protocols support that you specify the name and password in the URL |
| 343 |
itself. libcurl will detect this and use them accordingly. This is written |
| 344 |
like this: |
| 345 |
|
| 346 |
protocol://user:[email protected]/path/ |
| 347 |
|
| 348 |
If you need any odd letters in your user name or password, you should enter |
| 349 |
them URL encoded, as %XX where XX is a two-digit hexadecimal number. |
| 350 |
|
| 351 |
libcurl also provides options to set various passwords. The user name and |
| 352 |
password as shown embedded in the URL can instead get set with the |
| 353 |
\fICURLOPT_USERPWD(3)\fP option. The argument passed to libcurl should be a |
| 354 |
char * to a string in the format "user:password". In a manner like this: |
| 355 |
|
| 356 |
curl_easy_setopt(easyhandle, CURLOPT_USERPWD, "myname:thesecret"); |
| 357 |
|
| 358 |
Another case where name and password might be needed at times, is for those |
| 359 |
users who need to authenticate themselves to a proxy they use. libcurl offers |
| 360 |
another option for this, the \fICURLOPT_PROXYUSERPWD(3)\fP. It is used quite |
| 361 |
similar to the \fICURLOPT_USERPWD(3)\fP option like this: |
| 362 |
|
| 363 |
curl_easy_setopt(easyhandle, CURLOPT_PROXYUSERPWD, "myname:thesecret"); |
| 364 |
|
| 365 |
There's a long time Unix "standard" way of storing FTP user names and |
| 366 |
passwords, namely in the $HOME/.netrc file. The file should be made private |
| 367 |
so that only the user may read it (see also the "Security Considerations" |
| 368 |
chapter), as it might contain the password in plain text. libcurl has the |
| 369 |
ability to use this file to figure out what set of user name and password to |
| 370 |
use for a particular host. As an extension to the normal functionality, |
| 371 |
libcurl also supports this file for non-FTP protocols such as HTTP. To make |
| 372 |
curl use this file, use the \fICURLOPT_NETRC(3)\fP option: |
| 373 |
|
| 374 |
curl_easy_setopt(easyhandle, CURLOPT_NETRC, 1L); |
| 375 |
|
| 376 |
And a very basic example of how such a .netrc file may look like: |
| 377 |
|
| 378 |
.nf |
| 379 |
machine myhost.mydomain.com |
| 380 |
login userlogin |
| 381 |
password secretword |
| 382 |
.fi |
| 383 |
|
| 384 |
All these examples have been cases where the password has been optional, or |
| 385 |
at least you could leave it out and have libcurl attempt to do its job |
| 386 |
without it. There are times when the password isn't optional, like when |
| 387 |
you're using an SSL private key for secure transfers. |
| 388 |
|
| 389 |
To pass the known private key password to libcurl: |
| 390 |
|
| 391 |
curl_easy_setopt(easyhandle, CURLOPT_KEYPASSWD, "keypassword"); |
| 392 |
|
| 393 |
.SH "HTTP Authentication" |
| 394 |
The previous chapter showed how to set user name and password for getting |
| 395 |
URLs that require authentication. When using the HTTP protocol, there are |
| 396 |
many different ways a client can provide those credentials to the server and |
| 397 |
you can control which way libcurl will (attempt to) use them. The default HTTP |
| 398 |
authentication method is called 'Basic', which is sending the name and |
| 399 |
password in clear-text in the HTTP request, base64-encoded. This is insecure. |
| 400 |
|
| 401 |
At the time of this writing, libcurl can be built to use: Basic, Digest, NTLM, |
| 402 |
Negotiate (SPNEGO). You can tell libcurl which one to use |
| 403 |
with \fICURLOPT_HTTPAUTH(3)\fP as in: |
| 404 |
|
| 405 |
curl_easy_setopt(easyhandle, CURLOPT_HTTPAUTH, CURLAUTH_DIGEST); |
| 406 |
|
| 407 |
And when you send authentication to a proxy, you can also set authentication |
| 408 |
type the same way but instead with \fICURLOPT_PROXYAUTH(3)\fP: |
| 409 |
|
| 410 |
curl_easy_setopt(easyhandle, CURLOPT_PROXYAUTH, CURLAUTH_NTLM); |
| 411 |
|
| 412 |
Both these options allow you to set multiple types (by ORing them together), |
| 413 |
to make libcurl pick the most secure one out of the types the server/proxy |
| 414 |
claims to support. This method does however add a round-trip since libcurl |
| 415 |
must first ask the server what it supports: |
| 416 |
|
| 417 |
curl_easy_setopt(easyhandle, CURLOPT_HTTPAUTH, |
| 418 |
CURLAUTH_DIGEST|CURLAUTH_BASIC); |
| 419 |
|
| 420 |
For convenience, you can use the 'CURLAUTH_ANY' define (instead of a list |
| 421 |
with specific types) which allows libcurl to use whatever method it wants. |
| 422 |
|
| 423 |
When asking for multiple types, libcurl will pick the available one it |
| 424 |
considers "best" in its own internal order of preference. |
| 425 |
|
| 426 |
.SH "HTTP POSTing" |
| 427 |
We get many questions regarding how to issue HTTP POSTs with libcurl the |
| 428 |
proper way. This chapter will thus include examples using both different |
| 429 |
versions of HTTP POST that libcurl supports. |
| 430 |
|
| 431 |
The first version is the simple POST, the most common version, that most HTML |
| 432 |
pages using the <form> tag uses. We provide a pointer to the data and tell |
| 433 |
libcurl to post it all to the remote site: |
| 434 |
|
| 435 |
.nf |
| 436 |
char *data="name=daniel&project=curl"; |
| 437 |
curl_easy_setopt(easyhandle, CURLOPT_POSTFIELDS, data); |
| 438 |
curl_easy_setopt(easyhandle, CURLOPT_URL, "http://posthere.com/"); |
| 439 |
|
| 440 |
curl_easy_perform(easyhandle); /* post away! */ |
| 441 |
.fi |
| 442 |
|
| 443 |
Simple enough, huh? Since you set the POST options with the |
| 444 |
\fICURLOPT_POSTFIELDS(3)\fP, this automatically switches the handle to use |
| 445 |
POST in the upcoming request. |
| 446 |
|
| 447 |
Ok, so what if you want to post binary data that also requires you to set the |
| 448 |
Content-Type: header of the post? Well, binary posts prevent libcurl from |
| 449 |
being able to do strlen() on the data to figure out the size, so therefore we |
| 450 |
must tell libcurl the size of the post data. Setting headers in libcurl |
| 451 |
requests are done in a generic way, by building a list of our own headers and |
| 452 |
then passing that list to libcurl. |
| 453 |
|
| 454 |
.nf |
| 455 |
struct curl_slist *headers=NULL; |
| 456 |
headers = curl_slist_append(headers, "Content-Type: text/xml"); |
| 457 |
|
| 458 |
/* post binary data */ |
| 459 |
curl_easy_setopt(easyhandle, CURLOPT_POSTFIELDS, binaryptr); |
| 460 |
|
| 461 |
/* set the size of the postfields data */ |
| 462 |
curl_easy_setopt(easyhandle, CURLOPT_POSTFIELDSIZE, 23L); |
| 463 |
|
| 464 |
/* pass our list of custom made headers */ |
| 465 |
curl_easy_setopt(easyhandle, CURLOPT_HTTPHEADER, headers); |
| 466 |
|
| 467 |
curl_easy_perform(easyhandle); /* post away! */ |
| 468 |
|
| 469 |
curl_slist_free_all(headers); /* free the header list */ |
| 470 |
.fi |
| 471 |
|
| 472 |
While the simple examples above cover the majority of all cases where HTTP |
| 473 |
POST operations are required, they don't do multi-part formposts. Multi-part |
| 474 |
formposts were introduced as a better way to post (possibly large) binary data |
| 475 |
and were first documented in the RFC1867 (updated in RFC2388). They're called |
| 476 |
multi-part because they're built by a chain of parts, each part being a single |
| 477 |
unit of data. Each part has its own name and contents. You can in fact create |
| 478 |
and post a multi-part formpost with the regular libcurl POST support described |
| 479 |
above, but that would require that you build a formpost yourself and provide |
| 480 |
to libcurl. To make that easier, libcurl provides a MIME API consisting in |
| 481 |
several functions: using those, you can create and fill a multi-part form. |
| 482 |
Function \fIcurl_mime_init(3)\fP creates a multi-part body; you can then |
| 483 |
append new parts to a multi-part body using \fIcurl_mime_addpart(3)\fP. |
| 484 |
There are three possible data sources for a part: memory using |
| 485 |
\fIcurl_mime_data(3)\fP, file using \fIcurl_mime_filedata(3)\fP and |
| 486 |
user-defined data read callback using \fIcurl_mime_data_cb(3)\fP. |
| 487 |
\fIcurl_mime_name(3)\fP sets a part's (i.e.: form field) name, while |
| 488 |
\fIcurl_mime_filename(3)\fP fills in the remote file name. With |
| 489 |
\fIcurl_mime_type(3)\fP, you can tell the MIME type of a part, |
| 490 |
\fIcurl_mime_headers(3)\fP allows defining the part's headers. When a |
| 491 |
multi-part body is no longer needed, you can destroy it using |
| 492 |
\fIcurl_mime_free(3)\fP. |
| 493 |
|
| 494 |
The following example sets two simple text parts with plain textual contents, |
| 495 |
and then a file with binary contents and uploads the whole thing. |
| 496 |
|
| 497 |
.nf |
| 498 |
curl_mime *multipart = curl_mime_init(easyhandle); |
| 499 |
curl_mimepart *part = curl_mime_addpart(multipart); |
| 500 |
curl_mime_name(part, "name"); |
| 501 |
curl_mime_data(part, "daniel", CURL_ZERO_TERMINATED); |
| 502 |
part = curl_mime_addpart(multipart); |
| 503 |
curl_mime_name(part, "project"); |
| 504 |
curl_mime_data(part, "curl", CURL_ZERO_TERMINATED); |
| 505 |
part = curl_mime_addpart(multipart); |
| 506 |
curl_mime_name(part, "logotype-image"); |
| 507 |
curl_mime_filedata(part, "curl.png"); |
| 508 |
|
| 509 |
/* Set the form info */ |
| 510 |
curl_easy_setopt(easyhandle, CURLOPT_MIMEPOST, multipart); |
| 511 |
|
| 512 |
curl_easy_perform(easyhandle); /* post away! */ |
| 513 |
|
| 514 |
/* free the post data again */ |
| 515 |
curl_mime_free(multipart); |
| 516 |
.fi |
| 517 |
|
| 518 |
To post multiple files for a single form field, you must supply each file in |
| 519 |
a separate part, all with the same field name. Although function |
| 520 |
\fIcurl_mime_subparts(3)\fP implements nested multi-parts, this way of |
| 521 |
multiple files posting is deprecated by RFC 7578, chapter 4.3. |
| 522 |
|
| 523 |
To set the data source from an already opened FILE pointer, use: |
| 524 |
|
| 525 |
.nf |
| 526 |
curl_mime_data_cb(part, filesize, (curl_read_callback) fread, |
| 527 |
(curl_seek_callback) fseek, NULL, filepointer); |
| 528 |
.fi |
| 529 |
|
| 530 |
A deprecated \fIcurl_formadd(3)\fP function is still supported in libcurl. |
| 531 |
It should however not be used anymore for new designs and programs using it |
| 532 |
ought to be converted to the MIME API. It is however described here as an |
| 533 |
aid to conversion. |
| 534 |
|
| 535 |
Using \fIcurl_formadd\fP, you add parts to the form. When you're done adding |
| 536 |
parts, you post the whole form. |
| 537 |
|
| 538 |
The MIME API example above is expressed as follows using this function: |
| 539 |
|
| 540 |
.nf |
| 541 |
struct curl_httppost *post=NULL; |
| 542 |
struct curl_httppost *last=NULL; |
| 543 |
curl_formadd(&post, &last, |
| 544 |
CURLFORM_COPYNAME, "name", |
| 545 |
CURLFORM_COPYCONTENTS, "daniel", CURLFORM_END); |
| 546 |
curl_formadd(&post, &last, |
| 547 |
CURLFORM_COPYNAME, "project", |
| 548 |
CURLFORM_COPYCONTENTS, "curl", CURLFORM_END); |
| 549 |
curl_formadd(&post, &last, |
| 550 |
CURLFORM_COPYNAME, "logotype-image", |
| 551 |
CURLFORM_FILECONTENT, "curl.png", CURLFORM_END); |
| 552 |
|
| 553 |
/* Set the form info */ |
| 554 |
curl_easy_setopt(easyhandle, CURLOPT_HTTPPOST, post); |
| 555 |
|
| 556 |
curl_easy_perform(easyhandle); /* post away! */ |
| 557 |
|
| 558 |
/* free the post data again */ |
| 559 |
curl_formfree(post); |
| 560 |
.fi |
| 561 |
|
| 562 |
Multipart formposts are chains of parts using MIME-style separators and |
| 563 |
headers. It means that each one of these separate parts get a few headers set |
| 564 |
that describe the individual content-type, size etc. To enable your |
| 565 |
application to handicraft this formpost even more, libcurl allows you to |
| 566 |
supply your own set of custom headers to such an individual form part. You can |
| 567 |
of course supply headers to as many parts as you like, but this little example |
| 568 |
will show how you set headers to one specific part when you add that to the |
| 569 |
post handle: |
| 570 |
|
| 571 |
.nf |
| 572 |
struct curl_slist *headers=NULL; |
| 573 |
headers = curl_slist_append(headers, "Content-Type: text/xml"); |
| 574 |
|
| 575 |
curl_formadd(&post, &last, |
| 576 |
CURLFORM_COPYNAME, "logotype-image", |
| 577 |
CURLFORM_FILECONTENT, "curl.xml", |
| 578 |
CURLFORM_CONTENTHEADER, headers, |
| 579 |
CURLFORM_END); |
| 580 |
|
| 581 |
curl_easy_perform(easyhandle); /* post away! */ |
| 582 |
|
| 583 |
curl_formfree(post); /* free post */ |
| 584 |
curl_slist_free_all(headers); /* free custom header list */ |
| 585 |
.fi |
| 586 |
|
| 587 |
Since all options on an easyhandle are "sticky", they remain the same until |
| 588 |
changed even if you do call \fIcurl_easy_perform(3)\fP, you may need to tell |
| 589 |
curl to go back to a plain GET request if you intend to do one as your next |
| 590 |
request. You force an easyhandle to go back to GET by using the |
| 591 |
\fICURLOPT_HTTPGET(3)\fP option: |
| 592 |
|
| 593 |
curl_easy_setopt(easyhandle, CURLOPT_HTTPGET, 1L); |
| 594 |
|
| 595 |
Just setting \fICURLOPT_POSTFIELDS(3)\fP to "" or NULL will *not* stop libcurl |
| 596 |
from doing a POST. It will just make it POST without any data to send! |
| 597 |
|
| 598 |
.SH "Converting from deprecated form API to MIME API" |
| 599 |
Four rules have to be respected in building the multi-part: |
| 600 |
.br |
| 601 |
- The easy handle must be created before building the multi-part. |
| 602 |
.br |
| 603 |
- The multi-part is always created by a call to curl_mime_init(easyhandle). |
| 604 |
.br |
| 605 |
- Each part is created by a call to curl_mime_addpart(multipart). |
| 606 |
.br |
| 607 |
- When complete, the multi-part must be bound to the easy handle using |
| 608 |
\fICURLOPT_MIMEPOST(3)\fP instead of \fICURLOPT_HTTPPOST(3)\fP. |
| 609 |
|
| 610 |
Here are some example of \fIcurl_formadd\fP calls to MIME API sequences: |
| 611 |
|
| 612 |
.nf |
| 613 |
curl_formadd(&post, &last, |
| 614 |
CURLFORM_COPYNAME, "id", |
| 615 |
CURLFORM_COPYCONTENTS, "daniel", CURLFORM_END); |
| 616 |
CURLFORM_CONTENTHEADER, headers, |
| 617 |
CURLFORM_END); |
| 618 |
.fi |
| 619 |
becomes: |
| 620 |
.nf |
| 621 |
part = curl_mime_addpart(multipart); |
| 622 |
curl_mime_name(part, "id"); |
| 623 |
curl_mime_data(part, "daniel", CURL_ZERO_TERMINATED); |
| 624 |
curl_mime_headers(part, headers, FALSE); |
| 625 |
.fi |
| 626 |
|
| 627 |
Setting the last \fIcurl_mime_headers\fP argument to TRUE would have caused |
| 628 |
the headers to be automatically released upon destroyed the multi-part, thus |
| 629 |
saving a clean-up call to \fIcurl_slist_free_all(3)\fP. |
| 630 |
|
| 631 |
.nf |
| 632 |
curl_formadd(&post, &last, |
| 633 |
CURLFORM_PTRNAME, "logotype-image", |
| 634 |
CURLFORM_FILECONTENT, "-", |
| 635 |
CURLFORM_END); |
| 636 |
.fi |
| 637 |
becomes: |
| 638 |
.nf |
| 639 |
part = curl_mime_addpart(multipart); |
| 640 |
curl_mime_name(part, "logotype-image"); |
| 641 |
curl_mime_data_cb(part, (curl_off_t) -1, fread, fseek, NULL, stdin); |
| 642 |
.fi |
| 643 |
|
| 644 |
\fIcurl_mime_name\fP always copies the field name. The special file name "-" |
| 645 |
is not supported by \fIcurl_mime_file\fP: to read an open file, use |
| 646 |
a callback source using fread(). The transfer will be chunked since the data |
| 647 |
size is unknown. |
| 648 |
|
| 649 |
.nf |
| 650 |
curl_formadd(&post, &last, |
| 651 |
CURLFORM_COPYNAME, "datafile[]", |
| 652 |
CURLFORM_FILE, "file1", |
| 653 |
CURLFORM_FILE, "file2", |
| 654 |
CURLFORM_END); |
| 655 |
.fi |
| 656 |
becomes: |
| 657 |
.nf |
| 658 |
part = curl_mime_addpart(multipart); |
| 659 |
curl_mime_name(part, "datafile[]"); |
| 660 |
curl_mime_filedata(part, "file1"); |
| 661 |
part = curl_mime_addpart(multipart); |
| 662 |
curl_mime_name(part, "datafile[]"); |
| 663 |
curl_mime_filedata(part, "file2"); |
| 664 |
.fi |
| 665 |
|
| 666 |
The deprecated multipart/mixed implementation of multiple files field is |
| 667 |
translated to two distinct parts with the same name. |
| 668 |
|
| 669 |
.nf |
| 670 |
curl_easy_setopt(easyhandle, CURLOPT_READFUNCTION, myreadfunc); |
| 671 |
curl_formadd(&post, &last, |
| 672 |
CURLFORM_COPYNAME, "stream", |
| 673 |
CURLFORM_STREAM, arg, |
| 674 |
CURLFORM_CONTENTLEN, (curl_off_t) datasize, |
| 675 |
CURLFORM_FILENAME, "archive.zip", |
| 676 |
CURLFORM_CONTENTTYPE, "application/zip", |
| 677 |
CURLFORM_END); |
| 678 |
.fi |
| 679 |
becomes: |
| 680 |
.nf |
| 681 |
part = curl_mime_addpart(multipart); |
| 682 |
curl_mime_name(part, "stream"); |
| 683 |
curl_mime_data_cb(part, (curl_off_t) datasize, |
| 684 |
myreadfunc, NULL, NULL, arg); |
| 685 |
curl_mime_filename(part, "archive.zip"); |
| 686 |
curl_mime_type(part, "application/zip"); |
| 687 |
.fi |
| 688 |
|
| 689 |
\fICURLOPT_READFUNCTION\fP callback is not used: it is replace by directly |
| 690 |
setting the part source data from the callback read function. |
| 691 |
|
| 692 |
.nf |
| 693 |
curl_formadd(&post, &last, |
| 694 |
CURLFORM_COPYNAME, "memfile", |
| 695 |
CURLFORM_BUFFER, "memfile.bin", |
| 696 |
CURLFORM_BUFFERPTR, databuffer, |
| 697 |
CURLFORM_BUFFERLENGTH, (long) sizeof databuffer, |
| 698 |
CURLFORM_END); |
| 699 |
.fi |
| 700 |
becomes: |
| 701 |
.nf |
| 702 |
part = curl_mime_addpart(multipart); |
| 703 |
curl_mime_name(part, "memfile"); |
| 704 |
curl_mime_data(part, databuffer, (curl_off_t) sizeof databuffer); |
| 705 |
curl_mime_filename(part, "memfile.bin"); |
| 706 |
.fi |
| 707 |
|
| 708 |
\fIcurl_mime_data\fP always copies the initial data: data buffer is thus |
| 709 |
free for immediate reuse. |
| 710 |
|
| 711 |
.nf |
| 712 |
curl_formadd(&post, &last, |
| 713 |
CURLFORM_COPYNAME, "message", |
| 714 |
CURLFORM_FILECONTENT, "msg.txt", |
| 715 |
CURLFORM_END); |
| 716 |
.fi |
| 717 |
becomes: |
| 718 |
.nf |
| 719 |
part = curl_mime_addpart(multipart); |
| 720 |
curl_mime_name(part, "message"); |
| 721 |
curl_mime_filedata(part, "msg.txt"); |
| 722 |
curl_mime_filename(part, NULL); |
| 723 |
.fi |
| 724 |
|
| 725 |
Use of \fIcurl_mime_filedata\fP sets the remote file name as a side effect: it |
| 726 |
is therefore necessary to clear it for \fICURLFORM_FILECONTENT\fP emulation. |
| 727 |
|
| 728 |
.SH "Showing Progress" |
| 729 |
|
| 730 |
For historical and traditional reasons, libcurl has a built-in progress meter |
| 731 |
that can be switched on and then makes it present a progress meter in your |
| 732 |
terminal. |
| 733 |
|
| 734 |
Switch on the progress meter by, oddly enough, setting |
| 735 |
\fICURLOPT_NOPROGRESS(3)\fP to zero. This option is set to 1 by default. |
| 736 |
|
| 737 |
For most applications however, the built-in progress meter is useless and |
| 738 |
what instead is interesting is the ability to specify a progress |
| 739 |
callback. The function pointer you pass to libcurl will then be called on |
| 740 |
irregular intervals with information about the current transfer. |
| 741 |
|
| 742 |
Set the progress callback by using \fICURLOPT_PROGRESSFUNCTION(3)\fP. And pass |
| 743 |
a pointer to a function that matches this prototype: |
| 744 |
|
| 745 |
.nf |
| 746 |
int progress_callback(void *clientp, |
| 747 |
double dltotal, |
| 748 |
double dlnow, |
| 749 |
double ultotal, |
| 750 |
double ulnow); |
| 751 |
.fi |
| 752 |
|
| 753 |
If any of the input arguments is unknown, a 0 will be passed. The first |
| 754 |
argument, the 'clientp' is the pointer you pass to libcurl with |
| 755 |
\fICURLOPT_PROGRESSDATA(3)\fP. libcurl won't touch it. |
| 756 |
|
| 757 |
.SH "libcurl with C++" |
| 758 |
|
| 759 |
There's basically only one thing to keep in mind when using C++ instead of C |
| 760 |
when interfacing libcurl: |
| 761 |
|
| 762 |
The callbacks CANNOT be non-static class member functions |
| 763 |
|
| 764 |
Example C++ code: |
| 765 |
|
| 766 |
.nf |
| 767 |
class AClass { |
| 768 |
static size_t write_data(void *ptr, size_t size, size_t nmemb, |
| 769 |
void *ourpointer) |
| 770 |
{ |
| 771 |
/* do what you want with the data */ |
| 772 |
} |
| 773 |
} |
| 774 |
.fi |
| 775 |
|
| 776 |
.SH "Proxies" |
| 777 |
|
| 778 |
What "proxy" means according to Merriam-Webster: "a person authorized to act |
| 779 |
for another" but also "the agency, function, or office of a deputy who acts as |
| 780 |
a substitute for another". |
| 781 |
|
| 782 |
Proxies are exceedingly common these days. Companies often only offer Internet |
| 783 |
access to employees through their proxies. Network clients or user-agents ask |
| 784 |
the proxy for documents, the proxy does the actual request and then it returns |
| 785 |
them. |
| 786 |
|
| 787 |
libcurl supports SOCKS and HTTP proxies. When a given URL is wanted, libcurl |
| 788 |
will ask the proxy for it instead of trying to connect to the actual host |
| 789 |
identified in the URL. |
| 790 |
|
| 791 |
If you're using a SOCKS proxy, you may find that libcurl doesn't quite support |
| 792 |
all operations through it. |
| 793 |
|
| 794 |
For HTTP proxies: the fact that the proxy is an HTTP proxy puts certain |
| 795 |
restrictions on what can actually happen. A requested URL that might not be a |
| 796 |
HTTP URL will be still be passed to the HTTP proxy to deliver back to |
| 797 |
libcurl. This happens transparently, and an application may not need to |
| 798 |
know. I say "may", because at times it is very important to understand that |
| 799 |
all operations over an HTTP proxy use the HTTP protocol. For example, you |
| 800 |
can't invoke your own custom FTP commands or even proper FTP directory |
| 801 |
listings. |
| 802 |
|
| 803 |
.IP "Proxy Options" |
| 804 |
|
| 805 |
To tell libcurl to use a proxy at a given port number: |
| 806 |
|
| 807 |
curl_easy_setopt(easyhandle, CURLOPT_PROXY, "proxy-host.com:8080"); |
| 808 |
|
| 809 |
Some proxies require user authentication before allowing a request, and you |
| 810 |
pass that information similar to this: |
| 811 |
|
| 812 |
curl_easy_setopt(easyhandle, CURLOPT_PROXYUSERPWD, "user:password"); |
| 813 |
|
| 814 |
If you want to, you can specify the host name only in the |
| 815 |
\fICURLOPT_PROXY(3)\fP option, and set the port number separately with |
| 816 |
\fICURLOPT_PROXYPORT(3)\fP. |
| 817 |
|
| 818 |
Tell libcurl what kind of proxy it is with \fICURLOPT_PROXYTYPE(3)\fP (if not, |
| 819 |
it will default to assume an HTTP proxy): |
| 820 |
|
| 821 |
curl_easy_setopt(easyhandle, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS4); |
| 822 |
|
| 823 |
.IP "Environment Variables" |
| 824 |
|
| 825 |
libcurl automatically checks and uses a set of environment variables to know |
| 826 |
what proxies to use for certain protocols. The names of the variables are |
| 827 |
following an ancient de facto standard and are built up as "[protocol]_proxy" |
| 828 |
(note the lower casing). Which makes the variable \&'http_proxy' checked for a |
| 829 |
name of a proxy to use when the input URL is HTTP. Following the same rule, |
| 830 |
the variable named 'ftp_proxy' is checked for FTP URLs. Again, the proxies are |
| 831 |
always HTTP proxies, the different names of the variables simply allows |
| 832 |
different HTTP proxies to be used. |
| 833 |
|
| 834 |
The proxy environment variable contents should be in the format |
| 835 |
\&"[protocol://][user:password@]machine[:port]". Where the protocol:// part is |
| 836 |
simply ignored if present (so http://proxy and bluerk://proxy will do the |
| 837 |
same) and the optional port number specifies on which port the proxy operates |
| 838 |
on the host. If not specified, the internal default port number will be used |
| 839 |
and that is most likely *not* the one you would like it to be. |
| 840 |
|
| 841 |
There are two special environment variables. 'all_proxy' is what sets proxy |
| 842 |
for any URL in case the protocol specific variable wasn't set, and |
| 843 |
\&'no_proxy' defines a list of hosts that should not use a proxy even though a |
| 844 |
variable may say so. If 'no_proxy' is a plain asterisk ("*") it matches all |
| 845 |
hosts. |
| 846 |
|
| 847 |
To explicitly disable libcurl's checking for and using the proxy environment |
| 848 |
variables, set the proxy name to "" - an empty string - with |
| 849 |
\fICURLOPT_PROXY(3)\fP. |
| 850 |
.IP "SSL and Proxies" |
| 851 |
|
| 852 |
SSL is for secure point-to-point connections. This involves strong encryption |
| 853 |
and similar things, which effectively makes it impossible for a proxy to |
| 854 |
operate as a "man in between" which the proxy's task is, as previously |
| 855 |
discussed. Instead, the only way to have SSL work over an HTTP proxy is to ask |
| 856 |
the proxy to tunnel trough everything without being able to check or fiddle |
| 857 |
with the traffic. |
| 858 |
|
| 859 |
Opening an SSL connection over an HTTP proxy is therefore a matter of asking the |
| 860 |
proxy for a straight connection to the target host on a specified port. This |
| 861 |
is made with the HTTP request CONNECT. ("please mr proxy, connect me to that |
| 862 |
remote host"). |
| 863 |
|
| 864 |
Because of the nature of this operation, where the proxy has no idea what kind |
| 865 |
of data that is passed in and out through this tunnel, this breaks some of the |
| 866 |
very few advantages that come from using a proxy, such as caching. Many |
| 867 |
organizations prevent this kind of tunneling to other destination port numbers |
| 868 |
than 443 (which is the default HTTPS port number). |
| 869 |
|
| 870 |
.IP "Tunneling Through Proxy" |
| 871 |
As explained above, tunneling is required for SSL to work and often even |
| 872 |
restricted to the operation intended for SSL; HTTPS. |
| 873 |
|
| 874 |
This is however not the only time proxy-tunneling might offer benefits to |
| 875 |
you or your application. |
| 876 |
|
| 877 |
As tunneling opens a direct connection from your application to the remote |
| 878 |
machine, it suddenly also re-introduces the ability to do non-HTTP |
| 879 |
operations over an HTTP proxy. You can in fact use things such as FTP |
| 880 |
upload or FTP custom commands this way. |
| 881 |
|
| 882 |
Again, this is often prevented by the administrators of proxies and is |
| 883 |
rarely allowed. |
| 884 |
|
| 885 |
Tell libcurl to use proxy tunneling like this: |
| 886 |
|
| 887 |
curl_easy_setopt(easyhandle, CURLOPT_HTTPPROXYTUNNEL, 1L); |
| 888 |
|
| 889 |
In fact, there might even be times when you want to do plain HTTP |
| 890 |
operations using a tunnel like this, as it then enables you to operate on |
| 891 |
the remote server instead of asking the proxy to do so. libcurl will not |
| 892 |
stand in the way for such innovative actions either! |
| 893 |
|
| 894 |
.IP "Proxy Auto-Config" |
| 895 |
|
| 896 |
Netscape first came up with this. It is basically a web page (usually using a |
| 897 |
\&.pac extension) with a Javascript that when executed by the browser with the |
| 898 |
requested URL as input, returns information to the browser on how to connect |
| 899 |
to the URL. The returned information might be "DIRECT" (which means no proxy |
| 900 |
should be used), "PROXY host:port" (to tell the browser where the proxy for |
| 901 |
this particular URL is) or "SOCKS host:port" (to direct the browser to a SOCKS |
| 902 |
proxy). |
| 903 |
|
| 904 |
libcurl has no means to interpret or evaluate Javascript and thus it doesn't |
| 905 |
support this. If you get yourself in a position where you face this nasty |
| 906 |
invention, the following advice have been mentioned and used in the past: |
| 907 |
|
| 908 |
- Depending on the Javascript complexity, write up a script that translates it |
| 909 |
to another language and execute that. |
| 910 |
|
| 911 |
- Read the Javascript code and rewrite the same logic in another language. |
| 912 |
|
| 913 |
- Implement a Javascript interpreter; people have successfully used the |
| 914 |
Mozilla Javascript engine in the past. |
| 915 |
|
| 916 |
- Ask your admins to stop this, for a static proxy setup or similar. |
| 917 |
|
| 918 |
.SH "Persistence Is The Way to Happiness" |
| 919 |
|
| 920 |
Re-cycling the same easy handle several times when doing multiple requests is |
| 921 |
the way to go. |
| 922 |
|
| 923 |
After each single \fIcurl_easy_perform(3)\fP operation, libcurl will keep the |
| 924 |
connection alive and open. A subsequent request using the same easy handle to |
| 925 |
the same host might just be able to use the already open connection! This |
| 926 |
reduces network impact a lot. |
| 927 |
|
| 928 |
Even if the connection is dropped, all connections involving SSL to the same |
| 929 |
host again, will benefit from libcurl's session ID cache that drastically |
| 930 |
reduces re-connection time. |
| 931 |
|
| 932 |
FTP connections that are kept alive save a lot of time, as the command- |
| 933 |
response round-trips are skipped, and also you don't risk getting blocked |
| 934 |
without permission to login again like on many FTP servers only allowing N |
| 935 |
persons to be logged in at the same time. |
| 936 |
|
| 937 |
libcurl caches DNS name resolving results, to make lookups of a previously |
| 938 |
looked up name a lot faster. |
| 939 |
|
| 940 |
Other interesting details that improve performance for subsequent requests |
| 941 |
may also be added in the future. |
| 942 |
|
| 943 |
Each easy handle will attempt to keep the last few connections alive for a |
| 944 |
while in case they are to be used again. You can set the size of this "cache" |
| 945 |
with the \fICURLOPT_MAXCONNECTS(3)\fP option. Default is 5. There is very |
| 946 |
seldom any point in changing this value, and if you think of changing this it |
| 947 |
is often just a matter of thinking again. |
| 948 |
|
| 949 |
To force your upcoming request to not use an already existing connection (it |
| 950 |
will even close one first if there happens to be one alive to the same host |
| 951 |
you're about to operate on), you can do that by setting |
| 952 |
\fICURLOPT_FRESH_CONNECT(3)\fP to 1. In a similar spirit, you can also forbid |
| 953 |
the upcoming request to be "lying" around and possibly get re-used after the |
| 954 |
request by setting \fICURLOPT_FORBID_REUSE(3)\fP to 1. |
| 955 |
|
| 956 |
.SH "HTTP Headers Used by libcurl" |
| 957 |
When you use libcurl to do HTTP requests, it'll pass along a series of headers |
| 958 |
automatically. It might be good for you to know and understand these. You |
| 959 |
can replace or remove them by using the \fICURLOPT_HTTPHEADER(3)\fP option. |
| 960 |
|
| 961 |
.IP "Host" |
| 962 |
This header is required by HTTP 1.1 and even many 1.0 servers and should be |
| 963 |
the name of the server we want to talk to. This includes the port number if |
| 964 |
anything but default. |
| 965 |
|
| 966 |
.IP "Accept" |
| 967 |
\&"*/*". |
| 968 |
|
| 969 |
.IP "Expect" |
| 970 |
When doing POST requests, libcurl sets this header to \&"100-continue" to ask |
| 971 |
the server for an "OK" message before it proceeds with sending the data part |
| 972 |
of the post. If the POSTed data amount is deemed "small", libcurl will not use |
| 973 |
this header. |
| 974 |
|
| 975 |
.SH "Customizing Operations" |
| 976 |
There is an ongoing development today where more and more protocols are built |
| 977 |
upon HTTP for transport. This has obvious benefits as HTTP is a tested and |
| 978 |
reliable protocol that is widely deployed and has excellent proxy-support. |
| 979 |
|
| 980 |
When you use one of these protocols, and even when doing other kinds of |
| 981 |
programming you may need to change the traditional HTTP (or FTP or...) |
| 982 |
manners. You may need to change words, headers or various data. |
| 983 |
|
| 984 |
libcurl is your friend here too. |
| 985 |
|
| 986 |
.IP CUSTOMREQUEST |
| 987 |
If just changing the actual HTTP request keyword is what you want, like when |
| 988 |
GET, HEAD or POST is not good enough for you, \fICURLOPT_CUSTOMREQUEST(3)\fP |
| 989 |
is there for you. It is very simple to use: |
| 990 |
|
| 991 |
curl_easy_setopt(easyhandle, CURLOPT_CUSTOMREQUEST, "MYOWNREQUEST"); |
| 992 |
|
| 993 |
When using the custom request, you change the request keyword of the actual |
| 994 |
request you are performing. Thus, by default you make a GET request but you can |
| 995 |
also make a POST operation (as described before) and then replace the POST |
| 996 |
keyword if you want to. You're the boss. |
| 997 |
|
| 998 |
.IP "Modify Headers" |
| 999 |
HTTP-like protocols pass a series of headers to the server when doing the |
| 1000 |
request, and you're free to pass any amount of extra headers that you |
| 1001 |
think fit. Adding headers is this easy: |
| 1002 |
|
| 1003 |
.nf |
| 1004 |
struct curl_slist *headers=NULL; /* init to NULL is important */ |
| 1005 |
|
| 1006 |
headers = curl_slist_append(headers, "Hey-server-hey: how are you?"); |
| 1007 |
headers = curl_slist_append(headers, "X-silly-content: yes"); |
| 1008 |
|
| 1009 |
/* pass our list of custom made headers */ |
| 1010 |
curl_easy_setopt(easyhandle, CURLOPT_HTTPHEADER, headers); |
| 1011 |
|
| 1012 |
curl_easy_perform(easyhandle); /* transfer http */ |
| 1013 |
|
| 1014 |
curl_slist_free_all(headers); /* free the header list */ |
| 1015 |
.fi |
| 1016 |
|
| 1017 |
\&... and if you think some of the internally generated headers, such as |
| 1018 |
Accept: or Host: don't contain the data you want them to contain, you can |
| 1019 |
replace them by simply setting them too: |
| 1020 |
|
| 1021 |
.nf |
| 1022 |
headers = curl_slist_append(headers, "Accept: Agent-007"); |
| 1023 |
headers = curl_slist_append(headers, "Host: munged.host.line"); |
| 1024 |
.fi |
| 1025 |
|
| 1026 |
.IP "Delete Headers" |
| 1027 |
If you replace an existing header with one with no contents, you will prevent |
| 1028 |
the header from being sent. For instance, if you want to completely prevent the |
| 1029 |
\&"Accept:" header from being sent, you can disable it with code similar to this: |
| 1030 |
|
| 1031 |
headers = curl_slist_append(headers, "Accept:"); |
| 1032 |
|
| 1033 |
Both replacing and canceling internal headers should be done with careful |
| 1034 |
consideration and you should be aware that you may violate the HTTP protocol |
| 1035 |
when doing so. |
| 1036 |
|
| 1037 |
.IP "Enforcing chunked transfer-encoding" |
| 1038 |
|
| 1039 |
By making sure a request uses the custom header "Transfer-Encoding: chunked" |
| 1040 |
when doing a non-GET HTTP operation, libcurl will switch over to "chunked" |
| 1041 |
upload, even though the size of the data to upload might be known. By default, |
| 1042 |
libcurl usually switches over to chunked upload automatically if the upload |
| 1043 |
data size is unknown. |
| 1044 |
|
| 1045 |
.IP "HTTP Version" |
| 1046 |
|
| 1047 |
All HTTP requests includes the version number to tell the server which version |
| 1048 |
we support. libcurl speaks HTTP 1.1 by default. Some very old servers don't |
| 1049 |
like getting 1.1-requests and when dealing with stubborn old things like that, |
| 1050 |
you can tell libcurl to use 1.0 instead by doing something like this: |
| 1051 |
|
| 1052 |
curl_easy_setopt(easyhandle, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_0); |
| 1053 |
|
| 1054 |
.IP "FTP Custom Commands" |
| 1055 |
|
| 1056 |
Not all protocols are HTTP-like, and thus the above may not help you when |
| 1057 |
you want to make, for example, your FTP transfers to behave differently. |
| 1058 |
|
| 1059 |
Sending custom commands to an FTP server means that you need to send the |
| 1060 |
commands exactly as the FTP server expects them (RFC959 is a good guide |
| 1061 |
here), and you can only use commands that work on the control-connection |
| 1062 |
alone. All kinds of commands that require data interchange and thus need |
| 1063 |
a data-connection must be left to libcurl's own judgement. Also be aware |
| 1064 |
that libcurl will do its very best to change directory to the target |
| 1065 |
directory before doing any transfer, so if you change directory (with CWD |
| 1066 |
or similar) you might confuse libcurl and then it might not attempt to |
| 1067 |
transfer the file in the correct remote directory. |
| 1068 |
|
| 1069 |
A little example that deletes a given file before an operation: |
| 1070 |
|
| 1071 |
.nf |
| 1072 |
headers = curl_slist_append(headers, "DELE file-to-remove"); |
| 1073 |
|
| 1074 |
/* pass the list of custom commands to the handle */ |
| 1075 |
curl_easy_setopt(easyhandle, CURLOPT_QUOTE, headers); |
| 1076 |
|
| 1077 |
curl_easy_perform(easyhandle); /* transfer ftp data! */ |
| 1078 |
|
| 1079 |
curl_slist_free_all(headers); /* free the header list */ |
| 1080 |
.fi |
| 1081 |
|
| 1082 |
If you would instead want this operation (or chain of operations) to happen |
| 1083 |
_after_ the data transfer took place the option to \fIcurl_easy_setopt(3)\fP |
| 1084 |
would instead be called \fICURLOPT_POSTQUOTE(3)\fP and used the exact same |
| 1085 |
way. |
| 1086 |
|
| 1087 |
The custom FTP command will be issued to the server in the same order they are |
| 1088 |
added to the list, and if a command gets an error code returned back from the |
| 1089 |
server, no more commands will be issued and libcurl will bail out with an |
| 1090 |
error code (CURLE_QUOTE_ERROR). Note that if you use \fICURLOPT_QUOTE(3)\fP to |
| 1091 |
send commands before a transfer, no transfer will actually take place when a |
| 1092 |
quote command has failed. |
| 1093 |
|
| 1094 |
If you set the \fICURLOPT_HEADER(3)\fP to 1, you will tell libcurl to get |
| 1095 |
information about the target file and output "headers" about it. The headers |
| 1096 |
will be in "HTTP-style", looking like they do in HTTP. |
| 1097 |
|
| 1098 |
The option to enable headers or to run custom FTP commands may be useful to |
| 1099 |
combine with \fICURLOPT_NOBODY(3)\fP. If this option is set, no actual file |
| 1100 |
content transfer will be performed. |
| 1101 |
|
| 1102 |
.IP "FTP Custom CUSTOMREQUEST" |
| 1103 |
If you do want to list the contents of an FTP directory using your own defined |
| 1104 |
FTP command, \fICURLOPT_CUSTOMREQUEST(3)\fP will do just that. "NLST" is the |
| 1105 |
default one for listing directories but you're free to pass in your idea of a |
| 1106 |
good alternative. |
| 1107 |
|
| 1108 |
.SH "Cookies Without Chocolate Chips" |
| 1109 |
In the HTTP sense, a cookie is a name with an associated value. A server sends |
| 1110 |
the name and value to the client, and expects it to get sent back on every |
| 1111 |
subsequent request to the server that matches the particular conditions |
| 1112 |
set. The conditions include that the domain name and path match and that the |
| 1113 |
cookie hasn't become too old. |
| 1114 |
|
| 1115 |
In real-world cases, servers send new cookies to replace existing ones to |
| 1116 |
update them. Server use cookies to "track" users and to keep "sessions". |
| 1117 |
|
| 1118 |
Cookies are sent from server to clients with the header Set-Cookie: and |
| 1119 |
they're sent from clients to servers with the Cookie: header. |
| 1120 |
|
| 1121 |
To just send whatever cookie you want to a server, you can use |
| 1122 |
\fICURLOPT_COOKIE(3)\fP to set a cookie string like this: |
| 1123 |
|
| 1124 |
curl_easy_setopt(easyhandle, CURLOPT_COOKIE, "name1=var1; name2=var2;"); |
| 1125 |
|
| 1126 |
In many cases, that is not enough. You might want to dynamically save |
| 1127 |
whatever cookies the remote server passes to you, and make sure those cookies |
| 1128 |
are then used accordingly on later requests. |
| 1129 |
|
| 1130 |
One way to do this, is to save all headers you receive in a plain file and |
| 1131 |
when you make a request, you tell libcurl to read the previous headers to |
| 1132 |
figure out which cookies to use. Set the header file to read cookies from with |
| 1133 |
\fICURLOPT_COOKIEFILE(3)\fP. |
| 1134 |
|
| 1135 |
The \fICURLOPT_COOKIEFILE(3)\fP option also automatically enables the cookie |
| 1136 |
parser in libcurl. Until the cookie parser is enabled, libcurl will not parse |
| 1137 |
or understand incoming cookies and they will just be ignored. However, when |
| 1138 |
the parser is enabled the cookies will be understood and the cookies will be |
| 1139 |
kept in memory and used properly in subsequent requests when the same handle |
| 1140 |
is used. Many times this is enough, and you may not have to save the cookies |
| 1141 |
to disk at all. Note that the file you specify to \fICURLOPT_COOKIEFILE(3)\fP |
| 1142 |
doesn't have to exist to enable the parser, so a common way to just enable the |
| 1143 |
parser and not read any cookies is to use the name of a file you know doesn't |
| 1144 |
exist. |
| 1145 |
|
| 1146 |
If you would rather use existing cookies that you've previously received with |
| 1147 |
your Netscape or Mozilla browsers, you can make libcurl use that cookie file |
| 1148 |
as input. The \fICURLOPT_COOKIEFILE(3)\fP is used for that too, as libcurl |
| 1149 |
will automatically find out what kind of file it is and act accordingly. |
| 1150 |
|
| 1151 |
Perhaps the most advanced cookie operation libcurl offers, is saving the |
| 1152 |
entire internal cookie state back into a Netscape/Mozilla formatted cookie |
| 1153 |
file. We call that the cookie-jar. When you set a file name with |
| 1154 |
\fICURLOPT_COOKIEJAR(3)\fP, that file name will be created and all received |
| 1155 |
cookies will be stored in it when \fIcurl_easy_cleanup(3)\fP is called. This |
| 1156 |
enables cookies to get passed on properly between multiple handles without any |
| 1157 |
information getting lost. |
| 1158 |
|
| 1159 |
.SH "FTP Peculiarities We Need" |
| 1160 |
|
| 1161 |
FTP transfers use a second TCP/IP connection for the data transfer. This is |
| 1162 |
usually a fact you can forget and ignore but at times this fact will come |
| 1163 |
back to haunt you. libcurl offers several different ways to customize how the |
| 1164 |
second connection is being made. |
| 1165 |
|
| 1166 |
libcurl can either connect to the server a second time or tell the server to |
| 1167 |
connect back to it. The first option is the default and it is also what works |
| 1168 |
best for all the people behind firewalls, NATs or IP-masquerading setups. |
| 1169 |
libcurl then tells the server to open up a new port and wait for a second |
| 1170 |
connection. This is by default attempted with EPSV first, and if that doesn't |
| 1171 |
work it tries PASV instead. (EPSV is an extension to the original FTP spec |
| 1172 |
and does not exist nor work on all FTP servers.) |
| 1173 |
|
| 1174 |
You can prevent libcurl from first trying the EPSV command by setting |
| 1175 |
\fICURLOPT_FTP_USE_EPSV(3)\fP to zero. |
| 1176 |
|
| 1177 |
In some cases, you will prefer to have the server connect back to you for the |
| 1178 |
second connection. This might be when the server is perhaps behind a firewall |
| 1179 |
or something and only allows connections on a single port. libcurl then |
| 1180 |
informs the remote server which IP address and port number to connect to. |
| 1181 |
This is made with the \fICURLOPT_FTPPORT(3)\fP option. If you set it to "-", |
| 1182 |
libcurl will use your system's "default IP address". If you want to use a |
| 1183 |
particular IP, you can set the full IP address, a host name to resolve to an |
| 1184 |
IP address or even a local network interface name that libcurl will get the IP |
| 1185 |
address from. |
| 1186 |
|
| 1187 |
When doing the "PORT" approach, libcurl will attempt to use the EPRT and the |
| 1188 |
LPRT before trying PORT, as they work with more protocols. You can disable |
| 1189 |
this behavior by setting \fICURLOPT_FTP_USE_EPRT(3)\fP to zero. |
| 1190 |
|
| 1191 |
.SH "MIME API revisited for SMTP and IMAP" |
| 1192 |
In addition to support HTTP multi-part form fields, the MIME API can be used |
| 1193 |
to build structured e-mail messages and send them via SMTP or append such |
| 1194 |
messages to IMAP directories. |
| 1195 |
|
| 1196 |
A structured e-mail message may contain several parts: some are displayed |
| 1197 |
inline by the MUA, some are attachments. Parts can also be structured as |
| 1198 |
multi-part, for example to include another e-mail message or to offer several |
| 1199 |
text formats alternatives. This can be nested to any level. |
| 1200 |
|
| 1201 |
To build such a message, you prepare the nth-level multi-part and then include |
| 1202 |
it as a source to the parent multi-part using function |
| 1203 |
\fIcurl_mime_subparts(3)\fP. Once it has been |
| 1204 |
bound to its parent multi-part, a nth-level multi-part belongs to it and |
| 1205 |
should not be freed explicitly. |
| 1206 |
|
| 1207 |
E-mail messages data is not supposed to be non-ascii and line length is |
| 1208 |
limited: fortunately, some transfer encodings are defined by the standards |
| 1209 |
to support the transmission of such incompatible data. Function |
| 1210 |
\fIcurl_mime_encoder(3)\fP tells a part that its source data must be encoded |
| 1211 |
before being sent. It also generates the corresponding header for that part. |
| 1212 |
If the part data you want to send is already encoded in such a scheme, |
| 1213 |
do not use this function (this would over-encode it), but explicitly set the |
| 1214 |
corresponding part header. |
| 1215 |
|
| 1216 |
Upon sending such a message, libcurl prepends it with the header list |
| 1217 |
set with \fICURLOPT_HTTPHEADER(3)\fP, as 0th-level mime part headers. |
| 1218 |
|
| 1219 |
Here is an example building an e-mail message with an inline plain/html text |
| 1220 |
alternative and a file attachment encoded in base64: |
| 1221 |
|
| 1222 |
.nf |
| 1223 |
curl_mime *message = curl_mime_init(easyhandle); |
| 1224 |
|
| 1225 |
/* The inline part is an alternative proposing the html and the text |
| 1226 |
versions of the e-mail. */ |
| 1227 |
curl_mime *alt = curl_mime_init(easyhandle); |
| 1228 |
|
| 1229 |
/* HTML message. */ |
| 1230 |
curl_mimepart *part = curl_mime_addpart(alt); |
| 1231 |
curl_mime_data(part, "<html><body><p>This is HTML</p></body></html>", |
| 1232 |
CURL_ZERO_TERMINATED); |
| 1233 |
curl_mime_type(part, "text/html"); |
| 1234 |
|
| 1235 |
/* Text message. */ |
| 1236 |
part = curl_mime_addpart(alt); |
| 1237 |
curl_mime_data(part, "This is plain text message", |
| 1238 |
CURL_ZERO_TERMINATED); |
| 1239 |
|
| 1240 |
/* Create the inline part. */ |
| 1241 |
part = curl_mime_addpart(message); |
| 1242 |
curl_mime_subparts(part, alt); |
| 1243 |
curl_mime_type(part, "multipart/alternative"); |
| 1244 |
struct curl_slist *headers = curl_slist_append(NULL, |
| 1245 |
"Content-Disposition: inline"); |
| 1246 |
curl_mime_headers(part, headers, TRUE); |
| 1247 |
|
| 1248 |
/* Add the attachment. */ |
| 1249 |
part = curl_mime_addpart(message); |
| 1250 |
curl_mime_filedata(part, "manual.pdf"); |
| 1251 |
curl_mime_encoder(part, "base64"); |
| 1252 |
|
| 1253 |
/* Build the mail headers. */ |
| 1254 |
headers = curl_slist_append(NULL, "From: [email protected]"); |
| 1255 |
headers = curl_slist_append(headers, "To: [email protected]"); |
| 1256 |
|
| 1257 |
/* Set these into the easy handle. */ |
| 1258 |
curl_easy_setopt(easyhandle, CURLOPT_HTTPHEADER, headers); |
| 1259 |
curl_easy_setopt(easyhandle, CURLOPT_MIMEPOST, mime); |
| 1260 |
.fi |
| 1261 |
|
| 1262 |
It should be noted that appending a message to an IMAP directory requires |
| 1263 |
the message size to be known prior upload. It is therefore not possible to |
| 1264 |
include parts with unknown data size in this context. |
| 1265 |
|
| 1266 |
.SH "Headers Equal Fun" |
| 1267 |
|
| 1268 |
Some protocols provide "headers", meta-data separated from the normal |
| 1269 |
data. These headers are by default not included in the normal data stream, but |
| 1270 |
you can make them appear in the data stream by setting \fICURLOPT_HEADER(3)\fP |
| 1271 |
to 1. |
| 1272 |
|
| 1273 |
What might be even more useful, is libcurl's ability to separate the headers |
| 1274 |
from the data and thus make the callbacks differ. You can for example set a |
| 1275 |
different pointer to pass to the ordinary write callback by setting |
| 1276 |
\fICURLOPT_HEADERDATA(3)\fP. |
| 1277 |
|
| 1278 |
Or, you can set an entirely separate function to receive the headers, by using |
| 1279 |
\fICURLOPT_HEADERFUNCTION(3)\fP. |
| 1280 |
|
| 1281 |
The headers are passed to the callback function one by one, and you can |
| 1282 |
depend on that fact. It makes it easier for you to add custom header parsers |
| 1283 |
etc. |
| 1284 |
|
| 1285 |
\&"Headers" for FTP transfers equal all the FTP server responses. They aren't |
| 1286 |
actually true headers, but in this case we pretend they are! ;-) |
| 1287 |
|
| 1288 |
.SH "Post Transfer Information" |
| 1289 |
See \fIcurl_easy_getinfo(3)\fP. |
| 1290 |
.SH "The multi Interface" |
| 1291 |
The easy interface as described in detail in this document is a synchronous |
| 1292 |
interface that transfers one file at a time and doesn't return until it is |
| 1293 |
done. |
| 1294 |
|
| 1295 |
The multi interface, on the other hand, allows your program to transfer |
| 1296 |
multiple files in both directions at the same time, without forcing you to use |
| 1297 |
multiple threads. The name might make it seem that the multi interface is for |
| 1298 |
multi-threaded programs, but the truth is almost the reverse. The multi |
| 1299 |
interface allows a single-threaded application to perform the same kinds of |
| 1300 |
multiple, simultaneous transfers that multi-threaded programs can perform. It |
| 1301 |
allows many of the benefits of multi-threaded transfers without the complexity |
| 1302 |
of managing and synchronizing many threads. |
| 1303 |
|
| 1304 |
To complicate matters somewhat more, there are even two versions of the multi |
| 1305 |
interface. The event based one, also called multi_socket and the "normal one" |
| 1306 |
designed for using with select(). See the libcurl-multi.3 man page for details |
| 1307 |
on the multi_socket event based API, this description here is for the select() |
| 1308 |
oriented one. |
| 1309 |
|
| 1310 |
To use this interface, you are better off if you first understand the basics |
| 1311 |
of how to use the easy interface. The multi interface is simply a way to make |
| 1312 |
multiple transfers at the same time by adding up multiple easy handles into |
| 1313 |
a "multi stack". |
| 1314 |
|
| 1315 |
You create the easy handles you want, one for each concurrent transfer, and |
| 1316 |
you set all the options just like you learned above, and then you create a |
| 1317 |
multi handle with \fIcurl_multi_init(3)\fP and add all those easy handles to |
| 1318 |
that multi handle with \fIcurl_multi_add_handle(3)\fP. |
| 1319 |
|
| 1320 |
When you've added the handles you have for the moment (you can still add new |
| 1321 |
ones at any time), you start the transfers by calling |
| 1322 |
\fIcurl_multi_perform(3)\fP. |
| 1323 |
|
| 1324 |
\fIcurl_multi_perform(3)\fP is asynchronous. It will only perform what can be |
| 1325 |
done now and then return back control to your program. It is designed to never |
| 1326 |
block. You need to keep calling the function until all transfers are |
| 1327 |
completed. |
| 1328 |
|
| 1329 |
The best usage of this interface is when you do a select() on all possible |
| 1330 |
file descriptors or sockets to know when to call libcurl again. This also |
| 1331 |
makes it easy for you to wait and respond to actions on your own application's |
| 1332 |
sockets/handles. You figure out what to select() for by using |
| 1333 |
\fIcurl_multi_fdset(3)\fP, that fills in a set of fd_set variables for you |
| 1334 |
with the particular file descriptors libcurl uses for the moment. |
| 1335 |
|
| 1336 |
When you then call select(), it'll return when one of the file handles signal |
| 1337 |
action and you then call \fIcurl_multi_perform(3)\fP to allow libcurl to do |
| 1338 |
what it wants to do. Take note that libcurl does also feature some time-out |
| 1339 |
code so we advise you to never use very long timeouts on select() before you |
| 1340 |
call \fIcurl_multi_perform(3)\fP again. \fIcurl_multi_timeout(3)\fP is |
| 1341 |
provided to help you get a suitable timeout period. |
| 1342 |
|
| 1343 |
Another precaution you should use: always call \fIcurl_multi_fdset(3)\fP |
| 1344 |
immediately before the select() call since the current set of file descriptors |
| 1345 |
may change in any curl function invoke. |
| 1346 |
|
| 1347 |
If you want to stop the transfer of one of the easy handles in the stack, you |
| 1348 |
can use \fIcurl_multi_remove_handle(3)\fP to remove individual easy |
| 1349 |
handles. Remember that easy handles should be \fIcurl_easy_cleanup(3)\fPed. |
| 1350 |
|
| 1351 |
When a transfer within the multi stack has finished, the counter of running |
| 1352 |
transfers (as filled in by \fIcurl_multi_perform(3)\fP) will decrease. When |
| 1353 |
the number reaches zero, all transfers are done. |
| 1354 |
|
| 1355 |
\fIcurl_multi_info_read(3)\fP can be used to get information about completed |
| 1356 |
transfers. It then returns the CURLcode for each easy transfer, to allow you |
| 1357 |
to figure out success on each individual transfer. |
| 1358 |
|
| 1359 |
.SH "SSL, Certificates and Other Tricks" |
| 1360 |
|
| 1361 |
[ seeding, passwords, keys, certificates, ENGINE, ca certs ] |
| 1362 |
|
| 1363 |
.SH "Sharing Data Between Easy Handles" |
| 1364 |
You can share some data between easy handles when the easy interface is used, |
| 1365 |
and some data is share automatically when you use the multi interface. |
| 1366 |
|
| 1367 |
When you add easy handles to a multi handle, these easy handles will |
| 1368 |
automatically share a lot of the data that otherwise would be kept on a |
| 1369 |
per-easy handle basis when the easy interface is used. |
| 1370 |
|
| 1371 |
The DNS cache is shared between handles within a multi handle, making |
| 1372 |
subsequent name resolving faster, and the connection pool that is kept to |
| 1373 |
better allow persistent connections and connection re-use is also shared. If |
| 1374 |
you're using the easy interface, you can still share these between specific |
| 1375 |
easy handles by using the share interface, see \fIlibcurl-share(3)\fP. |
| 1376 |
|
| 1377 |
Some things are never shared automatically, not within multi handles, like for |
| 1378 |
example cookies so the only way to share that is with the share interface. |
| 1379 |
.SH "Footnotes" |
| 1380 |
|
| 1381 |
.IP "[1]" |
| 1382 |
libcurl 7.10.3 and later have the ability to switch over to chunked |
| 1383 |
Transfer-Encoding in cases where HTTP uploads are done with data of an unknown |
| 1384 |
size. |
| 1385 |
.IP "[2]" |
| 1386 |
This happens on Windows machines when libcurl is built and used as a |
| 1387 |
DLL. However, you can still do this on Windows if you link with a static |
| 1388 |
library. |
| 1389 |
.IP "[3]" |
| 1390 |
The curl-config tool is generated at build-time (on Unix-like systems) and |
| 1391 |
should be installed with the 'make install' or similar instruction that |
| 1392 |
installs the library, header files, man pages etc. |
| 1393 |
.IP "[4]" |
| 1394 |
This behavior was different in versions before 7.17.0, where strings had to |
| 1395 |
remain valid past the end of the \fIcurl_easy_setopt(3)\fP call. |
| 1396 |
.SH "SEE ALSO" |
| 1397 |
.BR libcurl-errors "(3), " libcurl-multi "(3), " libcurl-easy "(3) " |