| 1 |
_ _ ____ _ |
| 2 |
___| | | | _ \| | |
| 3 |
/ __| | | | |_) | | |
| 4 |
| (__| |_| | _ <| |___ |
| 5 |
\___|\___/|_| \_\_____| |
| 6 |
|
| 7 |
Things that could be nice to do in the future |
| 8 |
|
| 9 |
Things to do in project curl. Please tell us what you think, contribute and |
| 10 |
send us patches that improve things! |
| 11 |
|
| 12 |
Be aware that these are things that we could do, or have once been considered |
| 13 |
things we could do. If you want to work on any of these areas, please |
| 14 |
consider bringing it up for discussions first on the mailing list so that we |
| 15 |
all agree it is still a good idea for the project! |
| 16 |
|
| 17 |
All bugs documented in the KNOWN_BUGS document are subject for fixing! |
| 18 |
|
| 19 |
1. libcurl |
| 20 |
1.1 TFO support on Windows |
| 21 |
1.2 Consult %APPDATA% also for .netrc |
| 22 |
1.3 struct lifreq |
| 23 |
1.4 alt-svc sharing |
| 24 |
1.5 get rid of PATH_MAX |
| 25 |
1.7 Support HTTP/2 for HTTP(S) proxies |
| 26 |
1.8 CURLOPT_RESOLVE for any port number |
| 27 |
1.9 Cache negative name resolves |
| 28 |
1.10 auto-detect proxy |
| 29 |
1.11 minimize dependencies with dynamically loaded modules |
| 30 |
1.12 updated DNS server while running |
| 31 |
1.13 c-ares and CURLOPT_OPENSOCKETFUNCTION |
| 32 |
1.14 Typesafe curl_easy_setopt() |
| 33 |
1.15 Monitor connections in the connection pool |
| 34 |
1.16 Try to URL encode given URL |
| 35 |
1.17 Add support for IRIs |
| 36 |
1.18 try next proxy if one doesn't work |
| 37 |
1.20 SRV and URI DNS records |
| 38 |
1.22 CURLINFO_PAUSE_STATE |
| 39 |
1.23 Offer API to flush the connection pool |
| 40 |
1.24 TCP Fast Open for windows |
| 41 |
1.25 Expose tried IP addresses that failed |
| 42 |
1.27 hardcode the "localhost" addresses |
| 43 |
1.28 FD_CLOEXEC |
| 44 |
1.29 Upgrade to websockets |
| 45 |
1.30 config file parsing |
| 46 |
|
| 47 |
2. libcurl - multi interface |
| 48 |
2.1 More non-blocking |
| 49 |
2.2 Better support for same name resolves |
| 50 |
2.3 Non-blocking curl_multi_remove_handle() |
| 51 |
2.4 Split connect and authentication process |
| 52 |
2.5 Edge-triggered sockets should work |
| 53 |
2.6 multi upkeep |
| 54 |
|
| 55 |
3. Documentation |
| 56 |
3.2 Provide cmake config-file |
| 57 |
|
| 58 |
4. FTP |
| 59 |
4.1 HOST |
| 60 |
4.2 Alter passive/active on failure and retry |
| 61 |
4.3 Earlier bad letter detection |
| 62 |
4.5 ASCII support |
| 63 |
4.6 GSSAPI via Windows SSPI |
| 64 |
4.7 STAT for LIST without data connection |
| 65 |
4.8 Option to ignore private IP addresses in PASV response |
| 66 |
|
| 67 |
5. HTTP |
| 68 |
5.1 Better persistency for HTTP 1.0 |
| 69 |
5.3 Rearrange request header order |
| 70 |
5.4 Allow SAN names in HTTP/2 server push |
| 71 |
5.5 auth= in URLs |
| 72 |
|
| 73 |
6. TELNET |
| 74 |
6.1 ditch stdin |
| 75 |
6.2 ditch telnet-specific select |
| 76 |
6.3 feature negotiation debug data |
| 77 |
|
| 78 |
7. SMTP |
| 79 |
7.2 Enhanced capability support |
| 80 |
7.3 Add CURLOPT_MAIL_CLIENT option |
| 81 |
|
| 82 |
8. POP3 |
| 83 |
8.2 Enhanced capability support |
| 84 |
|
| 85 |
9. IMAP |
| 86 |
9.1 Enhanced capability support |
| 87 |
|
| 88 |
10. LDAP |
| 89 |
10.1 SASL based authentication mechanisms |
| 90 |
10.2 CURLOPT_SSL_CTX_FUNCTION for LDAPS |
| 91 |
10.3 Paged searches on LDAP server |
| 92 |
|
| 93 |
11. SMB |
| 94 |
11.1 File listing support |
| 95 |
11.2 Honor file timestamps |
| 96 |
11.3 Use NTLMv2 |
| 97 |
11.4 Create remote directories |
| 98 |
|
| 99 |
12. New protocols |
| 100 |
|
| 101 |
13. SSL |
| 102 |
13.2 Provide mutex locking API |
| 103 |
13.3 Support in-memory certs/ca certs/keys |
| 104 |
13.4 Cache/share OpenSSL contexts |
| 105 |
13.5 Export session ids |
| 106 |
13.6 Provide callback for cert verification |
| 107 |
13.7 improve configure --with-ssl |
| 108 |
13.8 Support DANE |
| 109 |
13.10 Support Authority Information Access certificate extension (AIA) |
| 110 |
13.11 Support intermediate & root pinning for PINNEDPUBLICKEY |
| 111 |
13.12 Support HSTS |
| 112 |
13.14 Support the clienthello extension |
| 113 |
|
| 114 |
14. GnuTLS |
| 115 |
14.2 check connection |
| 116 |
|
| 117 |
15. WinSSL/SChannel |
| 118 |
15.1 Add support for client certificate authentication |
| 119 |
15.3 Add support for the --ciphers option |
| 120 |
15.4 Add option to disable client certificate auto-send |
| 121 |
|
| 122 |
16. SASL |
| 123 |
16.1 Other authentication mechanisms |
| 124 |
16.2 Add QOP support to GSSAPI authentication |
| 125 |
16.3 Support binary messages (i.e.: non-base64) |
| 126 |
|
| 127 |
17. SSH protocols |
| 128 |
17.1 Multiplexing |
| 129 |
17.2 Handle growing SFTP files |
| 130 |
17.3 Support better than MD5 hostkey hash |
| 131 |
17.4 Support CURLOPT_PREQUOTE |
| 132 |
|
| 133 |
18. Command line tool |
| 134 |
18.1 sync |
| 135 |
18.2 glob posts |
| 136 |
18.3 prevent file overwriting |
| 137 |
18.4 --proxycommand |
| 138 |
18.5 UTF-8 filenames in Content-Disposition |
| 139 |
18.7 at least N milliseconds between requests |
| 140 |
18.9 Choose the name of file in braces for complex URLs |
| 141 |
18.10 improve how curl works in a windows console window |
| 142 |
18.11 Windows: set attribute 'archive' for completed downloads |
| 143 |
18.12 keep running, read instructions from pipe/socket |
| 144 |
18.15 --retry should resume |
| 145 |
18.16 send only part of --data |
| 146 |
18.17 consider file name from the redirected URL with -O ? |
| 147 |
18.18 retry on network is unreachable |
| 148 |
18.19 expand ~/ in config files |
| 149 |
18.20 host name sections in config files |
| 150 |
|
| 151 |
19. Build |
| 152 |
19.1 roffit |
| 153 |
19.2 Enable PIE and RELRO by default |
| 154 |
19.3 cmake test suite improvements |
| 155 |
|
| 156 |
20. Test suite |
| 157 |
20.1 SSL tunnel |
| 158 |
20.2 nicer lacking perl message |
| 159 |
20.3 more protocols supported |
| 160 |
20.4 more platforms supported |
| 161 |
20.5 Add support for concurrent connections |
| 162 |
20.6 Use the RFC6265 test suite |
| 163 |
20.7 Support LD_PRELOAD on macOS |
| 164 |
20.8 Run web-platform-tests url tests |
| 165 |
|
| 166 |
21. Next SONAME bump |
| 167 |
21.1 http-style HEAD output for FTP |
| 168 |
21.2 combine error codes |
| 169 |
21.3 extend CURLOPT_SOCKOPTFUNCTION prototype |
| 170 |
|
| 171 |
22. Next major release |
| 172 |
22.1 cleanup return codes |
| 173 |
22.2 remove obsolete defines |
| 174 |
22.3 size_t |
| 175 |
22.4 remove several functions |
| 176 |
22.5 remove CURLOPT_FAILONERROR |
| 177 |
22.7 remove progress meter from libcurl |
| 178 |
22.8 remove 'curl_httppost' from public |
| 179 |
|
| 180 |
============================================================================== |
| 181 |
|
| 182 |
1. libcurl |
| 183 |
|
| 184 |
1.1 TFO support on Windows |
| 185 |
|
| 186 |
TCP Fast Open is supported on several platforms but not on Windows. Work on |
| 187 |
this was once started but never finished. |
| 188 |
|
| 189 |
See https://github.com/curl/curl/pull/3378 |
| 190 |
|
| 191 |
1.2 Consult %APPDATA% also for .netrc |
| 192 |
|
| 193 |
%APPDATA%\.netrc is not considered when running on Windows. Shouldn't it? |
| 194 |
|
| 195 |
See https://github.com/curl/curl/issues/4016 |
| 196 |
|
| 197 |
1.3 struct lifreq |
| 198 |
|
| 199 |
Use 'struct lifreq' and SIOCGLIFADDR instead of 'struct ifreq' and |
| 200 |
SIOCGIFADDR on newer Solaris versions as they claim the latter is obsolete. |
| 201 |
To support IPv6 interface addresses for network interfaces properly. |
| 202 |
|
| 203 |
1.4 alt-svc sharing |
| 204 |
|
| 205 |
The share interface could benefit from allowing the alt-svc cache to be |
| 206 |
possible to share between easy handles. |
| 207 |
|
| 208 |
See https://github.com/curl/curl/issues/4476 |
| 209 |
|
| 210 |
1.5 get rid of PATH_MAX |
| 211 |
|
| 212 |
Having code use and rely on PATH_MAX is not nice: |
| 213 |
https://insanecoding.blogspot.com/2007/11/pathmax-simply-isnt.html |
| 214 |
|
| 215 |
Currently the libssh2 SSH based code uses it, but to remove PATH_MAX from |
| 216 |
there we need libssh2 to properly tell us when we pass in a too small buffer |
| 217 |
and its current API (as of libssh2 1.2.7) doesn't. |
| 218 |
|
| 219 |
1.7 Support HTTP/2 for HTTP(S) proxies |
| 220 |
|
| 221 |
Support for doing HTTP/2 to HTTP and HTTPS proxies is still missing. |
| 222 |
|
| 223 |
See https://github.com/curl/curl/issues/3570 |
| 224 |
|
| 225 |
1.8 CURLOPT_RESOLVE for any port number |
| 226 |
|
| 227 |
This option allows applications to set a replacement IP address for a given |
| 228 |
host + port pair. Consider making support for providing a replacement address |
| 229 |
for the host name on all port numbers. |
| 230 |
|
| 231 |
See https://github.com/curl/curl/issues/1264 |
| 232 |
|
| 233 |
1.9 Cache negative name resolves |
| 234 |
|
| 235 |
A name resolve that has failed is likely to fail when made again within a |
| 236 |
short period of time. Currently we only cache positive responses. |
| 237 |
|
| 238 |
1.10 auto-detect proxy |
| 239 |
|
| 240 |
libcurl could be made to detect the system proxy setup automatically and use |
| 241 |
that. On Windows, macOS and Linux desktops for example. |
| 242 |
|
| 243 |
The pull-request to use libproxy for this was deferred due to doubts on the |
| 244 |
reliability of the dependency and how to use it: |
| 245 |
https://github.com/curl/curl/pull/977 |
| 246 |
|
| 247 |
libdetectproxy is a (C++) library for detecting the proxy on Windows |
| 248 |
https://github.com/paulharris/libdetectproxy |
| 249 |
|
| 250 |
1.11 minimize dependencies with dynamically loaded modules |
| 251 |
|
| 252 |
We can create a system with loadable modules/plug-ins, where these modules |
| 253 |
would be the ones that link to 3rd party libs. That would allow us to avoid |
| 254 |
having to load ALL dependencies since only the necessary ones for this |
| 255 |
app/invoke/used protocols would be necessary to load. See |
| 256 |
https://github.com/curl/curl/issues/349 |
| 257 |
|
| 258 |
1.12 updated DNS server while running |
| 259 |
|
| 260 |
If /etc/resolv.conf gets updated while a program using libcurl is running, it |
| 261 |
is may cause name resolves to fail unless res_init() is called. We should |
| 262 |
consider calling res_init() + retry once unconditionally on all name resolve |
| 263 |
failures to mitigate against this. Firefox works like that. Note that Windows |
| 264 |
doesn't have res_init() or an alternative. |
| 265 |
|
| 266 |
https://github.com/curl/curl/issues/2251 |
| 267 |
|
| 268 |
1.13 c-ares and CURLOPT_OPENSOCKETFUNCTION |
| 269 |
|
| 270 |
curl will create most sockets via the CURLOPT_OPENSOCKETFUNCTION callback and |
| 271 |
close them with the CURLOPT_CLOSESOCKETFUNCTION callback. However, c-ares |
| 272 |
does not use those functions and instead opens and closes the sockets |
| 273 |
itself. This means that when curl passes the c-ares socket to the |
| 274 |
CURLMOPT_SOCKETFUNCTION it isn't owned by the application like other sockets. |
| 275 |
|
| 276 |
See https://github.com/curl/curl/issues/2734 |
| 277 |
|
| 278 |
1.14 Typesafe curl_easy_setopt() |
| 279 |
|
| 280 |
One of the most common problems in libcurl using applications is the lack of |
| 281 |
type checks for curl_easy_setopt() which happens because it accepts varargs |
| 282 |
and thus can take any type. |
| 283 |
|
| 284 |
One possible solution to this is to introduce a few different versions of the |
| 285 |
setopt version for the different kinds of data you can set. |
| 286 |
|
| 287 |
curl_easy_set_num() - sets a long value |
| 288 |
|
| 289 |
curl_easy_set_large() - sets a curl_off_t value |
| 290 |
|
| 291 |
curl_easy_set_ptr() - sets a pointer |
| 292 |
|
| 293 |
curl_easy_set_cb() - sets a callback PLUS its callback data |
| 294 |
|
| 295 |
1.15 Monitor connections in the connection pool |
| 296 |
|
| 297 |
libcurl's connection cache or pool holds a number of open connections for the |
| 298 |
purpose of possible subsequent connection reuse. It may contain a few up to a |
| 299 |
significant amount of connections. Currently, libcurl leaves all connections |
| 300 |
as they are and first when a connection is iterated over for matching or |
| 301 |
reuse purpose it is verified that it is still alive. |
| 302 |
|
| 303 |
Those connections may get closed by the server side for idleness or they may |
| 304 |
get a HTTP/2 ping from the peer to verify that they're still alive. By adding |
| 305 |
monitoring of the connections while in the pool, libcurl can detect dead |
| 306 |
connections (and close them) better and earlier, and it can handle HTTP/2 |
| 307 |
pings to keep such ones alive even when not actively doing transfers on them. |
| 308 |
|
| 309 |
1.16 Try to URL encode given URL |
| 310 |
|
| 311 |
Given a URL that for example contains spaces, libcurl could have an option |
| 312 |
that would try somewhat harder than it does now and convert spaces to %20 and |
| 313 |
perhaps URL encoded byte values over 128 etc (basically do what the redirect |
| 314 |
following code already does). |
| 315 |
|
| 316 |
https://github.com/curl/curl/issues/514 |
| 317 |
|
| 318 |
1.17 Add support for IRIs |
| 319 |
|
| 320 |
IRIs (RFC 3987) allow localized, non-ascii, names in the URL. To properly |
| 321 |
support this, curl/libcurl would need to translate/encode the given input |
| 322 |
from the input string encoding into percent encoded output "over the wire". |
| 323 |
|
| 324 |
To make that work smoothly for curl users even on Windows, curl would |
| 325 |
probably need to be able to convert from several input encodings. |
| 326 |
|
| 327 |
1.18 try next proxy if one doesn't work |
| 328 |
|
| 329 |
Allow an application to specify a list of proxies to try, and failing to |
| 330 |
connect to the first go on and try the next instead until the list is |
| 331 |
exhausted. Browsers support this feature at least when they specify proxies |
| 332 |
using PACs. |
| 333 |
|
| 334 |
https://github.com/curl/curl/issues/896 |
| 335 |
|
| 336 |
1.20 SRV and URI DNS records |
| 337 |
|
| 338 |
Offer support for resolving SRV and URI DNS records for libcurl to know which |
| 339 |
server to connect to for various protocols (including HTTP!). |
| 340 |
|
| 341 |
1.22 CURLINFO_PAUSE_STATE |
| 342 |
|
| 343 |
Return information about the transfer's current pause state, in both |
| 344 |
directions. https://github.com/curl/curl/issues/2588 |
| 345 |
|
| 346 |
1.23 Offer API to flush the connection pool |
| 347 |
|
| 348 |
Sometimes applications want to flush all the existing connections kept alive. |
| 349 |
An API could allow a forced flush or just a forced loop that would properly |
| 350 |
close all connections that have been closed by the server already. |
| 351 |
|
| 352 |
1.24 TCP Fast Open for windows |
| 353 |
|
| 354 |
libcurl supports the CURLOPT_TCP_FASTOPEN option since 7.49.0 for Linux and |
| 355 |
Mac OS. Windows supports TCP Fast Open starting with Windows 10, version 1607 |
| 356 |
and we should add support for it. |
| 357 |
|
| 358 |
1.25 Expose tried IP addresses that failed |
| 359 |
|
| 360 |
When libcurl fails to connect to a host, it should be able to offer the |
| 361 |
application the list of IP addresses that were used in the attempt. |
| 362 |
|
| 363 |
https://github.com/curl/curl/issues/2126 |
| 364 |
|
| 365 |
1.27 hardcode the "localhost" addresses |
| 366 |
|
| 367 |
There's this new spec getting adopted that says "localhost" should always and |
| 368 |
unconditionally be a local address and not get resolved by a DNS server. A |
| 369 |
fine way for curl to fix this would be to simply hard-code the response to |
| 370 |
127.0.0.1 and/or ::1 (depending on what IP versions that are requested). This |
| 371 |
is what the browsers probably will do with this hostname. |
| 372 |
|
| 373 |
https://bugzilla.mozilla.org/show_bug.cgi?id=1220810 |
| 374 |
|
| 375 |
https://tools.ietf.org/html/draft-ietf-dnsop-let-localhost-be-localhost-02 |
| 376 |
|
| 377 |
1.28 FD_CLOEXEC |
| 378 |
|
| 379 |
It sets the close-on-exec flag for the file descriptor, which causes the file |
| 380 |
descriptor to be automatically (and atomically) closed when any of the |
| 381 |
exec-family functions succeed. Should probably be set by default? |
| 382 |
|
| 383 |
https://github.com/curl/curl/issues/2252 |
| 384 |
|
| 385 |
1.29 Upgrade to websockets |
| 386 |
|
| 387 |
libcurl could offer a smoother path to get to a websocket connection. |
| 388 |
See https://github.com/curl/curl/issues/3523 |
| 389 |
|
| 390 |
Michael Kaufmann suggestion here: |
| 391 |
https://curl.haxx.se/video/curlup-2017/2017-03-19_05_Michael_Kaufmann_Websocket_support_for_curl.mp4 |
| 392 |
|
| 393 |
1.30 config file parsing |
| 394 |
|
| 395 |
Consider providing an API, possibly in a separate companion library, for |
| 396 |
parsing a config file like curl's -K/--config option to allow applications to |
| 397 |
get the same ability to read curl options from files. |
| 398 |
|
| 399 |
See https://github.com/curl/curl/issues/3698 |
| 400 |
|
| 401 |
2. libcurl - multi interface |
| 402 |
|
| 403 |
2.1 More non-blocking |
| 404 |
|
| 405 |
Make sure we don't ever loop because of non-blocking sockets returning |
| 406 |
EWOULDBLOCK or similar. Blocking cases include: |
| 407 |
|
| 408 |
- Name resolves on non-windows unless c-ares or the threaded resolver is used |
| 409 |
- file:// transfers |
| 410 |
- TELNET transfers |
| 411 |
- The "DONE" operation (post transfer protocol-specific actions) for the |
| 412 |
protocols SFTP, SMTP, FTP. Fixing Curl_done() for this is a worthy task. |
| 413 |
|
| 414 |
2.2 Better support for same name resolves |
| 415 |
|
| 416 |
If a name resolve has been initiated for name NN and a second easy handle |
| 417 |
wants to resolve that name as well, make it wait for the first resolve to end |
| 418 |
up in the cache instead of doing a second separate resolve. This is |
| 419 |
especially needed when adding many simultaneous handles using the same host |
| 420 |
name when the DNS resolver can get flooded. |
| 421 |
|
| 422 |
2.3 Non-blocking curl_multi_remove_handle() |
| 423 |
|
| 424 |
The multi interface has a few API calls that assume a blocking behavior, like |
| 425 |
add_handle() and remove_handle() which limits what we can do internally. The |
| 426 |
multi API need to be moved even more into a single function that "drives" |
| 427 |
everything in a non-blocking manner and signals when something is done. A |
| 428 |
remove or add would then only ask for the action to get started and then |
| 429 |
multi_perform() etc still be called until the add/remove is completed. |
| 430 |
|
| 431 |
2.4 Split connect and authentication process |
| 432 |
|
| 433 |
The multi interface treats the authentication process as part of the connect |
| 434 |
phase. As such any failures during authentication won't trigger the relevant |
| 435 |
QUIT or LOGOFF for protocols such as IMAP, POP3 and SMTP. |
| 436 |
|
| 437 |
2.5 Edge-triggered sockets should work |
| 438 |
|
| 439 |
The multi_socket API should work with edge-triggered socket events. One of |
| 440 |
the internal actions that need to be improved for this to work perfectly is |
| 441 |
the 'maxloops' handling in transfer.c:readwrite_data(). |
| 442 |
|
| 443 |
2.6 multi upkeep |
| 444 |
|
| 445 |
In libcurl 7.62.0 we introduced curl_easy_upkeep. It unfortunately only works |
| 446 |
on easy handles. We should introduces a version of that for the multi handle, |
| 447 |
and also consider doing "upkeep" automatically on connections in the |
| 448 |
connection pool when the multi handle is in used. |
| 449 |
|
| 450 |
See https://github.com/curl/curl/issues/3199 |
| 451 |
|
| 452 |
3. Documentation |
| 453 |
|
| 454 |
3.2 Provide cmake config-file |
| 455 |
|
| 456 |
A config-file package is a set of files provided by us to allow applications |
| 457 |
to write cmake scripts to find and use libcurl easier. See |
| 458 |
https://github.com/curl/curl/issues/885 |
| 459 |
|
| 460 |
4. FTP |
| 461 |
|
| 462 |
4.1 HOST |
| 463 |
|
| 464 |
HOST is a command for a client to tell which host name to use, to offer FTP |
| 465 |
servers named-based virtual hosting: |
| 466 |
|
| 467 |
https://tools.ietf.org/html/rfc7151 |
| 468 |
|
| 469 |
4.2 Alter passive/active on failure and retry |
| 470 |
|
| 471 |
When trying to connect passively to a server which only supports active |
| 472 |
connections, libcurl returns CURLE_FTP_WEIRD_PASV_REPLY and closes the |
| 473 |
connection. There could be a way to fallback to an active connection (and |
| 474 |
vice versa). https://curl.haxx.se/bug/feature.cgi?id=1754793 |
| 475 |
|
| 476 |
4.3 Earlier bad letter detection |
| 477 |
|
| 478 |
Make the detection of (bad) %0d and %0a codes in FTP URL parts earlier in the |
| 479 |
process to avoid doing a resolve and connect in vain. |
| 480 |
|
| 481 |
4.5 ASCII support |
| 482 |
|
| 483 |
FTP ASCII transfers do not follow RFC959. They don't convert the data |
| 484 |
accordingly. |
| 485 |
|
| 486 |
4.6 GSSAPI via Windows SSPI |
| 487 |
|
| 488 |
In addition to currently supporting the SASL GSSAPI mechanism (Kerberos V5) |
| 489 |
via third-party GSS-API libraries, such as Heimdal or MIT Kerberos, also add |
| 490 |
support for GSSAPI authentication via Windows SSPI. |
| 491 |
|
| 492 |
4.7 STAT for LIST without data connection |
| 493 |
|
| 494 |
Some FTP servers allow STAT for listing directories instead of using LIST, |
| 495 |
and the response is then sent over the control connection instead of as the |
| 496 |
otherwise usedw data connection: https://www.nsftools.com/tips/RawFTP.htm#STAT |
| 497 |
|
| 498 |
This is not detailed in any FTP specification. |
| 499 |
|
| 500 |
4.8 Option to ignore private IP addresses in PASV response |
| 501 |
|
| 502 |
Some servers respond with and some other FTP client implementations can |
| 503 |
ignore private (RFC 1918 style) IP addresses when received in PASV responses. |
| 504 |
To consider for libcurl as well. See https://github.com/curl/curl/issues/1455 |
| 505 |
|
| 506 |
5. HTTP |
| 507 |
|
| 508 |
5.1 Better persistency for HTTP 1.0 |
| 509 |
|
| 510 |
"Better" support for persistent connections over HTTP 1.0 |
| 511 |
https://curl.haxx.se/bug/feature.cgi?id=1089001 |
| 512 |
|
| 513 |
5.3 Rearrange request header order |
| 514 |
|
| 515 |
Server implementors often make an effort to detect browser and to reject |
| 516 |
clients it can detect to not match. One of the last details we cannot yet |
| 517 |
control in libcurl's HTTP requests, which also can be exploited to detect |
| 518 |
that libcurl is in fact used even when it tries to impersonate a browser, is |
| 519 |
the order of the request headers. I propose that we introduce a new option in |
| 520 |
which you give headers a value, and then when the HTTP request is built it |
| 521 |
sorts the headers based on that number. We could then have internally created |
| 522 |
headers use a default value so only headers that need to be moved have to be |
| 523 |
specified. |
| 524 |
|
| 525 |
5.4 Allow SAN names in HTTP/2 server push |
| 526 |
|
| 527 |
curl only allows HTTP/2 push promise if the provided :authority header value |
| 528 |
exactly matches the host name given in the URL. It could be extended to allow |
| 529 |
any name that would match the Subject Alternative Names in the server's TLS |
| 530 |
certificate. |
| 531 |
|
| 532 |
See https://github.com/curl/curl/pull/3581 |
| 533 |
|
| 534 |
5.5 auth= in URLs |
| 535 |
|
| 536 |
Add the ability to specify the preferred authentication mechanism to use by |
| 537 |
using ;auth=<mech> in the login part of the URL. |
| 538 |
|
| 539 |
For example: |
| 540 |
|
| 541 |
http://test:pass;[email protected] would be equivalent to specifying |
| 542 |
--user test:pass;auth=NTLM or --user test:pass --ntlm from the command line. |
| 543 |
|
| 544 |
Additionally this should be implemented for proxy base URLs as well. |
| 545 |
|
| 546 |
|
| 547 |
6. TELNET |
| 548 |
|
| 549 |
6.1 ditch stdin |
| 550 |
|
| 551 |
Reading input (to send to the remote server) on stdin is a crappy solution |
| 552 |
for library purposes. We need to invent a good way for the application to be |
| 553 |
able to provide the data to send. |
| 554 |
|
| 555 |
6.2 ditch telnet-specific select |
| 556 |
|
| 557 |
Move the telnet support's network select() loop go away and merge the code |
| 558 |
into the main transfer loop. Until this is done, the multi interface won't |
| 559 |
work for telnet. |
| 560 |
|
| 561 |
6.3 feature negotiation debug data |
| 562 |
|
| 563 |
Add telnet feature negotiation data to the debug callback as header data. |
| 564 |
|
| 565 |
|
| 566 |
7. SMTP |
| 567 |
|
| 568 |
7.2 Enhanced capability support |
| 569 |
|
| 570 |
Add the ability, for an application that uses libcurl, to obtain the list of |
| 571 |
capabilities returned from the EHLO command. |
| 572 |
|
| 573 |
7.3 Add CURLOPT_MAIL_CLIENT option |
| 574 |
|
| 575 |
Rather than use the URL to specify the mail client string to present in the |
| 576 |
HELO and EHLO commands, libcurl should support a new CURLOPT specifically for |
| 577 |
specifying this data as the URL is non-standard and to be honest a bit of a |
| 578 |
hack ;-) |
| 579 |
|
| 580 |
Please see the following thread for more information: |
| 581 |
https://curl.haxx.se/mail/lib-2012-05/0178.html |
| 582 |
|
| 583 |
|
| 584 |
8. POP3 |
| 585 |
|
| 586 |
8.2 Enhanced capability support |
| 587 |
|
| 588 |
Add the ability, for an application that uses libcurl, to obtain the list of |
| 589 |
capabilities returned from the CAPA command. |
| 590 |
|
| 591 |
9. IMAP |
| 592 |
|
| 593 |
9.1 Enhanced capability support |
| 594 |
|
| 595 |
Add the ability, for an application that uses libcurl, to obtain the list of |
| 596 |
capabilities returned from the CAPABILITY command. |
| 597 |
|
| 598 |
10. LDAP |
| 599 |
|
| 600 |
10.1 SASL based authentication mechanisms |
| 601 |
|
| 602 |
Currently the LDAP module only supports ldap_simple_bind_s() in order to bind |
| 603 |
to an LDAP server. However, this function sends username and password details |
| 604 |
using the simple authentication mechanism (as clear text). However, it should |
| 605 |
be possible to use ldap_bind_s() instead specifying the security context |
| 606 |
information ourselves. |
| 607 |
|
| 608 |
10.2 CURLOPT_SSL_CTX_FUNCTION for LDAPS |
| 609 |
|
| 610 |
CURLOPT_SSL_CTX_FUNCTION works perfectly for HTTPS and email protocols, but |
| 611 |
it has no effect for LDAPS connections. |
| 612 |
|
| 613 |
https://github.com/curl/curl/issues/4108 |
| 614 |
|
| 615 |
10.3 Paged searches on LDAP server |
| 616 |
|
| 617 |
https://github.com/curl/curl/issues/4452 |
| 618 |
|
| 619 |
11. SMB |
| 620 |
|
| 621 |
11.1 File listing support |
| 622 |
|
| 623 |
Add support for listing the contents of a SMB share. The output should probably |
| 624 |
be the same as/similar to FTP. |
| 625 |
|
| 626 |
11.2 Honor file timestamps |
| 627 |
|
| 628 |
The timestamp of the transferred file should reflect that of the original file. |
| 629 |
|
| 630 |
11.3 Use NTLMv2 |
| 631 |
|
| 632 |
Currently the SMB authentication uses NTLMv1. |
| 633 |
|
| 634 |
11.4 Create remote directories |
| 635 |
|
| 636 |
Support for creating remote directories when uploading a file to a directory |
| 637 |
that doesn't exist on the server, just like --ftp-create-dirs. |
| 638 |
|
| 639 |
12. New protocols |
| 640 |
|
| 641 |
13. SSL |
| 642 |
|
| 643 |
13.2 Provide mutex locking API |
| 644 |
|
| 645 |
Provide a libcurl API for setting mutex callbacks in the underlying SSL |
| 646 |
library, so that the same application code can use mutex-locking |
| 647 |
independently of OpenSSL or GnutTLS being used. |
| 648 |
|
| 649 |
13.3 Support in-memory certs/ca certs/keys |
| 650 |
|
| 651 |
You can specify the private and public keys for SSH/SSL as file paths. Some |
| 652 |
programs want to avoid using files and instead just pass them as in-memory |
| 653 |
data blobs. There's probably a challenge to make this work across the |
| 654 |
plethory of different TLS and SSH backends that curl supports. |
| 655 |
https://github.com/curl/curl/issues/2310 |
| 656 |
|
| 657 |
13.4 Cache/share OpenSSL contexts |
| 658 |
|
| 659 |
"Look at SSL cafile - quick traces look to me like these are done on every |
| 660 |
request as well, when they should only be necessary once per SSL context (or |
| 661 |
once per handle)". The major improvement we can rather easily do is to make |
| 662 |
sure we don't create and kill a new SSL "context" for every request, but |
| 663 |
instead make one for every connection and re-use that SSL context in the same |
| 664 |
style connections are re-used. It will make us use slightly more memory but |
| 665 |
it will libcurl do less creations and deletions of SSL contexts. |
| 666 |
|
| 667 |
Technically, the "caching" is probably best implemented by getting added to |
| 668 |
the share interface so that easy handles who want to and can reuse the |
| 669 |
context specify that by sharing with the right properties set. |
| 670 |
|
| 671 |
https://github.com/curl/curl/issues/1110 |
| 672 |
|
| 673 |
13.5 Export session ids |
| 674 |
|
| 675 |
Add an interface to libcurl that enables "session IDs" to get |
| 676 |
exported/imported. Cris Bailiff said: "OpenSSL has functions which can |
| 677 |
serialise the current SSL state to a buffer of your choice, and recover/reset |
| 678 |
the state from such a buffer at a later date - this is used by mod_ssl for |
| 679 |
apache to implement and SSL session ID cache". |
| 680 |
|
| 681 |
13.6 Provide callback for cert verification |
| 682 |
|
| 683 |
OpenSSL supports a callback for customised verification of the peer |
| 684 |
certificate, but this doesn't seem to be exposed in the libcurl APIs. Could |
| 685 |
it be? There's so much that could be done if it were! |
| 686 |
|
| 687 |
13.7 improve configure --with-ssl |
| 688 |
|
| 689 |
make the configure --with-ssl option first check for OpenSSL, then GnuTLS, |
| 690 |
then NSS... |
| 691 |
|
| 692 |
13.8 Support DANE |
| 693 |
|
| 694 |
DNS-Based Authentication of Named Entities (DANE) is a way to provide SSL |
| 695 |
keys and certs over DNS using DNSSEC as an alternative to the CA model. |
| 696 |
https://www.rfc-editor.org/rfc/rfc6698.txt |
| 697 |
|
| 698 |
An initial patch was posted by Suresh Krishnaswamy on March 7th 2013 |
| 699 |
(https://curl.haxx.se/mail/lib-2013-03/0075.html) but it was a too simple |
| 700 |
approach. See Daniel's comments: |
| 701 |
https://curl.haxx.se/mail/lib-2013-03/0103.html . libunbound may be the |
| 702 |
correct library to base this development on. |
| 703 |
|
| 704 |
Björn Stenberg wrote a separate initial take on DANE that was never |
| 705 |
completed. |
| 706 |
|
| 707 |
13.10 Support Authority Information Access certificate extension (AIA) |
| 708 |
|
| 709 |
AIA can provide various things like CRLs but more importantly information |
| 710 |
about intermediate CA certificates that can allow validation path to be |
| 711 |
fulfilled when the HTTPS server doesn't itself provide them. |
| 712 |
|
| 713 |
Since AIA is about downloading certs on demand to complete a TLS handshake, |
| 714 |
it is probably a bit tricky to get done right. |
| 715 |
|
| 716 |
See https://github.com/curl/curl/issues/2793 |
| 717 |
|
| 718 |
13.11 Support intermediate & root pinning for PINNEDPUBLICKEY |
| 719 |
|
| 720 |
CURLOPT_PINNEDPUBLICKEY does not consider the hashes of intermediate & root |
| 721 |
certificates when comparing the pinned keys. Therefore it is not compatible |
| 722 |
with "HTTP Public Key Pinning" as there also intermediate and root certificates |
| 723 |
can be pinned. This is very useful as it prevents webadmins from "locking |
| 724 |
themself out of their servers". |
| 725 |
|
| 726 |
Adding this feature would make curls pinning 100% compatible to HPKP and allow |
| 727 |
more flexible pinning. |
| 728 |
|
| 729 |
13.12 Support HSTS |
| 730 |
|
| 731 |
"HTTP Strict Transport Security" is TOFU (trust on first use), time-based |
| 732 |
features indicated by a HTTP header send by the webserver. It is widely used |
| 733 |
in browsers and it's purpose is to prevent insecure HTTP connections after |
| 734 |
a previous HTTPS connection. It protects against SSLStripping attacks. |
| 735 |
|
| 736 |
Doc: https://developer.mozilla.org/en-US/docs/Web/Security/HTTP_strict_transport_security |
| 737 |
RFC 6797: https://tools.ietf.org/html/rfc6797 |
| 738 |
|
| 739 |
13.14 Support the clienthello extension |
| 740 |
|
| 741 |
Certain stupid networks and middle boxes have a problem with SSL handshake |
| 742 |
pakets that are within a certain size range because how that sets some bits |
| 743 |
that previously (in older TLS version) were not set. The clienthello |
| 744 |
extension adds padding to avoid that size range. |
| 745 |
|
| 746 |
https://tools.ietf.org/html/rfc7685 |
| 747 |
https://github.com/curl/curl/issues/2299 |
| 748 |
|
| 749 |
14. GnuTLS |
| 750 |
|
| 751 |
14.2 check connection |
| 752 |
|
| 753 |
Add a way to check if the connection seems to be alive, to correspond to the |
| 754 |
SSL_peak() way we use with OpenSSL. |
| 755 |
|
| 756 |
15. WinSSL/SChannel |
| 757 |
|
| 758 |
15.1 Add support for client certificate authentication |
| 759 |
|
| 760 |
WinSSL/SChannel currently makes use of the OS-level system and user |
| 761 |
certificate and private key stores. This does not allow the application |
| 762 |
or the user to supply a custom client certificate using curl or libcurl. |
| 763 |
|
| 764 |
Therefore support for the existing -E/--cert and --key options should be |
| 765 |
implemented by supplying a custom certificate to the SChannel APIs, see: |
| 766 |
- Getting a Certificate for Schannel |
| 767 |
https://msdn.microsoft.com/en-us/library/windows/desktop/aa375447.aspx |
| 768 |
|
| 769 |
15.3 Add support for the --ciphers option |
| 770 |
|
| 771 |
The cipher suites used by WinSSL/SChannel are configured on an OS-level |
| 772 |
instead of an application-level. This does not allow the application or |
| 773 |
the user to customize the configured cipher suites using curl or libcurl. |
| 774 |
|
| 775 |
Therefore support for the existing --ciphers option should be implemented |
| 776 |
by mapping the OpenSSL/GnuTLS cipher suites to the SChannel APIs, see |
| 777 |
- Specifying Schannel Ciphers and Cipher Strengths |
| 778 |
https://msdn.microsoft.com/en-us/library/windows/desktop/aa380161.aspx |
| 779 |
|
| 780 |
15.4 Add option to disable client certificate auto-send |
| 781 |
|
| 782 |
Microsoft says "By default, Schannel will, with no notification to the client, |
| 783 |
attempt to locate a client certificate and send it to the server." That could |
| 784 |
be considered a privacy violation and unexpected. |
| 785 |
|
| 786 |
Some Windows users have come to expect that default behavior and to change the |
| 787 |
default to make it consistent with other SSL backends would be a breaking |
| 788 |
change. An option should be added that can be used to disable the default |
| 789 |
Schannel auto-send behavior. |
| 790 |
|
| 791 |
https://github.com/curl/curl/issues/2262 |
| 792 |
|
| 793 |
16. SASL |
| 794 |
|
| 795 |
16.1 Other authentication mechanisms |
| 796 |
|
| 797 |
Add support for other authentication mechanisms such as OLP, |
| 798 |
GSS-SPNEGO and others. |
| 799 |
|
| 800 |
16.2 Add QOP support to GSSAPI authentication |
| 801 |
|
| 802 |
Currently the GSSAPI authentication only supports the default QOP of auth |
| 803 |
(Authentication), whilst Kerberos V5 supports both auth-int (Authentication |
| 804 |
with integrity protection) and auth-conf (Authentication with integrity and |
| 805 |
privacy protection). |
| 806 |
|
| 807 |
16.3 Support binary messages (i.e.: non-base64) |
| 808 |
|
| 809 |
Mandatory to support LDAP SASL authentication. |
| 810 |
|
| 811 |
|
| 812 |
17. SSH protocols |
| 813 |
|
| 814 |
17.1 Multiplexing |
| 815 |
|
| 816 |
SSH is a perfectly fine multiplexed protocols which would allow libcurl to do |
| 817 |
multiple parallel transfers from the same host using the same connection, |
| 818 |
much in the same spirit as HTTP/2 does. libcurl however does not take |
| 819 |
advantage of that ability but will instead always create a new connection for |
| 820 |
new transfers even if an existing connection already exists to the host. |
| 821 |
|
| 822 |
To fix this, libcurl would have to detect an existing connection and "attach" |
| 823 |
the new transfer to the existing one. |
| 824 |
|
| 825 |
17.2 Handle growing SFTP files |
| 826 |
|
| 827 |
The SFTP code in libcurl checks the file size *before* a transfer starts and |
| 828 |
then proceeds to transfer exactly that amount of data. If the remote file |
| 829 |
grows while the transfer is in progress libcurl won't notice and will not |
| 830 |
adapt. The OpenSSH SFTP command line tool does and libcurl could also just |
| 831 |
attempt to download more to see if there is more to get... |
| 832 |
|
| 833 |
https://github.com/curl/curl/issues/4344 |
| 834 |
|
| 835 |
17.3 Support better than MD5 hostkey hash |
| 836 |
|
| 837 |
libcurl offers the CURLOPT_SSH_HOST_PUBLIC_KEY_MD5 option for verifying the |
| 838 |
server's key. MD5 is generally being deprecated so we should implement |
| 839 |
support for stronger hashing algorithms. libssh2 itself is what provides this |
| 840 |
underlying functionality and it supports at least SHA-1 as an alternative. |
| 841 |
SHA-1 is also being deprecated these days so we should consider working with |
| 842 |
libssh2 to instead offer support for SHA-256 or similar. |
| 843 |
|
| 844 |
17.4 Support CURLOPT_PREQUOTE |
| 845 |
|
| 846 |
The two other QUOTE options are supported for SFTP, but this was left out for |
| 847 |
unknown reasons! |
| 848 |
|
| 849 |
18. Command line tool |
| 850 |
|
| 851 |
18.1 sync |
| 852 |
|
| 853 |
"curl --sync http://example.com/feed[1-100].rss" or |
| 854 |
"curl --sync http://example.net/{index,calendar,history}.html" |
| 855 |
|
| 856 |
Downloads a range or set of URLs using the remote name, but only if the |
| 857 |
remote file is newer than the local file. A Last-Modified HTTP date header |
| 858 |
should also be used to set the mod date on the downloaded file. |
| 859 |
|
| 860 |
18.2 glob posts |
| 861 |
|
| 862 |
Globbing support for -d and -F, as in 'curl -d "name=foo[0-9]" URL'. |
| 863 |
This is easily scripted though. |
| 864 |
|
| 865 |
18.3 prevent file overwriting |
| 866 |
|
| 867 |
Add an option that prevents curl from overwriting existing local files. When |
| 868 |
used, and there already is an existing file with the target file name |
| 869 |
(either -O or -o), a number should be appended (and increased if already |
| 870 |
existing). So that index.html becomes first index.html.1 and then |
| 871 |
index.html.2 etc. |
| 872 |
|
| 873 |
18.4 --proxycommand |
| 874 |
|
| 875 |
Allow the user to make curl run a command and use its stdio to make requests |
| 876 |
and not do any network connection by itself. Example: |
| 877 |
|
| 878 |
curl --proxycommand 'ssh [email protected] -W 10.1.1.75 80' \ |
| 879 |
http://some/otherwise/unavailable/service.php |
| 880 |
|
| 881 |
See https://github.com/curl/curl/issues/4941 |
| 882 |
|
| 883 |
18.5 UTF-8 filenames in Content-Disposition |
| 884 |
|
| 885 |
RFC 6266 documents how UTF-8 names can be passed to a client in the |
| 886 |
Content-Disposition header, and curl does not support this. |
| 887 |
|
| 888 |
https://github.com/curl/curl/issues/1888 |
| 889 |
|
| 890 |
18.7 at least N milliseconds between requests |
| 891 |
|
| 892 |
Allow curl command lines issue a lot of request against services that limit |
| 893 |
users to no more than N requests/second or similar. Could be implemented with |
| 894 |
an option asking that at least a certain time has elapsed since the previous |
| 895 |
request before the next one will be performed. Example: |
| 896 |
|
| 897 |
$ curl "https://example.com/api?input=[1-1000]" -d yadayada --after 500 |
| 898 |
|
| 899 |
See https://github.com/curl/curl/issues/3920 |
| 900 |
|
| 901 |
18.9 Choose the name of file in braces for complex URLs |
| 902 |
|
| 903 |
When using braces to download a list of URLs and you use complicated names |
| 904 |
in the list of alternatives, it could be handy to allow curl to use other |
| 905 |
names when saving. |
| 906 |
|
| 907 |
Consider a way to offer that. Possibly like |
| 908 |
{partURL1:name1,partURL2:name2,partURL3:name3} where the name following the |
| 909 |
colon is the output name. |
| 910 |
|
| 911 |
See https://github.com/curl/curl/issues/221 |
| 912 |
|
| 913 |
18.10 improve how curl works in a windows console window |
| 914 |
|
| 915 |
If you pull the scrollbar when transferring with curl in a Windows console |
| 916 |
window, the transfer is interrupted and can get disconnected. This can |
| 917 |
probably be improved. See https://github.com/curl/curl/issues/322 |
| 918 |
|
| 919 |
18.11 Windows: set attribute 'archive' for completed downloads |
| 920 |
|
| 921 |
The archive bit (FILE_ATTRIBUTE_ARCHIVE, 0x20) separates files that shall be |
| 922 |
backed up from those that are either not ready or have not changed. |
| 923 |
|
| 924 |
Downloads in progress are neither ready to be backed up, nor should they be |
| 925 |
opened by a different process. Only after a download has been completed it's |
| 926 |
sensible to include it in any integer snapshot or backup of the system. |
| 927 |
|
| 928 |
See https://github.com/curl/curl/issues/3354 |
| 929 |
|
| 930 |
18.12 keep running, read instructions from pipe/socket |
| 931 |
|
| 932 |
Provide an option that makes curl not exit after the last URL (or even work |
| 933 |
without a given URL), and then make it read instructions passed on a pipe or |
| 934 |
over a socket to make further instructions so that a second subsequent curl |
| 935 |
invoke can talk to the still running instance and ask for transfers to get |
| 936 |
done, and thus maintain its connection pool, DNS cache and more. |
| 937 |
|
| 938 |
18.15 --retry should resume |
| 939 |
|
| 940 |
When --retry is used and curl actually retries transfer, it should use the |
| 941 |
already transferred data and do a resumed transfer for the rest (when |
| 942 |
possible) so that it doesn't have to transfer the same data again that was |
| 943 |
already transferred before the retry. |
| 944 |
|
| 945 |
See https://github.com/curl/curl/issues/1084 |
| 946 |
|
| 947 |
18.16 send only part of --data |
| 948 |
|
| 949 |
When the user only wants to send a small piece of the data provided with |
| 950 |
--data or --data-binary, like when that data is a huge file, consider a way |
| 951 |
to specify that curl should only send a piece of that. One suggested syntax |
| 952 |
would be: "--data-binary @largefile.zip!1073741823-2147483647". |
| 953 |
|
| 954 |
See https://github.com/curl/curl/issues/1200 |
| 955 |
|
| 956 |
18.17 consider file name from the redirected URL with -O ? |
| 957 |
|
| 958 |
When a user gives a URL and uses -O, and curl follows a redirect to a new |
| 959 |
URL, the file name is not extracted and used from the newly redirected-to URL |
| 960 |
even if the new URL may have a much more sensible file name. |
| 961 |
|
| 962 |
This is clearly documented and helps for security since there's no surprise |
| 963 |
to users which file name that might get overwritten. But maybe a new option |
| 964 |
could allow for this or maybe -J should imply such a treatment as well as -J |
| 965 |
already allows for the server to decide what file name to use so it already |
| 966 |
provides the "may overwrite any file" risk. |
| 967 |
|
| 968 |
This is extra tricky if the original URL has no file name part at all since |
| 969 |
then the current code path will error out with an error message, and we can't |
| 970 |
*know* already at that point if curl will be redirected to a URL that has a |
| 971 |
file name... |
| 972 |
|
| 973 |
See https://github.com/curl/curl/issues/1241 |
| 974 |
|
| 975 |
18.18 retry on network is unreachable |
| 976 |
|
| 977 |
The --retry option retries transfers on "transient failures". We later added |
| 978 |
--retry-connrefused to also retry for "connection refused" errors. |
| 979 |
|
| 980 |
Suggestions have been brought to also allow retry on "network is unreachable" |
| 981 |
errors and while totally reasonable, maybe we should consider a way to make |
| 982 |
this more configurable than to add a new option for every new error people |
| 983 |
want to retry for? |
| 984 |
|
| 985 |
https://github.com/curl/curl/issues/1603 |
| 986 |
|
| 987 |
18.19 expand ~/ in config files |
| 988 |
|
| 989 |
For example .curlrc could benefit from being able to do this. |
| 990 |
|
| 991 |
See https://github.com/curl/curl/issues/2317 |
| 992 |
|
| 993 |
18.20 host name sections in config files |
| 994 |
|
| 995 |
config files would be more powerful if they could set different |
| 996 |
configurations depending on used URLs, host name or possibly origin. Then a |
| 997 |
default .curlrc could a specific user-agent only when doing requests against |
| 998 |
a certain site. |
| 999 |
|
| 1000 |
|
| 1001 |
19. Build |
| 1002 |
|
| 1003 |
19.1 roffit |
| 1004 |
|
| 1005 |
Consider extending 'roffit' to produce decent ASCII output, and use that |
| 1006 |
instead of (g)nroff when building src/tool_hugehelp.c |
| 1007 |
|
| 1008 |
19.2 Enable PIE and RELRO by default |
| 1009 |
|
| 1010 |
Especially when having programs that execute curl via the command line, PIE |
| 1011 |
renders the exploitation of memory corruption vulnerabilities a lot more |
| 1012 |
difficult. This can be attributed to the additional information leaks being |
| 1013 |
required to conduct a successful attack. RELRO, on the other hand, masks |
| 1014 |
different binary sections like the GOT as read-only and thus kills a handful |
| 1015 |
of techniques that come in handy when attackers are able to arbitrarily |
| 1016 |
overwrite memory. A few tests showed that enabling these features had close |
| 1017 |
to no impact, neither on the performance nor on the general functionality of |
| 1018 |
curl. |
| 1019 |
|
| 1020 |
19.3 cmake test suite improvements |
| 1021 |
|
| 1022 |
The cmake build doesn't support 'make show' so it doesn't know which tests |
| 1023 |
are in the makefile or not (making appveyor builds do many false warnings |
| 1024 |
about it) nor does it support running the test suite if building out-of-tree. |
| 1025 |
|
| 1026 |
See https://github.com/curl/curl/issues/3109 |
| 1027 |
|
| 1028 |
20. Test suite |
| 1029 |
|
| 1030 |
20.1 SSL tunnel |
| 1031 |
|
| 1032 |
Make our own version of stunnel for simple port forwarding to enable HTTPS |
| 1033 |
and FTP-SSL tests without the stunnel dependency, and it could allow us to |
| 1034 |
provide test tools built with either OpenSSL or GnuTLS |
| 1035 |
|
| 1036 |
20.2 nicer lacking perl message |
| 1037 |
|
| 1038 |
If perl wasn't found by the configure script, don't attempt to run the tests |
| 1039 |
but explain something nice why it doesn't. |
| 1040 |
|
| 1041 |
20.3 more protocols supported |
| 1042 |
|
| 1043 |
Extend the test suite to include more protocols. The telnet could just do FTP |
| 1044 |
or http operations (for which we have test servers). |
| 1045 |
|
| 1046 |
20.4 more platforms supported |
| 1047 |
|
| 1048 |
Make the test suite work on more platforms. OpenBSD and Mac OS. Remove |
| 1049 |
fork()s and it should become even more portable. |
| 1050 |
|
| 1051 |
20.5 Add support for concurrent connections |
| 1052 |
|
| 1053 |
Tests 836, 882 and 938 were designed to verify that separate connections |
| 1054 |
aren't used when using different login credentials in protocols that |
| 1055 |
shouldn't re-use a connection under such circumstances. |
| 1056 |
|
| 1057 |
Unfortunately, ftpserver.pl doesn't appear to support multiple concurrent |
| 1058 |
connections. The read while() loop seems to loop until it receives a |
| 1059 |
disconnect from the client, where it then enters the waiting for connections |
| 1060 |
loop. When the client opens a second connection to the server, the first |
| 1061 |
connection hasn't been dropped (unless it has been forced - which we |
| 1062 |
shouldn't do in these tests) and thus the wait for connections loop is never |
| 1063 |
entered to receive the second connection. |
| 1064 |
|
| 1065 |
20.6 Use the RFC6265 test suite |
| 1066 |
|
| 1067 |
A test suite made for HTTP cookies (RFC 6265) by Adam Barth is available at |
| 1068 |
https://github.com/abarth/http-state/tree/master/tests |
| 1069 |
|
| 1070 |
It'd be really awesome if someone would write a script/setup that would run |
| 1071 |
curl with that test suite and detect deviances. Ideally, that would even be |
| 1072 |
incorporated into our regular test suite. |
| 1073 |
|
| 1074 |
20.7 Support LD_PRELOAD on macOS |
| 1075 |
|
| 1076 |
LD_RELOAD doesn't work on macOS, but there are tests which require it to run |
| 1077 |
properly. Look into making the preload support in runtests.pl portable such |
| 1078 |
that it uses DYLD_INSERT_LIBRARIES on macOS. |
| 1079 |
|
| 1080 |
20.8 Run web-platform-tests url tests |
| 1081 |
|
| 1082 |
Run web-platform-tests url tests and compare results with browsers on wpt.fyi |
| 1083 |
|
| 1084 |
It would help us find issues to fix and help us document where our parser |
| 1085 |
differs from the WHATWG URL spec parsers. |
| 1086 |
|
| 1087 |
See https://github.com/curl/curl/issues/4477 |
| 1088 |
|
| 1089 |
21. Next SONAME bump |
| 1090 |
|
| 1091 |
21.1 http-style HEAD output for FTP |
| 1092 |
|
| 1093 |
#undef CURL_FTP_HTTPSTYLE_HEAD in lib/ftp.c to remove the HTTP-style headers |
| 1094 |
from being output in NOBODY requests over FTP |
| 1095 |
|
| 1096 |
21.2 combine error codes |
| 1097 |
|
| 1098 |
Combine some of the error codes to remove duplicates. The original |
| 1099 |
numbering should not be changed, and the old identifiers would be |
| 1100 |
macroed to the new ones in an CURL_NO_OLDIES section to help with |
| 1101 |
backward compatibility. |
| 1102 |
|
| 1103 |
Candidates for removal and their replacements: |
| 1104 |
|
| 1105 |
CURLE_FILE_COULDNT_READ_FILE => CURLE_REMOTE_FILE_NOT_FOUND |
| 1106 |
|
| 1107 |
CURLE_FTP_COULDNT_RETR_FILE => CURLE_REMOTE_FILE_NOT_FOUND |
| 1108 |
|
| 1109 |
CURLE_FTP_COULDNT_USE_REST => CURLE_RANGE_ERROR |
| 1110 |
|
| 1111 |
CURLE_FUNCTION_NOT_FOUND => CURLE_FAILED_INIT |
| 1112 |
|
| 1113 |
CURLE_LDAP_INVALID_URL => CURLE_URL_MALFORMAT |
| 1114 |
|
| 1115 |
CURLE_TFTP_NOSUCHUSER => CURLE_TFTP_ILLEGAL |
| 1116 |
|
| 1117 |
CURLE_TFTP_NOTFOUND => CURLE_REMOTE_FILE_NOT_FOUND |
| 1118 |
|
| 1119 |
CURLE_TFTP_PERM => CURLE_REMOTE_ACCESS_DENIED |
| 1120 |
|
| 1121 |
21.3 extend CURLOPT_SOCKOPTFUNCTION prototype |
| 1122 |
|
| 1123 |
The current prototype only provides 'purpose' that tells what the |
| 1124 |
connection/socket is for, but not any protocol or similar. It makes it hard |
| 1125 |
for applications to differentiate on TCP vs UDP and even HTTP vs FTP and |
| 1126 |
similar. |
| 1127 |
|
| 1128 |
22. Next major release |
| 1129 |
|
| 1130 |
22.1 cleanup return codes |
| 1131 |
|
| 1132 |
curl_easy_cleanup() returns void, but curl_multi_cleanup() returns a |
| 1133 |
CURLMcode. These should be changed to be the same. |
| 1134 |
|
| 1135 |
22.2 remove obsolete defines |
| 1136 |
|
| 1137 |
remove obsolete defines from curl/curl.h |
| 1138 |
|
| 1139 |
22.3 size_t |
| 1140 |
|
| 1141 |
make several functions use size_t instead of int in their APIs |
| 1142 |
|
| 1143 |
22.4 remove several functions |
| 1144 |
|
| 1145 |
remove the following functions from the public API: |
| 1146 |
|
| 1147 |
curl_getenv |
| 1148 |
|
| 1149 |
curl_mprintf (and variations) |
| 1150 |
|
| 1151 |
curl_strequal |
| 1152 |
|
| 1153 |
curl_strnequal |
| 1154 |
|
| 1155 |
They will instead become curlx_ - alternatives. That makes the curl app |
| 1156 |
still capable of using them, by building with them from source. |
| 1157 |
|
| 1158 |
These functions have no purpose anymore: |
| 1159 |
|
| 1160 |
curl_multi_socket |
| 1161 |
|
| 1162 |
curl_multi_socket_all |
| 1163 |
|
| 1164 |
22.5 remove CURLOPT_FAILONERROR |
| 1165 |
|
| 1166 |
Remove support for CURLOPT_FAILONERROR, it has gotten too kludgy and weird |
| 1167 |
internally. Let the app judge success or not for itself. |
| 1168 |
|
| 1169 |
22.7 remove progress meter from libcurl |
| 1170 |
|
| 1171 |
The internally provided progress meter output doesn't belong in the library. |
| 1172 |
Basically no application wants it (apart from curl) but instead applications |
| 1173 |
can and should do their own progress meters using the progress callback. |
| 1174 |
|
| 1175 |
The progress callback should then be bumped as well to get proper 64bit |
| 1176 |
variable types passed to it instead of doubles so that big files work |
| 1177 |
correctly. |
| 1178 |
|
| 1179 |
22.8 remove 'curl_httppost' from public |
| 1180 |
|
| 1181 |
curl_formadd() was made to fill in a public struct, but the fact that the |
| 1182 |
struct is public is never really used by application for their own advantage |
| 1183 |
but instead often restricts how the form functions can or can't be modified. |
| 1184 |
|
| 1185 |
Changing them to return a private handle will benefit the implementation and |
| 1186 |
allow us much greater freedoms while still maintaining a solid API and ABI. |