| 1 |
Things TODO |
| 2 |
=========== |
| 3 |
|
| 4 |
* Fix -Wsign-conversion warnings in src |
| 5 |
|
| 6 |
* Fix the numerous malloc+copy operations for sending data, see "Buffering |
| 7 |
Improvements" below for details |
| 8 |
|
| 9 |
* make sure the windowing code adapts better to slow situations so that it |
| 10 |
does not then use as much memory as today. Possibly by an app-controllable |
| 11 |
"Window mode"? |
| 12 |
|
| 13 |
* Decrease the number of mallocs. Everywhere. Will get easier once the |
| 14 |
buffering improvements have been done. |
| 15 |
|
| 16 |
* Use SO_NOSIGPIPE for Mac OS/BSD systems where MSG_NOSIGNAL does not |
| 17 |
exist/work |
| 18 |
|
| 19 |
* Extend the test suite to actually test lots of aspects of libssh2 |
| 20 |
|
| 21 |
* Update public API to drop casts added to fix compiler warnings |
| 22 |
|
| 23 |
* Expose error messages sent by the server |
| 24 |
|
| 25 |
* select() is troublesome with libssh2 when using multiple channels over |
| 26 |
the same session. See "New Transport API" below for more details. |
| 27 |
|
| 28 |
* for obsolete/weak/insecure algorithms: either stop enabling them by default |
| 29 |
at build-time, or delete support for them completely. |
| 30 |
|
| 31 |
At next SONAME bump |
| 32 |
=================== |
| 33 |
|
| 34 |
* stop using #defined macros as part of the official API. The macros should |
| 35 |
either be turned into real functions or discarded from the API. |
| 36 |
|
| 37 |
* delete or deprecate libssh2_session_callback_set() |
| 38 |
|
| 39 |
* bump length arguments in callback functions to size_t/ssize_t |
| 40 |
|
| 41 |
* remove the following functions from the API/ABI |
| 42 |
|
| 43 |
libssh2_base64_decode() |
| 44 |
libssh2_session_flag() |
| 45 |
libssh2_channel_handle_extended_data() |
| 46 |
libssh2_channel_receive_window_adjust() |
| 47 |
libssh2_poll() |
| 48 |
libssh2_poll_channel_read() |
| 49 |
libssh2_session_startup() (libssh2_session_handshake() is the replacement) |
| 50 |
libssh2_banner_set() (libssh2_session_banner_set() is the replacement) |
| 51 |
|
| 52 |
* Rename a few function: |
| 53 |
|
| 54 |
libssh2_hostkey_hash => libssh2_session_hostkey_hash |
| 55 |
libssh2_banner_set => libssh2_session_banner_set |
| 56 |
|
| 57 |
* change 'int' to 'libssh2_socket_t' in the public API for sockets. |
| 58 |
|
| 59 |
* Use 'size_t' for string lengths in all functions. |
| 60 |
|
| 61 |
* Add a comment field to struct libssh2_knownhost. |
| 62 |
|
| 63 |
* remove the existing libssh2_knownhost_add() function and rename |
| 64 |
libssh2_knownhost_addc to become the new libssh2_knownhost_add instead |
| 65 |
|
| 66 |
* remove the existing libssh2_scp_send_ex() function and rename |
| 67 |
libssh2_scp_send64 to become the new libssh2_scp_send instead. |
| 68 |
|
| 69 |
* remove the existing libssh2_knownhost_check() function and rename |
| 70 |
libssh2_knownhost_checkp() to become the new libssh2_knownhost_check instead |
| 71 |
|
| 72 |
Buffering Improvements |
| 73 |
====================== |
| 74 |
|
| 75 |
transport_write |
| 76 |
|
| 77 |
- If this function gets called with a total packet size that is larger than |
| 78 |
32K, it should create more than one SSH packet so that it keeps the largest |
| 79 |
one below 32K |
| 80 |
|
| 81 |
sftp_write |
| 82 |
|
| 83 |
- should not copy/allocate anything for the data, only create a header chunk |
| 84 |
and pass on the payload data to channel_write "pointed to" |
| 85 |
|
| 86 |
New Transport API |
| 87 |
================= |
| 88 |
|
| 89 |
THE PROBLEM |
| 90 |
|
| 91 |
The problem in a nutshell is that when an application opens up multiple |
| 92 |
channels over a single session, those are all using the same socket. If the |
| 93 |
application is then using select() to wait for traffic (like any sensible app |
| 94 |
does) and wants to act on the data when select() tells there is something to |
| 95 |
for example read, what does an application do? |
| 96 |
|
| 97 |
With our current API, you have to loop over all the channels and read from |
| 98 |
them to see if they have data. This effectively makes blocking reads |
| 99 |
impossible. If the app has many channels in a setup like this, it even becomes |
| 100 |
slow. (The original API had the libssh2_poll_channel_read() and libssh2_poll() |
| 101 |
to somewhat overcome this hurdle, but they too have pretty much the same |
| 102 |
problems plus a few others.) |
| 103 |
|
| 104 |
Traffic in the other direction is similarly limited: the app has to try |
| 105 |
sending to all channels, even though some of them may very well not accept any |
| 106 |
data at that point. |
| 107 |
|
| 108 |
A SOLUTION |
| 109 |
|
| 110 |
I suggest we introduce two new helper functions: |
| 111 |
|
| 112 |
libssh2_transport_read() |
| 113 |
|
| 114 |
- Read "a bunch" of data from the given socket and returns information to the |
| 115 |
app about what channels that are now readable (ie they will not block when |
| 116 |
read from). The function can be called over and over and it will repeatedly |
| 117 |
return info about what channels that are readable at that moment. |
| 118 |
|
| 119 |
libssh2_transport_write() |
| 120 |
|
| 121 |
- Returns information about what channels that are writable, in the sense |
| 122 |
that they have windows set from the remote side that allows data to get |
| 123 |
sent. Writing to one of those channels will not block. Of course, the |
| 124 |
underlying socket may only accept a certain amount of data, so at the first |
| 125 |
short return, nothing more should be attempted to get sent until select() |
| 126 |
(or equivalent) has been used on the master socket again. |
| 127 |
|
| 128 |
I have not yet figured out a sensible API for how these functions should return |
| 129 |
that info, but if we agree on the general principles I guess we can work that |
| 130 |
out. |
| 131 |
|
| 132 |
VOLUNTARY |
| 133 |
|
| 134 |
I wanted to mention that these two helper functions would not be mandatory |
| 135 |
in any way. They would just be there for those who want them, and existing |
| 136 |
programs can remain using the old functions only if they prefer to. |
| 137 |
|
| 138 |
New SFTP API |
| 139 |
============ |
| 140 |
|
| 141 |
PURPOSE |
| 142 |
|
| 143 |
Provide API functions that explicitly tells at once that a (full) SFTP file |
| 144 |
transfer is wanted, to allow libssh2 to leverage on that knowledge to speed |
| 145 |
up things internally. It can for example do read ahead, buffer writes (merge |
| 146 |
small writes into larger chunks), better tune the SSH window and more. This |
| 147 |
sort of API is already provided for SCP transfers. |
| 148 |
|
| 149 |
API |
| 150 |
|
| 151 |
New functions: |
| 152 |
|
| 153 |
LIBSSH2_SFTP_HANDLE *libssh2_sftp_send(SFTP_SESSION *sftp, |
| 154 |
libssh2_uint64_t filesize, |
| 155 |
char *remote_path, |
| 156 |
size_t remote_path_len, |
| 157 |
long mode); |
| 158 |
|
| 159 |
Tell libssh2 that a local file with a given size is about to get sent to |
| 160 |
the SFTP server. |
| 161 |
|
| 162 |
LIBSSH2_SFTP_HANDLE *libssh2_sftp_recv(); |
| 163 |
|
| 164 |
Tell libssh2 that a remote file is requested to get downloaded from the SFTP |
| 165 |
server. |
| 166 |
|
| 167 |
Only the setup of the file transfer is different from an application's point |
| 168 |
of view. Depending on direction of the transfer(s), the following already |
| 169 |
existing functions should then be used until the transfer is complete: |
| 170 |
|
| 171 |
libssh2_sftp_read() |
| 172 |
libssh2_sftp_write() |
| 173 |
|
| 174 |
HOW TO USE |
| 175 |
|
| 176 |
1. Setup the transfer using one of the two new functions. |
| 177 |
|
| 178 |
2. Loop through the reading or writing of data. |
| 179 |
|
| 180 |
3. Cleanup the transfer |