Debian Codegraph @

docs/TODO

raw
1 Things TODO
2 ===========
3
4 * Fix -Wsign-conversion warnings in src
5
6 * Fix the numerous malloc+copy operations for sending data, see "Buffering
7 Improvements" below for details
8
9 * make sure the windowing code adapts better to slow situations so that it
10 does not then use as much memory as today. Possibly by an app-controllable
11 "Window mode"?
12
13 * Decrease the number of mallocs. Everywhere. Will get easier once the
14 buffering improvements have been done.
15
16 * Use SO_NOSIGPIPE for Mac OS/BSD systems where MSG_NOSIGNAL does not
17 exist/work
18
19 * Extend the test suite to actually test lots of aspects of libssh2
20
21 * Update public API to drop casts added to fix compiler warnings
22
23 * Expose error messages sent by the server
24
25 * select() is troublesome with libssh2 when using multiple channels over
26 the same session. See "New Transport API" below for more details.
27
28 * for obsolete/weak/insecure algorithms: either stop enabling them by default
29 at build-time, or delete support for them completely.
30
31 At next SONAME bump
32 ===================
33
34 * stop using #defined macros as part of the official API. The macros should
35 either be turned into real functions or discarded from the API.
36
37 * delete or deprecate libssh2_session_callback_set()
38
39 * bump length arguments in callback functions to size_t/ssize_t
40
41 * remove the following functions from the API/ABI
42
43 libssh2_base64_decode()
44 libssh2_session_flag()
45 libssh2_channel_handle_extended_data()
46 libssh2_channel_receive_window_adjust()
47 libssh2_poll()
48 libssh2_poll_channel_read()
49 libssh2_session_startup() (libssh2_session_handshake() is the replacement)
50 libssh2_banner_set() (libssh2_session_banner_set() is the replacement)
51
52 * Rename a few function:
53
54 libssh2_hostkey_hash => libssh2_session_hostkey_hash
55 libssh2_banner_set => libssh2_session_banner_set
56
57 * change 'int' to 'libssh2_socket_t' in the public API for sockets.
58
59 * Use 'size_t' for string lengths in all functions.
60
61 * Add a comment field to struct libssh2_knownhost.
62
63 * remove the existing libssh2_knownhost_add() function and rename
64 libssh2_knownhost_addc to become the new libssh2_knownhost_add instead
65
66 * remove the existing libssh2_scp_send_ex() function and rename
67 libssh2_scp_send64 to become the new libssh2_scp_send instead.
68
69 * remove the existing libssh2_knownhost_check() function and rename
70 libssh2_knownhost_checkp() to become the new libssh2_knownhost_check instead
71
72 Buffering Improvements
73 ======================
74
75 transport_write
76
77 - If this function gets called with a total packet size that is larger than
78 32K, it should create more than one SSH packet so that it keeps the largest
79 one below 32K
80
81 sftp_write
82
83 - should not copy/allocate anything for the data, only create a header chunk
84 and pass on the payload data to channel_write "pointed to"
85
86 New Transport API
87 =================
88
89 THE PROBLEM
90
91 The problem in a nutshell is that when an application opens up multiple
92 channels over a single session, those are all using the same socket. If the
93 application is then using select() to wait for traffic (like any sensible app
94 does) and wants to act on the data when select() tells there is something to
95 for example read, what does an application do?
96
97 With our current API, you have to loop over all the channels and read from
98 them to see if they have data. This effectively makes blocking reads
99 impossible. If the app has many channels in a setup like this, it even becomes
100 slow. (The original API had the libssh2_poll_channel_read() and libssh2_poll()
101 to somewhat overcome this hurdle, but they too have pretty much the same
102 problems plus a few others.)
103
104 Traffic in the other direction is similarly limited: the app has to try
105 sending to all channels, even though some of them may very well not accept any
106 data at that point.
107
108 A SOLUTION
109
110 I suggest we introduce two new helper functions:
111
112 libssh2_transport_read()
113
114 - Read "a bunch" of data from the given socket and returns information to the
115 app about what channels that are now readable (ie they will not block when
116 read from). The function can be called over and over and it will repeatedly
117 return info about what channels that are readable at that moment.
118
119 libssh2_transport_write()
120
121 - Returns information about what channels that are writable, in the sense
122 that they have windows set from the remote side that allows data to get
123 sent. Writing to one of those channels will not block. Of course, the
124 underlying socket may only accept a certain amount of data, so at the first
125 short return, nothing more should be attempted to get sent until select()
126 (or equivalent) has been used on the master socket again.
127
128 I have not yet figured out a sensible API for how these functions should return
129 that info, but if we agree on the general principles I guess we can work that
130 out.
131
132 VOLUNTARY
133
134 I wanted to mention that these two helper functions would not be mandatory
135 in any way. They would just be there for those who want them, and existing
136 programs can remain using the old functions only if they prefer to.
137
138 New SFTP API
139 ============
140
141 PURPOSE
142
143 Provide API functions that explicitly tells at once that a (full) SFTP file
144 transfer is wanted, to allow libssh2 to leverage on that knowledge to speed
145 up things internally. It can for example do read ahead, buffer writes (merge
146 small writes into larger chunks), better tune the SSH window and more. This
147 sort of API is already provided for SCP transfers.
148
149 API
150
151 New functions:
152
153 LIBSSH2_SFTP_HANDLE *libssh2_sftp_send(SFTP_SESSION *sftp,
154 libssh2_uint64_t filesize,
155 char *remote_path,
156 size_t remote_path_len,
157 long mode);
158
159 Tell libssh2 that a local file with a given size is about to get sent to
160 the SFTP server.
161
162 LIBSSH2_SFTP_HANDLE *libssh2_sftp_recv();
163
164 Tell libssh2 that a remote file is requested to get downloaded from the SFTP
165 server.
166
167 Only the setup of the file transfer is different from an application's point
168 of view. Depending on direction of the transfer(s), the following already
169 existing functions should then be used until the transfer is complete:
170
171 libssh2_sftp_read()
172 libssh2_sftp_write()
173
174 HOW TO USE
175
176 1. Setup the transfer using one of the two new functions.
177
178 2. Loop through the reading or writing of data.
179
180 3. Cleanup the transfer