| 1 |
# Transport Explainer |
| 2 |
|
| 3 |
@vjpai |
| 4 |
|
| 5 |
## Existing Transports |
| 6 |
|
| 7 |
[gRPC |
| 8 |
transports](https://github.com/grpc/grpc/tree/master/src/core/ext/transport) |
| 9 |
plug in below the core API (one level below the C++ or other wrapped-language |
| 10 |
API). You can write your transport in C or C++ though; currently (Nov 2017) all |
| 11 |
the transports are nominally written in C++ though they are idiomatically C. The |
| 12 |
existing transports are: |
| 13 |
|
| 14 |
* [HTTP/2](https://github.com/grpc/grpc/tree/master/src/core/ext/transport/chttp2) |
| 15 |
* [Cronet](https://github.com/grpc/grpc/tree/master/src/core/ext/transport/cronet) |
| 16 |
* [In-process](https://github.com/grpc/grpc/tree/master/src/core/ext/transport/inproc) |
| 17 |
|
| 18 |
Among these, the in-process is likely the easiest to understand, though arguably |
| 19 |
also the least similar to a "real" sockets-based transport since it is only used |
| 20 |
in a single process. |
| 21 |
|
| 22 |
## Transport stream ops |
| 23 |
|
| 24 |
In the gRPC core implementation, a fundamental struct is the |
| 25 |
`grpc_transport_stream_op_batch` which represents a collection of stream |
| 26 |
operations sent to a transport. (Note that in gRPC, _stream_ and _RPC_ are used |
| 27 |
synonymously since all RPCs are actually streams internally.) The ops in a batch |
| 28 |
can include: |
| 29 |
|
| 30 |
* send\_initial\_metadata |
| 31 |
- Client: initiate an RPC |
| 32 |
- Server: supply response headers |
| 33 |
* recv\_initial\_metadata |
| 34 |
- Client: get response headers |
| 35 |
- Server: accept an RPC |
| 36 |
* send\_message (zero or more) : send a data buffer |
| 37 |
* recv\_message (zero or more) : receive a data buffer |
| 38 |
* send\_trailing\_metadata |
| 39 |
- Client: half-close indicating that no more messages will be coming |
| 40 |
- Server: full-close providing final status for the RPC |
| 41 |
* recv\_trailing\_metadata: get final status for the RPC |
| 42 |
- Server extra: This op shouldn't actually be considered complete until the |
| 43 |
server has also sent trailing metadata to provide the other side with final |
| 44 |
status |
| 45 |
* cancel\_stream: Attempt to cancel an RPC |
| 46 |
* collect\_stats: Get stats |
| 47 |
|
| 48 |
The fundamental responsibility of the transport is to transform between this |
| 49 |
internal format and an actual wire format, so the processing of these operations |
| 50 |
is largely transport-specific. |
| 51 |
|
| 52 |
One or more of these ops are grouped into a batch. Applications can start all of |
| 53 |
a call's ops in a single batch, or they can split them up into multiple |
| 54 |
batches. Results of each batch are returned asynchronously via a completion |
| 55 |
queue. |
| 56 |
|
| 57 |
Internally, we use callbacks to indicate completion. The surface layer creates a |
| 58 |
callback when starting a new batch and sends it down the filter stack along with |
| 59 |
the batch. The transport must invoke this callback when the batch is complete, |
| 60 |
and then the surface layer returns an event to the application via the |
| 61 |
completion queue. Each batch can have up to 3 callbacks: |
| 62 |
|
| 63 |
* recv\_initial\_metadata\_ready (called by the transport when the |
| 64 |
recv\_initial\_metadata op is complete) |
| 65 |
* recv\_message\_ready (called by the transport when the recv_message op is |
| 66 |
complete) |
| 67 |
* on\_complete (called by the transport when the entire batch is complete) |
| 68 |
|
| 69 |
## Timelines of transport stream op batches |
| 70 |
|
| 71 |
The transport's job is to sequence and interpret various possible interleavings |
| 72 |
of the basic stream ops. For example, a sample timeline of batches would be: |
| 73 |
|
| 74 |
1. Client send\_initial\_metadata: Initiate an RPC with a path (method) and authority |
| 75 |
1. Server recv\_initial\_metadata: accept an RPC |
| 76 |
1. Client send\_message: Supply the input proto for the RPC |
| 77 |
1. Server recv\_message: Get the input proto from the RPC |
| 78 |
1. Client send\_trailing\_metadata: This is a half-close indicating that the |
| 79 |
client will not be sending any more messages |
| 80 |
1. Server recv\_trailing\_metadata: The server sees this from the client and |
| 81 |
knows that it will not get any more messages. This won't complete yet though, |
| 82 |
as described above. |
| 83 |
1. Server send\_initial\_metadata, send\_message, send\_trailing\_metadata: A |
| 84 |
batch can contain multiple ops, and this batch provides the RPC response |
| 85 |
headers, response content, and status. Note that sending the trailing |
| 86 |
metadata will also complete the server's receive of trailing metadata. |
| 87 |
1. Client recv\_initial\_metadata: The number of ops in one side of the batch |
| 88 |
has no relation with the number of ops on the other side of the batch. In |
| 89 |
this case, the client is just collecting the response headers. |
| 90 |
1. Client recv\_message, recv\_trailing\_metadata: Get the data response and |
| 91 |
status |
| 92 |
|
| 93 |
|
| 94 |
There are other possible sample timelines. For example, for client-side streaming, a "typical" sequence would be: |
| 95 |
|
| 96 |
1. Server: recv\_initial\_metadata |
| 97 |
- At API-level, that would be the server requesting an RPC |
| 98 |
1. Server: recv\_trailing\_metadata |
| 99 |
- This is for when the server wants to know the final completion of the RPC |
| 100 |
through an `AsyncNotifyWhenDone` API in C++ |
| 101 |
1. Client: send\_initial\_metadata, recv\_message, recv\_trailing\_metadata |
| 102 |
- At API-level, that's a client invoking a client-side streaming call. The |
| 103 |
send\_initial\_metadata is the call invocation, the recv\_message collects |
| 104 |
the final response from the server, and the recv\_trailing\_metadata gets |
| 105 |
the `grpc::Status` value that will be returned from the call |
| 106 |
1. Client: send\_message / Server: recv\_message |
| 107 |
- Repeat the above step numerous times; these correspond to a client issuing |
| 108 |
`Write` in a loop and a server doing `Read` in a loop until `Read` fails |
| 109 |
1. Client: send\_trailing\_metadata / Server: recv\_message that indicates doneness (NULL) |
| 110 |
- These correspond to a client issuing `WritesDone` which causes the server's |
| 111 |
`Read` to fail |
| 112 |
1. Server: send\_message, send\_trailing\_metadata |
| 113 |
- These correspond to the server doing `Finish` |
| 114 |
|
| 115 |
The sends on one side will call their own callbacks when complete, and they will |
| 116 |
in turn trigger actions that cause the other side's recv operations to |
| 117 |
complete. In some transports, a send can sometimes complete before the recv on |
| 118 |
the other side (e.g., in HTTP/2 if there is sufficient flow-control buffer space |
| 119 |
available) |
| 120 |
|
| 121 |
## Other transport duties |
| 122 |
|
| 123 |
In addition to these basic stream ops, the transport must handle cancellations |
| 124 |
of a stream at any time and pass their effects to the other side. For example, |
| 125 |
in HTTP/2, this triggers a `RST_STREAM` being sent on the wire. The transport |
| 126 |
must perform operations like pings and statistics that are used to shape |
| 127 |
transport-level characteristics like flow control (see, for example, their use |
| 128 |
in the HTTP/2 transport). |
| 129 |
|
| 130 |
## Putting things together with detail: Sending Metadata |
| 131 |
|
| 132 |
* API layer: `map<string, string>` that is specific to this RPC |
| 133 |
* Core surface layer: array of `{slice, slice}` pairs where each slice |
| 134 |
references an underlying string |
| 135 |
* [Core transport |
| 136 |
layer](https://github.com/grpc/grpc/tree/master/src/core/lib/transport): list |
| 137 |
of `{slice, slice}` pairs that includes the above plus possibly some general |
| 138 |
metadata (e.g., Method and Authority for initial metadata) |
| 139 |
* [Specific transport |
| 140 |
layer](https://github.com/grpc/grpc/tree/master/src/core/ext/transport): |
| 141 |
- Either send it to the other side using transport-specific API (e.g., Cronet) |
| 142 |
- Or have it sent through the [iomgr/endpoint |
| 143 |
layer](https://github.com/grpc/grpc/tree/master/src/core/lib/iomgr) (e.g., |
| 144 |
HTTP/2) |
| 145 |
- Or just manipulate pointers to get it from one side to the other (e.g., |
| 146 |
In-process) |
| 147 |
|
| 148 |
## Requirements for any transport |
| 149 |
|
| 150 |
Each transport implements several operations in a vtbl (may change to actual |
| 151 |
virtual functions as transport moves to idiomatic C++). |
| 152 |
|
| 153 |
The most important and common one is `perform_stream_op`. This function |
| 154 |
processes a single stream op batch on a specific stream that is associated with |
| 155 |
a specific transport: |
| 156 |
|
| 157 |
* Gets the 6 ops/cancel passed down from the surface |
| 158 |
* Pass metadata from one side to the other as described above |
| 159 |
* Transform messages between slice buffer structure and stream of bytes to pass |
| 160 |
to other side |
| 161 |
- May require insertion of extra bytes (e.g., per-message headers in HTTP/2) |
| 162 |
* React to metadata to preserve expected orderings (*) |
| 163 |
* Schedule invocation of completion callbacks |
| 164 |
|
| 165 |
There are other functions in the vtbl as well. |
| 166 |
|
| 167 |
* `perform_transport_op` |
| 168 |
- Configure the transport instance for the connectivity state change notifier |
| 169 |
or the server-side accept callback |
| 170 |
- Disconnect transport or set up a goaway for later streams |
| 171 |
* `init_stream` |
| 172 |
- Starts a stream from the client-side |
| 173 |
- (*) Server-side of the transport must call `accept_stream_cb` when a new |
| 174 |
stream is available |
| 175 |
* Triggers request-matcher |
| 176 |
* `destroy_stream`, `destroy_transport` |
| 177 |
- Free up data related to a stream or transport |
| 178 |
* `set_pollset`, `set_pollset_set`, `get_endpoint` |
| 179 |
- Map each specific instance of the transport to FDs being used by iomgr (for |
| 180 |
HTTP/2) |
| 181 |
- Get a pointer to the endpoint structure that actually moves the data |
| 182 |
(wrapper around a socket for HTTP/2) |
| 183 |
|
| 184 |
## Book-keeping responsibilities of the transport layer |
| 185 |
|
| 186 |
A given transport must keep all of its transport and streams ref-counted. This |
| 187 |
is essential to make sure that no struct disappears before it is done being |
| 188 |
used. |
| 189 |
|
| 190 |
A transport must also preserve relevant orders for the different categories of |
| 191 |
ops on a stream, as described above. A transport must also make sure that all |
| 192 |
relevant batch operations have completed before scheduling the `on_complete` |
| 193 |
closure for a batch. Further examples include the idea that the server logic |
| 194 |
expects to not complete recv\_trailing\_metadata until after it actually sends |
| 195 |
trailing metadata since it would have already found this out by seeing a NULL’ed |
| 196 |
recv\_message. This is considered part of the transport's duties in preserving |
| 197 |
orders. |