| 1 |
# gRPC Completion Queue |
| 2 |
|
| 3 |
_Author: Sree Kuchibhotla (@sreecha) - Sep 2018_ |
| 4 |
|
| 5 |
Code: [completion_queue.cc](https://github.com/grpc/grpc/blob/v1.15.1/src/core/lib/surface/completion_queue.cc) |
| 6 |
|
| 7 |
This document gives an overview of completion queue architecture and focuses mainly on the interaction between completion queue and the Polling engine layer. |
| 8 |
|
| 9 |
## Completion queue attributes |
| 10 |
Completion queue has two attributes |
| 11 |
|
| 12 |
- Completion_type: |
| 13 |
- GRPC_CQ_NEXT: grpc_completion_queue_next() can be called (but not grpc_completion_queue_pluck()) |
| 14 |
- GRPC_CQ_PLUCK: grpc_completion_queue_pluck() can be called (but not grpc_completion_queue_next()) |
| 15 |
- GRPC_CQ_CALLBACK: The tags in the queue are function pointers to callbacks. Also, neither next() nor pluck() can be called on this |
| 16 |
|
| 17 |
- Polling_type: |
| 18 |
- GRPC_CQ_NON_POLLING: Threads calling completion_queue_next/pluck do not do any polling |
| 19 |
- GRPC_CQ_DEFAULT_POLLING: Threads calling completion_queue_next/pluck do polling |
| 20 |
- GRPC_CQ_NON_LISTENING: Functionally similar to default polling except for a boolean attribute that states that the cq is non-listening. This is used by the grpc-server code to not associate any listening sockets with this completion-queue’s pollset |
| 21 |
|
| 22 |
|
| 23 |
## Details |
| 24 |
|
| 25 |
 |
| 26 |
|
| 27 |
|
| 28 |
### **grpc\_completion\_queue\_next()** & **grpc_completion_queue_pluck()** APIS |
| 29 |
|
| 30 |
|
| 31 |
``` C++ |
| 32 |
grpc_completion_queue_next(cq, deadline)/pluck(cq, deadline, tag) { |
| 33 |
while(true) { |
| 34 |
\\ 1. If an event is queued in the completion queue, dequeue and return |
| 35 |
\\ (in case of pluck() dequeue only if the tag is the one we are interested in) |
| 36 |
|
| 37 |
\\ 2. If completion queue shutdown return |
| 38 |
|
| 39 |
\\ 3. In case of pluck, add (tag, worker) pair to the tag<->worker map on the cq |
| 40 |
|
| 41 |
\\ 4. Call grpc_pollset_work(cq’s-pollset, deadline) to do polling |
| 42 |
\\ Note that if this function found some fds to be readable/writable/error, |
| 43 |
\\ it would have scheduled those closures (which may queue completion events |
| 44 |
\\ on SOME completion queue - not necessarily this one) |
| 45 |
} |
| 46 |
} |
| 47 |
``` |
| 48 |
|
| 49 |
### Queuing a completion event (i.e., "tag") |
| 50 |
|
| 51 |
``` C++ |
| 52 |
grpc_cq_end_op(cq, tag) { |
| 53 |
\\ 1. Queue the tag in the event queue |
| 54 |
|
| 55 |
\\ 2. Find the pollset corresponding to the completion queue |
| 56 |
\\ (i) If the cq is of type GRPC_CQ_NEXT, then KICK ANY worker |
| 57 |
\\ i.e., call grpc_pollset_kick(pollset, nullptr) |
| 58 |
\\ (ii) If the cq is of type GRPC_CQ_PLUCK, then search the tag<->worker |
| 59 |
\\ map on the completion queue to find the worker. Then specifically |
| 60 |
\\ kick that worker i.e call grpc_pollset_kick(pollset, worker) |
| 61 |
} |
| 62 |
|
| 63 |
``` |
| 64 |
|