| 1 |
This file describes the RESTful compile-server API. The API looks like |
| 2 |
the following: |
| 3 |
|
| 4 |
Method URI DESCRIPTION |
| 5 |
POST /builds create build |
| 6 |
GET /builds/123 retrieve build info |
| 7 |
DELETE /builds/123 delete build |
| 8 |
GET /results/789 retrieve result info |
| 9 |
GET /results/789/FILE retrieve result files |
| 10 |
|
| 11 |
All the individual item numbers in the URIs will be UUIDs. |
| 12 |
|
| 13 |
You'll notice that DELETE is missing from most the above. For now |
| 14 |
anyway, we'll let the server decide when to delete the |
| 15 |
containers. Also note that the server will decide when a new |
| 16 |
container needs to be created (based on the build id information). |
| 17 |
|
| 18 |
Also note at this point that we're not going to authenticate |
| 19 |
users. We're going to leave that level of authentication to a |
| 20 |
front-end reverse-proxy server, rather than worry about it ourselves. |
| 21 |
|
| 22 |
How are we going to handle the perhaps long creation time, if we |
| 23 |
have to create a new docker container for instance? It would look |
| 24 |
like this: |
| 25 |
|
| 26 |
client: POST /builds (all the parameters, kernel version, |
| 27 |
arch, etc.) |
| 28 |
server: 202 Accepted - URI of queued build task (like /builds/456) |
| 29 |
|
| 30 |
Note that the server could return an error here, especially if the |
| 31 |
architectures don't match. Also note that there could be a GET needed |
| 32 |
here on the returned URI to upload a script file (if that wasn't done |
| 33 |
in the POST data). |
| 34 |
|
| 35 |
So then how does the client know when the queued build task is |
| 36 |
finished? It polls the queued build task: |
| 37 |
|
| 38 |
client: GET /builds/456 |
| 39 |
server: 200 OK (along with buildid details) |
| 40 |
|
| 41 |
Eventually, the client will get the following: |
| 42 |
|
| 43 |
client: GET /builds/456 |
| 44 |
server: 303 See Other - along with the URI of the result info. |
| 45 |
|
| 46 |
The client would then retrieve the result info, which will list all |
| 47 |
the files produced. |
| 48 |
|
| 49 |
After retrieving the module (using GET), the client can delete the |
| 50 |
buildid (at this point the server would also delete the linked |
| 51 |
result files). |
| 52 |
|
| 53 |
client: DELETE /builds/456 |
| 54 |
server: 300 OK |
| 55 |
|
| 56 |
One semi-tricky server note. Let's say a job comes in to build a |
| 57 |
module for kernel X. While building a container for kernel X, a |
| 58 |
second request comes in to build a module for kernel X. The server |
| 59 |
should reuse the in-progress container from the first build, not |
| 60 |
try to build a 2nd container for the same kernel. This implies that |
| 61 |
the containers should contain a reference count. That information |
| 62 |
should be persistent across server shutdown and startup, so we'll |
| 63 |
need a small database. |
| 64 |
|
| 65 |
FIXME: at some point we'll need to add other stuff the current |
| 66 |
client/server code supports, like certificates used for module |
| 67 |
signing, MOK signing, etc. |
| 68 |
|
| 69 |
The server could support several possible "back ends": |
| 70 |
|
| 71 |
- LOCAL: use the kernel devel environment(s) on the local |
| 72 |
machine. If nothing else, this could be used for testing the |
| 73 |
server. The server wouldn't add any more kernel devel |
| 74 |
environments here, but if the devel environment for kernel X is |
| 75 |
already installed here, it seems odd to build a container. |
| 76 |
- MOCK: build a mock "container" to build the module in |
| 77 |
- DOCKER: build a docker container to build the module in |
| 78 |
- VAGRANT: build a vagrant container to build the module in |
| 79 |
|
| 80 |
The user wouldn't be bothered with picking from these, the server will |
| 81 |
decide which one to use based on which one matches up with the server |
| 82 |
system. In other words, if the docker application isn't present the |
| 83 |
server shouldn't try to use it. |
| 84 |
|
| 85 |
The server will also need to "garbage collect" at some point, |
| 86 |
making sure the we only keep X containers around and deleting ones |
| 87 |
that haven't been used recently. |
| 88 |
|
| 89 |
Here's an example run: |
| 90 |
|
| 91 |
- The client sends a POST /builds request: |
| 92 |
|
| 93 |
POST /builds kver=4.12.0-0.rc3.git0.2.fc27.x86_64&arch=x86_64&cmd_args=-vp4&cmd_args=-e&cmd_args=probe+begin+%7B+exit%28%29+%7D' |
| 94 |
|
| 95 |
The server responds with: |
| 96 |
|
| 97 |
HTTP/1.1 202 Accepted |
| 98 |
Retry-After: 10 |
| 99 |
Location: /builds/ad64 |
| 100 |
|
| 101 |
This tells the client to wait 10 seconds then do a GET on |
| 102 |
/builds/ad64. |
| 103 |
|
| 104 |
- The client does a GET: |
| 105 |
|
| 106 |
GET /builds/ad64 |
| 107 |
|
| 108 |
The server responds with a JSON encoded body: |
| 109 |
|
| 110 |
HTTP/1.1 200 OK |
| 111 |
Location: /builds/ad64 |
| 112 |
Body: { |
| 113 |
"uuid": "ad64", |
| 114 |
"kver": "4.12.0-0.rc3.git0.2.fc27.x86_64", |
| 115 |
"arch": "x86_64", |
| 116 |
"cmd_args": [ |
| 117 |
"-vp4", |
| 118 |
"-e", |
| 119 |
"probe begin { exit() }" |
| 120 |
] |
| 121 |
} |
| 122 |
|
| 123 |
The client keeps doing the same GET until... |
| 124 |
|
| 125 |
- When the build is finished, the server will respond with a 303: |
| 126 |
|
| 127 |
HTTP/1.1 303 See Other |
| 128 |
Location: /results/39ff |
| 129 |
|
| 130 |
The client then does a GET on the /results/URI: |
| 131 |
|
| 132 |
GET /results/39ff |
| 133 |
|
| 134 |
The server responds: |
| 135 |
|
| 136 |
HTTP/1.1 200 OK |
| 137 |
Body: { |
| 138 |
"uuid": "39ff", |
| 139 |
"rc": 0, |
| 140 |
"stdout_location": "/results/39ff/stdout", |
| 141 |
"stderr_location": "/results/39ff/stderr", |
| 142 |
"files": [ |
| 143 |
{ "location": "/results/39ff/stap_91e5911967e16d8c350000d9f1d7de71_1105.ko", "mode": 436 } |
| 144 |
] |
| 145 |
} |
| 146 |
|
| 147 |
- The client can then to a GET on the module itself: |
| 148 |
|
| 149 |
GET /results/39ff/stap_91e5911967e16d8c350000d9f1d7de71_1105.ko |
| 150 |
|
| 151 |
The server responds (along with the module data): |
| 152 |
|
| 153 |
HTTP/1.1 200 OK |
| 154 |
|
| 155 |
- At this point the client could issue a DELETE on /builds/ad64. |