Debian Codegraph @

doc/web/man/include_server_1.html

raw
1 <HTML>
2 <!-- this file was generated by troffcvt and tc2html -->
3 <HEAD>
4 <TITLE>
5 include_server manual page
6 </TITLE>
7 </HEAD>
8 <BODY>
9 <H1>
10 include_server manual page<BR>
11 </H1>
12 <P>
13 <H2>
14 Table of Contents
15 </H2>
16 <!-- INSERT TOC HERE, MAYBE -->
17 <!-- TOC BEGIN -->
18 <UL>
19 <LI>
20 <A HREF=#TOC_1> NAME</A>
21 <LI>
22 <A HREF=#TOC_2> SYNOPSIS</A>
23 <LI>
24 <A HREF=#TOC_3> DESCRIPTION</A>
25 <LI>
26 <A HREF=#TOC_4> OPTION SUMMARY</A>
27 <LI>
28 <A HREF=#TOC_5> INCLUDE SERVER SYMPTOMS AND ISSUES</A>
29 <LI>
30 <A HREF=#TOC_6> DISTCC DISCREPANCY SYMPTOMS</A>
31 <LI>
32 <A HREF=#TOC_7> EXIT CODES</A>
33 <LI>
34 <A HREF=#TOC_8> ENVIRONMENT VARIABLES</A>
35 <LI>
36 <A HREF=#TOC_9> BUGS</A>
37 <LI>
38 <A HREF=#TOC_10> AUTHOR</A>
39 <LI>
40 <A HREF=#TOC_11> LICENCE</A>
41 <LI>
42 <A HREF=#TOC_12> SEE ALSO</A>
43 </UL>
44 <!-- TOC END -->
45 <BR>
46 <H2>
47 <A NAME=TOC_1>
48 NAME</A>
49 </H2>
50 include_server.py - conservative approximation of include dependencies
51 for C/C++<BR>
52 <H2>
53 <A NAME=TOC_2>
54 SYNOPSIS</A>
55 </H2>
56 <B>include_server</B> --port <I> INCLUDE_SERVER_PORT [OPTIONS]</I><BR>
57 <H2>
58 <A NAME=TOC_3>
59 DESCRIPTION</A>
60 </H2>
61 include_server.py starts an include server process. This process
62 answers queries from <B>distcc</B>(1) clients about what files
63 to include in C/C++ compilations. The include_server.py command
64 itself terminates as soon as the include server has been spawned.<BR>
65 <P>
66 The INCLUDE_SERVER_PORT argument is the name of a socket used
67 for all communication between distcc clients and the include server.
68 The <B>pump</B>(1) command is responsible for creating the socket
69 location, for passing it to this script, and for passing it to
70 all distcc clients via the environment variable named INCLUDE_SERVER_PORT.<BR>
71 <P>
72 The protocol used by the include server uses distcc's RPC implementation.
73 Each distcc request consists of (1) the current directory and
74 (2) the list of arguments of the compilation command.<BR>
75 <P>
76 If the include server is able to process the request, then it
77 answers the distcc client by sending a list of filepaths. The
78 filepaths are those of the compressed source and header files
79 found to be necessary for compilation through include analysis.
80 The list also comprises symbolic links and even dummy files needed
81 for the compilation server to construct an accurate replica of
82 the parts of the filesystem needed for compilation. In this way,
83 a needed header file like /path/foo.h is compressed, renamed,
84 and stored in a temporary location, such as /dev/shm/tmpiAvfGv.include_server-9368-1/path/foo.h.lzo.
85 The distcc client will pass these files on to a compilation server,
86 where they will be uncompressed and mounted temporarily.<BR>
87 <P>
88 If the include server is not able to process the request, then
89 it returns the empty list to the distcc client.<BR>
90 <P>
91 There are two kinds of failures that relate to the include server.
92 The include server may fail to compute the includes or fail in
93 other ways, see section <B>INCLUDE SERVER SYMPTOMS</B>. Also,
94 the compilation on the remove server may fail due to inadequacy
95 of the calculated include closure, but then succeed when locally
96 retried, see section <B>DISTCC DISCREPANCY SYMPTOMS</B>.<BR>
97 <H2>
98 <A NAME=TOC_4>
99 OPTION SUMMARY</A>
100 </H2>
101 The following options are understood by include_server.py.<BR>
102 <DL>
103 <DT>
104 <B>-dPAT, --debug_pattern=PAT</B>
105 </DT>
106 <DD>
107 Bit vector for turning on warnings and debugging<BR>
108 1 = warnings<BR>
109 2 = trace some functions<BR>
110 other powers of two: see include_server/basics.py.<BR>
111 </DD>
112 <DT>
113 <B>-e, --email</B>
114 </DT>
115 <DD>
116 Send email to 'distcc-pump-errors' or if defined, the value of
117 environment variable DISTCC_EMAILLOG_WHOM_TO_BLAME, when include
118 server gets in trouble. The default is to not send email.<BR>
119 </DD>
120 <DT>
121 <B>--email_bound NUMBER</B>
122 </DT>
123 <DD>
124 Maximal number of emails to send (in addition to a final email).
125 Default: 3.<BR>
126 </DD>
127 <DT>
128 <B>--no-email</B>
129 </DT>
130 <DD>
131 Do not send email. This is the default.<BR>
132 </DD>
133 <DT>
134 <B>--path_observation_re=RE</B>
135 </DT>
136 <DD>
137 Issue warning message whenever a filename is resolved to a realpath
138 that is matched by RE, which is a regular expression in Python
139 syntax. This is useful for finding out where files included actually
140 come from. Use RE=&quot;&quot; to find them all. Note: warnings
141 must be enabled with at least -d1.<BR>
142 </DD>
143 <DT>
144 <B>--pid_file FILEPATH</B>
145 </DT>
146 <DD>
147 The pid of the include server is written to file FILEPATH. This
148 allows a script such a <B>pump</B> to tear down the include server.<BR>
149 </DD>
150 <DT>
151 <B>-s, --statistics</B>
152 </DT>
153 <DD>
154 Print information to stdout about include analysis.<BR>
155 </DD>
156 <DT>
157 <B>--stat_reset_triggers=LIST</B>
158 </DT>
159 <DD>
160 Flush stat caches when the timestamp of any filepath in LIST changes
161 or the filepath comes in or out of existence. LIST is a colon
162 separated string of filepaths, possibly containing simple globs
163 (as allowed by Python's glob module). Print a warning whenever
164 such a change happens (if warnings are enabled). This option allows
165 limited exceptions to distcc_pump's normal assumption that source
166 files are not modified during the build.<BR>
167 </DD>
168 <DT>
169 <B>-t, --time</B>
170 </DT>
171 <DD>
172 Print elapsed, user, and system time to stderr.<BR>
173 </DD>
174 <DT>
175 <B>--unsafe_absolute_includes</B>
176 </DT>
177 <DD>
178 Do preprocessing on the compilation server even if includes of
179 absolute filepaths are encountered. Normally the include-server
180 will fall back on local preprocessing if it detects any absolute
181 includes. Thus, this flag is useful for preventing such fallbacks
182 when the absolute includes are a false alarm, either because the
183 absolute include is discarded during preprocessing or because
184 the absolutely included file exists on the compilation servers.<BR>
185 </DD>
186 <DT>
187 <BR>
188 </DT>
189 <DD>
190 More precisely, with --unsafe_absolute_includes absolute includes
191 are ignored for the purposes of gathering the include closure.
192 Using this option may lead to incorrect results because (1) the
193 header may actually be included on the compilation server and
194 it may not be the same as on the client, (2) the include directives
195 of the header are not further analyzed.<BR>
196 </DD>
197 <DT>
198 <BR>
199 </DT>
200 <DD>
201 The option is useful for compiling code that has such hardcoded
202 absolute locations of header files inside conditional directives
203 (e.g. &quot;#ifdef&quot;) that render the includes irrelevant.
204 More precisely, these includes must be eliminated during preprocessing
205 for the actual configuration. Then the question of existence of
206 the header file is moot and the remote compilation is sound. This
207 is often the case if such includes are meant for unusual configurations
208 different from the actual configuration.<BR>
209 </DD>
210 <DT>
211 <B>-v, --verify</B>
212 </DT>
213 <DD>
214 Verify that files in CPP closure are contained in closure calculated
215 by include processor.<BR>
216 </DD>
217 <DT>
218 <B>-w, --write_include_closure</B>
219 </DT>
220 <DD>
221 Write a .d_approx file which lists all the included files calculated
222 by the include server; with -x, additionally write the included
223 files as calculated by CPP to a .d_exact file.<BR>
224 </DD>
225 <DT>
226 <B>-x, --exact_analysis</B>
227 </DT>
228 <DD>
229 Use CPP instead, do not omit system headers files.<BR>
230 </DD>
231 </DL>
232 <H2>
233 <A NAME=TOC_5>
234 INCLUDE SERVER SYMPTOMS AND ISSUES</A>
235 </H2>
236 <BR>
237 The most likely messages and warnings to come from the include
238 processor are listed below.<BR>
239 <P>
240 <DL>
241 <DT>
242 <B>Preprocessing locally. Include server not covering: Couldn't
243 determine default system include directories</B>
244 </DT>
245 <DD>
246 To determine the default system header directories, the include
247 server runs the compiler once for each language needed during
248 its session. This message indicates that the compiler specified
249 to distcc is not present on the client.<BR>
250 </DD>
251 </DL>
252 <P>
253 <DL>
254 <DT>
255 <B>Preprocessing locally. Include server not covering: Bailing
256 out because include server spent more than ...s user time handling
257 request</B>
258 </DT>
259 <DD>
260 In uncommon situations, the include server fails to analyze very
261 complicated macro expressions. The distcc client will use plain
262 distcc mode.<BR>
263 </DD>
264 </DL>
265 <P>
266 <DL>
267 <DT>
268 <B>Warning: Filepath must be relative but isn't</B>
269 </DT>
270 <DD>
271 The include server does not accept absolute filepaths, such as
272 /usr/include/stdio.h, in include directives, because there is
273 no guarantee that this header on the compilation server machine
274 will be the same as that on the client. The include server gives
275 up analyzing the include closure. The distcc client cannot use
276 pump-mode.<BR>
277 </DD>
278 <DT>
279 <BR>
280 </DT>
281 <DD>
282 To overcome this problem in a not always reliable way, set the
283 environment variable INCLUDE_SERVER_ARGS='--unsafe_absolute_includes'
284 when invoking the pump script to pass the --unsafe_absolute_includes
285 option to the include server.<BR>
286 </DD>
287 </DL>
288 <P>
289 <DL>
290 <DT>
291 <B>Warning: Absolute filepath ... was IGNORED</B>
292 </DT>
293 <DD>
294 The --unsafe_absolute_includes is in use. This situation happens
295 under the same circumstances as when &quot;Filepath must be relative
296 but isn't&quot; is issued, but in this case the include will provide
297 an answer to the distcc client.<BR>
298 </DD>
299 </DL>
300 <P>
301 <DL>
302 <DT>
303 <B>Warning: Path '/PATH/FILE' changed/came into existence/no longer
304 exists</B>
305 </DT>
306 <DD>
307 These warnings are issued when using stat reset triggers. Because
308 /PATH/FILE changed, the include server clears its caches; the
309 new version of the file (or the lack of it) renders the include
310 analysis invalid. This message can usually be ignored; it does
311 signify a somewhat precarious use of files by the build system.
312 It is recommended to fix the build system so that files are not
313 rewritten.<BR>
314 </DD>
315 </DL>
316 <P>
317 <DL>
318 <DT>
319 <B>Warning: For translation unit ..., lookup of file ... resolved
320 to ... whose realpath is ...</B>
321 </DT>
322 <DD>
323 This warning occurs with --path_observation_re when a new realpath
324 matching a source or header file is observed.<BR>
325 </DD>
326 </DL>
327 <H2>
328 <A NAME=TOC_6>
329 DISTCC DISCREPANCY SYMPTOMS</A>
330 </H2>
331 <BR>
332 The interactions between the build system, distcc, and the include
333 server is somewhat complex. When a distcc commands receives a
334 failing compilation from the remote server it retries the compilation
335 locally. This section discusses the causes of discrepancies between
336 remote and local compilation. These are flagged by the demotion
337 message:<BR>
338 <P>
339 <B>__________Warning: ... pump-mode compilation(s) failed on server,</B>
340 <B>but succeeded locally.</B><BR>
341 <B>__________Distcc-pump was demoted to plain mode.</B> <B> See
342 the Distcc Discrepancy Symptoms section in the include_server(1)
343 man</B> <B> page.</B><BR>
344 <P>
345 The pump script issues this message at the end of the build. This
346 means that for at least one distcc invocation a local compilation
347 succeeded after the remote compilation failed. Each distcc invocation
348 for which such a discrepancy occurred in turn also issues a message
349 such as:<BR>
350 <P>
351 <B>Warning: remote compilation of '...' failed,</B> <B> retried
352 locally and got a different result.</B><BR>
353 <P>
354 The demotion makes subsequent distcc invocations use plain distcc
355 mode. Thus preprocessing will take place on the local machine
356 for the remainder of the build. This technique prevents very
357 slow builds where all compilations end up on the local machine
358 after failing remotely.<BR>
359 <P>
360 Of course, if the local compilations fails after the remote failures,
361 then the distcc invocation exits with the non-zero status of the
362 local compilation. The error messages printed are also those of
363 the local compilation.<BR>
364 <P>
365 The fallback behavior for distcc-pump mode to local compilation
366 can be disabled by setting the environment variable DISTCC_FALLBACK
367 to 0, which makes the distcc command fail as soon as the remote
368 compilation has failed. This setting is very useful for debugging
369 why the remote compilation went wrong, because now the output
370 from the server will be printed.<BR>
371 <P>
372 Next we discuss the possible causes of discrepancies.<BR>
373 <P>
374 <DL>
375 <DT>
376 <B>The user changed a source or header file during the build.</B>
377 </DT>
378 <DD>
379 This yields inconsistent results of course.<BR>
380 </DD>
381 </DL>
382 <P>
383 <DL>
384 <DT>
385 <B>A source or header file changed during the build.</B>
386 </DT>
387 <DD>
388 The build system rewrites a file. For Linux kernel 2.6, this
389 happens for 'include/linux/compile.h' and 'include/asm/asm-offsets.h'.
390 This condition is fixed by letting the include server know that
391 it must reset its caches when a stat of any of the files changes.
392 Practically, this is done by gathering the files in a colon-separated
393 list and then setting the INCLUDE_SERVER_ARGS environment variable
394 when invoking the pump script, so that it passes the <B> --stat_reset_triggers</B>
395 option; for example,<BR>
396 INCLUDE_SERVER_ARGS=&quot;--stat_reset_triggers=include/linux/compile.h:include/asm/asm-offsets.h&quot;<BR>
397 </DD>
398 </DL>
399 <P>
400 <DL>
401 <DT>
402 <B>A header file is potentially included, but does not exist,
403 and is then later included.</B>
404 </DT>
405 <DD>
406 This occurs when some header foo.h includes another header file
407 trick.h, but the trick.h file has not yet been generated and the
408 inclusion is actually ignored because of preprocessing directives.
409 The include server will probe for the existence of trick.h, because
410 it overapproximates all possible ways directives actually evaluate.
411 The file trick.h is determined not to exist. If it is later generated,
412 and then really included, then the include server will falsely
413 believe that the file still does not exist. The solution to this
414 problem is to make the build system generate trick.h before the
415 first time any header file is included that makes a syntactic
416 reference to trick.h<BR>
417 </DD>
418 </DL>
419 <P>
420 <DL>
421 <DT>
422 <B>The include server was started with --unsafe_absolute_includes</B>.<B>
423 </DT>
424 <DD>
425 This is a problem if there are header files locally that do not
426 exist remotely and that are actually used. Such includes are
427 often protected by conditional directives that evaluate so that
428 are actually used on only specific and often uncommon platforms.
429 If you are not compiling for such a platform, then it may be correct
430 to use --unsafe_absolute_include</B>.<BR>
431 </DD>
432 </DL>
433 <P>
434 <DL>
435 <DT>
436 <B>The include server has calculated the wrong includes.</B>
437 </DT>
438 <DD>
439 We do not know of such a situation.<BR>
440 </DD>
441 </DL>
442 <H2>
443 <A NAME=TOC_7>
444 EXIT CODES</A>
445 </H2>
446 The exit code of include_server.py is usually 0. That the include
447 server has been started properly is communicated through the existence
448 of the pid_file.<BR>
449 <H2>
450 <A NAME=TOC_8>
451 ENVIRONMENT VARIABLES</A>
452 </H2>
453 <BR>
454 <B>DISTCC_EMAILLOG_WHOM_TO_BLAME</B> The email address to use
455 for include server automated emails. The default is 'distcc-pump-errors'
456 (which is an email address that probably will not exist in your
457 domain).<BR>
458 <P>
459 Additionally, the invocation of the compiler may use additional
460 environment variables.<BR>
461 <H2>
462 <A NAME=TOC_9>
463 BUGS</A>
464 </H2>
465 If you think you have found a distcc bug, please see the file
466 <I>reporting-bugs.txt</I> in the documentation directory for information
467 on how to report it.<BR>
468 <P>
469 In distcc-pump mode, the include server is unable to handle certain
470 very complicated computed includes as found in parts of the Boost
471 library. The include server will time out and distcc will revert
472 to plain mode.<BR>
473 <P>
474 Other known bugs may be documented on <I> http://code.google.com/p/distcc/</I><BR>
475 <H2>
476 <A NAME=TOC_10>
477 AUTHOR</A>
478 </H2>
479 The include server was written by Nils Klarlund, with assistance
480 from Fergus Henderson, Manos Renieris, and Craig Silverstein.
481 Please report bugs to &lt;[email protected]&gt;.<BR>
482 <H2>
483 <A NAME=TOC_11>
484 LICENCE</A>
485 </H2>
486 You are free to use distcc. distcc (including this manual) may
487 be copied, modified or distributed only under the terms of the
488 GNU General Public Licence version 2 or later. distcc comes with
489 absolutely no warrany. A copy of the GPL is included in the file
490 COPYING.<BR>
491 <H2>
492 <A NAME=TOC_12>
493 SEE ALSO</A>
494 </H2>
495 <B>distcc</B>(1), <B>distccd</B>(1), <B>include_server</B>(1),
496 and <B>gcc</B>(1). http://code.google.com/p/distcc/ https://ccache.dev/<BR>
497 </BODY>
498 </HTML>