Debian Codegraph @

doc/web/benchmark.html

raw
1 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"
2 "http://www.w3.org/TR/REC-html40/loose.dtd">
3 <html>
4 <head><title>benchmark results for distcc</title>
5 <link rel="SHORTCUT ICON" href="favicon.ico">
6 <link type="text/css" rel="stylesheet" href="distcc-green.css">
7 <link type="text/html" rel="top" href="index.html">
8 <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"></head>
9 <body vlink="#003300" text="#000000" link="#003300" bgcolor="#ffffff" alink="#336633"><p><table width="100%"><tr><td valign="top"><div class="navbar"><ul>
10 <li><a href="index.html">distcc</a></li>
11 <li><a href="http://code.google.com/p/distcc/source/browse/trunk">source</a>
12 <li><a href="http://code.google.com/p/distcc/downloads/list">binaries</a></li>
13 <li><a href="http://distcc.googlecode.com/svn/trunk/NEWS">news</a></li>
14 <li><a href="scenarios.html">scenarios</a></li>
15 <li><a href="results.html">testimonials</a></li>
16 <li><a href="benchmark.html"><b>benchmarks</b></a></li>
17 <li><a href="security.html">security</a></li>
18 <li><a href="distcc-lca-2004.html">whitepaper</a></li>
19 <li><a href="http://distcc.googlecode.com/svn/trunk/doc/web/man/">Man pages</a></li>
20 <li><a href="faq.html">FAQ</a></li>
21
22 <p>
23 <li><a href="http://code.google.com/p/distcc/issues/list">report a bug</a></li>
24 <li><a href="https://lists.samba.org/mailman/listinfo/distcc">mailing list</a>
25 <ul><li><a href="mailto:[email protected]">post</a></li>
26 <li><a href="http://lists.samba.org/archive/distcc/">archive</a></li>
27 <li><a href="news://news.gmane.org/gmane.comp.compilers.distcc">newsgroup</a></li></ul>
28 </li>
29 <li><a href="http://freshmeat.net/projects/distcc">Freshmeat</a>
30 <ul><li><a href="http://freshmeat.net/subscribe/28140">be notified</a></li></ul>
31 </li>
32
33 <p>
34 <li>related:</li>
35 <ul>
36 <li><a title="How to use distcc and Gentoo" href="http://www.gentoo.org/doc/en/distcc.xml">Gentoo</a></li>
37 <li><a title="Script to ease building cross-compilers and toolchains" href="http://kegel.com/crosstool/">crosstool</a></li>
38 <li><a title="Cache compiler results" href="https://ccache.dev/">ccache</a></li>
39 <li><a title="Centralized control over distcc and ccache" href="http://ccontrol.ozlabs.org/">ccontrol</a></li>
40 <li><a title="Smart job scheduler" href="http://dmucs.sourceforge.net/">dmucs</a></li>
41 </ul>
42 <p><p></td><td valign="top">
43
44 <div class="body"><h1 class="title">benchmark results for distcc</h1>
45
46 <p>The following table shows the effect of distcc on several
47 real-world compilation tasks; in this case, compiling various
48 open-source projects. In each case, we compiled in four modes:</p>
49 <ol>
50 <li> <b>local_01</b>: all compilation done locally
51 <li> <b>dist_h38_j40</b>: using distcc with 38 compilation servers (152
52 CPUs), and <code>make -j40</code>, which gives 40 parallel
53 compilations. distcc is used in non-pump mode (local preprocessing).
54 <li> <b>pump_h38_j40</b>: like <code>dist_h38_j40</code>, but using
55 distcc in pump (remote preprocessing) mode.
56 <li> <b>pump_h38_j80</b>: like <code>pump_h38_j40</code>, but using
57 <code>make -j80</code> instead of <code>make -j40</code>.
58 </ol>
59
60 <p>The client machine is an 2.8GHz Intel Pentium 4 machine, with 2G of
61 memory, running Linux 2.6.18.5 (modified Ubuntu 6.06). The server
62 machines were all 4 CPU 2GHz AMD Dual Core Processor 270 machines,
63 with 2G of memory, running Linux 2.6.18.5 (modified Ubuntu 6.06).</p>
64
65 <p>Each test was run 5 times. We report the average time over the 5
66 runs, as well as the standard deviation. In order to minimize disk
67 caching effects on walltime results, we read all the files in each
68 project's tarball before building.</p>
69
70
71 <table width=100%>
72 <tr valign=bottom align=right><th align=left>Project</th><th align=left>mode</th>
73 <th>wall<br>time</th><th>std<br>dev</th><th>cpu<br>time</th><th>cpu<br>%</th>
74 <tr><td colspan=6><hr></td></tr>
75
76 <tr align=right><td align=left>binutils-2.18</td><td align=left>local_01 </td>
77 <td>125.8s</td> <td>4.0s</td> <td>102.9s</td> <td>81.9%</td></tr>
78 <tr align=right><td align=left>binutils-2.18</td><td align=left>dist_h38_j40</td>
79 <td>35.4s</td> <td>1.0s</td> <td>25.6s</td> <td>72.2%</td></tr>
80 <tr align=right><td align=left>binutils-2.18</td><td align=left>pump_h38_j40</td>
81 <td>28.7s</td> <td>1.5s</td> <td>16.6s</td> <td>57.7%</td></tr>
82 <tr align=right><td align=left>binutils-2.18</td><td align=left>pump_h38_j80</td>
83 <td>29.8s</td> <td>1.5s</td> <td>16.6s</td> <td>55.9%</td></tr>
84 <tr><td colspan=6><hr></td></tr>
85
86 <tr align=right><td align=left>glibc-2.6 </td><td align=left>local_01 </td>
87 <td>946.4s</td> <td>5.1s</td> <td>568.7s</td> <td>60.1%</td></tr>
88 <tr align=right><td align=left>glibc-2.6 </td><td align=left>dist_h38_j40</td>
89 <td>534.0s</td> <td>7.0s</td> <td>390.5s</td> <td>74.7%</td></tr>
90 <tr align=right><td align=left>glibc-2.6 </td><td align=left>pump_h38_j40</td>
91 <td>357.8s</td> <td>6.5s</td> <td>241.6s</td> <td>67.5%</td></tr>
92 <tr align=right><td align=left>glibc-2.6 </td><td align=left>pump_h38_j80</td>
93 <td>365.4s</td> <td>4.2s</td> <td>243.0s</td> <td>66.6%</td></tr>
94 <tr><td colspan=6><hr></td></tr>
95
96 <tr align=right><td align=left>hello-2.1.1 </td><td align=left>local_01 </td>
97 <td>0.7s</td> <td>0.1s</td> <td>0.4s</td> <td>62.8%</td></tr>
98 <tr align=right><td align=left>hello-2.1.1 </td><td align=left>dist_h38_j40</td>
99 <td>0.9s</td> <td>0.2s</td> <td>0.3s</td> <td>40.3%</td></tr>
100 <tr align=right><td align=left>hello-2.1.1 </td><td align=left>pump_h38_j40</td>
101 <td>1.1s</td> <td>0.1s</td> <td>0.4s</td> <td>31.3%</td></tr>
102 <tr align=right><td align=left>hello-2.1.1 </td><td align=left>pump_h38_j80</td>
103 <td>1.1s</td> <td>0.0s</td> <td>0.4s</td> <td>31.6%</td></tr>
104 <tr><td colspan=6><hr></td></tr>
105
106 <tr align=right><td align=left>httpd-2.0.43</td><td align=left>local_01 </td>
107 <td>139.8s</td> <td>1.4s</td> <td>117.2s</td> <td>83.9%</td></tr>
108 <tr align=right><td align=left>httpd-2.0.43</td><td align=left>dist_h38_j40</td>
109 <td>85.1s</td> <td>2.1s</td> <td>51.3s</td> <td>60.4%</td></tr>
110 <tr align=right><td align=left>httpd-2.0.43</td><td align=left>pump_h38_j40</td>
111 <td>80.1s</td> <td>2.0s</td> <td>33.7s</td> <td>42.1%</td></tr>
112 <tr align=right><td align=left>httpd-2.0.43</td><td align=left>pump_h38_j80</td>
113 <td>81.7s</td> <td>4.1s</td> <td>33.7s</td> <td>41.4%</td></tr>
114 <tr><td colspan=6><hr></td></tr>
115
116 <tr align=right><td align=left>linux-2.6.25</td><td align=left>local_01 </td>
117 <td>818.3s</td> <td>7.9s</td> <td>543.9s</td> <td>66.5%</td></tr>
118 <tr align=right><td align=left>linux-2.6.25</td><td align=left>dist_h38_j40</td>
119 <td>203.2s</td> <td>1.7s</td> <td>185.3s</td> <td>91.2%</td></tr>
120 <tr align=right><td align=left>linux-2.6.25</td><td align=left>pump_h38_j40</td>
121 <td>134.9s</td> <td>5.3s</td> <td>96.5s</td> <td>71.6%</td></tr>
122 <tr align=right><td align=left>linux-2.6.25</td><td align=left>pump_h38_j80</td>
123 <td>135.6s</td> <td>4.3s</td> <td>97.7s</td> <td>72.1%</td></tr>
124 <tr><td colspan=6><hr></td></tr>
125
126 <tr align=right><td align=left>samba-3.0.20</td><td align=left>local_01 </td>
127 <td>314.0s</td> <td>2.4s</td> <td>258.8s</td> <td>82.4%</td></tr>
128 <tr align=right><td align=left>samba-3.0.20</td><td align=left>dist_h38_j40</td>
129 <td>101.8s</td> <td>0.6s</td> <td>94.0s</td> <td>92.3%</td></tr>
130 <tr align=right><td align=left>samba-3.0.20</td><td align=left>pump_h38_j40</td>
131 <td>31.4s</td> <td>2.4s</td> <td>21.1s</td> <td>67.4%</td></tr>
132 <tr align=right><td align=left>samba-3.0.20</td><td align=left>pump_h38_j80</td>
133 <td>31.8s</td> <td>1.7s</td> <td>21.2s</td> <td>66.9%</td></tr>
134 <tr><td colspan=6><hr></td></tr>
135
136 </table>
137
138 <h2><a name="discussion" href="#discussion">Discussion</a></h2>
139
140 <ul>
141
142 <li> For all but the smallest projects, <code>distcc</code> results in a
143 significant speedup in compilation time over compiling locally. This is
144 because distcc is able to compile many source files in parallel. Such
145 parallelism may not possible for small projects such as
146 <code>hello</code>, which involves four small compilations.
147 <br><br></li>
148
149 <li> In almost all cases, distcc's new pump mode results in a significant
150 speedup in compilation time over non-pump mode. Sometimes, as for
151 Samba 3.0, the speedup is more than a factor of three!<br><br></li>
152
153 <li> The parallelism of the various projects' Makefiles affects
154 the obtained speedup significantly. Makefiles that
155 run <code>make</code> sequentially in subdirectories benefit less from
156 distcc. They will see little added benefit from distcc's pump mode
157 because the sequentiality of their execution allows only some,
158 but not many, compilations to be issued near
159 simultaneously. <br><br></li>
160
161 <li> All these opensource projects are built by running
162 <code>configure</code> plus <code>make</code>. In each case,
163 we count only the <code>make</code> time, not the
164 <code>configure</code> time. However, projects such as
165 binutils run extra configuration during the <code>make</code>
166 step (for binutils, the initial <code>configure</code> run is
167 trivial, and the <code>make</code> command does more intensive
168 configuration in various subdirectories before building). This
169 will affect times as described above -- especially reducing the
170 benefit of pump mode -- since the configuration
171 steps are not run in parallel. <br><br></li>
172
173 <li> Even when pump mode does not speed up the build much, as for
174 <code>httpd</code>, it still reduces the CPU burden on the
175 local machine, making it more usable during compiles. Note,
176 however, that this is balanced by an increased CPU burden on
177 the server machines (about 10% in our tests), and may also
178 require more memory on the host machine than non-pump mode
179 does. <br><br></li>
180
181 <li> With a multi-processor client machine, the speed-ups would have been
182 less, both for non-pump distcc over local compilation as for
183 distcc-pump over non-pump distcc. Still, with four-processor client
184 machines, distcc's pump-mode is up to 2 1/2 times faster than its
185 non-pump mode for large projects benchmarked at Google.<br><br></li>
186
187 <li> As a side note, while collecting benchmark results, we found sometimes
188 that pump mode did not give the expected speedup. On analyzing the
189 logs, we discovered the reason: that distcc had encountered an error
190 running the test in pump mode, and had fallen back to plain distcc
191 mode. This can happen for
192 <a href="man/include_server_1.html#TOC_5">several
193 reasons</a>. For example, the Linux kernel needed special attention
194 because it rewrites header files during the build.
195 </ul>
196
197 </td>
198 </tr></table>
199
200 <div class="thanks">Copyright &copy; 2008 Google Inc.<br>
201 distcc is a trademark of Martin Pool.
202
203 <p>Send comments to <tt>distcc(at)lists.samba.org</tt>.
204
205 </body>
206 </html>