| 1 |
# Arm performance test |
| 2 |
|
| 3 |
Performance differences in build systems become more apparent on |
| 4 |
slower platforms. To examine this difference we compared the |
| 5 |
performance of Meson with GNU Autotools. We took the GLib software |
| 6 |
project and rewrote its build setup with Meson. GLib was chosen |
| 7 |
because it is a relatively large C code base which requires lots of |
| 8 |
low level configuration. |
| 9 |
|
| 10 |
The Meson version of the build system is not fully equivalent to the |
| 11 |
original Autotools one. It does not do all the same configuration |
| 12 |
steps and does not build all the same targets. The biggest missing |
| 13 |
piece being internationalisation support with Gettext. However it does |
| 14 |
configure the system enough to build all C source and run all unit |
| 15 |
tests. |
| 16 |
|
| 17 |
All measurements were done on a Nexus 4 smart phone running the latest |
| 18 |
Ubuntu touch image (updated on September 9th 2013). |
| 19 |
|
| 20 |
Measurements |
| 21 |
------ |
| 22 |
|
| 23 |
The first thing we measured was the time it took to run the configure step. |
| 24 |
|
| 25 |
 |
| 26 |
|
| 27 |
Meson takes roughly 20 seconds whereas Autotools takes 220. This is a |
| 28 |
difference of one order of magnitude. Autotools' time contains both |
| 29 |
autogen and configure. Again it should be remembered that Meson does |
| 30 |
not do all the configure steps that Autotools does. It does do about |
| 31 |
90% of them and it takes just 10% of the time to do it. |
| 32 |
|
| 33 |
Then we measured the build times. Two parallel compilation processes |
| 34 |
were used for both systems. |
| 35 |
|
| 36 |
 |
| 37 |
|
| 38 |
On desktop machines Ninja based build systems are 10-20% faster than |
| 39 |
Make based ones. On this platform the difference grows to 50%. The |
| 40 |
difference is probably caused by Make's inefficient disk access |
| 41 |
patterns. Ninja is better at keeping both cores running all the time |
| 42 |
which yields impressive performance improvements. |
| 43 |
|
| 44 |
 |
| 45 |
|
| 46 |
Next we measured the "empty build" case. That is, how long does it |
| 47 |
take for the build system to detect that no changes need to be |
| 48 |
made. This is one of the most important metrics of build systems |
| 49 |
because it places a hard limit on how fast you can iterate on your |
| 50 |
code. Autotools takes 14 seconds to determine that no work needs to be |
| 51 |
done. Meson (or, rather, Ninja) takes just one quarter of a second. |
| 52 |
|
| 53 |
 |
| 54 |
|
| 55 |
One step which takes quite a lot of time is linking. A common case is |
| 56 |
that you are working on a library and there are tens of small test |
| 57 |
executables that link to it. Even if the compilation step would be |
| 58 |
fast, relinking all of the test executables takes time. It is common |
| 59 |
for people to manually compile only one test application with a |
| 60 |
command such as `make sometest` rather than rebuild everything. |
| 61 |
|
| 62 |
Meson has an optimization for this case. Whenever a library is |
| 63 |
rebuilt, Meson inspects the ABI it exports. If it has not changed, |
| 64 |
Meson will skip all relinking steps as unnecessary. The difference |
| 65 |
this makes can be clearly seen in the chart above. In that test the |
| 66 |
source was fully built, then the file `glib/gbytes.c` was touched to |
| 67 |
force the rebuild of the base glib shared library. As can be seen, |
| 68 |
Autotools then relinks all test executables that link with glib. Since |
| 69 |
Meson can detect that the ABI is the same it can skip those steps. The |
| 70 |
end result being that Meson is almost one hundred times faster on this |
| 71 |
very common use case. |
| 72 |
|
| 73 |
Conclusions |
| 74 |
----- |
| 75 |
|
| 76 |
One of the main drawbacks of C and C++ compared to languages such as |
| 77 |
Java are long compilation times. However at least some of the blame |
| 78 |
can be found in the build tools used rather than the languages |
| 79 |
themselves or their compilers. Choosing proper tools can bring C and |
| 80 |
C++ compilation very close to instantaneous rebuilds. This has a |
| 81 |
direct impact on programmer productivity. |