Debian Codegraph @

subprojects/meson-docs/docs/markdown/Design-rationale.md

raw
1 ---
2 title: Design rationale
3 ...
4
5 This is the original design rationale for Meson. The syntax it
6 describes does not match the released version
7 ==
8
9 A software developer's most important tool is the editor. If you talk
10 to coders about the editors they use, you are usually met with massive
11 enthusiasm and praise. You will hear how Emacs is the greatest thing
12 ever or how vi is so elegant or how Eclipse's integration features
13 make you so much more productive. You can sense the enthusiasm and
14 affection that the people feel towards these programs.
15
16 The second most important tool, even more important than the compiler,
17 is the build system.
18
19 Those are pretty much universally despised.
20
21 The most positive statement on build systems you can usually get (and
22 it might require some coaxing) is something along the lines of *well,
23 it's a terrible system, but all other options are even worse*. It is
24 easy to see why this is the case. For starters, commonly used free
25 build systems have obtuse syntaxes. They use for the most part global
26 variables that are set in random locations so you can never really be
27 sure what a given line of code does. They do strange and unpredictable
28 things at every turn.
29
30 Let's illustrate this with a simple example. Suppose we want to run a
31 program built with GNU Autotools under GDB. The instinctive thing to
32 do is to just run `gdb programname`. The problem is that this may or
33 may not work. In some cases the executable file is a binary whereas at
34 other times it is a wrapper shell script that invokes the real binary
35 which resides in a hidden subdirectory. GDB invocation fails if the
36 binary is a script but succeeds if it is not. The user has to remember
37 the type of each one of his executables (which is an implementation
38 detail of the build system) just to be able to debug them. Several
39 other such pain points can be found in [this blog
40 post](http://voices.canonical.com/jussi.pakkanen/2011/09/13/autotools/).
41
42 Given these idiosyncrasies it is no wonder that most people don't want
43 to have anything to do with build systems. They'll just copy-paste
44 code that works (somewhat) in one place to another and hope for the
45 best. They actively go out of their way not to understand the system
46 because the mere thought of it is repulsive. Doing this also provides
47 a kind of inverse job security. If you don't know tool X, there's less
48 chance of finding yourself responsible for its use in your
49 organisation. Instead you get to work on more enjoyable things.
50
51 This leads to a vicious circle. Since people avoid the tools and don't
52 want to deal with them, very few work on improving them. The result is
53 apathy and stagnation.
54
55 Can we do better?
56 --
57
58 At its core, building C and C++ code is not a terribly difficult
59 task. In fact, writing a text editor is a lot more complicated and
60 takes more effort. Yet we have lots of very high quality editors but
61 only few build systems with questionable quality and usability.
62
63 So, in the grand tradition of own-itch-scratching, I decided to run a
64 scientific experiment. The purpose of this experiment was to explore
65 what would it take to build a "good" build system. What kind of syntax
66 would suit this problem? What sort of problems would this application
67 need to solve? What sort of solutions would be the most appropriate?
68
69 To get things started, here is a list of requirements any modern
70 cross-platform build system needs to provide.
71
72 ### 1. Must be simple to use
73
74 One of the great virtues of Python is the fact that it is very
75 readable. It is easy to see what a given block of code does. It is
76 concise, clear and easy to understand. The proposed build system must
77 be syntactically and semantically clean. Side effects, global state
78 and interrelations must be kept at a minimum or, if possible,
79 eliminated entirely.
80
81 ### 2. Must do the right thing by default
82
83 Most builds are done by developers working on the code. Therefore the
84 defaults must be tailored towards that use case. As an example the
85 system shall build objects without optimization and with debug
86 information. It shall make binaries that can be run directly from the
87 build directory without linker tricks, shell scripts or magic
88 environment variables.
89
90 ### 3. Must enforce established best practices
91
92 There really is no reason to compile source code without the
93 equivalent of `-Wall`. So enable it by default. A different kind of
94 best practice is the total separation of source and build
95 directories. All build artifacts must be stored in the build
96 directory. Writing stray files in the source directory is not
97 permitted under any circumstances.
98
99 ### 4. Must have native support for platforms that are in common use
100
101 A lot of free software projects can be used on non-free platforms such
102 as Windows or OSX. The system must provide native support for the
103 tools of choice on those platforms. In practice this means native
104 support for Visual Studio and XCode. Having said IDEs invoke external
105 builder binaries does not count as native support.
106
107 ### 5. Must not add complexity due to obsolete platforms
108
109 Work on this build system started during the Christmas holidays of 2012.
110 This provides a natural hard cutoff line of 2012/12/24. Any
111 platform, tool or library that was not in active use at that time is
112 explicitly not supported. These include Unixes such as IRIX, SunOS,
113 OSF-1, Ubuntu versions older than 12/10, GCC versions older than 4.7
114 and so on. If these old versions happen to work, great. If they don't,
115 not a single line of code will be added to the system to work around
116 their bugs.
117
118 ### 6. Must be fast
119
120 Running the configuration step on a moderate sized project must not
121 take more than five seconds. Running the compile command on a fully up
122 to date tree of 1000 source files must not take more than 0.1 seconds.
123
124 ### 7. Must provide easy to use support for modern sw development features
125
126 An example is precompiled headers. Currently no free software build
127 system provides native support for them. Other examples could include
128 easy integration of Valgrind and unit tests, test coverage reporting
129 and so on.
130
131 ### 8. Must allow override of default values
132
133 Sometimes you just have to compile files with only given compiler
134 flags and no others, or install files in weird places. The system must
135 allow the user to do this if he really wants to.
136
137 Overview of the solution
138 --
139
140 Going over these requirements it becomes quite apparent that the only
141 viable approach is roughly the same as taken by CMake: having a domain
142 specific language to declare the build system. Out of this declaration
143 a configuration is generated for the backend build system. This can be
144 a Makefile, Visual Studio or XCode project or anything else.
145
146 The difference between the proposed DSL and existing ones is that the
147 new one is declarative. It also tries to work on a higher level of
148 abstraction than existing systems. As an example, using external
149 libraries in current build systems means manually extracting and
150 passing around compiler flags and linker flags. In the proposed system
151 the user just declares that a given build target uses a given external
152 dependency. The build system then takes care of passing all flags and
153 settings to their proper locations. This means that the user can focus
154 on his own code rather than marshalling command line arguments from
155 one place to another.
156
157 A DSL is more work than the approach taken by SCons, which is to
158 provide the system as a Python library. However it allows us to make
159 the syntax more expressive and prevent certain types of bugs by
160 e.g. making certain objects truly immutable. The end result is again
161 the same: less work for the user.
162
163 The backend for Unix requires a bit more thought. The default choice
164 would be Make. However it is extremely slow. It is not uncommon on
165 large code bases for Make to take several minutes just to determine
166 that nothing needs to be done. Instead of Make we use
167 [Ninja](https://ninja-build.org/), which is extremely fast. The
168 backend code is abstracted away from the core, so other backends can
169 be added with relatively little effort.
170
171 Sample code
172 --
173
174 Enough design talk, let's get to the code. Before looking at the
175 examples we would like to emphasize that this is not in any way the
176 final code. It is proof of concept code that works in the system as it
177 currently exists (February 2013), but may change at any time.
178
179 Let's start simple. Here is the code to compile a single executable
180 binary.
181
182 ```meson
183 project('compile one', 'c')
184 executable('program', 'prog.c')
185 ```
186
187 This is about as simple as one can get. First you declare the project
188 name and the languages it uses. Then you specify the binary to build
189 and its sources. The build system will do all the rest. It will add
190 proper suffixes (e.g. '.exe' on Windows), set the default compiler
191 flags and so on.
192
193 Usually programs have more than one source file. Listing them all in
194 the function call can become unwieldy. That is why the system supports
195 keyword arguments. They look like this.
196
197 ```meson
198 project('compile several', 'c')
199 sourcelist = ['main.c', 'file1.c', 'file2.c', 'file3.c']
200 executable('program', sources : sourcelist)
201 ```
202
203 External dependencies are simple to use.
204
205 ```meson
206 project('external lib', 'c')
207 libdep = find_dep('extlibrary', required : true)
208 sourcelist = ['main.c', 'file1.c', 'file2.c', 'file3.c']
209 executable('program', sources : sourcelist, dep : libdep)
210 ```
211
212 In other build systems you have to manually add the compile and link
213 flags from external dependencies to targets. In this system you just
214 declare that extlibrary is mandatory and that the generated program
215 uses that. The build system does all the plumbing for you.
216
217 Here's a slightly more complicated definition. It should still be
218 understandable.
219
220 ```meson
221 project('build library', 'c')
222 foolib = shared_library('foobar', sources : 'foobar.c',\
223 install : true)
224 exe = executable('testfoobar', 'tester.c', link : foolib)
225 add_test('test library', exe)
226 ```
227
228 First we build a shared library named foobar. It is marked
229 installable, so running `meson install` installs it to the library
230 directory (the system knows which one so the user does not have to
231 care). Then we build a test executable which is linked against the
232 library. It will not be installed, but instead it is added to the list
233 of unit tests, which can be run with the command `meson test`.
234
235 Above we mentioned precompiled headers as a feature not supported by
236 other build systems. Here's how you would use them.
237
238 ```meson
239 project('pch demo', 'cpp')
240 executable('myapp', 'myapp.cpp', pch : 'pch/myapp.hh')
241 ```
242
243 The main reason other build systems cannot provide pch support this
244 easily is because they don't enforce certain best practices. Due to
245 the way include paths work, it is impossible to provide pch support
246 that always works with both in-source and out-of-source
247 builds. Mandating separate build and source directories makes this and
248 many other problems a lot easier.
249
250 Get the code
251 --
252
253 The code for this experiment can be found at [the Meson
254 repository](https://github.com/mesonbuild/meson). It should be noted
255 that (at the time of writing) it is not a build system. It is only
256 a proposal for one. It does not work reliably yet. You probably
257 should not use it as the build system of your project.
258
259 All that said I hope that this experiment will eventually turn into a
260 full blown build system. For that I need your help. Comments and
261 especially patches are more than welcome.