Debian Codegraph @

doc/HOWTO-precompile.adoc

raw
1 = Precompile (a.k.a. ahead-of-time compile) Scheme code
2
3 NOTE: This is a feature still in development and very likely
4 change over time.
5 Especially, we usually keep ABI compatibility across micro versions
6 so that you don't need to recompile extensions for every new
7 release, but we don't guarantee that for precompiled binary yet --
8 if you precompile your library to distribute, let users know
9 that they need to recompile it for new Gauche releases, even if
10 it's just a micro version update.
11
12 As a script engine, the default operating mode of Gauche is to read
13 Scheme source code at run-time, compile it on-the-fly and execute.
14 However, that incurs some overhead of loading and compiling, and
15 you might want to cut the time if you have tons of Scheme code.
16
17 Gauche has a feature to compile Scheme code and dump the result
18 as C code, which can be compiled by a C compiler into DSO and dlopen-ed
19 at runtime. At this moment, we don't really "compile Scheme to C" --
20 what we do is to compile Scheme into Gauche's VM instruction sequence,
21 as we do at run-time, and to dump the result as a C static data structure.
22 Thus run-time performance won't improve by precompilation, but you do
23 cut the compile time, including macro expansion, and the parsing time.
24 In future, we may improve performance of precompiled code, for
25 we can spend more time on optimization.
26
27 == Things to consider before precompiling
28
29 1. The precompiler needs to make sense of source statically (i.e. without
30 executing it), so we recommend the source to follow certain constraints:
31 +
32 ** Each scm source begins with a `define-module` form.
33 ** One scm source defines one module.
34
35 2. In the normal operation, Gauche compiles and executes each
36 toplevel form. It allows you to use a value computed at
37 run-time to be used in the macro expansion of the subsequent
38 toplevel forms in the same file. It doesn't work in precompilation,
39 since macro expansion precedes execution. This limitation
40 includes procedure definitions. For example:
41 +
42 [source,scheme]
43 ----
44 (define (quote-it x) `(quote ,x))
45
46 (define-syntax quote-self
47 (er-macro-transformer
48 (^[f r c] (quote-it f))))
49
50 (define (test)
51 (quote-self a b c))
52 ----
53 +
54 This works by loading the source, but won't work in precompilation,
55 since the expansion of `quote-self` requires to call `quote-it`,
56 which is only available at run-time.
57 +
58 You need to split the file, one contains the values the macro
59 expander depends, and another contains the forms that uses the
60 macro. The macro definition itself can be in either file.
61 (If the macro expander does not depend on the runtime value
62 computed in the same file, you can put the macro definition
63 and macro use in the same file.)
64
65 3. The `cond-expand` macro is handy to switch architecture-dependent
66 code. Since it is a macro, it's expanded at the precompile time,
67 so the features are selected based on the platform you're
68 precompiling, not the platform you actually run the program on.
69 That means you can't simply precompile a Scheme file and carry over
70 the generated C file to other platform, if the Scheme file needs
71 to select architecture-dependent part with cond-expand. C code
72 itself is portable, but the unportable selection is to be done
73 before C file is created.
74 +
75 If you need to distribute generated C code and you need to use
76 `cond-expand`, just extract the part that uses `cond-expand` into
77 a Scheme file and keep it from being precompiled.
78
79
80 == The first step
81
82 Let's begin with the simplest case. Precompiling a single Scheme file.
83
84 Suppose we have `greet.scm`:
85
86 [source,scheme]
87 ---------------------------------
88 (define-module greet
89 (export greeting))
90 (select-module greet)
91
92 (define *message* '("Hello, " ". How are you?"))
93
94 (define (greeting who)
95 #"~(car *message*)~|who|~(cadr *message*)")
96 ---------------------------------
97
98 Run `tools/precomp` subcommand as follows. The `-e` option is important -- that
99 tells `precomp` to create a C file suitable for Gauche extension module
100 on its own.
101
102 [source,console]
103 ----
104 $ gosh tools/precomp -e greet.scm
105 ----
106
107 Now you see two files are generated: `greet.c` and `greet.sci`
108
109 [source,console]
110 ----
111 $ ls
112 greet.c greet.sci greet.scm
113 ----
114
115 The `.sci` suffix stands for **SC**heme **I**nterface, which contains a small
116 Scheme code fragment for module definitions, plus forms that can't be
117 compiled to C, and the call to dynamic-load that loads DSO file:
118
119 [source,console]
120 ----
121 $ cat greet.sci
122 ;; generated automatically. DO NOT EDIT
123 #!no-fold-case
124 (define-module greet (export greeting))
125 (select-module greet)
126 (dynamic-load "greet")
127 ----
128
129 Precompiled DSOs are always loaded via its interface file -- it resolves
130 module dependency. That means you have to distribute the interface
131 file along the DSO. You don't usually see `+*.sci+`, though, since
132 `gauche-install` command can rename `+*.sci+` to `+*.scm+` while it copies
133 the file (with `-C` option). We use `+*.sci+` suffix only because we
134 want to be able to generate the file in the same directory as
135 the source `+*.scm+` file.
136
137 Next, you can compile `greet.c` to generate greet.so (or `greet.dylib`
138 or `greet.dll`, depending on your platform):
139
140 [source,console]
141 ----
142 $ gauche-package compile greet greet.c
143 $ ls
144 greet.c greet.o greet.sci greet.scm greet.so
145 ----
146
147 `gauche-package compile` command knows which C compiler and flags
148 are used when Gauche is compiled, where the Gauche header files are,
149 etc. You have to compile the `+*.c+` file with the same compiler and
150 same `CFLAGS`, otherwise Gauche can't safely dlopen the resulting DSO.
151 You can give extra flags to the `gauche-package compile`; just type
152 this command with no arguments and it tells the usage.
153
154 Now, let's try the precompiled module. We give `-I.` option to `gosh`
155 so that it can find `greet.sci` and `greet.so`. We also give `-fload-verbose`
156 option to check if it really loads the precompiled file, not the
157 source file:
158
159 [source,console]
160 ----
161 $ gosh -I. -fload-verbose
162 ;;Loading /usr/share/gauche-0.9/0.9.5/lib/gauche/interactive.scm...
163 ;; Loading /usr/share/gauche-0.9/0.9.5/lib/gauche/common-macros.scm...
164 ;; Loading /usr/share/gauche-0.9/0.9.5/lib/gauche/defvalues.scm...
165 gosh> (use greet)
166 ;;Loading ./greet.sci...
167 ;; Dynamically Loading ./greet.so...
168 gosh> (greeting "John")
169 "Hello, John. How are you?"
170 ----
171
172 Woohoo! Note that `+*.sci+` takes precedence over `+*.scm+`, so even we have
173 both `greet.sci` and `greet.scm`, gosh loads `greet.sci` (and subsequently,
174 `greet.so`).
175
176 In practice, you have to install `+*.sci+` file and `+*.so+` file to
177 appropriate places so that other's code can find them. The standard
178 places can be queried to with `gauche-config` command:
179
180 [source,console]
181 ----
182 $ gauche-config --sitelibdir
183 /usr/share/gauche-0.9/site/lib
184 $ gauche-config --sitearchdir
185 /usr/lib/gauche-0.9/site/x86_64-pc-linux-gnu
186 ----
187
188 You can use gauche-install to copy the files:
189
190 [source,console]
191 ----
192 $ gauche-install -T `gauche-config --sitelibdir` -C greet.sci
193 $ gauche-install -T `gauche-config --sitearchdir` greet.so
194 ----
195
196 The `-C` option renames `greet.sci` to `greet.scm` in the target directory.
197
198 (If you use the template makefile generated by `gauche-package generate`,
199 it has the skeleton to set up the installation destination.)
200
201
202 == Multiple source files
203
204 The previous section shows converting one Scheme source into one DSO.
205 However, usually you want to make one DSO per library, which may
206 contain lots of source files.
207
208 Suppose you have the following source code structure, and `foo.scm`
209 uses sub files as `foo.boo` and `foo.woo`.
210
211 ----
212 foo.scm
213 foo/boo.scm
214 foo/woo.scm
215 ----
216
217 You have to generate C file for each Scheme source, then compile and
218 link all together.
219
220 [source,console]
221 ----
222 $ gosh tools/precomp -I. -e foo.scm foo/boo.scm foo/woo.scm
223 $ ls
224 foo/ foo--boo.c foo--woo.c foo.c foo.sci foo.scm
225 $ ls foo
226 boo.sci boo.scm woo.sci woo.scm
227 ----
228
229 Note the `-I.` option, which allows `precomp` to find
230 `foo/util1.scm` and `foo/util2.scm` referred from `foo.scm`.
231
232 You see C files are generated in the current directory, while
233 SCI files are on the side of each file under the subdirectory.
234
235 Now you can compile those C files into a single DSO:
236
237 [source,console]
238 ----
239 $ gauche-package compile foo foo.c foo--boo.c foo--woo.c
240 $ ls
241 foo/ foo--boo.o foo--woo.o foo.o foo.scm
242 foo--boo.c foo--woo.c foo.c foo.sci foo.so
243 ----
244
245 You still have an SCI file per each Scheme source, but there's only
246 one DSO and that can be loaded at once.
247
248
249 == But I don't want a bunch of SCI files!
250
251 The reason we have one `+*.sci+` file per one `+*.scm+` file is to guarantee
252 the consitent behavior between source form and precompiled form.
253 With keeping the interface file in the same relative path as the source,
254 we can guarantee the library user can say not only `(use foo)`,
255 but also `(use foo.boo)` and `(use foo.woo)`. The latter two still
256 work, since we have `foo/boo.sci` and `foo/woo.sci`, with appropriate
257 initialization in it.
258
259 However, if you have hundreds of source files and you *know*
260 the external user will only use the toplevel module and not
261 directly `use` the submodules, then you can consolidate those
262 SCI files into single one. Just give `--single-interface`
263 option to precomp.
264
265 [source,console]
266 ----
267 $ gosh tools/precomp -I. -e --single-interface foo.scm foo/boo.scm foo/woo.scm
268 $ ls
269 foo/ foo--boo.c foo--woo.c foo.c foo.sci foo.scm
270 $ ls foo
271 boo.scm woo.scm
272 ----
273
274 Having this in `foo.sci`, you only need it and `foo.so` for the library
275 to work:
276
277 [source,console]
278 ----
279 $ gosh -I. -fload-verbose
280 ;;Loading /usr/share/gauche-0.9/0.9.5/lib/gauche/interactive.scm...
281 ;; Loading /usr/share/gauche-0.9/0.9.5/lib/gauche/common-macros.scm...
282 ;; Loading /usr/share/gauche-0.9/0.9.5/lib/gauche/defvalues.scm...
283 gosh> (use foo)
284 ;;Loading ./foo.sci...
285 ;; Dynamically Loading ./foo.so...
286 ----