| 1 |
= Precompile (a.k.a. ahead-of-time compile) Scheme code |
| 2 |
|
| 3 |
NOTE: This is a feature still in development and very likely |
| 4 |
change over time. |
| 5 |
Especially, we usually keep ABI compatibility across micro versions |
| 6 |
so that you don't need to recompile extensions for every new |
| 7 |
release, but we don't guarantee that for precompiled binary yet -- |
| 8 |
if you precompile your library to distribute, let users know |
| 9 |
that they need to recompile it for new Gauche releases, even if |
| 10 |
it's just a micro version update. |
| 11 |
|
| 12 |
As a script engine, the default operating mode of Gauche is to read |
| 13 |
Scheme source code at run-time, compile it on-the-fly and execute. |
| 14 |
However, that incurs some overhead of loading and compiling, and |
| 15 |
you might want to cut the time if you have tons of Scheme code. |
| 16 |
|
| 17 |
Gauche has a feature to compile Scheme code and dump the result |
| 18 |
as C code, which can be compiled by a C compiler into DSO and dlopen-ed |
| 19 |
at runtime. At this moment, we don't really "compile Scheme to C" -- |
| 20 |
what we do is to compile Scheme into Gauche's VM instruction sequence, |
| 21 |
as we do at run-time, and to dump the result as a C static data structure. |
| 22 |
Thus run-time performance won't improve by precompilation, but you do |
| 23 |
cut the compile time, including macro expansion, and the parsing time. |
| 24 |
In future, we may improve performance of precompiled code, for |
| 25 |
we can spend more time on optimization. |
| 26 |
|
| 27 |
== Things to consider before precompiling |
| 28 |
|
| 29 |
1. The precompiler needs to make sense of source statically (i.e. without |
| 30 |
executing it), so we recommend the source to follow certain constraints: |
| 31 |
+ |
| 32 |
** Each scm source begins with a `define-module` form. |
| 33 |
** One scm source defines one module. |
| 34 |
|
| 35 |
2. In the normal operation, Gauche compiles and executes each |
| 36 |
toplevel form. It allows you to use a value computed at |
| 37 |
run-time to be used in the macro expansion of the subsequent |
| 38 |
toplevel forms in the same file. It doesn't work in precompilation, |
| 39 |
since macro expansion precedes execution. This limitation |
| 40 |
includes procedure definitions. For example: |
| 41 |
+ |
| 42 |
[source,scheme] |
| 43 |
---- |
| 44 |
(define (quote-it x) `(quote ,x)) |
| 45 |
|
| 46 |
(define-syntax quote-self |
| 47 |
(er-macro-transformer |
| 48 |
(^[f r c] (quote-it f)))) |
| 49 |
|
| 50 |
(define (test) |
| 51 |
(quote-self a b c)) |
| 52 |
---- |
| 53 |
+ |
| 54 |
This works by loading the source, but won't work in precompilation, |
| 55 |
since the expansion of `quote-self` requires to call `quote-it`, |
| 56 |
which is only available at run-time. |
| 57 |
+ |
| 58 |
You need to split the file, one contains the values the macro |
| 59 |
expander depends, and another contains the forms that uses the |
| 60 |
macro. The macro definition itself can be in either file. |
| 61 |
(If the macro expander does not depend on the runtime value |
| 62 |
computed in the same file, you can put the macro definition |
| 63 |
and macro use in the same file.) |
| 64 |
|
| 65 |
3. The `cond-expand` macro is handy to switch architecture-dependent |
| 66 |
code. Since it is a macro, it's expanded at the precompile time, |
| 67 |
so the features are selected based on the platform you're |
| 68 |
precompiling, not the platform you actually run the program on. |
| 69 |
That means you can't simply precompile a Scheme file and carry over |
| 70 |
the generated C file to other platform, if the Scheme file needs |
| 71 |
to select architecture-dependent part with cond-expand. C code |
| 72 |
itself is portable, but the unportable selection is to be done |
| 73 |
before C file is created. |
| 74 |
+ |
| 75 |
If you need to distribute generated C code and you need to use |
| 76 |
`cond-expand`, just extract the part that uses `cond-expand` into |
| 77 |
a Scheme file and keep it from being precompiled. |
| 78 |
|
| 79 |
|
| 80 |
== The first step |
| 81 |
|
| 82 |
Let's begin with the simplest case. Precompiling a single Scheme file. |
| 83 |
|
| 84 |
Suppose we have `greet.scm`: |
| 85 |
|
| 86 |
[source,scheme] |
| 87 |
--------------------------------- |
| 88 |
(define-module greet |
| 89 |
(export greeting)) |
| 90 |
(select-module greet) |
| 91 |
|
| 92 |
(define *message* '("Hello, " ". How are you?")) |
| 93 |
|
| 94 |
(define (greeting who) |
| 95 |
#"~(car *message*)~|who|~(cadr *message*)") |
| 96 |
--------------------------------- |
| 97 |
|
| 98 |
Run `tools/precomp` subcommand as follows. The `-e` option is important -- that |
| 99 |
tells `precomp` to create a C file suitable for Gauche extension module |
| 100 |
on its own. |
| 101 |
|
| 102 |
[source,console] |
| 103 |
---- |
| 104 |
$ gosh tools/precomp -e greet.scm |
| 105 |
---- |
| 106 |
|
| 107 |
Now you see two files are generated: `greet.c` and `greet.sci` |
| 108 |
|
| 109 |
[source,console] |
| 110 |
---- |
| 111 |
$ ls |
| 112 |
greet.c greet.sci greet.scm |
| 113 |
---- |
| 114 |
|
| 115 |
The `.sci` suffix stands for **SC**heme **I**nterface, which contains a small |
| 116 |
Scheme code fragment for module definitions, plus forms that can't be |
| 117 |
compiled to C, and the call to dynamic-load that loads DSO file: |
| 118 |
|
| 119 |
[source,console] |
| 120 |
---- |
| 121 |
$ cat greet.sci |
| 122 |
;; generated automatically. DO NOT EDIT |
| 123 |
#!no-fold-case |
| 124 |
(define-module greet (export greeting)) |
| 125 |
(select-module greet) |
| 126 |
(dynamic-load "greet") |
| 127 |
---- |
| 128 |
|
| 129 |
Precompiled DSOs are always loaded via its interface file -- it resolves |
| 130 |
module dependency. That means you have to distribute the interface |
| 131 |
file along the DSO. You don't usually see `+*.sci+`, though, since |
| 132 |
`gauche-install` command can rename `+*.sci+` to `+*.scm+` while it copies |
| 133 |
the file (with `-C` option). We use `+*.sci+` suffix only because we |
| 134 |
want to be able to generate the file in the same directory as |
| 135 |
the source `+*.scm+` file. |
| 136 |
|
| 137 |
Next, you can compile `greet.c` to generate greet.so (or `greet.dylib` |
| 138 |
or `greet.dll`, depending on your platform): |
| 139 |
|
| 140 |
[source,console] |
| 141 |
---- |
| 142 |
$ gauche-package compile greet greet.c |
| 143 |
$ ls |
| 144 |
greet.c greet.o greet.sci greet.scm greet.so |
| 145 |
---- |
| 146 |
|
| 147 |
`gauche-package compile` command knows which C compiler and flags |
| 148 |
are used when Gauche is compiled, where the Gauche header files are, |
| 149 |
etc. You have to compile the `+*.c+` file with the same compiler and |
| 150 |
same `CFLAGS`, otherwise Gauche can't safely dlopen the resulting DSO. |
| 151 |
You can give extra flags to the `gauche-package compile`; just type |
| 152 |
this command with no arguments and it tells the usage. |
| 153 |
|
| 154 |
Now, let's try the precompiled module. We give `-I.` option to `gosh` |
| 155 |
so that it can find `greet.sci` and `greet.so`. We also give `-fload-verbose` |
| 156 |
option to check if it really loads the precompiled file, not the |
| 157 |
source file: |
| 158 |
|
| 159 |
[source,console] |
| 160 |
---- |
| 161 |
$ gosh -I. -fload-verbose |
| 162 |
;;Loading /usr/share/gauche-0.9/0.9.5/lib/gauche/interactive.scm... |
| 163 |
;; Loading /usr/share/gauche-0.9/0.9.5/lib/gauche/common-macros.scm... |
| 164 |
;; Loading /usr/share/gauche-0.9/0.9.5/lib/gauche/defvalues.scm... |
| 165 |
gosh> (use greet) |
| 166 |
;;Loading ./greet.sci... |
| 167 |
;; Dynamically Loading ./greet.so... |
| 168 |
gosh> (greeting "John") |
| 169 |
"Hello, John. How are you?" |
| 170 |
---- |
| 171 |
|
| 172 |
Woohoo! Note that `+*.sci+` takes precedence over `+*.scm+`, so even we have |
| 173 |
both `greet.sci` and `greet.scm`, gosh loads `greet.sci` (and subsequently, |
| 174 |
`greet.so`). |
| 175 |
|
| 176 |
In practice, you have to install `+*.sci+` file and `+*.so+` file to |
| 177 |
appropriate places so that other's code can find them. The standard |
| 178 |
places can be queried to with `gauche-config` command: |
| 179 |
|
| 180 |
[source,console] |
| 181 |
---- |
| 182 |
$ gauche-config --sitelibdir |
| 183 |
/usr/share/gauche-0.9/site/lib |
| 184 |
$ gauche-config --sitearchdir |
| 185 |
/usr/lib/gauche-0.9/site/x86_64-pc-linux-gnu |
| 186 |
---- |
| 187 |
|
| 188 |
You can use gauche-install to copy the files: |
| 189 |
|
| 190 |
[source,console] |
| 191 |
---- |
| 192 |
$ gauche-install -T `gauche-config --sitelibdir` -C greet.sci |
| 193 |
$ gauche-install -T `gauche-config --sitearchdir` greet.so |
| 194 |
---- |
| 195 |
|
| 196 |
The `-C` option renames `greet.sci` to `greet.scm` in the target directory. |
| 197 |
|
| 198 |
(If you use the template makefile generated by `gauche-package generate`, |
| 199 |
it has the skeleton to set up the installation destination.) |
| 200 |
|
| 201 |
|
| 202 |
== Multiple source files |
| 203 |
|
| 204 |
The previous section shows converting one Scheme source into one DSO. |
| 205 |
However, usually you want to make one DSO per library, which may |
| 206 |
contain lots of source files. |
| 207 |
|
| 208 |
Suppose you have the following source code structure, and `foo.scm` |
| 209 |
uses sub files as `foo.boo` and `foo.woo`. |
| 210 |
|
| 211 |
---- |
| 212 |
foo.scm |
| 213 |
foo/boo.scm |
| 214 |
foo/woo.scm |
| 215 |
---- |
| 216 |
|
| 217 |
You have to generate C file for each Scheme source, then compile and |
| 218 |
link all together. |
| 219 |
|
| 220 |
[source,console] |
| 221 |
---- |
| 222 |
$ gosh tools/precomp -I. -e foo.scm foo/boo.scm foo/woo.scm |
| 223 |
$ ls |
| 224 |
foo/ foo--boo.c foo--woo.c foo.c foo.sci foo.scm |
| 225 |
$ ls foo |
| 226 |
boo.sci boo.scm woo.sci woo.scm |
| 227 |
---- |
| 228 |
|
| 229 |
Note the `-I.` option, which allows `precomp` to find |
| 230 |
`foo/util1.scm` and `foo/util2.scm` referred from `foo.scm`. |
| 231 |
|
| 232 |
You see C files are generated in the current directory, while |
| 233 |
SCI files are on the side of each file under the subdirectory. |
| 234 |
|
| 235 |
Now you can compile those C files into a single DSO: |
| 236 |
|
| 237 |
[source,console] |
| 238 |
---- |
| 239 |
$ gauche-package compile foo foo.c foo--boo.c foo--woo.c |
| 240 |
$ ls |
| 241 |
foo/ foo--boo.o foo--woo.o foo.o foo.scm |
| 242 |
foo--boo.c foo--woo.c foo.c foo.sci foo.so |
| 243 |
---- |
| 244 |
|
| 245 |
You still have an SCI file per each Scheme source, but there's only |
| 246 |
one DSO and that can be loaded at once. |
| 247 |
|
| 248 |
|
| 249 |
== But I don't want a bunch of SCI files! |
| 250 |
|
| 251 |
The reason we have one `+*.sci+` file per one `+*.scm+` file is to guarantee |
| 252 |
the consitent behavior between source form and precompiled form. |
| 253 |
With keeping the interface file in the same relative path as the source, |
| 254 |
we can guarantee the library user can say not only `(use foo)`, |
| 255 |
but also `(use foo.boo)` and `(use foo.woo)`. The latter two still |
| 256 |
work, since we have `foo/boo.sci` and `foo/woo.sci`, with appropriate |
| 257 |
initialization in it. |
| 258 |
|
| 259 |
However, if you have hundreds of source files and you *know* |
| 260 |
the external user will only use the toplevel module and not |
| 261 |
directly `use` the submodules, then you can consolidate those |
| 262 |
SCI files into single one. Just give `--single-interface` |
| 263 |
option to precomp. |
| 264 |
|
| 265 |
[source,console] |
| 266 |
---- |
| 267 |
$ gosh tools/precomp -I. -e --single-interface foo.scm foo/boo.scm foo/woo.scm |
| 268 |
$ ls |
| 269 |
foo/ foo--boo.c foo--woo.c foo.c foo.sci foo.scm |
| 270 |
$ ls foo |
| 271 |
boo.scm woo.scm |
| 272 |
---- |
| 273 |
|
| 274 |
Having this in `foo.sci`, you only need it and `foo.so` for the library |
| 275 |
to work: |
| 276 |
|
| 277 |
[source,console] |
| 278 |
---- |
| 279 |
$ gosh -I. -fload-verbose |
| 280 |
;;Loading /usr/share/gauche-0.9/0.9.5/lib/gauche/interactive.scm... |
| 281 |
;; Loading /usr/share/gauche-0.9/0.9.5/lib/gauche/common-macros.scm... |
| 282 |
;; Loading /usr/share/gauche-0.9/0.9.5/lib/gauche/defvalues.scm... |
| 283 |
gosh> (use foo) |
| 284 |
;;Loading ./foo.sci... |
| 285 |
;; Dynamically Loading ./foo.so... |
| 286 |
---- |