= Precompile (a.k.a. ahead-of-time compile) Scheme code NOTE: This is a feature still in development and very likely change over time. Especially, we usually keep ABI compatibility across micro versions so that you don't need to recompile extensions for every new release, but we don't guarantee that for precompiled binary yet -- if you precompile your library to distribute, let users know that they need to recompile it for new Gauche releases, even if it's just a micro version update. As a script engine, the default operating mode of Gauche is to read Scheme source code at run-time, compile it on-the-fly and execute. However, that incurs some overhead of loading and compiling, and you might want to cut the time if you have tons of Scheme code. Gauche has a feature to compile Scheme code and dump the result as C code, which can be compiled by a C compiler into DSO and dlopen-ed at runtime. At this moment, we don't really "compile Scheme to C" -- what we do is to compile Scheme into Gauche's VM instruction sequence, as we do at run-time, and to dump the result as a C static data structure. Thus run-time performance won't improve by precompilation, but you do cut the compile time, including macro expansion, and the parsing time. In future, we may improve performance of precompiled code, for we can spend more time on optimization. == Things to consider before precompiling 1. The precompiler needs to make sense of source statically (i.e. without executing it), so we recommend the source to follow certain constraints: + ** Each scm source begins with a `define-module` form. ** One scm source defines one module. 2. In the normal operation, Gauche compiles and executes each toplevel form. It allows you to use a value computed at run-time to be used in the macro expansion of the subsequent toplevel forms in the same file. It doesn't work in precompilation, since macro expansion precedes execution. This limitation includes procedure definitions. For example: + [source,scheme] ---- (define (quote-it x) `(quote ,x)) (define-syntax quote-self (er-macro-transformer (^[f r c] (quote-it f)))) (define (test) (quote-self a b c)) ---- + This works by loading the source, but won't work in precompilation, since the expansion of `quote-self` requires to call `quote-it`, which is only available at run-time. + You need to split the file, one contains the values the macro expander depends, and another contains the forms that uses the macro. The macro definition itself can be in either file. (If the macro expander does not depend on the runtime value computed in the same file, you can put the macro definition and macro use in the same file.) 3. The `cond-expand` macro is handy to switch architecture-dependent code. Since it is a macro, it's expanded at the precompile time, so the features are selected based on the platform you're precompiling, not the platform you actually run the program on. That means you can't simply precompile a Scheme file and carry over the generated C file to other platform, if the Scheme file needs to select architecture-dependent part with cond-expand. C code itself is portable, but the unportable selection is to be done before C file is created. + If you need to distribute generated C code and you need to use `cond-expand`, just extract the part that uses `cond-expand` into a Scheme file and keep it from being precompiled. == The first step Let's begin with the simplest case. Precompiling a single Scheme file. Suppose we have `greet.scm`: [source,scheme] --------------------------------- (define-module greet (export greeting)) (select-module greet) (define *message* '("Hello, " ". How are you?")) (define (greeting who) #"~(car *message*)~|who|~(cadr *message*)") --------------------------------- Run `tools/precomp` subcommand as follows. The `-e` option is important -- that tells `precomp` to create a C file suitable for Gauche extension module on its own. [source,console] ---- $ gosh tools/precomp -e greet.scm ---- Now you see two files are generated: `greet.c` and `greet.sci` [source,console] ---- $ ls greet.c greet.sci greet.scm ---- The `.sci` suffix stands for **SC**heme **I**nterface, which contains a small Scheme code fragment for module definitions, plus forms that can't be compiled to C, and the call to dynamic-load that loads DSO file: [source,console] ---- $ cat greet.sci ;; generated automatically. DO NOT EDIT #!no-fold-case (define-module greet (export greeting)) (select-module greet) (dynamic-load "greet") ---- Precompiled DSOs are always loaded via its interface file -- it resolves module dependency. That means you have to distribute the interface file along the DSO. You don't usually see `+*.sci+`, though, since `gauche-install` command can rename `+*.sci+` to `+*.scm+` while it copies the file (with `-C` option). We use `+*.sci+` suffix only because we want to be able to generate the file in the same directory as the source `+*.scm+` file. Next, you can compile `greet.c` to generate greet.so (or `greet.dylib` or `greet.dll`, depending on your platform): [source,console] ---- $ gauche-package compile greet greet.c $ ls greet.c greet.o greet.sci greet.scm greet.so ---- `gauche-package compile` command knows which C compiler and flags are used when Gauche is compiled, where the Gauche header files are, etc. You have to compile the `+*.c+` file with the same compiler and same `CFLAGS`, otherwise Gauche can't safely dlopen the resulting DSO. You can give extra flags to the `gauche-package compile`; just type this command with no arguments and it tells the usage. Now, let's try the precompiled module. We give `-I.` option to `gosh` so that it can find `greet.sci` and `greet.so`. We also give `-fload-verbose` option to check if it really loads the precompiled file, not the source file: [source,console] ---- $ gosh -I. -fload-verbose ;;Loading /usr/share/gauche-0.9/0.9.5/lib/gauche/interactive.scm... ;; Loading /usr/share/gauche-0.9/0.9.5/lib/gauche/common-macros.scm... ;; Loading /usr/share/gauche-0.9/0.9.5/lib/gauche/defvalues.scm... gosh> (use greet) ;;Loading ./greet.sci... ;; Dynamically Loading ./greet.so... gosh> (greeting "John") "Hello, John. How are you?" ---- Woohoo! Note that `+*.sci+` takes precedence over `+*.scm+`, so even we have both `greet.sci` and `greet.scm`, gosh loads `greet.sci` (and subsequently, `greet.so`). In practice, you have to install `+*.sci+` file and `+*.so+` file to appropriate places so that other's code can find them. The standard places can be queried to with `gauche-config` command: [source,console] ---- $ gauche-config --sitelibdir /usr/share/gauche-0.9/site/lib $ gauche-config --sitearchdir /usr/lib/gauche-0.9/site/x86_64-pc-linux-gnu ---- You can use gauche-install to copy the files: [source,console] ---- $ gauche-install -T `gauche-config --sitelibdir` -C greet.sci $ gauche-install -T `gauche-config --sitearchdir` greet.so ---- The `-C` option renames `greet.sci` to `greet.scm` in the target directory. (If you use the template makefile generated by `gauche-package generate`, it has the skeleton to set up the installation destination.) == Multiple source files The previous section shows converting one Scheme source into one DSO. However, usually you want to make one DSO per library, which may contain lots of source files. Suppose you have the following source code structure, and `foo.scm` uses sub files as `foo.boo` and `foo.woo`. ---- foo.scm foo/boo.scm foo/woo.scm ---- You have to generate C file for each Scheme source, then compile and link all together. [source,console] ---- $ gosh tools/precomp -I. -e foo.scm foo/boo.scm foo/woo.scm $ ls foo/ foo--boo.c foo--woo.c foo.c foo.sci foo.scm $ ls foo boo.sci boo.scm woo.sci woo.scm ---- Note the `-I.` option, which allows `precomp` to find `foo/util1.scm` and `foo/util2.scm` referred from `foo.scm`. You see C files are generated in the current directory, while SCI files are on the side of each file under the subdirectory. Now you can compile those C files into a single DSO: [source,console] ---- $ gauche-package compile foo foo.c foo--boo.c foo--woo.c $ ls foo/ foo--boo.o foo--woo.o foo.o foo.scm foo--boo.c foo--woo.c foo.c foo.sci foo.so ---- You still have an SCI file per each Scheme source, but there's only one DSO and that can be loaded at once. == But I don't want a bunch of SCI files! The reason we have one `+*.sci+` file per one `+*.scm+` file is to guarantee the consitent behavior between source form and precompiled form. With keeping the interface file in the same relative path as the source, we can guarantee the library user can say not only `(use foo)`, but also `(use foo.boo)` and `(use foo.woo)`. The latter two still work, since we have `foo/boo.sci` and `foo/woo.sci`, with appropriate initialization in it. However, if you have hundreds of source files and you *know* the external user will only use the toplevel module and not directly `use` the submodules, then you can consolidate those SCI files into single one. Just give `--single-interface` option to precomp. [source,console] ---- $ gosh tools/precomp -I. -e --single-interface foo.scm foo/boo.scm foo/woo.scm $ ls foo/ foo--boo.c foo--woo.c foo.c foo.sci foo.scm $ ls foo boo.scm woo.scm ---- Having this in `foo.sci`, you only need it and `foo.so` for the library to work: [source,console] ---- $ gosh -I. -fload-verbose ;;Loading /usr/share/gauche-0.9/0.9.5/lib/gauche/interactive.scm... ;; Loading /usr/share/gauche-0.9/0.9.5/lib/gauche/common-macros.scm... ;; Loading /usr/share/gauche-0.9/0.9.5/lib/gauche/defvalues.scm... gosh> (use foo) ;;Loading ./foo.sci... ;; Dynamically Loading ./foo.so... ----