Debian Codegraph @

HACKING.adoc

raw
1 = How to hack the cutting-edge Gauche
2
3 NOTE: This document is for those who wants to hack the bleeding-edge
4 Gauche source from the git repository. If you just want to
5 compile from the distribution, you don't need to read any further.
6
7
8 == TOOLS REQUIRED FOR BUILDING
9
10 === On Unix platforms
11
12 In order to build Gauche from the git sources, instead of from
13 the distribution, you need to have the *latest* release of
14 Gauche installed on your machine, for it is required to generate
15 some C files. If you don't have one, grab the release tarball
16 and install it.
17
18 You also need a C compiler supporting C11, e.g. `gcc` (>= 4.9) or `clang` (>= 3.3),
19 `autoconf` (>= 2.71), `automake` (>= 1.14.1),
20 `libtool` (>= 2.4.2), and `pkg-config` (>= 0.26).
21 (Note on macOS users: Xcode now only includes subset
22 of `autotools`. You have to install full set either manually
23 or thru MacPorts or Homebrew. Xcode's `libtool` *does not work*).
24
25 The git source tree doesn't include `configure` scripts.
26 You can use the `DIST` script like the following to generate them.
27
28 [source,console]
29 ----
30 $ ./DIST gen
31 ----
32
33 Afterwards, you just do the usual `./configure` + `make`. For
34 the configure options, see the link:INSTALL.adoc[].
35
36 In case if the latest release of Gauche can't be built on your
37 target machine but the fix is already in the HEAD, you need to
38 take more steps -- you need the source with that fix, but in order
39 to build it, you need the latest release of Gauche installed,
40 which you can't. If you stuck in this Catch-22 situation, you
41 have to generate tarball on other machines. Find a Linux box
42 (or BSD, or most Unix box would do) and take the following steps:
43
44 1. Build and install the latest release of Gauche on that box
45
46 2. Clone the source on that box, and build the HEAD as explained above.
47
48 3. Run
49 +
50 [source,console]
51 ----
52 $ ./DIST tgz
53 ----
54 +
55 This creates a tarball `Gauche-_$VERSION_.tgz` in the parent
56 directory of the source tree.
57
58 4. Copy the created tarball to your target machine, untar,
59 configure and build. The created tarball includes generated
60 files, so you don't need Gauche in this step.
61
62 Note for seasoned libtool/automake users: the git source tree
63 includes files that are usually generated automatically by
64 autotools, such as `aclocal.m4`, `ltmain.sh`, `src/gauche/config.h.in`,
65 etc. Avoid clobbering these files, unless you're sure you want
66 to do so. If your autotools version doesn't match mine,
67 it tends to break.
68
69
70 === On Windows
71
72 The recommended way to build Gauche on Windows is mingw-w64 and
73 MSYS2. If you want to build from git source, you need Gauche
74 of the latest release to be installed. The easiest way is to
75 grab the windows installer from https://practical-scheme.net/gauche.
76 For the details of build procedure, see link:doc/HOWTO-mingw.adoc[].
77
78 You can build Gauche with Cygwin, but it is no longer actively supported.
79
80 Some effort was made to build Gauche with MSVC. You can find
81 project files under winnt. Unfortunately though, it is so
82 much pain to make it work and keep it updated with newer Gauche
83 source tree, and now they are quite obsolete.
84
85
86 == BUILD STEPS
87
88 After `confiugre`, just running `make` build everything. But you want to
89 know how build process proceeds if you got into trouble hacking it. Here's
90 the internal steps after you type `make`.
91
92 1. The Boehm-Demers-Weiser garbage collector (bdwgc) is built under `gc/`.
93 We included the bdwgc source instead of relying on the shared library,
94 since we need a custom configure options to be optimal for Gauche memory
95 usage pattern.
96
97 2. The 'core' engine is built under `src/`. This includes `libgauche-X.so`
98 and `libgauche-static-X.a`, and the `gosh` command. Some sources are
99 written in Gauche, so they need to be processed with working Gauche;
100 thus you require the latest release of Gauche installed on your system.
101 Note that, at this point, no precompiled extensions under `ext/` are
102 available. If you simply 'use' one of those modules during the building,
103 it can't load the module. To avoid this, the host `gosh` loads
104 a file link:src/preload.scm[], which lists the modules the host gosh
105 need to load from the host environment, not from the source tree.
106
107 3. The extensions are built under `ext/`. If it's not cross compiling,
108 this process uses the freshly built `gosh` in `src/`. Some modules under
109 this directory have C code, while others are pure Scheme. The pure Scheme
110 source may be placed under `libsrc/`, so that those modules are available
111 during the build process; that is, during the build process the modules
112 are loaded in the source form from `libsrc/`, and once the build is
113 completed, they are dynamically loaded as precompiled modules.
114
115 4. Documents under `doc/` is built. Info documents are built only when
116 the system has newer makeinfo.
117
118
119 Knowing which module is used during build, the one in the host environment
120 or the one in the source tree, is important. Modules listed in `preload.scm`
121 are from the host environment, which means you can't use the feature
122 added after the latest release of Gauche to build the current source.
123
124 You can make sure that the source tree is buildable both by the latest
125 relase _and_ the current built, by running `./DIST self-host-test`.
126 It also checks if out-of-tree build works. It performs the following
127 opertaions:
128
129 - It cleans the source tree (by `make maintainer-clean` and regenerate
130 configure scripts.
131 - It creates a temporary directory and run `configure` in it. This
132 sets up out-of-tree build.
133 - It builds Gauche using the host environment (latest release) and
134 runs the tests.
135 - It installs the built Gauche inside the temporary directory.
136 - It builds Gauche again, using the freshly built Gauche, and
137 runs the tests again.
138
139 It leaves a temporary directory `../Gauche-tmp-self-host-test` afterwards,
140 so that it's easier to investigate if something goes wrong. You can
141 remove that directory if build succeeds.
142
143 === `*.sci` files
144
145 After building the source, you'll see files with `sci` suffix. It stands
146 for 'SCheme Interface'. It is generated from `*.scm` file with
147 `precomp`, and contains `define-module` form with using modules and
148 exporting symbols, and a form to dynamically load precompiled file.
149
150 If you have `foo.scm` and `foo.sci` in the same directory and you
151 try to load `foo`, `sci` take precedence. Usually, if you change
152 the source `\*.scm` file and run `make`, the `*.sci` file is
153 regenerated. But if you forgot to run `make`, you may wonder
154 why the changes you did in `*.scm` file aren't reflected.
155
156 If you're not sure which file is actually loaded, run `gosh` with
157 `-fload-verbose` option; it reports the exact pathname of each
158 loaded file.
159
160 The `\*.sci` files are renamed to `*.scm` when installed, so you won't
161 see them in the installation directory.
162
163 The reason that we use `*.sci` files is for the precompiler not to
164 overwrite the source file. See link:doc/HOWTO-precompile.adoc[] for
165 the details.
166
167
168 == DEPENDENCIES
169
170 Quite a few files are generated by Gauche itself if you build
171 Gauche from scratch. Consequently, there are quite a few nasty
172 dependency issues. If you modify files listed below, be careful
173 not to introduce undesirable dependencies.
174
175 - The compiler (`compile*.scm`) shouldn't depend on anything that
176 are not compiled into the libgauche core. An exception is
177 `util.match`; it's ok since it is just a macro and all `match`
178 forms are expanded fully in precompilation.
179
180 - `precomp`, `genstub`, `geninsn` shouldn't depend on the extension
181 modules except the '`pre-loaded`' ones in `BUILD_GOSH`.
182
183 - An extension compiled by `precomp` shouldn't depend on other
184 extensions compiled by `precomp`. This is because `precomp` is
185 run by `BUILD_GOSH` and it may not be able to load the other
186 extensions compiled for the target `gosh`.
187
188 - `ext/xlink` shouldn't depend on anything that requires loading
189 extension modules, since it is called before all the extension
190 modules are generated.
191
192
193 == CROSS COMPILATION
194
195 In a normal compilation, extension modules (`+ext/*+`) are build
196 using the new gosh just built in `src/`. However, we can't
197 run `src/gosh` when we're cross compiling. So you need to
198 install *this version of Gauche* compiled on your platform beforehand.
199 Then, configure with the ordinary cross-compiling options.
200
201
202 == CHANGELOGS
203
204 We have link:ChangeLog[] file to record changes of new features, APIs, or
205 internal changes large enough so that other developers need to be
206 aware of. Small changes like '`quick fix of the bug introduced
207 in the last commit`', or documentation additions, don't need to
208 be recorded.
209
210 In the age of distributed development, ChangeLog seems becoming
211 obsolete, for it causes conflicts easily. Yet I still see it worth,
212 for it is one-stop place where somebody who tries to grasp what was
213 changed since some point of time. SCM's commit log doesn't replace
214 that role -- commit logs comes with every commit, however little it
215 is, and those small trivial logs are just noise to those want to
216 know changes that matter. Also, commit logs are inherently per-commit,
217 so it's not a good place to describe overall intention or a plan
218 of series of changes.
219
220 To keep ChangeLog in that role *and* to be friendly to distributed
221 development, I suggest that you don't update ChangeLog unless you
222 feel you really want to. If I see your change is better mentioned
223 in ChangeLog, I'll update it separately.
224
225
226 == CODING STYLE
227
228 We aren't picky about coding style, but adopt some Gauche-specific
229 style, especially for Gauche macros. For Emacs users, such style is
230 written in `.dir-locals.el` in the top source directory. The Emacs
231 package `lisp-local` (available from MELPA) can automatically apply
232 the settings.
233
234
235 == TROUBLESHOOTING
236
237 If your modification break something fundamental and you can't even
238 get to REPL, there are a couple of tricks to isolate the problem.
239
240 - Run gosh with `-q` option, e.g. `./gosh -ftest -q`. It suppresses
241 loading startup files, e.g. it won't read any external Scheme files,
242 nor loading DSO files, before getting to a REPL prompt. If you get
243 `gosh>` prompt, then the issue is the loading of runtime files.
244 -- Run gosh with `./gosh -ftest -fload-verbose` option and see which
245 startup file causes the trouble.
246 -- With the bare REPL from `./gosh -ftest -q`, try to load or evaluate
247 offending files/expressions to figure out the issue. Note that
248 the 'bare' REPL only have minimal features---no toplevel commands,
249 no line editing, etc.
250
251 - If you can't even get to REPL with `-q`, it is likely that something
252 is broken during initialization. Run `gosh` with an environment variable
253 `GAUCHE_DEBUG_INITIALIZATION` set. It reports which component is being
254 initialized.