| 1 |
= How to hack the cutting-edge Gauche |
| 2 |
|
| 3 |
NOTE: This document is for those who wants to hack the bleeding-edge |
| 4 |
Gauche source from the git repository. If you just want to |
| 5 |
compile from the distribution, you don't need to read any further. |
| 6 |
|
| 7 |
|
| 8 |
== TOOLS REQUIRED FOR BUILDING |
| 9 |
|
| 10 |
=== On Unix platforms |
| 11 |
|
| 12 |
In order to build Gauche from the git sources, instead of from |
| 13 |
the distribution, you need to have the *latest* release of |
| 14 |
Gauche installed on your machine, for it is required to generate |
| 15 |
some C files. If you don't have one, grab the release tarball |
| 16 |
and install it. |
| 17 |
|
| 18 |
You also need a C compiler supporting C11, e.g. `gcc` (>= 4.9) or `clang` (>= 3.3), |
| 19 |
`autoconf` (>= 2.71), `automake` (>= 1.14.1), |
| 20 |
`libtool` (>= 2.4.2), and `pkg-config` (>= 0.26). |
| 21 |
(Note on macOS users: Xcode now only includes subset |
| 22 |
of `autotools`. You have to install full set either manually |
| 23 |
or thru MacPorts or Homebrew. Xcode's `libtool` *does not work*). |
| 24 |
|
| 25 |
The git source tree doesn't include `configure` scripts. |
| 26 |
You can use the `DIST` script like the following to generate them. |
| 27 |
|
| 28 |
[source,console] |
| 29 |
---- |
| 30 |
$ ./DIST gen |
| 31 |
---- |
| 32 |
|
| 33 |
Afterwards, you just do the usual `./configure` + `make`. For |
| 34 |
the configure options, see the link:INSTALL.adoc[]. |
| 35 |
|
| 36 |
In case if the latest release of Gauche can't be built on your |
| 37 |
target machine but the fix is already in the HEAD, you need to |
| 38 |
take more steps -- you need the source with that fix, but in order |
| 39 |
to build it, you need the latest release of Gauche installed, |
| 40 |
which you can't. If you stuck in this Catch-22 situation, you |
| 41 |
have to generate tarball on other machines. Find a Linux box |
| 42 |
(or BSD, or most Unix box would do) and take the following steps: |
| 43 |
|
| 44 |
1. Build and install the latest release of Gauche on that box |
| 45 |
|
| 46 |
2. Clone the source on that box, and build the HEAD as explained above. |
| 47 |
|
| 48 |
3. Run |
| 49 |
+ |
| 50 |
[source,console] |
| 51 |
---- |
| 52 |
$ ./DIST tgz |
| 53 |
---- |
| 54 |
+ |
| 55 |
This creates a tarball `Gauche-_$VERSION_.tgz` in the parent |
| 56 |
directory of the source tree. |
| 57 |
|
| 58 |
4. Copy the created tarball to your target machine, untar, |
| 59 |
configure and build. The created tarball includes generated |
| 60 |
files, so you don't need Gauche in this step. |
| 61 |
|
| 62 |
Note for seasoned libtool/automake users: the git source tree |
| 63 |
includes files that are usually generated automatically by |
| 64 |
autotools, such as `aclocal.m4`, `ltmain.sh`, `src/gauche/config.h.in`, |
| 65 |
etc. Avoid clobbering these files, unless you're sure you want |
| 66 |
to do so. If your autotools version doesn't match mine, |
| 67 |
it tends to break. |
| 68 |
|
| 69 |
|
| 70 |
=== On Windows |
| 71 |
|
| 72 |
The recommended way to build Gauche on Windows is mingw-w64 and |
| 73 |
MSYS2. If you want to build from git source, you need Gauche |
| 74 |
of the latest release to be installed. The easiest way is to |
| 75 |
grab the windows installer from https://practical-scheme.net/gauche. |
| 76 |
For the details of build procedure, see link:doc/HOWTO-mingw.adoc[]. |
| 77 |
|
| 78 |
You can build Gauche with Cygwin, but it is no longer actively supported. |
| 79 |
|
| 80 |
Some effort was made to build Gauche with MSVC. You can find |
| 81 |
project files under winnt. Unfortunately though, it is so |
| 82 |
much pain to make it work and keep it updated with newer Gauche |
| 83 |
source tree, and now they are quite obsolete. |
| 84 |
|
| 85 |
|
| 86 |
== BUILD STEPS |
| 87 |
|
| 88 |
After `confiugre`, just running `make` build everything. But you want to |
| 89 |
know how build process proceeds if you got into trouble hacking it. Here's |
| 90 |
the internal steps after you type `make`. |
| 91 |
|
| 92 |
1. The Boehm-Demers-Weiser garbage collector (bdwgc) is built under `gc/`. |
| 93 |
We included the bdwgc source instead of relying on the shared library, |
| 94 |
since we need a custom configure options to be optimal for Gauche memory |
| 95 |
usage pattern. |
| 96 |
|
| 97 |
2. The 'core' engine is built under `src/`. This includes `libgauche-X.so` |
| 98 |
and `libgauche-static-X.a`, and the `gosh` command. Some sources are |
| 99 |
written in Gauche, so they need to be processed with working Gauche; |
| 100 |
thus you require the latest release of Gauche installed on your system. |
| 101 |
Note that, at this point, no precompiled extensions under `ext/` are |
| 102 |
available. If you simply 'use' one of those modules during the building, |
| 103 |
it can't load the module. To avoid this, the host `gosh` loads |
| 104 |
a file link:src/preload.scm[], which lists the modules the host gosh |
| 105 |
need to load from the host environment, not from the source tree. |
| 106 |
|
| 107 |
3. The extensions are built under `ext/`. If it's not cross compiling, |
| 108 |
this process uses the freshly built `gosh` in `src/`. Some modules under |
| 109 |
this directory have C code, while others are pure Scheme. The pure Scheme |
| 110 |
source may be placed under `libsrc/`, so that those modules are available |
| 111 |
during the build process; that is, during the build process the modules |
| 112 |
are loaded in the source form from `libsrc/`, and once the build is |
| 113 |
completed, they are dynamically loaded as precompiled modules. |
| 114 |
|
| 115 |
4. Documents under `doc/` is built. Info documents are built only when |
| 116 |
the system has newer makeinfo. |
| 117 |
|
| 118 |
|
| 119 |
Knowing which module is used during build, the one in the host environment |
| 120 |
or the one in the source tree, is important. Modules listed in `preload.scm` |
| 121 |
are from the host environment, which means you can't use the feature |
| 122 |
added after the latest release of Gauche to build the current source. |
| 123 |
|
| 124 |
You can make sure that the source tree is buildable both by the latest |
| 125 |
relase _and_ the current built, by running `./DIST self-host-test`. |
| 126 |
It also checks if out-of-tree build works. It performs the following |
| 127 |
opertaions: |
| 128 |
|
| 129 |
- It cleans the source tree (by `make maintainer-clean` and regenerate |
| 130 |
configure scripts. |
| 131 |
- It creates a temporary directory and run `configure` in it. This |
| 132 |
sets up out-of-tree build. |
| 133 |
- It builds Gauche using the host environment (latest release) and |
| 134 |
runs the tests. |
| 135 |
- It installs the built Gauche inside the temporary directory. |
| 136 |
- It builds Gauche again, using the freshly built Gauche, and |
| 137 |
runs the tests again. |
| 138 |
|
| 139 |
It leaves a temporary directory `../Gauche-tmp-self-host-test` afterwards, |
| 140 |
so that it's easier to investigate if something goes wrong. You can |
| 141 |
remove that directory if build succeeds. |
| 142 |
|
| 143 |
=== `*.sci` files |
| 144 |
|
| 145 |
After building the source, you'll see files with `sci` suffix. It stands |
| 146 |
for 'SCheme Interface'. It is generated from `*.scm` file with |
| 147 |
`precomp`, and contains `define-module` form with using modules and |
| 148 |
exporting symbols, and a form to dynamically load precompiled file. |
| 149 |
|
| 150 |
If you have `foo.scm` and `foo.sci` in the same directory and you |
| 151 |
try to load `foo`, `sci` take precedence. Usually, if you change |
| 152 |
the source `\*.scm` file and run `make`, the `*.sci` file is |
| 153 |
regenerated. But if you forgot to run `make`, you may wonder |
| 154 |
why the changes you did in `*.scm` file aren't reflected. |
| 155 |
|
| 156 |
If you're not sure which file is actually loaded, run `gosh` with |
| 157 |
`-fload-verbose` option; it reports the exact pathname of each |
| 158 |
loaded file. |
| 159 |
|
| 160 |
The `\*.sci` files are renamed to `*.scm` when installed, so you won't |
| 161 |
see them in the installation directory. |
| 162 |
|
| 163 |
The reason that we use `*.sci` files is for the precompiler not to |
| 164 |
overwrite the source file. See link:doc/HOWTO-precompile.adoc[] for |
| 165 |
the details. |
| 166 |
|
| 167 |
|
| 168 |
== DEPENDENCIES |
| 169 |
|
| 170 |
Quite a few files are generated by Gauche itself if you build |
| 171 |
Gauche from scratch. Consequently, there are quite a few nasty |
| 172 |
dependency issues. If you modify files listed below, be careful |
| 173 |
not to introduce undesirable dependencies. |
| 174 |
|
| 175 |
- The compiler (`compile*.scm`) shouldn't depend on anything that |
| 176 |
are not compiled into the libgauche core. An exception is |
| 177 |
`util.match`; it's ok since it is just a macro and all `match` |
| 178 |
forms are expanded fully in precompilation. |
| 179 |
|
| 180 |
- `precomp`, `genstub`, `geninsn` shouldn't depend on the extension |
| 181 |
modules except the '`pre-loaded`' ones in `BUILD_GOSH`. |
| 182 |
|
| 183 |
- An extension compiled by `precomp` shouldn't depend on other |
| 184 |
extensions compiled by `precomp`. This is because `precomp` is |
| 185 |
run by `BUILD_GOSH` and it may not be able to load the other |
| 186 |
extensions compiled for the target `gosh`. |
| 187 |
|
| 188 |
- `ext/xlink` shouldn't depend on anything that requires loading |
| 189 |
extension modules, since it is called before all the extension |
| 190 |
modules are generated. |
| 191 |
|
| 192 |
|
| 193 |
== CROSS COMPILATION |
| 194 |
|
| 195 |
In a normal compilation, extension modules (`+ext/*+`) are build |
| 196 |
using the new gosh just built in `src/`. However, we can't |
| 197 |
run `src/gosh` when we're cross compiling. So you need to |
| 198 |
install *this version of Gauche* compiled on your platform beforehand. |
| 199 |
Then, configure with the ordinary cross-compiling options. |
| 200 |
|
| 201 |
|
| 202 |
== CHANGELOGS |
| 203 |
|
| 204 |
We have link:ChangeLog[] file to record changes of new features, APIs, or |
| 205 |
internal changes large enough so that other developers need to be |
| 206 |
aware of. Small changes like '`quick fix of the bug introduced |
| 207 |
in the last commit`', or documentation additions, don't need to |
| 208 |
be recorded. |
| 209 |
|
| 210 |
In the age of distributed development, ChangeLog seems becoming |
| 211 |
obsolete, for it causes conflicts easily. Yet I still see it worth, |
| 212 |
for it is one-stop place where somebody who tries to grasp what was |
| 213 |
changed since some point of time. SCM's commit log doesn't replace |
| 214 |
that role -- commit logs comes with every commit, however little it |
| 215 |
is, and those small trivial logs are just noise to those want to |
| 216 |
know changes that matter. Also, commit logs are inherently per-commit, |
| 217 |
so it's not a good place to describe overall intention or a plan |
| 218 |
of series of changes. |
| 219 |
|
| 220 |
To keep ChangeLog in that role *and* to be friendly to distributed |
| 221 |
development, I suggest that you don't update ChangeLog unless you |
| 222 |
feel you really want to. If I see your change is better mentioned |
| 223 |
in ChangeLog, I'll update it separately. |
| 224 |
|
| 225 |
|
| 226 |
== CODING STYLE |
| 227 |
|
| 228 |
We aren't picky about coding style, but adopt some Gauche-specific |
| 229 |
style, especially for Gauche macros. For Emacs users, such style is |
| 230 |
written in `.dir-locals.el` in the top source directory. The Emacs |
| 231 |
package `lisp-local` (available from MELPA) can automatically apply |
| 232 |
the settings. |
| 233 |
|
| 234 |
|
| 235 |
== TROUBLESHOOTING |
| 236 |
|
| 237 |
If your modification break something fundamental and you can't even |
| 238 |
get to REPL, there are a couple of tricks to isolate the problem. |
| 239 |
|
| 240 |
- Run gosh with `-q` option, e.g. `./gosh -ftest -q`. It suppresses |
| 241 |
loading startup files, e.g. it won't read any external Scheme files, |
| 242 |
nor loading DSO files, before getting to a REPL prompt. If you get |
| 243 |
`gosh>` prompt, then the issue is the loading of runtime files. |
| 244 |
-- Run gosh with `./gosh -ftest -fload-verbose` option and see which |
| 245 |
startup file causes the trouble. |
| 246 |
-- With the bare REPL from `./gosh -ftest -q`, try to load or evaluate |
| 247 |
offending files/expressions to figure out the issue. Note that |
| 248 |
the 'bare' REPL only have minimal features---no toplevel commands, |
| 249 |
no line editing, etc. |
| 250 |
|
| 251 |
- If you can't even get to REPL with `-q`, it is likely that something |
| 252 |
is broken during initialization. Run `gosh` with an environment variable |
| 253 |
`GAUCHE_DEBUG_INITIALIZATION` set. It reports which component is being |
| 254 |
initialized. |