| 1 |
// some workarounds for words contain `++` |
| 2 |
// https://github.com/asciidoctor/asciidoctor/issues/1208 |
| 3 |
:cpp: C++ |
| 4 |
:gpp: g++ |
| 5 |
|
| 6 |
This directory contains a sample extension module code to bridge |
| 7 |
Gauche and {cpp} libraries. |
| 8 |
|
| 9 |
The two files, link:mqueue.h[] and link:mqueue.cpp[], are our hypothetical |
| 10 |
external {cpp} library. It implements a simple message queue. |
| 11 |
It is independent from Gauche. Our mission here is, given this |
| 12 |
mqueue library, to write a Gauche binding for it. |
| 13 |
|
| 14 |
The following files are needed for our extension: |
| 15 |
|
| 16 |
[horizontal] |
| 17 |
link:Makefile.in[]:: template makefile |
| 18 |
link:configure[]:: "configure" script |
| 19 |
link:mqueue_glue.cpp[]:: a bridge (glue) between {cpp} library and Gauche |
| 20 |
link:mqueue_glue.h[]:: ditto |
| 21 |
link:mqueue_lib.scm[]:: Scheme binding definition |
| 22 |
link:test.scm[]:: unit test |
| 23 |
link:example/mqueue-cpp.scm[]:: Scheme module definition |
| 24 |
|
| 25 |
The skeleton of those files can be generated by |
| 26 |
`gauche-package` command, like this: |
| 27 |
|
| 28 |
[source,console] |
| 29 |
---- |
| 30 |
% gauche-package generate mqueue-cpp example.mqueue-cpp |
| 31 |
---- |
| 32 |
|
| 33 |
I renamed the generated `mqueue_cpp.h`, `mqueue_cpp.c`, `mqueue_cpplib.scm` |
| 34 |
to more descriptive names, `mqueue_glue.h`, `mqueue_glue.cpp`, |
| 35 |
and `mqueue_lib.scm`, respectively. |
| 36 |
|
| 37 |
The generated files don't know about {cpp}, so you have to tweak link:configure[] |
| 38 |
and link:Makefile.in[] to enable {cpp}. |
| 39 |
|
| 40 |
1. Add (`cf-prog-cxx`) in link:configure[]. This tries to figure out how |
| 41 |
to call {cpp} compiler, and sets up a substitution variable `CXX`. |
| 42 |
+ |
| 43 |
+NOTE:+ the {cpp} compiler has to be 'compatible' with the C |
| 44 |
compiler used to build Gauche, e.g. it must accept the |
| 45 |
same set of command-line options to generate a shared |
| 46 |
library. Gcc and {gpp} are such compatible ones. |
| 47 |
|
| 48 |
2. In link:Makefile.in[], added the following line to receive the {cpp} |
| 49 |
compiler name `configure.in` finds. |
| 50 |
+ |
| 51 |
[source,Makefile] |
| 52 |
---- |
| 53 |
CXX = @CXX@ |
| 54 |
---- |
| 55 |
|
| 56 |
3. Also in link:Makefile.in[], when builing `mqueue_cpp.so`, give `--c++=$(CXX)` |
| 57 |
option to the `gauche-package` script to have it use the {cpp} compiler |
| 58 |
instead of the default C compiler. |
| 59 |
+ |
| 60 |
[source,Makefile] |
| 61 |
---- |
| 62 |
mqueue_cpp.$(SOEXT): $(mqueue_SRCS) $(mqueue_HDRS) |
| 63 |
$(GAUCHE_PACKAGE) compile --c++=$(CXX) --verbose mqueue_cpp $(mqueue_SRCS) |
| 64 |
---- |
| 65 |
|
| 66 |
|
| 67 |
== Glue code |
| 68 |
|
| 69 |
Now, let's take a look at the 'glue' code. |
| 70 |
|
| 71 |
The header file link:mqueue_glue.h[] begins with including both the Gauche |
| 72 |
interface and the library interface. |
| 73 |
|
| 74 |
[source,c++] |
| 75 |
---- |
| 76 |
#include <gauche.h> |
| 77 |
#include <gauche/extend.h> |
| 78 |
#include "mqueue.h" |
| 79 |
---- |
| 80 |
|
| 81 |
It is important to include `+<gauche.h>+` *before any other system |
| 82 |
header files*. Since `gauche.h` includes some configuration |
| 83 |
information (from `gauche/config.h`) which may affect the definitions |
| 84 |
of the standard header files; one example is `_FILE_OFFSET_BITS`, |
| 85 |
which is 32 on most 32bit Linux by default, but Gauche sets it to 64, |
| 86 |
that causes the system library calls to call 64bit versions. |
| 87 |
|
| 88 |
We need a Gauche class that represents `MQueue` object in {cpp} world. |
| 89 |
We're going to use Gauche's foreign pointer feature to implement it. |
| 90 |
The pointer to the Scheme `<mqueue>` class is held in this external |
| 91 |
variable. It is set in the extension initialization routine. |
| 92 |
|
| 93 |
[source,c++] |
| 94 |
---- |
| 95 |
extern ScmClass *MQueueClass; |
| 96 |
---- |
| 97 |
|
| 98 |
Some conveninece macros. The `<mqueue>` object in Scheme world is |
| 99 |
wrapped in `ScmForeignPointer.` 'Unboxing' is to retrieve the {cpp} |
| 100 |
`+MQueue*+` object from it, and 'boxing' is to wrap {cpp} object by |
| 101 |
`ScmForeignPointer`. |
| 102 |
|
| 103 |
[source,c++] |
| 104 |
---- |
| 105 |
#define MQUEUE_P(obj) SCM_XTYPEP(obj, MQueueClass) |
| 106 |
#define MQUEUE_UNBOX(obj) SCM_FOREIGN_POINTER_REF(MQueue*, obj) |
| 107 |
#define MQUEUE_BOX(ptr) Scm_MakeForeignPointer(MQueueClass, ptr) |
| 108 |
---- |
| 109 |
|
| 110 |
Next comes the declaration of the extension initialization routine. |
| 111 |
It is important for it to be declared/defined in `"C"` linkage scope. |
| 112 |
Macros `SCM_DECL_BEGIN` and `SCM_DECL_END` ensures it (they are the same |
| 113 |
as `extern "C" {` and `}`.) |
| 114 |
|
| 115 |
[source,c++] |
| 116 |
---- |
| 117 |
extern void Scm_Init_mqueue_cpp(); |
| 118 |
---- |
| 119 |
|
| 120 |
The source link:mqueue_glue.cpp[] implements a bridging machinery. The |
| 121 |
important part is this statement: |
| 122 |
|
| 123 |
[source,c++] |
| 124 |
---- |
| 125 |
MQueueClass = |
| 126 |
Scm_MakeForeignPointerClass(mod, "<mqueue>", |
| 127 |
mqueue_print, |
| 128 |
mqueue_cleanup, |
| 129 |
SCM_FOREIGN_POINTER_KEEP_IDENTITY|SCM_FOREIGN_POINTER_MAP_NULL); |
| 130 |
---- |
| 131 |
|
| 132 |
This creates a new Scheme class, named `<mqueue>`, as a foreign pointer |
| 133 |
class and stores its pointer to `MQueueClass`. Scheme variable `<mqueue>` |
| 134 |
is bound to the newly created class in the module `mod`. |
| 135 |
|
| 136 |
The second argument is a print procedure, called whenever the |
| 137 |
instance of `<mqueue>` is printed. |
| 138 |
|
| 139 |
The tricky part here is the 'cleanup' procedure. If you pass |
| 140 |
a cleanup procedure, it is called when the Scheme `<mqueue>` object |
| 141 |
is about to be GC-ed, so that you can free the resources in the {cpp} |
| 142 |
world. Although it sounds simple, it requires more attention than |
| 143 |
its superficial simplicity. If you do something wrong here, |
| 144 |
you'll get a nasty bug which is very hard to track. |
| 145 |
|
| 146 |
Typically, there are a few cases with regard to the foreign resource |
| 147 |
management. |
| 148 |
|
| 149 |
Case 1:: |
| 150 |
You allocate the foreign object via Gauche's allocator. |
| 151 |
(`SCM_NEW` etc.) In this case, Gauche's GC can take care |
| 152 |
of deallocation, so you don't need specify 'cleanup' procedure |
| 153 |
except you have other resources to free (e.g. file descriptors). |
| 154 |
|
| 155 |
anchor:frm-case-2[]Case 2:: |
| 156 |
The foreign object is allocated by the foreign library, and |
| 157 |
it is only pointed from the Scheme world. In this case, |
| 158 |
you are responsible to free the foreign object when there's |
| 159 |
no reference from the Scheme world to it. You can use cleanup |
| 160 |
procedure to do so. |
| 161 |
+ |
| 162 |
Note that the cleanup procedure is called when there's |
| 163 |
no reference to the Scheme `<mqueue>` object here, but it doesn't |
| 164 |
necessarily means that the {cpp} `MQueue` object pointed by it can be |
| 165 |
freed -- what if there are other Scheme `<mqueue>` object which |
| 166 |
points to the same {cpp} `MQueue` object? Gauche's GC can't detect |
| 167 |
such case, since the {cpp} `MQueue` object is outside the scope |
| 168 |
of Gauche's memory management. |
| 169 |
+ |
| 170 |
There's no universal solution for it, but Gauche provides a |
| 171 |
convenience mehchanism that works in typical cases. If you |
| 172 |
give a flag `SCM_FOREIGN_POINTER_KEEP_IDENTITY` to the |
| 173 |
`Scm_MakeForeignPointerClass`, then Gauche guarantees that |
| 174 |
`Scm_MakeForeignPointer` returns exactly same Scheme object |
| 175 |
for the same pointer (internally, it uses a weak hash table to |
| 176 |
map the `+void*+` pointer value to the created Scheme object). |
| 177 |
If your code never assigns the wrapped foreign pointer to |
| 178 |
other Scheme objects, then you can be sure that whenever the |
| 179 |
Scheme `<mqueue>` object is GC-ed, the {cpp} `MQueue` object pointed |
| 180 |
by it can also be destroyed. |
| 181 |
|
| 182 |
Case 3:: |
| 183 |
The foreign object is allocated by the foreign library, and |
| 184 |
it may be pointed from both Scheme and {cpp} worlds. In this case, |
| 185 |
you cannot destroy the foreign object even when the Scheme |
| 186 |
object is GC-ed, since there might be a pointer in {cpp} world |
| 187 |
that points to the same foreign object. Gauche's GC cannot |
| 188 |
track it. |
| 189 |
+ |
| 190 |
This is very difficult problem indeed. Usually, if this is |
| 191 |
the case, such foreign library provides some sort of resource |
| 192 |
management infrastructure such as reference counting. If so, |
| 193 |
you can drop the reference count in the cleanup routine, |
| 194 |
provided that you do increment the reference count in the |
| 195 |
boxing routine. |
| 196 |
+ |
| 197 |
(`SCM_FOREIGN_POINTER_KEEP_IDENTITY`, for there can be more than |
| 198 |
one Scheme object that points to the same foreign object). |
| 199 |
+ |
| 200 |
Unfortunately this won't work sometimes; there are cases that |
| 201 |
you also have to pass a Scheme object to the foreign library |
| 202 |
(e.g. closures for callback), and sometimes they form a loop |
| 203 |
of reference, spanning into both Scheme and foreign library |
| 204 |
world. But it is beyond the scope of this document and I'll |
| 205 |
leave the further discussion for some other time. |
| 206 |
|
| 207 |
|
| 208 |
The use of `SCM_FOREIGN_POINTER_KEEP_IDENTITY` is convenient even |
| 209 |
for case 1 or 3, since you can use `eq?` to test the pointer identity |
| 210 |
of the foreign object. However, using it incurs overhead of |
| 211 |
bookkeeping. |
| 212 |
|
| 213 |
The other flag, `SCM_FOREIGN_POINTER_MAP_NULL`, is a convenience flag. |
| 214 |
If you specify this flag, `Scm_MakeForeignPointer` returns `SCM_FALSE` |
| 215 |
if `NULL` is given. It is handy if you call a foreign routine that |
| 216 |
may return `NULL`; with this flag, such case can be seen from |
| 217 |
Scheme as if the foreign routine returned Scheme `#f`. |
| 218 |
(Be careful, however, that when you use this flag, you cannot assume |
| 219 |
the returned value of `Scm_MakeForeignPointer(KLASS, PTR)` is the |
| 220 |
instance of `KLASS`. If you play with the returned value in C code |
| 221 |
you have to check its type.) |
| 222 |
|
| 223 |
|
| 224 |
== Stub code |
| 225 |
|
| 226 |
The link:mqueue_lib.scm[] file binds the Scheme procedures and the |
| 227 |
foreign library functions. It looks like a Scheme source, and you |
| 228 |
can write Scheme code in it as well, but it also contains several forms |
| 229 |
that can't be executed if you load it directly to Gauche. |
| 230 |
|
| 231 |
Such forms needs to be _precompiled_ by Gauche's AOT compiler. |
| 232 |
The `gauche-package compile` command takes care of it. |
| 233 |
|
| 234 |
Let's see those forms specific to be precompiled. Open link:mqueue_lib.scm[] |
| 235 |
in a separete window and read along. |
| 236 |
|
| 237 |
The first form, `(in-module example.mqueue-cpp)`, sets up the current |
| 238 |
module much like `select-module`. However, unlike the latter, `in-module` |
| 239 |
doesn't need the specified module to exist beforehand. You can't |
| 240 |
just `(use example.mqueue-cpp)`, since it will try to load the compiled |
| 241 |
extension, which is exactly what we're about to produce. |
| 242 |
|
| 243 |
A quick side note: If you put Scheme code in it (which will also |
| 244 |
be precompiled), be aware that you need to `use` dependent modules |
| 245 |
after `in-module` form; since we don't read |
| 246 |
`(define-module example.mqueue-cpp ...)`. In the example file, |
| 247 |
|
| 248 |
The second form is `(inline-stub ...)`. It is the way to write |
| 249 |
C/C++ interface code inside Scheme source. The code inside |
| 250 |
`inline-stub` isn't Scheme at all; we call it _stub code_ for |
| 251 |
historical reasons. |
| 252 |
|
| 253 |
The first thing in the stub code is `(.include "mqueue_glue.h")`, which |
| 254 |
is C/C++ `#include "mqueue_glue.h"`. |
| 255 |
|
| 256 |
The `declare-stub-type` directive tells the stub generator about the |
| 257 |
`<mqueue>` class you create in this extension. |
| 258 |
|
| 259 |
[source,scheme] |
| 260 |
---- |
| 261 |
(declare-stub-type <mqueue> "MQueue*" "mqueue" |
| 262 |
"MQUEUE_P" "MQUEUE_UNBOX" "MQUEUE_BOX") |
| 263 |
---- |
| 264 |
|
| 265 |
The arguments are: Scheme name, C type name, description (used in |
| 266 |
error messages), C function or macro to check the type, an unboxer, |
| 267 |
and a boxer. |
| 268 |
|
| 269 |
NOTE: I'm planning to have better way to define foreign types. |
| 270 |
Consider this `declare-stub-type` as temporary solution. |
| 271 |
|
| 272 |
The `define-cproc` directive defines the foreign function interface: |
| 273 |
|
| 274 |
[source,scheme] |
| 275 |
---- |
| 276 |
(define-cproc SCHEME-NAME (ARGSPEC ...) ::RETURN-TYPE |
| 277 |
CLAUSE ...) |
| 278 |
---- |
| 279 |
|
| 280 |
`SCHEME-NAME` becomes the name of the Scheme function. `ARGSPEC` |
| 281 |
specifies the arguments and its type. |
| 282 |
|
| 283 |
---- |
| 284 |
ARGSPEC := name::type |
| 285 |
---- |
| 286 |
|
| 287 |
To see the exact meanings of type and how it is mapped in the |
| 288 |
C world, peek the source of `src/genstub` (search "Type handling"). |
| 289 |
You can use `:rest`, `:optional` and `:keyword` a la Common Lisp. |
| 290 |
|
| 291 |
`::RETURN-TYPE` specifies what type of value the function body |
| 292 |
returns. If the function doesn't need to generate a value, |
| 293 |
specify `::<void>`. The Scheme function returns `#<undef>` then. |
| 294 |
If the function generates a `ScmObj` value, you can omit `::RETURN-TYPE`; |
| 295 |
or give `::<top>`. If you give other types, such as `::<int>`, |
| 296 |
the stub generator can generate a code to convert C `int` to |
| 297 |
Scheme `integer`. |
| 298 |
|
| 299 |
`CLAUSE` gives the information about how to call the C stuff. |
| 300 |
You can write C/C++ code in S-exression (CiSE) here. Consult |
| 301 |
Gauche document for details of CiSE. |
| 302 |
|
| 303 |
Other clause type worth to mention here is `catch` clause. |
| 304 |
It handles {cpp} exceptions. |
| 305 |
|
| 306 |
Gauche has its own exception handling system, and although both |
| 307 |
can coexist, it is not allowed that one nonlocal exit jumps the |
| 308 |
dynamic environment of the other. For example, Suppose you call {cpp} |
| 309 |
function (2) from Scheme (1), which in turn calls Scheme function (3), |
| 310 |
which calls {cpp} function (4), which calls {cpp} function (5). |
| 311 |
And the {cpp} function (5) raised an exception. |
| 312 |
|
| 313 |
Scheme(1) --> C++(2) --> Scheme(3) --> C++(4) --> C++(5) |
| 314 |
throw! |
| 315 |
|
| 316 |
It is fine as far as the exception is caught within (5) or (4). |
| 317 |
|
| 318 |
Scheme(1) --> C++(2) --> Scheme(3) --> C++(4) --> C++(5) |
| 319 |
catch <----- throw |
| 320 |
|
| 321 |
|
| 322 |
However, it is not allowed to catch the exception in the function (2), |
| 323 |
since it will jump the Gauche's exception frame set up for Scheme (3). |
| 324 |
|
| 325 |
Scheme(1) --> C++(2) --> Scheme(3) --> C++(4) --> C++(5) |
| 326 |
catch <----------------------------- throw |
| 327 |
|
| 328 |
This fact mandates that if you call a {cpp} function that may throw |
| 329 |
an exception, you have to catch it within the stub routine. |
| 330 |
The typical way is to convert the caught exception to Gauche's |
| 331 |
exception. The Gauche's exception can then be caught in Scheme's |
| 332 |
`guard` form. |
| 333 |
|
| 334 |
Scheme(1) --> C++(2) --> Scheme(3) --> C++(4) --> C++(5) |
| 335 |
catch <---- throw |
| 336 |
guard <----- raise |
| 337 |
|
| 338 |
For the convenience, you can write a `catch` spec within the stub |
| 339 |
description, such as this: |
| 340 |
|
| 341 |
[source,scheme] |
| 342 |
---- |
| 343 |
(define-cproc mqueue-pop! (mq::<mqueue>) ::<const-cstring> |
| 344 |
(catch ("MQueueException& e" |
| 345 |
(Scm_Error "mqueue-pop!: %s" (ref e reason (c_str))))) |
| 346 |
(return (SCM_MAKE_STR_COPYING (ref (-> mq (popMessage)) (c_str))))) |
| 347 |
---- |
| 348 |
|
| 349 |
The `catch` spec causes the body of stub function to be surrounded |
| 350 |
by try, and appropriate catch clauses to be generated. The generated |
| 351 |
code roughly will be something like this: |
| 352 |
|
| 353 |
[source,c++] |
| 354 |
---- |
| 355 |
try { |
| 356 |
const char* result; |
| 357 |
result = mq->popMessage().c_str(); |
| 358 |
return SCM_MAKE_STR_COPYING(result); |
| 359 |
} |
| 360 |
catch (MQueueException& e) { |
| 361 |
Scm_Error("mqueue-pop!: %s", e.reason.c_str()); |
| 362 |
} |
| 363 |
catch (std::exception& e) { |
| 364 |
Scm_Error("mqueue-pop!: %s", e.what()); |
| 365 |
} |
| 366 |
catch (...) { |
| 367 |
Scm_Error("C++ exception is thrown in mqueue-pop!"); |
| 368 |
} |
| 369 |
---- |
| 370 |
|
| 371 |
Note that mere existence of `catch` spec causes the last two |
| 372 |
catch clause (`std::exception` and `+...+`) to be generated. |
| 373 |
|
| 374 |
|
| 375 |
== Caveats |
| 376 |
|
| 377 |
Be always careful about the ownership of resources. If you're |
| 378 |
within Gauche's world, most things are taken care of by its garbage |
| 379 |
collector. But once you step into the foreign land, it's up |
| 380 |
to you again to make sure all resources are managed. |
| 381 |
|
| 382 |
Especially, make sure the memory owned by Gauche is always visible |
| 383 |
from Gauche. For example, if you specify `<const-cstring>` in the |
| 384 |
argument type, Gauche converts Scheme string to NUL-terminated |
| 385 |
C string, but Gauche still owns the resulted string. So, if the |
| 386 |
foreign function retains the passed pointer within itself, such as |
| 387 |
this fictious code: |
| 388 |
|
| 389 |
[source,c++] |
| 390 |
---- |
| 391 |
/* foreign code */ |
| 392 |
static const char *ss; |
| 393 |
|
| 394 |
void foo(const char *s) |
| 395 |
{ |
| 396 |
ss = s; |
| 397 |
} |
| 398 |
---- |
| 399 |
|
| 400 |
Then, it is wrong to write a stub function like this: |
| 401 |
|
| 402 |
[source,scheme] |
| 403 |
---- |
| 404 |
;; stub function |
| 405 |
(define-cproc foo (s::<const-cstring>) ::<void> |
| 406 |
(call "foo")) |
| 407 |
---- |
| 408 |
|
| 409 |
It compiles, but the string passed to `foo` is stored |
| 410 |
in the location that Gauche doesn't know, so later the string body |
| 411 |
is GC-ed, leaving the foreign pointer dangling. |
| 412 |
|
| 413 |
The more subtle case is to pass the foreign object pointer, which |
| 414 |
itself is allocated via foreign allocator, to the foreign function. |
| 415 |
If we adopt the <<#frm-case-2,Case 2>> scheme described above, the foreign object |
| 416 |
would be destroyed when its wrapping Scheme object is GC-ed, even |
| 417 |
if the foreign object itself isn't allocated by Gauche. |