Debian Codegraph @

examples/mqueue-cpp/README.adoc

raw
1 // some workarounds for words contain `++`
2 // https://github.com/asciidoctor/asciidoctor/issues/1208
3 :cpp: C++
4 :gpp: g++
5
6 This directory contains a sample extension module code to bridge
7 Gauche and {cpp} libraries.
8
9 The two files, link:mqueue.h[] and link:mqueue.cpp[], are our hypothetical
10 external {cpp} library. It implements a simple message queue.
11 It is independent from Gauche. Our mission here is, given this
12 mqueue library, to write a Gauche binding for it.
13
14 The following files are needed for our extension:
15
16 [horizontal]
17 link:Makefile.in[]:: template makefile
18 link:configure[]:: "configure" script
19 link:mqueue_glue.cpp[]:: a bridge (glue) between {cpp} library and Gauche
20 link:mqueue_glue.h[]:: ditto
21 link:mqueue_lib.scm[]:: Scheme binding definition
22 link:test.scm[]:: unit test
23 link:example/mqueue-cpp.scm[]:: Scheme module definition
24
25 The skeleton of those files can be generated by
26 `gauche-package` command, like this:
27
28 [source,console]
29 ----
30 % gauche-package generate mqueue-cpp example.mqueue-cpp
31 ----
32
33 I renamed the generated `mqueue_cpp.h`, `mqueue_cpp.c`, `mqueue_cpplib.scm`
34 to more descriptive names, `mqueue_glue.h`, `mqueue_glue.cpp`,
35 and `mqueue_lib.scm`, respectively.
36
37 The generated files don't know about {cpp}, so you have to tweak link:configure[]
38 and link:Makefile.in[] to enable {cpp}.
39
40 1. Add (`cf-prog-cxx`) in link:configure[]. This tries to figure out how
41 to call {cpp} compiler, and sets up a substitution variable `CXX`.
42 +
43 +NOTE:+ the {cpp} compiler has to be 'compatible' with the C
44 compiler used to build Gauche, e.g. it must accept the
45 same set of command-line options to generate a shared
46 library. Gcc and {gpp} are such compatible ones.
47
48 2. In link:Makefile.in[], added the following line to receive the {cpp}
49 compiler name `configure.in` finds.
50 +
51 [source,Makefile]
52 ----
53 CXX = @CXX@
54 ----
55
56 3. Also in link:Makefile.in[], when builing `mqueue_cpp.so`, give `--c++=$(CXX)`
57 option to the `gauche-package` script to have it use the {cpp} compiler
58 instead of the default C compiler.
59 +
60 [source,Makefile]
61 ----
62 mqueue_cpp.$(SOEXT): $(mqueue_SRCS) $(mqueue_HDRS)
63 $(GAUCHE_PACKAGE) compile --c++=$(CXX) --verbose mqueue_cpp $(mqueue_SRCS)
64 ----
65
66
67 == Glue code
68
69 Now, let's take a look at the 'glue' code.
70
71 The header file link:mqueue_glue.h[] begins with including both the Gauche
72 interface and the library interface.
73
74 [source,c++]
75 ----
76 #include <gauche.h>
77 #include <gauche/extend.h>
78 #include "mqueue.h"
79 ----
80
81 It is important to include `+<gauche.h>+` *before any other system
82 header files*. Since `gauche.h` includes some configuration
83 information (from `gauche/config.h`) which may affect the definitions
84 of the standard header files; one example is `_FILE_OFFSET_BITS`,
85 which is 32 on most 32bit Linux by default, but Gauche sets it to 64,
86 that causes the system library calls to call 64bit versions.
87
88 We need a Gauche class that represents `MQueue` object in {cpp} world.
89 We're going to use Gauche's foreign pointer feature to implement it.
90 The pointer to the Scheme `<mqueue>` class is held in this external
91 variable. It is set in the extension initialization routine.
92
93 [source,c++]
94 ----
95 extern ScmClass *MQueueClass;
96 ----
97
98 Some conveninece macros. The `<mqueue>` object in Scheme world is
99 wrapped in `ScmForeignPointer.` 'Unboxing' is to retrieve the {cpp}
100 `+MQueue*+` object from it, and 'boxing' is to wrap {cpp} object by
101 `ScmForeignPointer`.
102
103 [source,c++]
104 ----
105 #define MQUEUE_P(obj) SCM_XTYPEP(obj, MQueueClass)
106 #define MQUEUE_UNBOX(obj) SCM_FOREIGN_POINTER_REF(MQueue*, obj)
107 #define MQUEUE_BOX(ptr) Scm_MakeForeignPointer(MQueueClass, ptr)
108 ----
109
110 Next comes the declaration of the extension initialization routine.
111 It is important for it to be declared/defined in `"C"` linkage scope.
112 Macros `SCM_DECL_BEGIN` and `SCM_DECL_END` ensures it (they are the same
113 as `extern "C" {` and `}`.)
114
115 [source,c++]
116 ----
117 extern void Scm_Init_mqueue_cpp();
118 ----
119
120 The source link:mqueue_glue.cpp[] implements a bridging machinery. The
121 important part is this statement:
122
123 [source,c++]
124 ----
125 MQueueClass =
126 Scm_MakeForeignPointerClass(mod, "<mqueue>",
127 mqueue_print,
128 mqueue_cleanup,
129 SCM_FOREIGN_POINTER_KEEP_IDENTITY|SCM_FOREIGN_POINTER_MAP_NULL);
130 ----
131
132 This creates a new Scheme class, named `<mqueue>`, as a foreign pointer
133 class and stores its pointer to `MQueueClass`. Scheme variable `<mqueue>`
134 is bound to the newly created class in the module `mod`.
135
136 The second argument is a print procedure, called whenever the
137 instance of `<mqueue>` is printed.
138
139 The tricky part here is the 'cleanup' procedure. If you pass
140 a cleanup procedure, it is called when the Scheme `<mqueue>` object
141 is about to be GC-ed, so that you can free the resources in the {cpp}
142 world. Although it sounds simple, it requires more attention than
143 its superficial simplicity. If you do something wrong here,
144 you'll get a nasty bug which is very hard to track.
145
146 Typically, there are a few cases with regard to the foreign resource
147 management.
148
149 Case 1::
150 You allocate the foreign object via Gauche's allocator.
151 (`SCM_NEW` etc.) In this case, Gauche's GC can take care
152 of deallocation, so you don't need specify 'cleanup' procedure
153 except you have other resources to free (e.g. file descriptors).
154
155 anchor:frm-case-2[]Case 2::
156 The foreign object is allocated by the foreign library, and
157 it is only pointed from the Scheme world. In this case,
158 you are responsible to free the foreign object when there's
159 no reference from the Scheme world to it. You can use cleanup
160 procedure to do so.
161 +
162 Note that the cleanup procedure is called when there's
163 no reference to the Scheme `<mqueue>` object here, but it doesn't
164 necessarily means that the {cpp} `MQueue` object pointed by it can be
165 freed -- what if there are other Scheme `<mqueue>` object which
166 points to the same {cpp} `MQueue` object? Gauche's GC can't detect
167 such case, since the {cpp} `MQueue` object is outside the scope
168 of Gauche's memory management.
169 +
170 There's no universal solution for it, but Gauche provides a
171 convenience mehchanism that works in typical cases. If you
172 give a flag `SCM_FOREIGN_POINTER_KEEP_IDENTITY` to the
173 `Scm_MakeForeignPointerClass`, then Gauche guarantees that
174 `Scm_MakeForeignPointer` returns exactly same Scheme object
175 for the same pointer (internally, it uses a weak hash table to
176 map the `+void*+` pointer value to the created Scheme object).
177 If your code never assigns the wrapped foreign pointer to
178 other Scheme objects, then you can be sure that whenever the
179 Scheme `<mqueue>` object is GC-ed, the {cpp} `MQueue` object pointed
180 by it can also be destroyed.
181
182 Case 3::
183 The foreign object is allocated by the foreign library, and
184 it may be pointed from both Scheme and {cpp} worlds. In this case,
185 you cannot destroy the foreign object even when the Scheme
186 object is GC-ed, since there might be a pointer in {cpp} world
187 that points to the same foreign object. Gauche's GC cannot
188 track it.
189 +
190 This is very difficult problem indeed. Usually, if this is
191 the case, such foreign library provides some sort of resource
192 management infrastructure such as reference counting. If so,
193 you can drop the reference count in the cleanup routine,
194 provided that you do increment the reference count in the
195 boxing routine.
196 +
197 (`SCM_FOREIGN_POINTER_KEEP_IDENTITY`, for there can be more than
198 one Scheme object that points to the same foreign object).
199 +
200 Unfortunately this won't work sometimes; there are cases that
201 you also have to pass a Scheme object to the foreign library
202 (e.g. closures for callback), and sometimes they form a loop
203 of reference, spanning into both Scheme and foreign library
204 world. But it is beyond the scope of this document and I'll
205 leave the further discussion for some other time.
206
207
208 The use of `SCM_FOREIGN_POINTER_KEEP_IDENTITY` is convenient even
209 for case 1 or 3, since you can use `eq?` to test the pointer identity
210 of the foreign object. However, using it incurs overhead of
211 bookkeeping.
212
213 The other flag, `SCM_FOREIGN_POINTER_MAP_NULL`, is a convenience flag.
214 If you specify this flag, `Scm_MakeForeignPointer` returns `SCM_FALSE`
215 if `NULL` is given. It is handy if you call a foreign routine that
216 may return `NULL`; with this flag, such case can be seen from
217 Scheme as if the foreign routine returned Scheme `#f`.
218 (Be careful, however, that when you use this flag, you cannot assume
219 the returned value of `Scm_MakeForeignPointer(KLASS, PTR)` is the
220 instance of `KLASS`. If you play with the returned value in C code
221 you have to check its type.)
222
223
224 == Stub code
225
226 The link:mqueue_lib.scm[] file binds the Scheme procedures and the
227 foreign library functions. It looks like a Scheme source, and you
228 can write Scheme code in it as well, but it also contains several forms
229 that can't be executed if you load it directly to Gauche.
230
231 Such forms needs to be _precompiled_ by Gauche's AOT compiler.
232 The `gauche-package compile` command takes care of it.
233
234 Let's see those forms specific to be precompiled. Open link:mqueue_lib.scm[]
235 in a separete window and read along.
236
237 The first form, `(in-module example.mqueue-cpp)`, sets up the current
238 module much like `select-module`. However, unlike the latter, `in-module`
239 doesn't need the specified module to exist beforehand. You can't
240 just `(use example.mqueue-cpp)`, since it will try to load the compiled
241 extension, which is exactly what we're about to produce.
242
243 A quick side note: If you put Scheme code in it (which will also
244 be precompiled), be aware that you need to `use` dependent modules
245 after `in-module` form; since we don't read
246 `(define-module example.mqueue-cpp ...)`. In the example file,
247
248 The second form is `(inline-stub ...)`. It is the way to write
249 C/C++ interface code inside Scheme source. The code inside
250 `inline-stub` isn't Scheme at all; we call it _stub code_ for
251 historical reasons.
252
253 The first thing in the stub code is `(.include "mqueue_glue.h")`, which
254 is C/C++ `#include "mqueue_glue.h"`.
255
256 The `declare-stub-type` directive tells the stub generator about the
257 `<mqueue>` class you create in this extension.
258
259 [source,scheme]
260 ----
261 (declare-stub-type <mqueue> "MQueue*" "mqueue"
262 "MQUEUE_P" "MQUEUE_UNBOX" "MQUEUE_BOX")
263 ----
264
265 The arguments are: Scheme name, C type name, description (used in
266 error messages), C function or macro to check the type, an unboxer,
267 and a boxer.
268
269 NOTE: I'm planning to have better way to define foreign types.
270 Consider this `declare-stub-type` as temporary solution.
271
272 The `define-cproc` directive defines the foreign function interface:
273
274 [source,scheme]
275 ----
276 (define-cproc SCHEME-NAME (ARGSPEC ...) ::RETURN-TYPE
277 CLAUSE ...)
278 ----
279
280 `SCHEME-NAME` becomes the name of the Scheme function. `ARGSPEC`
281 specifies the arguments and its type.
282
283 ----
284 ARGSPEC := name::type
285 ----
286
287 To see the exact meanings of type and how it is mapped in the
288 C world, peek the source of `src/genstub` (search "Type handling").
289 You can use `:rest`, `:optional` and `:keyword` a la Common Lisp.
290
291 `::RETURN-TYPE` specifies what type of value the function body
292 returns. If the function doesn't need to generate a value,
293 specify `::<void>`. The Scheme function returns `#<undef>` then.
294 If the function generates a `ScmObj` value, you can omit `::RETURN-TYPE`;
295 or give `::<top>`. If you give other types, such as `::<int>`,
296 the stub generator can generate a code to convert C `int` to
297 Scheme `integer`.
298
299 `CLAUSE` gives the information about how to call the C stuff.
300 You can write C/C++ code in S-exression (CiSE) here. Consult
301 Gauche document for details of CiSE.
302
303 Other clause type worth to mention here is `catch` clause.
304 It handles {cpp} exceptions.
305
306 Gauche has its own exception handling system, and although both
307 can coexist, it is not allowed that one nonlocal exit jumps the
308 dynamic environment of the other. For example, Suppose you call {cpp}
309 function (2) from Scheme (1), which in turn calls Scheme function (3),
310 which calls {cpp} function (4), which calls {cpp} function (5).
311 And the {cpp} function (5) raised an exception.
312
313 Scheme(1) --> C++(2) --> Scheme(3) --> C++(4) --> C++(5)
314 throw!
315
316 It is fine as far as the exception is caught within (5) or (4).
317
318 Scheme(1) --> C++(2) --> Scheme(3) --> C++(4) --> C++(5)
319 catch <----- throw
320
321
322 However, it is not allowed to catch the exception in the function (2),
323 since it will jump the Gauche's exception frame set up for Scheme (3).
324
325 Scheme(1) --> C++(2) --> Scheme(3) --> C++(4) --> C++(5)
326 catch <----------------------------- throw
327
328 This fact mandates that if you call a {cpp} function that may throw
329 an exception, you have to catch it within the stub routine.
330 The typical way is to convert the caught exception to Gauche's
331 exception. The Gauche's exception can then be caught in Scheme's
332 `guard` form.
333
334 Scheme(1) --> C++(2) --> Scheme(3) --> C++(4) --> C++(5)
335 catch <---- throw
336 guard <----- raise
337
338 For the convenience, you can write a `catch` spec within the stub
339 description, such as this:
340
341 [source,scheme]
342 ----
343 (define-cproc mqueue-pop! (mq::<mqueue>) ::<const-cstring>
344 (catch ("MQueueException& e"
345 (Scm_Error "mqueue-pop!: %s" (ref e reason (c_str)))))
346 (return (SCM_MAKE_STR_COPYING (ref (-> mq (popMessage)) (c_str)))))
347 ----
348
349 The `catch` spec causes the body of stub function to be surrounded
350 by try, and appropriate catch clauses to be generated. The generated
351 code roughly will be something like this:
352
353 [source,c++]
354 ----
355 try {
356 const char* result;
357 result = mq->popMessage().c_str();
358 return SCM_MAKE_STR_COPYING(result);
359 }
360 catch (MQueueException& e) {
361 Scm_Error("mqueue-pop!: %s", e.reason.c_str());
362 }
363 catch (std::exception& e) {
364 Scm_Error("mqueue-pop!: %s", e.what());
365 }
366 catch (...) {
367 Scm_Error("C++ exception is thrown in mqueue-pop!");
368 }
369 ----
370
371 Note that mere existence of `catch` spec causes the last two
372 catch clause (`std::exception` and `+...+`) to be generated.
373
374
375 == Caveats
376
377 Be always careful about the ownership of resources. If you're
378 within Gauche's world, most things are taken care of by its garbage
379 collector. But once you step into the foreign land, it's up
380 to you again to make sure all resources are managed.
381
382 Especially, make sure the memory owned by Gauche is always visible
383 from Gauche. For example, if you specify `<const-cstring>` in the
384 argument type, Gauche converts Scheme string to NUL-terminated
385 C string, but Gauche still owns the resulted string. So, if the
386 foreign function retains the passed pointer within itself, such as
387 this fictious code:
388
389 [source,c++]
390 ----
391 /* foreign code */
392 static const char *ss;
393
394 void foo(const char *s)
395 {
396 ss = s;
397 }
398 ----
399
400 Then, it is wrong to write a stub function like this:
401
402 [source,scheme]
403 ----
404 ;; stub function
405 (define-cproc foo (s::<const-cstring>) ::<void>
406 (call "foo"))
407 ----
408
409 It compiles, but the string passed to `foo` is stored
410 in the location that Gauche doesn't know, so later the string body
411 is GC-ed, leaving the foreign pointer dangling.
412
413 The more subtle case is to pass the foreign object pointer, which
414 itself is allocated via foreign allocator, to the foreign function.
415 If we adopt the <<#frm-case-2,Case 2>> scheme described above, the foreign object
416 would be destroyed when its wrapping Scheme object is GC-ed, even
417 if the foreign object itself isn't allocated by Gauche.