// some workarounds for words contain `++` // https://github.com/asciidoctor/asciidoctor/issues/1208 :cpp: C++ :gpp: g++ This directory contains a sample extension module code to bridge Gauche and {cpp} libraries. The two files, link:mqueue.h[] and link:mqueue.cpp[], are our hypothetical external {cpp} library. It implements a simple message queue. It is independent from Gauche. Our mission here is, given this mqueue library, to write a Gauche binding for it. The following files are needed for our extension: [horizontal] link:Makefile.in[]:: template makefile link:configure[]:: "configure" script link:mqueue_glue.cpp[]:: a bridge (glue) between {cpp} library and Gauche link:mqueue_glue.h[]:: ditto link:mqueue_lib.scm[]:: Scheme binding definition link:test.scm[]:: unit test link:example/mqueue-cpp.scm[]:: Scheme module definition The skeleton of those files can be generated by `gauche-package` command, like this: [source,console] ---- % gauche-package generate mqueue-cpp example.mqueue-cpp ---- I renamed the generated `mqueue_cpp.h`, `mqueue_cpp.c`, `mqueue_cpplib.scm` to more descriptive names, `mqueue_glue.h`, `mqueue_glue.cpp`, and `mqueue_lib.scm`, respectively. The generated files don't know about {cpp}, so you have to tweak link:configure[] and link:Makefile.in[] to enable {cpp}. 1. Add (`cf-prog-cxx`) in link:configure[]. This tries to figure out how to call {cpp} compiler, and sets up a substitution variable `CXX`. + +NOTE:+ the {cpp} compiler has to be 'compatible' with the C compiler used to build Gauche, e.g. it must accept the same set of command-line options to generate a shared library. Gcc and {gpp} are such compatible ones. 2. In link:Makefile.in[], added the following line to receive the {cpp} compiler name `configure.in` finds. + [source,Makefile] ---- CXX = @CXX@ ---- 3. Also in link:Makefile.in[], when builing `mqueue_cpp.so`, give `--c++=$(CXX)` option to the `gauche-package` script to have it use the {cpp} compiler instead of the default C compiler. + [source,Makefile] ---- mqueue_cpp.$(SOEXT): $(mqueue_SRCS) $(mqueue_HDRS) $(GAUCHE_PACKAGE) compile --c++=$(CXX) --verbose mqueue_cpp $(mqueue_SRCS) ---- == Glue code Now, let's take a look at the 'glue' code. The header file link:mqueue_glue.h[] begins with including both the Gauche interface and the library interface. [source,c++] ---- #include #include #include "mqueue.h" ---- It is important to include `++` *before any other system header files*. Since `gauche.h` includes some configuration information (from `gauche/config.h`) which may affect the definitions of the standard header files; one example is `_FILE_OFFSET_BITS`, which is 32 on most 32bit Linux by default, but Gauche sets it to 64, that causes the system library calls to call 64bit versions. We need a Gauche class that represents `MQueue` object in {cpp} world. We're going to use Gauche's foreign pointer feature to implement it. The pointer to the Scheme `` class is held in this external variable. It is set in the extension initialization routine. [source,c++] ---- extern ScmClass *MQueueClass; ---- Some conveninece macros. The `` object in Scheme world is wrapped in `ScmForeignPointer.` 'Unboxing' is to retrieve the {cpp} `+MQueue*+` object from it, and 'boxing' is to wrap {cpp} object by `ScmForeignPointer`. [source,c++] ---- #define MQUEUE_P(obj) SCM_XTYPEP(obj, MQueueClass) #define MQUEUE_UNBOX(obj) SCM_FOREIGN_POINTER_REF(MQueue*, obj) #define MQUEUE_BOX(ptr) Scm_MakeForeignPointer(MQueueClass, ptr) ---- Next comes the declaration of the extension initialization routine. It is important for it to be declared/defined in `"C"` linkage scope. Macros `SCM_DECL_BEGIN` and `SCM_DECL_END` ensures it (they are the same as `extern "C" {` and `}`.) [source,c++] ---- extern void Scm_Init_mqueue_cpp(); ---- The source link:mqueue_glue.cpp[] implements a bridging machinery. The important part is this statement: [source,c++] ---- MQueueClass = Scm_MakeForeignPointerClass(mod, "", mqueue_print, mqueue_cleanup, SCM_FOREIGN_POINTER_KEEP_IDENTITY|SCM_FOREIGN_POINTER_MAP_NULL); ---- This creates a new Scheme class, named ``, as a foreign pointer class and stores its pointer to `MQueueClass`. Scheme variable `` is bound to the newly created class in the module `mod`. The second argument is a print procedure, called whenever the instance of `` is printed. The tricky part here is the 'cleanup' procedure. If you pass a cleanup procedure, it is called when the Scheme `` object is about to be GC-ed, so that you can free the resources in the {cpp} world. Although it sounds simple, it requires more attention than its superficial simplicity. If you do something wrong here, you'll get a nasty bug which is very hard to track. Typically, there are a few cases with regard to the foreign resource management. Case 1:: You allocate the foreign object via Gauche's allocator. (`SCM_NEW` etc.) In this case, Gauche's GC can take care of deallocation, so you don't need specify 'cleanup' procedure except you have other resources to free (e.g. file descriptors). anchor:frm-case-2[]Case 2:: The foreign object is allocated by the foreign library, and it is only pointed from the Scheme world. In this case, you are responsible to free the foreign object when there's no reference from the Scheme world to it. You can use cleanup procedure to do so. + Note that the cleanup procedure is called when there's no reference to the Scheme `` object here, but it doesn't necessarily means that the {cpp} `MQueue` object pointed by it can be freed -- what if there are other Scheme `` object which points to the same {cpp} `MQueue` object? Gauche's GC can't detect such case, since the {cpp} `MQueue` object is outside the scope of Gauche's memory management. + There's no universal solution for it, but Gauche provides a convenience mehchanism that works in typical cases. If you give a flag `SCM_FOREIGN_POINTER_KEEP_IDENTITY` to the `Scm_MakeForeignPointerClass`, then Gauche guarantees that `Scm_MakeForeignPointer` returns exactly same Scheme object for the same pointer (internally, it uses a weak hash table to map the `+void*+` pointer value to the created Scheme object). If your code never assigns the wrapped foreign pointer to other Scheme objects, then you can be sure that whenever the Scheme `` object is GC-ed, the {cpp} `MQueue` object pointed by it can also be destroyed. Case 3:: The foreign object is allocated by the foreign library, and it may be pointed from both Scheme and {cpp} worlds. In this case, you cannot destroy the foreign object even when the Scheme object is GC-ed, since there might be a pointer in {cpp} world that points to the same foreign object. Gauche's GC cannot track it. + This is very difficult problem indeed. Usually, if this is the case, such foreign library provides some sort of resource management infrastructure such as reference counting. If so, you can drop the reference count in the cleanup routine, provided that you do increment the reference count in the boxing routine. + (`SCM_FOREIGN_POINTER_KEEP_IDENTITY`, for there can be more than one Scheme object that points to the same foreign object). + Unfortunately this won't work sometimes; there are cases that you also have to pass a Scheme object to the foreign library (e.g. closures for callback), and sometimes they form a loop of reference, spanning into both Scheme and foreign library world. But it is beyond the scope of this document and I'll leave the further discussion for some other time. The use of `SCM_FOREIGN_POINTER_KEEP_IDENTITY` is convenient even for case 1 or 3, since you can use `eq?` to test the pointer identity of the foreign object. However, using it incurs overhead of bookkeeping. The other flag, `SCM_FOREIGN_POINTER_MAP_NULL`, is a convenience flag. If you specify this flag, `Scm_MakeForeignPointer` returns `SCM_FALSE` if `NULL` is given. It is handy if you call a foreign routine that may return `NULL`; with this flag, such case can be seen from Scheme as if the foreign routine returned Scheme `#f`. (Be careful, however, that when you use this flag, you cannot assume the returned value of `Scm_MakeForeignPointer(KLASS, PTR)` is the instance of `KLASS`. If you play with the returned value in C code you have to check its type.) == Stub code The link:mqueue_lib.scm[] file binds the Scheme procedures and the foreign library functions. It looks like a Scheme source, and you can write Scheme code in it as well, but it also contains several forms that can't be executed if you load it directly to Gauche. Such forms needs to be _precompiled_ by Gauche's AOT compiler. The `gauche-package compile` command takes care of it. Let's see those forms specific to be precompiled. Open link:mqueue_lib.scm[] in a separete window and read along. The first form, `(in-module example.mqueue-cpp)`, sets up the current module much like `select-module`. However, unlike the latter, `in-module` doesn't need the specified module to exist beforehand. You can't just `(use example.mqueue-cpp)`, since it will try to load the compiled extension, which is exactly what we're about to produce. A quick side note: If you put Scheme code in it (which will also be precompiled), be aware that you need to `use` dependent modules after `in-module` form; since we don't read `(define-module example.mqueue-cpp ...)`. In the example file, The second form is `(inline-stub ...)`. It is the way to write C/C++ interface code inside Scheme source. The code inside `inline-stub` isn't Scheme at all; we call it _stub code_ for historical reasons. The first thing in the stub code is `(.include "mqueue_glue.h")`, which is C/C++ `#include "mqueue_glue.h"`. The `declare-stub-type` directive tells the stub generator about the `` class you create in this extension. [source,scheme] ---- (declare-stub-type "MQueue*" "mqueue" "MQUEUE_P" "MQUEUE_UNBOX" "MQUEUE_BOX") ---- The arguments are: Scheme name, C type name, description (used in error messages), C function or macro to check the type, an unboxer, and a boxer. NOTE: I'm planning to have better way to define foreign types. Consider this `declare-stub-type` as temporary solution. The `define-cproc` directive defines the foreign function interface: [source,scheme] ---- (define-cproc SCHEME-NAME (ARGSPEC ...) ::RETURN-TYPE CLAUSE ...) ---- `SCHEME-NAME` becomes the name of the Scheme function. `ARGSPEC` specifies the arguments and its type. ---- ARGSPEC := name::type ---- To see the exact meanings of type and how it is mapped in the C world, peek the source of `src/genstub` (search "Type handling"). You can use `:rest`, `:optional` and `:keyword` a la Common Lisp. `::RETURN-TYPE` specifies what type of value the function body returns. If the function doesn't need to generate a value, specify `::`. The Scheme function returns `#` then. If the function generates a `ScmObj` value, you can omit `::RETURN-TYPE`; or give `::`. If you give other types, such as `::`, the stub generator can generate a code to convert C `int` to Scheme `integer`. `CLAUSE` gives the information about how to call the C stuff. You can write C/C++ code in S-exression (CiSE) here. Consult Gauche document for details of CiSE. Other clause type worth to mention here is `catch` clause. It handles {cpp} exceptions. Gauche has its own exception handling system, and although both can coexist, it is not allowed that one nonlocal exit jumps the dynamic environment of the other. For example, Suppose you call {cpp} function (2) from Scheme (1), which in turn calls Scheme function (3), which calls {cpp} function (4), which calls {cpp} function (5). And the {cpp} function (5) raised an exception. Scheme(1) --> C++(2) --> Scheme(3) --> C++(4) --> C++(5) throw! It is fine as far as the exception is caught within (5) or (4). Scheme(1) --> C++(2) --> Scheme(3) --> C++(4) --> C++(5) catch <----- throw However, it is not allowed to catch the exception in the function (2), since it will jump the Gauche's exception frame set up for Scheme (3). Scheme(1) --> C++(2) --> Scheme(3) --> C++(4) --> C++(5) catch <----------------------------- throw This fact mandates that if you call a {cpp} function that may throw an exception, you have to catch it within the stub routine. The typical way is to convert the caught exception to Gauche's exception. The Gauche's exception can then be caught in Scheme's `guard` form. Scheme(1) --> C++(2) --> Scheme(3) --> C++(4) --> C++(5) catch <---- throw guard <----- raise For the convenience, you can write a `catch` spec within the stub description, such as this: [source,scheme] ---- (define-cproc mqueue-pop! (mq::) :: (catch ("MQueueException& e" (Scm_Error "mqueue-pop!: %s" (ref e reason (c_str))))) (return (SCM_MAKE_STR_COPYING (ref (-> mq (popMessage)) (c_str))))) ---- The `catch` spec causes the body of stub function to be surrounded by try, and appropriate catch clauses to be generated. The generated code roughly will be something like this: [source,c++] ---- try { const char* result; result = mq->popMessage().c_str(); return SCM_MAKE_STR_COPYING(result); } catch (MQueueException& e) { Scm_Error("mqueue-pop!: %s", e.reason.c_str()); } catch (std::exception& e) { Scm_Error("mqueue-pop!: %s", e.what()); } catch (...) { Scm_Error("C++ exception is thrown in mqueue-pop!"); } ---- Note that mere existence of `catch` spec causes the last two catch clause (`std::exception` and `+...+`) to be generated. == Caveats Be always careful about the ownership of resources. If you're within Gauche's world, most things are taken care of by its garbage collector. But once you step into the foreign land, it's up to you again to make sure all resources are managed. Especially, make sure the memory owned by Gauche is always visible from Gauche. For example, if you specify `` in the argument type, Gauche converts Scheme string to NUL-terminated C string, but Gauche still owns the resulted string. So, if the foreign function retains the passed pointer within itself, such as this fictious code: [source,c++] ---- /* foreign code */ static const char *ss; void foo(const char *s) { ss = s; } ---- Then, it is wrong to write a stub function like this: [source,scheme] ---- ;; stub function (define-cproc foo (s::) :: (call "foo")) ---- It compiles, but the string passed to `foo` is stored in the location that Gauche doesn't know, so later the string body is GC-ed, leaving the foreign pointer dangling. The more subtle case is to pass the foreign object pointer, which itself is allocated via foreign allocator, to the foreign function. If we adopt the <<#frm-case-2,Case 2>> scheme described above, the foreign object would be destroyed when its wrapping Scheme object is GC-ed, even if the foreign object itself isn't allocated by Gauche.