Debian Codegraph @

external/bdwgc/doc/porting.md

raw
1 # Conservative Garbage Collector Porting Directions
2
3 The collector is designed to be relatively easy to port, but is not portable
4 code per se. The collector inherently has to perform operations, such
5 as scanning the stack(s), that are not possible in portable C code.
6
7 All of the following assumes that the collector is being ported to
8 a byte-addressable 32- or 64-bit machine. Currently all successful ports
9 to 64-bit machines involve LP64 targets. The code base includes some
10 provisions for P64 targets (notably Win64), but that has not been tested. You
11 are hereby discouraged from attempting a port to non-byte-addressable,
12 or 8-bit, or 16-bit machines.
13
14 The difficulty of porting the collector varies greatly depending on the needed
15 functionality. In the simplest case, only some small additions are needed for
16 the `include/private/gcconfig.h` file. This is described in the following
17 section. Later sections discuss some of the optional features, which typically
18 involve more porting effort.
19
20 Note that the collector makes heavy use of `ifdef`s. Unlike some other
21 software projects, we have concluded repeatedly that this is preferable
22 to system dependent files, with code duplicated between the files. However,
23 to keep this manageable, we do strongly believe in indenting `ifdef`s
24 correctly (for historical reasons usually without the leading sharp sign).
25 (Separate source files are of course fine if they do not result in code
26 duplication.)
27
28 ## Adding Platforms to gcconfig.h
29
30 If neither thread support, nor tracing of dynamic library data is required,
31 these are often the only changes you will need to make.
32
33 The `gcconfig.h` file consists of three sections:
34
35 1. A section that defines GC-internal macros that identify the architecture
36 (e.g. `IA64` or `I386`) and operating system (e.g. `LINUX` or `MSWIN32`).
37 This is usually done by testing predefined macros. By defining our own
38 macros instead of using the predefined ones directly, we can impose a bit
39 more consistency, and somewhat isolate ourselves from compiler differences.
40 It is relatively straightforward to add a new entry here. But please try
41 to be consistent with the existing code. In particular, 64-bit variants
42 of 32-bit architectures general are _not_ treated as a new architecture.
43 Instead we explicitly test for 64-bit-ness in the few places in which
44 it matters. (The notable exception here is `I386` and `X86_64`. This
45 is partially historical, and partially justified by the fact that there are
46 arguably more substantial architecture and ABI differences here than for
47 RISC variants.) On GNU-based systems, `cpp -dM empty_source_file.c` seems
48 to generate a set of predefined macros. On some other systems, the "verbose"
49 compiler option may do so, or the manual page may list them.
50 2. A section that defines a small number of platform-specific macros, which
51 are then used directly by the collector. For simple ports, this is where
52 most of the effort is required. We describe the macros below. This section
53 contains a subsection for each architecture (enclosed in a suitable `ifdef`.
54 Each subsection usually contains some architecture-dependent defines,
55 followed by several sets of OS-dependent defines, again enclosed in
56 `ifdef`s.
57
58 3. A section that fills in defaults for some macros left undefined in the
59 preceding section, and defines some other macros that rarely need adjustment
60 for new platforms. You will typically not have to touch these. If you are
61 porting to an OS that was previously completely unsupported, it is likely
62 that you will need to add another clause to the definition of `GET_MEM`.
63
64 The following macros must be defined correctly for each architecture and
65 operating system:
66
67 * `MACH_TYPE` - Defined to a string that represents the machine
68 architecture. Usually just the macro name used to identify the architecture,
69 but enclosed in quotes.
70 * `OS_TYPE` - Defined to a string that represents the operating system name.
71 Usually just the macro name used to identify the operating system, but
72 enclosed in quotes.
73 * `CPP_WORDSZ` - The word size in bits as a constant suitable for
74 preprocessor tests, i.e. without casts or `sizeof` expressions. Currently
75 always defined as either 64 or 32. For platforms supporting both 32- and
76 64-bit ABIs, this should be conditionally defined depending on the current
77 ABI. There is a default of 32.
78 * `ALIGNMENT` - Defined to be the largest _N_ such that all pointer
79 are guaranteed to be aligned on _N_-byte boundaries. Defining it to be _1_
80 will always work, but perform poorly. For all modern 32-bit platforms, this
81 is 4. For all modern 64-bit platforms, this is 8. Whether or not X86
82 qualifies as a modern architecture here is compiler- and OS-dependent.
83 * `DATASTART` - The beginning of the main data segment. The collector will
84 trace all memory between `DATASTART` and `DATAEND` for root pointers.
85 On some platforms, this can be defined to a constant address, though
86 experience has shown that to be risky. Ideally the linker will define
87 a symbol (e.g. `_data` whose address is the beginning of the data segment.
88 Sometimes the value can be computed using the `GC_SysVGetDataStart`
89 function. Not used if either the next macro is defined, or if dynamic
90 loading is supported, and the dynamic loading support defines a function
91 `GC_register_main_static_data` which returns false.
92 * `SEARCH_FOR_DATA_START` - If this is defined `DATASTART` will be defined
93 to a dynamically computed value which is obtained by starting with the
94 address of `_end` and walking backwards until non-addressable memory
95 is found. This often works on Posix-like platforms. It makes it harder
96 to debug client programs, since startup involves generating and catching
97 a segmentation fault, which tends to confuse users.
98 * `DATAEND` - Set to the end of the main data segment. Defaults to `end`,
99 where that is declared as an array. This works in some cases, since the
100 linker introduces a suitable symbol.
101 * `DATASTART2`, `DATAEND2` - Some platforms have two discontiguous main data
102 segments, e.g. for initialized and uninitialized data. If so, these two
103 macros should be defined to the limits of the second main data segment.
104 * `STACK_GROWS_UP` - Should be defined if the stack (or thread stacks) grow
105 towards higher addresses. (This appears to be true only on PA-RISC. If your
106 architecture has more than one stack per thread, and is not supported yet,
107 you will need to do more work. Grep for "IA64" in the source for an
108 example.)
109 * `STACKBOTTOM` - Defined to be the cool end of the stack, which is usually
110 the highest address in the stack. It must bound the region of the stack that
111 contains pointers into the GC heap. With thread support, this must be the
112 cold end of the main stack, which typically cannot be found in the same way
113 as the other thread stacks. If this is not defined and none of the following
114 three macros is defined, client code must explicitly set `GC_stackbottom`
115 to an appropriate value before calling `GC_INIT` or any other `GC_` routine.
116 * `LINUX_STACKBOTTOM` - May be defined instead of `STACKBOTTOM`. If defined,
117 then the cold end of the stack will be determined Currently we usually read
118 it from `/proc`.
119 * `HEURISTIC1` - May be defined instead of `STACKBOTTOM`. `STACK_GRAN`
120 should generally also be redefined. The cold end of the stack is determined
121 by taking an address inside `GC_init`s frame, and rounding it up to the next
122 multiple of `STACK_GRAN`. This works well if the stack base is always
123 aligned to a large power of two. (`STACK_GRAN` is predefined to 0x1000000,
124 which is rarely optimal.)
125 * `HEURISTIC2` - May be defined instead of `STACKBOTTOM`. The cold end
126 of the stack is determined by taking an address inside `GC_init`s frame,
127 incrementing it repeatedly in small steps (decrement if `STACK_GROWS_UP`),
128 and reading the value at each location. We remember the value when the first
129 Segmentation violation or Bus error is signaled, round that to the nearest
130 plausible page boundary, and use that as the stack base.
131 * `DYNAMIC_LOADING` - Should be defined if `dyn_load.c` has been updated for
132 this platform and tracing of dynamic library roots is supported.
133 * `MPROTECT_VDB`, `PROC_VDB` - May be defined if the corresponding
134 _virtual dirty bit_ implementation in `os_dep.c` is usable on this platform.
135 This allows incremental/generational garbage collection. `MPROTECT_VDB`
136 identifies modified pages by write protecting the heap and catching faults.
137 `PROC_VDB` uses the /proc primitives to read dirty bits.
138 * `PREFETCH`, `GC_PREFETCH_FOR_WRITE` - The collector uses `PREFETCH(x)`
139 to preload the cache with the data at _x_ address. This defaults to a no-op.
140 * `CLEAR_DOUBLE` - If `CLEAR_DOUBLE` is defined, then `CLEAR_DOUBLE(x)`
141 is used as a fast way to clear the two words at `GC_malloc`-aligned address
142 _x_. By default, word stores of 0 are used instead.
143 * `HEAP_START` - May be defined as the initial address hint for mmap-based
144 allocation.
145
146 ## Additional requirements for a basic port
147
148 In some cases, you may have to add additional platform-specific code to other
149 files. A likely candidate is the implementation
150 of `GC_with_callee_saves_pushed` in `mach_dep.c`. This ensure that register
151 contents that the collector must trace from are copied to the stack. Typically
152 this can be done portably, but on some platforms it may require assembly code,
153 or just tweaking of conditional compilation tests.
154
155 For GC v7, if your platform supports `getcontext`, then defining the macro
156 `UNIX_LIKE` for your OS in `gcconfig.h` (if it is not defined there yet)
157 is likely to solve the problem. otherwise, if you are using gcc,
158 `_builtin_unwind_init` will be used, and should work fine. If that is not
159 applicable either, the implementation will try to use `setjmp`. This will work
160 if your `setjmp` implementation saves all possibly pointer-valued registers
161 into the buffer, as opposed to trying to unwind the stack at `longjmp` time.
162 The `setjmp_test` test tries to determine this, but often does not get it
163 right.
164
165 In GC v6.x versions of the collector, tracing of registers was more commonly
166 handled with assembly code. In GC v7, this is generally to be avoided.
167
168 Most commonly `os_dep.c` will not require attention, but see below.
169
170 ## Thread support
171
172 Supporting threads requires that the collector be able to find and suspend all
173 threads potentially accessing the garbage-collected heap, and locate any state
174 associated with each thread that must be traced.
175
176 The functionality needed for thread support is generally implemented in one or
177 more files specific to the particular thread interface. For example, somewhat
178 portable pthread support is implemented in `pthread_support.c` and
179 `pthread_stop_world.c`. The essential functionality consists of:
180
181 * `GC_stop_world` - Stops all threads which may access the garbage collected
182 heap, other than the caller;
183 * `GC_start_world` - Restart other threads;
184 * `GC_push_all_stacks` - Push the contents of all thread stacks (or,
185 at least, of pointer-containing regions in the thread stacks) onto the mark
186 stack.
187
188 These very often require that the garbage collector maintain its own data
189 structures to track active threads.
190
191 In addition, `LOCK` and `UNLOCK` must be implemented in `gc_locks.h`.
192
193 The easiest case is probably a new pthreads platform on which threads can be
194 stopped with signals. In this case, the changes involve:
195
196 1. Introducing a suitable `GC_xxx_THREADS` macro, which should
197 be automatically defined by `gc_config_macros.h` in the right cases.
198 It should also result in a definition of `GC_PTHREADS`, as for the existing
199 cases.
200 2. For GC v7, ensuring that the `atomic_ops` package at least minimally
201 supports the platform. If incremental GC is needed, or if pthread locks
202 do not perform adequately as the allocation lock, you will probably need
203 to ensure that a sufficient `atomic_ops` port exists for the platform
204 to provided an atomic test and set operation. The latest GC code can use
205 GCC atomic intrinsics instead of `atomic_ops` package (see
206 `include/private/gc_atomic_ops.h`).
207 3. Making any needed adjustments to `pthread_stop_world.c` and
208 `pthread_support.c`. Ideally none should be needed. In fact, not all of this
209 is as well standardized as one would like, and outright bugs requiring
210 workarounds are common. Non-preemptive threads packages will probably
211 require further work. Similarly thread-local allocation and parallel marking
212 requires further work in `pthread_support.c`, and may require better
213 `atomic_ops` support.
214
215 ## Dynamic library support
216
217 So long as `DATASTART` and `DATAEND` are defined correctly, the collector will
218 trace memory reachable from file scope or `static` variables defined as part
219 of the main executable. This is sufficient if either the program is statically
220 linked, or if pointers to the garbage-collected heap are never stored
221 in non-stack variables defined in dynamic libraries.
222
223 If dynamic library data sections must also be traced, then:
224
225 * `DYNAMIC_LOADING` must be defined in the appropriate section of
226 `gcconfig.h`.
227 * An appropriate versions of the functions `GC_register_dynamic_libraries`
228 should be defined in `dyn_load.c`. This function should invoke
229 `GC_cond_add_roots(_region_start, region_end_, TRUE)` on each dynamic
230 library data section.
231
232 Implementations that scan for writable data segments are error prone,
233 particularly in the presence of threads. They frequently result in race
234 conditions when threads exit and stacks disappear. They may also accidentally
235 trace large regions of graphics memory, or mapped files. On at least one
236 occasion they have been known to try to trace device memory that could not
237 safely be read in the manner the GC wanted to read it.
238
239 It is usually safer to walk the dynamic linker data structure, especially
240 if the linker exports an interface to do so. But beware of poorly documented
241 locking behavior in this case.
242
243 ## Incremental GC support
244
245 For incremental and generational collection to work, `os_dep.c` must contain
246 a suitable _virtual dirty bit_ implementation, which allows the collector
247 to track which heap pages (assumed to be a multiple of the collectors block
248 size) have been written during a certain time interval. The collector provides
249 several implementations, which might be adapted. The default (`DEFAULT_VDB`)
250 is a placeholder which treats all pages as having been written. This ensures
251 correctness, but renders incremental and generational collection essentially
252 useless.
253
254 ## Stack traces for debug support
255
256 If stack traces in objects are need for debug support, `GC_dave_callers` and
257 `GC_print_callers` must be implemented.
258
259 ## Disclaimer
260
261 This is an initial pass at porting guidelines. Some things have no doubt been
262 overlooked.