Debian Codegraph @

Doc/c-api/gcsupport.rst

raw
1 .. highlight:: c
2
3 .. _supporting-cycle-detection:
4
5 Supporting Cyclic Garbage Collection
6 ====================================
7
8 Python's support for detecting and collecting garbage which involves circular
9 references requires support from object types which are "containers" for other
10 objects which may also be containers. Types which do not store references to
11 other objects, or which only store references to atomic types (such as numbers
12 or strings), do not need to provide any explicit support for garbage
13 collection.
14
15 To create a container type, the :c:member:`~PyTypeObject.tp_flags` field of the type object must
16 include the :c:macro:`Py_TPFLAGS_HAVE_GC` and provide an implementation of the
17 :c:member:`~PyTypeObject.tp_traverse` handler. If instances of the type are mutable, a
18 :c:member:`~PyTypeObject.tp_clear` implementation must also be provided.
19
20
21 :c:macro:`Py_TPFLAGS_HAVE_GC`
22 Objects with a type with this flag set must conform with the rules
23 documented here. For convenience these objects will be referred to as
24 container objects.
25
26 Constructors for container types must conform to two rules:
27
28 #. The memory for the object must be allocated using :c:macro:`PyObject_GC_New`
29 or :c:macro:`PyObject_GC_NewVar`.
30
31 #. Once all the fields which may contain references to other containers are
32 initialized, it must call :c:func:`PyObject_GC_Track`.
33
34 Similarly, the deallocator for the object must conform to a similar pair of
35 rules:
36
37 #. Before fields which refer to other containers are invalidated,
38 :c:func:`PyObject_GC_UnTrack` must be called.
39
40 #. The object's memory must be deallocated using :c:func:`PyObject_GC_Del`.
41
42 .. warning::
43 If a type adds the Py_TPFLAGS_HAVE_GC, then it *must* implement at least
44 a :c:member:`~PyTypeObject.tp_traverse` handler or explicitly use one
45 from its subclass or subclasses.
46
47 When calling :c:func:`PyType_Ready` or some of the APIs that indirectly
48 call it like :c:func:`PyType_FromSpecWithBases` or
49 :c:func:`PyType_FromSpec` the interpreter will automatically populate the
50 :c:member:`~PyTypeObject.tp_flags`, :c:member:`~PyTypeObject.tp_traverse`
51 and :c:member:`~PyTypeObject.tp_clear` fields if the type inherits from a
52 class that implements the garbage collector protocol and the child class
53 does *not* include the :c:macro:`Py_TPFLAGS_HAVE_GC` flag.
54
55 .. c:macro:: PyObject_GC_New(TYPE, typeobj)
56
57 Analogous to :c:macro:`PyObject_New` but for container objects with the
58 :c:macro:`Py_TPFLAGS_HAVE_GC` flag set.
59
60 Do not call this directly to allocate memory for an object; call the type's
61 :c:member:`~PyTypeObject.tp_alloc` slot instead.
62
63 When populating a type's :c:member:`~PyTypeObject.tp_alloc` slot,
64 :c:func:`PyType_GenericAlloc` is preferred over a custom function that
65 simply calls this macro.
66
67 Memory allocated by this macro must be freed with
68 :c:func:`PyObject_GC_Del` (usually called via the object's
69 :c:member:`~PyTypeObject.tp_free` slot).
70
71 .. seealso::
72
73 * :c:func:`PyObject_GC_Del`
74 * :c:macro:`PyObject_New`
75 * :c:func:`PyType_GenericAlloc`
76 * :c:member:`~PyTypeObject.tp_alloc`
77
78
79 .. c:macro:: PyObject_GC_NewVar(TYPE, typeobj, size)
80
81 Analogous to :c:macro:`PyObject_NewVar` but for container objects with the
82 :c:macro:`Py_TPFLAGS_HAVE_GC` flag set.
83
84 Do not call this directly to allocate memory for an object; call the type's
85 :c:member:`~PyTypeObject.tp_alloc` slot instead.
86
87 When populating a type's :c:member:`~PyTypeObject.tp_alloc` slot,
88 :c:func:`PyType_GenericAlloc` is preferred over a custom function that
89 simply calls this macro.
90
91 Memory allocated by this macro must be freed with
92 :c:func:`PyObject_GC_Del` (usually called via the object's
93 :c:member:`~PyTypeObject.tp_free` slot).
94
95 .. seealso::
96
97 * :c:func:`PyObject_GC_Del`
98 * :c:macro:`PyObject_NewVar`
99 * :c:func:`PyType_GenericAlloc`
100 * :c:member:`~PyTypeObject.tp_alloc`
101
102
103 .. c:function:: PyObject* PyUnstable_Object_GC_NewWithExtraData(PyTypeObject *type, size_t extra_size)
104
105 Analogous to :c:macro:`PyObject_GC_New` but allocates *extra_size*
106 bytes at the end of the object (at offset
107 :c:member:`~PyTypeObject.tp_basicsize`).
108 The allocated memory is initialized to zeros,
109 except for the :c:type:`Python object header <PyObject>`.
110
111 The extra data will be deallocated with the object, but otherwise it is
112 not managed by Python.
113
114 Memory allocated by this function must be freed with
115 :c:func:`PyObject_GC_Del` (usually called via the object's
116 :c:member:`~PyTypeObject.tp_free` slot).
117
118 .. warning::
119 The function is marked as unstable because the final mechanism
120 for reserving extra data after an instance is not yet decided.
121 For allocating a variable number of fields, prefer using
122 :c:type:`PyVarObject` and :c:member:`~PyTypeObject.tp_itemsize`
123 instead.
124
125 .. versionadded:: 3.12
126
127
128 .. c:macro:: PyObject_GC_Resize(TYPE, op, newsize)
129
130 Resize an object allocated by :c:macro:`PyObject_NewVar`.
131 Returns the resized object of type ``TYPE*`` (refers to any C type)
132 or ``NULL`` on failure.
133
134 *op* must be of type :c:expr:`PyVarObject *`
135 and must not be tracked by the collector yet.
136 *newsize* must be of type :c:type:`Py_ssize_t`.
137
138
139 .. c:function:: void PyObject_GC_Track(PyObject *op)
140
141 Adds the object *op* to the set of container objects tracked by the
142 collector. The collector can run at unexpected times so objects must be
143 valid while being tracked. This should be called once all the fields
144 followed by the :c:member:`~PyTypeObject.tp_traverse` handler become valid, usually near the
145 end of the constructor.
146
147
148 .. c:function:: int PyObject_IS_GC(PyObject *obj)
149
150 Returns non-zero if the object implements the garbage collector protocol,
151 otherwise returns 0.
152
153 The object cannot be tracked by the garbage collector if this function returns 0.
154
155
156 .. c:function:: int PyObject_GC_IsTracked(PyObject *op)
157
158 Returns 1 if the object type of *op* implements the GC protocol and *op* is being
159 currently tracked by the garbage collector and 0 otherwise.
160
161 This is analogous to the Python function :func:`gc.is_tracked`.
162
163 .. versionadded:: 3.9
164
165
166 .. c:function:: int PyObject_GC_IsFinalized(PyObject *op)
167
168 Returns 1 if the object type of *op* implements the GC protocol and *op* has been
169 already finalized by the garbage collector and 0 otherwise.
170
171 This is analogous to the Python function :func:`gc.is_finalized`.
172
173 .. versionadded:: 3.9
174
175
176 .. c:function:: void PyObject_GC_Del(void *op)
177
178 Releases memory allocated to an object using :c:macro:`PyObject_GC_New` or
179 :c:macro:`PyObject_GC_NewVar`.
180
181 Do not call this directly to free an object's memory; call the type's
182 :c:member:`~PyTypeObject.tp_free` slot instead.
183
184 Do not use this for memory allocated by :c:macro:`PyObject_New`,
185 :c:macro:`PyObject_NewVar`, or related allocation functions; use
186 :c:func:`PyObject_Free` instead.
187
188 .. seealso::
189
190 * :c:func:`PyObject_Free` is the non-GC equivalent of this function.
191 * :c:macro:`PyObject_GC_New`
192 * :c:macro:`PyObject_GC_NewVar`
193 * :c:func:`PyType_GenericAlloc`
194 * :c:member:`~PyTypeObject.tp_free`
195
196
197 .. c:function:: void PyObject_GC_UnTrack(void *op)
198
199 Remove the object *op* from the set of container objects tracked by the
200 collector. Note that :c:func:`PyObject_GC_Track` can be called again on
201 this object to add it back to the set of tracked objects. The deallocator
202 (:c:member:`~PyTypeObject.tp_dealloc` handler) should call this for the object before any of
203 the fields used by the :c:member:`~PyTypeObject.tp_traverse` handler become invalid.
204
205
206 .. versionchanged:: 3.8
207
208 The :c:func:`!_PyObject_GC_TRACK` and :c:func:`!_PyObject_GC_UNTRACK` macros
209 have been removed from the public C API.
210
211 The :c:member:`~PyTypeObject.tp_traverse` handler accepts a function parameter of this type:
212
213
214 .. c:type:: int (*visitproc)(PyObject *object, void *arg)
215
216 Type of the visitor function passed to the :c:member:`~PyTypeObject.tp_traverse` handler.
217 The function should be called with an object to traverse as *object* and
218 the third parameter to the :c:member:`~PyTypeObject.tp_traverse` handler as *arg*. The
219 Python core uses several visitor functions to implement cyclic garbage
220 detection; it's not expected that users will need to write their own
221 visitor functions.
222
223 The :c:member:`~PyTypeObject.tp_traverse` handler must have the following type:
224
225
226 .. c:type:: int (*traverseproc)(PyObject *self, visitproc visit, void *arg)
227
228 Traversal function for a container object. Implementations must call the
229 *visit* function for each object directly contained by *self*, with the
230 parameters to *visit* being the contained object and the *arg* value passed
231 to the handler. The *visit* function must not be called with a ``NULL``
232 object argument. If *visit* returns a non-zero value that value should be
233 returned immediately.
234
235 The traversal function must not have any side effects. Implementations
236 may not modify the reference counts of any Python objects nor create or
237 destroy any Python objects.
238
239 To simplify writing :c:member:`~PyTypeObject.tp_traverse` handlers, a :c:func:`Py_VISIT` macro is
240 provided. In order to use this macro, the :c:member:`~PyTypeObject.tp_traverse` implementation
241 must name its arguments exactly *visit* and *arg*:
242
243
244 .. c:macro:: Py_VISIT(o)
245
246 If the :c:expr:`PyObject *` *o* is not ``NULL``, call the *visit* callback, with arguments *o*
247 and *arg*. If *visit* returns a non-zero value, then return it.
248 Using this macro, :c:member:`~PyTypeObject.tp_traverse` handlers
249 look like::
250
251 static int
252 my_traverse(Noddy *self, visitproc visit, void *arg)
253 {
254 Py_VISIT(self->foo);
255 Py_VISIT(self->bar);
256 return 0;
257 }
258
259 The :c:member:`~PyTypeObject.tp_clear` handler must be of the :c:type:`inquiry` type, or ``NULL``
260 if the object is immutable.
261
262
263 .. c:type:: int (*inquiry)(PyObject *self)
264
265 Drop references that may have created reference cycles. Immutable objects
266 do not have to define this method since they can never directly create
267 reference cycles. Note that the object must still be valid after calling
268 this method (don't just call :c:func:`Py_DECREF` on a reference). The
269 collector will call this method if it detects that this object is involved
270 in a reference cycle.
271
272
273 Controlling the Garbage Collector State
274 ---------------------------------------
275
276 The C-API provides the following functions for controlling
277 garbage collection runs.
278
279 .. c:function:: Py_ssize_t PyGC_Collect(void)
280
281 Perform a full garbage collection, if the garbage collector is enabled.
282 (Note that :func:`gc.collect` runs it unconditionally.)
283
284 Returns the number of collected + unreachable objects which cannot
285 be collected.
286 If the garbage collector is disabled or already collecting,
287 returns ``0`` immediately.
288 Errors during garbage collection are passed to :data:`sys.unraisablehook`.
289 This function does not raise exceptions.
290
291
292 .. c:function:: int PyGC_Enable(void)
293
294 Enable the garbage collector: similar to :func:`gc.enable`.
295 Returns the previous state, 0 for disabled and 1 for enabled.
296
297 .. versionadded:: 3.10
298
299
300 .. c:function:: int PyGC_Disable(void)
301
302 Disable the garbage collector: similar to :func:`gc.disable`.
303 Returns the previous state, 0 for disabled and 1 for enabled.
304
305 .. versionadded:: 3.10
306
307
308 .. c:function:: int PyGC_IsEnabled(void)
309
310 Query the state of the garbage collector: similar to :func:`gc.isenabled`.
311 Returns the current state, 0 for disabled and 1 for enabled.
312
313 .. versionadded:: 3.10
314
315
316 Querying Garbage Collector State
317 --------------------------------
318
319 The C-API provides the following interface for querying information about
320 the garbage collector.
321
322 .. c:function:: void PyUnstable_GC_VisitObjects(gcvisitobjects_t callback, void *arg)
323
324 Run supplied *callback* on all live GC-capable objects. *arg* is passed through to
325 all invocations of *callback*.
326
327 .. warning::
328 If new objects are (de)allocated by the callback it is undefined if they
329 will be visited.
330
331 Garbage collection is disabled during operation. Explicitly running a collection
332 in the callback may lead to undefined behaviour e.g. visiting the same objects
333 multiple times or not at all.
334
335 .. versionadded:: 3.12
336
337 .. c:type:: int (*gcvisitobjects_t)(PyObject *object, void *arg)
338
339 Type of the visitor function to be passed to :c:func:`PyUnstable_GC_VisitObjects`.
340 *arg* is the same as the *arg* passed to ``PyUnstable_GC_VisitObjects``.
341 Return ``1`` to continue iteration, return ``0`` to stop iteration. Other return
342 values are reserved for now so behavior on returning anything else is undefined.
343
344 .. versionadded:: 3.12
345
346