| 1 |
.. highlight:: c |
| 2 |
|
| 3 |
.. _supporting-cycle-detection: |
| 4 |
|
| 5 |
Supporting Cyclic Garbage Collection |
| 6 |
==================================== |
| 7 |
|
| 8 |
Python's support for detecting and collecting garbage which involves circular |
| 9 |
references requires support from object types which are "containers" for other |
| 10 |
objects which may also be containers. Types which do not store references to |
| 11 |
other objects, or which only store references to atomic types (such as numbers |
| 12 |
or strings), do not need to provide any explicit support for garbage |
| 13 |
collection. |
| 14 |
|
| 15 |
To create a container type, the :c:member:`~PyTypeObject.tp_flags` field of the type object must |
| 16 |
include the :c:macro:`Py_TPFLAGS_HAVE_GC` and provide an implementation of the |
| 17 |
:c:member:`~PyTypeObject.tp_traverse` handler. If instances of the type are mutable, a |
| 18 |
:c:member:`~PyTypeObject.tp_clear` implementation must also be provided. |
| 19 |
|
| 20 |
|
| 21 |
:c:macro:`Py_TPFLAGS_HAVE_GC` |
| 22 |
Objects with a type with this flag set must conform with the rules |
| 23 |
documented here. For convenience these objects will be referred to as |
| 24 |
container objects. |
| 25 |
|
| 26 |
Constructors for container types must conform to two rules: |
| 27 |
|
| 28 |
#. The memory for the object must be allocated using :c:macro:`PyObject_GC_New` |
| 29 |
or :c:macro:`PyObject_GC_NewVar`. |
| 30 |
|
| 31 |
#. Once all the fields which may contain references to other containers are |
| 32 |
initialized, it must call :c:func:`PyObject_GC_Track`. |
| 33 |
|
| 34 |
Similarly, the deallocator for the object must conform to a similar pair of |
| 35 |
rules: |
| 36 |
|
| 37 |
#. Before fields which refer to other containers are invalidated, |
| 38 |
:c:func:`PyObject_GC_UnTrack` must be called. |
| 39 |
|
| 40 |
#. The object's memory must be deallocated using :c:func:`PyObject_GC_Del`. |
| 41 |
|
| 42 |
.. warning:: |
| 43 |
If a type adds the Py_TPFLAGS_HAVE_GC, then it *must* implement at least |
| 44 |
a :c:member:`~PyTypeObject.tp_traverse` handler or explicitly use one |
| 45 |
from its subclass or subclasses. |
| 46 |
|
| 47 |
When calling :c:func:`PyType_Ready` or some of the APIs that indirectly |
| 48 |
call it like :c:func:`PyType_FromSpecWithBases` or |
| 49 |
:c:func:`PyType_FromSpec` the interpreter will automatically populate the |
| 50 |
:c:member:`~PyTypeObject.tp_flags`, :c:member:`~PyTypeObject.tp_traverse` |
| 51 |
and :c:member:`~PyTypeObject.tp_clear` fields if the type inherits from a |
| 52 |
class that implements the garbage collector protocol and the child class |
| 53 |
does *not* include the :c:macro:`Py_TPFLAGS_HAVE_GC` flag. |
| 54 |
|
| 55 |
.. c:macro:: PyObject_GC_New(TYPE, typeobj) |
| 56 |
|
| 57 |
Analogous to :c:macro:`PyObject_New` but for container objects with the |
| 58 |
:c:macro:`Py_TPFLAGS_HAVE_GC` flag set. |
| 59 |
|
| 60 |
Do not call this directly to allocate memory for an object; call the type's |
| 61 |
:c:member:`~PyTypeObject.tp_alloc` slot instead. |
| 62 |
|
| 63 |
When populating a type's :c:member:`~PyTypeObject.tp_alloc` slot, |
| 64 |
:c:func:`PyType_GenericAlloc` is preferred over a custom function that |
| 65 |
simply calls this macro. |
| 66 |
|
| 67 |
Memory allocated by this macro must be freed with |
| 68 |
:c:func:`PyObject_GC_Del` (usually called via the object's |
| 69 |
:c:member:`~PyTypeObject.tp_free` slot). |
| 70 |
|
| 71 |
.. seealso:: |
| 72 |
|
| 73 |
* :c:func:`PyObject_GC_Del` |
| 74 |
* :c:macro:`PyObject_New` |
| 75 |
* :c:func:`PyType_GenericAlloc` |
| 76 |
* :c:member:`~PyTypeObject.tp_alloc` |
| 77 |
|
| 78 |
|
| 79 |
.. c:macro:: PyObject_GC_NewVar(TYPE, typeobj, size) |
| 80 |
|
| 81 |
Analogous to :c:macro:`PyObject_NewVar` but for container objects with the |
| 82 |
:c:macro:`Py_TPFLAGS_HAVE_GC` flag set. |
| 83 |
|
| 84 |
Do not call this directly to allocate memory for an object; call the type's |
| 85 |
:c:member:`~PyTypeObject.tp_alloc` slot instead. |
| 86 |
|
| 87 |
When populating a type's :c:member:`~PyTypeObject.tp_alloc` slot, |
| 88 |
:c:func:`PyType_GenericAlloc` is preferred over a custom function that |
| 89 |
simply calls this macro. |
| 90 |
|
| 91 |
Memory allocated by this macro must be freed with |
| 92 |
:c:func:`PyObject_GC_Del` (usually called via the object's |
| 93 |
:c:member:`~PyTypeObject.tp_free` slot). |
| 94 |
|
| 95 |
.. seealso:: |
| 96 |
|
| 97 |
* :c:func:`PyObject_GC_Del` |
| 98 |
* :c:macro:`PyObject_NewVar` |
| 99 |
* :c:func:`PyType_GenericAlloc` |
| 100 |
* :c:member:`~PyTypeObject.tp_alloc` |
| 101 |
|
| 102 |
|
| 103 |
.. c:function:: PyObject* PyUnstable_Object_GC_NewWithExtraData(PyTypeObject *type, size_t extra_size) |
| 104 |
|
| 105 |
Analogous to :c:macro:`PyObject_GC_New` but allocates *extra_size* |
| 106 |
bytes at the end of the object (at offset |
| 107 |
:c:member:`~PyTypeObject.tp_basicsize`). |
| 108 |
The allocated memory is initialized to zeros, |
| 109 |
except for the :c:type:`Python object header <PyObject>`. |
| 110 |
|
| 111 |
The extra data will be deallocated with the object, but otherwise it is |
| 112 |
not managed by Python. |
| 113 |
|
| 114 |
Memory allocated by this function must be freed with |
| 115 |
:c:func:`PyObject_GC_Del` (usually called via the object's |
| 116 |
:c:member:`~PyTypeObject.tp_free` slot). |
| 117 |
|
| 118 |
.. warning:: |
| 119 |
The function is marked as unstable because the final mechanism |
| 120 |
for reserving extra data after an instance is not yet decided. |
| 121 |
For allocating a variable number of fields, prefer using |
| 122 |
:c:type:`PyVarObject` and :c:member:`~PyTypeObject.tp_itemsize` |
| 123 |
instead. |
| 124 |
|
| 125 |
.. versionadded:: 3.12 |
| 126 |
|
| 127 |
|
| 128 |
.. c:macro:: PyObject_GC_Resize(TYPE, op, newsize) |
| 129 |
|
| 130 |
Resize an object allocated by :c:macro:`PyObject_NewVar`. |
| 131 |
Returns the resized object of type ``TYPE*`` (refers to any C type) |
| 132 |
or ``NULL`` on failure. |
| 133 |
|
| 134 |
*op* must be of type :c:expr:`PyVarObject *` |
| 135 |
and must not be tracked by the collector yet. |
| 136 |
*newsize* must be of type :c:type:`Py_ssize_t`. |
| 137 |
|
| 138 |
|
| 139 |
.. c:function:: void PyObject_GC_Track(PyObject *op) |
| 140 |
|
| 141 |
Adds the object *op* to the set of container objects tracked by the |
| 142 |
collector. The collector can run at unexpected times so objects must be |
| 143 |
valid while being tracked. This should be called once all the fields |
| 144 |
followed by the :c:member:`~PyTypeObject.tp_traverse` handler become valid, usually near the |
| 145 |
end of the constructor. |
| 146 |
|
| 147 |
|
| 148 |
.. c:function:: int PyObject_IS_GC(PyObject *obj) |
| 149 |
|
| 150 |
Returns non-zero if the object implements the garbage collector protocol, |
| 151 |
otherwise returns 0. |
| 152 |
|
| 153 |
The object cannot be tracked by the garbage collector if this function returns 0. |
| 154 |
|
| 155 |
|
| 156 |
.. c:function:: int PyObject_GC_IsTracked(PyObject *op) |
| 157 |
|
| 158 |
Returns 1 if the object type of *op* implements the GC protocol and *op* is being |
| 159 |
currently tracked by the garbage collector and 0 otherwise. |
| 160 |
|
| 161 |
This is analogous to the Python function :func:`gc.is_tracked`. |
| 162 |
|
| 163 |
.. versionadded:: 3.9 |
| 164 |
|
| 165 |
|
| 166 |
.. c:function:: int PyObject_GC_IsFinalized(PyObject *op) |
| 167 |
|
| 168 |
Returns 1 if the object type of *op* implements the GC protocol and *op* has been |
| 169 |
already finalized by the garbage collector and 0 otherwise. |
| 170 |
|
| 171 |
This is analogous to the Python function :func:`gc.is_finalized`. |
| 172 |
|
| 173 |
.. versionadded:: 3.9 |
| 174 |
|
| 175 |
|
| 176 |
.. c:function:: void PyObject_GC_Del(void *op) |
| 177 |
|
| 178 |
Releases memory allocated to an object using :c:macro:`PyObject_GC_New` or |
| 179 |
:c:macro:`PyObject_GC_NewVar`. |
| 180 |
|
| 181 |
Do not call this directly to free an object's memory; call the type's |
| 182 |
:c:member:`~PyTypeObject.tp_free` slot instead. |
| 183 |
|
| 184 |
Do not use this for memory allocated by :c:macro:`PyObject_New`, |
| 185 |
:c:macro:`PyObject_NewVar`, or related allocation functions; use |
| 186 |
:c:func:`PyObject_Free` instead. |
| 187 |
|
| 188 |
.. seealso:: |
| 189 |
|
| 190 |
* :c:func:`PyObject_Free` is the non-GC equivalent of this function. |
| 191 |
* :c:macro:`PyObject_GC_New` |
| 192 |
* :c:macro:`PyObject_GC_NewVar` |
| 193 |
* :c:func:`PyType_GenericAlloc` |
| 194 |
* :c:member:`~PyTypeObject.tp_free` |
| 195 |
|
| 196 |
|
| 197 |
.. c:function:: void PyObject_GC_UnTrack(void *op) |
| 198 |
|
| 199 |
Remove the object *op* from the set of container objects tracked by the |
| 200 |
collector. Note that :c:func:`PyObject_GC_Track` can be called again on |
| 201 |
this object to add it back to the set of tracked objects. The deallocator |
| 202 |
(:c:member:`~PyTypeObject.tp_dealloc` handler) should call this for the object before any of |
| 203 |
the fields used by the :c:member:`~PyTypeObject.tp_traverse` handler become invalid. |
| 204 |
|
| 205 |
|
| 206 |
.. versionchanged:: 3.8 |
| 207 |
|
| 208 |
The :c:func:`!_PyObject_GC_TRACK` and :c:func:`!_PyObject_GC_UNTRACK` macros |
| 209 |
have been removed from the public C API. |
| 210 |
|
| 211 |
The :c:member:`~PyTypeObject.tp_traverse` handler accepts a function parameter of this type: |
| 212 |
|
| 213 |
|
| 214 |
.. c:type:: int (*visitproc)(PyObject *object, void *arg) |
| 215 |
|
| 216 |
Type of the visitor function passed to the :c:member:`~PyTypeObject.tp_traverse` handler. |
| 217 |
The function should be called with an object to traverse as *object* and |
| 218 |
the third parameter to the :c:member:`~PyTypeObject.tp_traverse` handler as *arg*. The |
| 219 |
Python core uses several visitor functions to implement cyclic garbage |
| 220 |
detection; it's not expected that users will need to write their own |
| 221 |
visitor functions. |
| 222 |
|
| 223 |
The :c:member:`~PyTypeObject.tp_traverse` handler must have the following type: |
| 224 |
|
| 225 |
|
| 226 |
.. c:type:: int (*traverseproc)(PyObject *self, visitproc visit, void *arg) |
| 227 |
|
| 228 |
Traversal function for a container object. Implementations must call the |
| 229 |
*visit* function for each object directly contained by *self*, with the |
| 230 |
parameters to *visit* being the contained object and the *arg* value passed |
| 231 |
to the handler. The *visit* function must not be called with a ``NULL`` |
| 232 |
object argument. If *visit* returns a non-zero value that value should be |
| 233 |
returned immediately. |
| 234 |
|
| 235 |
The traversal function must not have any side effects. Implementations |
| 236 |
may not modify the reference counts of any Python objects nor create or |
| 237 |
destroy any Python objects. |
| 238 |
|
| 239 |
To simplify writing :c:member:`~PyTypeObject.tp_traverse` handlers, a :c:func:`Py_VISIT` macro is |
| 240 |
provided. In order to use this macro, the :c:member:`~PyTypeObject.tp_traverse` implementation |
| 241 |
must name its arguments exactly *visit* and *arg*: |
| 242 |
|
| 243 |
|
| 244 |
.. c:macro:: Py_VISIT(o) |
| 245 |
|
| 246 |
If the :c:expr:`PyObject *` *o* is not ``NULL``, call the *visit* callback, with arguments *o* |
| 247 |
and *arg*. If *visit* returns a non-zero value, then return it. |
| 248 |
Using this macro, :c:member:`~PyTypeObject.tp_traverse` handlers |
| 249 |
look like:: |
| 250 |
|
| 251 |
static int |
| 252 |
my_traverse(Noddy *self, visitproc visit, void *arg) |
| 253 |
{ |
| 254 |
Py_VISIT(self->foo); |
| 255 |
Py_VISIT(self->bar); |
| 256 |
return 0; |
| 257 |
} |
| 258 |
|
| 259 |
The :c:member:`~PyTypeObject.tp_clear` handler must be of the :c:type:`inquiry` type, or ``NULL`` |
| 260 |
if the object is immutable. |
| 261 |
|
| 262 |
|
| 263 |
.. c:type:: int (*inquiry)(PyObject *self) |
| 264 |
|
| 265 |
Drop references that may have created reference cycles. Immutable objects |
| 266 |
do not have to define this method since they can never directly create |
| 267 |
reference cycles. Note that the object must still be valid after calling |
| 268 |
this method (don't just call :c:func:`Py_DECREF` on a reference). The |
| 269 |
collector will call this method if it detects that this object is involved |
| 270 |
in a reference cycle. |
| 271 |
|
| 272 |
|
| 273 |
Controlling the Garbage Collector State |
| 274 |
--------------------------------------- |
| 275 |
|
| 276 |
The C-API provides the following functions for controlling |
| 277 |
garbage collection runs. |
| 278 |
|
| 279 |
.. c:function:: Py_ssize_t PyGC_Collect(void) |
| 280 |
|
| 281 |
Perform a full garbage collection, if the garbage collector is enabled. |
| 282 |
(Note that :func:`gc.collect` runs it unconditionally.) |
| 283 |
|
| 284 |
Returns the number of collected + unreachable objects which cannot |
| 285 |
be collected. |
| 286 |
If the garbage collector is disabled or already collecting, |
| 287 |
returns ``0`` immediately. |
| 288 |
Errors during garbage collection are passed to :data:`sys.unraisablehook`. |
| 289 |
This function does not raise exceptions. |
| 290 |
|
| 291 |
|
| 292 |
.. c:function:: int PyGC_Enable(void) |
| 293 |
|
| 294 |
Enable the garbage collector: similar to :func:`gc.enable`. |
| 295 |
Returns the previous state, 0 for disabled and 1 for enabled. |
| 296 |
|
| 297 |
.. versionadded:: 3.10 |
| 298 |
|
| 299 |
|
| 300 |
.. c:function:: int PyGC_Disable(void) |
| 301 |
|
| 302 |
Disable the garbage collector: similar to :func:`gc.disable`. |
| 303 |
Returns the previous state, 0 for disabled and 1 for enabled. |
| 304 |
|
| 305 |
.. versionadded:: 3.10 |
| 306 |
|
| 307 |
|
| 308 |
.. c:function:: int PyGC_IsEnabled(void) |
| 309 |
|
| 310 |
Query the state of the garbage collector: similar to :func:`gc.isenabled`. |
| 311 |
Returns the current state, 0 for disabled and 1 for enabled. |
| 312 |
|
| 313 |
.. versionadded:: 3.10 |
| 314 |
|
| 315 |
|
| 316 |
Querying Garbage Collector State |
| 317 |
-------------------------------- |
| 318 |
|
| 319 |
The C-API provides the following interface for querying information about |
| 320 |
the garbage collector. |
| 321 |
|
| 322 |
.. c:function:: void PyUnstable_GC_VisitObjects(gcvisitobjects_t callback, void *arg) |
| 323 |
|
| 324 |
Run supplied *callback* on all live GC-capable objects. *arg* is passed through to |
| 325 |
all invocations of *callback*. |
| 326 |
|
| 327 |
.. warning:: |
| 328 |
If new objects are (de)allocated by the callback it is undefined if they |
| 329 |
will be visited. |
| 330 |
|
| 331 |
Garbage collection is disabled during operation. Explicitly running a collection |
| 332 |
in the callback may lead to undefined behaviour e.g. visiting the same objects |
| 333 |
multiple times or not at all. |
| 334 |
|
| 335 |
.. versionadded:: 3.12 |
| 336 |
|
| 337 |
.. c:type:: int (*gcvisitobjects_t)(PyObject *object, void *arg) |
| 338 |
|
| 339 |
Type of the visitor function to be passed to :c:func:`PyUnstable_GC_VisitObjects`. |
| 340 |
*arg* is the same as the *arg* passed to ``PyUnstable_GC_VisitObjects``. |
| 341 |
Return ``1`` to continue iteration, return ``0`` to stop iteration. Other return |
| 342 |
values are reserved for now so behavior on returning anything else is undefined. |
| 343 |
|
| 344 |
.. versionadded:: 3.12 |
| 345 |
|
| 346 |
|