Debian Codegraph @

Doc/c-api/lifecycle.rst

raw
1 .. highlight:: c
2
3 .. _life-cycle:
4
5 Object Life Cycle
6 =================
7
8 This section explains how a type's slots relate to each other throughout the
9 life of an object. It is not intended to be a complete canonical reference for
10 the slots; instead, refer to the slot-specific documentation in
11 :ref:`type-structs` for details about a particular slot.
12
13
14 Life Events
15 -----------
16
17 The figure below illustrates the order of events that can occur throughout an
18 object's life. An arrow from *A* to *B* indicates that event *B* can occur
19 after event *A* has occurred, with the arrow's label indicating the condition
20 that must be true for *B* to occur after *A*.
21
22 .. only:: html and not epub
23
24 .. raw:: html
25
26 <style type="text/css">
27
28 .. raw:: html
29 :file: lifecycle.dot.css
30
31 .. raw:: html
32
33 </style>
34
35 .. raw:: html
36 :file: lifecycle.dot.svg
37
38 .. raw:: html
39
40 <script>
41 (() => {
42 const g = document.getElementById('life_events_graph');
43 const title = g.querySelector(':scope > title');
44 title.id = 'life-events-graph-title';
45 const svg = g.closest('svg');
46 svg.role = 'img';
47 svg.setAttribute('aria-describedby',
48 'life-events-graph-description');
49 svg.setAttribute('aria-labelledby', 'life-events-graph-title');
50 })();
51 </script>
52
53 .. only:: epub or not (html or latex)
54
55 .. image:: lifecycle.dot.svg
56 :align: center
57 :class: invert-in-dark-mode
58 :alt: Diagram showing events in an object's life. Explained in detail below.
59
60 .. only:: latex
61
62 .. image:: lifecycle.dot.pdf
63 :align: center
64 :class: invert-in-dark-mode
65 :alt: Diagram showing events in an object's life. Explained in detail below.
66
67 .. container::
68 :name: life-events-graph-description
69
70 Explanation:
71
72 * When a new object is constructed by calling its type:
73
74 #. :c:member:`~PyTypeObject.tp_new` is called to create a new object.
75 #. :c:member:`~PyTypeObject.tp_alloc` is directly called by
76 :c:member:`~PyTypeObject.tp_new` to allocate the memory for the new
77 object.
78 #. :c:member:`~PyTypeObject.tp_init` initializes the newly created object.
79 :c:member:`!tp_init` can be called again to re-initialize an object, if
80 desired. The :c:member:`!tp_init` call can also be skipped entirely,
81 for example by Python code calling :py:meth:`~object.__new__`.
82
83 * After :c:member:`!tp_init` completes, the object is ready to use.
84 * Some time after the last reference to an object is removed:
85
86 #. If an object is not marked as *finalized*, it might be finalized by
87 marking it as *finalized* and calling its
88 :c:member:`~PyTypeObject.tp_finalize` function. Python does
89 *not* finalize an object when the last reference to it is deleted; use
90 :c:func:`PyObject_CallFinalizerFromDealloc` to ensure that
91 :c:member:`~PyTypeObject.tp_finalize` is always called.
92 #. If the object is marked as finalized,
93 :c:member:`~PyTypeObject.tp_clear` might be called by the garbage collector
94 to clear references held by the object. It is *not* called when the
95 object's reference count reaches zero.
96 #. :c:member:`~PyTypeObject.tp_dealloc` is called to destroy the object.
97 To avoid code duplication, :c:member:`~PyTypeObject.tp_dealloc` typically
98 calls into :c:member:`~PyTypeObject.tp_clear` to free up the object's
99 references.
100 #. When :c:member:`~PyTypeObject.tp_dealloc` finishes object destruction,
101 it directly calls :c:member:`~PyTypeObject.tp_free` (usually set to
102 :c:func:`PyObject_Free` or :c:func:`PyObject_GC_Del` automatically as
103 appropriate for the type) to deallocate the memory.
104
105 * The :c:member:`~PyTypeObject.tp_finalize` function is permitted to add a
106 reference to the object if desired. If it does, the object is
107 *resurrected*, preventing its pending destruction. (Only
108 :c:member:`!tp_finalize` is allowed to resurrect an object;
109 :c:member:`~PyTypeObject.tp_clear` and
110 :c:member:`~PyTypeObject.tp_dealloc` cannot without calling into
111 :c:member:`!tp_finalize`.) Resurrecting an object may
112 or may not cause the object's *finalized* mark to be removed. Currently,
113 Python does not remove the *finalized* mark from a resurrected object if
114 it supports garbage collection (i.e., the :c:macro:`Py_TPFLAGS_HAVE_GC`
115 flag is set) but does remove the mark if the object does not support
116 garbage collection; either or both of these behaviors may change in the
117 future.
118 * :c:member:`~PyTypeObject.tp_dealloc` can optionally call
119 :c:member:`~PyTypeObject.tp_finalize` via
120 :c:func:`PyObject_CallFinalizerFromDealloc` if it wishes to reuse that
121 code to help with object destruction. This is recommended because it
122 guarantees that :c:member:`!tp_finalize` is always called before
123 destruction. See the :c:member:`~PyTypeObject.tp_dealloc` documentation
124 for example code.
125 * If the object is a member of a :term:`cyclic isolate` and either
126 :c:member:`~PyTypeObject.tp_clear` fails to break the reference cycle or
127 the cyclic isolate is not detected (perhaps :func:`gc.disable` was called,
128 or the :c:macro:`Py_TPFLAGS_HAVE_GC` flag was erroneously omitted in one
129 of the involved types), the objects remain indefinitely uncollectable
130 (they "leak"). See :data:`gc.garbage`.
131
132 If the object is marked as supporting garbage collection (the
133 :c:macro:`Py_TPFLAGS_HAVE_GC` flag is set in
134 :c:member:`~PyTypeObject.tp_flags`), the following events are also possible:
135
136 * The garbage collector occasionally calls
137 :c:member:`~PyTypeObject.tp_traverse` to identify :term:`cyclic isolates
138 <cyclic isolate>`.
139 * When the garbage collector discovers a :term:`cyclic isolate`, it
140 finalizes one of the objects in the group by marking it as *finalized* and
141 calling its :c:member:`~PyTypeObject.tp_finalize` function, if it has one.
142 This repeats until the cyclic isolate doesn't exist or all of the objects
143 have been finalized.
144 * :c:member:`~PyTypeObject.tp_finalize` is permitted to resurrect the object
145 by adding a reference from outside the :term:`cyclic isolate`. The new
146 reference causes the group of objects to no longer form a cyclic isolate
147 (the reference cycle may still exist, but if it does the objects are no
148 longer isolated).
149 * When the garbage collector discovers a :term:`cyclic isolate` and all of
150 the objects in the group have already been marked as *finalized*, the
151 garbage collector clears one or more of the uncleared objects in the group
152 (possibly concurrently) by calling each's
153 :c:member:`~PyTypeObject.tp_clear` function. This repeats as long as the
154 cyclic isolate still exists and not all of the objects have been cleared.
155
156
157 Cyclic Isolate Destruction
158 --------------------------
159
160 Listed below are the stages of life of a hypothetical :term:`cyclic isolate`
161 that continues to exist after each member object is finalized or cleared. It
162 is a memory leak if a cyclic isolate progresses through all of these stages; it should
163 vanish once all objects are cleared, if not sooner. A cyclic isolate can
164 vanish either because the reference cycle is broken or because the objects are
165 no longer isolated due to finalizer resurrection (see
166 :c:member:`~PyTypeObject.tp_finalize`).
167
168 0. **Reachable** (not yet a cyclic isolate): All objects are in their normal,
169 reachable state. A reference cycle could exist, but an external reference
170 means the objects are not yet isolated.
171 #. **Unreachable but consistent:** The final reference from outside the cyclic
172 group of objects has been removed, causing the objects to become isolated
173 (thus a cyclic isolate is born). None of the group's objects have been
174 finalized or cleared yet. The cyclic isolate remains at this stage until
175 some future run of the garbage collector (not necessarily the next run
176 because the next run might not scan every object).
177 #. **Mix of finalized and not finalized:** Objects in a cyclic isolate are
178 finalized one at a time, which means that there is a period of time when the
179 cyclic isolate is composed of a mix of finalized and non-finalized objects.
180 Finalization order is unspecified, so it can appear random. A finalized
181 object must behave in a sane manner when non-finalized objects interact with
182 it, and a non-finalized object must be able to tolerate the finalization of
183 an arbitrary subset of its referents.
184 #. **All finalized:** All objects in a cyclic isolate are finalized before any
185 of them are cleared.
186 #. **Mix of finalized and cleared:** The objects can be cleared serially or
187 concurrently (but with the :term:`GIL` held); either way, some will finish
188 before others. A finalized object must be able to tolerate the clearing of
189 a subset of its referents. :pep:`442` calls this stage "cyclic trash".
190 #. **Leaked:** If a cyclic isolate still exists after all objects in the group
191 have been finalized and cleared, then the objects remain indefinitely
192 uncollectable (see :data:`gc.garbage`). It is a bug if a cyclic isolate
193 reaches this stage---it means the :c:member:`~PyTypeObject.tp_clear` methods
194 of the participating objects have failed to break the reference cycle as
195 required.
196
197 If :c:member:`~PyTypeObject.tp_clear` did not exist, then Python would have no
198 way to safely break a reference cycle. Simply destroying an object in a cyclic
199 isolate would result in a dangling pointer, triggering undefined behavior when
200 an object referencing the destroyed object is itself destroyed. The clearing
201 step makes object destruction a two-phase process: first
202 :c:member:`~PyTypeObject.tp_clear` is called to partially destroy the objects
203 enough to detangle them from each other, then
204 :c:member:`~PyTypeObject.tp_dealloc` is called to complete the destruction.
205
206 Unlike clearing, finalization is not a phase of destruction. A finalized
207 object must still behave properly by continuing to fulfill its design
208 contracts. An object's finalizer is allowed to execute arbitrary Python code,
209 and is even allowed to prevent the impending destruction by adding a reference.
210 The finalizer is only related to destruction by call order---if it runs, it runs
211 before destruction, which starts with :c:member:`~PyTypeObject.tp_clear` (if
212 called) and concludes with :c:member:`~PyTypeObject.tp_dealloc`.
213
214 The finalization step is not necessary to safely reclaim the objects in a
215 cyclic isolate, but its existence makes it easier to design types that behave
216 in a sane manner when objects are cleared. Clearing an object might
217 necessarily leave it in a broken, partially destroyed state---it might be
218 unsafe to call any of the cleared object's methods or access any of its
219 attributes. With finalization, only finalized objects can possibly interact
220 with cleared objects; non-finalized objects are guaranteed to interact with
221 only non-cleared (but potentially finalized) objects.
222
223 To summarize the possible interactions:
224
225 * A non-finalized object might have references to or from non-finalized and
226 finalized objects, but not to or from cleared objects.
227 * A finalized object might have references to or from non-finalized, finalized,
228 and cleared objects.
229 * A cleared object might have references to or from finalized and cleared
230 objects, but not to or from non-finalized objects.
231
232 Without any reference cycles, an object can be simply destroyed once its last
233 reference is deleted; the finalization and clearing steps are not necessary to
234 safely reclaim unused objects. However, it can be useful to automatically call
235 :c:member:`~PyTypeObject.tp_finalize` and :c:member:`~PyTypeObject.tp_clear`
236 before destruction anyway because type design is simplified when all objects
237 always experience the same series of events regardless of whether they
238 participated in a cyclic isolate. Python currently only calls
239 :c:member:`~PyTypeObject.tp_finalize` and :c:member:`~PyTypeObject.tp_clear` as
240 needed to destroy a cyclic isolate; this may change in a future version.
241
242
243 Functions
244 ---------
245
246 To allocate and free memory, see :ref:`allocating-objects`.
247
248
249 .. c:function:: void PyObject_CallFinalizer(PyObject *op)
250
251 Finalizes the object as described in :c:member:`~PyTypeObject.tp_finalize`.
252 Call this function (or :c:func:`PyObject_CallFinalizerFromDealloc`) instead
253 of calling :c:member:`~PyTypeObject.tp_finalize` directly because this
254 function may deduplicate multiple calls to :c:member:`!tp_finalize`.
255 Currently, calls are only deduplicated if the type supports garbage
256 collection (i.e., the :c:macro:`Py_TPFLAGS_HAVE_GC` flag is set); this may
257 change in the future.
258
259 .. versionadded:: 3.4
260
261
262 .. c:function:: int PyObject_CallFinalizerFromDealloc(PyObject *op)
263
264 Same as :c:func:`PyObject_CallFinalizer` but meant to be called at the
265 beginning of the object's destructor (:c:member:`~PyTypeObject.tp_dealloc`).
266 There must not be any references to the object. If the object's finalizer
267 resurrects the object, this function returns -1; no further destruction
268 should happen. Otherwise, this function returns 0 and destruction can
269 continue normally.
270
271 .. versionadded:: 3.4
272
273 .. seealso::
274
275 :c:member:`~PyTypeObject.tp_dealloc` for example code.