| 1 |
.. highlight:: c |
| 2 |
|
| 3 |
.. _life-cycle: |
| 4 |
|
| 5 |
Object Life Cycle |
| 6 |
================= |
| 7 |
|
| 8 |
This section explains how a type's slots relate to each other throughout the |
| 9 |
life of an object. It is not intended to be a complete canonical reference for |
| 10 |
the slots; instead, refer to the slot-specific documentation in |
| 11 |
:ref:`type-structs` for details about a particular slot. |
| 12 |
|
| 13 |
|
| 14 |
Life Events |
| 15 |
----------- |
| 16 |
|
| 17 |
The figure below illustrates the order of events that can occur throughout an |
| 18 |
object's life. An arrow from *A* to *B* indicates that event *B* can occur |
| 19 |
after event *A* has occurred, with the arrow's label indicating the condition |
| 20 |
that must be true for *B* to occur after *A*. |
| 21 |
|
| 22 |
.. only:: html and not epub |
| 23 |
|
| 24 |
.. raw:: html |
| 25 |
|
| 26 |
<style type="text/css"> |
| 27 |
|
| 28 |
.. raw:: html |
| 29 |
:file: lifecycle.dot.css |
| 30 |
|
| 31 |
.. raw:: html |
| 32 |
|
| 33 |
</style> |
| 34 |
|
| 35 |
.. raw:: html |
| 36 |
:file: lifecycle.dot.svg |
| 37 |
|
| 38 |
.. raw:: html |
| 39 |
|
| 40 |
<script> |
| 41 |
(() => { |
| 42 |
const g = document.getElementById('life_events_graph'); |
| 43 |
const title = g.querySelector(':scope > title'); |
| 44 |
title.id = 'life-events-graph-title'; |
| 45 |
const svg = g.closest('svg'); |
| 46 |
svg.role = 'img'; |
| 47 |
svg.setAttribute('aria-describedby', |
| 48 |
'life-events-graph-description'); |
| 49 |
svg.setAttribute('aria-labelledby', 'life-events-graph-title'); |
| 50 |
})(); |
| 51 |
</script> |
| 52 |
|
| 53 |
.. only:: epub or not (html or latex) |
| 54 |
|
| 55 |
.. image:: lifecycle.dot.svg |
| 56 |
:align: center |
| 57 |
:class: invert-in-dark-mode |
| 58 |
:alt: Diagram showing events in an object's life. Explained in detail below. |
| 59 |
|
| 60 |
.. only:: latex |
| 61 |
|
| 62 |
.. image:: lifecycle.dot.pdf |
| 63 |
:align: center |
| 64 |
:class: invert-in-dark-mode |
| 65 |
:alt: Diagram showing events in an object's life. Explained in detail below. |
| 66 |
|
| 67 |
.. container:: |
| 68 |
:name: life-events-graph-description |
| 69 |
|
| 70 |
Explanation: |
| 71 |
|
| 72 |
* When a new object is constructed by calling its type: |
| 73 |
|
| 74 |
#. :c:member:`~PyTypeObject.tp_new` is called to create a new object. |
| 75 |
#. :c:member:`~PyTypeObject.tp_alloc` is directly called by |
| 76 |
:c:member:`~PyTypeObject.tp_new` to allocate the memory for the new |
| 77 |
object. |
| 78 |
#. :c:member:`~PyTypeObject.tp_init` initializes the newly created object. |
| 79 |
:c:member:`!tp_init` can be called again to re-initialize an object, if |
| 80 |
desired. The :c:member:`!tp_init` call can also be skipped entirely, |
| 81 |
for example by Python code calling :py:meth:`~object.__new__`. |
| 82 |
|
| 83 |
* After :c:member:`!tp_init` completes, the object is ready to use. |
| 84 |
* Some time after the last reference to an object is removed: |
| 85 |
|
| 86 |
#. If an object is not marked as *finalized*, it might be finalized by |
| 87 |
marking it as *finalized* and calling its |
| 88 |
:c:member:`~PyTypeObject.tp_finalize` function. Python does |
| 89 |
*not* finalize an object when the last reference to it is deleted; use |
| 90 |
:c:func:`PyObject_CallFinalizerFromDealloc` to ensure that |
| 91 |
:c:member:`~PyTypeObject.tp_finalize` is always called. |
| 92 |
#. If the object is marked as finalized, |
| 93 |
:c:member:`~PyTypeObject.tp_clear` might be called by the garbage collector |
| 94 |
to clear references held by the object. It is *not* called when the |
| 95 |
object's reference count reaches zero. |
| 96 |
#. :c:member:`~PyTypeObject.tp_dealloc` is called to destroy the object. |
| 97 |
To avoid code duplication, :c:member:`~PyTypeObject.tp_dealloc` typically |
| 98 |
calls into :c:member:`~PyTypeObject.tp_clear` to free up the object's |
| 99 |
references. |
| 100 |
#. When :c:member:`~PyTypeObject.tp_dealloc` finishes object destruction, |
| 101 |
it directly calls :c:member:`~PyTypeObject.tp_free` (usually set to |
| 102 |
:c:func:`PyObject_Free` or :c:func:`PyObject_GC_Del` automatically as |
| 103 |
appropriate for the type) to deallocate the memory. |
| 104 |
|
| 105 |
* The :c:member:`~PyTypeObject.tp_finalize` function is permitted to add a |
| 106 |
reference to the object if desired. If it does, the object is |
| 107 |
*resurrected*, preventing its pending destruction. (Only |
| 108 |
:c:member:`!tp_finalize` is allowed to resurrect an object; |
| 109 |
:c:member:`~PyTypeObject.tp_clear` and |
| 110 |
:c:member:`~PyTypeObject.tp_dealloc` cannot without calling into |
| 111 |
:c:member:`!tp_finalize`.) Resurrecting an object may |
| 112 |
or may not cause the object's *finalized* mark to be removed. Currently, |
| 113 |
Python does not remove the *finalized* mark from a resurrected object if |
| 114 |
it supports garbage collection (i.e., the :c:macro:`Py_TPFLAGS_HAVE_GC` |
| 115 |
flag is set) but does remove the mark if the object does not support |
| 116 |
garbage collection; either or both of these behaviors may change in the |
| 117 |
future. |
| 118 |
* :c:member:`~PyTypeObject.tp_dealloc` can optionally call |
| 119 |
:c:member:`~PyTypeObject.tp_finalize` via |
| 120 |
:c:func:`PyObject_CallFinalizerFromDealloc` if it wishes to reuse that |
| 121 |
code to help with object destruction. This is recommended because it |
| 122 |
guarantees that :c:member:`!tp_finalize` is always called before |
| 123 |
destruction. See the :c:member:`~PyTypeObject.tp_dealloc` documentation |
| 124 |
for example code. |
| 125 |
* If the object is a member of a :term:`cyclic isolate` and either |
| 126 |
:c:member:`~PyTypeObject.tp_clear` fails to break the reference cycle or |
| 127 |
the cyclic isolate is not detected (perhaps :func:`gc.disable` was called, |
| 128 |
or the :c:macro:`Py_TPFLAGS_HAVE_GC` flag was erroneously omitted in one |
| 129 |
of the involved types), the objects remain indefinitely uncollectable |
| 130 |
(they "leak"). See :data:`gc.garbage`. |
| 131 |
|
| 132 |
If the object is marked as supporting garbage collection (the |
| 133 |
:c:macro:`Py_TPFLAGS_HAVE_GC` flag is set in |
| 134 |
:c:member:`~PyTypeObject.tp_flags`), the following events are also possible: |
| 135 |
|
| 136 |
* The garbage collector occasionally calls |
| 137 |
:c:member:`~PyTypeObject.tp_traverse` to identify :term:`cyclic isolates |
| 138 |
<cyclic isolate>`. |
| 139 |
* When the garbage collector discovers a :term:`cyclic isolate`, it |
| 140 |
finalizes one of the objects in the group by marking it as *finalized* and |
| 141 |
calling its :c:member:`~PyTypeObject.tp_finalize` function, if it has one. |
| 142 |
This repeats until the cyclic isolate doesn't exist or all of the objects |
| 143 |
have been finalized. |
| 144 |
* :c:member:`~PyTypeObject.tp_finalize` is permitted to resurrect the object |
| 145 |
by adding a reference from outside the :term:`cyclic isolate`. The new |
| 146 |
reference causes the group of objects to no longer form a cyclic isolate |
| 147 |
(the reference cycle may still exist, but if it does the objects are no |
| 148 |
longer isolated). |
| 149 |
* When the garbage collector discovers a :term:`cyclic isolate` and all of |
| 150 |
the objects in the group have already been marked as *finalized*, the |
| 151 |
garbage collector clears one or more of the uncleared objects in the group |
| 152 |
(possibly concurrently) by calling each's |
| 153 |
:c:member:`~PyTypeObject.tp_clear` function. This repeats as long as the |
| 154 |
cyclic isolate still exists and not all of the objects have been cleared. |
| 155 |
|
| 156 |
|
| 157 |
Cyclic Isolate Destruction |
| 158 |
-------------------------- |
| 159 |
|
| 160 |
Listed below are the stages of life of a hypothetical :term:`cyclic isolate` |
| 161 |
that continues to exist after each member object is finalized or cleared. It |
| 162 |
is a memory leak if a cyclic isolate progresses through all of these stages; it should |
| 163 |
vanish once all objects are cleared, if not sooner. A cyclic isolate can |
| 164 |
vanish either because the reference cycle is broken or because the objects are |
| 165 |
no longer isolated due to finalizer resurrection (see |
| 166 |
:c:member:`~PyTypeObject.tp_finalize`). |
| 167 |
|
| 168 |
0. **Reachable** (not yet a cyclic isolate): All objects are in their normal, |
| 169 |
reachable state. A reference cycle could exist, but an external reference |
| 170 |
means the objects are not yet isolated. |
| 171 |
#. **Unreachable but consistent:** The final reference from outside the cyclic |
| 172 |
group of objects has been removed, causing the objects to become isolated |
| 173 |
(thus a cyclic isolate is born). None of the group's objects have been |
| 174 |
finalized or cleared yet. The cyclic isolate remains at this stage until |
| 175 |
some future run of the garbage collector (not necessarily the next run |
| 176 |
because the next run might not scan every object). |
| 177 |
#. **Mix of finalized and not finalized:** Objects in a cyclic isolate are |
| 178 |
finalized one at a time, which means that there is a period of time when the |
| 179 |
cyclic isolate is composed of a mix of finalized and non-finalized objects. |
| 180 |
Finalization order is unspecified, so it can appear random. A finalized |
| 181 |
object must behave in a sane manner when non-finalized objects interact with |
| 182 |
it, and a non-finalized object must be able to tolerate the finalization of |
| 183 |
an arbitrary subset of its referents. |
| 184 |
#. **All finalized:** All objects in a cyclic isolate are finalized before any |
| 185 |
of them are cleared. |
| 186 |
#. **Mix of finalized and cleared:** The objects can be cleared serially or |
| 187 |
concurrently (but with the :term:`GIL` held); either way, some will finish |
| 188 |
before others. A finalized object must be able to tolerate the clearing of |
| 189 |
a subset of its referents. :pep:`442` calls this stage "cyclic trash". |
| 190 |
#. **Leaked:** If a cyclic isolate still exists after all objects in the group |
| 191 |
have been finalized and cleared, then the objects remain indefinitely |
| 192 |
uncollectable (see :data:`gc.garbage`). It is a bug if a cyclic isolate |
| 193 |
reaches this stage---it means the :c:member:`~PyTypeObject.tp_clear` methods |
| 194 |
of the participating objects have failed to break the reference cycle as |
| 195 |
required. |
| 196 |
|
| 197 |
If :c:member:`~PyTypeObject.tp_clear` did not exist, then Python would have no |
| 198 |
way to safely break a reference cycle. Simply destroying an object in a cyclic |
| 199 |
isolate would result in a dangling pointer, triggering undefined behavior when |
| 200 |
an object referencing the destroyed object is itself destroyed. The clearing |
| 201 |
step makes object destruction a two-phase process: first |
| 202 |
:c:member:`~PyTypeObject.tp_clear` is called to partially destroy the objects |
| 203 |
enough to detangle them from each other, then |
| 204 |
:c:member:`~PyTypeObject.tp_dealloc` is called to complete the destruction. |
| 205 |
|
| 206 |
Unlike clearing, finalization is not a phase of destruction. A finalized |
| 207 |
object must still behave properly by continuing to fulfill its design |
| 208 |
contracts. An object's finalizer is allowed to execute arbitrary Python code, |
| 209 |
and is even allowed to prevent the impending destruction by adding a reference. |
| 210 |
The finalizer is only related to destruction by call order---if it runs, it runs |
| 211 |
before destruction, which starts with :c:member:`~PyTypeObject.tp_clear` (if |
| 212 |
called) and concludes with :c:member:`~PyTypeObject.tp_dealloc`. |
| 213 |
|
| 214 |
The finalization step is not necessary to safely reclaim the objects in a |
| 215 |
cyclic isolate, but its existence makes it easier to design types that behave |
| 216 |
in a sane manner when objects are cleared. Clearing an object might |
| 217 |
necessarily leave it in a broken, partially destroyed state---it might be |
| 218 |
unsafe to call any of the cleared object's methods or access any of its |
| 219 |
attributes. With finalization, only finalized objects can possibly interact |
| 220 |
with cleared objects; non-finalized objects are guaranteed to interact with |
| 221 |
only non-cleared (but potentially finalized) objects. |
| 222 |
|
| 223 |
To summarize the possible interactions: |
| 224 |
|
| 225 |
* A non-finalized object might have references to or from non-finalized and |
| 226 |
finalized objects, but not to or from cleared objects. |
| 227 |
* A finalized object might have references to or from non-finalized, finalized, |
| 228 |
and cleared objects. |
| 229 |
* A cleared object might have references to or from finalized and cleared |
| 230 |
objects, but not to or from non-finalized objects. |
| 231 |
|
| 232 |
Without any reference cycles, an object can be simply destroyed once its last |
| 233 |
reference is deleted; the finalization and clearing steps are not necessary to |
| 234 |
safely reclaim unused objects. However, it can be useful to automatically call |
| 235 |
:c:member:`~PyTypeObject.tp_finalize` and :c:member:`~PyTypeObject.tp_clear` |
| 236 |
before destruction anyway because type design is simplified when all objects |
| 237 |
always experience the same series of events regardless of whether they |
| 238 |
participated in a cyclic isolate. Python currently only calls |
| 239 |
:c:member:`~PyTypeObject.tp_finalize` and :c:member:`~PyTypeObject.tp_clear` as |
| 240 |
needed to destroy a cyclic isolate; this may change in a future version. |
| 241 |
|
| 242 |
|
| 243 |
Functions |
| 244 |
--------- |
| 245 |
|
| 246 |
To allocate and free memory, see :ref:`allocating-objects`. |
| 247 |
|
| 248 |
|
| 249 |
.. c:function:: void PyObject_CallFinalizer(PyObject *op) |
| 250 |
|
| 251 |
Finalizes the object as described in :c:member:`~PyTypeObject.tp_finalize`. |
| 252 |
Call this function (or :c:func:`PyObject_CallFinalizerFromDealloc`) instead |
| 253 |
of calling :c:member:`~PyTypeObject.tp_finalize` directly because this |
| 254 |
function may deduplicate multiple calls to :c:member:`!tp_finalize`. |
| 255 |
Currently, calls are only deduplicated if the type supports garbage |
| 256 |
collection (i.e., the :c:macro:`Py_TPFLAGS_HAVE_GC` flag is set); this may |
| 257 |
change in the future. |
| 258 |
|
| 259 |
.. versionadded:: 3.4 |
| 260 |
|
| 261 |
|
| 262 |
.. c:function:: int PyObject_CallFinalizerFromDealloc(PyObject *op) |
| 263 |
|
| 264 |
Same as :c:func:`PyObject_CallFinalizer` but meant to be called at the |
| 265 |
beginning of the object's destructor (:c:member:`~PyTypeObject.tp_dealloc`). |
| 266 |
There must not be any references to the object. If the object's finalizer |
| 267 |
resurrects the object, this function returns -1; no further destruction |
| 268 |
should happen. Otherwise, this function returns 0 and destruction can |
| 269 |
continue normally. |
| 270 |
|
| 271 |
.. versionadded:: 3.4 |
| 272 |
|
| 273 |
.. seealso:: |
| 274 |
|
| 275 |
:c:member:`~PyTypeObject.tp_dealloc` for example code. |