Debian Codegraph @

Doc/howto/free-threading-python.rst

raw
1 .. _freethreading-python-howto:
2
3 *********************************
4 Python support for free threading
5 *********************************
6
7 Starting with the 3.13 release, CPython has support for a build of
8 Python called :term:`free threading` where the :term:`global interpreter lock`
9 (GIL) is disabled. Free-threaded execution allows for full utilization of the
10 available processing power by running threads in parallel on available CPU cores.
11 While not all software will benefit from this automatically, programs
12 designed with threading in mind will run faster on multi-core hardware.
13
14 Some third-party packages, in particular ones
15 with an :term:`extension module`, may not be ready for use in a
16 free-threaded build, and will re-enable the :term:`GIL`.
17
18 This document describes the implications of free threading
19 for Python code. See :ref:`freethreading-extensions-howto` for information on
20 how to write C extensions that support the free-threaded build.
21
22 .. seealso::
23
24 :pep:`703` – Making the Global Interpreter Lock Optional in CPython for an
25 overall description of free-threaded Python.
26
27
28 Installation
29 ============
30
31 Starting with Python 3.13, the official macOS and Windows installers
32 optionally support installing free-threaded Python binaries. The installers
33 are available at https://www.python.org/downloads/.
34
35 For information on other platforms, see the `Installing a Free-Threaded Python
36 <https://py-free-threading.github.io/installing-cpython/>`_, a
37 community-maintained installation guide for installing free-threaded Python.
38
39 When building CPython from source, the :option:`--disable-gil` configure option
40 should be used to build a free-threaded Python interpreter.
41
42
43 Identifying free-threaded Python
44 ================================
45
46 To check if the current interpreter supports free-threading, :option:`python -VV <-V>`
47 and :data:`sys.version` contain "free-threading build".
48 The new :func:`sys._is_gil_enabled` function can be used to check whether
49 the GIL is actually disabled in the running process.
50
51 The ``sysconfig.get_config_var("Py_GIL_DISABLED")`` configuration variable can
52 be used to determine whether the build supports free threading. If the variable
53 is set to ``1``, then the build supports free threading. This is the recommended
54 mechanism for decisions related to the build configuration.
55
56
57 The global interpreter lock in free-threaded Python
58 ===================================================
59
60 Free-threaded builds of CPython support optionally running with the GIL enabled
61 at runtime using the environment variable :envvar:`PYTHON_GIL` or
62 the command-line option :option:`-X gil`.
63
64 The GIL may also automatically be enabled when importing a C-API extension
65 module that is not explicitly marked as supporting free threading. A warning
66 will be printed in this case.
67
68 In addition to individual package documentation, the following websites track
69 the status of popular packages support for free threading:
70
71 * https://py-free-threading.github.io/tracking/
72 * https://hugovk.github.io/free-threaded-wheels/
73
74
75 Thread safety
76 =============
77
78 The free-threaded build of CPython aims to provide similar thread-safety
79 behavior at the Python level to the default GIL-enabled build. Built-in
80 types like :class:`dict`, :class:`list`, and :class:`set` use internal locks
81 to protect against concurrent modifications in ways that behave similarly to
82 the GIL. However, Python has not historically guaranteed specific behavior for
83 concurrent modifications to these built-in types, so this should be treated
84 as a description of the current implementation, not a guarantee of current or
85 future behavior.
86
87 .. note::
88
89 It's recommended to use the :class:`threading.Lock` or other synchronization
90 primitives instead of relying on the internal locks of built-in types, when
91 possible.
92
93
94 Known limitations
95 =================
96
97 This section describes known limitations of the free-threaded CPython build.
98
99 Immortalization
100 ---------------
101
102 In the free-threaded build, some objects are :term:`immortal`.
103 Immortal objects are not deallocated and have reference counts that are
104 never modified. This is done to avoid reference count contention that would
105 prevent efficient multi-threaded scaling.
106
107 As of the 3.14 release, immortalization is limited to:
108
109 * Code constants: numeric literals, string literals, and tuple literals
110 composed of other constants.
111 * Strings interned by :func:`sys.intern`.
112
113
114 Frame objects
115 -------------
116
117 It is not safe to access :attr:`frame.f_locals` from a :ref:`frame <frame-objects>`
118 object if that frame is currently executing in another thread, and doing so may
119 crash the interpreter.
120
121
122 Iterators
123 ---------
124
125 It is generally not thread-safe to access the same iterator object from
126 multiple threads concurrently, and threads may see duplicate or missing
127 elements.
128
129
130 Single-threaded performance
131 ---------------------------
132
133 The free-threaded build has additional overhead when executing Python code
134 compared to the default GIL-enabled build. The amount of overhead depends
135 on the workload and hardware. On the pyperformance benchmark suite, the
136 average overhead ranges from about 1% on macOS aarch64 to 8% on x86-64 Linux
137 systems.
138
139
140 Behavioral changes
141 ==================
142
143 This section describes CPython behavioural changes with the free-threaded
144 build.
145
146
147 Context variables
148 -----------------
149
150 In the free-threaded build, the flag :data:`~sys.flags.thread_inherit_context`
151 is set to true by default which causes threads created with
152 :class:`threading.Thread` to start with a copy of the
153 :class:`~contextvars.Context()` of the caller of
154 :meth:`~threading.Thread.start`. In the default GIL-enabled build, the flag
155 defaults to false so threads start with an
156 empty :class:`~contextvars.Context()`.
157
158
159 Warning filters
160 ---------------
161
162 In the free-threaded build, the flag :data:`~sys.flags.context_aware_warnings`
163 is set to true by default. In the default GIL-enabled build, the flag defaults
164 to false. If the flag is true then the :class:`warnings.catch_warnings`
165 context manager uses a context variable for warning filters. If the flag is
166 false then :class:`~warnings.catch_warnings` modifies the global filters list,
167 which is not thread-safe. See the :mod:`warnings` module for more details.
168
169
170 Increased memory usage
171 ----------------------
172
173 The free-threaded build will typically use more memory compared to the default
174 build. There are multiple reasons for this, mostly due to design decisions.
175
176
177 All interned strings are immortal
178 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
179
180 For modern Python versions (since version 2.3), interning a string (e.g. with
181 :func:`sys.intern`) does not cause it to become immortal. Instead, if the last
182 reference to that string disappears, it will be removed from the interned
183 string table. This is not the case for the free-threaded build and any interned
184 string will become immortal, surviving until interpreter shutdown.
185
186
187 Non-GC objects have a larger object header
188 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
189
190 The free-threaded build uses a different :c:type:`PyObject` structure. Instead
191 of having the GC related information allocated before the :c:type:`PyObject`
192 structure, like in the default build, the GC related info is part of the normal
193 object header. For example, on the AMD64 platform, ``None`` uses 32 bytes on
194 the free-threaded build vs 16 bytes for the default build. GC objects (such as
195 dicts and lists) are the same size for both builds since the free-threaded
196 build does not use additional space for the GC info.
197
198
199 QSBR can delay freeing of memory
200 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
201
202 In order to safely implement lock-free data structures, a safe memory
203 reclamation (SMR) scheme is used, known as quiescent state-based reclamation
204 (QSBR). This means that the memory backing data structures allowing lock-free
205 access will use QSBR, which defers the free operation, rather than immediately
206 freeing the memory. Two examples of these data structures are the list object
207 and the dictionary keys object. See ``InternalDocs/qsbr.md`` in the CPython
208 source tree for more details on how QSBR is implemented. Running
209 :func:`gc.collect` should cause all memory being held by QSBR to be actually
210 freed. Note that even when QSBR frees the memory, the underlying memory
211 allocator may not immediately return that memory to the OS and so the resident
212 set size (RSS) of the process might not decrease.
213
214
215 mimalloc allocator vs pymalloc
216 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
217
218 The default build will normally use the "pymalloc" memory allocator for small
219 allocations (512 bytes or smaller). The free-threaded build does not use
220 pymalloc and allocates all Python objects using the "mimalloc" allocator. The
221 pymalloc allocator has the following properties that help keep memory usage
222 low: small per-allocated-block overhead, effective memory fragmentation
223 prevention, and quick return of free memory to the operating system. The
224 mimalloc allocator does quite well in these respects as well but can have some
225 more overhead.
226
227 In the free-threaded build, mimalloc manages memory in a number of separate
228 heaps (currently four). For example, all GC supporting objects are allocated
229 from their own heap. Using separate heaps means that free memory in one heap
230 cannot be used for an allocation that uses another heap. Also, some heaps are
231 configured to use QSBR (quiescent-state based reclamation) when freeing the
232 memory that backs up the heap (known as "pages" in mimalloc terminology). The
233 use of QSBR creates a delay between all memory blocks for a page being freed
234 and the memory page being released, either for new allocations or back to the
235 OS.
236
237 The mimalloc allocator also defers returning freed memory back to the OS. You
238 can reduce that delay by setting the environment variable
239 :envvar:`!MIMALLOC_PURGE_DELAY` to ``0``. Note that this will likely reduce
240 the performance of the allocator.
241
242
243 Free-threaded reference counting can cause objects to live longer
244 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
245
246 In the default build, when an object's reference count reaches zero, it is
247 normally deallocated. The free-threaded build uses "biased reference
248 counting", with a fast-path for objects "owned" by the current thread and a
249 slow path for other objects. See :pep:`703` for additional details. Any time
250 an object's reference count ends up in a "queued" state, deallocation can be
251 deferred. The queued state is cleared from the "eval breaker" section of the
252 bytecode evaluator.
253
254 The free-threaded build also allows a different mode of reference counting,
255 known as "deferred reference counting". This mode is enabled by setting a flag
256 on a per-object basis. Deferred reference counting is enabled for the
257 following types:
258
259 * module objects
260 * module top-level functions
261 * class methods defined in the class scope
262 * descriptor objects
263 * thread-local objects, created by :class:`threading.local`
264
265 When deferred reference counting is enabled, references from Python function
266 stacks are not added to the reference count. This scheme reduces the overhead
267 of reference counting, especially for objects used from multiple threads.
268 Because the stack references are not counted, objects with deferred reference
269 counting are not immediately freed when their internal reference count goes to
270 zero. Instead, they are examined by the next GC run and, if no stack
271 references to them are found, they are freed. This means these objects are
272 freed by the GC and not when their reference count goes to zero, as is typical.
273
274
275 Per-thread reference counting can delay freeing objects
276 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
277
278 To avoid contention on the reference count fields of frequently shared
279 objects, the free-threaded build also uses "per-thread reference counting"
280 for a few selected object types. Rather than updating a single shared
281 reference count, each thread maintains its own local reference count array,
282 indexed by a unique id assigned to the object. The true reference count is
283 only computed by summing the per-thread counts when the object's local
284 count drops to zero. Per-thread reference counting is currently used for:
285
286 * heap type objects (classes created in Python)
287 * code objects
288 * the ``__dict__`` of module objects
289
290 Because the per-thread counts must be merged back to the object before it
291 can be deallocated, objects using per-thread reference counting are
292 typically freed later than they would be in the default build. In
293 particular, such an object is usually not freed until the thread that
294 referenced it reaches a safe point (for example, in the "eval breaker"
295 section of the bytecode evaluator) or exits. Running :func:`gc.collect`
296 will merge the per-thread counts and allow these objects to be freed.