Debian Codegraph @

Doc/howto/free-threading-extensions.rst

raw
1 .. highlight:: c
2
3 .. _freethreading-extensions-howto:
4
5 ******************************************
6 C API Extension Support for Free Threading
7 ******************************************
8
9 Starting with the 3.13 release, CPython has support for running with
10 the :term:`global interpreter lock` (GIL) disabled in a configuration
11 called :term:`free threading`. This document describes how to adapt C API
12 extensions to support free threading.
13
14
15 Identifying the Free-Threaded Build in C
16 ========================================
17
18 The CPython C API exposes the ``Py_GIL_DISABLED`` macro: in the free-threaded
19 build it's defined to ``1``, and in the regular build it's not defined.
20 You can use it to enable code that only runs under the free-threaded build::
21
22 #ifdef Py_GIL_DISABLED
23 /* code that only runs in the free-threaded build */
24 #endif
25
26 .. note::
27
28 On Windows, this macro is not defined automatically, but must be specified
29 to the compiler when building. The :func:`sysconfig.get_config_var` function
30 can be used to determine whether the current running interpreter had the
31 macro defined.
32
33
34 Module Initialization
35 =====================
36
37 Extension modules need to explicitly indicate that they support running with
38 the GIL disabled; otherwise importing the extension will raise a warning and
39 enable the GIL at runtime.
40
41 There are two ways to indicate that an extension module supports running with
42 the GIL disabled depending on whether the extension uses multi-phase or
43 single-phase initialization.
44
45 Multi-Phase Initialization
46 ..........................
47
48 Extensions that use multi-phase initialization (i.e.,
49 :c:func:`PyModuleDef_Init`) should add a :c:data:`Py_mod_gil` slot in the
50 module definition. If your extension supports older versions of CPython,
51 you should guard the slot with a :c:data:`PY_VERSION_HEX` check.
52
53 ::
54
55 static struct PyModuleDef_Slot module_slots[] = {
56 ...
57 #if PY_VERSION_HEX >= 0x030D0000
58 {Py_mod_gil, Py_MOD_GIL_NOT_USED},
59 #endif
60 {0, NULL}
61 };
62
63 static struct PyModuleDef moduledef = {
64 PyModuleDef_HEAD_INIT,
65 .m_slots = module_slots,
66 ...
67 };
68
69
70 Single-Phase Initialization
71 ...........................
72
73 Extensions that use single-phase initialization (i.e.,
74 :c:func:`PyModule_Create`) should call :c:func:`PyUnstable_Module_SetGIL` to
75 indicate that they support running with the GIL disabled. The function is
76 only defined in the free-threaded build, so you should guard the call with
77 ``#ifdef Py_GIL_DISABLED`` to avoid compilation errors in the regular build.
78
79 ::
80
81 static struct PyModuleDef moduledef = {
82 PyModuleDef_HEAD_INIT,
83 ...
84 };
85
86 PyMODINIT_FUNC
87 PyInit_mymodule(void)
88 {
89 PyObject *m = PyModule_Create(&moduledef);
90 if (m == NULL) {
91 return NULL;
92 }
93 #ifdef Py_GIL_DISABLED
94 PyUnstable_Module_SetGIL(m, Py_MOD_GIL_NOT_USED);
95 #endif
96 return m;
97 }
98
99
100 General API Guidelines
101 ======================
102
103 Most of the C API is thread-safe, but there are some exceptions.
104
105 * **Struct Fields**: Accessing fields in Python C API objects or structs
106 directly is not thread-safe if the field may be concurrently modified.
107 * **Macros**: Accessor macros like :c:macro:`PyList_GET_ITEM`,
108 :c:macro:`PyList_SET_ITEM`, and macros like
109 :c:macro:`PySequence_Fast_GET_SIZE` that use the object returned by
110 :c:func:`PySequence_Fast` do not perform any error checking or locking.
111 These macros are not thread-safe if the container object may be modified
112 concurrently.
113 * **Borrowed References**: C API functions that return
114 :term:`borrowed references <borrowed reference>` may not be thread-safe if
115 the containing object is modified concurrently. See the section on
116 :ref:`borrowed references <borrowed-references>` for more information.
117
118
119 Container Thread Safety
120 .......................
121
122 Containers like :c:struct:`PyListObject`,
123 :c:struct:`PyDictObject`, and :c:struct:`PySetObject` perform internal locking
124 in the free-threaded build. For example, the :c:func:`PyList_Append` will
125 lock the list before appending an item.
126
127 .. _PyDict_Next:
128
129 ``PyDict_Next``
130 '''''''''''''''
131
132 A notable exception is :c:func:`PyDict_Next`, which does not lock the
133 dictionary. You should use :c:macro:`Py_BEGIN_CRITICAL_SECTION` to protect
134 the dictionary while iterating over it if the dictionary may be concurrently
135 modified::
136
137 Py_BEGIN_CRITICAL_SECTION(dict);
138 PyObject *key, *value;
139 Py_ssize_t pos = 0;
140 while (PyDict_Next(dict, &pos, &key, &value)) {
141 ...
142 }
143 Py_END_CRITICAL_SECTION();
144
145
146 Borrowed References
147 ===================
148
149 .. _borrowed-references:
150
151 Some C API functions return :term:`borrowed references <borrowed reference>`.
152 These APIs are not thread-safe if the containing object is modified
153 concurrently. For example, it's not safe to use :c:func:`PyList_GetItem`
154 if the list may be modified concurrently.
155
156 The following table lists some borrowed reference APIs and their replacements
157 that return :term:`strong references <strong reference>`.
158
159 +-----------------------------------+-----------------------------------+
160 | Borrowed reference API | Strong reference API |
161 +===================================+===================================+
162 | :c:func:`PyList_GetItem` | :c:func:`PyList_GetItemRef` |
163 +-----------------------------------+-----------------------------------+
164 | :c:func:`PyList_GET_ITEM` | :c:func:`PyList_GetItemRef` |
165 +-----------------------------------+-----------------------------------+
166 | :c:func:`PyDict_GetItem` | :c:func:`PyDict_GetItemRef` |
167 +-----------------------------------+-----------------------------------+
168 | :c:func:`PyDict_GetItemWithError` | :c:func:`PyDict_GetItemRef` |
169 +-----------------------------------+-----------------------------------+
170 | :c:func:`PyDict_GetItemString` | :c:func:`PyDict_GetItemStringRef` |
171 +-----------------------------------+-----------------------------------+
172 | :c:func:`PyDict_SetDefault` | :c:func:`PyDict_SetDefaultRef` |
173 +-----------------------------------+-----------------------------------+
174 | :c:func:`PyDict_Next` | none (see :ref:`PyDict_Next`) |
175 +-----------------------------------+-----------------------------------+
176 | :c:func:`PyWeakref_GetObject` | :c:func:`PyWeakref_GetRef` |
177 +-----------------------------------+-----------------------------------+
178 | :c:func:`PyWeakref_GET_OBJECT` | :c:func:`PyWeakref_GetRef` |
179 +-----------------------------------+-----------------------------------+
180 | :c:func:`PyImport_AddModule` | :c:func:`PyImport_AddModuleRef` |
181 +-----------------------------------+-----------------------------------+
182 | :c:func:`PyCell_GET` | :c:func:`PyCell_Get` |
183 +-----------------------------------+-----------------------------------+
184
185 Not all APIs that return borrowed references are problematic. For
186 example, :c:func:`PyTuple_GetItem` is safe because tuples are immutable.
187 Similarly, not all uses of the above APIs are problematic. For example,
188 :c:func:`PyDict_GetItem` is often used for parsing keyword argument
189 dictionaries in function calls; those keyword argument dictionaries are
190 effectively private (not accessible by other threads), so using borrowed
191 references in that context is safe.
192
193 Some of these functions were added in Python 3.13. You can use the
194 `pythoncapi-compat <https://github.com/python/pythoncapi-compat>`_ package
195 to provide implementations of these functions for older Python versions.
196
197
198 .. _free-threaded-memory-allocation:
199
200 Memory Allocation APIs
201 ======================
202
203 Python's memory management C API provides functions in three different
204 :ref:`allocation domains <allocator-domains>`: "raw", "mem", and "object".
205 For thread-safety, the free-threaded build requires that only Python objects
206 are allocated using the object domain, and that all Python objects are
207 allocated using that domain. This differs from the prior Python versions,
208 where this was only a best practice and not a hard requirement.
209
210 .. note::
211
212 Search for uses of :c:func:`PyObject_Malloc` in your
213 extension and check that the allocated memory is used for Python objects.
214 Use :c:func:`PyMem_Malloc` to allocate buffers instead of
215 :c:func:`PyObject_Malloc`.
216
217
218 Thread State and GIL APIs
219 =========================
220
221 Python provides a set of functions and macros to manage thread state and the
222 GIL, such as:
223
224 * :c:func:`PyGILState_Ensure` and :c:func:`PyGILState_Release`
225 * :c:func:`PyEval_SaveThread` and :c:func:`PyEval_RestoreThread`
226 * :c:macro:`Py_BEGIN_ALLOW_THREADS` and :c:macro:`Py_END_ALLOW_THREADS`
227
228 These functions should still be used in the free-threaded build to manage
229 thread state even when the :term:`GIL` is disabled. For example, if you
230 create a thread outside of Python, you must call :c:func:`PyGILState_Ensure`
231 before calling into the Python API to ensure that the thread has a valid
232 Python thread state.
233
234 You should continue to call :c:func:`PyEval_SaveThread` or
235 :c:macro:`Py_BEGIN_ALLOW_THREADS` around blocking operations, such as I/O or
236 lock acquisitions, to allow other threads to run the
237 :term:`cyclic garbage collector <garbage collection>`.
238
239
240 Protecting Internal Extension State
241 ===================================
242
243 Your extension may have internal state that was previously protected by the
244 GIL. You may need to add locking to protect this state. The approach will
245 depend on your extension, but some common patterns include:
246
247 * **Caches**: global caches are a common source of shared state. Consider
248 using a lock to protect the cache or disabling it in the free-threaded build
249 if the cache is not critical for performance.
250 * **Global State**: global state may need to be protected by a lock or moved
251 to thread local storage. C11 and C++11 provide the ``thread_local`` or
252 ``_Thread_local`` for
253 `thread-local storage <https://en.cppreference.com/w/c/language/storage_duration>`_.
254
255
256 Critical Sections
257 =================
258
259 .. _critical-sections:
260
261 In the free-threaded build, CPython provides a mechanism called "critical
262 sections" to protect data that would otherwise be protected by the GIL.
263 While extension authors may not interact with the internal critical section
264 implementation directly, understanding their behavior is crucial when using
265 certain C API functions or managing shared state in the free-threaded build.
266
267 What Are Critical Sections?
268 ...........................
269
270 Conceptually, critical sections act as a deadlock avoidance layer built on
271 top of simple mutexes. Each thread maintains a stack of active critical
272 sections. When a thread needs to acquire a lock associated with a critical
273 section (e.g., implicitly when calling a thread-safe C API function like
274 :c:func:`PyDict_SetItem`, or explicitly using macros), it attempts to acquire
275 the underlying mutex.
276
277 Using Critical Sections
278 .......................
279
280 The primary APIs for using critical sections are:
281
282 * :c:macro:`Py_BEGIN_CRITICAL_SECTION` and :c:macro:`Py_END_CRITICAL_SECTION` -
283 For locking a single object
284
285 * :c:macro:`Py_BEGIN_CRITICAL_SECTION2` and :c:macro:`Py_END_CRITICAL_SECTION2`
286 - For locking two objects simultaneously
287
288 These macros must be used in matching pairs and must appear in the same C
289 scope, since they establish a new local scope. These macros are no-ops in
290 non-free-threaded builds, so they can be safely added to code that needs to
291 support both build types.
292
293 A common use of a critical section would be to lock an object while accessing
294 an internal attribute of it. For example, if an extension type has an internal
295 count field, you could use a critical section while reading or writing that
296 field::
297
298 // read the count, returns new reference to internal count value
299 PyObject *result;
300 Py_BEGIN_CRITICAL_SECTION(obj);
301 result = Py_NewRef(obj->count);
302 Py_END_CRITICAL_SECTION();
303 return result;
304
305 // write the count, consumes reference from new_count
306 Py_BEGIN_CRITICAL_SECTION(obj);
307 obj->count = new_count;
308 Py_END_CRITICAL_SECTION();
309
310
311 How Critical Sections Work
312 ..........................
313
314 Unlike traditional locks, critical sections do not guarantee exclusive access
315 throughout their entire duration. If a thread would block while holding a
316 critical section (e.g., by acquiring another lock or performing I/O), the
317 critical section is temporarily suspended—all locks are released—and then
318 resumed when the blocking operation completes.
319
320 This behavior is similar to what happens with the GIL when a thread makes a
321 blocking call. The key differences are:
322
323 * Critical sections operate on a per-object basis rather than globally
324
325 * Critical sections follow a stack discipline within each thread (the "begin" and
326 "end" macros enforce this since they must be paired and within the same scope)
327
328 * Critical sections automatically release and reacquire locks around potential
329 blocking operations
330
331 Deadlock Avoidance
332 ..................
333
334 Critical sections help avoid deadlocks in two ways:
335
336 1. If a thread tries to acquire a lock that's already held by another thread,
337 it first suspends all of its active critical sections, temporarily releasing
338 their locks
339
340 2. When the blocking operation completes, only the top-most critical section is
341 reacquired first
342
343 This means you cannot rely on nested critical sections to lock multiple objects
344 at once, as the inner critical section may suspend the outer ones. Instead, use
345 :c:macro:`Py_BEGIN_CRITICAL_SECTION2` to lock two objects simultaneously.
346
347 Note that the locks described above are only :c:type:`PyMutex` based locks.
348 The critical section implementation does not know about or affect other locking
349 mechanisms that might be in use, like POSIX mutexes. Also note that while
350 blocking on any :c:type:`PyMutex` causes the critical sections to be
351 suspended, only the mutexes that are part of the critical sections are
352 released. If :c:type:`PyMutex` is used without a critical section, it will
353 not be released and therefore does not get the same deadlock avoidance.
354
355 Important Considerations
356 ........................
357
358 * Critical sections may temporarily release their locks, allowing other threads
359 to modify the protected data. Be careful about making assumptions about the
360 state of the data after operations that might block.
361
362 * Because locks can be temporarily released (suspended), entering a critical
363 section does not guarantee exclusive access to the protected resource
364 throughout the section's duration. If code within a critical section calls
365 another function that blocks (e.g., acquires another lock, performs blocking
366 I/O), all locks held by the thread via critical sections will be released.
367 This is similar to how the GIL can be released during blocking calls.
368
369 * Only the lock(s) associated with the most recently entered (top-most)
370 critical section are guaranteed to be held at any given time. Locks for
371 outer, nested critical sections might have been suspended.
372
373 * You can lock at most two objects simultaneously with these APIs. If you need
374 to lock more objects, you'll need to restructure your code.
375
376 * While critical sections will not deadlock if you attempt to lock the same
377 object twice, they are less efficient than purpose-built reentrant locks for
378 this use case.
379
380 * When using :c:macro:`Py_BEGIN_CRITICAL_SECTION2`, the order of the objects
381 doesn't affect correctness (the implementation handles deadlock avoidance),
382 but it's good practice to always lock objects in a consistent order.
383
384 * Remember that the critical section macros are primarily for protecting access
385 to *Python objects* that might be involved in internal CPython operations
386 susceptible to the deadlock scenarios described above. For protecting purely
387 internal extension state, standard mutexes or other synchronization
388 primitives might be more appropriate.
389
390
391 Building Extensions for the Free-Threaded Build
392 ===============================================
393
394 C API extensions need to be built specifically for the free-threaded build.
395 The wheels, shared libraries, and binaries are indicated by a ``t`` suffix.
396
397 * `pypa/manylinux <https://github.com/pypa/manylinux>`_ supports the
398 free-threaded build, with the ``t`` suffix, such as ``python3.14t``.
399 * `pypa/cibuildwheel <https://github.com/pypa/cibuildwheel>`_ supports
400 building wheels for the free-threaded build of Python 3.14 and newer.
401
402 Limited C API and Stable ABI
403 ............................
404
405 The free-threaded build does not currently support the
406 :ref:`Limited C API <limited-c-api>` or the stable ABI. If you use
407 `setuptools <https://setuptools.pypa.io/en/latest/setuptools.html>`_ to build
408 your extension and currently set ``py_limited_api=True`` you can use
409 ``py_limited_api=not sysconfig.get_config_var("Py_GIL_DISABLED")`` to opt out
410 of the limited API when building with the free-threaded build.
411
412 .. note::
413 You will need to build separate wheels specifically for the free-threaded
414 build. If you currently use the stable ABI, you can continue to build a
415 single wheel for multiple non-free-threaded Python versions.
416
417
418 Windows
419 .......
420
421 Due to a limitation of the official Windows installer, you will need to
422 manually define ``Py_GIL_DISABLED=1`` when building extensions from source.
423
424 .. seealso::
425
426 `Porting Extension Modules to Support Free-Threading
427 <https://py-free-threading.github.io/porting/>`_:
428 A community-maintained porting guide for extension authors.