| 1 |
.. highlight:: c |
| 2 |
|
| 3 |
.. _synchronization: |
| 4 |
|
| 5 |
Synchronization primitives |
| 6 |
========================== |
| 7 |
|
| 8 |
The C-API provides a basic mutual exclusion lock. |
| 9 |
|
| 10 |
.. c:type:: PyMutex |
| 11 |
|
| 12 |
A mutual exclusion lock. The :c:type:`!PyMutex` should be initialized to |
| 13 |
zero to represent the unlocked state. For example:: |
| 14 |
|
| 15 |
PyMutex mutex = {0}; |
| 16 |
|
| 17 |
Instances of :c:type:`!PyMutex` should not be copied or moved. Both the |
| 18 |
contents and address of a :c:type:`!PyMutex` are meaningful, and it must |
| 19 |
remain at a fixed, writable location in memory. |
| 20 |
|
| 21 |
.. note:: |
| 22 |
|
| 23 |
A :c:type:`!PyMutex` currently occupies one byte, but the size should be |
| 24 |
considered unstable. The size may change in future Python releases |
| 25 |
without a deprecation period. |
| 26 |
|
| 27 |
.. versionadded:: 3.13 |
| 28 |
|
| 29 |
.. c:function:: void PyMutex_Lock(PyMutex *m) |
| 30 |
|
| 31 |
Lock mutex *m*. If another thread has already locked it, the calling |
| 32 |
thread will block until the mutex is unlocked. While blocked, the thread |
| 33 |
will temporarily detach the :term:`thread state <attached thread state>` if one exists. |
| 34 |
|
| 35 |
.. versionadded:: 3.13 |
| 36 |
|
| 37 |
.. c:function:: void PyMutex_Unlock(PyMutex *m) |
| 38 |
|
| 39 |
Unlock mutex *m*. The mutex must be locked --- otherwise, the function will |
| 40 |
issue a fatal error. |
| 41 |
|
| 42 |
.. versionadded:: 3.13 |
| 43 |
|
| 44 |
.. c:function:: int PyMutex_IsLocked(PyMutex *m) |
| 45 |
|
| 46 |
Returns non-zero if the mutex *m* is currently locked, zero otherwise. |
| 47 |
|
| 48 |
.. note:: |
| 49 |
|
| 50 |
This function is intended for use in assertions and debugging only and |
| 51 |
should not be used to make concurrency control decisions, as the lock |
| 52 |
state may change immediately after the check. |
| 53 |
|
| 54 |
.. versionadded:: 3.14 |
| 55 |
|
| 56 |
.. _python-critical-section-api: |
| 57 |
|
| 58 |
Python critical section API |
| 59 |
--------------------------- |
| 60 |
|
| 61 |
The critical section API provides a deadlock avoidance layer on top of |
| 62 |
per-object locks for :term:`free-threaded <free threading>` CPython. They are |
| 63 |
intended to replace reliance on the :term:`global interpreter lock`, and are |
| 64 |
no-ops in versions of Python with the global interpreter lock. |
| 65 |
|
| 66 |
Critical sections are intended to be used for custom types implemented |
| 67 |
in C-API extensions. They should generally not be used with built-in types like |
| 68 |
:class:`list` and :class:`dict` because their public C-APIs |
| 69 |
already use critical sections internally, with the notable |
| 70 |
exception of :c:func:`PyDict_Next`, which requires critical section |
| 71 |
to be acquired externally. |
| 72 |
|
| 73 |
Critical sections avoid deadlocks by implicitly suspending active critical |
| 74 |
sections, hence, they do not provide exclusive access such as provided by |
| 75 |
traditional locks like :c:type:`PyMutex`. When a critical section is started, |
| 76 |
the per-object lock for the object is acquired. If the code executed inside the |
| 77 |
critical section calls C-API functions then it can suspend the critical section thereby |
| 78 |
releasing the per-object lock, so other threads can acquire the per-object lock |
| 79 |
for the same object. |
| 80 |
|
| 81 |
Variants that accept :c:type:`PyMutex` pointers rather than Python objects are also |
| 82 |
available. Use these variants to start a critical section in a situation where |
| 83 |
there is no :c:type:`PyObject` -- for example, when working with a C type that |
| 84 |
does not extend or wrap :c:type:`PyObject` but still needs to call into the C |
| 85 |
API in a manner that might lead to deadlocks. |
| 86 |
|
| 87 |
The functions and structs used by the macros are exposed for cases |
| 88 |
where C macros are not available. They should only be used as in the |
| 89 |
given macro expansions. Note that the sizes and contents of the structures may |
| 90 |
change in future Python versions. |
| 91 |
|
| 92 |
.. note:: |
| 93 |
|
| 94 |
Operations that need to lock two objects at once must use |
| 95 |
:c:macro:`Py_BEGIN_CRITICAL_SECTION2`. You *cannot* use nested critical |
| 96 |
sections to lock more than one object at once, because the inner critical |
| 97 |
section may suspend the outer critical sections. This API does not provide |
| 98 |
a way to lock more than two objects at once. |
| 99 |
|
| 100 |
Example usage:: |
| 101 |
|
| 102 |
static PyObject * |
| 103 |
set_field(MyObject *self, PyObject *value) |
| 104 |
{ |
| 105 |
Py_BEGIN_CRITICAL_SECTION(self); |
| 106 |
Py_SETREF(self->field, Py_XNewRef(value)); |
| 107 |
Py_END_CRITICAL_SECTION(); |
| 108 |
Py_RETURN_NONE; |
| 109 |
} |
| 110 |
|
| 111 |
In the above example, :c:macro:`Py_SETREF` calls :c:macro:`Py_DECREF`, which |
| 112 |
can call arbitrary code through an object's deallocation function. The critical |
| 113 |
section API avoids potential deadlocks due to reentrancy and lock ordering |
| 114 |
by allowing the runtime to temporarily suspend the critical section if the |
| 115 |
code triggered by the finalizer blocks and calls :c:func:`PyEval_SaveThread`. |
| 116 |
|
| 117 |
.. c:macro:: Py_BEGIN_CRITICAL_SECTION(op) |
| 118 |
|
| 119 |
Acquires the per-object lock for the object *op* and begins a |
| 120 |
critical section. |
| 121 |
|
| 122 |
In the free-threaded build, this macro expands to:: |
| 123 |
|
| 124 |
{ |
| 125 |
PyCriticalSection _py_cs; |
| 126 |
PyCriticalSection_Begin(&_py_cs, (PyObject*)(op)) |
| 127 |
|
| 128 |
In the default build, this macro expands to ``{``. |
| 129 |
|
| 130 |
.. versionadded:: 3.13 |
| 131 |
|
| 132 |
.. c:macro:: Py_BEGIN_CRITICAL_SECTION_MUTEX(m) |
| 133 |
|
| 134 |
Locks the mutex *m* and begins a critical section. |
| 135 |
|
| 136 |
In the free-threaded build, this macro expands to:: |
| 137 |
|
| 138 |
{ |
| 139 |
PyCriticalSection _py_cs; |
| 140 |
PyCriticalSection_BeginMutex(&_py_cs, m) |
| 141 |
|
| 142 |
Note that unlike :c:macro:`Py_BEGIN_CRITICAL_SECTION`, there is no cast for |
| 143 |
the argument of the macro - it must be a :c:type:`PyMutex` pointer. |
| 144 |
|
| 145 |
On the default build, this macro expands to ``{``. |
| 146 |
|
| 147 |
.. versionadded:: 3.14 |
| 148 |
|
| 149 |
.. c:macro:: Py_END_CRITICAL_SECTION() |
| 150 |
|
| 151 |
Ends the critical section and releases the per-object lock. |
| 152 |
|
| 153 |
In the free-threaded build, this macro expands to:: |
| 154 |
|
| 155 |
PyCriticalSection_End(&_py_cs); |
| 156 |
} |
| 157 |
|
| 158 |
In the default build, this macro expands to ``}``. |
| 159 |
|
| 160 |
.. versionadded:: 3.13 |
| 161 |
|
| 162 |
.. c:macro:: Py_BEGIN_CRITICAL_SECTION2(a, b) |
| 163 |
|
| 164 |
Acquires the per-object locks for the objects *a* and *b* and begins a |
| 165 |
critical section. The locks are acquired in a consistent order (lowest |
| 166 |
address first) to avoid lock ordering deadlocks. |
| 167 |
|
| 168 |
In the free-threaded build, this macro expands to:: |
| 169 |
|
| 170 |
{ |
| 171 |
PyCriticalSection2 _py_cs2; |
| 172 |
PyCriticalSection2_Begin(&_py_cs2, (PyObject*)(a), (PyObject*)(b)) |
| 173 |
|
| 174 |
In the default build, this macro expands to ``{``. |
| 175 |
|
| 176 |
.. versionadded:: 3.13 |
| 177 |
|
| 178 |
.. c:macro:: Py_BEGIN_CRITICAL_SECTION2_MUTEX(m1, m2) |
| 179 |
|
| 180 |
Locks the mutexes *m1* and *m2* and begins a critical section. |
| 181 |
|
| 182 |
In the free-threaded build, this macro expands to:: |
| 183 |
|
| 184 |
{ |
| 185 |
PyCriticalSection2 _py_cs2; |
| 186 |
PyCriticalSection2_BeginMutex(&_py_cs2, m1, m2) |
| 187 |
|
| 188 |
Note that unlike :c:macro:`Py_BEGIN_CRITICAL_SECTION2`, there is no cast for |
| 189 |
the arguments of the macro - they must be :c:type:`PyMutex` pointers. |
| 190 |
|
| 191 |
On the default build, this macro expands to ``{``. |
| 192 |
|
| 193 |
.. versionadded:: 3.14 |
| 194 |
|
| 195 |
.. c:macro:: Py_END_CRITICAL_SECTION2() |
| 196 |
|
| 197 |
Ends the critical section and releases the per-object locks. |
| 198 |
|
| 199 |
In the free-threaded build, this macro expands to:: |
| 200 |
|
| 201 |
PyCriticalSection2_End(&_py_cs2); |
| 202 |
} |
| 203 |
|
| 204 |
In the default build, this macro expands to ``}``. |
| 205 |
|
| 206 |
.. versionadded:: 3.13 |
| 207 |
|
| 208 |
|
| 209 |
Legacy locking APIs |
| 210 |
------------------- |
| 211 |
|
| 212 |
These APIs are obsolete since Python 3.13 with the introduction of |
| 213 |
:c:type:`PyMutex`. |
| 214 |
|
| 215 |
|
| 216 |
.. c:type:: PyThread_type_lock |
| 217 |
|
| 218 |
A pointer to a mutual exclusion lock. |
| 219 |
|
| 220 |
|
| 221 |
.. c:type:: PyLockStatus |
| 222 |
|
| 223 |
The result of acquiring a lock with a timeout. |
| 224 |
|
| 225 |
.. c:namespace:: NULL |
| 226 |
|
| 227 |
.. c:enumerator:: PY_LOCK_FAILURE |
| 228 |
|
| 229 |
Failed to acquire the lock. |
| 230 |
|
| 231 |
.. c:enumerator:: PY_LOCK_ACQUIRED |
| 232 |
|
| 233 |
The lock was successfully acquired. |
| 234 |
|
| 235 |
.. c:enumerator:: PY_LOCK_INTR |
| 236 |
|
| 237 |
The lock was interrupted by a signal. |
| 238 |
|
| 239 |
|
| 240 |
.. c:function:: PyThread_type_lock PyThread_allocate_lock(void) |
| 241 |
|
| 242 |
Allocate a new lock. |
| 243 |
|
| 244 |
On success, this function returns a lock; on failure, this |
| 245 |
function returns ``0`` without an exception set. |
| 246 |
|
| 247 |
The caller does not need to hold an :term:`attached thread state`. |
| 248 |
|
| 249 |
|
| 250 |
.. c:function:: void PyThread_free_lock(PyThread_type_lock lock) |
| 251 |
|
| 252 |
Destroy *lock*. The lock should not be held by any thread when calling |
| 253 |
this. |
| 254 |
|
| 255 |
The caller does not need to hold an :term:`attached thread state`. |
| 256 |
|
| 257 |
|
| 258 |
.. c:function:: PyLockStatus PyThread_acquire_lock_timed(PyThread_type_lock lock, long long microseconds, int intr_flag) |
| 259 |
|
| 260 |
Acquire *lock* with a timeout. |
| 261 |
|
| 262 |
This will wait for *microseconds* microseconds to acquire the lock. If the |
| 263 |
timeout expires, this function returns :c:enumerator:`PY_LOCK_FAILURE`. |
| 264 |
If *microseconds* is ``-1``, this will wait indefinitely until the lock has |
| 265 |
been released. |
| 266 |
|
| 267 |
If *intr_flag* is ``1``, acquiring the lock may be interrupted by a signal, |
| 268 |
in which case this function returns :c:enumerator:`PY_LOCK_INTR`. Upon |
| 269 |
interruption, it's generally expected that the caller makes a call to |
| 270 |
:c:func:`Py_MakePendingCalls` to propagate an exception to Python code. |
| 271 |
|
| 272 |
If the lock is successfully acquired, this function returns |
| 273 |
:c:enumerator:`PY_LOCK_ACQUIRED`. |
| 274 |
|
| 275 |
The caller does not need to hold an :term:`attached thread state`. |
| 276 |
|
| 277 |
|
| 278 |
.. c:function:: int PyThread_acquire_lock(PyThread_type_lock lock, int waitflag) |
| 279 |
|
| 280 |
Acquire *lock*. |
| 281 |
|
| 282 |
If *waitflag* is ``1`` and another thread currently holds the lock, this |
| 283 |
function will wait until the lock can be acquired and will always return |
| 284 |
``1``. |
| 285 |
|
| 286 |
If *waitflag* is ``0`` and another thread holds the lock, this function will |
| 287 |
not wait and instead return ``0``. If the lock is not held by any other |
| 288 |
thread, then this function will acquire it and return ``1``. |
| 289 |
|
| 290 |
Unlike :c:func:`PyThread_acquire_lock_timed`, acquiring the lock cannot be |
| 291 |
interrupted by a signal. |
| 292 |
|
| 293 |
The caller does not need to hold an :term:`attached thread state`. |
| 294 |
|
| 295 |
|
| 296 |
.. c:function:: int PyThread_release_lock(PyThread_type_lock lock) |
| 297 |
|
| 298 |
Release *lock*. If *lock* is not held, then this function issues a |
| 299 |
fatal error. |
| 300 |
|
| 301 |
The caller does not need to hold an :term:`attached thread state`. |