| 1 |
.. highlight:: c |
| 2 |
|
| 3 |
.. _stable: |
| 4 |
|
| 5 |
*************** |
| 6 |
C API Stability |
| 7 |
*************** |
| 8 |
|
| 9 |
Unless documented otherwise, Python's C API is covered by the Backwards |
| 10 |
Compatibility Policy, :pep:`387`. |
| 11 |
Most changes to it are source-compatible (typically by only adding new API). |
| 12 |
Changing existing API or removing API is only done after a deprecation period |
| 13 |
or to fix serious issues. |
| 14 |
|
| 15 |
CPython's Application Binary Interface (ABI) is forward- and |
| 16 |
backwards-compatible across a minor release (if these are compiled the same |
| 17 |
way; see :ref:`stable-abi-platform` below). |
| 18 |
So, code compiled for Python 3.10.0 will work on 3.10.8 and vice versa, |
| 19 |
but will need to be compiled separately for 3.9.x and 3.11.x. |
| 20 |
|
| 21 |
There are two tiers of C API with different stability expectations: |
| 22 |
|
| 23 |
- :ref:`Unstable API <unstable-c-api>`, may change in minor versions without |
| 24 |
a deprecation period. It is marked by the ``PyUnstable`` prefix in names. |
| 25 |
- :ref:`Limited API <limited-c-api>`, is compatible across several minor releases. |
| 26 |
When :c:macro:`Py_LIMITED_API` is defined, only this subset is exposed |
| 27 |
from ``Python.h``. |
| 28 |
|
| 29 |
These are discussed in more detail below. |
| 30 |
|
| 31 |
Names prefixed by an underscore, such as ``_Py_InternalState``, |
| 32 |
are private API that can change without notice even in patch releases. |
| 33 |
If you need to use this API, consider reaching out to |
| 34 |
`CPython developers <https://discuss.python.org/c/core-dev/c-api/30>`_ |
| 35 |
to discuss adding public API for your use case. |
| 36 |
|
| 37 |
.. _unstable-c-api: |
| 38 |
|
| 39 |
Unstable C API |
| 40 |
============== |
| 41 |
|
| 42 |
.. index:: single: PyUnstable |
| 43 |
|
| 44 |
Any API named with the ``PyUnstable`` prefix exposes CPython implementation |
| 45 |
details, and may change in every minor release (e.g. from 3.9 to 3.10) without |
| 46 |
any deprecation warnings. |
| 47 |
However, it will not change in a bugfix release (e.g. from 3.10.0 to 3.10.1). |
| 48 |
|
| 49 |
It is generally intended for specialized, low-level tools like debuggers. |
| 50 |
|
| 51 |
Projects that use this API are expected to follow |
| 52 |
CPython development and spend extra effort adjusting to changes. |
| 53 |
|
| 54 |
.. _stable-application-binary-interface: |
| 55 |
|
| 56 |
Stable Application Binary Interface |
| 57 |
=================================== |
| 58 |
|
| 59 |
For simplicity, this document talks about *extensions*, but the Limited API |
| 60 |
and Stable ABI work the same way for all uses of the API – for example, |
| 61 |
embedding Python. |
| 62 |
|
| 63 |
.. _limited-c-api: |
| 64 |
|
| 65 |
Limited C API |
| 66 |
------------- |
| 67 |
|
| 68 |
Python 3.2 introduced the *Limited API*, a subset of Python's C API. |
| 69 |
Extensions that only use the Limited API can be |
| 70 |
compiled once and be loaded on multiple versions of Python. |
| 71 |
Contents of the Limited API are :ref:`listed below <limited-api-list>`. |
| 72 |
|
| 73 |
.. c:macro:: Py_LIMITED_API |
| 74 |
|
| 75 |
Define this macro before including ``Python.h`` to opt in to only use |
| 76 |
the Limited API, and to select the Limited API version. |
| 77 |
|
| 78 |
Define ``Py_LIMITED_API`` to the value of :c:macro:`PY_VERSION_HEX` |
| 79 |
corresponding to the lowest Python version your extension supports. |
| 80 |
The extension will be ABI-compatible with all Python 3 releases |
| 81 |
from the specified one onward, and can use Limited API introduced up to that |
| 82 |
version. |
| 83 |
|
| 84 |
Rather than using the ``PY_VERSION_HEX`` macro directly, hardcode a minimum |
| 85 |
minor version (e.g. ``0x030A0000`` for Python 3.10) for stability when |
| 86 |
compiling with future Python versions. |
| 87 |
|
| 88 |
You can also define ``Py_LIMITED_API`` to ``3``. This works the same as |
| 89 |
``0x03020000`` (Python 3.2, the version that introduced Limited API). |
| 90 |
|
| 91 |
|
| 92 |
.. _stable-abi: |
| 93 |
|
| 94 |
Stable ABI |
| 95 |
---------- |
| 96 |
|
| 97 |
To enable this, Python provides a *Stable ABI*: a set of symbols that will |
| 98 |
remain ABI-compatible across Python 3.x versions. |
| 99 |
|
| 100 |
.. note:: |
| 101 |
|
| 102 |
The Stable ABI prevents ABI issues, like linker errors due to missing |
| 103 |
symbols or data corruption due to changes in structure layouts or function |
| 104 |
signatures. |
| 105 |
However, other changes in Python can change the *behavior* of extensions. |
| 106 |
See Python's Backwards Compatibility Policy (:pep:`387`) for details. |
| 107 |
|
| 108 |
The Stable ABI contains symbols exposed in the :ref:`Limited API |
| 109 |
<limited-c-api>`, but also other ones – for example, functions necessary to |
| 110 |
support older versions of the Limited API. |
| 111 |
|
| 112 |
On Windows, extensions that use the Stable ABI should be linked against |
| 113 |
``python3.dll`` rather than a version-specific library such as |
| 114 |
``python39.dll``. |
| 115 |
|
| 116 |
On some platforms, Python will look for and load shared library files named |
| 117 |
with the ``abi3`` tag (e.g. ``mymodule.abi3.so``). |
| 118 |
It does not check if such extensions conform to a Stable ABI. |
| 119 |
The user (or their packaging tools) need to ensure that, for example, |
| 120 |
extensions built with the 3.10+ Limited API are not installed for lower |
| 121 |
versions of Python. |
| 122 |
|
| 123 |
All functions in the Stable ABI are present as functions in Python's shared |
| 124 |
library, not solely as macros. This makes them usable from languages that don't |
| 125 |
use the C preprocessor. |
| 126 |
|
| 127 |
|
| 128 |
Limited API Scope and Performance |
| 129 |
--------------------------------- |
| 130 |
|
| 131 |
The goal for the Limited API is to allow everything that is possible with the |
| 132 |
full C API, but possibly with a performance penalty. |
| 133 |
|
| 134 |
For example, while :c:func:`PyList_GetItem` is available, its “unsafe” macro |
| 135 |
variant :c:func:`PyList_GET_ITEM` is not. |
| 136 |
The macro can be faster because it can rely on version-specific implementation |
| 137 |
details of the list object. |
| 138 |
|
| 139 |
Without ``Py_LIMITED_API`` defined, some C API functions are inlined or |
| 140 |
replaced by macros. |
| 141 |
Defining ``Py_LIMITED_API`` disables this inlining, allowing stability as |
| 142 |
Python's data structures are improved, but possibly reducing performance. |
| 143 |
|
| 144 |
By leaving out the ``Py_LIMITED_API`` definition, it is possible to compile |
| 145 |
a Limited API extension with a version-specific ABI. This can improve |
| 146 |
performance for that Python version, but will limit compatibility. |
| 147 |
Compiling with ``Py_LIMITED_API`` will then yield an extension that can be |
| 148 |
distributed where a version-specific one is not available – for example, |
| 149 |
for prereleases of an upcoming Python version. |
| 150 |
|
| 151 |
|
| 152 |
Limited API Caveats |
| 153 |
------------------- |
| 154 |
|
| 155 |
Note that compiling with ``Py_LIMITED_API`` is *not* a complete guarantee that |
| 156 |
code conforms to the :ref:`Limited API <limited-c-api>` or the :ref:`Stable ABI |
| 157 |
<stable-abi>`. ``Py_LIMITED_API`` only covers definitions, but an API also |
| 158 |
includes other issues, such as expected semantics. |
| 159 |
|
| 160 |
One issue that ``Py_LIMITED_API`` does not guard against is calling a function |
| 161 |
with arguments that are invalid in a lower Python version. |
| 162 |
For example, consider a function that starts accepting ``NULL`` for an |
| 163 |
argument. In Python 3.9, ``NULL`` now selects a default behavior, but in |
| 164 |
Python 3.8, the argument will be used directly, causing a ``NULL`` dereference |
| 165 |
and crash. A similar argument works for fields of structs. |
| 166 |
|
| 167 |
Another issue is that some struct fields are currently not hidden when |
| 168 |
``Py_LIMITED_API`` is defined, even though they're part of the Limited API. |
| 169 |
|
| 170 |
For these reasons, we recommend testing an extension with *all* minor Python |
| 171 |
versions it supports, and preferably to build with the *lowest* such version. |
| 172 |
|
| 173 |
We also recommend reviewing documentation of all used API to check |
| 174 |
if it is explicitly part of the Limited API. Even with ``Py_LIMITED_API`` |
| 175 |
defined, a few private declarations are exposed for technical reasons (or |
| 176 |
even unintentionally, as bugs). |
| 177 |
|
| 178 |
Also note that the Limited API is not necessarily stable: compiling with |
| 179 |
``Py_LIMITED_API`` with Python 3.8 means that the extension will |
| 180 |
run with Python 3.12, but it will not necessarily *compile* with Python 3.12. |
| 181 |
In particular, parts of the Limited API may be deprecated and removed, |
| 182 |
provided that the Stable ABI stays stable. |
| 183 |
|
| 184 |
|
| 185 |
.. _stable-abi-platform: |
| 186 |
|
| 187 |
Platform Considerations |
| 188 |
======================= |
| 189 |
|
| 190 |
ABI stability depends not only on Python, but also on the compiler used, |
| 191 |
lower-level libraries and compiler options. For the purposes of |
| 192 |
the :ref:`Stable ABI <stable-abi>`, these details define a “platform”. They |
| 193 |
usually depend on the OS type and processor architecture |
| 194 |
|
| 195 |
It is the responsibility of each particular distributor of Python |
| 196 |
to ensure that all Python versions on a particular platform are built |
| 197 |
in a way that does not break the Stable ABI. |
| 198 |
This is the case with Windows and macOS releases from ``python.org`` and many |
| 199 |
third-party distributors. |
| 200 |
|
| 201 |
|
| 202 |
.. _limited-api-list: |
| 203 |
|
| 204 |
Contents of Limited API |
| 205 |
======================= |
| 206 |
|
| 207 |
|
| 208 |
Currently, the :ref:`Limited API <limited-c-api>` includes the following items: |
| 209 |
|
| 210 |
.. limited-api-list:: |