| 1 |
.. _using-ios: |
| 2 |
|
| 3 |
=================== |
| 4 |
Using Python on iOS |
| 5 |
=================== |
| 6 |
|
| 7 |
:Authors: |
| 8 |
Russell Keith-Magee (2024-03) |
| 9 |
|
| 10 |
Python on iOS is unlike Python on desktop platforms. On a desktop platform, |
| 11 |
Python is generally installed as a system resource that can be used by any user |
| 12 |
of that computer. Users then interact with Python by running a :program:`python` |
| 13 |
executable and entering commands at an interactive prompt, or by running a |
| 14 |
Python script. |
| 15 |
|
| 16 |
On iOS, there is no concept of installing as a system resource. The only unit |
| 17 |
of software distribution is an "app". There is also no console where you could |
| 18 |
run a :program:`python` executable, or interact with a Python REPL. |
| 19 |
|
| 20 |
As a result, the only way you can use Python on iOS is in embedded mode - that |
| 21 |
is, by writing a native iOS application, and embedding a Python interpreter |
| 22 |
using ``libPython``, and invoking Python code using the :ref:`Python embedding |
| 23 |
API <embedding>`. The full Python interpreter, the standard library, and all |
| 24 |
your Python code is then packaged as a standalone bundle that can be |
| 25 |
distributed via the iOS App Store. |
| 26 |
|
| 27 |
If you're looking to experiment for the first time with writing an iOS app in |
| 28 |
Python, projects such as `BeeWare <https://beeware.org>`__ and `Kivy |
| 29 |
<https://kivy.org>`__ will provide a much more approachable user experience. |
| 30 |
These projects manage the complexities associated with getting an iOS project |
| 31 |
running, so you only need to deal with the Python code itself. |
| 32 |
|
| 33 |
Python at runtime on iOS |
| 34 |
======================== |
| 35 |
|
| 36 |
iOS version compatibility |
| 37 |
------------------------- |
| 38 |
|
| 39 |
The minimum supported iOS version is specified at compile time, using the |
| 40 |
:option:`--host` option to ``configure``. By default, when compiled for iOS, |
| 41 |
Python will be compiled with a minimum supported iOS version of 13.0. To use a |
| 42 |
different minimum iOS version, provide the version number as part of the |
| 43 |
:option:`!--host` argument - for example, |
| 44 |
``--host=arm64-apple-ios15.4-simulator`` would compile an ARM64 simulator build |
| 45 |
with a deployment target of 15.4. |
| 46 |
|
| 47 |
Platform identification |
| 48 |
----------------------- |
| 49 |
|
| 50 |
When executing on iOS, ``sys.platform`` will report as ``ios``. This value will |
| 51 |
be returned on an iPhone or iPad, regardless of whether the app is running on |
| 52 |
the simulator or a physical device. |
| 53 |
|
| 54 |
Information about the specific runtime environment, including the iOS version, |
| 55 |
device model, and whether the device is a simulator, can be obtained using |
| 56 |
:func:`platform.ios_ver`. :func:`platform.system` will report ``iOS`` or |
| 57 |
``iPadOS``, depending on the device. |
| 58 |
|
| 59 |
:func:`os.uname` reports kernel-level details; it will report a name of |
| 60 |
``Darwin``. |
| 61 |
|
| 62 |
Standard library availability |
| 63 |
----------------------------- |
| 64 |
|
| 65 |
The Python standard library has some notable omissions and restrictions on |
| 66 |
iOS. See the :ref:`API availability guide for iOS <mobile-availability>` for |
| 67 |
details. |
| 68 |
|
| 69 |
Binary extension modules |
| 70 |
------------------------ |
| 71 |
|
| 72 |
One notable difference about iOS as a platform is that App Store distribution |
| 73 |
imposes hard requirements on the packaging of an application. One of these |
| 74 |
requirements governs how binary extension modules are distributed. |
| 75 |
|
| 76 |
The iOS App Store requires that *all* binary modules in an iOS app must be |
| 77 |
dynamic libraries, contained in a framework with appropriate metadata, stored |
| 78 |
in the ``Frameworks`` folder of the packaged app. There can be only a single |
| 79 |
binary per framework, and there can be no executable binary material outside |
| 80 |
the ``Frameworks`` folder. |
| 81 |
|
| 82 |
This conflicts with the usual Python approach for distributing binaries, which |
| 83 |
allows a binary extension module to be loaded from any location on |
| 84 |
``sys.path``. To ensure compliance with App Store policies, an iOS project must |
| 85 |
post-process any Python packages, converting ``.so`` binary modules into |
| 86 |
individual standalone frameworks with appropriate metadata and signing. For |
| 87 |
details on how to perform this post-processing, see the guide for :ref:`adding |
| 88 |
Python to your project <adding-ios>`. |
| 89 |
|
| 90 |
To help Python discover binaries in their new location, the original ``.so`` |
| 91 |
file on ``sys.path`` is replaced with a ``.fwork`` file. This file is a text |
| 92 |
file containing the location of the framework binary, relative to the app |
| 93 |
bundle. To allow the framework to resolve back to the original location, the |
| 94 |
framework must contain a ``.origin`` file that contains the location of the |
| 95 |
``.fwork`` file, relative to the app bundle. |
| 96 |
|
| 97 |
For example, consider the case of an import ``from foo.bar import _whiz``, |
| 98 |
where ``_whiz`` is implemented with the binary module |
| 99 |
``sources/foo/bar/_whiz.abi3.so``, with ``sources`` being the location |
| 100 |
registered on ``sys.path``, relative to the application bundle. This module |
| 101 |
*must* be distributed as ``Frameworks/foo.bar._whiz.framework/foo.bar._whiz`` |
| 102 |
(creating the framework name from the full import path of the module), with an |
| 103 |
``Info.plist`` file in the ``.framework`` directory identifying the binary as a |
| 104 |
framework. The ``foo.bar._whiz`` module would be represented in the original |
| 105 |
location with a ``sources/foo/bar/_whiz.abi3.fwork`` marker file, containing |
| 106 |
the path ``Frameworks/foo.bar._whiz/foo.bar._whiz``. The framework would also |
| 107 |
contain ``Frameworks/foo.bar._whiz.framework/foo.bar._whiz.origin``, containing |
| 108 |
the path to the ``.fwork`` file. |
| 109 |
|
| 110 |
When running on iOS, the Python interpreter will install an |
| 111 |
:class:`~importlib.machinery.AppleFrameworkLoader` that is able to read and |
| 112 |
import ``.fwork`` files. Once imported, the ``__file__`` attribute of the |
| 113 |
binary module will report as the location of the ``.fwork`` file. However, the |
| 114 |
:class:`~importlib.machinery.ModuleSpec` for the loaded module will report the |
| 115 |
``origin`` as the location of the binary in the framework folder. |
| 116 |
|
| 117 |
Compiler stub binaries |
| 118 |
---------------------- |
| 119 |
|
| 120 |
Xcode doesn't expose explicit compilers for iOS; instead, it uses an ``xcrun`` |
| 121 |
script that resolves to a full compiler path (e.g., ``xcrun --sdk iphoneos |
| 122 |
clang`` to get the ``clang`` for an iPhone device). However, using this script |
| 123 |
poses two problems: |
| 124 |
|
| 125 |
* The output of ``xcrun`` includes paths that are machine specific, resulting |
| 126 |
in a sysconfig module that cannot be shared between users; and |
| 127 |
|
| 128 |
* It results in ``CC``/``CPP``/``LD``/``AR`` definitions that include spaces. |
| 129 |
There is a lot of C ecosystem tooling that assumes that you can split a |
| 130 |
command line at the first space to get the path to the compiler executable; |
| 131 |
this isn't the case when using ``xcrun``. |
| 132 |
|
| 133 |
To avoid these problems, Python provided stubs for these tools. These stubs are |
| 134 |
shell script wrappers around the underingly ``xcrun`` tools, distributed in a |
| 135 |
``bin`` folder distributed alongside the compiled iOS framework. These scripts |
| 136 |
are relocatable, and will always resolve to the appropriate local system paths. |
| 137 |
By including these scripts in the bin folder that accompanies a framework, the |
| 138 |
contents of the ``sysconfig`` module becomes useful for end-users to compile |
| 139 |
their own modules. When compiling third-party Python modules for iOS, you |
| 140 |
should ensure these stub binaries are on your path. |
| 141 |
|
| 142 |
Installing Python on iOS |
| 143 |
======================== |
| 144 |
|
| 145 |
Tools for building iOS apps |
| 146 |
--------------------------- |
| 147 |
|
| 148 |
Building for iOS requires the use of Apple's Xcode tooling. It is strongly |
| 149 |
recommended that you use the most recent stable release of Xcode. This will |
| 150 |
require the use of the most (or second-most) recently released macOS version, |
| 151 |
as Apple does not maintain Xcode for older macOS versions. The Xcode Command |
| 152 |
Line Tools are not sufficient for iOS development; you need a *full* Xcode |
| 153 |
install. |
| 154 |
|
| 155 |
If you want to run your code on the iOS simulator, you'll also need to install |
| 156 |
an iOS Simulator Platform. You should be prompted to select an iOS Simulator |
| 157 |
Platform when you first run Xcode. Alternatively, you can add an iOS Simulator |
| 158 |
Platform by selecting from the Platforms tab of the Xcode Settings panel. |
| 159 |
|
| 160 |
.. _adding-ios: |
| 161 |
|
| 162 |
Adding Python to an iOS project |
| 163 |
------------------------------- |
| 164 |
|
| 165 |
Python can be added to any iOS project, using either Swift or Objective C. The |
| 166 |
following examples will use Objective C; if you are using Swift, you may find a |
| 167 |
library like `PythonKit <https://github.com/pvieito/PythonKit>`__ to be |
| 168 |
helpful. |
| 169 |
|
| 170 |
To add Python to an iOS Xcode project: |
| 171 |
|
| 172 |
1. Build or obtain a Python ``XCFramework``. See the instructions in |
| 173 |
:source:`iOS/README.rst` (in the CPython source distribution) for details on |
| 174 |
how to build a Python ``XCFramework``. At a minimum, you will need a build |
| 175 |
that supports ``arm64-apple-ios``, plus one of either |
| 176 |
``arm64-apple-ios-simulator`` or ``x86_64-apple-ios-simulator``. |
| 177 |
|
| 178 |
2. Drag the ``XCframework`` into your iOS project. In the following |
| 179 |
instructions, we'll assume you've dropped the ``XCframework`` into the root |
| 180 |
of your project; however, you can use any other location that you want by |
| 181 |
adjusting paths as needed. |
| 182 |
|
| 183 |
3. Drag the ``iOS/Resources/dylib-Info-template.plist`` file into your project, |
| 184 |
and ensure it is associated with the app target. |
| 185 |
|
| 186 |
4. Add your application code as a folder in your Xcode project. In the |
| 187 |
following instructions, we'll assume that your user code is in a folder |
| 188 |
named ``app`` in the root of your project; you can use any other location by |
| 189 |
adjusting paths as needed. Ensure that this folder is associated with your |
| 190 |
app target. |
| 191 |
|
| 192 |
5. Select the app target by selecting the root node of your Xcode project, then |
| 193 |
the target name in the sidebar that appears. |
| 194 |
|
| 195 |
6. In the "General" settings, under "Frameworks, Libraries and Embedded |
| 196 |
Content", add ``Python.xcframework``, with "Embed & Sign" selected. |
| 197 |
|
| 198 |
7. In the "Build Settings" tab, modify the following: |
| 199 |
|
| 200 |
- Build Options |
| 201 |
|
| 202 |
* User Script Sandboxing: No |
| 203 |
* Enable Testability: Yes |
| 204 |
|
| 205 |
- Search Paths |
| 206 |
|
| 207 |
* Framework Search Paths: ``$(PROJECT_DIR)`` |
| 208 |
* Header Search Paths: ``"$(BUILT_PRODUCTS_DIR)/Python.framework/Headers"`` |
| 209 |
|
| 210 |
- Apple Clang - Warnings - All languages |
| 211 |
|
| 212 |
* Quoted Include In Framework Header: No |
| 213 |
|
| 214 |
8. Add a build step that copies the Python standard library into your app. In |
| 215 |
the "Build Phases" tab, add a new "Run Script" build step *before* the |
| 216 |
"Embed Frameworks" step, but *after* the "Copy Bundle Resources" step. Name |
| 217 |
the step "Install Target Specific Python Standard Library", disable the |
| 218 |
"Based on dependency analysis" checkbox, and set the script content to: |
| 219 |
|
| 220 |
.. code-block:: bash |
| 221 |
|
| 222 |
set -e |
| 223 |
|
| 224 |
mkdir -p "$CODESIGNING_FOLDER_PATH/python/lib" |
| 225 |
if [ "$EFFECTIVE_PLATFORM_NAME" = "-iphonesimulator" ]; then |
| 226 |
echo "Installing Python modules for iOS Simulator" |
| 227 |
rsync -au --delete "$PROJECT_DIR/Python.xcframework/ios-arm64_x86_64-simulator/lib/" "$CODESIGNING_FOLDER_PATH/python/lib/" |
| 228 |
else |
| 229 |
echo "Installing Python modules for iOS Device" |
| 230 |
rsync -au --delete "$PROJECT_DIR/Python.xcframework/ios-arm64/lib/" "$CODESIGNING_FOLDER_PATH/python/lib/" |
| 231 |
fi |
| 232 |
|
| 233 |
Note that the name of the simulator "slice" in the XCframework may be |
| 234 |
different, depending the CPU architectures your ``XCFramework`` supports. |
| 235 |
|
| 236 |
9. Add a second build step that processes the binary extension modules in the |
| 237 |
standard library into "Framework" format. Add a "Run Script" build step |
| 238 |
*directly after* the one you added in step 8, named "Prepare Python Binary |
| 239 |
Modules". It should also have "Based on dependency analysis" unchecked, with |
| 240 |
the following script content: |
| 241 |
|
| 242 |
.. code-block:: bash |
| 243 |
|
| 244 |
set -e |
| 245 |
|
| 246 |
install_dylib () { |
| 247 |
INSTALL_BASE=$1 |
| 248 |
FULL_EXT=$2 |
| 249 |
|
| 250 |
# The name of the extension file |
| 251 |
EXT=$(basename "$FULL_EXT") |
| 252 |
# The location of the extension file, relative to the bundle |
| 253 |
RELATIVE_EXT=${FULL_EXT#$CODESIGNING_FOLDER_PATH/} |
| 254 |
# The path to the extension file, relative to the install base |
| 255 |
PYTHON_EXT=${RELATIVE_EXT/$INSTALL_BASE/} |
| 256 |
# The full dotted name of the extension module, constructed from the file path. |
| 257 |
FULL_MODULE_NAME=$(echo $PYTHON_EXT | cut -d "." -f 1 | tr "/" "."); |
| 258 |
# A bundle identifier; not actually used, but required by Xcode framework packaging |
| 259 |
FRAMEWORK_BUNDLE_ID=$(echo $PRODUCT_BUNDLE_IDENTIFIER.$FULL_MODULE_NAME | tr "_" "-") |
| 260 |
# The name of the framework folder. |
| 261 |
FRAMEWORK_FOLDER="Frameworks/$FULL_MODULE_NAME.framework" |
| 262 |
|
| 263 |
# If the framework folder doesn't exist, create it. |
| 264 |
if [ ! -d "$CODESIGNING_FOLDER_PATH/$FRAMEWORK_FOLDER" ]; then |
| 265 |
echo "Creating framework for $RELATIVE_EXT" |
| 266 |
mkdir -p "$CODESIGNING_FOLDER_PATH/$FRAMEWORK_FOLDER" |
| 267 |
cp "$CODESIGNING_FOLDER_PATH/dylib-Info-template.plist" "$CODESIGNING_FOLDER_PATH/$FRAMEWORK_FOLDER/Info.plist" |
| 268 |
plutil -replace CFBundleExecutable -string "$FULL_MODULE_NAME" "$CODESIGNING_FOLDER_PATH/$FRAMEWORK_FOLDER/Info.plist" |
| 269 |
plutil -replace CFBundleIdentifier -string "$FRAMEWORK_BUNDLE_ID" "$CODESIGNING_FOLDER_PATH/$FRAMEWORK_FOLDER/Info.plist" |
| 270 |
fi |
| 271 |
|
| 272 |
echo "Installing binary for $FRAMEWORK_FOLDER/$FULL_MODULE_NAME" |
| 273 |
mv "$FULL_EXT" "$CODESIGNING_FOLDER_PATH/$FRAMEWORK_FOLDER/$FULL_MODULE_NAME" |
| 274 |
# Create a placeholder .fwork file where the .so was |
| 275 |
echo "$FRAMEWORK_FOLDER/$FULL_MODULE_NAME" > ${FULL_EXT%.so}.fwork |
| 276 |
# Create a back reference to the .so file location in the framework |
| 277 |
echo "${RELATIVE_EXT%.so}.fwork" > "$CODESIGNING_FOLDER_PATH/$FRAMEWORK_FOLDER/$FULL_MODULE_NAME.origin" |
| 278 |
} |
| 279 |
|
| 280 |
PYTHON_VER=$(ls -1 "$CODESIGNING_FOLDER_PATH/python/lib") |
| 281 |
echo "Install Python $PYTHON_VER standard library extension modules..." |
| 282 |
find "$CODESIGNING_FOLDER_PATH/python/lib/$PYTHON_VER/lib-dynload" -name "*.so" | while read FULL_EXT; do |
| 283 |
install_dylib python/lib/$PYTHON_VER/lib-dynload/ "$FULL_EXT" |
| 284 |
done |
| 285 |
|
| 286 |
# Clean up dylib template |
| 287 |
rm -f "$CODESIGNING_FOLDER_PATH/dylib-Info-template.plist" |
| 288 |
|
| 289 |
echo "Signing frameworks as $EXPANDED_CODE_SIGN_IDENTITY_NAME ($EXPANDED_CODE_SIGN_IDENTITY)..." |
| 290 |
find "$CODESIGNING_FOLDER_PATH/Frameworks" -name "*.framework" -exec /usr/bin/codesign --force --sign "$EXPANDED_CODE_SIGN_IDENTITY" ${OTHER_CODE_SIGN_FLAGS:-} -o runtime --timestamp=none --preserve-metadata=identifier,entitlements,flags --generate-entitlement-der "{}" \; |
| 291 |
|
| 292 |
10. Add Objective C code to initialize and use a Python interpreter in embedded |
| 293 |
mode. You should ensure that: |
| 294 |
|
| 295 |
* UTF-8 mode (:c:member:`PyPreConfig.utf8_mode`) is *enabled*; |
| 296 |
* Buffered stdio (:c:member:`PyConfig.buffered_stdio`) is *disabled*; |
| 297 |
* Writing bytecode (:c:member:`PyConfig.write_bytecode`) is *disabled*; |
| 298 |
* Signal handlers (:c:member:`PyConfig.install_signal_handlers`) are *enabled*; |
| 299 |
* :envvar:`PYTHONHOME` for the interpreter is configured to point at the |
| 300 |
``python`` subfolder of your app's bundle; and |
| 301 |
* The :envvar:`PYTHONPATH` for the interpreter includes: |
| 302 |
|
| 303 |
- the ``python/lib/python3.X`` subfolder of your app's bundle, |
| 304 |
- the ``python/lib/python3.X/lib-dynload`` subfolder of your app's bundle, and |
| 305 |
- the ``app`` subfolder of your app's bundle |
| 306 |
|
| 307 |
Your app's bundle location can be determined using ``[[NSBundle mainBundle] |
| 308 |
resourcePath]``. |
| 309 |
|
| 310 |
Steps 8, 9 and 10 of these instructions assume that you have a single folder of |
| 311 |
pure Python application code, named ``app``. If you have third-party binary |
| 312 |
modules in your app, some additional steps will be required: |
| 313 |
|
| 314 |
* You need to ensure that any folders containing third-party binaries are |
| 315 |
either associated with the app target, or copied in as part of step 8. Step 8 |
| 316 |
should also purge any binaries that are not appropriate for the platform a |
| 317 |
specific build is targeting (i.e., delete any device binaries if you're |
| 318 |
building an app targeting the simulator). |
| 319 |
|
| 320 |
* Any folders that contain third-party binaries must be processed into |
| 321 |
framework form by step 9. The invocation of ``install_dylib`` that processes |
| 322 |
the ``lib-dynload`` folder can be copied and adapted for this purpose. |
| 323 |
|
| 324 |
* If you're using a separate folder for third-party packages, ensure that folder |
| 325 |
is included as part of the :envvar:`PYTHONPATH` configuration in step 10. |
| 326 |
|
| 327 |
* If any of the folders that contain third-party packages will contain ``.pth`` |
| 328 |
files, you should add that folder as a *site directory* (using |
| 329 |
:meth:`site.addsitedir`), rather than adding to :envvar:`PYTHONPATH` or |
| 330 |
:attr:`sys.path` directly. |
| 331 |
|
| 332 |
Testing a Python package |
| 333 |
------------------------ |
| 334 |
|
| 335 |
The CPython source tree contains :source:`a testbed project <iOS/testbed>` that |
| 336 |
is used to run the CPython test suite on the iOS simulator. This testbed can also |
| 337 |
be used as a testbed project for running your Python library's test suite on iOS. |
| 338 |
|
| 339 |
After building or obtaining an iOS XCFramework (See :source:`iOS/README.rst` |
| 340 |
for details), create a clone of the Python iOS testbed project by running: |
| 341 |
|
| 342 |
.. code-block:: bash |
| 343 |
|
| 344 |
$ python iOS/testbed clone --framework <path/to/Python.xcframework> --app <path/to/module1> --app <path/to/module2> app-testbed |
| 345 |
|
| 346 |
You will need to modify the ``iOS/testbed`` reference to point to that |
| 347 |
directory in the CPython source tree; any folders specified with the ``--app`` |
| 348 |
flag will be copied into the cloned testbed project. The resulting testbed will |
| 349 |
be created in the ``app-testbed`` folder. In this example, the ``module1`` and |
| 350 |
``module2`` would be importable modules at runtime. If your project has |
| 351 |
additional dependencies, they can be installed into the |
| 352 |
``app-testbed/iOSTestbed/app_packages`` folder (using ``pip install --target |
| 353 |
app-testbed/iOSTestbed/app_packages`` or similar). |
| 354 |
|
| 355 |
You can then use the ``app-testbed`` folder to run the test suite for your app, |
| 356 |
For example, if ``module1.tests`` was the entry point to your test suite, you |
| 357 |
could run: |
| 358 |
|
| 359 |
.. code-block:: bash |
| 360 |
|
| 361 |
$ python app-testbed run -- module1.tests |
| 362 |
|
| 363 |
This is the equivalent of running ``python -m module1.tests`` on a desktop |
| 364 |
Python build. Any arguments after the ``--`` will be passed to the testbed as |
| 365 |
if they were arguments to ``python -m`` on a desktop machine. |
| 366 |
|
| 367 |
You can also open the testbed project in Xcode by running: |
| 368 |
|
| 369 |
.. code-block:: bash |
| 370 |
|
| 371 |
$ open app-testbed/iOSTestbed.xcodeproj |
| 372 |
|
| 373 |
This will allow you to use the full Xcode suite of tools for debugging. |
| 374 |
|
| 375 |
The arguments used to run the test suite are defined as part of the test plan. |
| 376 |
To modify the test plan, select the test plan node of the project tree (it |
| 377 |
should be the first child of the root node), and select the "Configurations" |
| 378 |
tab. Modify the "Arguments Passed On Launch" value to change the testing |
| 379 |
arguments. |
| 380 |
|
| 381 |
The test plan also disables parallel testing, and specifies the use of the |
| 382 |
``iOSTestbed.lldbinit`` file for providing configuration of the debugger. The |
| 383 |
default debugger configuration disables automatic breakpoints on the |
| 384 |
``SIGINT``, ``SIGUSR1``, ``SIGUSR2``, and ``SIGXFSZ`` signals. |
| 385 |
|
| 386 |
App Store Compliance |
| 387 |
==================== |
| 388 |
|
| 389 |
The only mechanism for distributing apps to third-party iOS devices is to |
| 390 |
submit the app to the iOS App Store; apps submitted for distribution must pass |
| 391 |
Apple's app review process. This process includes a set of automated validation |
| 392 |
rules that inspect the submitted application bundle for problematic code. |
| 393 |
|
| 394 |
The Python standard library contains some code that is known to violate these |
| 395 |
automated rules. While these violations appear to be false positives, Apple's |
| 396 |
review rules cannot be challenged; so, it is necessary to modify the Python |
| 397 |
standard library for an app to pass App Store review. |
| 398 |
|
| 399 |
The Python source tree contains |
| 400 |
:source:`a patch file <Mac/Resources/app-store-compliance.patch>` that will remove |
| 401 |
all code that is known to cause issues with the App Store review process. This |
| 402 |
patch is applied automatically when building for iOS. |