Debian Codegraph @

Doc/using/ios.rst

raw
1 .. _using-ios:
2
3 ===================
4 Using Python on iOS
5 ===================
6
7 :Authors:
8 Russell Keith-Magee (2024-03)
9
10 Python on iOS is unlike Python on desktop platforms. On a desktop platform,
11 Python is generally installed as a system resource that can be used by any user
12 of that computer. Users then interact with Python by running a :program:`python`
13 executable and entering commands at an interactive prompt, or by running a
14 Python script.
15
16 On iOS, there is no concept of installing as a system resource. The only unit
17 of software distribution is an "app". There is also no console where you could
18 run a :program:`python` executable, or interact with a Python REPL.
19
20 As a result, the only way you can use Python on iOS is in embedded mode - that
21 is, by writing a native iOS application, and embedding a Python interpreter
22 using ``libPython``, and invoking Python code using the :ref:`Python embedding
23 API <embedding>`. The full Python interpreter, the standard library, and all
24 your Python code is then packaged as a standalone bundle that can be
25 distributed via the iOS App Store.
26
27 If you're looking to experiment for the first time with writing an iOS app in
28 Python, projects such as `BeeWare <https://beeware.org>`__ and `Kivy
29 <https://kivy.org>`__ will provide a much more approachable user experience.
30 These projects manage the complexities associated with getting an iOS project
31 running, so you only need to deal with the Python code itself.
32
33 Python at runtime on iOS
34 ========================
35
36 iOS version compatibility
37 -------------------------
38
39 The minimum supported iOS version is specified at compile time, using the
40 :option:`--host` option to ``configure``. By default, when compiled for iOS,
41 Python will be compiled with a minimum supported iOS version of 13.0. To use a
42 different minimum iOS version, provide the version number as part of the
43 :option:`!--host` argument - for example,
44 ``--host=arm64-apple-ios15.4-simulator`` would compile an ARM64 simulator build
45 with a deployment target of 15.4.
46
47 Platform identification
48 -----------------------
49
50 When executing on iOS, ``sys.platform`` will report as ``ios``. This value will
51 be returned on an iPhone or iPad, regardless of whether the app is running on
52 the simulator or a physical device.
53
54 Information about the specific runtime environment, including the iOS version,
55 device model, and whether the device is a simulator, can be obtained using
56 :func:`platform.ios_ver`. :func:`platform.system` will report ``iOS`` or
57 ``iPadOS``, depending on the device.
58
59 :func:`os.uname` reports kernel-level details; it will report a name of
60 ``Darwin``.
61
62 Standard library availability
63 -----------------------------
64
65 The Python standard library has some notable omissions and restrictions on
66 iOS. See the :ref:`API availability guide for iOS <mobile-availability>` for
67 details.
68
69 Binary extension modules
70 ------------------------
71
72 One notable difference about iOS as a platform is that App Store distribution
73 imposes hard requirements on the packaging of an application. One of these
74 requirements governs how binary extension modules are distributed.
75
76 The iOS App Store requires that *all* binary modules in an iOS app must be
77 dynamic libraries, contained in a framework with appropriate metadata, stored
78 in the ``Frameworks`` folder of the packaged app. There can be only a single
79 binary per framework, and there can be no executable binary material outside
80 the ``Frameworks`` folder.
81
82 This conflicts with the usual Python approach for distributing binaries, which
83 allows a binary extension module to be loaded from any location on
84 ``sys.path``. To ensure compliance with App Store policies, an iOS project must
85 post-process any Python packages, converting ``.so`` binary modules into
86 individual standalone frameworks with appropriate metadata and signing. For
87 details on how to perform this post-processing, see the guide for :ref:`adding
88 Python to your project <adding-ios>`.
89
90 To help Python discover binaries in their new location, the original ``.so``
91 file on ``sys.path`` is replaced with a ``.fwork`` file. This file is a text
92 file containing the location of the framework binary, relative to the app
93 bundle. To allow the framework to resolve back to the original location, the
94 framework must contain a ``.origin`` file that contains the location of the
95 ``.fwork`` file, relative to the app bundle.
96
97 For example, consider the case of an import ``from foo.bar import _whiz``,
98 where ``_whiz`` is implemented with the binary module
99 ``sources/foo/bar/_whiz.abi3.so``, with ``sources`` being the location
100 registered on ``sys.path``, relative to the application bundle. This module
101 *must* be distributed as ``Frameworks/foo.bar._whiz.framework/foo.bar._whiz``
102 (creating the framework name from the full import path of the module), with an
103 ``Info.plist`` file in the ``.framework`` directory identifying the binary as a
104 framework. The ``foo.bar._whiz`` module would be represented in the original
105 location with a ``sources/foo/bar/_whiz.abi3.fwork`` marker file, containing
106 the path ``Frameworks/foo.bar._whiz/foo.bar._whiz``. The framework would also
107 contain ``Frameworks/foo.bar._whiz.framework/foo.bar._whiz.origin``, containing
108 the path to the ``.fwork`` file.
109
110 When running on iOS, the Python interpreter will install an
111 :class:`~importlib.machinery.AppleFrameworkLoader` that is able to read and
112 import ``.fwork`` files. Once imported, the ``__file__`` attribute of the
113 binary module will report as the location of the ``.fwork`` file. However, the
114 :class:`~importlib.machinery.ModuleSpec` for the loaded module will report the
115 ``origin`` as the location of the binary in the framework folder.
116
117 Compiler stub binaries
118 ----------------------
119
120 Xcode doesn't expose explicit compilers for iOS; instead, it uses an ``xcrun``
121 script that resolves to a full compiler path (e.g., ``xcrun --sdk iphoneos
122 clang`` to get the ``clang`` for an iPhone device). However, using this script
123 poses two problems:
124
125 * The output of ``xcrun`` includes paths that are machine specific, resulting
126 in a sysconfig module that cannot be shared between users; and
127
128 * It results in ``CC``/``CPP``/``LD``/``AR`` definitions that include spaces.
129 There is a lot of C ecosystem tooling that assumes that you can split a
130 command line at the first space to get the path to the compiler executable;
131 this isn't the case when using ``xcrun``.
132
133 To avoid these problems, Python provided stubs for these tools. These stubs are
134 shell script wrappers around the underingly ``xcrun`` tools, distributed in a
135 ``bin`` folder distributed alongside the compiled iOS framework. These scripts
136 are relocatable, and will always resolve to the appropriate local system paths.
137 By including these scripts in the bin folder that accompanies a framework, the
138 contents of the ``sysconfig`` module becomes useful for end-users to compile
139 their own modules. When compiling third-party Python modules for iOS, you
140 should ensure these stub binaries are on your path.
141
142 Installing Python on iOS
143 ========================
144
145 Tools for building iOS apps
146 ---------------------------
147
148 Building for iOS requires the use of Apple's Xcode tooling. It is strongly
149 recommended that you use the most recent stable release of Xcode. This will
150 require the use of the most (or second-most) recently released macOS version,
151 as Apple does not maintain Xcode for older macOS versions. The Xcode Command
152 Line Tools are not sufficient for iOS development; you need a *full* Xcode
153 install.
154
155 If you want to run your code on the iOS simulator, you'll also need to install
156 an iOS Simulator Platform. You should be prompted to select an iOS Simulator
157 Platform when you first run Xcode. Alternatively, you can add an iOS Simulator
158 Platform by selecting from the Platforms tab of the Xcode Settings panel.
159
160 .. _adding-ios:
161
162 Adding Python to an iOS project
163 -------------------------------
164
165 Python can be added to any iOS project, using either Swift or Objective C. The
166 following examples will use Objective C; if you are using Swift, you may find a
167 library like `PythonKit <https://github.com/pvieito/PythonKit>`__ to be
168 helpful.
169
170 To add Python to an iOS Xcode project:
171
172 1. Build or obtain a Python ``XCFramework``. See the instructions in
173 :source:`iOS/README.rst` (in the CPython source distribution) for details on
174 how to build a Python ``XCFramework``. At a minimum, you will need a build
175 that supports ``arm64-apple-ios``, plus one of either
176 ``arm64-apple-ios-simulator`` or ``x86_64-apple-ios-simulator``.
177
178 2. Drag the ``XCframework`` into your iOS project. In the following
179 instructions, we'll assume you've dropped the ``XCframework`` into the root
180 of your project; however, you can use any other location that you want by
181 adjusting paths as needed.
182
183 3. Drag the ``iOS/Resources/dylib-Info-template.plist`` file into your project,
184 and ensure it is associated with the app target.
185
186 4. Add your application code as a folder in your Xcode project. In the
187 following instructions, we'll assume that your user code is in a folder
188 named ``app`` in the root of your project; you can use any other location by
189 adjusting paths as needed. Ensure that this folder is associated with your
190 app target.
191
192 5. Select the app target by selecting the root node of your Xcode project, then
193 the target name in the sidebar that appears.
194
195 6. In the "General" settings, under "Frameworks, Libraries and Embedded
196 Content", add ``Python.xcframework``, with "Embed & Sign" selected.
197
198 7. In the "Build Settings" tab, modify the following:
199
200 - Build Options
201
202 * User Script Sandboxing: No
203 * Enable Testability: Yes
204
205 - Search Paths
206
207 * Framework Search Paths: ``$(PROJECT_DIR)``
208 * Header Search Paths: ``"$(BUILT_PRODUCTS_DIR)/Python.framework/Headers"``
209
210 - Apple Clang - Warnings - All languages
211
212 * Quoted Include In Framework Header: No
213
214 8. Add a build step that copies the Python standard library into your app. In
215 the "Build Phases" tab, add a new "Run Script" build step *before* the
216 "Embed Frameworks" step, but *after* the "Copy Bundle Resources" step. Name
217 the step "Install Target Specific Python Standard Library", disable the
218 "Based on dependency analysis" checkbox, and set the script content to:
219
220 .. code-block:: bash
221
222 set -e
223
224 mkdir -p "$CODESIGNING_FOLDER_PATH/python/lib"
225 if [ "$EFFECTIVE_PLATFORM_NAME" = "-iphonesimulator" ]; then
226 echo "Installing Python modules for iOS Simulator"
227 rsync -au --delete "$PROJECT_DIR/Python.xcframework/ios-arm64_x86_64-simulator/lib/" "$CODESIGNING_FOLDER_PATH/python/lib/"
228 else
229 echo "Installing Python modules for iOS Device"
230 rsync -au --delete "$PROJECT_DIR/Python.xcframework/ios-arm64/lib/" "$CODESIGNING_FOLDER_PATH/python/lib/"
231 fi
232
233 Note that the name of the simulator "slice" in the XCframework may be
234 different, depending the CPU architectures your ``XCFramework`` supports.
235
236 9. Add a second build step that processes the binary extension modules in the
237 standard library into "Framework" format. Add a "Run Script" build step
238 *directly after* the one you added in step 8, named "Prepare Python Binary
239 Modules". It should also have "Based on dependency analysis" unchecked, with
240 the following script content:
241
242 .. code-block:: bash
243
244 set -e
245
246 install_dylib () {
247 INSTALL_BASE=$1
248 FULL_EXT=$2
249
250 # The name of the extension file
251 EXT=$(basename "$FULL_EXT")
252 # The location of the extension file, relative to the bundle
253 RELATIVE_EXT=${FULL_EXT#$CODESIGNING_FOLDER_PATH/}
254 # The path to the extension file, relative to the install base
255 PYTHON_EXT=${RELATIVE_EXT/$INSTALL_BASE/}
256 # The full dotted name of the extension module, constructed from the file path.
257 FULL_MODULE_NAME=$(echo $PYTHON_EXT | cut -d "." -f 1 | tr "/" ".");
258 # A bundle identifier; not actually used, but required by Xcode framework packaging
259 FRAMEWORK_BUNDLE_ID=$(echo $PRODUCT_BUNDLE_IDENTIFIER.$FULL_MODULE_NAME | tr "_" "-")
260 # The name of the framework folder.
261 FRAMEWORK_FOLDER="Frameworks/$FULL_MODULE_NAME.framework"
262
263 # If the framework folder doesn't exist, create it.
264 if [ ! -d "$CODESIGNING_FOLDER_PATH/$FRAMEWORK_FOLDER" ]; then
265 echo "Creating framework for $RELATIVE_EXT"
266 mkdir -p "$CODESIGNING_FOLDER_PATH/$FRAMEWORK_FOLDER"
267 cp "$CODESIGNING_FOLDER_PATH/dylib-Info-template.plist" "$CODESIGNING_FOLDER_PATH/$FRAMEWORK_FOLDER/Info.plist"
268 plutil -replace CFBundleExecutable -string "$FULL_MODULE_NAME" "$CODESIGNING_FOLDER_PATH/$FRAMEWORK_FOLDER/Info.plist"
269 plutil -replace CFBundleIdentifier -string "$FRAMEWORK_BUNDLE_ID" "$CODESIGNING_FOLDER_PATH/$FRAMEWORK_FOLDER/Info.plist"
270 fi
271
272 echo "Installing binary for $FRAMEWORK_FOLDER/$FULL_MODULE_NAME"
273 mv "$FULL_EXT" "$CODESIGNING_FOLDER_PATH/$FRAMEWORK_FOLDER/$FULL_MODULE_NAME"
274 # Create a placeholder .fwork file where the .so was
275 echo "$FRAMEWORK_FOLDER/$FULL_MODULE_NAME" > ${FULL_EXT%.so}.fwork
276 # Create a back reference to the .so file location in the framework
277 echo "${RELATIVE_EXT%.so}.fwork" > "$CODESIGNING_FOLDER_PATH/$FRAMEWORK_FOLDER/$FULL_MODULE_NAME.origin"
278 }
279
280 PYTHON_VER=$(ls -1 "$CODESIGNING_FOLDER_PATH/python/lib")
281 echo "Install Python $PYTHON_VER standard library extension modules..."
282 find "$CODESIGNING_FOLDER_PATH/python/lib/$PYTHON_VER/lib-dynload" -name "*.so" | while read FULL_EXT; do
283 install_dylib python/lib/$PYTHON_VER/lib-dynload/ "$FULL_EXT"
284 done
285
286 # Clean up dylib template
287 rm -f "$CODESIGNING_FOLDER_PATH/dylib-Info-template.plist"
288
289 echo "Signing frameworks as $EXPANDED_CODE_SIGN_IDENTITY_NAME ($EXPANDED_CODE_SIGN_IDENTITY)..."
290 find "$CODESIGNING_FOLDER_PATH/Frameworks" -name "*.framework" -exec /usr/bin/codesign --force --sign "$EXPANDED_CODE_SIGN_IDENTITY" ${OTHER_CODE_SIGN_FLAGS:-} -o runtime --timestamp=none --preserve-metadata=identifier,entitlements,flags --generate-entitlement-der "{}" \;
291
292 10. Add Objective C code to initialize and use a Python interpreter in embedded
293 mode. You should ensure that:
294
295 * UTF-8 mode (:c:member:`PyPreConfig.utf8_mode`) is *enabled*;
296 * Buffered stdio (:c:member:`PyConfig.buffered_stdio`) is *disabled*;
297 * Writing bytecode (:c:member:`PyConfig.write_bytecode`) is *disabled*;
298 * Signal handlers (:c:member:`PyConfig.install_signal_handlers`) are *enabled*;
299 * :envvar:`PYTHONHOME` for the interpreter is configured to point at the
300 ``python`` subfolder of your app's bundle; and
301 * The :envvar:`PYTHONPATH` for the interpreter includes:
302
303 - the ``python/lib/python3.X`` subfolder of your app's bundle,
304 - the ``python/lib/python3.X/lib-dynload`` subfolder of your app's bundle, and
305 - the ``app`` subfolder of your app's bundle
306
307 Your app's bundle location can be determined using ``[[NSBundle mainBundle]
308 resourcePath]``.
309
310 Steps 8, 9 and 10 of these instructions assume that you have a single folder of
311 pure Python application code, named ``app``. If you have third-party binary
312 modules in your app, some additional steps will be required:
313
314 * You need to ensure that any folders containing third-party binaries are
315 either associated with the app target, or copied in as part of step 8. Step 8
316 should also purge any binaries that are not appropriate for the platform a
317 specific build is targeting (i.e., delete any device binaries if you're
318 building an app targeting the simulator).
319
320 * Any folders that contain third-party binaries must be processed into
321 framework form by step 9. The invocation of ``install_dylib`` that processes
322 the ``lib-dynload`` folder can be copied and adapted for this purpose.
323
324 * If you're using a separate folder for third-party packages, ensure that folder
325 is included as part of the :envvar:`PYTHONPATH` configuration in step 10.
326
327 * If any of the folders that contain third-party packages will contain ``.pth``
328 files, you should add that folder as a *site directory* (using
329 :meth:`site.addsitedir`), rather than adding to :envvar:`PYTHONPATH` or
330 :attr:`sys.path` directly.
331
332 Testing a Python package
333 ------------------------
334
335 The CPython source tree contains :source:`a testbed project <iOS/testbed>` that
336 is used to run the CPython test suite on the iOS simulator. This testbed can also
337 be used as a testbed project for running your Python library's test suite on iOS.
338
339 After building or obtaining an iOS XCFramework (See :source:`iOS/README.rst`
340 for details), create a clone of the Python iOS testbed project by running:
341
342 .. code-block:: bash
343
344 $ python iOS/testbed clone --framework <path/to/Python.xcframework> --app <path/to/module1> --app <path/to/module2> app-testbed
345
346 You will need to modify the ``iOS/testbed`` reference to point to that
347 directory in the CPython source tree; any folders specified with the ``--app``
348 flag will be copied into the cloned testbed project. The resulting testbed will
349 be created in the ``app-testbed`` folder. In this example, the ``module1`` and
350 ``module2`` would be importable modules at runtime. If your project has
351 additional dependencies, they can be installed into the
352 ``app-testbed/iOSTestbed/app_packages`` folder (using ``pip install --target
353 app-testbed/iOSTestbed/app_packages`` or similar).
354
355 You can then use the ``app-testbed`` folder to run the test suite for your app,
356 For example, if ``module1.tests`` was the entry point to your test suite, you
357 could run:
358
359 .. code-block:: bash
360
361 $ python app-testbed run -- module1.tests
362
363 This is the equivalent of running ``python -m module1.tests`` on a desktop
364 Python build. Any arguments after the ``--`` will be passed to the testbed as
365 if they were arguments to ``python -m`` on a desktop machine.
366
367 You can also open the testbed project in Xcode by running:
368
369 .. code-block:: bash
370
371 $ open app-testbed/iOSTestbed.xcodeproj
372
373 This will allow you to use the full Xcode suite of tools for debugging.
374
375 The arguments used to run the test suite are defined as part of the test plan.
376 To modify the test plan, select the test plan node of the project tree (it
377 should be the first child of the root node), and select the "Configurations"
378 tab. Modify the "Arguments Passed On Launch" value to change the testing
379 arguments.
380
381 The test plan also disables parallel testing, and specifies the use of the
382 ``iOSTestbed.lldbinit`` file for providing configuration of the debugger. The
383 default debugger configuration disables automatic breakpoints on the
384 ``SIGINT``, ``SIGUSR1``, ``SIGUSR2``, and ``SIGXFSZ`` signals.
385
386 App Store Compliance
387 ====================
388
389 The only mechanism for distributing apps to third-party iOS devices is to
390 submit the app to the iOS App Store; apps submitted for distribution must pass
391 Apple's app review process. This process includes a set of automated validation
392 rules that inspect the submitted application bundle for problematic code.
393
394 The Python standard library contains some code that is known to violate these
395 automated rules. While these violations appear to be false positives, Apple's
396 review rules cannot be challenged; so, it is necessary to modify the Python
397 standard library for an app to pass App Store review.
398
399 The Python source tree contains
400 :source:`a patch file <Mac/Resources/app-store-compliance.patch>` that will remove
401 all code that is known to cause issues with the App Store review process. This
402 patch is applied automatically when building for iOS.