Debian Codegraph @

iOS/README.rst

raw
1 ====================
2 Python on iOS README
3 ====================
4
5 :Authors:
6 Russell Keith-Magee (2023-11)
7
8 This document provides a quick overview of some iOS specific features in the
9 Python distribution.
10
11 These instructions are only needed if you're planning to compile Python for iOS
12 yourself. Most users should *not* need to do this. If you're looking to
13 experiment with writing an iOS app in Python, tools such as `BeeWare's Briefcase
14 <https://briefcase.readthedocs.io>`__ and `Kivy's Buildozer
15 <https://buildozer.readthedocs.io>`__ will provide a much more approachable
16 user experience.
17
18 Compilers for building on iOS
19 =============================
20
21 Building for iOS requires the use of Apple's Xcode tooling. It is strongly
22 recommended that you use the most recent stable release of Xcode. This will
23 require the use of the most (or second-most) recently released macOS version,
24 as Apple does not maintain Xcode for older macOS versions. The Xcode Command
25 Line Tools are not sufficient for iOS development; you need a *full* Xcode
26 install.
27
28 If you want to run your code on the iOS simulator, you'll also need to install
29 an iOS Simulator Platform. You should be prompted to select an iOS Simulator
30 Platform when you first run Xcode. Alternatively, you can add an iOS Simulator
31 Platform by selecting an open the Platforms tab of the Xcode Settings panel.
32
33 iOS specific arguments to configure
34 ===================================
35
36 * ``--enable-framework[=DIR]``
37
38 This argument specifies the location where the Python.framework will be
39 installed. If ``DIR`` is not specified, the framework will be installed into
40 a subdirectory of the ``iOS/Frameworks`` folder.
41
42 This argument *must* be provided when configuring iOS builds. iOS does not
43 support non-framework builds.
44
45 * ``--with-framework-name=NAME``
46
47 Specify the name for the Python framework; defaults to ``Python``.
48
49 .. admonition:: Use this option with care!
50
51 Unless you know what you're doing, changing the name of the Python
52 framework on iOS is not advised. If you use this option, you won't be able
53 to run the ``make testios`` target without making significant manual
54 alterations, and you won't be able to use any binary packages unless you
55 compile them yourself using your own framework name.
56
57 Building Python on iOS
58 ======================
59
60 ABIs and Architectures
61 ----------------------
62
63 iOS apps can be deployed on physical devices, and on the iOS simulator. Although
64 the API used on these devices is identical, the ABI is different - you need to
65 link against different libraries for an iOS device build (``iphoneos``) or an
66 iOS simulator build (``iphonesimulator``).
67
68 Apple uses the ``XCframework`` format to allow specifying a single dependency
69 that supports multiple ABIs. An ``XCframework`` is a wrapper around multiple
70 ABI-specific frameworks that share a common API.
71
72 iOS can also support different CPU architectures within each ABI. At present,
73 there is only a single supported architecture on physical devices - ARM64.
74 However, the *simulator* supports 2 architectures - ARM64 (for running on Apple
75 Silicon machines), and x86_64 (for running on older Intel-based machines).
76
77 To support multiple CPU architectures on a single platform, Apple uses a "fat
78 binary" format - a single physical file that contains support for multiple
79 architectures. It is possible to compile and use a "thin" single architecture
80 version of a binary for testing purposes; however, the "thin" binary will not be
81 portable to machines using other architectures.
82
83 Building a single-architecture framework
84 ----------------------------------------
85
86 The Python build system will create a ``Python.framework`` that supports a
87 *single* ABI with a *single* architecture. Unlike macOS, iOS does not allow a
88 framework to contain non-library content, so the iOS build will produce a
89 ``bin`` and ``lib`` folder in the same output folder as ``Python.framework``.
90 The ``lib`` folder will be needed at runtime to support the Python library.
91
92 If you want to use Python in a real iOS project, you need to produce multiple
93 ``Python.framework`` builds, one for each ABI and architecture. iOS builds of
94 Python *must* be constructed as framework builds. To support this, you must
95 provide the ``--enable-framework`` flag when configuring the build. The build
96 also requires the use of cross-compilation. The minimal commands for building
97 Python for the ARM64 iOS simulator will look something like::
98
99 $ export PATH="$(pwd)/iOS/Resources/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin"
100 $ ./configure \
101 --enable-framework \
102 --host=arm64-apple-ios-simulator \
103 --build=arm64-apple-darwin \
104 --with-build-python=/path/to/python.exe
105 $ make
106 $ make install
107
108 In this invocation:
109
110 * ``iOS/Resources/bin`` has been added to the path, providing some shims for the
111 compilers and linkers needed by the build. Xcode requires the use of ``xcrun``
112 to invoke compiler tooling. However, if ``xcrun`` is pre-evaluated and the
113 result passed to ``configure``, these results can embed user- and
114 version-specific paths into the sysconfig data, which limits the portability
115 of the compiled Python. Alternatively, if ``xcrun`` is used *as* the compiler,
116 it requires that compiler variables like ``CC`` include spaces, which can
117 cause significant problems with many C configuration systems which assume that
118 ``CC`` will be a single executable.
119
120 To work around this problem, the ``iOS/Resources/bin`` folder contains some
121 wrapper scripts that present as simple compilers and linkers, but wrap
122 underlying calls to ``xcrun``. This allows configure to use a ``CC``
123 definition without spaces, and without user- or version-specific paths, while
124 retaining the ability to adapt to the local Xcode install. These scripts are
125 included in the ``bin`` directory of an iOS install.
126
127 These scripts will, by default, use the currently active Xcode installation.
128 If you want to use a different Xcode installation, you can use
129 ``xcode-select`` to set a new default Xcode globally, or you can use the
130 ``DEVELOPER_DIR`` environment variable to specify an Xcode install. The
131 scripts will use the default ``iphoneos``/``iphonesimulator`` SDK version for
132 the select Xcode install; if you want to use a different SDK, you can set the
133 ``IOS_SDK_VERSION`` environment variable. (e.g, setting
134 ``IOS_SDK_VERSION=17.1`` would cause the scripts to use the ``iphoneos17.1``
135 and ``iphonesimulator17.1`` SDKs, regardless of the Xcode default.)
136
137 The path has also been cleared of any user customizations. A common source of
138 bugs is for tools like Homebrew to accidentally leak macOS binaries into an iOS
139 build. Resetting the path to a known "bare bones" value is the easiest way to
140 avoid these problems.
141
142 * ``--host`` is the architecture and ABI that you want to build, in GNU compiler
143 triple format. This will be one of:
144
145 - ``arm64-apple-ios`` for ARM64 iOS devices.
146 - ``arm64-apple-ios-simulator`` for the iOS simulator running on Apple
147 Silicon devices.
148 - ``x86_64-apple-ios-simulator`` for the iOS simulator running on Intel
149 devices.
150
151 * ``--build`` is the GNU compiler triple for the machine that will be running
152 the compiler. This is one of:
153
154 - ``arm64-apple-darwin`` for Apple Silicon devices.
155 - ``x86_64-apple-darwin`` for Intel devices.
156
157 * ``/path/to/python.exe`` is the path to a Python binary on the machine that
158 will be running the compiler. This is needed because the Python compilation
159 process involves running some Python code. On a normal desktop build of
160 Python, you can compile a python interpreter and then use that interpreter to
161 run Python code. However, the binaries produced for iOS won't run on macOS, so
162 you need to provide an external Python interpreter. This interpreter must be
163 the same version as the Python that is being compiled. To be completely safe,
164 this should be the *exact* same commit hash. However, the longer a Python
165 release has been stable, the more likely it is that this constraint can be
166 relaxed - the same micro version will often be sufficient.
167
168 * The ``install`` target for iOS builds is slightly different to other
169 platforms. On most platforms, ``make install`` will install the build into
170 the final runtime location. This won't be the case for iOS, as the final
171 runtime location will be on a physical device.
172
173 However, you still need to run the ``install`` target for iOS builds, as it
174 performs some final framework assembly steps. The location specified with
175 ``--enable-framework`` will be the location where ``make install`` will
176 assemble the complete iOS framework. This completed framework can then
177 be copied and relocated as required.
178
179 For a full CPython build, you also need to specify the paths to iOS builds of
180 the binary libraries that CPython depends on (XZ, BZip2, LibFFI and OpenSSL).
181 This can be done by defining the ``LIBLZMA_CFLAGS``, ``LIBLZMA_LIBS``,
182 ``BZIP2_CFLAGS``, ``BZIP2_LIBS``, ``LIBFFI_CFLAGS``, and ``LIBFFI_LIBS``
183 environment variables, and the ``--with-openssl`` configure option. Versions of
184 these libraries pre-compiled for iOS can be found in `this repository
185 <https://github.com/beeware/cpython-apple-source-deps/releases>`__. LibFFI is
186 especially important, as many parts of the standard library (including the
187 ``platform``, ``sysconfig`` and ``webbrowser`` modules) require the use of the
188 ``ctypes`` module at runtime.
189
190 By default, Python will be compiled with an iOS deployment target (i.e., the
191 minimum supported iOS version) of 13.0. To specify a different deployment
192 target, provide the version number as part of the ``--host`` argument - for
193 example, ``--host=arm64-apple-ios15.4-simulator`` would compile an ARM64
194 simulator build with a deployment target of 15.4.
195
196 Merge thin frameworks into fat frameworks
197 -----------------------------------------
198
199 Once you've built a ``Python.framework`` for each ABI and architecture, you
200 must produce a "fat" framework for each ABI that contains all the architectures
201 for that ABI.
202
203 The ``iphoneos`` build only needs to support a single architecture, so it can be
204 used without modification.
205
206 If you only want to support a single simulator architecture, (e.g., only support
207 ARM64 simulators), you can use a single architecture ``Python.framework`` build.
208 However, if you want to create ``Python.xcframework`` that supports *all*
209 architectures, you'll need to merge the ``iphonesimulator`` builds for ARM64 and
210 x86_64 into a single "fat" framework.
211
212 The "fat" framework can be constructed by performing a directory merge of the
213 content of the two "thin" ``Python.framework`` directories, plus the ``bin`` and
214 ``lib`` folders for each thin framework. When performing this merge:
215
216 * The pure Python standard library content is identical for each architecture,
217 except for a handful of platform-specific files (such as the ``sysconfig``
218 module). Ensure that the "fat" framework has the union of all standard library
219 files.
220
221 * Any binary files in the standard library, plus the main
222 ``libPython3.X.dylib``, can be merged using the ``lipo`` tool, provide by
223 Xcode::
224
225 $ lipo -create -output module.dylib path/to/x86_64/module.dylib path/to/arm64/module.dylib
226
227 * The header files will be identical on both architectures, except for
228 ``pyconfig.h``. Copy all the headers from one platform (say, arm64), rename
229 ``pyconfig.h`` to ``pyconfig-arm64.h``, and copy the ``pyconfig.h`` for the
230 other architecture into the merged header folder as ``pyconfig-x86_64.h``.
231 Then copy the ``iOS/Resources/pyconfig.h`` file from the CPython sources into
232 the merged headers folder. This will allow the two Python architectures to
233 share a common ``pyconfig.h`` header file.
234
235 At this point, you should have 2 Python.framework folders - one for ``iphoneos``,
236 and one for ``iphonesimulator`` that is a merge of x86+64 and ARM64 content.
237
238 Merge frameworks into an XCframework
239 ------------------------------------
240
241 Now that we have 2 (potentially fat) ABI-specific frameworks, we can merge those
242 frameworks into a single ``XCframework``.
243
244 The initial skeleton of an ``XCframework`` is built using::
245
246 xcodebuild -create-xcframework -output Python.xcframework -framework path/to/iphoneos/Python.framework -framework path/to/iphonesimulator/Python.framework
247
248 Then, copy the ``bin`` and ``lib`` folders into the architecture-specific slices of
249 the XCframework::
250
251 cp path/to/iphoneos/bin Python.xcframework/ios-arm64
252 cp path/to/iphoneos/lib Python.xcframework/ios-arm64
253
254 cp path/to/iphonesimulator/bin Python.xcframework/ios-arm64_x86_64-simulator
255 cp path/to/iphonesimulator/lib Python.xcframework/ios-arm64_x86_64-simulator
256
257 Note that the name of the architecture-specific slice for the simulator will
258 depend on the CPU architecture(s) that you build.
259
260 You now have a Python.xcframework that can be used in a project.
261
262 Testing Python on iOS
263 =====================
264
265 The ``iOS/testbed`` folder that contains an Xcode project that is able to run
266 the iOS test suite. This project converts the Python test suite into a single
267 test case in Xcode's XCTest framework. The single XCTest passes if the test
268 suite passes.
269
270 To run the test suite, configure a Python build for an iOS simulator (i.e.,
271 ``--host=arm64-apple-ios-simulator`` or ``--host=x86_64-apple-ios-simulator``
272 ), specifying a framework build (i.e. ``--enable-framework``). Ensure that your
273 ``PATH`` has been configured to include the ``iOS/Resources/bin`` folder and
274 exclude any non-iOS tools, then run::
275
276 $ make all
277 $ make install
278 $ make testios
279
280 This will:
281
282 * Build an iOS framework for your chosen architecture;
283 * Finalize the single-platform framework;
284 * Make a clean copy of the testbed project;
285 * Install the Python iOS framework into the copy of the testbed project; and
286 * Run the test suite on an "iPhone SE (3rd generation)" simulator.
287
288 On success, the test suite will exit and report successful completion of the
289 test suite. On a 2022 M1 MacBook Pro, the test suite takes approximately 15
290 minutes to run; a couple of extra minutes is required to compile the testbed
291 project, and then boot and prepare the iOS simulator.
292
293 Debugging test failures
294 -----------------------
295
296 Running ``make testios`` generates a standalone version of the ``iOS/testbed``
297 project, and runs the full test suite. It does this using ``iOS/testbed``
298 itself - the folder is an executable module that can be used to create and run
299 a clone of the testbed project.
300
301 You can generate your own standalone testbed instance by running::
302
303 $ python iOS/testbed clone --framework iOS/Frameworks/arm64-iphonesimulator my-testbed
304
305 This invocation assumes that ``iOS/Frameworks/arm64-iphonesimulator`` is the
306 path to the iOS simulator framework for your platform (ARM64 in this case);
307 ``my-testbed`` is the name of the folder for the new testbed clone.
308
309 You can then use the ``my-testbed`` folder to run the Python test suite,
310 passing in any command line arguments you may require. For example, if you're
311 trying to diagnose a failure in the ``os`` module, you might run::
312
313 $ python my-testbed run -- test -W test_os
314
315 This is the equivalent of running ``python -m test -W test_os`` on a desktop
316 Python build. Any arguments after the ``--`` will be passed to testbed as if
317 they were arguments to ``python -m`` on a desktop machine.
318
319 Testing in Xcode
320 ^^^^^^^^^^^^^^^^
321
322 You can also open the testbed project in Xcode by running::
323
324 $ open my-testbed/iOSTestbed.xcodeproj
325
326 This will allow you to use the full Xcode suite of tools for debugging.
327
328 The arguments used to run the test suite are defined as part of the test plan.
329 To modify the test plan, select the test plan node of the project tree (it
330 should be the first child of the root node), and select the "Configurations"
331 tab. Modify the "Arguments Passed On Launch" value to change the testing
332 arguments.
333
334 The test plan also disables parallel testing, and specifies the use of the
335 ``iOSTestbed.lldbinit`` file for providing configuration of the debugger. The
336 default debugger configuration disables automatic breakpoints on the
337 ``SIGINT``, ``SIGUSR1``, ``SIGUSR2``, and ``SIGXFSZ`` signals.
338
339 Testing on an iOS device
340 ^^^^^^^^^^^^^^^^^^^^^^^^
341
342 To test on an iOS device, the app needs to be signed with known developer
343 credentials. To obtain these credentials, you must have an iOS Developer
344 account, and your Xcode install will need to be logged into your account (see
345 the Accounts tab of the Preferences dialog).
346
347 Once the project is open, and you're signed into your Apple Developer account,
348 select the root node of the project tree (labeled "iOSTestbed"), then the
349 "Signing & Capabilities" tab in the details page. Select a development team
350 (this will likely be your own name), and plug in a physical device to your
351 macOS machine with a USB cable. You should then be able to select your physical
352 device from the list of targets in the pulldown in the Xcode titlebar.