Debian Codegraph @

Tools/msi/README.txt

raw
1 Quick Build Info
2 ================
3
4 For testing, the installer should be built with the Tools/msi/build.bat
5 script:
6
7 build.bat [-x86] [-x64] [-ARM64] [--doc]
8
9 For an official release, the installer should be built with the
10 Tools/msi/buildrelease.bat script and environment variables:
11
12 set PYTHON=<path to Python 3.10 or later>
13 set SPHINXBUILD=<path to sphinx-build.exe>
14 set PATH=<path to Git (git.exe)>;%PATH%
15
16 buildrelease.bat [-x86] [-x64] [-ARM64] [-D] [-B]
17 [-o <output directory>] [-c <certificate name>]
18
19 See the Building the Installer section for more information.
20
21 Overview
22 ========
23
24 Python is distributed on Windows as an installer that will configure the
25 user's system. This allows users to have a functioning copy of Python
26 without having to build it themselves.
27
28 The main tasks of the installer are:
29
30 * copy required files into the expected layout
31 * configure system settings so the installation can be located by
32 other programs
33 * add entry points for modifying, repairing and uninstalling Python
34 * make it easy to launch Python, its documentation, and IDLE
35
36 Each of these is discussed in a later section of this document.
37
38 Structure of the Installer
39 ==========================
40
41 The installer is structured as a 'layout', which consists of a number of
42 CAB and MSI files and a single EXE.
43
44 The EXE is the main entry point into the installer. It contains the UI
45 and command-line logic, as well as the ability to locate and optionally
46 download other parts of the layout.
47
48 Each MSI contains the logic required to install a component or feature
49 of Python. These MSIs should not be launched directly by users. MSIs can
50 be embedded into the EXE or automatically downloaded as needed.
51
52 Each CAB contains the files making up a Python installation. CABs are
53 embedded into their associated MSI and are never seen by users.
54
55 MSIs are only required when the related feature or component is being
56 installed. When components are not selected for installation, the
57 associated MSI is not downloaded. This allows the installer to offer
58 options to install debugging symbols and binaries without increasing
59 the initial download size by separating them into their own MSIs.
60
61 Building the Installer
62 ======================
63
64 Before building the installer, download the extra build dependencies
65 using Tools\msi\get_externals.bat. (Note that this is in addition to the
66 similarly named file in PCbuild.)
67
68 One of the dependencies used in the build process is WiX, a toolset that
69 allows developers to create installers for Windows Installer, the
70 Windows installation engine. If you're not using the copy of WiX
71 installed by Tools\msi\get_externals.bat, you'll need to set the
72 "WixInstallPath" environment variable before building.
73
74 Additionally, ensure that "MSVC v14x - VS 20xx C++ ARM64/ARM64EC build tools"
75 is selected under "Desktop Development with C++" in the "Visual Studio Installer",
76 even if you're not building on ARM64. This is required because we've upgraded
77 to WiX 3.14, which requires these tools for Python 3.10 and later versions.
78
79 For testing, the installer should be built with the Tools/msi/build.bat
80 script:
81
82 build.bat [-x86] [-x64] [-ARM64] [--doc] [--test-marker] [--pack]
83
84 This script will build the required configurations of Python and
85 generate an installer layout in PCbuild/(win32|amd64)/en-us.
86
87 Specify -x86, -x64 and/or -ARM64 to build for each platform. If none are
88 specified, both x64 and x86 will be built. Currently, both the debug and
89 release versions of Python are required for the installer.
90
91 Specify --doc to include the documentation files. Ensure %PYTHON% and
92 %SPHINXBUILD% are set when passing this option.
93
94 Specify --test-marker to build an installer that works side-by-side with
95 an official Python release. All registry keys and install locations will
96 include an extra marker to avoid overwriting files. This marker is
97 currently an 'x' prefix, but may change at any time.
98
99 Specify --pack to build an installer that does not require all MSIs to
100 be available alongside. This takes longer, but is easier to share.
101
102
103 For an official release, the installer should be built with the
104 Tools/msi/buildrelease.bat script:
105
106 set PYTHON=<path to Python 3.10 or later>
107 set SPHINXBUILD=<path to sphinx-build.exe>
108 set PATH=<path to Git (git.exe)>;%PATH%
109
110 buildrelease.bat [-x86] [-x64] [-ARM64] [-D] [-B]
111 [-o <output directory>] [-c <certificate name>]
112
113 Specify -x86, -x64 and/or -ARM64 to build for each platform. If none are
114 specified, both x64 and x86 will be built. Currently, both the debug and
115 release versions of Python are required for the installer.
116
117 Specify -D to skip rebuilding the documentation. The documentation is
118 required for a release and the build will fail if it is not available.
119 Ensure %PYTHON% and %SPHINXBUILD% are set if you omit this option.
120
121 Specify -B to skip rebuilding Python. This is useful to only rebuild the
122 installer layout after a previous call to buildrelease.bat.
123
124 Specify -o to set an output directory. The installer layouts will be
125 copied to platform-specific subdirectories of this path.
126
127 Specify -c to choose a code-signing certificate to be used for all the
128 signable binaries in Python as well as each file making up the
129 installer. Official releases of Python must be signed.
130
131
132 If WiX is not found on your system, it will be automatically downloaded
133 and extracted to the externals/ directory.
134
135 To manually build layouts of the installer, build one of the projects in
136 the bundle folder.
137
138 msbuild bundle\snapshot.wixproj
139 msbuild bundle\releaseweb.wixproj
140 msbuild bundle\releaselocal.wixproj
141 msbuild bundle\full.wixproj
142
143 snapshot.wixproj produces a test installer versioned based on the date.
144
145 releaseweb.wixproj produces a release installer that does not embed any
146 of the layout.
147
148 releaselocal.wixproj produces a release installer that embeds the files
149 required for a default installation.
150
151 full.wixproj produces a test installer that embeds the entire layout.
152
153 The following properties may be passed when building these projects.
154
155 /p:BuildForRelease=(true|false)
156 When true, adds extra verification to ensure a complete installer is
157 produced. Defaults to false.
158
159 /p:ReleaseUri=(any URI)
160 Used to generate unique IDs for the installers to allow side-by-side
161 installation. Forks of Python can use the same installer infrastructure
162 by providing a unique URI for this property. It does not need to be an
163 active internet address. Defaults to $(ComputerName).
164
165 Official releases use https://www.python.org/(architecture name)
166
167 /p:DownloadUrlBase=(any URI)
168 Specifies the base of a URL where missing parts of the installer layout
169 can be downloaded from. The build version and architecture will be
170 appended to create the full address. If omitted, missing components will
171 not be automatically downloaded.
172
173 /p:DownloadUrl=(any URI)
174 Specifies the full URL where missing parts of the installer layout can
175 be downloaded from. Should normally include '{2}', which will be
176 substituted for the filename. If omitted, missing components will not be
177 automatically downloaded. If specified, this value overrides
178 DownloadUrlBase.
179
180 /p:SigningCertificate=(certificate name)
181 Specifies the certificate to sign the installer layout with. If omitted,
182 the layout will not be signed.
183
184 /p:RebuildAll=(true|false)
185 When true, rebuilds all of the MSIs making up the layout. Defaults to
186 true.
187
188 Uploading the Installer
189 =======================
190
191 For official releases, the uploadrelease.bat script should be used.
192
193 You will require PuTTY so that plink.exe and pscp.exe can be used, and your
194 SSH key can be activated in pageant.exe. PuTTY should be either on your path
195 or in %ProgramFiles(x86)%\PuTTY.
196
197 To include signatures for each uploaded file, you will need gpg2.exe on your
198 path or have run get_externals.bat. You may also need to "gpg2.exe --import"
199 your key before running the upload script.
200
201 uploadrelease.bat --host <host> --user <username> [--dry-run] [--no-gpg]
202
203 The host is the URL to the server. This can be provided by the Release
204 Manager. You should be able to SSH to this address.
205
206 The username is your own username, which you have permission to SSH into
207 the server containing downloads.
208
209 Use --dry-run to display the generated upload commands without executing
210 them. Signatures for each file will be generated but not uploaded unless
211 --no-gpg is also passed.
212
213 Use --no-gpg to suppress signature generation and upload.
214
215 The default target directory (which appears in uploadrelease.proj) is
216 correct for official Python releases, but may be overridden with
217 --target <path> for other purposes. This path should generally not include
218 any version specifier, as that will be added automatically.
219
220 Modifying the Installer
221 =======================
222
223 The code for the installer is divided into three main groups: packages,
224 the bundle and the bootstrap application.
225
226 Packages
227 --------
228
229 Packages appear as subdirectories of Tools/msi (other than the bundle/
230 directory). The project file is a .wixproj and the build output is a
231 single MSI. Packages are built with the WiX Toolset. Some project files
232 share source files and use preprocessor directives to enable particular
233 features. These are typically used to keep the sources close when the
234 files are related, but produce multiple independent packages.
235
236 A package is the smallest element that may be independently installed or
237 uninstalled (as used in this installer). For example, the test suite has
238 its own package, as users can choose to add or remove it after the
239 initial installation.
240
241 All the files installed by a single package should be related, though
242 some packages may not install any files. For example, the pip package
243 executes the ensurepip package, but does not add or remove any of its
244 own files. (It is represented as a package because of its
245 installed/uninstalled nature, as opposed to the "precompile standard
246 library" option, for example.) Dependencies between packages are handled
247 by the bundle, but packages should detect when dependencies are missing
248 and raise an error.
249
250 Packages that include a lot of files may use an InstallFiles element in
251 the .wixproj file to generate sources. See lib/lib.wixproj for an
252 example, and msi.targets and csv_to_wxs.py for the implementation. This
253 element is also responsible for generating the code for cleaning up and
254 removing __pycache__ folders in any directory containing .py files.
255
256 All packages are built with the Tools/msi/common.wxs file, and so any
257 directory or property in this file may be referenced. Of particular
258 interest:
259
260 REGISTRYKEY (property)
261 The registry key for the current installation.
262
263 InstallDirectory (directory)
264 The root install directory for the current installation. Subdirectories
265 are also specified in this file (DLLs, Lib, etc.)
266
267 MenuDir (directory)
268 The Start Menu folder for the current installation.
269
270 UpgradeTable (property)
271 Every package should reference this property to include upgrade
272 information.
273
274 OptionalFeature (Component)
275 Packages that may be enabled or disabled should reference this component
276 and have an OPTIONAL_FEATURES entry in the bootstrap application to
277 properly handle Modify and Upgrade.
278
279 The .wxl_template file is specially handled by the build system for this
280 project to perform {{substitutions}} as defined in msi.targets. They
281 should be included in projects as <WxlTemplate> items, where .wxl files
282 are normally included as <EmbeddedResource> items.
283
284 Bundle
285 ------
286
287 The bundle is compiled to the main EXE entry point that for most users
288 will represent the Python installer. It is built from Tools/msi/bundle
289 with packages references in Tools/msi/bundle/packagegroups.
290
291 Build logic for the bundle is in bundle.targets, but should be invoked
292 through one of the .wixproj files as described in Building the
293 Installer.
294
295 The UI is separated between Default.thm (UI layout), Default.wxl
296 (strings), bundle.wxs (properties) and the bootstrap application.
297 Bundle.wxs also contains the chain, which is the list of packages to
298 install and the order they should be installed in. These refer to named
299 package groups in bundle/packagegroups.
300
301 Each package group specifies one or more packages to install. Most
302 packages require two separate entries to support both per-user and
303 all-users installations. Because these reuse the same package, it does
304 not increase the overall size of the package.
305
306 Package groups refer to payload groups, which allow better control over
307 embedding and downloading files than the default settings. Whether files
308 are embedded and where they are downloaded from depends on settings
309 created by the project files.
310
311 Package references can include install conditions that determine when to
312 install the package. When a package is a dependency for others, the
313 condition should be crafted to ensure it is installed.
314
315 MSI packages are installed or uninstalled based on their current state
316 and the install condition. This makes them most suitable for features
317 that are clearly present or absent from the user's machine.
318
319 EXE packages are executed based on a customisable condition that can be
320 omitted. This makes them suitable for pre- or post-install tasks that
321 need to run regardless of whether features have been added or removed.
322
323 Bootstrap Application
324 ---------------------
325
326 The bootstrap application is a C++ application that controls the UI and
327 installation. While it does not directly compile into the main EXE of
328 the installer, it forms the main active component. Most of the
329 installation functionality is provided by WiX, and so the bootstrap
330 application is predominantly responsible for the code behind the UI that
331 is defined in the Default.thm file. The bootstrap application code is in
332 bundle/bootstrap and is built automatically when building the bundle.
333
334 Installation Layout
335 ===================
336
337 There are two installation layouts for Python on Windows, with the only
338 differences being supporting files. A layout is selected implicitly
339 based on whether the install is for all users of the machine or just for
340 the user performing the installation.
341
342 The default installation location when installing for all users is
343 "%ProgramFiles%\Python3X" for the 64-bit interpreter and
344 "%ProgramFiles(x86)%\Python3X-32" for the 32-bit interpreter. (Note that
345 the latter path is equivalent to "%ProgramFiles%\Python3X-32" when
346 running a 32-bit version of Windows.) This location requires
347 administrative privileges to install or later modify the installation.
348
349 The default installation location when installing for the current user
350 is "%LocalAppData%\Programs\Python\Python3X" for the 64-bit interpreter
351 and "%LocalAppData%\Programs\Python\Python3X-32" for the 32-bit
352 interpreter. Only the current user can access this location. This
353 provides a suitable level of protection against malicious modification
354 of Python's files.
355
356 (Default installation locations are set in Tools\msi\bundle\bundle.wxs.)
357
358 Within this install directory is the following approximate layout:
359
360 .\python[w].exe The core executable files
361 .\python3x.dll The core interpreter
362 .\python3.dll The stable ABI reference
363 .\DLLs Stdlib extensions (*.pyd) and dependencies
364 .\Doc Documentation (*.html)
365 .\include Development headers (*.h)
366 .\Lib Standard library
367 .\Lib\test Test suite
368 .\libs Development libraries (*.lib)
369 .\Scripts Launcher scripts (*.exe, *.py)
370 .\tcl Tcl dependencies (*.dll, *.tcl and others)
371 .\Tools Tool scripts (*.py)
372
373 When installed for all users, the following files are installed to
374 "%SystemRoot%" (typically "C:\Windows") to ensure they are always
375 available on PATH. (See Launching Python below.) For the current user,
376 they are installed in "%LocalAppData%\Programs\Python\PyLauncher".
377
378 .\py[w].exe PEP 397 launcher
379
380
381 System Settings
382 ===============
383
384 On installation, registry keys are created so that other applications
385 can locate and identify installations of Python. The locations of these
386 keys vary based on the install type.
387
388 For 64-bit interpreters installed for all users, the root key is:
389 HKEY_LOCAL_MACHINE\Software\Python\PythonCore\3.X
390
391 For 32-bit interpreters installed for all users on a 64-bit operating
392 system, the root key is:
393 HKEY_LOCAL_MACHINE\Software\Wow6432Node\Python\PythonCore\3.X-32
394
395 For 32-bit interpreters installed for all users on a 32-bit operating
396 system, the root key is:
397 HKEY_LOCAL_MACHINE\Software\Python\PythonCore\3.X-32
398
399 For 64-bit interpreters installed for the current user:
400 HKEY_CURRENT_USER\Software\Python\PythonCore\3.X
401
402 For 32-bit interpreters installed for the current user:
403 HKEY_CURRENT_USER\Software\Python\PythonCore\3.X-32
404
405 When the core Python executables are installed, a key "InstallPath" is
406 created within the root key with its default value set to the
407 executable's install directory. A value named "ExecutablePath" is added
408 with the full path to the main Python interpreter, and a key
409 "InstallGroup" is created with its default value set to the product
410 name "Python 3.X".
411
412 When the Python standard library is installed, a key "PythonPath" is
413 created within the root key with its default value set to the full path
414 to the Lib folder followed by the path to the DLLs folder, separated by
415 a semicolon.
416
417 When the documentation is installed, a key "Help" is created within the
418 root key, with a subkey "Main Python Documentation" with its default
419 value set to the full path to the main index.html file.
420
421
422 The py.exe launcher is installed as part of a regular Python install,
423 but using a separate mechanism that allows it to more easily span
424 versions of Python. As a result, it has different root keys for its
425 registry entries:
426
427 When installed for all users on a 64-bit operating system, the
428 launcher's root key is:
429 HKEY_LOCAL_MACHINE\Software\Wow6432Node\Python\Launcher
430
431 When installed for all users on a 32-bit operating system, the
432 launcher's root key is:
433 HKEY_LOCAL_MACHINE\Software\Python\Launcher
434
435 When installed for the current user:
436 HKEY_CURRENT_USER\Software\Python\Launcher
437
438 When the launcher is installed, a key "InstallPath" is created within
439 its root key with its default value set to the launcher's install
440 directory. File associations are also created for .py, .pyw, .pyc and
441 .pyo files.
442
443 Launching Python
444 ================
445
446 When a feature offering user entry points in the Start Menu is
447 installed, a folder "Python 3.X" is created. Every shortcut should be
448 created within this folder, and each shortcut should include the version
449 and platform to allow users to identify the shortcut in a search results
450 page.
451
452 The core Python executables creates a shortcut "Python 3.X (32-bit)" or
453 "Python 3.X (64-bit)" depending on the interpreter.
454
455 The documentation creates a shortcut "Python 3.X 32-bit Manuals" or
456 "Python 3.X 64-bit Manuals". The documentation is identical for all
457 platforms, but the shortcuts need to be separate to avoid uninstallation
458 conflicts.
459
460 Installing IDLE creates a shortcut "IDLE (Python 3.X 32-bit)" or "IDLE
461 (Python 3.X 64-bit)" depending on the interpreter.
462
463
464 For users who often launch Python from a Command Prompt, an option is
465 provided to add the directory containing python.exe to the user or
466 system PATH variable. If the option is selected, the install directory
467 and the Scripts directory will be added at the start of the system PATH
468 for an all users install and the user PATH for a per-user install.
469
470 When the user only has one version of Python installed, this will behave
471 as expected. However, because Windows searches the system PATH before
472 the user PATH, users cannot override a system-wide installation of
473 Python on their PATH. Further, because the installer can only prepend to
474 the path, later installations of Python will take precedence over
475 earlier installations, regardless of interpreter version.
476
477 Because it is not possible to automatically create a sensible PATH
478 configuration, users are recommended to use the py.exe launcher and
479 manually modify their PATH variable to add Scripts directories in their
480 preferred order. System-wide installations of Python should consider not
481 modifying PATH, or using an alternative technology to modify their
482 users' PATH variables.
483
484
485 The py.exe launcher is recommended because it uses a consistent and
486 sensible search order for Python installations. User installations are
487 preferred over system-wide installs, and later versions are preferred
488 regardless of installation order (with the exception that py.exe
489 currently prefers 2.x versions over 3.x versions without the -3 command
490 line argument).
491
492 For both 32-bit and 64-bit interpreters, the 32-bit version of the
493 launcher is installed. This ensures that the search order is always
494 consistent (as the 64-bit launcher is subtly different from the 32-bit
495 launcher) and also avoids the need to install it multiple times. Future
496 versions of Python will upgrade the launcher in-place, using Windows
497 Installer's upgrade functionality to avoid conflicts with earlier
498 installed versions.
499
500 When installed, file associations are created for .py, .pyc and .pyo
501 files to launch with py.exe and .pyw files to launch with pyw.exe. This
502 makes Python files respect shebang lines by default and also avoids
503 conflicts between multiple Python installations.
504
505
506 Repair, Modify and Uninstall
507 ============================
508
509 After installation, Python may be modified, repaired or uninstalled by
510 running the original EXE again or via the Programs and Features applet
511 (formerly known as Add or Remove Programs).
512
513 Modifications allow features to be added or removed. The install
514 directory and kind (all users/single user) cannot be modified. Because
515 Windows Installer caches installation packages, removing features will
516 not require internet access unless the package cache has been corrupted
517 or deleted. Adding features that were not previously installed and are
518 not embedded or otherwise available will require internet access.
519
520 Repairing will rerun the installation for all currently installed
521 features, restoring files and registry keys that have been modified or
522 removed. This operation generally will not redownload any files unless
523 the cached packages have been corrupted or deleted.
524
525 Removing Python will clean up all the files and registry keys that were
526 created by the installer, as well as __pycache__ folders that are
527 explicitly handled by the installer. Python packages installed later
528 using a tool like pip will not be removed. Some components may be
529 installed by other installers and these will not be removed if another
530 product has a dependency on them.
531