Debian Codegraph @

Tools/msi/README.txt

raw
1 Quick Build Info
2 ================
3
4 For testing, the installer should be built with the Tools/msi/build.bat
5 script:
6
7 build.bat [-x86] [-x64] [-ARM64] [--doc]
8
9 For an official release, the installer should be built with the
10 Tools/msi/buildrelease.bat script and environment variables:
11
12 set PYTHON=<path to Python 3.8 or later>
13 set SPHINXBUILD=<path to sphinx-build.exe>
14 set PATH=<path to Git (git.exe)>;%PATH%
15
16 buildrelease.bat [-x86] [-x64] [-ARM64] [-D] [-B]
17 [-o <output directory>] [-c <certificate name>]
18
19 See the Building the Installer section for more information.
20
21 Overview
22 ========
23
24 Python is distributed on Windows as an installer that will configure the
25 user's system. This allows users to have a functioning copy of Python
26 without having to build it themselves.
27
28 The main tasks of the installer are:
29
30 * copy required files into the expected layout
31 * configure system settings so the installation can be located by
32 other programs
33 * add entry points for modifying, repairing and uninstalling Python
34 * make it easy to launch Python, its documentation, and IDLE
35
36 Each of these is discussed in a later section of this document.
37
38 Structure of the Installer
39 ==========================
40
41 The installer is structured as a 'layout', which consists of a number of
42 CAB and MSI files and a single EXE.
43
44 The EXE is the main entry point into the installer. It contains the UI
45 and command-line logic, as well as the ability to locate and optionally
46 download other parts of the layout.
47
48 Each MSI contains the logic required to install a component or feature
49 of Python. These MSIs should not be launched directly by users. MSIs can
50 be embedded into the EXE or automatically downloaded as needed.
51
52 Each CAB contains the files making up a Python installation. CABs are
53 embedded into their associated MSI and are never seen by users.
54
55 MSIs are only required when the related feature or component is being
56 installed. When components are not selected for installation, the
57 associated MSI is not downloaded. This allows the installer to offer
58 options to install debugging symbols and binaries without increasing
59 the initial download size by separating them into their own MSIs.
60
61 Building the Installer
62 ======================
63
64 Before building the installer, download extra build dependencies using
65 Tools\msi\get_externals.bat. (Note that this is in addition to the
66 similarly named file in PCbuild.)
67
68 One of the dependencies used in builds is WiX, a toolset that lets developers
69 create installers for Windows Installer, the Windows installation engine. WiX
70 has a dependency on the Microsoft .NET Framework Version 3.5 (which may not be
71 configured on recent versions of Windows, such as Windows 10). If you are
72 building on a recent Windows version, use the Control Panel (Programs | Programs
73 and Features | Turn Windows Features on or off) and ensure that the entry
74 ".NET Framework 3.5 (includes .NET 2.0 and 3.0)" is enabled.
75
76 For Python 3.11.x and above, enable "Microsoft .NET Framework 4.8 Advanced Services"
77 instead of "Microsoft .NET Framework Version 3.5" available for Windows 10 and above.
78 Also make sure "MSVC v143 - VS 2022 C++ ARM64 build tools" are selected under
79 "Desktop Development with C++" in "Visual Studio installer" even if you are not
80 building on ARM64 along with other x64 related v143 build tools. This is because for
81 3.11.x and above we have upgraded to Wix-3.14.
82
83 For testing, the installer should be built with the Tools/msi/build.bat
84 script:
85
86 build.bat [-x86] [-x64] [-ARM64] [--doc] [--test-marker] [--pack]
87
88 This script will build the required configurations of Python and
89 generate an installer layout in PCbuild/(win32|amd64)/en-us.
90
91 Specify -x86, -x64 and/or -ARM64 to build for each platform. If none are
92 specified, both x64 and x86 will be built. Currently, both the debug and
93 release versions of Python are required for the installer.
94
95 Specify --doc to include the documentation files. Ensure %PYTHON% and
96 %SPHINXBUILD% are set when passing this option.
97
98 Specify --test-marker to build an installer that works side-by-side with
99 an official Python release. All registry keys and install locations will
100 include an extra marker to avoid overwriting files. This marker is
101 currently an 'x' prefix, but may change at any time.
102
103 Specify --pack to build an installer that does not require all MSIs to
104 be available alongside. This takes longer, but is easier to share.
105
106
107 For an official release, the installer should be built with the
108 Tools/msi/buildrelease.bat script:
109
110 set PYTHON=<path to Python 2.7 or 3.4>
111 set SPHINXBUILD=<path to sphinx-build.exe>
112 set PATH=<path to Git (git.exe)>;%PATH%
113
114 buildrelease.bat [-x86] [-x64] [-ARM64] [-D] [-B]
115 [-o <output directory>] [-c <certificate name>]
116
117 Specify -x86, -x64 and/or -ARM64 to build for each platform. If none are
118 specified, both x64 and x86 will be built. Currently, both the debug and
119 release versions of Python are required for the installer.
120
121 Specify -D to skip rebuilding the documentation. The documentation is
122 required for a release and the build will fail if it is not available.
123 Ensure %PYTHON% and %SPHINXBUILD% are set if you omit this option.
124
125 Specify -B to skip rebuilding Python. This is useful to only rebuild the
126 installer layout after a previous call to buildrelease.bat.
127
128 Specify -o to set an output directory. The installer layouts will be
129 copied to platform-specific subdirectories of this path.
130
131 Specify -c to choose a code-signing certificate to be used for all the
132 signable binaries in Python as well as each file making up the
133 installer. Official releases of Python must be signed.
134
135
136 If WiX is not found on your system, it will be automatically downloaded
137 and extracted to the externals/ directory.
138
139 To manually build layouts of the installer, build one of the projects in
140 the bundle folder.
141
142 msbuild bundle\snapshot.wixproj
143 msbuild bundle\releaseweb.wixproj
144 msbuild bundle\releaselocal.wixproj
145 msbuild bundle\full.wixproj
146
147 snapshot.wixproj produces a test installer versioned based on the date.
148
149 releaseweb.wixproj produces a release installer that does not embed any
150 of the layout.
151
152 releaselocal.wixproj produces a release installer that embeds the files
153 required for a default installation.
154
155 full.wixproj produces a test installer that embeds the entire layout.
156
157 The following properties may be passed when building these projects.
158
159 /p:BuildForRelease=(true|false)
160 When true, adds extra verification to ensure a complete installer is
161 produced. Defaults to false.
162
163 /p:ReleaseUri=(any URI)
164 Used to generate unique IDs for the installers to allow side-by-side
165 installation. Forks of Python can use the same installer infrastructure
166 by providing a unique URI for this property. It does not need to be an
167 active internet address. Defaults to $(ComputerName).
168
169 Official releases use https://www.python.org/(architecture name)
170
171 /p:DownloadUrlBase=(any URI)
172 Specifies the base of a URL where missing parts of the installer layout
173 can be downloaded from. The build version and architecture will be
174 appended to create the full address. If omitted, missing components will
175 not be automatically downloaded.
176
177 /p:DownloadUrl=(any URI)
178 Specifies the full URL where missing parts of the installer layout can
179 be downloaded from. Should normally include '{2}', which will be
180 substituted for the filename. If omitted, missing components will not be
181 automatically downloaded. If specified, this value overrides
182 DownloadUrlBase.
183
184 /p:SigningCertificate=(certificate name)
185 Specifies the certificate to sign the installer layout with. If omitted,
186 the layout will not be signed.
187
188 /p:RebuildAll=(true|false)
189 When true, rebuilds all of the MSIs making up the layout. Defaults to
190 true.
191
192 Uploading the Installer
193 =======================
194
195 For official releases, the uploadrelease.bat script should be used.
196
197 You will require PuTTY so that plink.exe and pscp.exe can be used, and your
198 SSH key can be activated in pageant.exe. PuTTY should be either on your path
199 or in %ProgramFiles(x86)%\PuTTY.
200
201 To include signatures for each uploaded file, you will need gpg2.exe on your
202 path or have run get_externals.bat. You may also need to "gpg2.exe --import"
203 your key before running the upload script.
204
205 uploadrelease.bat --host <host> --user <username> [--dry-run] [--no-gpg]
206
207 The host is the URL to the server. This can be provided by the Release
208 Manager. You should be able to SSH to this address.
209
210 The username is your own username, which you have permission to SSH into
211 the server containing downloads.
212
213 Use --dry-run to display the generated upload commands without executing
214 them. Signatures for each file will be generated but not uploaded unless
215 --no-gpg is also passed.
216
217 Use --no-gpg to suppress signature generation and upload.
218
219 The default target directory (which appears in uploadrelease.proj) is
220 correct for official Python releases, but may be overridden with
221 --target <path> for other purposes. This path should generally not include
222 any version specifier, as that will be added automatically.
223
224 Modifying the Installer
225 =======================
226
227 The code for the installer is divided into three main groups: packages,
228 the bundle and the bootstrap application.
229
230 Packages
231 --------
232
233 Packages appear as subdirectories of Tools/msi (other than the bundle/
234 directory). The project file is a .wixproj and the build output is a
235 single MSI. Packages are built with the WiX Toolset. Some project files
236 share source files and use preprocessor directives to enable particular
237 features. These are typically used to keep the sources close when the
238 files are related, but produce multiple independent packages.
239
240 A package is the smallest element that may be independently installed or
241 uninstalled (as used in this installer). For example, the test suite has
242 its own package, as users can choose to add or remove it after the
243 initial installation.
244
245 All the files installed by a single package should be related, though
246 some packages may not install any files. For example, the pip package
247 executes the ensurepip package, but does not add or remove any of its
248 own files. (It is represented as a package because of its
249 installed/uninstalled nature, as opposed to the "precompile standard
250 library" option, for example.) Dependencies between packages are handled
251 by the bundle, but packages should detect when dependencies are missing
252 and raise an error.
253
254 Packages that include a lot of files may use an InstallFiles element in
255 the .wixproj file to generate sources. See lib/lib.wixproj for an
256 example, and msi.targets and csv_to_wxs.py for the implementation. This
257 element is also responsible for generating the code for cleaning up and
258 removing __pycache__ folders in any directory containing .py files.
259
260 All packages are built with the Tools/msi/common.wxs file, and so any
261 directory or property in this file may be referenced. Of particular
262 interest:
263
264 REGISTRYKEY (property)
265 The registry key for the current installation.
266
267 InstallDirectory (directory)
268 The root install directory for the current installation. Subdirectories
269 are also specified in this file (DLLs, Lib, etc.)
270
271 MenuDir (directory)
272 The Start Menu folder for the current installation.
273
274 UpgradeTable (property)
275 Every package should reference this property to include upgrade
276 information.
277
278 OptionalFeature (Component)
279 Packages that may be enabled or disabled should reference this component
280 and have an OPTIONAL_FEATURES entry in the bootstrap application to
281 properly handle Modify and Upgrade.
282
283 The .wxl_template file is specially handled by the build system for this
284 project to perform {{substitutions}} as defined in msi.targets. They
285 should be included in projects as <WxlTemplate> items, where .wxl files
286 are normally included as <EmbeddedResource> items.
287
288 Bundle
289 ------
290
291 The bundle is compiled to the main EXE entry point that for most users
292 will represent the Python installer. It is built from Tools/msi/bundle
293 with packages references in Tools/msi/bundle/packagegroups.
294
295 Build logic for the bundle is in bundle.targets, but should be invoked
296 through one of the .wixproj files as described in Building the
297 Installer.
298
299 The UI is separated between Default.thm (UI layout), Default.wxl
300 (strings), bundle.wxs (properties) and the bootstrap application.
301 Bundle.wxs also contains the chain, which is the list of packages to
302 install and the order they should be installed in. These refer to named
303 package groups in bundle/packagegroups.
304
305 Each package group specifies one or more packages to install. Most
306 packages require two separate entries to support both per-user and
307 all-users installations. Because these reuse the same package, it does
308 not increase the overall size of the package.
309
310 Package groups refer to payload groups, which allow better control over
311 embedding and downloading files than the default settings. Whether files
312 are embedded and where they are downloaded from depends on settings
313 created by the project files.
314
315 Package references can include install conditions that determine when to
316 install the package. When a package is a dependency for others, the
317 condition should be crafted to ensure it is installed.
318
319 MSI packages are installed or uninstalled based on their current state
320 and the install condition. This makes them most suitable for features
321 that are clearly present or absent from the user's machine.
322
323 EXE packages are executed based on a customisable condition that can be
324 omitted. This makes them suitable for pre- or post-install tasks that
325 need to run regardless of whether features have been added or removed.
326
327 Bootstrap Application
328 ---------------------
329
330 The bootstrap application is a C++ application that controls the UI and
331 installation. While it does not directly compile into the main EXE of
332 the installer, it forms the main active component. Most of the
333 installation functionality is provided by WiX, and so the bootstrap
334 application is predominantly responsible for the code behind the UI that
335 is defined in the Default.thm file. The bootstrap application code is in
336 bundle/bootstrap and is built automatically when building the bundle.
337
338 Installation Layout
339 ===================
340
341 There are two installation layouts for Python on Windows, with the only
342 differences being supporting files. A layout is selected implicitly
343 based on whether the install is for all users of the machine or just for
344 the user performing the installation.
345
346 The default installation location when installing for all users is
347 "%ProgramFiles%\Python3X" for the 64-bit interpreter and
348 "%ProgramFiles(x86)%\Python3X-32" for the 32-bit interpreter. (Note that
349 the latter path is equivalent to "%ProgramFiles%\Python3X-32" when
350 running a 32-bit version of Windows.) This location requires
351 administrative privileges to install or later modify the installation.
352
353 The default installation location when installing for the current user
354 is "%LocalAppData%\Programs\Python\Python3X" for the 64-bit interpreter
355 and "%LocalAppData%\Programs\Python\Python3X-32" for the 32-bit
356 interpreter. Only the current user can access this location. This
357 provides a suitable level of protection against malicious modification
358 of Python's files.
359
360 (Default installation locations are set in Tools\msi\bundle\bundle.wxs.)
361
362 Within this install directory is the following approximate layout:
363
364 .\python[w].exe The core executable files
365 .\python3x.dll The core interpreter
366 .\python3.dll The stable ABI reference
367 .\DLLs Stdlib extensions (*.pyd) and dependencies
368 .\Doc Documentation (*.html)
369 .\include Development headers (*.h)
370 .\Lib Standard library
371 .\Lib\test Test suite
372 .\libs Development libraries (*.lib)
373 .\Scripts Launcher scripts (*.exe, *.py)
374 .\tcl Tcl dependencies (*.dll, *.tcl and others)
375 .\Tools Tool scripts (*.py)
376
377 When installed for all users, the following files are installed to
378 "%SystemRoot%" (typically "C:\Windows") to ensure they are always
379 available on PATH. (See Launching Python below.) For the current user,
380 they are installed in "%LocalAppData%\Programs\Python\PyLauncher".
381
382 .\py[w].exe PEP 397 launcher
383
384
385 System Settings
386 ===============
387
388 On installation, registry keys are created so that other applications
389 can locate and identify installations of Python. The locations of these
390 keys vary based on the install type.
391
392 For 64-bit interpreters installed for all users, the root key is:
393 HKEY_LOCAL_MACHINE\Software\Python\PythonCore\3.X
394
395 For 32-bit interpreters installed for all users on a 64-bit operating
396 system, the root key is:
397 HKEY_LOCAL_MACHINE\Software\Wow6432Node\Python\PythonCore\3.X-32
398
399 For 32-bit interpreters installed for all users on a 32-bit operating
400 system, the root key is:
401 HKEY_LOCAL_MACHINE\Software\Python\PythonCore\3.X-32
402
403 For 64-bit interpreters installed for the current user:
404 HKEY_CURRENT_USER\Software\Python\PythonCore\3.X
405
406 For 32-bit interpreters installed for the current user:
407 HKEY_CURRENT_USER\Software\Python\PythonCore\3.X-32
408
409 When the core Python executables are installed, a key "InstallPath" is
410 created within the root key with its default value set to the
411 executable's install directory. A value named "ExecutablePath" is added
412 with the full path to the main Python interpreter, and a key
413 "InstallGroup" is created with its default value set to the product
414 name "Python 3.X".
415
416 When the Python standard library is installed, a key "PythonPath" is
417 created within the root key with its default value set to the full path
418 to the Lib folder followed by the path to the DLLs folder, separated by
419 a semicolon.
420
421 When the documentation is installed, a key "Help" is created within the
422 root key, with a subkey "Main Python Documentation" with its default
423 value set to the full path to the main index.html file.
424
425
426 The py.exe launcher is installed as part of a regular Python install,
427 but using a separate mechanism that allows it to more easily span
428 versions of Python. As a result, it has different root keys for its
429 registry entries:
430
431 When installed for all users on a 64-bit operating system, the
432 launcher's root key is:
433 HKEY_LOCAL_MACHINE\Software\Wow6432Node\Python\Launcher
434
435 When installed for all users on a 32-bit operating system, the
436 launcher's root key is:
437 HKEY_LOCAL_MACHINE\Software\Python\Launcher
438
439 When installed for the current user:
440 HKEY_CURRENT_USER\Software\Python\Launcher
441
442 When the launcher is installed, a key "InstallPath" is created within
443 its root key with its default value set to the launcher's install
444 directory. File associations are also created for .py, .pyw, .pyc and
445 .pyo files.
446
447 Launching Python
448 ================
449
450 When a feature offering user entry points in the Start Menu is
451 installed, a folder "Python 3.X" is created. Every shortcut should be
452 created within this folder, and each shortcut should include the version
453 and platform to allow users to identify the shortcut in a search results
454 page.
455
456 The core Python executables creates a shortcut "Python 3.X (32-bit)" or
457 "Python 3.X (64-bit)" depending on the interpreter.
458
459 The documentation creates a shortcut "Python 3.X 32-bit Manuals" or
460 "Python 3.X 64-bit Manuals". The documentation is identical for all
461 platforms, but the shortcuts need to be separate to avoid uninstallation
462 conflicts.
463
464 Installing IDLE creates a shortcut "IDLE (Python 3.X 32-bit)" or "IDLE
465 (Python 3.X 64-bit)" depending on the interpreter.
466
467
468 For users who often launch Python from a Command Prompt, an option is
469 provided to add the directory containing python.exe to the user or
470 system PATH variable. If the option is selected, the install directory
471 and the Scripts directory will be added at the start of the system PATH
472 for an all users install and the user PATH for a per-user install.
473
474 When the user only has one version of Python installed, this will behave
475 as expected. However, because Windows searches the system PATH before
476 the user PATH, users cannot override a system-wide installation of
477 Python on their PATH. Further, because the installer can only prepend to
478 the path, later installations of Python will take precedence over
479 earlier installations, regardless of interpreter version.
480
481 Because it is not possible to automatically create a sensible PATH
482 configuration, users are recommended to use the py.exe launcher and
483 manually modify their PATH variable to add Scripts directories in their
484 preferred order. System-wide installations of Python should consider not
485 modifying PATH, or using an alternative technology to modify their
486 users' PATH variables.
487
488
489 The py.exe launcher is recommended because it uses a consistent and
490 sensible search order for Python installations. User installations are
491 preferred over system-wide installs, and later versions are preferred
492 regardless of installation order (with the exception that py.exe
493 currently prefers 2.x versions over 3.x versions without the -3 command
494 line argument).
495
496 For both 32-bit and 64-bit interpreters, the 32-bit version of the
497 launcher is installed. This ensures that the search order is always
498 consistent (as the 64-bit launcher is subtly different from the 32-bit
499 launcher) and also avoids the need to install it multiple times. Future
500 versions of Python will upgrade the launcher in-place, using Windows
501 Installer's upgrade functionality to avoid conflicts with earlier
502 installed versions.
503
504 When installed, file associations are created for .py, .pyc and .pyo
505 files to launch with py.exe and .pyw files to launch with pyw.exe. This
506 makes Python files respect shebang lines by default and also avoids
507 conflicts between multiple Python installations.
508
509
510 Repair, Modify and Uninstall
511 ============================
512
513 After installation, Python may be modified, repaired or uninstalled by
514 running the original EXE again or via the Programs and Features applet
515 (formerly known as Add or Remove Programs).
516
517 Modifications allow features to be added or removed. The install
518 directory and kind (all users/single user) cannot be modified. Because
519 Windows Installer caches installation packages, removing features will
520 not require internet access unless the package cache has been corrupted
521 or deleted. Adding features that were not previously installed and are
522 not embedded or otherwise available will require internet access.
523
524 Repairing will rerun the installation for all currently installed
525 features, restoring files and registry keys that have been modified or
526 removed. This operation generally will not redownload any files unless
527 the cached packages have been corrupted or deleted.
528
529 Removing Python will clean up all the files and registry keys that were
530 created by the installer, as well as __pycache__ folders that are
531 explicitly handled by the installer. Python packages installed later
532 using a tool like pip will not be removed. Some components may be
533 installed by other installers and these will not be removed if another
534 product has a dependency on them.
535