Debian Codegraph @

Doc/whatsnew/3.9.rst

raw
1 ****************************
2 What's New In Python 3.9
3 ****************************
4
5 :Editor: Łukasz Langa
6
7 .. Rules for maintenance:
8
9 * Anyone can add text to this document. Your text might get
10 rewritten to some degree.
11
12 * The maintainer will go through Misc/NEWS periodically and add
13 changes; it's therefore more important to add your changes to
14 Misc/NEWS than to this file.
15
16 * This is not a complete list of every single change; completeness
17 is the purpose of Misc/NEWS. Some changes will be too small
18 or esoteric to include. If such a change is added to the text,
19 it might get removed during final editing.
20
21 * If you want to draw your new text to the attention of the
22 maintainer, add 'XXX' to the beginning of the paragraph or
23 section.
24
25 * It's OK to just add a fragmentary note about a change. For
26 example: "XXX Describe the transmogrify() function added to the
27 socket module." The maintainer will research the change and
28 write the necessary text.
29
30 * You can comment out your additions if you like, but it's not
31 necessary (especially when a final release is some months away).
32
33 * Credit the author of a patch or bugfix. Just the name is
34 sufficient; the e-mail address isn't necessary.
35
36 * It's helpful to add the bug/patch number as a comment:
37
38 XXX Describe the transmogrify() function added to the socket
39 module.
40 (Contributed by P.Y. Developer in :issue:`12345`.)
41
42 This saves the maintainer the effort of going through the Mercurial log
43 when researching a change.
44
45 This article explains the new features in Python 3.9, compared to 3.8.
46 Python 3.9 was released on October 5, 2020.
47 For full details, see the :ref:`changelog <changelog>`.
48
49 .. seealso::
50
51 :pep:`596` - Python 3.9 Release Schedule
52
53
54 Summary -- Release highlights
55 =============================
56
57 .. This section singles out the most important changes in Python 3.9.
58 Brevity is key.
59
60 New syntax features:
61
62 * :pep:`584`, union operators added to ``dict``;
63 * :pep:`585`, type hinting generics in standard collections;
64 * :pep:`614`, relaxed grammar restrictions on decorators.
65
66 New built-in features:
67
68 * :pep:`616`, string methods to remove prefixes and suffixes.
69
70 New features in the standard library:
71
72 * :pep:`593`, flexible function and variable annotations;
73 * :func:`os.pidfd_open` added that allows process management without races
74 and signals.
75
76 Interpreter improvements:
77
78 * :pep:`573`, fast access to module state from methods of C extension
79 types;
80 * :pep:`617`, CPython now uses a new parser based on PEG;
81 * a number of Python builtins (range, tuple, set, frozenset, list, dict) are
82 now sped up using :pep:`590` vectorcall;
83 * garbage collection does not block on resurrected objects;
84 * a number of Python modules (:mod:`!_abc`, :mod:`!audioop`, :mod:`!_bz2`,
85 :mod:`!_codecs`, :mod:`!_contextvars`, :mod:`!_crypt`, :mod:`!_functools`,
86 :mod:`!_json`, :mod:`!_locale`, :mod:`math`, :mod:`operator`, :mod:`resource`,
87 :mod:`time`, :mod:`!_weakref`) now use multiphase initialization as defined
88 by PEP 489;
89 * a number of standard library modules (:mod:`!audioop`, :mod:`ast`, :mod:`grp`,
90 :mod:`!_hashlib`, :mod:`pwd`, :mod:`!_posixsubprocess`, :mod:`random`,
91 :mod:`select`, :mod:`struct`, :mod:`termios`, :mod:`zlib`) are now using
92 the stable ABI defined by PEP 384.
93
94 New library modules:
95
96 * :pep:`615`, the IANA Time Zone Database is now present in the standard
97 library in the :mod:`zoneinfo` module;
98 * an implementation of a topological sort of a graph is now provided in
99 the new :mod:`graphlib` module.
100
101 Release process changes:
102
103 * :pep:`602`, CPython adopts an annual release cycle.
104
105
106 You should check for DeprecationWarning in your code
107 ====================================================
108
109 When Python 2.7 was still supported, a lot of functionality in Python 3
110 was kept for backward compatibility with Python 2.7. With the end of Python
111 2 support, these backward compatibility layers have been removed, or will
112 be removed soon. Most of them emitted a :exc:`DeprecationWarning` warning for
113 several years. For example, using ``collections.Mapping`` instead of
114 ``collections.abc.Mapping`` emits a :exc:`DeprecationWarning` since Python
115 3.3, released in 2012.
116
117 Test your application with the :option:`-W` ``default`` command-line option to see
118 :exc:`DeprecationWarning` and :exc:`PendingDeprecationWarning`, or even with
119 :option:`-W` ``error`` to treat them as errors. :ref:`Warnings Filter
120 <warning-filter>` can be used to ignore warnings from third-party code.
121
122 Python 3.9 is the last version providing those Python 2 backward compatibility
123 layers, to give more time to Python projects maintainers to organize the
124 removal of the Python 2 support and add support for Python 3.9.
125
126 Aliases to :ref:`Abstract Base Classes <collections-abstract-base-classes>` in
127 the :mod:`collections` module, like ``collections.Mapping`` alias to
128 :class:`collections.abc.Mapping`, are kept for one last release for backward
129 compatibility. They will be removed from Python 3.10.
130
131 More generally, try to run your tests in the :ref:`Python Development Mode
132 <devmode>` which helps to prepare your code to make it compatible with the
133 next Python version.
134
135 Note: a number of pre-existing deprecations were removed in this version of
136 Python as well. Consult the :ref:`removed-in-python-39` section.
137
138
139 New Features
140 ============
141
142 Dictionary Merge & Update Operators
143 -----------------------------------
144
145 Merge (``|``) and update (``|=``) operators have been added to the built-in
146 :class:`dict` class. Those complement the existing ``dict.update`` and
147 ``{**d1, **d2}`` methods of merging dictionaries.
148
149 Example::
150
151 >>> x = {"key1": "value1 from x", "key2": "value2 from x"}
152 >>> y = {"key2": "value2 from y", "key3": "value3 from y"}
153 >>> x | y
154 {'key1': 'value1 from x', 'key2': 'value2 from y', 'key3': 'value3 from y'}
155 >>> y | x
156 {'key2': 'value2 from x', 'key3': 'value3 from y', 'key1': 'value1 from x'}
157
158 See :pep:`584` for a full description.
159 (Contributed by Brandt Bucher in :issue:`36144`.)
160
161 New String Methods to Remove Prefixes and Suffixes
162 --------------------------------------------------
163
164 :meth:`str.removeprefix(prefix)<str.removeprefix>` and
165 :meth:`str.removesuffix(suffix)<str.removesuffix>` have been added
166 to easily remove an unneeded prefix or a suffix from a string. Corresponding
167 ``bytes``, ``bytearray``, and ``collections.UserString`` methods have also been
168 added. See :pep:`616` for a full description. (Contributed by Dennis Sweeney in
169 :issue:`39939`.)
170
171 Type Hinting Generics in Standard Collections
172 ---------------------------------------------
173
174 In type annotations you can now use built-in collection types such as
175 ``list`` and ``dict`` as generic types instead of importing the
176 corresponding capitalized types (e.g. ``List`` or ``Dict``) from
177 ``typing``. Some other types in the standard library are also now generic,
178 for example ``queue.Queue``.
179
180 Example:
181
182 .. code-block:: python
183
184 def greet_all(names: list[str]) -> None:
185 for name in names:
186 print("Hello", name)
187
188 See :pep:`585` for more details. (Contributed by Guido van Rossum,
189 Ethan Smith, and Batuhan Taşkaya in :issue:`39481`.)
190
191 New Parser
192 ----------
193
194 Python 3.9 uses a new parser, based on `PEG
195 <https://en.wikipedia.org/wiki/Parsing_expression_grammar>`_ instead
196 of `LL(1) <https://en.wikipedia.org/wiki/LL_parser>`_. The new
197 parser's performance is roughly comparable to that of the old parser,
198 but the PEG formalism is more flexible than LL(1) when it comes to
199 designing new language features. We'll start using this flexibility
200 in Python 3.10 and later.
201
202 The :mod:`ast` module uses the new parser and produces the same AST as
203 the old parser.
204
205 In Python 3.10, the old parser will be deleted and so will all
206 functionality that depends on it (primarily the :mod:`!parser` module,
207 which has long been deprecated). In Python 3.9 *only*, you can switch
208 back to the LL(1) parser using a command line switch (``-X
209 oldparser``) or an environment variable (``PYTHONOLDPARSER=1``).
210
211 See :pep:`617` for more details. (Contributed by Guido van Rossum,
212 Pablo Galindo and Lysandros Nikolaou in :issue:`40334`.)
213
214
215 Other Language Changes
216 ======================
217
218 * :func:`__import__` now raises :exc:`ImportError` instead of
219 :exc:`ValueError`, which used to occur when a relative import went past
220 its top-level package.
221 (Contributed by Ngalim Siregar in :issue:`37444`.)
222
223 * Python now gets the absolute path of the script filename specified on
224 the command line (ex: ``python3 script.py``): the ``__file__`` attribute of
225 the :mod:`__main__` module became an absolute path, rather than a relative
226 path. These paths now remain valid after the current directory is changed
227 by :func:`os.chdir`. As a side effect, the traceback also displays the
228 absolute path for :mod:`__main__` module frames in this case.
229 (Contributed by Victor Stinner in :issue:`20443`.)
230
231 * In the :ref:`Python Development Mode <devmode>` and in :ref:`debug build <debug-build>`, the
232 *encoding* and *errors* arguments are now checked for string encoding and
233 decoding operations. Examples: :func:`open`, :meth:`str.encode` and
234 :meth:`bytes.decode`.
235
236 By default, for best performance, the *errors* argument is only checked at
237 the first encoding/decoding error and the *encoding* argument is sometimes
238 ignored for empty strings.
239 (Contributed by Victor Stinner in :issue:`37388`.)
240
241 * ``"".replace("", s, n)`` now returns ``s`` instead of an empty string for
242 all non-zero ``n``. It is now consistent with ``"".replace("", s)``.
243 There are similar changes for :class:`bytes` and :class:`bytearray` objects.
244 (Contributed by Serhiy Storchaka in :issue:`28029`.)
245
246 * Any valid expression can now be used as a :term:`decorator`. Previously, the
247 grammar was much more restrictive. See :pep:`614` for details.
248 (Contributed by Brandt Bucher in :issue:`39702`.)
249
250 * Improved help for the :mod:`typing` module. Docstrings are now shown for
251 all special forms and special generic aliases (like ``Union`` and ``List``).
252 Using :func:`help` with generic alias like ``List[int]`` will show the help
253 for the correspondent concrete type (``list`` in this case).
254 (Contributed by Serhiy Storchaka in :issue:`40257`.)
255
256 * Parallel running of :meth:`~agen.aclose` / :meth:`~agen.asend` /
257 :meth:`~agen.athrow` is now prohibited, and ``ag_running`` now reflects
258 the actual running status of the async generator.
259 (Contributed by Yury Selivanov in :issue:`30773`.)
260
261 * Unexpected errors in calling the ``__iter__`` method are no longer masked by
262 ``TypeError`` in the :keyword:`in` operator and functions
263 :func:`~operator.contains`, :func:`~operator.indexOf` and
264 :func:`~operator.countOf` of the :mod:`operator` module.
265 (Contributed by Serhiy Storchaka in :issue:`40824`.)
266
267 * Unparenthesized lambda expressions can no longer be the expression part in an
268 ``if`` clause in comprehensions and generator expressions. See :issue:`41848`
269 and :issue:`43755` for details.
270
271
272 New Modules
273 ===========
274
275 zoneinfo
276 --------
277
278 The :mod:`zoneinfo` module brings support for the IANA time zone database to
279 the standard library. It adds :class:`zoneinfo.ZoneInfo`, a concrete
280 :class:`datetime.tzinfo` implementation backed by the system's time zone data.
281
282 Example::
283
284 >>> from zoneinfo import ZoneInfo
285 >>> import datetime as dt
286
287 >>> # Daylight saving time
288 >>> when = dt.datetime(2020, 10, 31, 12, tzinfo=ZoneInfo("America/Los_Angeles"))
289 >>> print(when)
290 2020-10-31 12:00:00-07:00
291 >>> when.tzname()
292 'PDT'
293
294 >>> # Standard time
295 >>> when += dt.timedelta(days=7)
296 >>> print(when)
297 2020-11-07 12:00:00-08:00
298 >>> print(when.tzname())
299 PST
300
301
302 As a fall-back source of data for platforms that don't ship the IANA database,
303 the :pypi:`tzdata` module was released as a first-party package -- distributed via
304 PyPI and maintained by the CPython core team.
305
306 .. seealso::
307
308 :pep:`615` -- Support for the IANA Time Zone Database in the Standard Library
309 PEP written and implemented by Paul Ganssle
310
311
312 graphlib
313 ---------
314
315 A new module, :mod:`graphlib`, was added that contains the
316 :class:`graphlib.TopologicalSorter` class to offer functionality to perform
317 topological sorting of graphs. (Contributed by Pablo Galindo, Tim Peters and
318 Larry Hastings in :issue:`17005`.)
319
320
321 Improved Modules
322 ================
323
324 ast
325 ---
326
327 Added the *indent* option to :func:`~ast.dump` which allows it to produce a
328 multiline indented output.
329 (Contributed by Serhiy Storchaka in :issue:`37995`.)
330
331 Added :func:`ast.unparse` as a function in the :mod:`ast` module that can
332 be used to unparse an :class:`ast.AST` object and produce a string with code
333 that would produce an equivalent :class:`ast.AST` object when parsed.
334 (Contributed by Pablo Galindo and Batuhan Taskaya in :issue:`38870`.)
335
336 Added docstrings to AST nodes that contains the ASDL signature used to
337 construct that node. (Contributed by Batuhan Taskaya in :issue:`39638`.)
338
339 asyncio
340 -------
341
342 Due to significant security concerns, the *reuse_address* parameter of
343 :meth:`asyncio.loop.create_datagram_endpoint` is no longer supported. This is
344 because of the behavior of the socket option ``SO_REUSEADDR`` in UDP. For more
345 details, see the documentation for ``loop.create_datagram_endpoint()``.
346 (Contributed by Kyle Stanley, Antoine Pitrou, and Yury Selivanov in
347 :issue:`37228`.)
348
349 Added a new :term:`coroutine` :meth:`~asyncio.loop.shutdown_default_executor`
350 that schedules a shutdown for the default executor that waits on the
351 :class:`~concurrent.futures.ThreadPoolExecutor` to finish closing. Also,
352 :func:`asyncio.run` has been updated to use the new :term:`coroutine`.
353 (Contributed by Kyle Stanley in :issue:`34037`.)
354
355 Added :class:`!asyncio.PidfdChildWatcher`, a Linux-specific child watcher
356 implementation that polls process file descriptors. (:issue:`38692`)
357
358 Added a new :term:`coroutine` :func:`asyncio.to_thread`. It is mainly used for
359 running IO-bound functions in a separate thread to avoid blocking the event
360 loop, and essentially works as a high-level version of
361 :meth:`~asyncio.loop.run_in_executor` that can directly take keyword arguments.
362 (Contributed by Kyle Stanley and Yury Selivanov in :issue:`32309`.)
363
364 When cancelling the task due to a timeout, :meth:`asyncio.wait_for` will now
365 wait until the cancellation is complete also in the case when *timeout* is
366 <= 0, like it does with positive timeouts.
367 (Contributed by Elvis Pranskevichus in :issue:`32751`.)
368
369 :mod:`asyncio` now raises :exc:`TypeError` when calling incompatible
370 methods with an :class:`ssl.SSLSocket` socket.
371 (Contributed by Ido Michael in :issue:`37404`.)
372
373 compileall
374 ----------
375
376 Added new possibility to use hardlinks for duplicated ``.pyc`` files: *hardlink_dupes* parameter and --hardlink-dupes command line option.
377 (Contributed by Lumír 'Frenzy' Balhar in :issue:`40495`.)
378
379 Added new options for path manipulation in resulting ``.pyc`` files: *stripdir*, *prependdir*, *limit_sl_dest* parameters and -s, -p, -e command line options.
380 Added the possibility to specify the option for an optimization level multiple times.
381 (Contributed by Lumír 'Frenzy' Balhar in :issue:`38112`.)
382
383 concurrent.futures
384 ------------------
385
386 Added a new *cancel_futures* parameter to
387 :meth:`concurrent.futures.Executor.shutdown` that cancels all pending futures
388 which have not started running, instead of waiting for them to complete before
389 shutting down the executor.
390 (Contributed by Kyle Stanley in :issue:`39349`.)
391
392 Removed daemon threads from :class:`~concurrent.futures.ThreadPoolExecutor`
393 and :class:`~concurrent.futures.ProcessPoolExecutor`. This improves
394 compatibility with subinterpreters and predictability in their shutdown
395 processes. (Contributed by Kyle Stanley in :issue:`39812`.)
396
397 Workers in :class:`~concurrent.futures.ProcessPoolExecutor` are now spawned on
398 demand, only when there are no available idle workers to reuse. This optimizes
399 startup overhead and reduces the amount of lost CPU time to idle workers.
400 (Contributed by Kyle Stanley in :issue:`39207`.)
401
402 curses
403 ------
404
405 Added :func:`curses.get_escdelay`, :func:`curses.set_escdelay`,
406 :func:`curses.get_tabsize`, and :func:`curses.set_tabsize` functions.
407 (Contributed by Anthony Sottile in :issue:`38312`.)
408
409 datetime
410 --------
411 The :meth:`~datetime.date.isocalendar` of :class:`datetime.date`
412 and :meth:`~datetime.datetime.isocalendar` of :class:`datetime.datetime`
413 methods now returns a :func:`~collections.namedtuple` instead of a :class:`tuple`.
414 (Contributed by Donghee Na in :issue:`24416`.)
415
416 distutils
417 ---------
418
419 The :command:`upload` command now creates SHA2-256 and Blake2b-256 hash
420 digests. It skips MD5 on platforms that block MD5 digest.
421 (Contributed by Christian Heimes in :issue:`40698`.)
422
423 fcntl
424 -----
425
426 Added constants :const:`!fcntl.F_OFD_GETLK`, :const:`!fcntl.F_OFD_SETLK`
427 and :const:`!fcntl.F_OFD_SETLKW`.
428 (Contributed by Donghee Na in :issue:`38602`.)
429
430 ftplib
431 -------
432
433 :class:`~ftplib.FTP` and :class:`~ftplib.FTP_TLS` now raise a :class:`ValueError`
434 if the given timeout for their constructor is zero to prevent the creation of
435 a non-blocking socket. (Contributed by Donghee Na in :issue:`39259`.)
436
437 gc
438 --
439
440 When the garbage collector makes a collection in which some objects resurrect
441 (they are reachable from outside the isolated cycles after the finalizers have
442 been executed), do not block the collection of all objects that are still
443 unreachable. (Contributed by Pablo Galindo and Tim Peters in :issue:`38379`.)
444
445 Added a new function :func:`gc.is_finalized` to check if an object has been
446 finalized by the garbage collector. (Contributed by Pablo Galindo in
447 :issue:`39322`.)
448
449 hashlib
450 -------
451
452 The :mod:`hashlib` module can now use SHA3 hashes and SHAKE XOF from OpenSSL
453 when available.
454 (Contributed by Christian Heimes in :issue:`37630`.)
455
456 Builtin hash modules can now be disabled with
457 ``./configure --without-builtin-hashlib-hashes`` or selectively enabled with
458 e.g. ``./configure --with-builtin-hashlib-hashes=sha3,blake2`` to force use
459 of OpenSSL based implementation.
460 (Contributed by Christian Heimes in :issue:`40479`)
461
462
463 http
464 ----
465
466 HTTP status codes ``103 EARLY_HINTS``, ``418 IM_A_TEAPOT`` and ``425 TOO_EARLY`` are added to
467 :class:`http.HTTPStatus`. (Contributed by Donghee Na in :issue:`39509` and Ross Rhodes in :issue:`39507`.)
468
469 IDLE and idlelib
470 ----------------
471
472 Added option to toggle cursor blink off. (Contributed by Zackery Spytz
473 in :issue:`4603`.)
474
475 Escape key now closes IDLE completion windows. (Contributed by Johnny
476 Najera in :issue:`38944`.)
477
478 Added keywords to module name completion list. (Contributed by Terry J.
479 Reedy in :issue:`37765`.)
480
481 New in 3.9 maintenance releases
482
483 Make IDLE invoke :func:`sys.excepthook` (when started without '-n').
484 User hooks were previously ignored. (Contributed by Ken Hilton in
485 :issue:`43008`.)
486
487 The changes above have been backported to 3.8 maintenance releases.
488
489 Rearrange the settings dialog. Split the General tab into Windows
490 and Shell/Ed tabs. Move help sources, which extend the Help menu, to the
491 Extensions tab. Make space for new options and shorten the dialog. The
492 latter makes the dialog better fit small screens. (Contributed by Terry Jan
493 Reedy in :issue:`40468`.) Move the indent space setting from the Font tab to
494 the new Windows tab. (Contributed by Mark Roseman and Terry Jan Reedy in
495 :issue:`33962`.)
496
497 Apply syntax highlighting to ``.pyi`` files. (Contributed by Alex
498 Waygood and Terry Jan Reedy in :issue:`45447`.)
499
500 imaplib
501 -------
502
503 :class:`~imaplib.IMAP4` and :class:`~imaplib.IMAP4_SSL` now have
504 an optional *timeout* parameter for their constructors.
505 Also, the :meth:`~imaplib.IMAP4.open` method now has an optional *timeout* parameter
506 with this change. The overridden methods of :class:`~imaplib.IMAP4_SSL` and
507 :class:`~imaplib.IMAP4_stream` were applied to this change.
508 (Contributed by Donghee Na in :issue:`38615`.)
509
510 :meth:`imaplib.IMAP4.unselect` is added.
511 :meth:`imaplib.IMAP4.unselect` frees server's resources associated with the
512 selected mailbox and returns the server to the authenticated
513 state. This command performs the same actions as :meth:`imaplib.IMAP4.close`, except
514 that no messages are permanently removed from the currently
515 selected mailbox. (Contributed by Donghee Na in :issue:`40375`.)
516
517 importlib
518 ---------
519
520 To improve consistency with import statements, :func:`importlib.util.resolve_name`
521 now raises :exc:`ImportError` instead of :exc:`ValueError` for invalid relative
522 import attempts.
523 (Contributed by Ngalim Siregar in :issue:`37444`.)
524
525 Import loaders which publish immutable module objects can now publish
526 immutable packages in addition to individual modules.
527 (Contributed by Dino Viehland in :issue:`39336`.)
528
529 Added :func:`importlib.resources.files` function with support for
530 subdirectories in package data, matching backport in ``importlib_resources``
531 version 1.5.
532 (Contributed by Jason R. Coombs in :issue:`39791`.)
533
534 Refreshed ``importlib.metadata`` from ``importlib_metadata`` version 1.6.1.
535
536 inspect
537 -------
538
539 :attr:`inspect.BoundArguments.arguments` is changed from ``OrderedDict`` to regular
540 dict. (Contributed by Inada Naoki in :issue:`36350` and :issue:`39775`.)
541
542 ipaddress
543 ---------
544
545 :mod:`ipaddress` now supports IPv6 Scoped Addresses (IPv6 address with suffix ``%<scope_id>``).
546
547 Scoped IPv6 addresses can be parsed using :class:`ipaddress.IPv6Address`.
548 If present, scope zone ID is available through the :attr:`~ipaddress.IPv6Address.scope_id` attribute.
549 (Contributed by Oleksandr Pavliuk in :issue:`34788`.)
550
551 Starting with Python 3.9.5 the :mod:`ipaddress` module no longer
552 accepts any leading zeros in IPv4 address strings.
553 (Contributed by Christian Heimes in :issue:`36384`).
554
555 math
556 ----
557
558 Expanded the :func:`math.gcd` function to handle multiple arguments.
559 Formerly, it only supported two arguments.
560 (Contributed by Serhiy Storchaka in :issue:`39648`.)
561
562 Added :func:`math.lcm`: return the least common multiple of specified arguments.
563 (Contributed by Mark Dickinson, Ananthakrishnan and Serhiy Storchaka in
564 :issue:`39479` and :issue:`39648`.)
565
566 Added :func:`math.nextafter`: return the next floating-point value after *x*
567 towards *y*.
568 (Contributed by Victor Stinner in :issue:`39288`.)
569
570 Added :func:`math.ulp`: return the value of the least significant bit
571 of a float.
572 (Contributed by Victor Stinner in :issue:`39310`.)
573
574 multiprocessing
575 ---------------
576
577 The :class:`multiprocessing.SimpleQueue` class has a new
578 :meth:`~multiprocessing.SimpleQueue.close` method to explicitly close the
579 queue.
580 (Contributed by Victor Stinner in :issue:`30966`.)
581
582 nntplib
583 -------
584
585 :class:`!NNTP` and :class:`!NNTP_SSL` now raise a :class:`ValueError`
586 if the given timeout for their constructor is zero to prevent the creation of
587 a non-blocking socket. (Contributed by Donghee Na in :issue:`39259`.)
588
589 os
590 --
591
592 Added :const:`~os.CLD_KILLED` and :const:`~os.CLD_STOPPED` for :attr:`!si_code`.
593 (Contributed by Donghee Na in :issue:`38493`.)
594
595 Exposed the Linux-specific :func:`os.pidfd_open` (:issue:`38692`) and
596 :const:`os.P_PIDFD` (:issue:`38713`) for process management with file
597 descriptors.
598
599 The :func:`os.unsetenv` function is now also available on Windows.
600 (Contributed by Victor Stinner in :issue:`39413`.)
601
602 The :func:`os.putenv` and :func:`os.unsetenv` functions are now always
603 available.
604 (Contributed by Victor Stinner in :issue:`39395`.)
605
606 Added :func:`os.waitstatus_to_exitcode` function:
607 convert a wait status to an exit code.
608 (Contributed by Victor Stinner in :issue:`40094`.)
609
610 pathlib
611 -------
612
613 Added :meth:`pathlib.Path.readlink` which acts similarly to
614 :func:`os.readlink`.
615 (Contributed by Girts Folkmanis in :issue:`30618`)
616
617 pdb
618 ---
619
620 On Windows now :class:`~pdb.Pdb` supports ``~/.pdbrc``.
621 (Contributed by Tim Hopper and Dan Lidral-Porter in :issue:`20523`.)
622
623 poplib
624 ------
625
626 :class:`~poplib.POP3` and :class:`~poplib.POP3_SSL` now raise a :class:`ValueError`
627 if the given timeout for their constructor is zero to prevent the creation of
628 a non-blocking socket. (Contributed by Donghee Na in :issue:`39259`.)
629
630 pprint
631 ------
632
633 :mod:`pprint` can now pretty-print :class:`types.SimpleNamespace`.
634 (Contributed by Carl Bordum Hansen in :issue:`37376`.)
635
636 pydoc
637 -----
638
639 The documentation string is now shown not only for class, function,
640 method etc, but for any object that has its own :attr:`~definition.__doc__`
641 attribute.
642 (Contributed by Serhiy Storchaka in :issue:`40257`.)
643
644 random
645 ------
646
647 Added a new :meth:`random.Random.randbytes` method: generate random bytes.
648 (Contributed by Victor Stinner in :issue:`40286`.)
649
650 signal
651 ------
652
653 Exposed the Linux-specific :func:`signal.pidfd_send_signal` for sending to
654 signals to a process using a file descriptor instead of a pid. (:issue:`38712`)
655
656 smtplib
657 -------
658
659 :class:`~smtplib.SMTP` and :class:`~smtplib.SMTP_SSL` now raise a :class:`ValueError`
660 if the given timeout for their constructor is zero to prevent the creation of
661 a non-blocking socket. (Contributed by Donghee Na in :issue:`39259`.)
662
663 :class:`~smtplib.LMTP` constructor now has an optional *timeout* parameter.
664 (Contributed by Donghee Na in :issue:`39329`.)
665
666 socket
667 ------
668
669 The :mod:`socket` module now exports the :const:`~socket.CAN_RAW_JOIN_FILTERS`
670 constant on Linux 4.1 and greater.
671 (Contributed by Stefan Tatschner and Zackery Spytz in :issue:`25780`.)
672
673 The socket module now supports the :const:`~socket.CAN_J1939` protocol on
674 platforms that support it. (Contributed by Karl Ding in :issue:`40291`.)
675
676 The socket module now has the :func:`socket.send_fds` and
677 :func:`socket.recv_fds` functions. (Contributed by Joannah Nanjekye, Shinya
678 Okano and Victor Stinner in :issue:`28724`.)
679
680
681 time
682 ----
683
684 On AIX, :func:`~time.thread_time` is now implemented with ``thread_cputime()``
685 which has nanosecond resolution, rather than
686 ``clock_gettime(CLOCK_THREAD_CPUTIME_ID)`` which has a resolution of 10 milliseconds.
687 (Contributed by Batuhan Taskaya in :issue:`40192`)
688
689 sys
690 ---
691
692 Added a new :data:`sys.platlibdir` attribute: name of the platform-specific
693 library directory. It is used to build the path of standard library and the
694 paths of installed extension modules. It is equal to ``"lib"`` on most
695 platforms. On Fedora and SuSE, it is equal to ``"lib64"`` on 64-bit platforms.
696 (Contributed by Jan Matějek, Matěj Cepl, Charalampos Stratakis and Victor Stinner in :issue:`1294959`.)
697
698 Previously, :data:`sys.stderr` was block-buffered when non-interactive. Now
699 ``stderr`` defaults to always being line-buffered.
700 (Contributed by Jendrik Seipp in :issue:`13601`.)
701
702 tracemalloc
703 -----------
704
705 Added :func:`tracemalloc.reset_peak` to set the peak size of traced memory
706 blocks to the current size, to measure the peak of specific pieces of code.
707 (Contributed by Huon Wilson in :issue:`40630`.)
708
709 typing
710 ------
711
712 :pep:`593` introduced an :data:`typing.Annotated` type to decorate existing
713 types with context-specific metadata and new ``include_extras`` parameter to
714 :func:`typing.get_type_hints` to access the metadata at runtime. (Contributed
715 by Till Varoquaux and Konstantin Kashin.)
716
717 unicodedata
718 -----------
719
720 The Unicode database has been updated to version 13.0.0. (:issue:`39926`).
721
722 venv
723 ----
724
725 The activation scripts provided by :mod:`venv` now all specify their prompt
726 customization consistently by always using the value specified by
727 ``__VENV_PROMPT__``. Previously some scripts unconditionally used
728 ``__VENV_PROMPT__``, others only if it happened to be set (which was the default
729 case), and one used ``__VENV_NAME__`` instead.
730 (Contributed by Brett Cannon in :issue:`37663`.)
731
732 xml
733 ---
734
735 White space characters within attributes are now preserved when serializing
736 :mod:`xml.etree.ElementTree` to XML file. EOLNs are no longer normalized
737 to "\n". This is the result of discussion about how to interpret
738 section 2.11 of XML spec.
739 (Contributed by Mefistotelis in :issue:`39011`.)
740
741
742 Optimizations
743 =============
744
745 * Optimized the idiom for assignment a temporary variable in comprehensions.
746 Now ``for y in [expr]`` in comprehensions is as fast as a simple assignment
747 ``y = expr``. For example:
748
749 sums = [s for s in [0] for x in data for s in [s + x]]
750
751 Unlike the ``:=`` operator this idiom does not leak a variable to the
752 outer scope.
753
754 (Contributed by Serhiy Storchaka in :issue:`32856`.)
755
756 * Optimized signal handling in multithreaded applications. If a thread different
757 than the main thread gets a signal, the bytecode evaluation loop is no longer
758 interrupted at each bytecode instruction to check for pending signals which
759 cannot be handled. Only the main thread of the main interpreter can handle
760 signals.
761
762 Previously, the bytecode evaluation loop was interrupted at each instruction
763 until the main thread handles signals.
764 (Contributed by Victor Stinner in :issue:`40010`.)
765
766 * Optimized the :mod:`subprocess` module on FreeBSD using ``closefrom()``.
767 (Contributed by Ed Maste, Conrad Meyer, Kyle Evans, Kubilay Kocak and Victor
768 Stinner in :issue:`38061`.)
769
770 * :c:func:`PyLong_FromDouble` is now up to 1.87x faster for values that
771 fit into :c:expr:`long`.
772 (Contributed by Sergey Fedoseev in :issue:`37986`.)
773
774 * A number of Python builtins (:class:`range`, :class:`tuple`, :class:`set`,
775 :class:`frozenset`, :class:`list`, :class:`dict`) are now sped up by using
776 :pep:`590` vectorcall protocol.
777 (Contributed by Donghee Na, Mark Shannon, Jeroen Demeyer and Petr Viktorin in :issue:`37207`.)
778
779 * Optimized :meth:`!set.difference_update` for the case when the other set
780 is much larger than the base set.
781 (Suggested by Evgeny Kapun with code contributed by Michele Orrù in :issue:`8425`.)
782
783 * Python's small object allocator (``obmalloc.c``) now allows (no more than)
784 one empty arena to remain available for immediate reuse, without returning
785 it to the OS. This prevents thrashing in simple loops where an arena could
786 be created and destroyed anew on each iteration.
787 (Contributed by Tim Peters in :issue:`37257`.)
788
789 * :term:`floor division` of float operation now has a better performance. Also
790 the message of :exc:`ZeroDivisionError` for this operation is updated.
791 (Contributed by Donghee Na in :issue:`39434`.)
792
793 * Decoding short ASCII strings with UTF-8 and ascii codecs is now about
794 15% faster. (Contributed by Inada Naoki in :issue:`37348`.)
795
796 Here's a summary of performance improvements from Python 3.4 through Python 3.9:
797
798 .. code-block:: none
799
800 Python version 3.4 3.5 3.6 3.7 3.8 3.9
801 -------------- --- --- --- --- --- ---
802
803 Variable and attribute read access:
804 read_local 7.1 7.1 5.4 5.1 3.9 3.9
805 read_nonlocal 7.1 8.1 5.8 5.4 4.4 4.5
806 read_global 15.5 19.0 14.3 13.6 7.6 7.8
807 read_builtin 21.1 21.6 18.5 19.0 7.5 7.8
808 read_classvar_from_class 25.6 26.5 20.7 19.5 18.4 17.9
809 read_classvar_from_instance 22.8 23.5 18.8 17.1 16.4 16.9
810 read_instancevar 32.4 33.1 28.0 26.3 25.4 25.3
811 read_instancevar_slots 27.8 31.3 20.8 20.8 20.2 20.5
812 read_namedtuple 73.8 57.5 45.0 46.8 18.4 18.7
813 read_boundmethod 37.6 37.9 29.6 26.9 27.7 41.1
814
815 Variable and attribute write access:
816 write_local 8.7 9.3 5.5 5.3 4.3 4.3
817 write_nonlocal 10.5 11.1 5.6 5.5 4.7 4.8
818 write_global 19.7 21.2 18.0 18.0 15.8 16.7
819 write_classvar 92.9 96.0 104.6 102.1 39.2 39.8
820 write_instancevar 44.6 45.8 40.0 38.9 35.5 37.4
821 write_instancevar_slots 35.6 36.1 27.3 26.6 25.7 25.8
822
823 Data structure read access:
824 read_list 24.2 24.5 20.8 20.8 19.0 19.5
825 read_deque 24.7 25.5 20.2 20.6 19.8 20.2
826 read_dict 24.3 25.7 22.3 23.0 21.0 22.4
827 read_strdict 22.6 24.3 19.5 21.2 18.9 21.5
828
829 Data structure write access:
830 write_list 27.1 28.5 22.5 21.6 20.0 20.0
831 write_deque 28.7 30.1 22.7 21.8 23.5 21.7
832 write_dict 31.4 33.3 29.3 29.2 24.7 25.4
833 write_strdict 28.4 29.9 27.5 25.2 23.1 24.5
834
835 Stack (or queue) operations:
836 list_append_pop 93.4 112.7 75.4 74.2 50.8 50.6
837 deque_append_pop 43.5 57.0 49.4 49.2 42.5 44.2
838 deque_append_popleft 43.7 57.3 49.7 49.7 42.8 46.4
839
840 Timing loop:
841 loop_overhead 0.5 0.6 0.4 0.3 0.3 0.3
842
843 These results were generated from the variable access benchmark script at:
844 ``Tools/scripts/var_access_benchmark.py``. The benchmark script displays timings
845 in nanoseconds. The benchmarks were measured on an
846 `Intel® Core™ i7-4960HQ processor
847 <https://ark.intel.com/content/www/us/en/ark/products/76088/intel-core-i7-4960hq-processor-6m-cache-up-to-3-80-ghz.html>`_
848 running the macOS 64-bit builds found at
849 `python.org <https://www.python.org/downloads/macos/>`_.
850
851
852 Deprecated
853 ==========
854
855 * The distutils ``bdist_msi`` command is now deprecated, use
856 ``bdist_wheel`` (wheel packages) instead.
857 (Contributed by Hugo van Kemenade in :issue:`39586`.)
858
859 * Currently :func:`math.factorial` accepts :class:`float` instances with
860 non-negative integer values (like ``5.0``). It raises a :exc:`ValueError`
861 for non-integral and negative floats. It is now deprecated. In future
862 Python versions it will raise a :exc:`TypeError` for all floats.
863 (Contributed by Serhiy Storchaka in :issue:`37315`.)
864
865 * The :mod:`!parser` and :mod:`!symbol` modules are deprecated and will be
866 removed in future versions of Python. For the majority of use cases,
867 users can leverage the Abstract Syntax Tree (AST) generation and compilation
868 stage, using the :mod:`ast` module.
869
870 * The Public C API functions :c:func:`!PyParser_SimpleParseStringFlags`,
871 :c:func:`!PyParser_SimpleParseStringFlagsFilename`,
872 :c:func:`!PyParser_SimpleParseFileFlags` and :c:func:`!PyNode_Compile`
873 are deprecated and will be removed in Python 3.10 together with the old parser.
874
875 * Using :data:`NotImplemented` in a boolean context has been deprecated,
876 as it is almost exclusively the result of incorrect rich comparator
877 implementations. It will be made a :exc:`TypeError` in a future version
878 of Python.
879 (Contributed by Josh Rosenberg in :issue:`35712`.)
880
881 * The :mod:`random` module currently accepts any hashable type as a
882 possible seed value. Unfortunately, some of those types are not
883 guaranteed to have a deterministic hash value. After Python 3.9,
884 the module will restrict its seeds to :const:`None`, :class:`int`,
885 :class:`float`, :class:`str`, :class:`bytes`, and :class:`bytearray`.
886
887 * Opening the :class:`~gzip.GzipFile` file for writing without specifying
888 the *mode* argument is deprecated. In future Python versions it will always
889 be opened for reading by default. Specify the *mode* argument for opening
890 it for writing and silencing a warning.
891 (Contributed by Serhiy Storchaka in :issue:`28286`.)
892
893 * Deprecated the ``split()`` method of :class:`!_tkinter.TkappType` in
894 favour of the ``splitlist()`` method which has more consistent and
895 predictable behavior.
896 (Contributed by Serhiy Storchaka in :issue:`38371`.)
897
898 * The explicit passing of coroutine objects to :func:`asyncio.wait` has been
899 deprecated and will be removed in version 3.11.
900 (Contributed by Yury Selivanov and Kyle Stanley in :issue:`34790`.)
901
902 * binhex4 and hexbin4 standards are now deprecated. The :mod:`!binhex` module
903 and the following :mod:`binascii` functions are now deprecated:
904
905 * :func:`!b2a_hqx`, :func:`!a2b_hqx`
906 * :func:`!rlecode_hqx`, :func:`!rledecode_hqx`
907
908 (Contributed by Victor Stinner in :issue:`39353`.)
909
910 * :mod:`ast` classes ``slice``, ``Index`` and ``ExtSlice`` are considered deprecated
911 and will be removed in future Python versions. ``value`` itself should be
912 used instead of ``Index(value)``. ``Tuple(slices, Load())`` should be
913 used instead of ``ExtSlice(slices)``.
914 (Contributed by Serhiy Storchaka in :issue:`34822`.)
915
916 * :mod:`ast` classes ``Suite``, ``Param``, ``AugLoad`` and ``AugStore``
917 are considered deprecated and will be removed in future Python versions.
918 They were not generated by the parser and not accepted by the code
919 generator in Python 3.
920 (Contributed by Batuhan Taskaya in :issue:`39639` and :issue:`39969`
921 and Serhiy Storchaka in :issue:`39988`.)
922
923 * The :c:func:`!PyEval_InitThreads` and :c:func:`!PyEval_ThreadsInitialized`
924 functions are now deprecated and will be removed in Python 3.11. Calling
925 :c:func:`!PyEval_InitThreads` now does nothing. The :term:`GIL` is initialized
926 by :c:func:`Py_Initialize` since Python 3.7.
927 (Contributed by Victor Stinner in :issue:`39877`.)
928
929 * Passing ``None`` as the first argument to the :func:`shlex.split` function
930 has been deprecated. (Contributed by Zackery Spytz in :issue:`33262`.)
931
932 * :func:`!smtpd.MailmanProxy` is now deprecated as it is unusable without
933 an external module, ``mailman``. (Contributed by Samuel Colvin in :issue:`35800`.)
934
935 * The :mod:`!lib2to3` module now emits a :exc:`PendingDeprecationWarning`.
936 Python 3.9 switched to a PEG parser (see :pep:`617`), and Python 3.10 may
937 include new language syntax that is not parsable by lib2to3's LL(1) parser.
938 The :mod:`!lib2to3` module may be removed from the standard library in a future
939 Python version. Consider third-party alternatives such as `LibCST`_ or
940 `parso`_.
941 (Contributed by Carl Meyer in :issue:`40360`.)
942
943 * The *random* parameter of :func:`random.shuffle` has been deprecated.
944 (Contributed by Raymond Hettinger in :issue:`40465`)
945
946 .. _LibCST: https://libcst.readthedocs.io/
947 .. _parso: https://parso.readthedocs.io/
948
949 .. _removed-in-python-39:
950
951 Removed
952 =======
953
954 * The erroneous version at :data:`!unittest.mock.__version__` has been removed.
955
956 * :class:`!nntplib.NNTP`: ``xpath()`` and ``xgtitle()`` methods have been removed.
957 These methods are deprecated since Python 3.3. Generally, these extensions
958 are not supported or not enabled by NNTP server administrators.
959 For ``xgtitle()``, please use :meth:`!nntplib.NNTP.descriptions` or
960 :meth:`!nntplib.NNTP.description` instead.
961 (Contributed by Donghee Na in :issue:`39366`.)
962
963 * :class:`array.array`: ``tostring()`` and ``fromstring()`` methods have been
964 removed. They were aliases to ``tobytes()`` and ``frombytes()``, deprecated
965 since Python 3.2.
966 (Contributed by Victor Stinner in :issue:`38916`.)
967
968 * The undocumented ``sys.callstats()`` function has been removed. Since Python
969 3.7, it was deprecated and always returned :const:`None`. It required a special
970 build option ``CALL_PROFILE`` which was already removed in Python 3.7.
971 (Contributed by Victor Stinner in :issue:`37414`.)
972
973 * The ``sys.getcheckinterval()`` and ``sys.setcheckinterval()`` functions have
974 been removed. They were deprecated since Python 3.2. Use
975 :func:`sys.getswitchinterval` and :func:`sys.setswitchinterval` instead.
976 (Contributed by Victor Stinner in :issue:`37392`.)
977
978 * The C function ``PyImport_Cleanup()`` has been removed. It was documented as:
979 "Empty the module table. For internal use only."
980 (Contributed by Victor Stinner in :issue:`36710`.)
981
982 * ``_dummy_thread`` and ``dummy_threading`` modules have been removed. These
983 modules were deprecated since Python 3.7 which requires threading support.
984 (Contributed by Victor Stinner in :issue:`37312`.)
985
986 * ``aifc.openfp()`` alias to ``aifc.open()``, ``sunau.openfp()`` alias to
987 ``sunau.open()``, and ``wave.openfp()`` alias to :func:`wave.open` have been
988 removed. They were deprecated since Python 3.7.
989 (Contributed by Victor Stinner in :issue:`37320`.)
990
991 * The :meth:`!isAlive` method of :class:`threading.Thread`
992 has been removed. It was deprecated since Python 3.8.
993 Use :meth:`~threading.Thread.is_alive` instead.
994 (Contributed by Donghee Na in :issue:`37804`.)
995
996 * Methods ``getchildren()`` and ``getiterator()`` of classes
997 :class:`~xml.etree.ElementTree.ElementTree` and
998 :class:`~xml.etree.ElementTree.Element` in the :mod:`~xml.etree.ElementTree`
999 module have been removed. They were deprecated in Python 3.2.
1000 Use ``iter(x)`` or ``list(x)`` instead of ``x.getchildren()`` and
1001 ``x.iter()`` or ``list(x.iter())`` instead of ``x.getiterator()``.
1002 (Contributed by Serhiy Storchaka in :issue:`36543`.)
1003
1004 * The old :mod:`plistlib` API has been removed, it was deprecated since Python
1005 3.4. Use the :func:`~plistlib.load`, :func:`~plistlib.loads`, :func:`~plistlib.dump`, and
1006 :func:`~plistlib.dumps` functions. Additionally, the *use_builtin_types* parameter was
1007 removed, standard :class:`bytes` objects are always used instead.
1008 (Contributed by Jon Janzen in :issue:`36409`.)
1009
1010 * The C function ``PyGen_NeedsFinalizing`` has been removed. It was not
1011 documented, tested, or used anywhere within CPython after the implementation
1012 of :pep:`442`. Patch by Joannah Nanjekye.
1013 (Contributed by Joannah Nanjekye in :issue:`15088`)
1014
1015 * ``base64.encodestring()`` and ``base64.decodestring()``, aliases deprecated
1016 since Python 3.1, have been removed: use :func:`base64.encodebytes` and
1017 :func:`base64.decodebytes` instead.
1018 (Contributed by Victor Stinner in :issue:`39351`.)
1019
1020 * ``fractions.gcd()`` function has been removed, it was deprecated since Python
1021 3.5 (:issue:`22486`): use :func:`math.gcd` instead.
1022 (Contributed by Victor Stinner in :issue:`39350`.)
1023
1024 * The *buffering* parameter of :class:`bz2.BZ2File` has been removed. Since
1025 Python 3.0, it was ignored and using it emitted a :exc:`DeprecationWarning`.
1026 Pass an open file object to control how the file is opened.
1027 (Contributed by Victor Stinner in :issue:`39357`.)
1028
1029 * The *encoding* parameter of :func:`json.loads` has been removed.
1030 As of Python 3.1, it was deprecated and ignored; using it has emitted a
1031 :exc:`DeprecationWarning` since Python 3.8.
1032 (Contributed by Inada Naoki in :issue:`39377`)
1033
1034 * ``with (await asyncio.lock):`` and ``with (yield from asyncio.lock):`` statements are
1035 not longer supported, use ``async with lock`` instead. The same is correct for
1036 ``asyncio.Condition`` and ``asyncio.Semaphore``.
1037 (Contributed by Andrew Svetlov in :issue:`34793`.)
1038
1039 * The :func:`!sys.getcounts` function, the ``-X showalloccount`` command line
1040 option and the ``show_alloc_count`` field of the C structure
1041 :c:type:`PyConfig` have been removed. They required a special Python build by
1042 defining ``COUNT_ALLOCS`` macro.
1043 (Contributed by Victor Stinner in :issue:`39489`.)
1044
1045 * The ``_field_types`` attribute of the :class:`typing.NamedTuple` class
1046 has been removed. It was deprecated since Python 3.8. Use
1047 the ``__annotations__`` attribute instead.
1048 (Contributed by Serhiy Storchaka in :issue:`40182`.)
1049
1050 * The :meth:`!symtable.SymbolTable.has_exec` method has been removed. It was
1051 deprecated since 2006, and only returning ``False`` when it's called.
1052 (Contributed by Batuhan Taskaya in :issue:`40208`)
1053
1054 * The :meth:`!asyncio.Task.current_task` and :meth:`!asyncio.Task.all_tasks`
1055 have been removed. They were deprecated since Python 3.7 and you can use
1056 :func:`asyncio.current_task` and :func:`asyncio.all_tasks` instead.
1057 (Contributed by Rémi Lapeyre in :issue:`40967`)
1058
1059 * The ``unescape()`` method in the :class:`html.parser.HTMLParser` class
1060 has been removed (it was deprecated since Python 3.4). :func:`html.unescape`
1061 should be used for converting character references to the corresponding
1062 unicode characters.
1063
1064
1065 Porting to Python 3.9
1066 =====================
1067
1068 This section lists previously described changes and other bugfixes
1069 that may require changes to your code.
1070
1071
1072 Changes in the Python API
1073 -------------------------
1074
1075 * :func:`__import__` and :func:`importlib.util.resolve_name` now raise
1076 :exc:`ImportError` where it previously raised :exc:`ValueError`. Callers
1077 catching the specific exception type and supporting both Python 3.9 and
1078 earlier versions will need to catch both using ``except (ImportError, ValueError):``.
1079
1080 * The :mod:`venv` activation scripts no longer special-case when
1081 ``__VENV_PROMPT__`` is set to ``""``.
1082
1083 * The :meth:`select.epoll.unregister` method no longer ignores the
1084 :const:`~errno.EBADF` error.
1085 (Contributed by Victor Stinner in :issue:`39239`.)
1086
1087 * The *compresslevel* parameter of :class:`bz2.BZ2File` became keyword-only,
1088 since the *buffering* parameter has been removed.
1089 (Contributed by Victor Stinner in :issue:`39357`.)
1090
1091 * Simplified AST for subscription. Simple indices will be represented by
1092 their value, extended slices will be represented as tuples.
1093 ``Index(value)`` will return a ``value`` itself, ``ExtSlice(slices)``
1094 will return ``Tuple(slices, Load())``.
1095 (Contributed by Serhiy Storchaka in :issue:`34822`.)
1096
1097 * The :mod:`importlib` module now ignores the :envvar:`PYTHONCASEOK`
1098 environment variable when the :option:`-E` or :option:`-I` command line
1099 options are being used.
1100
1101 * The *encoding* parameter has been added to the classes :class:`ftplib.FTP` and
1102 :class:`ftplib.FTP_TLS` as a keyword-only parameter, and the default encoding
1103 is changed from Latin-1 to UTF-8 to follow :rfc:`2640`.
1104
1105 * :meth:`asyncio.loop.shutdown_default_executor` has been added to
1106 :class:`~asyncio.AbstractEventLoop`, meaning alternative event loops that
1107 inherit from it should have this method defined.
1108 (Contributed by Kyle Stanley in :issue:`34037`.)
1109
1110 * The constant values of future flags in the :mod:`__future__` module
1111 is updated in order to prevent collision with compiler flags. Previously
1112 ``PyCF_ALLOW_TOP_LEVEL_AWAIT`` was clashing with ``CO_FUTURE_DIVISION``.
1113 (Contributed by Batuhan Taskaya in :issue:`39562`)
1114
1115 * ``array('u')`` now uses :c:type:`wchar_t` as C type instead of ``Py_UNICODE``.
1116 This change doesn't affect to its behavior because ``Py_UNICODE`` is alias
1117 of :c:type:`wchar_t` since Python 3.3.
1118 (Contributed by Inada Naoki in :issue:`34538`.)
1119
1120 * The :func:`logging.getLogger` API now returns the root logger when passed
1121 the name ``'root'``, whereas previously it returned a non-root logger named
1122 ``'root'``. This could affect cases where user code explicitly wants a
1123 non-root logger named ``'root'``, or instantiates a logger using
1124 ``logging.getLogger(__name__)`` in some top-level module called ``'root.py'``.
1125 (Contributed by Vinay Sajip in :issue:`37742`.)
1126
1127 * Division handling of :class:`~pathlib.PurePath` now returns :data:`NotImplemented`
1128 instead of raising a :exc:`TypeError` when passed something other than an
1129 instance of ``str`` or :class:`~pathlib.PurePath`. This allows creating
1130 compatible classes that don't inherit from those mentioned types.
1131 (Contributed by Roger Aiudi in :issue:`34775`).
1132
1133 * Starting with Python 3.9.5 the :mod:`ipaddress` module no longer
1134 accepts any leading zeros in IPv4 address strings. Leading zeros are
1135 ambiguous and interpreted as octal notation by some libraries. For example
1136 the legacy function :func:`socket.inet_aton` treats leading zeros as octal
1137 notatation. glibc implementation of modern :func:`~socket.inet_pton` does
1138 not accept any leading zeros.
1139 (Contributed by Christian Heimes in :issue:`36384`).
1140
1141 * :func:`codecs.lookup` now normalizes the encoding name the same way as
1142 :func:`encodings.normalize_encoding`, except that :func:`codecs.lookup` also
1143 converts the name to lower case. For example, ``"latex+latin1"`` encoding
1144 name is now normalized to ``"latex_latin1"``.
1145 (Contributed by Jordon Xu in :issue:`37751`.)
1146
1147
1148 Changes in the C API
1149 --------------------
1150
1151 * Instances of :ref:`heap-allocated types <heap-types>` (such as those created with
1152 :c:func:`PyType_FromSpec` and similar APIs) hold a reference to their type
1153 object since Python 3.8. As indicated in the "Changes in the C API" of Python
1154 3.8, for the vast majority of cases, there should be no side effect but for
1155 types that have a custom :c:member:`~PyTypeObject.tp_traverse` function,
1156 ensure that all custom ``tp_traverse`` functions of heap-allocated types
1157 visit the object's type.
1158
1159 Example:
1160
1161 .. code-block:: c
1162
1163 int
1164 foo_traverse(PyObject *self, visitproc visit, void *arg)
1165 {
1166 // Rest of the traverse function
1167 #if PY_VERSION_HEX >= 0x03090000
1168 // This was not needed before Python 3.9 (Python issue 35810 and 40217)
1169 Py_VISIT(Py_TYPE(self));
1170 #endif
1171 }
1172
1173 If your traverse function delegates to ``tp_traverse`` of its base class
1174 (or another type), ensure that ``Py_TYPE(self)`` is visited only once.
1175 Note that only :ref:`heap type <heap-types>` are expected to visit the type
1176 in ``tp_traverse``.
1177
1178 For example, if your ``tp_traverse`` function includes:
1179
1180 .. code-block:: c
1181
1182 base->tp_traverse(self, visit, arg)
1183
1184 then add:
1185
1186 .. code-block:: c
1187
1188 #if PY_VERSION_HEX >= 0x03090000
1189 // This was not needed before Python 3.9 (bpo-35810 and bpo-40217)
1190 if (base->tp_flags & Py_TPFLAGS_HEAPTYPE) {
1191 // a heap type's tp_traverse already visited Py_TYPE(self)
1192 } else {
1193 Py_VISIT(Py_TYPE(self));
1194 }
1195 #else
1196
1197 (See :issue:`35810` and :issue:`40217` for more information.)
1198
1199 * The functions ``PyEval_CallObject``, ``PyEval_CallFunction``,
1200 ``PyEval_CallMethod`` and ``PyEval_CallObjectWithKeywords`` are deprecated.
1201 Use :c:func:`PyObject_Call` and its variants instead.
1202 (See more details in :issue:`29548`.)
1203
1204 CPython bytecode changes
1205 ------------------------
1206
1207 * The :opcode:`!LOAD_ASSERTION_ERROR` opcode was added for handling the
1208 :keyword:`assert` statement. Previously, the assert statement would not work
1209 correctly if the :exc:`AssertionError` exception was being shadowed.
1210 (Contributed by Zackery Spytz in :issue:`34880`.)
1211
1212 * The :opcode:`COMPARE_OP` opcode was split into four distinct instructions:
1213
1214 * ``COMPARE_OP`` for rich comparisons
1215 * ``IS_OP`` for 'is' and 'is not' tests
1216 * ``CONTAINS_OP`` for 'in' and 'not in' tests
1217 * ``JUMP_IF_NOT_EXC_MATCH`` for checking exceptions in 'try-except'
1218 statements.
1219
1220 (Contributed by Mark Shannon in :issue:`39156`.)
1221
1222
1223 Build Changes
1224 =============
1225
1226 * Added ``--with-platlibdir`` option to the ``configure`` script: name of the
1227 platform-specific library directory, stored in the new :data:`sys.platlibdir`
1228 attribute. See :data:`sys.platlibdir` attribute for more information.
1229 (Contributed by Jan Matějek, Matěj Cepl, Charalampos Stratakis
1230 and Victor Stinner in :issue:`1294959`.)
1231
1232 * The ``COUNT_ALLOCS`` special build macro has been removed.
1233 (Contributed by Victor Stinner in :issue:`39489`.)
1234
1235 * On non-Windows platforms, the :c:func:`!setenv` and :c:func:`!unsetenv`
1236 functions are now required to build Python.
1237 (Contributed by Victor Stinner in :issue:`39395`.)
1238
1239 * On non-Windows platforms, creating ``bdist_wininst`` installers is now
1240 officially unsupported. (See :issue:`10945` for more details.)
1241
1242 * When building Python on macOS from source, ``_tkinter`` now links with
1243 non-system Tcl and Tk frameworks if they are installed in
1244 ``/Library/Frameworks``, as had been the case on older releases
1245 of macOS. If a macOS SDK is explicitly configured, by using
1246 :option:`--enable-universalsdk` or ``-isysroot``, only the SDK itself is
1247 searched. The default behavior can still be overridden with
1248 ``--with-tcltk-includes`` and ``--with-tcltk-libs``.
1249 (Contributed by Ned Deily in :issue:`34956`.)
1250
1251 * Python can now be built for Windows 10 ARM64.
1252 (Contributed by Steve Dower in :issue:`33125`.)
1253
1254 * Some individual tests are now skipped when ``--pgo`` is used. The tests
1255 in question increased the PGO task time significantly and likely
1256 didn't help improve optimization of the final executable. This
1257 speeds up the task by a factor of about 15x. Running the full unit test
1258 suite is slow. This change may result in a slightly less optimized build
1259 since not as many code branches will be executed. If you are willing to
1260 wait for the much slower build, the old behavior can be restored using
1261 ``./configure [..] PROFILE_TASK="-m test --pgo-extended"``. We make no
1262 guarantees as to which PGO task set produces a faster build. Users who care
1263 should run their own relevant benchmarks as results can depend on the
1264 environment, workload, and compiler tool chain.
1265 (See :issue:`36044` and :issue:`37707` for more details.)
1266
1267
1268 C API Changes
1269 =============
1270
1271 New Features
1272 ------------
1273
1274 * :pep:`573`: Added :c:func:`PyType_FromModuleAndSpec` to associate
1275 a module with a class; :c:func:`PyType_GetModule` and
1276 :c:func:`PyType_GetModuleState` to retrieve the module and its state; and
1277 :c:type:`PyCMethod` and :c:macro:`METH_METHOD` to allow a method to
1278 access the class it was defined in.
1279 (Contributed by Marcel Plch and Petr Viktorin in :issue:`38787`.)
1280
1281 * Added :c:func:`PyFrame_GetCode` function: get a frame code.
1282 Added :c:func:`PyFrame_GetBack` function: get the frame next outer frame.
1283 (Contributed by Victor Stinner in :issue:`40421`.)
1284
1285 * Added :c:func:`PyFrame_GetLineNumber` to the limited C API.
1286 (Contributed by Victor Stinner in :issue:`40421`.)
1287
1288 * Added :c:func:`PyThreadState_GetInterpreter` and
1289 :c:func:`PyInterpreterState_Get` functions to get the interpreter.
1290 Added :c:func:`PyThreadState_GetFrame` function to get the current frame of a
1291 Python thread state.
1292 Added :c:func:`PyThreadState_GetID` function: get the unique identifier of a
1293 Python thread state.
1294 (Contributed by Victor Stinner in :issue:`39947`.)
1295
1296 * Added a new public :c:func:`PyObject_CallNoArgs` function to the C API, which
1297 calls a callable Python object without any arguments. It is the most efficient
1298 way to call a callable Python object without any argument.
1299 (Contributed by Victor Stinner in :issue:`37194`.)
1300
1301 * Changes in the limited C API (if ``Py_LIMITED_API`` macro is defined):
1302
1303 * Provide :c:func:`Py_EnterRecursiveCall` and :c:func:`Py_LeaveRecursiveCall`
1304 as regular functions for the limited API. Previously, there were defined as
1305 macros, but these macros didn't compile with the limited C API which cannot
1306 access ``PyThreadState.recursion_depth`` field (the structure is opaque in
1307 the limited C API).
1308
1309 * ``PyObject_INIT()`` and ``PyObject_INIT_VAR()`` become regular "opaque"
1310 function to hide implementation details.
1311
1312 (Contributed by Victor Stinner in :issue:`38644` and :issue:`39542`.)
1313
1314 * The :c:func:`PyModule_AddType` function is added to help adding a type
1315 to a module.
1316 (Contributed by Donghee Na in :issue:`40024`.)
1317
1318 * Added the functions :c:func:`PyObject_GC_IsTracked` and
1319 :c:func:`PyObject_GC_IsFinalized` to the public API to allow to query if
1320 Python objects are being currently tracked or have been already finalized by
1321 the garbage collector respectively.
1322 (Contributed by Pablo Galindo Salgado in :issue:`40241`.)
1323
1324 * Added :c:func:`!_PyObject_FunctionStr` to get a user-friendly string
1325 representation of a function-like object.
1326 (Patch by Jeroen Demeyer in :issue:`37645`.)
1327
1328 * Added :c:func:`PyObject_CallOneArg` for calling an object with one
1329 positional argument
1330 (Patch by Jeroen Demeyer in :issue:`37483`.)
1331
1332
1333 Porting to Python 3.9
1334 ---------------------
1335
1336 * ``PyInterpreterState.eval_frame`` (:pep:`523`) now requires a new mandatory
1337 *tstate* parameter (``PyThreadState*``).
1338 (Contributed by Victor Stinner in :issue:`38500`.)
1339
1340 * Extension modules: :c:member:`~PyModuleDef.m_traverse`,
1341 :c:member:`~PyModuleDef.m_clear` and :c:member:`~PyModuleDef.m_free`
1342 functions of :c:type:`PyModuleDef` are no longer called if the module state
1343 was requested but is not allocated yet. This is the case immediately after
1344 the module is created and before the module is executed
1345 (:c:data:`Py_mod_exec` function). More precisely, these functions are not called
1346 if :c:member:`~PyModuleDef.m_size` is greater than 0 and the module state (as
1347 returned by :c:func:`PyModule_GetState`) is ``NULL``.
1348
1349 Extension modules without module state (``m_size <= 0``) are not affected.
1350
1351 * If :c:func:`Py_AddPendingCall` is called in a subinterpreter, the function is
1352 now scheduled to be called from the subinterpreter, rather than being called
1353 from the main interpreter. Each subinterpreter now has its own list of
1354 scheduled calls.
1355 (Contributed by Victor Stinner in :issue:`39984`.)
1356
1357 * The Windows registry is no longer used to initialize :data:`sys.path` when
1358 the ``-E`` option is used (if :c:member:`PyConfig.use_environment` is set to
1359 ``0``). This is significant when embedding Python on Windows.
1360 (Contributed by Zackery Spytz in :issue:`8901`.)
1361
1362 * The global variable :c:data:`PyStructSequence_UnnamedField` is now a constant
1363 and refers to a constant string.
1364 (Contributed by Serhiy Storchaka in :issue:`38650`.)
1365
1366 * The :c:type:`!PyGC_Head` structure is now opaque. It is only defined in the
1367 internal C API (``pycore_gc.h``).
1368 (Contributed by Victor Stinner in :issue:`40241`.)
1369
1370 * The ``Py_UNICODE_COPY``, ``Py_UNICODE_FILL``, ``PyUnicode_WSTR_LENGTH``,
1371 :c:func:`!PyUnicode_FromUnicode`, :c:func:`!PyUnicode_AsUnicode`,
1372 ``_PyUnicode_AsUnicode``, and :c:func:`!PyUnicode_AsUnicodeAndSize` are
1373 marked as deprecated in C. They have been deprecated by :pep:`393` since
1374 Python 3.3.
1375 (Contributed by Inada Naoki in :issue:`36346`.)
1376
1377 * The :c:func:`Py_FatalError` function is replaced with a macro which logs
1378 automatically the name of the current function, unless the
1379 ``Py_LIMITED_API`` macro is defined.
1380 (Contributed by Victor Stinner in :issue:`39882`.)
1381
1382 * The vectorcall protocol now requires that the caller passes only strings as
1383 keyword names. (See :issue:`37540` for more information.)
1384
1385 * Implementation details of a number of macros and functions are now hidden:
1386
1387 * :c:func:`PyObject_IS_GC` macro was converted to a function.
1388
1389 * The :c:func:`!PyObject_NEW` macro becomes an alias to the
1390 :c:macro:`PyObject_New` macro, and the :c:func:`!PyObject_NEW_VAR` macro
1391 becomes an alias to the :c:macro:`PyObject_NewVar` macro. They no longer
1392 access directly the :c:member:`PyTypeObject.tp_basicsize` member.
1393
1394 * :c:func:`!PyObject_GET_WEAKREFS_LISTPTR` macro was converted to a function:
1395 the macro accessed directly the :c:member:`PyTypeObject.tp_weaklistoffset`
1396 member.
1397
1398 * :c:func:`PyObject_CheckBuffer` macro was converted to a function: the macro
1399 accessed directly the :c:member:`PyTypeObject.tp_as_buffer` member.
1400
1401 * :c:func:`PyIndex_Check` is now always declared as an opaque function to hide
1402 implementation details: removed the ``PyIndex_Check()`` macro. The macro accessed
1403 directly the :c:member:`PyTypeObject.tp_as_number` member.
1404
1405 (See :issue:`40170` for more details.)
1406
1407 Removed
1408 -------
1409
1410 * Excluded ``PyFPE_START_PROTECT()`` and ``PyFPE_END_PROTECT()`` macros of
1411 ``pyfpe.h`` from the limited C API.
1412 (Contributed by Victor Stinner in :issue:`38835`.)
1413
1414 * The ``tp_print`` slot of :ref:`PyTypeObject <type-structs>` has been removed.
1415 It was used for printing objects to files in Python 2.7 and before. Since
1416 Python 3.0, it has been ignored and unused.
1417 (Contributed by Jeroen Demeyer in :issue:`36974`.)
1418
1419 * Changes in the limited C API (if ``Py_LIMITED_API`` macro is defined):
1420
1421 * Excluded the following functions from the limited C API:
1422
1423 * ``PyThreadState_DeleteCurrent()``
1424 (Contributed by Joannah Nanjekye in :issue:`37878`.)
1425 * ``_Py_CheckRecursionLimit``
1426 * ``_Py_NewReference()``
1427 * ``_Py_ForgetReference()``
1428 * ``_PyTraceMalloc_NewReference()``
1429 * ``_Py_GetRefTotal()``
1430 * The trashcan mechanism which never worked in the limited C API.
1431 * ``PyTrash_UNWIND_LEVEL``
1432 * ``Py_TRASHCAN_BEGIN_CONDITION``
1433 * ``Py_TRASHCAN_BEGIN``
1434 * ``Py_TRASHCAN_END``
1435 * ``Py_TRASHCAN_SAFE_BEGIN``
1436 * ``Py_TRASHCAN_SAFE_END``
1437
1438 * Moved following functions and definitions to the internal C API:
1439
1440 * ``_PyDebug_PrintTotalRefs()``
1441 * ``_Py_PrintReferences()``
1442 * ``_Py_PrintReferenceAddresses()``
1443 * ``_Py_tracemalloc_config``
1444 * ``_Py_AddToAllObjects()`` (specific to ``Py_TRACE_REFS`` build)
1445
1446 (Contributed by Victor Stinner in :issue:`38644` and :issue:`39542`.)
1447
1448 * Removed ``_PyRuntime.getframe`` hook and removed ``_PyThreadState_GetFrame``
1449 macro which was an alias to ``_PyRuntime.getframe``. They were only exposed
1450 by the internal C API. Removed also ``PyThreadFrameGetter`` type.
1451 (Contributed by Victor Stinner in :issue:`39946`.)
1452
1453 * Removed the following functions from the C API. Call :c:func:`PyGC_Collect`
1454 explicitly to clear all free lists.
1455 (Contributed by Inada Naoki and Victor Stinner in :issue:`37340`,
1456 :issue:`38896` and :issue:`40428`.)
1457
1458 * ``PyAsyncGen_ClearFreeLists()``
1459 * ``PyContext_ClearFreeList()``
1460 * ``PyDict_ClearFreeList()``
1461 * ``PyFloat_ClearFreeList()``
1462 * ``PyFrame_ClearFreeList()``
1463 * ``PyList_ClearFreeList()``
1464 * ``PyMethod_ClearFreeList()`` and ``PyCFunction_ClearFreeList()``:
1465 the free lists of bound method objects have been removed.
1466 * ``PySet_ClearFreeList()``: the set free list has been removed
1467 in Python 3.4.
1468 * ``PyTuple_ClearFreeList()``
1469 * ``PyUnicode_ClearFreeList()``: the Unicode free list has been removed in
1470 Python 3.3.
1471
1472 * Removed ``_PyUnicode_ClearStaticStrings()`` function.
1473 (Contributed by Victor Stinner in :issue:`39465`.)
1474
1475 * Removed ``Py_UNICODE_MATCH``. It has been deprecated by :pep:`393`, and
1476 broken since Python 3.3. The :c:func:`PyUnicode_Tailmatch` function can be
1477 used instead.
1478 (Contributed by Inada Naoki in :issue:`36346`.)
1479
1480 * Cleaned header files of interfaces defined but with no implementation.
1481 The public API symbols being removed are:
1482 ``_PyBytes_InsertThousandsGroupingLocale``,
1483 ``_PyBytes_InsertThousandsGrouping``, ``_Py_InitializeFromArgs``,
1484 ``_Py_InitializeFromWideArgs``, ``_PyFloat_Repr``, ``_PyFloat_Digits``,
1485 ``_PyFloat_DigitsInit``, ``PyFrame_ExtendStack``, ``_PyAIterWrapper_Type``,
1486 ``PyNullImporter_Type``, ``PyCmpWrapper_Type``, ``PySortWrapper_Type``,
1487 ``PyNoArgsFunction``.
1488 (Contributed by Pablo Galindo Salgado in :issue:`39372`.)
1489
1490 Notable changes in Python 3.9.1
1491 ===============================
1492
1493 typing
1494 ------
1495
1496 The behavior of :class:`typing.Literal` was changed to conform with :pep:`586`
1497 and to match the behavior of static type checkers specified in the PEP.
1498
1499 1. ``Literal`` now de-duplicates parameters.
1500 2. Equality comparisons between ``Literal`` objects are now order independent.
1501 3. ``Literal`` comparisons now respect types. For example,
1502 ``Literal[0] == Literal[False]`` previously evaluated to ``True``. It is
1503 now ``False``. To support this change, the internally used type cache now
1504 supports differentiating types.
1505 4. ``Literal`` objects will now raise a :exc:`TypeError` exception during
1506 equality comparisons if any of their parameters are not :term:`hashable`.
1507 Note that declaring ``Literal`` with mutable parameters will not throw
1508 an error::
1509
1510 >>> from typing import Literal
1511 >>> Literal[{0}]
1512 >>> Literal[{0}] == Literal[{False}]
1513 Traceback (most recent call last):
1514 File "<stdin>", line 1, in <module>
1515 TypeError: unhashable type: 'set'
1516
1517 (Contributed by Yurii Karabas in :issue:`42345`.)
1518
1519 macOS 11.0 (Big Sur) and Apple Silicon Mac support
1520 --------------------------------------------------
1521
1522 As of 3.9.1, Python now fully supports building and running on macOS 11.0
1523 (Big Sur) and on Apple Silicon Macs (based on the ``ARM64`` architecture).
1524 A new universal build variant, ``universal2``, is now available to natively
1525 support both ``ARM64`` and ``Intel 64`` in one set of executables. Binaries
1526 can also now be built on current versions of macOS to be deployed on a range
1527 of older macOS versions (tested to 10.9) while making some newer OS
1528 functions and options conditionally available based on the operating system
1529 version in use at runtime ("weaklinking").
1530
1531 (Contributed by Ronald Oussoren and Lawrence D'Anna in :issue:`41100`.)
1532
1533 Notable changes in Python 3.9.2
1534 ===============================
1535
1536 collections.abc
1537 ---------------
1538
1539 :class:`collections.abc.Callable` generic now flattens type parameters, similar
1540 to what :data:`typing.Callable` currently does. This means that
1541 ``collections.abc.Callable[[int, str], str]`` will have ``__args__`` of
1542 ``(int, str, str)``; previously this was ``([int, str], str)``. To allow this
1543 change, :class:`types.GenericAlias` can now be subclassed, and a subclass will
1544 be returned when subscripting the :class:`collections.abc.Callable` type.
1545 Code which accesses the arguments via :func:`typing.get_args` or ``__args__``
1546 need to account for this change. A :exc:`DeprecationWarning` may be emitted for
1547 invalid forms of parameterizing :class:`collections.abc.Callable` which may have
1548 passed silently in Python 3.9.1. This :exc:`DeprecationWarning` will
1549 become a :exc:`TypeError` in Python 3.10.
1550 (Contributed by Ken Jin in :issue:`42195`.)
1551
1552 urllib.parse
1553 ------------
1554
1555 Earlier Python versions allowed using both ``;`` and ``&`` as
1556 query parameter separators in :func:`urllib.parse.parse_qs` and
1557 :func:`urllib.parse.parse_qsl`. Due to security concerns, and to conform with
1558 newer W3C recommendations, this has been changed to allow only a single
1559 separator key, with ``&`` as the default. This change also affects
1560 :func:`!cgi.parse` and :func:`!cgi.parse_multipart` as they use the affected
1561 functions internally. For more details, please see their respective
1562 documentation.
1563 (Contributed by Adam Goldschmidt, Senthil Kumaran and Ken Jin in :issue:`42967`.)
1564
1565 Notable changes in Python 3.9.3
1566 ===============================
1567
1568 A security fix alters the :class:`ftplib.FTP` behavior to not trust the
1569 IPv4 address sent from the remote server when setting up a passive data
1570 channel. We reuse the ftp server IP address instead. For unusual code
1571 requiring the old behavior, set a ``trust_server_pasv_ipv4_address``
1572 attribute on your FTP instance to ``True``. (See :gh:`87451`)
1573
1574 Notable changes in Python 3.9.5
1575 ===============================
1576
1577 urllib.parse
1578 ------------
1579
1580 The presence of newline or tab characters in parts of a URL allows for some
1581 forms of attacks. Following the WHATWG specification that updates :rfc:`3986`,
1582 ASCII newline ``\n``, ``\r`` and tab ``\t`` characters are stripped from the
1583 URL by the parser in :mod:`urllib.parse` preventing such attacks. The removal
1584 characters are controlled by a new module level variable
1585 ``urllib.parse._UNSAFE_URL_BYTES_TO_REMOVE``. (See :gh:`88048`)
1586
1587 Notable security feature in 3.9.14
1588 ==================================
1589
1590 Converting between :class:`int` and :class:`str` in bases other than 2
1591 (binary), 4, 8 (octal), 16 (hexadecimal), or 32 such as base 10 (decimal)
1592 now raises a :exc:`ValueError` if the number of digits in string form is
1593 above a limit to avoid potential denial of service attacks due to the
1594 algorithmic complexity. This is a mitigation for :cve:`2020-10735`.
1595 This limit can be configured or disabled by environment variable, command
1596 line flag, or :mod:`sys` APIs. See the :ref:`integer string conversion
1597 length limitation <int_max_str_digits>` documentation. The default limit
1598 is 4300 digits in string form.
1599
1600 Notable changes in 3.9.17
1601 =========================
1602
1603 tarfile
1604 -------
1605
1606 * The extraction methods in :mod:`tarfile`, and :func:`shutil.unpack_archive`,
1607 have a new a *filter* argument that allows limiting tar features than may be
1608 surprising or dangerous, such as creating files outside the destination
1609 directory.
1610 See :ref:`tarfile-extraction-filter` for details.
1611 In Python 3.12, use without the *filter* argument will show a
1612 :exc:`DeprecationWarning`.
1613 In Python 3.14, the default will switch to ``'data'``.
1614 (Contributed by Petr Viktorin in :pep:`706`.)