| 1 |
**************************** |
| 2 |
What's New In Python 3.9 |
| 3 |
**************************** |
| 4 |
|
| 5 |
:Editor: Łukasz Langa |
| 6 |
|
| 7 |
.. Rules for maintenance: |
| 8 |
|
| 9 |
* Anyone can add text to this document. Your text might get |
| 10 |
rewritten to some degree. |
| 11 |
|
| 12 |
* The maintainer will go through Misc/NEWS periodically and add |
| 13 |
changes; it's therefore more important to add your changes to |
| 14 |
Misc/NEWS than to this file. |
| 15 |
|
| 16 |
* This is not a complete list of every single change; completeness |
| 17 |
is the purpose of Misc/NEWS. Some changes will be too small |
| 18 |
or esoteric to include. If such a change is added to the text, |
| 19 |
it might get removed during final editing. |
| 20 |
|
| 21 |
* If you want to draw your new text to the attention of the |
| 22 |
maintainer, add 'XXX' to the beginning of the paragraph or |
| 23 |
section. |
| 24 |
|
| 25 |
* It's OK to just add a fragmentary note about a change. For |
| 26 |
example: "XXX Describe the transmogrify() function added to the |
| 27 |
socket module." The maintainer will research the change and |
| 28 |
write the necessary text. |
| 29 |
|
| 30 |
* You can comment out your additions if you like, but it's not |
| 31 |
necessary (especially when a final release is some months away). |
| 32 |
|
| 33 |
* Credit the author of a patch or bugfix. Just the name is |
| 34 |
sufficient; the e-mail address isn't necessary. |
| 35 |
|
| 36 |
* It's helpful to add the bug/patch number as a comment: |
| 37 |
|
| 38 |
XXX Describe the transmogrify() function added to the socket |
| 39 |
module. |
| 40 |
(Contributed by P.Y. Developer in :issue:`12345`.) |
| 41 |
|
| 42 |
This saves the maintainer the effort of going through the Mercurial log |
| 43 |
when researching a change. |
| 44 |
|
| 45 |
This article explains the new features in Python 3.9, compared to 3.8. |
| 46 |
Python 3.9 was released on October 5, 2020. |
| 47 |
For full details, see the :ref:`changelog <changelog>`. |
| 48 |
|
| 49 |
.. seealso:: |
| 50 |
|
| 51 |
:pep:`596` - Python 3.9 Release Schedule |
| 52 |
|
| 53 |
|
| 54 |
Summary -- Release highlights |
| 55 |
============================= |
| 56 |
|
| 57 |
.. This section singles out the most important changes in Python 3.9. |
| 58 |
Brevity is key. |
| 59 |
|
| 60 |
New syntax features: |
| 61 |
|
| 62 |
* :pep:`584`, union operators added to ``dict``; |
| 63 |
* :pep:`585`, type hinting generics in standard collections; |
| 64 |
* :pep:`614`, relaxed grammar restrictions on decorators. |
| 65 |
|
| 66 |
New built-in features: |
| 67 |
|
| 68 |
* :pep:`616`, string methods to remove prefixes and suffixes. |
| 69 |
|
| 70 |
New features in the standard library: |
| 71 |
|
| 72 |
* :pep:`593`, flexible function and variable annotations; |
| 73 |
* :func:`os.pidfd_open` added that allows process management without races |
| 74 |
and signals. |
| 75 |
|
| 76 |
Interpreter improvements: |
| 77 |
|
| 78 |
* :pep:`573`, fast access to module state from methods of C extension |
| 79 |
types; |
| 80 |
* :pep:`617`, CPython now uses a new parser based on PEG; |
| 81 |
* a number of Python builtins (range, tuple, set, frozenset, list, dict) are |
| 82 |
now sped up using :pep:`590` vectorcall; |
| 83 |
* garbage collection does not block on resurrected objects; |
| 84 |
* a number of Python modules (:mod:`!_abc`, :mod:`!audioop`, :mod:`!_bz2`, |
| 85 |
:mod:`!_codecs`, :mod:`!_contextvars`, :mod:`!_crypt`, :mod:`!_functools`, |
| 86 |
:mod:`!_json`, :mod:`!_locale`, :mod:`math`, :mod:`operator`, :mod:`resource`, |
| 87 |
:mod:`time`, :mod:`!_weakref`) now use multiphase initialization as defined |
| 88 |
by PEP 489; |
| 89 |
* a number of standard library modules (:mod:`!audioop`, :mod:`ast`, :mod:`grp`, |
| 90 |
:mod:`!_hashlib`, :mod:`pwd`, :mod:`!_posixsubprocess`, :mod:`random`, |
| 91 |
:mod:`select`, :mod:`struct`, :mod:`termios`, :mod:`zlib`) are now using |
| 92 |
the stable ABI defined by PEP 384. |
| 93 |
|
| 94 |
New library modules: |
| 95 |
|
| 96 |
* :pep:`615`, the IANA Time Zone Database is now present in the standard |
| 97 |
library in the :mod:`zoneinfo` module; |
| 98 |
* an implementation of a topological sort of a graph is now provided in |
| 99 |
the new :mod:`graphlib` module. |
| 100 |
|
| 101 |
Release process changes: |
| 102 |
|
| 103 |
* :pep:`602`, CPython adopts an annual release cycle. |
| 104 |
|
| 105 |
|
| 106 |
You should check for DeprecationWarning in your code |
| 107 |
==================================================== |
| 108 |
|
| 109 |
When Python 2.7 was still supported, a lot of functionality in Python 3 |
| 110 |
was kept for backward compatibility with Python 2.7. With the end of Python |
| 111 |
2 support, these backward compatibility layers have been removed, or will |
| 112 |
be removed soon. Most of them emitted a :exc:`DeprecationWarning` warning for |
| 113 |
several years. For example, using ``collections.Mapping`` instead of |
| 114 |
``collections.abc.Mapping`` emits a :exc:`DeprecationWarning` since Python |
| 115 |
3.3, released in 2012. |
| 116 |
|
| 117 |
Test your application with the :option:`-W` ``default`` command-line option to see |
| 118 |
:exc:`DeprecationWarning` and :exc:`PendingDeprecationWarning`, or even with |
| 119 |
:option:`-W` ``error`` to treat them as errors. :ref:`Warnings Filter |
| 120 |
<warning-filter>` can be used to ignore warnings from third-party code. |
| 121 |
|
| 122 |
Python 3.9 is the last version providing those Python 2 backward compatibility |
| 123 |
layers, to give more time to Python projects maintainers to organize the |
| 124 |
removal of the Python 2 support and add support for Python 3.9. |
| 125 |
|
| 126 |
Aliases to :ref:`Abstract Base Classes <collections-abstract-base-classes>` in |
| 127 |
the :mod:`collections` module, like ``collections.Mapping`` alias to |
| 128 |
:class:`collections.abc.Mapping`, are kept for one last release for backward |
| 129 |
compatibility. They will be removed from Python 3.10. |
| 130 |
|
| 131 |
More generally, try to run your tests in the :ref:`Python Development Mode |
| 132 |
<devmode>` which helps to prepare your code to make it compatible with the |
| 133 |
next Python version. |
| 134 |
|
| 135 |
Note: a number of pre-existing deprecations were removed in this version of |
| 136 |
Python as well. Consult the :ref:`removed-in-python-39` section. |
| 137 |
|
| 138 |
|
| 139 |
New Features |
| 140 |
============ |
| 141 |
|
| 142 |
Dictionary Merge & Update Operators |
| 143 |
----------------------------------- |
| 144 |
|
| 145 |
Merge (``|``) and update (``|=``) operators have been added to the built-in |
| 146 |
:class:`dict` class. Those complement the existing ``dict.update`` and |
| 147 |
``{**d1, **d2}`` methods of merging dictionaries. |
| 148 |
|
| 149 |
Example:: |
| 150 |
|
| 151 |
>>> x = {"key1": "value1 from x", "key2": "value2 from x"} |
| 152 |
>>> y = {"key2": "value2 from y", "key3": "value3 from y"} |
| 153 |
>>> x | y |
| 154 |
{'key1': 'value1 from x', 'key2': 'value2 from y', 'key3': 'value3 from y'} |
| 155 |
>>> y | x |
| 156 |
{'key2': 'value2 from x', 'key3': 'value3 from y', 'key1': 'value1 from x'} |
| 157 |
|
| 158 |
See :pep:`584` for a full description. |
| 159 |
(Contributed by Brandt Bucher in :issue:`36144`.) |
| 160 |
|
| 161 |
New String Methods to Remove Prefixes and Suffixes |
| 162 |
-------------------------------------------------- |
| 163 |
|
| 164 |
:meth:`str.removeprefix(prefix)<str.removeprefix>` and |
| 165 |
:meth:`str.removesuffix(suffix)<str.removesuffix>` have been added |
| 166 |
to easily remove an unneeded prefix or a suffix from a string. Corresponding |
| 167 |
``bytes``, ``bytearray``, and ``collections.UserString`` methods have also been |
| 168 |
added. See :pep:`616` for a full description. (Contributed by Dennis Sweeney in |
| 169 |
:issue:`39939`.) |
| 170 |
|
| 171 |
Type Hinting Generics in Standard Collections |
| 172 |
--------------------------------------------- |
| 173 |
|
| 174 |
In type annotations you can now use built-in collection types such as |
| 175 |
``list`` and ``dict`` as generic types instead of importing the |
| 176 |
corresponding capitalized types (e.g. ``List`` or ``Dict``) from |
| 177 |
``typing``. Some other types in the standard library are also now generic, |
| 178 |
for example ``queue.Queue``. |
| 179 |
|
| 180 |
Example: |
| 181 |
|
| 182 |
.. code-block:: python |
| 183 |
|
| 184 |
def greet_all(names: list[str]) -> None: |
| 185 |
for name in names: |
| 186 |
print("Hello", name) |
| 187 |
|
| 188 |
See :pep:`585` for more details. (Contributed by Guido van Rossum, |
| 189 |
Ethan Smith, and Batuhan Taşkaya in :issue:`39481`.) |
| 190 |
|
| 191 |
New Parser |
| 192 |
---------- |
| 193 |
|
| 194 |
Python 3.9 uses a new parser, based on `PEG |
| 195 |
<https://en.wikipedia.org/wiki/Parsing_expression_grammar>`_ instead |
| 196 |
of `LL(1) <https://en.wikipedia.org/wiki/LL_parser>`_. The new |
| 197 |
parser's performance is roughly comparable to that of the old parser, |
| 198 |
but the PEG formalism is more flexible than LL(1) when it comes to |
| 199 |
designing new language features. We'll start using this flexibility |
| 200 |
in Python 3.10 and later. |
| 201 |
|
| 202 |
The :mod:`ast` module uses the new parser and produces the same AST as |
| 203 |
the old parser. |
| 204 |
|
| 205 |
In Python 3.10, the old parser will be deleted and so will all |
| 206 |
functionality that depends on it (primarily the :mod:`!parser` module, |
| 207 |
which has long been deprecated). In Python 3.9 *only*, you can switch |
| 208 |
back to the LL(1) parser using a command line switch (``-X |
| 209 |
oldparser``) or an environment variable (``PYTHONOLDPARSER=1``). |
| 210 |
|
| 211 |
See :pep:`617` for more details. (Contributed by Guido van Rossum, |
| 212 |
Pablo Galindo and Lysandros Nikolaou in :issue:`40334`.) |
| 213 |
|
| 214 |
|
| 215 |
Other Language Changes |
| 216 |
====================== |
| 217 |
|
| 218 |
* :func:`__import__` now raises :exc:`ImportError` instead of |
| 219 |
:exc:`ValueError`, which used to occur when a relative import went past |
| 220 |
its top-level package. |
| 221 |
(Contributed by Ngalim Siregar in :issue:`37444`.) |
| 222 |
|
| 223 |
* Python now gets the absolute path of the script filename specified on |
| 224 |
the command line (ex: ``python3 script.py``): the ``__file__`` attribute of |
| 225 |
the :mod:`__main__` module became an absolute path, rather than a relative |
| 226 |
path. These paths now remain valid after the current directory is changed |
| 227 |
by :func:`os.chdir`. As a side effect, the traceback also displays the |
| 228 |
absolute path for :mod:`__main__` module frames in this case. |
| 229 |
(Contributed by Victor Stinner in :issue:`20443`.) |
| 230 |
|
| 231 |
* In the :ref:`Python Development Mode <devmode>` and in :ref:`debug build <debug-build>`, the |
| 232 |
*encoding* and *errors* arguments are now checked for string encoding and |
| 233 |
decoding operations. Examples: :func:`open`, :meth:`str.encode` and |
| 234 |
:meth:`bytes.decode`. |
| 235 |
|
| 236 |
By default, for best performance, the *errors* argument is only checked at |
| 237 |
the first encoding/decoding error and the *encoding* argument is sometimes |
| 238 |
ignored for empty strings. |
| 239 |
(Contributed by Victor Stinner in :issue:`37388`.) |
| 240 |
|
| 241 |
* ``"".replace("", s, n)`` now returns ``s`` instead of an empty string for |
| 242 |
all non-zero ``n``. It is now consistent with ``"".replace("", s)``. |
| 243 |
There are similar changes for :class:`bytes` and :class:`bytearray` objects. |
| 244 |
(Contributed by Serhiy Storchaka in :issue:`28029`.) |
| 245 |
|
| 246 |
* Any valid expression can now be used as a :term:`decorator`. Previously, the |
| 247 |
grammar was much more restrictive. See :pep:`614` for details. |
| 248 |
(Contributed by Brandt Bucher in :issue:`39702`.) |
| 249 |
|
| 250 |
* Improved help for the :mod:`typing` module. Docstrings are now shown for |
| 251 |
all special forms and special generic aliases (like ``Union`` and ``List``). |
| 252 |
Using :func:`help` with generic alias like ``List[int]`` will show the help |
| 253 |
for the correspondent concrete type (``list`` in this case). |
| 254 |
(Contributed by Serhiy Storchaka in :issue:`40257`.) |
| 255 |
|
| 256 |
* Parallel running of :meth:`~agen.aclose` / :meth:`~agen.asend` / |
| 257 |
:meth:`~agen.athrow` is now prohibited, and ``ag_running`` now reflects |
| 258 |
the actual running status of the async generator. |
| 259 |
(Contributed by Yury Selivanov in :issue:`30773`.) |
| 260 |
|
| 261 |
* Unexpected errors in calling the ``__iter__`` method are no longer masked by |
| 262 |
``TypeError`` in the :keyword:`in` operator and functions |
| 263 |
:func:`~operator.contains`, :func:`~operator.indexOf` and |
| 264 |
:func:`~operator.countOf` of the :mod:`operator` module. |
| 265 |
(Contributed by Serhiy Storchaka in :issue:`40824`.) |
| 266 |
|
| 267 |
* Unparenthesized lambda expressions can no longer be the expression part in an |
| 268 |
``if`` clause in comprehensions and generator expressions. See :issue:`41848` |
| 269 |
and :issue:`43755` for details. |
| 270 |
|
| 271 |
|
| 272 |
New Modules |
| 273 |
=========== |
| 274 |
|
| 275 |
zoneinfo |
| 276 |
-------- |
| 277 |
|
| 278 |
The :mod:`zoneinfo` module brings support for the IANA time zone database to |
| 279 |
the standard library. It adds :class:`zoneinfo.ZoneInfo`, a concrete |
| 280 |
:class:`datetime.tzinfo` implementation backed by the system's time zone data. |
| 281 |
|
| 282 |
Example:: |
| 283 |
|
| 284 |
>>> from zoneinfo import ZoneInfo |
| 285 |
>>> import datetime as dt |
| 286 |
|
| 287 |
>>> # Daylight saving time |
| 288 |
>>> when = dt.datetime(2020, 10, 31, 12, tzinfo=ZoneInfo("America/Los_Angeles")) |
| 289 |
>>> print(when) |
| 290 |
2020-10-31 12:00:00-07:00 |
| 291 |
>>> when.tzname() |
| 292 |
'PDT' |
| 293 |
|
| 294 |
>>> # Standard time |
| 295 |
>>> when += dt.timedelta(days=7) |
| 296 |
>>> print(when) |
| 297 |
2020-11-07 12:00:00-08:00 |
| 298 |
>>> print(when.tzname()) |
| 299 |
PST |
| 300 |
|
| 301 |
|
| 302 |
As a fall-back source of data for platforms that don't ship the IANA database, |
| 303 |
the :pypi:`tzdata` module was released as a first-party package -- distributed via |
| 304 |
PyPI and maintained by the CPython core team. |
| 305 |
|
| 306 |
.. seealso:: |
| 307 |
|
| 308 |
:pep:`615` -- Support for the IANA Time Zone Database in the Standard Library |
| 309 |
PEP written and implemented by Paul Ganssle |
| 310 |
|
| 311 |
|
| 312 |
graphlib |
| 313 |
--------- |
| 314 |
|
| 315 |
A new module, :mod:`graphlib`, was added that contains the |
| 316 |
:class:`graphlib.TopologicalSorter` class to offer functionality to perform |
| 317 |
topological sorting of graphs. (Contributed by Pablo Galindo, Tim Peters and |
| 318 |
Larry Hastings in :issue:`17005`.) |
| 319 |
|
| 320 |
|
| 321 |
Improved Modules |
| 322 |
================ |
| 323 |
|
| 324 |
ast |
| 325 |
--- |
| 326 |
|
| 327 |
Added the *indent* option to :func:`~ast.dump` which allows it to produce a |
| 328 |
multiline indented output. |
| 329 |
(Contributed by Serhiy Storchaka in :issue:`37995`.) |
| 330 |
|
| 331 |
Added :func:`ast.unparse` as a function in the :mod:`ast` module that can |
| 332 |
be used to unparse an :class:`ast.AST` object and produce a string with code |
| 333 |
that would produce an equivalent :class:`ast.AST` object when parsed. |
| 334 |
(Contributed by Pablo Galindo and Batuhan Taskaya in :issue:`38870`.) |
| 335 |
|
| 336 |
Added docstrings to AST nodes that contains the ASDL signature used to |
| 337 |
construct that node. (Contributed by Batuhan Taskaya in :issue:`39638`.) |
| 338 |
|
| 339 |
asyncio |
| 340 |
------- |
| 341 |
|
| 342 |
Due to significant security concerns, the *reuse_address* parameter of |
| 343 |
:meth:`asyncio.loop.create_datagram_endpoint` is no longer supported. This is |
| 344 |
because of the behavior of the socket option ``SO_REUSEADDR`` in UDP. For more |
| 345 |
details, see the documentation for ``loop.create_datagram_endpoint()``. |
| 346 |
(Contributed by Kyle Stanley, Antoine Pitrou, and Yury Selivanov in |
| 347 |
:issue:`37228`.) |
| 348 |
|
| 349 |
Added a new :term:`coroutine` :meth:`~asyncio.loop.shutdown_default_executor` |
| 350 |
that schedules a shutdown for the default executor that waits on the |
| 351 |
:class:`~concurrent.futures.ThreadPoolExecutor` to finish closing. Also, |
| 352 |
:func:`asyncio.run` has been updated to use the new :term:`coroutine`. |
| 353 |
(Contributed by Kyle Stanley in :issue:`34037`.) |
| 354 |
|
| 355 |
Added :class:`!asyncio.PidfdChildWatcher`, a Linux-specific child watcher |
| 356 |
implementation that polls process file descriptors. (:issue:`38692`) |
| 357 |
|
| 358 |
Added a new :term:`coroutine` :func:`asyncio.to_thread`. It is mainly used for |
| 359 |
running IO-bound functions in a separate thread to avoid blocking the event |
| 360 |
loop, and essentially works as a high-level version of |
| 361 |
:meth:`~asyncio.loop.run_in_executor` that can directly take keyword arguments. |
| 362 |
(Contributed by Kyle Stanley and Yury Selivanov in :issue:`32309`.) |
| 363 |
|
| 364 |
When cancelling the task due to a timeout, :meth:`asyncio.wait_for` will now |
| 365 |
wait until the cancellation is complete also in the case when *timeout* is |
| 366 |
<= 0, like it does with positive timeouts. |
| 367 |
(Contributed by Elvis Pranskevichus in :issue:`32751`.) |
| 368 |
|
| 369 |
:mod:`asyncio` now raises :exc:`TypeError` when calling incompatible |
| 370 |
methods with an :class:`ssl.SSLSocket` socket. |
| 371 |
(Contributed by Ido Michael in :issue:`37404`.) |
| 372 |
|
| 373 |
compileall |
| 374 |
---------- |
| 375 |
|
| 376 |
Added new possibility to use hardlinks for duplicated ``.pyc`` files: *hardlink_dupes* parameter and --hardlink-dupes command line option. |
| 377 |
(Contributed by Lumír 'Frenzy' Balhar in :issue:`40495`.) |
| 378 |
|
| 379 |
Added new options for path manipulation in resulting ``.pyc`` files: *stripdir*, *prependdir*, *limit_sl_dest* parameters and -s, -p, -e command line options. |
| 380 |
Added the possibility to specify the option for an optimization level multiple times. |
| 381 |
(Contributed by Lumír 'Frenzy' Balhar in :issue:`38112`.) |
| 382 |
|
| 383 |
concurrent.futures |
| 384 |
------------------ |
| 385 |
|
| 386 |
Added a new *cancel_futures* parameter to |
| 387 |
:meth:`concurrent.futures.Executor.shutdown` that cancels all pending futures |
| 388 |
which have not started running, instead of waiting for them to complete before |
| 389 |
shutting down the executor. |
| 390 |
(Contributed by Kyle Stanley in :issue:`39349`.) |
| 391 |
|
| 392 |
Removed daemon threads from :class:`~concurrent.futures.ThreadPoolExecutor` |
| 393 |
and :class:`~concurrent.futures.ProcessPoolExecutor`. This improves |
| 394 |
compatibility with subinterpreters and predictability in their shutdown |
| 395 |
processes. (Contributed by Kyle Stanley in :issue:`39812`.) |
| 396 |
|
| 397 |
Workers in :class:`~concurrent.futures.ProcessPoolExecutor` are now spawned on |
| 398 |
demand, only when there are no available idle workers to reuse. This optimizes |
| 399 |
startup overhead and reduces the amount of lost CPU time to idle workers. |
| 400 |
(Contributed by Kyle Stanley in :issue:`39207`.) |
| 401 |
|
| 402 |
curses |
| 403 |
------ |
| 404 |
|
| 405 |
Added :func:`curses.get_escdelay`, :func:`curses.set_escdelay`, |
| 406 |
:func:`curses.get_tabsize`, and :func:`curses.set_tabsize` functions. |
| 407 |
(Contributed by Anthony Sottile in :issue:`38312`.) |
| 408 |
|
| 409 |
datetime |
| 410 |
-------- |
| 411 |
The :meth:`~datetime.date.isocalendar` of :class:`datetime.date` |
| 412 |
and :meth:`~datetime.datetime.isocalendar` of :class:`datetime.datetime` |
| 413 |
methods now returns a :func:`~collections.namedtuple` instead of a :class:`tuple`. |
| 414 |
(Contributed by Donghee Na in :issue:`24416`.) |
| 415 |
|
| 416 |
distutils |
| 417 |
--------- |
| 418 |
|
| 419 |
The :command:`upload` command now creates SHA2-256 and Blake2b-256 hash |
| 420 |
digests. It skips MD5 on platforms that block MD5 digest. |
| 421 |
(Contributed by Christian Heimes in :issue:`40698`.) |
| 422 |
|
| 423 |
fcntl |
| 424 |
----- |
| 425 |
|
| 426 |
Added constants :const:`!fcntl.F_OFD_GETLK`, :const:`!fcntl.F_OFD_SETLK` |
| 427 |
and :const:`!fcntl.F_OFD_SETLKW`. |
| 428 |
(Contributed by Donghee Na in :issue:`38602`.) |
| 429 |
|
| 430 |
ftplib |
| 431 |
------- |
| 432 |
|
| 433 |
:class:`~ftplib.FTP` and :class:`~ftplib.FTP_TLS` now raise a :class:`ValueError` |
| 434 |
if the given timeout for their constructor is zero to prevent the creation of |
| 435 |
a non-blocking socket. (Contributed by Donghee Na in :issue:`39259`.) |
| 436 |
|
| 437 |
gc |
| 438 |
-- |
| 439 |
|
| 440 |
When the garbage collector makes a collection in which some objects resurrect |
| 441 |
(they are reachable from outside the isolated cycles after the finalizers have |
| 442 |
been executed), do not block the collection of all objects that are still |
| 443 |
unreachable. (Contributed by Pablo Galindo and Tim Peters in :issue:`38379`.) |
| 444 |
|
| 445 |
Added a new function :func:`gc.is_finalized` to check if an object has been |
| 446 |
finalized by the garbage collector. (Contributed by Pablo Galindo in |
| 447 |
:issue:`39322`.) |
| 448 |
|
| 449 |
hashlib |
| 450 |
------- |
| 451 |
|
| 452 |
The :mod:`hashlib` module can now use SHA3 hashes and SHAKE XOF from OpenSSL |
| 453 |
when available. |
| 454 |
(Contributed by Christian Heimes in :issue:`37630`.) |
| 455 |
|
| 456 |
Builtin hash modules can now be disabled with |
| 457 |
``./configure --without-builtin-hashlib-hashes`` or selectively enabled with |
| 458 |
e.g. ``./configure --with-builtin-hashlib-hashes=sha3,blake2`` to force use |
| 459 |
of OpenSSL based implementation. |
| 460 |
(Contributed by Christian Heimes in :issue:`40479`) |
| 461 |
|
| 462 |
|
| 463 |
http |
| 464 |
---- |
| 465 |
|
| 466 |
HTTP status codes ``103 EARLY_HINTS``, ``418 IM_A_TEAPOT`` and ``425 TOO_EARLY`` are added to |
| 467 |
:class:`http.HTTPStatus`. (Contributed by Donghee Na in :issue:`39509` and Ross Rhodes in :issue:`39507`.) |
| 468 |
|
| 469 |
IDLE and idlelib |
| 470 |
---------------- |
| 471 |
|
| 472 |
Added option to toggle cursor blink off. (Contributed by Zackery Spytz |
| 473 |
in :issue:`4603`.) |
| 474 |
|
| 475 |
Escape key now closes IDLE completion windows. (Contributed by Johnny |
| 476 |
Najera in :issue:`38944`.) |
| 477 |
|
| 478 |
Added keywords to module name completion list. (Contributed by Terry J. |
| 479 |
Reedy in :issue:`37765`.) |
| 480 |
|
| 481 |
New in 3.9 maintenance releases |
| 482 |
|
| 483 |
Make IDLE invoke :func:`sys.excepthook` (when started without '-n'). |
| 484 |
User hooks were previously ignored. (Contributed by Ken Hilton in |
| 485 |
:issue:`43008`.) |
| 486 |
|
| 487 |
The changes above have been backported to 3.8 maintenance releases. |
| 488 |
|
| 489 |
Rearrange the settings dialog. Split the General tab into Windows |
| 490 |
and Shell/Ed tabs. Move help sources, which extend the Help menu, to the |
| 491 |
Extensions tab. Make space for new options and shorten the dialog. The |
| 492 |
latter makes the dialog better fit small screens. (Contributed by Terry Jan |
| 493 |
Reedy in :issue:`40468`.) Move the indent space setting from the Font tab to |
| 494 |
the new Windows tab. (Contributed by Mark Roseman and Terry Jan Reedy in |
| 495 |
:issue:`33962`.) |
| 496 |
|
| 497 |
Apply syntax highlighting to ``.pyi`` files. (Contributed by Alex |
| 498 |
Waygood and Terry Jan Reedy in :issue:`45447`.) |
| 499 |
|
| 500 |
imaplib |
| 501 |
------- |
| 502 |
|
| 503 |
:class:`~imaplib.IMAP4` and :class:`~imaplib.IMAP4_SSL` now have |
| 504 |
an optional *timeout* parameter for their constructors. |
| 505 |
Also, the :meth:`~imaplib.IMAP4.open` method now has an optional *timeout* parameter |
| 506 |
with this change. The overridden methods of :class:`~imaplib.IMAP4_SSL` and |
| 507 |
:class:`~imaplib.IMAP4_stream` were applied to this change. |
| 508 |
(Contributed by Donghee Na in :issue:`38615`.) |
| 509 |
|
| 510 |
:meth:`imaplib.IMAP4.unselect` is added. |
| 511 |
:meth:`imaplib.IMAP4.unselect` frees server's resources associated with the |
| 512 |
selected mailbox and returns the server to the authenticated |
| 513 |
state. This command performs the same actions as :meth:`imaplib.IMAP4.close`, except |
| 514 |
that no messages are permanently removed from the currently |
| 515 |
selected mailbox. (Contributed by Donghee Na in :issue:`40375`.) |
| 516 |
|
| 517 |
importlib |
| 518 |
--------- |
| 519 |
|
| 520 |
To improve consistency with import statements, :func:`importlib.util.resolve_name` |
| 521 |
now raises :exc:`ImportError` instead of :exc:`ValueError` for invalid relative |
| 522 |
import attempts. |
| 523 |
(Contributed by Ngalim Siregar in :issue:`37444`.) |
| 524 |
|
| 525 |
Import loaders which publish immutable module objects can now publish |
| 526 |
immutable packages in addition to individual modules. |
| 527 |
(Contributed by Dino Viehland in :issue:`39336`.) |
| 528 |
|
| 529 |
Added :func:`importlib.resources.files` function with support for |
| 530 |
subdirectories in package data, matching backport in ``importlib_resources`` |
| 531 |
version 1.5. |
| 532 |
(Contributed by Jason R. Coombs in :issue:`39791`.) |
| 533 |
|
| 534 |
Refreshed ``importlib.metadata`` from ``importlib_metadata`` version 1.6.1. |
| 535 |
|
| 536 |
inspect |
| 537 |
------- |
| 538 |
|
| 539 |
:attr:`inspect.BoundArguments.arguments` is changed from ``OrderedDict`` to regular |
| 540 |
dict. (Contributed by Inada Naoki in :issue:`36350` and :issue:`39775`.) |
| 541 |
|
| 542 |
ipaddress |
| 543 |
--------- |
| 544 |
|
| 545 |
:mod:`ipaddress` now supports IPv6 Scoped Addresses (IPv6 address with suffix ``%<scope_id>``). |
| 546 |
|
| 547 |
Scoped IPv6 addresses can be parsed using :class:`ipaddress.IPv6Address`. |
| 548 |
If present, scope zone ID is available through the :attr:`~ipaddress.IPv6Address.scope_id` attribute. |
| 549 |
(Contributed by Oleksandr Pavliuk in :issue:`34788`.) |
| 550 |
|
| 551 |
Starting with Python 3.9.5 the :mod:`ipaddress` module no longer |
| 552 |
accepts any leading zeros in IPv4 address strings. |
| 553 |
(Contributed by Christian Heimes in :issue:`36384`). |
| 554 |
|
| 555 |
math |
| 556 |
---- |
| 557 |
|
| 558 |
Expanded the :func:`math.gcd` function to handle multiple arguments. |
| 559 |
Formerly, it only supported two arguments. |
| 560 |
(Contributed by Serhiy Storchaka in :issue:`39648`.) |
| 561 |
|
| 562 |
Added :func:`math.lcm`: return the least common multiple of specified arguments. |
| 563 |
(Contributed by Mark Dickinson, Ananthakrishnan and Serhiy Storchaka in |
| 564 |
:issue:`39479` and :issue:`39648`.) |
| 565 |
|
| 566 |
Added :func:`math.nextafter`: return the next floating-point value after *x* |
| 567 |
towards *y*. |
| 568 |
(Contributed by Victor Stinner in :issue:`39288`.) |
| 569 |
|
| 570 |
Added :func:`math.ulp`: return the value of the least significant bit |
| 571 |
of a float. |
| 572 |
(Contributed by Victor Stinner in :issue:`39310`.) |
| 573 |
|
| 574 |
multiprocessing |
| 575 |
--------------- |
| 576 |
|
| 577 |
The :class:`multiprocessing.SimpleQueue` class has a new |
| 578 |
:meth:`~multiprocessing.SimpleQueue.close` method to explicitly close the |
| 579 |
queue. |
| 580 |
(Contributed by Victor Stinner in :issue:`30966`.) |
| 581 |
|
| 582 |
nntplib |
| 583 |
------- |
| 584 |
|
| 585 |
:class:`!NNTP` and :class:`!NNTP_SSL` now raise a :class:`ValueError` |
| 586 |
if the given timeout for their constructor is zero to prevent the creation of |
| 587 |
a non-blocking socket. (Contributed by Donghee Na in :issue:`39259`.) |
| 588 |
|
| 589 |
os |
| 590 |
-- |
| 591 |
|
| 592 |
Added :const:`~os.CLD_KILLED` and :const:`~os.CLD_STOPPED` for :attr:`!si_code`. |
| 593 |
(Contributed by Donghee Na in :issue:`38493`.) |
| 594 |
|
| 595 |
Exposed the Linux-specific :func:`os.pidfd_open` (:issue:`38692`) and |
| 596 |
:const:`os.P_PIDFD` (:issue:`38713`) for process management with file |
| 597 |
descriptors. |
| 598 |
|
| 599 |
The :func:`os.unsetenv` function is now also available on Windows. |
| 600 |
(Contributed by Victor Stinner in :issue:`39413`.) |
| 601 |
|
| 602 |
The :func:`os.putenv` and :func:`os.unsetenv` functions are now always |
| 603 |
available. |
| 604 |
(Contributed by Victor Stinner in :issue:`39395`.) |
| 605 |
|
| 606 |
Added :func:`os.waitstatus_to_exitcode` function: |
| 607 |
convert a wait status to an exit code. |
| 608 |
(Contributed by Victor Stinner in :issue:`40094`.) |
| 609 |
|
| 610 |
pathlib |
| 611 |
------- |
| 612 |
|
| 613 |
Added :meth:`pathlib.Path.readlink` which acts similarly to |
| 614 |
:func:`os.readlink`. |
| 615 |
(Contributed by Girts Folkmanis in :issue:`30618`) |
| 616 |
|
| 617 |
pdb |
| 618 |
--- |
| 619 |
|
| 620 |
On Windows now :class:`~pdb.Pdb` supports ``~/.pdbrc``. |
| 621 |
(Contributed by Tim Hopper and Dan Lidral-Porter in :issue:`20523`.) |
| 622 |
|
| 623 |
poplib |
| 624 |
------ |
| 625 |
|
| 626 |
:class:`~poplib.POP3` and :class:`~poplib.POP3_SSL` now raise a :class:`ValueError` |
| 627 |
if the given timeout for their constructor is zero to prevent the creation of |
| 628 |
a non-blocking socket. (Contributed by Donghee Na in :issue:`39259`.) |
| 629 |
|
| 630 |
pprint |
| 631 |
------ |
| 632 |
|
| 633 |
:mod:`pprint` can now pretty-print :class:`types.SimpleNamespace`. |
| 634 |
(Contributed by Carl Bordum Hansen in :issue:`37376`.) |
| 635 |
|
| 636 |
pydoc |
| 637 |
----- |
| 638 |
|
| 639 |
The documentation string is now shown not only for class, function, |
| 640 |
method etc, but for any object that has its own :attr:`~definition.__doc__` |
| 641 |
attribute. |
| 642 |
(Contributed by Serhiy Storchaka in :issue:`40257`.) |
| 643 |
|
| 644 |
random |
| 645 |
------ |
| 646 |
|
| 647 |
Added a new :meth:`random.Random.randbytes` method: generate random bytes. |
| 648 |
(Contributed by Victor Stinner in :issue:`40286`.) |
| 649 |
|
| 650 |
signal |
| 651 |
------ |
| 652 |
|
| 653 |
Exposed the Linux-specific :func:`signal.pidfd_send_signal` for sending to |
| 654 |
signals to a process using a file descriptor instead of a pid. (:issue:`38712`) |
| 655 |
|
| 656 |
smtplib |
| 657 |
------- |
| 658 |
|
| 659 |
:class:`~smtplib.SMTP` and :class:`~smtplib.SMTP_SSL` now raise a :class:`ValueError` |
| 660 |
if the given timeout for their constructor is zero to prevent the creation of |
| 661 |
a non-blocking socket. (Contributed by Donghee Na in :issue:`39259`.) |
| 662 |
|
| 663 |
:class:`~smtplib.LMTP` constructor now has an optional *timeout* parameter. |
| 664 |
(Contributed by Donghee Na in :issue:`39329`.) |
| 665 |
|
| 666 |
socket |
| 667 |
------ |
| 668 |
|
| 669 |
The :mod:`socket` module now exports the :const:`~socket.CAN_RAW_JOIN_FILTERS` |
| 670 |
constant on Linux 4.1 and greater. |
| 671 |
(Contributed by Stefan Tatschner and Zackery Spytz in :issue:`25780`.) |
| 672 |
|
| 673 |
The socket module now supports the :const:`~socket.CAN_J1939` protocol on |
| 674 |
platforms that support it. (Contributed by Karl Ding in :issue:`40291`.) |
| 675 |
|
| 676 |
The socket module now has the :func:`socket.send_fds` and |
| 677 |
:func:`socket.recv_fds` functions. (Contributed by Joannah Nanjekye, Shinya |
| 678 |
Okano and Victor Stinner in :issue:`28724`.) |
| 679 |
|
| 680 |
|
| 681 |
time |
| 682 |
---- |
| 683 |
|
| 684 |
On AIX, :func:`~time.thread_time` is now implemented with ``thread_cputime()`` |
| 685 |
which has nanosecond resolution, rather than |
| 686 |
``clock_gettime(CLOCK_THREAD_CPUTIME_ID)`` which has a resolution of 10 milliseconds. |
| 687 |
(Contributed by Batuhan Taskaya in :issue:`40192`) |
| 688 |
|
| 689 |
sys |
| 690 |
--- |
| 691 |
|
| 692 |
Added a new :data:`sys.platlibdir` attribute: name of the platform-specific |
| 693 |
library directory. It is used to build the path of standard library and the |
| 694 |
paths of installed extension modules. It is equal to ``"lib"`` on most |
| 695 |
platforms. On Fedora and SuSE, it is equal to ``"lib64"`` on 64-bit platforms. |
| 696 |
(Contributed by Jan Matějek, Matěj Cepl, Charalampos Stratakis and Victor Stinner in :issue:`1294959`.) |
| 697 |
|
| 698 |
Previously, :data:`sys.stderr` was block-buffered when non-interactive. Now |
| 699 |
``stderr`` defaults to always being line-buffered. |
| 700 |
(Contributed by Jendrik Seipp in :issue:`13601`.) |
| 701 |
|
| 702 |
tracemalloc |
| 703 |
----------- |
| 704 |
|
| 705 |
Added :func:`tracemalloc.reset_peak` to set the peak size of traced memory |
| 706 |
blocks to the current size, to measure the peak of specific pieces of code. |
| 707 |
(Contributed by Huon Wilson in :issue:`40630`.) |
| 708 |
|
| 709 |
typing |
| 710 |
------ |
| 711 |
|
| 712 |
:pep:`593` introduced an :data:`typing.Annotated` type to decorate existing |
| 713 |
types with context-specific metadata and new ``include_extras`` parameter to |
| 714 |
:func:`typing.get_type_hints` to access the metadata at runtime. (Contributed |
| 715 |
by Till Varoquaux and Konstantin Kashin.) |
| 716 |
|
| 717 |
unicodedata |
| 718 |
----------- |
| 719 |
|
| 720 |
The Unicode database has been updated to version 13.0.0. (:issue:`39926`). |
| 721 |
|
| 722 |
venv |
| 723 |
---- |
| 724 |
|
| 725 |
The activation scripts provided by :mod:`venv` now all specify their prompt |
| 726 |
customization consistently by always using the value specified by |
| 727 |
``__VENV_PROMPT__``. Previously some scripts unconditionally used |
| 728 |
``__VENV_PROMPT__``, others only if it happened to be set (which was the default |
| 729 |
case), and one used ``__VENV_NAME__`` instead. |
| 730 |
(Contributed by Brett Cannon in :issue:`37663`.) |
| 731 |
|
| 732 |
xml |
| 733 |
--- |
| 734 |
|
| 735 |
White space characters within attributes are now preserved when serializing |
| 736 |
:mod:`xml.etree.ElementTree` to XML file. EOLNs are no longer normalized |
| 737 |
to "\n". This is the result of discussion about how to interpret |
| 738 |
section 2.11 of XML spec. |
| 739 |
(Contributed by Mefistotelis in :issue:`39011`.) |
| 740 |
|
| 741 |
|
| 742 |
Optimizations |
| 743 |
============= |
| 744 |
|
| 745 |
* Optimized the idiom for assignment a temporary variable in comprehensions. |
| 746 |
Now ``for y in [expr]`` in comprehensions is as fast as a simple assignment |
| 747 |
``y = expr``. For example: |
| 748 |
|
| 749 |
sums = [s for s in [0] for x in data for s in [s + x]] |
| 750 |
|
| 751 |
Unlike the ``:=`` operator this idiom does not leak a variable to the |
| 752 |
outer scope. |
| 753 |
|
| 754 |
(Contributed by Serhiy Storchaka in :issue:`32856`.) |
| 755 |
|
| 756 |
* Optimized signal handling in multithreaded applications. If a thread different |
| 757 |
than the main thread gets a signal, the bytecode evaluation loop is no longer |
| 758 |
interrupted at each bytecode instruction to check for pending signals which |
| 759 |
cannot be handled. Only the main thread of the main interpreter can handle |
| 760 |
signals. |
| 761 |
|
| 762 |
Previously, the bytecode evaluation loop was interrupted at each instruction |
| 763 |
until the main thread handles signals. |
| 764 |
(Contributed by Victor Stinner in :issue:`40010`.) |
| 765 |
|
| 766 |
* Optimized the :mod:`subprocess` module on FreeBSD using ``closefrom()``. |
| 767 |
(Contributed by Ed Maste, Conrad Meyer, Kyle Evans, Kubilay Kocak and Victor |
| 768 |
Stinner in :issue:`38061`.) |
| 769 |
|
| 770 |
* :c:func:`PyLong_FromDouble` is now up to 1.87x faster for values that |
| 771 |
fit into :c:expr:`long`. |
| 772 |
(Contributed by Sergey Fedoseev in :issue:`37986`.) |
| 773 |
|
| 774 |
* A number of Python builtins (:class:`range`, :class:`tuple`, :class:`set`, |
| 775 |
:class:`frozenset`, :class:`list`, :class:`dict`) are now sped up by using |
| 776 |
:pep:`590` vectorcall protocol. |
| 777 |
(Contributed by Donghee Na, Mark Shannon, Jeroen Demeyer and Petr Viktorin in :issue:`37207`.) |
| 778 |
|
| 779 |
* Optimized :meth:`!set.difference_update` for the case when the other set |
| 780 |
is much larger than the base set. |
| 781 |
(Suggested by Evgeny Kapun with code contributed by Michele Orrù in :issue:`8425`.) |
| 782 |
|
| 783 |
* Python's small object allocator (``obmalloc.c``) now allows (no more than) |
| 784 |
one empty arena to remain available for immediate reuse, without returning |
| 785 |
it to the OS. This prevents thrashing in simple loops where an arena could |
| 786 |
be created and destroyed anew on each iteration. |
| 787 |
(Contributed by Tim Peters in :issue:`37257`.) |
| 788 |
|
| 789 |
* :term:`floor division` of float operation now has a better performance. Also |
| 790 |
the message of :exc:`ZeroDivisionError` for this operation is updated. |
| 791 |
(Contributed by Donghee Na in :issue:`39434`.) |
| 792 |
|
| 793 |
* Decoding short ASCII strings with UTF-8 and ascii codecs is now about |
| 794 |
15% faster. (Contributed by Inada Naoki in :issue:`37348`.) |
| 795 |
|
| 796 |
Here's a summary of performance improvements from Python 3.4 through Python 3.9: |
| 797 |
|
| 798 |
.. code-block:: none |
| 799 |
|
| 800 |
Python version 3.4 3.5 3.6 3.7 3.8 3.9 |
| 801 |
-------------- --- --- --- --- --- --- |
| 802 |
|
| 803 |
Variable and attribute read access: |
| 804 |
read_local 7.1 7.1 5.4 5.1 3.9 3.9 |
| 805 |
read_nonlocal 7.1 8.1 5.8 5.4 4.4 4.5 |
| 806 |
read_global 15.5 19.0 14.3 13.6 7.6 7.8 |
| 807 |
read_builtin 21.1 21.6 18.5 19.0 7.5 7.8 |
| 808 |
read_classvar_from_class 25.6 26.5 20.7 19.5 18.4 17.9 |
| 809 |
read_classvar_from_instance 22.8 23.5 18.8 17.1 16.4 16.9 |
| 810 |
read_instancevar 32.4 33.1 28.0 26.3 25.4 25.3 |
| 811 |
read_instancevar_slots 27.8 31.3 20.8 20.8 20.2 20.5 |
| 812 |
read_namedtuple 73.8 57.5 45.0 46.8 18.4 18.7 |
| 813 |
read_boundmethod 37.6 37.9 29.6 26.9 27.7 41.1 |
| 814 |
|
| 815 |
Variable and attribute write access: |
| 816 |
write_local 8.7 9.3 5.5 5.3 4.3 4.3 |
| 817 |
write_nonlocal 10.5 11.1 5.6 5.5 4.7 4.8 |
| 818 |
write_global 19.7 21.2 18.0 18.0 15.8 16.7 |
| 819 |
write_classvar 92.9 96.0 104.6 102.1 39.2 39.8 |
| 820 |
write_instancevar 44.6 45.8 40.0 38.9 35.5 37.4 |
| 821 |
write_instancevar_slots 35.6 36.1 27.3 26.6 25.7 25.8 |
| 822 |
|
| 823 |
Data structure read access: |
| 824 |
read_list 24.2 24.5 20.8 20.8 19.0 19.5 |
| 825 |
read_deque 24.7 25.5 20.2 20.6 19.8 20.2 |
| 826 |
read_dict 24.3 25.7 22.3 23.0 21.0 22.4 |
| 827 |
read_strdict 22.6 24.3 19.5 21.2 18.9 21.5 |
| 828 |
|
| 829 |
Data structure write access: |
| 830 |
write_list 27.1 28.5 22.5 21.6 20.0 20.0 |
| 831 |
write_deque 28.7 30.1 22.7 21.8 23.5 21.7 |
| 832 |
write_dict 31.4 33.3 29.3 29.2 24.7 25.4 |
| 833 |
write_strdict 28.4 29.9 27.5 25.2 23.1 24.5 |
| 834 |
|
| 835 |
Stack (or queue) operations: |
| 836 |
list_append_pop 93.4 112.7 75.4 74.2 50.8 50.6 |
| 837 |
deque_append_pop 43.5 57.0 49.4 49.2 42.5 44.2 |
| 838 |
deque_append_popleft 43.7 57.3 49.7 49.7 42.8 46.4 |
| 839 |
|
| 840 |
Timing loop: |
| 841 |
loop_overhead 0.5 0.6 0.4 0.3 0.3 0.3 |
| 842 |
|
| 843 |
These results were generated from the variable access benchmark script at: |
| 844 |
``Tools/scripts/var_access_benchmark.py``. The benchmark script displays timings |
| 845 |
in nanoseconds. The benchmarks were measured on an |
| 846 |
`Intel® Core™ i7-4960HQ processor |
| 847 |
<https://ark.intel.com/content/www/us/en/ark/products/76088/intel-core-i7-4960hq-processor-6m-cache-up-to-3-80-ghz.html>`_ |
| 848 |
running the macOS 64-bit builds found at |
| 849 |
`python.org <https://www.python.org/downloads/macos/>`_. |
| 850 |
|
| 851 |
|
| 852 |
Deprecated |
| 853 |
========== |
| 854 |
|
| 855 |
* The distutils ``bdist_msi`` command is now deprecated, use |
| 856 |
``bdist_wheel`` (wheel packages) instead. |
| 857 |
(Contributed by Hugo van Kemenade in :issue:`39586`.) |
| 858 |
|
| 859 |
* Currently :func:`math.factorial` accepts :class:`float` instances with |
| 860 |
non-negative integer values (like ``5.0``). It raises a :exc:`ValueError` |
| 861 |
for non-integral and negative floats. It is now deprecated. In future |
| 862 |
Python versions it will raise a :exc:`TypeError` for all floats. |
| 863 |
(Contributed by Serhiy Storchaka in :issue:`37315`.) |
| 864 |
|
| 865 |
* The :mod:`!parser` and :mod:`!symbol` modules are deprecated and will be |
| 866 |
removed in future versions of Python. For the majority of use cases, |
| 867 |
users can leverage the Abstract Syntax Tree (AST) generation and compilation |
| 868 |
stage, using the :mod:`ast` module. |
| 869 |
|
| 870 |
* The Public C API functions :c:func:`!PyParser_SimpleParseStringFlags`, |
| 871 |
:c:func:`!PyParser_SimpleParseStringFlagsFilename`, |
| 872 |
:c:func:`!PyParser_SimpleParseFileFlags` and :c:func:`!PyNode_Compile` |
| 873 |
are deprecated and will be removed in Python 3.10 together with the old parser. |
| 874 |
|
| 875 |
* Using :data:`NotImplemented` in a boolean context has been deprecated, |
| 876 |
as it is almost exclusively the result of incorrect rich comparator |
| 877 |
implementations. It will be made a :exc:`TypeError` in a future version |
| 878 |
of Python. |
| 879 |
(Contributed by Josh Rosenberg in :issue:`35712`.) |
| 880 |
|
| 881 |
* The :mod:`random` module currently accepts any hashable type as a |
| 882 |
possible seed value. Unfortunately, some of those types are not |
| 883 |
guaranteed to have a deterministic hash value. After Python 3.9, |
| 884 |
the module will restrict its seeds to :const:`None`, :class:`int`, |
| 885 |
:class:`float`, :class:`str`, :class:`bytes`, and :class:`bytearray`. |
| 886 |
|
| 887 |
* Opening the :class:`~gzip.GzipFile` file for writing without specifying |
| 888 |
the *mode* argument is deprecated. In future Python versions it will always |
| 889 |
be opened for reading by default. Specify the *mode* argument for opening |
| 890 |
it for writing and silencing a warning. |
| 891 |
(Contributed by Serhiy Storchaka in :issue:`28286`.) |
| 892 |
|
| 893 |
* Deprecated the ``split()`` method of :class:`!_tkinter.TkappType` in |
| 894 |
favour of the ``splitlist()`` method which has more consistent and |
| 895 |
predictable behavior. |
| 896 |
(Contributed by Serhiy Storchaka in :issue:`38371`.) |
| 897 |
|
| 898 |
* The explicit passing of coroutine objects to :func:`asyncio.wait` has been |
| 899 |
deprecated and will be removed in version 3.11. |
| 900 |
(Contributed by Yury Selivanov and Kyle Stanley in :issue:`34790`.) |
| 901 |
|
| 902 |
* binhex4 and hexbin4 standards are now deprecated. The :mod:`!binhex` module |
| 903 |
and the following :mod:`binascii` functions are now deprecated: |
| 904 |
|
| 905 |
* :func:`!b2a_hqx`, :func:`!a2b_hqx` |
| 906 |
* :func:`!rlecode_hqx`, :func:`!rledecode_hqx` |
| 907 |
|
| 908 |
(Contributed by Victor Stinner in :issue:`39353`.) |
| 909 |
|
| 910 |
* :mod:`ast` classes ``slice``, ``Index`` and ``ExtSlice`` are considered deprecated |
| 911 |
and will be removed in future Python versions. ``value`` itself should be |
| 912 |
used instead of ``Index(value)``. ``Tuple(slices, Load())`` should be |
| 913 |
used instead of ``ExtSlice(slices)``. |
| 914 |
(Contributed by Serhiy Storchaka in :issue:`34822`.) |
| 915 |
|
| 916 |
* :mod:`ast` classes ``Suite``, ``Param``, ``AugLoad`` and ``AugStore`` |
| 917 |
are considered deprecated and will be removed in future Python versions. |
| 918 |
They were not generated by the parser and not accepted by the code |
| 919 |
generator in Python 3. |
| 920 |
(Contributed by Batuhan Taskaya in :issue:`39639` and :issue:`39969` |
| 921 |
and Serhiy Storchaka in :issue:`39988`.) |
| 922 |
|
| 923 |
* The :c:func:`!PyEval_InitThreads` and :c:func:`!PyEval_ThreadsInitialized` |
| 924 |
functions are now deprecated and will be removed in Python 3.11. Calling |
| 925 |
:c:func:`!PyEval_InitThreads` now does nothing. The :term:`GIL` is initialized |
| 926 |
by :c:func:`Py_Initialize` since Python 3.7. |
| 927 |
(Contributed by Victor Stinner in :issue:`39877`.) |
| 928 |
|
| 929 |
* Passing ``None`` as the first argument to the :func:`shlex.split` function |
| 930 |
has been deprecated. (Contributed by Zackery Spytz in :issue:`33262`.) |
| 931 |
|
| 932 |
* :func:`!smtpd.MailmanProxy` is now deprecated as it is unusable without |
| 933 |
an external module, ``mailman``. (Contributed by Samuel Colvin in :issue:`35800`.) |
| 934 |
|
| 935 |
* The :mod:`!lib2to3` module now emits a :exc:`PendingDeprecationWarning`. |
| 936 |
Python 3.9 switched to a PEG parser (see :pep:`617`), and Python 3.10 may |
| 937 |
include new language syntax that is not parsable by lib2to3's LL(1) parser. |
| 938 |
The :mod:`!lib2to3` module may be removed from the standard library in a future |
| 939 |
Python version. Consider third-party alternatives such as `LibCST`_ or |
| 940 |
`parso`_. |
| 941 |
(Contributed by Carl Meyer in :issue:`40360`.) |
| 942 |
|
| 943 |
* The *random* parameter of :func:`random.shuffle` has been deprecated. |
| 944 |
(Contributed by Raymond Hettinger in :issue:`40465`) |
| 945 |
|
| 946 |
.. _LibCST: https://libcst.readthedocs.io/ |
| 947 |
.. _parso: https://parso.readthedocs.io/ |
| 948 |
|
| 949 |
.. _removed-in-python-39: |
| 950 |
|
| 951 |
Removed |
| 952 |
======= |
| 953 |
|
| 954 |
* The erroneous version at :data:`!unittest.mock.__version__` has been removed. |
| 955 |
|
| 956 |
* :class:`!nntplib.NNTP`: ``xpath()`` and ``xgtitle()`` methods have been removed. |
| 957 |
These methods are deprecated since Python 3.3. Generally, these extensions |
| 958 |
are not supported or not enabled by NNTP server administrators. |
| 959 |
For ``xgtitle()``, please use :meth:`!nntplib.NNTP.descriptions` or |
| 960 |
:meth:`!nntplib.NNTP.description` instead. |
| 961 |
(Contributed by Donghee Na in :issue:`39366`.) |
| 962 |
|
| 963 |
* :class:`array.array`: ``tostring()`` and ``fromstring()`` methods have been |
| 964 |
removed. They were aliases to ``tobytes()`` and ``frombytes()``, deprecated |
| 965 |
since Python 3.2. |
| 966 |
(Contributed by Victor Stinner in :issue:`38916`.) |
| 967 |
|
| 968 |
* The undocumented ``sys.callstats()`` function has been removed. Since Python |
| 969 |
3.7, it was deprecated and always returned :const:`None`. It required a special |
| 970 |
build option ``CALL_PROFILE`` which was already removed in Python 3.7. |
| 971 |
(Contributed by Victor Stinner in :issue:`37414`.) |
| 972 |
|
| 973 |
* The ``sys.getcheckinterval()`` and ``sys.setcheckinterval()`` functions have |
| 974 |
been removed. They were deprecated since Python 3.2. Use |
| 975 |
:func:`sys.getswitchinterval` and :func:`sys.setswitchinterval` instead. |
| 976 |
(Contributed by Victor Stinner in :issue:`37392`.) |
| 977 |
|
| 978 |
* The C function ``PyImport_Cleanup()`` has been removed. It was documented as: |
| 979 |
"Empty the module table. For internal use only." |
| 980 |
(Contributed by Victor Stinner in :issue:`36710`.) |
| 981 |
|
| 982 |
* ``_dummy_thread`` and ``dummy_threading`` modules have been removed. These |
| 983 |
modules were deprecated since Python 3.7 which requires threading support. |
| 984 |
(Contributed by Victor Stinner in :issue:`37312`.) |
| 985 |
|
| 986 |
* ``aifc.openfp()`` alias to ``aifc.open()``, ``sunau.openfp()`` alias to |
| 987 |
``sunau.open()``, and ``wave.openfp()`` alias to :func:`wave.open` have been |
| 988 |
removed. They were deprecated since Python 3.7. |
| 989 |
(Contributed by Victor Stinner in :issue:`37320`.) |
| 990 |
|
| 991 |
* The :meth:`!isAlive` method of :class:`threading.Thread` |
| 992 |
has been removed. It was deprecated since Python 3.8. |
| 993 |
Use :meth:`~threading.Thread.is_alive` instead. |
| 994 |
(Contributed by Donghee Na in :issue:`37804`.) |
| 995 |
|
| 996 |
* Methods ``getchildren()`` and ``getiterator()`` of classes |
| 997 |
:class:`~xml.etree.ElementTree.ElementTree` and |
| 998 |
:class:`~xml.etree.ElementTree.Element` in the :mod:`~xml.etree.ElementTree` |
| 999 |
module have been removed. They were deprecated in Python 3.2. |
| 1000 |
Use ``iter(x)`` or ``list(x)`` instead of ``x.getchildren()`` and |
| 1001 |
``x.iter()`` or ``list(x.iter())`` instead of ``x.getiterator()``. |
| 1002 |
(Contributed by Serhiy Storchaka in :issue:`36543`.) |
| 1003 |
|
| 1004 |
* The old :mod:`plistlib` API has been removed, it was deprecated since Python |
| 1005 |
3.4. Use the :func:`~plistlib.load`, :func:`~plistlib.loads`, :func:`~plistlib.dump`, and |
| 1006 |
:func:`~plistlib.dumps` functions. Additionally, the *use_builtin_types* parameter was |
| 1007 |
removed, standard :class:`bytes` objects are always used instead. |
| 1008 |
(Contributed by Jon Janzen in :issue:`36409`.) |
| 1009 |
|
| 1010 |
* The C function ``PyGen_NeedsFinalizing`` has been removed. It was not |
| 1011 |
documented, tested, or used anywhere within CPython after the implementation |
| 1012 |
of :pep:`442`. Patch by Joannah Nanjekye. |
| 1013 |
(Contributed by Joannah Nanjekye in :issue:`15088`) |
| 1014 |
|
| 1015 |
* ``base64.encodestring()`` and ``base64.decodestring()``, aliases deprecated |
| 1016 |
since Python 3.1, have been removed: use :func:`base64.encodebytes` and |
| 1017 |
:func:`base64.decodebytes` instead. |
| 1018 |
(Contributed by Victor Stinner in :issue:`39351`.) |
| 1019 |
|
| 1020 |
* ``fractions.gcd()`` function has been removed, it was deprecated since Python |
| 1021 |
3.5 (:issue:`22486`): use :func:`math.gcd` instead. |
| 1022 |
(Contributed by Victor Stinner in :issue:`39350`.) |
| 1023 |
|
| 1024 |
* The *buffering* parameter of :class:`bz2.BZ2File` has been removed. Since |
| 1025 |
Python 3.0, it was ignored and using it emitted a :exc:`DeprecationWarning`. |
| 1026 |
Pass an open file object to control how the file is opened. |
| 1027 |
(Contributed by Victor Stinner in :issue:`39357`.) |
| 1028 |
|
| 1029 |
* The *encoding* parameter of :func:`json.loads` has been removed. |
| 1030 |
As of Python 3.1, it was deprecated and ignored; using it has emitted a |
| 1031 |
:exc:`DeprecationWarning` since Python 3.8. |
| 1032 |
(Contributed by Inada Naoki in :issue:`39377`) |
| 1033 |
|
| 1034 |
* ``with (await asyncio.lock):`` and ``with (yield from asyncio.lock):`` statements are |
| 1035 |
not longer supported, use ``async with lock`` instead. The same is correct for |
| 1036 |
``asyncio.Condition`` and ``asyncio.Semaphore``. |
| 1037 |
(Contributed by Andrew Svetlov in :issue:`34793`.) |
| 1038 |
|
| 1039 |
* The :func:`!sys.getcounts` function, the ``-X showalloccount`` command line |
| 1040 |
option and the ``show_alloc_count`` field of the C structure |
| 1041 |
:c:type:`PyConfig` have been removed. They required a special Python build by |
| 1042 |
defining ``COUNT_ALLOCS`` macro. |
| 1043 |
(Contributed by Victor Stinner in :issue:`39489`.) |
| 1044 |
|
| 1045 |
* The ``_field_types`` attribute of the :class:`typing.NamedTuple` class |
| 1046 |
has been removed. It was deprecated since Python 3.8. Use |
| 1047 |
the ``__annotations__`` attribute instead. |
| 1048 |
(Contributed by Serhiy Storchaka in :issue:`40182`.) |
| 1049 |
|
| 1050 |
* The :meth:`!symtable.SymbolTable.has_exec` method has been removed. It was |
| 1051 |
deprecated since 2006, and only returning ``False`` when it's called. |
| 1052 |
(Contributed by Batuhan Taskaya in :issue:`40208`) |
| 1053 |
|
| 1054 |
* The :meth:`!asyncio.Task.current_task` and :meth:`!asyncio.Task.all_tasks` |
| 1055 |
have been removed. They were deprecated since Python 3.7 and you can use |
| 1056 |
:func:`asyncio.current_task` and :func:`asyncio.all_tasks` instead. |
| 1057 |
(Contributed by Rémi Lapeyre in :issue:`40967`) |
| 1058 |
|
| 1059 |
* The ``unescape()`` method in the :class:`html.parser.HTMLParser` class |
| 1060 |
has been removed (it was deprecated since Python 3.4). :func:`html.unescape` |
| 1061 |
should be used for converting character references to the corresponding |
| 1062 |
unicode characters. |
| 1063 |
|
| 1064 |
|
| 1065 |
Porting to Python 3.9 |
| 1066 |
===================== |
| 1067 |
|
| 1068 |
This section lists previously described changes and other bugfixes |
| 1069 |
that may require changes to your code. |
| 1070 |
|
| 1071 |
|
| 1072 |
Changes in the Python API |
| 1073 |
------------------------- |
| 1074 |
|
| 1075 |
* :func:`__import__` and :func:`importlib.util.resolve_name` now raise |
| 1076 |
:exc:`ImportError` where it previously raised :exc:`ValueError`. Callers |
| 1077 |
catching the specific exception type and supporting both Python 3.9 and |
| 1078 |
earlier versions will need to catch both using ``except (ImportError, ValueError):``. |
| 1079 |
|
| 1080 |
* The :mod:`venv` activation scripts no longer special-case when |
| 1081 |
``__VENV_PROMPT__`` is set to ``""``. |
| 1082 |
|
| 1083 |
* The :meth:`select.epoll.unregister` method no longer ignores the |
| 1084 |
:const:`~errno.EBADF` error. |
| 1085 |
(Contributed by Victor Stinner in :issue:`39239`.) |
| 1086 |
|
| 1087 |
* The *compresslevel* parameter of :class:`bz2.BZ2File` became keyword-only, |
| 1088 |
since the *buffering* parameter has been removed. |
| 1089 |
(Contributed by Victor Stinner in :issue:`39357`.) |
| 1090 |
|
| 1091 |
* Simplified AST for subscription. Simple indices will be represented by |
| 1092 |
their value, extended slices will be represented as tuples. |
| 1093 |
``Index(value)`` will return a ``value`` itself, ``ExtSlice(slices)`` |
| 1094 |
will return ``Tuple(slices, Load())``. |
| 1095 |
(Contributed by Serhiy Storchaka in :issue:`34822`.) |
| 1096 |
|
| 1097 |
* The :mod:`importlib` module now ignores the :envvar:`PYTHONCASEOK` |
| 1098 |
environment variable when the :option:`-E` or :option:`-I` command line |
| 1099 |
options are being used. |
| 1100 |
|
| 1101 |
* The *encoding* parameter has been added to the classes :class:`ftplib.FTP` and |
| 1102 |
:class:`ftplib.FTP_TLS` as a keyword-only parameter, and the default encoding |
| 1103 |
is changed from Latin-1 to UTF-8 to follow :rfc:`2640`. |
| 1104 |
|
| 1105 |
* :meth:`asyncio.loop.shutdown_default_executor` has been added to |
| 1106 |
:class:`~asyncio.AbstractEventLoop`, meaning alternative event loops that |
| 1107 |
inherit from it should have this method defined. |
| 1108 |
(Contributed by Kyle Stanley in :issue:`34037`.) |
| 1109 |
|
| 1110 |
* The constant values of future flags in the :mod:`__future__` module |
| 1111 |
is updated in order to prevent collision with compiler flags. Previously |
| 1112 |
``PyCF_ALLOW_TOP_LEVEL_AWAIT`` was clashing with ``CO_FUTURE_DIVISION``. |
| 1113 |
(Contributed by Batuhan Taskaya in :issue:`39562`) |
| 1114 |
|
| 1115 |
* ``array('u')`` now uses :c:type:`wchar_t` as C type instead of ``Py_UNICODE``. |
| 1116 |
This change doesn't affect to its behavior because ``Py_UNICODE`` is alias |
| 1117 |
of :c:type:`wchar_t` since Python 3.3. |
| 1118 |
(Contributed by Inada Naoki in :issue:`34538`.) |
| 1119 |
|
| 1120 |
* The :func:`logging.getLogger` API now returns the root logger when passed |
| 1121 |
the name ``'root'``, whereas previously it returned a non-root logger named |
| 1122 |
``'root'``. This could affect cases where user code explicitly wants a |
| 1123 |
non-root logger named ``'root'``, or instantiates a logger using |
| 1124 |
``logging.getLogger(__name__)`` in some top-level module called ``'root.py'``. |
| 1125 |
(Contributed by Vinay Sajip in :issue:`37742`.) |
| 1126 |
|
| 1127 |
* Division handling of :class:`~pathlib.PurePath` now returns :data:`NotImplemented` |
| 1128 |
instead of raising a :exc:`TypeError` when passed something other than an |
| 1129 |
instance of ``str`` or :class:`~pathlib.PurePath`. This allows creating |
| 1130 |
compatible classes that don't inherit from those mentioned types. |
| 1131 |
(Contributed by Roger Aiudi in :issue:`34775`). |
| 1132 |
|
| 1133 |
* Starting with Python 3.9.5 the :mod:`ipaddress` module no longer |
| 1134 |
accepts any leading zeros in IPv4 address strings. Leading zeros are |
| 1135 |
ambiguous and interpreted as octal notation by some libraries. For example |
| 1136 |
the legacy function :func:`socket.inet_aton` treats leading zeros as octal |
| 1137 |
notatation. glibc implementation of modern :func:`~socket.inet_pton` does |
| 1138 |
not accept any leading zeros. |
| 1139 |
(Contributed by Christian Heimes in :issue:`36384`). |
| 1140 |
|
| 1141 |
* :func:`codecs.lookup` now normalizes the encoding name the same way as |
| 1142 |
:func:`encodings.normalize_encoding`, except that :func:`codecs.lookup` also |
| 1143 |
converts the name to lower case. For example, ``"latex+latin1"`` encoding |
| 1144 |
name is now normalized to ``"latex_latin1"``. |
| 1145 |
(Contributed by Jordon Xu in :issue:`37751`.) |
| 1146 |
|
| 1147 |
|
| 1148 |
Changes in the C API |
| 1149 |
-------------------- |
| 1150 |
|
| 1151 |
* Instances of :ref:`heap-allocated types <heap-types>` (such as those created with |
| 1152 |
:c:func:`PyType_FromSpec` and similar APIs) hold a reference to their type |
| 1153 |
object since Python 3.8. As indicated in the "Changes in the C API" of Python |
| 1154 |
3.8, for the vast majority of cases, there should be no side effect but for |
| 1155 |
types that have a custom :c:member:`~PyTypeObject.tp_traverse` function, |
| 1156 |
ensure that all custom ``tp_traverse`` functions of heap-allocated types |
| 1157 |
visit the object's type. |
| 1158 |
|
| 1159 |
Example: |
| 1160 |
|
| 1161 |
.. code-block:: c |
| 1162 |
|
| 1163 |
int |
| 1164 |
foo_traverse(PyObject *self, visitproc visit, void *arg) |
| 1165 |
{ |
| 1166 |
// Rest of the traverse function |
| 1167 |
#if PY_VERSION_HEX >= 0x03090000 |
| 1168 |
// This was not needed before Python 3.9 (Python issue 35810 and 40217) |
| 1169 |
Py_VISIT(Py_TYPE(self)); |
| 1170 |
#endif |
| 1171 |
} |
| 1172 |
|
| 1173 |
If your traverse function delegates to ``tp_traverse`` of its base class |
| 1174 |
(or another type), ensure that ``Py_TYPE(self)`` is visited only once. |
| 1175 |
Note that only :ref:`heap type <heap-types>` are expected to visit the type |
| 1176 |
in ``tp_traverse``. |
| 1177 |
|
| 1178 |
For example, if your ``tp_traverse`` function includes: |
| 1179 |
|
| 1180 |
.. code-block:: c |
| 1181 |
|
| 1182 |
base->tp_traverse(self, visit, arg) |
| 1183 |
|
| 1184 |
then add: |
| 1185 |
|
| 1186 |
.. code-block:: c |
| 1187 |
|
| 1188 |
#if PY_VERSION_HEX >= 0x03090000 |
| 1189 |
// This was not needed before Python 3.9 (bpo-35810 and bpo-40217) |
| 1190 |
if (base->tp_flags & Py_TPFLAGS_HEAPTYPE) { |
| 1191 |
// a heap type's tp_traverse already visited Py_TYPE(self) |
| 1192 |
} else { |
| 1193 |
Py_VISIT(Py_TYPE(self)); |
| 1194 |
} |
| 1195 |
#else |
| 1196 |
|
| 1197 |
(See :issue:`35810` and :issue:`40217` for more information.) |
| 1198 |
|
| 1199 |
* The functions ``PyEval_CallObject``, ``PyEval_CallFunction``, |
| 1200 |
``PyEval_CallMethod`` and ``PyEval_CallObjectWithKeywords`` are deprecated. |
| 1201 |
Use :c:func:`PyObject_Call` and its variants instead. |
| 1202 |
(See more details in :issue:`29548`.) |
| 1203 |
|
| 1204 |
CPython bytecode changes |
| 1205 |
------------------------ |
| 1206 |
|
| 1207 |
* The :opcode:`!LOAD_ASSERTION_ERROR` opcode was added for handling the |
| 1208 |
:keyword:`assert` statement. Previously, the assert statement would not work |
| 1209 |
correctly if the :exc:`AssertionError` exception was being shadowed. |
| 1210 |
(Contributed by Zackery Spytz in :issue:`34880`.) |
| 1211 |
|
| 1212 |
* The :opcode:`COMPARE_OP` opcode was split into four distinct instructions: |
| 1213 |
|
| 1214 |
* ``COMPARE_OP`` for rich comparisons |
| 1215 |
* ``IS_OP`` for 'is' and 'is not' tests |
| 1216 |
* ``CONTAINS_OP`` for 'in' and 'not in' tests |
| 1217 |
* ``JUMP_IF_NOT_EXC_MATCH`` for checking exceptions in 'try-except' |
| 1218 |
statements. |
| 1219 |
|
| 1220 |
(Contributed by Mark Shannon in :issue:`39156`.) |
| 1221 |
|
| 1222 |
|
| 1223 |
Build Changes |
| 1224 |
============= |
| 1225 |
|
| 1226 |
* Added ``--with-platlibdir`` option to the ``configure`` script: name of the |
| 1227 |
platform-specific library directory, stored in the new :data:`sys.platlibdir` |
| 1228 |
attribute. See :data:`sys.platlibdir` attribute for more information. |
| 1229 |
(Contributed by Jan Matějek, Matěj Cepl, Charalampos Stratakis |
| 1230 |
and Victor Stinner in :issue:`1294959`.) |
| 1231 |
|
| 1232 |
* The ``COUNT_ALLOCS`` special build macro has been removed. |
| 1233 |
(Contributed by Victor Stinner in :issue:`39489`.) |
| 1234 |
|
| 1235 |
* On non-Windows platforms, the :c:func:`!setenv` and :c:func:`!unsetenv` |
| 1236 |
functions are now required to build Python. |
| 1237 |
(Contributed by Victor Stinner in :issue:`39395`.) |
| 1238 |
|
| 1239 |
* On non-Windows platforms, creating ``bdist_wininst`` installers is now |
| 1240 |
officially unsupported. (See :issue:`10945` for more details.) |
| 1241 |
|
| 1242 |
* When building Python on macOS from source, ``_tkinter`` now links with |
| 1243 |
non-system Tcl and Tk frameworks if they are installed in |
| 1244 |
``/Library/Frameworks``, as had been the case on older releases |
| 1245 |
of macOS. If a macOS SDK is explicitly configured, by using |
| 1246 |
:option:`--enable-universalsdk` or ``-isysroot``, only the SDK itself is |
| 1247 |
searched. The default behavior can still be overridden with |
| 1248 |
``--with-tcltk-includes`` and ``--with-tcltk-libs``. |
| 1249 |
(Contributed by Ned Deily in :issue:`34956`.) |
| 1250 |
|
| 1251 |
* Python can now be built for Windows 10 ARM64. |
| 1252 |
(Contributed by Steve Dower in :issue:`33125`.) |
| 1253 |
|
| 1254 |
* Some individual tests are now skipped when ``--pgo`` is used. The tests |
| 1255 |
in question increased the PGO task time significantly and likely |
| 1256 |
didn't help improve optimization of the final executable. This |
| 1257 |
speeds up the task by a factor of about 15x. Running the full unit test |
| 1258 |
suite is slow. This change may result in a slightly less optimized build |
| 1259 |
since not as many code branches will be executed. If you are willing to |
| 1260 |
wait for the much slower build, the old behavior can be restored using |
| 1261 |
``./configure [..] PROFILE_TASK="-m test --pgo-extended"``. We make no |
| 1262 |
guarantees as to which PGO task set produces a faster build. Users who care |
| 1263 |
should run their own relevant benchmarks as results can depend on the |
| 1264 |
environment, workload, and compiler tool chain. |
| 1265 |
(See :issue:`36044` and :issue:`37707` for more details.) |
| 1266 |
|
| 1267 |
|
| 1268 |
C API Changes |
| 1269 |
============= |
| 1270 |
|
| 1271 |
New Features |
| 1272 |
------------ |
| 1273 |
|
| 1274 |
* :pep:`573`: Added :c:func:`PyType_FromModuleAndSpec` to associate |
| 1275 |
a module with a class; :c:func:`PyType_GetModule` and |
| 1276 |
:c:func:`PyType_GetModuleState` to retrieve the module and its state; and |
| 1277 |
:c:type:`PyCMethod` and :c:macro:`METH_METHOD` to allow a method to |
| 1278 |
access the class it was defined in. |
| 1279 |
(Contributed by Marcel Plch and Petr Viktorin in :issue:`38787`.) |
| 1280 |
|
| 1281 |
* Added :c:func:`PyFrame_GetCode` function: get a frame code. |
| 1282 |
Added :c:func:`PyFrame_GetBack` function: get the frame next outer frame. |
| 1283 |
(Contributed by Victor Stinner in :issue:`40421`.) |
| 1284 |
|
| 1285 |
* Added :c:func:`PyFrame_GetLineNumber` to the limited C API. |
| 1286 |
(Contributed by Victor Stinner in :issue:`40421`.) |
| 1287 |
|
| 1288 |
* Added :c:func:`PyThreadState_GetInterpreter` and |
| 1289 |
:c:func:`PyInterpreterState_Get` functions to get the interpreter. |
| 1290 |
Added :c:func:`PyThreadState_GetFrame` function to get the current frame of a |
| 1291 |
Python thread state. |
| 1292 |
Added :c:func:`PyThreadState_GetID` function: get the unique identifier of a |
| 1293 |
Python thread state. |
| 1294 |
(Contributed by Victor Stinner in :issue:`39947`.) |
| 1295 |
|
| 1296 |
* Added a new public :c:func:`PyObject_CallNoArgs` function to the C API, which |
| 1297 |
calls a callable Python object without any arguments. It is the most efficient |
| 1298 |
way to call a callable Python object without any argument. |
| 1299 |
(Contributed by Victor Stinner in :issue:`37194`.) |
| 1300 |
|
| 1301 |
* Changes in the limited C API (if ``Py_LIMITED_API`` macro is defined): |
| 1302 |
|
| 1303 |
* Provide :c:func:`Py_EnterRecursiveCall` and :c:func:`Py_LeaveRecursiveCall` |
| 1304 |
as regular functions for the limited API. Previously, there were defined as |
| 1305 |
macros, but these macros didn't compile with the limited C API which cannot |
| 1306 |
access ``PyThreadState.recursion_depth`` field (the structure is opaque in |
| 1307 |
the limited C API). |
| 1308 |
|
| 1309 |
* ``PyObject_INIT()`` and ``PyObject_INIT_VAR()`` become regular "opaque" |
| 1310 |
function to hide implementation details. |
| 1311 |
|
| 1312 |
(Contributed by Victor Stinner in :issue:`38644` and :issue:`39542`.) |
| 1313 |
|
| 1314 |
* The :c:func:`PyModule_AddType` function is added to help adding a type |
| 1315 |
to a module. |
| 1316 |
(Contributed by Donghee Na in :issue:`40024`.) |
| 1317 |
|
| 1318 |
* Added the functions :c:func:`PyObject_GC_IsTracked` and |
| 1319 |
:c:func:`PyObject_GC_IsFinalized` to the public API to allow to query if |
| 1320 |
Python objects are being currently tracked or have been already finalized by |
| 1321 |
the garbage collector respectively. |
| 1322 |
(Contributed by Pablo Galindo Salgado in :issue:`40241`.) |
| 1323 |
|
| 1324 |
* Added :c:func:`!_PyObject_FunctionStr` to get a user-friendly string |
| 1325 |
representation of a function-like object. |
| 1326 |
(Patch by Jeroen Demeyer in :issue:`37645`.) |
| 1327 |
|
| 1328 |
* Added :c:func:`PyObject_CallOneArg` for calling an object with one |
| 1329 |
positional argument |
| 1330 |
(Patch by Jeroen Demeyer in :issue:`37483`.) |
| 1331 |
|
| 1332 |
|
| 1333 |
Porting to Python 3.9 |
| 1334 |
--------------------- |
| 1335 |
|
| 1336 |
* ``PyInterpreterState.eval_frame`` (:pep:`523`) now requires a new mandatory |
| 1337 |
*tstate* parameter (``PyThreadState*``). |
| 1338 |
(Contributed by Victor Stinner in :issue:`38500`.) |
| 1339 |
|
| 1340 |
* Extension modules: :c:member:`~PyModuleDef.m_traverse`, |
| 1341 |
:c:member:`~PyModuleDef.m_clear` and :c:member:`~PyModuleDef.m_free` |
| 1342 |
functions of :c:type:`PyModuleDef` are no longer called if the module state |
| 1343 |
was requested but is not allocated yet. This is the case immediately after |
| 1344 |
the module is created and before the module is executed |
| 1345 |
(:c:data:`Py_mod_exec` function). More precisely, these functions are not called |
| 1346 |
if :c:member:`~PyModuleDef.m_size` is greater than 0 and the module state (as |
| 1347 |
returned by :c:func:`PyModule_GetState`) is ``NULL``. |
| 1348 |
|
| 1349 |
Extension modules without module state (``m_size <= 0``) are not affected. |
| 1350 |
|
| 1351 |
* If :c:func:`Py_AddPendingCall` is called in a subinterpreter, the function is |
| 1352 |
now scheduled to be called from the subinterpreter, rather than being called |
| 1353 |
from the main interpreter. Each subinterpreter now has its own list of |
| 1354 |
scheduled calls. |
| 1355 |
(Contributed by Victor Stinner in :issue:`39984`.) |
| 1356 |
|
| 1357 |
* The Windows registry is no longer used to initialize :data:`sys.path` when |
| 1358 |
the ``-E`` option is used (if :c:member:`PyConfig.use_environment` is set to |
| 1359 |
``0``). This is significant when embedding Python on Windows. |
| 1360 |
(Contributed by Zackery Spytz in :issue:`8901`.) |
| 1361 |
|
| 1362 |
* The global variable :c:data:`PyStructSequence_UnnamedField` is now a constant |
| 1363 |
and refers to a constant string. |
| 1364 |
(Contributed by Serhiy Storchaka in :issue:`38650`.) |
| 1365 |
|
| 1366 |
* The :c:type:`!PyGC_Head` structure is now opaque. It is only defined in the |
| 1367 |
internal C API (``pycore_gc.h``). |
| 1368 |
(Contributed by Victor Stinner in :issue:`40241`.) |
| 1369 |
|
| 1370 |
* The ``Py_UNICODE_COPY``, ``Py_UNICODE_FILL``, ``PyUnicode_WSTR_LENGTH``, |
| 1371 |
:c:func:`!PyUnicode_FromUnicode`, :c:func:`!PyUnicode_AsUnicode`, |
| 1372 |
``_PyUnicode_AsUnicode``, and :c:func:`!PyUnicode_AsUnicodeAndSize` are |
| 1373 |
marked as deprecated in C. They have been deprecated by :pep:`393` since |
| 1374 |
Python 3.3. |
| 1375 |
(Contributed by Inada Naoki in :issue:`36346`.) |
| 1376 |
|
| 1377 |
* The :c:func:`Py_FatalError` function is replaced with a macro which logs |
| 1378 |
automatically the name of the current function, unless the |
| 1379 |
``Py_LIMITED_API`` macro is defined. |
| 1380 |
(Contributed by Victor Stinner in :issue:`39882`.) |
| 1381 |
|
| 1382 |
* The vectorcall protocol now requires that the caller passes only strings as |
| 1383 |
keyword names. (See :issue:`37540` for more information.) |
| 1384 |
|
| 1385 |
* Implementation details of a number of macros and functions are now hidden: |
| 1386 |
|
| 1387 |
* :c:func:`PyObject_IS_GC` macro was converted to a function. |
| 1388 |
|
| 1389 |
* The :c:func:`!PyObject_NEW` macro becomes an alias to the |
| 1390 |
:c:macro:`PyObject_New` macro, and the :c:func:`!PyObject_NEW_VAR` macro |
| 1391 |
becomes an alias to the :c:macro:`PyObject_NewVar` macro. They no longer |
| 1392 |
access directly the :c:member:`PyTypeObject.tp_basicsize` member. |
| 1393 |
|
| 1394 |
* :c:func:`!PyObject_GET_WEAKREFS_LISTPTR` macro was converted to a function: |
| 1395 |
the macro accessed directly the :c:member:`PyTypeObject.tp_weaklistoffset` |
| 1396 |
member. |
| 1397 |
|
| 1398 |
* :c:func:`PyObject_CheckBuffer` macro was converted to a function: the macro |
| 1399 |
accessed directly the :c:member:`PyTypeObject.tp_as_buffer` member. |
| 1400 |
|
| 1401 |
* :c:func:`PyIndex_Check` is now always declared as an opaque function to hide |
| 1402 |
implementation details: removed the ``PyIndex_Check()`` macro. The macro accessed |
| 1403 |
directly the :c:member:`PyTypeObject.tp_as_number` member. |
| 1404 |
|
| 1405 |
(See :issue:`40170` for more details.) |
| 1406 |
|
| 1407 |
Removed |
| 1408 |
------- |
| 1409 |
|
| 1410 |
* Excluded ``PyFPE_START_PROTECT()`` and ``PyFPE_END_PROTECT()`` macros of |
| 1411 |
``pyfpe.h`` from the limited C API. |
| 1412 |
(Contributed by Victor Stinner in :issue:`38835`.) |
| 1413 |
|
| 1414 |
* The ``tp_print`` slot of :ref:`PyTypeObject <type-structs>` has been removed. |
| 1415 |
It was used for printing objects to files in Python 2.7 and before. Since |
| 1416 |
Python 3.0, it has been ignored and unused. |
| 1417 |
(Contributed by Jeroen Demeyer in :issue:`36974`.) |
| 1418 |
|
| 1419 |
* Changes in the limited C API (if ``Py_LIMITED_API`` macro is defined): |
| 1420 |
|
| 1421 |
* Excluded the following functions from the limited C API: |
| 1422 |
|
| 1423 |
* ``PyThreadState_DeleteCurrent()`` |
| 1424 |
(Contributed by Joannah Nanjekye in :issue:`37878`.) |
| 1425 |
* ``_Py_CheckRecursionLimit`` |
| 1426 |
* ``_Py_NewReference()`` |
| 1427 |
* ``_Py_ForgetReference()`` |
| 1428 |
* ``_PyTraceMalloc_NewReference()`` |
| 1429 |
* ``_Py_GetRefTotal()`` |
| 1430 |
* The trashcan mechanism which never worked in the limited C API. |
| 1431 |
* ``PyTrash_UNWIND_LEVEL`` |
| 1432 |
* ``Py_TRASHCAN_BEGIN_CONDITION`` |
| 1433 |
* ``Py_TRASHCAN_BEGIN`` |
| 1434 |
* ``Py_TRASHCAN_END`` |
| 1435 |
* ``Py_TRASHCAN_SAFE_BEGIN`` |
| 1436 |
* ``Py_TRASHCAN_SAFE_END`` |
| 1437 |
|
| 1438 |
* Moved following functions and definitions to the internal C API: |
| 1439 |
|
| 1440 |
* ``_PyDebug_PrintTotalRefs()`` |
| 1441 |
* ``_Py_PrintReferences()`` |
| 1442 |
* ``_Py_PrintReferenceAddresses()`` |
| 1443 |
* ``_Py_tracemalloc_config`` |
| 1444 |
* ``_Py_AddToAllObjects()`` (specific to ``Py_TRACE_REFS`` build) |
| 1445 |
|
| 1446 |
(Contributed by Victor Stinner in :issue:`38644` and :issue:`39542`.) |
| 1447 |
|
| 1448 |
* Removed ``_PyRuntime.getframe`` hook and removed ``_PyThreadState_GetFrame`` |
| 1449 |
macro which was an alias to ``_PyRuntime.getframe``. They were only exposed |
| 1450 |
by the internal C API. Removed also ``PyThreadFrameGetter`` type. |
| 1451 |
(Contributed by Victor Stinner in :issue:`39946`.) |
| 1452 |
|
| 1453 |
* Removed the following functions from the C API. Call :c:func:`PyGC_Collect` |
| 1454 |
explicitly to clear all free lists. |
| 1455 |
(Contributed by Inada Naoki and Victor Stinner in :issue:`37340`, |
| 1456 |
:issue:`38896` and :issue:`40428`.) |
| 1457 |
|
| 1458 |
* ``PyAsyncGen_ClearFreeLists()`` |
| 1459 |
* ``PyContext_ClearFreeList()`` |
| 1460 |
* ``PyDict_ClearFreeList()`` |
| 1461 |
* ``PyFloat_ClearFreeList()`` |
| 1462 |
* ``PyFrame_ClearFreeList()`` |
| 1463 |
* ``PyList_ClearFreeList()`` |
| 1464 |
* ``PyMethod_ClearFreeList()`` and ``PyCFunction_ClearFreeList()``: |
| 1465 |
the free lists of bound method objects have been removed. |
| 1466 |
* ``PySet_ClearFreeList()``: the set free list has been removed |
| 1467 |
in Python 3.4. |
| 1468 |
* ``PyTuple_ClearFreeList()`` |
| 1469 |
* ``PyUnicode_ClearFreeList()``: the Unicode free list has been removed in |
| 1470 |
Python 3.3. |
| 1471 |
|
| 1472 |
* Removed ``_PyUnicode_ClearStaticStrings()`` function. |
| 1473 |
(Contributed by Victor Stinner in :issue:`39465`.) |
| 1474 |
|
| 1475 |
* Removed ``Py_UNICODE_MATCH``. It has been deprecated by :pep:`393`, and |
| 1476 |
broken since Python 3.3. The :c:func:`PyUnicode_Tailmatch` function can be |
| 1477 |
used instead. |
| 1478 |
(Contributed by Inada Naoki in :issue:`36346`.) |
| 1479 |
|
| 1480 |
* Cleaned header files of interfaces defined but with no implementation. |
| 1481 |
The public API symbols being removed are: |
| 1482 |
``_PyBytes_InsertThousandsGroupingLocale``, |
| 1483 |
``_PyBytes_InsertThousandsGrouping``, ``_Py_InitializeFromArgs``, |
| 1484 |
``_Py_InitializeFromWideArgs``, ``_PyFloat_Repr``, ``_PyFloat_Digits``, |
| 1485 |
``_PyFloat_DigitsInit``, ``PyFrame_ExtendStack``, ``_PyAIterWrapper_Type``, |
| 1486 |
``PyNullImporter_Type``, ``PyCmpWrapper_Type``, ``PySortWrapper_Type``, |
| 1487 |
``PyNoArgsFunction``. |
| 1488 |
(Contributed by Pablo Galindo Salgado in :issue:`39372`.) |
| 1489 |
|
| 1490 |
Notable changes in Python 3.9.1 |
| 1491 |
=============================== |
| 1492 |
|
| 1493 |
typing |
| 1494 |
------ |
| 1495 |
|
| 1496 |
The behavior of :class:`typing.Literal` was changed to conform with :pep:`586` |
| 1497 |
and to match the behavior of static type checkers specified in the PEP. |
| 1498 |
|
| 1499 |
1. ``Literal`` now de-duplicates parameters. |
| 1500 |
2. Equality comparisons between ``Literal`` objects are now order independent. |
| 1501 |
3. ``Literal`` comparisons now respect types. For example, |
| 1502 |
``Literal[0] == Literal[False]`` previously evaluated to ``True``. It is |
| 1503 |
now ``False``. To support this change, the internally used type cache now |
| 1504 |
supports differentiating types. |
| 1505 |
4. ``Literal`` objects will now raise a :exc:`TypeError` exception during |
| 1506 |
equality comparisons if any of their parameters are not :term:`hashable`. |
| 1507 |
Note that declaring ``Literal`` with mutable parameters will not throw |
| 1508 |
an error:: |
| 1509 |
|
| 1510 |
>>> from typing import Literal |
| 1511 |
>>> Literal[{0}] |
| 1512 |
>>> Literal[{0}] == Literal[{False}] |
| 1513 |
Traceback (most recent call last): |
| 1514 |
File "<stdin>", line 1, in <module> |
| 1515 |
TypeError: unhashable type: 'set' |
| 1516 |
|
| 1517 |
(Contributed by Yurii Karabas in :issue:`42345`.) |
| 1518 |
|
| 1519 |
macOS 11.0 (Big Sur) and Apple Silicon Mac support |
| 1520 |
-------------------------------------------------- |
| 1521 |
|
| 1522 |
As of 3.9.1, Python now fully supports building and running on macOS 11.0 |
| 1523 |
(Big Sur) and on Apple Silicon Macs (based on the ``ARM64`` architecture). |
| 1524 |
A new universal build variant, ``universal2``, is now available to natively |
| 1525 |
support both ``ARM64`` and ``Intel 64`` in one set of executables. Binaries |
| 1526 |
can also now be built on current versions of macOS to be deployed on a range |
| 1527 |
of older macOS versions (tested to 10.9) while making some newer OS |
| 1528 |
functions and options conditionally available based on the operating system |
| 1529 |
version in use at runtime ("weaklinking"). |
| 1530 |
|
| 1531 |
(Contributed by Ronald Oussoren and Lawrence D'Anna in :issue:`41100`.) |
| 1532 |
|
| 1533 |
Notable changes in Python 3.9.2 |
| 1534 |
=============================== |
| 1535 |
|
| 1536 |
collections.abc |
| 1537 |
--------------- |
| 1538 |
|
| 1539 |
:class:`collections.abc.Callable` generic now flattens type parameters, similar |
| 1540 |
to what :data:`typing.Callable` currently does. This means that |
| 1541 |
``collections.abc.Callable[[int, str], str]`` will have ``__args__`` of |
| 1542 |
``(int, str, str)``; previously this was ``([int, str], str)``. To allow this |
| 1543 |
change, :class:`types.GenericAlias` can now be subclassed, and a subclass will |
| 1544 |
be returned when subscripting the :class:`collections.abc.Callable` type. |
| 1545 |
Code which accesses the arguments via :func:`typing.get_args` or ``__args__`` |
| 1546 |
need to account for this change. A :exc:`DeprecationWarning` may be emitted for |
| 1547 |
invalid forms of parameterizing :class:`collections.abc.Callable` which may have |
| 1548 |
passed silently in Python 3.9.1. This :exc:`DeprecationWarning` will |
| 1549 |
become a :exc:`TypeError` in Python 3.10. |
| 1550 |
(Contributed by Ken Jin in :issue:`42195`.) |
| 1551 |
|
| 1552 |
urllib.parse |
| 1553 |
------------ |
| 1554 |
|
| 1555 |
Earlier Python versions allowed using both ``;`` and ``&`` as |
| 1556 |
query parameter separators in :func:`urllib.parse.parse_qs` and |
| 1557 |
:func:`urllib.parse.parse_qsl`. Due to security concerns, and to conform with |
| 1558 |
newer W3C recommendations, this has been changed to allow only a single |
| 1559 |
separator key, with ``&`` as the default. This change also affects |
| 1560 |
:func:`!cgi.parse` and :func:`!cgi.parse_multipart` as they use the affected |
| 1561 |
functions internally. For more details, please see their respective |
| 1562 |
documentation. |
| 1563 |
(Contributed by Adam Goldschmidt, Senthil Kumaran and Ken Jin in :issue:`42967`.) |
| 1564 |
|
| 1565 |
Notable changes in Python 3.9.3 |
| 1566 |
=============================== |
| 1567 |
|
| 1568 |
A security fix alters the :class:`ftplib.FTP` behavior to not trust the |
| 1569 |
IPv4 address sent from the remote server when setting up a passive data |
| 1570 |
channel. We reuse the ftp server IP address instead. For unusual code |
| 1571 |
requiring the old behavior, set a ``trust_server_pasv_ipv4_address`` |
| 1572 |
attribute on your FTP instance to ``True``. (See :gh:`87451`) |
| 1573 |
|
| 1574 |
Notable changes in Python 3.9.5 |
| 1575 |
=============================== |
| 1576 |
|
| 1577 |
urllib.parse |
| 1578 |
------------ |
| 1579 |
|
| 1580 |
The presence of newline or tab characters in parts of a URL allows for some |
| 1581 |
forms of attacks. Following the WHATWG specification that updates :rfc:`3986`, |
| 1582 |
ASCII newline ``\n``, ``\r`` and tab ``\t`` characters are stripped from the |
| 1583 |
URL by the parser in :mod:`urllib.parse` preventing such attacks. The removal |
| 1584 |
characters are controlled by a new module level variable |
| 1585 |
``urllib.parse._UNSAFE_URL_BYTES_TO_REMOVE``. (See :gh:`88048`) |
| 1586 |
|
| 1587 |
Notable security feature in 3.9.14 |
| 1588 |
================================== |
| 1589 |
|
| 1590 |
Converting between :class:`int` and :class:`str` in bases other than 2 |
| 1591 |
(binary), 4, 8 (octal), 16 (hexadecimal), or 32 such as base 10 (decimal) |
| 1592 |
now raises a :exc:`ValueError` if the number of digits in string form is |
| 1593 |
above a limit to avoid potential denial of service attacks due to the |
| 1594 |
algorithmic complexity. This is a mitigation for :cve:`2020-10735`. |
| 1595 |
This limit can be configured or disabled by environment variable, command |
| 1596 |
line flag, or :mod:`sys` APIs. See the :ref:`integer string conversion |
| 1597 |
length limitation <int_max_str_digits>` documentation. The default limit |
| 1598 |
is 4300 digits in string form. |
| 1599 |
|
| 1600 |
Notable changes in 3.9.17 |
| 1601 |
========================= |
| 1602 |
|
| 1603 |
tarfile |
| 1604 |
------- |
| 1605 |
|
| 1606 |
* The extraction methods in :mod:`tarfile`, and :func:`shutil.unpack_archive`, |
| 1607 |
have a new a *filter* argument that allows limiting tar features than may be |
| 1608 |
surprising or dangerous, such as creating files outside the destination |
| 1609 |
directory. |
| 1610 |
See :ref:`tarfile-extraction-filter` for details. |
| 1611 |
In Python 3.12, use without the *filter* argument will show a |
| 1612 |
:exc:`DeprecationWarning`. |
| 1613 |
In Python 3.14, the default will switch to ``'data'``. |
| 1614 |
(Contributed by Petr Viktorin in :pep:`706`.) |