| 1 |
**************************** |
| 2 |
What's New In Python 3.11 |
| 3 |
**************************** |
| 4 |
|
| 5 |
:Editor: Pablo Galindo Salgado |
| 6 |
|
| 7 |
.. Rules for maintenance: |
| 8 |
|
| 9 |
* Anyone can add text to this document. Do not spend very much time |
| 10 |
on the wording of your changes, because your text will probably |
| 11 |
get rewritten to some degree. |
| 12 |
|
| 13 |
* The maintainer will go through Misc/NEWS periodically and add |
| 14 |
changes; it's therefore more important to add your changes to |
| 15 |
Misc/NEWS than to this file. |
| 16 |
|
| 17 |
* This is not a complete list of every single change; completeness |
| 18 |
is the purpose of Misc/NEWS. Some changes I consider too small |
| 19 |
or esoteric to include. If such a change is added to the text, |
| 20 |
I'll just remove it. (This is another reason you shouldn't spend |
| 21 |
too much time on writing your addition.) |
| 22 |
|
| 23 |
* If you want to draw your new text to the attention of the |
| 24 |
maintainer, add 'XXX' to the beginning of the paragraph or |
| 25 |
section. |
| 26 |
|
| 27 |
* It's OK to just add a fragmentary note about a change. For |
| 28 |
example: "XXX Describe the transmogrify() function added to the |
| 29 |
socket module." The maintainer will research the change and |
| 30 |
write the necessary text. |
| 31 |
|
| 32 |
* You can comment out your additions if you like, but it's not |
| 33 |
necessary (especially when a final release is some months away). |
| 34 |
|
| 35 |
* Credit the author of a patch or bugfix. Just the name is |
| 36 |
sufficient; the e-mail address isn't necessary. |
| 37 |
|
| 38 |
* It's helpful to add the bug/patch number as a comment: |
| 39 |
|
| 40 |
XXX Describe the transmogrify() function added to the socket |
| 41 |
module. |
| 42 |
(Contributed by P.Y. Developer in :issue:`12345`.) |
| 43 |
|
| 44 |
This saves the maintainer the effort of going through the Mercurial log |
| 45 |
when researching a change. |
| 46 |
|
| 47 |
This article explains the new features in Python 3.11, compared to 3.10. |
| 48 |
Python 3.11 was released on October 24, 2022. |
| 49 |
For full details, see the :ref:`changelog <changelog>`. |
| 50 |
|
| 51 |
|
| 52 |
.. _whatsnew311-summary: |
| 53 |
|
| 54 |
Summary -- Release highlights |
| 55 |
============================= |
| 56 |
|
| 57 |
.. This section singles out the most important changes in Python 3.11. |
| 58 |
Brevity is key. |
| 59 |
|
| 60 |
* Python 3.11 is between 10-60% faster than Python 3.10. |
| 61 |
On average, we measured a 1.25x speedup on the standard benchmark suite. |
| 62 |
See :ref:`whatsnew311-faster-cpython` for details. |
| 63 |
|
| 64 |
.. PEP-sized items next. |
| 65 |
|
| 66 |
New syntax features: |
| 67 |
|
| 68 |
* :ref:`whatsnew311-pep654` |
| 69 |
|
| 70 |
New built-in features: |
| 71 |
|
| 72 |
* :ref:`whatsnew311-pep678` |
| 73 |
|
| 74 |
New standard library modules: |
| 75 |
|
| 76 |
* :pep:`680`: :mod:`tomllib` — |
| 77 |
Support for parsing `TOML <https://toml.io/>`_ in the Standard Library |
| 78 |
|
| 79 |
Interpreter improvements: |
| 80 |
|
| 81 |
* :ref:`whatsnew311-pep657` |
| 82 |
* New :option:`-P` command line option and :envvar:`PYTHONSAFEPATH` environment |
| 83 |
variable to :ref:`disable automatically prepending potentially unsafe paths |
| 84 |
<whatsnew311-pythonsafepath>` to :data:`sys.path` |
| 85 |
|
| 86 |
New typing features: |
| 87 |
|
| 88 |
* :ref:`whatsnew311-pep646` |
| 89 |
* :ref:`whatsnew311-pep655` |
| 90 |
* :ref:`whatsnew311-pep673` |
| 91 |
* :ref:`whatsnew311-pep675` |
| 92 |
* :ref:`whatsnew311-pep681` |
| 93 |
|
| 94 |
Important deprecations, removals and restrictions: |
| 95 |
|
| 96 |
* :pep:`594`: |
| 97 |
:ref:`Many legacy standard library modules have been deprecated |
| 98 |
<whatsnew311-pep594>` and will be removed in Python 3.13 |
| 99 |
* :pep:`624`: |
| 100 |
:ref:`Py_UNICODE encoder APIs have been removed <whatsnew311-pep624>` |
| 101 |
* :pep:`670`: |
| 102 |
:ref:`Macros converted to static inline functions <whatsnew311-pep670>` |
| 103 |
|
| 104 |
|
| 105 |
.. _whatsnew311-features: |
| 106 |
|
| 107 |
New Features |
| 108 |
============ |
| 109 |
|
| 110 |
.. _whatsnew311-pep657: |
| 111 |
|
| 112 |
PEP 657: Fine-grained error locations in tracebacks |
| 113 |
--------------------------------------------------- |
| 114 |
|
| 115 |
When printing tracebacks, the interpreter will now point to the exact expression |
| 116 |
that caused the error, instead of just the line. For example: |
| 117 |
|
| 118 |
.. code-block:: python |
| 119 |
|
| 120 |
Traceback (most recent call last): |
| 121 |
File "distance.py", line 11, in <module> |
| 122 |
print(manhattan_distance(p1, p2)) |
| 123 |
^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 124 |
File "distance.py", line 6, in manhattan_distance |
| 125 |
return abs(point_1.x - point_2.x) + abs(point_1.y - point_2.y) |
| 126 |
^^^^^^^^^ |
| 127 |
AttributeError: 'NoneType' object has no attribute 'x' |
| 128 |
|
| 129 |
Previous versions of the interpreter would point to just the line, making it |
| 130 |
ambiguous which object was ``None``. These enhanced errors can also be helpful |
| 131 |
when dealing with deeply nested :class:`dict` objects and multiple function calls: |
| 132 |
|
| 133 |
.. code-block:: python |
| 134 |
|
| 135 |
Traceback (most recent call last): |
| 136 |
File "query.py", line 37, in <module> |
| 137 |
magic_arithmetic('foo') |
| 138 |
File "query.py", line 18, in magic_arithmetic |
| 139 |
return add_counts(x) / 25 |
| 140 |
^^^^^^^^^^^^^ |
| 141 |
File "query.py", line 24, in add_counts |
| 142 |
return 25 + query_user(user1) + query_user(user2) |
| 143 |
^^^^^^^^^^^^^^^^^ |
| 144 |
File "query.py", line 32, in query_user |
| 145 |
return 1 + query_count(db, response['a']['b']['c']['user'], retry=True) |
| 146 |
~~~~~~~~~~~~~~~~~~^^^^^ |
| 147 |
TypeError: 'NoneType' object is not subscriptable |
| 148 |
|
| 149 |
As well as complex arithmetic expressions: |
| 150 |
|
| 151 |
.. code-block:: python |
| 152 |
|
| 153 |
Traceback (most recent call last): |
| 154 |
File "calculation.py", line 54, in <module> |
| 155 |
result = (x / y / z) * (a / b / c) |
| 156 |
~~~~~~^~~ |
| 157 |
ZeroDivisionError: division by zero |
| 158 |
|
| 159 |
Additionally, the information used by the enhanced traceback feature |
| 160 |
is made available via a general API, that can be used to correlate |
| 161 |
:term:`bytecode` :ref:`instructions <bytecodes>` with source code location. |
| 162 |
This information can be retrieved using: |
| 163 |
|
| 164 |
- The :meth:`codeobject.co_positions` method in Python. |
| 165 |
- The :c:func:`PyCode_Addr2Location` function in the C API. |
| 166 |
|
| 167 |
See :pep:`657` for more details. (Contributed by Pablo Galindo, Batuhan Taskaya |
| 168 |
and Ammar Askar in :issue:`43950`.) |
| 169 |
|
| 170 |
.. note:: |
| 171 |
This feature requires storing column positions in :ref:`codeobjects`, |
| 172 |
which may result in a small increase in interpreter memory usage |
| 173 |
and disk usage for compiled Python files. |
| 174 |
To avoid storing the extra information |
| 175 |
and deactivate printing the extra traceback information, |
| 176 |
use the :option:`-X no_debug_ranges <-X>` command line option |
| 177 |
or the :envvar:`PYTHONNODEBUGRANGES` environment variable. |
| 178 |
|
| 179 |
|
| 180 |
.. _whatsnew311-pep654: |
| 181 |
|
| 182 |
PEP 654: Exception Groups and ``except*`` |
| 183 |
----------------------------------------- |
| 184 |
|
| 185 |
:pep:`654` introduces language features that enable a program |
| 186 |
to raise and handle multiple unrelated exceptions simultaneously. |
| 187 |
The builtin types :exc:`ExceptionGroup` and :exc:`BaseExceptionGroup` |
| 188 |
make it possible to group exceptions and raise them together, |
| 189 |
and the new :keyword:`except* <except_star>` syntax generalizes |
| 190 |
:keyword:`except` to match subgroups of exception groups. |
| 191 |
|
| 192 |
See :pep:`654` for more details. |
| 193 |
|
| 194 |
(Contributed by Irit Katriel in :issue:`45292`. PEP written by |
| 195 |
Irit Katriel, Yury Selivanov and Guido van Rossum.) |
| 196 |
|
| 197 |
|
| 198 |
.. _whatsnew311-pep678: |
| 199 |
|
| 200 |
PEP 678: Exceptions can be enriched with notes |
| 201 |
---------------------------------------------- |
| 202 |
|
| 203 |
The :meth:`~BaseException.add_note` method is added to :exc:`BaseException`. |
| 204 |
It can be used to enrich exceptions with context information |
| 205 |
that is not available at the time when the exception is raised. |
| 206 |
The added notes appear in the default traceback. |
| 207 |
|
| 208 |
See :pep:`678` for more details. |
| 209 |
|
| 210 |
(Contributed by Irit Katriel in :issue:`45607`. |
| 211 |
PEP written by Zac Hatfield-Dodds.) |
| 212 |
|
| 213 |
|
| 214 |
.. _whatsnew311-windows-launcher: |
| 215 |
|
| 216 |
Windows ``py.exe`` launcher improvements |
| 217 |
---------------------------------------- |
| 218 |
|
| 219 |
The copy of the :ref:`launcher` included with Python 3.11 has been significantly |
| 220 |
updated. It now supports company/tag syntax as defined in :pep:`514` using the |
| 221 |
:samp:`-V:{<company>}/{<tag>}` argument instead of the limited :samp:`-{<major>}.{<minor>}`. |
| 222 |
This allows launching distributions other than ``PythonCore``, |
| 223 |
the one hosted on `python.org <https://www.python.org>`_. |
| 224 |
|
| 225 |
When using ``-V:`` selectors, either company or tag can be omitted, but all |
| 226 |
installs will be searched. For example, ``-V:OtherPython/`` will select the |
| 227 |
"best" tag registered for ``OtherPython``, while ``-V:3.11`` or ``-V:/3.11`` |
| 228 |
will select the "best" distribution with tag ``3.11``. |
| 229 |
|
| 230 |
When using the legacy :samp:`-{<major>}`, :samp:`-{<major>}.{<minor>}`, |
| 231 |
:samp:`-{<major>}-{<bitness>}` or :samp:`-{<major>}.{<minor>}-{<bitness>}` arguments, |
| 232 |
all existing behaviour should be preserved from past versions, |
| 233 |
and only releases from ``PythonCore`` will be selected. |
| 234 |
However, the ``-64`` suffix now implies "not 32-bit" (not necessarily x86-64), |
| 235 |
as there are multiple supported 64-bit platforms. |
| 236 |
32-bit runtimes are detected by checking the runtime's tag for a ``-32`` suffix. |
| 237 |
All releases of Python since 3.5 have included this in their 32-bit builds. |
| 238 |
|
| 239 |
|
| 240 |
.. _new-feat-related-type-hints-311: |
| 241 |
.. _whatsnew311-typing-features: |
| 242 |
|
| 243 |
New Features Related to Type Hints |
| 244 |
================================== |
| 245 |
|
| 246 |
This section covers major changes affecting :pep:`484` type hints and |
| 247 |
the :mod:`typing` module. |
| 248 |
|
| 249 |
|
| 250 |
.. _whatsnew311-pep646: |
| 251 |
|
| 252 |
PEP 646: Variadic generics |
| 253 |
-------------------------- |
| 254 |
|
| 255 |
:pep:`484` previously introduced :data:`~typing.TypeVar`, enabling creation |
| 256 |
of generics parameterised with a single type. :pep:`646` adds |
| 257 |
:data:`~typing.TypeVarTuple`, enabling parameterisation |
| 258 |
with an *arbitrary* number of types. In other words, |
| 259 |
a :data:`~typing.TypeVarTuple` is a *variadic* type variable, |
| 260 |
enabling *variadic* generics. |
| 261 |
|
| 262 |
This enables a wide variety of use cases. |
| 263 |
In particular, it allows the type of array-like structures |
| 264 |
in numerical computing libraries such as NumPy and TensorFlow to be |
| 265 |
parameterised with the array *shape*. Static type checkers will now |
| 266 |
be able to catch shape-related bugs in code that uses these libraries. |
| 267 |
|
| 268 |
See :pep:`646` for more details. |
| 269 |
|
| 270 |
(Contributed by Matthew Rahtz in :issue:`43224`, with contributions by |
| 271 |
Serhiy Storchaka and Jelle Zijlstra. PEP written by Mark Mendoza, Matthew |
| 272 |
Rahtz, Pradeep Kumar Srinivasan, and Vincent Siles.) |
| 273 |
|
| 274 |
|
| 275 |
.. _whatsnew311-pep655: |
| 276 |
|
| 277 |
PEP 655: Marking individual ``TypedDict`` items as required or not-required |
| 278 |
--------------------------------------------------------------------------- |
| 279 |
|
| 280 |
:data:`~typing.Required` and :data:`~typing.NotRequired` provide a |
| 281 |
straightforward way to mark whether individual items in a |
| 282 |
:class:`~typing.TypedDict` must be present. Previously, this was only possible |
| 283 |
using inheritance. |
| 284 |
|
| 285 |
All fields are still required by default, |
| 286 |
unless the *total* parameter is set to ``False``, |
| 287 |
in which case all fields are still not-required by default. |
| 288 |
For example, the following specifies a :class:`!TypedDict` |
| 289 |
with one required and one not-required key:: |
| 290 |
|
| 291 |
class Movie(TypedDict): |
| 292 |
title: str |
| 293 |
year: NotRequired[int] |
| 294 |
|
| 295 |
m1: Movie = {"title": "Black Panther", "year": 2018} # OK |
| 296 |
m2: Movie = {"title": "Star Wars"} # OK (year is not required) |
| 297 |
m3: Movie = {"year": 2022} # ERROR (missing required field title) |
| 298 |
|
| 299 |
The following definition is equivalent:: |
| 300 |
|
| 301 |
class Movie(TypedDict, total=False): |
| 302 |
title: Required[str] |
| 303 |
year: int |
| 304 |
|
| 305 |
See :pep:`655` for more details. |
| 306 |
|
| 307 |
(Contributed by David Foster and Jelle Zijlstra in :issue:`47087`. PEP |
| 308 |
written by David Foster.) |
| 309 |
|
| 310 |
|
| 311 |
.. _whatsnew311-pep673: |
| 312 |
|
| 313 |
PEP 673: ``Self`` type |
| 314 |
---------------------- |
| 315 |
|
| 316 |
The new :data:`~typing.Self` annotation provides a simple and intuitive |
| 317 |
way to annotate methods that return an instance of their class. This |
| 318 |
behaves the same as the :class:`~typing.TypeVar`-based approach |
| 319 |
:pep:`specified in PEP 484 <484#annotating-instance-and-class-methods>`, |
| 320 |
but is more concise and easier to follow. |
| 321 |
|
| 322 |
Common use cases include alternative constructors provided as |
| 323 |
:func:`classmethod <classmethod>`\s, |
| 324 |
and :meth:`~object.__enter__` methods that return ``self``:: |
| 325 |
|
| 326 |
class MyLock: |
| 327 |
def __enter__(self) -> Self: |
| 328 |
self.lock() |
| 329 |
return self |
| 330 |
|
| 331 |
... |
| 332 |
|
| 333 |
class MyInt: |
| 334 |
@classmethod |
| 335 |
def fromhex(cls, s: str) -> Self: |
| 336 |
return cls(int(s, 16)) |
| 337 |
|
| 338 |
... |
| 339 |
|
| 340 |
:data:`~typing.Self` can also be used to annotate method parameters |
| 341 |
or attributes of the same type as their enclosing class. |
| 342 |
|
| 343 |
See :pep:`673` for more details. |
| 344 |
|
| 345 |
(Contributed by James Hilton-Balfe in :issue:`46534`. PEP written by |
| 346 |
Pradeep Kumar Srinivasan and James Hilton-Balfe.) |
| 347 |
|
| 348 |
|
| 349 |
.. _whatsnew311-pep675: |
| 350 |
|
| 351 |
PEP 675: Arbitrary literal string type |
| 352 |
-------------------------------------- |
| 353 |
|
| 354 |
The new :data:`~typing.LiteralString` annotation may be used to indicate |
| 355 |
that a function parameter can be of any literal string type. This allows |
| 356 |
a function to accept arbitrary literal string types, as well as strings |
| 357 |
created from other literal strings. Type checkers can then |
| 358 |
enforce that sensitive functions, such as those that execute SQL |
| 359 |
statements or shell commands, are called only with static arguments, |
| 360 |
providing protection against injection attacks. |
| 361 |
|
| 362 |
For example, a SQL query function could be annotated as follows:: |
| 363 |
|
| 364 |
def run_query(sql: LiteralString) -> ... |
| 365 |
... |
| 366 |
|
| 367 |
def caller( |
| 368 |
arbitrary_string: str, |
| 369 |
query_string: LiteralString, |
| 370 |
table_name: LiteralString, |
| 371 |
) -> None: |
| 372 |
run_query("SELECT * FROM students") # ok |
| 373 |
run_query(query_string) # ok |
| 374 |
run_query("SELECT * FROM " + table_name) # ok |
| 375 |
run_query(arbitrary_string) # type checker error |
| 376 |
run_query( # type checker error |
| 377 |
f"SELECT * FROM students WHERE name = {arbitrary_string}" |
| 378 |
) |
| 379 |
|
| 380 |
See :pep:`675` for more details. |
| 381 |
|
| 382 |
(Contributed by Jelle Zijlstra in :issue:`47088`. PEP written by Pradeep |
| 383 |
Kumar Srinivasan and Graham Bleaney.) |
| 384 |
|
| 385 |
|
| 386 |
.. _whatsnew311-pep681: |
| 387 |
|
| 388 |
PEP 681: Data class transforms |
| 389 |
------------------------------ |
| 390 |
|
| 391 |
:data:`~typing.dataclass_transform` may be used to |
| 392 |
decorate a class, metaclass, or a function that is itself a decorator. |
| 393 |
The presence of ``@dataclass_transform()`` tells a static type checker that the |
| 394 |
decorated object performs runtime "magic" that transforms a class, |
| 395 |
giving it :func:`dataclass <dataclasses.dataclass>`-like behaviors. |
| 396 |
|
| 397 |
For example:: |
| 398 |
|
| 399 |
# The create_model decorator is defined by a library. |
| 400 |
@typing.dataclass_transform() |
| 401 |
def create_model(cls: Type[T]) -> Type[T]: |
| 402 |
cls.__init__ = ... |
| 403 |
cls.__eq__ = ... |
| 404 |
cls.__ne__ = ... |
| 405 |
return cls |
| 406 |
|
| 407 |
# The create_model decorator can now be used to create new model classes: |
| 408 |
@create_model |
| 409 |
class CustomerModel: |
| 410 |
id: int |
| 411 |
name: str |
| 412 |
|
| 413 |
c = CustomerModel(id=327, name="Eric Idle") |
| 414 |
|
| 415 |
See :pep:`681` for more details. |
| 416 |
|
| 417 |
(Contributed by Jelle Zijlstra in :gh:`91860`. PEP written by |
| 418 |
Erik De Bonte and Eric Traut.) |
| 419 |
|
| 420 |
|
| 421 |
.. _whatsnew311-pep563-deferred: |
| 422 |
|
| 423 |
PEP 563 may not be the future |
| 424 |
----------------------------- |
| 425 |
|
| 426 |
:pep:`563` Postponed Evaluation of Annotations |
| 427 |
(the ``from __future__ import annotations`` :ref:`future statement <future>`) |
| 428 |
that was originally planned for release in Python 3.10 |
| 429 |
has been put on hold indefinitely. |
| 430 |
See `this message from the Steering Council <https://mail.python.org/archives/list/[email protected]/message/VIZEBX5EYMSYIJNDBF6DMUMZOCWHARSO/>`__ |
| 431 |
for more information. |
| 432 |
|
| 433 |
|
| 434 |
.. _whatsnew311-other-lang-changes: |
| 435 |
|
| 436 |
Other Language Changes |
| 437 |
====================== |
| 438 |
|
| 439 |
* Starred unpacking expressions can now be used in :keyword:`for` statements. |
| 440 |
(See :issue:`46725` for more details.) |
| 441 |
|
| 442 |
* Asynchronous :ref:`comprehensions <comprehensions>` are now allowed |
| 443 |
inside comprehensions in :ref:`asynchronous functions <async def>`. |
| 444 |
Outer comprehensions implicitly become asynchronous in this case. |
| 445 |
(Contributed by Serhiy Storchaka in :issue:`33346`.) |
| 446 |
|
| 447 |
* A :exc:`TypeError` is now raised instead of an :exc:`AttributeError` in |
| 448 |
:keyword:`with` statements and :meth:`contextlib.ExitStack.enter_context` |
| 449 |
for objects that do not support the :term:`context manager` protocol, |
| 450 |
and in :keyword:`async with` statements and |
| 451 |
:meth:`contextlib.AsyncExitStack.enter_async_context` |
| 452 |
for objects not supporting the :term:`asynchronous context manager` protocol. |
| 453 |
(Contributed by Serhiy Storchaka in :issue:`12022` and :issue:`44471`.) |
| 454 |
|
| 455 |
* Added :meth:`object.__getstate__`, which provides the default |
| 456 |
implementation of the :meth:`!__getstate__` method. :mod:`copy`\ing |
| 457 |
and :mod:`pickle`\ing instances of subclasses of builtin types |
| 458 |
:class:`bytearray`, :class:`set`, :class:`frozenset`, |
| 459 |
:class:`collections.OrderedDict`, :class:`collections.deque`, |
| 460 |
:class:`weakref.WeakSet`, and :class:`datetime.tzinfo` now copies and |
| 461 |
pickles instance attributes implemented as :term:`slots <__slots__>`. |
| 462 |
This change has an unintended side effect: It trips up a small minority |
| 463 |
of existing Python projects not expecting :meth:`object.__getstate__` to |
| 464 |
exist. See the later comments on :gh:`70766` for discussions of what |
| 465 |
workarounds such code may need. |
| 466 |
(Contributed by Serhiy Storchaka in :issue:`26579`.) |
| 467 |
|
| 468 |
.. _whatsnew311-pythonsafepath: |
| 469 |
|
| 470 |
* Added a :option:`-P` command line option |
| 471 |
and a :envvar:`PYTHONSAFEPATH` environment variable, |
| 472 |
which disable the automatic prepending to :data:`sys.path` |
| 473 |
of the script's directory when running a script, |
| 474 |
or the current directory when using :option:`-c` and :option:`-m`. |
| 475 |
This ensures only stdlib and installed modules |
| 476 |
are picked up by :keyword:`import`, |
| 477 |
and avoids unintentionally or maliciously shadowing modules |
| 478 |
with those in a local (and typically user-writable) directory. |
| 479 |
(Contributed by Victor Stinner in :gh:`57684`.) |
| 480 |
|
| 481 |
* A ``"z"`` option was added to the :ref:`formatspec` that |
| 482 |
coerces negative to positive zero after rounding to the format precision. |
| 483 |
See :pep:`682` for more details. |
| 484 |
(Contributed by John Belmonte in :gh:`90153`.) |
| 485 |
|
| 486 |
* Bytes are no longer accepted on :data:`sys.path`. Support broke sometime |
| 487 |
between Python 3.2 and 3.6, with no one noticing until after Python 3.10.0 |
| 488 |
was released. In addition, bringing back support would be problematic due to |
| 489 |
interactions between :option:`-b` and :data:`sys.path_importer_cache` when |
| 490 |
there is a mixture of :class:`str` and :class:`bytes` keys. |
| 491 |
(Contributed by Thomas Grainger in :gh:`91181`.) |
| 492 |
|
| 493 |
|
| 494 |
.. _whatsnew311-other-implementation-changes: |
| 495 |
|
| 496 |
Other CPython Implementation Changes |
| 497 |
==================================== |
| 498 |
|
| 499 |
* The special methods :meth:`~object.__complex__` for :class:`complex` |
| 500 |
and :meth:`~object.__bytes__` for :class:`bytes` are implemented to support |
| 501 |
the :class:`typing.SupportsComplex` and :class:`typing.SupportsBytes` protocols. |
| 502 |
(Contributed by Mark Dickinson and Donghee Na in :issue:`24234`.) |
| 503 |
|
| 504 |
* ``siphash13`` is added as a new internal hashing algorithm. |
| 505 |
It has similar security properties as ``siphash24``, |
| 506 |
but it is slightly faster for long inputs. |
| 507 |
:class:`str`, :class:`bytes`, and some other types |
| 508 |
now use it as the default algorithm for :func:`hash`. |
| 509 |
:pep:`552` :ref:`hash-based .pyc files <pyc-invalidation>` |
| 510 |
now use ``siphash13`` too. |
| 511 |
(Contributed by Inada Naoki in :issue:`29410`.) |
| 512 |
|
| 513 |
* When an active exception is re-raised by a :keyword:`raise` statement with no parameters, |
| 514 |
the traceback attached to this exception is now always ``sys.exc_info()[1].__traceback__``. |
| 515 |
This means that changes made to the traceback in the current :keyword:`except` clause are |
| 516 |
reflected in the re-raised exception. |
| 517 |
(Contributed by Irit Katriel in :issue:`45711`.) |
| 518 |
|
| 519 |
* The interpreter state's representation of handled exceptions |
| 520 |
(aka ``exc_info`` or ``_PyErr_StackItem``) |
| 521 |
now only has the ``exc_value`` field; ``exc_type`` and ``exc_traceback`` |
| 522 |
have been removed, as they can be derived from ``exc_value``. |
| 523 |
(Contributed by Irit Katriel in :issue:`45711`.) |
| 524 |
|
| 525 |
* A new :ref:`command line option <install-quiet-option>`, ``AppendPath``, |
| 526 |
has been added for the Windows installer. |
| 527 |
It behaves similarly to ``PrependPath``, |
| 528 |
but appends the install and scripts directories instead of prepending them. |
| 529 |
(Contributed by Bastian Neuburger in :issue:`44934`.) |
| 530 |
|
| 531 |
* The :c:member:`PyConfig.module_search_paths_set` field must now be set to ``1`` for |
| 532 |
initialization to use :c:member:`PyConfig.module_search_paths` to initialize |
| 533 |
:data:`sys.path`. Otherwise, initialization will recalculate the path and replace |
| 534 |
any values added to ``module_search_paths``. |
| 535 |
|
| 536 |
* The output of the :option:`--help` option now fits in 50 lines/80 columns. |
| 537 |
Information about :ref:`Python environment variables <using-on-envvars>` |
| 538 |
and :option:`-X` options is now available using the respective |
| 539 |
:option:`--help-env` and :option:`--help-xoptions` flags, |
| 540 |
and with the new :option:`--help-all`. |
| 541 |
(Contributed by Éric Araujo in :issue:`46142`.) |
| 542 |
|
| 543 |
* Converting between :class:`int` and :class:`str` in bases other than 2 |
| 544 |
(binary), 4, 8 (octal), 16 (hexadecimal), or 32 such as base 10 (decimal) |
| 545 |
now raises a :exc:`ValueError` if the number of digits in string form is |
| 546 |
above a limit to avoid potential denial of service attacks due to the |
| 547 |
algorithmic complexity. This is a mitigation for :cve:`2020-10735`. |
| 548 |
This limit can be configured or disabled by environment variable, command |
| 549 |
line flag, or :mod:`sys` APIs. See the :ref:`integer string conversion |
| 550 |
length limitation <int_max_str_digits>` documentation. The default limit |
| 551 |
is 4300 digits in string form. |
| 552 |
|
| 553 |
|
| 554 |
.. _whatsnew311-new-modules: |
| 555 |
|
| 556 |
New Modules |
| 557 |
=========== |
| 558 |
|
| 559 |
* :mod:`tomllib`: For parsing `TOML <https://toml.io/>`_. |
| 560 |
See :pep:`680` for more details. |
| 561 |
(Contributed by Taneli Hukkinen in :issue:`40059`.) |
| 562 |
|
| 563 |
* :mod:`wsgiref.types`: |
| 564 |
:pep:`WSGI <3333>`-specific types for static type checking. |
| 565 |
(Contributed by Sebastian Rittau in :issue:`42012`.) |
| 566 |
|
| 567 |
|
| 568 |
.. _whatsnew311-improved-modules: |
| 569 |
|
| 570 |
Improved Modules |
| 571 |
================ |
| 572 |
|
| 573 |
.. _whatsnew311-asyncio: |
| 574 |
|
| 575 |
asyncio |
| 576 |
------- |
| 577 |
|
| 578 |
* Added the :class:`~asyncio.TaskGroup` class, |
| 579 |
an :ref:`asynchronous context manager <async-context-managers>` |
| 580 |
holding a group of tasks that will wait for all of them upon exit. |
| 581 |
For new code this is recommended over using |
| 582 |
:func:`~asyncio.create_task` and :func:`~asyncio.gather` directly. |
| 583 |
(Contributed by Yury Selivanov and others in :gh:`90908`.) |
| 584 |
|
| 585 |
* Added :func:`~asyncio.timeout`, an asynchronous context manager for |
| 586 |
setting a timeout on asynchronous operations. For new code this is |
| 587 |
recommended over using :func:`~asyncio.wait_for` directly. |
| 588 |
(Contributed by Andrew Svetlov in :gh:`90927`.) |
| 589 |
|
| 590 |
* Added the :class:`~asyncio.Runner` class, which exposes the machinery |
| 591 |
used by :func:`~asyncio.run`. |
| 592 |
(Contributed by Andrew Svetlov in :gh:`91218`.) |
| 593 |
|
| 594 |
* Added the :class:`~asyncio.Barrier` class to the synchronization |
| 595 |
primitives in the asyncio library, and the related |
| 596 |
:exc:`~asyncio.BrokenBarrierError` exception. |
| 597 |
(Contributed by Yves Duprat and Andrew Svetlov in :gh:`87518`.) |
| 598 |
|
| 599 |
* Added keyword argument *all_errors* to :meth:`asyncio.loop.create_connection` |
| 600 |
so that multiple connection errors can be raised as an :exc:`ExceptionGroup`. |
| 601 |
|
| 602 |
* Added the :meth:`asyncio.StreamWriter.start_tls` method for |
| 603 |
upgrading existing stream-based connections to TLS. |
| 604 |
(Contributed by Ian Good in :issue:`34975`.) |
| 605 |
|
| 606 |
* Added raw datagram socket functions to the event loop: |
| 607 |
:meth:`~asyncio.loop.sock_sendto`, |
| 608 |
:meth:`~asyncio.loop.sock_recvfrom` and |
| 609 |
:meth:`~asyncio.loop.sock_recvfrom_into`. |
| 610 |
These have implementations in :class:`~asyncio.SelectorEventLoop` and |
| 611 |
:class:`~asyncio.ProactorEventLoop`. |
| 612 |
(Contributed by Alex Grönholm in :issue:`46805`.) |
| 613 |
|
| 614 |
* Added :meth:`~asyncio.Task.cancelling` and |
| 615 |
:meth:`~asyncio.Task.uncancel` methods to :class:`~asyncio.Task`. |
| 616 |
These are primarily intended for internal use, |
| 617 |
notably by :class:`~asyncio.TaskGroup`. |
| 618 |
|
| 619 |
|
| 620 |
.. _whatsnew311-contextlib: |
| 621 |
|
| 622 |
contextlib |
| 623 |
---------- |
| 624 |
|
| 625 |
* Added non parallel-safe :func:`~contextlib.chdir` context manager to change |
| 626 |
the current working directory and then restore it on exit. Simple wrapper |
| 627 |
around :func:`~os.chdir`. (Contributed by Filipe Laíns in :issue:`25625`) |
| 628 |
|
| 629 |
|
| 630 |
.. _whatsnew311-dataclasses: |
| 631 |
|
| 632 |
dataclasses |
| 633 |
----------- |
| 634 |
|
| 635 |
* Change field default mutability check, allowing only defaults which are |
| 636 |
:term:`hashable` instead of any object which is not an instance of |
| 637 |
:class:`dict`, :class:`list` or :class:`set`. (Contributed by Eric V. Smith in |
| 638 |
:issue:`44674`.) |
| 639 |
|
| 640 |
|
| 641 |
.. _whatsnew311-datetime: |
| 642 |
|
| 643 |
datetime |
| 644 |
-------- |
| 645 |
|
| 646 |
* Add :const:`datetime.UTC`, a convenience alias for |
| 647 |
:attr:`datetime.timezone.utc`. (Contributed by Kabir Kwatra in :gh:`91973`.) |
| 648 |
|
| 649 |
* :meth:`datetime.date.fromisoformat`, :meth:`datetime.time.fromisoformat` and |
| 650 |
:meth:`datetime.datetime.fromisoformat` can now be used to parse most ISO 8601 |
| 651 |
formats (barring only those that support fractional hours and minutes). |
| 652 |
(Contributed by Paul Ganssle in :gh:`80010`.) |
| 653 |
|
| 654 |
|
| 655 |
.. _whatsnew311-enum: |
| 656 |
|
| 657 |
enum |
| 658 |
---- |
| 659 |
|
| 660 |
* Renamed :class:`!EnumMeta` to :class:`~enum.EnumType` |
| 661 |
(:class:`!EnumMeta` kept as an alias). |
| 662 |
|
| 663 |
* Added :class:`~enum.StrEnum`, |
| 664 |
with members that can be used as (and must be) strings. |
| 665 |
|
| 666 |
* Added :class:`~enum.ReprEnum`, |
| 667 |
which only modifies the :meth:`~object.__repr__` of members |
| 668 |
while returning their literal values (rather than names) |
| 669 |
for :meth:`~object.__str__` and :meth:`~object.__format__` |
| 670 |
(used by :func:`str`, :func:`format` and :term:`f-string`\s). |
| 671 |
|
| 672 |
* Changed :meth:`Enum.__format__() <enum.Enum.__format__>` (the default for |
| 673 |
:func:`format`, :meth:`str.format` and :term:`f-string`\s) to always produce |
| 674 |
the same result as :meth:`Enum.__str__() <enum.Enum.__str__>`: for enums inheriting from |
| 675 |
:class:`~enum.ReprEnum` it will be the member's value; for all other enums |
| 676 |
it will be the enum and member name (e.g. ``Color.RED``). |
| 677 |
|
| 678 |
* Added a new *boundary* class parameter to :class:`~enum.Flag` enums |
| 679 |
and the :class:`~enum.FlagBoundary` enum with its options, |
| 680 |
to control how to handle out-of-range flag values. |
| 681 |
|
| 682 |
* Added the :func:`~enum.verify` enum decorator |
| 683 |
and the :class:`~enum.EnumCheck` enum with its options, |
| 684 |
to check enum classes against several specific constraints. |
| 685 |
|
| 686 |
* Added the :func:`~enum.member` and :func:`~enum.nonmember` decorators, |
| 687 |
to ensure the decorated object is/is not converted to an enum member. |
| 688 |
|
| 689 |
* Added the :func:`~enum.property` decorator, |
| 690 |
which works like :func:`property` except for enums. |
| 691 |
Use this instead of :func:`types.DynamicClassAttribute`. |
| 692 |
|
| 693 |
* Added the :func:`~enum.global_enum` enum decorator, |
| 694 |
which adjusts :meth:`~object.__repr__` and :meth:`~object.__str__` |
| 695 |
to show values as members of their module rather than the enum class. |
| 696 |
For example, ``'re.ASCII'`` for the :const:`~re.ASCII` member |
| 697 |
of :class:`re.RegexFlag` rather than ``'RegexFlag.ASCII'``. |
| 698 |
|
| 699 |
* Enhanced :class:`~enum.Flag` to support |
| 700 |
:func:`len`, iteration and :keyword:`in`/:keyword:`not in` on its members. |
| 701 |
For example, the following now works: |
| 702 |
``len(AFlag(3)) == 2 and list(AFlag(3)) == (AFlag.ONE, AFlag.TWO)`` |
| 703 |
|
| 704 |
* Changed :class:`~enum.Enum` and :class:`~enum.Flag` |
| 705 |
so that members are now defined |
| 706 |
before :meth:`~object.__init_subclass__` is called; |
| 707 |
:func:`dir` now includes methods, etc., from mixed-in data types. |
| 708 |
|
| 709 |
* Changed :class:`~enum.Flag` |
| 710 |
to only consider primary values (power of two) canonical |
| 711 |
while composite values (``3``, ``6``, ``10``, etc.) are considered aliases; |
| 712 |
inverted flags are coerced to their positive equivalent. |
| 713 |
|
| 714 |
|
| 715 |
.. _whatsnew311-fcntl: |
| 716 |
|
| 717 |
fcntl |
| 718 |
----- |
| 719 |
|
| 720 |
* On FreeBSD, the :data:`!F_DUP2FD` and :data:`!F_DUP2FD_CLOEXEC` flags respectively |
| 721 |
are supported, the former equals to ``dup2`` usage while the latter set |
| 722 |
the ``FD_CLOEXEC`` flag in addition. |
| 723 |
|
| 724 |
|
| 725 |
.. _whatsnew311-fractions: |
| 726 |
|
| 727 |
fractions |
| 728 |
--------- |
| 729 |
|
| 730 |
* Support :PEP:`515`-style initialization of :class:`~fractions.Fraction` from |
| 731 |
string. (Contributed by Sergey B Kirpichev in :issue:`44258`.) |
| 732 |
|
| 733 |
* :class:`~fractions.Fraction` now implements an ``__int__`` method, so |
| 734 |
that an ``isinstance(some_fraction, typing.SupportsInt)`` check passes. |
| 735 |
(Contributed by Mark Dickinson in :issue:`44547`.) |
| 736 |
|
| 737 |
|
| 738 |
.. _whatsnew311-functools: |
| 739 |
|
| 740 |
functools |
| 741 |
--------- |
| 742 |
|
| 743 |
* :func:`functools.singledispatch` now supports :class:`types.UnionType` |
| 744 |
and :class:`typing.Union` as annotations to the dispatch argument.:: |
| 745 |
|
| 746 |
>>> from functools import singledispatch |
| 747 |
>>> @singledispatch |
| 748 |
... def fun(arg, verbose=False): |
| 749 |
... if verbose: |
| 750 |
... print("Let me just say,", end=" ") |
| 751 |
... print(arg) |
| 752 |
... |
| 753 |
>>> @fun.register |
| 754 |
... def _(arg: int | float, verbose=False): |
| 755 |
... if verbose: |
| 756 |
... print("Strength in numbers, eh?", end=" ") |
| 757 |
... print(arg) |
| 758 |
... |
| 759 |
>>> from typing import Union |
| 760 |
>>> @fun.register |
| 761 |
... def _(arg: Union[list, set], verbose=False): |
| 762 |
... if verbose: |
| 763 |
... print("Enumerate this:") |
| 764 |
... for i, elem in enumerate(arg): |
| 765 |
... print(i, elem) |
| 766 |
... |
| 767 |
|
| 768 |
(Contributed by Yurii Karabas in :issue:`46014`.) |
| 769 |
|
| 770 |
|
| 771 |
.. _whatsnew311-gzip: |
| 772 |
|
| 773 |
gzip |
| 774 |
---- |
| 775 |
|
| 776 |
* The :func:`gzip.compress` function is now faster when used with the |
| 777 |
**mtime=0** argument as it delegates the compression entirely to a single |
| 778 |
:func:`zlib.compress` operation. There is one side effect of this change: The |
| 779 |
gzip file header contains an "OS" byte in its header. That was traditionally |
| 780 |
always set to a value of 255 representing "unknown" by the :mod:`gzip` |
| 781 |
module. Now, when using :func:`~gzip.compress` with **mtime=0**, it may be |
| 782 |
set to a different value by the underlying zlib C library Python was linked |
| 783 |
against. |
| 784 |
(See :gh:`112346` for details on the side effect.) |
| 785 |
|
| 786 |
.. _whatsnew311-hashlib: |
| 787 |
|
| 788 |
hashlib |
| 789 |
------- |
| 790 |
|
| 791 |
* :func:`hashlib.blake2b` and :func:`hashlib.blake2s` now prefer `libb2`_ |
| 792 |
over Python's vendored copy. |
| 793 |
(Contributed by Christian Heimes in :issue:`47095`.) |
| 794 |
|
| 795 |
* The internal ``_sha3`` module with SHA3 and SHAKE algorithms now uses |
| 796 |
*tiny_sha3* instead of the *Keccak Code Package* to reduce code and binary |
| 797 |
size. The :mod:`hashlib` module prefers optimized SHA3 and SHAKE |
| 798 |
implementations from OpenSSL. The change affects only installations without |
| 799 |
OpenSSL support. |
| 800 |
(Contributed by Christian Heimes in :issue:`47098`.) |
| 801 |
|
| 802 |
* Add :func:`hashlib.file_digest`, a helper function for efficient hashing |
| 803 |
of files or file-like objects. |
| 804 |
(Contributed by Christian Heimes in :gh:`89313`.) |
| 805 |
|
| 806 |
|
| 807 |
.. _whatsnew311-idle: |
| 808 |
|
| 809 |
IDLE and idlelib |
| 810 |
---------------- |
| 811 |
|
| 812 |
* Apply syntax highlighting to ``.pyi`` files. (Contributed by Alex |
| 813 |
Waygood and Terry Jan Reedy in :issue:`45447`.) |
| 814 |
|
| 815 |
* Include prompts when saving Shell with inputs and outputs. |
| 816 |
(Contributed by Terry Jan Reedy in :gh:`95191`.) |
| 817 |
|
| 818 |
|
| 819 |
.. _whatsnew311-inspect: |
| 820 |
|
| 821 |
inspect |
| 822 |
------- |
| 823 |
|
| 824 |
* Add :func:`~inspect.getmembers_static` to return all members without |
| 825 |
triggering dynamic lookup via the descriptor protocol. (Contributed by |
| 826 |
Weipeng Hong in :issue:`30533`.) |
| 827 |
|
| 828 |
* Add :func:`~inspect.ismethodwrapper` |
| 829 |
for checking if the type of an object is a :class:`~types.MethodWrapperType`. |
| 830 |
(Contributed by Hakan Çelik in :issue:`29418`.) |
| 831 |
|
| 832 |
* Change the frame-related functions in the :mod:`inspect` module to return new |
| 833 |
:class:`~inspect.FrameInfo` and :class:`~inspect.Traceback` class instances |
| 834 |
(backwards compatible with the previous :term:`named tuple`-like interfaces) |
| 835 |
that includes the extended :pep:`657` position information (end |
| 836 |
line number, column and end column). The affected functions are: |
| 837 |
|
| 838 |
* :func:`inspect.getframeinfo` |
| 839 |
* :func:`inspect.getouterframes` |
| 840 |
* :func:`inspect.getinnerframes`, |
| 841 |
* :func:`inspect.stack` |
| 842 |
* :func:`inspect.trace` |
| 843 |
|
| 844 |
(Contributed by Pablo Galindo in :gh:`88116`.) |
| 845 |
|
| 846 |
|
| 847 |
.. _whatsnew311-locale: |
| 848 |
|
| 849 |
locale |
| 850 |
------ |
| 851 |
|
| 852 |
* Add :func:`locale.getencoding` to get the current locale encoding. It is similar to |
| 853 |
``locale.getpreferredencoding(False)`` but ignores the |
| 854 |
:ref:`Python UTF-8 Mode <utf8-mode>`. |
| 855 |
|
| 856 |
|
| 857 |
.. _whatsnew311-logging: |
| 858 |
|
| 859 |
logging |
| 860 |
------- |
| 861 |
|
| 862 |
* Added :func:`~logging.getLevelNamesMapping` |
| 863 |
to return a mapping from logging level names (e.g. ``'CRITICAL'``) |
| 864 |
to the values of their corresponding :ref:`levels` (e.g. ``50``, by default). |
| 865 |
(Contributed by Andrei Kulakovin in :gh:`88024`.) |
| 866 |
|
| 867 |
* Added a :meth:`~logging.handlers.SysLogHandler.createSocket` method |
| 868 |
to :class:`~logging.handlers.SysLogHandler`, to match |
| 869 |
:meth:`SocketHandler.createSocket() |
| 870 |
<logging.handlers.SocketHandler.createSocket>`. |
| 871 |
It is called automatically during handler initialization |
| 872 |
and when emitting an event, if there is no active socket. |
| 873 |
(Contributed by Kirill Pinchuk in :gh:`88457`.) |
| 874 |
|
| 875 |
|
| 876 |
.. _whatsnew311-math: |
| 877 |
|
| 878 |
math |
| 879 |
---- |
| 880 |
|
| 881 |
* Add :func:`math.exp2`: return 2 raised to the power of x. |
| 882 |
(Contributed by Gideon Mitchell in :issue:`45917`.) |
| 883 |
|
| 884 |
* Add :func:`math.cbrt`: return the cube root of x. |
| 885 |
(Contributed by Ajith Ramachandran in :issue:`44357`.) |
| 886 |
|
| 887 |
* The behaviour of two :func:`math.pow` corner cases was changed, for |
| 888 |
consistency with the IEEE 754 specification. The operations |
| 889 |
``math.pow(0.0, -math.inf)`` and ``math.pow(-0.0, -math.inf)`` now return |
| 890 |
``inf``. Previously they raised :exc:`ValueError`. (Contributed by Mark |
| 891 |
Dickinson in :issue:`44339`.) |
| 892 |
|
| 893 |
* The :data:`math.nan` value is now always available. |
| 894 |
(Contributed by Victor Stinner in :issue:`46917`.) |
| 895 |
|
| 896 |
|
| 897 |
.. _whatsnew311-operator: |
| 898 |
|
| 899 |
operator |
| 900 |
-------- |
| 901 |
|
| 902 |
* A new function ``operator.call`` has been added, such that |
| 903 |
``operator.call(obj, *args, **kwargs) == obj(*args, **kwargs)``. |
| 904 |
(Contributed by Antony Lee in :issue:`44019`.) |
| 905 |
|
| 906 |
|
| 907 |
.. _whatsnew311-os: |
| 908 |
|
| 909 |
os |
| 910 |
-- |
| 911 |
|
| 912 |
* On Windows, :func:`os.urandom` now uses ``BCryptGenRandom()``, |
| 913 |
instead of ``CryptGenRandom()`` which is deprecated. |
| 914 |
(Contributed by Donghee Na in :issue:`44611`.) |
| 915 |
|
| 916 |
|
| 917 |
.. _whatsnew311-pathlib: |
| 918 |
|
| 919 |
pathlib |
| 920 |
------- |
| 921 |
|
| 922 |
* :meth:`~pathlib.Path.glob` and :meth:`~pathlib.Path.rglob` return only |
| 923 |
directories if *pattern* ends with a pathname components separator: |
| 924 |
:data:`~os.sep` or :data:`~os.altsep`. |
| 925 |
(Contributed by Eisuke Kawasima in :issue:`22276` and :issue:`33392`.) |
| 926 |
|
| 927 |
|
| 928 |
.. _whatsnew311-re: |
| 929 |
|
| 930 |
re |
| 931 |
-- |
| 932 |
|
| 933 |
* Atomic grouping (``(?>...)``) and possessive quantifiers (``*+``, ``++``, |
| 934 |
``?+``, ``{m,n}+``) are now supported in regular expressions. |
| 935 |
(Contributed by Jeffrey C. Jacobs and Serhiy Storchaka in :issue:`433030`.) |
| 936 |
|
| 937 |
|
| 938 |
.. _whatsnew311-shutil: |
| 939 |
|
| 940 |
shutil |
| 941 |
------ |
| 942 |
|
| 943 |
* Add optional parameter *dir_fd* in :func:`shutil.rmtree`. |
| 944 |
(Contributed by Serhiy Storchaka in :issue:`46245`.) |
| 945 |
|
| 946 |
|
| 947 |
.. _whatsnew311-socket: |
| 948 |
|
| 949 |
socket |
| 950 |
------ |
| 951 |
|
| 952 |
* Add CAN Socket support for NetBSD. |
| 953 |
(Contributed by Thomas Klausner in :issue:`30512`.) |
| 954 |
|
| 955 |
* :meth:`~socket.create_connection` has an option to raise, in case of |
| 956 |
failure to connect, an :exc:`ExceptionGroup` containing all errors |
| 957 |
instead of only raising the last error. |
| 958 |
(Contributed by Irit Katriel in :issue:`29980`.) |
| 959 |
|
| 960 |
|
| 961 |
.. _whatsnew311-sqlite3: |
| 962 |
|
| 963 |
sqlite3 |
| 964 |
------- |
| 965 |
|
| 966 |
* You can now disable the authorizer by passing :const:`None` to |
| 967 |
:meth:`~sqlite3.Connection.set_authorizer`. |
| 968 |
(Contributed by Erlend E. Aasland in :issue:`44491`.) |
| 969 |
|
| 970 |
* Collation name :meth:`~sqlite3.Connection.create_collation` can now |
| 971 |
contain any Unicode character. Collation names with invalid characters |
| 972 |
now raise :exc:`UnicodeEncodeError` instead of :exc:`sqlite3.ProgrammingError`. |
| 973 |
(Contributed by Erlend E. Aasland in :issue:`44688`.) |
| 974 |
|
| 975 |
* :mod:`sqlite3` exceptions now include the SQLite extended error code as |
| 976 |
:attr:`~sqlite3.Error.sqlite_errorcode` and the SQLite error name as |
| 977 |
:attr:`~sqlite3.Error.sqlite_errorname`. |
| 978 |
(Contributed by Aviv Palivoda, Daniel Shahaf, and Erlend E. Aasland in |
| 979 |
:issue:`16379` and :issue:`24139`.) |
| 980 |
|
| 981 |
* Add :meth:`~sqlite3.Connection.setlimit` and |
| 982 |
:meth:`~sqlite3.Connection.getlimit` to :class:`sqlite3.Connection` for |
| 983 |
setting and getting SQLite limits by connection basis. |
| 984 |
(Contributed by Erlend E. Aasland in :issue:`45243`.) |
| 985 |
|
| 986 |
* :mod:`sqlite3` now sets :attr:`sqlite3.threadsafety` based on the default |
| 987 |
threading mode the underlying SQLite library has been compiled with. |
| 988 |
(Contributed by Erlend E. Aasland in :issue:`45613`.) |
| 989 |
|
| 990 |
* :mod:`sqlite3` C callbacks now use unraisable exceptions if callback |
| 991 |
tracebacks are enabled. Users can now register an |
| 992 |
:func:`unraisable hook handler <sys.unraisablehook>` to improve their debug |
| 993 |
experience. |
| 994 |
(Contributed by Erlend E. Aasland in :issue:`45828`.) |
| 995 |
|
| 996 |
* Fetch across rollback no longer raises :exc:`~sqlite3.InterfaceError`. |
| 997 |
Instead we leave it to the SQLite library to handle these cases. |
| 998 |
(Contributed by Erlend E. Aasland in :issue:`44092`.) |
| 999 |
|
| 1000 |
* Add :meth:`~sqlite3.Connection.serialize` and |
| 1001 |
:meth:`~sqlite3.Connection.deserialize` to :class:`sqlite3.Connection` for |
| 1002 |
serializing and deserializing databases. |
| 1003 |
(Contributed by Erlend E. Aasland in :issue:`41930`.) |
| 1004 |
|
| 1005 |
* Add :meth:`~sqlite3.Connection.create_window_function` to |
| 1006 |
:class:`sqlite3.Connection` for creating aggregate window functions. |
| 1007 |
(Contributed by Erlend E. Aasland in :issue:`34916`.) |
| 1008 |
|
| 1009 |
* Add :meth:`~sqlite3.Connection.blobopen` to :class:`sqlite3.Connection`. |
| 1010 |
:class:`sqlite3.Blob` allows incremental I/O operations on blobs. |
| 1011 |
(Contributed by Aviv Palivoda and Erlend E. Aasland in :issue:`24905`.) |
| 1012 |
|
| 1013 |
|
| 1014 |
.. _whatsnew311-string: |
| 1015 |
|
| 1016 |
string |
| 1017 |
------ |
| 1018 |
|
| 1019 |
* Add :meth:`~string.Template.get_identifiers` |
| 1020 |
and :meth:`~string.Template.is_valid` to :class:`string.Template`, |
| 1021 |
which respectively return all valid placeholders, |
| 1022 |
and whether any invalid placeholders are present. |
| 1023 |
(Contributed by Ben Kehoe in :gh:`90465`.) |
| 1024 |
|
| 1025 |
|
| 1026 |
.. _whatsnew311-sys: |
| 1027 |
|
| 1028 |
sys |
| 1029 |
--- |
| 1030 |
|
| 1031 |
* :func:`sys.exc_info` now derives the ``type`` and ``traceback`` fields |
| 1032 |
from the ``value`` (the exception instance), so when an exception is |
| 1033 |
modified while it is being handled, the changes are reflected in |
| 1034 |
the results of subsequent calls to :func:`!exc_info`. |
| 1035 |
(Contributed by Irit Katriel in :issue:`45711`.) |
| 1036 |
|
| 1037 |
* Add :func:`sys.exception` which returns the active exception instance |
| 1038 |
(equivalent to ``sys.exc_info()[1]``). |
| 1039 |
(Contributed by Irit Katriel in :issue:`46328`.) |
| 1040 |
|
| 1041 |
* Add the :data:`sys.flags.safe_path <sys.flags>` flag. |
| 1042 |
(Contributed by Victor Stinner in :gh:`57684`.) |
| 1043 |
|
| 1044 |
|
| 1045 |
.. _whatsnew311-sysconfig: |
| 1046 |
|
| 1047 |
sysconfig |
| 1048 |
--------- |
| 1049 |
|
| 1050 |
* Three new :ref:`installation schemes <installation_paths>` |
| 1051 |
(*posix_venv*, *nt_venv* and *venv*) were added and are used when Python |
| 1052 |
creates new virtual environments or when it is running from a virtual |
| 1053 |
environment. |
| 1054 |
The first two schemes (*posix_venv* and *nt_venv*) are OS-specific |
| 1055 |
for non-Windows and Windows, the *venv* is essentially an alias to one of |
| 1056 |
them according to the OS Python runs on. |
| 1057 |
This is useful for downstream distributors who modify |
| 1058 |
:func:`sysconfig.get_preferred_scheme`. |
| 1059 |
Third party code that creates new virtual environments should use the new |
| 1060 |
*venv* installation scheme to determine the paths, as does :mod:`venv`. |
| 1061 |
(Contributed by Miro Hrončok in :issue:`45413`.) |
| 1062 |
|
| 1063 |
|
| 1064 |
.. _whatsnew311-tempfile: |
| 1065 |
|
| 1066 |
tempfile |
| 1067 |
-------- |
| 1068 |
|
| 1069 |
* :class:`~tempfile.SpooledTemporaryFile` objects now fully implement the methods |
| 1070 |
of :class:`io.BufferedIOBase` or :class:`io.TextIOBase` |
| 1071 |
(depending on file mode). |
| 1072 |
This lets them work correctly with APIs that expect file-like objects, |
| 1073 |
such as compression modules. |
| 1074 |
(Contributed by Carey Metcalfe in :gh:`70363`.) |
| 1075 |
|
| 1076 |
|
| 1077 |
.. _whatsnew311-threading: |
| 1078 |
|
| 1079 |
threading |
| 1080 |
--------- |
| 1081 |
|
| 1082 |
* On Unix, if the ``sem_clockwait()`` function is available in the C library |
| 1083 |
(glibc 2.30 and newer), the :meth:`threading.Lock.acquire` method now uses |
| 1084 |
the monotonic clock (:const:`time.CLOCK_MONOTONIC`) for the timeout, rather |
| 1085 |
than using the system clock (:const:`time.CLOCK_REALTIME`), to not be affected |
| 1086 |
by system clock changes. |
| 1087 |
(Contributed by Victor Stinner in :issue:`41710`.) |
| 1088 |
|
| 1089 |
|
| 1090 |
.. _whatsnew311-time: |
| 1091 |
|
| 1092 |
time |
| 1093 |
---- |
| 1094 |
|
| 1095 |
* On Unix, :func:`time.sleep` now uses the ``clock_nanosleep()`` or |
| 1096 |
``nanosleep()`` function, if available, which has a resolution of 1 nanosecond |
| 1097 |
(10\ :sup:`-9` seconds), rather than using ``select()`` which has a resolution |
| 1098 |
of 1 microsecond (10\ :sup:`-6` seconds). |
| 1099 |
(Contributed by Benjamin Szőke and Victor Stinner in :issue:`21302`.) |
| 1100 |
|
| 1101 |
* On Windows 8.1 and newer, :func:`time.sleep` now uses a waitable timer based |
| 1102 |
on `high-resolution timers |
| 1103 |
<https://docs.microsoft.com/en-us/windows-hardware/drivers/kernel/high-resolution-timers>`_ |
| 1104 |
which has a resolution of 100 nanoseconds (10\ :sup:`-7` seconds). Previously, |
| 1105 |
it had a resolution of 1 millisecond (10\ :sup:`-3` seconds). |
| 1106 |
(Contributed by Benjamin Szőke, Donghee Na, Eryk Sun and Victor Stinner in :issue:`21302` and :issue:`45429`.) |
| 1107 |
|
| 1108 |
|
| 1109 |
.. _whatsnew311-tkinter: |
| 1110 |
|
| 1111 |
tkinter |
| 1112 |
------- |
| 1113 |
|
| 1114 |
* Added method ``info_patchlevel()`` which returns the exact version of |
| 1115 |
the Tcl library as a named tuple similar to :data:`sys.version_info`. |
| 1116 |
(Contributed by Serhiy Storchaka in :gh:`91827`.) |
| 1117 |
|
| 1118 |
|
| 1119 |
.. _whatsnew311-traceback: |
| 1120 |
|
| 1121 |
traceback |
| 1122 |
--------- |
| 1123 |
|
| 1124 |
* Add :func:`traceback.StackSummary.format_frame_summary` to allow users |
| 1125 |
to override which frames appear in the traceback, and how they are |
| 1126 |
formatted. |
| 1127 |
(Contributed by Ammar Askar in :issue:`44569`.) |
| 1128 |
|
| 1129 |
* Add :func:`traceback.TracebackException.print`, which prints the |
| 1130 |
formatted :exc:`~traceback.TracebackException` instance to a file. |
| 1131 |
(Contributed by Irit Katriel in :issue:`33809`.) |
| 1132 |
|
| 1133 |
|
| 1134 |
.. _whatsnew311-typing: |
| 1135 |
|
| 1136 |
typing |
| 1137 |
------ |
| 1138 |
|
| 1139 |
For major changes, see :ref:`new-feat-related-type-hints-311`. |
| 1140 |
|
| 1141 |
* Add :func:`typing.assert_never` and :class:`typing.Never`. |
| 1142 |
:func:`typing.assert_never` is useful for asking a type checker to confirm |
| 1143 |
that a line of code is not reachable. At runtime, it raises an |
| 1144 |
:exc:`AssertionError`. |
| 1145 |
(Contributed by Jelle Zijlstra in :gh:`90633`.) |
| 1146 |
|
| 1147 |
* Add :func:`typing.reveal_type`. This is useful for asking a type checker |
| 1148 |
what type it has inferred for a given expression. At runtime it prints |
| 1149 |
the type of the received value. |
| 1150 |
(Contributed by Jelle Zijlstra in :gh:`90572`.) |
| 1151 |
|
| 1152 |
* Add :func:`typing.assert_type`. This is useful for asking a type checker |
| 1153 |
to confirm that the type it has inferred for a given expression matches |
| 1154 |
the given type. At runtime it simply returns the received value. |
| 1155 |
(Contributed by Jelle Zijlstra in :gh:`90638`.) |
| 1156 |
|
| 1157 |
* :data:`typing.TypedDict` types can now be generic. (Contributed by |
| 1158 |
Samodya Abeysiriwardane in :gh:`89026`.) |
| 1159 |
|
| 1160 |
* :class:`~typing.NamedTuple` types can now be generic. |
| 1161 |
(Contributed by Serhiy Storchaka in :issue:`43923`.) |
| 1162 |
|
| 1163 |
* Allow subclassing of :class:`typing.Any`. This is useful for avoiding |
| 1164 |
type checker errors related to highly dynamic class, such as mocks. |
| 1165 |
(Contributed by Shantanu Jain in :gh:`91154`.) |
| 1166 |
|
| 1167 |
* The :func:`typing.final` decorator now sets the ``__final__`` attributed on |
| 1168 |
the decorated object. |
| 1169 |
(Contributed by Jelle Zijlstra in :gh:`90500`.) |
| 1170 |
|
| 1171 |
* The :func:`typing.get_overloads` function can be used for introspecting |
| 1172 |
the overloads of a function. :func:`typing.clear_overloads` can be used |
| 1173 |
to clear all registered overloads of a function. |
| 1174 |
(Contributed by Jelle Zijlstra in :gh:`89263`.) |
| 1175 |
|
| 1176 |
* The :meth:`~object.__init__` method of :class:`~typing.Protocol` subclasses |
| 1177 |
is now preserved. (Contributed by Adrian Garcia Badarasco in :gh:`88970`.) |
| 1178 |
|
| 1179 |
* The representation of empty tuple types (``Tuple[()]``) is simplified. |
| 1180 |
This affects introspection, e.g. ``get_args(Tuple[()])`` now evaluates |
| 1181 |
to ``()`` instead of ``((),)``. |
| 1182 |
(Contributed by Serhiy Storchaka in :gh:`91137`.) |
| 1183 |
|
| 1184 |
* Loosen runtime requirements for type annotations by removing the callable |
| 1185 |
check in the private ``typing._type_check`` function. (Contributed by |
| 1186 |
Gregory Beauregard in :gh:`90802`.) |
| 1187 |
|
| 1188 |
* :func:`typing.get_type_hints` now supports evaluating strings as forward |
| 1189 |
references in :ref:`PEP 585 generic aliases <types-genericalias>`. |
| 1190 |
(Contributed by Niklas Rosenstein in :gh:`85542`.) |
| 1191 |
|
| 1192 |
* :func:`typing.get_type_hints` no longer adds :data:`~typing.Optional` |
| 1193 |
to parameters with ``None`` as a default. (Contributed by Nikita Sobolev |
| 1194 |
in :gh:`90353`.) |
| 1195 |
|
| 1196 |
* :func:`typing.get_type_hints` now supports evaluating bare stringified |
| 1197 |
:data:`~typing.ClassVar` annotations. (Contributed by Gregory Beauregard |
| 1198 |
in :gh:`90711`.) |
| 1199 |
|
| 1200 |
* :func:`typing.no_type_check` no longer modifies external classes and functions. |
| 1201 |
It also now correctly marks classmethods as not to be type checked. (Contributed |
| 1202 |
by Nikita Sobolev in :gh:`90729`.) |
| 1203 |
|
| 1204 |
|
| 1205 |
.. _whatsnew311-unicodedata: |
| 1206 |
|
| 1207 |
unicodedata |
| 1208 |
----------- |
| 1209 |
|
| 1210 |
* The Unicode database has been updated to version 14.0.0. |
| 1211 |
(Contributed by Benjamin Peterson in :issue:`45190`). |
| 1212 |
|
| 1213 |
|
| 1214 |
.. _whatsnew311-unittest: |
| 1215 |
|
| 1216 |
unittest |
| 1217 |
-------- |
| 1218 |
|
| 1219 |
* Added methods :meth:`~unittest.TestCase.enterContext` and |
| 1220 |
:meth:`~unittest.TestCase.enterClassContext` of class |
| 1221 |
:class:`~unittest.TestCase`, method |
| 1222 |
:meth:`~unittest.IsolatedAsyncioTestCase.enterAsyncContext` of |
| 1223 |
class :class:`~unittest.IsolatedAsyncioTestCase` and function |
| 1224 |
:func:`unittest.enterModuleContext`. |
| 1225 |
(Contributed by Serhiy Storchaka in :issue:`45046`.) |
| 1226 |
|
| 1227 |
|
| 1228 |
.. _whatsnew311-venv: |
| 1229 |
|
| 1230 |
venv |
| 1231 |
---- |
| 1232 |
|
| 1233 |
* When new Python virtual environments are created, the *venv* |
| 1234 |
:ref:`sysconfig installation scheme <installation_paths>` is used |
| 1235 |
to determine the paths inside the environment. |
| 1236 |
When Python runs in a virtual environment, the same installation scheme |
| 1237 |
is the default. |
| 1238 |
That means that downstream distributors can change the default sysconfig install |
| 1239 |
scheme without changing behavior of virtual environments. |
| 1240 |
Third party code that also creates new virtual environments should do the same. |
| 1241 |
(Contributed by Miro Hrončok in :issue:`45413`.) |
| 1242 |
|
| 1243 |
|
| 1244 |
.. _whatsnew311-warnings: |
| 1245 |
|
| 1246 |
warnings |
| 1247 |
-------- |
| 1248 |
|
| 1249 |
* :func:`warnings.catch_warnings` now accepts arguments for :func:`warnings.simplefilter`, |
| 1250 |
providing a more concise way to locally ignore warnings or convert them to errors. |
| 1251 |
(Contributed by Zac Hatfield-Dodds in :issue:`47074`.) |
| 1252 |
|
| 1253 |
|
| 1254 |
.. _whatsnew311-zipfile: |
| 1255 |
|
| 1256 |
zipfile |
| 1257 |
------- |
| 1258 |
|
| 1259 |
* Added support for specifying member name encoding for reading metadata |
| 1260 |
in a :class:`~zipfile.ZipFile`'s directory and file headers. |
| 1261 |
(Contributed by Stephen J. Turnbull and Serhiy Storchaka in :issue:`28080`.) |
| 1262 |
|
| 1263 |
* Added :meth:`ZipFile.mkdir() <zipfile.ZipFile.mkdir>` |
| 1264 |
for creating new directories inside ZIP archives. |
| 1265 |
(Contributed by Sam Ezeh in :gh:`49083`.) |
| 1266 |
|
| 1267 |
* Added :attr:`~zipfile.Path.stem`, :attr:`~zipfile.Path.suffix` |
| 1268 |
and :attr:`~zipfile.Path.suffixes` to :class:`zipfile.Path`. |
| 1269 |
(Contributed by Miguel Brito in :gh:`88261`.) |
| 1270 |
|
| 1271 |
|
| 1272 |
.. _whatsnew311-optimizations: |
| 1273 |
|
| 1274 |
Optimizations |
| 1275 |
============= |
| 1276 |
|
| 1277 |
This section covers specific optimizations independent of the |
| 1278 |
:ref:`whatsnew311-faster-cpython` project, which is covered in its own section. |
| 1279 |
|
| 1280 |
* The compiler now optimizes simple |
| 1281 |
:ref:`printf-style % formatting <old-string-formatting>` on string literals |
| 1282 |
containing only the format codes ``%s``, ``%r`` and ``%a`` and makes it as |
| 1283 |
fast as a corresponding :term:`f-string` expression. |
| 1284 |
(Contributed by Serhiy Storchaka in :issue:`28307`.) |
| 1285 |
|
| 1286 |
* Integer division (``//``) is better tuned for optimization by compilers. |
| 1287 |
It is now around 20% faster on x86-64 when dividing an :class:`int` |
| 1288 |
by a value smaller than ``2**30``. |
| 1289 |
(Contributed by Gregory P. Smith and Tim Peters in :gh:`90564`.) |
| 1290 |
|
| 1291 |
* :func:`sum` is now nearly 30% faster for integers smaller than ``2**30``. |
| 1292 |
(Contributed by Stefan Behnel in :gh:`68264`.) |
| 1293 |
|
| 1294 |
* Resizing lists is streamlined for the common case, |
| 1295 |
speeding up :meth:`list.append` by ≈15% |
| 1296 |
and simple :term:`list comprehension`\s by up to 20-30% |
| 1297 |
(Contributed by Dennis Sweeney in :gh:`91165`.) |
| 1298 |
|
| 1299 |
* Dictionaries don't store hash values when all keys are Unicode objects, |
| 1300 |
decreasing :class:`dict` size. |
| 1301 |
For example, ``sys.getsizeof(dict.fromkeys("abcdefg"))`` |
| 1302 |
is reduced from 352 bytes to 272 bytes (23% smaller) on 64-bit platforms. |
| 1303 |
(Contributed by Inada Naoki in :issue:`46845`.) |
| 1304 |
|
| 1305 |
* Using :class:`asyncio.DatagramProtocol` is now orders of magnitude faster |
| 1306 |
when transferring large files over UDP, |
| 1307 |
with speeds over 100 times higher for a ≈60 MiB file. |
| 1308 |
(Contributed by msoxzw in :gh:`91487`.) |
| 1309 |
|
| 1310 |
* :mod:`math` functions :func:`~math.comb` and :func:`~math.perm` are now |
| 1311 |
≈10 times faster for large arguments (with a larger speedup for larger *k*). |
| 1312 |
(Contributed by Serhiy Storchaka in :issue:`37295`.) |
| 1313 |
|
| 1314 |
* The :mod:`statistics` functions :func:`~statistics.mean`, |
| 1315 |
:func:`~statistics.variance` and :func:`~statistics.stdev` now consume |
| 1316 |
iterators in one pass rather than converting them to a :class:`list` first. |
| 1317 |
This is twice as fast and can save substantial memory. |
| 1318 |
(Contributed by Raymond Hettinger in :gh:`90415`.) |
| 1319 |
|
| 1320 |
* :func:`unicodedata.normalize` |
| 1321 |
now normalizes pure-ASCII strings in constant time. |
| 1322 |
(Contributed by Donghee Na in :issue:`44987`.) |
| 1323 |
|
| 1324 |
|
| 1325 |
.. _whatsnew311-faster-cpython: |
| 1326 |
|
| 1327 |
Faster CPython |
| 1328 |
============== |
| 1329 |
|
| 1330 |
CPython 3.11 is an average of |
| 1331 |
`25% faster <https://github.com/faster-cpython/ideas#published-results>`_ |
| 1332 |
than CPython 3.10 as measured with the |
| 1333 |
`pyperformance <https://github.com/python/pyperformance>`_ benchmark suite, |
| 1334 |
when compiled with GCC on Ubuntu Linux. |
| 1335 |
Depending on your workload, the overall speedup could be 10-60%. |
| 1336 |
|
| 1337 |
This project focuses on two major areas in Python: |
| 1338 |
:ref:`whatsnew311-faster-startup` and :ref:`whatsnew311-faster-runtime`. |
| 1339 |
Optimizations not covered by this project are listed separately under |
| 1340 |
:ref:`whatsnew311-optimizations`. |
| 1341 |
|
| 1342 |
|
| 1343 |
.. _whatsnew311-faster-startup: |
| 1344 |
|
| 1345 |
Faster Startup |
| 1346 |
-------------- |
| 1347 |
|
| 1348 |
.. _whatsnew311-faster-imports: |
| 1349 |
|
| 1350 |
Frozen imports / Static code objects |
| 1351 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 1352 |
|
| 1353 |
Python caches :term:`bytecode` in the :ref:`__pycache__ <tut-pycache>` |
| 1354 |
directory to speed up module loading. |
| 1355 |
|
| 1356 |
Previously in 3.10, Python module execution looked like this: |
| 1357 |
|
| 1358 |
.. code-block:: text |
| 1359 |
|
| 1360 |
Read __pycache__ -> Unmarshal -> Heap allocated code object -> Evaluate |
| 1361 |
|
| 1362 |
In Python 3.11, the core modules essential for Python startup are "frozen". |
| 1363 |
This means that their :ref:`codeobjects` (and bytecode) |
| 1364 |
are statically allocated by the interpreter. |
| 1365 |
This reduces the steps in module execution process to: |
| 1366 |
|
| 1367 |
.. code-block:: text |
| 1368 |
|
| 1369 |
Statically allocated code object -> Evaluate |
| 1370 |
|
| 1371 |
Interpreter startup is now 10-15% faster in Python 3.11. This has a big |
| 1372 |
impact for short-running programs using Python. |
| 1373 |
|
| 1374 |
(Contributed by Eric Snow, Guido van Rossum and Kumar Aditya in many issues.) |
| 1375 |
|
| 1376 |
|
| 1377 |
.. _whatsnew311-faster-runtime: |
| 1378 |
|
| 1379 |
Faster Runtime |
| 1380 |
-------------- |
| 1381 |
|
| 1382 |
.. _whatsnew311-lazy-python-frames: |
| 1383 |
|
| 1384 |
Cheaper, lazy Python frames |
| 1385 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 1386 |
|
| 1387 |
Python frames, holding execution information, |
| 1388 |
are created whenever Python calls a Python function. |
| 1389 |
The following are new frame optimizations: |
| 1390 |
|
| 1391 |
- Streamlined the frame creation process. |
| 1392 |
- Avoided memory allocation by generously re-using frame space on the C stack. |
| 1393 |
- Streamlined the internal frame struct to contain only essential information. |
| 1394 |
Frames previously held extra debugging and memory management information. |
| 1395 |
|
| 1396 |
Old-style :ref:`frame objects <frame-objects>` |
| 1397 |
are now created only when requested by debuggers |
| 1398 |
or by Python introspection functions such as :func:`sys._getframe` and |
| 1399 |
:func:`inspect.currentframe`. For most user code, no frame objects are |
| 1400 |
created at all. As a result, nearly all Python functions calls have sped |
| 1401 |
up significantly. We measured a 3-7% speedup in pyperformance. |
| 1402 |
|
| 1403 |
(Contributed by Mark Shannon in :issue:`44590`.) |
| 1404 |
|
| 1405 |
|
| 1406 |
.. _inline-calls: |
| 1407 |
.. _whatsnew311-inline-calls: |
| 1408 |
|
| 1409 |
Inlined Python function calls |
| 1410 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 1411 |
|
| 1412 |
During a Python function call, Python will call an evaluating C function to |
| 1413 |
interpret that function's code. This effectively limits pure Python recursion to |
| 1414 |
what's safe for the C stack. |
| 1415 |
|
| 1416 |
In 3.11, when CPython detects Python code calling another Python function, |
| 1417 |
it sets up a new frame, and "jumps" to the new code inside the new frame. This |
| 1418 |
avoids calling the C interpreting function altogether. |
| 1419 |
|
| 1420 |
Most Python function calls now consume no C stack space, speeding them up. |
| 1421 |
In simple recursive functions like fibonacci or |
| 1422 |
factorial, we observed a 1.7x speedup. This also means recursive functions |
| 1423 |
can recurse significantly deeper |
| 1424 |
(if the user increases the recursion limit with :func:`sys.setrecursionlimit`). |
| 1425 |
We measured a 1-3% improvement in pyperformance. |
| 1426 |
|
| 1427 |
(Contributed by Pablo Galindo and Mark Shannon in :issue:`45256`.) |
| 1428 |
|
| 1429 |
|
| 1430 |
.. _whatsnew311-pep659: |
| 1431 |
|
| 1432 |
PEP 659: Specializing Adaptive Interpreter |
| 1433 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 1434 |
|
| 1435 |
:pep:`659` is one of the key parts of the Faster CPython project. The general |
| 1436 |
idea is that while Python is a dynamic language, most code has regions where |
| 1437 |
objects and types rarely change. This concept is known as *type stability*. |
| 1438 |
|
| 1439 |
At runtime, Python will try to look for common patterns and type stability |
| 1440 |
in the executing code. Python will then replace the current operation with a |
| 1441 |
more specialized one. This specialized operation uses fast paths available only |
| 1442 |
to those use cases/types, which generally outperform their generic |
| 1443 |
counterparts. This also brings in another concept called *inline caching*, where |
| 1444 |
Python caches the results of expensive operations directly in the |
| 1445 |
:term:`bytecode`. |
| 1446 |
|
| 1447 |
The specializer will also combine certain common instruction pairs into one |
| 1448 |
superinstruction, reducing the overhead during execution. |
| 1449 |
|
| 1450 |
Python will only specialize |
| 1451 |
when it sees code that is "hot" (executed multiple times). This prevents Python |
| 1452 |
from wasting time on run-once code. Python can also de-specialize when code is |
| 1453 |
too dynamic or when the use changes. Specialization is attempted periodically, |
| 1454 |
and specialization attempts are not too expensive, |
| 1455 |
allowing specialization to adapt to new circumstances. |
| 1456 |
|
| 1457 |
(PEP written by Mark Shannon, with ideas inspired by Stefan Brunthaler. |
| 1458 |
See :pep:`659` for more information. Implementation by Mark Shannon and Brandt |
| 1459 |
Bucher, with additional help from Irit Katriel and Dennis Sweeney.) |
| 1460 |
|
| 1461 |
.. |
| 1462 |
If I missed out anyone, please add them. |
| 1463 |
|
| 1464 |
+---------------+--------------------+-------------------------------------------------------+-------------------+-------------------+ |
| 1465 |
| Operation | Form | Specialization | Operation speedup | Contributor(s) | |
| 1466 |
| | | | (up to) | | |
| 1467 |
+===============+====================+=======================================================+===================+===================+ |
| 1468 |
| Binary | ``x + x`` | Binary add, multiply and subtract for common types | 10% | Mark Shannon, | |
| 1469 |
| operations | | such as :class:`int`, :class:`float` and :class:`str` | | Donghee Na, | |
| 1470 |
| | ``x - x`` | take custom fast paths for their underlying types. | | Brandt Bucher, | |
| 1471 |
| | | | | Dennis Sweeney | |
| 1472 |
| | ``x * x`` | | | | |
| 1473 |
+---------------+--------------------+-------------------------------------------------------+-------------------+-------------------+ |
| 1474 |
| Subscript | ``a[i]`` | Subscripting container types such as :class:`list`, | 10-25% | Irit Katriel, | |
| 1475 |
| | | :class:`tuple` and :class:`dict` directly index | | Mark Shannon | |
| 1476 |
| | | the underlying data structures. | | | |
| 1477 |
| | | | | | |
| 1478 |
| | | Subscripting custom :meth:`~object.__getitem__` | | | |
| 1479 |
| | | is also inlined similar to :ref:`inline-calls`. | | | |
| 1480 |
+---------------+--------------------+-------------------------------------------------------+-------------------+-------------------+ |
| 1481 |
| Store | ``a[i] = z`` | Similar to subscripting specialization above. | 10-25% | Dennis Sweeney | |
| 1482 |
| subscript | | | | | |
| 1483 |
+---------------+--------------------+-------------------------------------------------------+-------------------+-------------------+ |
| 1484 |
| Calls | ``f(arg)`` | Calls to common builtin (C) functions and types such | 20% | Mark Shannon, | |
| 1485 |
| | | as :func:`len` and :class:`str` directly call their | | Ken Jin | |
| 1486 |
| | ``C(arg)`` | underlying C version. This avoids going through the | | | |
| 1487 |
| | | internal calling convention. | | | |
| 1488 |
+---------------+--------------------+-------------------------------------------------------+-------------------+-------------------+ |
| 1489 |
| Load | ``print`` | The object's index in the globals/builtins namespace | [#load-global]_ | Mark Shannon | |
| 1490 |
| global | | is cached. Loading globals and builtins require | | | |
| 1491 |
| variable | ``len`` | zero namespace lookups. | | | |
| 1492 |
+---------------+--------------------+-------------------------------------------------------+-------------------+-------------------+ |
| 1493 |
| Load | ``o.attr`` | Similar to loading global variables. The attribute's | [#load-attr]_ | Mark Shannon | |
| 1494 |
| attribute | | index inside the class/object's namespace is cached. | | | |
| 1495 |
| | | In most cases, attribute loading will require zero | | | |
| 1496 |
| | | namespace lookups. | | | |
| 1497 |
+---------------+--------------------+-------------------------------------------------------+-------------------+-------------------+ |
| 1498 |
| Load | ``o.meth()`` | The actual address of the method is cached. Method | 10-20% | Ken Jin, | |
| 1499 |
| methods for | | loading now has no namespace lookups -- even for | | Mark Shannon | |
| 1500 |
| call | | classes with long inheritance chains. | | | |
| 1501 |
+---------------+--------------------+-------------------------------------------------------+-------------------+-------------------+ |
| 1502 |
| Store | ``o.attr = z`` | Similar to load attribute optimization. | 2% | Mark Shannon | |
| 1503 |
| attribute | | | in pyperformance | | |
| 1504 |
+---------------+--------------------+-------------------------------------------------------+-------------------+-------------------+ |
| 1505 |
| Unpack | ``*seq`` | Specialized for common containers such as | 8% | Brandt Bucher | |
| 1506 |
| Sequence | | :class:`list` and :class:`tuple`. | | | |
| 1507 |
| | | Avoids internal calling convention. | | | |
| 1508 |
+---------------+--------------------+-------------------------------------------------------+-------------------+-------------------+ |
| 1509 |
|
| 1510 |
.. [#load-global] A similar optimization already existed since Python 3.8. |
| 1511 |
3.11 specializes for more forms and reduces some overhead. |
| 1512 |
|
| 1513 |
.. [#load-attr] A similar optimization already existed since Python 3.10. |
| 1514 |
3.11 specializes for more forms. Furthermore, all attribute loads should |
| 1515 |
be sped up by :issue:`45947`. |
| 1516 |
|
| 1517 |
|
| 1518 |
.. _whatsnew311-faster-cpython-misc: |
| 1519 |
|
| 1520 |
Misc |
| 1521 |
---- |
| 1522 |
|
| 1523 |
* Objects now require less memory due to lazily created object namespaces. |
| 1524 |
Their namespace dictionaries now also share keys more freely. |
| 1525 |
(Contributed Mark Shannon in :issue:`45340` and :issue:`40116`.) |
| 1526 |
|
| 1527 |
* "Zero-cost" exceptions are implemented, eliminating the cost |
| 1528 |
of :keyword:`try` statements when no exception is raised. |
| 1529 |
(Contributed by Mark Shannon in :issue:`40222`.) |
| 1530 |
|
| 1531 |
* A more concise representation of exceptions in the interpreter reduced the |
| 1532 |
time required for catching an exception by about 10%. |
| 1533 |
(Contributed by Irit Katriel in :issue:`45711`.) |
| 1534 |
|
| 1535 |
* :mod:`re`'s regular expression matching engine has been partially refactored, |
| 1536 |
and now uses computed gotos (or "threaded code") on supported platforms. As a |
| 1537 |
result, Python 3.11 executes the `pyperformance regular expression benchmarks |
| 1538 |
<https://pyperformance.readthedocs.io/benchmarks.html#regex-dna>`_ up to 10% |
| 1539 |
faster than Python 3.10. |
| 1540 |
(Contributed by Brandt Bucher in :gh:`91404`.) |
| 1541 |
|
| 1542 |
|
| 1543 |
.. _whatsnew311-faster-cpython-faq: |
| 1544 |
|
| 1545 |
FAQ |
| 1546 |
--- |
| 1547 |
|
| 1548 |
.. _faster-cpython-faq-my-code: |
| 1549 |
|
| 1550 |
How should I write my code to utilize these speedups? |
| 1551 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 1552 |
|
| 1553 |
Write Pythonic code that follows common best practices; |
| 1554 |
you don't have to change your code. |
| 1555 |
The Faster CPython project optimizes for common code patterns we observe. |
| 1556 |
|
| 1557 |
|
| 1558 |
.. _faster-cpython-faq-memory: |
| 1559 |
|
| 1560 |
Will CPython 3.11 use more memory? |
| 1561 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 1562 |
|
| 1563 |
Maybe not; we don't expect memory use to exceed 20% higher than 3.10. |
| 1564 |
This is offset by memory optimizations for frame objects and object |
| 1565 |
dictionaries as mentioned above. |
| 1566 |
|
| 1567 |
|
| 1568 |
.. _faster-cpython-ymmv: |
| 1569 |
|
| 1570 |
I don't see any speedups in my workload. Why? |
| 1571 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 1572 |
|
| 1573 |
Certain code won't have noticeable benefits. If your code spends most of |
| 1574 |
its time on I/O operations, or already does most of its |
| 1575 |
computation in a C extension library like NumPy, there won't be significant |
| 1576 |
speedups. This project currently benefits pure-Python workloads the most. |
| 1577 |
|
| 1578 |
Furthermore, the pyperformance figures are a geometric mean. Even within the |
| 1579 |
pyperformance benchmarks, certain benchmarks have slowed down slightly, while |
| 1580 |
others have sped up by nearly 2x! |
| 1581 |
|
| 1582 |
|
| 1583 |
.. _faster-cpython-jit: |
| 1584 |
|
| 1585 |
Is there a JIT compiler? |
| 1586 |
^^^^^^^^^^^^^^^^^^^^^^^^ |
| 1587 |
|
| 1588 |
No. We're still exploring other optimizations. |
| 1589 |
|
| 1590 |
|
| 1591 |
.. _whatsnew311-faster-cpython-about: |
| 1592 |
|
| 1593 |
About |
| 1594 |
----- |
| 1595 |
|
| 1596 |
Faster CPython explores optimizations for :term:`CPython`. The main team is |
| 1597 |
funded by Microsoft to work on this full-time. Pablo Galindo Salgado is also |
| 1598 |
funded by Bloomberg LP to work on the project part-time. Finally, many |
| 1599 |
contributors are volunteers from the community. |
| 1600 |
|
| 1601 |
|
| 1602 |
.. _whatsnew311-bytecode-changes: |
| 1603 |
|
| 1604 |
CPython bytecode changes |
| 1605 |
======================== |
| 1606 |
|
| 1607 |
The bytecode now contains inline cache entries, |
| 1608 |
which take the form of the newly-added :opcode:`CACHE` instructions. |
| 1609 |
Many opcodes expect to be followed by an exact number of caches, |
| 1610 |
and instruct the interpreter to skip over them at runtime. |
| 1611 |
Populated caches can look like arbitrary instructions, |
| 1612 |
so great care should be taken when reading or modifying |
| 1613 |
raw, adaptive bytecode containing quickened data. |
| 1614 |
|
| 1615 |
|
| 1616 |
.. _whatsnew311-added-opcodes: |
| 1617 |
|
| 1618 |
New opcodes |
| 1619 |
----------- |
| 1620 |
|
| 1621 |
* :opcode:`!ASYNC_GEN_WRAP`, :opcode:`RETURN_GENERATOR` and :opcode:`SEND`, |
| 1622 |
used in generators and co-routines. |
| 1623 |
|
| 1624 |
* :opcode:`COPY_FREE_VARS`, |
| 1625 |
which avoids needing special caller-side code for closures. |
| 1626 |
|
| 1627 |
* :opcode:`JUMP_BACKWARD_NO_INTERRUPT`, |
| 1628 |
for use in certain loops where handling interrupts is undesirable. |
| 1629 |
|
| 1630 |
* :opcode:`MAKE_CELL`, to create :ref:`cell-objects`. |
| 1631 |
|
| 1632 |
* :opcode:`CHECK_EG_MATCH` and :opcode:`!PREP_RERAISE_STAR`, |
| 1633 |
to handle the :ref:`new exception groups and except* <whatsnew311-pep654>` |
| 1634 |
added in :pep:`654`. |
| 1635 |
|
| 1636 |
* :opcode:`PUSH_EXC_INFO`, for use in exception handlers. |
| 1637 |
|
| 1638 |
* :opcode:`RESUME`, a no-op, |
| 1639 |
for internal tracing, debugging and optimization checks. |
| 1640 |
|
| 1641 |
|
| 1642 |
.. _whatsnew311-replaced-opcodes: |
| 1643 |
|
| 1644 |
Replaced opcodes |
| 1645 |
---------------- |
| 1646 |
|
| 1647 |
+------------------------------------+------------------------------------+-----------------------------------------+ |
| 1648 |
| Replaced Opcode(s) | New Opcode(s) | Notes | |
| 1649 |
+====================================+====================================+=========================================+ |
| 1650 |
| | :opcode:`!BINARY_*` | :opcode:`BINARY_OP` | Replaced all numeric binary/in-place | |
| 1651 |
| | :opcode:`!INPLACE_*` | | opcodes with a single opcode | |
| 1652 |
+------------------------------------+------------------------------------+-----------------------------------------+ |
| 1653 |
| | :opcode:`!CALL_FUNCTION` | | :opcode:`CALL` | Decouples argument shifting for methods | |
| 1654 |
| | :opcode:`!CALL_FUNCTION_KW` | | :opcode:`!KW_NAMES` | from handling of keyword arguments; | |
| 1655 |
| | :opcode:`!CALL_METHOD` | | :opcode:`!PRECALL` | allows better specialization of calls | |
| 1656 |
| | | :opcode:`PUSH_NULL` | | |
| 1657 |
+------------------------------------+------------------------------------+-----------------------------------------+ |
| 1658 |
| | :opcode:`!DUP_TOP` | | :opcode:`COPY` | Stack manipulation instructions | |
| 1659 |
| | :opcode:`!DUP_TOP_TWO` | | :opcode:`SWAP` | | |
| 1660 |
| | :opcode:`!ROT_TWO` | | | |
| 1661 |
| | :opcode:`!ROT_THREE` | | | |
| 1662 |
| | :opcode:`!ROT_FOUR` | | | |
| 1663 |
| | :opcode:`!ROT_N` | | | |
| 1664 |
+------------------------------------+------------------------------------+-----------------------------------------+ |
| 1665 |
| | :opcode:`!JUMP_IF_NOT_EXC_MATCH` | | :opcode:`CHECK_EXC_MATCH` | Now performs check but doesn't jump | |
| 1666 |
+------------------------------------+------------------------------------+-----------------------------------------+ |
| 1667 |
| | :opcode:`!JUMP_ABSOLUTE` | | :opcode:`JUMP_BACKWARD` | See [#bytecode-jump]_; | |
| 1668 |
| | :opcode:`!POP_JUMP_IF_FALSE` | | :opcode:`!POP_JUMP_BACKWARD_IF_*`| ``TRUE``, ``FALSE``, | |
| 1669 |
| | :opcode:`!POP_JUMP_IF_TRUE` | | :opcode:`!POP_JUMP_FORWARD_IF_*` | ``NONE`` and ``NOT_NONE`` variants | |
| 1670 |
| | | for each direction | |
| 1671 |
| | | | |
| 1672 |
+------------------------------------+------------------------------------+-----------------------------------------+ |
| 1673 |
| | :opcode:`!SETUP_WITH` | :opcode:`!BEFORE_WITH` | :keyword:`with` block setup | |
| 1674 |
| | :opcode:`!SETUP_ASYNC_WITH` | | | |
| 1675 |
+------------------------------------+------------------------------------+-----------------------------------------+ |
| 1676 |
|
| 1677 |
.. [#bytecode-jump] All jump opcodes are now relative, including the |
| 1678 |
existing :opcode:`!JUMP_IF_TRUE_OR_POP` and :opcode:`!JUMP_IF_FALSE_OR_POP`. |
| 1679 |
The argument is now an offset from the current instruction |
| 1680 |
rather than an absolute location. |
| 1681 |
|
| 1682 |
|
| 1683 |
.. _whatsnew311-changed-opcodes: |
| 1684 |
.. _whatsnew311-removed-opcodes: |
| 1685 |
.. _whatsnew311-changed-removed-opcodes: |
| 1686 |
|
| 1687 |
Changed/removed opcodes |
| 1688 |
----------------------- |
| 1689 |
|
| 1690 |
* Changed :opcode:`MATCH_CLASS` and :opcode:`MATCH_KEYS` |
| 1691 |
to no longer push an additional boolean value to indicate success/failure. |
| 1692 |
Instead, ``None`` is pushed on failure |
| 1693 |
in place of the tuple of extracted values. |
| 1694 |
|
| 1695 |
* Changed opcodes that work with exceptions to reflect them |
| 1696 |
now being represented as one item on the stack instead of three |
| 1697 |
(see :gh:`89874`). |
| 1698 |
|
| 1699 |
* Removed :opcode:`!COPY_DICT_WITHOUT_KEYS`, :opcode:`!GEN_START`, |
| 1700 |
:opcode:`!POP_BLOCK`, :opcode:`!SETUP_FINALLY` and :opcode:`!YIELD_FROM`. |
| 1701 |
|
| 1702 |
|
| 1703 |
.. _whatsnew311-deprecated: |
| 1704 |
.. _whatsnew311-python-api-deprecated: |
| 1705 |
|
| 1706 |
Deprecated |
| 1707 |
========== |
| 1708 |
|
| 1709 |
This section lists Python APIs that have been deprecated in Python 3.11. |
| 1710 |
|
| 1711 |
Deprecated C APIs are :ref:`listed separately <whatsnew311-c-api-deprecated>`. |
| 1712 |
|
| 1713 |
|
| 1714 |
.. _whatsnew311-deprecated-language: |
| 1715 |
.. _whatsnew311-deprecated-builtins: |
| 1716 |
|
| 1717 |
Language/Builtins |
| 1718 |
----------------- |
| 1719 |
|
| 1720 |
* Chaining :class:`classmethod` descriptors (introduced in :issue:`19072`) |
| 1721 |
is now deprecated. It can no longer be used to wrap other descriptors |
| 1722 |
such as :class:`property`. The core design of this feature was flawed |
| 1723 |
and caused a number of downstream problems. To "pass-through" a |
| 1724 |
:class:`classmethod`, consider using the :attr:`!__wrapped__` attribute |
| 1725 |
that was added in Python 3.10. |
| 1726 |
(Contributed by Raymond Hettinger in :gh:`89519`.) |
| 1727 |
|
| 1728 |
* Octal escapes in string and bytes literals with values larger than ``0o377`` |
| 1729 |
(255 in decimal) now produce a :exc:`DeprecationWarning`. |
| 1730 |
In a future Python version, they will raise a :exc:`SyntaxWarning` and |
| 1731 |
eventually a :exc:`SyntaxError`. |
| 1732 |
(Contributed by Serhiy Storchaka in :gh:`81548`.) |
| 1733 |
|
| 1734 |
* The delegation of :func:`int` to :meth:`~object.__trunc__` is now deprecated. |
| 1735 |
Calling ``int(a)`` when ``type(a)`` implements :meth:`!__trunc__` but not |
| 1736 |
:meth:`~object.__int__` or :meth:`~object.__index__` now raises |
| 1737 |
a :exc:`DeprecationWarning`. |
| 1738 |
(Contributed by Zackery Spytz in :issue:`44977`.) |
| 1739 |
|
| 1740 |
|
| 1741 |
.. _whatsnew311-deprecated-modules: |
| 1742 |
|
| 1743 |
Modules |
| 1744 |
------- |
| 1745 |
|
| 1746 |
.. _whatsnew311-pep594: |
| 1747 |
|
| 1748 |
* :pep:`594` led to the deprecations of the following modules |
| 1749 |
slated for removal in Python 3.13: |
| 1750 |
|
| 1751 |
+---------------------+---------------------+---------------------+---------------------+---------------------+ |
| 1752 |
| :mod:`!aifc` | :mod:`!chunk` | :mod:`!msilib` | :mod:`!pipes` | :mod:`!telnetlib` | |
| 1753 |
+---------------------+---------------------+---------------------+---------------------+---------------------+ |
| 1754 |
| :mod:`!audioop` | :mod:`!crypt` | :mod:`!nis` | :mod:`!sndhdr` | :mod:`!uu` | |
| 1755 |
+---------------------+---------------------+---------------------+---------------------+---------------------+ |
| 1756 |
| :mod:`!cgi` | :mod:`!imghdr` | :mod:`!nntplib` | :mod:`!spwd` | :mod:`!xdrlib` | |
| 1757 |
+---------------------+---------------------+---------------------+---------------------+---------------------+ |
| 1758 |
| :mod:`!cgitb` | :mod:`!mailcap` | :mod:`!ossaudiodev` | :mod:`!sunau` | | |
| 1759 |
+---------------------+---------------------+---------------------+---------------------+---------------------+ |
| 1760 |
|
| 1761 |
(Contributed by Brett Cannon in :issue:`47061` and Victor Stinner in |
| 1762 |
:gh:`68966`.) |
| 1763 |
|
| 1764 |
* The :mod:`!asynchat`, :mod:`!asyncore` and :mod:`!smtpd` modules have been |
| 1765 |
deprecated since at least Python 3.6. Their documentation and deprecation |
| 1766 |
warnings have now been updated to note they will be removed in Python 3.12. |
| 1767 |
(Contributed by Hugo van Kemenade in :issue:`47022`.) |
| 1768 |
|
| 1769 |
* The :mod:`!lib2to3` package and ``2to3`` tool |
| 1770 |
are now deprecated and may not be able to parse Python 3.10 or newer. |
| 1771 |
See :pep:`617`, introducing the new PEG parser, for details. |
| 1772 |
(Contributed by Victor Stinner in :issue:`40360`.) |
| 1773 |
|
| 1774 |
* Undocumented modules :mod:`!sre_compile`, :mod:`!sre_constants` |
| 1775 |
and :mod:`!sre_parse` are now deprecated. |
| 1776 |
(Contributed by Serhiy Storchaka in :issue:`47152`.) |
| 1777 |
|
| 1778 |
|
| 1779 |
.. _whatsnew311-deprecated-stdlib: |
| 1780 |
|
| 1781 |
Standard Library |
| 1782 |
---------------- |
| 1783 |
|
| 1784 |
* The following have been deprecated in :mod:`configparser` since Python 3.2. |
| 1785 |
Their deprecation warnings have now been updated to note they will be removed |
| 1786 |
in Python 3.12: |
| 1787 |
|
| 1788 |
* the :class:`!configparser.SafeConfigParser` class |
| 1789 |
* the :attr:`!configparser.ParsingError.filename` property |
| 1790 |
* the :meth:`!configparser.RawConfigParser.readfp` method |
| 1791 |
|
| 1792 |
(Contributed by Hugo van Kemenade in :issue:`45173`.) |
| 1793 |
|
| 1794 |
* :class:`!configparser.LegacyInterpolation` has been deprecated in the docstring |
| 1795 |
since Python 3.2, and is not listed in the :mod:`configparser` documentation. |
| 1796 |
It now emits a :exc:`DeprecationWarning` and will be removed |
| 1797 |
in Python 3.13. Use :class:`configparser.BasicInterpolation` or |
| 1798 |
:class:`configparser.ExtendedInterpolation` instead. |
| 1799 |
(Contributed by Hugo van Kemenade in :issue:`46607`.) |
| 1800 |
|
| 1801 |
* The older set of :mod:`importlib.resources` functions were deprecated |
| 1802 |
in favor of the replacements added in Python 3.9 |
| 1803 |
and will be removed in a future Python version, |
| 1804 |
due to not supporting resources located within package subdirectories: |
| 1805 |
|
| 1806 |
* :func:`!importlib.resources.contents` |
| 1807 |
* :func:`!importlib.resources.is_resource` |
| 1808 |
* :func:`!importlib.resources.open_binary` |
| 1809 |
* :func:`!importlib.resources.open_text` |
| 1810 |
* :func:`!importlib.resources.read_binary` |
| 1811 |
* :func:`!importlib.resources.read_text` |
| 1812 |
* :func:`!importlib.resources.path` |
| 1813 |
|
| 1814 |
* The :func:`locale.getdefaultlocale` function is deprecated and will be |
| 1815 |
removed in Python 3.15. Use :func:`locale.setlocale`, |
| 1816 |
:func:`locale.getpreferredencoding(False) <locale.getpreferredencoding>` and |
| 1817 |
:func:`locale.getlocale` functions instead. |
| 1818 |
(Contributed by Victor Stinner in :gh:`90817`.) |
| 1819 |
|
| 1820 |
* The :func:`!locale.resetlocale` function is deprecated and will be |
| 1821 |
removed in Python 3.13. Use ``locale.setlocale(locale.LC_ALL, "")`` instead. |
| 1822 |
(Contributed by Victor Stinner in :gh:`90817`.) |
| 1823 |
|
| 1824 |
* Stricter rules will now be applied for numerical group references |
| 1825 |
and group names in :ref:`regular expressions <re-syntax>`. |
| 1826 |
Only sequences of ASCII digits will now be accepted as a numerical reference, |
| 1827 |
and the group name in :class:`bytes` patterns and replacement strings |
| 1828 |
can only contain ASCII letters, digits and underscores. |
| 1829 |
For now, a deprecation warning is raised for syntax violating these rules. |
| 1830 |
(Contributed by Serhiy Storchaka in :gh:`91760`.) |
| 1831 |
|
| 1832 |
* In the :mod:`re` module, the :func:`!re.template` function |
| 1833 |
and the corresponding :const:`!re.TEMPLATE` and :const:`!re.T` flags |
| 1834 |
are deprecated, as they were undocumented and lacked an obvious purpose. |
| 1835 |
They will be removed in Python 3.13. |
| 1836 |
(Contributed by Serhiy Storchaka and Miro Hrončok in :gh:`92728`.) |
| 1837 |
|
| 1838 |
* :func:`!turtle.settiltangle` has been deprecated since Python 3.1; |
| 1839 |
it now emits a deprecation warning and will be removed in Python 3.13. Use |
| 1840 |
:func:`turtle.tiltangle` instead (it was earlier incorrectly marked |
| 1841 |
as deprecated, and its docstring is now corrected). |
| 1842 |
(Contributed by Hugo van Kemenade in :issue:`45837`.) |
| 1843 |
|
| 1844 |
* :class:`typing.Text`, which exists solely to provide compatibility support |
| 1845 |
between Python 2 and Python 3 code, is now deprecated. Its removal is |
| 1846 |
currently unplanned, but users are encouraged to use :class:`str` instead |
| 1847 |
wherever possible. |
| 1848 |
(Contributed by Alex Waygood in :gh:`92332`.) |
| 1849 |
|
| 1850 |
* The keyword argument syntax for constructing :data:`typing.TypedDict` types |
| 1851 |
is now deprecated. Support will be removed in Python 3.13. (Contributed by |
| 1852 |
Jingchen Ye in :gh:`90224`.) |
| 1853 |
|
| 1854 |
* :class:`!webbrowser.MacOSX` is deprecated and will be removed in Python 3.13. |
| 1855 |
It is untested, undocumented, and not used by :mod:`webbrowser` itself. |
| 1856 |
(Contributed by Donghee Na in :issue:`42255`.) |
| 1857 |
|
| 1858 |
* The behavior of returning a value from a :class:`~unittest.TestCase` and |
| 1859 |
:class:`~unittest.IsolatedAsyncioTestCase` test methods (other than the |
| 1860 |
default ``None`` value) is now deprecated. |
| 1861 |
|
| 1862 |
* Deprecated the following not-formally-documented :mod:`unittest` functions, |
| 1863 |
scheduled for removal in Python 3.13: |
| 1864 |
|
| 1865 |
* :func:`!unittest.findTestCases` |
| 1866 |
* :func:`!unittest.makeSuite` |
| 1867 |
* :func:`!unittest.getTestCaseNames` |
| 1868 |
|
| 1869 |
Use :class:`~unittest.TestLoader` methods instead: |
| 1870 |
|
| 1871 |
* :meth:`unittest.TestLoader.loadTestsFromModule` |
| 1872 |
* :meth:`unittest.TestLoader.loadTestsFromTestCase` |
| 1873 |
* :meth:`unittest.TestLoader.getTestCaseNames` |
| 1874 |
|
| 1875 |
(Contributed by Erlend E. Aasland in :issue:`5846`.) |
| 1876 |
|
| 1877 |
* :meth:`!unittest.TestProgram.usageExit` is marked deprecated, to be removed |
| 1878 |
in 3.13. |
| 1879 |
(Contributed by Carlos Damázio in :gh:`67048`.) |
| 1880 |
|
| 1881 |
|
| 1882 |
.. _whatsnew311-pending-removal: |
| 1883 |
.. _whatsnew311-python-api-pending-removal: |
| 1884 |
|
| 1885 |
Pending Removal in Python 3.12 |
| 1886 |
============================== |
| 1887 |
|
| 1888 |
The following Python APIs have been deprecated in earlier Python releases, |
| 1889 |
and will be removed in Python 3.12. |
| 1890 |
|
| 1891 |
C APIs pending removal are |
| 1892 |
:ref:`listed separately <whatsnew311-c-api-pending-removal>`. |
| 1893 |
|
| 1894 |
* The :mod:`!asynchat` module |
| 1895 |
* The :mod:`!asyncore` module |
| 1896 |
* The :ref:`entire distutils package <distutils-deprecated>` |
| 1897 |
* The :mod:`!imp` module |
| 1898 |
* The :class:`typing.io <typing.IO>` namespace |
| 1899 |
* The :class:`typing.re <typing.Pattern>` namespace |
| 1900 |
* :func:`!cgi.log` |
| 1901 |
* :func:`!importlib.find_loader` |
| 1902 |
* :meth:`!importlib.abc.Loader.module_repr` |
| 1903 |
* :meth:`!importlib.abc.MetaPathFinder.find_module` |
| 1904 |
* :meth:`!importlib.abc.PathEntryFinder.find_loader` |
| 1905 |
* :meth:`!importlib.abc.PathEntryFinder.find_module` |
| 1906 |
* :meth:`!importlib.machinery.BuiltinImporter.find_module` |
| 1907 |
* :meth:`!importlib.machinery.BuiltinLoader.module_repr` |
| 1908 |
* :meth:`!importlib.machinery.FileFinder.find_loader` |
| 1909 |
* :meth:`!importlib.machinery.FileFinder.find_module` |
| 1910 |
* :meth:`!importlib.machinery.FrozenImporter.find_module` |
| 1911 |
* :meth:`!importlib.machinery.FrozenLoader.module_repr` |
| 1912 |
* :meth:`!importlib.machinery.PathFinder.find_module` |
| 1913 |
* :meth:`!importlib.machinery.WindowsRegistryFinder.find_module` |
| 1914 |
* :func:`!importlib.util.module_for_loader` |
| 1915 |
* :func:`!importlib.util.set_loader_wrapper` |
| 1916 |
* :func:`!importlib.util.set_package_wrapper` |
| 1917 |
* :class:`!pkgutil.ImpImporter` |
| 1918 |
* :class:`!pkgutil.ImpLoader` |
| 1919 |
* :meth:`!pathlib.Path.link_to` |
| 1920 |
* :func:`!sqlite3.enable_shared_cache` |
| 1921 |
* :func:`!sqlite3.OptimizedUnicode` |
| 1922 |
* :envvar:`!PYTHONTHREADDEBUG` environment variable |
| 1923 |
* The following deprecated aliases in :mod:`unittest`: |
| 1924 |
|
| 1925 |
============================ =============================== =============== |
| 1926 |
Deprecated alias Method Name Deprecated in |
| 1927 |
============================ =============================== =============== |
| 1928 |
``failUnless`` :meth:`.assertTrue` 3.1 |
| 1929 |
``failIf`` :meth:`.assertFalse` 3.1 |
| 1930 |
``failUnlessEqual`` :meth:`.assertEqual` 3.1 |
| 1931 |
``failIfEqual`` :meth:`.assertNotEqual` 3.1 |
| 1932 |
``failUnlessAlmostEqual`` :meth:`.assertAlmostEqual` 3.1 |
| 1933 |
``failIfAlmostEqual`` :meth:`.assertNotAlmostEqual` 3.1 |
| 1934 |
``failUnlessRaises`` :meth:`.assertRaises` 3.1 |
| 1935 |
``assert_`` :meth:`.assertTrue` 3.2 |
| 1936 |
``assertEquals`` :meth:`.assertEqual` 3.2 |
| 1937 |
``assertNotEquals`` :meth:`.assertNotEqual` 3.2 |
| 1938 |
``assertAlmostEquals`` :meth:`.assertAlmostEqual` 3.2 |
| 1939 |
``assertNotAlmostEquals`` :meth:`.assertNotAlmostEqual` 3.2 |
| 1940 |
``assertRegexpMatches`` :meth:`.assertRegex` 3.2 |
| 1941 |
``assertRaisesRegexp`` :meth:`.assertRaisesRegex` 3.2 |
| 1942 |
``assertNotRegexpMatches`` :meth:`.assertNotRegex` 3.5 |
| 1943 |
============================ =============================== =============== |
| 1944 |
|
| 1945 |
.. _whatsnew311-removed: |
| 1946 |
.. _whatsnew311-python-api-removed: |
| 1947 |
|
| 1948 |
Removed |
| 1949 |
======= |
| 1950 |
|
| 1951 |
This section lists Python APIs that have been removed in Python 3.11. |
| 1952 |
|
| 1953 |
Removed C APIs are :ref:`listed separately <whatsnew311-c-api-removed>`. |
| 1954 |
|
| 1955 |
* Removed the :func:`[email protected]` :term:`decorator` |
| 1956 |
enabling legacy generator-based coroutines to be compatible with |
| 1957 |
:keyword:`async` / :keyword:`await` code. |
| 1958 |
The function has been deprecated since Python 3.8 and the removal was |
| 1959 |
initially scheduled for Python 3.10. Use :keyword:`async def` instead. |
| 1960 |
(Contributed by Illia Volochii in :issue:`43216`.) |
| 1961 |
|
| 1962 |
* Removed :class:`!asyncio.coroutines.CoroWrapper` used for wrapping legacy |
| 1963 |
generator-based coroutine objects in the debug mode. |
| 1964 |
(Contributed by Illia Volochii in :issue:`43216`.) |
| 1965 |
|
| 1966 |
* Due to significant security concerns, the *reuse_address* parameter of |
| 1967 |
:meth:`asyncio.loop.create_datagram_endpoint`, disabled in Python 3.9, is |
| 1968 |
now entirely removed. This is because of the behavior of the socket option |
| 1969 |
``SO_REUSEADDR`` in UDP. |
| 1970 |
(Contributed by Hugo van Kemenade in :issue:`45129`.) |
| 1971 |
|
| 1972 |
* Removed the :mod:`!binhex` module, deprecated in Python 3.9. |
| 1973 |
Also removed the related, similarly-deprecated :mod:`binascii` functions: |
| 1974 |
|
| 1975 |
* :func:`!binascii.a2b_hqx` |
| 1976 |
* :func:`!binascii.b2a_hqx` |
| 1977 |
* :func:`!binascii.rlecode_hqx` |
| 1978 |
* :func:`!binascii.rldecode_hqx` |
| 1979 |
|
| 1980 |
The :func:`binascii.crc_hqx` function remains available. |
| 1981 |
|
| 1982 |
(Contributed by Victor Stinner in :issue:`45085`.) |
| 1983 |
|
| 1984 |
* Removed the :mod:`!distutils` ``bdist_msi`` command deprecated in Python 3.9. |
| 1985 |
Use ``bdist_wheel`` (wheel packages) instead. |
| 1986 |
(Contributed by Hugo van Kemenade in :issue:`45124`.) |
| 1987 |
|
| 1988 |
* Removed the :meth:`~object.__getitem__` methods of |
| 1989 |
:class:`xml.dom.pulldom.DOMEventStream`, :class:`wsgiref.util.FileWrapper` |
| 1990 |
and :class:`fileinput.FileInput`, deprecated since Python 3.9. |
| 1991 |
(Contributed by Hugo van Kemenade in :issue:`45132`.) |
| 1992 |
|
| 1993 |
* Removed the deprecated :mod:`gettext` functions |
| 1994 |
:func:`!lgettext`, :func:`!ldgettext`, |
| 1995 |
:func:`!lngettext` and :func:`!ldngettext`. |
| 1996 |
Also removed the :func:`!bind_textdomain_codeset` function, |
| 1997 |
the :meth:`!NullTranslations.output_charset` and |
| 1998 |
:meth:`!NullTranslations.set_output_charset` methods, |
| 1999 |
and the *codeset* parameter of :func:`!translation` and :func:`!install`, |
| 2000 |
since they are only used for the :func:`!l*gettext` functions. |
| 2001 |
(Contributed by Donghee Na and Serhiy Storchaka in :issue:`44235`.) |
| 2002 |
|
| 2003 |
* Removed from the :mod:`inspect` module: |
| 2004 |
|
| 2005 |
* The :func:`!getargspec` function, deprecated since Python 3.0; |
| 2006 |
use :func:`inspect.signature` or :func:`inspect.getfullargspec` instead. |
| 2007 |
|
| 2008 |
* The :func:`!formatargspec` function, deprecated since Python 3.5; |
| 2009 |
use the :func:`inspect.signature` function |
| 2010 |
or the :class:`inspect.Signature` object directly. |
| 2011 |
|
| 2012 |
* The undocumented :meth:`!Signature.from_builtin` |
| 2013 |
and :meth:`!Signature.from_function` methods, deprecated since Python 3.5; |
| 2014 |
use the :meth:`Signature.from_callable() <inspect.Signature.from_callable>` |
| 2015 |
method instead. |
| 2016 |
|
| 2017 |
(Contributed by Hugo van Kemenade in :issue:`45320`.) |
| 2018 |
|
| 2019 |
* Removed the :meth:`~object.__class_getitem__` method |
| 2020 |
from :class:`pathlib.PurePath`, |
| 2021 |
because it was not used and added by mistake in previous versions. |
| 2022 |
(Contributed by Nikita Sobolev in :issue:`46483`.) |
| 2023 |
|
| 2024 |
* Removed the :class:`!MailmanProxy` class in the :mod:`!smtpd` module, |
| 2025 |
as it is unusable without the external :mod:`!mailman` package. |
| 2026 |
(Contributed by Donghee Na in :issue:`35800`.) |
| 2027 |
|
| 2028 |
* Removed the deprecated :meth:`!split` method of :class:`!_tkinter.TkappType`. |
| 2029 |
(Contributed by Erlend E. Aasland in :issue:`38371`.) |
| 2030 |
|
| 2031 |
* Removed namespace package support from :mod:`unittest` discovery. |
| 2032 |
It was introduced in Python 3.4 but has been broken since Python 3.7. |
| 2033 |
(Contributed by Inada Naoki in :issue:`23882`.) |
| 2034 |
|
| 2035 |
* Removed the undocumented private :meth:`!float.__set_format__` method, |
| 2036 |
previously known as :meth:`!float.__setformat__` in Python 3.7. |
| 2037 |
Its docstring said: "You probably don't want to use this function. |
| 2038 |
It exists mainly to be used in Python's test suite." |
| 2039 |
(Contributed by Victor Stinner in :issue:`46852`.) |
| 2040 |
|
| 2041 |
* The :option:`!--experimental-isolated-subinterpreters` configure flag |
| 2042 |
(and corresponding :c:macro:`!EXPERIMENTAL_ISOLATED_SUBINTERPRETERS` macro) |
| 2043 |
have been removed. |
| 2044 |
|
| 2045 |
* :pypi:`Pynche` |
| 2046 |
--- The Pythonically Natural Color and Hue Editor --- has been moved out |
| 2047 |
of ``Tools/scripts`` and is `being developed independently |
| 2048 |
<https://gitlab.com/warsaw/pynche/-/tree/main>`_ from the Python source tree. |
| 2049 |
|
| 2050 |
|
| 2051 |
.. _whatsnew311-porting: |
| 2052 |
.. _whatsnew311-python-api-porting: |
| 2053 |
|
| 2054 |
Porting to Python 3.11 |
| 2055 |
====================== |
| 2056 |
|
| 2057 |
This section lists previously described changes and other bugfixes |
| 2058 |
in the Python API that may require changes to your Python code. |
| 2059 |
|
| 2060 |
Porting notes for the C API are |
| 2061 |
:ref:`listed separately <whatsnew311-c-api-porting>`. |
| 2062 |
|
| 2063 |
* :func:`open`, :func:`io.open`, :func:`codecs.open` and |
| 2064 |
:class:`fileinput.FileInput` no longer accept ``'U'`` ("universal newline") |
| 2065 |
in the file mode. In Python 3, "universal newline" mode is used by default |
| 2066 |
whenever a file is opened in text mode, |
| 2067 |
and the ``'U'`` flag has been deprecated since Python 3.3. |
| 2068 |
The :ref:`newline parameter <open-newline-parameter>` |
| 2069 |
to these functions controls how universal newlines work. |
| 2070 |
(Contributed by Victor Stinner in :issue:`37330`.) |
| 2071 |
|
| 2072 |
* :class:`ast.AST` node positions are now validated when provided to |
| 2073 |
:func:`compile` and other related functions. If invalid positions are detected, |
| 2074 |
a :exc:`ValueError` will be raised. (Contributed by Pablo Galindo in :gh:`93351`) |
| 2075 |
|
| 2076 |
* Prohibited passing non-:class:`concurrent.futures.ThreadPoolExecutor` |
| 2077 |
executors to :meth:`asyncio.loop.set_default_executor` |
| 2078 |
following a deprecation in Python 3.8. |
| 2079 |
(Contributed by Illia Volochii in :issue:`43234`.) |
| 2080 |
|
| 2081 |
* :mod:`calendar`: The :class:`calendar.LocaleTextCalendar` and |
| 2082 |
:class:`calendar.LocaleHTMLCalendar` classes now use |
| 2083 |
:func:`locale.getlocale`, instead of using :func:`locale.getdefaultlocale`, |
| 2084 |
if no locale is specified. |
| 2085 |
(Contributed by Victor Stinner in :issue:`46659`.) |
| 2086 |
|
| 2087 |
* The :mod:`pdb` module now reads the :file:`.pdbrc` configuration file with |
| 2088 |
the ``'UTF-8'`` encoding. |
| 2089 |
(Contributed by Srinivas Reddy Thatiparthy (శ్రీనివాస్ రెడ్డి తాటిపర్తి) in :issue:`41137`.) |
| 2090 |
|
| 2091 |
* The *population* parameter of :func:`random.sample` must be a sequence, |
| 2092 |
and automatic conversion of :class:`set`\s to :class:`list`\s |
| 2093 |
is no longer supported. Also, if the sample size |
| 2094 |
is larger than the population size, a :exc:`ValueError` is raised. |
| 2095 |
(Contributed by Raymond Hettinger in :issue:`40465`.) |
| 2096 |
|
| 2097 |
* The *random* optional parameter of :func:`random.shuffle` was removed. |
| 2098 |
It was previously an arbitrary random function to use for the shuffle; |
| 2099 |
now, :func:`random.random` (its previous default) will always be used. |
| 2100 |
|
| 2101 |
* In :mod:`re` :ref:`re-syntax`, global inline flags (e.g. ``(?i)``) |
| 2102 |
can now only be used at the start of regular expressions. |
| 2103 |
Using them elsewhere has been deprecated since Python 3.6. |
| 2104 |
(Contributed by Serhiy Storchaka in :issue:`47066`.) |
| 2105 |
|
| 2106 |
* In the :mod:`re` module, several long-standing bugs where fixed that, |
| 2107 |
in rare cases, could cause capture groups to get the wrong result. |
| 2108 |
Therefore, this could change the captured output in these cases. |
| 2109 |
(Contributed by Ma Lin in :issue:`35859`.) |
| 2110 |
|
| 2111 |
|
| 2112 |
.. _whatsnew311-build-changes: |
| 2113 |
|
| 2114 |
Build Changes |
| 2115 |
============= |
| 2116 |
|
| 2117 |
* CPython now has :pep:`11` :pep:`Tier 3 support <11#tier-3>` for |
| 2118 |
cross compiling to the `WebAssembly <https://webassembly.org/>`_ platforms |
| 2119 |
`Emscripten <https://emscripten.org/>`_ |
| 2120 |
(``wasm32-unknown-emscripten``, i.e. Python in the browser) |
| 2121 |
and `WebAssembly System Interface (WASI) <https://wasi.dev/>`_ |
| 2122 |
(``wasm32-unknown-wasi``). |
| 2123 |
The effort is inspired by previous work like `Pyodide <https://pyodide.org/>`_. |
| 2124 |
These platforms provide a limited subset of POSIX APIs; Python standard |
| 2125 |
libraries features and modules related to networking, processes, threading, |
| 2126 |
signals, mmap, and users/groups are not available or don't work. |
| 2127 |
(Emscripten contributed by Christian Heimes and Ethan Smith in :gh:`84461` |
| 2128 |
and WASI contributed by Christian Heimes in :gh:`90473`; |
| 2129 |
platforms promoted in :gh:`95085`) |
| 2130 |
|
| 2131 |
* Building CPython now requires: |
| 2132 |
|
| 2133 |
* A `C11 <https://en.cppreference.com/w/c/11>`_ compiler and standard library. |
| 2134 |
`Optional C11 features |
| 2135 |
<https://en.wikipedia.org/wiki/C11_(C_standard_revision)#Optional_features>`_ |
| 2136 |
are not required. |
| 2137 |
(Contributed by Victor Stinner in :issue:`46656`, |
| 2138 |
:issue:`45440` and :issue:`46640`.) |
| 2139 |
|
| 2140 |
* Support for `IEEE 754 <https://en.wikipedia.org/wiki/IEEE_754>`_ |
| 2141 |
floating-point numbers. |
| 2142 |
(Contributed by Victor Stinner in :issue:`46917`.) |
| 2143 |
|
| 2144 |
* The :c:macro:`!Py_NO_NAN` macro has been removed. |
| 2145 |
Since CPython now requires IEEE 754 floats, NaN values are always available. |
| 2146 |
(Contributed by Victor Stinner in :issue:`46656`.) |
| 2147 |
|
| 2148 |
* The :mod:`tkinter` package now requires `Tcl/Tk <https://www.tcl.tk>`_ |
| 2149 |
version 8.5.12 or newer. |
| 2150 |
(Contributed by Serhiy Storchaka in :issue:`46996`.) |
| 2151 |
|
| 2152 |
* Build dependencies, compiler flags, and linker flags for most stdlib |
| 2153 |
extension modules are now detected by :program:`configure`. libffi, libnsl, |
| 2154 |
libsqlite3, zlib, bzip2, liblzma, libcrypt, Tcl/Tk, and uuid flags |
| 2155 |
are detected by `pkg-config |
| 2156 |
<https://www.freedesktop.org/wiki/Software/pkg-config/>`_ (when available). |
| 2157 |
:mod:`tkinter` now requires a pkg-config command |
| 2158 |
to detect development settings for `Tcl/Tk`_ headers and libraries. |
| 2159 |
(Contributed by Christian Heimes and Erlend Egeberg Aasland in |
| 2160 |
:issue:`45847`, :issue:`45747`, and :issue:`45763`.) |
| 2161 |
|
| 2162 |
* libpython is no longer linked against libcrypt. |
| 2163 |
(Contributed by Mike Gilbert in :issue:`45433`.) |
| 2164 |
|
| 2165 |
* CPython can now be built with the |
| 2166 |
`ThinLTO <https://clang.llvm.org/docs/ThinLTO.html>`_ option |
| 2167 |
via passing ``thin`` to :option:`--with-lto`, i.e. ``--with-lto=thin``. |
| 2168 |
(Contributed by Donghee Na and Brett Holman in :issue:`44340`.) |
| 2169 |
|
| 2170 |
* Freelists for object structs can now be disabled. A new :program:`configure` |
| 2171 |
option ``--without-freelists`` can be used to disable all freelists |
| 2172 |
except empty tuple singleton. |
| 2173 |
(Contributed by Christian Heimes in :issue:`45522`.) |
| 2174 |
|
| 2175 |
* ``Modules/Setup`` and ``Modules/makesetup`` have been improved and tied up. |
| 2176 |
Extension modules can now be built through ``makesetup``. All except some |
| 2177 |
test modules can be linked statically into a main binary or library. |
| 2178 |
(Contributed by Brett Cannon and Christian Heimes in :issue:`45548`, |
| 2179 |
:issue:`45570`, :issue:`45571`, and :issue:`43974`.) |
| 2180 |
|
| 2181 |
.. note:: |
| 2182 |
Use the environment variables :envvar:`!TCLTK_CFLAGS` and |
| 2183 |
:envvar:`!TCLTK_LIBS` to manually specify the location of Tcl/Tk headers |
| 2184 |
and libraries. The :program:`configure` options |
| 2185 |
:option:`!--with-tcltk-includes` and :option:`!--with-tcltk-libs` |
| 2186 |
have been removed. |
| 2187 |
|
| 2188 |
On RHEL 7 and CentOS 7 the development packages do not provide ``tcl.pc`` |
| 2189 |
and ``tk.pc``; use ``TCLTK_LIBS="-ltk8.5 -ltkstub8.5 -ltcl8.5"``. |
| 2190 |
The directory ``Misc/rhel7`` contains ``.pc`` files and instructions |
| 2191 |
on how to build Python with RHEL 7's and CentOS 7's Tcl/Tk and OpenSSL. |
| 2192 |
|
| 2193 |
* CPython will now use 30-bit digits by default for the Python :class:`int` |
| 2194 |
implementation. Previously, the default was to use 30-bit digits on platforms |
| 2195 |
with ``SIZEOF_VOID_P >= 8``, and 15-bit digits otherwise. It's still possible |
| 2196 |
to explicitly request use of 15-bit digits via either the |
| 2197 |
:option:`--enable-big-digits` option to the configure script |
| 2198 |
or (for Windows) the ``PYLONG_BITS_IN_DIGIT`` variable in ``PC/pyconfig.h``, |
| 2199 |
but this option may be removed at some point in the future. |
| 2200 |
(Contributed by Mark Dickinson in :issue:`45569`.) |
| 2201 |
|
| 2202 |
|
| 2203 |
.. _whatsnew311-c-api: |
| 2204 |
|
| 2205 |
C API Changes |
| 2206 |
============= |
| 2207 |
|
| 2208 |
.. _whatsnew311-c-api-new-features: |
| 2209 |
|
| 2210 |
New Features |
| 2211 |
------------ |
| 2212 |
|
| 2213 |
* Add a new :c:func:`PyType_GetName` function to get type's short name. |
| 2214 |
(Contributed by Hai Shi in :issue:`42035`.) |
| 2215 |
|
| 2216 |
* Add a new :c:func:`PyType_GetQualName` function to get type's qualified name. |
| 2217 |
(Contributed by Hai Shi in :issue:`42035`.) |
| 2218 |
|
| 2219 |
* Add new :c:func:`PyThreadState_EnterTracing` and |
| 2220 |
:c:func:`PyThreadState_LeaveTracing` functions to the limited C API to |
| 2221 |
suspend and resume tracing and profiling. |
| 2222 |
(Contributed by Victor Stinner in :issue:`43760`.) |
| 2223 |
|
| 2224 |
* Added the :c:data:`Py_Version` constant which bears the same value as |
| 2225 |
:c:macro:`PY_VERSION_HEX`. |
| 2226 |
(Contributed by Gabriele N. Tornetta in :issue:`43931`.) |
| 2227 |
|
| 2228 |
* :c:type:`Py_buffer` and APIs are now part of the limited API and the stable |
| 2229 |
ABI: |
| 2230 |
|
| 2231 |
* :c:func:`PyObject_CheckBuffer` |
| 2232 |
* :c:func:`PyObject_GetBuffer` |
| 2233 |
* :c:func:`PyBuffer_GetPointer` |
| 2234 |
* :c:func:`PyBuffer_SizeFromFormat` |
| 2235 |
* :c:func:`PyBuffer_ToContiguous` |
| 2236 |
* :c:func:`PyBuffer_FromContiguous` |
| 2237 |
* :c:func:`PyObject_CopyData` |
| 2238 |
* :c:func:`PyBuffer_IsContiguous` |
| 2239 |
* :c:func:`PyBuffer_FillContiguousStrides` |
| 2240 |
* :c:func:`PyBuffer_FillInfo` |
| 2241 |
* :c:func:`PyBuffer_Release` |
| 2242 |
* :c:func:`PyMemoryView_FromBuffer` |
| 2243 |
* :c:member:`~PyBufferProcs.bf_getbuffer` and |
| 2244 |
:c:member:`~PyBufferProcs.bf_releasebuffer` type slots |
| 2245 |
|
| 2246 |
(Contributed by Christian Heimes in :issue:`45459`.) |
| 2247 |
|
| 2248 |
* Added the :c:func:`PyType_GetModuleByDef` function, used to get the module |
| 2249 |
in which a method was defined, in cases where this information is not |
| 2250 |
available directly (via :c:type:`PyCMethod`). |
| 2251 |
(Contributed by Petr Viktorin in :issue:`46613`.) |
| 2252 |
|
| 2253 |
* Add new functions to pack and unpack C double (serialize and deserialize): |
| 2254 |
:c:func:`PyFloat_Pack2`, :c:func:`PyFloat_Pack4`, :c:func:`PyFloat_Pack8`, |
| 2255 |
:c:func:`PyFloat_Unpack2`, :c:func:`PyFloat_Unpack4` and |
| 2256 |
:c:func:`PyFloat_Unpack8`. |
| 2257 |
(Contributed by Victor Stinner in :issue:`46906`.) |
| 2258 |
|
| 2259 |
* Add new functions to get frame object attributes: |
| 2260 |
:c:func:`PyFrame_GetBuiltins`, :c:func:`PyFrame_GetGenerator`, |
| 2261 |
:c:func:`PyFrame_GetGlobals`, :c:func:`PyFrame_GetLasti`. |
| 2262 |
|
| 2263 |
* Added two new functions to get and set the active exception instance: |
| 2264 |
:c:func:`PyErr_GetHandledException` and :c:func:`PyErr_SetHandledException`. |
| 2265 |
These are alternatives to :c:func:`PyErr_SetExcInfo()` and |
| 2266 |
:c:func:`PyErr_GetExcInfo()` which work with the legacy 3-tuple |
| 2267 |
representation of exceptions. |
| 2268 |
(Contributed by Irit Katriel in :issue:`46343`.) |
| 2269 |
|
| 2270 |
* Added the :c:member:`PyConfig.safe_path` member. |
| 2271 |
(Contributed by Victor Stinner in :gh:`57684`.) |
| 2272 |
|
| 2273 |
|
| 2274 |
.. _whatsnew311-c-api-porting: |
| 2275 |
|
| 2276 |
Porting to Python 3.11 |
| 2277 |
---------------------- |
| 2278 |
|
| 2279 |
.. _whatsnew311-pep670: |
| 2280 |
|
| 2281 |
* Some macros have been converted to static inline functions to avoid |
| 2282 |
`macro pitfalls <https://gcc.gnu.org/onlinedocs/cpp/Macro-Pitfalls.html>`_. |
| 2283 |
The change should be mostly transparent to users, |
| 2284 |
as the replacement functions will cast their arguments to the expected types |
| 2285 |
to avoid compiler warnings due to static type checks. |
| 2286 |
However, when the limited C API is set to >=3.11, |
| 2287 |
these casts are not done, |
| 2288 |
and callers will need to cast arguments to their expected types. |
| 2289 |
See :pep:`670` for more details. |
| 2290 |
(Contributed by Victor Stinner and Erlend E. Aasland in :gh:`89653`.) |
| 2291 |
|
| 2292 |
* :c:func:`PyErr_SetExcInfo()` no longer uses the ``type`` and ``traceback`` |
| 2293 |
arguments, the interpreter now derives those values from the exception |
| 2294 |
instance (the ``value`` argument). The function still steals references |
| 2295 |
of all three arguments. |
| 2296 |
(Contributed by Irit Katriel in :issue:`45711`.) |
| 2297 |
|
| 2298 |
* :c:func:`PyErr_GetExcInfo()` now derives the ``type`` and ``traceback`` |
| 2299 |
fields of the result from the exception instance (the ``value`` field). |
| 2300 |
(Contributed by Irit Katriel in :issue:`45711`.) |
| 2301 |
|
| 2302 |
* :c:struct:`_frozen` has a new ``is_package`` field to indicate whether |
| 2303 |
or not the frozen module is a package. Previously, a negative value |
| 2304 |
in the ``size`` field was the indicator. Now only non-negative values |
| 2305 |
be used for ``size``. |
| 2306 |
(Contributed by Kumar Aditya in :issue:`46608`.) |
| 2307 |
|
| 2308 |
* :c:func:`_PyFrameEvalFunction` now takes ``_PyInterpreterFrame*`` |
| 2309 |
as its second parameter, instead of ``PyFrameObject*``. |
| 2310 |
See :pep:`523` for more details of how to use this function pointer type. |
| 2311 |
|
| 2312 |
* :c:func:`!PyCode_New` and :c:func:`!PyCode_NewWithPosOnlyArgs` now take |
| 2313 |
an additional ``exception_table`` argument. |
| 2314 |
Using these functions should be avoided, if at all possible. |
| 2315 |
To get a custom code object: create a code object using the compiler, |
| 2316 |
then get a modified version with the ``replace`` method. |
| 2317 |
|
| 2318 |
* :c:type:`PyCodeObject` no longer has the ``co_code``, ``co_varnames``, |
| 2319 |
``co_cellvars`` and ``co_freevars`` fields. Instead, use |
| 2320 |
:c:func:`PyCode_GetCode`, :c:func:`PyCode_GetVarnames`, |
| 2321 |
:c:func:`PyCode_GetCellvars` and :c:func:`PyCode_GetFreevars` respectively |
| 2322 |
to access them via the C API. |
| 2323 |
(Contributed by Brandt Bucher in :issue:`46841` and Ken Jin in :gh:`92154` |
| 2324 |
and :gh:`94936`.) |
| 2325 |
|
| 2326 |
* The old trashcan macros (``Py_TRASHCAN_SAFE_BEGIN``/``Py_TRASHCAN_SAFE_END``) |
| 2327 |
are now deprecated. They should be replaced by the new macros |
| 2328 |
``Py_TRASHCAN_BEGIN`` and ``Py_TRASHCAN_END``. |
| 2329 |
|
| 2330 |
A tp_dealloc function that has the old macros, such as:: |
| 2331 |
|
| 2332 |
static void |
| 2333 |
mytype_dealloc(mytype *p) |
| 2334 |
{ |
| 2335 |
PyObject_GC_UnTrack(p); |
| 2336 |
Py_TRASHCAN_SAFE_BEGIN(p); |
| 2337 |
... |
| 2338 |
Py_TRASHCAN_SAFE_END |
| 2339 |
} |
| 2340 |
|
| 2341 |
should migrate to the new macros as follows:: |
| 2342 |
|
| 2343 |
static void |
| 2344 |
mytype_dealloc(mytype *p) |
| 2345 |
{ |
| 2346 |
PyObject_GC_UnTrack(p); |
| 2347 |
Py_TRASHCAN_BEGIN(p, mytype_dealloc) |
| 2348 |
... |
| 2349 |
Py_TRASHCAN_END |
| 2350 |
} |
| 2351 |
|
| 2352 |
Note that ``Py_TRASHCAN_BEGIN`` has a second argument which |
| 2353 |
should be the deallocation function it is in. |
| 2354 |
|
| 2355 |
To support older Python versions in the same codebase, you |
| 2356 |
can define the following macros and use them throughout |
| 2357 |
the code (credit: these were copied from the ``mypy`` codebase):: |
| 2358 |
|
| 2359 |
#if PY_VERSION_HEX >= 0x03080000 |
| 2360 |
# define CPy_TRASHCAN_BEGIN(op, dealloc) Py_TRASHCAN_BEGIN(op, dealloc) |
| 2361 |
# define CPy_TRASHCAN_END(op) Py_TRASHCAN_END |
| 2362 |
#else |
| 2363 |
# define CPy_TRASHCAN_BEGIN(op, dealloc) Py_TRASHCAN_SAFE_BEGIN(op) |
| 2364 |
# define CPy_TRASHCAN_END(op) Py_TRASHCAN_SAFE_END(op) |
| 2365 |
#endif |
| 2366 |
|
| 2367 |
* The :c:func:`PyType_Ready` function now raises an error if a type is defined |
| 2368 |
with the :c:macro:`Py_TPFLAGS_HAVE_GC` flag set but has no traverse function |
| 2369 |
(:c:member:`PyTypeObject.tp_traverse`). |
| 2370 |
(Contributed by Victor Stinner in :issue:`44263`.) |
| 2371 |
|
| 2372 |
* Heap types with the :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` flag can now inherit |
| 2373 |
the :pep:`590` vectorcall protocol. Previously, this was only possible for |
| 2374 |
:ref:`static types <static-types>`. |
| 2375 |
(Contributed by Erlend E. Aasland in :issue:`43908`) |
| 2376 |
|
| 2377 |
* Since :c:func:`Py_TYPE()` is changed to a inline static function, |
| 2378 |
``Py_TYPE(obj) = new_type`` must be replaced with |
| 2379 |
``Py_SET_TYPE(obj, new_type)``: see the :c:func:`Py_SET_TYPE()` function |
| 2380 |
(available since Python 3.9). For backward compatibility, this macro can be |
| 2381 |
used:: |
| 2382 |
|
| 2383 |
#if PY_VERSION_HEX < 0x030900A4 && !defined(Py_SET_TYPE) |
| 2384 |
static inline void _Py_SET_TYPE(PyObject *ob, PyTypeObject *type) |
| 2385 |
{ ob->ob_type = type; } |
| 2386 |
#define Py_SET_TYPE(ob, type) _Py_SET_TYPE((PyObject*)(ob), type) |
| 2387 |
#endif |
| 2388 |
|
| 2389 |
(Contributed by Victor Stinner in :issue:`39573`.) |
| 2390 |
|
| 2391 |
* Since :c:func:`Py_SIZE()` is changed to a inline static function, |
| 2392 |
``Py_SIZE(obj) = new_size`` must be replaced with |
| 2393 |
``Py_SET_SIZE(obj, new_size)``: see the :c:func:`Py_SET_SIZE()` function |
| 2394 |
(available since Python 3.9). For backward compatibility, this macro can be |
| 2395 |
used:: |
| 2396 |
|
| 2397 |
#if PY_VERSION_HEX < 0x030900A4 && !defined(Py_SET_SIZE) |
| 2398 |
static inline void _Py_SET_SIZE(PyVarObject *ob, Py_ssize_t size) |
| 2399 |
{ ob->ob_size = size; } |
| 2400 |
#define Py_SET_SIZE(ob, size) _Py_SET_SIZE((PyVarObject*)(ob), size) |
| 2401 |
#endif |
| 2402 |
|
| 2403 |
(Contributed by Victor Stinner in :issue:`39573`.) |
| 2404 |
|
| 2405 |
* ``<Python.h>`` no longer includes the header files ``<stdlib.h>``, |
| 2406 |
``<stdio.h>``, ``<errno.h>`` and ``<string.h>`` when the ``Py_LIMITED_API`` |
| 2407 |
macro is set to ``0x030b0000`` (Python 3.11) or higher. C extensions should |
| 2408 |
explicitly include the header files after ``#include <Python.h>``. |
| 2409 |
(Contributed by Victor Stinner in :issue:`45434`.) |
| 2410 |
|
| 2411 |
* The non-limited API files ``cellobject.h``, ``classobject.h``, ``code.h``, ``context.h``, |
| 2412 |
``funcobject.h``, ``genobject.h`` and ``longintrepr.h`` have been moved to |
| 2413 |
the ``Include/cpython`` directory. Moreover, the ``eval.h`` header file was |
| 2414 |
removed. These files must not be included directly, as they are already |
| 2415 |
included in ``Python.h``: :ref:`Include Files <api-includes>`. If they have |
| 2416 |
been included directly, consider including ``Python.h`` instead. |
| 2417 |
(Contributed by Victor Stinner in :issue:`35134`.) |
| 2418 |
|
| 2419 |
* The :c:func:`!PyUnicode_CHECK_INTERNED` macro has been excluded from the |
| 2420 |
limited C API. It was never usable there, because it used internal structures |
| 2421 |
which are not available in the limited C API. |
| 2422 |
(Contributed by Victor Stinner in :issue:`46007`.) |
| 2423 |
|
| 2424 |
* The following frame functions and type are now directly available with |
| 2425 |
``#include <Python.h>``, it's no longer needed to add |
| 2426 |
``#include <frameobject.h>``: |
| 2427 |
|
| 2428 |
* :c:func:`PyFrame_Check` |
| 2429 |
* :c:func:`PyFrame_GetBack` |
| 2430 |
* :c:func:`PyFrame_GetBuiltins` |
| 2431 |
* :c:func:`PyFrame_GetGenerator` |
| 2432 |
* :c:func:`PyFrame_GetGlobals` |
| 2433 |
* :c:func:`PyFrame_GetLasti` |
| 2434 |
* :c:func:`PyFrame_GetLocals` |
| 2435 |
* :c:type:`PyFrame_Type` |
| 2436 |
|
| 2437 |
(Contributed by Victor Stinner in :gh:`93937`.) |
| 2438 |
|
| 2439 |
.. _pyframeobject-3.11-hiding: |
| 2440 |
|
| 2441 |
* The :c:type:`PyFrameObject` structure members have been removed from the |
| 2442 |
public C API. |
| 2443 |
|
| 2444 |
While the documentation notes that the :c:type:`PyFrameObject` fields are |
| 2445 |
subject to change at any time, they have been stable for a long time and were |
| 2446 |
used in several popular extensions. |
| 2447 |
|
| 2448 |
In Python 3.11, the frame struct was reorganized to allow performance |
| 2449 |
optimizations. Some fields were removed entirely, as they were details of the |
| 2450 |
old implementation. |
| 2451 |
|
| 2452 |
:c:type:`PyFrameObject` fields: |
| 2453 |
|
| 2454 |
* ``f_back``: use :c:func:`PyFrame_GetBack`. |
| 2455 |
* ``f_blockstack``: removed. |
| 2456 |
* ``f_builtins``: use :c:func:`PyFrame_GetBuiltins`. |
| 2457 |
* ``f_code``: use :c:func:`PyFrame_GetCode`. |
| 2458 |
* ``f_gen``: use :c:func:`PyFrame_GetGenerator`. |
| 2459 |
* ``f_globals``: use :c:func:`PyFrame_GetGlobals`. |
| 2460 |
* ``f_iblock``: removed. |
| 2461 |
* ``f_lasti``: use :c:func:`PyFrame_GetLasti`. |
| 2462 |
Code using ``f_lasti`` with ``PyCode_Addr2Line()`` should use |
| 2463 |
:c:func:`PyFrame_GetLineNumber` instead; it may be faster. |
| 2464 |
* ``f_lineno``: use :c:func:`PyFrame_GetLineNumber` |
| 2465 |
* ``f_locals``: use :c:func:`PyFrame_GetLocals`. |
| 2466 |
* ``f_stackdepth``: removed. |
| 2467 |
* ``f_state``: no public API (renamed to ``f_frame.f_state``). |
| 2468 |
* ``f_trace``: no public API. |
| 2469 |
* ``f_trace_lines``: use ``PyObject_GetAttrString((PyObject*)frame, "f_trace_lines")``. |
| 2470 |
* ``f_trace_opcodes``: use ``PyObject_GetAttrString((PyObject*)frame, "f_trace_opcodes")``. |
| 2471 |
* ``f_localsplus``: no public API (renamed to ``f_frame.localsplus``). |
| 2472 |
* ``f_valuestack``: removed. |
| 2473 |
|
| 2474 |
The Python frame object is now created lazily. A side effect is that the |
| 2475 |
:attr:`~frame.f_back` member must not be accessed directly, |
| 2476 |
since its value is now also |
| 2477 |
computed lazily. The :c:func:`PyFrame_GetBack` function must be called |
| 2478 |
instead. |
| 2479 |
|
| 2480 |
Debuggers that accessed the :attr:`~frame.f_locals` directly *must* call |
| 2481 |
:c:func:`PyFrame_GetLocals` instead. They no longer need to call |
| 2482 |
:c:func:`!PyFrame_FastToLocalsWithError` or :c:func:`!PyFrame_LocalsToFast`, |
| 2483 |
in fact they should not call those functions. The necessary updating of the |
| 2484 |
frame is now managed by the virtual machine. |
| 2485 |
|
| 2486 |
Code defining ``PyFrame_GetCode()`` on Python 3.8 and older:: |
| 2487 |
|
| 2488 |
#if PY_VERSION_HEX < 0x030900B1 |
| 2489 |
static inline PyCodeObject* PyFrame_GetCode(PyFrameObject *frame) |
| 2490 |
{ |
| 2491 |
Py_INCREF(frame->f_code); |
| 2492 |
return frame->f_code; |
| 2493 |
} |
| 2494 |
#endif |
| 2495 |
|
| 2496 |
Code defining ``PyFrame_GetBack()`` on Python 3.8 and older:: |
| 2497 |
|
| 2498 |
#if PY_VERSION_HEX < 0x030900B1 |
| 2499 |
static inline PyFrameObject* PyFrame_GetBack(PyFrameObject *frame) |
| 2500 |
{ |
| 2501 |
Py_XINCREF(frame->f_back); |
| 2502 |
return frame->f_back; |
| 2503 |
} |
| 2504 |
#endif |
| 2505 |
|
| 2506 |
Or use the `pythoncapi_compat project |
| 2507 |
<https://github.com/python/pythoncapi-compat>`__ to get these two |
| 2508 |
functions on older Python versions. |
| 2509 |
|
| 2510 |
* Changes of the :c:type:`PyThreadState` structure members: |
| 2511 |
|
| 2512 |
* ``frame``: removed, use :c:func:`PyThreadState_GetFrame` (function added |
| 2513 |
to Python 3.9 by :issue:`40429`). |
| 2514 |
Warning: the function returns a :term:`strong reference`, need to call |
| 2515 |
:c:func:`Py_XDECREF`. |
| 2516 |
* ``tracing``: changed, use :c:func:`PyThreadState_EnterTracing` |
| 2517 |
and :c:func:`PyThreadState_LeaveTracing` |
| 2518 |
(functions added to Python 3.11 by :issue:`43760`). |
| 2519 |
* ``recursion_depth``: removed, |
| 2520 |
use ``(tstate->recursion_limit - tstate->recursion_remaining)`` instead. |
| 2521 |
* ``stackcheck_counter``: removed. |
| 2522 |
|
| 2523 |
Code defining ``PyThreadState_GetFrame()`` on Python 3.8 and older:: |
| 2524 |
|
| 2525 |
#if PY_VERSION_HEX < 0x030900B1 |
| 2526 |
static inline PyFrameObject* PyThreadState_GetFrame(PyThreadState *tstate) |
| 2527 |
{ |
| 2528 |
Py_XINCREF(tstate->frame); |
| 2529 |
return tstate->frame; |
| 2530 |
} |
| 2531 |
#endif |
| 2532 |
|
| 2533 |
Code defining ``PyThreadState_EnterTracing()`` and |
| 2534 |
``PyThreadState_LeaveTracing()`` on Python 3.10 and older:: |
| 2535 |
|
| 2536 |
#if PY_VERSION_HEX < 0x030B00A2 |
| 2537 |
static inline void PyThreadState_EnterTracing(PyThreadState *tstate) |
| 2538 |
{ |
| 2539 |
tstate->tracing++; |
| 2540 |
#if PY_VERSION_HEX >= 0x030A00A1 |
| 2541 |
tstate->cframe->use_tracing = 0; |
| 2542 |
#else |
| 2543 |
tstate->use_tracing = 0; |
| 2544 |
#endif |
| 2545 |
} |
| 2546 |
|
| 2547 |
static inline void PyThreadState_LeaveTracing(PyThreadState *tstate) |
| 2548 |
{ |
| 2549 |
int use_tracing = (tstate->c_tracefunc != NULL || tstate->c_profilefunc != NULL); |
| 2550 |
tstate->tracing--; |
| 2551 |
#if PY_VERSION_HEX >= 0x030A00A1 |
| 2552 |
tstate->cframe->use_tracing = use_tracing; |
| 2553 |
#else |
| 2554 |
tstate->use_tracing = use_tracing; |
| 2555 |
#endif |
| 2556 |
} |
| 2557 |
#endif |
| 2558 |
|
| 2559 |
Or use `the pythoncapi-compat project |
| 2560 |
<https://github.com/python/pythoncapi-compat>`__ to get these functions |
| 2561 |
on old Python functions. |
| 2562 |
|
| 2563 |
* Distributors are encouraged to build Python with the optimized Blake2 |
| 2564 |
library `libb2`_. |
| 2565 |
|
| 2566 |
* The :c:member:`PyConfig.module_search_paths_set` field must now be set to 1 for |
| 2567 |
initialization to use :c:member:`PyConfig.module_search_paths` to initialize |
| 2568 |
:data:`sys.path`. Otherwise, initialization will recalculate the path and replace |
| 2569 |
any values added to ``module_search_paths``. |
| 2570 |
|
| 2571 |
* :c:func:`PyConfig_Read` no longer calculates the initial search path, and will not |
| 2572 |
fill any values into :c:member:`PyConfig.module_search_paths`. To calculate default |
| 2573 |
paths and then modify them, finish initialization and use :c:func:`PySys_GetObject` |
| 2574 |
to retrieve :data:`sys.path` as a Python list object and modify it directly. |
| 2575 |
|
| 2576 |
|
| 2577 |
.. _whatsnew311-c-api-deprecated: |
| 2578 |
|
| 2579 |
Deprecated |
| 2580 |
---------- |
| 2581 |
|
| 2582 |
* Deprecate the following functions to configure the Python initialization: |
| 2583 |
|
| 2584 |
* :c:func:`!PySys_AddWarnOptionUnicode` |
| 2585 |
* :c:func:`!PySys_AddWarnOption` |
| 2586 |
* :c:func:`!PySys_AddXOption` |
| 2587 |
* :c:func:`!PySys_HasWarnOptions` |
| 2588 |
* :c:func:`!PySys_SetArgvEx` |
| 2589 |
* :c:func:`!PySys_SetArgv` |
| 2590 |
* :c:func:`!PySys_SetPath` |
| 2591 |
* :c:func:`!Py_SetPath` |
| 2592 |
* :c:func:`!Py_SetProgramName` |
| 2593 |
* :c:func:`!Py_SetPythonHome` |
| 2594 |
* :c:func:`!Py_SetStandardStreamEncoding` |
| 2595 |
* :c:func:`!_Py_SetProgramFullPath` |
| 2596 |
|
| 2597 |
Use the new :c:type:`PyConfig` API of the :ref:`Python Initialization Configuration |
| 2598 |
<init-config>` instead (:pep:`587`). |
| 2599 |
(Contributed by Victor Stinner in :gh:`88279`.) |
| 2600 |
|
| 2601 |
* Deprecate the ``ob_shash`` member of the :c:type:`PyBytesObject`. Use :c:func:`PyObject_Hash` instead. |
| 2602 |
(Contributed by Inada Naoki in :issue:`46864`.) |
| 2603 |
|
| 2604 |
|
| 2605 |
.. _whatsnew311-c-api-pending-removal: |
| 2606 |
|
| 2607 |
Pending Removal in Python 3.12 |
| 2608 |
------------------------------ |
| 2609 |
|
| 2610 |
The following C APIs have been deprecated in earlier Python releases, |
| 2611 |
and will be removed in Python 3.12. |
| 2612 |
|
| 2613 |
* :c:func:`!PyUnicode_AS_DATA` |
| 2614 |
* :c:func:`!PyUnicode_AS_UNICODE` |
| 2615 |
* :c:func:`!PyUnicode_AsUnicodeAndSize` |
| 2616 |
* :c:func:`!PyUnicode_AsUnicode` |
| 2617 |
* :c:func:`!PyUnicode_FromUnicode` |
| 2618 |
* :c:func:`!PyUnicode_GET_DATA_SIZE` |
| 2619 |
* :c:func:`!PyUnicode_GET_SIZE` |
| 2620 |
* :c:func:`!PyUnicode_GetSize` |
| 2621 |
* :c:func:`!PyUnicode_IS_COMPACT` |
| 2622 |
* :c:func:`!PyUnicode_IS_READY` |
| 2623 |
* :c:func:`PyUnicode_READY` |
| 2624 |
* :c:func:`!PyUnicode_WSTR_LENGTH` |
| 2625 |
* :c:func:`!_PyUnicode_AsUnicode` |
| 2626 |
* :c:macro:`!PyUnicode_WCHAR_KIND` |
| 2627 |
* :c:type:`PyUnicodeObject` |
| 2628 |
* :c:func:`!PyUnicode_InternImmortal` |
| 2629 |
|
| 2630 |
|
| 2631 |
.. _whatsnew311-c-api-removed: |
| 2632 |
|
| 2633 |
Removed |
| 2634 |
------- |
| 2635 |
|
| 2636 |
* :c:func:`!PyFrame_BlockSetup` and :c:func:`!PyFrame_BlockPop` have been |
| 2637 |
removed. |
| 2638 |
(Contributed by Mark Shannon in :issue:`40222`.) |
| 2639 |
|
| 2640 |
* Remove the following math macros using the ``errno`` variable: |
| 2641 |
|
| 2642 |
* ``Py_ADJUST_ERANGE1()`` |
| 2643 |
* ``Py_ADJUST_ERANGE2()`` |
| 2644 |
* ``Py_OVERFLOWED()`` |
| 2645 |
* ``Py_SET_ERANGE_IF_OVERFLOW()`` |
| 2646 |
* ``Py_SET_ERRNO_ON_MATH_ERROR()`` |
| 2647 |
|
| 2648 |
(Contributed by Victor Stinner in :issue:`45412`.) |
| 2649 |
|
| 2650 |
* Remove ``Py_UNICODE_COPY()`` and ``Py_UNICODE_FILL()`` macros, deprecated |
| 2651 |
since Python 3.3. Use ``PyUnicode_CopyCharacters()`` or ``memcpy()`` |
| 2652 |
(``wchar_t*`` string), and ``PyUnicode_Fill()`` functions instead. |
| 2653 |
(Contributed by Victor Stinner in :issue:`41123`.) |
| 2654 |
|
| 2655 |
* Remove the ``pystrhex.h`` header file. It only contains private functions. |
| 2656 |
C extensions should only include the main ``<Python.h>`` header file. |
| 2657 |
(Contributed by Victor Stinner in :issue:`45434`.) |
| 2658 |
|
| 2659 |
* Remove the ``Py_FORCE_DOUBLE()`` macro. It was used by the |
| 2660 |
``Py_IS_INFINITY()`` macro. |
| 2661 |
(Contributed by Victor Stinner in :issue:`45440`.) |
| 2662 |
|
| 2663 |
* The following items are no longer available when :c:macro:`Py_LIMITED_API` |
| 2664 |
is defined: |
| 2665 |
|
| 2666 |
* :c:func:`PyMarshal_WriteLongToFile` |
| 2667 |
* :c:func:`PyMarshal_WriteObjectToFile` |
| 2668 |
* :c:func:`PyMarshal_ReadObjectFromString` |
| 2669 |
* :c:func:`PyMarshal_WriteObjectToString` |
| 2670 |
* the ``Py_MARSHAL_VERSION`` macro |
| 2671 |
|
| 2672 |
These are not part of the :ref:`limited API <limited-api-list>`. |
| 2673 |
|
| 2674 |
(Contributed by Victor Stinner in :issue:`45474`.) |
| 2675 |
|
| 2676 |
* Exclude :c:func:`PyWeakref_GET_OBJECT` from the limited C API. It never |
| 2677 |
worked since the :c:type:`!PyWeakReference` structure is opaque in the |
| 2678 |
limited C API. |
| 2679 |
(Contributed by Victor Stinner in :issue:`35134`.) |
| 2680 |
|
| 2681 |
* Remove the ``PyHeapType_GET_MEMBERS()`` macro. It was exposed in the |
| 2682 |
public C API by mistake, it must only be used by Python internally. |
| 2683 |
Use the ``PyTypeObject.tp_members`` member instead. |
| 2684 |
(Contributed by Victor Stinner in :issue:`40170`.) |
| 2685 |
|
| 2686 |
* Remove the ``HAVE_PY_SET_53BIT_PRECISION`` macro (moved to the internal C |
| 2687 |
API). |
| 2688 |
(Contributed by Victor Stinner in :issue:`45412`.) |
| 2689 |
|
| 2690 |
.. _whatsnew311-pep624: |
| 2691 |
|
| 2692 |
* Remove the :c:type:`Py_UNICODE` encoder APIs, |
| 2693 |
as they have been deprecated since Python 3.3, |
| 2694 |
are little used |
| 2695 |
and are inefficient relative to the recommended alternatives. |
| 2696 |
|
| 2697 |
The removed functions are: |
| 2698 |
|
| 2699 |
* :func:`!PyUnicode_Encode` |
| 2700 |
* :func:`!PyUnicode_EncodeASCII` |
| 2701 |
* :func:`!PyUnicode_EncodeLatin1` |
| 2702 |
* :func:`!PyUnicode_EncodeUTF7` |
| 2703 |
* :func:`!PyUnicode_EncodeUTF8` |
| 2704 |
* :func:`!PyUnicode_EncodeUTF16` |
| 2705 |
* :func:`!PyUnicode_EncodeUTF32` |
| 2706 |
* :func:`!PyUnicode_EncodeUnicodeEscape` |
| 2707 |
* :func:`!PyUnicode_EncodeRawUnicodeEscape` |
| 2708 |
* :func:`!PyUnicode_EncodeCharmap` |
| 2709 |
* :func:`!PyUnicode_TranslateCharmap` |
| 2710 |
* :func:`!PyUnicode_EncodeDecimal` |
| 2711 |
* :func:`!PyUnicode_TransformDecimalToASCII` |
| 2712 |
|
| 2713 |
See :pep:`624` for details and |
| 2714 |
:pep:`migration guidance <624#alternative-apis>`. |
| 2715 |
(Contributed by Inada Naoki in :issue:`44029`.) |
| 2716 |
|
| 2717 |
|
| 2718 |
Notable changes in 3.11.4 |
| 2719 |
========================= |
| 2720 |
|
| 2721 |
tarfile |
| 2722 |
------- |
| 2723 |
|
| 2724 |
* The extraction methods in :mod:`tarfile`, and :func:`shutil.unpack_archive`, |
| 2725 |
have a new a *filter* argument that allows limiting tar features than may be |
| 2726 |
surprising or dangerous, such as creating files outside the destination |
| 2727 |
directory. |
| 2728 |
See :ref:`tarfile-extraction-filter` for details. |
| 2729 |
In Python 3.12, use without the *filter* argument will show a |
| 2730 |
:exc:`DeprecationWarning`. |
| 2731 |
In Python 3.14, the default will switch to ``'data'``. |
| 2732 |
(Contributed by Petr Viktorin in :pep:`706`.) |
| 2733 |
|
| 2734 |
|
| 2735 |
Notable changes in 3.11.5 |
| 2736 |
========================= |
| 2737 |
|
| 2738 |
OpenSSL |
| 2739 |
------- |
| 2740 |
|
| 2741 |
* Windows builds and macOS installers from python.org now use OpenSSL 3.0. |
| 2742 |
|
| 2743 |
|
| 2744 |
.. _libb2: https://www.blake2.net/ |