Debian Codegraph @

Doc/whatsnew/3.11.rst

raw
1 ****************************
2 What's New In Python 3.11
3 ****************************
4
5 :Editor: Pablo Galindo Salgado
6
7 .. Rules for maintenance:
8
9 * Anyone can add text to this document. Do not spend very much time
10 on the wording of your changes, because your text will probably
11 get rewritten to some degree.
12
13 * The maintainer will go through Misc/NEWS periodically and add
14 changes; it's therefore more important to add your changes to
15 Misc/NEWS than to this file.
16
17 * This is not a complete list of every single change; completeness
18 is the purpose of Misc/NEWS. Some changes I consider too small
19 or esoteric to include. If such a change is added to the text,
20 I'll just remove it. (This is another reason you shouldn't spend
21 too much time on writing your addition.)
22
23 * If you want to draw your new text to the attention of the
24 maintainer, add 'XXX' to the beginning of the paragraph or
25 section.
26
27 * It's OK to just add a fragmentary note about a change. For
28 example: "XXX Describe the transmogrify() function added to the
29 socket module." The maintainer will research the change and
30 write the necessary text.
31
32 * You can comment out your additions if you like, but it's not
33 necessary (especially when a final release is some months away).
34
35 * Credit the author of a patch or bugfix. Just the name is
36 sufficient; the e-mail address isn't necessary.
37
38 * It's helpful to add the bug/patch number as a comment:
39
40 XXX Describe the transmogrify() function added to the socket
41 module.
42 (Contributed by P.Y. Developer in :issue:`12345`.)
43
44 This saves the maintainer the effort of going through the Mercurial log
45 when researching a change.
46
47 This article explains the new features in Python 3.11, compared to 3.10.
48 Python 3.11 was released on October 24, 2022.
49 For full details, see the :ref:`changelog <changelog>`.
50
51
52 .. _whatsnew311-summary:
53
54 Summary -- Release highlights
55 =============================
56
57 .. This section singles out the most important changes in Python 3.11.
58 Brevity is key.
59
60 * Python 3.11 is between 10-60% faster than Python 3.10.
61 On average, we measured a 1.25x speedup on the standard benchmark suite.
62 See :ref:`whatsnew311-faster-cpython` for details.
63
64 .. PEP-sized items next.
65
66 New syntax features:
67
68 * :ref:`whatsnew311-pep654`
69
70 New built-in features:
71
72 * :ref:`whatsnew311-pep678`
73
74 New standard library modules:
75
76 * :pep:`680`: :mod:`tomllib` —
77 Support for parsing `TOML <https://toml.io/>`_ in the Standard Library
78
79 Interpreter improvements:
80
81 * :ref:`whatsnew311-pep657`
82 * New :option:`-P` command line option and :envvar:`PYTHONSAFEPATH` environment
83 variable to :ref:`disable automatically prepending potentially unsafe paths
84 <whatsnew311-pythonsafepath>` to :data:`sys.path`
85
86 New typing features:
87
88 * :ref:`whatsnew311-pep646`
89 * :ref:`whatsnew311-pep655`
90 * :ref:`whatsnew311-pep673`
91 * :ref:`whatsnew311-pep675`
92 * :ref:`whatsnew311-pep681`
93
94 Important deprecations, removals and restrictions:
95
96 * :pep:`594`:
97 :ref:`Many legacy standard library modules have been deprecated
98 <whatsnew311-pep594>` and will be removed in Python 3.13
99 * :pep:`624`:
100 :ref:`Py_UNICODE encoder APIs have been removed <whatsnew311-pep624>`
101 * :pep:`670`:
102 :ref:`Macros converted to static inline functions <whatsnew311-pep670>`
103
104
105 .. _whatsnew311-features:
106
107 New Features
108 ============
109
110 .. _whatsnew311-pep657:
111
112 PEP 657: Fine-grained error locations in tracebacks
113 ---------------------------------------------------
114
115 When printing tracebacks, the interpreter will now point to the exact expression
116 that caused the error, instead of just the line. For example:
117
118 .. code-block:: python
119
120 Traceback (most recent call last):
121 File "distance.py", line 11, in <module>
122 print(manhattan_distance(p1, p2))
123 ^^^^^^^^^^^^^^^^^^^^^^^^^^
124 File "distance.py", line 6, in manhattan_distance
125 return abs(point_1.x - point_2.x) + abs(point_1.y - point_2.y)
126 ^^^^^^^^^
127 AttributeError: 'NoneType' object has no attribute 'x'
128
129 Previous versions of the interpreter would point to just the line, making it
130 ambiguous which object was ``None``. These enhanced errors can also be helpful
131 when dealing with deeply nested :class:`dict` objects and multiple function calls:
132
133 .. code-block:: python
134
135 Traceback (most recent call last):
136 File "query.py", line 37, in <module>
137 magic_arithmetic('foo')
138 File "query.py", line 18, in magic_arithmetic
139 return add_counts(x) / 25
140 ^^^^^^^^^^^^^
141 File "query.py", line 24, in add_counts
142 return 25 + query_user(user1) + query_user(user2)
143 ^^^^^^^^^^^^^^^^^
144 File "query.py", line 32, in query_user
145 return 1 + query_count(db, response['a']['b']['c']['user'], retry=True)
146 ~~~~~~~~~~~~~~~~~~^^^^^
147 TypeError: 'NoneType' object is not subscriptable
148
149 As well as complex arithmetic expressions:
150
151 .. code-block:: python
152
153 Traceback (most recent call last):
154 File "calculation.py", line 54, in <module>
155 result = (x / y / z) * (a / b / c)
156 ~~~~~~^~~
157 ZeroDivisionError: division by zero
158
159 Additionally, the information used by the enhanced traceback feature
160 is made available via a general API, that can be used to correlate
161 :term:`bytecode` :ref:`instructions <bytecodes>` with source code location.
162 This information can be retrieved using:
163
164 - The :meth:`codeobject.co_positions` method in Python.
165 - The :c:func:`PyCode_Addr2Location` function in the C API.
166
167 See :pep:`657` for more details. (Contributed by Pablo Galindo, Batuhan Taskaya
168 and Ammar Askar in :issue:`43950`.)
169
170 .. note::
171 This feature requires storing column positions in :ref:`codeobjects`,
172 which may result in a small increase in interpreter memory usage
173 and disk usage for compiled Python files.
174 To avoid storing the extra information
175 and deactivate printing the extra traceback information,
176 use the :option:`-X no_debug_ranges <-X>` command line option
177 or the :envvar:`PYTHONNODEBUGRANGES` environment variable.
178
179
180 .. _whatsnew311-pep654:
181
182 PEP 654: Exception Groups and ``except*``
183 -----------------------------------------
184
185 :pep:`654` introduces language features that enable a program
186 to raise and handle multiple unrelated exceptions simultaneously.
187 The builtin types :exc:`ExceptionGroup` and :exc:`BaseExceptionGroup`
188 make it possible to group exceptions and raise them together,
189 and the new :keyword:`except* <except_star>` syntax generalizes
190 :keyword:`except` to match subgroups of exception groups.
191
192 See :pep:`654` for more details.
193
194 (Contributed by Irit Katriel in :issue:`45292`. PEP written by
195 Irit Katriel, Yury Selivanov and Guido van Rossum.)
196
197
198 .. _whatsnew311-pep678:
199
200 PEP 678: Exceptions can be enriched with notes
201 ----------------------------------------------
202
203 The :meth:`~BaseException.add_note` method is added to :exc:`BaseException`.
204 It can be used to enrich exceptions with context information
205 that is not available at the time when the exception is raised.
206 The added notes appear in the default traceback.
207
208 See :pep:`678` for more details.
209
210 (Contributed by Irit Katriel in :issue:`45607`.
211 PEP written by Zac Hatfield-Dodds.)
212
213
214 .. _whatsnew311-windows-launcher:
215
216 Windows ``py.exe`` launcher improvements
217 ----------------------------------------
218
219 The copy of the :ref:`launcher` included with Python 3.11 has been significantly
220 updated. It now supports company/tag syntax as defined in :pep:`514` using the
221 :samp:`-V:{<company>}/{<tag>}` argument instead of the limited :samp:`-{<major>}.{<minor>}`.
222 This allows launching distributions other than ``PythonCore``,
223 the one hosted on `python.org <https://www.python.org>`_.
224
225 When using ``-V:`` selectors, either company or tag can be omitted, but all
226 installs will be searched. For example, ``-V:OtherPython/`` will select the
227 "best" tag registered for ``OtherPython``, while ``-V:3.11`` or ``-V:/3.11``
228 will select the "best" distribution with tag ``3.11``.
229
230 When using the legacy :samp:`-{<major>}`, :samp:`-{<major>}.{<minor>}`,
231 :samp:`-{<major>}-{<bitness>}` or :samp:`-{<major>}.{<minor>}-{<bitness>}` arguments,
232 all existing behaviour should be preserved from past versions,
233 and only releases from ``PythonCore`` will be selected.
234 However, the ``-64`` suffix now implies "not 32-bit" (not necessarily x86-64),
235 as there are multiple supported 64-bit platforms.
236 32-bit runtimes are detected by checking the runtime's tag for a ``-32`` suffix.
237 All releases of Python since 3.5 have included this in their 32-bit builds.
238
239
240 .. _new-feat-related-type-hints-311:
241 .. _whatsnew311-typing-features:
242
243 New Features Related to Type Hints
244 ==================================
245
246 This section covers major changes affecting :pep:`484` type hints and
247 the :mod:`typing` module.
248
249
250 .. _whatsnew311-pep646:
251
252 PEP 646: Variadic generics
253 --------------------------
254
255 :pep:`484` previously introduced :data:`~typing.TypeVar`, enabling creation
256 of generics parameterised with a single type. :pep:`646` adds
257 :data:`~typing.TypeVarTuple`, enabling parameterisation
258 with an *arbitrary* number of types. In other words,
259 a :data:`~typing.TypeVarTuple` is a *variadic* type variable,
260 enabling *variadic* generics.
261
262 This enables a wide variety of use cases.
263 In particular, it allows the type of array-like structures
264 in numerical computing libraries such as NumPy and TensorFlow to be
265 parameterised with the array *shape*. Static type checkers will now
266 be able to catch shape-related bugs in code that uses these libraries.
267
268 See :pep:`646` for more details.
269
270 (Contributed by Matthew Rahtz in :issue:`43224`, with contributions by
271 Serhiy Storchaka and Jelle Zijlstra. PEP written by Mark Mendoza, Matthew
272 Rahtz, Pradeep Kumar Srinivasan, and Vincent Siles.)
273
274
275 .. _whatsnew311-pep655:
276
277 PEP 655: Marking individual ``TypedDict`` items as required or not-required
278 ---------------------------------------------------------------------------
279
280 :data:`~typing.Required` and :data:`~typing.NotRequired` provide a
281 straightforward way to mark whether individual items in a
282 :class:`~typing.TypedDict` must be present. Previously, this was only possible
283 using inheritance.
284
285 All fields are still required by default,
286 unless the *total* parameter is set to ``False``,
287 in which case all fields are still not-required by default.
288 For example, the following specifies a :class:`!TypedDict`
289 with one required and one not-required key::
290
291 class Movie(TypedDict):
292 title: str
293 year: NotRequired[int]
294
295 m1: Movie = {"title": "Black Panther", "year": 2018} # OK
296 m2: Movie = {"title": "Star Wars"} # OK (year is not required)
297 m3: Movie = {"year": 2022} # ERROR (missing required field title)
298
299 The following definition is equivalent::
300
301 class Movie(TypedDict, total=False):
302 title: Required[str]
303 year: int
304
305 See :pep:`655` for more details.
306
307 (Contributed by David Foster and Jelle Zijlstra in :issue:`47087`. PEP
308 written by David Foster.)
309
310
311 .. _whatsnew311-pep673:
312
313 PEP 673: ``Self`` type
314 ----------------------
315
316 The new :data:`~typing.Self` annotation provides a simple and intuitive
317 way to annotate methods that return an instance of their class. This
318 behaves the same as the :class:`~typing.TypeVar`-based approach
319 :pep:`specified in PEP 484 <484#annotating-instance-and-class-methods>`,
320 but is more concise and easier to follow.
321
322 Common use cases include alternative constructors provided as
323 :func:`classmethod <classmethod>`\s,
324 and :meth:`~object.__enter__` methods that return ``self``::
325
326 class MyLock:
327 def __enter__(self) -> Self:
328 self.lock()
329 return self
330
331 ...
332
333 class MyInt:
334 @classmethod
335 def fromhex(cls, s: str) -> Self:
336 return cls(int(s, 16))
337
338 ...
339
340 :data:`~typing.Self` can also be used to annotate method parameters
341 or attributes of the same type as their enclosing class.
342
343 See :pep:`673` for more details.
344
345 (Contributed by James Hilton-Balfe in :issue:`46534`. PEP written by
346 Pradeep Kumar Srinivasan and James Hilton-Balfe.)
347
348
349 .. _whatsnew311-pep675:
350
351 PEP 675: Arbitrary literal string type
352 --------------------------------------
353
354 The new :data:`~typing.LiteralString` annotation may be used to indicate
355 that a function parameter can be of any literal string type. This allows
356 a function to accept arbitrary literal string types, as well as strings
357 created from other literal strings. Type checkers can then
358 enforce that sensitive functions, such as those that execute SQL
359 statements or shell commands, are called only with static arguments,
360 providing protection against injection attacks.
361
362 For example, a SQL query function could be annotated as follows::
363
364 def run_query(sql: LiteralString) -> ...
365 ...
366
367 def caller(
368 arbitrary_string: str,
369 query_string: LiteralString,
370 table_name: LiteralString,
371 ) -> None:
372 run_query("SELECT * FROM students") # ok
373 run_query(query_string) # ok
374 run_query("SELECT * FROM " + table_name) # ok
375 run_query(arbitrary_string) # type checker error
376 run_query( # type checker error
377 f"SELECT * FROM students WHERE name = {arbitrary_string}"
378 )
379
380 See :pep:`675` for more details.
381
382 (Contributed by Jelle Zijlstra in :issue:`47088`. PEP written by Pradeep
383 Kumar Srinivasan and Graham Bleaney.)
384
385
386 .. _whatsnew311-pep681:
387
388 PEP 681: Data class transforms
389 ------------------------------
390
391 :data:`~typing.dataclass_transform` may be used to
392 decorate a class, metaclass, or a function that is itself a decorator.
393 The presence of ``@dataclass_transform()`` tells a static type checker that the
394 decorated object performs runtime "magic" that transforms a class,
395 giving it :func:`dataclass <dataclasses.dataclass>`-like behaviors.
396
397 For example::
398
399 # The create_model decorator is defined by a library.
400 @typing.dataclass_transform()
401 def create_model(cls: Type[T]) -> Type[T]:
402 cls.__init__ = ...
403 cls.__eq__ = ...
404 cls.__ne__ = ...
405 return cls
406
407 # The create_model decorator can now be used to create new model classes:
408 @create_model
409 class CustomerModel:
410 id: int
411 name: str
412
413 c = CustomerModel(id=327, name="Eric Idle")
414
415 See :pep:`681` for more details.
416
417 (Contributed by Jelle Zijlstra in :gh:`91860`. PEP written by
418 Erik De Bonte and Eric Traut.)
419
420
421 .. _whatsnew311-pep563-deferred:
422
423 PEP 563 may not be the future
424 -----------------------------
425
426 :pep:`563` Postponed Evaluation of Annotations
427 (the ``from __future__ import annotations`` :ref:`future statement <future>`)
428 that was originally planned for release in Python 3.10
429 has been put on hold indefinitely.
430 See `this message from the Steering Council <https://mail.python.org/archives/list/[email protected]/message/VIZEBX5EYMSYIJNDBF6DMUMZOCWHARSO/>`__
431 for more information.
432
433
434 .. _whatsnew311-other-lang-changes:
435
436 Other Language Changes
437 ======================
438
439 * Starred unpacking expressions can now be used in :keyword:`for` statements.
440 (See :issue:`46725` for more details.)
441
442 * Asynchronous :ref:`comprehensions <comprehensions>` are now allowed
443 inside comprehensions in :ref:`asynchronous functions <async def>`.
444 Outer comprehensions implicitly become asynchronous in this case.
445 (Contributed by Serhiy Storchaka in :issue:`33346`.)
446
447 * A :exc:`TypeError` is now raised instead of an :exc:`AttributeError` in
448 :keyword:`with` statements and :meth:`contextlib.ExitStack.enter_context`
449 for objects that do not support the :term:`context manager` protocol,
450 and in :keyword:`async with` statements and
451 :meth:`contextlib.AsyncExitStack.enter_async_context`
452 for objects not supporting the :term:`asynchronous context manager` protocol.
453 (Contributed by Serhiy Storchaka in :issue:`12022` and :issue:`44471`.)
454
455 * Added :meth:`object.__getstate__`, which provides the default
456 implementation of the :meth:`!__getstate__` method. :mod:`copy`\ing
457 and :mod:`pickle`\ing instances of subclasses of builtin types
458 :class:`bytearray`, :class:`set`, :class:`frozenset`,
459 :class:`collections.OrderedDict`, :class:`collections.deque`,
460 :class:`weakref.WeakSet`, and :class:`datetime.tzinfo` now copies and
461 pickles instance attributes implemented as :term:`slots <__slots__>`.
462 This change has an unintended side effect: It trips up a small minority
463 of existing Python projects not expecting :meth:`object.__getstate__` to
464 exist. See the later comments on :gh:`70766` for discussions of what
465 workarounds such code may need.
466 (Contributed by Serhiy Storchaka in :issue:`26579`.)
467
468 .. _whatsnew311-pythonsafepath:
469
470 * Added a :option:`-P` command line option
471 and a :envvar:`PYTHONSAFEPATH` environment variable,
472 which disable the automatic prepending to :data:`sys.path`
473 of the script's directory when running a script,
474 or the current directory when using :option:`-c` and :option:`-m`.
475 This ensures only stdlib and installed modules
476 are picked up by :keyword:`import`,
477 and avoids unintentionally or maliciously shadowing modules
478 with those in a local (and typically user-writable) directory.
479 (Contributed by Victor Stinner in :gh:`57684`.)
480
481 * A ``"z"`` option was added to the :ref:`formatspec` that
482 coerces negative to positive zero after rounding to the format precision.
483 See :pep:`682` for more details.
484 (Contributed by John Belmonte in :gh:`90153`.)
485
486 * Bytes are no longer accepted on :data:`sys.path`. Support broke sometime
487 between Python 3.2 and 3.6, with no one noticing until after Python 3.10.0
488 was released. In addition, bringing back support would be problematic due to
489 interactions between :option:`-b` and :data:`sys.path_importer_cache` when
490 there is a mixture of :class:`str` and :class:`bytes` keys.
491 (Contributed by Thomas Grainger in :gh:`91181`.)
492
493
494 .. _whatsnew311-other-implementation-changes:
495
496 Other CPython Implementation Changes
497 ====================================
498
499 * The special methods :meth:`~object.__complex__` for :class:`complex`
500 and :meth:`~object.__bytes__` for :class:`bytes` are implemented to support
501 the :class:`typing.SupportsComplex` and :class:`typing.SupportsBytes` protocols.
502 (Contributed by Mark Dickinson and Donghee Na in :issue:`24234`.)
503
504 * ``siphash13`` is added as a new internal hashing algorithm.
505 It has similar security properties as ``siphash24``,
506 but it is slightly faster for long inputs.
507 :class:`str`, :class:`bytes`, and some other types
508 now use it as the default algorithm for :func:`hash`.
509 :pep:`552` :ref:`hash-based .pyc files <pyc-invalidation>`
510 now use ``siphash13`` too.
511 (Contributed by Inada Naoki in :issue:`29410`.)
512
513 * When an active exception is re-raised by a :keyword:`raise` statement with no parameters,
514 the traceback attached to this exception is now always ``sys.exc_info()[1].__traceback__``.
515 This means that changes made to the traceback in the current :keyword:`except` clause are
516 reflected in the re-raised exception.
517 (Contributed by Irit Katriel in :issue:`45711`.)
518
519 * The interpreter state's representation of handled exceptions
520 (aka ``exc_info`` or ``_PyErr_StackItem``)
521 now only has the ``exc_value`` field; ``exc_type`` and ``exc_traceback``
522 have been removed, as they can be derived from ``exc_value``.
523 (Contributed by Irit Katriel in :issue:`45711`.)
524
525 * A new :ref:`command line option <install-quiet-option>`, ``AppendPath``,
526 has been added for the Windows installer.
527 It behaves similarly to ``PrependPath``,
528 but appends the install and scripts directories instead of prepending them.
529 (Contributed by Bastian Neuburger in :issue:`44934`.)
530
531 * The :c:member:`PyConfig.module_search_paths_set` field must now be set to ``1`` for
532 initialization to use :c:member:`PyConfig.module_search_paths` to initialize
533 :data:`sys.path`. Otherwise, initialization will recalculate the path and replace
534 any values added to ``module_search_paths``.
535
536 * The output of the :option:`--help` option now fits in 50 lines/80 columns.
537 Information about :ref:`Python environment variables <using-on-envvars>`
538 and :option:`-X` options is now available using the respective
539 :option:`--help-env` and :option:`--help-xoptions` flags,
540 and with the new :option:`--help-all`.
541 (Contributed by Éric Araujo in :issue:`46142`.)
542
543 * Converting between :class:`int` and :class:`str` in bases other than 2
544 (binary), 4, 8 (octal), 16 (hexadecimal), or 32 such as base 10 (decimal)
545 now raises a :exc:`ValueError` if the number of digits in string form is
546 above a limit to avoid potential denial of service attacks due to the
547 algorithmic complexity. This is a mitigation for :cve:`2020-10735`.
548 This limit can be configured or disabled by environment variable, command
549 line flag, or :mod:`sys` APIs. See the :ref:`integer string conversion
550 length limitation <int_max_str_digits>` documentation. The default limit
551 is 4300 digits in string form.
552
553
554 .. _whatsnew311-new-modules:
555
556 New Modules
557 ===========
558
559 * :mod:`tomllib`: For parsing `TOML <https://toml.io/>`_.
560 See :pep:`680` for more details.
561 (Contributed by Taneli Hukkinen in :issue:`40059`.)
562
563 * :mod:`wsgiref.types`:
564 :pep:`WSGI <3333>`-specific types for static type checking.
565 (Contributed by Sebastian Rittau in :issue:`42012`.)
566
567
568 .. _whatsnew311-improved-modules:
569
570 Improved Modules
571 ================
572
573 .. _whatsnew311-asyncio:
574
575 asyncio
576 -------
577
578 * Added the :class:`~asyncio.TaskGroup` class,
579 an :ref:`asynchronous context manager <async-context-managers>`
580 holding a group of tasks that will wait for all of them upon exit.
581 For new code this is recommended over using
582 :func:`~asyncio.create_task` and :func:`~asyncio.gather` directly.
583 (Contributed by Yury Selivanov and others in :gh:`90908`.)
584
585 * Added :func:`~asyncio.timeout`, an asynchronous context manager for
586 setting a timeout on asynchronous operations. For new code this is
587 recommended over using :func:`~asyncio.wait_for` directly.
588 (Contributed by Andrew Svetlov in :gh:`90927`.)
589
590 * Added the :class:`~asyncio.Runner` class, which exposes the machinery
591 used by :func:`~asyncio.run`.
592 (Contributed by Andrew Svetlov in :gh:`91218`.)
593
594 * Added the :class:`~asyncio.Barrier` class to the synchronization
595 primitives in the asyncio library, and the related
596 :exc:`~asyncio.BrokenBarrierError` exception.
597 (Contributed by Yves Duprat and Andrew Svetlov in :gh:`87518`.)
598
599 * Added keyword argument *all_errors* to :meth:`asyncio.loop.create_connection`
600 so that multiple connection errors can be raised as an :exc:`ExceptionGroup`.
601
602 * Added the :meth:`asyncio.StreamWriter.start_tls` method for
603 upgrading existing stream-based connections to TLS.
604 (Contributed by Ian Good in :issue:`34975`.)
605
606 * Added raw datagram socket functions to the event loop:
607 :meth:`~asyncio.loop.sock_sendto`,
608 :meth:`~asyncio.loop.sock_recvfrom` and
609 :meth:`~asyncio.loop.sock_recvfrom_into`.
610 These have implementations in :class:`~asyncio.SelectorEventLoop` and
611 :class:`~asyncio.ProactorEventLoop`.
612 (Contributed by Alex Grönholm in :issue:`46805`.)
613
614 * Added :meth:`~asyncio.Task.cancelling` and
615 :meth:`~asyncio.Task.uncancel` methods to :class:`~asyncio.Task`.
616 These are primarily intended for internal use,
617 notably by :class:`~asyncio.TaskGroup`.
618
619
620 .. _whatsnew311-contextlib:
621
622 contextlib
623 ----------
624
625 * Added non parallel-safe :func:`~contextlib.chdir` context manager to change
626 the current working directory and then restore it on exit. Simple wrapper
627 around :func:`~os.chdir`. (Contributed by Filipe Laíns in :issue:`25625`)
628
629
630 .. _whatsnew311-dataclasses:
631
632 dataclasses
633 -----------
634
635 * Change field default mutability check, allowing only defaults which are
636 :term:`hashable` instead of any object which is not an instance of
637 :class:`dict`, :class:`list` or :class:`set`. (Contributed by Eric V. Smith in
638 :issue:`44674`.)
639
640
641 .. _whatsnew311-datetime:
642
643 datetime
644 --------
645
646 * Add :const:`datetime.UTC`, a convenience alias for
647 :attr:`datetime.timezone.utc`. (Contributed by Kabir Kwatra in :gh:`91973`.)
648
649 * :meth:`datetime.date.fromisoformat`, :meth:`datetime.time.fromisoformat` and
650 :meth:`datetime.datetime.fromisoformat` can now be used to parse most ISO 8601
651 formats (barring only those that support fractional hours and minutes).
652 (Contributed by Paul Ganssle in :gh:`80010`.)
653
654
655 .. _whatsnew311-enum:
656
657 enum
658 ----
659
660 * Renamed :class:`!EnumMeta` to :class:`~enum.EnumType`
661 (:class:`!EnumMeta` kept as an alias).
662
663 * Added :class:`~enum.StrEnum`,
664 with members that can be used as (and must be) strings.
665
666 * Added :class:`~enum.ReprEnum`,
667 which only modifies the :meth:`~object.__repr__` of members
668 while returning their literal values (rather than names)
669 for :meth:`~object.__str__` and :meth:`~object.__format__`
670 (used by :func:`str`, :func:`format` and :term:`f-string`\s).
671
672 * Changed :meth:`Enum.__format__() <enum.Enum.__format__>` (the default for
673 :func:`format`, :meth:`str.format` and :term:`f-string`\s) to always produce
674 the same result as :meth:`Enum.__str__() <enum.Enum.__str__>`: for enums inheriting from
675 :class:`~enum.ReprEnum` it will be the member's value; for all other enums
676 it will be the enum and member name (e.g. ``Color.RED``).
677
678 * Added a new *boundary* class parameter to :class:`~enum.Flag` enums
679 and the :class:`~enum.FlagBoundary` enum with its options,
680 to control how to handle out-of-range flag values.
681
682 * Added the :func:`~enum.verify` enum decorator
683 and the :class:`~enum.EnumCheck` enum with its options,
684 to check enum classes against several specific constraints.
685
686 * Added the :func:`~enum.member` and :func:`~enum.nonmember` decorators,
687 to ensure the decorated object is/is not converted to an enum member.
688
689 * Added the :func:`~enum.property` decorator,
690 which works like :func:`property` except for enums.
691 Use this instead of :func:`types.DynamicClassAttribute`.
692
693 * Added the :func:`~enum.global_enum` enum decorator,
694 which adjusts :meth:`~object.__repr__` and :meth:`~object.__str__`
695 to show values as members of their module rather than the enum class.
696 For example, ``'re.ASCII'`` for the :const:`~re.ASCII` member
697 of :class:`re.RegexFlag` rather than ``'RegexFlag.ASCII'``.
698
699 * Enhanced :class:`~enum.Flag` to support
700 :func:`len`, iteration and :keyword:`in`/:keyword:`not in` on its members.
701 For example, the following now works:
702 ``len(AFlag(3)) == 2 and list(AFlag(3)) == (AFlag.ONE, AFlag.TWO)``
703
704 * Changed :class:`~enum.Enum` and :class:`~enum.Flag`
705 so that members are now defined
706 before :meth:`~object.__init_subclass__` is called;
707 :func:`dir` now includes methods, etc., from mixed-in data types.
708
709 * Changed :class:`~enum.Flag`
710 to only consider primary values (power of two) canonical
711 while composite values (``3``, ``6``, ``10``, etc.) are considered aliases;
712 inverted flags are coerced to their positive equivalent.
713
714
715 .. _whatsnew311-fcntl:
716
717 fcntl
718 -----
719
720 * On FreeBSD, the :data:`!F_DUP2FD` and :data:`!F_DUP2FD_CLOEXEC` flags respectively
721 are supported, the former equals to ``dup2`` usage while the latter set
722 the ``FD_CLOEXEC`` flag in addition.
723
724
725 .. _whatsnew311-fractions:
726
727 fractions
728 ---------
729
730 * Support :PEP:`515`-style initialization of :class:`~fractions.Fraction` from
731 string. (Contributed by Sergey B Kirpichev in :issue:`44258`.)
732
733 * :class:`~fractions.Fraction` now implements an ``__int__`` method, so
734 that an ``isinstance(some_fraction, typing.SupportsInt)`` check passes.
735 (Contributed by Mark Dickinson in :issue:`44547`.)
736
737
738 .. _whatsnew311-functools:
739
740 functools
741 ---------
742
743 * :func:`functools.singledispatch` now supports :class:`types.UnionType`
744 and :class:`typing.Union` as annotations to the dispatch argument.::
745
746 >>> from functools import singledispatch
747 >>> @singledispatch
748 ... def fun(arg, verbose=False):
749 ... if verbose:
750 ... print("Let me just say,", end=" ")
751 ... print(arg)
752 ...
753 >>> @fun.register
754 ... def _(arg: int | float, verbose=False):
755 ... if verbose:
756 ... print("Strength in numbers, eh?", end=" ")
757 ... print(arg)
758 ...
759 >>> from typing import Union
760 >>> @fun.register
761 ... def _(arg: Union[list, set], verbose=False):
762 ... if verbose:
763 ... print("Enumerate this:")
764 ... for i, elem in enumerate(arg):
765 ... print(i, elem)
766 ...
767
768 (Contributed by Yurii Karabas in :issue:`46014`.)
769
770
771 .. _whatsnew311-gzip:
772
773 gzip
774 ----
775
776 * The :func:`gzip.compress` function is now faster when used with the
777 **mtime=0** argument as it delegates the compression entirely to a single
778 :func:`zlib.compress` operation. There is one side effect of this change: The
779 gzip file header contains an "OS" byte in its header. That was traditionally
780 always set to a value of 255 representing "unknown" by the :mod:`gzip`
781 module. Now, when using :func:`~gzip.compress` with **mtime=0**, it may be
782 set to a different value by the underlying zlib C library Python was linked
783 against.
784 (See :gh:`112346` for details on the side effect.)
785
786 .. _whatsnew311-hashlib:
787
788 hashlib
789 -------
790
791 * :func:`hashlib.blake2b` and :func:`hashlib.blake2s` now prefer `libb2`_
792 over Python's vendored copy.
793 (Contributed by Christian Heimes in :issue:`47095`.)
794
795 * The internal ``_sha3`` module with SHA3 and SHAKE algorithms now uses
796 *tiny_sha3* instead of the *Keccak Code Package* to reduce code and binary
797 size. The :mod:`hashlib` module prefers optimized SHA3 and SHAKE
798 implementations from OpenSSL. The change affects only installations without
799 OpenSSL support.
800 (Contributed by Christian Heimes in :issue:`47098`.)
801
802 * Add :func:`hashlib.file_digest`, a helper function for efficient hashing
803 of files or file-like objects.
804 (Contributed by Christian Heimes in :gh:`89313`.)
805
806
807 .. _whatsnew311-idle:
808
809 IDLE and idlelib
810 ----------------
811
812 * Apply syntax highlighting to ``.pyi`` files. (Contributed by Alex
813 Waygood and Terry Jan Reedy in :issue:`45447`.)
814
815 * Include prompts when saving Shell with inputs and outputs.
816 (Contributed by Terry Jan Reedy in :gh:`95191`.)
817
818
819 .. _whatsnew311-inspect:
820
821 inspect
822 -------
823
824 * Add :func:`~inspect.getmembers_static` to return all members without
825 triggering dynamic lookup via the descriptor protocol. (Contributed by
826 Weipeng Hong in :issue:`30533`.)
827
828 * Add :func:`~inspect.ismethodwrapper`
829 for checking if the type of an object is a :class:`~types.MethodWrapperType`.
830 (Contributed by Hakan Çelik in :issue:`29418`.)
831
832 * Change the frame-related functions in the :mod:`inspect` module to return new
833 :class:`~inspect.FrameInfo` and :class:`~inspect.Traceback` class instances
834 (backwards compatible with the previous :term:`named tuple`-like interfaces)
835 that includes the extended :pep:`657` position information (end
836 line number, column and end column). The affected functions are:
837
838 * :func:`inspect.getframeinfo`
839 * :func:`inspect.getouterframes`
840 * :func:`inspect.getinnerframes`,
841 * :func:`inspect.stack`
842 * :func:`inspect.trace`
843
844 (Contributed by Pablo Galindo in :gh:`88116`.)
845
846
847 .. _whatsnew311-locale:
848
849 locale
850 ------
851
852 * Add :func:`locale.getencoding` to get the current locale encoding. It is similar to
853 ``locale.getpreferredencoding(False)`` but ignores the
854 :ref:`Python UTF-8 Mode <utf8-mode>`.
855
856
857 .. _whatsnew311-logging:
858
859 logging
860 -------
861
862 * Added :func:`~logging.getLevelNamesMapping`
863 to return a mapping from logging level names (e.g. ``'CRITICAL'``)
864 to the values of their corresponding :ref:`levels` (e.g. ``50``, by default).
865 (Contributed by Andrei Kulakovin in :gh:`88024`.)
866
867 * Added a :meth:`~logging.handlers.SysLogHandler.createSocket` method
868 to :class:`~logging.handlers.SysLogHandler`, to match
869 :meth:`SocketHandler.createSocket()
870 <logging.handlers.SocketHandler.createSocket>`.
871 It is called automatically during handler initialization
872 and when emitting an event, if there is no active socket.
873 (Contributed by Kirill Pinchuk in :gh:`88457`.)
874
875
876 .. _whatsnew311-math:
877
878 math
879 ----
880
881 * Add :func:`math.exp2`: return 2 raised to the power of x.
882 (Contributed by Gideon Mitchell in :issue:`45917`.)
883
884 * Add :func:`math.cbrt`: return the cube root of x.
885 (Contributed by Ajith Ramachandran in :issue:`44357`.)
886
887 * The behaviour of two :func:`math.pow` corner cases was changed, for
888 consistency with the IEEE 754 specification. The operations
889 ``math.pow(0.0, -math.inf)`` and ``math.pow(-0.0, -math.inf)`` now return
890 ``inf``. Previously they raised :exc:`ValueError`. (Contributed by Mark
891 Dickinson in :issue:`44339`.)
892
893 * The :data:`math.nan` value is now always available.
894 (Contributed by Victor Stinner in :issue:`46917`.)
895
896
897 .. _whatsnew311-operator:
898
899 operator
900 --------
901
902 * A new function ``operator.call`` has been added, such that
903 ``operator.call(obj, *args, **kwargs) == obj(*args, **kwargs)``.
904 (Contributed by Antony Lee in :issue:`44019`.)
905
906
907 .. _whatsnew311-os:
908
909 os
910 --
911
912 * On Windows, :func:`os.urandom` now uses ``BCryptGenRandom()``,
913 instead of ``CryptGenRandom()`` which is deprecated.
914 (Contributed by Donghee Na in :issue:`44611`.)
915
916
917 .. _whatsnew311-pathlib:
918
919 pathlib
920 -------
921
922 * :meth:`~pathlib.Path.glob` and :meth:`~pathlib.Path.rglob` return only
923 directories if *pattern* ends with a pathname components separator:
924 :data:`~os.sep` or :data:`~os.altsep`.
925 (Contributed by Eisuke Kawasima in :issue:`22276` and :issue:`33392`.)
926
927
928 .. _whatsnew311-re:
929
930 re
931 --
932
933 * Atomic grouping (``(?>...)``) and possessive quantifiers (``*+``, ``++``,
934 ``?+``, ``{m,n}+``) are now supported in regular expressions.
935 (Contributed by Jeffrey C. Jacobs and Serhiy Storchaka in :issue:`433030`.)
936
937
938 .. _whatsnew311-shutil:
939
940 shutil
941 ------
942
943 * Add optional parameter *dir_fd* in :func:`shutil.rmtree`.
944 (Contributed by Serhiy Storchaka in :issue:`46245`.)
945
946
947 .. _whatsnew311-socket:
948
949 socket
950 ------
951
952 * Add CAN Socket support for NetBSD.
953 (Contributed by Thomas Klausner in :issue:`30512`.)
954
955 * :meth:`~socket.create_connection` has an option to raise, in case of
956 failure to connect, an :exc:`ExceptionGroup` containing all errors
957 instead of only raising the last error.
958 (Contributed by Irit Katriel in :issue:`29980`.)
959
960
961 .. _whatsnew311-sqlite3:
962
963 sqlite3
964 -------
965
966 * You can now disable the authorizer by passing :const:`None` to
967 :meth:`~sqlite3.Connection.set_authorizer`.
968 (Contributed by Erlend E. Aasland in :issue:`44491`.)
969
970 * Collation name :meth:`~sqlite3.Connection.create_collation` can now
971 contain any Unicode character. Collation names with invalid characters
972 now raise :exc:`UnicodeEncodeError` instead of :exc:`sqlite3.ProgrammingError`.
973 (Contributed by Erlend E. Aasland in :issue:`44688`.)
974
975 * :mod:`sqlite3` exceptions now include the SQLite extended error code as
976 :attr:`~sqlite3.Error.sqlite_errorcode` and the SQLite error name as
977 :attr:`~sqlite3.Error.sqlite_errorname`.
978 (Contributed by Aviv Palivoda, Daniel Shahaf, and Erlend E. Aasland in
979 :issue:`16379` and :issue:`24139`.)
980
981 * Add :meth:`~sqlite3.Connection.setlimit` and
982 :meth:`~sqlite3.Connection.getlimit` to :class:`sqlite3.Connection` for
983 setting and getting SQLite limits by connection basis.
984 (Contributed by Erlend E. Aasland in :issue:`45243`.)
985
986 * :mod:`sqlite3` now sets :attr:`sqlite3.threadsafety` based on the default
987 threading mode the underlying SQLite library has been compiled with.
988 (Contributed by Erlend E. Aasland in :issue:`45613`.)
989
990 * :mod:`sqlite3` C callbacks now use unraisable exceptions if callback
991 tracebacks are enabled. Users can now register an
992 :func:`unraisable hook handler <sys.unraisablehook>` to improve their debug
993 experience.
994 (Contributed by Erlend E. Aasland in :issue:`45828`.)
995
996 * Fetch across rollback no longer raises :exc:`~sqlite3.InterfaceError`.
997 Instead we leave it to the SQLite library to handle these cases.
998 (Contributed by Erlend E. Aasland in :issue:`44092`.)
999
1000 * Add :meth:`~sqlite3.Connection.serialize` and
1001 :meth:`~sqlite3.Connection.deserialize` to :class:`sqlite3.Connection` for
1002 serializing and deserializing databases.
1003 (Contributed by Erlend E. Aasland in :issue:`41930`.)
1004
1005 * Add :meth:`~sqlite3.Connection.create_window_function` to
1006 :class:`sqlite3.Connection` for creating aggregate window functions.
1007 (Contributed by Erlend E. Aasland in :issue:`34916`.)
1008
1009 * Add :meth:`~sqlite3.Connection.blobopen` to :class:`sqlite3.Connection`.
1010 :class:`sqlite3.Blob` allows incremental I/O operations on blobs.
1011 (Contributed by Aviv Palivoda and Erlend E. Aasland in :issue:`24905`.)
1012
1013
1014 .. _whatsnew311-string:
1015
1016 string
1017 ------
1018
1019 * Add :meth:`~string.Template.get_identifiers`
1020 and :meth:`~string.Template.is_valid` to :class:`string.Template`,
1021 which respectively return all valid placeholders,
1022 and whether any invalid placeholders are present.
1023 (Contributed by Ben Kehoe in :gh:`90465`.)
1024
1025
1026 .. _whatsnew311-sys:
1027
1028 sys
1029 ---
1030
1031 * :func:`sys.exc_info` now derives the ``type`` and ``traceback`` fields
1032 from the ``value`` (the exception instance), so when an exception is
1033 modified while it is being handled, the changes are reflected in
1034 the results of subsequent calls to :func:`!exc_info`.
1035 (Contributed by Irit Katriel in :issue:`45711`.)
1036
1037 * Add :func:`sys.exception` which returns the active exception instance
1038 (equivalent to ``sys.exc_info()[1]``).
1039 (Contributed by Irit Katriel in :issue:`46328`.)
1040
1041 * Add the :data:`sys.flags.safe_path <sys.flags>` flag.
1042 (Contributed by Victor Stinner in :gh:`57684`.)
1043
1044
1045 .. _whatsnew311-sysconfig:
1046
1047 sysconfig
1048 ---------
1049
1050 * Three new :ref:`installation schemes <installation_paths>`
1051 (*posix_venv*, *nt_venv* and *venv*) were added and are used when Python
1052 creates new virtual environments or when it is running from a virtual
1053 environment.
1054 The first two schemes (*posix_venv* and *nt_venv*) are OS-specific
1055 for non-Windows and Windows, the *venv* is essentially an alias to one of
1056 them according to the OS Python runs on.
1057 This is useful for downstream distributors who modify
1058 :func:`sysconfig.get_preferred_scheme`.
1059 Third party code that creates new virtual environments should use the new
1060 *venv* installation scheme to determine the paths, as does :mod:`venv`.
1061 (Contributed by Miro Hrončok in :issue:`45413`.)
1062
1063
1064 .. _whatsnew311-tempfile:
1065
1066 tempfile
1067 --------
1068
1069 * :class:`~tempfile.SpooledTemporaryFile` objects now fully implement the methods
1070 of :class:`io.BufferedIOBase` or :class:`io.TextIOBase`
1071 (depending on file mode).
1072 This lets them work correctly with APIs that expect file-like objects,
1073 such as compression modules.
1074 (Contributed by Carey Metcalfe in :gh:`70363`.)
1075
1076
1077 .. _whatsnew311-threading:
1078
1079 threading
1080 ---------
1081
1082 * On Unix, if the ``sem_clockwait()`` function is available in the C library
1083 (glibc 2.30 and newer), the :meth:`threading.Lock.acquire` method now uses
1084 the monotonic clock (:const:`time.CLOCK_MONOTONIC`) for the timeout, rather
1085 than using the system clock (:const:`time.CLOCK_REALTIME`), to not be affected
1086 by system clock changes.
1087 (Contributed by Victor Stinner in :issue:`41710`.)
1088
1089
1090 .. _whatsnew311-time:
1091
1092 time
1093 ----
1094
1095 * On Unix, :func:`time.sleep` now uses the ``clock_nanosleep()`` or
1096 ``nanosleep()`` function, if available, which has a resolution of 1 nanosecond
1097 (10\ :sup:`-9` seconds), rather than using ``select()`` which has a resolution
1098 of 1 microsecond (10\ :sup:`-6` seconds).
1099 (Contributed by Benjamin Szőke and Victor Stinner in :issue:`21302`.)
1100
1101 * On Windows 8.1 and newer, :func:`time.sleep` now uses a waitable timer based
1102 on `high-resolution timers
1103 <https://docs.microsoft.com/en-us/windows-hardware/drivers/kernel/high-resolution-timers>`_
1104 which has a resolution of 100 nanoseconds (10\ :sup:`-7` seconds). Previously,
1105 it had a resolution of 1 millisecond (10\ :sup:`-3` seconds).
1106 (Contributed by Benjamin Szőke, Donghee Na, Eryk Sun and Victor Stinner in :issue:`21302` and :issue:`45429`.)
1107
1108
1109 .. _whatsnew311-tkinter:
1110
1111 tkinter
1112 -------
1113
1114 * Added method ``info_patchlevel()`` which returns the exact version of
1115 the Tcl library as a named tuple similar to :data:`sys.version_info`.
1116 (Contributed by Serhiy Storchaka in :gh:`91827`.)
1117
1118
1119 .. _whatsnew311-traceback:
1120
1121 traceback
1122 ---------
1123
1124 * Add :func:`traceback.StackSummary.format_frame_summary` to allow users
1125 to override which frames appear in the traceback, and how they are
1126 formatted.
1127 (Contributed by Ammar Askar in :issue:`44569`.)
1128
1129 * Add :func:`traceback.TracebackException.print`, which prints the
1130 formatted :exc:`~traceback.TracebackException` instance to a file.
1131 (Contributed by Irit Katriel in :issue:`33809`.)
1132
1133
1134 .. _whatsnew311-typing:
1135
1136 typing
1137 ------
1138
1139 For major changes, see :ref:`new-feat-related-type-hints-311`.
1140
1141 * Add :func:`typing.assert_never` and :class:`typing.Never`.
1142 :func:`typing.assert_never` is useful for asking a type checker to confirm
1143 that a line of code is not reachable. At runtime, it raises an
1144 :exc:`AssertionError`.
1145 (Contributed by Jelle Zijlstra in :gh:`90633`.)
1146
1147 * Add :func:`typing.reveal_type`. This is useful for asking a type checker
1148 what type it has inferred for a given expression. At runtime it prints
1149 the type of the received value.
1150 (Contributed by Jelle Zijlstra in :gh:`90572`.)
1151
1152 * Add :func:`typing.assert_type`. This is useful for asking a type checker
1153 to confirm that the type it has inferred for a given expression matches
1154 the given type. At runtime it simply returns the received value.
1155 (Contributed by Jelle Zijlstra in :gh:`90638`.)
1156
1157 * :data:`typing.TypedDict` types can now be generic. (Contributed by
1158 Samodya Abeysiriwardane in :gh:`89026`.)
1159
1160 * :class:`~typing.NamedTuple` types can now be generic.
1161 (Contributed by Serhiy Storchaka in :issue:`43923`.)
1162
1163 * Allow subclassing of :class:`typing.Any`. This is useful for avoiding
1164 type checker errors related to highly dynamic class, such as mocks.
1165 (Contributed by Shantanu Jain in :gh:`91154`.)
1166
1167 * The :func:`typing.final` decorator now sets the ``__final__`` attributed on
1168 the decorated object.
1169 (Contributed by Jelle Zijlstra in :gh:`90500`.)
1170
1171 * The :func:`typing.get_overloads` function can be used for introspecting
1172 the overloads of a function. :func:`typing.clear_overloads` can be used
1173 to clear all registered overloads of a function.
1174 (Contributed by Jelle Zijlstra in :gh:`89263`.)
1175
1176 * The :meth:`~object.__init__` method of :class:`~typing.Protocol` subclasses
1177 is now preserved. (Contributed by Adrian Garcia Badarasco in :gh:`88970`.)
1178
1179 * The representation of empty tuple types (``Tuple[()]``) is simplified.
1180 This affects introspection, e.g. ``get_args(Tuple[()])`` now evaluates
1181 to ``()`` instead of ``((),)``.
1182 (Contributed by Serhiy Storchaka in :gh:`91137`.)
1183
1184 * Loosen runtime requirements for type annotations by removing the callable
1185 check in the private ``typing._type_check`` function. (Contributed by
1186 Gregory Beauregard in :gh:`90802`.)
1187
1188 * :func:`typing.get_type_hints` now supports evaluating strings as forward
1189 references in :ref:`PEP 585 generic aliases <types-genericalias>`.
1190 (Contributed by Niklas Rosenstein in :gh:`85542`.)
1191
1192 * :func:`typing.get_type_hints` no longer adds :data:`~typing.Optional`
1193 to parameters with ``None`` as a default. (Contributed by Nikita Sobolev
1194 in :gh:`90353`.)
1195
1196 * :func:`typing.get_type_hints` now supports evaluating bare stringified
1197 :data:`~typing.ClassVar` annotations. (Contributed by Gregory Beauregard
1198 in :gh:`90711`.)
1199
1200 * :func:`typing.no_type_check` no longer modifies external classes and functions.
1201 It also now correctly marks classmethods as not to be type checked. (Contributed
1202 by Nikita Sobolev in :gh:`90729`.)
1203
1204
1205 .. _whatsnew311-unicodedata:
1206
1207 unicodedata
1208 -----------
1209
1210 * The Unicode database has been updated to version 14.0.0.
1211 (Contributed by Benjamin Peterson in :issue:`45190`).
1212
1213
1214 .. _whatsnew311-unittest:
1215
1216 unittest
1217 --------
1218
1219 * Added methods :meth:`~unittest.TestCase.enterContext` and
1220 :meth:`~unittest.TestCase.enterClassContext` of class
1221 :class:`~unittest.TestCase`, method
1222 :meth:`~unittest.IsolatedAsyncioTestCase.enterAsyncContext` of
1223 class :class:`~unittest.IsolatedAsyncioTestCase` and function
1224 :func:`unittest.enterModuleContext`.
1225 (Contributed by Serhiy Storchaka in :issue:`45046`.)
1226
1227
1228 .. _whatsnew311-venv:
1229
1230 venv
1231 ----
1232
1233 * When new Python virtual environments are created, the *venv*
1234 :ref:`sysconfig installation scheme <installation_paths>` is used
1235 to determine the paths inside the environment.
1236 When Python runs in a virtual environment, the same installation scheme
1237 is the default.
1238 That means that downstream distributors can change the default sysconfig install
1239 scheme without changing behavior of virtual environments.
1240 Third party code that also creates new virtual environments should do the same.
1241 (Contributed by Miro Hrončok in :issue:`45413`.)
1242
1243
1244 .. _whatsnew311-warnings:
1245
1246 warnings
1247 --------
1248
1249 * :func:`warnings.catch_warnings` now accepts arguments for :func:`warnings.simplefilter`,
1250 providing a more concise way to locally ignore warnings or convert them to errors.
1251 (Contributed by Zac Hatfield-Dodds in :issue:`47074`.)
1252
1253
1254 .. _whatsnew311-zipfile:
1255
1256 zipfile
1257 -------
1258
1259 * Added support for specifying member name encoding for reading metadata
1260 in a :class:`~zipfile.ZipFile`'s directory and file headers.
1261 (Contributed by Stephen J. Turnbull and Serhiy Storchaka in :issue:`28080`.)
1262
1263 * Added :meth:`ZipFile.mkdir() <zipfile.ZipFile.mkdir>`
1264 for creating new directories inside ZIP archives.
1265 (Contributed by Sam Ezeh in :gh:`49083`.)
1266
1267 * Added :attr:`~zipfile.Path.stem`, :attr:`~zipfile.Path.suffix`
1268 and :attr:`~zipfile.Path.suffixes` to :class:`zipfile.Path`.
1269 (Contributed by Miguel Brito in :gh:`88261`.)
1270
1271
1272 .. _whatsnew311-optimizations:
1273
1274 Optimizations
1275 =============
1276
1277 This section covers specific optimizations independent of the
1278 :ref:`whatsnew311-faster-cpython` project, which is covered in its own section.
1279
1280 * The compiler now optimizes simple
1281 :ref:`printf-style % formatting <old-string-formatting>` on string literals
1282 containing only the format codes ``%s``, ``%r`` and ``%a`` and makes it as
1283 fast as a corresponding :term:`f-string` expression.
1284 (Contributed by Serhiy Storchaka in :issue:`28307`.)
1285
1286 * Integer division (``//``) is better tuned for optimization by compilers.
1287 It is now around 20% faster on x86-64 when dividing an :class:`int`
1288 by a value smaller than ``2**30``.
1289 (Contributed by Gregory P. Smith and Tim Peters in :gh:`90564`.)
1290
1291 * :func:`sum` is now nearly 30% faster for integers smaller than ``2**30``.
1292 (Contributed by Stefan Behnel in :gh:`68264`.)
1293
1294 * Resizing lists is streamlined for the common case,
1295 speeding up :meth:`list.append` by ≈15%
1296 and simple :term:`list comprehension`\s by up to 20-30%
1297 (Contributed by Dennis Sweeney in :gh:`91165`.)
1298
1299 * Dictionaries don't store hash values when all keys are Unicode objects,
1300 decreasing :class:`dict` size.
1301 For example, ``sys.getsizeof(dict.fromkeys("abcdefg"))``
1302 is reduced from 352 bytes to 272 bytes (23% smaller) on 64-bit platforms.
1303 (Contributed by Inada Naoki in :issue:`46845`.)
1304
1305 * Using :class:`asyncio.DatagramProtocol` is now orders of magnitude faster
1306 when transferring large files over UDP,
1307 with speeds over 100 times higher for a ≈60 MiB file.
1308 (Contributed by msoxzw in :gh:`91487`.)
1309
1310 * :mod:`math` functions :func:`~math.comb` and :func:`~math.perm` are now
1311 ≈10 times faster for large arguments (with a larger speedup for larger *k*).
1312 (Contributed by Serhiy Storchaka in :issue:`37295`.)
1313
1314 * The :mod:`statistics` functions :func:`~statistics.mean`,
1315 :func:`~statistics.variance` and :func:`~statistics.stdev` now consume
1316 iterators in one pass rather than converting them to a :class:`list` first.
1317 This is twice as fast and can save substantial memory.
1318 (Contributed by Raymond Hettinger in :gh:`90415`.)
1319
1320 * :func:`unicodedata.normalize`
1321 now normalizes pure-ASCII strings in constant time.
1322 (Contributed by Donghee Na in :issue:`44987`.)
1323
1324
1325 .. _whatsnew311-faster-cpython:
1326
1327 Faster CPython
1328 ==============
1329
1330 CPython 3.11 is an average of
1331 `25% faster <https://github.com/faster-cpython/ideas#published-results>`_
1332 than CPython 3.10 as measured with the
1333 `pyperformance <https://github.com/python/pyperformance>`_ benchmark suite,
1334 when compiled with GCC on Ubuntu Linux.
1335 Depending on your workload, the overall speedup could be 10-60%.
1336
1337 This project focuses on two major areas in Python:
1338 :ref:`whatsnew311-faster-startup` and :ref:`whatsnew311-faster-runtime`.
1339 Optimizations not covered by this project are listed separately under
1340 :ref:`whatsnew311-optimizations`.
1341
1342
1343 .. _whatsnew311-faster-startup:
1344
1345 Faster Startup
1346 --------------
1347
1348 .. _whatsnew311-faster-imports:
1349
1350 Frozen imports / Static code objects
1351 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1352
1353 Python caches :term:`bytecode` in the :ref:`__pycache__ <tut-pycache>`
1354 directory to speed up module loading.
1355
1356 Previously in 3.10, Python module execution looked like this:
1357
1358 .. code-block:: text
1359
1360 Read __pycache__ -> Unmarshal -> Heap allocated code object -> Evaluate
1361
1362 In Python 3.11, the core modules essential for Python startup are "frozen".
1363 This means that their :ref:`codeobjects` (and bytecode)
1364 are statically allocated by the interpreter.
1365 This reduces the steps in module execution process to:
1366
1367 .. code-block:: text
1368
1369 Statically allocated code object -> Evaluate
1370
1371 Interpreter startup is now 10-15% faster in Python 3.11. This has a big
1372 impact for short-running programs using Python.
1373
1374 (Contributed by Eric Snow, Guido van Rossum and Kumar Aditya in many issues.)
1375
1376
1377 .. _whatsnew311-faster-runtime:
1378
1379 Faster Runtime
1380 --------------
1381
1382 .. _whatsnew311-lazy-python-frames:
1383
1384 Cheaper, lazy Python frames
1385 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
1386
1387 Python frames, holding execution information,
1388 are created whenever Python calls a Python function.
1389 The following are new frame optimizations:
1390
1391 - Streamlined the frame creation process.
1392 - Avoided memory allocation by generously re-using frame space on the C stack.
1393 - Streamlined the internal frame struct to contain only essential information.
1394 Frames previously held extra debugging and memory management information.
1395
1396 Old-style :ref:`frame objects <frame-objects>`
1397 are now created only when requested by debuggers
1398 or by Python introspection functions such as :func:`sys._getframe` and
1399 :func:`inspect.currentframe`. For most user code, no frame objects are
1400 created at all. As a result, nearly all Python functions calls have sped
1401 up significantly. We measured a 3-7% speedup in pyperformance.
1402
1403 (Contributed by Mark Shannon in :issue:`44590`.)
1404
1405
1406 .. _inline-calls:
1407 .. _whatsnew311-inline-calls:
1408
1409 Inlined Python function calls
1410 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1411
1412 During a Python function call, Python will call an evaluating C function to
1413 interpret that function's code. This effectively limits pure Python recursion to
1414 what's safe for the C stack.
1415
1416 In 3.11, when CPython detects Python code calling another Python function,
1417 it sets up a new frame, and "jumps" to the new code inside the new frame. This
1418 avoids calling the C interpreting function altogether.
1419
1420 Most Python function calls now consume no C stack space, speeding them up.
1421 In simple recursive functions like fibonacci or
1422 factorial, we observed a 1.7x speedup. This also means recursive functions
1423 can recurse significantly deeper
1424 (if the user increases the recursion limit with :func:`sys.setrecursionlimit`).
1425 We measured a 1-3% improvement in pyperformance.
1426
1427 (Contributed by Pablo Galindo and Mark Shannon in :issue:`45256`.)
1428
1429
1430 .. _whatsnew311-pep659:
1431
1432 PEP 659: Specializing Adaptive Interpreter
1433 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1434
1435 :pep:`659` is one of the key parts of the Faster CPython project. The general
1436 idea is that while Python is a dynamic language, most code has regions where
1437 objects and types rarely change. This concept is known as *type stability*.
1438
1439 At runtime, Python will try to look for common patterns and type stability
1440 in the executing code. Python will then replace the current operation with a
1441 more specialized one. This specialized operation uses fast paths available only
1442 to those use cases/types, which generally outperform their generic
1443 counterparts. This also brings in another concept called *inline caching*, where
1444 Python caches the results of expensive operations directly in the
1445 :term:`bytecode`.
1446
1447 The specializer will also combine certain common instruction pairs into one
1448 superinstruction, reducing the overhead during execution.
1449
1450 Python will only specialize
1451 when it sees code that is "hot" (executed multiple times). This prevents Python
1452 from wasting time on run-once code. Python can also de-specialize when code is
1453 too dynamic or when the use changes. Specialization is attempted periodically,
1454 and specialization attempts are not too expensive,
1455 allowing specialization to adapt to new circumstances.
1456
1457 (PEP written by Mark Shannon, with ideas inspired by Stefan Brunthaler.
1458 See :pep:`659` for more information. Implementation by Mark Shannon and Brandt
1459 Bucher, with additional help from Irit Katriel and Dennis Sweeney.)
1460
1461 ..
1462 If I missed out anyone, please add them.
1463
1464 +---------------+--------------------+-------------------------------------------------------+-------------------+-------------------+
1465 | Operation | Form | Specialization | Operation speedup | Contributor(s) |
1466 | | | | (up to) | |
1467 +===============+====================+=======================================================+===================+===================+
1468 | Binary | ``x + x`` | Binary add, multiply and subtract for common types | 10% | Mark Shannon, |
1469 | operations | | such as :class:`int`, :class:`float` and :class:`str` | | Donghee Na, |
1470 | | ``x - x`` | take custom fast paths for their underlying types. | | Brandt Bucher, |
1471 | | | | | Dennis Sweeney |
1472 | | ``x * x`` | | | |
1473 +---------------+--------------------+-------------------------------------------------------+-------------------+-------------------+
1474 | Subscript | ``a[i]`` | Subscripting container types such as :class:`list`, | 10-25% | Irit Katriel, |
1475 | | | :class:`tuple` and :class:`dict` directly index | | Mark Shannon |
1476 | | | the underlying data structures. | | |
1477 | | | | | |
1478 | | | Subscripting custom :meth:`~object.__getitem__` | | |
1479 | | | is also inlined similar to :ref:`inline-calls`. | | |
1480 +---------------+--------------------+-------------------------------------------------------+-------------------+-------------------+
1481 | Store | ``a[i] = z`` | Similar to subscripting specialization above. | 10-25% | Dennis Sweeney |
1482 | subscript | | | | |
1483 +---------------+--------------------+-------------------------------------------------------+-------------------+-------------------+
1484 | Calls | ``f(arg)`` | Calls to common builtin (C) functions and types such | 20% | Mark Shannon, |
1485 | | | as :func:`len` and :class:`str` directly call their | | Ken Jin |
1486 | | ``C(arg)`` | underlying C version. This avoids going through the | | |
1487 | | | internal calling convention. | | |
1488 +---------------+--------------------+-------------------------------------------------------+-------------------+-------------------+
1489 | Load | ``print`` | The object's index in the globals/builtins namespace | [#load-global]_ | Mark Shannon |
1490 | global | | is cached. Loading globals and builtins require | | |
1491 | variable | ``len`` | zero namespace lookups. | | |
1492 +---------------+--------------------+-------------------------------------------------------+-------------------+-------------------+
1493 | Load | ``o.attr`` | Similar to loading global variables. The attribute's | [#load-attr]_ | Mark Shannon |
1494 | attribute | | index inside the class/object's namespace is cached. | | |
1495 | | | In most cases, attribute loading will require zero | | |
1496 | | | namespace lookups. | | |
1497 +---------------+--------------------+-------------------------------------------------------+-------------------+-------------------+
1498 | Load | ``o.meth()`` | The actual address of the method is cached. Method | 10-20% | Ken Jin, |
1499 | methods for | | loading now has no namespace lookups -- even for | | Mark Shannon |
1500 | call | | classes with long inheritance chains. | | |
1501 +---------------+--------------------+-------------------------------------------------------+-------------------+-------------------+
1502 | Store | ``o.attr = z`` | Similar to load attribute optimization. | 2% | Mark Shannon |
1503 | attribute | | | in pyperformance | |
1504 +---------------+--------------------+-------------------------------------------------------+-------------------+-------------------+
1505 | Unpack | ``*seq`` | Specialized for common containers such as | 8% | Brandt Bucher |
1506 | Sequence | | :class:`list` and :class:`tuple`. | | |
1507 | | | Avoids internal calling convention. | | |
1508 +---------------+--------------------+-------------------------------------------------------+-------------------+-------------------+
1509
1510 .. [#load-global] A similar optimization already existed since Python 3.8.
1511 3.11 specializes for more forms and reduces some overhead.
1512
1513 .. [#load-attr] A similar optimization already existed since Python 3.10.
1514 3.11 specializes for more forms. Furthermore, all attribute loads should
1515 be sped up by :issue:`45947`.
1516
1517
1518 .. _whatsnew311-faster-cpython-misc:
1519
1520 Misc
1521 ----
1522
1523 * Objects now require less memory due to lazily created object namespaces.
1524 Their namespace dictionaries now also share keys more freely.
1525 (Contributed Mark Shannon in :issue:`45340` and :issue:`40116`.)
1526
1527 * "Zero-cost" exceptions are implemented, eliminating the cost
1528 of :keyword:`try` statements when no exception is raised.
1529 (Contributed by Mark Shannon in :issue:`40222`.)
1530
1531 * A more concise representation of exceptions in the interpreter reduced the
1532 time required for catching an exception by about 10%.
1533 (Contributed by Irit Katriel in :issue:`45711`.)
1534
1535 * :mod:`re`'s regular expression matching engine has been partially refactored,
1536 and now uses computed gotos (or "threaded code") on supported platforms. As a
1537 result, Python 3.11 executes the `pyperformance regular expression benchmarks
1538 <https://pyperformance.readthedocs.io/benchmarks.html#regex-dna>`_ up to 10%
1539 faster than Python 3.10.
1540 (Contributed by Brandt Bucher in :gh:`91404`.)
1541
1542
1543 .. _whatsnew311-faster-cpython-faq:
1544
1545 FAQ
1546 ---
1547
1548 .. _faster-cpython-faq-my-code:
1549
1550 How should I write my code to utilize these speedups?
1551 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1552
1553 Write Pythonic code that follows common best practices;
1554 you don't have to change your code.
1555 The Faster CPython project optimizes for common code patterns we observe.
1556
1557
1558 .. _faster-cpython-faq-memory:
1559
1560 Will CPython 3.11 use more memory?
1561 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1562
1563 Maybe not; we don't expect memory use to exceed 20% higher than 3.10.
1564 This is offset by memory optimizations for frame objects and object
1565 dictionaries as mentioned above.
1566
1567
1568 .. _faster-cpython-ymmv:
1569
1570 I don't see any speedups in my workload. Why?
1571 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1572
1573 Certain code won't have noticeable benefits. If your code spends most of
1574 its time on I/O operations, or already does most of its
1575 computation in a C extension library like NumPy, there won't be significant
1576 speedups. This project currently benefits pure-Python workloads the most.
1577
1578 Furthermore, the pyperformance figures are a geometric mean. Even within the
1579 pyperformance benchmarks, certain benchmarks have slowed down slightly, while
1580 others have sped up by nearly 2x!
1581
1582
1583 .. _faster-cpython-jit:
1584
1585 Is there a JIT compiler?
1586 ^^^^^^^^^^^^^^^^^^^^^^^^
1587
1588 No. We're still exploring other optimizations.
1589
1590
1591 .. _whatsnew311-faster-cpython-about:
1592
1593 About
1594 -----
1595
1596 Faster CPython explores optimizations for :term:`CPython`. The main team is
1597 funded by Microsoft to work on this full-time. Pablo Galindo Salgado is also
1598 funded by Bloomberg LP to work on the project part-time. Finally, many
1599 contributors are volunteers from the community.
1600
1601
1602 .. _whatsnew311-bytecode-changes:
1603
1604 CPython bytecode changes
1605 ========================
1606
1607 The bytecode now contains inline cache entries,
1608 which take the form of the newly-added :opcode:`CACHE` instructions.
1609 Many opcodes expect to be followed by an exact number of caches,
1610 and instruct the interpreter to skip over them at runtime.
1611 Populated caches can look like arbitrary instructions,
1612 so great care should be taken when reading or modifying
1613 raw, adaptive bytecode containing quickened data.
1614
1615
1616 .. _whatsnew311-added-opcodes:
1617
1618 New opcodes
1619 -----------
1620
1621 * :opcode:`!ASYNC_GEN_WRAP`, :opcode:`RETURN_GENERATOR` and :opcode:`SEND`,
1622 used in generators and co-routines.
1623
1624 * :opcode:`COPY_FREE_VARS`,
1625 which avoids needing special caller-side code for closures.
1626
1627 * :opcode:`JUMP_BACKWARD_NO_INTERRUPT`,
1628 for use in certain loops where handling interrupts is undesirable.
1629
1630 * :opcode:`MAKE_CELL`, to create :ref:`cell-objects`.
1631
1632 * :opcode:`CHECK_EG_MATCH` and :opcode:`!PREP_RERAISE_STAR`,
1633 to handle the :ref:`new exception groups and except* <whatsnew311-pep654>`
1634 added in :pep:`654`.
1635
1636 * :opcode:`PUSH_EXC_INFO`, for use in exception handlers.
1637
1638 * :opcode:`RESUME`, a no-op,
1639 for internal tracing, debugging and optimization checks.
1640
1641
1642 .. _whatsnew311-replaced-opcodes:
1643
1644 Replaced opcodes
1645 ----------------
1646
1647 +------------------------------------+------------------------------------+-----------------------------------------+
1648 | Replaced Opcode(s) | New Opcode(s) | Notes |
1649 +====================================+====================================+=========================================+
1650 | | :opcode:`!BINARY_*` | :opcode:`BINARY_OP` | Replaced all numeric binary/in-place |
1651 | | :opcode:`!INPLACE_*` | | opcodes with a single opcode |
1652 +------------------------------------+------------------------------------+-----------------------------------------+
1653 | | :opcode:`!CALL_FUNCTION` | | :opcode:`CALL` | Decouples argument shifting for methods |
1654 | | :opcode:`!CALL_FUNCTION_KW` | | :opcode:`!KW_NAMES` | from handling of keyword arguments; |
1655 | | :opcode:`!CALL_METHOD` | | :opcode:`!PRECALL` | allows better specialization of calls |
1656 | | | :opcode:`PUSH_NULL` | |
1657 +------------------------------------+------------------------------------+-----------------------------------------+
1658 | | :opcode:`!DUP_TOP` | | :opcode:`COPY` | Stack manipulation instructions |
1659 | | :opcode:`!DUP_TOP_TWO` | | :opcode:`SWAP` | |
1660 | | :opcode:`!ROT_TWO` | | |
1661 | | :opcode:`!ROT_THREE` | | |
1662 | | :opcode:`!ROT_FOUR` | | |
1663 | | :opcode:`!ROT_N` | | |
1664 +------------------------------------+------------------------------------+-----------------------------------------+
1665 | | :opcode:`!JUMP_IF_NOT_EXC_MATCH` | | :opcode:`CHECK_EXC_MATCH` | Now performs check but doesn't jump |
1666 +------------------------------------+------------------------------------+-----------------------------------------+
1667 | | :opcode:`!JUMP_ABSOLUTE` | | :opcode:`JUMP_BACKWARD` | See [#bytecode-jump]_; |
1668 | | :opcode:`!POP_JUMP_IF_FALSE` | | :opcode:`!POP_JUMP_BACKWARD_IF_*`| ``TRUE``, ``FALSE``, |
1669 | | :opcode:`!POP_JUMP_IF_TRUE` | | :opcode:`!POP_JUMP_FORWARD_IF_*` | ``NONE`` and ``NOT_NONE`` variants |
1670 | | | for each direction |
1671 | | | |
1672 +------------------------------------+------------------------------------+-----------------------------------------+
1673 | | :opcode:`!SETUP_WITH` | :opcode:`!BEFORE_WITH` | :keyword:`with` block setup |
1674 | | :opcode:`!SETUP_ASYNC_WITH` | | |
1675 +------------------------------------+------------------------------------+-----------------------------------------+
1676
1677 .. [#bytecode-jump] All jump opcodes are now relative, including the
1678 existing :opcode:`!JUMP_IF_TRUE_OR_POP` and :opcode:`!JUMP_IF_FALSE_OR_POP`.
1679 The argument is now an offset from the current instruction
1680 rather than an absolute location.
1681
1682
1683 .. _whatsnew311-changed-opcodes:
1684 .. _whatsnew311-removed-opcodes:
1685 .. _whatsnew311-changed-removed-opcodes:
1686
1687 Changed/removed opcodes
1688 -----------------------
1689
1690 * Changed :opcode:`MATCH_CLASS` and :opcode:`MATCH_KEYS`
1691 to no longer push an additional boolean value to indicate success/failure.
1692 Instead, ``None`` is pushed on failure
1693 in place of the tuple of extracted values.
1694
1695 * Changed opcodes that work with exceptions to reflect them
1696 now being represented as one item on the stack instead of three
1697 (see :gh:`89874`).
1698
1699 * Removed :opcode:`!COPY_DICT_WITHOUT_KEYS`, :opcode:`!GEN_START`,
1700 :opcode:`!POP_BLOCK`, :opcode:`!SETUP_FINALLY` and :opcode:`!YIELD_FROM`.
1701
1702
1703 .. _whatsnew311-deprecated:
1704 .. _whatsnew311-python-api-deprecated:
1705
1706 Deprecated
1707 ==========
1708
1709 This section lists Python APIs that have been deprecated in Python 3.11.
1710
1711 Deprecated C APIs are :ref:`listed separately <whatsnew311-c-api-deprecated>`.
1712
1713
1714 .. _whatsnew311-deprecated-language:
1715 .. _whatsnew311-deprecated-builtins:
1716
1717 Language/Builtins
1718 -----------------
1719
1720 * Chaining :class:`classmethod` descriptors (introduced in :issue:`19072`)
1721 is now deprecated. It can no longer be used to wrap other descriptors
1722 such as :class:`property`. The core design of this feature was flawed
1723 and caused a number of downstream problems. To "pass-through" a
1724 :class:`classmethod`, consider using the :attr:`!__wrapped__` attribute
1725 that was added in Python 3.10.
1726 (Contributed by Raymond Hettinger in :gh:`89519`.)
1727
1728 * Octal escapes in string and bytes literals with values larger than ``0o377``
1729 (255 in decimal) now produce a :exc:`DeprecationWarning`.
1730 In a future Python version, they will raise a :exc:`SyntaxWarning` and
1731 eventually a :exc:`SyntaxError`.
1732 (Contributed by Serhiy Storchaka in :gh:`81548`.)
1733
1734 * The delegation of :func:`int` to :meth:`~object.__trunc__` is now deprecated.
1735 Calling ``int(a)`` when ``type(a)`` implements :meth:`!__trunc__` but not
1736 :meth:`~object.__int__` or :meth:`~object.__index__` now raises
1737 a :exc:`DeprecationWarning`.
1738 (Contributed by Zackery Spytz in :issue:`44977`.)
1739
1740
1741 .. _whatsnew311-deprecated-modules:
1742
1743 Modules
1744 -------
1745
1746 .. _whatsnew311-pep594:
1747
1748 * :pep:`594` led to the deprecations of the following modules
1749 slated for removal in Python 3.13:
1750
1751 +---------------------+---------------------+---------------------+---------------------+---------------------+
1752 | :mod:`!aifc` | :mod:`!chunk` | :mod:`!msilib` | :mod:`!pipes` | :mod:`!telnetlib` |
1753 +---------------------+---------------------+---------------------+---------------------+---------------------+
1754 | :mod:`!audioop` | :mod:`!crypt` | :mod:`!nis` | :mod:`!sndhdr` | :mod:`!uu` |
1755 +---------------------+---------------------+---------------------+---------------------+---------------------+
1756 | :mod:`!cgi` | :mod:`!imghdr` | :mod:`!nntplib` | :mod:`!spwd` | :mod:`!xdrlib` |
1757 +---------------------+---------------------+---------------------+---------------------+---------------------+
1758 | :mod:`!cgitb` | :mod:`!mailcap` | :mod:`!ossaudiodev` | :mod:`!sunau` | |
1759 +---------------------+---------------------+---------------------+---------------------+---------------------+
1760
1761 (Contributed by Brett Cannon in :issue:`47061` and Victor Stinner in
1762 :gh:`68966`.)
1763
1764 * The :mod:`!asynchat`, :mod:`!asyncore` and :mod:`!smtpd` modules have been
1765 deprecated since at least Python 3.6. Their documentation and deprecation
1766 warnings have now been updated to note they will be removed in Python 3.12.
1767 (Contributed by Hugo van Kemenade in :issue:`47022`.)
1768
1769 * The :mod:`!lib2to3` package and ``2to3`` tool
1770 are now deprecated and may not be able to parse Python 3.10 or newer.
1771 See :pep:`617`, introducing the new PEG parser, for details.
1772 (Contributed by Victor Stinner in :issue:`40360`.)
1773
1774 * Undocumented modules :mod:`!sre_compile`, :mod:`!sre_constants`
1775 and :mod:`!sre_parse` are now deprecated.
1776 (Contributed by Serhiy Storchaka in :issue:`47152`.)
1777
1778
1779 .. _whatsnew311-deprecated-stdlib:
1780
1781 Standard Library
1782 ----------------
1783
1784 * The following have been deprecated in :mod:`configparser` since Python 3.2.
1785 Their deprecation warnings have now been updated to note they will be removed
1786 in Python 3.12:
1787
1788 * the :class:`!configparser.SafeConfigParser` class
1789 * the :attr:`!configparser.ParsingError.filename` property
1790 * the :meth:`!configparser.RawConfigParser.readfp` method
1791
1792 (Contributed by Hugo van Kemenade in :issue:`45173`.)
1793
1794 * :class:`!configparser.LegacyInterpolation` has been deprecated in the docstring
1795 since Python 3.2, and is not listed in the :mod:`configparser` documentation.
1796 It now emits a :exc:`DeprecationWarning` and will be removed
1797 in Python 3.13. Use :class:`configparser.BasicInterpolation` or
1798 :class:`configparser.ExtendedInterpolation` instead.
1799 (Contributed by Hugo van Kemenade in :issue:`46607`.)
1800
1801 * The older set of :mod:`importlib.resources` functions were deprecated
1802 in favor of the replacements added in Python 3.9
1803 and will be removed in a future Python version,
1804 due to not supporting resources located within package subdirectories:
1805
1806 * :func:`!importlib.resources.contents`
1807 * :func:`!importlib.resources.is_resource`
1808 * :func:`!importlib.resources.open_binary`
1809 * :func:`!importlib.resources.open_text`
1810 * :func:`!importlib.resources.read_binary`
1811 * :func:`!importlib.resources.read_text`
1812 * :func:`!importlib.resources.path`
1813
1814 * The :func:`locale.getdefaultlocale` function is deprecated and will be
1815 removed in Python 3.15. Use :func:`locale.setlocale`,
1816 :func:`locale.getpreferredencoding(False) <locale.getpreferredencoding>` and
1817 :func:`locale.getlocale` functions instead.
1818 (Contributed by Victor Stinner in :gh:`90817`.)
1819
1820 * The :func:`!locale.resetlocale` function is deprecated and will be
1821 removed in Python 3.13. Use ``locale.setlocale(locale.LC_ALL, "")`` instead.
1822 (Contributed by Victor Stinner in :gh:`90817`.)
1823
1824 * Stricter rules will now be applied for numerical group references
1825 and group names in :ref:`regular expressions <re-syntax>`.
1826 Only sequences of ASCII digits will now be accepted as a numerical reference,
1827 and the group name in :class:`bytes` patterns and replacement strings
1828 can only contain ASCII letters, digits and underscores.
1829 For now, a deprecation warning is raised for syntax violating these rules.
1830 (Contributed by Serhiy Storchaka in :gh:`91760`.)
1831
1832 * In the :mod:`re` module, the :func:`!re.template` function
1833 and the corresponding :const:`!re.TEMPLATE` and :const:`!re.T` flags
1834 are deprecated, as they were undocumented and lacked an obvious purpose.
1835 They will be removed in Python 3.13.
1836 (Contributed by Serhiy Storchaka and Miro Hrončok in :gh:`92728`.)
1837
1838 * :func:`!turtle.settiltangle` has been deprecated since Python 3.1;
1839 it now emits a deprecation warning and will be removed in Python 3.13. Use
1840 :func:`turtle.tiltangle` instead (it was earlier incorrectly marked
1841 as deprecated, and its docstring is now corrected).
1842 (Contributed by Hugo van Kemenade in :issue:`45837`.)
1843
1844 * :class:`typing.Text`, which exists solely to provide compatibility support
1845 between Python 2 and Python 3 code, is now deprecated. Its removal is
1846 currently unplanned, but users are encouraged to use :class:`str` instead
1847 wherever possible.
1848 (Contributed by Alex Waygood in :gh:`92332`.)
1849
1850 * The keyword argument syntax for constructing :data:`typing.TypedDict` types
1851 is now deprecated. Support will be removed in Python 3.13. (Contributed by
1852 Jingchen Ye in :gh:`90224`.)
1853
1854 * :class:`!webbrowser.MacOSX` is deprecated and will be removed in Python 3.13.
1855 It is untested, undocumented, and not used by :mod:`webbrowser` itself.
1856 (Contributed by Donghee Na in :issue:`42255`.)
1857
1858 * The behavior of returning a value from a :class:`~unittest.TestCase` and
1859 :class:`~unittest.IsolatedAsyncioTestCase` test methods (other than the
1860 default ``None`` value) is now deprecated.
1861
1862 * Deprecated the following not-formally-documented :mod:`unittest` functions,
1863 scheduled for removal in Python 3.13:
1864
1865 * :func:`!unittest.findTestCases`
1866 * :func:`!unittest.makeSuite`
1867 * :func:`!unittest.getTestCaseNames`
1868
1869 Use :class:`~unittest.TestLoader` methods instead:
1870
1871 * :meth:`unittest.TestLoader.loadTestsFromModule`
1872 * :meth:`unittest.TestLoader.loadTestsFromTestCase`
1873 * :meth:`unittest.TestLoader.getTestCaseNames`
1874
1875 (Contributed by Erlend E. Aasland in :issue:`5846`.)
1876
1877 * :meth:`!unittest.TestProgram.usageExit` is marked deprecated, to be removed
1878 in 3.13.
1879 (Contributed by Carlos Damázio in :gh:`67048`.)
1880
1881
1882 .. _whatsnew311-pending-removal:
1883 .. _whatsnew311-python-api-pending-removal:
1884
1885 Pending Removal in Python 3.12
1886 ==============================
1887
1888 The following Python APIs have been deprecated in earlier Python releases,
1889 and will be removed in Python 3.12.
1890
1891 C APIs pending removal are
1892 :ref:`listed separately <whatsnew311-c-api-pending-removal>`.
1893
1894 * The :mod:`!asynchat` module
1895 * The :mod:`!asyncore` module
1896 * The :ref:`entire distutils package <distutils-deprecated>`
1897 * The :mod:`!imp` module
1898 * The :class:`typing.io <typing.IO>` namespace
1899 * The :class:`typing.re <typing.Pattern>` namespace
1900 * :func:`!cgi.log`
1901 * :func:`!importlib.find_loader`
1902 * :meth:`!importlib.abc.Loader.module_repr`
1903 * :meth:`!importlib.abc.MetaPathFinder.find_module`
1904 * :meth:`!importlib.abc.PathEntryFinder.find_loader`
1905 * :meth:`!importlib.abc.PathEntryFinder.find_module`
1906 * :meth:`!importlib.machinery.BuiltinImporter.find_module`
1907 * :meth:`!importlib.machinery.BuiltinLoader.module_repr`
1908 * :meth:`!importlib.machinery.FileFinder.find_loader`
1909 * :meth:`!importlib.machinery.FileFinder.find_module`
1910 * :meth:`!importlib.machinery.FrozenImporter.find_module`
1911 * :meth:`!importlib.machinery.FrozenLoader.module_repr`
1912 * :meth:`!importlib.machinery.PathFinder.find_module`
1913 * :meth:`!importlib.machinery.WindowsRegistryFinder.find_module`
1914 * :func:`!importlib.util.module_for_loader`
1915 * :func:`!importlib.util.set_loader_wrapper`
1916 * :func:`!importlib.util.set_package_wrapper`
1917 * :class:`!pkgutil.ImpImporter`
1918 * :class:`!pkgutil.ImpLoader`
1919 * :meth:`!pathlib.Path.link_to`
1920 * :func:`!sqlite3.enable_shared_cache`
1921 * :func:`!sqlite3.OptimizedUnicode`
1922 * :envvar:`!PYTHONTHREADDEBUG` environment variable
1923 * The following deprecated aliases in :mod:`unittest`:
1924
1925 ============================ =============================== ===============
1926 Deprecated alias Method Name Deprecated in
1927 ============================ =============================== ===============
1928 ``failUnless`` :meth:`.assertTrue` 3.1
1929 ``failIf`` :meth:`.assertFalse` 3.1
1930 ``failUnlessEqual`` :meth:`.assertEqual` 3.1
1931 ``failIfEqual`` :meth:`.assertNotEqual` 3.1
1932 ``failUnlessAlmostEqual`` :meth:`.assertAlmostEqual` 3.1
1933 ``failIfAlmostEqual`` :meth:`.assertNotAlmostEqual` 3.1
1934 ``failUnlessRaises`` :meth:`.assertRaises` 3.1
1935 ``assert_`` :meth:`.assertTrue` 3.2
1936 ``assertEquals`` :meth:`.assertEqual` 3.2
1937 ``assertNotEquals`` :meth:`.assertNotEqual` 3.2
1938 ``assertAlmostEquals`` :meth:`.assertAlmostEqual` 3.2
1939 ``assertNotAlmostEquals`` :meth:`.assertNotAlmostEqual` 3.2
1940 ``assertRegexpMatches`` :meth:`.assertRegex` 3.2
1941 ``assertRaisesRegexp`` :meth:`.assertRaisesRegex` 3.2
1942 ``assertNotRegexpMatches`` :meth:`.assertNotRegex` 3.5
1943 ============================ =============================== ===============
1944
1945 .. _whatsnew311-removed:
1946 .. _whatsnew311-python-api-removed:
1947
1948 Removed
1949 =======
1950
1951 This section lists Python APIs that have been removed in Python 3.11.
1952
1953 Removed C APIs are :ref:`listed separately <whatsnew311-c-api-removed>`.
1954
1955 * Removed the :func:`[email protected]` :term:`decorator`
1956 enabling legacy generator-based coroutines to be compatible with
1957 :keyword:`async` / :keyword:`await` code.
1958 The function has been deprecated since Python 3.8 and the removal was
1959 initially scheduled for Python 3.10. Use :keyword:`async def` instead.
1960 (Contributed by Illia Volochii in :issue:`43216`.)
1961
1962 * Removed :class:`!asyncio.coroutines.CoroWrapper` used for wrapping legacy
1963 generator-based coroutine objects in the debug mode.
1964 (Contributed by Illia Volochii in :issue:`43216`.)
1965
1966 * Due to significant security concerns, the *reuse_address* parameter of
1967 :meth:`asyncio.loop.create_datagram_endpoint`, disabled in Python 3.9, is
1968 now entirely removed. This is because of the behavior of the socket option
1969 ``SO_REUSEADDR`` in UDP.
1970 (Contributed by Hugo van Kemenade in :issue:`45129`.)
1971
1972 * Removed the :mod:`!binhex` module, deprecated in Python 3.9.
1973 Also removed the related, similarly-deprecated :mod:`binascii` functions:
1974
1975 * :func:`!binascii.a2b_hqx`
1976 * :func:`!binascii.b2a_hqx`
1977 * :func:`!binascii.rlecode_hqx`
1978 * :func:`!binascii.rldecode_hqx`
1979
1980 The :func:`binascii.crc_hqx` function remains available.
1981
1982 (Contributed by Victor Stinner in :issue:`45085`.)
1983
1984 * Removed the :mod:`!distutils` ``bdist_msi`` command deprecated in Python 3.9.
1985 Use ``bdist_wheel`` (wheel packages) instead.
1986 (Contributed by Hugo van Kemenade in :issue:`45124`.)
1987
1988 * Removed the :meth:`~object.__getitem__` methods of
1989 :class:`xml.dom.pulldom.DOMEventStream`, :class:`wsgiref.util.FileWrapper`
1990 and :class:`fileinput.FileInput`, deprecated since Python 3.9.
1991 (Contributed by Hugo van Kemenade in :issue:`45132`.)
1992
1993 * Removed the deprecated :mod:`gettext` functions
1994 :func:`!lgettext`, :func:`!ldgettext`,
1995 :func:`!lngettext` and :func:`!ldngettext`.
1996 Also removed the :func:`!bind_textdomain_codeset` function,
1997 the :meth:`!NullTranslations.output_charset` and
1998 :meth:`!NullTranslations.set_output_charset` methods,
1999 and the *codeset* parameter of :func:`!translation` and :func:`!install`,
2000 since they are only used for the :func:`!l*gettext` functions.
2001 (Contributed by Donghee Na and Serhiy Storchaka in :issue:`44235`.)
2002
2003 * Removed from the :mod:`inspect` module:
2004
2005 * The :func:`!getargspec` function, deprecated since Python 3.0;
2006 use :func:`inspect.signature` or :func:`inspect.getfullargspec` instead.
2007
2008 * The :func:`!formatargspec` function, deprecated since Python 3.5;
2009 use the :func:`inspect.signature` function
2010 or the :class:`inspect.Signature` object directly.
2011
2012 * The undocumented :meth:`!Signature.from_builtin`
2013 and :meth:`!Signature.from_function` methods, deprecated since Python 3.5;
2014 use the :meth:`Signature.from_callable() <inspect.Signature.from_callable>`
2015 method instead.
2016
2017 (Contributed by Hugo van Kemenade in :issue:`45320`.)
2018
2019 * Removed the :meth:`~object.__class_getitem__` method
2020 from :class:`pathlib.PurePath`,
2021 because it was not used and added by mistake in previous versions.
2022 (Contributed by Nikita Sobolev in :issue:`46483`.)
2023
2024 * Removed the :class:`!MailmanProxy` class in the :mod:`!smtpd` module,
2025 as it is unusable without the external :mod:`!mailman` package.
2026 (Contributed by Donghee Na in :issue:`35800`.)
2027
2028 * Removed the deprecated :meth:`!split` method of :class:`!_tkinter.TkappType`.
2029 (Contributed by Erlend E. Aasland in :issue:`38371`.)
2030
2031 * Removed namespace package support from :mod:`unittest` discovery.
2032 It was introduced in Python 3.4 but has been broken since Python 3.7.
2033 (Contributed by Inada Naoki in :issue:`23882`.)
2034
2035 * Removed the undocumented private :meth:`!float.__set_format__` method,
2036 previously known as :meth:`!float.__setformat__` in Python 3.7.
2037 Its docstring said: "You probably don't want to use this function.
2038 It exists mainly to be used in Python's test suite."
2039 (Contributed by Victor Stinner in :issue:`46852`.)
2040
2041 * The :option:`!--experimental-isolated-subinterpreters` configure flag
2042 (and corresponding :c:macro:`!EXPERIMENTAL_ISOLATED_SUBINTERPRETERS` macro)
2043 have been removed.
2044
2045 * :pypi:`Pynche`
2046 --- The Pythonically Natural Color and Hue Editor --- has been moved out
2047 of ``Tools/scripts`` and is `being developed independently
2048 <https://gitlab.com/warsaw/pynche/-/tree/main>`_ from the Python source tree.
2049
2050
2051 .. _whatsnew311-porting:
2052 .. _whatsnew311-python-api-porting:
2053
2054 Porting to Python 3.11
2055 ======================
2056
2057 This section lists previously described changes and other bugfixes
2058 in the Python API that may require changes to your Python code.
2059
2060 Porting notes for the C API are
2061 :ref:`listed separately <whatsnew311-c-api-porting>`.
2062
2063 * :func:`open`, :func:`io.open`, :func:`codecs.open` and
2064 :class:`fileinput.FileInput` no longer accept ``'U'`` ("universal newline")
2065 in the file mode. In Python 3, "universal newline" mode is used by default
2066 whenever a file is opened in text mode,
2067 and the ``'U'`` flag has been deprecated since Python 3.3.
2068 The :ref:`newline parameter <open-newline-parameter>`
2069 to these functions controls how universal newlines work.
2070 (Contributed by Victor Stinner in :issue:`37330`.)
2071
2072 * :class:`ast.AST` node positions are now validated when provided to
2073 :func:`compile` and other related functions. If invalid positions are detected,
2074 a :exc:`ValueError` will be raised. (Contributed by Pablo Galindo in :gh:`93351`)
2075
2076 * Prohibited passing non-:class:`concurrent.futures.ThreadPoolExecutor`
2077 executors to :meth:`asyncio.loop.set_default_executor`
2078 following a deprecation in Python 3.8.
2079 (Contributed by Illia Volochii in :issue:`43234`.)
2080
2081 * :mod:`calendar`: The :class:`calendar.LocaleTextCalendar` and
2082 :class:`calendar.LocaleHTMLCalendar` classes now use
2083 :func:`locale.getlocale`, instead of using :func:`locale.getdefaultlocale`,
2084 if no locale is specified.
2085 (Contributed by Victor Stinner in :issue:`46659`.)
2086
2087 * The :mod:`pdb` module now reads the :file:`.pdbrc` configuration file with
2088 the ``'UTF-8'`` encoding.
2089 (Contributed by Srinivas Reddy Thatiparthy (శ్రీనివాస్ రెడ్డి తాటిపర్తి) in :issue:`41137`.)
2090
2091 * The *population* parameter of :func:`random.sample` must be a sequence,
2092 and automatic conversion of :class:`set`\s to :class:`list`\s
2093 is no longer supported. Also, if the sample size
2094 is larger than the population size, a :exc:`ValueError` is raised.
2095 (Contributed by Raymond Hettinger in :issue:`40465`.)
2096
2097 * The *random* optional parameter of :func:`random.shuffle` was removed.
2098 It was previously an arbitrary random function to use for the shuffle;
2099 now, :func:`random.random` (its previous default) will always be used.
2100
2101 * In :mod:`re` :ref:`re-syntax`, global inline flags (e.g. ``(?i)``)
2102 can now only be used at the start of regular expressions.
2103 Using them elsewhere has been deprecated since Python 3.6.
2104 (Contributed by Serhiy Storchaka in :issue:`47066`.)
2105
2106 * In the :mod:`re` module, several long-standing bugs where fixed that,
2107 in rare cases, could cause capture groups to get the wrong result.
2108 Therefore, this could change the captured output in these cases.
2109 (Contributed by Ma Lin in :issue:`35859`.)
2110
2111
2112 .. _whatsnew311-build-changes:
2113
2114 Build Changes
2115 =============
2116
2117 * CPython now has :pep:`11` :pep:`Tier 3 support <11#tier-3>` for
2118 cross compiling to the `WebAssembly <https://webassembly.org/>`_ platforms
2119 `Emscripten <https://emscripten.org/>`_
2120 (``wasm32-unknown-emscripten``, i.e. Python in the browser)
2121 and `WebAssembly System Interface (WASI) <https://wasi.dev/>`_
2122 (``wasm32-unknown-wasi``).
2123 The effort is inspired by previous work like `Pyodide <https://pyodide.org/>`_.
2124 These platforms provide a limited subset of POSIX APIs; Python standard
2125 libraries features and modules related to networking, processes, threading,
2126 signals, mmap, and users/groups are not available or don't work.
2127 (Emscripten contributed by Christian Heimes and Ethan Smith in :gh:`84461`
2128 and WASI contributed by Christian Heimes in :gh:`90473`;
2129 platforms promoted in :gh:`95085`)
2130
2131 * Building CPython now requires:
2132
2133 * A `C11 <https://en.cppreference.com/w/c/11>`_ compiler and standard library.
2134 `Optional C11 features
2135 <https://en.wikipedia.org/wiki/C11_(C_standard_revision)#Optional_features>`_
2136 are not required.
2137 (Contributed by Victor Stinner in :issue:`46656`,
2138 :issue:`45440` and :issue:`46640`.)
2139
2140 * Support for `IEEE 754 <https://en.wikipedia.org/wiki/IEEE_754>`_
2141 floating-point numbers.
2142 (Contributed by Victor Stinner in :issue:`46917`.)
2143
2144 * The :c:macro:`!Py_NO_NAN` macro has been removed.
2145 Since CPython now requires IEEE 754 floats, NaN values are always available.
2146 (Contributed by Victor Stinner in :issue:`46656`.)
2147
2148 * The :mod:`tkinter` package now requires `Tcl/Tk <https://www.tcl.tk>`_
2149 version 8.5.12 or newer.
2150 (Contributed by Serhiy Storchaka in :issue:`46996`.)
2151
2152 * Build dependencies, compiler flags, and linker flags for most stdlib
2153 extension modules are now detected by :program:`configure`. libffi, libnsl,
2154 libsqlite3, zlib, bzip2, liblzma, libcrypt, Tcl/Tk, and uuid flags
2155 are detected by `pkg-config
2156 <https://www.freedesktop.org/wiki/Software/pkg-config/>`_ (when available).
2157 :mod:`tkinter` now requires a pkg-config command
2158 to detect development settings for `Tcl/Tk`_ headers and libraries.
2159 (Contributed by Christian Heimes and Erlend Egeberg Aasland in
2160 :issue:`45847`, :issue:`45747`, and :issue:`45763`.)
2161
2162 * libpython is no longer linked against libcrypt.
2163 (Contributed by Mike Gilbert in :issue:`45433`.)
2164
2165 * CPython can now be built with the
2166 `ThinLTO <https://clang.llvm.org/docs/ThinLTO.html>`_ option
2167 via passing ``thin`` to :option:`--with-lto`, i.e. ``--with-lto=thin``.
2168 (Contributed by Donghee Na and Brett Holman in :issue:`44340`.)
2169
2170 * Freelists for object structs can now be disabled. A new :program:`configure`
2171 option ``--without-freelists`` can be used to disable all freelists
2172 except empty tuple singleton.
2173 (Contributed by Christian Heimes in :issue:`45522`.)
2174
2175 * ``Modules/Setup`` and ``Modules/makesetup`` have been improved and tied up.
2176 Extension modules can now be built through ``makesetup``. All except some
2177 test modules can be linked statically into a main binary or library.
2178 (Contributed by Brett Cannon and Christian Heimes in :issue:`45548`,
2179 :issue:`45570`, :issue:`45571`, and :issue:`43974`.)
2180
2181 .. note::
2182 Use the environment variables :envvar:`!TCLTK_CFLAGS` and
2183 :envvar:`!TCLTK_LIBS` to manually specify the location of Tcl/Tk headers
2184 and libraries. The :program:`configure` options
2185 :option:`!--with-tcltk-includes` and :option:`!--with-tcltk-libs`
2186 have been removed.
2187
2188 On RHEL 7 and CentOS 7 the development packages do not provide ``tcl.pc``
2189 and ``tk.pc``; use ``TCLTK_LIBS="-ltk8.5 -ltkstub8.5 -ltcl8.5"``.
2190 The directory ``Misc/rhel7`` contains ``.pc`` files and instructions
2191 on how to build Python with RHEL 7's and CentOS 7's Tcl/Tk and OpenSSL.
2192
2193 * CPython will now use 30-bit digits by default for the Python :class:`int`
2194 implementation. Previously, the default was to use 30-bit digits on platforms
2195 with ``SIZEOF_VOID_P >= 8``, and 15-bit digits otherwise. It's still possible
2196 to explicitly request use of 15-bit digits via either the
2197 :option:`--enable-big-digits` option to the configure script
2198 or (for Windows) the ``PYLONG_BITS_IN_DIGIT`` variable in ``PC/pyconfig.h``,
2199 but this option may be removed at some point in the future.
2200 (Contributed by Mark Dickinson in :issue:`45569`.)
2201
2202
2203 .. _whatsnew311-c-api:
2204
2205 C API Changes
2206 =============
2207
2208 .. _whatsnew311-c-api-new-features:
2209
2210 New Features
2211 ------------
2212
2213 * Add a new :c:func:`PyType_GetName` function to get type's short name.
2214 (Contributed by Hai Shi in :issue:`42035`.)
2215
2216 * Add a new :c:func:`PyType_GetQualName` function to get type's qualified name.
2217 (Contributed by Hai Shi in :issue:`42035`.)
2218
2219 * Add new :c:func:`PyThreadState_EnterTracing` and
2220 :c:func:`PyThreadState_LeaveTracing` functions to the limited C API to
2221 suspend and resume tracing and profiling.
2222 (Contributed by Victor Stinner in :issue:`43760`.)
2223
2224 * Added the :c:data:`Py_Version` constant which bears the same value as
2225 :c:macro:`PY_VERSION_HEX`.
2226 (Contributed by Gabriele N. Tornetta in :issue:`43931`.)
2227
2228 * :c:type:`Py_buffer` and APIs are now part of the limited API and the stable
2229 ABI:
2230
2231 * :c:func:`PyObject_CheckBuffer`
2232 * :c:func:`PyObject_GetBuffer`
2233 * :c:func:`PyBuffer_GetPointer`
2234 * :c:func:`PyBuffer_SizeFromFormat`
2235 * :c:func:`PyBuffer_ToContiguous`
2236 * :c:func:`PyBuffer_FromContiguous`
2237 * :c:func:`PyObject_CopyData`
2238 * :c:func:`PyBuffer_IsContiguous`
2239 * :c:func:`PyBuffer_FillContiguousStrides`
2240 * :c:func:`PyBuffer_FillInfo`
2241 * :c:func:`PyBuffer_Release`
2242 * :c:func:`PyMemoryView_FromBuffer`
2243 * :c:member:`~PyBufferProcs.bf_getbuffer` and
2244 :c:member:`~PyBufferProcs.bf_releasebuffer` type slots
2245
2246 (Contributed by Christian Heimes in :issue:`45459`.)
2247
2248 * Added the :c:func:`PyType_GetModuleByDef` function, used to get the module
2249 in which a method was defined, in cases where this information is not
2250 available directly (via :c:type:`PyCMethod`).
2251 (Contributed by Petr Viktorin in :issue:`46613`.)
2252
2253 * Add new functions to pack and unpack C double (serialize and deserialize):
2254 :c:func:`PyFloat_Pack2`, :c:func:`PyFloat_Pack4`, :c:func:`PyFloat_Pack8`,
2255 :c:func:`PyFloat_Unpack2`, :c:func:`PyFloat_Unpack4` and
2256 :c:func:`PyFloat_Unpack8`.
2257 (Contributed by Victor Stinner in :issue:`46906`.)
2258
2259 * Add new functions to get frame object attributes:
2260 :c:func:`PyFrame_GetBuiltins`, :c:func:`PyFrame_GetGenerator`,
2261 :c:func:`PyFrame_GetGlobals`, :c:func:`PyFrame_GetLasti`.
2262
2263 * Added two new functions to get and set the active exception instance:
2264 :c:func:`PyErr_GetHandledException` and :c:func:`PyErr_SetHandledException`.
2265 These are alternatives to :c:func:`PyErr_SetExcInfo()` and
2266 :c:func:`PyErr_GetExcInfo()` which work with the legacy 3-tuple
2267 representation of exceptions.
2268 (Contributed by Irit Katriel in :issue:`46343`.)
2269
2270 * Added the :c:member:`PyConfig.safe_path` member.
2271 (Contributed by Victor Stinner in :gh:`57684`.)
2272
2273
2274 .. _whatsnew311-c-api-porting:
2275
2276 Porting to Python 3.11
2277 ----------------------
2278
2279 .. _whatsnew311-pep670:
2280
2281 * Some macros have been converted to static inline functions to avoid
2282 `macro pitfalls <https://gcc.gnu.org/onlinedocs/cpp/Macro-Pitfalls.html>`_.
2283 The change should be mostly transparent to users,
2284 as the replacement functions will cast their arguments to the expected types
2285 to avoid compiler warnings due to static type checks.
2286 However, when the limited C API is set to >=3.11,
2287 these casts are not done,
2288 and callers will need to cast arguments to their expected types.
2289 See :pep:`670` for more details.
2290 (Contributed by Victor Stinner and Erlend E. Aasland in :gh:`89653`.)
2291
2292 * :c:func:`PyErr_SetExcInfo()` no longer uses the ``type`` and ``traceback``
2293 arguments, the interpreter now derives those values from the exception
2294 instance (the ``value`` argument). The function still steals references
2295 of all three arguments.
2296 (Contributed by Irit Katriel in :issue:`45711`.)
2297
2298 * :c:func:`PyErr_GetExcInfo()` now derives the ``type`` and ``traceback``
2299 fields of the result from the exception instance (the ``value`` field).
2300 (Contributed by Irit Katriel in :issue:`45711`.)
2301
2302 * :c:struct:`_frozen` has a new ``is_package`` field to indicate whether
2303 or not the frozen module is a package. Previously, a negative value
2304 in the ``size`` field was the indicator. Now only non-negative values
2305 be used for ``size``.
2306 (Contributed by Kumar Aditya in :issue:`46608`.)
2307
2308 * :c:func:`_PyFrameEvalFunction` now takes ``_PyInterpreterFrame*``
2309 as its second parameter, instead of ``PyFrameObject*``.
2310 See :pep:`523` for more details of how to use this function pointer type.
2311
2312 * :c:func:`!PyCode_New` and :c:func:`!PyCode_NewWithPosOnlyArgs` now take
2313 an additional ``exception_table`` argument.
2314 Using these functions should be avoided, if at all possible.
2315 To get a custom code object: create a code object using the compiler,
2316 then get a modified version with the ``replace`` method.
2317
2318 * :c:type:`PyCodeObject` no longer has the ``co_code``, ``co_varnames``,
2319 ``co_cellvars`` and ``co_freevars`` fields. Instead, use
2320 :c:func:`PyCode_GetCode`, :c:func:`PyCode_GetVarnames`,
2321 :c:func:`PyCode_GetCellvars` and :c:func:`PyCode_GetFreevars` respectively
2322 to access them via the C API.
2323 (Contributed by Brandt Bucher in :issue:`46841` and Ken Jin in :gh:`92154`
2324 and :gh:`94936`.)
2325
2326 * The old trashcan macros (``Py_TRASHCAN_SAFE_BEGIN``/``Py_TRASHCAN_SAFE_END``)
2327 are now deprecated. They should be replaced by the new macros
2328 ``Py_TRASHCAN_BEGIN`` and ``Py_TRASHCAN_END``.
2329
2330 A tp_dealloc function that has the old macros, such as::
2331
2332 static void
2333 mytype_dealloc(mytype *p)
2334 {
2335 PyObject_GC_UnTrack(p);
2336 Py_TRASHCAN_SAFE_BEGIN(p);
2337 ...
2338 Py_TRASHCAN_SAFE_END
2339 }
2340
2341 should migrate to the new macros as follows::
2342
2343 static void
2344 mytype_dealloc(mytype *p)
2345 {
2346 PyObject_GC_UnTrack(p);
2347 Py_TRASHCAN_BEGIN(p, mytype_dealloc)
2348 ...
2349 Py_TRASHCAN_END
2350 }
2351
2352 Note that ``Py_TRASHCAN_BEGIN`` has a second argument which
2353 should be the deallocation function it is in.
2354
2355 To support older Python versions in the same codebase, you
2356 can define the following macros and use them throughout
2357 the code (credit: these were copied from the ``mypy`` codebase)::
2358
2359 #if PY_VERSION_HEX >= 0x03080000
2360 # define CPy_TRASHCAN_BEGIN(op, dealloc) Py_TRASHCAN_BEGIN(op, dealloc)
2361 # define CPy_TRASHCAN_END(op) Py_TRASHCAN_END
2362 #else
2363 # define CPy_TRASHCAN_BEGIN(op, dealloc) Py_TRASHCAN_SAFE_BEGIN(op)
2364 # define CPy_TRASHCAN_END(op) Py_TRASHCAN_SAFE_END(op)
2365 #endif
2366
2367 * The :c:func:`PyType_Ready` function now raises an error if a type is defined
2368 with the :c:macro:`Py_TPFLAGS_HAVE_GC` flag set but has no traverse function
2369 (:c:member:`PyTypeObject.tp_traverse`).
2370 (Contributed by Victor Stinner in :issue:`44263`.)
2371
2372 * Heap types with the :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` flag can now inherit
2373 the :pep:`590` vectorcall protocol. Previously, this was only possible for
2374 :ref:`static types <static-types>`.
2375 (Contributed by Erlend E. Aasland in :issue:`43908`)
2376
2377 * Since :c:func:`Py_TYPE()` is changed to a inline static function,
2378 ``Py_TYPE(obj) = new_type`` must be replaced with
2379 ``Py_SET_TYPE(obj, new_type)``: see the :c:func:`Py_SET_TYPE()` function
2380 (available since Python 3.9). For backward compatibility, this macro can be
2381 used::
2382
2383 #if PY_VERSION_HEX < 0x030900A4 && !defined(Py_SET_TYPE)
2384 static inline void _Py_SET_TYPE(PyObject *ob, PyTypeObject *type)
2385 { ob->ob_type = type; }
2386 #define Py_SET_TYPE(ob, type) _Py_SET_TYPE((PyObject*)(ob), type)
2387 #endif
2388
2389 (Contributed by Victor Stinner in :issue:`39573`.)
2390
2391 * Since :c:func:`Py_SIZE()` is changed to a inline static function,
2392 ``Py_SIZE(obj) = new_size`` must be replaced with
2393 ``Py_SET_SIZE(obj, new_size)``: see the :c:func:`Py_SET_SIZE()` function
2394 (available since Python 3.9). For backward compatibility, this macro can be
2395 used::
2396
2397 #if PY_VERSION_HEX < 0x030900A4 && !defined(Py_SET_SIZE)
2398 static inline void _Py_SET_SIZE(PyVarObject *ob, Py_ssize_t size)
2399 { ob->ob_size = size; }
2400 #define Py_SET_SIZE(ob, size) _Py_SET_SIZE((PyVarObject*)(ob), size)
2401 #endif
2402
2403 (Contributed by Victor Stinner in :issue:`39573`.)
2404
2405 * ``<Python.h>`` no longer includes the header files ``<stdlib.h>``,
2406 ``<stdio.h>``, ``<errno.h>`` and ``<string.h>`` when the ``Py_LIMITED_API``
2407 macro is set to ``0x030b0000`` (Python 3.11) or higher. C extensions should
2408 explicitly include the header files after ``#include <Python.h>``.
2409 (Contributed by Victor Stinner in :issue:`45434`.)
2410
2411 * The non-limited API files ``cellobject.h``, ``classobject.h``, ``code.h``, ``context.h``,
2412 ``funcobject.h``, ``genobject.h`` and ``longintrepr.h`` have been moved to
2413 the ``Include/cpython`` directory. Moreover, the ``eval.h`` header file was
2414 removed. These files must not be included directly, as they are already
2415 included in ``Python.h``: :ref:`Include Files <api-includes>`. If they have
2416 been included directly, consider including ``Python.h`` instead.
2417 (Contributed by Victor Stinner in :issue:`35134`.)
2418
2419 * The :c:func:`!PyUnicode_CHECK_INTERNED` macro has been excluded from the
2420 limited C API. It was never usable there, because it used internal structures
2421 which are not available in the limited C API.
2422 (Contributed by Victor Stinner in :issue:`46007`.)
2423
2424 * The following frame functions and type are now directly available with
2425 ``#include <Python.h>``, it's no longer needed to add
2426 ``#include <frameobject.h>``:
2427
2428 * :c:func:`PyFrame_Check`
2429 * :c:func:`PyFrame_GetBack`
2430 * :c:func:`PyFrame_GetBuiltins`
2431 * :c:func:`PyFrame_GetGenerator`
2432 * :c:func:`PyFrame_GetGlobals`
2433 * :c:func:`PyFrame_GetLasti`
2434 * :c:func:`PyFrame_GetLocals`
2435 * :c:type:`PyFrame_Type`
2436
2437 (Contributed by Victor Stinner in :gh:`93937`.)
2438
2439 .. _pyframeobject-3.11-hiding:
2440
2441 * The :c:type:`PyFrameObject` structure members have been removed from the
2442 public C API.
2443
2444 While the documentation notes that the :c:type:`PyFrameObject` fields are
2445 subject to change at any time, they have been stable for a long time and were
2446 used in several popular extensions.
2447
2448 In Python 3.11, the frame struct was reorganized to allow performance
2449 optimizations. Some fields were removed entirely, as they were details of the
2450 old implementation.
2451
2452 :c:type:`PyFrameObject` fields:
2453
2454 * ``f_back``: use :c:func:`PyFrame_GetBack`.
2455 * ``f_blockstack``: removed.
2456 * ``f_builtins``: use :c:func:`PyFrame_GetBuiltins`.
2457 * ``f_code``: use :c:func:`PyFrame_GetCode`.
2458 * ``f_gen``: use :c:func:`PyFrame_GetGenerator`.
2459 * ``f_globals``: use :c:func:`PyFrame_GetGlobals`.
2460 * ``f_iblock``: removed.
2461 * ``f_lasti``: use :c:func:`PyFrame_GetLasti`.
2462 Code using ``f_lasti`` with ``PyCode_Addr2Line()`` should use
2463 :c:func:`PyFrame_GetLineNumber` instead; it may be faster.
2464 * ``f_lineno``: use :c:func:`PyFrame_GetLineNumber`
2465 * ``f_locals``: use :c:func:`PyFrame_GetLocals`.
2466 * ``f_stackdepth``: removed.
2467 * ``f_state``: no public API (renamed to ``f_frame.f_state``).
2468 * ``f_trace``: no public API.
2469 * ``f_trace_lines``: use ``PyObject_GetAttrString((PyObject*)frame, "f_trace_lines")``.
2470 * ``f_trace_opcodes``: use ``PyObject_GetAttrString((PyObject*)frame, "f_trace_opcodes")``.
2471 * ``f_localsplus``: no public API (renamed to ``f_frame.localsplus``).
2472 * ``f_valuestack``: removed.
2473
2474 The Python frame object is now created lazily. A side effect is that the
2475 :attr:`~frame.f_back` member must not be accessed directly,
2476 since its value is now also
2477 computed lazily. The :c:func:`PyFrame_GetBack` function must be called
2478 instead.
2479
2480 Debuggers that accessed the :attr:`~frame.f_locals` directly *must* call
2481 :c:func:`PyFrame_GetLocals` instead. They no longer need to call
2482 :c:func:`!PyFrame_FastToLocalsWithError` or :c:func:`!PyFrame_LocalsToFast`,
2483 in fact they should not call those functions. The necessary updating of the
2484 frame is now managed by the virtual machine.
2485
2486 Code defining ``PyFrame_GetCode()`` on Python 3.8 and older::
2487
2488 #if PY_VERSION_HEX < 0x030900B1
2489 static inline PyCodeObject* PyFrame_GetCode(PyFrameObject *frame)
2490 {
2491 Py_INCREF(frame->f_code);
2492 return frame->f_code;
2493 }
2494 #endif
2495
2496 Code defining ``PyFrame_GetBack()`` on Python 3.8 and older::
2497
2498 #if PY_VERSION_HEX < 0x030900B1
2499 static inline PyFrameObject* PyFrame_GetBack(PyFrameObject *frame)
2500 {
2501 Py_XINCREF(frame->f_back);
2502 return frame->f_back;
2503 }
2504 #endif
2505
2506 Or use the `pythoncapi_compat project
2507 <https://github.com/python/pythoncapi-compat>`__ to get these two
2508 functions on older Python versions.
2509
2510 * Changes of the :c:type:`PyThreadState` structure members:
2511
2512 * ``frame``: removed, use :c:func:`PyThreadState_GetFrame` (function added
2513 to Python 3.9 by :issue:`40429`).
2514 Warning: the function returns a :term:`strong reference`, need to call
2515 :c:func:`Py_XDECREF`.
2516 * ``tracing``: changed, use :c:func:`PyThreadState_EnterTracing`
2517 and :c:func:`PyThreadState_LeaveTracing`
2518 (functions added to Python 3.11 by :issue:`43760`).
2519 * ``recursion_depth``: removed,
2520 use ``(tstate->recursion_limit - tstate->recursion_remaining)`` instead.
2521 * ``stackcheck_counter``: removed.
2522
2523 Code defining ``PyThreadState_GetFrame()`` on Python 3.8 and older::
2524
2525 #if PY_VERSION_HEX < 0x030900B1
2526 static inline PyFrameObject* PyThreadState_GetFrame(PyThreadState *tstate)
2527 {
2528 Py_XINCREF(tstate->frame);
2529 return tstate->frame;
2530 }
2531 #endif
2532
2533 Code defining ``PyThreadState_EnterTracing()`` and
2534 ``PyThreadState_LeaveTracing()`` on Python 3.10 and older::
2535
2536 #if PY_VERSION_HEX < 0x030B00A2
2537 static inline void PyThreadState_EnterTracing(PyThreadState *tstate)
2538 {
2539 tstate->tracing++;
2540 #if PY_VERSION_HEX >= 0x030A00A1
2541 tstate->cframe->use_tracing = 0;
2542 #else
2543 tstate->use_tracing = 0;
2544 #endif
2545 }
2546
2547 static inline void PyThreadState_LeaveTracing(PyThreadState *tstate)
2548 {
2549 int use_tracing = (tstate->c_tracefunc != NULL || tstate->c_profilefunc != NULL);
2550 tstate->tracing--;
2551 #if PY_VERSION_HEX >= 0x030A00A1
2552 tstate->cframe->use_tracing = use_tracing;
2553 #else
2554 tstate->use_tracing = use_tracing;
2555 #endif
2556 }
2557 #endif
2558
2559 Or use `the pythoncapi-compat project
2560 <https://github.com/python/pythoncapi-compat>`__ to get these functions
2561 on old Python functions.
2562
2563 * Distributors are encouraged to build Python with the optimized Blake2
2564 library `libb2`_.
2565
2566 * The :c:member:`PyConfig.module_search_paths_set` field must now be set to 1 for
2567 initialization to use :c:member:`PyConfig.module_search_paths` to initialize
2568 :data:`sys.path`. Otherwise, initialization will recalculate the path and replace
2569 any values added to ``module_search_paths``.
2570
2571 * :c:func:`PyConfig_Read` no longer calculates the initial search path, and will not
2572 fill any values into :c:member:`PyConfig.module_search_paths`. To calculate default
2573 paths and then modify them, finish initialization and use :c:func:`PySys_GetObject`
2574 to retrieve :data:`sys.path` as a Python list object and modify it directly.
2575
2576
2577 .. _whatsnew311-c-api-deprecated:
2578
2579 Deprecated
2580 ----------
2581
2582 * Deprecate the following functions to configure the Python initialization:
2583
2584 * :c:func:`!PySys_AddWarnOptionUnicode`
2585 * :c:func:`!PySys_AddWarnOption`
2586 * :c:func:`!PySys_AddXOption`
2587 * :c:func:`!PySys_HasWarnOptions`
2588 * :c:func:`!PySys_SetArgvEx`
2589 * :c:func:`!PySys_SetArgv`
2590 * :c:func:`!PySys_SetPath`
2591 * :c:func:`!Py_SetPath`
2592 * :c:func:`!Py_SetProgramName`
2593 * :c:func:`!Py_SetPythonHome`
2594 * :c:func:`!Py_SetStandardStreamEncoding`
2595 * :c:func:`!_Py_SetProgramFullPath`
2596
2597 Use the new :c:type:`PyConfig` API of the :ref:`Python Initialization Configuration
2598 <init-config>` instead (:pep:`587`).
2599 (Contributed by Victor Stinner in :gh:`88279`.)
2600
2601 * Deprecate the ``ob_shash`` member of the :c:type:`PyBytesObject`. Use :c:func:`PyObject_Hash` instead.
2602 (Contributed by Inada Naoki in :issue:`46864`.)
2603
2604
2605 .. _whatsnew311-c-api-pending-removal:
2606
2607 Pending Removal in Python 3.12
2608 ------------------------------
2609
2610 The following C APIs have been deprecated in earlier Python releases,
2611 and will be removed in Python 3.12.
2612
2613 * :c:func:`!PyUnicode_AS_DATA`
2614 * :c:func:`!PyUnicode_AS_UNICODE`
2615 * :c:func:`!PyUnicode_AsUnicodeAndSize`
2616 * :c:func:`!PyUnicode_AsUnicode`
2617 * :c:func:`!PyUnicode_FromUnicode`
2618 * :c:func:`!PyUnicode_GET_DATA_SIZE`
2619 * :c:func:`!PyUnicode_GET_SIZE`
2620 * :c:func:`!PyUnicode_GetSize`
2621 * :c:func:`!PyUnicode_IS_COMPACT`
2622 * :c:func:`!PyUnicode_IS_READY`
2623 * :c:func:`PyUnicode_READY`
2624 * :c:func:`!PyUnicode_WSTR_LENGTH`
2625 * :c:func:`!_PyUnicode_AsUnicode`
2626 * :c:macro:`!PyUnicode_WCHAR_KIND`
2627 * :c:type:`PyUnicodeObject`
2628 * :c:func:`!PyUnicode_InternImmortal`
2629
2630
2631 .. _whatsnew311-c-api-removed:
2632
2633 Removed
2634 -------
2635
2636 * :c:func:`!PyFrame_BlockSetup` and :c:func:`!PyFrame_BlockPop` have been
2637 removed.
2638 (Contributed by Mark Shannon in :issue:`40222`.)
2639
2640 * Remove the following math macros using the ``errno`` variable:
2641
2642 * ``Py_ADJUST_ERANGE1()``
2643 * ``Py_ADJUST_ERANGE2()``
2644 * ``Py_OVERFLOWED()``
2645 * ``Py_SET_ERANGE_IF_OVERFLOW()``
2646 * ``Py_SET_ERRNO_ON_MATH_ERROR()``
2647
2648 (Contributed by Victor Stinner in :issue:`45412`.)
2649
2650 * Remove ``Py_UNICODE_COPY()`` and ``Py_UNICODE_FILL()`` macros, deprecated
2651 since Python 3.3. Use ``PyUnicode_CopyCharacters()`` or ``memcpy()``
2652 (``wchar_t*`` string), and ``PyUnicode_Fill()`` functions instead.
2653 (Contributed by Victor Stinner in :issue:`41123`.)
2654
2655 * Remove the ``pystrhex.h`` header file. It only contains private functions.
2656 C extensions should only include the main ``<Python.h>`` header file.
2657 (Contributed by Victor Stinner in :issue:`45434`.)
2658
2659 * Remove the ``Py_FORCE_DOUBLE()`` macro. It was used by the
2660 ``Py_IS_INFINITY()`` macro.
2661 (Contributed by Victor Stinner in :issue:`45440`.)
2662
2663 * The following items are no longer available when :c:macro:`Py_LIMITED_API`
2664 is defined:
2665
2666 * :c:func:`PyMarshal_WriteLongToFile`
2667 * :c:func:`PyMarshal_WriteObjectToFile`
2668 * :c:func:`PyMarshal_ReadObjectFromString`
2669 * :c:func:`PyMarshal_WriteObjectToString`
2670 * the ``Py_MARSHAL_VERSION`` macro
2671
2672 These are not part of the :ref:`limited API <limited-api-list>`.
2673
2674 (Contributed by Victor Stinner in :issue:`45474`.)
2675
2676 * Exclude :c:func:`PyWeakref_GET_OBJECT` from the limited C API. It never
2677 worked since the :c:type:`!PyWeakReference` structure is opaque in the
2678 limited C API.
2679 (Contributed by Victor Stinner in :issue:`35134`.)
2680
2681 * Remove the ``PyHeapType_GET_MEMBERS()`` macro. It was exposed in the
2682 public C API by mistake, it must only be used by Python internally.
2683 Use the ``PyTypeObject.tp_members`` member instead.
2684 (Contributed by Victor Stinner in :issue:`40170`.)
2685
2686 * Remove the ``HAVE_PY_SET_53BIT_PRECISION`` macro (moved to the internal C
2687 API).
2688 (Contributed by Victor Stinner in :issue:`45412`.)
2689
2690 .. _whatsnew311-pep624:
2691
2692 * Remove the :c:type:`Py_UNICODE` encoder APIs,
2693 as they have been deprecated since Python 3.3,
2694 are little used
2695 and are inefficient relative to the recommended alternatives.
2696
2697 The removed functions are:
2698
2699 * :func:`!PyUnicode_Encode`
2700 * :func:`!PyUnicode_EncodeASCII`
2701 * :func:`!PyUnicode_EncodeLatin1`
2702 * :func:`!PyUnicode_EncodeUTF7`
2703 * :func:`!PyUnicode_EncodeUTF8`
2704 * :func:`!PyUnicode_EncodeUTF16`
2705 * :func:`!PyUnicode_EncodeUTF32`
2706 * :func:`!PyUnicode_EncodeUnicodeEscape`
2707 * :func:`!PyUnicode_EncodeRawUnicodeEscape`
2708 * :func:`!PyUnicode_EncodeCharmap`
2709 * :func:`!PyUnicode_TranslateCharmap`
2710 * :func:`!PyUnicode_EncodeDecimal`
2711 * :func:`!PyUnicode_TransformDecimalToASCII`
2712
2713 See :pep:`624` for details and
2714 :pep:`migration guidance <624#alternative-apis>`.
2715 (Contributed by Inada Naoki in :issue:`44029`.)
2716
2717
2718 Notable changes in 3.11.4
2719 =========================
2720
2721 tarfile
2722 -------
2723
2724 * The extraction methods in :mod:`tarfile`, and :func:`shutil.unpack_archive`,
2725 have a new a *filter* argument that allows limiting tar features than may be
2726 surprising or dangerous, such as creating files outside the destination
2727 directory.
2728 See :ref:`tarfile-extraction-filter` for details.
2729 In Python 3.12, use without the *filter* argument will show a
2730 :exc:`DeprecationWarning`.
2731 In Python 3.14, the default will switch to ``'data'``.
2732 (Contributed by Petr Viktorin in :pep:`706`.)
2733
2734
2735 Notable changes in 3.11.5
2736 =========================
2737
2738 OpenSSL
2739 -------
2740
2741 * Windows builds and macOS installers from python.org now use OpenSSL 3.0.
2742
2743
2744 .. _libb2: https://www.blake2.net/