| 1 |
**************************** |
| 2 |
What's New in Python 2.3 |
| 3 |
**************************** |
| 4 |
|
| 5 |
:Author: A.M. Kuchling |
| 6 |
|
| 7 |
.. |release| replace:: 1.01 |
| 8 |
|
| 9 |
.. $Id: whatsnew23.tex 54631 2007-03-31 11:58:36Z georg.brandl $ |
| 10 |
|
| 11 |
This article explains the new features in Python 2.3. Python 2.3 was released |
| 12 |
on July 29, 2003. |
| 13 |
|
| 14 |
The main themes for Python 2.3 are polishing some of the features added in 2.2, |
| 15 |
adding various small but useful enhancements to the core language, and expanding |
| 16 |
the standard library. The new object model introduced in the previous version |
| 17 |
has benefited from 18 months of bugfixes and from optimization efforts that have |
| 18 |
improved the performance of new-style classes. A few new built-in functions |
| 19 |
have been added such as :func:`sum` and :func:`enumerate`. The :keyword:`in` |
| 20 |
operator can now be used for substring searches (e.g. ``"ab" in "abc"`` returns |
| 21 |
:const:`True`). |
| 22 |
|
| 23 |
Some of the many new library features include Boolean, set, heap, and date/time |
| 24 |
data types, the ability to import modules from ZIP-format archives, metadata |
| 25 |
support for the long-awaited Python catalog, an updated version of IDLE, and |
| 26 |
modules for logging messages, wrapping text, parsing CSV files, processing |
| 27 |
command-line options, using BerkeleyDB databases... the list of new and |
| 28 |
enhanced modules is lengthy. |
| 29 |
|
| 30 |
This article doesn't attempt to provide a complete specification of the new |
| 31 |
features, but instead provides a convenient overview. For full details, you |
| 32 |
should refer to the documentation for Python 2.3, such as the Python Library |
| 33 |
Reference and the Python Reference Manual. If you want to understand the |
| 34 |
complete implementation and design rationale, refer to the PEP for a particular |
| 35 |
new feature. |
| 36 |
|
| 37 |
.. ====================================================================== |
| 38 |
|
| 39 |
|
| 40 |
PEP 218: A Standard Set Datatype |
| 41 |
================================ |
| 42 |
|
| 43 |
The new :mod:`!sets` module contains an implementation of a set datatype. The |
| 44 |
:class:`Set` class is for mutable sets, sets that can have members added and |
| 45 |
removed. The :class:`!ImmutableSet` class is for sets that can't be modified, |
| 46 |
and instances of :class:`!ImmutableSet` can therefore be used as dictionary keys. |
| 47 |
Sets are built on top of dictionaries, so the elements within a set must be |
| 48 |
hashable. |
| 49 |
|
| 50 |
Here's a simple example:: |
| 51 |
|
| 52 |
>>> import sets |
| 53 |
>>> S = sets.Set([1,2,3]) |
| 54 |
>>> S |
| 55 |
Set([1, 2, 3]) |
| 56 |
>>> 1 in S |
| 57 |
True |
| 58 |
>>> 0 in S |
| 59 |
False |
| 60 |
>>> S.add(5) |
| 61 |
>>> S.remove(3) |
| 62 |
>>> S |
| 63 |
Set([1, 2, 5]) |
| 64 |
>>> |
| 65 |
|
| 66 |
The union and intersection of sets can be computed with the :meth:`~frozenset.union` and |
| 67 |
:meth:`~frozenset.intersection` methods; an alternative notation uses the bitwise operators |
| 68 |
``&`` and ``|``. Mutable sets also have in-place versions of these methods, |
| 69 |
:meth:`!union_update` and :meth:`~set.intersection_update`. :: |
| 70 |
|
| 71 |
>>> S1 = sets.Set([1,2,3]) |
| 72 |
>>> S2 = sets.Set([4,5,6]) |
| 73 |
>>> S1.union(S2) |
| 74 |
Set([1, 2, 3, 4, 5, 6]) |
| 75 |
>>> S1 | S2 # Alternative notation |
| 76 |
Set([1, 2, 3, 4, 5, 6]) |
| 77 |
>>> S1.intersection(S2) |
| 78 |
Set([]) |
| 79 |
>>> S1 & S2 # Alternative notation |
| 80 |
Set([]) |
| 81 |
>>> S1.union_update(S2) |
| 82 |
>>> S1 |
| 83 |
Set([1, 2, 3, 4, 5, 6]) |
| 84 |
>>> |
| 85 |
|
| 86 |
It's also possible to take the symmetric difference of two sets. This is the |
| 87 |
set of all elements in the union that aren't in the intersection. Another way |
| 88 |
of putting it is that the symmetric difference contains all elements that are in |
| 89 |
exactly one set. Again, there's an alternative notation (``^``), and an |
| 90 |
in-place version with the ungainly name :meth:`~set.symmetric_difference_update`. :: |
| 91 |
|
| 92 |
>>> S1 = sets.Set([1,2,3,4]) |
| 93 |
>>> S2 = sets.Set([3,4,5,6]) |
| 94 |
>>> S1.symmetric_difference(S2) |
| 95 |
Set([1, 2, 5, 6]) |
| 96 |
>>> S1 ^ S2 |
| 97 |
Set([1, 2, 5, 6]) |
| 98 |
>>> |
| 99 |
|
| 100 |
There are also :meth:`!issubset` and :meth:`!issuperset` methods for checking |
| 101 |
whether one set is a subset or superset of another:: |
| 102 |
|
| 103 |
>>> S1 = sets.Set([1,2,3]) |
| 104 |
>>> S2 = sets.Set([2,3]) |
| 105 |
>>> S2.issubset(S1) |
| 106 |
True |
| 107 |
>>> S1.issubset(S2) |
| 108 |
False |
| 109 |
>>> S1.issuperset(S2) |
| 110 |
True |
| 111 |
>>> |
| 112 |
|
| 113 |
|
| 114 |
.. seealso:: |
| 115 |
|
| 116 |
:pep:`218` - Adding a Built-In Set Object Type |
| 117 |
PEP written by Greg V. Wilson. Implemented by Greg V. Wilson, Alex Martelli, and |
| 118 |
GvR. |
| 119 |
|
| 120 |
.. ====================================================================== |
| 121 |
|
| 122 |
|
| 123 |
.. _section-generators: |
| 124 |
|
| 125 |
PEP 255: Simple Generators |
| 126 |
========================== |
| 127 |
|
| 128 |
In Python 2.2, generators were added as an optional feature, to be enabled by a |
| 129 |
``from __future__ import generators`` directive. In 2.3 generators no longer |
| 130 |
need to be specially enabled, and are now always present; this means that |
| 131 |
:keyword:`yield` is now always a keyword. The rest of this section is a copy of |
| 132 |
the description of generators from the "What's New in Python 2.2" document; if |
| 133 |
you read it back when Python 2.2 came out, you can skip the rest of this |
| 134 |
section. |
| 135 |
|
| 136 |
You're doubtless familiar with how function calls work in Python or C. When you |
| 137 |
call a function, it gets a private namespace where its local variables are |
| 138 |
created. When the function reaches a :keyword:`return` statement, the local |
| 139 |
variables are destroyed and the resulting value is returned to the caller. A |
| 140 |
later call to the same function will get a fresh new set of local variables. |
| 141 |
But, what if the local variables weren't thrown away on exiting a function? |
| 142 |
What if you could later resume the function where it left off? This is what |
| 143 |
generators provide; they can be thought of as resumable functions. |
| 144 |
|
| 145 |
Here's the simplest example of a generator function:: |
| 146 |
|
| 147 |
def generate_ints(N): |
| 148 |
for i in range(N): |
| 149 |
yield i |
| 150 |
|
| 151 |
A new keyword, :keyword:`yield`, was introduced for generators. Any function |
| 152 |
containing a :keyword:`!yield` statement is a generator function; this is |
| 153 |
detected by Python's bytecode compiler which compiles the function specially as |
| 154 |
a result. |
| 155 |
|
| 156 |
When you call a generator function, it doesn't return a single value; instead it |
| 157 |
returns a generator object that supports the iterator protocol. On executing |
| 158 |
the :keyword:`yield` statement, the generator outputs the value of ``i``, |
| 159 |
similar to a :keyword:`return` statement. The big difference between |
| 160 |
:keyword:`!yield` and a :keyword:`!return` statement is that on reaching a |
| 161 |
:keyword:`!yield` the generator's state of execution is suspended and local |
| 162 |
variables are preserved. On the next call to the generator's ``.next()`` |
| 163 |
method, the function will resume executing immediately after the |
| 164 |
:keyword:`!yield` statement. (For complicated reasons, the :keyword:`!yield` |
| 165 |
statement isn't allowed inside the :keyword:`try` block of a |
| 166 |
:keyword:`!try`...\ :keyword:`!finally` statement; read :pep:`255` for a full |
| 167 |
explanation of the interaction between :keyword:`!yield` and exceptions.) |
| 168 |
|
| 169 |
Here's a sample usage of the :func:`!generate_ints` generator:: |
| 170 |
|
| 171 |
>>> gen = generate_ints(3) |
| 172 |
>>> gen |
| 173 |
<generator object at 0x8117f90> |
| 174 |
>>> gen.next() |
| 175 |
0 |
| 176 |
>>> gen.next() |
| 177 |
1 |
| 178 |
>>> gen.next() |
| 179 |
2 |
| 180 |
>>> gen.next() |
| 181 |
Traceback (most recent call last): |
| 182 |
File "stdin", line 1, in ? |
| 183 |
File "stdin", line 2, in generate_ints |
| 184 |
StopIteration |
| 185 |
|
| 186 |
You could equally write ``for i in generate_ints(5)``, or ``a,b,c = |
| 187 |
generate_ints(3)``. |
| 188 |
|
| 189 |
Inside a generator function, the :keyword:`return` statement can only be used |
| 190 |
without a value, and signals the end of the procession of values; afterwards the |
| 191 |
generator cannot return any further values. :keyword:`!return` with a value, such |
| 192 |
as ``return 5``, is a syntax error inside a generator function. The end of the |
| 193 |
generator's results can also be indicated by raising :exc:`StopIteration` |
| 194 |
manually, or by just letting the flow of execution fall off the bottom of the |
| 195 |
function. |
| 196 |
|
| 197 |
You could achieve the effect of generators manually by writing your own class |
| 198 |
and storing all the local variables of the generator as instance variables. For |
| 199 |
example, returning a list of integers could be done by setting ``self.count`` to |
| 200 |
0, and having the :meth:`next` method increment ``self.count`` and return it. |
| 201 |
However, for a moderately complicated generator, writing a corresponding class |
| 202 |
would be much messier. :file:`Lib/test/test_generators.py` contains a number of |
| 203 |
more interesting examples. The simplest one implements an in-order traversal of |
| 204 |
a tree using generators recursively. :: |
| 205 |
|
| 206 |
# A recursive generator that generates Tree leaves in in-order. |
| 207 |
def inorder(t): |
| 208 |
if t: |
| 209 |
for x in inorder(t.left): |
| 210 |
yield x |
| 211 |
yield t.label |
| 212 |
for x in inorder(t.right): |
| 213 |
yield x |
| 214 |
|
| 215 |
Two other examples in :file:`Lib/test/test_generators.py` produce solutions for |
| 216 |
the N-Queens problem (placing $N$ queens on an $NxN$ chess board so that no |
| 217 |
queen threatens another) and the Knight's Tour (a route that takes a knight to |
| 218 |
every square of an $NxN$ chessboard without visiting any square twice). |
| 219 |
|
| 220 |
The idea of generators comes from other programming languages, especially Icon |
| 221 |
(https://www2.cs.arizona.edu/icon/), where the idea of generators is central. In |
| 222 |
Icon, every expression and function call behaves like a generator. One example |
| 223 |
from "An Overview of the Icon Programming Language" at |
| 224 |
https://www2.cs.arizona.edu/icon/docs/ipd266.htm gives an idea of what this looks |
| 225 |
like:: |
| 226 |
|
| 227 |
sentence := "Store it in the neighboring harbor" |
| 228 |
if (i := find("or", sentence)) > 5 then write(i) |
| 229 |
|
| 230 |
In Icon the :func:`!find` function returns the indexes at which the substring |
| 231 |
"or" is found: 3, 23, 33. In the :keyword:`if` statement, ``i`` is first |
| 232 |
assigned a value of 3, but 3 is less than 5, so the comparison fails, and Icon |
| 233 |
retries it with the second value of 23. 23 is greater than 5, so the comparison |
| 234 |
now succeeds, and the code prints the value 23 to the screen. |
| 235 |
|
| 236 |
Python doesn't go nearly as far as Icon in adopting generators as a central |
| 237 |
concept. Generators are considered part of the core Python language, but |
| 238 |
learning or using them isn't compulsory; if they don't solve any problems that |
| 239 |
you have, feel free to ignore them. One novel feature of Python's interface as |
| 240 |
compared to Icon's is that a generator's state is represented as a concrete |
| 241 |
object (the iterator) that can be passed around to other functions or stored in |
| 242 |
a data structure. |
| 243 |
|
| 244 |
|
| 245 |
.. seealso:: |
| 246 |
|
| 247 |
:pep:`255` - Simple Generators |
| 248 |
Written by Neil Schemenauer, Tim Peters, Magnus Lie Hetland. Implemented mostly |
| 249 |
by Neil Schemenauer and Tim Peters, with other fixes from the Python Labs crew. |
| 250 |
|
| 251 |
.. ====================================================================== |
| 252 |
|
| 253 |
|
| 254 |
.. _section-encodings: |
| 255 |
|
| 256 |
PEP 263: Source Code Encodings |
| 257 |
============================== |
| 258 |
|
| 259 |
Python source files can now be declared as being in different character set |
| 260 |
encodings. Encodings are declared by including a specially formatted comment in |
| 261 |
the first or second line of the source file. For example, a UTF-8 file can be |
| 262 |
declared with:: |
| 263 |
|
| 264 |
#!/usr/bin/env python |
| 265 |
# -*- coding: UTF-8 -*- |
| 266 |
|
| 267 |
Without such an encoding declaration, the default encoding used is 7-bit ASCII. |
| 268 |
Executing or importing modules that contain string literals with 8-bit |
| 269 |
characters and have no encoding declaration will result in a |
| 270 |
:exc:`DeprecationWarning` being signalled by Python 2.3; in 2.4 this will be a |
| 271 |
syntax error. |
| 272 |
|
| 273 |
The encoding declaration only affects Unicode string literals, which will be |
| 274 |
converted to Unicode using the specified encoding. Note that Python identifiers |
| 275 |
are still restricted to ASCII characters, so you can't have variable names that |
| 276 |
use characters outside of the usual alphanumerics. |
| 277 |
|
| 278 |
|
| 279 |
.. seealso:: |
| 280 |
|
| 281 |
:pep:`263` - Defining Python Source Code Encodings |
| 282 |
Written by Marc-André Lemburg and Martin von Löwis; implemented by Suzuki Hisao |
| 283 |
and Martin von Löwis. |
| 284 |
|
| 285 |
.. ====================================================================== |
| 286 |
|
| 287 |
|
| 288 |
PEP 273: Importing Modules from ZIP Archives |
| 289 |
============================================ |
| 290 |
|
| 291 |
The new :mod:`zipimport` module adds support for importing modules from a |
| 292 |
ZIP-format archive. You don't need to import the module explicitly; it will be |
| 293 |
automatically imported if a ZIP archive's filename is added to ``sys.path``. |
| 294 |
For example: |
| 295 |
|
| 296 |
.. code-block:: shell-session |
| 297 |
|
| 298 |
amk@nyman:~/src/python$ unzip -l /tmp/example.zip |
| 299 |
Archive: /tmp/example.zip |
| 300 |
Length Date Time Name |
| 301 |
-------- ---- ---- ---- |
| 302 |
8467 11-26-02 22:30 jwzthreading.py |
| 303 |
-------- ------- |
| 304 |
8467 1 file |
| 305 |
amk@nyman:~/src/python$ ./python |
| 306 |
Python 2.3 (#1, Aug 1 2003, 19:54:32) |
| 307 |
>>> import sys |
| 308 |
>>> sys.path.insert(0, '/tmp/example.zip') # Add .zip file to front of path |
| 309 |
>>> import jwzthreading |
| 310 |
>>> jwzthreading.__file__ |
| 311 |
'/tmp/example.zip/jwzthreading.py' |
| 312 |
>>> |
| 313 |
|
| 314 |
An entry in ``sys.path`` can now be the filename of a ZIP archive. The ZIP |
| 315 |
archive can contain any kind of files, but only files named :file:`\*.py`, |
| 316 |
:file:`\*.pyc`, or :file:`\*.pyo` can be imported. If an archive only contains |
| 317 |
:file:`\*.py` files, Python will not attempt to modify the archive by adding the |
| 318 |
corresponding :file:`\*.pyc` file, meaning that if a ZIP archive doesn't contain |
| 319 |
:file:`\*.pyc` files, importing may be rather slow. |
| 320 |
|
| 321 |
A path within the archive can also be specified to only import from a |
| 322 |
subdirectory; for example, the path :file:`/tmp/example.zip/lib/` would only |
| 323 |
import from the :file:`lib/` subdirectory within the archive. |
| 324 |
|
| 325 |
|
| 326 |
.. seealso:: |
| 327 |
|
| 328 |
:pep:`273` - Import Modules from Zip Archives |
| 329 |
Written by James C. Ahlstrom, who also provided an implementation. Python 2.3 |
| 330 |
follows the specification in :pep:`273`, but uses an implementation written by |
| 331 |
Just van Rossum that uses the import hooks described in :pep:`302`. See section |
| 332 |
:ref:`section-pep302` for a description of the new import hooks. |
| 333 |
|
| 334 |
.. ====================================================================== |
| 335 |
|
| 336 |
|
| 337 |
PEP 277: Unicode file name support for Windows NT |
| 338 |
================================================= |
| 339 |
|
| 340 |
On Windows NT, 2000, and XP, the system stores file names as Unicode strings. |
| 341 |
Traditionally, Python has represented file names as byte strings, which is |
| 342 |
inadequate because it renders some file names inaccessible. |
| 343 |
|
| 344 |
Python now allows using arbitrary Unicode strings (within the limitations of the |
| 345 |
file system) for all functions that expect file names, most notably the |
| 346 |
:func:`open` built-in function. If a Unicode string is passed to |
| 347 |
:func:`os.listdir`, Python now returns a list of Unicode strings. A new |
| 348 |
function, :func:`!os.getcwdu`, returns the current directory as a Unicode string. |
| 349 |
|
| 350 |
Byte strings still work as file names, and on Windows Python will transparently |
| 351 |
convert them to Unicode using the ``mbcs`` encoding. |
| 352 |
|
| 353 |
Other systems also allow Unicode strings as file names but convert them to byte |
| 354 |
strings before passing them to the system, which can cause a :exc:`UnicodeError` |
| 355 |
to be raised. Applications can test whether arbitrary Unicode strings are |
| 356 |
supported as file names by checking :const:`os.path.supports_unicode_filenames`, |
| 357 |
a Boolean value. |
| 358 |
|
| 359 |
Under MacOS, :func:`os.listdir` may now return Unicode filenames. |
| 360 |
|
| 361 |
|
| 362 |
.. seealso:: |
| 363 |
|
| 364 |
:pep:`277` - Unicode file name support for Windows NT |
| 365 |
Written by Neil Hodgson; implemented by Neil Hodgson, Martin von Löwis, and Mark |
| 366 |
Hammond. |
| 367 |
|
| 368 |
.. ====================================================================== |
| 369 |
|
| 370 |
|
| 371 |
.. index:: |
| 372 |
single: universal newlines; What's new |
| 373 |
|
| 374 |
PEP 278: Universal Newline Support |
| 375 |
================================== |
| 376 |
|
| 377 |
The three major operating systems used today are Microsoft Windows, Apple's |
| 378 |
Macintosh OS, and the various Unix derivatives. A minor irritation of |
| 379 |
cross-platform work is that these three platforms all use different characters to |
| 380 |
mark the ends of lines in text files. Unix uses the linefeed (ASCII character |
| 381 |
10), MacOS uses the carriage return (ASCII character 13), and Windows uses a |
| 382 |
two-character sequence of a carriage return plus a newline. |
| 383 |
|
| 384 |
Python's file objects can now support end of line conventions other than the |
| 385 |
one followed by the platform on which Python is running. Opening a file with |
| 386 |
the mode ``'U'`` or ``'rU'`` will open a file for reading in :term:`universal |
| 387 |
newlines` mode. All three line ending conventions will be translated to a |
| 388 |
``'\n'`` in the strings returned by the various file methods such as |
| 389 |
:meth:`!read` and :meth:`!readline`. |
| 390 |
|
| 391 |
Universal newline support is also used when importing modules and when executing |
| 392 |
a file with the :func:`!execfile` function. This means that Python modules can |
| 393 |
be shared between all three operating systems without needing to convert the |
| 394 |
line-endings. |
| 395 |
|
| 396 |
This feature can be disabled when compiling Python by specifying the |
| 397 |
:option:`!--without-universal-newlines` switch when running Python's |
| 398 |
:program:`configure` script. |
| 399 |
|
| 400 |
|
| 401 |
.. seealso:: |
| 402 |
|
| 403 |
:pep:`278` - Universal Newline Support |
| 404 |
Written and implemented by Jack Jansen. |
| 405 |
|
| 406 |
.. ====================================================================== |
| 407 |
|
| 408 |
|
| 409 |
.. _section-enumerate: |
| 410 |
|
| 411 |
PEP 279: enumerate() |
| 412 |
==================== |
| 413 |
|
| 414 |
A new built-in function, :func:`enumerate`, will make certain loops a bit |
| 415 |
clearer. ``enumerate(thing)``, where *thing* is either an iterator or a |
| 416 |
sequence, returns an iterator that will return ``(0, thing[0])``, ``(1, |
| 417 |
thing[1])``, ``(2, thing[2])``, and so forth. |
| 418 |
|
| 419 |
A common idiom to change every element of a list looks like this:: |
| 420 |
|
| 421 |
for i in range(len(L)): |
| 422 |
item = L[i] |
| 423 |
# ... compute some result based on item ... |
| 424 |
L[i] = result |
| 425 |
|
| 426 |
This can be rewritten using :func:`enumerate` as:: |
| 427 |
|
| 428 |
for i, item in enumerate(L): |
| 429 |
# ... compute some result based on item ... |
| 430 |
L[i] = result |
| 431 |
|
| 432 |
|
| 433 |
.. seealso:: |
| 434 |
|
| 435 |
:pep:`279` - The enumerate() built-in function |
| 436 |
Written and implemented by Raymond D. Hettinger. |
| 437 |
|
| 438 |
.. ====================================================================== |
| 439 |
|
| 440 |
|
| 441 |
PEP 282: The logging Package |
| 442 |
============================ |
| 443 |
|
| 444 |
A standard package for writing logs, :mod:`logging`, has been added to Python |
| 445 |
2.3. It provides a powerful and flexible mechanism for generating logging |
| 446 |
output which can then be filtered and processed in various ways. A |
| 447 |
configuration file written in a standard format can be used to control the |
| 448 |
logging behavior of a program. Python includes handlers that will write log |
| 449 |
records to standard error or to a file or socket, send them to the system log, |
| 450 |
or even e-mail them to a particular address; of course, it's also possible to |
| 451 |
write your own handler classes. |
| 452 |
|
| 453 |
The :class:`~logging.Logger` class is the primary class. Most application code will deal |
| 454 |
with one or more :class:`~logging.Logger` objects, each one used by a particular |
| 455 |
subsystem of the application. Each :class:`~logging.Logger` is identified by a name, and |
| 456 |
names are organized into a hierarchy using ``.`` as the component separator. |
| 457 |
For example, you might have :class:`~logging.Logger` instances named ``server``, |
| 458 |
``server.auth`` and ``server.network``. The latter two instances are below |
| 459 |
``server`` in the hierarchy. This means that if you turn up the verbosity for |
| 460 |
``server`` or direct ``server`` messages to a different handler, the changes |
| 461 |
will also apply to records logged to ``server.auth`` and ``server.network``. |
| 462 |
There's also a root :class:`~logging.Logger` that's the parent of all other loggers. |
| 463 |
|
| 464 |
For simple uses, the :mod:`logging` package contains some convenience functions |
| 465 |
that always use the root log:: |
| 466 |
|
| 467 |
import logging |
| 468 |
|
| 469 |
logging.debug('Debugging information') |
| 470 |
logging.info('Informational message') |
| 471 |
logging.warning('Warning:config file %s not found', 'server.conf') |
| 472 |
logging.error('Error occurred') |
| 473 |
logging.critical('Critical error -- shutting down') |
| 474 |
|
| 475 |
This produces the following output:: |
| 476 |
|
| 477 |
WARNING:root:Warning:config file server.conf not found |
| 478 |
ERROR:root:Error occurred |
| 479 |
CRITICAL:root:Critical error -- shutting down |
| 480 |
|
| 481 |
In the default configuration, informational and debugging messages are |
| 482 |
suppressed and the output is sent to standard error. You can enable the display |
| 483 |
of informational and debugging messages by calling the :meth:`~logging.Logger.setLevel` method |
| 484 |
on the root logger. |
| 485 |
|
| 486 |
Notice the :func:`~logging.warning` call's use of string formatting operators; all of the |
| 487 |
functions for logging messages take the arguments ``(msg, arg1, arg2, ...)`` and |
| 488 |
log the string resulting from ``msg % (arg1, arg2, ...)``. |
| 489 |
|
| 490 |
There's also an :func:`~logging.exception` function that records the most recent |
| 491 |
traceback. Any of the other functions will also record the traceback if you |
| 492 |
specify a true value for the keyword argument *exc_info*. :: |
| 493 |
|
| 494 |
def f(): |
| 495 |
try: 1/0 |
| 496 |
except: logging.exception('Problem recorded') |
| 497 |
|
| 498 |
f() |
| 499 |
|
| 500 |
This produces the following output:: |
| 501 |
|
| 502 |
ERROR:root:Problem recorded |
| 503 |
Traceback (most recent call last): |
| 504 |
File "t.py", line 6, in f |
| 505 |
1/0 |
| 506 |
ZeroDivisionError: integer division or modulo by zero |
| 507 |
|
| 508 |
Slightly more advanced programs will use a logger other than the root logger. |
| 509 |
The ``getLogger(name)`` function is used to get a particular log, creating |
| 510 |
it if it doesn't exist yet. ``getLogger(None)`` returns the root logger. :: |
| 511 |
|
| 512 |
log = logging.getLogger('server') |
| 513 |
... |
| 514 |
log.info('Listening on port %i', port) |
| 515 |
... |
| 516 |
log.critical('Disk full') |
| 517 |
... |
| 518 |
|
| 519 |
Log records are usually propagated up the hierarchy, so a message logged to |
| 520 |
``server.auth`` is also seen by ``server`` and ``root``, but a :class:`~logging.Logger` |
| 521 |
can prevent this by setting its :attr:`~logging.Logger.propagate` attribute to :const:`False`. |
| 522 |
|
| 523 |
There are more classes provided by the :mod:`logging` package that can be |
| 524 |
customized. When a :class:`~logging.Logger` instance is told to log a message, it |
| 525 |
creates a :class:`~logging.LogRecord` instance that is sent to any number of different |
| 526 |
:class:`~logging.Handler` instances. Loggers and handlers can also have an attached list |
| 527 |
of filters, and each filter can cause the :class:`~logging.LogRecord` to be ignored or |
| 528 |
can modify the record before passing it along. When they're finally output, |
| 529 |
:class:`~logging.LogRecord` instances are converted to text by a :class:`~logging.Formatter` |
| 530 |
class. All of these classes can be replaced by your own specially written |
| 531 |
classes. |
| 532 |
|
| 533 |
With all of these features the :mod:`logging` package should provide enough |
| 534 |
flexibility for even the most complicated applications. This is only an |
| 535 |
incomplete overview of its features, so please see the package's reference |
| 536 |
documentation for all of the details. Reading :pep:`282` will also be helpful. |
| 537 |
|
| 538 |
|
| 539 |
.. seealso:: |
| 540 |
|
| 541 |
:pep:`282` - A Logging System |
| 542 |
Written by Vinay Sajip and Trent Mick; implemented by Vinay Sajip. |
| 543 |
|
| 544 |
.. ====================================================================== |
| 545 |
|
| 546 |
|
| 547 |
.. _section-bool: |
| 548 |
|
| 549 |
PEP 285: A Boolean Type |
| 550 |
======================= |
| 551 |
|
| 552 |
A Boolean type was added to Python 2.3. Two new constants were added to the |
| 553 |
:mod:`!__builtin__` module, :const:`True` and :const:`False`. (:const:`True` and |
| 554 |
:const:`False` constants were added to the built-ins in Python 2.2.1, but the |
| 555 |
2.2.1 versions are simply set to integer values of 1 and 0 and aren't a |
| 556 |
different type.) |
| 557 |
|
| 558 |
The type object for this new type is named :class:`bool`; the constructor for it |
| 559 |
takes any Python value and converts it to :const:`True` or :const:`False`. :: |
| 560 |
|
| 561 |
>>> bool(1) |
| 562 |
True |
| 563 |
>>> bool(0) |
| 564 |
False |
| 565 |
>>> bool([]) |
| 566 |
False |
| 567 |
>>> bool( (1,) ) |
| 568 |
True |
| 569 |
|
| 570 |
Most of the standard library modules and built-in functions have been changed to |
| 571 |
return Booleans. :: |
| 572 |
|
| 573 |
>>> obj = [] |
| 574 |
>>> hasattr(obj, 'append') |
| 575 |
True |
| 576 |
>>> isinstance(obj, list) |
| 577 |
True |
| 578 |
>>> isinstance(obj, tuple) |
| 579 |
False |
| 580 |
|
| 581 |
Python's Booleans were added with the primary goal of making code clearer. For |
| 582 |
example, if you're reading a function and encounter the statement ``return 1``, |
| 583 |
you might wonder whether the ``1`` represents a Boolean truth value, an index, |
| 584 |
or a coefficient that multiplies some other quantity. If the statement is |
| 585 |
``return True``, however, the meaning of the return value is quite clear. |
| 586 |
|
| 587 |
Python's Booleans were *not* added for the sake of strict type-checking. A very |
| 588 |
strict language such as Pascal would also prevent you performing arithmetic with |
| 589 |
Booleans, and would require that the expression in an :keyword:`if` statement |
| 590 |
always evaluate to a Boolean result. Python is not this strict and never will |
| 591 |
be, as :pep:`285` explicitly says. This means you can still use any expression |
| 592 |
in an :keyword:`!if` statement, even ones that evaluate to a list or tuple or |
| 593 |
some random object. The Boolean type is a subclass of the :class:`int` class so |
| 594 |
that arithmetic using a Boolean still works. :: |
| 595 |
|
| 596 |
>>> True + 1 |
| 597 |
2 |
| 598 |
>>> False + 1 |
| 599 |
1 |
| 600 |
>>> False * 75 |
| 601 |
0 |
| 602 |
>>> True * 75 |
| 603 |
75 |
| 604 |
|
| 605 |
To sum up :const:`True` and :const:`False` in a sentence: they're alternative |
| 606 |
ways to spell the integer values 1 and 0, with the single difference that |
| 607 |
:func:`str` and :func:`repr` return the strings ``'True'`` and ``'False'`` |
| 608 |
instead of ``'1'`` and ``'0'``. |
| 609 |
|
| 610 |
|
| 611 |
.. seealso:: |
| 612 |
|
| 613 |
:pep:`285` - Adding a bool type |
| 614 |
Written and implemented by GvR. |
| 615 |
|
| 616 |
.. ====================================================================== |
| 617 |
|
| 618 |
|
| 619 |
PEP 293: Codec Error Handling Callbacks |
| 620 |
======================================= |
| 621 |
|
| 622 |
When encoding a Unicode string into a byte string, unencodable characters may be |
| 623 |
encountered. So far, Python has allowed specifying the error processing as |
| 624 |
either "strict" (raising :exc:`UnicodeError`), "ignore" (skipping the |
| 625 |
character), or "replace" (using a question mark in the output string), with |
| 626 |
"strict" being the default behavior. It may be desirable to specify alternative |
| 627 |
processing of such errors, such as inserting an XML character reference or HTML |
| 628 |
entity reference into the converted string. |
| 629 |
|
| 630 |
Python now has a flexible framework to add different processing strategies. New |
| 631 |
error handlers can be added with :func:`codecs.register_error`, and codecs then |
| 632 |
can access the error handler with :func:`codecs.lookup_error`. An equivalent C |
| 633 |
API has been added for codecs written in C. The error handler gets the necessary |
| 634 |
state information such as the string being converted, the position in the string |
| 635 |
where the error was detected, and the target encoding. The handler can then |
| 636 |
either raise an exception or return a replacement string. |
| 637 |
|
| 638 |
Two additional error handlers have been implemented using this framework: |
| 639 |
"backslashreplace" uses Python backslash quoting to represent unencodable |
| 640 |
characters and "xmlcharrefreplace" emits XML character references. |
| 641 |
|
| 642 |
|
| 643 |
.. seealso:: |
| 644 |
|
| 645 |
:pep:`293` - Codec Error Handling Callbacks |
| 646 |
Written and implemented by Walter Dörwald. |
| 647 |
|
| 648 |
.. ====================================================================== |
| 649 |
|
| 650 |
|
| 651 |
.. _section-pep301: |
| 652 |
|
| 653 |
PEP 301: Package Index and Metadata for Distutils |
| 654 |
================================================= |
| 655 |
|
| 656 |
Support for the long-requested Python catalog makes its first appearance in 2.3. |
| 657 |
|
| 658 |
The heart of the catalog is the new Distutils :command:`register` command. |
| 659 |
Running ``python setup.py register`` will collect the metadata describing a |
| 660 |
package, such as its name, version, maintainer, description, &c., and send it to |
| 661 |
a central catalog server. The resulting catalog is available from |
| 662 |
https://pypi.org. |
| 663 |
|
| 664 |
To make the catalog a bit more useful, a new optional *classifiers* keyword |
| 665 |
argument has been added to the Distutils :func:`!setup` function. A list of |
| 666 |
`Trove <http://catb.org/~esr/trove/>`_-style strings can be supplied to help |
| 667 |
classify the software. |
| 668 |
|
| 669 |
Here's an example :file:`setup.py` with classifiers, written to be compatible |
| 670 |
with older versions of the Distutils:: |
| 671 |
|
| 672 |
from distutils import core |
| 673 |
kw = {'name': "Quixote", |
| 674 |
'version': "0.5.1", |
| 675 |
'description': "A highly Pythonic Web application framework", |
| 676 |
# ... |
| 677 |
} |
| 678 |
|
| 679 |
if (hasattr(core, 'setup_keywords') and |
| 680 |
'classifiers' in core.setup_keywords): |
| 681 |
kw['classifiers'] = \ |
| 682 |
['Topic :: Internet :: WWW/HTTP :: Dynamic Content', |
| 683 |
'Environment :: No Input/Output (Daemon)', |
| 684 |
'Intended Audience :: Developers'], |
| 685 |
|
| 686 |
core.setup(**kw) |
| 687 |
|
| 688 |
The full list of classifiers can be obtained by running ``python setup.py |
| 689 |
register --list-classifiers``. |
| 690 |
|
| 691 |
|
| 692 |
.. seealso:: |
| 693 |
|
| 694 |
:pep:`301` - Package Index and Metadata for Distutils |
| 695 |
Written and implemented by Richard Jones. |
| 696 |
|
| 697 |
.. ====================================================================== |
| 698 |
|
| 699 |
|
| 700 |
.. _section-pep302: |
| 701 |
|
| 702 |
PEP 302: New Import Hooks |
| 703 |
========================= |
| 704 |
|
| 705 |
While it's been possible to write custom import hooks ever since the |
| 706 |
:mod:`!ihooks` module was introduced in Python 1.3, no one has ever been really |
| 707 |
happy with it because writing new import hooks is difficult and messy. There |
| 708 |
have been various proposed alternatives such as the :mod:`!imputil` and :mod:`!iu` |
| 709 |
modules, but none of them has ever gained much acceptance, and none of them were |
| 710 |
easily usable from C code. |
| 711 |
|
| 712 |
:pep:`302` borrows ideas from its predecessors, especially from Gordon |
| 713 |
McMillan's :mod:`!iu` module. Three new items are added to the :mod:`sys` |
| 714 |
module: |
| 715 |
|
| 716 |
* ``sys.path_hooks`` is a list of callable objects; most often they'll be |
| 717 |
classes. Each callable takes a string containing a path and either returns an |
| 718 |
importer object that will handle imports from this path or raises an |
| 719 |
:exc:`ImportError` exception if it can't handle this path. |
| 720 |
|
| 721 |
* ``sys.path_importer_cache`` caches importer objects for each path, so |
| 722 |
``sys.path_hooks`` will only need to be traversed once for each path. |
| 723 |
|
| 724 |
* ``sys.meta_path`` is a list of importer objects that will be traversed before |
| 725 |
``sys.path`` is checked. This list is initially empty, but user code can add |
| 726 |
objects to it. Additional built-in and frozen modules can be imported by an |
| 727 |
object added to this list. |
| 728 |
|
| 729 |
Importer objects must have a single method, ``find_module(fullname, |
| 730 |
path=None)``. *fullname* will be a module or package name, e.g. ``string`` or |
| 731 |
``distutils.core``. :meth:`!find_module` must return a loader object that has a |
| 732 |
single method, ``load_module(fullname)``, that creates and returns the |
| 733 |
corresponding module object. |
| 734 |
|
| 735 |
Pseudo-code for Python's new import logic, therefore, looks something like this |
| 736 |
(simplified a bit; see :pep:`302` for the full details):: |
| 737 |
|
| 738 |
for mp in sys.meta_path: |
| 739 |
loader = mp(fullname) |
| 740 |
if loader is not None: |
| 741 |
<module> = loader.load_module(fullname) |
| 742 |
|
| 743 |
for path in sys.path: |
| 744 |
for hook in sys.path_hooks: |
| 745 |
try: |
| 746 |
importer = hook(path) |
| 747 |
except ImportError: |
| 748 |
# ImportError, so try the other path hooks |
| 749 |
pass |
| 750 |
else: |
| 751 |
loader = importer.find_module(fullname) |
| 752 |
<module> = loader.load_module(fullname) |
| 753 |
|
| 754 |
# Not found! |
| 755 |
raise ImportError |
| 756 |
|
| 757 |
|
| 758 |
.. seealso:: |
| 759 |
|
| 760 |
:pep:`302` - New Import Hooks |
| 761 |
Written by Just van Rossum and Paul Moore. Implemented by Just van Rossum. |
| 762 |
|
| 763 |
.. ====================================================================== |
| 764 |
|
| 765 |
|
| 766 |
.. _section-pep305: |
| 767 |
|
| 768 |
PEP 305: Comma-separated Files |
| 769 |
============================== |
| 770 |
|
| 771 |
Comma-separated files are a format frequently used for exporting data from |
| 772 |
databases and spreadsheets. Python 2.3 adds a parser for comma-separated files. |
| 773 |
|
| 774 |
Comma-separated format is deceptively simple at first glance:: |
| 775 |
|
| 776 |
Costs,150,200,3.95 |
| 777 |
|
| 778 |
Read a line and call ``line.split(',')``: what could be simpler? But toss in |
| 779 |
string data that can contain commas, and things get more complicated:: |
| 780 |
|
| 781 |
"Costs",150,200,3.95,"Includes taxes, shipping, and sundry items" |
| 782 |
|
| 783 |
A big ugly regular expression can parse this, but using the new :mod:`csv` |
| 784 |
package is much simpler:: |
| 785 |
|
| 786 |
import csv |
| 787 |
|
| 788 |
input = open('datafile', 'rb') |
| 789 |
reader = csv.reader(input) |
| 790 |
for line in reader: |
| 791 |
print line |
| 792 |
|
| 793 |
The :func:`~csv.reader` function takes a number of different options. The field |
| 794 |
separator isn't limited to the comma and can be changed to any character, and so |
| 795 |
can the quoting and line-ending characters. |
| 796 |
|
| 797 |
Different dialects of comma-separated files can be defined and registered; |
| 798 |
currently there are two dialects, both used by Microsoft Excel. A separate |
| 799 |
:class:`csv.writer` class will generate comma-separated files from a succession |
| 800 |
of tuples or lists, quoting strings that contain the delimiter. |
| 801 |
|
| 802 |
|
| 803 |
.. seealso:: |
| 804 |
|
| 805 |
:pep:`305` - CSV File API |
| 806 |
Written and implemented by Kevin Altis, Dave Cole, Andrew McNamara, Skip |
| 807 |
Montanaro, Cliff Wells. |
| 808 |
|
| 809 |
.. ====================================================================== |
| 810 |
|
| 811 |
|
| 812 |
.. _section-pep307: |
| 813 |
|
| 814 |
PEP 307: Pickle Enhancements |
| 815 |
============================ |
| 816 |
|
| 817 |
The :mod:`pickle` and :mod:`!cPickle` modules received some attention during the |
| 818 |
2.3 development cycle. In 2.2, new-style classes could be pickled without |
| 819 |
difficulty, but they weren't pickled very compactly; :pep:`307` quotes a trivial |
| 820 |
example where a new-style class results in a pickled string three times longer |
| 821 |
than that for a classic class. |
| 822 |
|
| 823 |
The solution was to invent a new pickle protocol. The :func:`pickle.dumps` |
| 824 |
function has supported a text-or-binary flag for a long time. In 2.3, this |
| 825 |
flag is redefined from a Boolean to an integer: 0 is the old text-mode pickle |
| 826 |
format, 1 is the old binary format, and now 2 is a new 2.3-specific format. A |
| 827 |
new constant, :const:`pickle.HIGHEST_PROTOCOL`, can be used to select the |
| 828 |
fanciest protocol available. |
| 829 |
|
| 830 |
Unpickling is no longer considered a safe operation. 2.2's :mod:`pickle` |
| 831 |
provided hooks for trying to prevent unsafe classes from being unpickled |
| 832 |
(specifically, a :attr:`!__safe_for_unpickling__` attribute), but none of this |
| 833 |
code was ever audited and therefore it's all been ripped out in 2.3. You should |
| 834 |
not unpickle untrusted data in any version of Python. |
| 835 |
|
| 836 |
To reduce the pickling overhead for new-style classes, a new interface for |
| 837 |
customizing pickling was added using three special methods: |
| 838 |
:meth:`~object.__getstate__`, :meth:`~object.__setstate__`, and :meth:`~object.__getnewargs__`. Consult |
| 839 |
:pep:`307` for the full semantics of these methods. |
| 840 |
|
| 841 |
As a way to compress pickles yet further, it's now possible to use integer codes |
| 842 |
instead of long strings to identify pickled classes. The Python Software |
| 843 |
Foundation will maintain a list of standardized codes; there's also a range of |
| 844 |
codes for private use. Currently no codes have been specified. |
| 845 |
|
| 846 |
|
| 847 |
.. seealso:: |
| 848 |
|
| 849 |
:pep:`307` - Extensions to the pickle protocol |
| 850 |
Written and implemented by Guido van Rossum and Tim Peters. |
| 851 |
|
| 852 |
.. ====================================================================== |
| 853 |
|
| 854 |
|
| 855 |
.. _section-slices: |
| 856 |
|
| 857 |
Extended Slices |
| 858 |
=============== |
| 859 |
|
| 860 |
Ever since Python 1.4, the slicing syntax has supported an optional third "step" |
| 861 |
or "stride" argument. For example, these are all legal Python syntax: |
| 862 |
``L[1:10:2]``, ``L[:-1:1]``, ``L[::-1]``. This was added to Python at the |
| 863 |
request of the developers of Numerical Python, which uses the third argument |
| 864 |
extensively. However, Python's built-in list, tuple, and string sequence types |
| 865 |
have never supported this feature, raising a :exc:`TypeError` if you tried it. |
| 866 |
Michael Hudson contributed a patch to fix this shortcoming. |
| 867 |
|
| 868 |
For example, you can now easily extract the elements of a list that have even |
| 869 |
indexes:: |
| 870 |
|
| 871 |
>>> L = range(10) |
| 872 |
>>> L[::2] |
| 873 |
[0, 2, 4, 6, 8] |
| 874 |
|
| 875 |
Negative values also work to make a copy of the same list in reverse order:: |
| 876 |
|
| 877 |
>>> L[::-1] |
| 878 |
[9, 8, 7, 6, 5, 4, 3, 2, 1, 0] |
| 879 |
|
| 880 |
This also works for tuples, arrays, and strings:: |
| 881 |
|
| 882 |
>>> s='abcd' |
| 883 |
>>> s[::2] |
| 884 |
'ac' |
| 885 |
>>> s[::-1] |
| 886 |
'dcba' |
| 887 |
|
| 888 |
If you have a mutable sequence such as a list or an array you can assign to or |
| 889 |
delete an extended slice, but there are some differences between assignment to |
| 890 |
extended and regular slices. Assignment to a regular slice can be used to |
| 891 |
change the length of the sequence:: |
| 892 |
|
| 893 |
>>> a = range(3) |
| 894 |
>>> a |
| 895 |
[0, 1, 2] |
| 896 |
>>> a[1:3] = [4, 5, 6] |
| 897 |
>>> a |
| 898 |
[0, 4, 5, 6] |
| 899 |
|
| 900 |
Extended slices aren't this flexible. When assigning to an extended slice, the |
| 901 |
list on the right hand side of the statement must contain the same number of |
| 902 |
items as the slice it is replacing:: |
| 903 |
|
| 904 |
>>> a = range(4) |
| 905 |
>>> a |
| 906 |
[0, 1, 2, 3] |
| 907 |
>>> a[::2] |
| 908 |
[0, 2] |
| 909 |
>>> a[::2] = [0, -1] |
| 910 |
>>> a |
| 911 |
[0, 1, -1, 3] |
| 912 |
>>> a[::2] = [0,1,2] |
| 913 |
Traceback (most recent call last): |
| 914 |
File "<stdin>", line 1, in ? |
| 915 |
ValueError: attempt to assign sequence of size 3 to extended slice of size 2 |
| 916 |
|
| 917 |
Deletion is more straightforward:: |
| 918 |
|
| 919 |
>>> a = range(4) |
| 920 |
>>> a |
| 921 |
[0, 1, 2, 3] |
| 922 |
>>> a[::2] |
| 923 |
[0, 2] |
| 924 |
>>> del a[::2] |
| 925 |
>>> a |
| 926 |
[1, 3] |
| 927 |
|
| 928 |
One can also now pass slice objects to the :meth:`~object.__getitem__` methods of the |
| 929 |
built-in sequences:: |
| 930 |
|
| 931 |
>>> range(10).__getitem__(slice(0, 5, 2)) |
| 932 |
[0, 2, 4] |
| 933 |
|
| 934 |
Or use slice objects directly in subscripts:: |
| 935 |
|
| 936 |
>>> range(10)[slice(0, 5, 2)] |
| 937 |
[0, 2, 4] |
| 938 |
|
| 939 |
To simplify implementing sequences that support extended slicing, slice objects |
| 940 |
now have a method ``indices(length)`` which, given the length of a sequence, |
| 941 |
returns a ``(start, stop, step)`` tuple that can be passed directly to |
| 942 |
:func:`range`. :meth:`!indices` handles omitted and out-of-bounds indices in a |
| 943 |
manner consistent with regular slices (and this innocuous phrase hides a welter |
| 944 |
of confusing details!). The method is intended to be used like this:: |
| 945 |
|
| 946 |
class FakeSeq: |
| 947 |
... |
| 948 |
def calc_item(self, i): |
| 949 |
... |
| 950 |
def __getitem__(self, item): |
| 951 |
if isinstance(item, slice): |
| 952 |
indices = item.indices(len(self)) |
| 953 |
return FakeSeq([self.calc_item(i) for i in range(*indices)]) |
| 954 |
else: |
| 955 |
return self.calc_item(i) |
| 956 |
|
| 957 |
From this example you can also see that the built-in :class:`slice` object is |
| 958 |
now the type object for the slice type, and is no longer a function. This is |
| 959 |
consistent with Python 2.2, where :class:`int`, :class:`str`, etc., underwent |
| 960 |
the same change. |
| 961 |
|
| 962 |
.. ====================================================================== |
| 963 |
|
| 964 |
|
| 965 |
Other Language Changes |
| 966 |
====================== |
| 967 |
|
| 968 |
Here are all of the changes that Python 2.3 makes to the core Python language. |
| 969 |
|
| 970 |
* The :keyword:`yield` statement is now always a keyword, as described in |
| 971 |
section :ref:`section-generators` of this document. |
| 972 |
|
| 973 |
* A new built-in function :func:`enumerate` was added, as described in section |
| 974 |
:ref:`section-enumerate` of this document. |
| 975 |
|
| 976 |
* Two new constants, :const:`True` and :const:`False` were added along with the |
| 977 |
built-in :class:`bool` type, as described in section :ref:`section-bool` of this |
| 978 |
document. |
| 979 |
|
| 980 |
* The :func:`int` type constructor will now return a long integer instead of |
| 981 |
raising an :exc:`OverflowError` when a string or floating-point number is too |
| 982 |
large to fit into an integer. This can lead to the paradoxical result that |
| 983 |
``isinstance(int(expression), int)`` is false, but that seems unlikely to cause |
| 984 |
problems in practice. |
| 985 |
|
| 986 |
* Built-in types now support the extended slicing syntax, as described in |
| 987 |
section :ref:`section-slices` of this document. |
| 988 |
|
| 989 |
* A new built-in function, ``sum(iterable, start=0)``, adds up the numeric |
| 990 |
items in the iterable object and returns their sum. :func:`sum` only accepts |
| 991 |
numbers, meaning that you can't use it to concatenate a bunch of strings. |
| 992 |
(Contributed by Alex Martelli.) |
| 993 |
|
| 994 |
* ``list.insert(pos, value)`` used to insert *value* at the front of the list |
| 995 |
when *pos* was negative. The behaviour has now been changed to be consistent |
| 996 |
with slice indexing, so when *pos* is -1 the value will be inserted before the |
| 997 |
last element, and so forth. |
| 998 |
|
| 999 |
* ``list.index(value)``, which searches for *value* within the list and returns |
| 1000 |
its index, now takes optional *start* and *stop* arguments to limit the search |
| 1001 |
to only part of the list. |
| 1002 |
|
| 1003 |
* Dictionaries have a new method, ``pop(key[, *default*])``, that returns |
| 1004 |
the value corresponding to *key* and removes that key/value pair from the |
| 1005 |
dictionary. If the requested key isn't present in the dictionary, *default* is |
| 1006 |
returned if it's specified and :exc:`KeyError` raised if it isn't. :: |
| 1007 |
|
| 1008 |
>>> d = {1:2} |
| 1009 |
>>> d |
| 1010 |
{1: 2} |
| 1011 |
>>> d.pop(4) |
| 1012 |
Traceback (most recent call last): |
| 1013 |
File "stdin", line 1, in ? |
| 1014 |
KeyError: 4 |
| 1015 |
>>> d.pop(1) |
| 1016 |
2 |
| 1017 |
>>> d.pop(1) |
| 1018 |
Traceback (most recent call last): |
| 1019 |
File "stdin", line 1, in ? |
| 1020 |
KeyError: 'pop(): dictionary is empty' |
| 1021 |
>>> d |
| 1022 |
{} |
| 1023 |
>>> |
| 1024 |
|
| 1025 |
There's also a new class method, ``dict.fromkeys(iterable, value)``, that |
| 1026 |
creates a dictionary with keys taken from the supplied iterator *iterable* and |
| 1027 |
all values set to *value*, defaulting to ``None``. |
| 1028 |
|
| 1029 |
(Patches contributed by Raymond Hettinger.) |
| 1030 |
|
| 1031 |
Also, the :func:`dict` constructor now accepts keyword arguments to simplify |
| 1032 |
creating small dictionaries:: |
| 1033 |
|
| 1034 |
>>> dict(red=1, blue=2, green=3, black=4) |
| 1035 |
{'blue': 2, 'black': 4, 'green': 3, 'red': 1} |
| 1036 |
|
| 1037 |
(Contributed by Just van Rossum.) |
| 1038 |
|
| 1039 |
* The :keyword:`assert` statement no longer checks the ``__debug__`` flag, so |
| 1040 |
you can no longer disable assertions by assigning to ``__debug__``. Running |
| 1041 |
Python with the :option:`-O` switch will still generate code that doesn't |
| 1042 |
execute any assertions. |
| 1043 |
|
| 1044 |
* Most type objects are now callable, so you can use them to create new objects |
| 1045 |
such as functions, classes, and modules. (This means that the :mod:`!new` module |
| 1046 |
can be deprecated in a future Python version, because you can now use the type |
| 1047 |
objects available in the :mod:`types` module.) For example, you can create a new |
| 1048 |
module object with the following code: |
| 1049 |
|
| 1050 |
:: |
| 1051 |
|
| 1052 |
>>> import types |
| 1053 |
>>> m = types.ModuleType('abc','docstring') |
| 1054 |
>>> m |
| 1055 |
<module 'abc' (built-in)> |
| 1056 |
>>> m.__doc__ |
| 1057 |
'docstring' |
| 1058 |
|
| 1059 |
* A new warning, :exc:`PendingDeprecationWarning` was added to indicate features |
| 1060 |
which are in the process of being deprecated. The warning will *not* be printed |
| 1061 |
by default. To check for use of features that will be deprecated in the future, |
| 1062 |
supply :option:`-Walways::PendingDeprecationWarning:: <-W>` on the command line or |
| 1063 |
use :func:`warnings.filterwarnings`. |
| 1064 |
|
| 1065 |
* The process of deprecating string-based exceptions, as in ``raise "Error |
| 1066 |
occurred"``, has begun. Raising a string will now trigger |
| 1067 |
:exc:`PendingDeprecationWarning`. |
| 1068 |
|
| 1069 |
* Using ``None`` as a variable name will now result in a :exc:`SyntaxWarning` |
| 1070 |
warning. In a future version of Python, ``None`` may finally become a keyword. |
| 1071 |
|
| 1072 |
* The :meth:`!xreadlines` method of file objects, introduced in Python 2.1, is no |
| 1073 |
longer necessary because files now behave as their own iterator. |
| 1074 |
:meth:`!xreadlines` was originally introduced as a faster way to loop over all |
| 1075 |
the lines in a file, but now you can simply write ``for line in file_obj``. |
| 1076 |
File objects also have a new read-only :attr:`!encoding` attribute that gives the |
| 1077 |
encoding used by the file; Unicode strings written to the file will be |
| 1078 |
automatically converted to bytes using the given encoding. |
| 1079 |
|
| 1080 |
* The method resolution order used by new-style classes has changed, though |
| 1081 |
you'll only notice the difference if you have a really complicated inheritance |
| 1082 |
hierarchy. Classic classes are unaffected by this change. Python 2.2 |
| 1083 |
originally used a topological sort of a class's ancestors, but 2.3 now uses the |
| 1084 |
C3 algorithm as described in the paper `"A Monotonic Superclass Linearization |
| 1085 |
for Dylan" <https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.19.3910>`_. To |
| 1086 |
understand the motivation for this change, read Michele Simionato's article |
| 1087 |
:ref:`python_2.3_mro`, or |
| 1088 |
read the thread on python-dev starting with the message at |
| 1089 |
https://mail.python.org/pipermail/python-dev/2002-October/029035.html. Samuele |
| 1090 |
Pedroni first pointed out the problem and also implemented the fix by coding the |
| 1091 |
C3 algorithm. |
| 1092 |
|
| 1093 |
* Python runs multithreaded programs by switching between threads after |
| 1094 |
executing N bytecodes. The default value for N has been increased from 10 to |
| 1095 |
100 bytecodes, speeding up single-threaded applications by reducing the |
| 1096 |
switching overhead. Some multithreaded applications may suffer slower response |
| 1097 |
time, but that's easily fixed by setting the limit back to a lower number using |
| 1098 |
``sys.setcheckinterval(N)``. The limit can be retrieved with the new |
| 1099 |
:func:`!sys.getcheckinterval` function. |
| 1100 |
|
| 1101 |
* One minor but far-reaching change is that the names of extension types defined |
| 1102 |
by the modules included with Python now contain the module and a ``'.'`` in |
| 1103 |
front of the type name. For example, in Python 2.2, if you created a socket and |
| 1104 |
printed its :attr:`!__class__`, you'd get this output:: |
| 1105 |
|
| 1106 |
>>> s = socket.socket() |
| 1107 |
>>> s.__class__ |
| 1108 |
<type 'socket'> |
| 1109 |
|
| 1110 |
In 2.3, you get this:: |
| 1111 |
|
| 1112 |
>>> s.__class__ |
| 1113 |
<type '_socket.socket'> |
| 1114 |
|
| 1115 |
* One of the noted incompatibilities between old- and new-style classes has been |
| 1116 |
removed: you can now assign to the :attr:`~type.__name__` and :attr:`~type.__bases__` |
| 1117 |
attributes of new-style classes. There are some restrictions on what can be |
| 1118 |
assigned to :attr:`!__bases__` along the lines of those relating to assigning to |
| 1119 |
an instance's :attr:`~object.__class__` attribute. |
| 1120 |
|
| 1121 |
.. ====================================================================== |
| 1122 |
|
| 1123 |
|
| 1124 |
String Changes |
| 1125 |
-------------- |
| 1126 |
|
| 1127 |
* The :keyword:`in` operator now works differently for strings. Previously, when |
| 1128 |
evaluating ``X in Y`` where *X* and *Y* are strings, *X* could only be a single |
| 1129 |
character. That's now changed; *X* can be a string of any length, and ``X in Y`` |
| 1130 |
will return :const:`True` if *X* is a substring of *Y*. If *X* is the empty |
| 1131 |
string, the result is always :const:`True`. :: |
| 1132 |
|
| 1133 |
>>> 'ab' in 'abcd' |
| 1134 |
True |
| 1135 |
>>> 'ad' in 'abcd' |
| 1136 |
False |
| 1137 |
>>> '' in 'abcd' |
| 1138 |
True |
| 1139 |
|
| 1140 |
Note that this doesn't tell you where the substring starts; if you need that |
| 1141 |
information, use the :meth:`~str.find` string method. |
| 1142 |
|
| 1143 |
* The :meth:`~str.strip`, :meth:`~str.lstrip`, and :meth:`~str.rstrip` string methods now have |
| 1144 |
an optional argument for specifying the characters to strip. The default is |
| 1145 |
still to remove all whitespace characters:: |
| 1146 |
|
| 1147 |
>>> ' abc '.strip() |
| 1148 |
'abc' |
| 1149 |
>>> '><><abc<><><>'.strip('<>') |
| 1150 |
'abc' |
| 1151 |
>>> '><><abc<><><>\n'.strip('<>') |
| 1152 |
'abc<><><>\n' |
| 1153 |
>>> u'\u4000\u4001abc\u4000'.strip(u'\u4000') |
| 1154 |
u'\u4001abc' |
| 1155 |
>>> |
| 1156 |
|
| 1157 |
(Suggested by Simon Brunning and implemented by Walter Dörwald.) |
| 1158 |
|
| 1159 |
* The :meth:`~str.startswith` and :meth:`~str.endswith` string methods now accept negative |
| 1160 |
numbers for the *start* and *end* parameters. |
| 1161 |
|
| 1162 |
* Another new string method is :meth:`~str.zfill`, originally a function in the |
| 1163 |
:mod:`string` module. :meth:`~str.zfill` pads a numeric string with zeros on the |
| 1164 |
left until it's the specified width. Note that the ``%`` operator is still more |
| 1165 |
flexible and powerful than :meth:`~str.zfill`. :: |
| 1166 |
|
| 1167 |
>>> '45'.zfill(4) |
| 1168 |
'0045' |
| 1169 |
>>> '12345'.zfill(4) |
| 1170 |
'12345' |
| 1171 |
>>> 'goofy'.zfill(6) |
| 1172 |
'0goofy' |
| 1173 |
|
| 1174 |
(Contributed by Walter Dörwald.) |
| 1175 |
|
| 1176 |
* A new type object, :class:`!basestring`, has been added. Both 8-bit strings and |
| 1177 |
Unicode strings inherit from this type, so ``isinstance(obj, basestring)`` will |
| 1178 |
return :const:`True` for either kind of string. It's a completely abstract |
| 1179 |
type, so you can't create :class:`!basestring` instances. |
| 1180 |
|
| 1181 |
* Interned strings are no longer immortal and will now be garbage-collected in |
| 1182 |
the usual way when the only reference to them is from the internal dictionary of |
| 1183 |
interned strings. (Implemented by Oren Tirosh.) |
| 1184 |
|
| 1185 |
.. ====================================================================== |
| 1186 |
|
| 1187 |
|
| 1188 |
Optimizations |
| 1189 |
------------- |
| 1190 |
|
| 1191 |
* The creation of new-style class instances has been made much faster; they're |
| 1192 |
now faster than classic classes! |
| 1193 |
|
| 1194 |
* The :meth:`~list.sort` method of list objects has been extensively rewritten by Tim |
| 1195 |
Peters, and the implementation is significantly faster. |
| 1196 |
|
| 1197 |
* Multiplication of large long integers is now much faster thanks to an |
| 1198 |
implementation of Karatsuba multiplication, an algorithm that scales better than |
| 1199 |
the *O*\ (*n*\ :sup:`2`) required for the grade-school multiplication algorithm. (Original |
| 1200 |
patch by Christopher A. Craig, and significantly reworked by Tim Peters.) |
| 1201 |
|
| 1202 |
* The ``SET_LINENO`` opcode is now gone. This may provide a small speed |
| 1203 |
increase, depending on your compiler's idiosyncrasies. See section |
| 1204 |
:ref:`23section-other` for a longer explanation. (Removed by Michael Hudson.) |
| 1205 |
|
| 1206 |
* :func:`!xrange` objects now have their own iterator, making ``for i in |
| 1207 |
xrange(n)`` slightly faster than ``for i in range(n)``. (Patch by Raymond |
| 1208 |
Hettinger.) |
| 1209 |
|
| 1210 |
* A number of small rearrangements have been made in various hotspots to improve |
| 1211 |
performance, such as inlining a function or removing some code. (Implemented |
| 1212 |
mostly by GvR, but lots of people have contributed single changes.) |
| 1213 |
|
| 1214 |
The net result of the 2.3 optimizations is that Python 2.3 runs the pystone |
| 1215 |
benchmark around 25% faster than Python 2.2. |
| 1216 |
|
| 1217 |
.. ====================================================================== |
| 1218 |
|
| 1219 |
|
| 1220 |
New, Improved, and Deprecated Modules |
| 1221 |
===================================== |
| 1222 |
|
| 1223 |
As usual, Python's standard library received a number of enhancements and bug |
| 1224 |
fixes. Here's a partial list of the most notable changes, sorted alphabetically |
| 1225 |
by module name. Consult the :file:`Misc/NEWS` file in the source tree for a more |
| 1226 |
complete list of changes, or look through the CVS logs for all the details. |
| 1227 |
|
| 1228 |
* The :mod:`array` module now supports arrays of Unicode characters using the |
| 1229 |
``'u'`` format character. Arrays also now support using the ``+=`` assignment |
| 1230 |
operator to add another array's contents, and the ``*=`` assignment operator to |
| 1231 |
repeat an array. (Contributed by Jason Orendorff.) |
| 1232 |
|
| 1233 |
* The :mod:`!bsddb` module has been replaced by version 4.1.6 of the `PyBSDDB |
| 1234 |
<https://pybsddb.sourceforge.net>`_ package, providing a more complete interface |
| 1235 |
to the transactional features of the BerkeleyDB library. |
| 1236 |
|
| 1237 |
The old version of the module has been renamed to :mod:`!bsddb185` and is no |
| 1238 |
longer built automatically; you'll have to edit :file:`Modules/Setup` to enable |
| 1239 |
it. Note that the new :mod:`!bsddb` package is intended to be compatible with |
| 1240 |
the old module, so be sure to file bugs if you discover any incompatibilities. |
| 1241 |
When upgrading to Python 2.3, if the new interpreter is compiled with a new |
| 1242 |
version of the underlying BerkeleyDB library, you will almost certainly have to |
| 1243 |
convert your database files to the new version. You can do this fairly easily |
| 1244 |
with the new scripts :file:`db2pickle.py` and :file:`pickle2db.py` which you |
| 1245 |
will find in the distribution's :file:`Tools/scripts` directory. If you've |
| 1246 |
already been using the PyBSDDB package and importing it as :mod:`!bsddb3`, you |
| 1247 |
will have to change your ``import`` statements to import it as :mod:`!bsddb`. |
| 1248 |
|
| 1249 |
* The new :mod:`bz2` module is an interface to the bz2 data compression library. |
| 1250 |
bz2-compressed data is usually smaller than corresponding |
| 1251 |
:mod:`zlib`\ -compressed data. (Contributed by Gustavo Niemeyer.) |
| 1252 |
|
| 1253 |
* A set of standard date/time types has been added in the new :mod:`datetime` |
| 1254 |
module. See the following section for more details. |
| 1255 |
|
| 1256 |
* The Distutils :class:`!Extension` class now supports an extra constructor |
| 1257 |
argument named *depends* for listing additional source files that an extension |
| 1258 |
depends on. This lets Distutils recompile the module if any of the dependency |
| 1259 |
files are modified. For example, if :file:`sampmodule.c` includes the header |
| 1260 |
file :file:`sample.h`, you would create the :class:`!Extension` object like |
| 1261 |
this:: |
| 1262 |
|
| 1263 |
ext = Extension("samp", |
| 1264 |
sources=["sampmodule.c"], |
| 1265 |
depends=["sample.h"]) |
| 1266 |
|
| 1267 |
Modifying :file:`sample.h` would then cause the module to be recompiled. |
| 1268 |
(Contributed by Jeremy Hylton.) |
| 1269 |
|
| 1270 |
* Other minor changes to Distutils: it now checks for the :envvar:`CC`, |
| 1271 |
:envvar:`CFLAGS`, :envvar:`!CPP`, :envvar:`LDFLAGS`, and :envvar:`CPPFLAGS` |
| 1272 |
environment variables, using them to override the settings in Python's |
| 1273 |
configuration (contributed by Robert Weber). |
| 1274 |
|
| 1275 |
* Previously the :mod:`doctest` module would only search the docstrings of |
| 1276 |
public methods and functions for test cases, but it now also examines private |
| 1277 |
ones as well. The :func:`~doctest.DocTestSuite` function creates a |
| 1278 |
:class:`unittest.TestSuite` object from a set of :mod:`doctest` tests. |
| 1279 |
|
| 1280 |
* The new ``gc.get_referents(object)`` function returns a list of all the |
| 1281 |
objects referenced by *object*. |
| 1282 |
|
| 1283 |
* The :mod:`getopt` module gained a new function, :func:`~getopt.gnu_getopt`, that |
| 1284 |
supports the same arguments as the existing :func:`~getopt.getopt` function but uses |
| 1285 |
GNU-style scanning mode. The existing :func:`~getopt.getopt` stops processing options as |
| 1286 |
soon as a non-option argument is encountered, but in GNU-style mode processing |
| 1287 |
continues, meaning that options and arguments can be mixed. For example:: |
| 1288 |
|
| 1289 |
>>> getopt.getopt(['-f', 'filename', 'output', '-v'], 'f:v') |
| 1290 |
([('-f', 'filename')], ['output', '-v']) |
| 1291 |
>>> getopt.gnu_getopt(['-f', 'filename', 'output', '-v'], 'f:v') |
| 1292 |
([('-f', 'filename'), ('-v', '')], ['output']) |
| 1293 |
|
| 1294 |
(Contributed by Peter Åstrand.) |
| 1295 |
|
| 1296 |
* The :mod:`grp`, :mod:`pwd`, and :mod:`resource` modules now return enhanced |
| 1297 |
tuples:: |
| 1298 |
|
| 1299 |
>>> import grp |
| 1300 |
>>> g = grp.getgrnam('amk') |
| 1301 |
>>> g.gr_name, g.gr_gid |
| 1302 |
('amk', 500) |
| 1303 |
|
| 1304 |
* The :mod:`gzip` module can now handle files exceeding 2 GiB. |
| 1305 |
|
| 1306 |
* The new :mod:`heapq` module contains an implementation of a heap queue |
| 1307 |
algorithm. A heap is an array-like data structure that keeps items in a |
| 1308 |
partially sorted order such that, for every index *k*, ``heap[k] <= |
| 1309 |
heap[2*k+1]`` and ``heap[k] <= heap[2*k+2]``. This makes it quick to remove the |
| 1310 |
smallest item, and inserting a new item while maintaining the heap property is |
| 1311 |
*O*\ (log *n*). (See https://xlinux.nist.gov/dads//HTML/priorityque.html for more |
| 1312 |
information about the priority queue data structure.) |
| 1313 |
|
| 1314 |
The :mod:`heapq` module provides :func:`~heapq.heappush` and :func:`~heapq.heappop` functions |
| 1315 |
for adding and removing items while maintaining the heap property on top of some |
| 1316 |
other mutable Python sequence type. Here's an example that uses a Python list:: |
| 1317 |
|
| 1318 |
>>> import heapq |
| 1319 |
>>> heap = [] |
| 1320 |
>>> for item in [3, 7, 5, 11, 1]: |
| 1321 |
... heapq.heappush(heap, item) |
| 1322 |
... |
| 1323 |
>>> heap |
| 1324 |
[1, 3, 5, 11, 7] |
| 1325 |
>>> heapq.heappop(heap) |
| 1326 |
1 |
| 1327 |
>>> heapq.heappop(heap) |
| 1328 |
3 |
| 1329 |
>>> heap |
| 1330 |
[5, 7, 11] |
| 1331 |
|
| 1332 |
(Contributed by Kevin O'Connor.) |
| 1333 |
|
| 1334 |
* The IDLE integrated development environment has been updated using the code |
| 1335 |
from the IDLEfork project (https://idlefork.sourceforge.net). The most notable feature is |
| 1336 |
that the code being developed is now executed in a subprocess, meaning that |
| 1337 |
there's no longer any need for manual ``reload()`` operations. IDLE's core code |
| 1338 |
has been incorporated into the standard library as the :mod:`idlelib` package. |
| 1339 |
|
| 1340 |
* The :mod:`imaplib` module now supports IMAP over SSL. (Contributed by Piers |
| 1341 |
Lauder and Tino Lange.) |
| 1342 |
|
| 1343 |
* The :mod:`itertools` contains a number of useful functions for use with |
| 1344 |
iterators, inspired by various functions provided by the ML and Haskell |
| 1345 |
languages. For example, ``itertools.ifilter(predicate, iterator)`` returns all |
| 1346 |
elements in the iterator for which the function :func:`!predicate` returns |
| 1347 |
:const:`True`, and ``itertools.repeat(obj, N)`` returns ``obj`` *N* times. |
| 1348 |
There are a number of other functions in the module; see the package's reference |
| 1349 |
documentation for details. |
| 1350 |
(Contributed by Raymond Hettinger.) |
| 1351 |
|
| 1352 |
* Two new functions in the :mod:`math` module, ``degrees(rads)`` and |
| 1353 |
``radians(degs)``, convert between radians and degrees. Other functions in |
| 1354 |
the :mod:`math` module such as :func:`math.sin` and :func:`math.cos` have always |
| 1355 |
required input values measured in radians. Also, an optional *base* argument |
| 1356 |
was added to :func:`math.log` to make it easier to compute logarithms for bases |
| 1357 |
other than ``e`` and ``10``. (Contributed by Raymond Hettinger.) |
| 1358 |
|
| 1359 |
* Several new POSIX functions (:func:`!getpgid`, :func:`!killpg`, :func:`!lchown`, |
| 1360 |
:func:`!loadavg`, :func:`!major`, :func:`!makedev`, :func:`!minor`, and |
| 1361 |
:func:`!mknod`) were added to the :mod:`posix` module that underlies the |
| 1362 |
:mod:`os` module. (Contributed by Gustavo Niemeyer, Geert Jansen, and Denis S. |
| 1363 |
Otkidach.) |
| 1364 |
|
| 1365 |
* In the :mod:`os` module, the :func:`!\*stat` family of functions can now report |
| 1366 |
fractions of a second in a timestamp. Such time stamps are represented as |
| 1367 |
floats, similar to the value returned by :func:`time.time`. |
| 1368 |
|
| 1369 |
During testing, it was found that some applications will break if time stamps |
| 1370 |
are floats. For compatibility, when using the tuple interface of the |
| 1371 |
:class:`~os.stat_result` time stamps will be represented as integers. When using |
| 1372 |
named fields (a feature first introduced in Python 2.2), time stamps are still |
| 1373 |
represented as integers, unless :func:`!os.stat_float_times` is invoked to enable |
| 1374 |
float return values:: |
| 1375 |
|
| 1376 |
>>> os.stat("/tmp").st_mtime |
| 1377 |
1034791200 |
| 1378 |
>>> os.stat_float_times(True) |
| 1379 |
>>> os.stat("/tmp").st_mtime |
| 1380 |
1034791200.6335014 |
| 1381 |
|
| 1382 |
In Python 2.4, the default will change to always returning floats. |
| 1383 |
|
| 1384 |
Application developers should enable this feature only if all their libraries |
| 1385 |
work properly when confronted with floating-point time stamps, or if they use |
| 1386 |
the tuple API. If used, the feature should be activated on an application level |
| 1387 |
instead of trying to enable it on a per-use basis. |
| 1388 |
|
| 1389 |
* The :mod:`optparse` module contains a new parser for command-line arguments |
| 1390 |
that can convert option values to a particular Python type and will |
| 1391 |
automatically generate a usage message. See the following section for more |
| 1392 |
details. |
| 1393 |
|
| 1394 |
* The old and never-documented :mod:`!linuxaudiodev` module has been deprecated, |
| 1395 |
and a new version named :mod:`!ossaudiodev` has been added. The module was |
| 1396 |
renamed because the OSS sound drivers can be used on platforms other than Linux, |
| 1397 |
and the interface has also been tidied and brought up to date in various ways. |
| 1398 |
(Contributed by Greg Ward and Nicholas FitzRoy-Dale.) |
| 1399 |
|
| 1400 |
* The new :mod:`platform` module contains a number of functions that try to |
| 1401 |
determine various properties of the platform you're running on. There are |
| 1402 |
functions for getting the architecture, CPU type, the Windows OS version, and |
| 1403 |
even the Linux distribution version. (Contributed by Marc-André Lemburg.) |
| 1404 |
|
| 1405 |
* The parser objects provided by the :mod:`pyexpat <xml.parsers.expat>` module can now optionally |
| 1406 |
buffer character data, resulting in fewer calls to your character data handler |
| 1407 |
and therefore faster performance. Setting the parser object's |
| 1408 |
:attr:`~xml.parsers.expat.xmlparser.buffer_text` attribute to :const:`True` will enable buffering. |
| 1409 |
|
| 1410 |
* The ``sample(population, k)`` function was added to the :mod:`random` |
| 1411 |
module. *population* is a sequence or :class:`!xrange` object containing the |
| 1412 |
elements of a population, and :func:`~random.sample` chooses *k* elements from the |
| 1413 |
population without replacing chosen elements. *k* can be any value up to |
| 1414 |
``len(population)``. For example:: |
| 1415 |
|
| 1416 |
>>> days = ['Mo', 'Tu', 'We', 'Th', 'Fr', 'St', 'Sn'] |
| 1417 |
>>> random.sample(days, 3) # Choose 3 elements |
| 1418 |
['St', 'Sn', 'Th'] |
| 1419 |
>>> random.sample(days, 7) # Choose 7 elements |
| 1420 |
['Tu', 'Th', 'Mo', 'We', 'St', 'Fr', 'Sn'] |
| 1421 |
>>> random.sample(days, 7) # Choose 7 again |
| 1422 |
['We', 'Mo', 'Sn', 'Fr', 'Tu', 'St', 'Th'] |
| 1423 |
>>> random.sample(days, 8) # Can't choose eight |
| 1424 |
Traceback (most recent call last): |
| 1425 |
File "<stdin>", line 1, in ? |
| 1426 |
File "random.py", line 414, in sample |
| 1427 |
raise ValueError, "sample larger than population" |
| 1428 |
ValueError: sample larger than population |
| 1429 |
>>> random.sample(xrange(1,10000,2), 10) # Choose ten odd nos. under 10000 |
| 1430 |
[3407, 3805, 1505, 7023, 2401, 2267, 9733, 3151, 8083, 9195] |
| 1431 |
|
| 1432 |
The :mod:`random` module now uses a new algorithm, the Mersenne Twister, |
| 1433 |
implemented in C. It's faster and more extensively studied than the previous |
| 1434 |
algorithm. |
| 1435 |
|
| 1436 |
(All changes contributed by Raymond Hettinger.) |
| 1437 |
|
| 1438 |
* The :mod:`readline` module also gained a number of new functions: |
| 1439 |
:func:`~readline.get_history_item`, :func:`~readline.get_current_history_length`, and |
| 1440 |
:func:`~readline.redisplay`. |
| 1441 |
|
| 1442 |
* The :mod:`!rexec` and :mod:`!Bastion` modules have been declared dead, and |
| 1443 |
attempts to import them will fail with a :exc:`RuntimeError`. New-style classes |
| 1444 |
provide new ways to break out of the restricted execution environment provided |
| 1445 |
by :mod:`!rexec`, and no one has interest in fixing them or time to do so. If |
| 1446 |
you have applications using :mod:`!rexec`, rewrite them to use something else. |
| 1447 |
|
| 1448 |
(Sticking with Python 2.2 or 2.1 will not make your applications any safer |
| 1449 |
because there are known bugs in the :mod:`!rexec` module in those versions. To |
| 1450 |
repeat: if you're using :mod:`!rexec`, stop using it immediately.) |
| 1451 |
|
| 1452 |
* The :mod:`!rotor` module has been deprecated because the algorithm it uses for |
| 1453 |
encryption is not believed to be secure. If you need encryption, use one of the |
| 1454 |
several AES Python modules that are available separately. |
| 1455 |
|
| 1456 |
* The :mod:`shutil` module gained a ``move(src, dest)`` function that |
| 1457 |
recursively moves a file or directory to a new location. |
| 1458 |
|
| 1459 |
* Support for more advanced POSIX signal handling was added to the :mod:`signal` |
| 1460 |
but then removed again as it proved impossible to make it work reliably across |
| 1461 |
platforms. |
| 1462 |
|
| 1463 |
* The :mod:`socket` module now supports timeouts. You can call the |
| 1464 |
``settimeout(t)`` method on a socket object to set a timeout of *t* seconds. |
| 1465 |
Subsequent socket operations that take longer than *t* seconds to complete will |
| 1466 |
abort and raise a :exc:`socket.timeout` exception. |
| 1467 |
|
| 1468 |
The original timeout implementation was by Tim O'Malley. Michael Gilfix |
| 1469 |
integrated it into the Python :mod:`socket` module and shepherded it through a |
| 1470 |
lengthy review. After the code was checked in, Guido van Rossum rewrote parts |
| 1471 |
of it. (This is a good example of a collaborative development process in |
| 1472 |
action.) |
| 1473 |
|
| 1474 |
* On Windows, the :mod:`socket` module now ships with Secure Sockets Layer |
| 1475 |
(SSL) support. |
| 1476 |
|
| 1477 |
* The value of the C :c:macro:`!PYTHON_API_VERSION` macro is now exposed at the |
| 1478 |
Python level as ``sys.api_version``. The current exception can be cleared by |
| 1479 |
calling the new :func:`!sys.exc_clear` function. |
| 1480 |
|
| 1481 |
* The new :mod:`tarfile` module allows reading from and writing to |
| 1482 |
:program:`tar`\ -format archive files. (Contributed by Lars Gustäbel.) |
| 1483 |
|
| 1484 |
* The new :mod:`textwrap` module contains functions for wrapping strings |
| 1485 |
containing paragraphs of text. The ``wrap(text, width)`` function takes a |
| 1486 |
string and returns a list containing the text split into lines of no more than |
| 1487 |
the chosen width. The ``fill(text, width)`` function returns a single |
| 1488 |
string, reformatted to fit into lines no longer than the chosen width. (As you |
| 1489 |
can guess, :func:`~textwrap.fill` is built on top of :func:`~textwrap.wrap`. For example:: |
| 1490 |
|
| 1491 |
>>> import textwrap |
| 1492 |
>>> paragraph = "Not a whit, we defy augury: ... more text ..." |
| 1493 |
>>> textwrap.wrap(paragraph, 60) |
| 1494 |
["Not a whit, we defy augury: there's a special providence in", |
| 1495 |
"the fall of a sparrow. If it be now, 'tis not to come; if it", |
| 1496 |
...] |
| 1497 |
>>> print textwrap.fill(paragraph, 35) |
| 1498 |
Not a whit, we defy augury: there's |
| 1499 |
a special providence in the fall of |
| 1500 |
a sparrow. If it be now, 'tis not |
| 1501 |
to come; if it be not to come, it |
| 1502 |
will be now; if it be not now, yet |
| 1503 |
it will come: the readiness is all. |
| 1504 |
>>> |
| 1505 |
|
| 1506 |
The module also contains a :class:`~textwrap.TextWrapper` class that actually implements |
| 1507 |
the text wrapping strategy. Both the :class:`~textwrap.TextWrapper` class and the |
| 1508 |
:func:`~textwrap.wrap` and :func:`~textwrap.fill` functions support a number of additional keyword |
| 1509 |
arguments for fine-tuning the formatting; consult the module's documentation |
| 1510 |
for details. (Contributed by Greg Ward.) |
| 1511 |
|
| 1512 |
* The :mod:`!thread` and :mod:`threading` modules now have companion modules, |
| 1513 |
:mod:`!dummy_thread` and :mod:`!dummy_threading`, that provide a do-nothing |
| 1514 |
implementation of the :mod:`!thread` module's interface for platforms where |
| 1515 |
threads are not supported. The intention is to simplify thread-aware modules |
| 1516 |
(ones that *don't* rely on threads to run) by putting the following code at the |
| 1517 |
top:: |
| 1518 |
|
| 1519 |
try: |
| 1520 |
import threading as _threading |
| 1521 |
except ImportError: |
| 1522 |
import dummy_threading as _threading |
| 1523 |
|
| 1524 |
In this example, :mod:`!_threading` is used as the module name to make it clear |
| 1525 |
that the module being used is not necessarily the actual :mod:`threading` |
| 1526 |
module. Code can call functions and use classes in :mod:`!_threading` whether or |
| 1527 |
not threads are supported, avoiding an :keyword:`if` statement and making the |
| 1528 |
code slightly clearer. This module will not magically make multithreaded code |
| 1529 |
run without threads; code that waits for another thread to return or to do |
| 1530 |
something will simply hang forever. |
| 1531 |
|
| 1532 |
* The :mod:`time` module's :func:`~time.strptime` function has long been an annoyance |
| 1533 |
because it uses the platform C library's :func:`~time.strptime` implementation, and |
| 1534 |
different platforms sometimes have odd bugs. Brett Cannon contributed a |
| 1535 |
portable implementation that's written in pure Python and should behave |
| 1536 |
identically on all platforms. |
| 1537 |
|
| 1538 |
* The new :mod:`timeit` module helps measure how long snippets of Python code |
| 1539 |
take to execute. The :file:`timeit.py` file can be run directly from the |
| 1540 |
command line, or the module's :class:`~timeit.Timer` class can be imported and used |
| 1541 |
directly. Here's a short example that figures out whether it's faster to |
| 1542 |
convert an 8-bit string to Unicode by appending an empty Unicode string to it or |
| 1543 |
by using the :func:`!unicode` function:: |
| 1544 |
|
| 1545 |
import timeit |
| 1546 |
|
| 1547 |
timer1 = timeit.Timer('unicode("abc")') |
| 1548 |
timer2 = timeit.Timer('"abc" + u""') |
| 1549 |
|
| 1550 |
# Run three trials |
| 1551 |
print timer1.repeat(repeat=3, number=100000) |
| 1552 |
print timer2.repeat(repeat=3, number=100000) |
| 1553 |
|
| 1554 |
# On my laptop this outputs: |
| 1555 |
# [0.36831796169281006, 0.37441694736480713, 0.35304892063140869] |
| 1556 |
# [0.17574405670166016, 0.18193507194519043, 0.17565798759460449] |
| 1557 |
|
| 1558 |
* The :mod:`!Tix` module has received various bug fixes and updates for the |
| 1559 |
current version of the Tix package. |
| 1560 |
|
| 1561 |
* The :mod:`!Tkinter` module now works with a thread-enabled version of Tcl. |
| 1562 |
Tcl's threading model requires that widgets only be accessed from the thread in |
| 1563 |
which they're created; accesses from another thread can cause Tcl to panic. For |
| 1564 |
certain Tcl interfaces, :mod:`!Tkinter` will now automatically avoid this when a |
| 1565 |
widget is accessed from a different thread by marshalling a command, passing it |
| 1566 |
to the correct thread, and waiting for the results. Other interfaces can't be |
| 1567 |
handled automatically but :mod:`!Tkinter` will now raise an exception on such an |
| 1568 |
access so that you can at least find out about the problem. See |
| 1569 |
https://mail.python.org/pipermail/python-dev/2002-December/031107.html for a more |
| 1570 |
detailed explanation of this change. (Implemented by Martin von Löwis.) |
| 1571 |
|
| 1572 |
* Calling Tcl methods through :mod:`!_tkinter` no longer returns only strings. |
| 1573 |
Instead, if Tcl returns other objects those objects are converted to their |
| 1574 |
Python equivalent, if one exists, or wrapped with a :class:`!_tkinter.Tcl_Obj` |
| 1575 |
object if no Python equivalent exists. This behavior can be controlled through |
| 1576 |
the :meth:`!wantobjects` method of :class:`!tkapp` objects. |
| 1577 |
|
| 1578 |
When using :mod:`!_tkinter` through the :mod:`!Tkinter` module (as most Tkinter |
| 1579 |
applications will), this feature is always activated. It should not cause |
| 1580 |
compatibility problems, since Tkinter would always convert string results to |
| 1581 |
Python types where possible. |
| 1582 |
|
| 1583 |
If any incompatibilities are found, the old behavior can be restored by setting |
| 1584 |
the :attr:`!wantobjects` variable in the :mod:`!Tkinter` module to false before |
| 1585 |
creating the first :class:`!tkapp` object. :: |
| 1586 |
|
| 1587 |
import Tkinter |
| 1588 |
Tkinter.wantobjects = 0 |
| 1589 |
|
| 1590 |
Any breakage caused by this change should be reported as a bug. |
| 1591 |
|
| 1592 |
* The :mod:`!UserDict` module has a new :class:`!DictMixin` class which defines |
| 1593 |
all dictionary methods for classes that already have a minimum mapping |
| 1594 |
interface. This greatly simplifies writing classes that need to be |
| 1595 |
substitutable for dictionaries, such as the classes in the :mod:`shelve` |
| 1596 |
module. |
| 1597 |
|
| 1598 |
Adding the mix-in as a superclass provides the full dictionary interface |
| 1599 |
whenever the class defines :meth:`~object.__getitem__`, :meth:`~object.__setitem__`, |
| 1600 |
:meth:`~object.__delitem__`, and :meth:`!keys`. For example:: |
| 1601 |
|
| 1602 |
>>> import UserDict |
| 1603 |
>>> class SeqDict(UserDict.DictMixin): |
| 1604 |
... """Dictionary lookalike implemented with lists.""" |
| 1605 |
... def __init__(self): |
| 1606 |
... self.keylist = [] |
| 1607 |
... self.valuelist = [] |
| 1608 |
... def __getitem__(self, key): |
| 1609 |
... try: |
| 1610 |
... i = self.keylist.index(key) |
| 1611 |
... except ValueError: |
| 1612 |
... raise KeyError |
| 1613 |
... return self.valuelist[i] |
| 1614 |
... def __setitem__(self, key, value): |
| 1615 |
... try: |
| 1616 |
... i = self.keylist.index(key) |
| 1617 |
... self.valuelist[i] = value |
| 1618 |
... except ValueError: |
| 1619 |
... self.keylist.append(key) |
| 1620 |
... self.valuelist.append(value) |
| 1621 |
... def __delitem__(self, key): |
| 1622 |
... try: |
| 1623 |
... i = self.keylist.index(key) |
| 1624 |
... except ValueError: |
| 1625 |
... raise KeyError |
| 1626 |
... self.keylist.pop(i) |
| 1627 |
... self.valuelist.pop(i) |
| 1628 |
... def keys(self): |
| 1629 |
... return list(self.keylist) |
| 1630 |
... |
| 1631 |
>>> s = SeqDict() |
| 1632 |
>>> dir(s) # See that other dictionary methods are implemented |
| 1633 |
['__cmp__', '__contains__', '__delitem__', '__doc__', '__getitem__', |
| 1634 |
'__init__', '__iter__', '__len__', '__module__', '__repr__', |
| 1635 |
'__setitem__', 'clear', 'get', 'has_key', 'items', 'iteritems', |
| 1636 |
'iterkeys', 'itervalues', 'keylist', 'keys', 'pop', 'popitem', |
| 1637 |
'setdefault', 'update', 'valuelist', 'values'] |
| 1638 |
|
| 1639 |
(Contributed by Raymond Hettinger.) |
| 1640 |
|
| 1641 |
* The DOM implementation in :mod:`xml.dom.minidom` can now generate XML output |
| 1642 |
in a particular encoding by providing an optional encoding argument to the |
| 1643 |
:meth:`~xml.dom.minidom.Node.toxml` and :meth:`~xml.dom.minidom.Node.toprettyxml` methods of DOM nodes. |
| 1644 |
|
| 1645 |
* The :mod:`!xmlrpclib` module now supports an XML-RPC extension for handling nil |
| 1646 |
data values such as Python's ``None``. Nil values are always supported on |
| 1647 |
unmarshalling an XML-RPC response. To generate requests containing ``None``, |
| 1648 |
you must supply a true value for the *allow_none* parameter when creating a |
| 1649 |
:class:`!Marshaller` instance. |
| 1650 |
|
| 1651 |
* The new :mod:`!DocXMLRPCServer` module allows writing self-documenting XML-RPC |
| 1652 |
servers. Run it in demo mode (as a program) to see it in action. Pointing the |
| 1653 |
web browser to the RPC server produces pydoc-style documentation; pointing |
| 1654 |
xmlrpclib to the server allows invoking the actual methods. (Contributed by |
| 1655 |
Brian Quinlan.) |
| 1656 |
|
| 1657 |
* Support for internationalized domain names (RFCs 3454, 3490, 3491, and 3492) |
| 1658 |
has been added. The "idna" encoding can be used to convert between a Unicode |
| 1659 |
domain name and the ASCII-compatible encoding (ACE) of that name. :: |
| 1660 |
|
| 1661 |
>{}>{}> u"www.Alliancefrançaise.nu".encode("idna") |
| 1662 |
'www.xn--alliancefranaise-npb.nu' |
| 1663 |
|
| 1664 |
The :mod:`socket` module has also been extended to transparently convert |
| 1665 |
Unicode hostnames to the ACE version before passing them to the C library. |
| 1666 |
Modules that deal with hostnames such as :mod:`!httplib` and :mod:`ftplib`) |
| 1667 |
also support Unicode host names; :mod:`!httplib` also sends HTTP ``Host`` |
| 1668 |
headers using the ACE version of the domain name. :mod:`urllib` supports |
| 1669 |
Unicode URLs with non-ASCII host names as long as the ``path`` part of the URL |
| 1670 |
is ASCII only. |
| 1671 |
|
| 1672 |
To implement this change, the :mod:`stringprep` module, the ``mkstringprep`` |
| 1673 |
tool and the ``punycode`` encoding have been added. |
| 1674 |
|
| 1675 |
.. ====================================================================== |
| 1676 |
|
| 1677 |
|
| 1678 |
Date/Time Type |
| 1679 |
-------------- |
| 1680 |
|
| 1681 |
Date and time types suitable for expressing timestamps were added as the |
| 1682 |
:mod:`datetime` module. The types don't support different calendars or many |
| 1683 |
fancy features, and just stick to the basics of representing time. |
| 1684 |
|
| 1685 |
The three primary types are: :class:`~datetime.date`, representing a day, month, and year; |
| 1686 |
:class:`~datetime.time`, consisting of hour, minute, and second; and :class:`~datetime.datetime`, |
| 1687 |
which contains all the attributes of both :class:`~datetime.date` and :class:`~datetime.time`. |
| 1688 |
There's also a :class:`~datetime.timedelta` class representing differences between two |
| 1689 |
points in time, and time zone logic is implemented by classes inheriting from |
| 1690 |
the abstract :class:`~datetime.tzinfo` class. |
| 1691 |
|
| 1692 |
You can create instances of :class:`~datetime.date` and :class:`~datetime.time` by either supplying |
| 1693 |
keyword arguments to the appropriate constructor, e.g. |
| 1694 |
``datetime.date(year=1972, month=10, day=15)``, or by using one of a number of |
| 1695 |
class methods. For example, the :meth:`~datetime.date.today` class method returns the |
| 1696 |
current local date. |
| 1697 |
|
| 1698 |
Once created, instances of the date/time classes are all immutable. There are a |
| 1699 |
number of methods for producing formatted strings from objects:: |
| 1700 |
|
| 1701 |
>>> import datetime as dt |
| 1702 |
>>> now = dt.datetime.now() |
| 1703 |
>>> now.isoformat() |
| 1704 |
'2002-12-30T21:27:03.994956' |
| 1705 |
>>> now.ctime() # Only available on date, datetime |
| 1706 |
'Mon Dec 30 21:27:03 2002' |
| 1707 |
>>> now.strftime('%Y %d %b') |
| 1708 |
'2002 30 Dec' |
| 1709 |
|
| 1710 |
The :meth:`~datetime.datetime.replace` method allows modifying one or more fields of a |
| 1711 |
:class:`~datetime.date` or :class:`~datetime.datetime` instance, returning a new instance:: |
| 1712 |
|
| 1713 |
>>> d = dt.datetime.now() |
| 1714 |
>>> d |
| 1715 |
datetime.datetime(2002, 12, 30, 22, 15, 38, 827738) |
| 1716 |
>>> d.replace(year=2001, hour=12) |
| 1717 |
datetime.datetime(2001, 12, 30, 12, 15, 38, 827738) |
| 1718 |
>>> |
| 1719 |
|
| 1720 |
Instances can be compared, hashed, and converted to strings (the result is the |
| 1721 |
same as that of :meth:`~datetime.datetime.isoformat`). :class:`~datetime.date` and :class:`~datetime.datetime` |
| 1722 |
instances can be subtracted from each other, and added to :class:`~datetime.timedelta` |
| 1723 |
instances. The largest missing feature is that there's no standard library |
| 1724 |
support for parsing strings and getting back a :class:`~datetime.date` or |
| 1725 |
:class:`~datetime.datetime`. |
| 1726 |
|
| 1727 |
For more information, refer to the module's reference documentation. |
| 1728 |
(Contributed by Tim Peters.) |
| 1729 |
|
| 1730 |
.. ====================================================================== |
| 1731 |
|
| 1732 |
|
| 1733 |
The optparse Module |
| 1734 |
------------------- |
| 1735 |
|
| 1736 |
The :mod:`getopt` module provides simple parsing of command-line arguments. The |
| 1737 |
new :mod:`optparse` module (originally named Optik) provides more elaborate |
| 1738 |
command-line parsing that follows the Unix conventions, automatically creates |
| 1739 |
the output for :option:`!--help`, and can perform different actions for different |
| 1740 |
options. |
| 1741 |
|
| 1742 |
You start by creating an instance of :class:`~optparse.OptionParser` and telling it what |
| 1743 |
your program's options are. :: |
| 1744 |
|
| 1745 |
import sys |
| 1746 |
from optparse import OptionParser |
| 1747 |
|
| 1748 |
op = OptionParser() |
| 1749 |
op.add_option('-i', '--input', |
| 1750 |
action='store', type='string', dest='input', |
| 1751 |
help='set input filename') |
| 1752 |
op.add_option('-l', '--length', |
| 1753 |
action='store', type='int', dest='length', |
| 1754 |
help='set maximum length of output') |
| 1755 |
|
| 1756 |
Parsing a command line is then done by calling the :meth:`~optparse.OptionParser.parse_args` method. :: |
| 1757 |
|
| 1758 |
options, args = op.parse_args(sys.argv[1:]) |
| 1759 |
print options |
| 1760 |
print args |
| 1761 |
|
| 1762 |
This returns an object containing all of the option values, and a list of |
| 1763 |
strings containing the remaining arguments. |
| 1764 |
|
| 1765 |
Invoking the script with the various arguments now works as you'd expect it to. |
| 1766 |
Note that the length argument is automatically converted to an integer. |
| 1767 |
|
| 1768 |
.. code-block:: shell-session |
| 1769 |
|
| 1770 |
$ ./python opt.py -i data arg1 |
| 1771 |
<Values at 0x400cad4c: {'input': 'data', 'length': None}> |
| 1772 |
['arg1'] |
| 1773 |
$ ./python opt.py --input=data --length=4 |
| 1774 |
<Values at 0x400cad2c: {'input': 'data', 'length': 4}> |
| 1775 |
[] |
| 1776 |
$ |
| 1777 |
|
| 1778 |
The help message is automatically generated for you: |
| 1779 |
|
| 1780 |
.. code-block:: shell-session |
| 1781 |
|
| 1782 |
$ ./python opt.py --help |
| 1783 |
usage: opt.py [options] |
| 1784 |
|
| 1785 |
options: |
| 1786 |
-h, --help show this help message and exit |
| 1787 |
-iINPUT, --input=INPUT |
| 1788 |
set input filename |
| 1789 |
-lLENGTH, --length=LENGTH |
| 1790 |
set maximum length of output |
| 1791 |
$ |
| 1792 |
|
| 1793 |
See the module's documentation for more details. |
| 1794 |
|
| 1795 |
|
| 1796 |
Optik was written by Greg Ward, with suggestions from the readers of the Getopt |
| 1797 |
SIG. |
| 1798 |
|
| 1799 |
.. ====================================================================== |
| 1800 |
|
| 1801 |
|
| 1802 |
.. _section-pymalloc: |
| 1803 |
|
| 1804 |
Pymalloc: A Specialized Object Allocator |
| 1805 |
======================================== |
| 1806 |
|
| 1807 |
Pymalloc, a specialized object allocator written by Vladimir Marangozov, was a |
| 1808 |
feature added to Python 2.1. Pymalloc is intended to be faster than the system |
| 1809 |
:c:func:`malloc` and to have less memory overhead for allocation patterns typical |
| 1810 |
of Python programs. The allocator uses C's :c:func:`malloc` function to get large |
| 1811 |
pools of memory and then fulfills smaller memory requests from these pools. |
| 1812 |
|
| 1813 |
In 2.1 and 2.2, pymalloc was an experimental feature and wasn't enabled by |
| 1814 |
default; you had to explicitly enable it when compiling Python by providing the |
| 1815 |
:option:`!--with-pymalloc` option to the :program:`configure` script. In 2.3, |
| 1816 |
pymalloc has had further enhancements and is now enabled by default; you'll have |
| 1817 |
to supply :option:`!--without-pymalloc` to disable it. |
| 1818 |
|
| 1819 |
This change is transparent to code written in Python; however, pymalloc may |
| 1820 |
expose bugs in C extensions. Authors of C extension modules should test their |
| 1821 |
code with pymalloc enabled, because some incorrect code may cause core dumps at |
| 1822 |
runtime. |
| 1823 |
|
| 1824 |
There's one particularly common error that causes problems. There are a number |
| 1825 |
of memory allocation functions in Python's C API that have previously just been |
| 1826 |
aliases for the C library's :c:func:`malloc` and :c:func:`free`, meaning that if |
| 1827 |
you accidentally called mismatched functions the error wouldn't be noticeable. |
| 1828 |
When the object allocator is enabled, these functions aren't aliases of |
| 1829 |
:c:func:`malloc` and :c:func:`free` any more, and calling the wrong function to |
| 1830 |
free memory may get you a core dump. For example, if memory was allocated using |
| 1831 |
:c:func:`PyObject_Malloc`, it has to be freed using :c:func:`PyObject_Free`, not |
| 1832 |
:c:func:`free`. A few modules included with Python fell afoul of this and had to |
| 1833 |
be fixed; doubtless there are more third-party modules that will have the same |
| 1834 |
problem. |
| 1835 |
|
| 1836 |
As part of this change, the confusing multiple interfaces for allocating memory |
| 1837 |
have been consolidated down into two API families. Memory allocated with one |
| 1838 |
family must not be manipulated with functions from the other family. There is |
| 1839 |
one family for allocating chunks of memory and another family of functions |
| 1840 |
specifically for allocating Python objects. |
| 1841 |
|
| 1842 |
* To allocate and free an undistinguished chunk of memory use the "raw memory" |
| 1843 |
family: :c:func:`PyMem_Malloc`, :c:func:`PyMem_Realloc`, and :c:func:`PyMem_Free`. |
| 1844 |
|
| 1845 |
* The "object memory" family is the interface to the pymalloc facility described |
| 1846 |
above and is biased towards a large number of "small" allocations: |
| 1847 |
:c:func:`PyObject_Malloc`, :c:func:`PyObject_Realloc`, and :c:func:`PyObject_Free`. |
| 1848 |
|
| 1849 |
* To allocate and free Python objects, use the "object" family |
| 1850 |
:c:macro:`PyObject_New`, :c:macro:`PyObject_NewVar`, and :c:func:`PyObject_Del`. |
| 1851 |
|
| 1852 |
Thanks to lots of work by Tim Peters, pymalloc in 2.3 also provides debugging |
| 1853 |
features to catch memory overwrites and doubled frees in both extension modules |
| 1854 |
and in the interpreter itself. To enable this support, compile a debugging |
| 1855 |
version of the Python interpreter by running :program:`configure` with |
| 1856 |
:option:`!--with-pydebug`. |
| 1857 |
|
| 1858 |
To aid extension writers, a header file :file:`Misc/pymemcompat.h` is |
| 1859 |
distributed with the source to Python 2.3 that allows Python extensions to use |
| 1860 |
the 2.3 interfaces to memory allocation while compiling against any version of |
| 1861 |
Python since 1.5.2. You would copy the file from Python's source distribution |
| 1862 |
and bundle it with the source of your extension. |
| 1863 |
|
| 1864 |
|
| 1865 |
.. seealso:: |
| 1866 |
|
| 1867 |
https://hg.python.org/cpython/file/default/Objects/obmalloc.c |
| 1868 |
For the full details of the pymalloc implementation, see the comments at |
| 1869 |
the top of the file :file:`Objects/obmalloc.c` in the Python source code. |
| 1870 |
The above link points to the file within the python.org SVN browser. |
| 1871 |
|
| 1872 |
.. ====================================================================== |
| 1873 |
|
| 1874 |
|
| 1875 |
Build and C API Changes |
| 1876 |
======================= |
| 1877 |
|
| 1878 |
Changes to Python's build process and to the C API include: |
| 1879 |
|
| 1880 |
* The cycle detection implementation used by the garbage collection has proven |
| 1881 |
to be stable, so it's now been made mandatory. You can no longer compile Python |
| 1882 |
without it, and the :option:`!--with-cycle-gc` switch to :program:`configure` has |
| 1883 |
been removed. |
| 1884 |
|
| 1885 |
* Python can now optionally be built as a shared library |
| 1886 |
(:file:`libpython2.3.so`) by supplying :option:`!--enable-shared` when running |
| 1887 |
Python's :program:`configure` script. (Contributed by Ondrej Palkovsky.) |
| 1888 |
|
| 1889 |
* The :c:macro:`!DL_EXPORT` and :c:macro:`!DL_IMPORT` macros are now deprecated. |
| 1890 |
Initialization functions for Python extension modules should now be declared |
| 1891 |
using the new macro :c:macro:`PyMODINIT_FUNC`, while the Python core will |
| 1892 |
generally use the :c:macro:`!PyAPI_FUNC` and :c:macro:`!PyAPI_DATA` macros. |
| 1893 |
|
| 1894 |
* The interpreter can be compiled without any docstrings for the built-in |
| 1895 |
functions and modules by supplying :option:`!--without-doc-strings` to the |
| 1896 |
:program:`configure` script. This makes the Python executable about 10% smaller, |
| 1897 |
but will also mean that you can't get help for Python's built-ins. (Contributed |
| 1898 |
by Gustavo Niemeyer.) |
| 1899 |
|
| 1900 |
* The :c:func:`!PyArg_NoArgs` macro is now deprecated, and code that uses it |
| 1901 |
should be changed. For Python 2.2 and later, the method definition table can |
| 1902 |
specify the :c:macro:`METH_NOARGS` flag, signalling that there are no arguments, |
| 1903 |
and the argument checking can then be removed. If compatibility with pre-2.2 |
| 1904 |
versions of Python is important, the code could use ``PyArg_ParseTuple(args, |
| 1905 |
"")`` instead, but this will be slower than using :c:macro:`METH_NOARGS`. |
| 1906 |
|
| 1907 |
* :c:func:`PyArg_ParseTuple` accepts new format characters for various sizes of |
| 1908 |
unsigned integers: ``B`` for :c:expr:`unsigned char`, ``H`` for :c:expr:`unsigned |
| 1909 |
short int`, ``I`` for :c:expr:`unsigned int`, and ``K`` for :c:expr:`unsigned |
| 1910 |
long long`. |
| 1911 |
|
| 1912 |
* A new function, ``PyObject_DelItemString(mapping, char *key)`` was added |
| 1913 |
as shorthand for ``PyObject_DelItem(mapping, PyString_New(key))``. |
| 1914 |
|
| 1915 |
* File objects now manage their internal string buffer differently, increasing |
| 1916 |
it exponentially when needed. This results in the benchmark tests in |
| 1917 |
:file:`Lib/test/test_bufio.py` speeding up considerably (from 57 seconds to 1.7 |
| 1918 |
seconds, according to one measurement). |
| 1919 |
|
| 1920 |
* It's now possible to define class and static methods for a C extension type by |
| 1921 |
setting either the :c:macro:`METH_CLASS` or :c:macro:`METH_STATIC` flags in a |
| 1922 |
method's :c:type:`PyMethodDef` structure. |
| 1923 |
|
| 1924 |
* Python now includes a copy of the Expat XML parser's source code, removing any |
| 1925 |
dependence on a system version or local installation of Expat. |
| 1926 |
|
| 1927 |
* If you dynamically allocate type objects in your extension, you should be |
| 1928 |
aware of a change in the rules relating to the :attr:`~type.__module__` and |
| 1929 |
:attr:`~type.__name__` attributes. In summary, you will want to ensure the type's |
| 1930 |
dictionary contains a ``'__module__'`` key; making the module name the part of |
| 1931 |
the type name leading up to the final period will no longer have the desired |
| 1932 |
effect. For more detail, read the API reference documentation or the source. |
| 1933 |
|
| 1934 |
.. ====================================================================== |
| 1935 |
|
| 1936 |
|
| 1937 |
Port-Specific Changes |
| 1938 |
--------------------- |
| 1939 |
|
| 1940 |
Support for a port to IBM's OS/2 using the EMX runtime environment was merged |
| 1941 |
into the main Python source tree. EMX is a POSIX emulation layer over the OS/2 |
| 1942 |
system APIs. The Python port for EMX tries to support all the POSIX-like |
| 1943 |
capability exposed by the EMX runtime, and mostly succeeds; :func:`!fork` and |
| 1944 |
:func:`fcntl` are restricted by the limitations of the underlying emulation |
| 1945 |
layer. The standard OS/2 port, which uses IBM's Visual Age compiler, also |
| 1946 |
gained support for case-sensitive import semantics as part of the integration of |
| 1947 |
the EMX port into CVS. (Contributed by Andrew MacIntyre.) |
| 1948 |
|
| 1949 |
On MacOS, most toolbox modules have been weaklinked to improve backward |
| 1950 |
compatibility. This means that modules will no longer fail to load if a single |
| 1951 |
routine is missing on the current OS version. Instead calling the missing |
| 1952 |
routine will raise an exception. (Contributed by Jack Jansen.) |
| 1953 |
|
| 1954 |
The RPM spec files, found in the :file:`Misc/RPM/` directory in the Python |
| 1955 |
source distribution, were updated for 2.3. (Contributed by Sean Reifschneider.) |
| 1956 |
|
| 1957 |
Other new platforms now supported by Python include AtheOS |
| 1958 |
(http://www.atheos.cx/), GNU/Hurd, and OpenVMS. |
| 1959 |
|
| 1960 |
.. ====================================================================== |
| 1961 |
|
| 1962 |
|
| 1963 |
.. _23section-other: |
| 1964 |
|
| 1965 |
Other Changes and Fixes |
| 1966 |
======================= |
| 1967 |
|
| 1968 |
As usual, there were a bunch of other improvements and bugfixes scattered |
| 1969 |
throughout the source tree. A search through the CVS change logs finds there |
| 1970 |
were 523 patches applied and 514 bugs fixed between Python 2.2 and 2.3. Both |
| 1971 |
figures are likely to be underestimates. |
| 1972 |
|
| 1973 |
Some of the more notable changes are: |
| 1974 |
|
| 1975 |
* If the :envvar:`PYTHONINSPECT` environment variable is set, the Python |
| 1976 |
interpreter will enter the interactive prompt after running a Python program, as |
| 1977 |
if Python had been invoked with the :option:`-i` option. The environment |
| 1978 |
variable can be set before running the Python interpreter, or it can be set by |
| 1979 |
the Python program as part of its execution. |
| 1980 |
|
| 1981 |
* The :file:`regrtest.py` script now provides a way to allow "all resources |
| 1982 |
except *foo*." A resource name passed to the :option:`!-u` option can now be |
| 1983 |
prefixed with a hyphen (``'-'``) to mean "remove this resource." For example, |
| 1984 |
the option '``-uall,-bsddb``' could be used to enable the use of all resources |
| 1985 |
except ``bsddb``. |
| 1986 |
|
| 1987 |
* The tools used to build the documentation now work under Cygwin as well as |
| 1988 |
Unix. |
| 1989 |
|
| 1990 |
* The ``SET_LINENO`` opcode has been removed. Back in the mists of time, this |
| 1991 |
opcode was needed to produce line numbers in tracebacks and support trace |
| 1992 |
functions (for, e.g., :mod:`pdb`). Since Python 1.5, the line numbers in |
| 1993 |
tracebacks have been computed using a different mechanism that works with |
| 1994 |
"python -O". For Python 2.3 Michael Hudson implemented a similar scheme to |
| 1995 |
determine when to call the trace function, removing the need for ``SET_LINENO`` |
| 1996 |
entirely. |
| 1997 |
|
| 1998 |
It would be difficult to detect any resulting difference from Python code, apart |
| 1999 |
from a slight speed up when Python is run without :option:`-O`. |
| 2000 |
|
| 2001 |
C extensions that access the :attr:`~frame.f_lineno` field of frame objects should |
| 2002 |
instead call ``PyCode_Addr2Line(f->f_code, f->f_lasti)``. This will have the |
| 2003 |
added effect of making the code work as desired under "python -O" in earlier |
| 2004 |
versions of Python. |
| 2005 |
|
| 2006 |
A nifty new feature is that trace functions can now assign to the |
| 2007 |
:attr:`~frame.f_lineno` attribute of frame objects, changing the line that will be |
| 2008 |
executed next. A ``jump`` command has been added to the :mod:`pdb` debugger |
| 2009 |
taking advantage of this new feature. (Implemented by Richie Hindle.) |
| 2010 |
|
| 2011 |
.. ====================================================================== |
| 2012 |
|
| 2013 |
|
| 2014 |
Porting to Python 2.3 |
| 2015 |
===================== |
| 2016 |
|
| 2017 |
This section lists previously described changes that may require changes to your |
| 2018 |
code: |
| 2019 |
|
| 2020 |
* :keyword:`yield` is now always a keyword; if it's used as a variable name in |
| 2021 |
your code, a different name must be chosen. |
| 2022 |
|
| 2023 |
* For strings *X* and *Y*, ``X in Y`` now works if *X* is more than one |
| 2024 |
character long. |
| 2025 |
|
| 2026 |
* The :func:`int` type constructor will now return a long integer instead of |
| 2027 |
raising an :exc:`OverflowError` when a string or floating-point number is too |
| 2028 |
large to fit into an integer. |
| 2029 |
|
| 2030 |
* If you have Unicode strings that contain 8-bit characters, you must declare |
| 2031 |
the file's encoding (UTF-8, Latin-1, or whatever) by adding a comment to the top |
| 2032 |
of the file. See section :ref:`section-encodings` for more information. |
| 2033 |
|
| 2034 |
* Calling Tcl methods through :mod:`!_tkinter` no longer returns only strings. |
| 2035 |
Instead, if Tcl returns other objects those objects are converted to their |
| 2036 |
Python equivalent, if one exists, or wrapped with a :class:`!_tkinter.Tcl_Obj` |
| 2037 |
object if no Python equivalent exists. |
| 2038 |
|
| 2039 |
* Large octal and hex literals such as ``0xffffffff`` now trigger a |
| 2040 |
:exc:`FutureWarning`. Currently they're stored as 32-bit numbers and result in a |
| 2041 |
negative value, but in Python 2.4 they'll become positive long integers. |
| 2042 |
|
| 2043 |
There are a few ways to fix this warning. If you really need a positive number, |
| 2044 |
just add an ``L`` to the end of the literal. If you're trying to get a 32-bit |
| 2045 |
integer with low bits set and have previously used an expression such as ``~(1 |
| 2046 |
<< 31)``, it's probably clearest to start with all bits set and clear the |
| 2047 |
desired upper bits. For example, to clear just the top bit (bit 31), you could |
| 2048 |
write ``0xffffffffL &~(1L<<31)``. |
| 2049 |
|
| 2050 |
* You can no longer disable assertions by assigning to ``__debug__``. |
| 2051 |
|
| 2052 |
* The Distutils :func:`!setup` function has gained various new keyword arguments |
| 2053 |
such as *depends*. Old versions of the Distutils will abort if passed unknown |
| 2054 |
keywords. A solution is to check for the presence of the new |
| 2055 |
:func:`!get_distutil_options` function in your :file:`setup.py` and only uses the |
| 2056 |
new keywords with a version of the Distutils that supports them:: |
| 2057 |
|
| 2058 |
from distutils import core |
| 2059 |
|
| 2060 |
kw = {'sources': 'foo.c', ...} |
| 2061 |
if hasattr(core, 'get_distutil_options'): |
| 2062 |
kw['depends'] = ['foo.h'] |
| 2063 |
ext = Extension(**kw) |
| 2064 |
|
| 2065 |
* Using ``None`` as a variable name will now result in a :exc:`SyntaxWarning` |
| 2066 |
warning. |
| 2067 |
|
| 2068 |
* Names of extension types defined by the modules included with Python now |
| 2069 |
contain the module and a ``'.'`` in front of the type name. |
| 2070 |
|
| 2071 |
.. ====================================================================== |
| 2072 |
|
| 2073 |
|
| 2074 |
.. _23acks: |
| 2075 |
|
| 2076 |
Acknowledgements |
| 2077 |
================ |
| 2078 |
|
| 2079 |
The author would like to thank the following people for offering suggestions, |
| 2080 |
corrections and assistance with various drafts of this article: Jeff Bauer, |
| 2081 |
Simon Brunning, Brett Cannon, Michael Chermside, Andrew Dalke, Scott David |
| 2082 |
Daniels, Fred L. Drake, Jr., David Fraser, Kelly Gerber, Raymond Hettinger, |
| 2083 |
Michael Hudson, Chris Lambert, Detlef Lannert, Martin von Löwis, Andrew |
| 2084 |
MacIntyre, Lalo Martins, Chad Netzer, Gustavo Niemeyer, Neal Norwitz, Hans |
| 2085 |
Nowak, Chris Reedy, Francesco Ricciardi, Vinay Sajip, Neil Schemenauer, Roman |
| 2086 |
Suzi, Jason Tishler, Just van Rossum. |