| 1 |
**************************** |
| 2 |
What's New in Python 2.4 |
| 3 |
**************************** |
| 4 |
|
| 5 |
:Author: A.M. Kuchling |
| 6 |
|
| 7 |
.. |release| replace:: 1.02 |
| 8 |
|
| 9 |
.. $Id: whatsnew24.tex 54632 2007-03-31 11:59:54Z georg.brandl $ |
| 10 |
.. Don't write extensive text for new sections; I'll do that. |
| 11 |
.. Feel free to add commented-out reminders of things that need |
| 12 |
.. to be covered. --amk |
| 13 |
|
| 14 |
This article explains the new features in Python 2.4.1, released on March 30, |
| 15 |
2005. |
| 16 |
|
| 17 |
Python 2.4 is a medium-sized release. It doesn't introduce as many changes as |
| 18 |
the radical Python 2.2, but introduces more features than the conservative 2.3 |
| 19 |
release. The most significant new language features are function decorators and |
| 20 |
generator expressions; most other changes are to the standard library. |
| 21 |
|
| 22 |
According to the CVS change logs, there were 481 patches applied and 502 bugs |
| 23 |
fixed between Python 2.3 and 2.4. Both figures are likely to be underestimates. |
| 24 |
|
| 25 |
This article doesn't attempt to provide a complete specification of every single |
| 26 |
new feature, but instead provides a brief introduction to each feature. For |
| 27 |
full details, you should refer to the documentation for Python 2.4, such as the |
| 28 |
Python Library Reference and the Python Reference Manual. Often you will be |
| 29 |
referred to the PEP for a particular new feature for explanations of the |
| 30 |
implementation and design rationale. |
| 31 |
|
| 32 |
.. ====================================================================== |
| 33 |
|
| 34 |
|
| 35 |
PEP 218: Built-In Set Objects |
| 36 |
============================= |
| 37 |
|
| 38 |
Python 2.3 introduced the :mod:`sets` module. C implementations of set data |
| 39 |
types have now been added to the Python core as two new built-in types, |
| 40 |
``set(iterable)`` and ``frozenset(iterable)``. They provide high speed |
| 41 |
operations for membership testing, for eliminating duplicates from sequences, |
| 42 |
and for mathematical operations like unions, intersections, differences, and |
| 43 |
symmetric differences. :: |
| 44 |
|
| 45 |
>>> a = set('abracadabra') # form a set from a string |
| 46 |
>>> 'z' in a # fast membership testing |
| 47 |
False |
| 48 |
>>> a # unique letters in a |
| 49 |
set(['a', 'r', 'b', 'c', 'd']) |
| 50 |
>>> ''.join(a) # convert back into a string |
| 51 |
'arbcd' |
| 52 |
|
| 53 |
>>> b = set('alacazam') # form a second set |
| 54 |
>>> a - b # letters in a but not in b |
| 55 |
set(['r', 'd', 'b']) |
| 56 |
>>> a | b # letters in either a or b |
| 57 |
set(['a', 'c', 'r', 'd', 'b', 'm', 'z', 'l']) |
| 58 |
>>> a & b # letters in both a and b |
| 59 |
set(['a', 'c']) |
| 60 |
>>> a ^ b # letters in a or b but not both |
| 61 |
set(['r', 'd', 'b', 'm', 'z', 'l']) |
| 62 |
|
| 63 |
>>> a.add('z') # add a new element |
| 64 |
>>> a.update('wxy') # add multiple new elements |
| 65 |
>>> a |
| 66 |
set(['a', 'c', 'b', 'd', 'r', 'w', 'y', 'x', 'z']) |
| 67 |
>>> a.remove('x') # take one element out |
| 68 |
>>> a |
| 69 |
set(['a', 'c', 'b', 'd', 'r', 'w', 'y', 'z']) |
| 70 |
|
| 71 |
The :func:`frozenset` type is an immutable version of :func:`set`. Since it is |
| 72 |
immutable and hashable, it may be used as a dictionary key or as a member of |
| 73 |
another set. |
| 74 |
|
| 75 |
The :mod:`sets` module remains in the standard library, and may be useful if you |
| 76 |
wish to subclass the :class:`Set` or :class:`ImmutableSet` classes. There are |
| 77 |
currently no plans to deprecate the module. |
| 78 |
|
| 79 |
|
| 80 |
.. seealso:: |
| 81 |
|
| 82 |
:pep:`218` - Adding a Built-In Set Object Type |
| 83 |
Originally proposed by Greg Wilson and ultimately implemented by Raymond |
| 84 |
Hettinger. |
| 85 |
|
| 86 |
.. ====================================================================== |
| 87 |
|
| 88 |
|
| 89 |
PEP 237: Unifying Long Integers and Integers |
| 90 |
============================================ |
| 91 |
|
| 92 |
The lengthy transition process for this PEP, begun in Python 2.2, takes another |
| 93 |
step forward in Python 2.4. In 2.3, certain integer operations that would |
| 94 |
behave differently after int/long unification triggered :exc:`FutureWarning` |
| 95 |
warnings and returned values limited to 32 or 64 bits (depending on your |
| 96 |
platform). In 2.4, these expressions no longer produce a warning and instead |
| 97 |
produce a different result that's usually a long integer. |
| 98 |
|
| 99 |
The problematic expressions are primarily left shifts and lengthy hexadecimal |
| 100 |
and octal constants. For example, ``2 << 32`` results in a warning in 2.3, |
| 101 |
evaluating to 0 on 32-bit platforms. In Python 2.4, this expression now returns |
| 102 |
the correct answer, 8589934592. |
| 103 |
|
| 104 |
|
| 105 |
.. seealso:: |
| 106 |
|
| 107 |
:pep:`237` - Unifying Long Integers and Integers |
| 108 |
Original PEP written by Moshe Zadka and GvR. The changes for 2.4 were |
| 109 |
implemented by Kalle Svensson. |
| 110 |
|
| 111 |
.. ====================================================================== |
| 112 |
|
| 113 |
|
| 114 |
PEP 289: Generator Expressions |
| 115 |
============================== |
| 116 |
|
| 117 |
The iterator feature introduced in Python 2.2 and the :mod:`itertools` module |
| 118 |
make it easier to write programs that loop through large data sets without |
| 119 |
having the entire data set in memory at one time. List comprehensions don't fit |
| 120 |
into this picture very well because they produce a Python list object containing |
| 121 |
all of the items. This unavoidably pulls all of the objects into memory, which |
| 122 |
can be a problem if your data set is very large. When trying to write a |
| 123 |
functionally styled program, it would be natural to write something like:: |
| 124 |
|
| 125 |
links = [link for link in get_all_links() if not link.followed] |
| 126 |
for link in links: |
| 127 |
... |
| 128 |
|
| 129 |
instead of :: |
| 130 |
|
| 131 |
for link in get_all_links(): |
| 132 |
if link.followed: |
| 133 |
continue |
| 134 |
... |
| 135 |
|
| 136 |
The first form is more concise and perhaps more readable, but if you're dealing |
| 137 |
with a large number of link objects you'd have to write the second form to avoid |
| 138 |
having all link objects in memory at the same time. |
| 139 |
|
| 140 |
Generator expressions work similarly to list comprehensions but don't |
| 141 |
materialize the entire list; instead they create a generator that will return |
| 142 |
elements one by one. The above example could be written as:: |
| 143 |
|
| 144 |
links = (link for link in get_all_links() if not link.followed) |
| 145 |
for link in links: |
| 146 |
... |
| 147 |
|
| 148 |
Generator expressions always have to be written inside parentheses, as in the |
| 149 |
above example. The parentheses signalling a function call also count, so if you |
| 150 |
want to create an iterator that will be immediately passed to a function you |
| 151 |
could write:: |
| 152 |
|
| 153 |
print sum(obj.count for obj in list_all_objects()) |
| 154 |
|
| 155 |
Generator expressions differ from list comprehensions in various small ways. |
| 156 |
Most notably, the loop variable (*obj* in the above example) is not accessible |
| 157 |
outside of the generator expression. List comprehensions leave the variable |
| 158 |
assigned to its last value; future versions of Python will change this, making |
| 159 |
list comprehensions match generator expressions in this respect. |
| 160 |
|
| 161 |
|
| 162 |
.. seealso:: |
| 163 |
|
| 164 |
:pep:`289` - Generator Expressions |
| 165 |
Proposed by Raymond Hettinger and implemented by Jiwon Seo with early efforts |
| 166 |
steered by Hye-Shik Chang. |
| 167 |
|
| 168 |
.. ====================================================================== |
| 169 |
|
| 170 |
|
| 171 |
PEP 292: Simpler String Substitutions |
| 172 |
===================================== |
| 173 |
|
| 174 |
Some new classes in the standard library provide an alternative mechanism for |
| 175 |
substituting variables into strings; this style of substitution may be better |
| 176 |
for applications where untrained users need to edit templates. |
| 177 |
|
| 178 |
The usual way of substituting variables by name is the ``%`` operator:: |
| 179 |
|
| 180 |
>>> '%(page)i: %(title)s' % {'page':2, 'title': 'The Best of Times'} |
| 181 |
'2: The Best of Times' |
| 182 |
|
| 183 |
When writing the template string, it can be easy to forget the ``i`` or ``s`` |
| 184 |
after the closing parenthesis. This isn't a big problem if the template is in a |
| 185 |
Python module, because you run the code, get an "Unsupported format character" |
| 186 |
:exc:`ValueError`, and fix the problem. However, consider an application such |
| 187 |
as Mailman where template strings or translations are being edited by users who |
| 188 |
aren't aware of the Python language. The format string's syntax is complicated |
| 189 |
to explain to such users, and if they make a mistake, it's difficult to provide |
| 190 |
helpful feedback to them. |
| 191 |
|
| 192 |
PEP 292 adds a :class:`Template` class to the :mod:`string` module that uses |
| 193 |
``$`` to indicate a substitution:: |
| 194 |
|
| 195 |
>>> import string |
| 196 |
>>> t = string.Template('$page: $title') |
| 197 |
>>> t.substitute({'page':2, 'title': 'The Best of Times'}) |
| 198 |
'2: The Best of Times' |
| 199 |
|
| 200 |
If a key is missing from the dictionary, the :meth:`substitute` method will |
| 201 |
raise a :exc:`KeyError`. There's also a :meth:`safe_substitute` method that |
| 202 |
ignores missing keys:: |
| 203 |
|
| 204 |
>>> t = string.Template('$page: $title') |
| 205 |
>>> t.safe_substitute({'page':3}) |
| 206 |
'3: $title' |
| 207 |
|
| 208 |
|
| 209 |
.. seealso:: |
| 210 |
|
| 211 |
:pep:`292` - Simpler String Substitutions |
| 212 |
Written and implemented by Barry Warsaw. |
| 213 |
|
| 214 |
.. ====================================================================== |
| 215 |
|
| 216 |
|
| 217 |
PEP 318: Decorators for Functions and Methods |
| 218 |
============================================= |
| 219 |
|
| 220 |
Python 2.2 extended Python's object model by adding static methods and class |
| 221 |
methods, but it didn't extend Python's syntax to provide any new way of defining |
| 222 |
static or class methods. Instead, you had to write a :keyword:`def` statement |
| 223 |
in the usual way, and pass the resulting method to a :func:`staticmethod` or |
| 224 |
:func:`classmethod` function that would wrap up the function as a method of the |
| 225 |
new type. Your code would look like this:: |
| 226 |
|
| 227 |
class C: |
| 228 |
def meth (cls): |
| 229 |
... |
| 230 |
|
| 231 |
meth = classmethod(meth) # Rebind name to wrapped-up class method |
| 232 |
|
| 233 |
If the method was very long, it would be easy to miss or forget the |
| 234 |
:func:`classmethod` invocation after the function body. |
| 235 |
|
| 236 |
The intention was always to add some syntax to make such definitions more |
| 237 |
readable, but at the time of 2.2's release a good syntax was not obvious. Today |
| 238 |
a good syntax *still* isn't obvious but users are asking for easier access to |
| 239 |
the feature; a new syntactic feature has been added to meet this need. |
| 240 |
|
| 241 |
The new feature is called "function decorators". The name comes from the idea |
| 242 |
that :func:`classmethod`, :func:`staticmethod`, and friends are storing |
| 243 |
additional information on a function object; they're *decorating* functions with |
| 244 |
more details. |
| 245 |
|
| 246 |
The notation borrows from Java and uses the ``'@'`` character as an indicator. |
| 247 |
Using the new syntax, the example above would be written:: |
| 248 |
|
| 249 |
class C: |
| 250 |
|
| 251 |
@classmethod |
| 252 |
def meth (cls): |
| 253 |
... |
| 254 |
|
| 255 |
|
| 256 |
The ``@classmethod`` is shorthand for the ``meth=classmethod(meth)`` assignment. |
| 257 |
More generally, if you have the following:: |
| 258 |
|
| 259 |
@A |
| 260 |
@B |
| 261 |
@C |
| 262 |
def f (): |
| 263 |
... |
| 264 |
|
| 265 |
It's equivalent to the following pre-decorator code:: |
| 266 |
|
| 267 |
def f(): ... |
| 268 |
f = A(B(C(f))) |
| 269 |
|
| 270 |
Decorators must come on the line before a function definition, one decorator per |
| 271 |
line, and can't be on the same line as the def statement, meaning that ``@A def |
| 272 |
f(): ...`` is illegal. You can only decorate function definitions, either at |
| 273 |
the module level or inside a class; you can't decorate class definitions. |
| 274 |
|
| 275 |
A decorator is just a function that takes the function to be decorated as an |
| 276 |
argument and returns either the same function or some new object. The return |
| 277 |
value of the decorator need not be callable (though it typically is), unless |
| 278 |
further decorators will be applied to the result. It's easy to write your own |
| 279 |
decorators. The following simple example just sets an attribute on the function |
| 280 |
object:: |
| 281 |
|
| 282 |
>>> def deco(func): |
| 283 |
... func.attr = 'decorated' |
| 284 |
... return func |
| 285 |
... |
| 286 |
>>> @deco |
| 287 |
... def f(): pass |
| 288 |
... |
| 289 |
>>> f |
| 290 |
<function f at 0x402ef0d4> |
| 291 |
>>> f.attr |
| 292 |
'decorated' |
| 293 |
>>> |
| 294 |
|
| 295 |
As a slightly more realistic example, the following decorator checks that the |
| 296 |
supplied argument is an integer:: |
| 297 |
|
| 298 |
def require_int (func): |
| 299 |
def wrapper (arg): |
| 300 |
assert isinstance(arg, int) |
| 301 |
return func(arg) |
| 302 |
|
| 303 |
return wrapper |
| 304 |
|
| 305 |
@require_int |
| 306 |
def p1 (arg): |
| 307 |
print arg |
| 308 |
|
| 309 |
@require_int |
| 310 |
def p2(arg): |
| 311 |
print arg*2 |
| 312 |
|
| 313 |
An example in :pep:`318` contains a fancier version of this idea that lets you |
| 314 |
both specify the required type and check the returned type. |
| 315 |
|
| 316 |
Decorator functions can take arguments. If arguments are supplied, your |
| 317 |
decorator function is called with only those arguments and must return a new |
| 318 |
decorator function; this function must take a single function and return a |
| 319 |
function, as previously described. In other words, ``@A @B @C(args)`` becomes:: |
| 320 |
|
| 321 |
def f(): ... |
| 322 |
_deco = C(args) |
| 323 |
f = A(B(_deco(f))) |
| 324 |
|
| 325 |
Getting this right can be slightly brain-bending, but it's not too difficult. |
| 326 |
|
| 327 |
A small related change makes the :attr:`func_name <function.__name__>` |
| 328 |
attribute of functions |
| 329 |
writable. This attribute is used to display function names in tracebacks, so |
| 330 |
decorators should change the name of any new function that's constructed and |
| 331 |
returned. |
| 332 |
|
| 333 |
|
| 334 |
.. seealso:: |
| 335 |
|
| 336 |
:pep:`318` - Decorators for Functions, Methods and Classes |
| 337 |
Written by Kevin D. Smith, Jim Jewett, and Skip Montanaro. Several people |
| 338 |
wrote patches implementing function decorators, but the one that was actually |
| 339 |
checked in was patch #979728, written by Mark Russell. |
| 340 |
|
| 341 |
https://wiki.python.org/moin/PythonDecoratorLibrary |
| 342 |
This Wiki page contains several examples of decorators. |
| 343 |
|
| 344 |
.. ====================================================================== |
| 345 |
|
| 346 |
|
| 347 |
PEP 322: Reverse Iteration |
| 348 |
========================== |
| 349 |
|
| 350 |
A new built-in function, ``reversed(seq)``, takes a sequence and returns an |
| 351 |
iterator that loops over the elements of the sequence in reverse order. :: |
| 352 |
|
| 353 |
>>> for i in reversed(xrange(1,4)): |
| 354 |
... print i |
| 355 |
... |
| 356 |
3 |
| 357 |
2 |
| 358 |
1 |
| 359 |
|
| 360 |
Compared to extended slicing, such as ``range(1,4)[::-1]``, :func:`reversed` is |
| 361 |
easier to read, runs faster, and uses substantially less memory. |
| 362 |
|
| 363 |
Note that :func:`reversed` only accepts sequences, not arbitrary iterators. If |
| 364 |
you want to reverse an iterator, first convert it to a list with :func:`list`. |
| 365 |
:: |
| 366 |
|
| 367 |
>>> input = open('/etc/passwd', 'r') |
| 368 |
>>> for line in reversed(list(input)): |
| 369 |
... print line |
| 370 |
... |
| 371 |
root:*:0:0:System Administrator:/var/root:/bin/tcsh |
| 372 |
... |
| 373 |
|
| 374 |
|
| 375 |
.. seealso:: |
| 376 |
|
| 377 |
:pep:`322` - Reverse Iteration |
| 378 |
Written and implemented by Raymond Hettinger. |
| 379 |
|
| 380 |
.. ====================================================================== |
| 381 |
|
| 382 |
|
| 383 |
PEP 324: New subprocess Module |
| 384 |
============================== |
| 385 |
|
| 386 |
The standard library provides a number of ways to execute a subprocess, offering |
| 387 |
different features and different levels of complexity. |
| 388 |
``os.system(command)`` is easy to use, but slow (it runs a shell process |
| 389 |
which executes the command) and dangerous (you have to be careful about escaping |
| 390 |
the shell's metacharacters). The :mod:`!popen2` module offers classes that can |
| 391 |
capture standard output and standard error from the subprocess, but the naming |
| 392 |
is confusing. The :mod:`subprocess` module cleans this up, providing a unified |
| 393 |
interface that offers all the features you might need. |
| 394 |
|
| 395 |
Instead of :mod:`!popen2`'s collection of classes, :mod:`subprocess` contains a |
| 396 |
single class called :class:`subprocess.Popen` whose constructor supports a number of |
| 397 |
different keyword arguments. :: |
| 398 |
|
| 399 |
class Popen(args, bufsize=0, executable=None, |
| 400 |
stdin=None, stdout=None, stderr=None, |
| 401 |
preexec_fn=None, close_fds=False, shell=False, |
| 402 |
cwd=None, env=None, universal_newlines=False, |
| 403 |
startupinfo=None, creationflags=0): |
| 404 |
|
| 405 |
*args* is commonly a sequence of strings that will be the arguments to the |
| 406 |
program executed as the subprocess. (If the *shell* argument is true, *args* |
| 407 |
can be a string which will then be passed on to the shell for interpretation, |
| 408 |
just as :func:`os.system` does.) |
| 409 |
|
| 410 |
*stdin*, *stdout*, and *stderr* specify what the subprocess's input, output, and |
| 411 |
error streams will be. You can provide a file object or a file descriptor, or |
| 412 |
you can use the constant ``subprocess.PIPE`` to create a pipe between the |
| 413 |
subprocess and the parent. |
| 414 |
|
| 415 |
.. index:: |
| 416 |
single: universal newlines; What's new |
| 417 |
|
| 418 |
The constructor has a number of handy options: |
| 419 |
|
| 420 |
* *close_fds* requests that all file descriptors be closed before running the |
| 421 |
subprocess. |
| 422 |
|
| 423 |
* *cwd* specifies the working directory in which the subprocess will be executed |
| 424 |
(defaulting to whatever the parent's working directory is). |
| 425 |
|
| 426 |
* *env* is a dictionary specifying environment variables. |
| 427 |
|
| 428 |
* *preexec_fn* is a function that gets called before the child is started. |
| 429 |
|
| 430 |
* *universal_newlines* opens the child's input and output using Python's |
| 431 |
:term:`universal newlines` feature. |
| 432 |
|
| 433 |
Once you've created the :class:`Popen` instance, you can call its :meth:`wait` |
| 434 |
method to pause until the subprocess has exited, :meth:`poll` to check if it's |
| 435 |
exited without pausing, or ``communicate(data)`` to send the string *data* |
| 436 |
to the subprocess's standard input. ``communicate(data)`` then reads any |
| 437 |
data that the subprocess has sent to its standard output or standard error, |
| 438 |
returning a tuple ``(stdout_data, stderr_data)``. |
| 439 |
|
| 440 |
:func:`call` is a shortcut that passes its arguments along to the :class:`Popen` |
| 441 |
constructor, waits for the command to complete, and returns the status code of |
| 442 |
the subprocess. It can serve as a safer analog to :func:`os.system`:: |
| 443 |
|
| 444 |
sts = subprocess.call(['dpkg', '-i', '/tmp/new-package.deb']) |
| 445 |
if sts == 0: |
| 446 |
# Success |
| 447 |
... |
| 448 |
else: |
| 449 |
# dpkg returned an error |
| 450 |
... |
| 451 |
|
| 452 |
The command is invoked without use of the shell. If you really do want to use |
| 453 |
the shell, you can add ``shell=True`` as a keyword argument and provide a string |
| 454 |
instead of a sequence:: |
| 455 |
|
| 456 |
sts = subprocess.call('dpkg -i /tmp/new-package.deb', shell=True) |
| 457 |
|
| 458 |
The PEP takes various examples of shell and Python code and shows how they'd be |
| 459 |
translated into Python code that uses :mod:`subprocess`. Reading this section |
| 460 |
of the PEP is highly recommended. |
| 461 |
|
| 462 |
|
| 463 |
.. seealso:: |
| 464 |
|
| 465 |
:pep:`324` - subprocess - New process module |
| 466 |
Written and implemented by Peter Åstrand, with assistance from Fredrik Lundh and |
| 467 |
others. |
| 468 |
|
| 469 |
.. ====================================================================== |
| 470 |
|
| 471 |
|
| 472 |
PEP 327: Decimal Data Type |
| 473 |
========================== |
| 474 |
|
| 475 |
Python has always supported floating-point (FP) numbers, based on the underlying |
| 476 |
C :c:expr:`double` type, as a data type. However, while most programming |
| 477 |
languages provide a floating-point type, many people (even programmers) are |
| 478 |
unaware that floating-point numbers don't represent certain decimal fractions |
| 479 |
accurately. The new :class:`Decimal` type can represent these fractions |
| 480 |
accurately, up to a user-specified precision limit. |
| 481 |
|
| 482 |
|
| 483 |
Why is Decimal needed? |
| 484 |
---------------------- |
| 485 |
|
| 486 |
The limitations arise from the representation used for floating-point numbers. |
| 487 |
FP numbers are made up of three components: |
| 488 |
|
| 489 |
* The sign, which is positive or negative. |
| 490 |
|
| 491 |
* The mantissa, which is a single-digit binary number followed by a fractional |
| 492 |
part. For example, ``1.01`` in base-2 notation is ``1 + 0/2 + 1/4``, or 1.25 in |
| 493 |
decimal notation. |
| 494 |
|
| 495 |
* The exponent, which tells where the decimal point is located in the number |
| 496 |
represented. |
| 497 |
|
| 498 |
For example, the number 1.25 has positive sign, a mantissa value of 1.01 (in |
| 499 |
binary), and an exponent of 0 (the decimal point doesn't need to be shifted). |
| 500 |
The number 5 has the same sign and mantissa, but the exponent is 2 because the |
| 501 |
mantissa is multiplied by 4 (2 to the power of the exponent 2); 1.25 \* 4 equals |
| 502 |
5. |
| 503 |
|
| 504 |
Modern systems usually provide floating-point support that conforms to a |
| 505 |
standard called IEEE 754. C's :c:expr:`double` type is usually implemented as a |
| 506 |
64-bit IEEE 754 number, which uses 52 bits of space for the mantissa. This |
| 507 |
means that numbers can only be specified to 52 bits of precision. If you're |
| 508 |
trying to represent numbers whose expansion repeats endlessly, the expansion is |
| 509 |
cut off after 52 bits. Unfortunately, most software needs to produce output in |
| 510 |
base 10, and common fractions in base 10 are often repeating decimals in binary. |
| 511 |
For example, 1.1 decimal is binary ``1.0001100110011 ...``; .1 = 1/16 + 1/32 + |
| 512 |
1/256 plus an infinite number of additional terms. IEEE 754 has to chop off |
| 513 |
that infinitely repeated decimal after 52 digits, so the representation is |
| 514 |
slightly inaccurate. |
| 515 |
|
| 516 |
Sometimes you can see this inaccuracy when the number is printed:: |
| 517 |
|
| 518 |
>>> 1.1 |
| 519 |
1.1000000000000001 |
| 520 |
|
| 521 |
The inaccuracy isn't always visible when you print the number because the |
| 522 |
FP-to-decimal-string conversion is provided by the C library, and most C libraries try |
| 523 |
to produce sensible output. Even if it's not displayed, however, the inaccuracy |
| 524 |
is still there and subsequent operations can magnify the error. |
| 525 |
|
| 526 |
For many applications this doesn't matter. If I'm plotting points and |
| 527 |
displaying them on my monitor, the difference between 1.1 and 1.1000000000000001 |
| 528 |
is too small to be visible. Reports often limit output to a certain number of |
| 529 |
decimal places, and if you round the number to two or three or even eight |
| 530 |
decimal places, the error is never apparent. However, for applications where it |
| 531 |
does matter, it's a lot of work to implement your own custom arithmetic |
| 532 |
routines. |
| 533 |
|
| 534 |
Hence, the :class:`Decimal` type was created. |
| 535 |
|
| 536 |
|
| 537 |
The :class:`Decimal` type |
| 538 |
------------------------- |
| 539 |
|
| 540 |
A new module, :mod:`decimal`, was added to Python's standard library. It |
| 541 |
contains two classes, :class:`Decimal` and :class:`Context`. :class:`Decimal` |
| 542 |
instances represent numbers, and :class:`Context` instances are used to wrap up |
| 543 |
various settings such as the precision and default rounding mode. |
| 544 |
|
| 545 |
:class:`Decimal` instances are immutable, like regular Python integers and FP |
| 546 |
numbers; once it's been created, you can't change the value an instance |
| 547 |
represents. :class:`Decimal` instances can be created from integers or |
| 548 |
strings:: |
| 549 |
|
| 550 |
>>> import decimal |
| 551 |
>>> decimal.Decimal(1972) |
| 552 |
Decimal("1972") |
| 553 |
>>> decimal.Decimal("1.1") |
| 554 |
Decimal("1.1") |
| 555 |
|
| 556 |
You can also provide tuples containing the sign, the mantissa represented as a |
| 557 |
tuple of decimal digits, and the exponent:: |
| 558 |
|
| 559 |
>>> decimal.Decimal((1, (1, 4, 7, 5), -2)) |
| 560 |
Decimal("-14.75") |
| 561 |
|
| 562 |
Cautionary note: the sign bit is a Boolean value, so 0 is positive and 1 is |
| 563 |
negative. |
| 564 |
|
| 565 |
Converting from floating-point numbers poses a bit of a problem: should the FP |
| 566 |
number representing 1.1 turn into the decimal number for exactly 1.1, or for 1.1 |
| 567 |
plus whatever inaccuracies are introduced? The decision was to dodge the issue |
| 568 |
and leave such a conversion out of the API. Instead, you should convert the |
| 569 |
floating-point number into a string using the desired precision and pass the |
| 570 |
string to the :class:`Decimal` constructor:: |
| 571 |
|
| 572 |
>>> f = 1.1 |
| 573 |
>>> decimal.Decimal(str(f)) |
| 574 |
Decimal("1.1") |
| 575 |
>>> decimal.Decimal('%.12f' % f) |
| 576 |
Decimal("1.100000000000") |
| 577 |
|
| 578 |
Once you have :class:`Decimal` instances, you can perform the usual mathematical |
| 579 |
operations on them. One limitation: exponentiation requires an integer |
| 580 |
exponent:: |
| 581 |
|
| 582 |
>>> a = decimal.Decimal('35.72') |
| 583 |
>>> b = decimal.Decimal('1.73') |
| 584 |
>>> a+b |
| 585 |
Decimal("37.45") |
| 586 |
>>> a-b |
| 587 |
Decimal("33.99") |
| 588 |
>>> a*b |
| 589 |
Decimal("61.7956") |
| 590 |
>>> a/b |
| 591 |
Decimal("20.64739884393063583815028902") |
| 592 |
>>> a ** 2 |
| 593 |
Decimal("1275.9184") |
| 594 |
>>> a**b |
| 595 |
Traceback (most recent call last): |
| 596 |
... |
| 597 |
decimal.InvalidOperation: x ** (non-integer) |
| 598 |
|
| 599 |
You can combine :class:`Decimal` instances with integers, but not with |
| 600 |
floating-point numbers:: |
| 601 |
|
| 602 |
>>> a + 4 |
| 603 |
Decimal("39.72") |
| 604 |
>>> a + 4.5 |
| 605 |
Traceback (most recent call last): |
| 606 |
... |
| 607 |
TypeError: You can interact Decimal only with int, long or Decimal data types. |
| 608 |
>>> |
| 609 |
|
| 610 |
:class:`Decimal` numbers can be used with the :mod:`math` and :mod:`cmath` |
| 611 |
modules, but note that they'll be immediately converted to floating-point |
| 612 |
numbers before the operation is performed, resulting in a possible loss of |
| 613 |
precision and accuracy. You'll also get back a regular floating-point number |
| 614 |
and not a :class:`Decimal`. :: |
| 615 |
|
| 616 |
>>> import math, cmath |
| 617 |
>>> d = decimal.Decimal('123456789012.345') |
| 618 |
>>> math.sqrt(d) |
| 619 |
351364.18288201344 |
| 620 |
>>> cmath.sqrt(-d) |
| 621 |
351364.18288201344j |
| 622 |
|
| 623 |
:class:`Decimal` instances have a :meth:`sqrt` method that returns a |
| 624 |
:class:`Decimal`, but if you need other things such as trigonometric functions |
| 625 |
you'll have to implement them. :: |
| 626 |
|
| 627 |
>>> d.sqrt() |
| 628 |
Decimal("351364.1828820134592177245001") |
| 629 |
|
| 630 |
|
| 631 |
The :class:`Context` type |
| 632 |
------------------------- |
| 633 |
|
| 634 |
Instances of the :class:`Context` class encapsulate several settings for |
| 635 |
decimal operations: |
| 636 |
|
| 637 |
* :attr:`prec` is the precision, the number of decimal places. |
| 638 |
|
| 639 |
* :attr:`rounding` specifies the rounding mode. The :mod:`decimal` module has |
| 640 |
constants for the various possibilities: :const:`ROUND_DOWN`, |
| 641 |
:const:`ROUND_CEILING`, :const:`ROUND_HALF_EVEN`, and various others. |
| 642 |
|
| 643 |
* :attr:`traps` is a dictionary specifying what happens on encountering certain |
| 644 |
error conditions: either an exception is raised or a value is returned. Some |
| 645 |
examples of error conditions are division by zero, loss of precision, and |
| 646 |
overflow. |
| 647 |
|
| 648 |
There's a thread-local default context available by calling :func:`getcontext`; |
| 649 |
you can change the properties of this context to alter the default precision, |
| 650 |
rounding, or trap handling. The following example shows the effect of changing |
| 651 |
the precision of the default context:: |
| 652 |
|
| 653 |
>>> decimal.getcontext().prec |
| 654 |
28 |
| 655 |
>>> decimal.Decimal(1) / decimal.Decimal(7) |
| 656 |
Decimal("0.1428571428571428571428571429") |
| 657 |
>>> decimal.getcontext().prec = 9 |
| 658 |
>>> decimal.Decimal(1) / decimal.Decimal(7) |
| 659 |
Decimal("0.142857143") |
| 660 |
|
| 661 |
The default action for error conditions is selectable; the module can either |
| 662 |
return a special value such as infinity or not-a-number, or exceptions can be |
| 663 |
raised:: |
| 664 |
|
| 665 |
>>> decimal.Decimal(1) / decimal.Decimal(0) |
| 666 |
Traceback (most recent call last): |
| 667 |
... |
| 668 |
decimal.DivisionByZero: x / 0 |
| 669 |
>>> decimal.getcontext().traps[decimal.DivisionByZero] = False |
| 670 |
>>> decimal.Decimal(1) / decimal.Decimal(0) |
| 671 |
Decimal("Infinity") |
| 672 |
>>> |
| 673 |
|
| 674 |
The :class:`Context` instance also has various methods for formatting numbers |
| 675 |
such as :meth:`to_eng_string` and :meth:`to_sci_string`. |
| 676 |
|
| 677 |
For more information, see the documentation for the :mod:`decimal` module, which |
| 678 |
includes a quick-start tutorial and a reference. |
| 679 |
|
| 680 |
|
| 681 |
.. seealso:: |
| 682 |
|
| 683 |
:pep:`327` - Decimal Data Type |
| 684 |
Written by Facundo Batista and implemented by Facundo Batista, Eric Price, |
| 685 |
Raymond Hettinger, Aahz, and Tim Peters. |
| 686 |
|
| 687 |
`http://www.lahey.com/float.htm <https://web.archive.org/web/20230604072523/http://www.lahey.com/float.htm>`__ |
| 688 |
The article uses Fortran code to illustrate many of the problems that |
| 689 |
floating-point inaccuracy can cause. |
| 690 |
|
| 691 |
https://speleotrove.com/decimal/ |
| 692 |
A description of a decimal-based representation. This representation is being |
| 693 |
proposed as a standard, and underlies the new Python decimal type. Much of this |
| 694 |
material was written by Mike Cowlishaw, designer of the Rexx language. |
| 695 |
|
| 696 |
.. ====================================================================== |
| 697 |
|
| 698 |
|
| 699 |
PEP 328: Multi-line Imports |
| 700 |
=========================== |
| 701 |
|
| 702 |
One language change is a small syntactic tweak aimed at making it easier to |
| 703 |
import many names from a module. In a ``from module import names`` statement, |
| 704 |
*names* is a sequence of names separated by commas. If the sequence is very |
| 705 |
long, you can either write multiple imports from the same module, or you can use |
| 706 |
backslashes to escape the line endings like this:: |
| 707 |
|
| 708 |
from SimpleXMLRPCServer import SimpleXMLRPCServer,\ |
| 709 |
SimpleXMLRPCRequestHandler,\ |
| 710 |
CGIXMLRPCRequestHandler,\ |
| 711 |
resolve_dotted_attribute |
| 712 |
|
| 713 |
The syntactic change in Python 2.4 simply allows putting the names within |
| 714 |
parentheses. Python ignores newlines within a parenthesized expression, so the |
| 715 |
backslashes are no longer needed:: |
| 716 |
|
| 717 |
from SimpleXMLRPCServer import (SimpleXMLRPCServer, |
| 718 |
SimpleXMLRPCRequestHandler, |
| 719 |
CGIXMLRPCRequestHandler, |
| 720 |
resolve_dotted_attribute) |
| 721 |
|
| 722 |
The PEP also proposes that all :keyword:`import` statements be absolute imports, |
| 723 |
with a leading ``.`` character to indicate a relative import. This part of the |
| 724 |
PEP was not implemented for Python 2.4, but was completed for Python 2.5. |
| 725 |
|
| 726 |
|
| 727 |
.. seealso:: |
| 728 |
|
| 729 |
:pep:`328` - Imports: Multi-Line and Absolute/Relative |
| 730 |
Written by Aahz. Multi-line imports were implemented by Dima Dorfman. |
| 731 |
|
| 732 |
.. ====================================================================== |
| 733 |
|
| 734 |
|
| 735 |
PEP 331: Locale-Independent Float/String Conversions |
| 736 |
==================================================== |
| 737 |
|
| 738 |
The :mod:`locale` modules lets Python software select various conversions and |
| 739 |
display conventions that are localized to a particular country or language. |
| 740 |
However, the module was careful to not change the numeric locale because various |
| 741 |
functions in Python's implementation required that the numeric locale remain set |
| 742 |
to the ``'C'`` locale. Often this was because the code was using the C |
| 743 |
library's :c:func:`atof` function. |
| 744 |
|
| 745 |
Not setting the numeric locale caused trouble for extensions that used third-party |
| 746 |
C libraries, however, because they wouldn't have the correct locale set. |
| 747 |
The motivating example was GTK+, whose user interface widgets weren't displaying |
| 748 |
numbers in the current locale. |
| 749 |
|
| 750 |
The solution described in the PEP is to add three new functions to the Python |
| 751 |
API that perform ASCII-only conversions, ignoring the locale setting: |
| 752 |
|
| 753 |
* ``PyOS_ascii_strtod(str, ptr)`` and ``PyOS_ascii_atof(str, ptr)`` |
| 754 |
both convert a string to a C :c:expr:`double`. |
| 755 |
|
| 756 |
* ``PyOS_ascii_formatd(buffer, buf_len, format, d)`` converts a |
| 757 |
:c:expr:`double` to an ASCII string. |
| 758 |
|
| 759 |
The code for these functions came from the GLib library |
| 760 |
(`https://developer-old.gnome.org/glib/2.26/ <http://web.archive.org/web/20210306104320/https://developer.gnome.org/glib/2.26/>`__), whose developers kindly |
| 761 |
relicensed the relevant functions and donated them to the Python Software |
| 762 |
Foundation. The :mod:`locale` module can now change the numeric locale, |
| 763 |
letting extensions such as GTK+ produce the correct results. |
| 764 |
|
| 765 |
|
| 766 |
.. seealso:: |
| 767 |
|
| 768 |
:pep:`331` - Locale-Independent Float/String Conversions |
| 769 |
Written by Christian R. Reis, and implemented by Gustavo Carneiro. |
| 770 |
|
| 771 |
.. ====================================================================== |
| 772 |
|
| 773 |
|
| 774 |
Other Language Changes |
| 775 |
====================== |
| 776 |
|
| 777 |
Here are all of the changes that Python 2.4 makes to the core Python language. |
| 778 |
|
| 779 |
* Decorators for functions and methods were added (:pep:`318`). |
| 780 |
|
| 781 |
* Built-in :func:`set` and :func:`frozenset` types were added (:pep:`218`). |
| 782 |
Other new built-ins include the ``reversed(seq)`` function (:pep:`322`). |
| 783 |
|
| 784 |
* Generator expressions were added (:pep:`289`). |
| 785 |
|
| 786 |
* Certain numeric expressions no longer return values restricted to 32 or 64 |
| 787 |
bits (:pep:`237`). |
| 788 |
|
| 789 |
* You can now put parentheses around the list of names in a ``from module import |
| 790 |
names`` statement (:pep:`328`). |
| 791 |
|
| 792 |
* The :meth:`dict.update` method now accepts the same argument forms as the |
| 793 |
:class:`dict` constructor. This includes any mapping, any iterable of key/value |
| 794 |
pairs, and keyword arguments. (Contributed by Raymond Hettinger.) |
| 795 |
|
| 796 |
* The string methods :meth:`ljust`, :meth:`rjust`, and :meth:`center` now take |
| 797 |
an optional argument for specifying a fill character other than a space. |
| 798 |
(Contributed by Raymond Hettinger.) |
| 799 |
|
| 800 |
* Strings also gained an :meth:`rsplit` method that works like the :meth:`split` |
| 801 |
method but splits from the end of the string. (Contributed by Sean |
| 802 |
Reifschneider.) :: |
| 803 |
|
| 804 |
>>> 'www.python.org'.split('.', 1) |
| 805 |
['www', 'python.org'] |
| 806 |
'www.python.org'.rsplit('.', 1) |
| 807 |
['www.python', 'org'] |
| 808 |
|
| 809 |
* Three keyword parameters, *cmp*, *key*, and *reverse*, were added to the |
| 810 |
:meth:`sort` method of lists. These parameters make some common usages of |
| 811 |
:meth:`sort` simpler. All of these parameters are optional. |
| 812 |
|
| 813 |
For the *cmp* parameter, the value should be a comparison function that takes |
| 814 |
two parameters and returns -1, 0, or +1 depending on how the parameters compare. |
| 815 |
This function will then be used to sort the list. Previously this was the only |
| 816 |
parameter that could be provided to :meth:`sort`. |
| 817 |
|
| 818 |
*key* should be a single-parameter function that takes a list element and |
| 819 |
returns a comparison key for the element. The list is then sorted using the |
| 820 |
comparison keys. The following example sorts a list case-insensitively:: |
| 821 |
|
| 822 |
>>> L = ['A', 'b', 'c', 'D'] |
| 823 |
>>> L.sort() # Case-sensitive sort |
| 824 |
>>> L |
| 825 |
['A', 'D', 'b', 'c'] |
| 826 |
>>> # Using 'key' parameter to sort list |
| 827 |
>>> L.sort(key=lambda x: x.lower()) |
| 828 |
>>> L |
| 829 |
['A', 'b', 'c', 'D'] |
| 830 |
>>> # Old-fashioned way |
| 831 |
>>> L.sort(cmp=lambda x,y: cmp(x.lower(), y.lower())) |
| 832 |
>>> L |
| 833 |
['A', 'b', 'c', 'D'] |
| 834 |
|
| 835 |
The last example, which uses the *cmp* parameter, is the old way to perform a |
| 836 |
case-insensitive sort. It works but is slower than using a *key* parameter. |
| 837 |
Using *key* calls :meth:`lower` method once for each element in the list while |
| 838 |
using *cmp* will call it twice for each comparison, so using *key* saves on |
| 839 |
invocations of the :meth:`lower` method. |
| 840 |
|
| 841 |
For simple key functions and comparison functions, it is often possible to avoid |
| 842 |
a :keyword:`lambda` expression by using an unbound method instead. For example, |
| 843 |
the above case-insensitive sort is best written as:: |
| 844 |
|
| 845 |
>>> L.sort(key=str.lower) |
| 846 |
>>> L |
| 847 |
['A', 'b', 'c', 'D'] |
| 848 |
|
| 849 |
Finally, the *reverse* parameter takes a Boolean value. If the value is true, |
| 850 |
the list will be sorted into reverse order. Instead of ``L.sort(); |
| 851 |
L.reverse()``, you can now write ``L.sort(reverse=True)``. |
| 852 |
|
| 853 |
The results of sorting are now guaranteed to be stable. This means that two |
| 854 |
entries with equal keys will be returned in the same order as they were input. |
| 855 |
For example, you can sort a list of people by name, and then sort the list by |
| 856 |
age, resulting in a list sorted by age where people with the same age are in |
| 857 |
name-sorted order. |
| 858 |
|
| 859 |
(All changes to :meth:`sort` contributed by Raymond Hettinger.) |
| 860 |
|
| 861 |
* There is a new built-in function ``sorted(iterable)`` that works like the |
| 862 |
in-place :meth:`list.sort` method but can be used in expressions. The |
| 863 |
differences are: |
| 864 |
|
| 865 |
* the input may be any iterable; |
| 866 |
|
| 867 |
* a newly formed copy is sorted, leaving the original intact; and |
| 868 |
|
| 869 |
* the expression returns the new sorted copy |
| 870 |
|
| 871 |
:: |
| 872 |
|
| 873 |
>>> L = [9,7,8,3,2,4,1,6,5] |
| 874 |
>>> [10+i for i in sorted(L)] # usable in a list comprehension |
| 875 |
[11, 12, 13, 14, 15, 16, 17, 18, 19] |
| 876 |
>>> L # original is left unchanged |
| 877 |
[9,7,8,3,2,4,1,6,5] |
| 878 |
>>> sorted('Monty Python') # any iterable may be an input |
| 879 |
[' ', 'M', 'P', 'h', 'n', 'n', 'o', 'o', 't', 't', 'y', 'y'] |
| 880 |
|
| 881 |
>>> # List the contents of a dict sorted by key values |
| 882 |
>>> colormap = dict(red=1, blue=2, green=3, black=4, yellow=5) |
| 883 |
>>> for k, v in sorted(colormap.iteritems()): |
| 884 |
... print k, v |
| 885 |
... |
| 886 |
black 4 |
| 887 |
blue 2 |
| 888 |
green 3 |
| 889 |
red 1 |
| 890 |
yellow 5 |
| 891 |
|
| 892 |
(Contributed by Raymond Hettinger.) |
| 893 |
|
| 894 |
* Integer operations will no longer trigger an :exc:`OverflowWarning`. The |
| 895 |
:exc:`OverflowWarning` warning will disappear in Python 2.5. |
| 896 |
|
| 897 |
* The interpreter gained a new switch, :option:`-m`, that takes a name, searches |
| 898 |
for the corresponding module on ``sys.path``, and runs the module as a script. |
| 899 |
For example, you can now run the Python profiler with ``python -m profile``. |
| 900 |
(Contributed by Nick Coghlan.) |
| 901 |
|
| 902 |
* The ``eval(expr, globals, locals)`` and ``execfile(filename, globals, |
| 903 |
locals)`` functions and the ``exec`` statement now accept any mapping type |
| 904 |
for the *locals* parameter. Previously this had to be a regular Python |
| 905 |
dictionary. (Contributed by Raymond Hettinger.) |
| 906 |
|
| 907 |
* The :func:`zip` built-in function and :func:`itertools.izip` now return an |
| 908 |
empty list if called with no arguments. Previously they raised a |
| 909 |
:exc:`TypeError` exception. This makes them more suitable for use with variable |
| 910 |
length argument lists:: |
| 911 |
|
| 912 |
>>> def transpose(array): |
| 913 |
... return zip(*array) |
| 914 |
... |
| 915 |
>>> transpose([(1,2,3), (4,5,6)]) |
| 916 |
[(1, 4), (2, 5), (3, 6)] |
| 917 |
>>> transpose([]) |
| 918 |
[] |
| 919 |
|
| 920 |
(Contributed by Raymond Hettinger.) |
| 921 |
|
| 922 |
* Encountering a failure while importing a module no longer leaves a partially initialized |
| 923 |
module object in ``sys.modules``. The incomplete module object left |
| 924 |
behind would fool further imports of the same module into succeeding, leading to |
| 925 |
confusing errors. (Fixed by Tim Peters.) |
| 926 |
|
| 927 |
* :const:`None` is now a constant; code that binds a new value to the name |
| 928 |
``None`` is now a syntax error. (Contributed by Raymond Hettinger.) |
| 929 |
|
| 930 |
.. ====================================================================== |
| 931 |
|
| 932 |
|
| 933 |
Optimizations |
| 934 |
------------- |
| 935 |
|
| 936 |
* The inner loops for list and tuple slicing were optimized and now run about |
| 937 |
one-third faster. The inner loops for dictionaries were also optimized, |
| 938 |
resulting in performance boosts for :meth:`keys`, :meth:`values`, :meth:`items`, |
| 939 |
:meth:`iterkeys`, :meth:`itervalues`, and :meth:`iteritems`. (Contributed by |
| 940 |
Raymond Hettinger.) |
| 941 |
|
| 942 |
* The machinery for growing and shrinking lists was optimized for speed and for |
| 943 |
space efficiency. Appending and popping from lists now runs faster due to more |
| 944 |
efficient code paths and less frequent use of the underlying system |
| 945 |
:c:func:`realloc`. List comprehensions also benefit. :meth:`list.extend` was |
| 946 |
also optimized and no longer converts its argument into a temporary list before |
| 947 |
extending the base list. (Contributed by Raymond Hettinger.) |
| 948 |
|
| 949 |
* :func:`list`, :func:`tuple`, :func:`map`, :func:`filter`, and :func:`zip` now |
| 950 |
run several times faster with non-sequence arguments that supply a |
| 951 |
:meth:`__len__` method. (Contributed by Raymond Hettinger.) |
| 952 |
|
| 953 |
* The methods :meth:`list.__getitem__`, :meth:`dict.__getitem__`, and |
| 954 |
:meth:`dict.__contains__` are now implemented as :class:`method_descriptor` |
| 955 |
objects rather than :class:`wrapper_descriptor` objects. This form of access |
| 956 |
doubles their performance and makes them more suitable for use as arguments to |
| 957 |
functionals: ``map(mydict.__getitem__, keylist)``. (Contributed by Raymond |
| 958 |
Hettinger.) |
| 959 |
|
| 960 |
* Added a new opcode, ``LIST_APPEND``, that simplifies the generated bytecode |
| 961 |
for list comprehensions and speeds them up by about a third. (Contributed by |
| 962 |
Raymond Hettinger.) |
| 963 |
|
| 964 |
* The peephole bytecode optimizer has been improved to produce shorter, faster |
| 965 |
bytecode; remarkably, the resulting bytecode is more readable. (Enhanced by |
| 966 |
Raymond Hettinger.) |
| 967 |
|
| 968 |
* String concatenations in statements of the form ``s = s + "abc"`` and ``s += |
| 969 |
"abc"`` are now performed more efficiently in certain circumstances. This |
| 970 |
optimization won't be present in other Python implementations such as Jython, so |
| 971 |
you shouldn't rely on it; using the :meth:`join` method of strings is still |
| 972 |
recommended when you want to efficiently glue a large number of strings |
| 973 |
together. (Contributed by Armin Rigo.) |
| 974 |
|
| 975 |
The net result of the 2.4 optimizations is that Python 2.4 runs the pystone |
| 976 |
benchmark around 5% faster than Python 2.3 and 35% faster than Python 2.2. |
| 977 |
(pystone is not a particularly good benchmark, but it's the most commonly used |
| 978 |
measurement of Python's performance. Your own applications may show greater or |
| 979 |
smaller benefits from Python 2.4.) |
| 980 |
|
| 981 |
.. pystone is almost useless for comparing different versions of Python; |
| 982 |
instead, it excels at predicting relative Python performance on different |
| 983 |
machines. So, this section would be more informative if it used other tools |
| 984 |
such as pybench and parrotbench. For a more application oriented benchmark, |
| 985 |
try comparing the timings of test_decimal.py under 2.3 and 2.4. |
| 986 |
|
| 987 |
.. ====================================================================== |
| 988 |
|
| 989 |
|
| 990 |
New, Improved, and Deprecated Modules |
| 991 |
===================================== |
| 992 |
|
| 993 |
As usual, Python's standard library received a number of enhancements and bug |
| 994 |
fixes. Here's a partial list of the most notable changes, sorted alphabetically |
| 995 |
by module name. Consult the :file:`Misc/NEWS` file in the source tree for a more |
| 996 |
complete list of changes, or look through the CVS logs for all the details. |
| 997 |
|
| 998 |
* The :mod:`!asyncore` module's :func:`!loop` function now has a *count* parameter |
| 999 |
that lets you perform a limited number of passes through the polling loop. The |
| 1000 |
default is still to loop forever. |
| 1001 |
|
| 1002 |
* The :mod:`base64` module now has more complete :rfc:`3548` support for Base64, |
| 1003 |
Base32, and Base16 encoding and decoding, including optional case folding and |
| 1004 |
optional alternative alphabets. (Contributed by Barry Warsaw.) |
| 1005 |
|
| 1006 |
* The :mod:`bisect` module now has an underlying C implementation for improved |
| 1007 |
performance. (Contributed by Dmitry Vasiliev.) |
| 1008 |
|
| 1009 |
* The CJKCodecs collections of East Asian codecs, maintained by Hye-Shik Chang, |
| 1010 |
was integrated into 2.4. The new encodings are: |
| 1011 |
|
| 1012 |
* Chinese (PRC): gb2312, gbk, gb18030, big5hkscs, hz |
| 1013 |
|
| 1014 |
* Chinese (ROC): big5, cp950 |
| 1015 |
|
| 1016 |
* Japanese: cp932, euc-jis-2004, euc-jp, euc-jisx0213, iso-2022-jp, |
| 1017 |
iso-2022-jp-1, iso-2022-jp-2, iso-2022-jp-3, iso-2022-jp-ext, iso-2022-jp-2004, |
| 1018 |
shift-jis, shift-jisx0213, shift-jis-2004 |
| 1019 |
|
| 1020 |
* Korean: cp949, euc-kr, johab, iso-2022-kr |
| 1021 |
|
| 1022 |
* Some other new encodings were added: HP Roman8, ISO_8859-11, ISO_8859-16, |
| 1023 |
PCTP-154, and TIS-620. |
| 1024 |
|
| 1025 |
* The UTF-8 and UTF-16 codecs now cope better with receiving partial input. |
| 1026 |
Previously the :class:`StreamReader` class would try to read more data, making |
| 1027 |
it impossible to resume decoding from the stream. The :meth:`read` method will |
| 1028 |
now return as much data as it can and future calls will resume decoding where |
| 1029 |
previous ones left off. (Implemented by Walter Dörwald.) |
| 1030 |
|
| 1031 |
* There is a new :mod:`collections` module for various specialized collection |
| 1032 |
datatypes. Currently it contains just one type, :class:`deque`, a double-ended |
| 1033 |
queue that supports efficiently adding and removing elements from either |
| 1034 |
end:: |
| 1035 |
|
| 1036 |
>>> from collections import deque |
| 1037 |
>>> d = deque('ghi') # make a new deque with three items |
| 1038 |
>>> d.append('j') # add a new entry to the right side |
| 1039 |
>>> d.appendleft('f') # add a new entry to the left side |
| 1040 |
>>> d # show the representation of the deque |
| 1041 |
deque(['f', 'g', 'h', 'i', 'j']) |
| 1042 |
>>> d.pop() # return and remove the rightmost item |
| 1043 |
'j' |
| 1044 |
>>> d.popleft() # return and remove the leftmost item |
| 1045 |
'f' |
| 1046 |
>>> list(d) # list the contents of the deque |
| 1047 |
['g', 'h', 'i'] |
| 1048 |
>>> 'h' in d # search the deque |
| 1049 |
True |
| 1050 |
|
| 1051 |
Several modules, such as the :mod:`Queue` and :mod:`threading` modules, now take |
| 1052 |
advantage of :class:`collections.deque` for improved performance. (Contributed |
| 1053 |
by Raymond Hettinger.) |
| 1054 |
|
| 1055 |
* The :mod:`ConfigParser <configparser>` classes have been enhanced slightly. The :meth:`~configparser.ConfigParser.read` |
| 1056 |
method now returns a list of the files that were successfully parsed, and the |
| 1057 |
:meth:`~configparser.ConfigParser.set` method raises :exc:`TypeError` if passed a *value* argument that |
| 1058 |
isn't a string. (Contributed by John Belmonte and David Goodger.) |
| 1059 |
|
| 1060 |
* The :mod:`curses` module now supports the ncurses extension |
| 1061 |
:func:`use_default_colors`. On platforms where the terminal supports |
| 1062 |
transparency, this makes it possible to use a transparent background. |
| 1063 |
(Contributed by Jörg Lehmann.) |
| 1064 |
|
| 1065 |
* The :mod:`difflib` module now includes an :class:`HtmlDiff` class that creates |
| 1066 |
an HTML table showing a side by side comparison of two versions of a text. |
| 1067 |
(Contributed by Dan Gass.) |
| 1068 |
|
| 1069 |
* The :mod:`email` package was updated to version 3.0, which dropped various |
| 1070 |
deprecated APIs and removes support for Python versions earlier than 2.3. The |
| 1071 |
3.0 version of the package uses a new incremental parser for MIME messages, |
| 1072 |
available in the :mod:`email.FeedParser` module. The new parser doesn't require |
| 1073 |
reading the entire message into memory, and doesn't raise exceptions if a |
| 1074 |
message is malformed; instead it records any problems in the :attr:`defect` |
| 1075 |
attribute of the message. (Developed by Anthony Baxter, Barry Warsaw, Thomas |
| 1076 |
Wouters, and others.) |
| 1077 |
|
| 1078 |
* The :mod:`heapq` module has been converted to C. The resulting tenfold |
| 1079 |
improvement in speed makes the module suitable for handling high volumes of |
| 1080 |
data. In addition, the module has two new functions :func:`nlargest` and |
| 1081 |
:func:`nsmallest` that use heaps to find the N largest or smallest values in a |
| 1082 |
dataset without the expense of a full sort. (Contributed by Raymond Hettinger.) |
| 1083 |
|
| 1084 |
* The :mod:`httplib <http>` module now contains constants for HTTP status codes defined |
| 1085 |
in various HTTP-related RFC documents. Constants have names such as |
| 1086 |
:const:`OK`, :const:`CREATED`, :const:`CONTINUE`, and |
| 1087 |
:const:`MOVED_PERMANENTLY`; use pydoc to get a full list. (Contributed by |
| 1088 |
Andrew Eland.) |
| 1089 |
|
| 1090 |
* The :mod:`imaplib` module now supports IMAP's THREAD command (contributed by |
| 1091 |
Yves Dionne) and new :meth:`deleteacl` and :meth:`myrights` methods (contributed |
| 1092 |
by Arnaud Mazin). |
| 1093 |
|
| 1094 |
* The :mod:`itertools` module gained a ``groupby(iterable[, *func*])`` |
| 1095 |
function. *iterable* is something that can be iterated over to return a stream |
| 1096 |
of elements, and the optional *func* parameter is a function that takes an |
| 1097 |
element and returns a key value; if omitted, the key is simply the element |
| 1098 |
itself. :func:`groupby` then groups the elements into subsequences which have |
| 1099 |
matching values of the key, and returns a series of 2-tuples containing the key |
| 1100 |
value and an iterator over the subsequence. |
| 1101 |
|
| 1102 |
Here's an example to make this clearer. The *key* function simply returns |
| 1103 |
whether a number is even or odd, so the result of :func:`groupby` is to return |
| 1104 |
consecutive runs of odd or even numbers. :: |
| 1105 |
|
| 1106 |
>>> import itertools |
| 1107 |
>>> L = [2, 4, 6, 7, 8, 9, 11, 12, 14] |
| 1108 |
>>> for key_val, it in itertools.groupby(L, lambda x: x % 2): |
| 1109 |
... print key_val, list(it) |
| 1110 |
... |
| 1111 |
0 [2, 4, 6] |
| 1112 |
1 [7] |
| 1113 |
0 [8] |
| 1114 |
1 [9, 11] |
| 1115 |
0 [12, 14] |
| 1116 |
>>> |
| 1117 |
|
| 1118 |
:func:`groupby` is typically used with sorted input. The logic for |
| 1119 |
:func:`groupby` is similar to the Unix ``uniq`` filter which makes it handy for |
| 1120 |
eliminating, counting, or identifying duplicate elements:: |
| 1121 |
|
| 1122 |
>>> word = 'abracadabra' |
| 1123 |
>>> letters = sorted(word) # Turn string into a sorted list of letters |
| 1124 |
>>> letters |
| 1125 |
['a', 'a', 'a', 'a', 'a', 'b', 'b', 'c', 'd', 'r', 'r'] |
| 1126 |
>>> for k, g in itertools.groupby(letters): |
| 1127 |
... print k, list(g) |
| 1128 |
... |
| 1129 |
a ['a', 'a', 'a', 'a', 'a'] |
| 1130 |
b ['b', 'b'] |
| 1131 |
c ['c'] |
| 1132 |
d ['d'] |
| 1133 |
r ['r', 'r'] |
| 1134 |
>>> # List unique letters |
| 1135 |
>>> [k for k, g in groupby(letters)] |
| 1136 |
['a', 'b', 'c', 'd', 'r'] |
| 1137 |
>>> # Count letter occurrences |
| 1138 |
>>> [(k, len(list(g))) for k, g in groupby(letters)] |
| 1139 |
[('a', 5), ('b', 2), ('c', 1), ('d', 1), ('r', 2)] |
| 1140 |
|
| 1141 |
(Contributed by Hye-Shik Chang.) |
| 1142 |
|
| 1143 |
* :mod:`itertools` also gained a function named ``tee(iterator, N)`` that |
| 1144 |
returns *N* independent iterators that replicate *iterator*. If *N* is omitted, |
| 1145 |
the default is 2. :: |
| 1146 |
|
| 1147 |
>>> L = [1,2,3] |
| 1148 |
>>> i1, i2 = itertools.tee(L) |
| 1149 |
>>> i1,i2 |
| 1150 |
(<itertools.tee object at 0x402c2080>, <itertools.tee object at 0x402c2090>) |
| 1151 |
>>> list(i1) # Run the first iterator to exhaustion |
| 1152 |
[1, 2, 3] |
| 1153 |
>>> list(i2) # Run the second iterator to exhaustion |
| 1154 |
[1, 2, 3] |
| 1155 |
|
| 1156 |
Note that :func:`tee` has to keep copies of the values returned by the |
| 1157 |
iterator; in the worst case, it may need to keep all of them. This should |
| 1158 |
therefore be used carefully if the leading iterator can run far ahead of the |
| 1159 |
trailing iterator in a long stream of inputs. If the separation is large, then |
| 1160 |
you might as well use :func:`list` instead. When the iterators track closely |
| 1161 |
with one another, :func:`tee` is ideal. Possible applications include |
| 1162 |
bookmarking, windowing, or lookahead iterators. (Contributed by Raymond |
| 1163 |
Hettinger.) |
| 1164 |
|
| 1165 |
* A number of functions were added to the :mod:`locale` module, such as |
| 1166 |
:func:`bind_textdomain_codeset` to specify a particular encoding and a family of |
| 1167 |
:func:`!l\*gettext` functions that return messages in the chosen encoding. |
| 1168 |
(Contributed by Gustavo Niemeyer.) |
| 1169 |
|
| 1170 |
* Some keyword arguments were added to the :mod:`logging` package's |
| 1171 |
:func:`basicConfig` function to simplify log configuration. The default |
| 1172 |
behavior is to log messages to standard error, but various keyword arguments can |
| 1173 |
be specified to log to a particular file, change the logging format, or set the |
| 1174 |
logging level. For example:: |
| 1175 |
|
| 1176 |
import logging |
| 1177 |
logging.basicConfig(filename='/var/log/application.log', |
| 1178 |
level=0, # Log all messages |
| 1179 |
format='%(levelname):%(process):%(thread):%(message)') |
| 1180 |
|
| 1181 |
Other additions to the :mod:`logging` package include a ``log(level, msg)`` |
| 1182 |
convenience method, as well as a :class:`TimedRotatingFileHandler` class that |
| 1183 |
rotates its log files at a timed interval. The module already had |
| 1184 |
:class:`RotatingFileHandler`, which rotated logs once the file exceeded a |
| 1185 |
certain size. Both classes derive from a new :class:`BaseRotatingHandler` class |
| 1186 |
that can be used to implement other rotating handlers. |
| 1187 |
|
| 1188 |
(Changes implemented by Vinay Sajip.) |
| 1189 |
|
| 1190 |
* The :mod:`marshal` module now shares interned strings on unpacking a data |
| 1191 |
structure. This may shrink the size of certain pickle strings, but the primary |
| 1192 |
effect is to make :file:`.pyc` files significantly smaller. (Contributed by |
| 1193 |
Martin von Löwis.) |
| 1194 |
|
| 1195 |
* The :mod:`!nntplib` module's :class:`NNTP` class gained :meth:`description` and |
| 1196 |
:meth:`descriptions` methods to retrieve newsgroup descriptions for a single |
| 1197 |
group or for a range of groups. (Contributed by Jürgen A. Erhard.) |
| 1198 |
|
| 1199 |
* Two new functions were added to the :mod:`operator` module, |
| 1200 |
``attrgetter(attr)`` and ``itemgetter(index)``. Both functions return |
| 1201 |
callables that take a single argument and return the corresponding attribute or |
| 1202 |
item; these callables make excellent data extractors when used with :func:`map` |
| 1203 |
or :func:`sorted`. For example:: |
| 1204 |
|
| 1205 |
>>> L = [('c', 2), ('d', 1), ('a', 4), ('b', 3)] |
| 1206 |
>>> map(operator.itemgetter(0), L) |
| 1207 |
['c', 'd', 'a', 'b'] |
| 1208 |
>>> map(operator.itemgetter(1), L) |
| 1209 |
[2, 1, 4, 3] |
| 1210 |
>>> sorted(L, key=operator.itemgetter(1)) # Sort list by second tuple item |
| 1211 |
[('d', 1), ('c', 2), ('b', 3), ('a', 4)] |
| 1212 |
|
| 1213 |
(Contributed by Raymond Hettinger.) |
| 1214 |
|
| 1215 |
* The :mod:`optparse` module was updated in various ways. The module now passes |
| 1216 |
its messages through :func:`gettext.gettext`, making it possible to |
| 1217 |
internationalize Optik's help and error messages. Help messages for options can |
| 1218 |
now include the string ``'%default'``, which will be replaced by the option's |
| 1219 |
default value. (Contributed by Greg Ward.) |
| 1220 |
|
| 1221 |
* The long-term plan is to deprecate the :mod:`!rfc822` module in some future |
| 1222 |
Python release in favor of the :mod:`email` package. To this end, the |
| 1223 |
:func:`email.Utils.formatdate <email.utils.formatdate>` function has been changed to make it usable as a |
| 1224 |
replacement for :func:`!rfc822.formatdate`. You may want to write new e-mail |
| 1225 |
processing code with this in mind. (Change implemented by Anthony Baxter.) |
| 1226 |
|
| 1227 |
* A new ``urandom(n)`` function was added to the :mod:`os` module, returning |
| 1228 |
a string containing *n* bytes of random data. This function provides access to |
| 1229 |
platform-specific sources of randomness such as :file:`/dev/urandom` on Linux or |
| 1230 |
the Windows CryptoAPI. (Contributed by Trevor Perrin.) |
| 1231 |
|
| 1232 |
* Another new function: ``os.path.lexists(path)`` returns true if the file |
| 1233 |
specified by *path* exists, whether or not it's a symbolic link. This differs |
| 1234 |
from the existing ``os.path.exists(path)`` function, which returns false if |
| 1235 |
*path* is a symlink that points to a destination that doesn't exist. |
| 1236 |
(Contributed by Beni Cherniavsky.) |
| 1237 |
|
| 1238 |
* A new :func:`getsid` function was added to the :mod:`posix` module that |
| 1239 |
underlies the :mod:`os` module. (Contributed by J. Raynor.) |
| 1240 |
|
| 1241 |
* The :mod:`poplib` module now supports POP over SSL. (Contributed by Hector |
| 1242 |
Urtubia.) |
| 1243 |
|
| 1244 |
* The :mod:`profile` module can now profile C extension functions. (Contributed |
| 1245 |
by Nick Bastin.) |
| 1246 |
|
| 1247 |
* The :mod:`random` module has a new method called ``getrandbits(N)`` that |
| 1248 |
returns a long integer *N* bits in length. The existing :meth:`randrange` |
| 1249 |
method now uses :meth:`getrandbits` where appropriate, making generation of |
| 1250 |
arbitrarily large random numbers more efficient. (Contributed by Raymond |
| 1251 |
Hettinger.) |
| 1252 |
|
| 1253 |
* The regular expression language accepted by the :mod:`re` module was extended |
| 1254 |
with simple conditional expressions, written as ``(?(group)A|B)``. *group* is |
| 1255 |
either a numeric group ID or a group name defined with ``(?P<group>...)`` |
| 1256 |
earlier in the expression. If the specified group matched, the regular |
| 1257 |
expression pattern *A* will be tested against the string; if the group didn't |
| 1258 |
match, the pattern *B* will be used instead. (Contributed by Gustavo Niemeyer.) |
| 1259 |
|
| 1260 |
* The :mod:`re` module is also no longer recursive, thanks to a massive amount |
| 1261 |
of work by Gustavo Niemeyer. In a recursive regular expression engine, certain |
| 1262 |
patterns result in a large amount of C stack space being consumed, and it was |
| 1263 |
possible to overflow the stack. For example, if you matched a 30000-byte string |
| 1264 |
of ``a`` characters against the expression ``(a|b)+``, one stack frame was |
| 1265 |
consumed per character. Python 2.3 tried to check for stack overflow and raise |
| 1266 |
a :exc:`RuntimeError` exception, but certain patterns could sidestep the |
| 1267 |
checking and if you were unlucky Python could segfault. Python 2.4's regular |
| 1268 |
expression engine can match this pattern without problems. |
| 1269 |
|
| 1270 |
* The :mod:`signal` module now performs tighter error-checking on the parameters |
| 1271 |
to the :func:`signal.signal` function. For example, you can't set a handler on |
| 1272 |
the :const:`SIGKILL` signal; previous versions of Python would quietly accept |
| 1273 |
this, but 2.4 will raise a :exc:`RuntimeError` exception. |
| 1274 |
|
| 1275 |
* Two new functions were added to the :mod:`socket` module. :func:`socketpair` |
| 1276 |
returns a pair of connected sockets and ``getservbyport(port)`` looks up the |
| 1277 |
service name for a given port number. (Contributed by Dave Cole and Barry |
| 1278 |
Warsaw.) |
| 1279 |
|
| 1280 |
* The :func:`sys.exitfunc` function has been deprecated. Code should be using |
| 1281 |
the existing :mod:`atexit` module, which correctly handles calling multiple exit |
| 1282 |
functions. Eventually :func:`sys.exitfunc` will become a purely internal |
| 1283 |
interface, accessed only by :mod:`atexit`. |
| 1284 |
|
| 1285 |
* The :mod:`tarfile` module now generates GNU-format tar files by default. |
| 1286 |
(Contributed by Lars Gustäbel.) |
| 1287 |
|
| 1288 |
* The :mod:`threading` module now has an elegantly simple way to support |
| 1289 |
thread-local data. The module contains a :class:`local` class whose attribute |
| 1290 |
values are local to different threads. :: |
| 1291 |
|
| 1292 |
import threading |
| 1293 |
|
| 1294 |
data = threading.local() |
| 1295 |
data.number = 42 |
| 1296 |
data.url = ('www.python.org', 80) |
| 1297 |
|
| 1298 |
Other threads can assign and retrieve their own values for the :attr:`number` |
| 1299 |
and :attr:`url` attributes. You can subclass :class:`local` to initialize |
| 1300 |
attributes or to add methods. (Contributed by Jim Fulton.) |
| 1301 |
|
| 1302 |
* The :mod:`timeit` module now automatically disables periodic garbage |
| 1303 |
collection during the timing loop. This change makes consecutive timings more |
| 1304 |
comparable. (Contributed by Raymond Hettinger.) |
| 1305 |
|
| 1306 |
* The :mod:`weakref` module now supports a wider variety of objects including |
| 1307 |
Python functions, class instances, sets, frozensets, deques, arrays, files, |
| 1308 |
sockets, and regular expression pattern objects. (Contributed by Raymond |
| 1309 |
Hettinger.) |
| 1310 |
|
| 1311 |
* The :mod:`xmlrpclib <xmlrpc.client>` module now supports a multi-call extension for |
| 1312 |
transmitting multiple XML-RPC calls in a single HTTP operation. (Contributed by |
| 1313 |
Brian Quinlan.) |
| 1314 |
|
| 1315 |
* The :mod:`mpz`, :mod:`rotor`, and :mod:`xreadlines` modules have been |
| 1316 |
removed. |
| 1317 |
|
| 1318 |
.. ====================================================================== |
| 1319 |
.. whole new modules get described in subsections here |
| 1320 |
.. ===================== |
| 1321 |
|
| 1322 |
|
| 1323 |
cookielib |
| 1324 |
--------- |
| 1325 |
|
| 1326 |
The :mod:`cookielib <http.cookiejar>` library supports client-side handling for HTTP cookies, |
| 1327 |
mirroring the :mod:`Cookie <http.cookies>` module's server-side cookie support. Cookies are |
| 1328 |
stored in cookie jars; the library transparently stores cookies offered by the |
| 1329 |
web server in the cookie jar, and fetches the cookie from the jar when |
| 1330 |
connecting to the server. As in web browsers, policy objects control whether |
| 1331 |
cookies are accepted or not. |
| 1332 |
|
| 1333 |
In order to store cookies across sessions, two implementations of cookie jars |
| 1334 |
are provided: one that stores cookies in the Netscape format so applications can |
| 1335 |
use the Mozilla or Lynx cookie files, and one that stores cookies in the same |
| 1336 |
format as the Perl libwww library. |
| 1337 |
|
| 1338 |
:mod:`urllib2 <urllib.request>` has been changed to interact with :mod:`cookielib <http.cookiejar>`: |
| 1339 |
:class:`HTTPCookieProcessor` manages a cookie jar that is used when accessing |
| 1340 |
URLs. |
| 1341 |
|
| 1342 |
This module was contributed by John J. Lee. |
| 1343 |
|
| 1344 |
.. ================== |
| 1345 |
|
| 1346 |
|
| 1347 |
doctest |
| 1348 |
------- |
| 1349 |
|
| 1350 |
The :mod:`doctest` module underwent considerable refactoring thanks to Edward |
| 1351 |
Loper and Tim Peters. Testing can still be as simple as running |
| 1352 |
:func:`doctest.testmod`, but the refactorings allow customizing the module's |
| 1353 |
operation in various ways |
| 1354 |
|
| 1355 |
The new :class:`DocTestFinder` class extracts the tests from a given object's |
| 1356 |
docstrings:: |
| 1357 |
|
| 1358 |
def f (x, y): |
| 1359 |
""">>> f(2,2) |
| 1360 |
4 |
| 1361 |
>>> f(3,2) |
| 1362 |
6 |
| 1363 |
""" |
| 1364 |
return x*y |
| 1365 |
|
| 1366 |
finder = doctest.DocTestFinder() |
| 1367 |
|
| 1368 |
# Get list of DocTest instances |
| 1369 |
tests = finder.find(f) |
| 1370 |
|
| 1371 |
The new :class:`DocTestRunner` class then runs individual tests and can produce |
| 1372 |
a summary of the results:: |
| 1373 |
|
| 1374 |
runner = doctest.DocTestRunner() |
| 1375 |
for t in tests: |
| 1376 |
tried, failed = runner.run(t) |
| 1377 |
|
| 1378 |
runner.summarize(verbose=1) |
| 1379 |
|
| 1380 |
The above example produces the following output:: |
| 1381 |
|
| 1382 |
1 items passed all tests: |
| 1383 |
2 tests in f |
| 1384 |
2 tests in 1 items. |
| 1385 |
2 passed and 0 failed. |
| 1386 |
Test passed. |
| 1387 |
|
| 1388 |
:class:`DocTestRunner` uses an instance of the :class:`OutputChecker` class to |
| 1389 |
compare the expected output with the actual output. This class takes a number |
| 1390 |
of different flags that customize its behaviour; ambitious users can also write |
| 1391 |
a completely new subclass of :class:`OutputChecker`. |
| 1392 |
|
| 1393 |
The default output checker provides a number of handy features. For example, |
| 1394 |
with the :const:`doctest.ELLIPSIS` option flag, an ellipsis (``...``) in the |
| 1395 |
expected output matches any substring, making it easier to accommodate outputs |
| 1396 |
that vary in minor ways:: |
| 1397 |
|
| 1398 |
def o (n): |
| 1399 |
""">>> o(1) |
| 1400 |
<__main__.C instance at 0x...> |
| 1401 |
>>> |
| 1402 |
""" |
| 1403 |
|
| 1404 |
Another special string, ``<BLANKLINE>``, matches a blank line:: |
| 1405 |
|
| 1406 |
def p (n): |
| 1407 |
""">>> p(1) |
| 1408 |
<BLANKLINE> |
| 1409 |
>>> |
| 1410 |
""" |
| 1411 |
|
| 1412 |
Another new capability is producing a diff-style display of the output by |
| 1413 |
specifying the :const:`doctest.REPORT_UDIFF` (unified diffs), |
| 1414 |
:const:`doctest.REPORT_CDIFF` (context diffs), or :const:`doctest.REPORT_NDIFF` |
| 1415 |
(delta-style) option flags. For example:: |
| 1416 |
|
| 1417 |
def g (n): |
| 1418 |
""">>> g(4) |
| 1419 |
here |
| 1420 |
is |
| 1421 |
a |
| 1422 |
lengthy |
| 1423 |
>>>""" |
| 1424 |
L = 'here is a rather lengthy list of words'.split() |
| 1425 |
for word in L[:n]: |
| 1426 |
print word |
| 1427 |
|
| 1428 |
Running the above function's tests with :const:`doctest.REPORT_UDIFF` specified, |
| 1429 |
you get the following output: |
| 1430 |
|
| 1431 |
.. code-block:: none |
| 1432 |
|
| 1433 |
********************************************************************** |
| 1434 |
File "t.py", line 15, in g |
| 1435 |
Failed example: |
| 1436 |
g(4) |
| 1437 |
Differences (unified diff with -expected +actual): |
| 1438 |
@@ -2,3 +2,3 @@ |
| 1439 |
is |
| 1440 |
a |
| 1441 |
-lengthy |
| 1442 |
+rather |
| 1443 |
********************************************************************** |
| 1444 |
|
| 1445 |
.. ====================================================================== |
| 1446 |
|
| 1447 |
|
| 1448 |
Build and C API Changes |
| 1449 |
======================= |
| 1450 |
|
| 1451 |
Some of the changes to Python's build process and to the C API are: |
| 1452 |
|
| 1453 |
* Three new convenience macros were added for common return values from |
| 1454 |
extension functions: :c:macro:`Py_RETURN_NONE`, :c:macro:`Py_RETURN_TRUE`, and |
| 1455 |
:c:macro:`Py_RETURN_FALSE`. (Contributed by Brett Cannon.) |
| 1456 |
|
| 1457 |
* Another new macro, :c:macro:`Py_CLEAR`, decreases the reference count of |
| 1458 |
*obj* and sets *obj* to the null pointer. (Contributed by Jim Fulton.) |
| 1459 |
|
| 1460 |
* A new function, ``PyTuple_Pack(N, obj1, obj2, ..., objN)``, constructs |
| 1461 |
tuples from a variable length argument list of Python objects. (Contributed by |
| 1462 |
Raymond Hettinger.) |
| 1463 |
|
| 1464 |
* A new function, ``PyDict_Contains(d, k)``, implements fast dictionary |
| 1465 |
lookups without masking exceptions raised during the look-up process. |
| 1466 |
(Contributed by Raymond Hettinger.) |
| 1467 |
|
| 1468 |
* The :c:expr:`Py_IS_NAN(X)` macro returns 1 if its float or double argument |
| 1469 |
*X* is a NaN. (Contributed by Tim Peters.) |
| 1470 |
|
| 1471 |
* C code can avoid unnecessary locking by using the new |
| 1472 |
:c:func:`!PyEval_ThreadsInitialized` function to tell if any thread operations |
| 1473 |
have been performed. If this function returns false, no lock operations are |
| 1474 |
needed. (Contributed by Nick Coghlan.) |
| 1475 |
|
| 1476 |
* A new function, :c:func:`PyArg_VaParseTupleAndKeywords`, is the same as |
| 1477 |
:c:func:`PyArg_ParseTupleAndKeywords` but takes a :c:type:`va_list` instead of a |
| 1478 |
number of arguments. (Contributed by Greg Chapman.) |
| 1479 |
|
| 1480 |
* A new method flag, :c:macro:`METH_COEXIST`, allows a function defined in slots |
| 1481 |
to co-exist with a :c:type:`PyCFunction` having the same name. This can halve |
| 1482 |
the access time for a method such as :meth:`set.__contains__`. (Contributed by |
| 1483 |
Raymond Hettinger.) |
| 1484 |
|
| 1485 |
* Python can now be built with additional profiling for the interpreter itself, |
| 1486 |
intended as an aid to people developing the Python core. Providing |
| 1487 |
:option:`!--enable-profiling` to the :program:`configure` script will let you |
| 1488 |
profile the interpreter with :program:`gprof`, and providing the |
| 1489 |
:option:`!--with-tsc` switch enables profiling using the Pentium's |
| 1490 |
Time-Stamp-Counter register. Note that the :option:`!--with-tsc` switch is slightly |
| 1491 |
misnamed, because the profiling feature also works on the PowerPC platform, |
| 1492 |
though that processor architecture doesn't call that register "the TSC |
| 1493 |
register". (Contributed by Jeremy Hylton.) |
| 1494 |
|
| 1495 |
* The :c:type:`!tracebackobject` type has been renamed to |
| 1496 |
:c:type:`PyTracebackObject`. |
| 1497 |
|
| 1498 |
.. ====================================================================== |
| 1499 |
|
| 1500 |
|
| 1501 |
Port-Specific Changes |
| 1502 |
--------------------- |
| 1503 |
|
| 1504 |
* The Windows port now builds under MSVC++ 7.1 as well as version 6. |
| 1505 |
(Contributed by Martin von Löwis.) |
| 1506 |
|
| 1507 |
.. ====================================================================== |
| 1508 |
|
| 1509 |
|
| 1510 |
Porting to Python 2.4 |
| 1511 |
===================== |
| 1512 |
|
| 1513 |
This section lists previously described changes that may require changes to your |
| 1514 |
code: |
| 1515 |
|
| 1516 |
* Left shifts and hexadecimal/octal constants that are too large no longer |
| 1517 |
trigger a :exc:`FutureWarning` and return a value limited to 32 or 64 bits; |
| 1518 |
instead they return a long integer. |
| 1519 |
|
| 1520 |
* Integer operations will no longer trigger an :exc:`OverflowWarning`. The |
| 1521 |
:exc:`OverflowWarning` warning will disappear in Python 2.5. |
| 1522 |
|
| 1523 |
* The :func:`zip` built-in function and :func:`itertools.izip` now return an |
| 1524 |
empty list instead of raising a :exc:`TypeError` exception if called with no |
| 1525 |
arguments. |
| 1526 |
|
| 1527 |
* You can no longer compare the :class:`date` and :class:`~datetime.datetime` instances |
| 1528 |
provided by the :mod:`datetime` module. Two instances of different classes |
| 1529 |
will now always be unequal, and relative comparisons (``<``, ``>``) will raise |
| 1530 |
a :exc:`TypeError`. |
| 1531 |
|
| 1532 |
* :func:`!dircache.listdir` now passes exceptions to the caller instead of |
| 1533 |
returning empty lists. |
| 1534 |
|
| 1535 |
* :func:`LexicalHandler.startDTD` used to receive the public and system IDs in |
| 1536 |
the wrong order. This has been corrected; applications relying on the wrong |
| 1537 |
order need to be fixed. |
| 1538 |
|
| 1539 |
* :func:`fcntl.ioctl` now warns if the *mutate* argument is omitted and |
| 1540 |
relevant. |
| 1541 |
|
| 1542 |
* The :mod:`tarfile` module now generates GNU-format tar files by default. |
| 1543 |
|
| 1544 |
* Encountering a failure while importing a module no longer leaves a |
| 1545 |
partially initialized module object in ``sys.modules``. |
| 1546 |
|
| 1547 |
* :const:`None` is now a constant; code that binds a new value to the name |
| 1548 |
``None`` is now a syntax error. |
| 1549 |
|
| 1550 |
* The :func:`signals.signal` function now raises a :exc:`RuntimeError` exception |
| 1551 |
for certain illegal values; previously these errors would pass silently. For |
| 1552 |
example, you can no longer set a handler on the :const:`SIGKILL` signal. |
| 1553 |
|
| 1554 |
.. ====================================================================== |
| 1555 |
|
| 1556 |
|
| 1557 |
.. _24acks: |
| 1558 |
|
| 1559 |
Acknowledgements |
| 1560 |
================ |
| 1561 |
|
| 1562 |
The author would like to thank the following people for offering suggestions, |
| 1563 |
corrections and assistance with various drafts of this article: Koray Can, |
| 1564 |
Hye-Shik Chang, Michael Dyck, Raymond Hettinger, Brian Hurt, Hamish Lawson, |
| 1565 |
Fredrik Lundh, Sean Reifschneider, Sadruddin Rejeb. |
| 1566 |
|