Debian Codegraph @

Doc/whatsnew/2.2.rst

raw
1 ****************************
2 What's New in Python 2.2
3 ****************************
4
5 :Author: A.M. Kuchling
6
7 .. |release| replace:: 1.02
8
9 .. $Id: whatsnew22.tex 37315 2004-09-10 19:33:00Z akuchling $
10
11
12 Introduction
13 ============
14
15 This article explains the new features in Python 2.2.2, released on October 14,
16 2002. Python 2.2.2 is a bugfix release of Python 2.2, originally released on
17 December 21, 2001.
18
19 Python 2.2 can be thought of as the "cleanup release". There are some features
20 such as generators and iterators that are completely new, but most of the
21 changes, significant and far-reaching though they may be, are aimed at cleaning
22 up irregularities and dark corners of the language design.
23
24 This article doesn't attempt to provide a complete specification of the new
25 features, but instead provides a convenient overview. For full details, you
26 should refer to the documentation for Python 2.2, such as the `Python Library
27 Reference <https://docs.python.org/2.2/lib/lib.html>`_ and the `Python
28 Reference Manual <https://docs.python.org/2.2/ref/ref.html>`_. If you want to
29 understand the complete implementation and design rationale for a change, refer
30 to the PEP for a particular new feature.
31
32
33 .. see also, now defunct
34
35 http://www.unixreview.com/documents/s=1356/urm0109h/0109h.htm
36 "What's So Special About Python 2.2?" is also about the new 2.2 features, and
37 was written by Cameron Laird and Kathryn Soraiz.
38
39 .. ======================================================================
40
41
42 PEPs 252 and 253: Type and Class Changes
43 ========================================
44
45 The largest and most far-reaching changes in Python 2.2 are to Python's model of
46 objects and classes. The changes should be backward compatible, so it's likely
47 that your code will continue to run unchanged, but the changes provide some
48 amazing new capabilities. Before beginning this, the longest and most
49 complicated section of this article, I'll provide an overview of the changes and
50 offer some comments.
51
52 A long time ago I wrote a web page listing flaws in Python's design. One of the
53 most significant flaws was that it's impossible to subclass Python types
54 implemented in C. In particular, it's not possible to subclass built-in types,
55 so you can't just subclass, say, lists in order to add a single useful method to
56 them. The :mod:`!UserList` module provides a class that supports all of the
57 methods of lists and that can be subclassed further, but there's lots of C code
58 that expects a regular Python list and won't accept a :class:`~collections.UserList`
59 instance.
60
61 Python 2.2 fixes this, and in the process adds some exciting new capabilities.
62 A brief summary:
63
64 * You can subclass built-in types such as lists and even integers, and your
65 subclasses should work in every place that requires the original type.
66
67 * It's now possible to define static and class methods, in addition to the
68 instance methods available in previous versions of Python.
69
70 * It's also possible to automatically call methods on accessing or setting an
71 instance attribute by using a new mechanism called :dfn:`properties`. Many uses
72 of :meth:`~object.__getattr__` can be rewritten to use properties instead, making the
73 resulting code simpler and faster. As a small side benefit, attributes can now
74 have docstrings, too.
75
76 * The list of legal attributes for an instance can be limited to a particular
77 set using :dfn:`slots`, making it possible to safeguard against typos and
78 perhaps make more optimizations possible in future versions of Python.
79
80 Some users have voiced concern about all these changes. Sure, they say, the new
81 features are neat and lend themselves to all sorts of tricks that weren't
82 possible in previous versions of Python, but they also make the language more
83 complicated. Some people have said that they've always recommended Python for
84 its simplicity, and feel that its simplicity is being lost.
85
86 Personally, I think there's no need to worry. Many of the new features are
87 quite esoteric, and you can write a lot of Python code without ever needed to be
88 aware of them. Writing a simple class is no more difficult than it ever was, so
89 you don't need to bother learning or teaching them unless they're actually
90 needed. Some very complicated tasks that were previously only possible from C
91 will now be possible in pure Python, and to my mind that's all for the better.
92
93 I'm not going to attempt to cover every single corner case and small change that
94 were required to make the new features work. Instead this section will paint
95 only the broad strokes. See section :ref:`sect-rellinks`, "Related Links", for
96 further sources of information about Python 2.2's new object model.
97
98
99 Old and New Classes
100 -------------------
101
102 First, you should know that Python 2.2 really has two kinds of classes: classic
103 or old-style classes, and new-style classes. The old-style class model is
104 exactly the same as the class model in earlier versions of Python. All the new
105 features described in this section apply only to new-style classes. This
106 divergence isn't intended to last forever; eventually old-style classes will be
107 dropped, possibly in Python 3.0.
108
109 So how do you define a new-style class? You do it by subclassing an existing
110 new-style class. Most of Python's built-in types, such as integers, lists,
111 dictionaries, and even files, are new-style classes now. A new-style class
112 named :class:`object`, the base class for all built-in types, has also been
113 added so if no built-in type is suitable, you can just subclass
114 :class:`object`::
115
116 class C(object):
117 def __init__ (self):
118 ...
119 ...
120
121 This means that :keyword:`class` statements that don't have any base classes are
122 always classic classes in Python 2.2. (Actually you can also change this by
123 setting a module-level variable named :attr:`!__metaclass__` --- see :pep:`253`
124 for the details --- but it's easier to just subclass :class:`object`.)
125
126 The type objects for the built-in types are available as built-ins, named using
127 a clever trick. Python has always had built-in functions named :func:`int`,
128 :func:`float`, and :func:`str`. In 2.2, they aren't functions any more, but
129 type objects that behave as factories when called. ::
130
131 >>> int
132 <type 'int'>
133 >>> int('123')
134 123
135
136 To make the set of types complete, new type objects such as :func:`dict` and
137 :func:`!file` have been added. Here's a more interesting example, adding a
138 :meth:`!lock` method to file objects::
139
140 class LockableFile(file):
141 def lock (self, operation, length=0, start=0, whence=0):
142 import fcntl
143 return fcntl.lockf(self.fileno(), operation,
144 length, start, whence)
145
146 The now-obsolete :mod:`!posixfile` module contained a class that emulated all of
147 a file object's methods and also added a :meth:`!lock` method, but this class
148 couldn't be passed to internal functions that expected a built-in file,
149 something which is possible with our new :class:`!LockableFile`.
150
151
152 Descriptors
153 -----------
154
155 In previous versions of Python, there was no consistent way to discover what
156 attributes and methods were supported by an object. There were some informal
157 conventions, such as defining :attr:`!__members__` and :attr:`!__methods__`
158 attributes that were lists of names, but often the author of an extension type
159 or a class wouldn't bother to define them. You could fall back on inspecting
160 the :attr:`~object.__dict__` of an object, but when class inheritance or an arbitrary
161 :meth:`!__getattr__` hook were in use this could still be inaccurate.
162
163 The one big idea underlying the new class model is that an API for describing
164 the attributes of an object using :dfn:`descriptors` has been formalized.
165 Descriptors specify the value of an attribute, stating whether it's a method or
166 a field. With the descriptor API, static methods and class methods become
167 possible, as well as more exotic constructs.
168
169 Attribute descriptors are objects that live inside class objects, and have a few
170 attributes of their own:
171
172 * :attr:`~definition.__name__` is the attribute's name.
173
174 * :attr:`~definition.__doc__` is the attribute's docstring.
175
176 * ``__get__(object)`` is a method that retrieves the attribute value from
177 *object*.
178
179 * ``__set__(object, value)`` sets the attribute on *object* to *value*.
180
181 * ``__delete__(object, value)`` deletes the *value* attribute of *object*.
182
183 For example, when you write ``obj.x``, the steps that Python actually performs
184 are::
185
186 descriptor = obj.__class__.x
187 descriptor.__get__(obj)
188
189 For methods, :meth:`descriptor.__get__ <object.__get__>` returns a temporary
190 object that's
191 callable, and wraps up the instance and the method to be called on it. This is
192 also why static methods and class methods are now possible; they have
193 descriptors that wrap up just the method, or the method and the class. As a
194 brief explanation of these new kinds of methods, static methods aren't passed
195 the instance, and therefore resemble regular functions. Class methods are
196 passed the class of the object, but not the object itself. Static and class
197 methods are defined like this::
198
199 class C(object):
200 def f(arg1, arg2):
201 ...
202 f = staticmethod(f)
203
204 def g(cls, arg1, arg2):
205 ...
206 g = classmethod(g)
207
208 The :func:`staticmethod` function takes the function :func:`!f`, and returns it
209 wrapped up in a descriptor so it can be stored in the class object. You might
210 expect there to be special syntax for creating such methods (``def static f``,
211 ``defstatic f()``, or something like that) but no such syntax has been defined
212 yet; that's been left for future versions of Python.
213
214 More new features, such as slots and properties, are also implemented as new
215 kinds of descriptors, and it's not difficult to write a descriptor class that
216 does something novel. For example, it would be possible to write a descriptor
217 class that made it possible to write Eiffel-style preconditions and
218 postconditions for a method. A class that used this feature might be defined
219 like this::
220
221 from eiffel import eiffelmethod
222
223 class C(object):
224 def f(self, arg1, arg2):
225 # The actual function
226 ...
227 def pre_f(self):
228 # Check preconditions
229 ...
230 def post_f(self):
231 # Check postconditions
232 ...
233
234 f = eiffelmethod(f, pre_f, post_f)
235
236 Note that a person using the new :func:`!eiffelmethod` doesn't have to understand
237 anything about descriptors. This is why I think the new features don't increase
238 the basic complexity of the language. There will be a few wizards who need to
239 know about it in order to write :func:`!eiffelmethod` or the ZODB or whatever,
240 but most users will just write code on top of the resulting libraries and ignore
241 the implementation details.
242
243
244 Multiple Inheritance: The Diamond Rule
245 --------------------------------------
246
247 Multiple inheritance has also been made more useful through changing the rules
248 under which names are resolved. Consider this set of classes (diagram taken
249 from :pep:`253` by Guido van Rossum)::
250
251 class A:
252 ^ ^ def save(self): ...
253 / \
254 / \
255 / \
256 / \
257 class B class C:
258 ^ ^ def save(self): ...
259 \ /
260 \ /
261 \ /
262 \ /
263 class D
264
265 The lookup rule for classic classes is simple but not very smart; the base
266 classes are searched depth-first, going from left to right. A reference to
267 :meth:`!D.save` will search the classes :class:`!D`, :class:`!B`, and then
268 :class:`!A`, where :meth:`!save` would be found and returned. :meth:`!C.save`
269 would never be found at all. This is bad, because if :class:`!C`'s :meth:`!save`
270 method is saving some internal state specific to :class:`!C`, not calling it will
271 result in that state never getting saved.
272
273 New-style classes follow a different algorithm that's a bit more complicated to
274 explain, but does the right thing in this situation. (Note that Python 2.3
275 changes this algorithm to one that produces the same results in most cases, but
276 produces more useful results for really complicated inheritance graphs.)
277
278 #. List all the base classes, following the classic lookup rule and include a
279 class multiple times if it's visited repeatedly. In the above example, the list
280 of visited classes is [:class:`!D`, :class:`!B`, :class:`!A`, :class:`!C`,
281 :class:`!A`].
282
283 #. Scan the list for duplicated classes. If any are found, remove all but one
284 occurrence, leaving the *last* one in the list. In the above example, the list
285 becomes [:class:`!D`, :class:`!B`, :class:`!C`, :class:`!A`] after dropping
286 duplicates.
287
288 Following this rule, referring to :meth:`!D.save` will return :meth:`!C.save`,
289 which is the behaviour we're after. This lookup rule is the same as the one
290 followed by Common Lisp. A new built-in function, :func:`super`, provides a way
291 to get at a class's superclasses without having to reimplement Python's
292 algorithm. The most commonly used form will be ``super(class, obj)``, which
293 returns a bound superclass object (not the actual class object). This form
294 will be used in methods to call a method in the superclass; for example,
295 :class:`!D`'s :meth:`!save` method would look like this::
296
297 class D (B,C):
298 def save (self):
299 # Call superclass .save()
300 super(D, self).save()
301 # Save D's private information here
302 ...
303
304 :func:`super` can also return unbound superclass objects when called as
305 ``super(class)`` or ``super(class1, class2)``, but this probably won't
306 often be useful.
307
308
309 Attribute Access
310 ----------------
311
312 A fair number of sophisticated Python classes define hooks for attribute access
313 using :meth:`~object.__getattr__`; most commonly this is done for convenience, to make
314 code more readable by automatically mapping an attribute access such as
315 ``obj.parent`` into a method call such as ``obj.get_parent``. Python 2.2 adds
316 some new ways of controlling attribute access.
317
318 First, ``__getattr__(attr_name)`` is still supported by new-style classes,
319 and nothing about it has changed. As before, it will be called when an attempt
320 is made to access ``obj.foo`` and no attribute named ``foo`` is found in the
321 instance's dictionary.
322
323 New-style classes also support a new method,
324 ``__getattribute__(attr_name)``. The difference between the two methods is
325 that :meth:`~object.__getattribute__` is *always* called whenever any attribute is
326 accessed, while the old :meth:`~object.__getattr__` is only called if ``foo`` isn't
327 found in the instance's dictionary.
328
329 However, Python 2.2's support for :dfn:`properties` will often be a simpler way
330 to trap attribute references. Writing a :meth:`!__getattr__` method is
331 complicated because to avoid recursion you can't use regular attribute accesses
332 inside them, and instead have to mess around with the contents of
333 :attr:`~object.__dict__`. :meth:`~object.__getattr__` methods also end up being called by Python
334 when it checks for other methods such as :meth:`~object.__repr__` or :meth:`!__coerce__`,
335 and so have to be written with this in mind. Finally, calling a function on
336 every attribute access results in a sizable performance loss.
337
338 :class:`property` is a new built-in type that packages up three functions that
339 get, set, or delete an attribute, and a docstring. For example, if you want to
340 define a :attr:`!size` attribute that's computed, but also settable, you could
341 write::
342
343 class C(object):
344 def get_size (self):
345 result = ... computation ...
346 return result
347 def set_size (self, size):
348 ... compute something based on the size
349 and set internal state appropriately ...
350
351 # Define a property. The 'delete this attribute'
352 # method is defined as None, so the attribute
353 # can't be deleted.
354 size = property(get_size, set_size,
355 None,
356 "Storage size of this instance")
357
358 That is certainly clearer and easier to write than a pair of
359 :meth:`!__getattr__`/:meth:`!__setattr__` methods that check for the :attr:`!size`
360 attribute and handle it specially while retrieving all other attributes from the
361 instance's :attr:`~object.__dict__`. Accesses to :attr:`!size` are also the only ones
362 which have to perform the work of calling a function, so references to other
363 attributes run at their usual speed.
364
365 Finally, it's possible to constrain the list of attributes that can be
366 referenced on an object using the new :attr:`~object.__slots__` class attribute. Python
367 objects are usually very dynamic; at any time it's possible to define a new
368 attribute on an instance by just doing ``obj.new_attr=1``. A new-style class
369 can define a class attribute named :attr:`~object.__slots__` to limit the legal
370 attributes to a particular set of names. An example will make this clear::
371
372 >>> class C(object):
373 ... __slots__ = ('template', 'name')
374 ...
375 >>> obj = C()
376 >>> print obj.template
377 None
378 >>> obj.template = 'Test'
379 >>> print obj.template
380 Test
381 >>> obj.newattr = None
382 Traceback (most recent call last):
383 File "<stdin>", line 1, in ?
384 AttributeError: 'C' object has no attribute 'newattr'
385
386 Note how you get an :exc:`AttributeError` on the attempt to assign to an
387 attribute not listed in :attr:`~object.__slots__`.
388
389
390 .. _sect-rellinks:
391
392 Related Links
393 -------------
394
395 This section has just been a quick overview of the new features, giving enough
396 of an explanation to start you programming, but many details have been
397 simplified or ignored. Where should you go to get a more complete picture?
398
399 The :ref:`descriptorhowto` is a lengthy tutorial introduction to
400 the descriptor features, written by Guido van Rossum. If my description has
401 whetted your appetite, go read this tutorial next, because it goes into much
402 more detail about the new features while still remaining quite easy to read.
403
404 Next, there are two relevant PEPs, :pep:`252` and :pep:`253`. :pep:`252` is
405 titled "Making Types Look More Like Classes", and covers the descriptor API.
406 :pep:`253` is titled "Subtyping Built-in Types", and describes the changes to
407 type objects that make it possible to subtype built-in objects. :pep:`253` is
408 the more complicated PEP of the two, and at a few points the necessary
409 explanations of types and meta-types may cause your head to explode. Both PEPs
410 were written and implemented by Guido van Rossum, with substantial assistance
411 from the rest of the Zope Corp. team.
412
413 Finally, there's the ultimate authority: the source code. Most of the machinery
414 for the type handling is in :file:`Objects/typeobject.c`, but you should only
415 resort to it after all other avenues have been exhausted, including posting a
416 question to python-list or python-dev.
417
418 .. ======================================================================
419
420
421 PEP 234: Iterators
422 ==================
423
424 Another significant addition to 2.2 is an iteration interface at both the C and
425 Python levels. Objects can define how they can be looped over by callers.
426
427 In Python versions up to 2.1, the usual way to make ``for item in obj`` work is
428 to define a :meth:`~object.__getitem__` method that looks something like this::
429
430 def __getitem__(self, index):
431 return <next item>
432
433 :meth:`~object.__getitem__` is more properly used to define an indexing operation on an
434 object so that you can write ``obj[5]`` to retrieve the sixth element. It's a
435 bit misleading when you're using this only to support :keyword:`for` loops.
436 Consider some file-like object that wants to be looped over; the *index*
437 parameter is essentially meaningless, as the class probably assumes that a
438 series of :meth:`~object.__getitem__` calls will be made with *index* incrementing by
439 one each time. In other words, the presence of the :meth:`~object.__getitem__` method
440 doesn't mean that using ``file[5]`` to randomly access the sixth element will
441 work, though it really should.
442
443 In Python 2.2, iteration can be implemented separately, and :meth:`~object.__getitem__`
444 methods can be limited to classes that really do support random access. The
445 basic idea of iterators is simple. A new built-in function, ``iter(obj)``
446 or ``iter(C, sentinel)``, is used to get an iterator. ``iter(obj)`` returns
447 an iterator for the object *obj*, while ``iter(C, sentinel)`` returns an
448 iterator that will invoke the callable object *C* until it returns *sentinel* to
449 signal that the iterator is done.
450
451 Python classes can define an :meth:`!__iter__` method, which should create and
452 return a new iterator for the object; if the object is its own iterator, this
453 method can just return ``self``. In particular, iterators will usually be their
454 own iterators. Extension types implemented in C can implement a :c:member:`~PyTypeObject.tp_iter`
455 function in order to return an iterator, and extension types that want to behave
456 as iterators can define a :c:member:`~PyTypeObject.tp_iternext` function.
457
458 So, after all this, what do iterators actually do? They have one required
459 method, :meth:`next`, which takes no arguments and returns the next value. When
460 there are no more values to be returned, calling :meth:`next` should raise the
461 :exc:`StopIteration` exception. ::
462
463 >>> L = [1,2,3]
464 >>> i = iter(L)
465 >>> print i
466 <iterator object at 0x8116870>
467 >>> i.next()
468 1
469 >>> i.next()
470 2
471 >>> i.next()
472 3
473 >>> i.next()
474 Traceback (most recent call last):
475 File "<stdin>", line 1, in ?
476 StopIteration
477 >>>
478
479 In 2.2, Python's :keyword:`for` statement no longer expects a sequence; it
480 expects something for which :func:`iter` will return an iterator. For backward
481 compatibility and convenience, an iterator is automatically constructed for
482 sequences that don't implement :meth:`!__iter__` or a :c:member:`~PyTypeObject.tp_iter` slot, so
483 ``for i in [1,2,3]`` will still work. Wherever the Python interpreter loops
484 over a sequence, it's been changed to use the iterator protocol. This means you
485 can do things like this::
486
487 >>> L = [1,2,3]
488 >>> i = iter(L)
489 >>> a,b,c = i
490 >>> a,b,c
491 (1, 2, 3)
492
493 Iterator support has been added to some of Python's basic types. Calling
494 :func:`iter` on a dictionary will return an iterator which loops over its keys::
495
496 >>> m = {'Jan': 1, 'Feb': 2, 'Mar': 3, 'Apr': 4, 'May': 5, 'Jun': 6,
497 ... 'Jul': 7, 'Aug': 8, 'Sep': 9, 'Oct': 10, 'Nov': 11, 'Dec': 12}
498 >>> for key in m: print key, m[key]
499 ...
500 Mar 3
501 Feb 2
502 Aug 8
503 Sep 9
504 May 5
505 Jun 6
506 Jul 7
507 Jan 1
508 Apr 4
509 Nov 11
510 Dec 12
511 Oct 10
512
513 That's just the default behaviour. If you want to iterate over keys, values, or
514 key/value pairs, you can explicitly call the :meth:`!iterkeys`,
515 :meth:`!itervalues`, or :meth:`!iteritems` methods to get an appropriate iterator.
516 In a minor related change, the :keyword:`in` operator now works on dictionaries,
517 so ``key in dict`` is now equivalent to ``dict.has_key(key)``.
518
519 Files also provide an iterator, which calls the :meth:`readline` method until
520 there are no more lines in the file. This means you can now read each line of a
521 file using code like this::
522
523 for line in file:
524 # do something for each line
525 ...
526
527 Note that you can only go forward in an iterator; there's no way to get the
528 previous element, reset the iterator, or make a copy of it. An iterator object
529 could provide such additional capabilities, but the iterator protocol only
530 requires a :meth:`next` method.
531
532
533 .. seealso::
534
535 :pep:`234` - Iterators
536 Written by Ka-Ping Yee and GvR; implemented by the Python Labs crew, mostly by
537 GvR and Tim Peters.
538
539 .. ======================================================================
540
541
542 PEP 255: Simple Generators
543 ==========================
544
545 Generators are another new feature, one that interacts with the introduction of
546 iterators.
547
548 You're doubtless familiar with how function calls work in Python or C. When you
549 call a function, it gets a private namespace where its local variables are
550 created. When the function reaches a :keyword:`return` statement, the local
551 variables are destroyed and the resulting value is returned to the caller. A
552 later call to the same function will get a fresh new set of local variables.
553 But, what if the local variables weren't thrown away on exiting a function?
554 What if you could later resume the function where it left off? This is what
555 generators provide; they can be thought of as resumable functions.
556
557 Here's the simplest example of a generator function::
558
559 def generate_ints(N):
560 for i in range(N):
561 yield i
562
563 A new keyword, :keyword:`yield`, was introduced for generators. Any function
564 containing a :keyword:`!yield` statement is a generator function; this is
565 detected by Python's bytecode compiler which compiles the function specially as
566 a result. Because a new keyword was introduced, generators must be explicitly
567 enabled in a module by including a ``from __future__ import generators``
568 statement near the top of the module's source code. In Python 2.3 this
569 statement will become unnecessary.
570
571 When you call a generator function, it doesn't return a single value; instead it
572 returns a generator object that supports the iterator protocol. On executing
573 the :keyword:`yield` statement, the generator outputs the value of ``i``,
574 similar to a :keyword:`return` statement. The big difference between
575 :keyword:`!yield` and a :keyword:`!return` statement is that on reaching a
576 :keyword:`!yield` the generator's state of execution is suspended and local
577 variables are preserved. On the next call to the generator's ``next()`` method,
578 the function will resume executing immediately after the :keyword:`!yield`
579 statement. (For complicated reasons, the :keyword:`!yield` statement isn't
580 allowed inside the :keyword:`!try` block of a
581 :keyword:`try`...\ :keyword:`finally` statement; read :pep:`255` for a full
582 explanation of the interaction between :keyword:`!yield` and exceptions.)
583
584 Here's a sample usage of the :func:`!generate_ints` generator::
585
586 >>> gen = generate_ints(3)
587 >>> gen
588 <generator object at 0x8117f90>
589 >>> gen.next()
590 0
591 >>> gen.next()
592 1
593 >>> gen.next()
594 2
595 >>> gen.next()
596 Traceback (most recent call last):
597 File "<stdin>", line 1, in ?
598 File "<stdin>", line 2, in generate_ints
599 StopIteration
600
601 You could equally write ``for i in generate_ints(5)``, or ``a,b,c =
602 generate_ints(3)``.
603
604 Inside a generator function, the :keyword:`return` statement can only be used
605 without a value, and signals the end of the procession of values; afterwards the
606 generator cannot return any further values. :keyword:`!return` with a value, such
607 as ``return 5``, is a syntax error inside a generator function. The end of the
608 generator's results can also be indicated by raising :exc:`StopIteration`
609 manually, or by just letting the flow of execution fall off the bottom of the
610 function.
611
612 You could achieve the effect of generators manually by writing your own class
613 and storing all the local variables of the generator as instance variables. For
614 example, returning a list of integers could be done by setting ``self.count`` to
615 0, and having the :meth:`next` method increment ``self.count`` and return it.
616 However, for a moderately complicated generator, writing a corresponding class
617 would be much messier. :file:`Lib/test/test_generators.py` contains a number of
618 more interesting examples. The simplest one implements an in-order traversal of
619 a tree using generators recursively. ::
620
621 # A recursive generator that generates Tree leaves in in-order.
622 def inorder(t):
623 if t:
624 for x in inorder(t.left):
625 yield x
626 yield t.label
627 for x in inorder(t.right):
628 yield x
629
630 Two other examples in :file:`Lib/test/test_generators.py` produce solutions for
631 the N-Queens problem (placing $N$ queens on an $NxN$ chess board so that no
632 queen threatens another) and the Knight's Tour (a route that takes a knight to
633 every square of an $NxN$ chessboard without visiting any square twice).
634
635 The idea of generators comes from other programming languages, especially Icon
636 (https://www2.cs.arizona.edu/icon/), where the idea of generators is central. In
637 Icon, every expression and function call behaves like a generator. One example
638 from "An Overview of the Icon Programming Language" at
639 https://www2.cs.arizona.edu/icon/docs/ipd266.htm gives an idea of what this looks
640 like::
641
642 sentence := "Store it in the neighboring harbor"
643 if (i := find("or", sentence)) > 5 then write(i)
644
645 In Icon the :func:`!find` function returns the indexes at which the substring
646 "or" is found: 3, 23, 33. In the :keyword:`if` statement, ``i`` is first
647 assigned a value of 3, but 3 is less than 5, so the comparison fails, and Icon
648 retries it with the second value of 23. 23 is greater than 5, so the comparison
649 now succeeds, and the code prints the value 23 to the screen.
650
651 Python doesn't go nearly as far as Icon in adopting generators as a central
652 concept. Generators are considered a new part of the core Python language, but
653 learning or using them isn't compulsory; if they don't solve any problems that
654 you have, feel free to ignore them. One novel feature of Python's interface as
655 compared to Icon's is that a generator's state is represented as a concrete
656 object (the iterator) that can be passed around to other functions or stored in
657 a data structure.
658
659
660 .. seealso::
661
662 :pep:`255` - Simple Generators
663 Written by Neil Schemenauer, Tim Peters, Magnus Lie Hetland. Implemented mostly
664 by Neil Schemenauer and Tim Peters, with other fixes from the Python Labs crew.
665
666 .. ======================================================================
667
668
669 PEP 237: Unifying Long Integers and Integers
670 ============================================
671
672 In recent versions, the distinction between regular integers, which are 32-bit
673 values on most machines, and long integers, which can be of arbitrary size, was
674 becoming an annoyance. For example, on platforms that support files larger than
675 ``2**32`` bytes, the :meth:`!tell` method of file objects has to return a long
676 integer. However, there were various bits of Python that expected plain integers
677 and would raise an error if a long integer was provided instead. For example,
678 in Python 1.5, only regular integers could be used as a slice index, and
679 ``'abc'[1L:]`` would raise a :exc:`TypeError` exception with the message 'slice
680 index must be int'.
681
682 Python 2.2 will shift values from short to long integers as required. The 'L'
683 suffix is no longer needed to indicate a long integer literal, as now the
684 compiler will choose the appropriate type. (Using the 'L' suffix will be
685 discouraged in future 2.x versions of Python, triggering a warning in Python
686 2.4, and probably dropped in Python 3.0.) Many operations that used to raise an
687 :exc:`OverflowError` will now return a long integer as their result. For
688 example::
689
690 >>> 1234567890123
691 1234567890123L
692 >>> 2 ** 64
693 18446744073709551616L
694
695 In most cases, integers and long integers will now be treated identically. You
696 can still distinguish them with the :func:`type` built-in function, but that's
697 rarely needed.
698
699
700 .. seealso::
701
702 :pep:`237` - Unifying Long Integers and Integers
703 Written by Moshe Zadka and Guido van Rossum. Implemented mostly by Guido van
704 Rossum.
705
706 .. ======================================================================
707
708
709 PEP 238: Changing the Division Operator
710 =======================================
711
712 The most controversial change in Python 2.2 heralds the start of an effort to
713 fix an old design flaw that's been in Python from the beginning. Currently
714 Python's division operator, ``/``, behaves like C's division operator when
715 presented with two integer arguments: it returns an integer result that's
716 truncated down when there would be a fractional part. For example, ``3/2`` is
717 1, not 1.5, and ``(-1)/2`` is -1, not -0.5. This means that the results of
718 division can vary unexpectedly depending on the type of the two operands and
719 because Python is dynamically typed, it can be difficult to determine the
720 possible types of the operands.
721
722 (The controversy is over whether this is *really* a design flaw, and whether
723 it's worth breaking existing code to fix this. It's caused endless discussions
724 on python-dev, and in July 2001 erupted into a storm of acidly sarcastic
725 postings on :newsgroup:`comp.lang.python`. I won't argue for either side here
726 and will stick to describing what's implemented in 2.2. Read :pep:`238` for a
727 summary of arguments and counter-arguments.)
728
729 Because this change might break code, it's being introduced very gradually.
730 Python 2.2 begins the transition, but the switch won't be complete until Python
731 3.0.
732
733 First, I'll borrow some terminology from :pep:`238`. "True division" is the
734 division that most non-programmers are familiar with: 3/2 is 1.5, 1/4 is 0.25,
735 and so forth. "Floor division" is what Python's ``/`` operator currently does
736 when given integer operands; the result is the floor of the value returned by
737 true division. "Classic division" is the current mixed behaviour of ``/``; it
738 returns the result of floor division when the operands are integers, and returns
739 the result of true division when one of the operands is a floating-point number.
740
741 Here are the changes 2.2 introduces:
742
743 * A new operator, ``//``, is the floor division operator. (Yes, we know it looks
744 like C++'s comment symbol.) ``//`` *always* performs floor division no matter
745 what the types of its operands are, so ``1 // 2`` is 0 and ``1.0 // 2.0`` is
746 also 0.0.
747
748 ``//`` is always available in Python 2.2; you don't need to enable it using a
749 ``__future__`` statement.
750
751 * By including a ``from __future__ import division`` in a module, the ``/``
752 operator will be changed to return the result of true division, so ``1/2`` is
753 0.5. Without the ``__future__`` statement, ``/`` still means classic division.
754 The default meaning of ``/`` will not change until Python 3.0.
755
756 * Classes can define methods called :meth:`~object.__truediv__` and :meth:`~object.__floordiv__`
757 to overload the two division operators. At the C level, there are also slots in
758 the :c:type:`PyNumberMethods` structure so extension types can define the two
759 operators.
760
761 * Python 2.2 supports some command-line arguments for testing whether code will
762 work with the changed division semantics. Running python with :option:`!-Q
763 warn` will cause a warning to be issued whenever division is applied to two
764 integers. You can use this to find code that's affected by the change and fix
765 it. By default, Python 2.2 will simply perform classic division without a
766 warning; the warning will be turned on by default in Python 2.3.
767
768
769 .. seealso::
770
771 :pep:`238` - Changing the Division Operator
772 Written by Moshe Zadka and Guido van Rossum. Implemented by Guido van Rossum..
773
774 .. ======================================================================
775
776
777 Unicode Changes
778 ===============
779
780 Python's Unicode support has been enhanced a bit in 2.2. Unicode strings are
781 usually stored as UCS-2, as 16-bit unsigned integers. Python 2.2 can also be
782 compiled to use UCS-4, 32-bit unsigned integers, as its internal encoding by
783 supplying :option:`!--enable-unicode=ucs4` to the configure script. (It's also
784 possible to specify :option:`!--disable-unicode` to completely disable Unicode
785 support.)
786
787 When built to use UCS-4 (a "wide Python"), the interpreter can natively handle
788 Unicode characters from U+000000 to U+110000, so the range of legal values for
789 the :func:`!unichr` function is expanded accordingly. Using an interpreter
790 compiled to use UCS-2 (a "narrow Python"), values greater than 65535 will still
791 cause :func:`!unichr` to raise a :exc:`ValueError` exception. This is all
792 described in :pep:`261`, "Support for 'wide' Unicode characters"; consult it for
793 further details.
794
795 Another change is simpler to explain. Since their introduction, Unicode strings
796 have supported an :meth:`!encode` method to convert the string to a selected
797 encoding such as UTF-8 or Latin-1. A symmetric ``decode([*encoding*])``
798 method has been added to 8-bit strings (though not to Unicode strings) in 2.2.
799 :meth:`!decode` assumes that the string is in the specified encoding and decodes
800 it, returning whatever is returned by the codec.
801
802 Using this new feature, codecs have been added for tasks not directly related to
803 Unicode. For example, codecs have been added for uu-encoding, MIME's base64
804 encoding, and compression with the :mod:`zlib` module::
805
806 >>> s = """Here is a lengthy piece of redundant, overly verbose,
807 ... and repetitive text.
808 ... """
809 >>> data = s.encode('zlib')
810 >>> data
811 'x\x9c\r\xc9\xc1\r\x80 \x10\x04\xc0?Ul...'
812 >>> data.decode('zlib')
813 'Here is a lengthy piece of redundant, overly verbose,\nand repetitive text.\n'
814 >>> print s.encode('uu')
815 begin 666 <data>
816 M2&5R92!I<R!A(&QE;F=T:'D@<&EE8V4@;V8@<F5D=6YD86YT+"!O=F5R;'D@
817 >=F5R8F]S92P*86YD(')E<&5T:71I=F4@=&5X="X*
818
819 end
820 >>> "sheesh".encode('rot-13')
821 'furrfu'
822
823 To convert a class instance to Unicode, a :meth:`!__unicode__` method can be
824 defined by a class, analogous to :meth:`!__str__`.
825
826 :meth:`!encode`, :meth:`!decode`, and :meth:`!__unicode__` were implemented by
827 Marc-André Lemburg. The changes to support using UCS-4 internally were
828 implemented by Fredrik Lundh and Martin von Löwis.
829
830
831 .. seealso::
832
833 :pep:`261` - Support for 'wide' Unicode characters
834 Written by Paul Prescod.
835
836 .. ======================================================================
837
838
839 PEP 227: Nested Scopes
840 ======================
841
842 In Python 2.1, statically nested scopes were added as an optional feature, to be
843 enabled by a ``from __future__ import nested_scopes`` directive. In 2.2 nested
844 scopes no longer need to be specially enabled, and are now always present. The
845 rest of this section is a copy of the description of nested scopes from my
846 "What's New in Python 2.1" document; if you read it when 2.1 came out, you can
847 skip the rest of this section.
848
849 The largest change introduced in Python 2.1, and made complete in 2.2, is to
850 Python's scoping rules. In Python 2.0, at any given time there are at most
851 three namespaces used to look up variable names: local, module-level, and the
852 built-in namespace. This often surprised people because it didn't match their
853 intuitive expectations. For example, a nested recursive function definition
854 doesn't work::
855
856 def f():
857 ...
858 def g(value):
859 ...
860 return g(value-1) + 1
861 ...
862
863 The function :func:`!g` will always raise a :exc:`NameError` exception, because
864 the binding of the name ``g`` isn't in either its local namespace or in the
865 module-level namespace. This isn't much of a problem in practice (how often do
866 you recursively define interior functions like this?), but this also made using
867 the :keyword:`lambda` expression clumsier, and this was a problem in practice.
868 In code which uses :keyword:`!lambda` you can often find local variables being
869 copied by passing them as the default values of arguments. ::
870
871 def find(self, name):
872 "Return list of any entries equal to 'name'"
873 L = filter(lambda x, name=name: x == name,
874 self.list_attribute)
875 return L
876
877 The readability of Python code written in a strongly functional style suffers
878 greatly as a result.
879
880 The most significant change to Python 2.2 is that static scoping has been added
881 to the language to fix this problem. As a first effect, the ``name=name``
882 default argument is now unnecessary in the above example. Put simply, when a
883 given variable name is not assigned a value within a function (by an assignment,
884 or the :keyword:`def`, :keyword:`class`, or :keyword:`import` statements),
885 references to the variable will be looked up in the local namespace of the
886 enclosing scope. A more detailed explanation of the rules, and a dissection of
887 the implementation, can be found in the PEP.
888
889 This change may cause some compatibility problems for code where the same
890 variable name is used both at the module level and as a local variable within a
891 function that contains further function definitions. This seems rather unlikely
892 though, since such code would have been pretty confusing to read in the first
893 place.
894
895 One side effect of the change is that the ``from module import *`` and
896 ``exec`` statements have been made illegal inside a function scope under
897 certain conditions. The Python reference manual has said all along that ``from
898 module import *`` is only legal at the top level of a module, but the CPython
899 interpreter has never enforced this before. As part of the implementation of
900 nested scopes, the compiler which turns Python source into bytecodes has to
901 generate different code to access variables in a containing scope. ``from
902 module import *`` and ``exec`` make it impossible for the compiler to
903 figure this out, because they add names to the local namespace that are
904 unknowable at compile time. Therefore, if a function contains function
905 definitions or :keyword:`lambda` expressions with free variables, the compiler
906 will flag this by raising a :exc:`SyntaxError` exception.
907
908 To make the preceding explanation a bit clearer, here's an example::
909
910 x = 1
911 def f():
912 # The next line is a syntax error
913 exec 'x=2'
914 def g():
915 return x
916
917 Line 4 containing the ``exec`` statement is a syntax error, since
918 ``exec`` would define a new local variable named ``x`` whose value should
919 be accessed by :func:`!g`.
920
921 This shouldn't be much of a limitation, since ``exec`` is rarely used in
922 most Python code (and when it is used, it's often a sign of a poor design
923 anyway).
924
925
926 .. seealso::
927
928 :pep:`227` - Statically Nested Scopes
929 Written and implemented by Jeremy Hylton.
930
931 .. ======================================================================
932
933
934 New and Improved Modules
935 ========================
936
937 * The :mod:`xmlrpclib <xmlrpc.client>` module was contributed to the standard library by Fredrik
938 Lundh, providing support for writing XML-RPC clients. XML-RPC is a simple
939 remote procedure call protocol built on top of HTTP and XML. For example, the
940 following snippet retrieves a list of RSS channels from the O'Reilly Network,
941 and then lists the recent headlines for one channel::
942
943 import xmlrpclib
944 s = xmlrpclib.Server(
945 'http://www.oreillynet.com/meerkat/xml-rpc/server.php')
946 channels = s.meerkat.getChannels()
947 # channels is a list of dictionaries, like this:
948 # [{'id': 4, 'title': 'Freshmeat Daily News'}
949 # {'id': 190, 'title': '32Bits Online'},
950 # {'id': 4549, 'title': '3DGamers'}, ... ]
951
952 # Get the items for one channel
953 items = s.meerkat.getItems( {'channel': 4} )
954
955 # 'items' is another list of dictionaries, like this:
956 # [{'link': 'http://freshmeat.net/releases/52719/',
957 # 'description': 'A utility which converts HTML to XSL FO.',
958 # 'title': 'html2fo 0.3 (Default)'}, ... ]
959
960 The :mod:`SimpleXMLRPCServer <xmlrpc.server>` module makes it easy to create straightforward
961 XML-RPC servers. See http://xmlrpc.scripting.com/ for more information about XML-RPC.
962
963 * The new :mod:`hmac` module implements the HMAC algorithm described by
964 :rfc:`2104`. (Contributed by Gerhard Häring.)
965
966 * Several functions that originally returned lengthy tuples now return
967 pseudo-sequences that still behave like tuples but also have mnemonic attributes such
968 as :attr:`!memberst_mtime` or :attr:`~time.struct_time.tm_year`. The enhanced functions include
969 :func:`~os.stat`, :func:`~os.fstat`, :func:`~os.statvfs`, and :func:`~os.fstatvfs` in the
970 :mod:`os` module, and :func:`~time.localtime`, :func:`~time.gmtime`, and :func:`~time.strptime` in
971 the :mod:`time` module.
972
973 For example, to obtain a file's size using the old tuples, you'd end up writing
974 something like ``file_size = os.stat(filename)[stat.ST_SIZE]``, but now this can
975 be written more clearly as ``file_size = os.stat(filename).st_size``.
976
977 The original patch for this feature was contributed by Nick Mathewson.
978
979 * The Python profiler has been extensively reworked and various errors in its
980 output have been corrected. (Contributed by Fred L. Drake, Jr. and Tim Peters.)
981
982 * The :mod:`socket` module can be compiled to support IPv6; specify the
983 :option:`!--enable-ipv6` option to Python's configure script. (Contributed by
984 Jun-ichiro "itojun" Hagino.)
985
986 * Two new format characters were added to the :mod:`struct` module for 64-bit
987 integers on platforms that support the C :c:expr:`long long` type. ``q`` is for
988 a signed 64-bit integer, and ``Q`` is for an unsigned one. The value is
989 returned in Python's long integer type. (Contributed by Tim Peters.)
990
991 * In the interpreter's interactive mode, there's a new built-in function
992 :func:`help` that uses the :mod:`pydoc` module introduced in Python 2.1 to
993 provide interactive help. ``help(object)`` displays any available help text
994 about *object*. :func:`help` with no argument puts you in an online help
995 utility, where you can enter the names of functions, classes, or modules to read
996 their help text. (Contributed by Guido van Rossum, using Ka-Ping Yee's
997 :mod:`pydoc` module.)
998
999 * Various bugfixes and performance improvements have been made to the SRE engine
1000 underlying the :mod:`re` module. For example, the :func:`re.sub` and
1001 :func:`re.split` functions have been rewritten in C. Another contributed patch
1002 speeds up certain Unicode character ranges by a factor of two, and a new
1003 :meth:`~re.finditer` method that returns an iterator over all the non-overlapping
1004 matches in a given string. (SRE is maintained by Fredrik Lundh. The
1005 BIGCHARSET patch was contributed by Martin von Löwis.)
1006
1007 * The :mod:`smtplib` module now supports :rfc:`2487`, "Secure SMTP over TLS", so
1008 it's now possible to encrypt the SMTP traffic between a Python program and the
1009 mail transport agent being handed a message. :mod:`smtplib` also supports SMTP
1010 authentication. (Contributed by Gerhard Häring.)
1011
1012 * The :mod:`imaplib` module, maintained by Piers Lauder, has support for several
1013 new extensions: the NAMESPACE extension defined in :rfc:`2342`, SORT, GETACL and
1014 SETACL. (Contributed by Anthony Baxter and Michel Pelletier.)
1015
1016 * The :mod:`!rfc822` module's parsing of email addresses is now compliant with
1017 :rfc:`2822`, an update to :rfc:`822`. (The module's name is *not* going to be
1018 changed to ``rfc2822``.) A new package, :mod:`email`, has also been added for
1019 parsing and generating e-mail messages. (Contributed by Barry Warsaw, and
1020 arising out of his work on Mailman.)
1021
1022 * The :mod:`difflib` module now contains a new :class:`!Differ` class for
1023 producing human-readable lists of changes (a "delta") between two sequences of
1024 lines of text. There are also two generator functions, :func:`!ndiff` and
1025 :func:`!restore`, which respectively return a delta from two sequences, or one of
1026 the original sequences from a delta. (Grunt work contributed by David Goodger,
1027 from ndiff.py code by Tim Peters who then did the generatorization.)
1028
1029 * New constants :const:`!ascii_letters`, :const:`!ascii_lowercase`, and
1030 :const:`!ascii_uppercase` were added to the :mod:`string` module. There were
1031 several modules in the standard library that used :const:`!string.letters` to
1032 mean the ranges A-Za-z, but that assumption is incorrect when locales are in
1033 use, because :const:`!string.letters` varies depending on the set of legal
1034 characters defined by the current locale. The buggy modules have all been fixed
1035 to use :const:`!ascii_letters` instead. (Reported by an unknown person; fixed by
1036 Fred L. Drake, Jr.)
1037
1038 * The :mod:`mimetypes` module now makes it easier to use alternative MIME-type
1039 databases by the addition of a :class:`~mimetypes.MimeTypes` class, which takes a list of
1040 filenames to be parsed. (Contributed by Fred L. Drake, Jr.)
1041
1042 * A :class:`~threading.Timer` class was added to the :mod:`threading` module that allows
1043 scheduling an activity to happen at some future time. (Contributed by Itamar
1044 Shtull-Trauring.)
1045
1046 .. ======================================================================
1047
1048
1049 Interpreter Changes and Fixes
1050 =============================
1051
1052 Some of the changes only affect people who deal with the Python interpreter at
1053 the C level because they're writing Python extension modules, embedding the
1054 interpreter, or just hacking on the interpreter itself. If you only write Python
1055 code, none of the changes described here will affect you very much.
1056
1057 * Profiling and tracing functions can now be implemented in C, which can operate
1058 at much higher speeds than Python-based functions and should reduce the overhead
1059 of profiling and tracing. This will be of interest to authors of development
1060 environments for Python. Two new C functions were added to Python's API,
1061 :c:func:`PyEval_SetProfile` and :c:func:`PyEval_SetTrace`. The existing
1062 :func:`sys.setprofile` and :func:`sys.settrace` functions still exist, and have
1063 simply been changed to use the new C-level interface. (Contributed by Fred L.
1064 Drake, Jr.)
1065
1066 * Another low-level API, primarily of interest to implementers of Python
1067 debuggers and development tools, was added. :c:func:`PyInterpreterState_Head` and
1068 :c:func:`PyInterpreterState_Next` let a caller walk through all the existing
1069 interpreter objects; :c:func:`PyInterpreterState_ThreadHead` and
1070 :c:func:`PyThreadState_Next` allow looping over all the thread states for a given
1071 interpreter. (Contributed by David Beazley.)
1072
1073 * The C-level interface to the garbage collector has been changed to make it
1074 easier to write extension types that support garbage collection and to debug
1075 misuses of the functions. Various functions have slightly different semantics,
1076 so a bunch of functions had to be renamed. Extensions that use the old API will
1077 still compile but will *not* participate in garbage collection, so updating them
1078 for 2.2 should be considered fairly high priority.
1079
1080 To upgrade an extension module to the new API, perform the following steps:
1081
1082 * Rename :c:macro:`!Py_TPFLAGS_GC` to :c:macro:`Py_TPFLAGS_HAVE_GC`.
1083
1084 * Use :c:func:`PyObject_GC_New` or :c:func:`PyObject_GC_NewVar` to allocate
1085 objects, and :c:func:`PyObject_GC_Del` to deallocate them.
1086
1087 * Rename :c:func:`!PyObject_GC_Init` to :c:func:`PyObject_GC_Track` and
1088 :c:func:`!PyObject_GC_Fini` to :c:func:`PyObject_GC_UnTrack`.
1089
1090 * Remove :c:macro:`!PyGC_HEAD_SIZE` from object size calculations.
1091
1092 * Remove calls to :c:func:`!PyObject_AS_GC` and :c:func:`!PyObject_FROM_GC`.
1093
1094 * A new ``et`` format sequence was added to :c:func:`PyArg_ParseTuple`; ``et``
1095 takes both a parameter and an encoding name, and converts the parameter to the
1096 given encoding if the parameter turns out to be a Unicode string, or leaves it
1097 alone if it's an 8-bit string, assuming it to already be in the desired
1098 encoding. This differs from the ``es`` format character, which assumes that
1099 8-bit strings are in Python's default ASCII encoding and converts them to the
1100 specified new encoding. (Contributed by M.-A. Lemburg, and used for the MBCS
1101 support on Windows described in the following section.)
1102
1103 * A different argument parsing function, :c:func:`PyArg_UnpackTuple`, has been
1104 added that's simpler and presumably faster. Instead of specifying a format
1105 string, the caller simply gives the minimum and maximum number of arguments
1106 expected, and a set of pointers to :c:expr:`PyObject*` variables that will be
1107 filled in with argument values.
1108
1109 * Two new flags :c:macro:`METH_NOARGS` and :c:macro:`METH_O` are available in method
1110 definition tables to simplify implementation of methods with no arguments or a
1111 single untyped argument. Calling such methods is more efficient than calling a
1112 corresponding method that uses :c:macro:`METH_VARARGS`. Also, the old
1113 :c:macro:`!METH_OLDARGS` style of writing C methods is now officially deprecated.
1114
1115 * Two new wrapper functions, :c:func:`PyOS_snprintf` and :c:func:`PyOS_vsnprintf`
1116 were added to provide cross-platform implementations for the relatively new
1117 :c:func:`snprintf` and :c:func:`vsnprintf` C lib APIs. In contrast to the standard
1118 :c:func:`sprintf` and :c:func:`!vsprintf` functions, the Python versions check the
1119 bounds of the buffer used to protect against buffer overruns. (Contributed by
1120 M.-A. Lemburg.)
1121
1122 * The :c:func:`_PyTuple_Resize` function has lost an unused parameter, so now it
1123 takes 2 parameters instead of 3. The third argument was never used, and can
1124 simply be discarded when porting code from earlier versions to Python 2.2.
1125
1126 .. ======================================================================
1127
1128
1129 Other Changes and Fixes
1130 =======================
1131
1132 As usual there were a bunch of other improvements and bugfixes scattered
1133 throughout the source tree. A search through the CVS change logs finds there
1134 were 527 patches applied and 683 bugs fixed between Python 2.1 and 2.2; 2.2.1
1135 applied 139 patches and fixed 143 bugs; 2.2.2 applied 106 patches and fixed 82
1136 bugs. These figures are likely to be underestimates.
1137
1138 Some of the more notable changes are:
1139
1140 * The code for the MacOS port for Python, maintained by Jack Jansen, is now kept
1141 in the main Python CVS tree, and many changes have been made to support MacOS X.
1142
1143 The most significant change is the ability to build Python as a framework,
1144 enabled by supplying the :option:`!--enable-framework` option to the configure
1145 script when compiling Python. According to Jack Jansen, "This installs a
1146 self-contained Python installation plus the OS X framework "glue" into
1147 :file:`/Library/Frameworks/Python.framework` (or another location of choice).
1148 For now there is little immediate added benefit to this (actually, there is the
1149 disadvantage that you have to change your PATH to be able to find Python), but
1150 it is the basis for creating a full-blown Python application, porting the
1151 MacPython IDE, possibly using Python as a standard OSA scripting language and
1152 much more."
1153
1154 Most of the MacPython toolbox modules, which interface to MacOS APIs such as
1155 windowing, QuickTime, scripting, etc. have been ported to OS X, but they've been
1156 left commented out in :file:`setup.py`. People who want to experiment with
1157 these modules can uncomment them manually.
1158
1159 .. Jack's original comments:
1160 The main change is the possibility to build Python as a
1161 framework. This installs a self-contained Python installation plus the
1162 OSX framework "glue" into /Library/Frameworks/Python.framework (or
1163 another location of choice). For now there is little immediate added
1164 benefit to this (actually, there is the disadvantage that you have to
1165 change your PATH to be able to find Python), but it is the basis for
1166 creating a fullblown Python application, porting the MacPython IDE,
1167 possibly using Python as a standard OSA scripting language and much
1168 more. You enable this with "configure --enable-framework".
1169 The other change is that most MacPython toolbox modules, which
1170 interface to all the MacOS APIs such as windowing, quicktime,
1171 scripting, etc. have been ported. Again, most of these are not of
1172 immediate use, as they need a full application to be really useful, so
1173 they have been commented out in setup.py. People wanting to experiment
1174 can uncomment them. Gestalt and Internet Config modules are enabled by
1175 default.
1176
1177 * Keyword arguments passed to built-in functions that don't take them now cause a
1178 :exc:`TypeError` exception to be raised, with the message "*function* takes no
1179 keyword arguments".
1180
1181 * Weak references, added in Python 2.1 as an extension module, are now part of
1182 the core because they're used in the implementation of new-style classes. The
1183 :exc:`ReferenceError` exception has therefore moved from the :mod:`weakref`
1184 module to become a built-in exception.
1185
1186 * A new script, :file:`Tools/scripts/cleanfuture.py` by Tim Peters,
1187 automatically removes obsolete ``__future__`` statements from Python source
1188 code.
1189
1190 * An additional *flags* argument has been added to the built-in function
1191 :func:`compile`, so the behaviour of ``__future__`` statements can now be
1192 correctly observed in simulated shells, such as those presented by IDLE and
1193 other development environments. This is described in :pep:`264`. (Contributed
1194 by Michael Hudson.)
1195
1196 * The new license introduced with Python 1.6 wasn't GPL-compatible. This is
1197 fixed by some minor textual changes to the 2.2 license, so it's now legal to
1198 embed Python inside a GPLed program again. Note that Python itself is not
1199 GPLed, but instead is under a license that's essentially equivalent to the BSD
1200 license, same as it always was. The license changes were also applied to the
1201 Python 2.0.1 and 2.1.1 releases.
1202
1203 * When presented with a Unicode filename on Windows, Python will now convert it
1204 to an MBCS encoded string, as used by the Microsoft file APIs. As MBCS is
1205 explicitly used by the file APIs, Python's choice of ASCII as the default
1206 encoding turns out to be an annoyance. On Unix, the locale's character set is
1207 used if ``locale.nl_langinfo(CODESET)`` is available. (Windows support was
1208 contributed by Mark Hammond with assistance from Marc-André Lemburg. Unix
1209 support was added by Martin von Löwis.)
1210
1211 * Large file support is now enabled on Windows. (Contributed by Tim Peters.)
1212
1213 * The :file:`Tools/scripts/ftpmirror.py` script now parses a :file:`.netrc`
1214 file, if you have one. (Contributed by Mike Romberg.)
1215
1216 * Some features of the object returned by the :func:`!xrange` function are now
1217 deprecated, and trigger warnings when they're accessed; they'll disappear in
1218 Python 2.3. :class:`!xrange` objects tried to pretend they were full sequence
1219 types by supporting slicing, sequence multiplication, and the :keyword:`in`
1220 operator, but these features were rarely used and therefore buggy. The
1221 :meth:`!tolist` method and the :attr:`!start`, :attr:`!stop`, and :attr:`!step`
1222 attributes are also being deprecated. At the C level, the fourth argument to
1223 the :c:func:`!PyRange_New` function, ``repeat``, has also been deprecated.
1224
1225 * There were a bunch of patches to the dictionary implementation, mostly to fix
1226 potential core dumps if a dictionary contains objects that sneakily changed
1227 their hash value, or mutated the dictionary they were contained in. For a while
1228 python-dev fell into a gentle rhythm of Michael Hudson finding a case that
1229 dumped core, Tim Peters fixing the bug, Michael finding another case, and round
1230 and round it went.
1231
1232 * On Windows, Python can now be compiled with Borland C thanks to a number of
1233 patches contributed by Stephen Hansen, though the result isn't fully functional
1234 yet. (But this *is* progress...)
1235
1236 * Another Windows enhancement: Wise Solutions generously offered PythonLabs use
1237 of their InstallerMaster 8.1 system. Earlier PythonLabs Windows installers used
1238 Wise 5.0a, which was beginning to show its age. (Packaged up by Tim Peters.)
1239
1240 * Files ending in ``.pyw`` can now be imported on Windows. ``.pyw`` is a
1241 Windows-only thing, used to indicate that a script needs to be run using
1242 PYTHONW.EXE instead of PYTHON.EXE in order to prevent a DOS console from popping
1243 up to display the output. This patch makes it possible to import such scripts,
1244 in case they're also usable as modules. (Implemented by David Bolen.)
1245
1246 * On platforms where Python uses the C :c:func:`dlopen` function to load
1247 extension modules, it's now possible to set the flags used by :c:func:`dlopen`
1248 using the :func:`sys.getdlopenflags` and :func:`sys.setdlopenflags` functions.
1249 (Contributed by Bram Stolk.)
1250
1251 * The :func:`pow` built-in function no longer supports 3 arguments when
1252 floating-point numbers are supplied. ``pow(x, y, z)`` returns ``(x**y) % z``,
1253 but this is never useful for floating-point numbers, and the final result varies
1254 unpredictably depending on the platform. A call such as ``pow(2.0, 8.0, 7.0)``
1255 will now raise a :exc:`TypeError` exception.
1256
1257 .. ======================================================================
1258
1259
1260 Acknowledgements
1261 ================
1262
1263 The author would like to thank the following people for offering suggestions,
1264 corrections and assistance with various drafts of this article: Fred Bremmer,
1265 Keith Briggs, Andrew Dalke, Fred L. Drake, Jr., Carel Fellinger, David Goodger,
1266 Mark Hammond, Stephen Hansen, Michael Hudson, Jack Jansen, Marc-André Lemburg,
1267 Martin von Löwis, Fredrik Lundh, Michael McLay, Nick Mathewson, Paul Moore,
1268 Gustavo Niemeyer, Don O'Donnell, Joonas Paalasma, Tim Peters, Jens Quade, Tom
1269 Reinhardt, Neil Schemenauer, Guido van Rossum, Greg Ward, Edward Welbourne.
1270