PageRenderTime 390ms CodeModel.GetById 141ms app.highlight 168ms RepoModel.GetById 33ms app.codeStats 1ms

ReStructuredText | 2078 lines | 1595 code | 483 blank | 0 comment | 0 complexity | 8db9b7b78675d404b6b3341c06d9e9af MD5 | raw file
   2  What's New in Python 2.3
   5:Author: A.M. Kuchling
   7.. |release| replace:: 1.01
   9.. $Id: whatsnew23.tex 54631 2007-03-31 11:58:36Z georg.brandl $
  11This article explains the new features in Python 2.3.  Python 2.3 was released
  12on July 29, 2003.
  14The main themes for Python 2.3 are polishing some of the features added in 2.2,
  15adding various small but useful enhancements to the core language, and expanding
  16the standard library.  The new object model introduced in the previous version
  17has benefited from 18 months of bugfixes and from optimization efforts that have
  18improved the performance of new-style classes.  A few new built-in functions
  19have been added such as :func:`sum` and :func:`enumerate`.  The :keyword:`in`
  20operator can now be used for substring searches (e.g. ``"ab" in "abc"`` returns
  23Some of the many new library features include Boolean, set, heap, and date/time
  24data types, the ability to import modules from ZIP-format archives, metadata
  25support for the long-awaited Python catalog, an updated version of IDLE, and
  26modules for logging messages, wrapping text, parsing CSV files, processing
  27command-line options, using BerkeleyDB databases...  the list of new and
  28enhanced modules is lengthy.
  30This article doesn't attempt to provide a complete specification of the new
  31features, but instead provides a convenient overview.  For full details, you
  32should refer to the documentation for Python 2.3, such as the Python Library
  33Reference and the Python Reference Manual.  If you want to understand the
  34complete implementation and design rationale, refer to the PEP for a particular
  35new feature.
  37.. ======================================================================
  40PEP 218: A Standard Set Datatype
  43The new :mod:`sets` module contains an implementation of a set datatype.  The
  44:class:`Set` class is for mutable sets, sets that can have members added and
  45removed.  The :class:`ImmutableSet` class is for sets that can't be modified,
  46and instances of :class:`ImmutableSet` can therefore be used as dictionary keys.
  47Sets are built on top of dictionaries, so the elements within a set must be
  50Here's a simple example::
  52   >>> import sets
  53   >>> S = sets.Set([1,2,3])
  54   >>> S
  55   Set([1, 2, 3])
  56   >>> 1 in S
  57   True
  58   >>> 0 in S
  59   False
  60   >>> S.add(5)
  61   >>> S.remove(3)
  62   >>> S
  63   Set([1, 2, 5])
  64   >>>
  66The union and intersection of sets can be computed with the :meth:`union` and
  67:meth:`intersection` methods; an alternative notation uses the bitwise operators
  68``&`` and ``|``. Mutable sets also have in-place versions of these methods,
  69:meth:`union_update` and :meth:`intersection_update`. ::
  71   >>> S1 = sets.Set([1,2,3])
  72   >>> S2 = sets.Set([4,5,6])
  73   >>> S1.union(S2)
  74   Set([1, 2, 3, 4, 5, 6])
  75   >>> S1 | S2                  # Alternative notation
  76   Set([1, 2, 3, 4, 5, 6])
  77   >>> S1.intersection(S2)
  78   Set([])
  79   >>> S1 & S2                  # Alternative notation
  80   Set([])
  81   >>> S1.union_update(S2)
  82   >>> S1
  83   Set([1, 2, 3, 4, 5, 6])
  84   >>>
  86It's also possible to take the symmetric difference of two sets.  This is the
  87set of all elements in the union that aren't in the intersection.  Another way
  88of putting it is that the symmetric difference contains all elements that are in
  89exactly one set.  Again, there's an alternative notation (``^``), and an in-
  90place version with the ungainly name :meth:`symmetric_difference_update`. ::
  92   >>> S1 = sets.Set([1,2,3,4])
  93   >>> S2 = sets.Set([3,4,5,6])
  94   >>> S1.symmetric_difference(S2)
  95   Set([1, 2, 5, 6])
  96   >>> S1 ^ S2
  97   Set([1, 2, 5, 6])
  98   >>>
 100There are also :meth:`issubset` and :meth:`issuperset` methods for checking
 101whether one set is a subset or superset of another::
 103   >>> S1 = sets.Set([1,2,3])
 104   >>> S2 = sets.Set([2,3])
 105   >>> S2.issubset(S1)
 106   True
 107   >>> S1.issubset(S2)
 108   False
 109   >>> S1.issuperset(S2)
 110   True
 111   >>>
 114.. seealso::
 116   :pep:`218` - Adding a Built-In Set Object Type
 117      PEP written by Greg V. Wilson. Implemented by Greg V. Wilson, Alex Martelli, and
 118      GvR.
 120.. ======================================================================
 123.. _section-generators:
 125PEP 255: Simple Generators
 128In Python 2.2, generators were added as an optional feature, to be enabled by a
 129``from __future__ import generators`` directive.  In 2.3 generators no longer
 130need to be specially enabled, and are now always present; this means that
 131:keyword:`yield` is now always a keyword.  The rest of this section is a copy of
 132the description of generators from the "What's New in Python 2.2" document; if
 133you read it back when Python 2.2 came out, you can skip the rest of this
 136You're doubtless familiar with how function calls work in Python or C. When you
 137call a function, it gets a private namespace where its local variables are
 138created.  When the function reaches a :keyword:`return` statement, the local
 139variables are destroyed and the resulting value is returned to the caller.  A
 140later call to the same function will get a fresh new set of local variables.
 141But, what if the local variables weren't thrown away on exiting a function?
 142What if you could later resume the function where it left off?  This is what
 143generators provide; they can be thought of as resumable functions.
 145Here's the simplest example of a generator function::
 147   def generate_ints(N):
 148       for i in range(N):
 149           yield i
 151A new keyword, :keyword:`yield`, was introduced for generators.  Any function
 152containing a :keyword:`yield` statement is a generator function; this is
 153detected by Python's bytecode compiler which compiles the function specially as
 154a result.
 156When you call a generator function, it doesn't return a single value; instead it
 157returns a generator object that supports the iterator protocol.  On executing
 158the :keyword:`yield` statement, the generator outputs the value of ``i``,
 159similar to a :keyword:`return` statement.  The big difference between
 160:keyword:`yield` and a :keyword:`return` statement is that on reaching a
 161:keyword:`yield` the generator's state of execution is suspended and local
 162variables are preserved.  On the next call to the generator's ``.next()``
 163method, the function will resume executing immediately after the
 164:keyword:`yield` statement.  (For complicated reasons, the :keyword:`yield`
 165statement isn't allowed inside the :keyword:`try` block of a :keyword:`try`...\
 166:keyword:`finally` statement; read :pep:`255` for a full explanation of the
 167interaction between :keyword:`yield` and exceptions.)
 169Here's a sample usage of the :func:`generate_ints` generator::
 171   >>> gen = generate_ints(3)
 172   >>> gen
 173   <generator object at 0x8117f90>
 174   >>>
 175   0
 176   >>>
 177   1
 178   >>>
 179   2
 180   >>>
 181   Traceback (most recent call last):
 182     File "stdin", line 1, in ?
 183     File "stdin", line 2, in generate_ints
 184   StopIteration
 186You could equally write ``for i in generate_ints(5)``, or ``a,b,c =
 189Inside a generator function, the :keyword:`return` statement can only be used
 190without a value, and signals the end of the procession of values; afterwards the
 191generator cannot return any further values. :keyword:`return` with a value, such
 192as ``return 5``, is a syntax error inside a generator function.  The end of the
 193generator's results can also be indicated by raising :exc:`StopIteration`
 194manually, or by just letting the flow of execution fall off the bottom of the
 197You could achieve the effect of generators manually by writing your own class
 198and storing all the local variables of the generator as instance variables.  For
 199example, returning a list of integers could be done by setting ``self.count`` to
 2000, and having the :meth:`next` method increment ``self.count`` and return it.
 201However, for a moderately complicated generator, writing a corresponding class
 202would be much messier. :file:`Lib/test/` contains a number of
 203more interesting examples.  The simplest one implements an in-order traversal of
 204a tree using generators recursively. ::
 206   # A recursive generator that generates Tree leaves in in-order.
 207   def inorder(t):
 208       if t:
 209           for x in inorder(t.left):
 210               yield x
 211           yield t.label
 212           for x in inorder(t.right):
 213               yield x
 215Two other examples in :file:`Lib/test/` produce solutions for
 216the N-Queens problem (placing $N$ queens on an $NxN$ chess board so that no
 217queen threatens another) and the Knight's Tour (a route that takes a knight to
 218every square of an $NxN$ chessboard without visiting any square twice).
 220The idea of generators comes from other programming languages, especially Icon
 221(, where the idea of generators is central.  In
 222Icon, every expression and function call behaves like a generator.  One example
 223from "An Overview of the Icon Programming Language" at
 224 gives an idea of what this looks
 227   sentence := "Store it in the neighboring harbor"
 228   if (i := find("or", sentence)) > 5 then write(i)
 230In Icon the :func:`find` function returns the indexes at which the substring
 231"or" is found: 3, 23, 33.  In the :keyword:`if` statement, ``i`` is first
 232assigned a value of 3, but 3 is less than 5, so the comparison fails, and Icon
 233retries it with the second value of 23.  23 is greater than 5, so the comparison
 234now succeeds, and the code prints the value 23 to the screen.
 236Python doesn't go nearly as far as Icon in adopting generators as a central
 237concept.  Generators are considered part of the core Python language, but
 238learning or using them isn't compulsory; if they don't solve any problems that
 239you have, feel free to ignore them. One novel feature of Python's interface as
 240compared to Icon's is that a generator's state is represented as a concrete
 241object (the iterator) that can be passed around to other functions or stored in
 242a data structure.
 245.. seealso::
 247   :pep:`255` - Simple Generators
 248      Written by Neil Schemenauer, Tim Peters, Magnus Lie Hetland.  Implemented mostly
 249      by Neil Schemenauer and Tim Peters, with other fixes from the Python Labs crew.
 251.. ======================================================================
 254.. _section-encodings:
 256PEP 263: Source Code Encodings
 259Python source files can now be declared as being in different character set
 260encodings.  Encodings are declared by including a specially formatted comment in
 261the first or second line of the source file.  For example, a UTF-8 file can be
 262declared with::
 264   #!/usr/bin/env python
 265   # -*- coding: UTF-8 -*-
 267Without such an encoding declaration, the default encoding used is 7-bit ASCII.
 268Executing or importing modules that contain string literals with 8-bit
 269characters and have no encoding declaration will result in a
 270:exc:`DeprecationWarning` being signalled by Python 2.3; in 2.4 this will be a
 271syntax error.
 273The encoding declaration only affects Unicode string literals, which will be
 274converted to Unicode using the specified encoding.  Note that Python identifiers
 275are still restricted to ASCII characters, so you can't have variable names that
 276use characters outside of the usual alphanumerics.
 279.. seealso::
 281   :pep:`263` - Defining Python Source Code Encodings
 282      Written by Marc-André Lemburg and Martin von Löwis; implemented by Suzuki Hisao
 283      and Martin von Löwis.
 285.. ======================================================================
 288PEP 273: Importing Modules from ZIP Archives
 291The new :mod:`zipimport` module adds support for importing modules from a ZIP-
 292format archive.  You don't need to import the module explicitly; it will be
 293automatically imported if a ZIP archive's filename is added to ``sys.path``.
 294For example::
 296   amk@nyman:~/src/python$ unzip -l /tmp/
 297   Archive:  /tmp/
 298     Length     Date   Time    Name
 299    --------    ----   ----    ----
 300        8467  11-26-02 22:30
 301    --------                   -------
 302        8467                   1 file
 303   amk@nyman:~/src/python$ ./python
 304   Python 2.3 (#1, Aug 1 2003, 19:54:32)
 305   >>> import sys
 306   >>> sys.path.insert(0, '/tmp/')  # Add .zip file to front of path
 307   >>> import jwzthreading
 308   >>> jwzthreading.__file__
 309   '/tmp/'
 310   >>>
 312An entry in ``sys.path`` can now be the filename of a ZIP archive. The ZIP
 313archive can contain any kind of files, but only files named :file:`\*.py`,
 314:file:`\*.pyc`, or :file:`\*.pyo` can be imported.  If an archive only contains
 315:file:`\*.py` files, Python will not attempt to modify the archive by adding the
 316corresponding :file:`\*.pyc` file, meaning that if a ZIP archive doesn't contain
 317:file:`\*.pyc` files, importing may be rather slow.
 319A path within the archive can also be specified to only import from a
 320subdirectory; for example, the path :file:`/tmp/` would only
 321import from the :file:`lib/` subdirectory within the archive.
 324.. seealso::
 326   :pep:`273` - Import Modules from Zip Archives
 327      Written by James C. Ahlstrom,  who also provided an implementation. Python 2.3
 328      follows the specification in :pep:`273`,  but uses an implementation written by
 329      Just van Rossum  that uses the import hooks described in :pep:`302`. See section
 330      :ref:`section-pep302` for a description of the new import hooks.
 332.. ======================================================================
 335PEP 277: Unicode file name support for Windows NT
 338On Windows NT, 2000, and XP, the system stores file names as Unicode strings.
 339Traditionally, Python has represented file names as byte strings, which is
 340inadequate because it renders some file names inaccessible.
 342Python now allows using arbitrary Unicode strings (within the limitations of the
 343file system) for all functions that expect file names, most notably the
 344:func:`open` built-in function. If a Unicode string is passed to
 345:func:`os.listdir`, Python now returns a list of Unicode strings.  A new
 346function, :func:`os.getcwdu`, returns the current directory as a Unicode string.
 348Byte strings still work as file names, and on Windows Python will transparently
 349convert them to Unicode using the ``mbcs`` encoding.
 351Other systems also allow Unicode strings as file names but convert them to byte
 352strings before passing them to the system, which can cause a :exc:`UnicodeError`
 353to be raised. Applications can test whether arbitrary Unicode strings are
 354supported as file names by checking :attr:`os.path.supports_unicode_filenames`,
 355a Boolean value.
 357Under MacOS, :func:`os.listdir` may now return Unicode filenames.
 360.. seealso::
 362   :pep:`277` - Unicode file name support for Windows NT
 363      Written by Neil Hodgson; implemented by Neil Hodgson, Martin von Löwis, and Mark
 364      Hammond.
 366.. ======================================================================
 369PEP 278: Universal Newline Support
 372The three major operating systems used today are Microsoft Windows, Apple's
 373Macintosh OS, and the various Unix derivatives.  A minor irritation of cross-
 374platform work  is that these three platforms all use different characters to
 375mark the ends of lines in text files.  Unix uses the linefeed (ASCII character
 37610), MacOS uses the carriage return (ASCII character 13), and Windows uses a
 377two-character sequence of a carriage return plus a newline.
 379Python's file objects can now support end of line conventions other than the one
 380followed by the platform on which Python is running. Opening a file with the
 381mode ``'U'`` or ``'rU'`` will open a file for reading in universal newline mode.
 382All three line ending conventions will be translated to a ``'\n'`` in the
 383strings returned by the various file methods such as :meth:`read` and
 386Universal newline support is also used when importing modules and when executing
 387a file with the :func:`execfile` function.  This means that Python modules can
 388be shared between all three operating systems without needing to convert the
 391This feature can be disabled when compiling Python by specifying the
 392:option:`--without-universal-newlines` switch when running Python's
 393:program:`configure` script.
 396.. seealso::
 398   :pep:`278` - Universal Newline Support
 399      Written and implemented by Jack Jansen.
 401.. ======================================================================
 404.. _section-enumerate:
 406PEP 279: enumerate()
 409A new built-in function, :func:`enumerate`, will make certain loops a bit
 410clearer.  ``enumerate(thing)``, where *thing* is either an iterator or a
 411sequence, returns a iterator that will return ``(0, thing[0])``, ``(1,
 412thing[1])``, ``(2, thing[2])``, and so forth.
 414A common idiom to change every element of a list looks like this::
 416   for i in range(len(L)):
 417       item = L[i]
 418       # ... compute some result based on item ...
 419       L[i] = result
 421This can be rewritten using :func:`enumerate` as::
 423   for i, item in enumerate(L):
 424       # ... compute some result based on item ...
 425       L[i] = result
 428.. seealso::
 430   :pep:`279` - The enumerate() built-in function
 431      Written and implemented by Raymond D. Hettinger.
 433.. ======================================================================
 436PEP 282: The logging Package
 439A standard package for writing logs, :mod:`logging`, has been added to Python
 4402.3.  It provides a powerful and flexible mechanism for generating logging
 441output which can then be filtered and processed in various ways.  A
 442configuration file written in a standard format can be used to control the
 443logging behavior of a program.  Python includes handlers that will write log
 444records to standard error or to a file or socket, send them to the system log,
 445or even e-mail them to a particular address; of course, it's also possible to
 446write your own handler classes.
 448The :class:`Logger` class is the primary class. Most application code will deal
 449with one or more :class:`Logger` objects, each one used by a particular
 450subsystem of the application. Each :class:`Logger` is identified by a name, and
 451names are organized into a hierarchy using ``.``  as the component separator.
 452For example, you might have :class:`Logger` instances named ``server``,
 453``server.auth`` and ````.  The latter two instances are below
 454``server`` in the hierarchy.  This means that if you turn up the verbosity for
 455``server`` or direct ``server`` messages to a different handler, the changes
 456will also apply to records logged to ``server.auth`` and ````.
 457There's also a root :class:`Logger` that's the parent of all other loggers.
 459For simple uses, the :mod:`logging` package contains some convenience functions
 460that always use the root log::
 462   import logging
 464   logging.debug('Debugging information')
 465'Informational message')
 466   logging.warning('Warning:config file %s not found', 'server.conf')
 467   logging.error('Error occurred')
 468   logging.critical('Critical error -- shutting down')
 470This produces the following output::
 472   WARNING:root:Warning:config file server.conf not found
 473   ERROR:root:Error occurred
 474   CRITICAL:root:Critical error -- shutting down
 476In the default configuration, informational and debugging messages are
 477suppressed and the output is sent to standard error.  You can enable the display
 478of informational and debugging messages by calling the :meth:`setLevel` method
 479on the root logger.
 481Notice the :func:`warning` call's use of string formatting operators; all of the
 482functions for logging messages take the arguments ``(msg, arg1, arg2, ...)`` and
 483log the string resulting from ``msg % (arg1, arg2, ...)``.
 485There's also an :func:`exception` function that records the most recent
 486traceback.  Any of the other functions will also record the traceback if you
 487specify a true value for the keyword argument *exc_info*. ::
 489   def f():
 490       try:    1/0
 491       except: logging.exception('Problem recorded')
 493   f()
 495This produces the following output::
 497   ERROR:root:Problem recorded
 498   Traceback (most recent call last):
 499     File "", line 6, in f
 500       1/0
 501   ZeroDivisionError: integer division or modulo by zero
 503Slightly more advanced programs will use a logger other than the root logger.
 504The :func:`getLogger(name)` function is used to get a particular log, creating
 505it if it doesn't exist yet. :func:`getLogger(None)` returns the root logger. ::
 507   log = logging.getLogger('server')
 508    ...
 509'Listening on port %i', port)
 510    ...
 511   log.critical('Disk full')
 512    ...
 514Log records are usually propagated up the hierarchy, so a message logged to
 515``server.auth`` is also seen by ``server`` and ``root``, but a :class:`Logger`
 516can prevent this by setting its :attr:`propagate` attribute to :const:`False`.
 518There are more classes provided by the :mod:`logging` package that can be
 519customized.  When a :class:`Logger` instance is told to log a message, it
 520creates a :class:`LogRecord` instance that is sent to any number of different
 521:class:`Handler` instances.  Loggers and handlers can also have an attached list
 522of filters, and each filter can cause the :class:`LogRecord` to be ignored or
 523can modify the record before passing it along.  When they're finally output,
 524:class:`LogRecord` instances are converted to text by a :class:`Formatter`
 525class.  All of these classes can be replaced by your own specially-written
 528With all of these features the :mod:`logging` package should provide enough
 529flexibility for even the most complicated applications.  This is only an
 530incomplete overview of its features, so please see the package's reference
 531documentation for all of the details.  Reading :pep:`282` will also be helpful.
 534.. seealso::
 536   :pep:`282` - A Logging System
 537      Written by Vinay Sajip and Trent Mick; implemented by Vinay Sajip.
 539.. ======================================================================
 542.. _section-bool:
 544PEP 285: A Boolean Type
 547A Boolean type was added to Python 2.3.  Two new constants were added to the
 548:mod:`__builtin__` module, :const:`True` and :const:`False`.  (:const:`True` and
 549:const:`False` constants were added to the built-ins in Python 2.2.1, but the
 5502.2.1 versions are simply set to integer values of 1 and 0 and aren't a
 551different type.)
 553The type object for this new type is named :class:`bool`; the constructor for it
 554takes any Python value and converts it to :const:`True` or :const:`False`. ::
 556   >>> bool(1)
 557   True
 558   >>> bool(0)
 559   False
 560   >>> bool([])
 561   False
 562   >>> bool( (1,) )
 563   True
 565Most of the standard library modules and built-in functions have been changed to
 566return Booleans. ::
 568   >>> obj = []
 569   >>> hasattr(obj, 'append')
 570   True
 571   >>> isinstance(obj, list)
 572   True
 573   >>> isinstance(obj, tuple)
 574   False
 576Python's Booleans were added with the primary goal of making code clearer.  For
 577example, if you're reading a function and encounter the statement ``return 1``,
 578you might wonder whether the ``1`` represents a Boolean truth value, an index,
 579or a coefficient that multiplies some other quantity.  If the statement is
 580``return True``, however, the meaning of the return value is quite clear.
 582Python's Booleans were *not* added for the sake of strict type-checking.  A very
 583strict language such as Pascal would also prevent you performing arithmetic with
 584Booleans, and would require that the expression in an :keyword:`if` statement
 585always evaluate to a Boolean result.  Python is not this strict and never will
 586be, as :pep:`285` explicitly says.  This means you can still use any expression
 587in an :keyword:`if` statement, even ones that evaluate to a list or tuple or
 588some random object.  The Boolean type is a subclass of the :class:`int` class so
 589that arithmetic using a Boolean still works. ::
 591   >>> True + 1
 592   2
 593   >>> False + 1
 594   1
 595   >>> False * 75
 596   0
 597   >>> True * 75
 598   75
 600To sum up :const:`True` and :const:`False` in a sentence: they're alternative
 601ways to spell the integer values 1 and 0, with the single difference that
 602:func:`str` and :func:`repr` return the strings ``'True'`` and ``'False'``
 603instead of ``'1'`` and ``'0'``.
 606.. seealso::
 608   :pep:`285` - Adding a bool type
 609      Written and implemented by GvR.
 611.. ======================================================================
 614PEP 293: Codec Error Handling Callbacks
 617When encoding a Unicode string into a byte string, unencodable characters may be
 618encountered.  So far, Python has allowed specifying the error processing as
 619either "strict" (raising :exc:`UnicodeError`), "ignore" (skipping the
 620character), or "replace" (using a question mark in the output string), with
 621"strict" being the default behavior. It may be desirable to specify alternative
 622processing of such errors, such as inserting an XML character reference or HTML
 623entity reference into the converted string.
 625Python now has a flexible framework to add different processing strategies.  New
 626error handlers can be added with :func:`codecs.register_error`, and codecs then
 627can access the error handler with :func:`codecs.lookup_error`. An equivalent C
 628API has been added for codecs written in C. The error handler gets the necessary
 629state information such as the string being converted, the position in the string
 630where the error was detected, and the target encoding.  The handler can then
 631either raise an exception or return a replacement string.
 633Two additional error handlers have been implemented using this framework:
 634"backslashreplace" uses Python backslash quoting to represent unencodable
 635characters and "xmlcharrefreplace" emits XML character references.
 638.. seealso::
 640   :pep:`293` - Codec Error Handling Callbacks
 641      Written and implemented by Walter Dörwald.
 643.. ======================================================================
 646.. _section-pep301:
 648PEP 301: Package Index and Metadata for Distutils
 651Support for the long-requested Python catalog makes its first appearance in 2.3.
 653The heart of the catalog is the new Distutils :command:`register` command.
 654Running ``python register`` will collect the metadata describing a
 655package, such as its name, version, maintainer, description, &c., and send it to
 656a central catalog server.  The resulting catalog is available from
 659To make the catalog a bit more useful, a new optional *classifiers* keyword
 660argument has been added to the Distutils :func:`setup` function.  A list of
 661`Trove <>`_-style strings can be supplied to help
 662classify the software.
 664Here's an example :file:`` with classifiers, written to be compatible
 665with older versions of the Distutils::
 667   from distutils import core
 668   kw = {'name': "Quixote",
 669         'version': "0.5.1",
 670         'description': "A highly Pythonic Web application framework",
 671         # ...
 672         }
 674   if (hasattr(core, 'setup_keywords') and
 675       'classifiers' in core.setup_keywords):
 676       kw['classifiers'] = \
 677           ['Topic :: Internet :: WWW/HTTP :: Dynamic Content',
 678            'Environment :: No Input/Output (Daemon)',
 679            'Intended Audience :: Developers'],
 681   core.setup(**kw)
 683The full list of classifiers can be obtained by running  ``python
 684register --list-classifiers``.
 687.. seealso::
 689   :pep:`301` - Package Index and Metadata for Distutils
 690      Written and implemented by Richard Jones.
 692.. ======================================================================
 695.. _section-pep302:
 697PEP 302: New Import Hooks
 700While it's been possible to write custom import hooks ever since the
 701:mod:`ihooks` module was introduced in Python 1.3, no one has ever been really
 702happy with it because writing new import hooks is difficult and messy.  There
 703have been various proposed alternatives such as the :mod:`imputil` and :mod:`iu`
 704modules, but none of them has ever gained much acceptance, and none of them were
 705easily usable from C code.
 707:pep:`302` borrows ideas from its predecessors, especially from Gordon
 708McMillan's :mod:`iu` module.  Three new items  are added to the :mod:`sys`
 711* ``sys.path_hooks`` is a list of callable objects; most  often they'll be
 712  classes.  Each callable takes a string containing a path and either returns an
 713  importer object that will handle imports from this path or raises an
 714  :exc:`ImportError` exception if it can't handle this path.
 716* ``sys.path_importer_cache`` caches importer objects for each path, so
 717  ``sys.path_hooks`` will only need to be traversed once for each path.
 719* ``sys.meta_path`` is a list of importer objects that will be traversed before
 720  ``sys.path`` is checked.  This list is initially empty, but user code can add
 721  objects to it.  Additional built-in and frozen modules can be imported by an
 722  object added to this list.
 724Importer objects must have a single method, :meth:`find_module(fullname,
 725path=None)`.  *fullname* will be a module or package name, e.g. ``string`` or
 726``distutils.core``.  :meth:`find_module` must return a loader object that has a
 727single method, :meth:`load_module(fullname)`, that creates and returns the
 728corresponding module object.
 730Pseudo-code for Python's new import logic, therefore, looks something like this
 731(simplified a bit; see :pep:`302` for the full details)::
 733   for mp in sys.meta_path:
 734       loader = mp(fullname)
 735       if loader is not None:
 736           <module> = loader.load_module(fullname)
 738   for path in sys.path:
 739       for hook in sys.path_hooks:
 740           try:
 741               importer = hook(path)
 742           except ImportError:
 743               # ImportError, so try the other path hooks
 744               pass
 745           else:
 746               loader = importer.find_module(fullname)
 747               <module> = loader.load_module(fullname)
 749   # Not found!
 750   raise ImportError
 753.. seealso::
 755   :pep:`302` - New Import Hooks
 756      Written by Just van Rossum and Paul Moore. Implemented by Just van Rossum.
 758.. ======================================================================
 761.. _section-pep305:
 763PEP 305: Comma-separated Files
 766Comma-separated files are a format frequently used for exporting data from
 767databases and spreadsheets.  Python 2.3 adds a parser for comma-separated files.
 769Comma-separated format is deceptively simple at first glance::
 771   Costs,150,200,3.95
 773Read a line and call ``line.split(',')``: what could be simpler? But toss in
 774string data that can contain commas, and things get more complicated::
 776   "Costs",150,200,3.95,"Includes taxes, shipping, and sundry items"
 778A big ugly regular expression can parse this, but using the new  :mod:`csv`
 779package is much simpler::
 781   import csv
 783   input = open('datafile', 'rb')
 784   reader = csv.reader(input)
 785   for line in reader:
 786       print line
 788The :func:`reader` function takes a number of different options. The field
 789separator isn't limited to the comma and can be changed to any character, and so
 790can the quoting and line-ending characters.
 792Different dialects of comma-separated files can be defined and registered;
 793currently there are two dialects, both used by Microsoft Excel. A separate
 794:class:`csv.writer` class will generate comma-separated files from a succession
 795of tuples or lists, quoting strings that contain the delimiter.
 798.. seealso::
 800   :pep:`305` - CSV File API
 801      Written and implemented  by Kevin Altis, Dave Cole, Andrew McNamara, Skip
 802      Montanaro, Cliff Wells.
 804.. ======================================================================
 807.. _section-pep307:
 809PEP 307: Pickle Enhancements
 812The :mod:`pickle` and :mod:`cPickle` modules received some attention during the
 8132.3 development cycle.  In 2.2, new-style classes could be pickled without
 814difficulty, but they weren't pickled very compactly; :pep:`307` quotes a trivial
 815example where a new-style class results in a pickled string three times longer
 816than that for a classic class.
 818The solution was to invent a new pickle protocol.  The :func:`pickle.dumps`
 819function has supported a text-or-binary flag  for a long time.  In 2.3, this
 820flag is redefined from a Boolean to an integer: 0 is the old text-mode pickle
 821format, 1 is the old binary format, and now 2 is a new 2.3-specific format.  A
 822new constant, :const:`pickle.HIGHEST_PROTOCOL`, can be used to select the
 823fanciest protocol available.
 825Unpickling is no longer considered a safe operation.  2.2's :mod:`pickle`
 826provided hooks for trying to prevent unsafe classes from being unpickled
 827(specifically, a :attr:`__safe_for_unpickling__` attribute), but none of this
 828code was ever audited and therefore it's all been ripped out in 2.3.  You should
 829not unpickle untrusted data in any version of Python.
 831To reduce the pickling overhead for new-style classes, a new interface for
 832customizing pickling was added using three special methods:
 833:meth:`__getstate__`, :meth:`__setstate__`, and :meth:`__getnewargs__`.  Consult
 834:pep:`307` for the full semantics  of these methods.
 836As a way to compress pickles yet further, it's now possible to use integer codes
 837instead of long strings to identify pickled classes. The Python Software
 838Foundation will maintain a list of standardized codes; there's also a range of
 839codes for private use.  Currently no codes have been specified.
 842.. seealso::
 844   :pep:`307` - Extensions to the pickle protocol
 845      Written and implemented  by Guido van Rossum and Tim Peters.
 847.. ======================================================================
 850.. _section-slices:
 852Extended Slices
 855Ever since Python 1.4, the slicing syntax has supported an optional third "step"
 856or "stride" argument.  For example, these are all legal Python syntax:
 857``L[1:10:2]``, ``L[:-1:1]``, ``L[::-1]``.  This was added to Python at the
 858request of the developers of Numerical Python, which uses the third argument
 859extensively.  However, Python's built-in list, tuple, and string sequence types
 860have never supported this feature, raising a :exc:`TypeError` if you tried it.
 861Michael Hudson contributed a patch to fix this shortcoming.
 863For example, you can now easily extract the elements of a list that have even
 866   >>> L = range(10)
 867   >>> L[::2]
 868   [0, 2, 4, 6, 8]
 870Negative values also work to make a copy of the same list in reverse order::
 872   >>> L[::-1]
 873   [9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
 875This also works for tuples, arrays, and strings::
 877   >>> s='abcd'
 878   >>> s[::2]
 879   'ac'
 880   >>> s[::-1]
 881   'dcba'
 883If you have a mutable sequence such as a list or an array you can assign to or
 884delete an extended slice, but there are some differences between assignment to
 885extended and regular slices.  Assignment to a regular slice can be used to
 886change the length of the sequence::
 888   >>> a = range(3)
 889   >>> a
 890   [0, 1, 2]
 891   >>> a[1:3] = [4, 5, 6]
 892   >>> a
 893   [0, 4, 5, 6]
 895Extended slices aren't this flexible.  When assigning to an extended slice, the
 896list on the right hand side of the statement must contain the same number of
 897items as the slice it is replacing::
 899   >>> a = range(4)
 900   >>> a
 901   [0, 1, 2, 3]
 902   >>> a[::2]
 903   [0, 2]
 904   >>> a[::2] = [0, -1]
 905   >>> a
 906   [0, 1, -1, 3]
 907   >>> a[::2] = [0,1,2]
 908   Traceback (most recent call last):
 909     File "<stdin>", line 1, in ?
 910   ValueError: attempt to assign sequence of size 3 to extended slice of size 2
 912Deletion is more straightforward::
 914   >>> a = range(4)
 915   >>> a
 916   [0, 1, 2, 3]
 917   >>> a[::2]
 918   [0, 2]
 919   >>> del a[::2]
 920   >>> a
 921   [1, 3]
 923One can also now pass slice objects to the :meth:`__getitem__` methods of the
 924built-in sequences::
 926   >>> range(10).__getitem__(slice(0, 5, 2))
 927   [0, 2, 4]
 929Or use slice objects directly in subscripts::
 931   >>> range(10)[slice(0, 5, 2)]
 932   [0, 2, 4]
 934To simplify implementing sequences that support extended slicing, slice objects
 935now have a method :meth:`indices(length)` which, given the length of a sequence,
 936returns a ``(start, stop, step)`` tuple that can be passed directly to
 937:func:`range`. :meth:`indices` handles omitted and out-of-bounds indices in a
 938manner consistent with regular slices (and this innocuous phrase hides a welter
 939of confusing details!).  The method is intended to be used like this::
 941   class FakeSeq:
 942       ...
 943       def calc_item(self, i):
 944           ...
 945       def __getitem__(self, item):
 946           if isinstance(item, slice):
 947               indices = item.indices(len(self))
 948               return FakeSeq([self.calc_item(i) for i in range(*indices)])
 949           else:
 950               return self.calc_item(i)
 952From this example you can also see that the built-in :class:`slice` object is
 953now the type object for the slice type, and is no longer a function.  This is
 954consistent with Python 2.2, where :class:`int`, :class:`str`, etc., underwent
 955the same change.
 957.. ======================================================================
 960Other Language Changes
 963Here are all of the changes that Python 2.3 makes to the core Python language.
 965* The :keyword:`yield` statement is now always a keyword, as described in
 966  section :ref:`section-generators` of this document.
 968* A new built-in function :func:`enumerate` was added, as described in section
 969  :ref:`section-enumerate` of this document.
 971* Two new constants, :const:`True` and :const:`False` were added along with the
 972  built-in :class:`bool` type, as described in section :ref:`section-bool` of this
 973  document.
 975* The :func:`int` type constructor will now return a long integer instead of
 976  raising an :exc:`OverflowError` when a string or floating-point number is too
 977  large to fit into an integer.  This can lead to the paradoxical result that
 978  ``isinstance(int(expression), int)`` is false, but that seems unlikely to cause
 979  problems in practice.
 981* Built-in types now support the extended slicing syntax, as described in
 982  section :ref:`section-slices` of this document.
 984* A new built-in function, :func:`sum(iterable, start=0)`,  adds up the numeric
 985  items in the iterable object and returns their sum.  :func:`sum` only accepts
 986  numbers, meaning that you can't use it to concatenate a bunch of strings.
 987  (Contributed by Alex Martelli.)
 989* ``list.insert(pos, value)`` used to  insert *value* at the front of the list
 990  when *pos* was negative.  The behaviour has now been changed to be consistent
 991  with slice indexing, so when *pos* is -1 the value will be inserted before the
 992  last element, and so forth.
 994* ``list.index(value)``, which searches for *value*  within the list and returns
 995  its index, now takes optional  *start* and *stop* arguments to limit the search
 996  to  only part of the list.
 998* Dictionaries have a new method, :meth:`pop(key[, *default*])`, that returns
 999  the value corresponding to *key* and removes that key/value pair from the
1000  dictionary.  If the requested key isn't present in the dictionary, *default* is
1001  returned if it's specified and :exc:`KeyError` raised if it isn't. ::
1003     >>> d = {1:2}
1004     >>> d
1005     {1: 2}
1006     >>> d.pop(4)
1007     Traceback (most recent call last):
1008       File "stdin", line 1, in ?
1009     KeyError: 4
1010     >>> d.pop(1)
1011     2
1012     >>> d.pop(1)
1013     Traceback (most recent call last):
1014       File "stdin", line 1, in ?
1015     KeyError: 'pop(): dictionary is empty'
1016     >>> d
1017     {}
1018     >>>
1020  There's also a new class method,  :meth:`dict.fromkeys(iterable, value)`, that
1021  creates a dictionary with keys taken from the supplied iterator *iterable* and
1022  all values set to *value*, defaulting to ``None``.
1024  (Patches contributed by Raymond Hettinger.)
1026  Also, the :func:`dict` constructor now accepts keyword arguments to simplify
1027  creating small dictionaries::
1029     >>> dict(red=1, blue=2, green=3, black=4)
1030     {'blue': 2, 'black': 4, 'green': 3, 'red': 1}
1032  (Contributed by Just van Rossum.)
1034* The :keyword:`assert` statement no longer checks the ``__debug__`` flag, so
1035  you can no longer disable assertions by assigning to ``__debug__``. Running
1036  Python with the :option:`-O` switch will still generate code that doesn't
1037  execute any assertions.
1039* Most type objects are now callable, so you can use them to create new objects
1040  such as functions, classes, and modules.  (This means that the :mod:`new` module
1041  can be deprecated in a future Python version, because you can now use the type
1042  objects available in the :mod:`types` module.) For example, you can create a new
1043  module object with the following code:
1045  ::
1047     >>> import types
1048     >>> m = types.ModuleType('abc','docstring')
1049     >>> m
1050     <module 'abc' (built-in)>
1051     >>> m.__doc__
1052     'docstring'
1054* A new warning, :exc:`PendingDeprecationWarning` was added to indicate features
1055  which are in the process of being deprecated.  The warning will *not* be printed
1056  by default.  To check for use of features that will be deprecated in the future,
1057  supply :option:`-Walways::PendingDeprecationWarning::` on the command line or
1058  use :func:`warnings.filterwarnings`.
1060* The process of deprecating string-based exceptions, as in ``raise "Error
1061  occurred"``, has begun.  Raising a string will now trigger
1062  :exc:`PendingDeprecationWarning`.
1064* Using ``None`` as a variable name will now result in a :exc:`SyntaxWarning`
1065  warning.  In a future version of Python, ``None`` may finally become a keyword.
1067* The :meth:`xreadlines` method of file objects, introduced in Python 2.1, is no
1068  longer necessary because files now behave as their own iterator.
1069  :meth:`xreadlines` was originally introduced as a faster way to loop over all
1070  the lines in a file, but now you can simply write ``for line in file_obj``.
1071  File objects also have a new read-only :attr:`encoding` attribute that gives the
1072  encoding used by the file; Unicode strings written to the file will be
1073  automatically  converted to bytes using the given encoding.
1075* The method resolution order used by new-style classes has changed, though
1076  you'll only notice the difference if you have a really complicated inheritance
1077  hierarchy.  Classic classes are unaffected by this change.  Python 2.2
1078  originally used a topological sort of a class's ancestors, but 2.3 now uses the
1079  C3 algorithm as described in the paper `"A Monotonic Superclass Linearization
1080  for Dylan" <>`_. To
1081  understand the motivation for this change,  read Michele Simionato's article
1082  `"Python 2.3 Method Resolution Order" <>`_, or
1083  read the thread on python-dev starting with the message at
1084 Samuele
1085  Pedroni first pointed out the problem and also implemented the fix by coding the
1086  C3 algorithm.
1088* Python runs multithreaded programs by switching between threads after
1089  executing N bytecodes.  The default value for N has been increased from 10 to
1090  100 bytecodes, speeding up single-threaded applications by reducing the
1091  switching overhead.  Some multithreaded applications may suffer slower response
1092  time, but that's easily fixed by setting the limit back to a lower number using
1093  :func:`sys.setcheckinterval(N)`. The limit can be retrieved with the new
1094  :func:`sys.getcheckinterval` function.
1096* One minor but far-reaching change is that the names of extension types defined
1097  by the modules included with Python now contain the module and a ``'.'`` in
1098  front of the type name.  For example, in Python 2.2, if you created a socket and
1099  printed its :attr:`__class__`, you'd get this output::
1101     >>> s = socket.socket()
1102     >>> s.__class__
1103     <type 'socket'>
1105  In 2.3, you get this::
1107     >>> s.__class__
1108     <type '_socket.socket'>
1110* One of the noted incompatibilities between old- and new-style classes has been
1111  removed: you can now assign to the :attr:`__name__` and :attr:`__bases__`
1112  attributes of new-style classes.  There are some restrictions on what can be
1113  assigned to :attr:`__bases__` along the lines of those relating to assigning to
1114  an instance's :attr:`__class__` attribute.
1116.. ======================================================================
1119String Changes
1122* The :keyword:`in` operator now works differently for strings. Previously, when
1123  evaluating ``X in Y`` where *X* and *Y* are strings, *X* could only be a single
1124  character. That's now changed; *X* can be a string of any length, and ``X in Y``
1125  will return :const:`True` if *X* is a substring of *Y*.  If *X* is the empty
1126  string, the result is always :const:`True`. ::
1128     >>> 'ab' in 'abcd'
1129     True
1130     >>> 'ad' in 'abcd'
1131     False
1132     >>> '' in 'abcd'
1133     True
1135  Note that this doesn't tell you where the substring starts; if you need that
1136  information, use the :meth:`find` string method.
1138* The :meth:`strip`, :meth:`lstrip`, and :meth:`rstrip` string methods now have
1139  an optional argument for specifying the characters to strip.  The default is
1140  still to remove all whitespace characters::
1142     >>> '   abc '.strip()
1143     'abc'
1144     >>> '><><abc<><><>'.strip('<>')
1145     'abc'
1146     >>> '><><abc<><><>\n'.strip('<>')
1147     'abc<><><>\n'
1148     >>> u'\u4000\u4001abc\u4000'.strip(u'\u4000')
1149     u'\u4001abc'
1150     >>>
1152  (Suggested by Simon Brunning and implemented by Walter Dörwald.)
1154* The :meth:`startswith` and :meth:`endswith` string methods now accept negative
1155  numbers for the *start* and *end* parameters.
1157* Another new string method is :meth:`zfill`, originally a function in the
1158  :mod:`string` module.  :meth:`zfill` pads a numeric string with zeros on the
1159  left until it's the specified width. Note that the ``%`` operator is still more
1160  flexible and powerful than :meth:`zfill`. ::
1162     >>> '45'.zfill(4)
1163     '0045'
1164     >>> '12345'.zfill(4)
1165     '12345'
1166     >>> 'goofy'.zfill(6)
1167     '0goofy'
1169  (Contributed by Walter Dörwald.)
1171* A new type object, :class:`basestring`, has been added. Both 8-bit strings and
1172  Unicode strings inherit from this type, so ``isinstance(obj, basestring)`` will
1173  return :const:`True` for either kind of string.  It's a completely abstract
1174  type, so you can't create :class:`basestring` instances.
1176* Interned strings are no longer immortal and will now be garbage-collected in
1177  the usual way when the only reference to them is from the internal dictionary of
1178  interned strings.  (Implemented by Oren Tirosh.)
1180.. ======================================================================
1186* The creation of new-style class instances has been made much faster; they're
1187  now faster than classic classes!
1189* The :meth:`sort` method of list objects has been extensively rewritten by Tim
1190  Peters, and the implementation is significantly faster.
1192* Multiplication of large long integers is now much faster thanks to an
1193  implementation of Karatsuba multiplication, an algorithm that scales better than
1194  the O(n\*n) required for the grade-school multiplication algorithm.  (Original
1195  patch by Christopher A. Craig, and significantly reworked by Tim Peters.)
1197* The ``SET_LINENO`` opcode is now gone.  This may provide a small speed
1198  increase, depending on your compiler's idiosyncrasies. See section
1199  :ref:`section-other` for a longer explanation. (Removed by Michael Hudson.)
1201* :func:`xrange` objects now have their own iterator, making ``for i in
1202  xrange(n)`` slightly faster than ``for i in range(n)``.  (Patch by Raymond
1203  Hettinger.)
1205* A number of small rearrangements have been made in various hotspots to improve
1206  performance, such as inlining a function or removing some code.  (Implemented
1207  mostly by GvR, but lots of people have contributed single changes.)
1209The net result of the 2.3 optimizations is that Python 2.3 runs the  pystone
1210benchmark around 25% faster than Python 2.2.
1212.. ======================================================================
1215New, Improved, and Deprecated Modules
1218As usual, Python's standard library received a number of enhancements and bug
1219fixes.  Here's a partial list of the most notable changes, sorted alphabetically
1220by module name. Consult the :file:`Misc/NEWS` file in the source tree for a more
1221complete list of changes, or look through the CVS logs for all the details.
1223* The :mod:`array` module now supports arrays of Unicode characters using the
1224  ``'u'`` format character.  Arrays also now support using the ``+=`` assignment
1225  operator to add another array's contents, and the ``*=`` assignment operator to
1226  repeat an array. (Contributed by Jason Orendorff.)
1228* The :mod:`bsddb` module has been replaced by version 4.1.6 of the `PyBSDDB
1229  <>`_ package, providing a more complete interface
1230  to the transactional features of the BerkeleyDB library.
1232  The old version of the module has been renamed to  :mod:`bsddb185` and is no
1233  longer built automatically; you'll  have to edit :file:`Modules/Setup` to enable
1234  it.  Note that the new :mod:`bsddb` package is intended to be compatible with
1235  the  old module, so be sure to file bugs if you discover any incompatibilities.
1236  When upgrading to Python 2.3, if the new interpreter is compiled with a new
1237  version of  the underlying BerkeleyDB library, you will almost certainly have to
1238  convert your database files to the new version.  You can do this fairly easily
1239  with the new scripts :file:`` and :file:`` which you
1240  will find in the distribution's :file:`Tools/scripts` directory.  If you've
1241  already been using the PyBSDDB package and importing it as :mod:`bsddb3`, you
1242  will have to change your ``import`` statements to import it as :mod:`bsddb`.
1244* The new :mod:`bz2` module is an interface to the bz2 data compression library.
1245  bz2-compressed data is usually smaller than  corresponding :mod:`zlib`\
1246  -compressed data. (Contributed by Gustavo Niemeyer.)
1248* A set of standard date/time types has been added in the new :mod:`datetime`
1249  module.  See the following section for more details.
1251* The Distutils :class:`Extension` class now supports an extra constructor
1252  argument named *depends* for listing additional source files that an extension
1253  depends on.  This lets Distutils recompile the module if any of the dependency
1254  files are modified.  For example, if :file:`sampmodule.c` includes the header
1255  file :file:`sample.h`, you would create the :class:`Extension` object like
1256  this::
1258     ext = Extension("samp",
1259                     sources=["sampmodule.c"],
1260                     depends=["sample.h"])
1262  Modifying :file:`sample.h` would then cause the module to be recompiled.
1263  (Contributed by Jeremy Hylton.)
1265* Other minor changes to Distutils: it now checks for the :envvar:`CC`,
1266  :envvar:`CFLAGS`, :envvar:`CPP`, :envvar:`LDFLAGS`, and :envvar:`CPPFLAGS`
1267  environment variables, using them to override the settings in Python's
1268  configuration (contributed by Robert Weber).
1270* Previously the :mod:`doctest` module would only search the docstrings of
1271  public methods and functions for test cases, but it now also examines private
1272  ones as well.  The :func:`DocTestSuite(` function creates a
1273  :class:`unittest.TestSuite` object from a set of :mod:`doctest` tests.
1275* The new :func:`gc.get_referents(object)` function returns a list of all the
1276  objects referenced by *object*.
1278* The :mod:`getopt` module gained a new function, :func:`gnu_getopt`, that
1279  supports the same arguments as the existing :func:`getopt` function but uses
1280  GNU-style scanning mode. The existing :func:`getopt` stops processing options as
1281  soon as a non-option argument is encountered, but in GNU-style mode processing
1282  continues, meaning that options and arguments can be mixed.  For example::
1284     >>> getopt.getopt(['-f', 'filename', 'output', '-v'], 'f:v')
1285     ([('-f', 'filename')], ['output', '-v'])
1286     >>> getopt.gnu_getopt(['-f', 'filename', 'output', '-v'], 'f:v')
1287     ([('-f', 'filename'), ('-v', '')], ['output'])
1289  (Contributed by Peter Åstrand.)
1291* The :mod:`grp`, :mod:`pwd`, and :mod:`resource` modules now return enhanced
1292  tuples::
1294     >>> import grp
1295     >>> g = grp.getgrnam('amk')
1296     >>> g.gr_name, g.gr_gid
1297     ('amk', 500)
1299* The :mod:`gzip` module can now handle files exceeding 2 GiB.
1301* The new :mod:`heapq` module contains an implementation of a heap queue
1302  algorithm.  A heap is an array-like data structure that keeps items in a
1303  partially sorted order such that, for every index *k*, ``heap[k] <=
1304  heap[2*k+1]`` and ``heap[k] <= heap[2*k+2]``.  This makes it quick to remove the
1305  smallest item, and inserting a new item while maintaining the heap property is
1306  O(lg n).  (See for more
1307  information about the priority queue data structure.)
1309  The :mod:`heapq` module provides :func:`heappush` and :func:`heappop` functions
1310  for adding and removing items while maintaining the heap property on top of some
1311  other mutable Python sequence type.  Here's an example that uses a Python list::
1313     >>> import heapq
1314     >>> heap = []
1315     >>> for item in [3, 7, 5, 11, 1]:
1316     ...    heapq.heappush(heap, item)
1317     ...
1318     >>> heap
1319     [1, 3, 5, 11, 7]
1320     >>> heapq.heappop(heap)
1321     1
1322     >>> heapq.heappop(heap)
1323     3
1324     >>> heap
1325     [5, 7, 11]
1327  (Contributed by Kevin O'Connor.)
1329* The IDLE integrated development environment has been updated using the code
1330  from the IDLEfork project (  The most notable feature is
1331  that the code being developed is now executed in a subprocess, meaning that
1332  there's no longer any need for manual ``reload()`` operations. IDLE's core code
1333  has been incorporated into the standard library as the :mod:`idlelib` package.
1335* The :mod:`imaplib` module now supports IMAP over SSL. (Contributed by Piers
1336  Lauder and Tino Lange.)
1338* The :mod:`itertools` contains a number of useful functions for use with
1339  iterators, inspired by various functions provided by the ML and Haskell
1340  languages.  For example, ``itertools.ifilter(predicate, iterator)`` returns all
1341  elements in the iterator for which the function :func:`predicate` returns
1342  :const:`True`, and ``itertools.repeat(obj, N)`` returns ``obj`` *N* times.
1343  There are a number of other functions in the module; see the package's reference
1344  documentation for details.
1345  (Contributed by Raymond Hettinger.)
1347* Two new functions in the :mod:`math` module, :func:`degrees(rads)` and
1348  :func:`radians(degs)`, convert between radians and degrees.  Other functions in
1349  the :mod:`math` module such as :func:`math.sin` and :func:`math.cos` have always
1350  required input values measured in radians.  Also, an optional *base* argument
1351  was added to :func:`math.log` to make it easier to compute logarithms for bases
1352  other than ``e`` and ``10``.  (Contributed by Raymond Hettinger.)
1354* Several new POSIX functions (:func:`getpgid`, :func:`killpg`, :func:`lchown`,
1355  :func:`loadavg`, :func:`major`, :func:`makedev`, :func:`minor`, and
1356  :func:`mknod`) were added to the :mod:`posix` module that underlies the
1357  :mod:`os` module. (Contributed by Gustavo Niemeyer, Geert Jansen, and Denis S.
1358  Otkidach.)
1360* In the :mod:`os` module, the :func:`\*stat` family of functions can now report
1361  fractions of a second in a timestamp.  Such time stamps are represented as
1362  floats, similar to the value returned by :func:`time.time`.
1364  During testing, it was found that some applications will break if time stamps
1365  are floats.  For compatibility, when using the tuple interface of the
1366  :class:`stat_result` time stamps will be represented as integers. When using
1367  named fields (a feature first introduced in Python 2.2), time stamps are still
1368  represented as integers, unless :func:`os.stat_float_times` is invoked to enable
1369  float return values::
1371     >>> os.stat("/tmp").st_mtime
1372     1034791200
1373     >>> os.stat_float_times(True)
1374     >>> os.stat("/tmp").st_mtime
1375     1034791200.6335014
1377  In Python 2.4, the default will change to always returning floats.
1379  Application developers should enable this feature only if all their libraries
1380  work properly when confronted with floating point time stamps, or if they use
1381  the tuple API. If used, the feature should be activated on an application level
1382  instead of trying to enable it on a per-use basis.
1384* The :mod:`optparse` module contains a new parser for command-line arguments
1385  that can convert option values to a particular Python type  and will
1386  automatically generate a usage message.  See the following section for  more
1387  details.
1389* The old and never-documented :mod:`linuxaudiodev` module has been deprecated,
1390  and a new version named :mod:`ossaudiodev` has been added.  The module was
1391  renamed because the OSS sound drivers can be used on platforms other than Linux,
1392  and the interface has also been tidied and brought up to date in various ways.
1393  (Contributed by Greg Ward and Nicholas FitzRoy-Dale.)
1395* The new :mod:`platform` module contains a number of functions that try to
1396  determine various properties of the platform you're running on.  There are
1397  functions for getting the architecture, CPU type, the Windows OS version, and
1398  even the Linux distribution version. (Contributed by Marc-André Lemburg.)
1400* The parser objects provided by the :mod:`pyexpat` module can now optionally
1401  buffer character data, resulting in fewer calls to your character data handler
1402  and therefore faster performance.  Setting the parser object's
1403  :attr:`buffer_text` attribute to :const:`True` will enable buffering.
1405* The :func:`sample(population, k)` function was added to the :mod:`random`
1406  module.  *population* is a sequence or :class:`xrange` object containing the
1407  elements of a population, and :func:`sample` chooses *k* elements from the
1408  population without replacing chosen elements.  *k* can be any value up to
1409  ``len(population)``. For example::
1411     >>> days = ['Mo', 'Tu', 'We', 'Th', 'Fr', 'St', 'Sn']
1412     >>> random.sample(days, 3)      # Choose 3 elements
1413     ['St', 'Sn', 'Th']
1414     >>> random.sample(days, 7)      # Choose 7 elements
1415     ['Tu', 'Th', 'Mo', 'We', 'St', 'Fr', 'Sn']
1416     >>> random.sample(days, 7)      # Choose 7 again
1417     ['We', 'Mo', 'Sn', 'Fr', 'Tu', 'St', 'Th']
1418     >>> random.sample(days, 8)      # Can't choose eight
1419     Traceback (most recent call last):
1420       File "<stdin>", line 1, in ?
1421       File "", line 414, in sample
1422           raise ValueError, "sample larger than population"
1423     ValueError: sample larger than population
1424     >>> random.sample(xrange(1,10000,2), 10)   # Choose ten odd nos. under 10000
1425     [3407, 3805, 1505, 7023, 2401, 2267, 9733, 3151, 8083, 9195]
1427  The :mod:`random` module now uses a new algorithm, the Mersenne Twister,
1428  implemented in C.  It's faster and more extensively studied than the previous
1429  algorithm.
1431  (All changes contributed by Raymond Hettinger.)
1433* The :mod:`readline` module also gained a number of new functions:
1434  :func:`get_history_item`, :func:`get_current_history_length`, and
1435  :func:`redisplay`.
1437* The :mod:`rexec` and :mod:`Bastion` modules have been declared dead, and
1438  attempts to import them will fail with a :exc:`RuntimeError`.  New-style classes
1439  provide new ways to break out of the restricted execution environment provided
1440  by :mod:`rexec`, and no one has interest in fixing them or time to do so.  If
1441  you have applications using :mod:`rexec`, rewrite them to use something else.
1443  (Sticking with Python 2.2 or 2.1 will not make your applications any safer
1444  because there are known bugs in the :mod:`rexec` module in those versions.  To
1445  repeat: if you're using :mod:`rexec`, stop using it immediately.)
1447* The :mod:`rotor` module has been deprecated because the  algorithm it uses for
1448  encryption is not believed to be secure.  If you need encryption, use one of the
1449  several AES Python modules that are available separately.
1451* The :mod:`shutil` module gained a :func:`move(src, dest)` function that
1452  recursively moves a file or directory to a new location.
1454* Support for more advanced POSIX signal handling was added to the :mod:`signal`
1455  but then removed again as it proved impossible to make it work reliably across
1456  platforms.
1458* The :mod:`socket` module now supports timeouts.  You can call the
1459  :meth:`settimeout(t)` method on a socket object to set a timeout of *t* seconds.
1460  Subsequent socket operations that take longer than *t* seconds to complete will
1461  abort and raise a :exc:`socket.timeout` exception.
1463  The original timeout implementation was by Tim O'Malley.  Michael Gilfix
1464  integrated it into the Python :mod:`socket` module and shepherded it through a
1465  lengthy review.  After the code was checked in, Guido van Rossum rewrote parts
1466  of it.  (This is a good example of a collaborative development process in
1467  action.)
1469* On Windows, the :mod:`socket` module now ships with Secure  Sockets Layer
1470  (SSL) support.
1472* The value of the C :const:`PYTHON_API_VERSION` macro is now exposed at the
1473  Python level as ``sys.api_version``.  The current exception can be cleared by
1474  calling the new :func:`sys.exc_clear` function.
1476* The new :mod:`tarfile` module  allows reading from and writing to
1477  :program:`tar`\ -format archive files. (Contributed by Lars Gustäbel.)
1479* The new :mod:`textwrap` module contains functions for wrapping strings
1480  containing paragraphs of text.  The :func:`wrap(text, width)` function takes a
1481  string and returns a list containing the text split into lines of no more than
1482  the chosen width.  The :func:`fill(text, width)` function returns a single
1483  string, reformatted to fit into lines no longer than the chosen width. (As you
1484  can guess, :func:`fill` is built on top of :func:`wrap`.  For example::
1486     >>> import textwrap
1487     >>> paragraph = "Not a whit, we defy augury: ... more text ..."
1488     >>> textwrap.wrap(paragraph, 60)
1489     ["Not a whit, we defy augury: there's a special providence in",
1490      "the fall of a sparrow. If it be now, 'tis not to come; if it",
1491      ...]
1492     >>> print textwrap.fill(paragraph, 35)
1493     Not a whit, we defy augury: there's
1494     a special providence in the fall of
1495     a sparrow. If it be now, 'tis not
1496     to come; if it be not to come, it
1497     will be now; if it be not now, yet
1498     it will come: the readiness is all.
1499     >>>
1501  The module also contains a :class:`TextWrapper` class that actually implements
1502  the text wrapping strategy.   Both the :class:`TextWrapper` class and the
1503  :func:`wrap` and :func:`fill` functions support a number of additional keyword
1504  arguments for fine-tuning the formatting; consult the module's documentation
1505  for details. (Contributed by Greg Ward.)
1507* The :mod:`thread` and :mod:`threading` modules now have companion modules,
1508  :mod:`dummy_thread` and :mod:`dummy_threading`, that provide a do-nothing
1509  implementation of the :mod:`thread` module's interface for platforms where
1510  threads are not supported.  The intention is to simplify thread-aware modules
1511  (ones that *don't* rely on threads to run) by putting the following code at the
1512  top::
1514     try:
1515         import threading as _threading
1516     except ImportError:
1517         import dummy_threading as _threading
1519  In this example, :mod:`_threading` is used as the module name to make it clear
1520  that the module being used is not necessarily the actual :mod:`threading`
1521  module. Code can call functions and use classes in :mod:`_threading` whether or
1522  not threads are supported, avoiding an :keyword:`if` statement and making the
1523  code slightly clearer.  This module will not magically make multithreaded code
1524  run without threads; code that waits for another thread to return or to do
1525  something will simply hang forever.
1527* The :mod:`time` module's :func:`strptime` function has long been an annoyance
1528  because it uses the platform C library's :func:`strptime` implementation, and
1529  different platforms sometimes have odd bugs.  Brett Cannon contributed a
1530  portable implementation that's written in pure Python and should behave
1531  identically on all platforms.
1533* The new :mod:`timeit` module helps measure how long snippets of Python code
1534  take to execute.  The :file:`` file can be run directly from the
1535  command line, or the module's :class:`Timer` class can be imported and used
1536  directly.  Here's a short example that figures out whether it's faster to
1537  convert an 8-bit string to Unicode by appending an empty Unicode string to it or
1538  by using the :func:`unicode` function::
1540     import timeit
1542     timer1 = timeit.Timer('unicode("abc")')
1543     timer2 = timeit.Timer('"abc" + u""')
1545     # Run three trials
1546     print timer1.repeat(repeat=3, number=100000)
1547     print timer2.repeat(repeat=3, number=100000)
1549     # On my laptop this outputs:
1550     # [0.36831796169281006, 0.37441694736480713, 0.35304892063140869]
1551     # [0.17574405670166016, 0.18193507194519043, 0.17565798759460449]
1553* The :mod:`Tix` module has received various bug fixes and updates for the
1554  current version of the Tix package.
1556* The :mod:`Tkinter` module now works with a thread-enabled  version of Tcl.
1557  Tcl's threading model requires that widgets only be accessed from the thread in
1558  which they're created; accesses from another thread can cause Tcl to panic.  For
1559  certain Tcl interfaces, :mod:`Tkinter` will now automatically avoid this  when a
1560  widget is accessed from a different thread by marshalling a command, passing it
1561  to the correct thread, and waiting for the results.  Other interfaces can't be
1562  handled automatically but :mod:`Tkinter` will now raise an exception on such an
1563  access so that you can at least find out about the problem.  See
1564 for a more
1565  detailed explanation of this change.  (Implemented by Martin von Löwis.)
1567* Calling Tcl methods through :mod:`_tkinter` no longer  returns only strings.
1568  Instead, if Tcl returns other objects those objects are converted to their
1569  Python equivalent, if one exists, or wrapped with a :class:`_tkinter.Tcl_Obj`
1570  object if no Python equivalent exists. This behavior can be controlled through
1571  the :meth:`wantobjects` method of :class:`tkapp` objects.
1573  When using :mod:`_tkinter` through the :mod:`Tkinter` module (as most Tkinter
1574  applications will), this feature is always activated. It should not cause
1575  compatibility problems, since Tkinter would always convert string results to
1576  Python types where possible.
1578  If any incompatibilities are found, the old behavior can be restored by setting
1579  the :attr:`wantobjects` variable in the :mod:`Tkinter` module to false before
1580  creating the first :class:`tkapp` object. ::
1582     import Tkinter
1583     Tkinter.wantobjects = 0
1585  Any breakage caused by this change should be reported as a bug.
1587* The :mod:`UserDict` module has a new :class:`DictMixin` class which defines
1588  all dictionary methods for classes that already have a minimum mapping
1589  interface.  This greatly simplifies writing classes that need to be
1590  substitutable for dictionaries, such as the classes in  the :mod:`shelve`
1591  module.
1593  Adding the mix-in as a superclass provides the full dictionary interface
1594  whenever the class defines :meth:`__getitem__`, :meth:`__setitem__`,
1595  :meth:`__delitem__`, and :meth:`keys`. For example::
1597     >>> import UserDict
1598     >>> class SeqDict(UserDict.DictMixin):
1599     ...     """Dictionary lookalike implemented with lists."""
1600     ...     def __init__(self):
1601     ...         self.keylist = []
1602     ...         self.valuelist = []
1603     ...     def __getitem__(self, key):
1604     ...         try:
1605     ...             i = self.keylist.index(key)
1606     ...         except ValueError:
1607     ...             raise KeyError
1608     ...         return self.valuelist[i]
1609     ...     def __setitem__(self, key, value):
1610     ...         try:
1611     ...             i = self.keylist.index(key)
1612     ...             self.valuelist[i] = value
1613     ...         except ValueError:
1614     ...             self.keylist.append(key)
1615     ...             self.valuelist.append(value)
1616     ...     def __delitem__(self, key):
1617     ...         try:
1618     ...             i = self.keylist.index(key)
1619     ...         except ValueError:
1620     ...             raise KeyError
1621     ...         self.keylist.pop(i)
1622     ...         self.valuelist.pop(i)
1623     ...     def keys(self):
1624     ...         return list(self.keylist)
1625     ...
1626     >>> s = SeqDict()
1627     >>> dir(s)      # See that other dictionary methods are implemented
1628     ['__cmp__', '__contains__', '__delitem__', '__doc__', '__getitem__',
1629      '__init__', '__iter__', '__len__', '__module__', '__repr__',
1630      '__setitem__', 'clear', 'get', 'has_key', 'items', 'iteritems',
1631      'iterkeys', 'itervalues', 'keylist', 'keys', 'pop', 'popitem',
1632      'setdefault', 'update', 'valuelist', 'values']
1634  (Contributed by Raymond Hettinger.)
1636* The DOM implementation in :mod:`xml.dom.minidom` can now generate XML output
1637  in a particular encoding by providing an optional encoding argument to the
1638  :meth:`toxml` and :meth:`toprettyxml` methods of DOM nodes.
1640* The :mod:`xmlrpclib` module now supports an XML-RPC extension for handling nil
1641  data values such as Python's ``None``.  Nil values are always supported on
1642  unmarshalling an XML-RPC response.  To generate requests containing ``None``,
1643  you must supply a true value for the *allow_none* parameter when creating a
1644  :class:`Marshaller` instance.
1646* The new :mod:`DocXMLRPCServer` module allows writing self-documenting XML-RPC
1647  servers. Run it in demo mode (as a program) to see it in action.   Pointing the
1648  Web browser to the RPC server produces pydoc-style documentation; pointing
1649  xmlrpclib to the server allows invoking the actual methods. (Contributed by
1650  Brian Quinlan.)
1652* Support for internationalized domain names (RFCs 3454, 3490, 3491, and 3492)
1653  has been added. The "idna" encoding can be used to convert between a Unicode
1654  domain name and the ASCII-compatible encoding (ACE) of that name. ::
1656     >{}>{}> u"www.Alliancefranç".encode("idna")
1657     ''
1659  The :mod:`socket` module has also been extended to transparently convert
1660  Unicode hostnames to the ACE version before passing them to the C library.
1661  Modules that deal with hostnames such as :mod:`httplib` and :mod:`ftplib`)
1662  also support Unicode host names; :mod:`httplib` also sends HTTP ``Host``
1663  headers using the ACE version of the domain name.  :mod:`urllib` supports
1664  Unicode URLs with non-ASCII host names as long as the ``path`` part of the URL
1665  is ASCII only.
1667  To implement this change, the :mod:`stringprep` module, the  ``mkstringprep``
1668  tool and the ``punycode`` encoding have been added.
1670.. ======================================================================
1673Date/Time Type
1676Date and time types suitable for expressing timestamps were added as the
1677:mod:`datetime` module.  The types don't support different calendars or many
1678fancy features, and just stick to the basics of representing time.
1680The three primary types are: :class:`date`, representing a day, month, and year;
1681:class:`time`, consisting of hour, minute, and second; and :class:`datetime`,
1682which contains all the attributes of both :class:`date` and :class:`time`.
1683There's also a :class:`timedelta` class representing differences between two
1684points in time, and time zone logic is implemented by classes inheriting from
1685the abstract :class:`tzinfo` class.
1687You can create instances of :class:`date` and :class:`time` by either supplying
1688keyword arguments to the appropriate constructor, e.g.
1689``, month=10, day=15)``, or by using one of a number of
1690class methods.  For example, the :meth:`` class method returns the
1691current local date.
1693Once created, instances of the date/time classes are all immutable. There are a
1694number of methods for producing formatted strings from objects::
1696   >>> import datetime
1697   >>> now =
1698   >>> now.isoformat()
1699   '2002-12-30T21:27:03.994956'
1700   >>> now.ctime()  # Only available on date, datetime
1701   'Mon Dec 30 21:27:03 2002'
1702   >>> now.strftime('%Y %d %b')
1703   '2002 30 Dec'
1705The :meth:`replace` method allows modifying one or more fields  of a
1706:class:`date` or :class:`datetime` instance, returning a new instance::
1708   >>> d =
1709   >>> d
1710   datetime.datetime(2002, 12, 30, 22, 15, 38, 827738)
1711   >>> d.replace(year=2001, hour = 12)
1712   datetime.datetime(2001, 12, 30, 12, 15, 38, 827738)
1713   >>>
1715Instances can be compared, hashed, and converted to strings (the result is the
1716same as that of :meth:`isoformat`).  :class:`date` and :class:`datetime`
1717instances can be subtracted from each other, and added to :class:`timedelta`
1718instances.  The largest missing feature is that there's no standard library
1719support for parsing strings and getting back a :class:`date` or
1722For more information, refer to the module's reference documentation.
1723(Contributed by Tim Peters.)
1725.. ======================================================================
1728The optparse Module
1731The :mod:`getopt` module provides simple parsing of command-line arguments.  The
1732new :mod:`optparse` module (originally named Optik) provides more elaborate
1733command-line parsing that follows the Unix conventions, automatically creates
1734the output for :option:`--help`, and can perform different actions for different
1737You start by creating an instance of :class:`OptionParser` and telling it what
1738your program's options are. ::
1740   import sys
1741   from optparse import OptionParser
1743   op = OptionParser()
1744   op.add_option('-i', '--input',
1745                 action='store', type='string', dest='input',
1746                 help='set input filename')
1747   op.add_option('-l', '--length',
1748                 action='store', type='int', dest='length',
1749                 help='set maximum length of output')
1751Parsing a command line is then done by calling the :meth:`parse_args` method. ::
1753   options, args = op.parse_args(sys.argv[1:])
1754   print options
1755   print args
1757This returns an object containing all of the option values, and a list of
1758strings containing the remaining arguments.
1760Invoking the script with the various arguments now works as you'd expect it to.
1761Note that the length argument is automatically converted to an integer. ::
1763   $ ./python -i data arg1
1764   <Values at 0x400cad4c: {'input': 'data', 'length': None}>
1765   ['arg1']
1766   $ ./python --input=data --length=4
1767   <Values at 0x400cad2c: {'input': 'data', 'length': 4}>
1768   []
1769   $
1771The help message is automatically generated for you::
1773   $ ./python --help
1774   usage: [options]
1776   options:
1777     -h, --help            show this help message and exit
1778     -iINPUT, --input=INPUT
1779                           set input filename
1780     -lLENGTH, --length=LENGTH
1781                           set maximum length of output
1782   $
1784See the module's documentation for more details.
1787Optik was written by Greg Ward, with suggestions from the readers of the Getopt
1790.. ======================================================================
1793.. _section-pymalloc:
1795Pymalloc: A Specialized Object Allocator
1798Pymalloc, a specialized object allocator written by Vladimir Marangozov, was a
1799feature added to Python 2.1.  Pymalloc is intended to be faster than the system
1800:cfunc:`malloc` and to have less memory overhead for allocation patterns typical
1801of Python programs. The allocator uses C's :cfunc:`malloc` function to get large
1802pools of memory and then fulfills smaller memory requests from these pools.
1804In 2.1 and 2.2, pymalloc was an experimental feature and wasn't enabled by
1805default; you had to explicitly enable it when compiling Python by providing the
1806:option:`--with-pymalloc` option to the :program:`configure` script.  In 2.3,
1807pymalloc has had further enhancements and is now enabled by default; you'll have
1808to supply :option:`--without-pymalloc` to disable it.
1810This change is transparent to code written in Python; however, pymalloc may
1811expose bugs in C extensions.  Authors of C extension modules should test their
1812code with pymalloc enabled, because some incorrect code may cause core dumps at
1815There's one particularly common error that causes problems.  There are a number
1816of memory allocation functions in Python's C API that have previously just been
1817aliases for the C library's :cfunc:`malloc` and :cfunc:`free`, meaning that if
1818you accidentally called mismatched functions the error wouldn't be noticeable.
1819When the object allocator is enabled, these functions aren't aliases of
1820:cfunc:`malloc` and :cfunc:`free` any more, and calling the wrong function to
1821free memory may get you a core dump.  For example, if memory was allocated using
1822:cfunc:`PyObject_Malloc`, it has to be freed using :cfunc:`PyObject_Free`, not
1823:cfunc:`free`.  A few modules included with Python fell afoul of this and had to
1824be fixed; doubtless there are more third-party modules that will have the same
1827As part of this change, the confusing multiple interfaces for allocating memory
1828have been consolidated down into two API families. Memory allocated with one
1829family must not be manipulated with functions from the other family.  There is
1830one family for allocating chunks of memory and another family of functions
1831specifically for allocating Python objects.
1833* To allocate and free an undistinguished chunk of memory use the "raw memory"
1834  family: :cfunc:`PyMem_Malloc`, :cfunc:`PyMem_Realloc`, and :cfunc:`PyMem_Free`.
1836* The "object memory" family is the interface to the pymalloc facility described
1837  above and is biased towards a large number of "small" allocations:
1838  :cfunc:`PyObject_Malloc`, :cfunc:`PyObject_Realloc`, and :cfunc:`PyObject_Free`.
1840* To allocate and free Python objects, use the "object" family
1841  :cfunc:`PyObject_New`, :cfunc:`PyObject_NewVar`, and :cfunc:`PyObject_Del`.
1843Thanks to lots of work by Tim Peters, pymalloc in 2.3 also provides debugging
1844features to catch memory overwrites and doubled frees in both extension modules
1845and in the interpreter itself.  To enable this support, compile a debugging
1846version of the Python interpreter by running :program:`configure` with
1849To aid extension writers, a header file :file:`Misc/pymemcompat.h` is
1850distributed with the source to Python 2.3 that allows Python extensions to use
1851the 2.3 interfaces to memory allocation while compiling against any version of
1852Python since 1.5.2.  You would copy the file from Python's source distribution
1853and bundle it with the source of your extension.
1856.. seealso::
1859      For the full details of the pymalloc implementation, see the comments at the top
1860      of the file :file:`Objects/obmalloc.c` in the Python source code.  The above
1861      link points to the file within the SourceForge CVS browser.
1863.. ======================================================================
1866Build and C API Changes
1869Changes to Python's build process and to the C API include:
1871* The cycle detection implementation used by the garbage collection has proven
1872  to be stable, so it's now been made mandatory.  You can no longer compile Python
1873  without it, and the :option:`--with-cycle-gc` switch to :program:`configure` has
1874  been removed.
1876* Python can now optionally be built as a shared library
1877  (:file:``) by supplying :option:`--enable-shared` when running
1878  Python's :program:`configure` script.  (Contributed by Ondrej Palkovsky.)
1880* The :cmacro:`DL_EXPORT` and :cmacro:`DL_IMPORT` macros are now deprecated.
1881  Initialization functions for Python extension modules should now be declared
1882  using the new macro :cmacro:`PyMODINIT_FUNC`, while the Python core will
1883  generally use the :cmacro:`PyAPI_FUNC` and :cmacro:`PyAPI_DATA` macros.
1885* The interpreter can be compiled without any docstrings for the built-in
1886  functions and modules by supplying :option:`--without-doc-strings` to the
1887  :program:`configure` script. This makes the Python executable about 10% smaller,
1888  but will also mean that you can't get help for Python's built-ins.  (Contributed
1889  by Gustavo Niemeyer.)
1891* The :cfunc:`PyArg_NoArgs` macro is now deprecated, and code that uses it
1892  should be changed.  For Python 2.2 and later, the method definition table can
1893  specify the :const:`METH_NOARGS` flag, signalling that there are no arguments,
1894  and the argument checking can then be removed.  If compatibility with pre-2.2
1895  versions of Python is important, the code could use ``PyArg_ParseTuple(args,
1896  "")`` instead, but this will be slower than using :const:`METH_NOARGS`.
1898* :cfunc:`PyArg_ParseTuple` accepts new format characters for various sizes of
1899  unsigned integers: ``B`` for :ctype:`unsigned char`, ``H`` for :ctype:`unsigned
1900  short int`,  ``I`` for :ctype:`unsigned int`,  and ``K`` for :ctype:`unsigned
1901  long long`.
1903* A new function, :cfunc:`PyObject_DelItemString(mapping, char \*key)` was added
1904  as shorthand for ``PyObject_DelItem(mapping, PyString_New(key))``.
1906* File objects now manage their internal string buffer differently, increasing
1907  it exponentially when needed.  This results in the benchmark tests in
1908  :file:`Lib/test/` speeding up considerably (from 57 seconds to 1.7
1909  seconds, according to one measurement).
1911* It's now possible to define class and static methods for a C extension type by
1912  setting either the :const:`METH_CLASS` or :const:`METH_STATIC` flags in a
1913  method's :ctype:`PyMethodDef` structure.
1915* Python now includes a copy of the Expat XML parser's source code, removing any
1916  dependence on a system version or local installation of Expat.
1918* If you dynamically allocate type objects in your extension, you should be
1919  aware of a change in the rules relating to the :attr:`__module__` and
1920  :attr:`__name__` attributes.  In summary, you will want to ensure the type's
1921  dictionary contains a ``'__module__'`` key; making the module name the part of
1922  the type name leading up to the final period will no longer have the desired
1923  effect.  For more detail, read the API reference documentation or the  source.
1925.. ======================================================================
1928Port-Specific Changes
1931Support for a port to IBM's OS/2 using the EMX runtime environment was merged
1932into the main Python source tree.  EMX is a POSIX emulation layer over the OS/2
1933system APIs.  The Python port for EMX tries to support all the POSIX-like
1934capability exposed by the EMX runtime, and mostly succeeds; :func:`fork` and
1935:func:`fcntl` are restricted by the limitations of the underlying emulation
1936layer.  The standard OS/2 port, which uses IBM's Visual Age compiler, also
1937gained support for case-sensitive import semantics as part of the integration of
1938the EMX port into CVS.  (Contributed by Andrew MacIntyre.)
1940On MacOS, most toolbox modules have been weaklinked to improve backward
1941compatibility.  This means that modules will no longer fail to load if a single
1942routine is missing on the current OS version. Instead calling the missing
1943routine will raise an exception. (Contributed by Jack Jansen.)
1945The RPM spec files, found in the :file:`Misc/RPM/` directory in the Python
1946source distribution, were updated for 2.3.  (Contributed by Sean Reifschneider.)
1948Other new platforms now supported by Python include AtheOS
1949(, GNU/Hurd, and OpenVMS.
1951.. ======================================================================
1954.. _section-other:
1956Other Changes and Fixes
1959As usual, there were a bunch of other improvements and bugfixes scattered
1960throughout the source tree.  A search through the CVS change logs finds there
1961were 523 patches applied and 514 bugs fixed between Python 2.2 and 2.3.  Both
1962figures are likely to be underestimates.
1964Some of the more notable changes are:
1966* If the :envvar:`PYTHONINSPECT` environment variable is set, the Python
1967  interpreter will enter the interactive prompt after running a Python program, as
1968  if Python had been invoked with the :option:`-i` option. The environment
1969  variable can be set before running the Python interpreter, or it can be set by
1970  the Python program as part of its execution.
1972* The :file:`` script now provides a way to allow "all resources
1973  except *foo*."  A resource name passed to the :option:`-u` option can now be
1974  prefixed with a hyphen (``'-'``) to mean "remove this resource."  For example,
1975  the option '``-uall,-bsddb``' could be used to enable the use of all resources
1976  except ``bsddb``.
1978* The tools used to build the documentation now work under Cygwin as well as
1979  Unix.
1981* The ``SET_LINENO`` opcode has been removed.  Back in the mists of time, this
1982  opcode was needed to produce line numbers in tracebacks and support trace
1983  functions (for, e.g., :mod:`pdb`). Since Python 1.5, the line numbers in
1984  tracebacks have been computed using a different mechanism that works with
1985  "python -O".  For Python 2.3 Michael Hudson implemented a similar scheme to
1986  determine when to call the trace function, removing the need for ``SET_LINENO``
1987  entirely.
1989  It would be difficult to detect any resulting difference from Python code, apart
1990  from a slight speed up when Python is run without :option:`-O`.
1992  C extensions that access the :attr:`f_lineno` field of frame objects should
1993  instead call ``PyCode_Addr2Line(f->f_code, f->f_lasti)``. This will have the
1994  added effect of making the code work as desired under "python -O" in earlier
1995  versions of Python.
1997  A nifty new feature is that trace functions can now assign to the
1998  :attr:`f_lineno` attribute of frame objects, changing the line that will be
1999  executed next.  A ``jump`` command has been added to the :mod:`pdb` debugger
2000  taking advantage of this new feature. (Implemented by Richie Hindle.)
2002.. ======================================================================
2005Porting to Python 2.3
2008This section lists previously described changes that may require changes to your
2011* :keyword:`yield` is now always a keyword; if it's used as a variable name in
2012  your code, a different name must be chosen.
2014* For strings *X* and *Y*, ``X in Y`` now works if *X* is more than one
2015  character long.
2017* The :func:`int` type constructor will now return a long integer instead of
2018  raising an :exc:`OverflowError` when a string or floating-point number is too
2019  large to fit into an integer.
2021* If you have Unicode strings that contain 8-bit characters, you must declare
2022  the file's encoding (UTF-8, Latin-1, or whatever) by adding a comment to the top
2023  of the file.  See section :ref:`section-encodings` for more information.
2025* Calling Tcl methods through :mod:`_tkinter` no longer  returns only strings.
2026  Instead, if Tcl returns other objects those objects are converted to their
2027  Python equivalent, if one exists, or wrapped with a :class:`_tkinter.Tcl_Obj`
2028  object if no Python equivalent exists.
2030* Large octal and hex literals such as ``0xffffffff`` now trigger a
2031  :exc:`FutureWarning`. Currently they're stored as 32-bit numbers and result in a
2032  negative value, but in Python 2.4 they'll become positive long integers.
2034  There are a few ways to fix this warning.  If you really need a positive number,
2035  just add an ``L`` to the end of the literal.  If you're trying to get a 32-bit
2036  integer with low bits set and have previously used an expression such as ``~(1
2037  << 31)``, it's probably clearest to start with all bits set and clear the
2038  desired upper bits. For example, to clear just the top bit (bit 31), you could
2039  write ``0xffffffffL &~(1L<<31)``.
2041* You can no longer disable assertions by assigning to ``__debug__``.
2043* The Distutils :func:`setup` function has gained various new keyword arguments
2044  such as *depends*.  Old versions of the Distutils will abort if passed unknown
2045  keywords.  A solution is to check for the presence of the new
2046  :func:`get_distutil_options` function in your :file:`` and only uses the
2047  new keywords with a version of the Distutils that supports them::
2049     from distutils import core
2051     kw = {'sources': 'foo.c', ...}
2052     if hasattr(core, 'get_distutil_options'):
2053         kw['depends'] = ['foo.h']
2054     ext = Extension(**kw)
2056* Using ``None`` as a variable name will now result in a :exc:`SyntaxWarning`
2057  warning.
2059* Names of extension types defined by the modules included with Python now
2060  contain the module and a ``'.'`` in front of the type name.
2062.. ======================================================================
2065.. _23acks:
2070The author would like to thank the following people for offering suggestions,
2071corrections and assistance with various drafts of this article: Jeff Bauer,
2072Simon Brunning, Brett Cannon, Michael Chermside, Andrew Dalke, Scott David
2073Daniels, Fred L. Drake, Jr., David Fraser,  Kelly Gerber, Raymond Hettinger,
2074Michael Hudson, Chris Lambert, Detlef Lannert, Martin von Löwis, Andrew
2075MacIntyre, Lalo Martins, Chad Netzer, Gustavo Niemeyer, Neal Norwitz, Hans
2076Nowak, Chris Reedy, Francesco Ricciardi, Vinay Sajip, Neil Schemenauer, Roman
2077Suzi, Jason Tishler, Just van Rossum.