/Doc/library/compiler.rst
http://unladen-swallow.googlecode.com/ · ReStructuredText · 644 lines · 534 code · 110 blank · 0 comment · 0 complexity · 9faa21e5e8e8032ff91bdf0c51c8f4f0 MD5 · raw file
- .. _compiler:
- ***********************
- Python compiler package
- ***********************
- .. deprecated:: 2.6
- The :mod:`compiler` package has been removed in Python 3.0.
- .. sectionauthor:: Jeremy Hylton <jeremy@zope.com>
- The Python compiler package is a tool for analyzing Python source code and
- generating Python bytecode. The compiler contains libraries to generate an
- abstract syntax tree from Python source code and to generate Python
- :term:`bytecode` from the tree.
- The :mod:`compiler` package is a Python source to bytecode translator written in
- Python. It uses the built-in parser and standard :mod:`parser` module to
- generated a concrete syntax tree. This tree is used to generate an abstract
- syntax tree (AST) and then Python bytecode.
- The full functionality of the package duplicates the builtin compiler provided
- with the Python interpreter. It is intended to match its behavior almost
- exactly. Why implement another compiler that does the same thing? The package
- is useful for a variety of purposes. It can be modified more easily than the
- builtin compiler. The AST it generates is useful for analyzing Python source
- code.
- This chapter explains how the various components of the :mod:`compiler` package
- work. It blends reference material with a tutorial.
- The basic interface
- ===================
- .. module:: compiler
- :synopsis: Python code compiler written in Python.
- :deprecated:
- The top-level of the package defines four functions. If you import
- :mod:`compiler`, you will get these functions and a collection of modules
- contained in the package.
- .. function:: parse(buf)
- Returns an abstract syntax tree for the Python source code in *buf*. The
- function raises :exc:`SyntaxError` if there is an error in the source code. The
- return value is a :class:`compiler.ast.Module` instance that contains the tree.
- .. function:: parseFile(path)
- Return an abstract syntax tree for the Python source code in the file specified
- by *path*. It is equivalent to ``parse(open(path).read())``.
- .. function:: walk(ast, visitor[, verbose])
- Do a pre-order walk over the abstract syntax tree *ast*. Call the appropriate
- method on the *visitor* instance for each node encountered.
- .. function:: compile(source, filename, mode, flags=None, dont_inherit=None)
- Compile the string *source*, a Python module, statement or expression, into a
- code object that can be executed by the exec statement or :func:`eval`. This
- function is a replacement for the built-in :func:`compile` function.
- The *filename* will be used for run-time error messages.
- The *mode* must be 'exec' to compile a module, 'single' to compile a single
- (interactive) statement, or 'eval' to compile an expression.
- The *flags* and *dont_inherit* arguments affect future-related statements, but
- are not supported yet.
- .. function:: compileFile(source)
- Compiles the file *source* and generates a .pyc file.
- The :mod:`compiler` package contains the following modules: :mod:`ast`,
- :mod:`consts`, :mod:`future`, :mod:`misc`, :mod:`pyassem`, :mod:`pycodegen`,
- :mod:`symbols`, :mod:`transformer`, and :mod:`visitor`.
- Limitations
- ===========
- There are some problems with the error checking of the compiler package. The
- interpreter detects syntax errors in two distinct phases. One set of errors is
- detected by the interpreter's parser, the other set by the compiler. The
- compiler package relies on the interpreter's parser, so it get the first phases
- of error checking for free. It implements the second phase itself, and that
- implementation is incomplete. For example, the compiler package does not raise
- an error if a name appears more than once in an argument list: ``def f(x, x):
- ...``
- A future version of the compiler should fix these problems.
- Python Abstract Syntax
- ======================
- The :mod:`compiler.ast` module defines an abstract syntax for Python. In the
- abstract syntax tree, each node represents a syntactic construct. The root of
- the tree is :class:`Module` object.
- The abstract syntax offers a higher level interface to parsed Python source
- code. The :mod:`parser` module and the compiler written in C for the Python
- interpreter use a concrete syntax tree. The concrete syntax is tied closely to
- the grammar description used for the Python parser. Instead of a single node
- for a construct, there are often several levels of nested nodes that are
- introduced by Python's precedence rules.
- The abstract syntax tree is created by the :mod:`compiler.transformer` module.
- The transformer relies on the builtin Python parser to generate a concrete
- syntax tree. It generates an abstract syntax tree from the concrete tree.
- .. index::
- single: Stein, Greg
- single: Tutt, Bill
- The :mod:`transformer` module was created by Greg Stein and Bill Tutt for an
- experimental Python-to-C compiler. The current version contains a number of
- modifications and improvements, but the basic form of the abstract syntax and of
- the transformer are due to Stein and Tutt.
- AST Nodes
- ---------
- .. module:: compiler.ast
- The :mod:`compiler.ast` module is generated from a text file that describes each
- node type and its elements. Each node type is represented as a class that
- inherits from the abstract base class :class:`compiler.ast.Node` and defines a
- set of named attributes for child nodes.
- .. class:: Node()
- The :class:`Node` instances are created automatically by the parser generator.
- The recommended interface for specific :class:`Node` instances is to use the
- public attributes to access child nodes. A public attribute may be bound to a
- single node or to a sequence of nodes, depending on the :class:`Node` type. For
- example, the :attr:`bases` attribute of the :class:`Class` node, is bound to a
- list of base class nodes, and the :attr:`doc` attribute is bound to a single
- node.
- Each :class:`Node` instance has a :attr:`lineno` attribute which may be
- ``None``. XXX Not sure what the rules are for which nodes will have a useful
- lineno.
- All :class:`Node` objects offer the following methods:
- .. method:: getChildren()
- Returns a flattened list of the child nodes and objects in the order they
- occur. Specifically, the order of the nodes is the order in which they
- appear in the Python grammar. Not all of the children are :class:`Node`
- instances. The names of functions and classes, for example, are plain
- strings.
- .. method:: getChildNodes()
- Returns a flattened list of the child nodes in the order they occur. This
- method is like :meth:`getChildren`, except that it only returns those
- children that are :class:`Node` instances.
- Two examples illustrate the general structure of :class:`Node` classes. The
- :keyword:`while` statement is defined by the following grammar production::
- while_stmt: "while" expression ":" suite
- ["else" ":" suite]
- The :class:`While` node has three attributes: :attr:`test`, :attr:`body`, and
- :attr:`else_`. (If the natural name for an attribute is also a Python reserved
- word, it can't be used as an attribute name. An underscore is appended to the
- word to make it a legal identifier, hence :attr:`else_` instead of
- :keyword:`else`.)
- The :keyword:`if` statement is more complicated because it can include several
- tests. ::
- if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite]
- The :class:`If` node only defines two attributes: :attr:`tests` and
- :attr:`else_`. The :attr:`tests` attribute is a sequence of test expression,
- consequent body pairs. There is one pair for each :keyword:`if`/:keyword:`elif`
- clause. The first element of the pair is the test expression. The second
- elements is a :class:`Stmt` node that contains the code to execute if the test
- is true.
- The :meth:`getChildren` method of :class:`If` returns a flat list of child
- nodes. If there are three :keyword:`if`/:keyword:`elif` clauses and no
- :keyword:`else` clause, then :meth:`getChildren` will return a list of six
- elements: the first test expression, the first :class:`Stmt`, the second text
- expression, etc.
- The following table lists each of the :class:`Node` subclasses defined in
- :mod:`compiler.ast` and each of the public attributes available on their
- instances. The values of most of the attributes are themselves :class:`Node`
- instances or sequences of instances. When the value is something other than an
- instance, the type is noted in the comment. The attributes are listed in the
- order in which they are returned by :meth:`getChildren` and
- :meth:`getChildNodes`.
- +-----------------------+--------------------+---------------------------------+
- | Node type | Attribute | Value |
- +=======================+====================+=================================+
- | :class:`Add` | :attr:`left` | left operand |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`right` | right operand |
- +-----------------------+--------------------+---------------------------------+
- | :class:`And` | :attr:`nodes` | list of operands |
- +-----------------------+--------------------+---------------------------------+
- | :class:`AssAttr` | | *attribute as target of |
- | | | assignment* |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`expr` | expression on the left-hand |
- | | | side of the dot |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`attrname` | the attribute name, a string |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`flags` | XXX |
- +-----------------------+--------------------+---------------------------------+
- | :class:`AssList` | :attr:`nodes` | list of list elements being |
- | | | assigned to |
- +-----------------------+--------------------+---------------------------------+
- | :class:`AssName` | :attr:`name` | name being assigned to |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`flags` | XXX |
- +-----------------------+--------------------+---------------------------------+
- | :class:`AssTuple` | :attr:`nodes` | list of tuple elements being |
- | | | assigned to |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Assert` | :attr:`test` | the expression to be tested |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`fail` | the value of the |
- | | | :exc:`AssertionError` |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Assign` | :attr:`nodes` | a list of assignment targets, |
- | | | one per equal sign |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`expr` | the value being assigned |
- +-----------------------+--------------------+---------------------------------+
- | :class:`AugAssign` | :attr:`node` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`op` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`expr` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Backquote` | :attr:`expr` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Bitand` | :attr:`nodes` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Bitor` | :attr:`nodes` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Bitxor` | :attr:`nodes` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Break` | | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`CallFunc` | :attr:`node` | expression for the callee |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`args` | a list of arguments |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`star_args` | the extended \*-arg value |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`dstar_args` | the extended \*\*-arg value |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Class` | :attr:`name` | the name of the class, a string |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`bases` | a list of base classes |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`doc` | doc string, a string or |
- | | | ``None`` |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`code` | the body of the class statement |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Compare` | :attr:`expr` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`ops` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Const` | :attr:`value` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Continue` | | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Decorators` | :attr:`nodes` | List of function decorator |
- | | | expressions |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Dict` | :attr:`items` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Discard` | :attr:`expr` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Div` | :attr:`left` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`right` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Ellipsis` | | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Expression` | :attr:`node` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Exec` | :attr:`expr` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`locals` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`globals` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`FloorDiv` | :attr:`left` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`right` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`For` | :attr:`assign` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`list` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`body` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`else_` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`From` | :attr:`modname` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`names` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Function` | :attr:`decorators` | :class:`Decorators` or ``None`` |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`name` | name used in def, a string |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`argnames` | list of argument names, as |
- | | | strings |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`defaults` | list of default values |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`flags` | xxx |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`doc` | doc string, a string or |
- | | | ``None`` |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`code` | the body of the function |
- +-----------------------+--------------------+---------------------------------+
- | :class:`GenExpr` | :attr:`code` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`GenExprFor` | :attr:`assign` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`iter` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`ifs` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`GenExprIf` | :attr:`test` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`GenExprInner` | :attr:`expr` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`quals` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Getattr` | :attr:`expr` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`attrname` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Global` | :attr:`names` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`If` | :attr:`tests` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`else_` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Import` | :attr:`names` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Invert` | :attr:`expr` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Keyword` | :attr:`name` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`expr` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Lambda` | :attr:`argnames` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`defaults` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`flags` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`code` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`LeftShift` | :attr:`left` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`right` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`List` | :attr:`nodes` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`ListComp` | :attr:`expr` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`quals` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`ListCompFor` | :attr:`assign` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`list` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`ifs` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`ListCompIf` | :attr:`test` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Mod` | :attr:`left` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`right` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Module` | :attr:`doc` | doc string, a string or |
- | | | ``None`` |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`node` | body of the module, a |
- | | | :class:`Stmt` |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Mul` | :attr:`left` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`right` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Name` | :attr:`name` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Not` | :attr:`expr` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Or` | :attr:`nodes` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Pass` | | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Power` | :attr:`left` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`right` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Print` | :attr:`nodes` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`dest` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Printnl` | :attr:`nodes` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`dest` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Raise` | :attr:`expr1` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`expr2` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`expr3` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Return` | :attr:`value` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`RightShift` | :attr:`left` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`right` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Slice` | :attr:`expr` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`flags` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`lower` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`upper` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Sliceobj` | :attr:`nodes` | list of statements |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Stmt` | :attr:`nodes` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Sub` | :attr:`left` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`right` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Subscript` | :attr:`expr` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`flags` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`subs` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`TryExcept` | :attr:`body` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`handlers` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`else_` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`TryFinally` | :attr:`body` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`final` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Tuple` | :attr:`nodes` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`UnaryAdd` | :attr:`expr` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`UnarySub` | :attr:`expr` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`While` | :attr:`test` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`body` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`else_` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`With` | :attr:`expr` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`vars` | |
- +-----------------------+--------------------+---------------------------------+
- | | :attr:`body` | |
- +-----------------------+--------------------+---------------------------------+
- | :class:`Yield` | :attr:`value` | |
- +-----------------------+--------------------+---------------------------------+
- Assignment nodes
- ----------------
- There is a collection of nodes used to represent assignments. Each assignment
- statement in the source code becomes a single :class:`Assign` node in the AST.
- The :attr:`nodes` attribute is a list that contains a node for each assignment
- target. This is necessary because assignment can be chained, e.g. ``a = b =
- 2``. Each :class:`Node` in the list will be one of the following classes:
- :class:`AssAttr`, :class:`AssList`, :class:`AssName`, or :class:`AssTuple`.
- Each target assignment node will describe the kind of object being assigned to:
- :class:`AssName` for a simple name, e.g. ``a = 1``. :class:`AssAttr` for an
- attribute assigned, e.g. ``a.x = 1``. :class:`AssList` and :class:`AssTuple` for
- list and tuple expansion respectively, e.g. ``a, b, c = a_tuple``.
- The target assignment nodes also have a :attr:`flags` attribute that indicates
- whether the node is being used for assignment or in a delete statement. The
- :class:`AssName` is also used to represent a delete statement, e.g. :class:`del
- x`.
- When an expression contains several attribute references, an assignment or
- delete statement will contain only one :class:`AssAttr` node -- for the final
- attribute reference. The other attribute references will be represented as
- :class:`Getattr` nodes in the :attr:`expr` attribute of the :class:`AssAttr`
- instance.
- Examples
- --------
- This section shows several simple examples of ASTs for Python source code. The
- examples demonstrate how to use the :func:`parse` function, what the repr of an
- AST looks like, and how to access attributes of an AST node.
- The first module defines a single function. Assume it is stored in
- :file:`/tmp/doublelib.py`. ::
- """This is an example module.
- This is the docstring.
- """
- def double(x):
- "Return twice the argument"
- return x * 2
- In the interactive interpreter session below, I have reformatted the long AST
- reprs for readability. The AST reprs use unqualified class names. If you want
- to create an instance from a repr, you must import the class names from the
- :mod:`compiler.ast` module. ::
- >>> import compiler
- >>> mod = compiler.parseFile("/tmp/doublelib.py")
- >>> mod
- Module('This is an example module.\n\nThis is the docstring.\n',
- Stmt([Function(None, 'double', ['x'], [], 0,
- 'Return twice the argument',
- Stmt([Return(Mul((Name('x'), Const(2))))]))]))
- >>> from compiler.ast import *
- >>> Module('This is an example module.\n\nThis is the docstring.\n',
- ... Stmt([Function(None, 'double', ['x'], [], 0,
- ... 'Return twice the argument',
- ... Stmt([Return(Mul((Name('x'), Const(2))))]))]))
- Module('This is an example module.\n\nThis is the docstring.\n',
- Stmt([Function(None, 'double', ['x'], [], 0,
- 'Return twice the argument',
- Stmt([Return(Mul((Name('x'), Const(2))))]))]))
- >>> mod.doc
- 'This is an example module.\n\nThis is the docstring.\n'
- >>> for node in mod.node.nodes:
- ... print node
- ...
- Function(None, 'double', ['x'], [], 0, 'Return twice the argument',
- Stmt([Return(Mul((Name('x'), Const(2))))]))
- >>> func = mod.node.nodes[0]
- >>> func.code
- Stmt([Return(Mul((Name('x'), Const(2))))])
- Using Visitors to Walk ASTs
- ===========================
- .. module:: compiler.visitor
- The visitor pattern is ... The :mod:`compiler` package uses a variant on the
- visitor pattern that takes advantage of Python's introspection features to
- eliminate the need for much of the visitor's infrastructure.
- The classes being visited do not need to be programmed to accept visitors. The
- visitor need only define visit methods for classes it is specifically interested
- in; a default visit method can handle the rest.
- XXX The magic :meth:`visit` method for visitors.
- .. function:: walk(tree, visitor[, verbose])
- .. class:: ASTVisitor()
- The :class:`ASTVisitor` is responsible for walking over the tree in the correct
- order. A walk begins with a call to :meth:`preorder`. For each node, it checks
- the *visitor* argument to :meth:`preorder` for a method named 'visitNodeType,'
- where NodeType is the name of the node's class, e.g. for a :class:`While` node a
- :meth:`visitWhile` would be called. If the method exists, it is called with the
- node as its first argument.
- The visitor method for a particular node type can control how child nodes are
- visited during the walk. The :class:`ASTVisitor` modifies the visitor argument
- by adding a visit method to the visitor; this method can be used to visit a
- particular child node. If no visitor is found for a particular node type, the
- :meth:`default` method is called.
- :class:`ASTVisitor` objects have the following methods:
- XXX describe extra arguments
- .. method:: default(node[, ...])
- .. method:: dispatch(node[, ...])
- .. method:: preorder(tree, visitor)
- Bytecode Generation
- ===================
- The code generator is a visitor that emits bytecodes. Each visit method can
- call the :meth:`emit` method to emit a new bytecode. The basic code generator
- is specialized for modules, classes, and functions. An assembler converts that
- emitted instructions to the low-level bytecode format. It handles things like
- generation of constant lists of code objects and calculation of jump offsets.