parserparser module provides an interface to Python's internal
parser and byte-code compiler. The primary purpose for this interface
is to allow Python code to edit the parse tree of a Python expression
and create executable code from this. This can be better than trying
to parse and modify an arbitrary Python code fragment as a string, and
ensures that parsing is performed in a manner identical to the code
forming the application. It's also faster.
There are a few things to note about this module which are important to making use of the data structures created. This is not a tutorial on editing the parse trees for Python code.
Most importantly, a good understanding of the Python grammar processed
by the internal parser is required. For full information on the
language syntax, refer to the Language Reference. The parser itself
is created from a grammar specification defined in the file
Grammar/Grammar in the standard Python distribution. The parse
trees stored in the ``AST objects'' created by this module are the
actual output from the internal parser when created by the
expr() or suite() functions, described below. The AST
objects created by tuple2ast() faithfully simulate those
structures.
Each element of the tuples returned by ast2tuple() has a simple
form. Tuples representing non-terminal elements in the grammar always
have a length greater than one. The first element is an integer which
identifies a production in the grammar. These integers are given
symbolic names in the C header file Include/graminit.h and the
Python module Lib/symbol.py. Each additional element of the
tuple represents a component of the production as recognized in the
input string: these are always tuples which have the same form as the
parent. An important aspect of this structure which should be noted
is that keywords used to identify the parent node type, such as the
keyword if in an if_stmt, are included in the node tree
without any special treatment. For example, the if keyword is
represented by the tuple (1, 'if'), where 1 is the
numeric value associated with all NAME elements, including
variable and function names defined by the user.
Terminal elements are represented in much the same way, but without
any child elements and the addition of the source text which was
identified. The example of the if keyword above is
representative. The various types of terminal symbols are defined in
the C header file Include/token.h and the Python module
Lib/token.py.
The AST objects are not actually required to support the functionality of this module, but are provided for three purposes: to allow an application to amortize the cost of processing complex parse trees, to provide a parse tree representation which conserves memory space when compared to the Python tuple representation, and to ease the creation of additional modules in C which manipulate parse trees. A simple ``wrapper'' module may be created in Python if desired to hide the use of AST objects.
The parser module defines the following functions:
ast and returns a Python tuple representing the
equivelent parse tree. The resulting tuple representation can be used
for inspection or the creation of a new parse tree in tuple form.
This function does not fail so long as memory is available to build
the tuple representation.
= '<ast>']) -- function of module parserexec statement or
a call to the built-in eval() function. This function provides
the interface to the compiler, passing the internal parse tree from
ast to the parser, using the source file name specified
by the filename parameter. The default value supplied
for filename indicates that the source was an AST object.
expr() function parses the parameter string
as if it were an input to compile(string, 'eval'). If
the parse succeeds, an AST object is created to hold the internal
parse tree representation, otherwise an appropriate exception is
thrown.
ast represents an 'eval' form, this function
returns a true value (1), otherwise it returns false
(0). This is useful, since code objects normally cannot be
queried for this information using existing built-in functions. Note
that the code objects created by compileast() cannot be queried
like this either, and are identical to those created by the built-in
compile() function.
isexpr() in that it reports whether an
AST object represents a suite of statements. It is not safe to assume
that this function is equivelent to not isexpr(ast), as
additional syntactic fragments may be supported in the future.
suite() function parses the parameter string
as if it were an input to compile(string, 'exec'). If
the parse succeeds, an AST object is created to hold the internal
parse tree representation, otherwise an appropriate exception is
thrown.
ParserError exception is thrown. An AST object
created this way should not be assumed to compile correctly; normal
exceptions thrown by compilation may still be initiated when the AST
object is passed to compileast(). This will normally indicate
problems not related to syntax (such as a MemoryError
exception).