/Doc/tutorial/classes.rst

  1.. _tut-classes:
  2
  3*******
  4Classes
  5*******
  6
  7Python's class mechanism adds classes to the language with a minimum of new
  8syntax and semantics.  It is a mixture of the class mechanisms found in C++ and
  9Modula-3.  As is true for modules, classes in Python do not put an absolute
 10barrier between definition and user, but rather rely on the politeness of the
 11user not to "break into the definition."  The most important features of classes
 12are retained with full power, however: the class inheritance mechanism allows
 13multiple base classes, a derived class can override any methods of its base
 14class or classes, and a method can call the method of a base class with the same
 15name.  Objects can contain an arbitrary amount of private data.
 16
 17In C++ terminology, all class members (including the data members) are *public*,
 18and all member functions are *virtual*.  There are no special constructors or
 19destructors.  As in Modula-3, there are no shorthands for referencing the
 20object's members from its methods: the method function is declared with an
 21explicit first argument representing the object, which is provided implicitly by
 22the call.  As in Smalltalk, classes themselves are objects, albeit in the wider
 23sense of the word: in Python, all data types are objects.  This provides
 24semantics for importing and renaming.  Unlike  C++ and Modula-3, built-in types
 25can be used as base classes for extension by the user.  Also, like in C++ but
 26unlike in Modula-3, most built-in operators with special syntax (arithmetic
 27operators, subscripting etc.) can be redefined for class instances.
 28
 29
 30.. _tut-terminology:
 31
 32A Word About Terminology
 33========================
 34
 35Lacking universally accepted terminology to talk about classes, I will make
 36occasional use of Smalltalk and C++ terms.  (I would use Modula-3 terms, since
 37its object-oriented semantics are closer to those of Python than C++, but I
 38expect that few readers have heard of it.)
 39
 40Objects have individuality, and multiple names (in multiple scopes) can be bound
 41to the same object.  This is known as aliasing in other languages.  This is
 42usually not appreciated on a first glance at Python, and can be safely ignored
 43when dealing with immutable basic types (numbers, strings, tuples).  However,
 44aliasing has an (intended!) effect on the semantics of Python code involving
 45mutable objects such as lists, dictionaries, and most types representing
 46entities outside the program (files, windows, etc.).  This is usually used to
 47the benefit of the program, since aliases behave like pointers in some respects.
 48For example, passing an object is cheap since only a pointer is passed by the
 49implementation; and if a function modifies an object passed as an argument, the
 50caller will see the change --- this eliminates the need for two different
 51argument passing mechanisms as in Pascal.
 52
 53
 54.. _tut-scopes:
 55
 56Python Scopes and Name Spaces
 57=============================
 58
 59Before introducing classes, I first have to tell you something about Python's
 60scope rules.  Class definitions play some neat tricks with namespaces, and you
 61need to know how scopes and namespaces work to fully understand what's going on.
 62Incidentally, knowledge about this subject is useful for any advanced Python
 63programmer.
 64
 65Let's begin with some definitions.
 66
 67A *namespace* is a mapping from names to objects.  Most namespaces are currently
 68implemented as Python dictionaries, but that's normally not noticeable in any
 69way (except for performance), and it may change in the future.  Examples of
 70namespaces are: the set of built-in names (functions such as :func:`abs`, and
 71built-in exception names); the global names in a module; and the local names in
 72a function invocation.  In a sense the set of attributes of an object also form
 73a namespace.  The important thing to know about namespaces is that there is
 74absolutely no relation between names in different namespaces; for instance, two
 75different modules may both define a function "maximize" without confusion ---
 76users of the modules must prefix it with the module name.
 77
 78By the way, I use the word *attribute* for any name following a dot --- for
 79example, in the expression ``z.real``, ``real`` is an attribute of the object
 80``z``.  Strictly speaking, references to names in modules are attribute
 81references: in the expression ``modname.funcname``, ``modname`` is a module
 82object and ``funcname`` is an attribute of it.  In this case there happens to be
 83a straightforward mapping between the module's attributes and the global names
 84defined in the module: they share the same namespace!  [#]_
 85
 86Attributes may be read-only or writable.  In the latter case, assignment to
 87attributes is possible.  Module attributes are writable: you can write
 88``modname.the_answer = 42``.  Writable attributes may also be deleted with the
 89:keyword:`del` statement.  For example, ``del modname.the_answer`` will remove
 90the attribute :attr:`the_answer` from the object named by ``modname``.
 91
 92Name spaces are created at different moments and have different lifetimes.  The
 93namespace containing the built-in names is created when the Python interpreter
 94starts up, and is never deleted.  The global namespace for a module is created
 95when the module definition is read in; normally, module namespaces also last
 96until the interpreter quits.  The statements executed by the top-level
 97invocation of the interpreter, either read from a script file or interactively,
 98are considered part of a module called :mod:`__main__`, so they have their own
 99global namespace.  (The built-in names actually also live in a module; this is
100called :mod:`__builtin__`.)
101
102The local namespace for a function is created when the function is called, and
103deleted when the function returns or raises an exception that is not handled
104within the function.  (Actually, forgetting would be a better way to describe
105what actually happens.)  Of course, recursive invocations each have their own
106local namespace.
107
108A *scope* is a textual region of a Python program where a namespace is directly
109accessible.  "Directly accessible" here means that an unqualified reference to a
110name attempts to find the name in the namespace.
111
112Although scopes are determined statically, they are used dynamically. At any
113time during execution, there are at least three nested scopes whose namespaces
114are directly accessible: the innermost scope, which is searched first, contains
115the local names; the namespaces of any enclosing functions, which are searched
116starting with the nearest enclosing scope; the middle scope, searched next,
117contains the current module's global names; and the outermost scope (searched
118last) is the namespace containing built-in names.
119
120If a name is declared global, then all references and assignments go directly to
121the middle scope containing the module's global names. Otherwise, all variables
122found outside of the innermost scope are read-only (an attempt to write to such
123a variable will simply create a *new* local variable in the innermost scope,
124leaving the identically named outer variable unchanged).
125
126Usually, the local scope references the local names of the (textually) current
127function.  Outside functions, the local scope references the same namespace as
128the global scope: the module's namespace. Class definitions place yet another
129namespace in the local scope.
130
131It is important to realize that scopes are determined textually: the global
132scope of a function defined in a module is that module's namespace, no matter
133from where or by what alias the function is called.  On the other hand, the
134actual search for names is done dynamically, at run time --- however, the
135language definition is evolving towards static name resolution, at "compile"
136time, so don't rely on dynamic name resolution!  (In fact, local variables are
137already determined statically.)
138
139A special quirk of Python is that -- if no :keyword:`global`
140statement is in effect -- assignments to names always go
141into the innermost scope.  Assignments do not copy data --- they just bind names
142to objects.  The same is true for deletions: the statement ``del x`` removes the
143binding of ``x`` from the namespace referenced by the local scope.  In fact, all
144operations that introduce new names use the local scope: in particular, import
145statements and function definitions bind the module or function name in the
146local scope.  (The :keyword:`global` statement can be used to indicate that
147particular variables live in the global scope.)
148
149
150.. _tut-firstclasses:
151
152A First Look at Classes
153=======================
154
155Classes introduce a little bit of new syntax, three new object types, and some
156new semantics.
157
158
159.. _tut-classdefinition:
160
161Class Definition Syntax
162-----------------------
163
164The simplest form of class definition looks like this::
165
166   class ClassName:
167       <statement-1>
168       .
169       .
170       .
171       <statement-N>
172
173Class definitions, like function definitions (:keyword:`def` statements) must be
174executed before they have any effect.  (You could conceivably place a class
175definition in a branch of an :keyword:`if` statement, or inside a function.)
176
177In practice, the statements inside a class definition will usually be function
178definitions, but other statements are allowed, and sometimes useful --- we'll
179come back to this later.  The function definitions inside a class normally have
180a peculiar form of argument list, dictated by the calling conventions for
181methods --- again, this is explained later.
182
183When a class definition is entered, a new namespace is created, and used as the
184local scope --- thus, all assignments to local variables go into this new
185namespace.  In particular, function definitions bind the name of the new
186function here.
187
188When a class definition is left normally (via the end), a *class object* is
189created.  This is basically a wrapper around the contents of the namespace
190created by the class definition; we'll learn more about class objects in the
191next section.  The original local scope (the one in effect just before the class
192definition was entered) is reinstated, and the class object is bound here to the
193class name given in the class definition header (:class:`ClassName` in the
194example).
195
196
197.. _tut-classobjects:
198
199Class Objects
200-------------
201
202Class objects support two kinds of operations: attribute references and
203instantiation.
204
205*Attribute references* use the standard syntax used for all attribute references
206in Python: ``obj.name``.  Valid attribute names are all the names that were in
207the class's namespace when the class object was created.  So, if the class
208definition looked like this::
209
210   class MyClass:
211       """A simple example class"""
212       i = 12345
213       def f(self):
214           return 'hello world'
215
216then ``MyClass.i`` and ``MyClass.f`` are valid attribute references, returning
217an integer and a function object, respectively. Class attributes can also be
218assigned to, so you can change the value of ``MyClass.i`` by assignment.
219:attr:`__doc__` is also a valid attribute, returning the docstring belonging to
220the class: ``"A simple example class"``.
221
222Class *instantiation* uses function notation.  Just pretend that the class
223object is a parameterless function that returns a new instance of the class.
224For example (assuming the above class)::
225
226   x = MyClass()
227
228creates a new *instance* of the class and assigns this object to the local
229variable ``x``.
230
231The instantiation operation ("calling" a class object) creates an empty object.
232Many classes like to create objects with instances customized to a specific
233initial state. Therefore a class may define a special method named
234:meth:`__init__`, like this::
235
236   def __init__(self):
237       self.data = []
238
239When a class defines an :meth:`__init__` method, class instantiation
240automatically invokes :meth:`__init__` for the newly-created class instance.  So
241in this example, a new, initialized instance can be obtained by::
242
243   x = MyClass()
244
245Of course, the :meth:`__init__` method may have arguments for greater
246flexibility.  In that case, arguments given to the class instantiation operator
247are passed on to :meth:`__init__`.  For example, ::
248
249   >>> class Complex:
250   ...     def __init__(self, realpart, imagpart):
251   ...         self.r = realpart
252   ...         self.i = imagpart
253   ...
254   >>> x = Complex(3.0, -4.5)
255   >>> x.r, x.i
256   (3.0, -4.5)
257
258
259.. _tut-instanceobjects:
260
261Instance Objects
262----------------
263
264Now what can we do with instance objects?  The only operations understood by
265instance objects are attribute references.  There are two kinds of valid
266attribute names, data attributes and methods.
267
268*data attributes* correspond to "instance variables" in Smalltalk, and to "data
269members" in C++.  Data attributes need not be declared; like local variables,
270they spring into existence when they are first assigned to.  For example, if
271``x`` is the instance of :class:`MyClass` created above, the following piece of
272code will print the value ``16``, without leaving a trace::
273
274   x.counter = 1
275   while x.counter < 10:
276       x.counter = x.counter * 2
277   print x.counter
278   del x.counter
279
280The other kind of instance attribute reference is a *method*. A method is a
281function that "belongs to" an object.  (In Python, the term method is not unique
282to class instances: other object types can have methods as well.  For example,
283list objects have methods called append, insert, remove, sort, and so on.
284However, in the following discussion, we'll use the term method exclusively to
285mean methods of class instance objects, unless explicitly stated otherwise.)
286
287.. index:: object: method
288
289Valid method names of an instance object depend on its class.  By definition,
290all attributes of a class that are function  objects define corresponding
291methods of its instances.  So in our example, ``x.f`` is a valid method
292reference, since ``MyClass.f`` is a function, but ``x.i`` is not, since
293``MyClass.i`` is not.  But ``x.f`` is not the same thing as ``MyClass.f`` --- it
294is a *method object*, not a function object.
295
296
297.. _tut-methodobjects:
298
299Method Objects
300--------------
301
302Usually, a method is called right after it is bound::
303
304   x.f()
305
306In the :class:`MyClass` example, this will return the string ``'hello world'``.
307However, it is not necessary to call a method right away: ``x.f`` is a method
308object, and can be stored away and called at a later time.  For example::
309
310   xf = x.f
311   while True:
312       print xf()
313
314will continue to print ``hello world`` until the end of time.
315
316What exactly happens when a method is called?  You may have noticed that
317``x.f()`` was called without an argument above, even though the function
318definition for :meth:`f` specified an argument.  What happened to the argument?
319Surely Python raises an exception when a function that requires an argument is
320called without any --- even if the argument isn't actually used...
321
322Actually, you may have guessed the answer: the special thing about methods is
323that the object is passed as the first argument of the function.  In our
324example, the call ``x.f()`` is exactly equivalent to ``MyClass.f(x)``.  In
325general, calling a method with a list of *n* arguments is equivalent to calling
326the corresponding function with an argument list that is created by inserting
327the method's object before the first argument.
328
329If you still don't understand how methods work, a look at the implementation can
330perhaps clarify matters.  When an instance attribute is referenced that isn't a
331data attribute, its class is searched.  If the name denotes a valid class
332attribute that is a function object, a method object is created by packing
333(pointers to) the instance object and the function object just found together in
334an abstract object: this is the method object.  When the method object is called
335with an argument list, it is unpacked again, a new argument list is constructed
336from the instance object and the original argument list, and the function object
337is called with this new argument list.
338
339
340.. _tut-remarks:
341
342Random Remarks
343==============
344
345.. These should perhaps be placed more carefully...
346
347Data attributes override method attributes with the same name; to avoid
348accidental name conflicts, which may cause hard-to-find bugs in large programs,
349it is wise to use some kind of convention that minimizes the chance of
350conflicts.  Possible conventions include capitalizing method names, prefixing
351data attribute names with a small unique string (perhaps just an underscore), or
352using verbs for methods and nouns for data attributes.
353
354Data attributes may be referenced by methods as well as by ordinary users
355("clients") of an object.  In other words, classes are not usable to implement
356pure abstract data types.  In fact, nothing in Python makes it possible to
357enforce data hiding --- it is all based upon convention.  (On the other hand,
358the Python implementation, written in C, can completely hide implementation
359details and control access to an object if necessary; this can be used by
360extensions to Python written in C.)
361
362Clients should use data attributes with care --- clients may mess up invariants
363maintained by the methods by stamping on their data attributes.  Note that
364clients may add data attributes of their own to an instance object without
365affecting the validity of the methods, as long as name conflicts are avoided ---
366again, a naming convention can save a lot of headaches here.
367
368There is no shorthand for referencing data attributes (or other methods!) from
369within methods.  I find that this actually increases the readability of methods:
370there is no chance of confusing local variables and instance variables when
371glancing through a method.
372
373Often, the first argument of a method is called ``self``.  This is nothing more
374than a convention: the name ``self`` has absolutely no special meaning to
375Python.  (Note, however, that by not following the convention your code may be
376less readable to other Python programmers, and it is also conceivable that a
377*class browser* program might be written that relies upon such a convention.)
378
379Any function object that is a class attribute defines a method for instances of
380that class.  It is not necessary that the function definition is textually
381enclosed in the class definition: assigning a function object to a local
382variable in the class is also ok.  For example::
383
384   # Function defined outside the class
385   def f1(self, x, y):
386       return min(x, x+y)
387
388   class C:
389       f = f1
390       def g(self):
391           return 'hello world'
392       h = g
393
394Now ``f``, ``g`` and ``h`` are all attributes of class :class:`C` that refer to
395function objects, and consequently they are all methods of instances of
396:class:`C` --- ``h`` being exactly equivalent to ``g``.  Note that this practice
397usually only serves to confuse the reader of a program.
398
399Methods may call other methods by using method attributes of the ``self``
400argument::
401
402   class Bag:
403       def __init__(self):
404           self.data = []
405       def add(self, x):
406           self.data.append(x)
407       def addtwice(self, x):
408           self.add(x)
409           self.add(x)
410
411Methods may reference global names in the same way as ordinary functions.  The
412global scope associated with a method is the module containing the class
413definition.  (The class itself is never used as a global scope!)  While one
414rarely encounters a good reason for using global data in a method, there are
415many legitimate uses of the global scope: for one thing, functions and modules
416imported into the global scope can be used by methods, as well as functions and
417classes defined in it.  Usually, the class containing the method is itself
418defined in this global scope, and in the next section we'll find some good
419reasons why a method would want to reference its own class!
420
421Each value is an object, and therefore has a *class* (also called its *type*).
422It is stored as ``object.__class__``.
423
424
425.. _tut-inheritance:
426
427Inheritance
428===========
429
430Of course, a language feature would not be worthy of the name "class" without
431supporting inheritance.  The syntax for a derived class definition looks like
432this::
433
434   class DerivedClassName(BaseClassName):
435       <statement-1>
436       .
437       .
438       .
439       <statement-N>
440
441The name :class:`BaseClassName` must be defined in a scope containing the
442derived class definition.  In place of a base class name, other arbitrary
443expressions are also allowed.  This can be useful, for example, when the base
444class is defined in another module::
445
446   class DerivedClassName(modname.BaseClassName):
447
448Execution of a derived class definition proceeds the same as for a base class.
449When the class object is constructed, the base class is remembered.  This is
450used for resolving attribute references: if a requested attribute is not found
451in the class, the search proceeds to look in the base class.  This rule is
452applied recursively if the base class itself is derived from some other class.
453
454There's nothing special about instantiation of derived classes:
455``DerivedClassName()`` creates a new instance of the class.  Method references
456are resolved as follows: the corresponding class attribute is searched,
457descending down the chain of base classes if necessary, and the method reference
458is valid if this yields a function object.
459
460Derived classes may override methods of their base classes.  Because methods
461have no special privileges when calling other methods of the same object, a
462method of a base class that calls another method defined in the same base class
463may end up calling a method of a derived class that overrides it.  (For C++
464programmers: all methods in Python are effectively ``virtual``.)
465
466An overriding method in a derived class may in fact want to extend rather than
467simply replace the base class method of the same name. There is a simple way to
468call the base class method directly: just call ``BaseClassName.methodname(self,
469arguments)``.  This is occasionally useful to clients as well.  (Note that this
470only works if the base class is defined or imported directly in the global
471scope.)
472
473Python has two built-in functions that work with inheritance:
474
475* Use :func:`isinstance` to check an object's type: ``isinstance(obj, int)``
476  will be ``True`` only if ``obj.__class__`` is :class:`int` or some class
477  derived from :class:`int`.
478
479* Use :func:`issubclass` to check class inheritance: ``issubclass(bool, int)``
480  is ``True`` since :class:`bool` is a subclass of :class:`int`.  However,
481  ``issubclass(unicode, str)`` is ``False`` since :class:`unicode` is not a
482  subclass of :class:`str` (they only share a common ancestor,
483  :class:`basestring`).
484
485
486
487.. _tut-multiple:
488
489Multiple Inheritance
490--------------------
491
492Python supports a limited form of multiple inheritance as well.  A class
493definition with multiple base classes looks like this::
494
495   class DerivedClassName(Base1, Base2, Base3):
496       <statement-1>
497       .
498       .
499       .
500       <statement-N>
501
502For old-style classes, the only rule is depth-first, left-to-right.  Thus, if an
503attribute is not found in :class:`DerivedClassName`, it is searched in
504:class:`Base1`, then (recursively) in the base classes of :class:`Base1`, and
505only if it is not found there, it is searched in :class:`Base2`, and so on.
506
507(To some people breadth first --- searching :class:`Base2` and :class:`Base3`
508before the base classes of :class:`Base1` --- looks more natural.  However, this
509would require you to know whether a particular attribute of :class:`Base1` is
510actually defined in :class:`Base1` or in one of its base classes before you can
511figure out the consequences of a name conflict with an attribute of
512:class:`Base2`.  The depth-first rule makes no differences between direct and
513inherited attributes of :class:`Base1`.)
514
515For :term:`new-style class`\es, the method resolution order changes dynamically
516to support cooperative calls to :func:`super`.  This approach is known in some
517other multiple-inheritance languages as call-next-method and is more powerful
518than the super call found in single-inheritance languages.
519
520With new-style classes, dynamic ordering is necessary because all  cases of
521multiple inheritance exhibit one or more diamond relationships (where one at
522least one of the parent classes can be accessed through multiple paths from the
523bottommost class).  For example, all new-style classes inherit from
524:class:`object`, so any case of multiple inheritance provides more than one path
525to reach :class:`object`.  To keep the base classes from being accessed more
526than once, the dynamic algorithm linearizes the search order in a way that
527preserves the left-to-right ordering specified in each class, that calls each
528parent only once, and that is monotonic (meaning that a class can be subclassed
529without affecting the precedence order of its parents).  Taken together, these
530properties make it possible to design reliable and extensible classes with
531multiple inheritance.  For more detail, see
532http://www.python.org/download/releases/2.3/mro/.
533
534
535.. _tut-private:
536
537Private Variables
538=================
539
540There is limited support for class-private identifiers.  Any identifier of the
541form ``__spam`` (at least two leading underscores, at most one trailing
542underscore) is textually replaced with ``_classname__spam``, where ``classname``
543is the current class name with leading underscore(s) stripped.  This mangling is
544done without regard to the syntactic position of the identifier, so it can be
545used to define class-private instance and class variables, methods, variables
546stored in globals, and even variables stored in instances. private to this class
547on instances of *other* classes.  Truncation may occur when the mangled name
548would be longer than 255 characters. Outside classes, or when the class name
549consists of only underscores, no mangling occurs.
550
551Name mangling is intended to give classes an easy way to define "private"
552instance variables and methods, without having to worry about instance variables
553defined by derived classes, or mucking with instance variables by code outside
554the class.  Note that the mangling rules are designed mostly to avoid accidents;
555it still is possible for a determined soul to access or modify a variable that
556is considered private.  This can even be useful in special circumstances, such
557as in the debugger, and that's one reason why this loophole is not closed.
558(Buglet: derivation of a class with the same name as the base class makes use of
559private variables of the base class possible.)
560
561Notice that code passed to ``exec``, ``eval()`` or ``execfile()`` does not
562consider the classname of the invoking  class to be the current class; this is
563similar to the effect of the  ``global`` statement, the effect of which is
564likewise restricted to  code that is byte-compiled together.  The same
565restriction applies to ``getattr()``, ``setattr()`` and ``delattr()``, as well
566as when referencing ``__dict__`` directly.
567
568
569.. _tut-odds:
570
571Odds and Ends
572=============
573
574Sometimes it is useful to have a data type similar to the Pascal "record" or C
575"struct", bundling together a few named data items.  An empty class definition
576will do nicely::
577
578   class Employee:
579       pass
580
581   john = Employee() # Create an empty employee record
582
583   # Fill the fields of the record
584   john.name = 'John Doe'
585   john.dept = 'computer lab'
586   john.salary = 1000
587
588A piece of Python code that expects a particular abstract data type can often be
589passed a class that emulates the methods of that data type instead.  For
590instance, if you have a function that formats some data from a file object, you
591can define a class with methods :meth:`read` and :meth:`readline` that get the
592data from a string buffer instead, and pass it as an argument.
593
594.. (Unfortunately, this technique has its limitations: a class can't define
595   operations that are accessed by special syntax such as sequence subscripting
596   or arithmetic operators, and assigning such a "pseudo-file" to sys.stdin will
597   not cause the interpreter to read further input from it.)
598
599Instance method objects have attributes, too: ``m.im_self`` is the instance
600object with the method :meth:`m`, and ``m.im_func`` is the function object
601corresponding to the method.
602
603
604.. _tut-exceptionclasses:
605
606Exceptions Are Classes Too
607==========================
608
609User-defined exceptions are identified by classes as well.  Using this mechanism
610it is possible to create extensible hierarchies of exceptions.
611
612There are two new valid (semantic) forms for the raise statement::
613
614   raise Class, instance
615
616   raise instance
617
618In the first form, ``instance`` must be an instance of :class:`Class` or of a
619class derived from it.  The second form is a shorthand for::
620
621   raise instance.__class__, instance
622
623A class in an except clause is compatible with an exception if it is the same
624class or a base class thereof (but not the other way around --- an except clause
625listing a derived class is not compatible with a base class).  For example, the
626following code will print B, C, D in that order::
627
628   class B:
629       pass
630   class C(B):
631       pass
632   class D(C):
633       pass
634
635   for c in [B, C, D]:
636       try:
637           raise c()
638       except D:
639           print "D"
640       except C:
641           print "C"
642       except B:
643           print "B"
644
645Note that if the except clauses were reversed (with ``except B`` first), it
646would have printed B, B, B --- the first matching except clause is triggered.
647
648When an error message is printed for an unhandled exception, the exception's
649class name is printed, then a colon and a space, and finally the instance
650converted to a string using the built-in function :func:`str`.
651
652
653.. _tut-iterators:
654
655Iterators
656=========
657
658By now you have probably noticed that most container objects can be looped over
659using a :keyword:`for` statement::
660
661   for element in [1, 2, 3]:
662       print element
663   for element in (1, 2, 3):
664       print element
665   for key in {'one':1, 'two':2}:
666       print key
667   for char in "123":
668       print char
669   for line in open("myfile.txt"):
670       print line
671
672This style of access is clear, concise, and convenient.  The use of iterators
673pervades and unifies Python.  Behind the scenes, the :keyword:`for` statement
674calls :func:`iter` on the container object.  The function returns an iterator
675object that defines the method :meth:`next` which accesses elements in the
676container one at a time.  When there are no more elements, :meth:`next` raises a
677:exc:`StopIteration` exception which tells the :keyword:`for` loop to terminate.
678This example shows how it all works::
679
680   >>> s = 'abc'
681   >>> it = iter(s)
682   >>> it
683   <iterator object at 0x00A1DB50>
684   >>> it.next()
685   'a'
686   >>> it.next()
687   'b'
688   >>> it.next()
689   'c'
690   >>> it.next()
691
692   Traceback (most recent call last):
693     File "<stdin>", line 1, in ?
694       it.next()
695   StopIteration
696
697Having seen the mechanics behind the iterator protocol, it is easy to add
698iterator behavior to your classes.  Define a :meth:`__iter__` method which
699returns an object with a :meth:`next` method.  If the class defines
700:meth:`next`, then :meth:`__iter__` can just return ``self``::
701
702   class Reverse:
703       "Iterator for looping over a sequence backwards"
704       def __init__(self, data):
705           self.data = data
706           self.index = len(data)
707       def __iter__(self):
708           return self
709       def next(self):
710           if self.index == 0:
711               raise StopIteration
712           self.index = self.index - 1
713           return self.data[self.index]
714
715   >>> for char in Reverse('spam'):
716   ...     print char
717   ...
718   m
719   a
720   p
721   s
722
723
724.. _tut-generators:
725
726Generators
727==========
728
729:term:`Generator`\s are a simple and powerful tool for creating iterators.  They
730are written like regular functions but use the :keyword:`yield` statement
731whenever they want to return data.  Each time :meth:`next` is called, the
732generator resumes where it left-off (it remembers all the data values and which
733statement was last executed).  An example shows that generators can be trivially
734easy to create::
735
736   def reverse(data):
737       for index in range(len(data)-1, -1, -1):
738           yield data[index]
739
740   >>> for char in reverse('golf'):
741   ...     print char
742   ...
743   f
744   l
745   o
746   g
747
748Anything that can be done with generators can also be done with class based
749iterators as described in the previous section.  What makes generators so
750compact is that the :meth:`__iter__` and :meth:`next` methods are created
751automatically.
752
753Another key feature is that the local variables and execution state are
754automatically saved between calls.  This made the function easier to write and
755much more clear than an approach using instance variables like ``self.index``
756and ``self.data``.
757
758In addition to automatic method creation and saving program state, when
759generators terminate, they automatically raise :exc:`StopIteration`. In
760combination, these features make it easy to create iterators with no more effort
761than writing a regular function.
762
763
764.. _tut-genexps:
765
766Generator Expressions
767=====================
768
769Some simple generators can be coded succinctly as expressions using a syntax
770similar to list comprehensions but with parentheses instead of brackets.  These
771expressions are designed for situations where the generator is used right away
772by an enclosing function.  Generator expressions are more compact but less
773versatile than full generator definitions and tend to be more memory friendly
774than equivalent list comprehensions.
775
776Examples::
777
778   >>> sum(i*i for i in range(10))                 # sum of squares
779   285
780
781   >>> xvec = [10, 20, 30]
782   >>> yvec = [7, 5, 3]
783   >>> sum(x*y for x,y in zip(xvec, yvec))         # dot product
784   260
785
786   >>> from math import pi, sin
787   >>> sine_table = dict((x, sin(x*pi/180)) for x in range(0, 91))
788
789   >>> unique_words = set(word  for line in page  for word in line.split())
790
791   >>> valedictorian = max((student.gpa, student.name) for student in graduates)
792
793   >>> data = 'golf'
794   >>> list(data[i] for i in range(len(data)-1,-1,-1))
795   ['f', 'l', 'o', 'g']
796
797
798
799.. rubric:: Footnotes
800
801.. [#] Except for one thing.  Module objects have a secret read-only attribute called
802   :attr:`__dict__` which returns the dictionary used to implement the module's
803   namespace; the name :attr:`__dict__` is an attribute but not a global name.
804   Obviously, using this violates the abstraction of namespace implementation, and
805   should be restricted to things like post-mortem debuggers.
806