/Doc/tutorial/modules.rst

http://unladen-swallow.googlecode.com/ · ReStructuredText · 550 lines · 422 code · 128 blank · 0 comment · 0 complexity · f6d8660295f0e8bae32bfce246658b27 MD5 · raw file

  1. .. _tut-modules:
  2. *******
  3. Modules
  4. *******
  5. If you quit from the Python interpreter and enter it again, the definitions you
  6. have made (functions and variables) are lost. Therefore, if you want to write a
  7. somewhat longer program, you are better off using a text editor to prepare the
  8. input for the interpreter and running it with that file as input instead. This
  9. is known as creating a *script*. As your program gets longer, you may want to
  10. split it into several files for easier maintenance. You may also want to use a
  11. handy function that you've written in several programs without copying its
  12. definition into each program.
  13. To support this, Python has a way to put definitions in a file and use them in a
  14. script or in an interactive instance of the interpreter. Such a file is called a
  15. *module*; definitions from a module can be *imported* into other modules or into
  16. the *main* module (the collection of variables that you have access to in a
  17. script executed at the top level and in calculator mode).
  18. A module is a file containing Python definitions and statements. The file name
  19. is the module name with the suffix :file:`.py` appended. Within a module, the
  20. module's name (as a string) is available as the value of the global variable
  21. ``__name__``. For instance, use your favorite text editor to create a file
  22. called :file:`fibo.py` in the current directory with the following contents::
  23. # Fibonacci numbers module
  24. def fib(n): # write Fibonacci series up to n
  25. a, b = 0, 1
  26. while b < n:
  27. print b,
  28. a, b = b, a+b
  29. def fib2(n): # return Fibonacci series up to n
  30. result = []
  31. a, b = 0, 1
  32. while b < n:
  33. result.append(b)
  34. a, b = b, a+b
  35. return result
  36. Now enter the Python interpreter and import this module with the following
  37. command::
  38. >>> import fibo
  39. This does not enter the names of the functions defined in ``fibo`` directly in
  40. the current symbol table; it only enters the module name ``fibo`` there. Using
  41. the module name you can access the functions::
  42. >>> fibo.fib(1000)
  43. 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987
  44. >>> fibo.fib2(100)
  45. [1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]
  46. >>> fibo.__name__
  47. 'fibo'
  48. If you intend to use a function often you can assign it to a local name::
  49. >>> fib = fibo.fib
  50. >>> fib(500)
  51. 1 1 2 3 5 8 13 21 34 55 89 144 233 377
  52. .. _tut-moremodules:
  53. More on Modules
  54. ===============
  55. A module can contain executable statements as well as function definitions.
  56. These statements are intended to initialize the module. They are executed only
  57. the *first* time the module is imported somewhere. [#]_
  58. Each module has its own private symbol table, which is used as the global symbol
  59. table by all functions defined in the module. Thus, the author of a module can
  60. use global variables in the module without worrying about accidental clashes
  61. with a user's global variables. On the other hand, if you know what you are
  62. doing you can touch a module's global variables with the same notation used to
  63. refer to its functions, ``modname.itemname``.
  64. Modules can import other modules. It is customary but not required to place all
  65. :keyword:`import` statements at the beginning of a module (or script, for that
  66. matter). The imported module names are placed in the importing module's global
  67. symbol table.
  68. There is a variant of the :keyword:`import` statement that imports names from a
  69. module directly into the importing module's symbol table. For example::
  70. >>> from fibo import fib, fib2
  71. >>> fib(500)
  72. 1 1 2 3 5 8 13 21 34 55 89 144 233 377
  73. This does not introduce the module name from which the imports are taken in the
  74. local symbol table (so in the example, ``fibo`` is not defined).
  75. There is even a variant to import all names that a module defines::
  76. >>> from fibo import *
  77. >>> fib(500)
  78. 1 1 2 3 5 8 13 21 34 55 89 144 233 377
  79. This imports all names except those beginning with an underscore (``_``).
  80. .. note::
  81. For efficiency reasons, each module is only imported once per interpreter
  82. session. Therefore, if you change your modules, you must restart the
  83. interpreter -- or, if it's just one module you want to test interactively,
  84. use :func:`reload`, e.g. ``reload(modulename)``.
  85. .. _tut-modulesasscripts:
  86. Executing modules as scripts
  87. ----------------------------
  88. When you run a Python module with ::
  89. python fibo.py <arguments>
  90. the code in the module will be executed, just as if you imported it, but with
  91. the ``__name__`` set to ``"__main__"``. That means that by adding this code at
  92. the end of your module::
  93. if __name__ == "__main__":
  94. import sys
  95. fib(int(sys.argv[1]))
  96. you can make the file usable as a script as well as an importable module,
  97. because the code that parses the command line only runs if the module is
  98. executed as the "main" file::
  99. $ python fibo.py 50
  100. 1 1 2 3 5 8 13 21 34
  101. If the module is imported, the code is not run::
  102. >>> import fibo
  103. >>>
  104. This is often used either to provide a convenient user interface to a module, or
  105. for testing purposes (running the module as a script executes a test suite).
  106. .. _tut-searchpath:
  107. The Module Search Path
  108. ----------------------
  109. .. index:: triple: module; search; path
  110. When a module named :mod:`spam` is imported, the interpreter searches for a file
  111. named :file:`spam.py` in the current directory, and then in the list of
  112. directories specified by the environment variable :envvar:`PYTHONPATH`. This
  113. has the same syntax as the shell variable :envvar:`PATH`, that is, a list of
  114. directory names. When :envvar:`PYTHONPATH` is not set, or when the file is not
  115. found there, the search continues in an installation-dependent default path; on
  116. Unix, this is usually :file:`.:/usr/local/lib/python`.
  117. Actually, modules are searched in the list of directories given by the variable
  118. ``sys.path`` which is initialized from the directory containing the input script
  119. (or the current directory), :envvar:`PYTHONPATH` and the installation- dependent
  120. default. This allows Python programs that know what they're doing to modify or
  121. replace the module search path. Note that because the directory containing the
  122. script being run is on the search path, it is important that the script not have
  123. the same name as a standard module, or Python will attempt to load the script as
  124. a module when that module is imported. This will generally be an error. See
  125. section :ref:`tut-standardmodules` for more information.
  126. "Compiled" Python files
  127. -----------------------
  128. As an important speed-up of the start-up time for short programs that use a lot
  129. of standard modules, if a file called :file:`spam.pyc` exists in the directory
  130. where :file:`spam.py` is found, this is assumed to contain an
  131. already-"byte-compiled" version of the module :mod:`spam`. The modification time
  132. of the version of :file:`spam.py` used to create :file:`spam.pyc` is recorded in
  133. :file:`spam.pyc`, and the :file:`.pyc` file is ignored if these don't match.
  134. Normally, you don't need to do anything to create the :file:`spam.pyc` file.
  135. Whenever :file:`spam.py` is successfully compiled, an attempt is made to write
  136. the compiled version to :file:`spam.pyc`. It is not an error if this attempt
  137. fails; if for any reason the file is not written completely, the resulting
  138. :file:`spam.pyc` file will be recognized as invalid and thus ignored later. The
  139. contents of the :file:`spam.pyc` file are platform independent, so a Python
  140. module directory can be shared by machines of different architectures.
  141. Some tips for experts:
  142. * When the Python interpreter is invoked with the :option:`-O` flag, optimized
  143. code is generated and stored in :file:`.pyo` files. The optimizer currently
  144. doesn't help much; it only removes :keyword:`assert` statements. When
  145. :option:`-O` is used, *all* :term:`bytecode` is optimized; ``.pyc`` files are
  146. ignored and ``.py`` files are compiled to optimized bytecode.
  147. * Passing two :option:`-O` flags to the Python interpreter (:option:`-OO`) will
  148. cause the bytecode compiler to perform optimizations that could in some rare
  149. cases result in malfunctioning programs. Currently only ``__doc__`` strings are
  150. removed from the bytecode, resulting in more compact :file:`.pyo` files. Since
  151. some programs may rely on having these available, you should only use this
  152. option if you know what you're doing.
  153. * A program doesn't run any faster when it is read from a :file:`.pyc` or
  154. :file:`.pyo` file than when it is read from a :file:`.py` file; the only thing
  155. that's faster about :file:`.pyc` or :file:`.pyo` files is the speed with which
  156. they are loaded.
  157. * When a script is run by giving its name on the command line, the bytecode for
  158. the script is never written to a :file:`.pyc` or :file:`.pyo` file. Thus, the
  159. startup time of a script may be reduced by moving most of its code to a module
  160. and having a small bootstrap script that imports that module. It is also
  161. possible to name a :file:`.pyc` or :file:`.pyo` file directly on the command
  162. line.
  163. * It is possible to have a file called :file:`spam.pyc` (or :file:`spam.pyo`
  164. when :option:`-O` is used) without a file :file:`spam.py` for the same module.
  165. This can be used to distribute a library of Python code in a form that is
  166. moderately hard to reverse engineer.
  167. .. index:: module: compileall
  168. * The module :mod:`compileall` can create :file:`.pyc` files (or :file:`.pyo`
  169. files when :option:`-O` is used) for all modules in a directory.
  170. .. _tut-standardmodules:
  171. Standard Modules
  172. ================
  173. .. index:: module: sys
  174. Python comes with a library of standard modules, described in a separate
  175. document, the Python Library Reference ("Library Reference" hereafter). Some
  176. modules are built into the interpreter; these provide access to operations that
  177. are not part of the core of the language but are nevertheless built in, either
  178. for efficiency or to provide access to operating system primitives such as
  179. system calls. The set of such modules is a configuration option which also
  180. depends on the underlying platform For example, the :mod:`winreg` module is only
  181. provided on Windows systems. One particular module deserves some attention:
  182. :mod:`sys`, which is built into every Python interpreter. The variables
  183. ``sys.ps1`` and ``sys.ps2`` define the strings used as primary and secondary
  184. prompts::
  185. >>> import sys
  186. >>> sys.ps1
  187. '>>> '
  188. >>> sys.ps2
  189. '... '
  190. >>> sys.ps1 = 'C> '
  191. C> print 'Yuck!'
  192. Yuck!
  193. C>
  194. These two variables are only defined if the interpreter is in interactive mode.
  195. The variable ``sys.path`` is a list of strings that determines the interpreter's
  196. search path for modules. It is initialized to a default path taken from the
  197. environment variable :envvar:`PYTHONPATH`, or from a built-in default if
  198. :envvar:`PYTHONPATH` is not set. You can modify it using standard list
  199. operations::
  200. >>> import sys
  201. >>> sys.path.append('/ufs/guido/lib/python')
  202. .. _tut-dir:
  203. The :func:`dir` Function
  204. ========================
  205. The built-in function :func:`dir` is used to find out which names a module
  206. defines. It returns a sorted list of strings::
  207. >>> import fibo, sys
  208. >>> dir(fibo)
  209. ['__name__', 'fib', 'fib2']
  210. >>> dir(sys)
  211. ['__displayhook__', '__doc__', '__excepthook__', '__name__', '__stderr__',
  212. '__stdin__', '__stdout__', '_getframe', 'api_version', 'argv',
  213. 'builtin_module_names', 'byteorder', 'callstats', 'copyright',
  214. 'displayhook', 'exc_clear', 'exc_info', 'exc_type', 'excepthook',
  215. 'exec_prefix', 'executable', 'exit', 'getdefaultencoding', 'getdlopenflags',
  216. 'getrecursionlimit', 'getrefcount', 'hexversion', 'maxint', 'maxunicode',
  217. 'meta_path', 'modules', 'path', 'path_hooks', 'path_importer_cache',
  218. 'platform', 'prefix', 'ps1', 'ps2', 'setcheckinterval', 'setdlopenflags',
  219. 'setprofile', 'setrecursionlimit', 'settrace', 'stderr', 'stdin', 'stdout',
  220. 'version', 'version_info', 'warnoptions']
  221. Without arguments, :func:`dir` lists the names you have defined currently::
  222. >>> a = [1, 2, 3, 4, 5]
  223. >>> import fibo
  224. >>> fib = fibo.fib
  225. >>> dir()
  226. ['__builtins__', '__doc__', '__file__', '__name__', 'a', 'fib', 'fibo', 'sys']
  227. Note that it lists all types of names: variables, modules, functions, etc.
  228. .. index:: module: __builtin__
  229. :func:`dir` does not list the names of built-in functions and variables. If you
  230. want a list of those, they are defined in the standard module
  231. :mod:`__builtin__`::
  232. >>> import __builtin__
  233. >>> dir(__builtin__)
  234. ['ArithmeticError', 'AssertionError', 'AttributeError', 'DeprecationWarning',
  235. 'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 'False',
  236. 'FloatingPointError', 'FutureWarning', 'IOError', 'ImportError',
  237. 'IndentationError', 'IndexError', 'KeyError', 'KeyboardInterrupt',
  238. 'LookupError', 'MemoryError', 'NameError', 'None', 'NotImplemented',
  239. 'NotImplementedError', 'OSError', 'OverflowError',
  240. 'PendingDeprecationWarning', 'ReferenceError', 'RuntimeError',
  241. 'RuntimeWarning', 'StandardError', 'StopIteration', 'SyntaxError',
  242. 'SyntaxWarning', 'SystemError', 'SystemExit', 'TabError', 'True',
  243. 'TypeError', 'UnboundLocalError', 'UnicodeDecodeError',
  244. 'UnicodeEncodeError', 'UnicodeError', 'UnicodeTranslateError',
  245. 'UserWarning', 'ValueError', 'Warning', 'WindowsError',
  246. 'ZeroDivisionError', '_', '__debug__', '__doc__', '__import__',
  247. '__name__', 'abs', 'apply', 'basestring', 'bool', 'buffer',
  248. 'callable', 'chr', 'classmethod', 'cmp', 'coerce', 'compile',
  249. 'complex', 'copyright', 'credits', 'delattr', 'dict', 'dir', 'divmod',
  250. 'enumerate', 'eval', 'execfile', 'exit', 'file', 'filter', 'float',
  251. 'frozenset', 'getattr', 'globals', 'hasattr', 'hash', 'help', 'hex',
  252. 'id', 'input', 'int', 'intern', 'isinstance', 'issubclass', 'iter',
  253. 'len', 'license', 'list', 'locals', 'long', 'map', 'max', 'min',
  254. 'object', 'oct', 'open', 'ord', 'pow', 'property', 'quit', 'range',
  255. 'raw_input', 'reduce', 'reload', 'repr', 'reversed', 'round', 'set',
  256. 'setattr', 'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super',
  257. 'tuple', 'type', 'unichr', 'unicode', 'vars', 'xrange', 'zip']
  258. .. _tut-packages:
  259. Packages
  260. ========
  261. Packages are a way of structuring Python's module namespace by using "dotted
  262. module names". For example, the module name :mod:`A.B` designates a submodule
  263. named ``B`` in a package named ``A``. Just like the use of modules saves the
  264. authors of different modules from having to worry about each other's global
  265. variable names, the use of dotted module names saves the authors of multi-module
  266. packages like NumPy or the Python Imaging Library from having to worry about
  267. each other's module names.
  268. Suppose you want to design a collection of modules (a "package") for the uniform
  269. handling of sound files and sound data. There are many different sound file
  270. formats (usually recognized by their extension, for example: :file:`.wav`,
  271. :file:`.aiff`, :file:`.au`), so you may need to create and maintain a growing
  272. collection of modules for the conversion between the various file formats.
  273. There are also many different operations you might want to perform on sound data
  274. (such as mixing, adding echo, applying an equalizer function, creating an
  275. artificial stereo effect), so in addition you will be writing a never-ending
  276. stream of modules to perform these operations. Here's a possible structure for
  277. your package (expressed in terms of a hierarchical filesystem)::
  278. sound/ Top-level package
  279. __init__.py Initialize the sound package
  280. formats/ Subpackage for file format conversions
  281. __init__.py
  282. wavread.py
  283. wavwrite.py
  284. aiffread.py
  285. aiffwrite.py
  286. auread.py
  287. auwrite.py
  288. ...
  289. effects/ Subpackage for sound effects
  290. __init__.py
  291. echo.py
  292. surround.py
  293. reverse.py
  294. ...
  295. filters/ Subpackage for filters
  296. __init__.py
  297. equalizer.py
  298. vocoder.py
  299. karaoke.py
  300. ...
  301. When importing the package, Python searches through the directories on
  302. ``sys.path`` looking for the package subdirectory.
  303. The :file:`__init__.py` files are required to make Python treat the directories
  304. as containing packages; this is done to prevent directories with a common name,
  305. such as ``string``, from unintentionally hiding valid modules that occur later
  306. on the module search path. In the simplest case, :file:`__init__.py` can just be
  307. an empty file, but it can also execute initialization code for the package or
  308. set the ``__all__`` variable, described later.
  309. Users of the package can import individual modules from the package, for
  310. example::
  311. import sound.effects.echo
  312. This loads the submodule :mod:`sound.effects.echo`. It must be referenced with
  313. its full name. ::
  314. sound.effects.echo.echofilter(input, output, delay=0.7, atten=4)
  315. An alternative way of importing the submodule is::
  316. from sound.effects import echo
  317. This also loads the submodule :mod:`echo`, and makes it available without its
  318. package prefix, so it can be used as follows::
  319. echo.echofilter(input, output, delay=0.7, atten=4)
  320. Yet another variation is to import the desired function or variable directly::
  321. from sound.effects.echo import echofilter
  322. Again, this loads the submodule :mod:`echo`, but this makes its function
  323. :func:`echofilter` directly available::
  324. echofilter(input, output, delay=0.7, atten=4)
  325. Note that when using ``from package import item``, the item can be either a
  326. submodule (or subpackage) of the package, or some other name defined in the
  327. package, like a function, class or variable. The ``import`` statement first
  328. tests whether the item is defined in the package; if not, it assumes it is a
  329. module and attempts to load it. If it fails to find it, an :exc:`ImportError`
  330. exception is raised.
  331. Contrarily, when using syntax like ``import item.subitem.subsubitem``, each item
  332. except for the last must be a package; the last item can be a module or a
  333. package but can't be a class or function or variable defined in the previous
  334. item.
  335. .. _tut-pkg-import-star:
  336. Importing \* From a Package
  337. ---------------------------
  338. .. index:: single: __all__
  339. Now what happens when the user writes ``from sound.effects import *``? Ideally,
  340. one would hope that this somehow goes out to the filesystem, finds which
  341. submodules are present in the package, and imports them all. Unfortunately,
  342. this operation does not work very well on Windows platforms, where the
  343. filesystem does not always have accurate information about the case of a
  344. filename! On these platforms, there is no guaranteed way to know whether a file
  345. :file:`ECHO.PY` should be imported as a module :mod:`echo`, :mod:`Echo` or
  346. :mod:`ECHO`. (For example, Windows 95 has the annoying practice of showing all
  347. file names with a capitalized first letter.) The DOS 8+3 filename restriction
  348. adds another interesting problem for long module names.
  349. The only solution is for the package author to provide an explicit index of the
  350. package. The import statement uses the following convention: if a package's
  351. :file:`__init__.py` code defines a list named ``__all__``, it is taken to be the
  352. list of module names that should be imported when ``from package import *`` is
  353. encountered. It is up to the package author to keep this list up-to-date when a
  354. new version of the package is released. Package authors may also decide not to
  355. support it, if they don't see a use for importing \* from their package. For
  356. example, the file :file:`sounds/effects/__init__.py` could contain the following
  357. code::
  358. __all__ = ["echo", "surround", "reverse"]
  359. This would mean that ``from sound.effects import *`` would import the three
  360. named submodules of the :mod:`sound` package.
  361. If ``__all__`` is not defined, the statement ``from sound.effects import *``
  362. does *not* import all submodules from the package :mod:`sound.effects` into the
  363. current namespace; it only ensures that the package :mod:`sound.effects` has
  364. been imported (possibly running any initialization code in :file:`__init__.py`)
  365. and then imports whatever names are defined in the package. This includes any
  366. names defined (and submodules explicitly loaded) by :file:`__init__.py`. It
  367. also includes any submodules of the package that were explicitly loaded by
  368. previous import statements. Consider this code::
  369. import sound.effects.echo
  370. import sound.effects.surround
  371. from sound.effects import *
  372. In this example, the echo and surround modules are imported in the current
  373. namespace because they are defined in the :mod:`sound.effects` package when the
  374. ``from...import`` statement is executed. (This also works when ``__all__`` is
  375. defined.)
  376. Note that in general the practice of importing ``*`` from a module or package is
  377. frowned upon, since it often causes poorly readable code. However, it is okay to
  378. use it to save typing in interactive sessions, and certain modules are designed
  379. to export only names that follow certain patterns.
  380. Remember, there is nothing wrong with using ``from Package import
  381. specific_submodule``! In fact, this is the recommended notation unless the
  382. importing module needs to use submodules with the same name from different
  383. packages.
  384. Intra-package References
  385. ------------------------
  386. The submodules often need to refer to each other. For example, the
  387. :mod:`surround` module might use the :mod:`echo` module. In fact, such
  388. references are so common that the :keyword:`import` statement first looks in the
  389. containing package before looking in the standard module search path. Thus, the
  390. :mod:`surround` module can simply use ``import echo`` or ``from echo import
  391. echofilter``. If the imported module is not found in the current package (the
  392. package of which the current module is a submodule), the :keyword:`import`
  393. statement looks for a top-level module with the given name.
  394. When packages are structured into subpackages (as with the :mod:`sound` package
  395. in the example), you can use absolute imports to refer to submodules of siblings
  396. packages. For example, if the module :mod:`sound.filters.vocoder` needs to use
  397. the :mod:`echo` module in the :mod:`sound.effects` package, it can use ``from
  398. sound.effects import echo``.
  399. Starting with Python 2.5, in addition to the implicit relative imports described
  400. above, you can write explicit relative imports with the ``from module import
  401. name`` form of import statement. These explicit relative imports use leading
  402. dots to indicate the current and parent packages involved in the relative
  403. import. From the :mod:`surround` module for example, you might use::
  404. from . import echo
  405. from .. import formats
  406. from ..filters import equalizer
  407. Note that both explicit and implicit relative imports are based on the name of
  408. the current module. Since the name of the main module is always ``"__main__"``,
  409. modules intended for use as the main module of a Python application should
  410. always use absolute imports.
  411. Packages in Multiple Directories
  412. --------------------------------
  413. Packages support one more special attribute, :attr:`__path__`. This is
  414. initialized to be a list containing the name of the directory holding the
  415. package's :file:`__init__.py` before the code in that file is executed. This
  416. variable can be modified; doing so affects future searches for modules and
  417. subpackages contained in the package.
  418. While this feature is not often needed, it can be used to extend the set of
  419. modules found in a package.
  420. .. rubric:: Footnotes
  421. .. [#] In fact function definitions are also 'statements' that are 'executed'; the
  422. execution enters the function name in the module's global symbol table.