/Doc/install/index.rst

http://unladen-swallow.googlecode.com/ · ReStructuredText · 997 lines · 742 code · 255 blank · 0 comment · 0 complexity · b02566c60416cf08796bd7f511f19b50 MD5 · raw file

  1. .. highlightlang:: none
  2. .. _install-index:
  3. *****************************
  4. Installing Python Modules
  5. *****************************
  6. :Author: Greg Ward
  7. :Release: |version|
  8. :Date: |today|
  9. .. TODO: Fill in XXX comments
  10. .. The audience for this document includes people who don't know anything
  11. about Python and aren't about to learn the language just in order to
  12. install and maintain it for their users, i.e. system administrators.
  13. Thus, I have to be sure to explain the basics at some point:
  14. sys.path and PYTHONPATH at least. Should probably give pointers to
  15. other docs on "import site", PYTHONSTARTUP, PYTHONHOME, etc.
  16. Finally, it might be useful to include all the material from my "Care
  17. and Feeding of a Python Installation" talk in here somewhere. Yow!
  18. .. topic:: Abstract
  19. This document describes the Python Distribution Utilities ("Distutils") from the
  20. end-user's point-of-view, describing how to extend the capabilities of a
  21. standard Python installation by building and installing third-party Python
  22. modules and extensions.
  23. .. _inst-intro:
  24. Introduction
  25. ============
  26. Although Python's extensive standard library covers many programming needs,
  27. there often comes a time when you need to add some new functionality to your
  28. Python installation in the form of third-party modules. This might be necessary
  29. to support your own programming, or to support an application that you want to
  30. use and that happens to be written in Python.
  31. In the past, there has been little support for adding third-party modules to an
  32. existing Python installation. With the introduction of the Python Distribution
  33. Utilities (Distutils for short) in Python 2.0, this changed.
  34. This document is aimed primarily at the people who need to install third-party
  35. Python modules: end-users and system administrators who just need to get some
  36. Python application running, and existing Python programmers who want to add some
  37. new goodies to their toolbox. You don't need to know Python to read this
  38. document; there will be some brief forays into using Python's interactive mode
  39. to explore your installation, but that's it. If you're looking for information
  40. on how to distribute your own Python modules so that others may use them, see
  41. the :ref:`distutils-index` manual.
  42. .. _inst-trivial-install:
  43. Best case: trivial installation
  44. -------------------------------
  45. In the best case, someone will have prepared a special version of the module
  46. distribution you want to install that is targeted specifically at your platform
  47. and is installed just like any other software on your platform. For example,
  48. the module developer might make an executable installer available for Windows
  49. users, an RPM package for users of RPM-based Linux systems (Red Hat, SuSE,
  50. Mandrake, and many others), a Debian package for users of Debian-based Linux
  51. systems, and so forth.
  52. In that case, you would download the installer appropriate to your platform and
  53. do the obvious thing with it: run it if it's an executable installer, ``rpm
  54. --install`` it if it's an RPM, etc. You don't need to run Python or a setup
  55. script, you don't need to compile anything---you might not even need to read any
  56. instructions (although it's always a good idea to do so anyways).
  57. Of course, things will not always be that easy. You might be interested in a
  58. module distribution that doesn't have an easy-to-use installer for your
  59. platform. In that case, you'll have to start with the source distribution
  60. released by the module's author/maintainer. Installing from a source
  61. distribution is not too hard, as long as the modules are packaged in the
  62. standard way. The bulk of this document is about building and installing
  63. modules from standard source distributions.
  64. .. _inst-new-standard:
  65. The new standard: Distutils
  66. ---------------------------
  67. If you download a module source distribution, you can tell pretty quickly if it
  68. was packaged and distributed in the standard way, i.e. using the Distutils.
  69. First, the distribution's name and version number will be featured prominently
  70. in the name of the downloaded archive, e.g. :file:`foo-1.0.tar.gz` or
  71. :file:`widget-0.9.7.zip`. Next, the archive will unpack into a similarly-named
  72. directory: :file:`foo-1.0` or :file:`widget-0.9.7`. Additionally, the
  73. distribution will contain a setup script :file:`setup.py`, and a file named
  74. :file:`README.txt` or possibly just :file:`README`, which should explain that
  75. building and installing the module distribution is a simple matter of running ::
  76. python setup.py install
  77. If all these things are true, then you already know how to build and install the
  78. modules you've just downloaded: Run the command above. Unless you need to
  79. install things in a non-standard way or customize the build process, you don't
  80. really need this manual. Or rather, the above command is everything you need to
  81. get out of this manual.
  82. .. _inst-standard-install:
  83. Standard Build and Install
  84. ==========================
  85. As described in section :ref:`inst-new-standard`, building and installing a module
  86. distribution using the Distutils is usually one simple command::
  87. python setup.py install
  88. On Unix, you'd run this command from a shell prompt; on Windows, you have to
  89. open a command prompt window ("DOS box") and do it there; on Mac OS X, you open
  90. a :command:`Terminal` window to get a shell prompt.
  91. .. _inst-platform-variations:
  92. Platform variations
  93. -------------------
  94. You should always run the setup command from the distribution root directory,
  95. i.e. the top-level subdirectory that the module source distribution unpacks
  96. into. For example, if you've just downloaded a module source distribution
  97. :file:`foo-1.0.tar.gz` onto a Unix system, the normal thing to do is::
  98. gunzip -c foo-1.0.tar.gz | tar xf - # unpacks into directory foo-1.0
  99. cd foo-1.0
  100. python setup.py install
  101. On Windows, you'd probably download :file:`foo-1.0.zip`. If you downloaded the
  102. archive file to :file:`C:\\Temp`, then it would unpack into
  103. :file:`C:\\Temp\\foo-1.0`; you can use either a archive manipulator with a
  104. graphical user interface (such as WinZip) or a command-line tool (such as
  105. :program:`unzip` or :program:`pkunzip`) to unpack the archive. Then, open a
  106. command prompt window ("DOS box"), and run::
  107. cd c:\Temp\foo-1.0
  108. python setup.py install
  109. .. _inst-splitting-up:
  110. Splitting the job up
  111. --------------------
  112. Running ``setup.py install`` builds and installs all modules in one run. If you
  113. prefer to work incrementally---especially useful if you want to customize the
  114. build process, or if things are going wrong---you can use the setup script to do
  115. one thing at a time. This is particularly helpful when the build and install
  116. will be done by different users---for example, you might want to build a module
  117. distribution and hand it off to a system administrator for installation (or do
  118. it yourself, with super-user privileges).
  119. For example, you can build everything in one step, and then install everything
  120. in a second step, by invoking the setup script twice::
  121. python setup.py build
  122. python setup.py install
  123. If you do this, you will notice that running the :command:`install` command
  124. first runs the :command:`build` command, which---in this case---quickly notices
  125. that it has nothing to do, since everything in the :file:`build` directory is
  126. up-to-date.
  127. You may not need this ability to break things down often if all you do is
  128. install modules downloaded off the 'net, but it's very handy for more advanced
  129. tasks. If you get into distributing your own Python modules and extensions,
  130. you'll run lots of individual Distutils commands on their own.
  131. .. _inst-how-build-works:
  132. How building works
  133. ------------------
  134. As implied above, the :command:`build` command is responsible for putting the
  135. files to install into a *build directory*. By default, this is :file:`build`
  136. under the distribution root; if you're excessively concerned with speed, or want
  137. to keep the source tree pristine, you can change the build directory with the
  138. :option:`--build-base` option. For example::
  139. python setup.py build --build-base=/tmp/pybuild/foo-1.0
  140. (Or you could do this permanently with a directive in your system or personal
  141. Distutils configuration file; see section :ref:`inst-config-files`.) Normally, this
  142. isn't necessary.
  143. The default layout for the build tree is as follows::
  144. --- build/ --- lib/
  145. or
  146. --- build/ --- lib.<plat>/
  147. temp.<plat>/
  148. where ``<plat>`` expands to a brief description of the current OS/hardware
  149. platform and Python version. The first form, with just a :file:`lib` directory,
  150. is used for "pure module distributions"---that is, module distributions that
  151. include only pure Python modules. If a module distribution contains any
  152. extensions (modules written in C/C++), then the second form, with two ``<plat>``
  153. directories, is used. In that case, the :file:`temp.{plat}` directory holds
  154. temporary files generated by the compile/link process that don't actually get
  155. installed. In either case, the :file:`lib` (or :file:`lib.{plat}`) directory
  156. contains all Python modules (pure Python and extensions) that will be installed.
  157. In the future, more directories will be added to handle Python scripts,
  158. documentation, binary executables, and whatever else is needed to handle the job
  159. of installing Python modules and applications.
  160. .. _inst-how-install-works:
  161. How installation works
  162. ----------------------
  163. After the :command:`build` command runs (whether you run it explicitly, or the
  164. :command:`install` command does it for you), the work of the :command:`install`
  165. command is relatively simple: all it has to do is copy everything under
  166. :file:`build/lib` (or :file:`build/lib.{plat}`) to your chosen installation
  167. directory.
  168. If you don't choose an installation directory---i.e., if you just run ``setup.py
  169. install``\ ---then the :command:`install` command installs to the standard
  170. location for third-party Python modules. This location varies by platform and
  171. by how you built/installed Python itself. On Unix (and Mac OS X, which is also
  172. Unix-based), it also depends on whether the module distribution being installed
  173. is pure Python or contains extensions ("non-pure"):
  174. +-----------------+-----------------------------------------------------+--------------------------------------------------+-------+
  175. | Platform | Standard installation location | Default value | Notes |
  176. +=================+=====================================================+==================================================+=======+
  177. | Unix (pure) | :file:`{prefix}/lib/python{X.Y}/site-packages` | :file:`/usr/local/lib/python{X.Y}/site-packages` | \(1) |
  178. +-----------------+-----------------------------------------------------+--------------------------------------------------+-------+
  179. | Unix (non-pure) | :file:`{exec-prefix}/lib/python{X.Y}/site-packages` | :file:`/usr/local/lib/python{X.Y}/site-packages` | \(1) |
  180. +-----------------+-----------------------------------------------------+--------------------------------------------------+-------+
  181. | Windows | :file:`{prefix}` | :file:`C:\\Python` | \(2) |
  182. +-----------------+-----------------------------------------------------+--------------------------------------------------+-------+
  183. Notes:
  184. (1)
  185. Most Linux distributions include Python as a standard part of the system, so
  186. :file:`{prefix}` and :file:`{exec-prefix}` are usually both :file:`/usr` on
  187. Linux. If you build Python yourself on Linux (or any Unix-like system), the
  188. default :file:`{prefix}` and :file:`{exec-prefix}` are :file:`/usr/local`.
  189. (2)
  190. The default installation directory on Windows was :file:`C:\\Program
  191. Files\\Python` under Python 1.6a1, 1.5.2, and earlier.
  192. :file:`{prefix}` and :file:`{exec-prefix}` stand for the directories that Python
  193. is installed to, and where it finds its libraries at run-time. They are always
  194. the same under Windows, and very often the same under Unix and Mac OS X. You
  195. can find out what your Python installation uses for :file:`{prefix}` and
  196. :file:`{exec-prefix}` by running Python in interactive mode and typing a few
  197. simple commands. Under Unix, just type ``python`` at the shell prompt. Under
  198. Windows, choose :menuselection:`Start --> Programs --> Python X.Y -->
  199. Python (command line)`. Once the interpreter is started, you type Python code
  200. at the prompt. For example, on my Linux system, I type the three Python
  201. statements shown below, and get the output as shown, to find out my
  202. :file:`{prefix}` and :file:`{exec-prefix}`::
  203. Python 2.4 (#26, Aug 7 2004, 17:19:02)
  204. Type "help", "copyright", "credits" or "license" for more information.
  205. >>> import sys
  206. >>> sys.prefix
  207. '/usr'
  208. >>> sys.exec_prefix
  209. '/usr'
  210. If you don't want to install modules to the standard location, or if you don't
  211. have permission to write there, then you need to read about alternate
  212. installations in section :ref:`inst-alt-install`. If you want to customize your
  213. installation directories more heavily, see section :ref:`inst-custom-install` on
  214. custom installations.
  215. .. _inst-alt-install:
  216. Alternate Installation
  217. ======================
  218. Often, it is necessary or desirable to install modules to a location other than
  219. the standard location for third-party Python modules. For example, on a Unix
  220. system you might not have permission to write to the standard third-party module
  221. directory. Or you might wish to try out a module before making it a standard
  222. part of your local Python installation. This is especially true when upgrading
  223. a distribution already present: you want to make sure your existing base of
  224. scripts still works with the new version before actually upgrading.
  225. The Distutils :command:`install` command is designed to make installing module
  226. distributions to an alternate location simple and painless. The basic idea is
  227. that you supply a base directory for the installation, and the
  228. :command:`install` command picks a set of directories (called an *installation
  229. scheme*) under this base directory in which to install files. The details
  230. differ across platforms, so read whichever of the following sections applies to
  231. you.
  232. .. _inst-alt-install-prefix:
  233. Alternate installation: the home scheme
  234. ---------------------------------------
  235. The idea behind the "home scheme" is that you build and maintain a personal
  236. stash of Python modules. This scheme's name is derived from the idea of a
  237. "home" directory on Unix, since it's not unusual for a Unix user to make their
  238. home directory have a layout similar to :file:`/usr/` or :file:`/usr/local/`.
  239. This scheme can be used by anyone, regardless of the operating system their
  240. installing for.
  241. Installing a new module distribution is as simple as ::
  242. python setup.py install --home=<dir>
  243. where you can supply any directory you like for the :option:`--home` option. On
  244. Unix, lazy typists can just type a tilde (``~``); the :command:`install` command
  245. will expand this to your home directory::
  246. python setup.py install --home=~
  247. The :option:`--home` option defines the installation base directory. Files are
  248. installed to the following directories under the installation base as follows:
  249. +------------------------------+---------------------------+-----------------------------+
  250. | Type of file | Installation Directory | Override option |
  251. +==============================+===========================+=============================+
  252. | pure module distribution | :file:`{home}/lib/python` | :option:`--install-purelib` |
  253. +------------------------------+---------------------------+-----------------------------+
  254. | non-pure module distribution | :file:`{home}/lib/python` | :option:`--install-platlib` |
  255. +------------------------------+---------------------------+-----------------------------+
  256. | scripts | :file:`{home}/bin` | :option:`--install-scripts` |
  257. +------------------------------+---------------------------+-----------------------------+
  258. | data | :file:`{home}/share` | :option:`--install-data` |
  259. +------------------------------+---------------------------+-----------------------------+
  260. .. versionchanged:: 2.4
  261. The :option:`--home` option used to be supported only on Unix.
  262. .. _inst-alt-install-home:
  263. Alternate installation: Unix (the prefix scheme)
  264. ------------------------------------------------
  265. The "prefix scheme" is useful when you wish to use one Python installation to
  266. perform the build/install (i.e., to run the setup script), but install modules
  267. into the third-party module directory of a different Python installation (or
  268. something that looks like a different Python installation). If this sounds a
  269. trifle unusual, it is---that's why the "home scheme" comes first. However,
  270. there are at least two known cases where the prefix scheme will be useful.
  271. First, consider that many Linux distributions put Python in :file:`/usr`, rather
  272. than the more traditional :file:`/usr/local`. This is entirely appropriate,
  273. since in those cases Python is part of "the system" rather than a local add-on.
  274. However, if you are installing Python modules from source, you probably want
  275. them to go in :file:`/usr/local/lib/python2.{X}` rather than
  276. :file:`/usr/lib/python2.{X}`. This can be done with ::
  277. /usr/bin/python setup.py install --prefix=/usr/local
  278. Another possibility is a network filesystem where the name used to write to a
  279. remote directory is different from the name used to read it: for example, the
  280. Python interpreter accessed as :file:`/usr/local/bin/python` might search for
  281. modules in :file:`/usr/local/lib/python2.{X}`, but those modules would have to
  282. be installed to, say, :file:`/mnt/{@server}/export/lib/python2.{X}`. This could
  283. be done with ::
  284. /usr/local/bin/python setup.py install --prefix=/mnt/@server/export
  285. In either case, the :option:`--prefix` option defines the installation base, and
  286. the :option:`--exec-prefix` option defines the platform-specific installation
  287. base, which is used for platform-specific files. (Currently, this just means
  288. non-pure module distributions, but could be expanded to C libraries, binary
  289. executables, etc.) If :option:`--exec-prefix` is not supplied, it defaults to
  290. :option:`--prefix`. Files are installed as follows:
  291. +------------------------------+-----------------------------------------------------+-----------------------------+
  292. | Type of file | Installation Directory | Override option |
  293. +==============================+=====================================================+=============================+
  294. | pure module distribution | :file:`{prefix}/lib/python{X.Y}/site-packages` | :option:`--install-purelib` |
  295. +------------------------------+-----------------------------------------------------+-----------------------------+
  296. | non-pure module distribution | :file:`{exec-prefix}/lib/python{X.Y}/site-packages` | :option:`--install-platlib` |
  297. +------------------------------+-----------------------------------------------------+-----------------------------+
  298. | scripts | :file:`{prefix}/bin` | :option:`--install-scripts` |
  299. +------------------------------+-----------------------------------------------------+-----------------------------+
  300. | data | :file:`{prefix}/share` | :option:`--install-data` |
  301. +------------------------------+-----------------------------------------------------+-----------------------------+
  302. There is no requirement that :option:`--prefix` or :option:`--exec-prefix`
  303. actually point to an alternate Python installation; if the directories listed
  304. above do not already exist, they are created at installation time.
  305. Incidentally, the real reason the prefix scheme is important is simply that a
  306. standard Unix installation uses the prefix scheme, but with :option:`--prefix`
  307. and :option:`--exec-prefix` supplied by Python itself as ``sys.prefix`` and
  308. ``sys.exec_prefix``. Thus, you might think you'll never use the prefix scheme,
  309. but every time you run ``python setup.py install`` without any other options,
  310. you're using it.
  311. Note that installing extensions to an alternate Python installation has no
  312. effect on how those extensions are built: in particular, the Python header files
  313. (:file:`Python.h` and friends) installed with the Python interpreter used to run
  314. the setup script will be used in compiling extensions. It is your
  315. responsibility to ensure that the interpreter used to run extensions installed
  316. in this way is compatible with the interpreter used to build them. The best way
  317. to do this is to ensure that the two interpreters are the same version of Python
  318. (possibly different builds, or possibly copies of the same build). (Of course,
  319. if your :option:`--prefix` and :option:`--exec-prefix` don't even point to an
  320. alternate Python installation, this is immaterial.)
  321. .. _inst-alt-install-windows:
  322. Alternate installation: Windows (the prefix scheme)
  323. ---------------------------------------------------
  324. Windows has no concept of a user's home directory, and since the standard Python
  325. installation under Windows is simpler than under Unix, the :option:`--prefix`
  326. option has traditionally been used to install additional packages in separate
  327. locations on Windows. ::
  328. python setup.py install --prefix="\Temp\Python"
  329. to install modules to the :file:`\\Temp\\Python` directory on the current drive.
  330. The installation base is defined by the :option:`--prefix` option; the
  331. :option:`--exec-prefix` option is not supported under Windows. Files are
  332. installed as follows:
  333. +------------------------------+---------------------------+-----------------------------+
  334. | Type of file | Installation Directory | Override option |
  335. +==============================+===========================+=============================+
  336. | pure module distribution | :file:`{prefix}` | :option:`--install-purelib` |
  337. +------------------------------+---------------------------+-----------------------------+
  338. | non-pure module distribution | :file:`{prefix}` | :option:`--install-platlib` |
  339. +------------------------------+---------------------------+-----------------------------+
  340. | scripts | :file:`{prefix}\\Scripts` | :option:`--install-scripts` |
  341. +------------------------------+---------------------------+-----------------------------+
  342. | data | :file:`{prefix}\\Data` | :option:`--install-data` |
  343. +------------------------------+---------------------------+-----------------------------+
  344. .. _inst-custom-install:
  345. Custom Installation
  346. ===================
  347. Sometimes, the alternate installation schemes described in section
  348. :ref:`inst-alt-install` just don't do what you want. You might want to tweak just
  349. one or two directories while keeping everything under the same base directory,
  350. or you might want to completely redefine the installation scheme. In either
  351. case, you're creating a *custom installation scheme*.
  352. You probably noticed the column of "override options" in the tables describing
  353. the alternate installation schemes above. Those options are how you define a
  354. custom installation scheme. These override options can be relative, absolute,
  355. or explicitly defined in terms of one of the installation base directories.
  356. (There are two installation base directories, and they are normally the same---
  357. they only differ when you use the Unix "prefix scheme" and supply different
  358. :option:`--prefix` and :option:`--exec-prefix` options.)
  359. For example, say you're installing a module distribution to your home directory
  360. under Unix---but you want scripts to go in :file:`~/scripts` rather than
  361. :file:`~/bin`. As you might expect, you can override this directory with the
  362. :option:`--install-scripts` option; in this case, it makes most sense to supply
  363. a relative path, which will be interpreted relative to the installation base
  364. directory (your home directory, in this case)::
  365. python setup.py install --home=~ --install-scripts=scripts
  366. Another Unix example: suppose your Python installation was built and installed
  367. with a prefix of :file:`/usr/local/python`, so under a standard installation
  368. scripts will wind up in :file:`/usr/local/python/bin`. If you want them in
  369. :file:`/usr/local/bin` instead, you would supply this absolute directory for the
  370. :option:`--install-scripts` option::
  371. python setup.py install --install-scripts=/usr/local/bin
  372. (This performs an installation using the "prefix scheme," where the prefix is
  373. whatever your Python interpreter was installed with--- :file:`/usr/local/python`
  374. in this case.)
  375. If you maintain Python on Windows, you might want third-party modules to live in
  376. a subdirectory of :file:`{prefix}`, rather than right in :file:`{prefix}`
  377. itself. This is almost as easy as customizing the script installation directory
  378. ---you just have to remember that there are two types of modules to worry about,
  379. pure modules and non-pure modules (i.e., modules from a non-pure distribution).
  380. For example::
  381. python setup.py install --install-purelib=Site --install-platlib=Site
  382. The specified installation directories are relative to :file:`{prefix}`. Of
  383. course, you also have to ensure that these directories are in Python's module
  384. search path, such as by putting a :file:`.pth` file in :file:`{prefix}`. See
  385. section :ref:`inst-search-path` to find out how to modify Python's search path.
  386. If you want to define an entire installation scheme, you just have to supply all
  387. of the installation directory options. The recommended way to do this is to
  388. supply relative paths; for example, if you want to maintain all Python
  389. module-related files under :file:`python` in your home directory, and you want a
  390. separate directory for each platform that you use your home directory from, you
  391. might define the following installation scheme::
  392. python setup.py install --home=~ \
  393. --install-purelib=python/lib \
  394. --install-platlib=python/lib.$PLAT \
  395. --install-scripts=python/scripts
  396. --install-data=python/data
  397. or, equivalently, ::
  398. python setup.py install --home=~/python \
  399. --install-purelib=lib \
  400. --install-platlib='lib.$PLAT' \
  401. --install-scripts=scripts
  402. --install-data=data
  403. ``$PLAT`` is not (necessarily) an environment variable---it will be expanded by
  404. the Distutils as it parses your command line options, just as it does when
  405. parsing your configuration file(s).
  406. Obviously, specifying the entire installation scheme every time you install a
  407. new module distribution would be very tedious. Thus, you can put these options
  408. into your Distutils config file (see section :ref:`inst-config-files`)::
  409. [install]
  410. install-base=$HOME
  411. install-purelib=python/lib
  412. install-platlib=python/lib.$PLAT
  413. install-scripts=python/scripts
  414. install-data=python/data
  415. or, equivalently, ::
  416. [install]
  417. install-base=$HOME/python
  418. install-purelib=lib
  419. install-platlib=lib.$PLAT
  420. install-scripts=scripts
  421. install-data=data
  422. Note that these two are *not* equivalent if you supply a different installation
  423. base directory when you run the setup script. For example, ::
  424. python setup.py install --install-base=/tmp
  425. would install pure modules to :file:`{/tmp/python/lib}` in the first case, and
  426. to :file:`{/tmp/lib}` in the second case. (For the second case, you probably
  427. want to supply an installation base of :file:`/tmp/python`.)
  428. You probably noticed the use of ``$HOME`` and ``$PLAT`` in the sample
  429. configuration file input. These are Distutils configuration variables, which
  430. bear a strong resemblance to environment variables. In fact, you can use
  431. environment variables in config files on platforms that have such a notion but
  432. the Distutils additionally define a few extra variables that may not be in your
  433. environment, such as ``$PLAT``. (And of course, on systems that don't have
  434. environment variables, such as Mac OS 9, the configuration variables supplied by
  435. the Distutils are the only ones you can use.) See section :ref:`inst-config-files`
  436. for details.
  437. .. XXX need some Windows examples---when would custom installation schemes be
  438. needed on those platforms?
  439. .. XXX I'm not sure where this section should go.
  440. .. _inst-search-path:
  441. Modifying Python's Search Path
  442. ------------------------------
  443. When the Python interpreter executes an :keyword:`import` statement, it searches
  444. for both Python code and extension modules along a search path. A default value
  445. for the path is configured into the Python binary when the interpreter is built.
  446. You can determine the path by importing the :mod:`sys` module and printing the
  447. value of ``sys.path``. ::
  448. $ python
  449. Python 2.2 (#11, Oct 3 2002, 13:31:27)
  450. [GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-112)] on linux2
  451. Type "help", "copyright", "credits" or "license" for more information.
  452. >>> import sys
  453. >>> sys.path
  454. ['', '/usr/local/lib/python2.3', '/usr/local/lib/python2.3/plat-linux2',
  455. '/usr/local/lib/python2.3/lib-tk', '/usr/local/lib/python2.3/lib-dynload',
  456. '/usr/local/lib/python2.3/site-packages']
  457. >>>
  458. The null string in ``sys.path`` represents the current working directory.
  459. The expected convention for locally installed packages is to put them in the
  460. :file:`{...}/site-packages/` directory, but you may want to install Python
  461. modules into some arbitrary directory. For example, your site may have a
  462. convention of keeping all software related to the web server under :file:`/www`.
  463. Add-on Python modules might then belong in :file:`/www/python`, and in order to
  464. import them, this directory must be added to ``sys.path``. There are several
  465. different ways to add the directory.
  466. The most convenient way is to add a path configuration file to a directory
  467. that's already on Python's path, usually to the :file:`.../site-packages/`
  468. directory. Path configuration files have an extension of :file:`.pth`, and each
  469. line must contain a single path that will be appended to ``sys.path``. (Because
  470. the new paths are appended to ``sys.path``, modules in the added directories
  471. will not override standard modules. This means you can't use this mechanism for
  472. installing fixed versions of standard modules.)
  473. Paths can be absolute or relative, in which case they're relative to the
  474. directory containing the :file:`.pth` file. See the documentation of
  475. the :mod:`site` module for more information.
  476. A slightly less convenient way is to edit the :file:`site.py` file in Python's
  477. standard library, and modify ``sys.path``. :file:`site.py` is automatically
  478. imported when the Python interpreter is executed, unless the :option:`-S` switch
  479. is supplied to suppress this behaviour. So you could simply edit
  480. :file:`site.py` and add two lines to it::
  481. import sys
  482. sys.path.append('/www/python/')
  483. However, if you reinstall the same major version of Python (perhaps when
  484. upgrading from 2.2 to 2.2.2, for example) :file:`site.py` will be overwritten by
  485. the stock version. You'd have to remember that it was modified and save a copy
  486. before doing the installation.
  487. There are two environment variables that can modify ``sys.path``.
  488. :envvar:`PYTHONHOME` sets an alternate value for the prefix of the Python
  489. installation. For example, if :envvar:`PYTHONHOME` is set to ``/www/python``,
  490. the search path will be set to ``['', '/www/python/lib/pythonX.Y/',
  491. '/www/python/lib/pythonX.Y/plat-linux2', ...]``.
  492. The :envvar:`PYTHONPATH` variable can be set to a list of paths that will be
  493. added to the beginning of ``sys.path``. For example, if :envvar:`PYTHONPATH` is
  494. set to ``/www/python:/opt/py``, the search path will begin with
  495. ``['/www/python', '/opt/py']``. (Note that directories must exist in order to
  496. be added to ``sys.path``; the :mod:`site` module removes paths that don't
  497. exist.)
  498. Finally, ``sys.path`` is just a regular Python list, so any Python application
  499. can modify it by adding or removing entries.
  500. .. _inst-config-files:
  501. Distutils Configuration Files
  502. =============================
  503. As mentioned above, you can use Distutils configuration files to record personal
  504. or site preferences for any Distutils options. That is, any option to any
  505. command can be stored in one of two or three (depending on your platform)
  506. configuration files, which will be consulted before the command-line is parsed.
  507. This means that configuration files will override default values, and the
  508. command-line will in turn override configuration files. Furthermore, if
  509. multiple configuration files apply, values from "earlier" files are overridden
  510. by "later" files.
  511. .. _inst-config-filenames:
  512. Location and names of config files
  513. ----------------------------------
  514. The names and locations of the configuration files vary slightly across
  515. platforms. On Unix and Mac OS X, the three configuration files (in the order
  516. they are processed) are:
  517. +--------------+----------------------------------------------------------+-------+
  518. | Type of file | Location and filename | Notes |
  519. +==============+==========================================================+=======+
  520. | system | :file:`{prefix}/lib/python{ver}/distutils/distutils.cfg` | \(1) |
  521. +--------------+----------------------------------------------------------+-------+
  522. | personal | :file:`$HOME/.pydistutils.cfg` | \(2) |
  523. +--------------+----------------------------------------------------------+-------+
  524. | local | :file:`setup.cfg` | \(3) |
  525. +--------------+----------------------------------------------------------+-------+
  526. And on Windows, the configuration files are:
  527. +--------------+-------------------------------------------------+-------+
  528. | Type of file | Location and filename | Notes |
  529. +==============+=================================================+=======+
  530. | system | :file:`{prefix}\\Lib\\distutils\\distutils.cfg` | \(4) |
  531. +--------------+-------------------------------------------------+-------+
  532. | personal | :file:`%HOME%\\pydistutils.cfg` | \(5) |
  533. +--------------+-------------------------------------------------+-------+
  534. | local | :file:`setup.cfg` | \(3) |
  535. +--------------+-------------------------------------------------+-------+
  536. Notes:
  537. (1)
  538. Strictly speaking, the system-wide configuration file lives in the directory
  539. where the Distutils are installed; under Python 1.6 and later on Unix, this is
  540. as shown. For Python 1.5.2, the Distutils will normally be installed to
  541. :file:`{prefix}/lib/python1.5/site-packages/distutils`, so the system
  542. configuration file should be put there under Python 1.5.2.
  543. (2)
  544. On Unix, if the :envvar:`HOME` environment variable is not defined, the user's
  545. home directory will be determined with the :func:`getpwuid` function from the
  546. standard :mod:`pwd` module. This is done by the :func:`os.path.expanduser`
  547. function used by Distutils.
  548. (3)
  549. I.e., in the current directory (usually the location of the setup script).
  550. (4)
  551. (See also note (1).) Under Python 1.6 and later, Python's default "installation
  552. prefix" is :file:`C:\\Python`, so the system configuration file is normally
  553. :file:`C:\\Python\\Lib\\distutils\\distutils.cfg`. Under Python 1.5.2, the
  554. default prefix was :file:`C:\\Program Files\\Python`, and the Distutils were not
  555. part of the standard library---so the system configuration file would be
  556. :file:`C:\\Program Files\\Python\\distutils\\distutils.cfg` in a standard Python
  557. 1.5.2 installation under Windows.
  558. (5)
  559. On Windows, if the :envvar:`HOME` environment variable is not defined,
  560. :envvar:`USERPROFILE` then :envvar:`HOMEDRIVE` and :envvar:`HOMEPATH` will
  561. be tried. This is done by the :func:`os.path.expanduser` function used
  562. by Distutils.
  563. .. _inst-config-syntax:
  564. Syntax of config files
  565. ----------------------
  566. The Distutils configuration files all have the same syntax. The config files
  567. are grouped into sections. There is one section for each Distutils command,
  568. plus a ``global`` section for global options that affect every command. Each
  569. section consists of one option per line, specified as ``option=value``.
  570. For example, the following is a complete config file that just forces all
  571. commands to run quietly by default::
  572. [global]
  573. verbose=0
  574. If this is installed as the system config file, it will affect all processing of
  575. any Python module distribution by any user on the current system. If it is
  576. installed as your personal config file (on systems that support them), it will
  577. affect only module distributions processed by you. And if it is used as the
  578. :file:`setup.cfg` for a particular module distribution, it affects only that
  579. distribution.
  580. You could override the default "build base" directory and make the
  581. :command:`build\*` commands always forcibly rebuild all files with the
  582. following::
  583. [build]
  584. build-base=blib
  585. force=1
  586. which corresponds to the command-line arguments ::
  587. python setup.py build --build-base=blib --force
  588. except that including the :command:`build` command on the command-line means
  589. that command will be run. Including a particular command in config files has no
  590. such implication; it only means that if the command is run, the options in the
  591. config file will apply. (Or if other commands that derive values from it are
  592. run, they will use the values in the config file.)
  593. You can find out the complete list of options for any command using the
  594. :option:`--help` option, e.g.::
  595. python setup.py build --help
  596. and you can find out the complete list of global options by using
  597. :option:`--help` without a command::
  598. python setup.py --help
  599. See also the "Reference" section of the "Distributing Python Modules" manual.
  600. .. _inst-building-ext:
  601. Building Extensions: Tips and Tricks
  602. ====================================
  603. Whenever possible, the Distutils try to use the configuration information made
  604. available by the Python interpreter used to run the :file:`setup.py` script.
  605. For example, the same compiler and linker flags used to compile Python will also
  606. be used for compiling extensions. Usually this will work well, but in
  607. complicated situations this might be inappropriate. This section discusses how
  608. to override the usual Distutils behaviour.
  609. .. _inst-tweak-flags:
  610. Tweaking compiler/linker flags
  611. ------------------------------
  612. Compiling a Python extension written in C or C++ will sometimes require
  613. specifying custom flags for the compiler and linker in order to use a particular
  614. library or produce a special kind of object code. This is especially true if the
  615. extension hasn't been tested on your platform, or if you're trying to
  616. cross-compile Python.
  617. In the most general case, the extension author might have foreseen that
  618. compiling the extensions would be complicated, and provided a :file:`Setup` file
  619. for you to edit. This will likely only be done if the module distribution
  620. contains many separate extension modules, or if they often require elaborate
  621. sets of compiler flags in order to work.
  622. A :file:`Setup` file, if present, is parsed in order to get a list of extensions
  623. to build. Each line in a :file:`Setup` describes a single module. Lines have
  624. the following structure::
  625. module ... [sourcefile ...] [cpparg ...] [library ...]
  626. Let's examine each of the fields in turn.
  627. * *module* is the name of the extension module to be built, and should be a
  628. valid Python identifier. You can't just change this in order to rename a module
  629. (edits to the source code would also be needed), so this should be left alone.
  630. * *sourcefile* is anything that's likely to be a source code file, at least
  631. judging by the filename. Filenames ending in :file:`.c` are assumed to be
  632. written in C, filenames ending in :file:`.C`, :file:`.cc`, and :file:`.c++` are
  633. assumed to be C++, and filenames ending in :file:`.m` or :file:`.mm` are assumed
  634. to be in Objective C.
  635. * *cpparg* is an argument for the C preprocessor, and is anything starting with
  636. :option:`-I`, :option:`-D`, :option:`-U` or :option:`-C`.
  637. * *library* is anything ending in :file:`.a` or beginning with :option:`-l` or
  638. :option:`-L`.
  639. If a particular platform requires a special library on your platform, you can
  640. add it by editing the :file:`Setup` file and running ``python setup.py build``.
  641. For example, if the module defined by the line ::
  642. foo foomodule.c
  643. must be linked with the math library :file:`libm.a` on your platform, simply add
  644. :option:`-lm` to the line::
  645. foo foomodule.c -lm
  646. Arbitrary switches intended for the compiler or the linker can be supplied with
  647. the :option:`-Xcompiler` *arg* and :option:`-Xlinker` *arg* options::
  648. foo foomodule.c -Xcompiler -o32 -Xlinker -shared -lm
  649. The next option after :option:`-Xcompiler` and :option:`-Xlinker` will be
  650. appended to the proper command line, so in the above example the compiler will
  651. be passed the :option:`-o32` option, and the linker will be passed
  652. :option:`-shared`. If a compiler option requires an argument, you'll have to
  653. supply multiple :option:`-Xcompiler` options; for example, to pass ``-x c++``
  654. the :file:`Setup` file would have to contain ``-Xcompiler -x -Xcompiler c++``.
  655. Compiler flags can also be supplied through setting the :envvar:`CFLAGS`
  656. environment variable. If set, the contents of :envvar:`CFLAGS` will be added to
  657. the compiler flags specified in the :file:`Setup` file.
  658. .. _inst-non-ms-compilers:
  659. Using non-Microsoft compilers on Windows
  660. ----------------------------------------
  661. .. sectionauthor:: Rene Liebscher <R.Liebscher@gmx.de>
  662. Borland/CodeGear C++
  663. ^^^^^^^^^^^^^^^^^^^^
  664. This subsection describes the necessary steps to use Distutils with the Borland
  665. C++ compiler version 5.5. First you have to know that Borland's object file
  666. format (OMF) is different from the format used by the Python version you can
  667. download from the Python or ActiveState Web site. (Python is built with
  668. Microsoft Visual C++, which uses COFF as the object file format.) For this
  669. reason you have to convert Python's library :file:`python25.lib` into the
  670. Borland format. You can do this as follows:
  671. .. Should we mention that users have to create cfg-files for the compiler?
  672. .. see also http://community.borland.com/article/0,1410,21205,00.html
  673. ::
  674. coff2omf python25.lib python25_bcpp.lib
  675. The :file:`coff2omf` program comes with the Borland compiler. The file
  676. :file:`python25.lib` is in the :file:`Libs` directory of your Python
  677. installation. If your extension uses other libraries (zlib, ...) you have to
  678. convert them too.
  679. The converted files have to reside in the same directories as the normal
  680. libraries.
  681. How does Distutils manage to use these libraries with their changed names? If
  682. the extension needs a library (eg. :file:`foo`) Distutils checks first if it
  683. finds a library with suffix :file:`_bcpp` (eg. :file:`foo_bcpp.lib`) and then
  684. uses this library. In the case it doesn't find such a special library it uses
  685. the default name (:file:`foo.lib`.) [#]_
  686. To let Distutils compile your extension with Borland C++ you now have to type::
  687. python setup.py build --compiler=bcpp
  688. If you want to use the Borland C++ compiler as the default, you could specify
  689. this in your personal or system-wide configuration file for Distutils (see
  690. section :ref:`inst-config-files`.)
  691. .. seealso::
  692. `C++Builder Compiler <http://www.codegear.com/downloads/free/cppbuilder>`_
  693. Information about the free C++ compiler from Borland, including links to the
  694. download pages.
  695. `Creating Python Extensions Using Borland's Free Compiler <http://www.cyberus.ca/~g_will/pyExtenDL.shtml>`_
  696. Document describing how to use Borland's free command-line C++ compiler to build
  697. Python.
  698. GNU C / Cygwin / MinGW
  699. ^^^^^^^^^^^^^^^^^^^^^^
  700. These instructions only apply if you're using a version of Python prior to
  701. 2.4.1 with a MinGW prior to 3.0.0 (with binutils-2.13.90-20030111-1).
  702. This section describes the necessary steps to use Distutils with the GNU C/C++
  703. compilers in their Cygwin and MinGW distributions. [#]_ For a Python interpreter
  704. that was built with Cygwin, everything should work without any of these
  705. following steps.
  706. These compilers require some special libraries. This task is more complex than
  707. for Borland's C++, because there is no program to convert the library. First
  708. you have to create a list of symbols which the Python DLL exports. (You can find
  709. a good program for this task at http://www.emmestech.com/software/cygwin/pexports-0.43/download_pexports.html)
  710. .. I don't understand what the next line means. --amk
  711. .. (inclusive the references on data structures.)
  712. ::
  713. pexports python25.dll >python25.def
  714. The location of an installed :file:`python25.dll` will depend on the
  715. installation options and the version and language of Windows. In a "just for
  716. me" installation, it will appear in the root of the installation directory. In
  717. a shared installation, it will be located in the system directory.
  718. Then you can create from these information an import library for gcc. ::
  719. /cygwin/bin/dlltool --dllname python25.dll --def python25.def --output-lib libpython25.a
  720. The resulting library has to be placed in the same directory as
  721. :file:`python25.lib`. (Should be the :file:`libs` directory under your Python
  722. installation directory.)
  723. If your extension uses other libraries (zlib,...) you might have to convert
  724. them too. The converted files have to reside in the same directories as the
  725. normal libraries do.
  726. To let Distutils compile your extension with Cygwin you now have to type ::
  727. python setup.py build --compiler=cygwin
  728. and for Cygwin in no-cygwin mode [#]_ or for MinGW type::
  729. python setup.py build --compiler=mingw32
  730. If you want to use any of these options/compilers as default, you should
  731. consider to write it in your personal or system-wide configuration file for
  732. Distutils (see section :ref:`inst-config-files`.)
  733. .. seealso::
  734. `Building Python modules on MS Windows platform with MinGW <http://www.zope.org/Members/als/tips/win32_mingw_modules>`_
  735. Information about building the required libraries for the MinGW environment.
  736. .. rubric:: Footnotes
  737. .. [#] This also means you could replace all existing COFF-libraries with OMF-libraries
  738. of the same name.
  739. .. [#] Check http://sources.redhat.com/cygwin/ and http://www.mingw.org/ for more
  740. information
  741. .. [#] Then you have no POSIX emulation available, but you also don't need
  742. :file:`cygwin1.dll`.