/Doc/install/index.rst

  1.. highlightlang:: none
  2
  3.. _install-index:
  4
  5*****************************
  6  Installing Python Modules
  7*****************************
  8
  9:Author: Greg Ward
 10:Release: |version|
 11:Date: |today|
 12
 13.. TODO: Fill in XXX comments
 14
 15.. The audience for this document includes people who don't know anything
 16   about Python and aren't about to learn the language just in order to
 17   install and maintain it for their users, i.e. system administrators.
 18   Thus, I have to be sure to explain the basics at some point:
 19   sys.path and PYTHONPATH at least.  Should probably give pointers to
 20   other docs on "import site", PYTHONSTARTUP, PYTHONHOME, etc.
 21
 22   Finally, it might be useful to include all the material from my "Care
 23   and Feeding of a Python Installation" talk in here somewhere.  Yow!
 24
 25.. topic:: Abstract
 26
 27   This document describes the Python Distribution Utilities ("Distutils") from the
 28   end-user's point-of-view, describing how to extend the capabilities of a
 29   standard Python installation by building and installing third-party Python
 30   modules and extensions.
 31
 32
 33.. _inst-intro:
 34
 35Introduction
 36============
 37
 38Although Python's extensive standard library covers many programming needs,
 39there often comes a time when you need to add some new functionality to your
 40Python installation in the form of third-party modules.  This might be necessary
 41to support your own programming, or to support an application that you want to
 42use and that happens to be written in Python.
 43
 44In the past, there has been little support for adding third-party modules to an
 45existing Python installation.  With the introduction of the Python Distribution
 46Utilities (Distutils for short) in Python 2.0, this changed.
 47
 48This document is aimed primarily at the people who need to install third-party
 49Python modules: end-users and system administrators who just need to get some
 50Python application running, and existing Python programmers who want to add some
 51new goodies to their toolbox.  You don't need to know Python to read this
 52document; there will be some brief forays into using Python's interactive mode
 53to explore your installation, but that's it.  If you're looking for information
 54on how to distribute your own Python modules so that others may use them, see
 55the :ref:`distutils-index` manual.
 56
 57
 58.. _inst-trivial-install:
 59
 60Best case: trivial installation
 61-------------------------------
 62
 63In the best case, someone will have prepared a special version of the module
 64distribution you want to install that is targeted specifically at your platform
 65and is installed just like any other software on your platform.  For example,
 66the module developer might make an executable installer available for Windows
 67users, an RPM package for users of RPM-based Linux systems (Red Hat, SuSE,
 68Mandrake, and many others), a Debian package for users of Debian-based Linux
 69systems, and so forth.
 70
 71In that case, you would download the installer appropriate to your platform and
 72do the obvious thing with it: run it if it's an executable installer, ``rpm
 73--install`` it if it's an RPM, etc.  You don't need to run Python or a setup
 74script, you don't need to compile anything---you might not even need to read any
 75instructions (although it's always a good idea to do so anyways).
 76
 77Of course, things will not always be that easy.  You might be interested in a
 78module distribution that doesn't have an easy-to-use installer for your
 79platform.  In that case, you'll have to start with the source distribution
 80released by the module's author/maintainer.  Installing from a source
 81distribution is not too hard, as long as the modules are packaged in the
 82standard way.  The bulk of this document is about building and installing
 83modules from standard source distributions.
 84
 85
 86.. _inst-new-standard:
 87
 88The new standard: Distutils
 89---------------------------
 90
 91If you download a module source distribution, you can tell pretty quickly if it
 92was packaged and distributed in the standard way, i.e. using the Distutils.
 93First, the distribution's name and version number will be featured prominently
 94in the name of the downloaded archive, e.g. :file:`foo-1.0.tar.gz` or
 95:file:`widget-0.9.7.zip`.  Next, the archive will unpack into a similarly-named
 96directory: :file:`foo-1.0` or :file:`widget-0.9.7`.  Additionally, the
 97distribution will contain a setup script :file:`setup.py`, and a file named
 98:file:`README.txt` or possibly just :file:`README`, which should explain that
 99building and installing the module distribution is a simple matter of running ::
100
101   python setup.py install
102
103If all these things are true, then you already know how to build and install the
104modules you've just downloaded:  Run the command above. Unless you need to
105install things in a non-standard way or customize the build process, you don't
106really need this manual.  Or rather, the above command is everything you need to
107get out of this manual.
108
109
110.. _inst-standard-install:
111
112Standard Build and Install
113==========================
114
115As described in section :ref:`inst-new-standard`, building and installing a module
116distribution using the Distutils is usually one simple command::
117
118   python setup.py install
119
120On Unix, you'd run this command from a shell prompt; on Windows, you have to
121open a command prompt window ("DOS box") and do it there; on Mac OS X, you open
122a :command:`Terminal` window to get a shell prompt.
123
124
125.. _inst-platform-variations:
126
127Platform variations
128-------------------
129
130You should always run the setup command from the distribution root directory,
131i.e. the top-level subdirectory that the module source distribution unpacks
132into.  For example, if you've just downloaded a module source distribution
133:file:`foo-1.0.tar.gz` onto a Unix system, the normal thing to do is::
134
135   gunzip -c foo-1.0.tar.gz | tar xf -    # unpacks into directory foo-1.0
136   cd foo-1.0
137   python setup.py install
138
139On Windows, you'd probably download :file:`foo-1.0.zip`.  If you downloaded the
140archive file to :file:`C:\\Temp`, then it would unpack into
141:file:`C:\\Temp\\foo-1.0`; you can use either a archive manipulator with a
142graphical user interface (such as WinZip) or a command-line tool (such as
143:program:`unzip` or :program:`pkunzip`) to unpack the archive.  Then, open a
144command prompt window ("DOS box"), and run::
145
146   cd c:\Temp\foo-1.0
147   python setup.py install
148
149
150.. _inst-splitting-up:
151
152Splitting the job up
153--------------------
154
155Running ``setup.py install`` builds and installs all modules in one run.  If you
156prefer to work incrementally---especially useful if you want to customize the
157build process, or if things are going wrong---you can use the setup script to do
158one thing at a time.  This is particularly helpful when the build and install
159will be done by different users---for example, you might want to build a module
160distribution and hand it off to a system administrator for installation (or do
161it yourself, with super-user privileges).
162
163For example, you can build everything in one step, and then install everything
164in a second step, by invoking the setup script twice::
165
166   python setup.py build
167   python setup.py install
168
169If you do this, you will notice that running the :command:`install` command
170first runs the :command:`build` command, which---in this case---quickly notices
171that it has nothing to do, since everything in the :file:`build` directory is
172up-to-date.
173
174You may not need this ability to break things down often if all you do is
175install modules downloaded off the 'net, but it's very handy for more advanced
176tasks.  If you get into distributing your own Python modules and extensions,
177you'll run lots of individual Distutils commands on their own.
178
179
180.. _inst-how-build-works:
181
182How building works
183------------------
184
185As implied above, the :command:`build` command is responsible for putting the
186files to install into a *build directory*.  By default, this is :file:`build`
187under the distribution root; if you're excessively concerned with speed, or want
188to keep the source tree pristine, you can change the build directory with the
189:option:`--build-base` option. For example::
190
191   python setup.py build --build-base=/tmp/pybuild/foo-1.0
192
193(Or you could do this permanently with a directive in your system or personal
194Distutils configuration file; see section :ref:`inst-config-files`.)  Normally, this
195isn't necessary.
196
197The default layout for the build tree is as follows::
198
199   --- build/ --- lib/
200   or
201   --- build/ --- lib.<plat>/
202                  temp.<plat>/
203
204where ``<plat>`` expands to a brief description of the current OS/hardware
205platform and Python version.  The first form, with just a :file:`lib` directory,
206is used for "pure module distributions"---that is, module distributions that
207include only pure Python modules.  If a module distribution contains any
208extensions (modules written in C/C++), then the second form, with two ``<plat>``
209directories, is used.  In that case, the :file:`temp.{plat}` directory holds
210temporary files generated by the compile/link process that don't actually get
211installed.  In either case, the :file:`lib` (or :file:`lib.{plat}`) directory
212contains all Python modules (pure Python and extensions) that will be installed.
213
214In the future, more directories will be added to handle Python scripts,
215documentation, binary executables, and whatever else is needed to handle the job
216of installing Python modules and applications.
217
218
219.. _inst-how-install-works:
220
221How installation works
222----------------------
223
224After the :command:`build` command runs (whether you run it explicitly, or the
225:command:`install` command does it for you), the work of the :command:`install`
226command is relatively simple: all it has to do is copy everything under
227:file:`build/lib` (or :file:`build/lib.{plat}`) to your chosen installation
228directory.
229
230If you don't choose an installation directory---i.e., if you just run ``setup.py
231install``\ ---then the :command:`install` command installs to the standard
232location for third-party Python modules.  This location varies by platform and
233by how you built/installed Python itself.  On Unix (and Mac OS X, which is also
234Unix-based), it also depends on whether the module distribution being installed
235is pure Python or contains extensions ("non-pure"):
236
237+-----------------+-----------------------------------------------------+--------------------------------------------------+-------+
238| Platform        | Standard installation location                      | Default value                                    | Notes |
239+=================+=====================================================+==================================================+=======+
240| Unix (pure)     | :file:`{prefix}/lib/python{X.Y}/site-packages`      | :file:`/usr/local/lib/python{X.Y}/site-packages` | \(1)  |
241+-----------------+-----------------------------------------------------+--------------------------------------------------+-------+
242| Unix (non-pure) | :file:`{exec-prefix}/lib/python{X.Y}/site-packages` | :file:`/usr/local/lib/python{X.Y}/site-packages` | \(1)  |
243+-----------------+-----------------------------------------------------+--------------------------------------------------+-------+
244| Windows         | :file:`{prefix}`                                    | :file:`C:\\Python`                               | \(2)  |
245+-----------------+-----------------------------------------------------+--------------------------------------------------+-------+
246
247Notes:
248
249(1)
250   Most Linux distributions include Python as a standard part of the system, so
251   :file:`{prefix}` and :file:`{exec-prefix}` are usually both :file:`/usr` on
252   Linux.  If you build Python yourself on Linux (or any Unix-like system), the
253   default :file:`{prefix}` and :file:`{exec-prefix}` are :file:`/usr/local`.
254
255(2)
256   The default installation directory on Windows was :file:`C:\\Program
257   Files\\Python` under Python 1.6a1, 1.5.2, and earlier.
258
259:file:`{prefix}` and :file:`{exec-prefix}` stand for the directories that Python
260is installed to, and where it finds its libraries at run-time.  They are always
261the same under Windows, and very often the same under Unix and Mac OS X.  You
262can find out what your Python installation uses for :file:`{prefix}` and
263:file:`{exec-prefix}` by running Python in interactive mode and typing a few
264simple commands. Under Unix, just type ``python`` at the shell prompt.  Under
265Windows, choose :menuselection:`Start --> Programs --> Python X.Y -->
266Python (command line)`.   Once the interpreter is started, you type Python code
267at the prompt.  For example, on my Linux system, I type the three Python
268statements shown below, and get the output as shown, to find out my
269:file:`{prefix}` and :file:`{exec-prefix}`::
270
271   Python 2.4 (#26, Aug  7 2004, 17:19:02)
272   Type "help", "copyright", "credits" or "license" for more information.
273   >>> import sys
274   >>> sys.prefix
275   '/usr'
276   >>> sys.exec_prefix
277   '/usr'
278
279If you don't want to install modules to the standard location, or if you don't
280have permission to write there, then you need to read about alternate
281installations in section :ref:`inst-alt-install`.  If you want to customize your
282installation directories more heavily, see section :ref:`inst-custom-install` on
283custom installations.
284
285
286.. _inst-alt-install:
287
288Alternate Installation
289======================
290
291Often, it is necessary or desirable to install modules to a location other than
292the standard location for third-party Python modules.  For example, on a Unix
293system you might not have permission to write to the standard third-party module
294directory.  Or you might wish to try out a module before making it a standard
295part of your local Python installation.  This is especially true when upgrading
296a distribution already present: you want to make sure your existing base of
297scripts still works with the new version before actually upgrading.
298
299The Distutils :command:`install` command is designed to make installing module
300distributions to an alternate location simple and painless.  The basic idea is
301that you supply a base directory for the installation, and the
302:command:`install` command picks a set of directories (called an *installation
303scheme*) under this base directory in which to install files.  The details
304differ across platforms, so read whichever of the following sections applies to
305you.
306
307
308.. _inst-alt-install-prefix:
309
310Alternate installation: the home scheme
311---------------------------------------
312
313The idea behind the "home scheme" is that you build and maintain a personal
314stash of Python modules.  This scheme's name is derived from the idea of a
315"home" directory on Unix, since it's not unusual for a Unix user to make their
316home directory have a layout similar to :file:`/usr/` or :file:`/usr/local/`.
317This scheme can be used by anyone, regardless of the operating system their
318installing for.
319
320Installing a new module distribution is as simple as ::
321
322   python setup.py install --home=<dir>
323
324where you can supply any directory you like for the :option:`--home` option.  On
325Unix, lazy typists can just type a tilde (``~``); the :command:`install` command
326will expand this to your home directory::
327
328   python setup.py install --home=~
329
330The :option:`--home` option defines the installation base directory.  Files are
331installed to the following directories under the installation base as follows:
332
333+------------------------------+---------------------------+-----------------------------+
334| Type of file                 | Installation Directory    | Override option             |
335+==============================+===========================+=============================+
336| pure module distribution     | :file:`{home}/lib/python` | :option:`--install-purelib` |
337+------------------------------+---------------------------+-----------------------------+
338| non-pure module distribution | :file:`{home}/lib/python` | :option:`--install-platlib` |
339+------------------------------+---------------------------+-----------------------------+
340| scripts                      | :file:`{home}/bin`        | :option:`--install-scripts` |
341+------------------------------+---------------------------+-----------------------------+
342| data                         | :file:`{home}/share`      | :option:`--install-data`    |
343+------------------------------+---------------------------+-----------------------------+
344
345.. versionchanged:: 2.4
346   The :option:`--home` option used to be supported only on Unix.
347
348
349.. _inst-alt-install-home:
350
351Alternate installation: Unix (the prefix scheme)
352------------------------------------------------
353
354The "prefix scheme" is useful when you wish to use one Python installation to
355perform the build/install (i.e., to run the setup script), but install modules
356into the third-party module directory of a different Python installation (or
357something that looks like a different Python installation).  If this sounds a
358trifle unusual, it is---that's why the "home scheme" comes first.  However,
359there are at least two known cases where the prefix scheme will be useful.
360
361First, consider that many Linux distributions put Python in :file:`/usr`, rather
362than the more traditional :file:`/usr/local`.  This is entirely appropriate,
363since in those cases Python is part of "the system" rather than a local add-on.
364However, if you are installing Python modules from source, you probably want
365them to go in :file:`/usr/local/lib/python2.{X}` rather than
366:file:`/usr/lib/python2.{X}`.  This can be done with ::
367
368   /usr/bin/python setup.py install --prefix=/usr/local
369
370Another possibility is a network filesystem where the name used to write to a
371remote directory is different from the name used to read it: for example, the
372Python interpreter accessed as :file:`/usr/local/bin/python` might search for
373modules in :file:`/usr/local/lib/python2.{X}`, but those modules would have to
374be installed to, say, :file:`/mnt/{@server}/export/lib/python2.{X}`.  This could
375be done with ::
376
377   /usr/local/bin/python setup.py install --prefix=/mnt/@server/export
378
379In either case, the :option:`--prefix` option defines the installation base, and
380the :option:`--exec-prefix` option defines the platform-specific installation
381base, which is used for platform-specific files.  (Currently, this just means
382non-pure module distributions, but could be expanded to C libraries, binary
383executables, etc.)  If :option:`--exec-prefix` is not supplied, it defaults to
384:option:`--prefix`.  Files are installed as follows:
385
386+------------------------------+-----------------------------------------------------+-----------------------------+
387| Type of file                 | Installation Directory                              | Override option             |
388+==============================+=====================================================+=============================+
389| pure module distribution     | :file:`{prefix}/lib/python{X.Y}/site-packages`      | :option:`--install-purelib` |
390+------------------------------+-----------------------------------------------------+-----------------------------+
391| non-pure module distribution | :file:`{exec-prefix}/lib/python{X.Y}/site-packages` | :option:`--install-platlib` |
392+------------------------------+-----------------------------------------------------+-----------------------------+
393| scripts                      | :file:`{prefix}/bin`                                | :option:`--install-scripts` |
394+------------------------------+-----------------------------------------------------+-----------------------------+
395| data                         | :file:`{prefix}/share`                              | :option:`--install-data`    |
396+------------------------------+-----------------------------------------------------+-----------------------------+
397
398There is no requirement that :option:`--prefix` or :option:`--exec-prefix`
399actually point to an alternate Python installation; if the directories listed
400above do not already exist, they are created at installation time.
401
402Incidentally, the real reason the prefix scheme is important is simply that a
403standard Unix installation uses the prefix scheme, but with :option:`--prefix`
404and :option:`--exec-prefix` supplied by Python itself as ``sys.prefix`` and
405``sys.exec_prefix``.  Thus, you might think you'll never use the prefix scheme,
406but every time you run ``python setup.py install`` without any other options,
407you're using it.
408
409Note that installing extensions to an alternate Python installation has no
410effect on how those extensions are built: in particular, the Python header files
411(:file:`Python.h` and friends) installed with the Python interpreter used to run
412the setup script will be used in compiling extensions.  It is your
413responsibility to ensure that the interpreter used to run extensions installed
414in this way is compatible with the interpreter used to build them.  The best way
415to do this is to ensure that the two interpreters are the same version of Python
416(possibly different builds, or possibly copies of the same build).  (Of course,
417if your :option:`--prefix` and :option:`--exec-prefix` don't even point to an
418alternate Python installation, this is immaterial.)
419
420
421.. _inst-alt-install-windows:
422
423Alternate installation: Windows (the prefix scheme)
424---------------------------------------------------
425
426Windows has no concept of a user's home directory, and since the standard Python
427installation under Windows is simpler than under Unix, the :option:`--prefix`
428option has traditionally been used to install additional packages in separate
429locations on Windows. ::
430
431   python setup.py install --prefix="\Temp\Python"
432
433to install modules to the :file:`\\Temp\\Python` directory on the current drive.
434
435The installation base is defined by the :option:`--prefix` option; the
436:option:`--exec-prefix` option is not supported under Windows. Files are
437installed as follows:
438
439+------------------------------+---------------------------+-----------------------------+
440| Type of file                 | Installation Directory    | Override option             |
441+==============================+===========================+=============================+
442| pure module distribution     | :file:`{prefix}`          | :option:`--install-purelib` |
443+------------------------------+---------------------------+-----------------------------+
444| non-pure module distribution | :file:`{prefix}`          | :option:`--install-platlib` |
445+------------------------------+---------------------------+-----------------------------+
446| scripts                      | :file:`{prefix}\\Scripts` | :option:`--install-scripts` |
447+------------------------------+---------------------------+-----------------------------+
448| data                         | :file:`{prefix}\\Data`    | :option:`--install-data`    |
449+------------------------------+---------------------------+-----------------------------+
450
451
452.. _inst-custom-install:
453
454Custom Installation
455===================
456
457Sometimes, the alternate installation schemes described in section
458:ref:`inst-alt-install` just don't do what you want.  You might want to tweak just
459one or two directories while keeping everything under the same base directory,
460or you might want to completely redefine the installation scheme.  In either
461case, you're creating a *custom installation scheme*.
462
463You probably noticed the column of "override options" in the tables describing
464the alternate installation schemes above.  Those options are how you define a
465custom installation scheme.  These override options can be relative, absolute,
466or explicitly defined in terms of one of the installation base directories.
467(There are two installation base directories, and they are normally the same---
468they only differ when you use the Unix "prefix scheme" and supply different
469:option:`--prefix` and :option:`--exec-prefix` options.)
470
471For example, say you're installing a module distribution to your home directory
472under Unix---but you want scripts to go in :file:`~/scripts` rather than
473:file:`~/bin`. As you might expect, you can override this directory with the
474:option:`--install-scripts` option; in this case, it makes most sense to supply
475a relative path, which will be interpreted relative to the installation base
476directory (your home directory, in this case)::
477
478   python setup.py install --home=~ --install-scripts=scripts
479
480Another Unix example: suppose your Python installation was built and installed
481with a prefix of :file:`/usr/local/python`, so under a standard  installation
482scripts will wind up in :file:`/usr/local/python/bin`.  If you want them in
483:file:`/usr/local/bin` instead, you would supply this absolute directory for the
484:option:`--install-scripts` option::
485
486   python setup.py install --install-scripts=/usr/local/bin
487
488(This performs an installation using the "prefix scheme," where the prefix is
489whatever your Python interpreter was installed with--- :file:`/usr/local/python`
490in this case.)
491
492If you maintain Python on Windows, you might want third-party modules to live in
493a subdirectory of :file:`{prefix}`, rather than right in :file:`{prefix}`
494itself.  This is almost as easy as customizing the script installation directory
495---you just have to remember that there are two types of modules to worry about,
496pure modules and non-pure modules (i.e., modules from a non-pure distribution).
497For example::
498
499   python setup.py install --install-purelib=Site --install-platlib=Site
500
501The specified installation directories are relative to :file:`{prefix}`.  Of
502course, you also have to ensure that these directories are in Python's module
503search path, such as by putting a :file:`.pth` file in :file:`{prefix}`.  See
504section :ref:`inst-search-path` to find out how to modify Python's search path.
505
506If you want to define an entire installation scheme, you just have to supply all
507of the installation directory options.  The recommended way to do this is to
508supply relative paths; for example, if you want to maintain all Python
509module-related files under :file:`python` in your home directory, and you want a
510separate directory for each platform that you use your home directory from, you
511might define the following installation scheme::
512
513   python setup.py install --home=~ \
514                           --install-purelib=python/lib \
515                           --install-platlib=python/lib.$PLAT \
516                           --install-scripts=python/scripts
517                           --install-data=python/data
518
519or, equivalently, ::
520
521   python setup.py install --home=~/python \
522                           --install-purelib=lib \
523                           --install-platlib='lib.$PLAT' \
524                           --install-scripts=scripts
525                           --install-data=data
526
527``$PLAT`` is not (necessarily) an environment variable---it will be expanded by
528the Distutils as it parses your command line options, just as it does when
529parsing your configuration file(s).
530
531Obviously, specifying the entire installation scheme every time you install a
532new module distribution would be very tedious.  Thus, you can put these options
533into your Distutils config file (see section :ref:`inst-config-files`)::
534
535   [install]
536   install-base=$HOME
537   install-purelib=python/lib
538   install-platlib=python/lib.$PLAT
539   install-scripts=python/scripts
540   install-data=python/data
541
542or, equivalently, ::
543
544   [install]
545   install-base=$HOME/python
546   install-purelib=lib
547   install-platlib=lib.$PLAT
548   install-scripts=scripts
549   install-data=data
550
551Note that these two are *not* equivalent if you supply a different installation
552base directory when you run the setup script.  For example, ::
553
554   python setup.py install --install-base=/tmp
555
556would install pure modules to :file:`{/tmp/python/lib}` in the first case, and
557to :file:`{/tmp/lib}` in the second case.  (For the second case, you probably
558want to supply an installation base of :file:`/tmp/python`.)
559
560You probably noticed the use of ``$HOME`` and ``$PLAT`` in the sample
561configuration file input.  These are Distutils configuration variables, which
562bear a strong resemblance to environment variables. In fact, you can use
563environment variables in config files on platforms that have such a notion but
564the Distutils additionally define a few extra variables that may not be in your
565environment, such as ``$PLAT``.  (And of course, on systems that don't have
566environment variables, such as Mac OS 9, the configuration variables supplied by
567the Distutils are the only ones you can use.) See section :ref:`inst-config-files`
568for details.
569
570.. XXX need some Windows examples---when would custom installation schemes be
571   needed on those platforms?
572
573
574.. XXX I'm not sure where this section should go.
575
576.. _inst-search-path:
577
578Modifying Python's Search Path
579------------------------------
580
581When the Python interpreter executes an :keyword:`import` statement, it searches
582for both Python code and extension modules along a search path.  A default value
583for the path is configured into the Python binary when the interpreter is built.
584You can determine the path by importing the :mod:`sys` module and printing the
585value of ``sys.path``.   ::
586
587   $ python
588   Python 2.2 (#11, Oct  3 2002, 13:31:27)
589   [GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-112)] on linux2
590   Type "help", "copyright", "credits" or "license" for more information.
591   >>> import sys
592   >>> sys.path
593   ['', '/usr/local/lib/python2.3', '/usr/local/lib/python2.3/plat-linux2',
594    '/usr/local/lib/python2.3/lib-tk', '/usr/local/lib/python2.3/lib-dynload',
595    '/usr/local/lib/python2.3/site-packages']
596   >>>
597
598The null string in ``sys.path`` represents the current working directory.
599
600The expected convention for locally installed packages is to put them in the
601:file:`{...}/site-packages/` directory, but you may want to install Python
602modules into some arbitrary directory.  For example, your site may have a
603convention of keeping all software related to the web server under :file:`/www`.
604Add-on Python modules might then belong in :file:`/www/python`, and in order to
605import them, this directory must be added to ``sys.path``.  There are several
606different ways to add the directory.
607
608The most convenient way is to add a path configuration file to a directory
609that's already on Python's path, usually to the :file:`.../site-packages/`
610directory.  Path configuration files have an extension of :file:`.pth`, and each
611line must contain a single path that will be appended to ``sys.path``.  (Because
612the new paths are appended to ``sys.path``, modules in the added directories
613will not override standard modules.  This means you can't use this mechanism for
614installing fixed versions of standard modules.)
615
616Paths can be absolute or relative, in which case they're relative to the
617directory containing the :file:`.pth` file.  See the documentation of
618the :mod:`site` module for more information.
619
620A slightly less convenient way is to edit the :file:`site.py` file in Python's
621standard library, and modify ``sys.path``.  :file:`site.py` is automatically
622imported when the Python interpreter is executed, unless the :option:`-S` switch
623is supplied to suppress this behaviour.  So you could simply edit
624:file:`site.py` and add two lines to it::
625
626   import sys
627   sys.path.append('/www/python/')
628
629However, if you reinstall the same major version of Python (perhaps when
630upgrading from 2.2 to 2.2.2, for example) :file:`site.py` will be overwritten by
631the stock version.  You'd have to remember that it was modified and save a copy
632before doing the installation.
633
634There are two environment variables that can modify ``sys.path``.
635:envvar:`PYTHONHOME` sets an alternate value for the prefix of the Python
636installation.  For example, if :envvar:`PYTHONHOME` is set to ``/www/python``,
637the search path will be set to ``['', '/www/python/lib/pythonX.Y/',
638'/www/python/lib/pythonX.Y/plat-linux2', ...]``.
639
640The :envvar:`PYTHONPATH` variable can be set to a list of paths that will be
641added to the beginning of ``sys.path``.  For example, if :envvar:`PYTHONPATH` is
642set to ``/www/python:/opt/py``, the search path will begin with
643``['/www/python', '/opt/py']``.  (Note that directories must exist in order to
644be added to ``sys.path``; the :mod:`site` module removes paths that don't
645exist.)
646
647Finally, ``sys.path`` is just a regular Python list, so any Python application
648can modify it by adding or removing entries.
649
650
651.. _inst-config-files:
652
653Distutils Configuration Files
654=============================
655
656As mentioned above, you can use Distutils configuration files to record personal
657or site preferences for any Distutils options.  That is, any option to any
658command can be stored in one of two or three (depending on your platform)
659configuration files, which will be consulted before the command-line is parsed.
660This means that configuration files will override default values, and the
661command-line will in turn override configuration files.  Furthermore, if
662multiple configuration files apply, values from "earlier" files are overridden
663by "later" files.
664
665
666.. _inst-config-filenames:
667
668Location and names of config files
669----------------------------------
670
671The names and locations of the configuration files vary slightly across
672platforms.  On Unix and Mac OS X, the three configuration files (in the order
673they are processed) are:
674
675+--------------+----------------------------------------------------------+-------+
676| Type of file | Location and filename                                    | Notes |
677+==============+==========================================================+=======+
678| system       | :file:`{prefix}/lib/python{ver}/distutils/distutils.cfg` | \(1)  |
679+--------------+----------------------------------------------------------+-------+
680| personal     | :file:`$HOME/.pydistutils.cfg`                           | \(2)  |
681+--------------+----------------------------------------------------------+-------+
682| local        | :file:`setup.cfg`                                        | \(3)  |
683+--------------+----------------------------------------------------------+-------+
684
685And on Windows, the configuration files are:
686
687+--------------+-------------------------------------------------+-------+
688| Type of file | Location and filename                           | Notes |
689+==============+=================================================+=======+
690| system       | :file:`{prefix}\\Lib\\distutils\\distutils.cfg` | \(4)  |
691+--------------+-------------------------------------------------+-------+
692| personal     | :file:`%HOME%\\pydistutils.cfg`                 | \(5)  |
693+--------------+-------------------------------------------------+-------+
694| local        | :file:`setup.cfg`                               | \(3)  |
695+--------------+-------------------------------------------------+-------+
696
697Notes:
698
699(1)
700   Strictly speaking, the system-wide configuration file lives in the directory
701   where the Distutils are installed; under Python 1.6 and later on Unix, this is
702   as shown. For Python 1.5.2, the Distutils will normally be installed to
703   :file:`{prefix}/lib/python1.5/site-packages/distutils`, so the system
704   configuration file should be put there under Python 1.5.2.
705
706(2)
707   On Unix, if the :envvar:`HOME` environment variable is not defined, the user's
708   home directory will be determined with the :func:`getpwuid` function from the
709   standard :mod:`pwd` module. This is done by the :func:`os.path.expanduser`
710   function used by Distutils.
711
712(3)
713   I.e., in the current directory (usually the location of the setup script).
714
715(4)
716   (See also note (1).)  Under Python 1.6 and later, Python's default "installation
717   prefix" is :file:`C:\\Python`, so the system configuration file is normally
718   :file:`C:\\Python\\Lib\\distutils\\distutils.cfg`. Under Python 1.5.2, the
719   default prefix was :file:`C:\\Program Files\\Python`, and the Distutils were not
720   part of the standard library---so the system configuration file would be
721   :file:`C:\\Program Files\\Python\\distutils\\distutils.cfg` in a standard Python
722   1.5.2 installation under Windows.
723
724(5)
725   On Windows, if the :envvar:`HOME` environment variable is not defined,
726   :envvar:`USERPROFILE` then :envvar:`HOMEDRIVE` and :envvar:`HOMEPATH` will
727   be tried. This is done by the :func:`os.path.expanduser` function used
728   by Distutils.
729
730
731.. _inst-config-syntax:
732
733Syntax of config files
734----------------------
735
736The Distutils configuration files all have the same syntax.  The config files
737are grouped into sections.  There is one section for each Distutils command,
738plus a ``global`` section for global options that affect every command.  Each
739section consists of one option per line, specified as ``option=value``.
740
741For example, the following is a complete config file that just forces all
742commands to run quietly by default::
743
744   [global]
745   verbose=0
746
747If this is installed as the system config file, it will affect all processing of
748any Python module distribution by any user on the current system.  If it is
749installed as your personal config file (on systems that support them), it will
750affect only module distributions processed by you.  And if it is used as the
751:file:`setup.cfg` for a particular module distribution, it affects only that
752distribution.
753
754You could override the default "build base" directory and make the
755:command:`build\*` commands always forcibly rebuild all files with the
756following::
757
758   [build]
759   build-base=blib
760   force=1
761
762which corresponds to the command-line arguments ::
763
764   python setup.py build --build-base=blib --force
765
766except that including the :command:`build` command on the command-line means
767that command will be run.  Including a particular command in config files has no
768such implication; it only means that if the command is run, the options in the
769config file will apply.  (Or if other commands that derive values from it are
770run, they will use the values in the config file.)
771
772You can find out the complete list of options for any command using the
773:option:`--help` option, e.g.::
774
775   python setup.py build --help
776
777and you can find out the complete list of global options by using
778:option:`--help` without a command::
779
780   python setup.py --help
781
782See also the "Reference" section of the "Distributing Python Modules" manual.
783
784
785.. _inst-building-ext:
786
787Building Extensions: Tips and Tricks
788====================================
789
790Whenever possible, the Distutils try to use the configuration information made
791available by the Python interpreter used to run the :file:`setup.py` script.
792For example, the same compiler and linker flags used to compile Python will also
793be used for compiling extensions.  Usually this will work well, but in
794complicated situations this might be inappropriate.  This section discusses how
795to override the usual Distutils behaviour.
796
797
798.. _inst-tweak-flags:
799
800Tweaking compiler/linker flags
801------------------------------
802
803Compiling a Python extension written in C or C++ will sometimes require
804specifying custom flags for the compiler and linker in order to use a particular
805library or produce a special kind of object code. This is especially true if the
806extension hasn't been tested on your platform, or if you're trying to
807cross-compile Python.
808
809In the most general case, the extension author might have foreseen that
810compiling the extensions would be complicated, and provided a :file:`Setup` file
811for you to edit.  This will likely only be done if the module distribution
812contains many separate extension modules, or if they often require elaborate
813sets of compiler flags in order to work.
814
815A :file:`Setup` file, if present, is parsed in order to get a list of extensions
816to build.  Each line in a :file:`Setup` describes a single module.  Lines have
817the following structure::
818
819   module ... [sourcefile ...] [cpparg ...] [library ...]
820
821
822Let's examine each of the fields in turn.
823
824* *module* is the name of the extension module to be built, and should be a
825  valid Python identifier.  You can't just change this in order to rename a module
826  (edits to the source code would also be needed), so this should be left alone.
827
828* *sourcefile* is anything that's likely to be a source code file, at least
829  judging by the filename.  Filenames ending in :file:`.c` are assumed to be
830  written in C, filenames ending in :file:`.C`, :file:`.cc`, and :file:`.c++` are
831  assumed to be C++, and filenames ending in :file:`.m` or :file:`.mm` are assumed
832  to be in Objective C.
833
834* *cpparg* is an argument for the C preprocessor,  and is anything starting with
835  :option:`-I`, :option:`-D`, :option:`-U` or :option:`-C`.
836
837* *library* is anything ending in :file:`.a` or beginning with :option:`-l` or
838  :option:`-L`.
839
840If a particular platform requires a special library on your platform, you can
841add it by editing the :file:`Setup` file and running ``python setup.py build``.
842For example, if the module defined by the line ::
843
844   foo foomodule.c
845
846must be linked with the math library :file:`libm.a` on your platform, simply add
847:option:`-lm` to the line::
848
849   foo foomodule.c -lm
850
851Arbitrary switches intended for the compiler or the linker can be supplied with
852the :option:`-Xcompiler` *arg* and :option:`-Xlinker` *arg* options::
853
854   foo foomodule.c -Xcompiler -o32 -Xlinker -shared -lm
855
856The next option after :option:`-Xcompiler` and :option:`-Xlinker` will be
857appended to the proper command line, so in the above example the compiler will
858be passed the :option:`-o32` option, and the linker will be passed
859:option:`-shared`.  If a compiler option requires an argument, you'll have to
860supply multiple :option:`-Xcompiler` options; for example, to pass ``-x c++``
861the :file:`Setup` file would have to contain ``-Xcompiler -x -Xcompiler c++``.
862
863Compiler flags can also be supplied through setting the :envvar:`CFLAGS`
864environment variable.  If set, the contents of :envvar:`CFLAGS` will be added to
865the compiler flags specified in the  :file:`Setup` file.
866
867
868.. _inst-non-ms-compilers:
869
870Using non-Microsoft compilers on Windows
871----------------------------------------
872
873.. sectionauthor:: Rene Liebscher <R.Liebscher@gmx.de>
874
875
876
877Borland/CodeGear C++
878^^^^^^^^^^^^^^^^^^^^
879
880This subsection describes the necessary steps to use Distutils with the Borland
881C++ compiler version 5.5.  First you have to know that Borland's object file
882format (OMF) is different from the format used by the Python version you can
883download from the Python or ActiveState Web site.  (Python is built with
884Microsoft Visual C++, which uses COFF as the object file format.) For this
885reason you have to convert Python's library :file:`python25.lib` into the
886Borland format.  You can do this as follows:
887
888.. Should we mention that users have to create cfg-files for the compiler?
889.. see also http://community.borland.com/article/0,1410,21205,00.html
890
891::
892
893   coff2omf python25.lib python25_bcpp.lib
894
895The :file:`coff2omf` program comes with the Borland compiler.  The file
896:file:`python25.lib` is in the :file:`Libs` directory of your Python
897installation.  If your extension uses other libraries (zlib, ...) you have to
898convert them too.
899
900The converted files have to reside in the same directories as the normal
901libraries.
902
903How does Distutils manage to use these libraries with their changed names?  If
904the extension needs a library (eg. :file:`foo`) Distutils checks first if it
905finds a library with suffix :file:`_bcpp` (eg. :file:`foo_bcpp.lib`) and then
906uses this library.  In the case it doesn't find such a special library it uses
907the default name (:file:`foo.lib`.) [#]_
908
909To let Distutils compile your extension with Borland C++ you now have to type::
910
911   python setup.py build --compiler=bcpp
912
913If you want to use the Borland C++ compiler as the default, you could specify
914this in your personal or system-wide configuration file for Distutils (see
915section :ref:`inst-config-files`.)
916
917
918.. seealso::
919
920   `C++Builder Compiler <http://www.codegear.com/downloads/free/cppbuilder>`_
921      Information about the free C++ compiler from Borland, including links to the
922      download pages.
923
924   `Creating Python Extensions Using Borland's Free Compiler <http://www.cyberus.ca/~g_will/pyExtenDL.shtml>`_
925      Document describing how to use Borland's free command-line C++ compiler to build
926      Python.
927
928
929GNU C / Cygwin / MinGW
930^^^^^^^^^^^^^^^^^^^^^^
931
932These instructions only apply if you're using a version of Python prior  to
9332.4.1 with a MinGW prior to 3.0.0 (with binutils-2.13.90-20030111-1).
934
935This section describes the necessary steps to use Distutils with the GNU C/C++
936compilers in their Cygwin and MinGW distributions. [#]_ For a Python interpreter
937that was built with Cygwin, everything should work without any of these
938following steps.
939
940These compilers require some special libraries. This task is more complex than
941for Borland's C++, because there is no program to convert the library.  First
942you have to create a list of symbols which the Python DLL exports. (You can find
943a good program for this task at http://www.emmestech.com/software/cygwin/pexports-0.43/download_pexports.html)
944
945.. I don't understand what the next line means. --amk
946.. (inclusive the references on data structures.)
947
948::
949
950   pexports python25.dll >python25.def
951
952The location of an installed :file:`python25.dll` will depend on the
953installation options and the version and language of Windows.  In a "just for
954me" installation, it will appear in the root of the installation directory.  In
955a shared installation, it will be located in the system directory.
956
957Then you can create from these information an import library for gcc. ::
958
959   /cygwin/bin/dlltool --dllname python25.dll --def python25.def --output-lib libpython25.a
960
961The resulting library has to be placed in the same directory as
962:file:`python25.lib`. (Should be the :file:`libs` directory under your Python
963installation directory.)
964
965If your extension uses other libraries (zlib,...) you might  have to convert
966them too. The converted files have to reside in the same directories as the
967normal libraries do.
968
969To let Distutils compile your extension with Cygwin you now have to type ::
970
971   python setup.py build --compiler=cygwin
972
973and for Cygwin in no-cygwin mode [#]_ or for MinGW type::
974
975   python setup.py build --compiler=mingw32
976
977If you want to use any of these options/compilers as default, you should
978consider to write it in your personal or system-wide configuration file for
979Distutils (see section :ref:`inst-config-files`.)
980
981
982.. seealso::
983
984   `Building Python modules on MS Windows platform with MinGW <http://www.zope.org/Members/als/tips/win32_mingw_modules>`_
985      Information about building the required libraries for the MinGW environment.
986
987
988.. rubric:: Footnotes
989
990.. [#] This also means you could replace all existing COFF-libraries with OMF-libraries
991   of the same name.
992
993.. [#] Check http://sources.redhat.com/cygwin/ and http://www.mingw.org/ for more
994   information
995
996.. [#] Then you have no POSIX emulation available, but you also don't need
997   :file:`cygwin1.dll`.