PageRenderTime 82ms CodeModel.GetById 4ms app.highlight 50ms RepoModel.GetById 2ms app.codeStats 0ms

/Doc/library/multiprocessing.rst

http://unladen-swallow.googlecode.com/
ReStructuredText | 2225 lines | 1520 code | 705 blank | 0 comment | 0 complexity | 47505cbd9f34fec5d0574573b59de8ba MD5 | raw file

Large files files are truncated, but you can click here to view the full file

   1:mod:`multiprocessing` --- Process-based "threading" interface
   2==============================================================
   3
   4.. module:: multiprocessing
   5   :synopsis: Process-based "threading" interface.
   6
   7.. versionadded:: 2.6
   8
   9
  10Introduction
  11----------------------
  12
  13:mod:`multiprocessing` is a package that supports spawning processes using an
  14API similar to the :mod:`threading` module.  The :mod:`multiprocessing` package
  15offers both local and remote concurrency, effectively side-stepping the
  16:term:`Global Interpreter Lock` by using subprocesses instead of threads.  Due
  17to this, the :mod:`multiprocessing` module allows the programmer to fully
  18leverage multiple processors on a given machine.  It runs on both Unix and
  19Windows.
  20
  21.. warning::
  22
  23    Some of this package's functionality requires a functioning shared semaphore
  24    implementation on the host operating system. Without one, the
  25    :mod:`multiprocessing.synchronize` module will be disabled, and attempts to
  26    import it will result in an :exc:`ImportError`. See
  27    :issue:`3770` for additional information.
  28
  29.. note::
  30
  31    Functionality within this package requires that the ``__main__`` method be
  32    importable by the children. This is covered in :ref:`multiprocessing-programming`
  33    however it is worth pointing out here. This means that some examples, such
  34    as the :class:`multiprocessing.Pool` examples will not work in the
  35    interactive interpreter. For example::
  36
  37        >>> from multiprocessing import Pool
  38        >>> p = Pool(5)
  39        >>> def f(x):
  40        ...     return x*x
  41        ...
  42        >>> p.map(f, [1,2,3])
  43        Process PoolWorker-1:
  44        Process PoolWorker-2:
  45        Process PoolWorker-3:
  46        Traceback (most recent call last):
  47        Traceback (most recent call last):
  48        Traceback (most recent call last):
  49        AttributeError: 'module' object has no attribute 'f'
  50        AttributeError: 'module' object has no attribute 'f'
  51        AttributeError: 'module' object has no attribute 'f'
  52
  53    (If you try this it will actually output three full tracebacks
  54    interleaved in a semi-random fashion, and then you may have to
  55    stop the master process somehow.)
  56
  57
  58The :class:`Process` class
  59~~~~~~~~~~~~~~~~~~~~~~~~~~
  60
  61In :mod:`multiprocessing`, processes are spawned by creating a :class:`Process`
  62object and then calling its :meth:`~Process.start` method.  :class:`Process`
  63follows the API of :class:`threading.Thread`.  A trivial example of a
  64multiprocess program is ::
  65
  66    from multiprocessing import Process
  67
  68    def f(name):
  69        print 'hello', name
  70
  71    if __name__ == '__main__':
  72        p = Process(target=f, args=('bob',))
  73        p.start()
  74        p.join()
  75
  76To show the individual process IDs involved, here is an expanded example::
  77
  78    from multiprocessing import Process
  79    import os
  80
  81    def info(title):
  82        print title
  83        print 'module name:', __name__
  84        print 'parent process:', os.getppid()
  85        print 'process id:', os.getpid()
  86
  87    def f(name):
  88        info('function f')
  89        print 'hello', name
  90
  91    if __name__ == '__main__':
  92        info('main line')
  93        p = Process(target=f, args=('bob',))
  94        p.start()
  95        p.join()
  96
  97For an explanation of why (on Windows) the ``if __name__ == '__main__'`` part is
  98necessary, see :ref:`multiprocessing-programming`.
  99
 100
 101
 102Exchanging objects between processes
 103~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 104
 105:mod:`multiprocessing` supports two types of communication channel between
 106processes:
 107
 108**Queues**
 109
 110   The :class:`Queue` class is a near clone of :class:`Queue.Queue`.  For
 111   example::
 112
 113      from multiprocessing import Process, Queue
 114
 115      def f(q):
 116          q.put([42, None, 'hello'])
 117
 118      if __name__ == '__main__':
 119          q = Queue()
 120          p = Process(target=f, args=(q,))
 121          p.start()
 122          print q.get()    # prints "[42, None, 'hello']"
 123          p.join()
 124
 125   Queues are thread and process safe.
 126
 127**Pipes**
 128
 129   The :func:`Pipe` function returns a pair of connection objects connected by a
 130   pipe which by default is duplex (two-way).  For example::
 131
 132      from multiprocessing import Process, Pipe
 133
 134      def f(conn):
 135          conn.send([42, None, 'hello'])
 136          conn.close()
 137
 138      if __name__ == '__main__':
 139          parent_conn, child_conn = Pipe()
 140          p = Process(target=f, args=(child_conn,))
 141          p.start()
 142          print parent_conn.recv()   # prints "[42, None, 'hello']"
 143          p.join()
 144
 145   The two connection objects returned by :func:`Pipe` represent the two ends of
 146   the pipe.  Each connection object has :meth:`~Connection.send` and
 147   :meth:`~Connection.recv` methods (among others).  Note that data in a pipe
 148   may become corrupted if two processes (or threads) try to read from or write
 149   to the *same* end of the pipe at the same time.  Of course there is no risk
 150   of corruption from processes using different ends of the pipe at the same
 151   time.
 152
 153
 154Synchronization between processes
 155~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 156
 157:mod:`multiprocessing` contains equivalents of all the synchronization
 158primitives from :mod:`threading`.  For instance one can use a lock to ensure
 159that only one process prints to standard output at a time::
 160
 161   from multiprocessing import Process, Lock
 162
 163   def f(l, i):
 164       l.acquire()
 165       print 'hello world', i
 166       l.release()
 167
 168   if __name__ == '__main__':
 169       lock = Lock()
 170
 171       for num in range(10):
 172           Process(target=f, args=(lock, num)).start()
 173
 174Without using the lock output from the different processes is liable to get all
 175mixed up.
 176
 177
 178Sharing state between processes
 179~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 180
 181As mentioned above, when doing concurrent programming it is usually best to
 182avoid using shared state as far as possible.  This is particularly true when
 183using multiple processes.
 184
 185However, if you really do need to use some shared data then
 186:mod:`multiprocessing` provides a couple of ways of doing so.
 187
 188**Shared memory**
 189
 190   Data can be stored in a shared memory map using :class:`Value` or
 191   :class:`Array`.  For example, the following code ::
 192
 193      from multiprocessing import Process, Value, Array
 194
 195      def f(n, a):
 196          n.value = 3.1415927
 197          for i in range(len(a)):
 198              a[i] = -a[i]
 199
 200      if __name__ == '__main__':
 201          num = Value('d', 0.0)
 202          arr = Array('i', range(10))
 203
 204          p = Process(target=f, args=(num, arr))
 205          p.start()
 206          p.join()
 207
 208          print num.value
 209          print arr[:]
 210
 211   will print ::
 212
 213      3.1415927
 214      [0, -1, -2, -3, -4, -5, -6, -7, -8, -9]
 215
 216   The ``'d'`` and ``'i'`` arguments used when creating ``num`` and ``arr`` are
 217   typecodes of the kind used by the :mod:`array` module: ``'d'`` indicates a
 218   double precision float and ``'i'`` indicates a signed integer.  These shared
 219   objects will be process and thread safe.
 220
 221   For more flexibility in using shared memory one can use the
 222   :mod:`multiprocessing.sharedctypes` module which supports the creation of
 223   arbitrary ctypes objects allocated from shared memory.
 224
 225**Server process**
 226
 227   A manager object returned by :func:`Manager` controls a server process which
 228   holds Python objects and allows other processes to manipulate them using
 229   proxies.
 230
 231   A manager returned by :func:`Manager` will support types :class:`list`,
 232   :class:`dict`, :class:`Namespace`, :class:`Lock`, :class:`RLock`,
 233   :class:`Semaphore`, :class:`BoundedSemaphore`, :class:`Condition`,
 234   :class:`Event`, :class:`Queue`, :class:`Value` and :class:`Array`.  For
 235   example, ::
 236
 237      from multiprocessing import Process, Manager
 238
 239      def f(d, l):
 240          d[1] = '1'
 241          d['2'] = 2
 242          d[0.25] = None
 243          l.reverse()
 244
 245      if __name__ == '__main__':
 246          manager = Manager()
 247
 248          d = manager.dict()
 249          l = manager.list(range(10))
 250
 251          p = Process(target=f, args=(d, l))
 252          p.start()
 253          p.join()
 254
 255          print d
 256          print l
 257
 258   will print ::
 259
 260       {0.25: None, 1: '1', '2': 2}
 261       [9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
 262
 263   Server process managers are more flexible than using shared memory objects
 264   because they can be made to support arbitrary object types.  Also, a single
 265   manager can be shared by processes on different computers over a network.
 266   They are, however, slower than using shared memory.
 267
 268
 269Using a pool of workers
 270~~~~~~~~~~~~~~~~~~~~~~~
 271
 272The :class:`~multiprocessing.pool.Pool` class represents a pool of worker
 273processes.  It has methods which allows tasks to be offloaded to the worker
 274processes in a few different ways.
 275
 276For example::
 277
 278   from multiprocessing import Pool
 279
 280   def f(x):
 281       return x*x
 282
 283   if __name__ == '__main__':
 284       pool = Pool(processes=4)              # start 4 worker processes
 285       result = pool.apply_async(f, [10])     # evaluate "f(10)" asynchronously
 286       print result.get(timeout=1)           # prints "100" unless your computer is *very* slow
 287       print pool.map(f, range(10))          # prints "[0, 1, 4,..., 81]"
 288
 289
 290Reference
 291---------
 292
 293The :mod:`multiprocessing` package mostly replicates the API of the
 294:mod:`threading` module.
 295
 296
 297:class:`Process` and exceptions
 298~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 299
 300.. class:: Process([group[, target[, name[, args[, kwargs]]]]])
 301
 302   Process objects represent activity that is run in a separate process. The
 303   :class:`Process` class has equivalents of all the methods of
 304   :class:`threading.Thread`.
 305
 306   The constructor should always be called with keyword arguments. *group*
 307   should always be ``None``; it exists solely for compatibility with
 308   :class:`threading.Thread`.  *target* is the callable object to be invoked by
 309   the :meth:`run()` method.  It defaults to ``None``, meaning nothing is
 310   called. *name* is the process name.  By default, a unique name is constructed
 311   of the form 'Process-N\ :sub:`1`:N\ :sub:`2`:...:N\ :sub:`k`' where N\
 312   :sub:`1`,N\ :sub:`2`,...,N\ :sub:`k` is a sequence of integers whose length
 313   is determined by the *generation* of the process.  *args* is the argument
 314   tuple for the target invocation.  *kwargs* is a dictionary of keyword
 315   arguments for the target invocation.  By default, no arguments are passed to
 316   *target*.
 317
 318   If a subclass overrides the constructor, it must make sure it invokes the
 319   base class constructor (:meth:`Process.__init__`) before doing anything else
 320   to the process.
 321
 322   .. method:: run()
 323
 324      Method representing the process's activity.
 325
 326      You may override this method in a subclass.  The standard :meth:`run`
 327      method invokes the callable object passed to the object's constructor as
 328      the target argument, if any, with sequential and keyword arguments taken
 329      from the *args* and *kwargs* arguments, respectively.
 330
 331   .. method:: start()
 332
 333      Start the process's activity.
 334
 335      This must be called at most once per process object.  It arranges for the
 336      object's :meth:`run` method to be invoked in a separate process.
 337
 338   .. method:: join([timeout])
 339
 340      Block the calling thread until the process whose :meth:`join` method is
 341      called terminates or until the optional timeout occurs.
 342
 343      If *timeout* is ``None`` then there is no timeout.
 344
 345      A process can be joined many times.
 346
 347      A process cannot join itself because this would cause a deadlock.  It is
 348      an error to attempt to join a process before it has been started.
 349
 350   .. attribute:: name
 351
 352      The process's name.
 353
 354      The name is a string used for identification purposes only.  It has no
 355      semantics.  Multiple processes may be given the same name.  The initial
 356      name is set by the constructor.
 357
 358   .. method:: is_alive
 359
 360      Return whether the process is alive.
 361
 362      Roughly, a process object is alive from the moment the :meth:`start`
 363      method returns until the child process terminates.
 364
 365   .. attribute:: daemon
 366
 367      The process's daemon flag, a Boolean value.  This must be set before
 368      :meth:`start` is called.
 369
 370      The initial value is inherited from the creating process.
 371
 372      When a process exits, it attempts to terminate all of its daemonic child
 373      processes.
 374
 375      Note that a daemonic process is not allowed to create child processes.
 376      Otherwise a daemonic process would leave its children orphaned if it gets
 377      terminated when its parent process exits.
 378
 379   In addition to the  :class:`Threading.Thread` API, :class:`Process` objects
 380   also support the following attributes and methods:
 381
 382   .. attribute:: pid
 383
 384      Return the process ID.  Before the process is spawned, this will be
 385      ``None``.
 386
 387   .. attribute:: exitcode
 388
 389      The child's exit code.  This will be ``None`` if the process has not yet
 390      terminated.  A negative value *-N* indicates that the child was terminated
 391      by signal *N*.
 392
 393   .. attribute:: authkey
 394
 395      The process's authentication key (a byte string).
 396
 397      When :mod:`multiprocessing` is initialized the main process is assigned a
 398      random string using :func:`os.random`.
 399
 400      When a :class:`Process` object is created, it will inherit the
 401      authentication key of its parent process, although this may be changed by
 402      setting :attr:`authkey` to another byte string.
 403
 404      See :ref:`multiprocessing-auth-keys`.
 405
 406   .. method:: terminate()
 407
 408      Terminate the process.  On Unix this is done using the ``SIGTERM`` signal;
 409      on Windows :cfunc:`TerminateProcess` is used.  Note that exit handlers and
 410      finally clauses, etc., will not be executed.
 411
 412      Note that descendant processes of the process will *not* be terminated --
 413      they will simply become orphaned.
 414
 415      .. warning::
 416
 417         If this method is used when the associated process is using a pipe or
 418         queue then the pipe or queue is liable to become corrupted and may
 419         become unusable by other process.  Similarly, if the process has
 420         acquired a lock or semaphore etc. then terminating it is liable to
 421         cause other processes to deadlock.
 422
 423   Note that the :meth:`start`, :meth:`join`, :meth:`is_alive` and
 424   :attr:`exit_code` methods should only be called by the process that created
 425   the process object.
 426
 427   Example usage of some of the methods of :class:`Process`:
 428
 429   .. doctest::
 430
 431       >>> import multiprocessing, time, signal
 432       >>> p = multiprocessing.Process(target=time.sleep, args=(1000,))
 433       >>> print p, p.is_alive()
 434       <Process(Process-1, initial)> False
 435       >>> p.start()
 436       >>> print p, p.is_alive()
 437       <Process(Process-1, started)> True
 438       >>> p.terminate()
 439       >>> time.sleep(0.1)
 440       >>> print p, p.is_alive()
 441       <Process(Process-1, stopped[SIGTERM])> False
 442       >>> p.exitcode == -signal.SIGTERM
 443       True
 444
 445
 446.. exception:: BufferTooShort
 447
 448   Exception raised by :meth:`Connection.recv_bytes_into()` when the supplied
 449   buffer object is too small for the message read.
 450
 451   If ``e`` is an instance of :exc:`BufferTooShort` then ``e.args[0]`` will give
 452   the message as a byte string.
 453
 454
 455Pipes and Queues
 456~~~~~~~~~~~~~~~~
 457
 458When using multiple processes, one generally uses message passing for
 459communication between processes and avoids having to use any synchronization
 460primitives like locks.
 461
 462For passing messages one can use :func:`Pipe` (for a connection between two
 463processes) or a queue (which allows multiple producers and consumers).
 464
 465The :class:`Queue` and :class:`JoinableQueue` types are multi-producer,
 466multi-consumer FIFO queues modelled on the :class:`Queue.Queue` class in the
 467standard library.  They differ in that :class:`Queue` lacks the
 468:meth:`~Queue.Queue.task_done` and :meth:`~Queue.Queue.join` methods introduced
 469into Python 2.5's :class:`Queue.Queue` class.
 470
 471If you use :class:`JoinableQueue` then you **must** call
 472:meth:`JoinableQueue.task_done` for each task removed from the queue or else the
 473semaphore used to count the number of unfinished tasks may eventually overflow
 474raising an exception.
 475
 476Note that one can also create a shared queue by using a manager object -- see
 477:ref:`multiprocessing-managers`.
 478
 479.. note::
 480
 481   :mod:`multiprocessing` uses the usual :exc:`Queue.Empty` and
 482   :exc:`Queue.Full` exceptions to signal a timeout.  They are not available in
 483   the :mod:`multiprocessing` namespace so you need to import them from
 484   :mod:`Queue`.
 485
 486
 487.. warning::
 488
 489   If a process is killed using :meth:`Process.terminate` or :func:`os.kill`
 490   while it is trying to use a :class:`Queue`, then the data in the queue is
 491   likely to become corrupted.  This may cause any other processes to get an
 492   exception when it tries to use the queue later on.
 493
 494.. warning::
 495
 496   As mentioned above, if a child process has put items on a queue (and it has
 497   not used :meth:`JoinableQueue.cancel_join_thread`), then that process will
 498   not terminate until all buffered items have been flushed to the pipe.
 499
 500   This means that if you try joining that process you may get a deadlock unless
 501   you are sure that all items which have been put on the queue have been
 502   consumed.  Similarly, if the child process is non-daemonic then the parent
 503   process may hang on exit when it tries to join all its non-daemonic children.
 504
 505   Note that a queue created using a manager does not have this issue.  See
 506   :ref:`multiprocessing-programming`.
 507
 508For an example of the usage of queues for interprocess communication see
 509:ref:`multiprocessing-examples`.
 510
 511
 512.. function:: Pipe([duplex])
 513
 514   Returns a pair ``(conn1, conn2)`` of :class:`Connection` objects representing
 515   the ends of a pipe.
 516
 517   If *duplex* is ``True`` (the default) then the pipe is bidirectional.  If
 518   *duplex* is ``False`` then the pipe is unidirectional: ``conn1`` can only be
 519   used for receiving messages and ``conn2`` can only be used for sending
 520   messages.
 521
 522
 523.. class:: Queue([maxsize])
 524
 525   Returns a process shared queue implemented using a pipe and a few
 526   locks/semaphores.  When a process first puts an item on the queue a feeder
 527   thread is started which transfers objects from a buffer into the pipe.
 528
 529   The usual :exc:`Queue.Empty` and :exc:`Queue.Full` exceptions from the
 530   standard library's :mod:`Queue` module are raised to signal timeouts.
 531
 532   :class:`Queue` implements all the methods of :class:`Queue.Queue` except for
 533   :meth:`~Queue.Queue.task_done` and :meth:`~Queue.Queue.join`.
 534
 535   .. method:: qsize()
 536
 537      Return the approximate size of the queue.  Because of
 538      multithreading/multiprocessing semantics, this number is not reliable.
 539
 540      Note that this may raise :exc:`NotImplementedError` on Unix platforms like
 541      Mac OS X where ``sem_getvalue()`` is not implemented.
 542
 543   .. method:: empty()
 544
 545      Return ``True`` if the queue is empty, ``False`` otherwise.  Because of
 546      multithreading/multiprocessing semantics, this is not reliable.
 547
 548   .. method:: full()
 549
 550      Return ``True`` if the queue is full, ``False`` otherwise.  Because of
 551      multithreading/multiprocessing semantics, this is not reliable.
 552
 553   .. method:: put(item[, block[, timeout]])
 554
 555      Put item into the queue.  If the optional argument *block* is ``True``
 556      (the default) and *timeout* is ``None`` (the default), block if necessary until
 557      a free slot is available.  If *timeout* is a positive number, it blocks at
 558      most *timeout* seconds and raises the :exc:`Queue.Full` exception if no
 559      free slot was available within that time.  Otherwise (*block* is
 560      ``False``), put an item on the queue if a free slot is immediately
 561      available, else raise the :exc:`Queue.Full` exception (*timeout* is
 562      ignored in that case).
 563
 564   .. method:: put_nowait(item)
 565
 566      Equivalent to ``put(item, False)``.
 567
 568   .. method:: get([block[, timeout]])
 569
 570      Remove and return an item from the queue.  If optional args *block* is
 571      ``True`` (the default) and *timeout* is ``None`` (the default), block if
 572      necessary until an item is available.  If *timeout* is a positive number,
 573      it blocks at most *timeout* seconds and raises the :exc:`Queue.Empty`
 574      exception if no item was available within that time.  Otherwise (block is
 575      ``False``), return an item if one is immediately available, else raise the
 576      :exc:`Queue.Empty` exception (*timeout* is ignored in that case).
 577
 578   .. method:: get_nowait()
 579               get_no_wait()
 580
 581      Equivalent to ``get(False)``.
 582
 583   :class:`multiprocessing.Queue` has a few additional methods not found in
 584   :class:`Queue.Queue`.  These methods are usually unnecessary for most
 585   code:
 586
 587   .. method:: close()
 588
 589      Indicate that no more data will be put on this queue by the current
 590      process.  The background thread will quit once it has flushed all buffered
 591      data to the pipe.  This is called automatically when the queue is garbage
 592      collected.
 593
 594   .. method:: join_thread()
 595
 596      Join the background thread.  This can only be used after :meth:`close` has
 597      been called.  It blocks until the background thread exits, ensuring that
 598      all data in the buffer has been flushed to the pipe.
 599
 600      By default if a process is not the creator of the queue then on exit it
 601      will attempt to join the queue's background thread.  The process can call
 602      :meth:`cancel_join_thread` to make :meth:`join_thread` do nothing.
 603
 604   .. method:: cancel_join_thread()
 605
 606      Prevent :meth:`join_thread` from blocking.  In particular, this prevents
 607      the background thread from being joined automatically when the process
 608      exits -- see :meth:`join_thread`.
 609
 610
 611.. class:: JoinableQueue([maxsize])
 612
 613   :class:`JoinableQueue`, a :class:`Queue` subclass, is a queue which
 614   additionally has :meth:`task_done` and :meth:`join` methods.
 615
 616   .. method:: task_done()
 617
 618      Indicate that a formerly enqueued task is complete. Used by queue consumer
 619      threads.  For each :meth:`~Queue.get` used to fetch a task, a subsequent
 620      call to :meth:`task_done` tells the queue that the processing on the task
 621      is complete.
 622
 623      If a :meth:`~Queue.join` is currently blocking, it will resume when all
 624      items have been processed (meaning that a :meth:`task_done` call was
 625      received for every item that had been :meth:`~Queue.put` into the queue).
 626
 627      Raises a :exc:`ValueError` if called more times than there were items
 628      placed in the queue.
 629
 630
 631   .. method:: join()
 632
 633      Block until all items in the queue have been gotten and processed.
 634
 635      The count of unfinished tasks goes up whenever an item is added to the
 636      queue.  The count goes down whenever a consumer thread calls
 637      :meth:`task_done` to indicate that the item was retrieved and all work on
 638      it is complete.  When the count of unfinished tasks drops to zero,
 639      :meth:`~Queue.join` unblocks.
 640
 641
 642Miscellaneous
 643~~~~~~~~~~~~~
 644
 645.. function:: active_children()
 646
 647   Return list of all live children of the current process.
 648
 649   Calling this has the side affect of "joining" any processes which have
 650   already finished.
 651
 652.. function:: cpu_count()
 653
 654   Return the number of CPUs in the system.  May raise
 655   :exc:`NotImplementedError`.
 656
 657.. function:: current_process()
 658
 659   Return the :class:`Process` object corresponding to the current process.
 660
 661   An analogue of :func:`threading.current_thread`.
 662
 663.. function:: freeze_support()
 664
 665   Add support for when a program which uses :mod:`multiprocessing` has been
 666   frozen to produce a Windows executable.  (Has been tested with **py2exe**,
 667   **PyInstaller** and **cx_Freeze**.)
 668
 669   One needs to call this function straight after the ``if __name__ ==
 670   '__main__'`` line of the main module.  For example::
 671
 672      from multiprocessing import Process, freeze_support
 673
 674      def f():
 675          print 'hello world!'
 676
 677      if __name__ == '__main__':
 678          freeze_support()
 679          Process(target=f).start()
 680
 681   If the ``freeze_support()`` line is omitted then trying to run the frozen
 682   executable will raise :exc:`RuntimeError`.
 683
 684   If the module is being run normally by the Python interpreter then
 685   :func:`freeze_support` has no effect.
 686
 687.. function:: set_executable()
 688
 689   Sets the path of the python interpreter to use when starting a child process.
 690   (By default :data:`sys.executable` is used).  Embedders will probably need to
 691   do some thing like ::
 692
 693      setExecutable(os.path.join(sys.exec_prefix, 'pythonw.exe'))
 694
 695   before they can create child processes.  (Windows only)
 696
 697
 698.. note::
 699
 700   :mod:`multiprocessing` contains no analogues of
 701   :func:`threading.active_count`, :func:`threading.enumerate`,
 702   :func:`threading.settrace`, :func:`threading.setprofile`,
 703   :class:`threading.Timer`, or :class:`threading.local`.
 704
 705
 706Connection Objects
 707~~~~~~~~~~~~~~~~~~
 708
 709Connection objects allow the sending and receiving of picklable objects or
 710strings.  They can be thought of as message oriented connected sockets.
 711
 712Connection objects usually created using :func:`Pipe` -- see also
 713:ref:`multiprocessing-listeners-clients`.
 714
 715.. class:: Connection
 716
 717   .. method:: send(obj)
 718
 719      Send an object to the other end of the connection which should be read
 720      using :meth:`recv`.
 721
 722      The object must be picklable.
 723
 724   .. method:: recv()
 725
 726      Return an object sent from the other end of the connection using
 727      :meth:`send`.  Raises :exc:`EOFError` if there is nothing left to receive
 728      and the other end was closed.
 729
 730   .. method:: fileno()
 731
 732      Returns the file descriptor or handle used by the connection.
 733
 734   .. method:: close()
 735
 736      Close the connection.
 737
 738      This is called automatically when the connection is garbage collected.
 739
 740   .. method:: poll([timeout])
 741
 742      Return whether there is any data available to be read.
 743
 744      If *timeout* is not specified then it will return immediately.  If
 745      *timeout* is a number then this specifies the maximum time in seconds to
 746      block.  If *timeout* is ``None`` then an infinite timeout is used.
 747
 748   .. method:: send_bytes(buffer[, offset[, size]])
 749
 750      Send byte data from an object supporting the buffer interface as a
 751      complete message.
 752
 753      If *offset* is given then data is read from that position in *buffer*.  If
 754      *size* is given then that many bytes will be read from buffer.
 755
 756   .. method:: recv_bytes([maxlength])
 757
 758      Return a complete message of byte data sent from the other end of the
 759      connection as a string.  Raises :exc:`EOFError` if there is nothing left
 760      to receive and the other end has closed.
 761
 762      If *maxlength* is specified and the message is longer than *maxlength*
 763      then :exc:`IOError` is raised and the connection will no longer be
 764      readable.
 765
 766   .. method:: recv_bytes_into(buffer[, offset])
 767
 768      Read into *buffer* a complete message of byte data sent from the other end
 769      of the connection and return the number of bytes in the message.  Raises
 770      :exc:`EOFError` if there is nothing left to receive and the other end was
 771      closed.
 772
 773      *buffer* must be an object satisfying the writable buffer interface.  If
 774      *offset* is given then the message will be written into the buffer from
 775      that position.  Offset must be a non-negative integer less than the
 776      length of *buffer* (in bytes).
 777
 778      If the buffer is too short then a :exc:`BufferTooShort` exception is
 779      raised and the complete message is available as ``e.args[0]`` where ``e``
 780      is the exception instance.
 781
 782
 783For example:
 784
 785.. doctest::
 786
 787    >>> from multiprocessing import Pipe
 788    >>> a, b = Pipe()
 789    >>> a.send([1, 'hello', None])
 790    >>> b.recv()
 791    [1, 'hello', None]
 792    >>> b.send_bytes('thank you')
 793    >>> a.recv_bytes()
 794    'thank you'
 795    >>> import array
 796    >>> arr1 = array.array('i', range(5))
 797    >>> arr2 = array.array('i', [0] * 10)
 798    >>> a.send_bytes(arr1)
 799    >>> count = b.recv_bytes_into(arr2)
 800    >>> assert count == len(arr1) * arr1.itemsize
 801    >>> arr2
 802    array('i', [0, 1, 2, 3, 4, 0, 0, 0, 0, 0])
 803
 804
 805.. warning::
 806
 807    The :meth:`Connection.recv` method automatically unpickles the data it
 808    receives, which can be a security risk unless you can trust the process
 809    which sent the message.
 810
 811    Therefore, unless the connection object was produced using :func:`Pipe` you
 812    should only use the :meth:`~Connection.recv` and :meth:`~Connection.send`
 813    methods after performing some sort of authentication.  See
 814    :ref:`multiprocessing-auth-keys`.
 815
 816.. warning::
 817
 818    If a process is killed while it is trying to read or write to a pipe then
 819    the data in the pipe is likely to become corrupted, because it may become
 820    impossible to be sure where the message boundaries lie.
 821
 822
 823Synchronization primitives
 824~~~~~~~~~~~~~~~~~~~~~~~~~~
 825
 826Generally synchronization primitives are not as necessary in a multiprocess
 827program as they are in a multithreaded program.  See the documentation for
 828:mod:`threading` module.
 829
 830Note that one can also create synchronization primitives by using a manager
 831object -- see :ref:`multiprocessing-managers`.
 832
 833.. class:: BoundedSemaphore([value])
 834
 835   A bounded semaphore object: a clone of :class:`threading.BoundedSemaphore`.
 836
 837   (On Mac OS X this is indistinguishable from :class:`Semaphore` because
 838   ``sem_getvalue()`` is not implemented on that platform).
 839
 840.. class:: Condition([lock])
 841
 842   A condition variable: a clone of :class:`threading.Condition`.
 843
 844   If *lock* is specified then it should be a :class:`Lock` or :class:`RLock`
 845   object from :mod:`multiprocessing`.
 846
 847.. class:: Event()
 848
 849   A clone of :class:`threading.Event`.
 850
 851.. class:: Lock()
 852
 853   A non-recursive lock object: a clone of :class:`threading.Lock`.
 854
 855.. class:: RLock()
 856
 857   A recursive lock object: a clone of :class:`threading.RLock`.
 858
 859.. class:: Semaphore([value])
 860
 861   A bounded semaphore object: a clone of :class:`threading.Semaphore`.
 862
 863.. note::
 864
 865   The :meth:`acquire` method of :class:`BoundedSemaphore`, :class:`Lock`,
 866   :class:`RLock` and :class:`Semaphore` has a timeout parameter not supported
 867   by the equivalents in :mod:`threading`.  The signature is
 868   ``acquire(block=True, timeout=None)`` with keyword parameters being
 869   acceptable.  If *block* is ``True`` and *timeout* is not ``None`` then it
 870   specifies a timeout in seconds.  If *block* is ``False`` then *timeout* is
 871   ignored.
 872
 873.. note::
 874   On OS/X ``sem_timedwait`` is unsupported, so timeout arguments for the
 875   aforementioned :meth:`acquire` methods will be ignored on OS/X.
 876
 877.. note::
 878
 879   If the SIGINT signal generated by Ctrl-C arrives while the main thread is
 880   blocked by a call to :meth:`BoundedSemaphore.acquire`, :meth:`Lock.acquire`,
 881   :meth:`RLock.acquire`, :meth:`Semaphore.acquire`, :meth:`Condition.acquire`
 882   or :meth:`Condition.wait` then the call will be immediately interrupted and
 883   :exc:`KeyboardInterrupt` will be raised.
 884
 885   This differs from the behaviour of :mod:`threading` where SIGINT will be
 886   ignored while the equivalent blocking calls are in progress.
 887
 888
 889Shared :mod:`ctypes` Objects
 890~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 891
 892It is possible to create shared objects using shared memory which can be
 893inherited by child processes.
 894
 895.. function:: Value(typecode_or_type, *args[, lock])
 896
 897   Return a :mod:`ctypes` object allocated from shared memory.  By default the
 898   return value is actually a synchronized wrapper for the object.
 899
 900   *typecode_or_type* determines the type of the returned object: it is either a
 901   ctypes type or a one character typecode of the kind used by the :mod:`array`
 902   module.  *\*args* is passed on to the constructor for the type.
 903
 904   If *lock* is ``True`` (the default) then a new lock object is created to
 905   synchronize access to the value.  If *lock* is a :class:`Lock` or
 906   :class:`RLock` object then that will be used to synchronize access to the
 907   value.  If *lock* is ``False`` then access to the returned object will not be
 908   automatically protected by a lock, so it will not necessarily be
 909   "process-safe".
 910
 911   Note that *lock* is a keyword-only argument.
 912
 913.. function:: Array(typecode_or_type, size_or_initializer, *, lock=True)
 914
 915   Return a ctypes array allocated from shared memory.  By default the return
 916   value is actually a synchronized wrapper for the array.
 917
 918   *typecode_or_type* determines the type of the elements of the returned array:
 919   it is either a ctypes type or a one character typecode of the kind used by
 920   the :mod:`array` module.  If *size_or_initializer* is an integer, then it
 921   determines the length of the array, and the array will be initially zeroed.
 922   Otherwise, *size_or_initializer* is a sequence which is used to initialize
 923   the array and whose length determines the length of the array.
 924
 925   If *lock* is ``True`` (the default) then a new lock object is created to
 926   synchronize access to the value.  If *lock* is a :class:`Lock` or
 927   :class:`RLock` object then that will be used to synchronize access to the
 928   value.  If *lock* is ``False`` then access to the returned object will not be
 929   automatically protected by a lock, so it will not necessarily be
 930   "process-safe".
 931
 932   Note that *lock* is a keyword only argument.
 933
 934   Note that an array of :data:`ctypes.c_char` has *value* and *raw*
 935   attributes which allow one to use it to store and retrieve strings.
 936
 937
 938The :mod:`multiprocessing.sharedctypes` module
 939>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
 940
 941.. module:: multiprocessing.sharedctypes
 942   :synopsis: Allocate ctypes objects from shared memory.
 943
 944The :mod:`multiprocessing.sharedctypes` module provides functions for allocating
 945:mod:`ctypes` objects from shared memory which can be inherited by child
 946processes.
 947
 948.. note::
 949
 950   Although it is possible to store a pointer in shared memory remember that
 951   this will refer to a location in the address space of a specific process.
 952   However, the pointer is quite likely to be invalid in the context of a second
 953   process and trying to dereference the pointer from the second process may
 954   cause a crash.
 955
 956.. function:: RawArray(typecode_or_type, size_or_initializer)
 957
 958   Return a ctypes array allocated from shared memory.
 959
 960   *typecode_or_type* determines the type of the elements of the returned array:
 961   it is either a ctypes type or a one character typecode of the kind used by
 962   the :mod:`array` module.  If *size_or_initializer* is an integer then it
 963   determines the length of the array, and the array will be initially zeroed.
 964   Otherwise *size_or_initializer* is a sequence which is used to initialize the
 965   array and whose length determines the length of the array.
 966
 967   Note that setting and getting an element is potentially non-atomic -- use
 968   :func:`Array` instead to make sure that access is automatically synchronized
 969   using a lock.
 970
 971.. function:: RawValue(typecode_or_type, *args)
 972
 973   Return a ctypes object allocated from shared memory.
 974
 975   *typecode_or_type* determines the type of the returned object: it is either a
 976   ctypes type or a one character typecode of the kind used by the :mod:`array`
 977   module.  *\*args* is passed on to the constructor for the type.
 978
 979   Note that setting and getting the value is potentially non-atomic -- use
 980   :func:`Value` instead to make sure that access is automatically synchronized
 981   using a lock.
 982
 983   Note that an array of :data:`ctypes.c_char` has ``value`` and ``raw``
 984   attributes which allow one to use it to store and retrieve strings -- see
 985   documentation for :mod:`ctypes`.
 986
 987.. function:: Array(typecode_or_type, size_or_initializer, *args[, lock])
 988
 989   The same as :func:`RawArray` except that depending on the value of *lock* a
 990   process-safe synchronization wrapper may be returned instead of a raw ctypes
 991   array.
 992
 993   If *lock* is ``True`` (the default) then a new lock object is created to
 994   synchronize access to the value.  If *lock* is a :class:`Lock` or
 995   :class:`RLock` object then that will be used to synchronize access to the
 996   value.  If *lock* is ``False`` then access to the returned object will not be
 997   automatically protected by a lock, so it will not necessarily be
 998   "process-safe".
 999
1000   Note that *lock* is a keyword-only argument.
1001
1002.. function:: Value(typecode_or_type, *args[, lock])
1003
1004   The same as :func:`RawValue` except that depending on the value of *lock* a
1005   process-safe synchronization wrapper may be returned instead of a raw ctypes
1006   object.
1007
1008   If *lock* is ``True`` (the default) then a new lock object is created to
1009   synchronize access to the value.  If *lock* is a :class:`Lock` or
1010   :class:`RLock` object then that will be used to synchronize access to the
1011   value.  If *lock* is ``False`` then access to the returned object will not be
1012   automatically protected by a lock, so it will not necessarily be
1013   "process-safe".
1014
1015   Note that *lock* is a keyword-only argument.
1016
1017.. function:: copy(obj)
1018
1019   Return a ctypes object allocated from shared memory which is a copy of the
1020   ctypes object *obj*.
1021
1022.. function:: synchronized(obj[, lock])
1023
1024   Return a process-safe wrapper object for a ctypes object which uses *lock* to
1025   synchronize access.  If *lock* is ``None`` (the default) then a
1026   :class:`multiprocessing.RLock` object is created automatically.
1027
1028   A synchronized wrapper will have two methods in addition to those of the
1029   object it wraps: :meth:`get_obj` returns the wrapped object and
1030   :meth:`get_lock` returns the lock object used for synchronization.
1031
1032   Note that accessing the ctypes object through the wrapper can be a lot slower
1033   than accessing the raw ctypes object.
1034
1035
1036The table below compares the syntax for creating shared ctypes objects from
1037shared memory with the normal ctypes syntax.  (In the table ``MyStruct`` is some
1038subclass of :class:`ctypes.Structure`.)
1039
1040==================== ========================== ===========================
1041ctypes               sharedctypes using type    sharedctypes using typecode
1042==================== ========================== ===========================
1043c_double(2.4)        RawValue(c_double, 2.4)    RawValue('d', 2.4)
1044MyStruct(4, 6)       RawValue(MyStruct, 4, 6)
1045(c_short * 7)()      RawArray(c_short, 7)       RawArray('h', 7)
1046(c_int * 3)(9, 2, 8) RawArray(c_int, (9, 2, 8)) RawArray('i', (9, 2, 8))
1047==================== ========================== ===========================
1048
1049
1050Below is an example where a number of ctypes objects are modified by a child
1051process::
1052
1053   from multiprocessing import Process, Lock
1054   from multiprocessing.sharedctypes import Value, Array
1055   from ctypes import Structure, c_double
1056
1057   class Point(Structure):
1058       _fields_ = [('x', c_double), ('y', c_double)]
1059
1060   def modify(n, x, s, A):
1061       n.value **= 2
1062       x.value **= 2
1063       s.value = s.value.upper()
1064       for a in A:
1065           a.x **= 2
1066           a.y **= 2
1067
1068   if __name__ == '__main__':
1069       lock = Lock()
1070
1071       n = Value('i', 7)
1072       x = Value(c_double, 1.0/3.0, lock=False)
1073       s = Array('c', 'hello world', lock=lock)
1074       A = Array(Point, [(1.875,-6.25), (-5.75,2.0), (2.375,9.5)], lock=lock)
1075
1076       p = Process(target=modify, args=(n, x, s, A))
1077       p.start()
1078       p.join()
1079
1080       print n.value
1081       print x.value
1082       print s.value
1083       print [(a.x, a.y) for a in A]
1084
1085
1086.. highlightlang:: none
1087
1088The results printed are ::
1089
1090    49
1091    0.1111111111111111
1092    HELLO WORLD
1093    [(3.515625, 39.0625), (33.0625, 4.0), (5.640625, 90.25)]
1094
1095.. highlightlang:: python
1096
1097
1098.. _multiprocessing-managers:
1099
1100Managers
1101~~~~~~~~
1102
1103Managers provide a way to create data which can be shared between different
1104processes. A manager object controls a server process which manages *shared
1105objects*.  Other processes can access the shared objects by using proxies.
1106
1107.. function:: multiprocessing.Manager()
1108
1109   Returns a started :class:`~multiprocessing.managers.SyncManager` object which
1110   can be used for sharing objects between processes.  The returned manager
1111   object corresponds to a spawned child process and has methods which will
1112   create shared objects and return corresponding proxies.
1113
1114.. module:: multiprocessing.managers
1115   :synopsis: Share data between process with shared objects.
1116
1117Manager processes will be shutdown as soon as they are garbage collected or
1118their parent process exits.  The manager classes are defined in the
1119:mod:`multiprocessing.managers` module:
1120
1121.. class:: BaseManager([address[, authkey]])
1122
1123   Create a BaseManager object.
1124
1125   Once created one should call :meth:`start` or :meth:`serve_forever` to ensure
1126   that the manager object refers to a started manager process.
1127
1128   *address* is the address on which the manager process listens for new
1129   connections.  If *address* is ``None`` then an arbitrary one is chosen.
1130
1131   *authkey* is the authentication key which will be used to check the validity
1132   of incoming connections to the server process.  If *authkey* is ``None`` then
1133   ``current_process().authkey``.  Otherwise *authkey* is used and it
1134   must be a string.
1135
1136   .. method:: start()
1137
1138      Start a subprocess to start the manager.
1139
1140   .. method:: serve_forever()
1141
1142      Run the server in the current process.
1143
1144   .. method:: get_server()
1145
1146      Returns a :class:`Server` object which represents the actual server under
1147      the control of the Manager. The :class:`Server` object supports the
1148      :meth:`serve_forever` method::
1149
1150      >>> from multiprocessing.managers import BaseManager
1151      >>> manager = BaseManager(address=('', 50000), authkey='abc')
1152      >>> server = manager.get_server()
1153      >>> server.serve_forever()
1154
1155      :class:`Server` additionally has an :attr:`address` attribute.
1156
1157   .. method:: connect()
1158
1159      Connect a local manager object to a remote manager process::
1160
1161      >>> from multiprocessing.managers import BaseManager
1162      >>> m = BaseManager(address=('127.0.0.1', 5000), authkey='abc')
1163      >>> m.connect()
1164
1165   .. method:: shutdown()
1166
1167      Stop the process used by the manager.  This is only available if
1168      :meth:`start` has been used to start the server process.
1169
1170      This can be called multiple times.
1171
1172   .. method:: register(typeid[, callable[, proxytype[, exposed[, method_to_typeid[, create_method]]]]])
1173
1174      A classmethod which can be used for registering a type or callable with
1175      the manager class.
1176
1177      *typeid* is a "type identifier" which is used to identify a particular
1178      type of shared object.  This must be a string.
1179
1180      *callable* is a callable used for creating objects for this type
1181      identifier.  If a manager instance will be created using the
1182      :meth:`from_address` classmethod or if the *create_method* argument is
1183      ``False`` then this can be left as ``None``.
1184
1185      *proxytype* is a subclass of :class:`BaseProxy` which is used to create
1186      proxies for shared objects with this *typeid*.  If ``None`` then a proxy
1187      class is created automatically.
1188
1189      *exposed* is used to specify a sequence of method names which proxies for
1190      this typeid should be allowed to access using
1191      :meth:`BaseProxy._callMethod`.  (If *exposed* is ``None`` then
1192      :attr:`proxytype._exposed_` is used instead if it exists.)  In the case
1193      where no exposed list is specified, all "public methods" of the shared
1194      object will be accessible.  (Here a "public method" means any attribute
1195      which has a :meth:`__call__` method and whose name does not begin with
1196      ``'_'``.)
1197
1198      *method_to_typeid* is a mapping used to specify the return type of those
1199      exposed methods which should return a proxy.  It maps method names to
1200      typeid strings.  (If *method_to_typeid* is ``None`` then
1201      :attr:`proxytype._method_to_typeid_` is used instead if it exists.)  If a
1202      method's name is not a key of this mapping or if the mapping is ``None``
1203      then the object returned by the method will be copied by value.
1204
1205      *create_method* determines whether a method should be created with name
1206      *typeid* which can be used to tell the server process to create a new
1207      shared object and return a proxy for it.  By default it is ``True``.
1208
1209   :class:`BaseManager` instances also have one read-only property:
1210
1211   .. attribute:: address
1212
1213      The address used by the manager.
1214
1215
1216.. class:: SyncManager
1217
1218   A subclass of :class:`BaseManager` which can be used for the synchronization
1219   of processes.  Objects of this type are returned by
1220   :func:`multiprocessing.Manager`.
1221
1222   It also supports creation of shared lists and dictionaries.
1223
1224   .. method:: BoundedSemaphore([value])
1225
1226      Create a shared :class:`threading.BoundedSemaphore` object and return a
1227      proxy for it.
1228
1229   .. method:: Condition([lock])
1230
1231      Create a shared :class:`threading.Condition` object and return a proxy for
1232      it.
1233
1234      If *lock* is supplied then it should be a proxy for a
1235      :class:`threading.Lock` or :class:`threading.RLock` object.
1236
1237   .. method:: Event()
1238
1239      Create a shared :class:`threading.Event` object and return a proxy for it.
1240
1241   .. method:: Lock()
1242
1243      Create a shared :class:`threading.Lock` object and return a proxy for it.
1244
1245   .. method:: Namespace()
1246
1247      Create a shared :class:`Namespace` object and return a proxy for it.
1248
1249   .. method:: Queue([maxsize])
1250
1251      Create a shared :class:`Queue.Queue` object and return a proxy for it.
1252
1253   .. method:: RLock()
1254
1255      Create a shared :class:`threading.RLock` object and return a proxy for it.
1256
1257   .. method:: Semaphore([value])
1258
1259      Create a shared :class:`threading.Semaphore` object and return a proxy for
1260      it.
1261
1262   .. method:: Array(typecode, sequence)
1263
1264      Create an array and return a proxy for it.
1265
1266   .. method:: Value(typecode, value)
1267
1268      Create an object with a writable ``value`` attribute and return a proxy
1269      for it.
1270
1271   .. method:: dict()
1272               dict(mapping)
1273               dict(sequence)
1274
1275      Create a shared ``dict`` object and return a proxy for it.
1276
1277   .. method:: list()
1278               list(sequence)
1279
1280      Create a shared ``list`` object and return a proxy for it.
1281
1282
1283Namespace objects
1284>>>>>>>>>>>>>>>>>
1285
1286A namespace object has no public methods, but does have writable attributes.
1287Its representation shows the values of its attributes.
1288
1289However, when using a proxy for a namespace object, an attribute beginning with
1290``'_'`` will be an attribute of the proxy and not an attribute of the referent:
1291
1292.. doctest::
1293
1294   >>> manager = multiprocessing.Manager()
1295   >>> Global = manager.Namespace()
1296   >>> Global.x = 10
1297   >>> Global.y = 'hello'
1298   >>> Global._z = 12.3    # this is an attribute of the proxy
1299   >>> print Global
1300   Namespace(x=10, y='hello')
1301
1302
1303Customized managers
1304>>>>>>>>>>>>>>>>>>>
1305
1306To create one's own manager, one creates a subclass of :class:`BaseManager` and
1307use the :meth:`~BaseManager.register` classmethod to register new types or
1308callables with the manager class.  For example::
1309
1310   from multiprocessing.managers import BaseManager
1311
1312   class MathsClass(object):
1313       def add(self, x, y):
1314           return x + y
1315       def mul(self, x, y):
1316           return x * y
1317
1318   class MyManager(BaseManager):
1319       pass
1320
1321   MyManager.register('Maths', MathsClass)
1322
1323   if __name__ == '__main__':
1324       manager = MyManager()
1325       manager.start()
1326       maths = manager.Maths()
1327       print maths.add(4, 3)         # prints 7
1328       print maths.mul(7, 8)         # prints 56
1329
1330
1331Using a remote manager
1332>>>>>>>>>>>>>>>>>>>>>>
1333
1334It is possible to run a manager server on one machine and have clients use it
1335from other machines (assuming that the firewalls involved allow it).
1336
1337Running the following commands creates a server for a single shared queue which
1338remote clients can access::
1339
1340   >>> from multiprocessing.managers import BaseManager
1341   >>> import Queue
1342   >>> queue = Queue.Queue()
1343   >>> class QueueManager(BaseManager): pass
1344   >>> QueueManager.register('get_queue', callable=lambda:queue)
1345   >>> m = QueueManager(address=('', 50000), authkey='abracadabra')
1346   >>> s = m.get_server()
1347   >>> s.serve_forever()
1348
1349One client can access the server as follows::
1350
1351   >>> from multiprocessing.managers import BaseManager
1352   >>> class QueueManager(BaseManager): pass
1353   >>> QueueManager.register('get_queue')
1354   >>> m = QueueManager(address=('foo.bar.org', 50000), authkey='abracadabra')
1355   >>> m.connect()
1356   >>> queue = m.get_queue()
1357   >>> queue.put('hello')
1358
1359Another client can also use it::
1360
1361   >>> from multiprocessing.managers import BaseManager
1362   >>> class QueueManager(BaseManager): pass
1363   >>> QueueManager.register('get_queue')
1364   >>> m = QueueManager(address=('foo.bar.org', 50000), authkey='abracadabra')
1365   >>> m.connect()
1366   >>> queue = m.get_queue()
1367   >>> queue.get()
1368   'hello'
1369
1370Local processes can also access that queue, using the code from above on the
1371client to access it remotely::
1372
1373    >>> from multiprocessing import Process, Queue
1374    >>> from multiprocessing.managers import BaseManager
1375    >>> class Worker(Process):
1376    ...     def __init__(self, q):
1377    ...         self.q = q
1378    ...         super(Worker, self).__init__()
1379    ...     def run(self):
1380    ...         self.q.put('local hello')
1381    ...
1382    >>> queue = Queue()
1383    >>> w = Worker(queue)
1384    >>> w.start()
1385    >>> class QueueManager(BaseManager): pass
1386    ...
1387    >>> QueueManager.register('get_queue', callable=lambda: queue)
1388    >>> m = QueueManager(address=('', 50000), authkey='abracadabra')
1389    >>> s = m.get_server()
1390    >>> s.serve_forever()
1391
1392Proxy Objects
1393~~~~~~~~~~~~~
1394
1395A proxy is an object which *refers* to a shared object which lives (presumably)
1396in a different process.  The shared object is said to be the *referent* of the
1397proxy.  Multiple proxy objects may have the same referent.
1398
1399A proxy object has methods which invoke corresponding methods of its referent
1400(although not every method of the referent will necessarily be available through
1401the proxy).  A proxy can usually be used in most of the same ways that its
1402referent can:
1403
1404.. doctest::
1405
1406   >>> from multiprocessing import Manager
1407   >>> manager = Manager()
1408   >>> l = manager.list([i*i for i in range(10)])
1409   >>> print l
1410   [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
1411   >>> print repr(l)
1412   <ListProxy object, typeid 'list' at 0x...>
1413   >>> l[4]
1414   16
1415   >>> l[2:5]
1416   [4, 9, 16]
1417
1418Notice that applying :func:`str` to a pro…

Large files files are truncated, but you can click here to view the full file