PageRenderTime 31ms CodeModel.GetById 15ms app.highlight 11ms RepoModel.GetById 1ms app.codeStats 1ms

/Doc/library/socketserver.rst

http://unladen-swallow.googlecode.com/
ReStructuredText | 547 lines | 378 code | 169 blank | 0 comment | 0 complexity | a16f78ae83385f1ea2eb87e75008a9cb MD5 | raw file
  1
  2:mod:`SocketServer` --- A framework for network servers
  3=======================================================
  4
  5.. module:: SocketServer
  6   :synopsis: A framework for network servers.
  7
  8.. note::
  9
 10   The :mod:`SocketServer` module has been renamed to :mod:`socketserver` in
 11   Python 3.0.  The :term:`2to3` tool will automatically adapt imports when
 12   converting your sources to 3.0.
 13
 14
 15The :mod:`SocketServer` module simplifies the task of writing network servers.
 16
 17There are four basic server classes: :class:`TCPServer` uses the Internet TCP
 18protocol, which provides for continuous streams of data between the client and
 19server.  :class:`UDPServer` uses datagrams, which are discrete packets of
 20information that may arrive out of order or be lost while in transit.  The more
 21infrequently used :class:`UnixStreamServer` and :class:`UnixDatagramServer`
 22classes are similar, but use Unix domain sockets; they're not available on
 23non-Unix platforms.  For more details on network programming, consult a book
 24such as
 25W. Richard Steven's UNIX Network Programming or Ralph Davis's Win32 Network
 26Programming.
 27
 28These four classes process requests :dfn:`synchronously`; each request must be
 29completed before the next request can be started.  This isn't suitable if each
 30request takes a long time to complete, because it requires a lot of computation,
 31or because it returns a lot of data which the client is slow to process.  The
 32solution is to create a separate process or thread to handle each request; the
 33:class:`ForkingMixIn` and :class:`ThreadingMixIn` mix-in classes can be used to
 34support asynchronous behaviour.
 35
 36Creating a server requires several steps.  First, you must create a request
 37handler class by subclassing the :class:`BaseRequestHandler` class and
 38overriding its :meth:`handle` method; this method will process incoming
 39requests.  Second, you must instantiate one of the server classes, passing it
 40the server's address and the request handler class.  Finally, call the
 41:meth:`handle_request` or :meth:`serve_forever` method of the server object to
 42process one or many requests.
 43
 44When inheriting from :class:`ThreadingMixIn` for threaded connection behavior,
 45you should explicitly declare how you want your threads to behave on an abrupt
 46shutdown. The :class:`ThreadingMixIn` class defines an attribute
 47*daemon_threads*, which indicates whether or not the server should wait for
 48thread termination. You should set the flag explicitly if you would like threads
 49to behave autonomously; the default is :const:`False`, meaning that Python will
 50not exit until all threads created by :class:`ThreadingMixIn` have exited.
 51
 52Server classes have the same external methods and attributes, no matter what
 53network protocol they use.
 54
 55
 56Server Creation Notes
 57---------------------
 58
 59There are five classes in an inheritance diagram, four of which represent
 60synchronous servers of four types::
 61
 62   +------------+
 63   | BaseServer |
 64   +------------+
 65         |
 66         v
 67   +-----------+        +------------------+
 68   | TCPServer |------->| UnixStreamServer |
 69   +-----------+        +------------------+
 70         |
 71         v
 72   +-----------+        +--------------------+
 73   | UDPServer |------->| UnixDatagramServer |
 74   +-----------+        +--------------------+
 75
 76Note that :class:`UnixDatagramServer` derives from :class:`UDPServer`, not from
 77:class:`UnixStreamServer` --- the only difference between an IP and a Unix
 78stream server is the address family, which is simply repeated in both Unix
 79server classes.
 80
 81Forking and threading versions of each type of server can be created using the
 82:class:`ForkingMixIn` and :class:`ThreadingMixIn` mix-in classes.  For instance,
 83a threading UDP server class is created as follows::
 84
 85   class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass
 86
 87The mix-in class must come first, since it overrides a method defined in
 88:class:`UDPServer`.  Setting the various member variables also changes the
 89behavior of the underlying server mechanism.
 90
 91To implement a service, you must derive a class from :class:`BaseRequestHandler`
 92and redefine its :meth:`handle` method.  You can then run various versions of
 93the service by combining one of the server classes with your request handler
 94class.  The request handler class must be different for datagram or stream
 95services.  This can be hidden by using the handler subclasses
 96:class:`StreamRequestHandler` or :class:`DatagramRequestHandler`.
 97
 98Of course, you still have to use your head!  For instance, it makes no sense to
 99use a forking server if the service contains state in memory that can be
100modified by different requests, since the modifications in the child process
101would never reach the initial state kept in the parent process and passed to
102each child.  In this case, you can use a threading server, but you will probably
103have to use locks to protect the integrity of the shared data.
104
105On the other hand, if you are building an HTTP server where all data is stored
106externally (for instance, in the file system), a synchronous class will
107essentially render the service "deaf" while one request is being handled --
108which may be for a very long time if a client is slow to receive all the data it
109has requested.  Here a threading or forking server is appropriate.
110
111In some cases, it may be appropriate to process part of a request synchronously,
112but to finish processing in a forked child depending on the request data.  This
113can be implemented by using a synchronous server and doing an explicit fork in
114the request handler class :meth:`handle` method.
115
116Another approach to handling multiple simultaneous requests in an environment
117that supports neither threads nor :func:`fork` (or where these are too expensive
118or inappropriate for the service) is to maintain an explicit table of partially
119finished requests and to use :func:`select` to decide which request to work on
120next (or whether to handle a new incoming request).  This is particularly
121important for stream services where each client can potentially be connected for
122a long time (if threads or subprocesses cannot be used). See :mod:`asyncore` for
123another way to manage this.
124
125.. XXX should data and methods be intermingled, or separate?
126   how should the distinction between class and instance variables be drawn?
127
128
129Server Objects
130--------------
131
132.. class:: BaseServer
133
134   This is the superclass of all Server objects in the module.  It defines the
135   interface, given below, but does not implement most of the methods, which is
136   done in subclasses.
137
138
139.. method:: BaseServer.fileno()
140
141   Return an integer file descriptor for the socket on which the server is
142   listening.  This function is most commonly passed to :func:`select.select`, to
143   allow monitoring multiple servers in the same process.
144
145
146.. method:: BaseServer.handle_request()
147
148   Process a single request.  This function calls the following methods in
149   order: :meth:`get_request`, :meth:`verify_request`, and
150   :meth:`process_request`.  If the user-provided :meth:`handle` method of the
151   handler class raises an exception, the server's :meth:`handle_error` method
152   will be called.  If no request is received within :attr:`self.timeout`
153   seconds, :meth:`handle_timeout` will be called and :meth:`handle_request`
154   will return.
155
156
157.. method:: BaseServer.serve_forever(poll_interval=0.5)
158
159   Handle requests until an explicit :meth:`shutdown` request.  Polls for
160   shutdown every *poll_interval* seconds.
161
162
163.. method:: BaseServer.shutdown()
164
165   Tells the :meth:`serve_forever` loop to stop and waits until it does.
166
167   .. versionadded:: 2.6
168
169
170.. attribute:: BaseServer.address_family
171
172   The family of protocols to which the server's socket belongs.
173   Common examples are :const:`socket.AF_INET` and :const:`socket.AF_UNIX`.
174
175
176.. attribute:: BaseServer.RequestHandlerClass
177
178   The user-provided request handler class; an instance of this class is created
179   for each request.
180
181
182.. attribute:: BaseServer.server_address
183
184   The address on which the server is listening.  The format of addresses varies
185   depending on the protocol family; see the documentation for the socket module
186   for details.  For Internet protocols, this is a tuple containing a string giving
187   the address, and an integer port number: ``('127.0.0.1', 80)``, for example.
188
189
190.. attribute:: BaseServer.socket
191
192   The socket object on which the server will listen for incoming requests.
193
194
195The server classes support the following class variables:
196
197.. XXX should class variables be covered before instance variables, or vice versa?
198
199.. attribute:: BaseServer.allow_reuse_address
200
201   Whether the server will allow the reuse of an address. This defaults to
202   :const:`False`, and can be set in subclasses to change the policy.
203
204
205.. attribute:: BaseServer.request_queue_size
206
207   The size of the request queue.  If it takes a long time to process a single
208   request, any requests that arrive while the server is busy are placed into a
209   queue, up to :attr:`request_queue_size` requests.  Once the queue is full,
210   further requests from clients will get a "Connection denied" error.  The default
211   value is usually 5, but this can be overridden by subclasses.
212
213
214.. attribute:: BaseServer.socket_type
215
216   The type of socket used by the server; :const:`socket.SOCK_STREAM` and
217   :const:`socket.SOCK_DGRAM` are two common values.
218
219
220.. attribute:: BaseServer.timeout
221
222   Timeout duration, measured in seconds, or :const:`None` if no timeout is
223   desired.  If :meth:`handle_request` receives no incoming requests within the
224   timeout period, the :meth:`handle_timeout` method is called.
225
226
227There are various server methods that can be overridden by subclasses of base
228server classes like :class:`TCPServer`; these methods aren't useful to external
229users of the server object.
230
231.. XXX should the default implementations of these be documented, or should
232   it be assumed that the user will look at SocketServer.py?
233
234.. method:: BaseServer.finish_request()
235
236   Actually processes the request by instantiating :attr:`RequestHandlerClass` and
237   calling its :meth:`handle` method.
238
239
240.. method:: BaseServer.get_request()
241
242   Must accept a request from the socket, and return a 2-tuple containing the *new*
243   socket object to be used to communicate with the client, and the client's
244   address.
245
246
247.. method:: BaseServer.handle_error(request, client_address)
248
249   This function is called if the :attr:`RequestHandlerClass`'s :meth:`handle`
250   method raises an exception.  The default action is to print the traceback to
251   standard output and continue handling further requests.
252
253
254.. method:: BaseServer.handle_timeout()
255
256   This function is called when the :attr:`timeout` attribute has been set to a
257   value other than :const:`None` and the timeout period has passed with no
258   requests being received.  The default action for forking servers is
259   to collect the status of any child processes that have exited, while
260   in threading servers this method does nothing.
261
262
263.. method:: BaseServer.process_request(request, client_address)
264
265   Calls :meth:`finish_request` to create an instance of the
266   :attr:`RequestHandlerClass`.  If desired, this function can create a new process
267   or thread to handle the request; the :class:`ForkingMixIn` and
268   :class:`ThreadingMixIn` classes do this.
269
270
271.. Is there any point in documenting the following two functions?
272   What would the purpose of overriding them be: initializing server
273   instance variables, adding new network families?
274
275.. method:: BaseServer.server_activate()
276
277   Called by the server's constructor to activate the server.  The default behavior
278   just :meth:`listen`\ s to the server's socket. May be overridden.
279
280
281.. method:: BaseServer.server_bind()
282
283   Called by the server's constructor to bind the socket to the desired address.
284   May be overridden.
285
286
287.. method:: BaseServer.verify_request(request, client_address)
288
289   Must return a Boolean value; if the value is :const:`True`, the request will be
290   processed, and if it's :const:`False`, the request will be denied. This function
291   can be overridden to implement access controls for a server. The default
292   implementation always returns :const:`True`.
293
294
295RequestHandler Objects
296----------------------
297
298The request handler class must define a new :meth:`handle` method, and can
299override any of the following methods.  A new instance is created for each
300request.
301
302
303.. method:: RequestHandler.finish()
304
305   Called after the :meth:`handle` method to perform any clean-up actions
306   required.  The default implementation does nothing.  If :meth:`setup` or
307   :meth:`handle` raise an exception, this function will not be called.
308
309
310.. method:: RequestHandler.handle()
311
312   This function must do all the work required to service a request.  The
313   default implementation does nothing.  Several instance attributes are
314   available to it; the request is available as :attr:`self.request`; the client
315   address as :attr:`self.client_address`; and the server instance as
316   :attr:`self.server`, in case it needs access to per-server information.
317
318   The type of :attr:`self.request` is different for datagram or stream
319   services.  For stream services, :attr:`self.request` is a socket object; for
320   datagram services, :attr:`self.request` is a pair of string and socket.
321   However, this can be hidden by using the request handler subclasses
322   :class:`StreamRequestHandler` or :class:`DatagramRequestHandler`, which
323   override the :meth:`setup` and :meth:`finish` methods, and provide
324   :attr:`self.rfile` and :attr:`self.wfile` attributes.  :attr:`self.rfile` and
325   :attr:`self.wfile` can be read or written, respectively, to get the request
326   data or return data to the client.
327
328
329.. method:: RequestHandler.setup()
330
331   Called before the :meth:`handle` method to perform any initialization actions
332   required.  The default implementation does nothing.
333
334
335Examples
336--------
337
338:class:`SocketServer.TCPServer` Example
339~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
340
341This is the server side::
342
343   import SocketServer
344
345   class MyTCPHandler(SocketServer.BaseRequestHandler):
346       """
347       The RequestHandler class for our server.
348
349       It is instantiated once per connection to the server, and must
350       override the handle() method to implement communication to the
351       client.
352       """
353
354       def handle(self):
355           # self.request is the TCP socket connected to the client
356           self.data = self.request.recv(1024).strip()
357           print "%s wrote:" % self.client_address[0]
358           print self.data
359           # just send back the same data, but upper-cased
360           self.request.send(self.data.upper())
361
362   if __name__ == "__main__":
363       HOST, PORT = "localhost", 9999
364
365       # Create the server, binding to localhost on port 9999
366       server = SocketServer.TCPServer((HOST, PORT), MyTCPHandler)
367
368       # Activate the server; this will keep running until you
369       # interrupt the program with Ctrl-C
370       server.serve_forever()
371
372An alternative request handler class that makes use of streams (file-like
373objects that simplify communication by providing the standard file interface)::
374
375   class MyTCPHandler(SocketServer.StreamRequestHandler):
376
377       def handle(self):
378           # self.rfile is a file-like object created by the handler;
379           # we can now use e.g. readline() instead of raw recv() calls
380           self.data = self.rfile.readline().strip()
381           print "%s wrote:" % self.client_address[0]
382           print self.data
383           # Likewise, self.wfile is a file-like object used to write back
384           # to the client
385           self.wfile.write(self.data.upper())
386
387The difference is that the ``readline()`` call in the second handler will call
388``recv()`` multiple times until it encounters a newline character, while the
389single ``recv()`` call in the first handler will just return what has been sent
390from the client in one ``send()`` call.
391
392
393This is the client side::
394
395   import socket
396   import sys
397
398   HOST, PORT = "localhost", 9999
399   data = " ".join(sys.argv[1:])
400
401   # Create a socket (SOCK_STREAM means a TCP socket)
402   sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
403
404   # Connect to server and send data
405   sock.connect((HOST, PORT))
406   sock.send(data + "\n")
407
408   # Receive data from the server and shut down
409   received = sock.recv(1024)
410   sock.close()
411
412   print "Sent:     %s" % data
413   print "Received: %s" % received
414
415
416The output of the example should look something like this:
417
418Server::
419
420   $ python TCPServer.py
421   127.0.0.1 wrote:
422   hello world with TCP
423   127.0.0.1 wrote:
424   python is nice
425
426Client::
427
428   $ python TCPClient.py hello world with TCP
429   Sent:     hello world with TCP
430   Received: HELLO WORLD WITH TCP
431   $ python TCPClient.py python is nice
432   Sent:     python is nice
433   Received: PYTHON IS NICE
434
435
436:class:`SocketServer.UDPServer` Example
437~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
438
439This is the server side::
440
441   import SocketServer
442
443   class MyUDPHandler(SocketServer.BaseRequestHandler):
444       """
445       This class works similar to the TCP handler class, except that
446       self.request consists of a pair of data and client socket, and since
447       there is no connection the client address must be given explicitly
448       when sending data back via sendto().
449       """
450
451       def handle(self):
452           data = self.request[0].strip()
453           socket = self.request[1]
454           print "%s wrote:" % self.client_address[0]
455           print data
456           socket.sendto(data.upper(), self.client_address)
457
458   if __name__ == "__main__":
459      HOST, PORT = "localhost", 9999
460      server = SocketServer.UDPServer((HOST, PORT), MyUDPHandler)
461      server.serve_forever()
462
463This is the client side::
464
465   import socket
466   import sys
467
468   HOST, PORT = "localhost"
469   data = " ".join(sys.argv[1:])
470
471   # SOCK_DGRAM is the socket type to use for UDP sockets
472   sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
473
474   # As you can see, there is no connect() call; UDP has no connections.
475   # Instead, data is directly sent to the recipient via sendto().
476   sock.sendto(data + "\n", (HOST, PORT))
477   received = sock.recv(1024)
478
479   print "Sent:     %s" % data
480   print "Received: %s" % received
481
482The output of the example should look exactly like for the TCP server example.
483
484
485Asynchronous Mixins
486~~~~~~~~~~~~~~~~~~~
487
488To build asynchronous handlers, use the :class:`ThreadingMixIn` and
489:class:`ForkingMixIn` classes.
490
491An example for the :class:`ThreadingMixIn` class::
492
493   import socket
494   import threading
495   import SocketServer
496
497   class ThreadedTCPRequestHandler(SocketServer.BaseRequestHandler):
498
499       def handle(self):
500           data = self.request.recv(1024)
501           cur_thread = threading.currentThread()
502           response = "%s: %s" % (cur_thread.getName(), data)
503           self.request.send(response)
504
505   class ThreadedTCPServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
506       pass
507
508   def client(ip, port, message):
509       sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
510       sock.connect((ip, port))
511       sock.send(message)
512       response = sock.recv(1024)
513       print "Received: %s" % response
514       sock.close()
515
516   if __name__ == "__main__":
517       # Port 0 means to select an arbitrary unused port
518       HOST, PORT = "localhost", 0
519
520       server = ThreadedTCPServer((HOST, PORT), ThreadedTCPRequestHandler)
521       ip, port = server.server_address
522
523       # Start a thread with the server -- that thread will then start one
524       # more thread for each request
525       server_thread = threading.Thread(target=server.serve_forever)
526       # Exit the server thread when the main thread terminates
527       server_thread.setDaemon(True)
528       server_thread.start()
529       print "Server loop running in thread:", server_thread.getName()
530
531       client(ip, port, "Hello World 1")
532       client(ip, port, "Hello World 2")
533       client(ip, port, "Hello World 3")
534
535       server.shutdown()
536
537The output of the example should look something like this::
538
539   $ python ThreadedTCPServer.py
540   Server loop running in thread: Thread-1
541   Received: Thread-2: Hello World 1
542   Received: Thread-3: Hello World 2
543   Received: Thread-4: Hello World 3
544
545
546The :class:`ForkingMixIn` class is used in the same way, except that the server
547will spawn a new process for each request.