/Doc/library/email.parser.rst

  1:mod:`email`: Parsing email messages
  2------------------------------------
  3
  4.. module:: email.parser
  5   :synopsis: Parse flat text email messages to produce a message object structure.
  6
  7
  8Message object structures can be created in one of two ways: they can be created
  9from whole cloth by instantiating :class:`~email.message.Message` objects and
 10stringing them together via :meth:`attach` and :meth:`set_payload` calls, or they
 11can be created by parsing a flat text representation of the email message.
 12
 13The :mod:`email` package provides a standard parser that understands most email
 14document structures, including MIME documents.  You can pass the parser a string
 15or a file object, and the parser will return to you the root
 16:class:`~email.message.Message` instance of the object structure.  For simple,
 17non-MIME messages the payload of this root object will likely be a string
 18containing the text of the message.  For MIME messages, the root object will
 19return ``True`` from its :meth:`is_multipart` method, and the subparts can be
 20accessed via the :meth:`get_payload` and :meth:`walk` methods.
 21
 22There are actually two parser interfaces available for use, the classic
 23:class:`Parser` API and the incremental :class:`FeedParser` API.  The classic
 24:class:`Parser` API is fine if you have the entire text of the message in memory
 25as a string, or if the entire message lives in a file on the file system.
 26:class:`FeedParser` is more appropriate for when you're reading the message from
 27a stream which might block waiting for more input (e.g. reading an email message
 28from a socket).  The :class:`FeedParser` can consume and parse the message
 29incrementally, and only returns the root object when you close the parser [#]_.
 30
 31Note that the parser can be extended in limited ways, and of course you can
 32implement your own parser completely from scratch.  There is no magical
 33connection between the :mod:`email` package's bundled parser and the
 34:class:`~email.message.Message` class, so your custom parser can create message
 35object trees any way it finds necessary.
 36
 37
 38FeedParser API
 39^^^^^^^^^^^^^^
 40
 41.. versionadded:: 2.4
 42
 43The :class:`FeedParser`, imported from the :mod:`email.feedparser` module,
 44provides an API that is conducive to incremental parsing of email messages, such
 45as would be necessary when reading the text of an email message from a source
 46that can block (e.g. a socket).  The :class:`FeedParser` can of course be used
 47to parse an email message fully contained in a string or a file, but the classic
 48:class:`Parser` API may be more convenient for such use cases.  The semantics
 49and results of the two parser APIs are identical.
 50
 51The :class:`FeedParser`'s API is simple; you create an instance, feed it a bunch
 52of text until there's no more to feed it, then close the parser to retrieve the
 53root message object.  The :class:`FeedParser` is extremely accurate when parsing
 54standards-compliant messages, and it does a very good job of parsing
 55non-compliant messages, providing information about how a message was deemed
 56broken.  It will populate a message object's *defects* attribute with a list of
 57any problems it found in a message.  See the :mod:`email.errors` module for the
 58list of defects that it can find.
 59
 60Here is the API for the :class:`FeedParser`:
 61
 62
 63.. class:: FeedParser([_factory])
 64
 65   Create a :class:`FeedParser` instance.  Optional *_factory* is a no-argument
 66   callable that will be called whenever a new message object is needed.  It
 67   defaults to the :class:`email.message.Message` class.
 68
 69
 70   .. method:: feed(data)
 71
 72      Feed the :class:`FeedParser` some more data.  *data* should be a string
 73      containing one or more lines.  The lines can be partial and the
 74      :class:`FeedParser` will stitch such partial lines together properly.  The
 75      lines in the string can have any of the common three line endings,
 76      carriage return, newline, or carriage return and newline (they can even be
 77      mixed).
 78
 79
 80   .. method:: close()
 81
 82      Closing a :class:`FeedParser` completes the parsing of all previously fed
 83      data, and returns the root message object.  It is undefined what happens
 84      if you feed more data to a closed :class:`FeedParser`.
 85
 86
 87Parser class API
 88^^^^^^^^^^^^^^^^
 89
 90The :class:`Parser` class, imported from the :mod:`email.parser` module,
 91provides an API that can be used to parse a message when the complete contents
 92of the message are available in a string or file.  The :mod:`email.parser`
 93module also provides a second class, called :class:`HeaderParser` which can be
 94used if you're only interested in the headers of the message.
 95:class:`HeaderParser` can be much faster in these situations, since it does not
 96attempt to parse the message body, instead setting the payload to the raw body
 97as a string. :class:`HeaderParser` has the same API as the :class:`Parser`
 98class.
 99
100
101.. class:: Parser([_class])
102
103   The constructor for the :class:`Parser` class takes an optional argument
104   *_class*.  This must be a callable factory (such as a function or a class), and
105   it is used whenever a sub-message object needs to be created.  It defaults to
106   :class:`~email.message.Message` (see :mod:`email.message`).  The factory will
107   be called without arguments.
108
109   The optional *strict* flag is ignored.
110
111   .. deprecated:: 2.4
112      Because the :class:`Parser` class is a backward compatible API wrapper
113      around the new-in-Python 2.4 :class:`FeedParser`, *all* parsing is
114      effectively non-strict.  You should simply stop passing a *strict* flag to
115      the :class:`Parser` constructor.
116
117   .. versionchanged:: 2.2.2
118      The *strict* flag was added.
119
120   .. versionchanged:: 2.4
121      The *strict* flag was deprecated.
122
123   The other public :class:`Parser` methods are:
124
125
126   .. method:: parse(fp[, headersonly])
127
128      Read all the data from the file-like object *fp*, parse the resulting
129      text, and return the root message object.  *fp* must support both the
130      :meth:`readline` and the :meth:`read` methods on file-like objects.
131
132      The text contained in *fp* must be formatted as a block of :rfc:`2822`
133      style headers and header continuation lines, optionally preceded by a
134      envelope header.  The header block is terminated either by the end of the
135      data or by a blank line.  Following the header block is the body of the
136      message (which may contain MIME-encoded subparts).
137
138      Optional *headersonly* is as with the :meth:`parse` method.
139
140      .. versionchanged:: 2.2.2
141         The *headersonly* flag was added.
142
143
144   .. method:: parsestr(text[, headersonly])
145
146      Similar to the :meth:`parse` method, except it takes a string object
147      instead of a file-like object.  Calling this method on a string is exactly
148      equivalent to wrapping *text* in a :class:`StringIO` instance first and
149      calling :meth:`parse`.
150
151      Optional *headersonly* is a flag specifying whether to stop parsing after
152      reading the headers or not.  The default is ``False``, meaning it parses
153      the entire contents of the file.
154
155      .. versionchanged:: 2.2.2
156         The *headersonly* flag was added.
157
158Since creating a message object structure from a string or a file object is such
159a common task, two functions are provided as a convenience.  They are available
160in the top-level :mod:`email` package namespace.
161
162.. currentmodule:: email
163
164.. function:: message_from_string(s[, _class[, strict]])
165
166   Return a message object structure from a string.  This is exactly equivalent to
167   ``Parser().parsestr(s)``.  Optional *_class* and *strict* are interpreted as
168   with the :class:`Parser` class constructor.
169
170   .. versionchanged:: 2.2.2
171      The *strict* flag was added.
172
173
174.. function:: message_from_file(fp[, _class[, strict]])
175
176   Return a message object structure tree from an open file object.  This is
177   exactly equivalent to ``Parser().parse(fp)``.  Optional *_class* and *strict*
178   are interpreted as with the :class:`Parser` class constructor.
179
180   .. versionchanged:: 2.2.2
181      The *strict* flag was added.
182
183Here's an example of how you might use this at an interactive Python prompt::
184
185   >>> import email
186   >>> msg = email.message_from_string(myString)
187
188
189Additional notes
190^^^^^^^^^^^^^^^^
191
192Here are some notes on the parsing semantics:
193
194* Most non-\ :mimetype:`multipart` type messages are parsed as a single message
195  object with a string payload.  These objects will return ``False`` for
196  :meth:`is_multipart`.  Their :meth:`get_payload` method will return a string
197  object.
198
199* All :mimetype:`multipart` type messages will be parsed as a container message
200  object with a list of sub-message objects for their payload.  The outer
201  container message will return ``True`` for :meth:`is_multipart` and their
202  :meth:`get_payload` method will return the list of :class:`~email.message.Message`
203  subparts.
204
205* Most messages with a content type of :mimetype:`message/\*` (e.g.
206  :mimetype:`message/delivery-status` and :mimetype:`message/rfc822`) will also be
207  parsed as container object containing a list payload of length 1.  Their
208  :meth:`is_multipart` method will return ``True``.  The single element in the
209  list payload will be a sub-message object.
210
211* Some non-standards compliant messages may not be internally consistent about
212  their :mimetype:`multipart`\ -edness.  Such messages may have a
213  :mailheader:`Content-Type` header of type :mimetype:`multipart`, but their
214  :meth:`is_multipart` method may return ``False``.  If such messages were parsed
215  with the :class:`FeedParser`, they will have an instance of the
216  :class:`MultipartInvariantViolationDefect` class in their *defects* attribute
217  list.  See :mod:`email.errors` for details.
218
219.. rubric:: Footnotes
220
221.. [#] As of email package version 3.0, introduced in Python 2.4, the classic
222   :class:`Parser` was re-implemented in terms of the :class:`FeedParser`, so the
223   semantics and results are identical between the two parsers.
224