Debian Codegraph @

Doc/library/email.parser.rst

raw
1 :mod:`!email.parser`: Parsing email messages
2 --------------------------------------------
3
4 .. module:: email.parser
5 :synopsis: Parse flat text email messages to produce a message object structure.
6
7 **Source code:** :source:`Lib/email/parser.py`
8
9 --------------
10
11 Message object structures can be created in one of two ways: they can be
12 created from whole cloth by creating an :class:`~email.message.EmailMessage`
13 object, adding headers using the dictionary interface, and adding payload(s)
14 using :meth:`~email.message.EmailMessage.set_content` and related methods, or
15 they can be created by parsing a serialized representation of the email
16 message.
17
18 The :mod:`email` package provides a standard parser that understands most email
19 document structures, including MIME documents. You can pass the parser a
20 bytes, string or file object, and the parser will return to you the root
21 :class:`~email.message.EmailMessage` instance of the object structure. For
22 simple, non-MIME messages the payload of this root object will likely be a
23 string containing the text of the message. For MIME messages, the root object
24 will return ``True`` from its :meth:`~email.message.EmailMessage.is_multipart`
25 method, and the subparts can be accessed via the payload manipulation methods,
26 such as :meth:`~email.message.EmailMessage.get_body`,
27 :meth:`~email.message.EmailMessage.iter_parts`, and
28 :meth:`~email.message.EmailMessage.walk`.
29
30 There are actually two parser interfaces available for use, the :class:`Parser`
31 API and the incremental :class:`FeedParser` API. The :class:`Parser` API is
32 most useful if you have the entire text of the message in memory, or if the
33 entire message lives in a file on the file system. :class:`FeedParser` is more
34 appropriate when you are reading the message from a stream which might block
35 waiting for more input (such as reading an email message from a socket). The
36 :class:`FeedParser` can consume and parse the message incrementally, and only
37 returns the root object when you close the parser.
38
39 Note that the parser can be extended in limited ways, and of course you can
40 implement your own parser completely from scratch. All of the logic that
41 connects the :mod:`email` package's bundled parser and the
42 :class:`~email.message.EmailMessage` class is embodied in the :class:`~email.policy.Policy`
43 class, so a custom parser can create message object trees any way it finds
44 necessary by implementing custom versions of the appropriate :class:`!Policy`
45 methods.
46
47
48 FeedParser API
49 ^^^^^^^^^^^^^^
50
51 The :class:`BytesFeedParser`, imported from the :mod:`email.feedparser` module,
52 provides an API that is conducive to incremental parsing of email messages,
53 such as would be necessary when reading the text of an email message from a
54 source that can block (such as a socket). The :class:`BytesFeedParser` can of
55 course be used to parse an email message fully contained in a :term:`bytes-like
56 object`, string, or file, but the :class:`BytesParser` API may be more
57 convenient for such use cases. The semantics and results of the two parser
58 APIs are identical.
59
60 The :class:`BytesFeedParser`'s API is simple; you create an instance, feed it a
61 bunch of bytes until there's no more to feed it, then close the parser to
62 retrieve the root message object. The :class:`BytesFeedParser` is extremely
63 accurate when parsing standards-compliant messages, and it does a very good job
64 of parsing non-compliant messages, providing information about how a message
65 was deemed broken. It will populate a message object's
66 :attr:`~email.message.EmailMessage.defects` attribute with a list of any
67 problems it found in a message. See the :mod:`email.errors` module for the
68 list of defects that it can find.
69
70 Here is the API for the :class:`BytesFeedParser`:
71
72
73 .. class:: BytesFeedParser(_factory=None, *, policy=policy.compat32)
74
75 Create a :class:`BytesFeedParser` instance. Optional *_factory* is a
76 no-argument callable; if not specified use the
77 :attr:`~email.policy.Policy.message_factory` from the *policy*. Call
78 *_factory* whenever a new message object is needed.
79
80 If *policy* is specified use the rules it specifies to update the
81 representation of the message. If *policy* is not set, use the
82 :class:`compat32 <email.policy.Compat32>` policy, which maintains backward
83 compatibility with the Python 3.2 version of the email package and provides
84 :class:`~email.message.Message` as the default factory. All other policies
85 provide :class:`~email.message.EmailMessage` as the default *_factory*. For
86 more information on what else *policy* controls, see the
87 :mod:`~email.policy` documentation.
88
89 Note: **The policy keyword should always be specified**; The default will
90 change to :data:`email.policy.default` in a future version of Python.
91
92 .. versionadded:: 3.2
93
94 .. versionchanged:: 3.3 Added the *policy* keyword.
95 .. versionchanged:: 3.6 *_factory* defaults to the policy ``message_factory``.
96
97
98 .. method:: feed(data)
99
100 Feed the parser some more data. *data* should be a :term:`bytes-like
101 object` containing one or more lines. The lines can be partial and the
102 parser will stitch such partial lines together properly. The lines can
103 have any of the three common line endings: carriage return, newline, or
104 carriage return and newline (they can even be mixed).
105
106
107 .. method:: close()
108
109 Complete the parsing of all previously fed data and return the root
110 message object. It is undefined what happens if :meth:`~feed` is called
111 after this method has been called.
112
113
114 .. class:: FeedParser(_factory=None, *, policy=policy.compat32)
115
116 Works like :class:`BytesFeedParser` except that the input to the
117 :meth:`~BytesFeedParser.feed` method must be a string. This is of limited
118 utility, since the only way for such a message to be valid is for it to
119 contain only ASCII text or, if :attr:`~email.policy.EmailPolicy.utf8` is
120 ``True``, no binary attachments.
121
122 .. versionchanged:: 3.3 Added the *policy* keyword.
123
124
125 Parser API
126 ^^^^^^^^^^
127
128 The :class:`BytesParser` class, imported from the :mod:`!email.parser` module,
129 provides an API that can be used to parse a message when the complete contents
130 of the message are available in a :term:`bytes-like object` or file. The
131 :mod:`!email.parser` module also provides :class:`Parser` for parsing strings,
132 and header-only parsers, :class:`BytesHeaderParser` and
133 :class:`HeaderParser`, which can be used if you're only interested in the
134 headers of the message. :class:`BytesHeaderParser` and :class:`HeaderParser`
135 can be much faster in these situations, since they do not attempt to parse the
136 message body, instead setting the payload to the raw body.
137
138
139 .. class:: BytesParser(_class=None, *, policy=policy.compat32)
140
141 Create a :class:`BytesParser` instance. The *_class* and *policy*
142 arguments have the same meaning and semantics as the *_factory*
143 and *policy* arguments of :class:`BytesFeedParser`.
144
145 Note: **The policy keyword should always be specified**; The default will
146 change to :data:`email.policy.default` in a future version of Python.
147
148 .. versionchanged:: 3.3
149 Removed the *strict* argument that was deprecated in 2.4. Added the
150 *policy* keyword.
151 .. versionchanged:: 3.6 *_class* defaults to the policy ``message_factory``.
152
153
154 .. method:: parse(fp, headersonly=False)
155
156 Read all the data from the binary file-like object *fp*, parse the
157 resulting bytes, and return the message object. *fp* must support
158 both the :meth:`~io.IOBase.readline` and the :meth:`~io.BufferedIOBase.read`
159 methods.
160
161 The bytes contained in *fp* must be formatted as a block of :rfc:`5322`
162 (or, if :attr:`~email.policy.EmailPolicy.utf8` is ``True``, :rfc:`6532`)
163 style headers and header continuation lines, optionally preceded by an
164 envelope header. The header block is terminated either by the end of the
165 data or by a blank line. Following the header block is the body of the
166 message (which may contain MIME-encoded subparts, including subparts
167 with a :mailheader:`Content-Transfer-Encoding` of ``8bit``).
168
169 Optional *headersonly* is a flag specifying whether to stop parsing after
170 reading the headers or not. The default is ``False``, meaning it parses
171 the entire contents of the file.
172
173
174 .. method:: parsebytes(bytes, headersonly=False)
175
176 Similar to the :meth:`parse` method, except it takes a :term:`bytes-like
177 object` instead of a file-like object. Calling this method on a
178 :term:`bytes-like object` is equivalent to wrapping *bytes* in a
179 :class:`~io.BytesIO` instance first and calling :meth:`parse`.
180
181 Optional *headersonly* is as with the :meth:`parse` method.
182
183 .. versionadded:: 3.2
184
185
186 .. class:: BytesHeaderParser(_class=None, *, policy=policy.compat32)
187
188 Exactly like :class:`BytesParser`, except that *headersonly*
189 defaults to ``True``.
190
191 .. versionadded:: 3.3
192
193
194 .. class:: Parser(_class=None, *, policy=policy.compat32)
195
196 This class is parallel to :class:`BytesParser`, but handles string input.
197
198 .. versionchanged:: 3.3
199 Removed the *strict* argument. Added the *policy* keyword.
200 .. versionchanged:: 3.6 *_class* defaults to the policy ``message_factory``.
201
202
203 .. method:: parse(fp, headersonly=False)
204
205 Read all the data from the text-mode file-like object *fp*, parse the
206 resulting text, and return the root message object. *fp* must support
207 both the :meth:`~io.TextIOBase.readline` and the
208 :meth:`~io.TextIOBase.read` methods on file-like objects.
209
210 Other than the text mode requirement, this method operates like
211 :meth:`BytesParser.parse`.
212
213
214 .. method:: parsestr(text, headersonly=False)
215
216 Similar to the :meth:`parse` method, except it takes a string object
217 instead of a file-like object. Calling this method on a string is
218 equivalent to wrapping *text* in a :class:`~io.StringIO` instance first
219 and calling :meth:`parse`.
220
221 Optional *headersonly* is as with the :meth:`parse` method.
222
223
224 .. class:: HeaderParser(_class=None, *, policy=policy.compat32)
225
226 Exactly like :class:`Parser`, except that *headersonly*
227 defaults to ``True``.
228
229
230 Since creating a message object structure from a string or a file object is such
231 a common task, four functions are provided as a convenience. They are available
232 in the top-level :mod:`email` package namespace.
233
234 .. currentmodule:: email
235
236
237 .. function:: message_from_bytes(s, _class=None, *, policy=policy.compat32)
238
239 Return a message object structure from a :term:`bytes-like object`. This is
240 equivalent to ``BytesParser().parsebytes(s)``. Optional *_class* and
241 *policy* are interpreted as with the :class:`~email.parser.BytesParser` class
242 constructor.
243
244 .. versionadded:: 3.2
245 .. versionchanged:: 3.3
246 Removed the *strict* argument. Added the *policy* keyword.
247
248
249 .. function:: message_from_binary_file(fp, _class=None, *, \
250 policy=policy.compat32)
251
252 Return a message object structure tree from an open binary :term:`file
253 object`. This is equivalent to ``BytesParser().parse(fp)``. *_class* and
254 *policy* are interpreted as with the :class:`~email.parser.BytesParser` class
255 constructor.
256
257 .. versionadded:: 3.2
258 .. versionchanged:: 3.3
259 Removed the *strict* argument. Added the *policy* keyword.
260
261
262 .. function:: message_from_string(s, _class=None, *, policy=policy.compat32)
263
264 Return a message object structure from a string. This is equivalent to
265 ``Parser().parsestr(s)``. *_class* and *policy* are interpreted as
266 with the :class:`~email.parser.Parser` class constructor.
267
268 .. versionchanged:: 3.3
269 Removed the *strict* argument. Added the *policy* keyword.
270
271
272 .. function:: message_from_file(fp, _class=None, *, policy=policy.compat32)
273
274 Return a message object structure tree from an open :term:`file object`.
275 This is equivalent to ``Parser().parse(fp)``. *_class* and *policy* are
276 interpreted as with the :class:`~email.parser.Parser` class constructor.
277
278 .. versionchanged:: 3.3
279 Removed the *strict* argument. Added the *policy* keyword.
280 .. versionchanged:: 3.6 *_class* defaults to the policy ``message_factory``.
281
282
283 Here's an example of how you might use :func:`message_from_bytes` at an
284 interactive Python prompt::
285
286 >>> import email
287 >>> msg = email.message_from_bytes(myBytes) # doctest: +SKIP
288
289
290 Additional notes
291 ^^^^^^^^^^^^^^^^
292
293 Here are some notes on the parsing semantics:
294
295 * Most non-\ :mimetype:`multipart` type messages are parsed as a single message
296 object with a string payload. These objects will return ``False`` for
297 :meth:`~email.message.EmailMessage.is_multipart`, and
298 :meth:`~email.message.EmailMessage.iter_parts` will yield an empty list.
299
300 * All :mimetype:`multipart` type messages will be parsed as a container message
301 object with a list of sub-message objects for their payload. The outer
302 container message will return ``True`` for
303 :meth:`~email.message.EmailMessage.is_multipart`, and
304 :meth:`~email.message.EmailMessage.iter_parts` will yield a list of subparts.
305
306 * Most messages with a content type of :mimetype:`message/\*` (such as
307 :mimetype:`message/delivery-status` and :mimetype:`message/rfc822`) will also
308 be parsed as container object containing a list payload of length 1. Their
309 :meth:`~email.message.EmailMessage.is_multipart` method will return ``True``.
310 The single element yielded by :meth:`~email.message.EmailMessage.iter_parts`
311 will be a sub-message object.
312
313 * Some non-standards-compliant messages may not be internally consistent about
314 their :mimetype:`multipart`\ -edness. Such messages may have a
315 :mailheader:`Content-Type` header of type :mimetype:`multipart`, but their
316 :meth:`~email.message.EmailMessage.is_multipart` method may return ``False``.
317 If such messages were parsed with the :class:`~email.parser.FeedParser`,
318 they will have an instance of the
319 :class:`~email.errors.MultipartInvariantViolationDefect` class in their
320 *defects* attribute list. See :mod:`email.errors` for details.