| 1 |
:mod:`!email.generator`: Generating MIME documents |
| 2 |
-------------------------------------------------- |
| 3 |
|
| 4 |
.. module:: email.generator |
| 5 |
:synopsis: Generate flat text email messages from a message structure. |
| 6 |
|
| 7 |
**Source code:** :source:`Lib/email/generator.py` |
| 8 |
|
| 9 |
-------------- |
| 10 |
|
| 11 |
One of the most common tasks is to generate the flat (serialized) version of |
| 12 |
the email message represented by a message object structure. You will need to |
| 13 |
do this if you want to send your message via :meth:`smtplib.SMTP.sendmail`, |
| 14 |
or print the message on the console. Taking a |
| 15 |
message object structure and producing a serialized representation is the job |
| 16 |
of the generator classes. |
| 17 |
|
| 18 |
As with the :mod:`email.parser` module, you aren't limited to the functionality |
| 19 |
of the bundled generator; you could write one from scratch yourself. However |
| 20 |
the bundled generator knows how to generate most email in a standards-compliant |
| 21 |
way, should handle MIME and non-MIME email messages just fine, and is designed |
| 22 |
so that the bytes-oriented parsing and generation operations are inverses, |
| 23 |
assuming the same non-transforming :mod:`~email.policy` is used for both. That |
| 24 |
is, parsing the serialized byte stream via the |
| 25 |
:class:`~email.parser.BytesParser` class and then regenerating the serialized |
| 26 |
byte stream using :class:`BytesGenerator` should produce output identical to |
| 27 |
the input [#]_. (On the other hand, using the generator on an |
| 28 |
:class:`~email.message.EmailMessage` constructed by program may result in |
| 29 |
changes to the :class:`~email.message.EmailMessage` object as defaults are |
| 30 |
filled in.) |
| 31 |
|
| 32 |
The :class:`Generator` class can be used to flatten a message into a text (as |
| 33 |
opposed to binary) serialized representation, but since Unicode cannot |
| 34 |
represent binary data directly, the message is of necessity transformed into |
| 35 |
something that contains only ASCII characters, using the standard email RFC |
| 36 |
Content Transfer Encoding techniques for encoding email messages for transport |
| 37 |
over channels that are not "8 bit clean". |
| 38 |
|
| 39 |
To accommodate reproducible processing of SMIME-signed messages |
| 40 |
:class:`Generator` disables header folding for message parts of type |
| 41 |
``multipart/signed`` and all subparts. |
| 42 |
|
| 43 |
|
| 44 |
.. class:: BytesGenerator(outfp, mangle_from_=None, maxheaderlen=None, *, \ |
| 45 |
policy=None) |
| 46 |
|
| 47 |
Return a :class:`BytesGenerator` object that will write any message provided |
| 48 |
to the :meth:`flatten` method, or any surrogateescape encoded text provided |
| 49 |
to the :meth:`write` method, to the :term:`file-like object` *outfp*. |
| 50 |
*outfp* must support a ``write`` method that accepts binary data. |
| 51 |
|
| 52 |
If optional *mangle_from_* is ``True``, put a ``>`` character in front of |
| 53 |
any line in the body that starts with the exact string ``"From "``, that is |
| 54 |
``From`` followed by a space at the beginning of a line. *mangle_from_* |
| 55 |
defaults to the value of the :attr:`~email.policy.Policy.mangle_from_` |
| 56 |
setting of the *policy* (which is ``True`` for the |
| 57 |
:data:`~email.policy.compat32` policy and ``False`` for all others). |
| 58 |
*mangle_from_* is intended for use when messages are stored in Unix mbox |
| 59 |
format (see :mod:`mailbox` and `WHY THE CONTENT-LENGTH FORMAT IS BAD |
| 60 |
<https://www.jwz.org/doc/content-length.html>`_). |
| 61 |
|
| 62 |
If *maxheaderlen* is not ``None``, refold any header lines that are longer |
| 63 |
than *maxheaderlen*, or if ``0``, do not rewrap any headers. If |
| 64 |
*manheaderlen* is ``None`` (the default), wrap headers and other message |
| 65 |
lines according to the *policy* settings. |
| 66 |
|
| 67 |
If *policy* is specified, use that policy to control message generation. If |
| 68 |
*policy* is ``None`` (the default), use the policy associated with the |
| 69 |
:class:`~email.message.Message` or :class:`~email.message.EmailMessage` |
| 70 |
object passed to ``flatten`` to control the message generation. See |
| 71 |
:mod:`email.policy` for details on what *policy* controls. |
| 72 |
|
| 73 |
.. versionadded:: 3.2 |
| 74 |
|
| 75 |
.. versionchanged:: 3.3 Added the *policy* keyword. |
| 76 |
|
| 77 |
.. versionchanged:: 3.6 The default behavior of the *mangle_from_* |
| 78 |
and *maxheaderlen* parameters is to follow the policy. |
| 79 |
|
| 80 |
|
| 81 |
.. method:: flatten(msg, unixfrom=False, linesep=None) |
| 82 |
|
| 83 |
Print the textual representation of the message object structure rooted |
| 84 |
at *msg* to the output file specified when the :class:`BytesGenerator` |
| 85 |
instance was created. |
| 86 |
|
| 87 |
If the :mod:`~email.policy` option :attr:`~email.policy.Policy.cte_type` |
| 88 |
is ``8bit`` (the default), copy any headers in the original parsed |
| 89 |
message that have not been modified to the output with any bytes with the |
| 90 |
high bit set reproduced as in the original, and preserve the non-ASCII |
| 91 |
:mailheader:`Content-Transfer-Encoding` of any body parts that have them. |
| 92 |
If ``cte_type`` is ``7bit``, convert the bytes with the high bit set as |
| 93 |
needed using an ASCII-compatible :mailheader:`Content-Transfer-Encoding`. |
| 94 |
That is, transform parts with non-ASCII |
| 95 |
:mailheader:`Content-Transfer-Encoding` |
| 96 |
(:mailheader:`Content-Transfer-Encoding: 8bit`) to an ASCII compatible |
| 97 |
:mailheader:`Content-Transfer-Encoding`, and encode RFC-invalid non-ASCII |
| 98 |
bytes in headers using the MIME ``unknown-8bit`` character set, thus |
| 99 |
rendering them RFC-compliant. |
| 100 |
|
| 101 |
.. XXX: There should be an option that just does the RFC |
| 102 |
compliance transformation on headers but leaves CTE 8bit parts alone. |
| 103 |
|
| 104 |
If *unixfrom* is ``True``, print the envelope header delimiter used by |
| 105 |
the Unix mailbox format (see :mod:`mailbox`) before the first of the |
| 106 |
:rfc:`5322` headers of the root message object. If the root object has |
| 107 |
no envelope header, craft a standard one. The default is ``False``. |
| 108 |
Note that for subparts, no envelope header is ever printed. |
| 109 |
|
| 110 |
If *linesep* is not ``None``, use it as the separator character between |
| 111 |
all the lines of the flattened message. If *linesep* is ``None`` (the |
| 112 |
default), use the value specified in the *policy*. |
| 113 |
|
| 114 |
.. XXX: flatten should take a *policy* keyword. |
| 115 |
|
| 116 |
|
| 117 |
.. method:: clone(fp) |
| 118 |
|
| 119 |
Return an independent clone of this :class:`BytesGenerator` instance with |
| 120 |
the exact same option settings, and *fp* as the new *outfp*. |
| 121 |
|
| 122 |
|
| 123 |
.. method:: write(s) |
| 124 |
|
| 125 |
Encode *s* using the ``ASCII`` codec and the ``surrogateescape`` error |
| 126 |
handler, and pass it to the *write* method of the *outfp* passed to the |
| 127 |
:class:`BytesGenerator`'s constructor. |
| 128 |
|
| 129 |
|
| 130 |
As a convenience, :class:`~email.message.EmailMessage` provides the methods |
| 131 |
:meth:`~email.message.EmailMessage.as_bytes` and ``bytes(aMessage)`` (a.k.a. |
| 132 |
:meth:`~email.message.EmailMessage.__bytes__`), which simplify the generation of |
| 133 |
a serialized binary representation of a message object. For more detail, see |
| 134 |
:mod:`email.message`. |
| 135 |
|
| 136 |
|
| 137 |
Because strings cannot represent binary data, the :class:`Generator` class must |
| 138 |
convert any binary data in any message it flattens to an ASCII compatible |
| 139 |
format, by converting them to an ASCII compatible |
| 140 |
:mailheader:`Content-Transfer_Encoding`. Using the terminology of the email |
| 141 |
RFCs, you can think of this as :class:`Generator` serializing to an I/O stream |
| 142 |
that is not "8 bit clean". In other words, most applications will want |
| 143 |
to be using :class:`BytesGenerator`, and not :class:`Generator`. |
| 144 |
|
| 145 |
.. class:: Generator(outfp, mangle_from_=None, maxheaderlen=None, *, \ |
| 146 |
policy=None) |
| 147 |
|
| 148 |
Return a :class:`Generator` object that will write any message provided |
| 149 |
to the :meth:`flatten` method, or any text provided to the :meth:`write` |
| 150 |
method, to the :term:`file-like object` *outfp*. *outfp* must support a |
| 151 |
``write`` method that accepts string data. |
| 152 |
|
| 153 |
If optional *mangle_from_* is ``True``, put a ``>`` character in front of |
| 154 |
any line in the body that starts with the exact string ``"From "``, that is |
| 155 |
``From`` followed by a space at the beginning of a line. *mangle_from_* |
| 156 |
defaults to the value of the :attr:`~email.policy.Policy.mangle_from_` |
| 157 |
setting of the *policy* (which is ``True`` for the |
| 158 |
:data:`~email.policy.compat32` policy and ``False`` for all others). |
| 159 |
*mangle_from_* is intended for use when messages are stored in Unix mbox |
| 160 |
format (see :mod:`mailbox` and `WHY THE CONTENT-LENGTH FORMAT IS BAD |
| 161 |
<https://www.jwz.org/doc/content-length.html>`_). |
| 162 |
|
| 163 |
If *maxheaderlen* is not ``None``, refold any header lines that are longer |
| 164 |
than *maxheaderlen*, or if ``0``, do not rewrap any headers. If |
| 165 |
*manheaderlen* is ``None`` (the default), wrap headers and other message |
| 166 |
lines according to the *policy* settings. |
| 167 |
|
| 168 |
If *policy* is specified, use that policy to control message generation. If |
| 169 |
*policy* is ``None`` (the default), use the policy associated with the |
| 170 |
:class:`~email.message.Message` or :class:`~email.message.EmailMessage` |
| 171 |
object passed to ``flatten`` to control the message generation. See |
| 172 |
:mod:`email.policy` for details on what *policy* controls. |
| 173 |
|
| 174 |
.. versionchanged:: 3.3 Added the *policy* keyword. |
| 175 |
|
| 176 |
.. versionchanged:: 3.6 The default behavior of the *mangle_from_* |
| 177 |
and *maxheaderlen* parameters is to follow the policy. |
| 178 |
|
| 179 |
|
| 180 |
.. method:: flatten(msg, unixfrom=False, linesep=None) |
| 181 |
|
| 182 |
Print the textual representation of the message object structure rooted |
| 183 |
at *msg* to the output file specified when the :class:`Generator` |
| 184 |
instance was created. |
| 185 |
|
| 186 |
If the :mod:`~email.policy` option :attr:`~email.policy.Policy.cte_type` |
| 187 |
is ``8bit``, generate the message as if the option were set to ``7bit``. |
| 188 |
(This is required because strings cannot represent non-ASCII bytes.) |
| 189 |
Convert any bytes with the high bit set as needed using an |
| 190 |
ASCII-compatible :mailheader:`Content-Transfer-Encoding`. That is, |
| 191 |
transform parts with non-ASCII :mailheader:`Content-Transfer-Encoding` |
| 192 |
(:mailheader:`Content-Transfer-Encoding: 8bit`) to an ASCII compatible |
| 193 |
:mailheader:`Content-Transfer-Encoding`, and encode RFC-invalid non-ASCII |
| 194 |
bytes in headers using the MIME ``unknown-8bit`` character set, thus |
| 195 |
rendering them RFC-compliant. |
| 196 |
|
| 197 |
If *unixfrom* is ``True``, print the envelope header delimiter used by |
| 198 |
the Unix mailbox format (see :mod:`mailbox`) before the first of the |
| 199 |
:rfc:`5322` headers of the root message object. If the root object has |
| 200 |
no envelope header, craft a standard one. The default is ``False``. |
| 201 |
Note that for subparts, no envelope header is ever printed. |
| 202 |
|
| 203 |
If *linesep* is not ``None``, use it as the separator character between |
| 204 |
all the lines of the flattened message. If *linesep* is ``None`` (the |
| 205 |
default), use the value specified in the *policy*. |
| 206 |
|
| 207 |
.. XXX: flatten should take a *policy* keyword. |
| 208 |
|
| 209 |
.. versionchanged:: 3.2 |
| 210 |
Added support for re-encoding ``8bit`` message bodies, and the |
| 211 |
*linesep* argument. |
| 212 |
|
| 213 |
|
| 214 |
.. method:: clone(fp) |
| 215 |
|
| 216 |
Return an independent clone of this :class:`Generator` instance with the |
| 217 |
exact same options, and *fp* as the new *outfp*. |
| 218 |
|
| 219 |
|
| 220 |
.. method:: write(s) |
| 221 |
|
| 222 |
Write *s* to the *write* method of the *outfp* passed to the |
| 223 |
:class:`Generator`'s constructor. This provides just enough file-like |
| 224 |
API for :class:`Generator` instances to be used in the :func:`print` |
| 225 |
function. |
| 226 |
|
| 227 |
|
| 228 |
As a convenience, :class:`~email.message.EmailMessage` provides the methods |
| 229 |
:meth:`~email.message.EmailMessage.as_string` and ``str(aMessage)`` (a.k.a. |
| 230 |
:meth:`~email.message.EmailMessage.__str__`), which simplify the generation of |
| 231 |
a formatted string representation of a message object. For more detail, see |
| 232 |
:mod:`email.message`. |
| 233 |
|
| 234 |
|
| 235 |
The :mod:`!email.generator` module also provides a derived class, |
| 236 |
:class:`DecodedGenerator`, which is like the :class:`Generator` base class, |
| 237 |
except that non-\ :mimetype:`text` parts are not serialized, but are instead |
| 238 |
represented in the output stream by a string derived from a template filled |
| 239 |
in with information about the part. |
| 240 |
|
| 241 |
.. class:: DecodedGenerator(outfp, mangle_from_=None, maxheaderlen=None, \ |
| 242 |
fmt=None, *, policy=None) |
| 243 |
|
| 244 |
Act like :class:`Generator`, except that for any subpart of the message |
| 245 |
passed to :meth:`Generator.flatten`, if the subpart is of main type |
| 246 |
:mimetype:`text`, print the decoded payload of the subpart, and if the main |
| 247 |
type is not :mimetype:`text`, instead of printing it fill in the string |
| 248 |
*fmt* using information from the part and print the resulting |
| 249 |
filled-in string. |
| 250 |
|
| 251 |
To fill in *fmt*, execute ``fmt % part_info``, where ``part_info`` |
| 252 |
is a dictionary composed of the following keys and values: |
| 253 |
|
| 254 |
* ``type`` -- Full MIME type of the non-\ :mimetype:`text` part |
| 255 |
|
| 256 |
* ``maintype`` -- Main MIME type of the non-\ :mimetype:`text` part |
| 257 |
|
| 258 |
* ``subtype`` -- Sub-MIME type of the non-\ :mimetype:`text` part |
| 259 |
|
| 260 |
* ``filename`` -- Filename of the non-\ :mimetype:`text` part |
| 261 |
|
| 262 |
* ``description`` -- Description associated with the non-\ :mimetype:`text` part |
| 263 |
|
| 264 |
* ``encoding`` -- Content transfer encoding of the non-\ :mimetype:`text` part |
| 265 |
|
| 266 |
If *fmt* is ``None``, use the following default *fmt*: |
| 267 |
|
| 268 |
"[Non-text (%(type)s) part of message omitted, filename %(filename)s]" |
| 269 |
|
| 270 |
Optional *_mangle_from_* and *maxheaderlen* are as with the |
| 271 |
:class:`Generator` base class. |
| 272 |
|
| 273 |
|
| 274 |
.. rubric:: Footnotes |
| 275 |
|
| 276 |
.. [#] This statement assumes that you use the appropriate setting for |
| 277 |
``unixfrom``, and that there are no :mod:`email.policy` settings calling for |
| 278 |
automatic adjustments (for example, |
| 279 |
:attr:`~email.policy.EmailPolicy.refold_source` must be ``none``, which is |
| 280 |
*not* the default). It is also not 100% true, since if the message |
| 281 |
does not conform to the RFC standards occasionally information about the |
| 282 |
exact original text is lost during parsing error recovery. It is a goal |
| 283 |
to fix these latter edge cases when possible. |