| 1 |
:mod:`!marshal` --- Internal Python object serialization |
| 2 |
======================================================== |
| 3 |
|
| 4 |
.. module:: marshal |
| 5 |
:synopsis: Convert Python objects to streams of bytes and back (with different |
| 6 |
constraints). |
| 7 |
|
| 8 |
-------------- |
| 9 |
|
| 10 |
This module contains functions that can read and write Python values in a binary |
| 11 |
format. The format is specific to Python, but independent of machine |
| 12 |
architecture issues (e.g., you can write a Python value to a file on a PC, |
| 13 |
transport the file to a Mac, and read it back there). Details of the format are |
| 14 |
undocumented on purpose; it may change between Python versions (although it |
| 15 |
rarely does). [#]_ |
| 16 |
|
| 17 |
.. index:: |
| 18 |
pair: module; pickle |
| 19 |
pair: module; shelve |
| 20 |
|
| 21 |
This is not a general "persistence" module. For general persistence and |
| 22 |
transfer of Python objects through RPC calls, see the modules :mod:`pickle` and |
| 23 |
:mod:`shelve`. The :mod:`marshal` module exists mainly to support reading and |
| 24 |
writing the "pseudo-compiled" code for Python modules of :file:`.pyc` files. |
| 25 |
Therefore, the Python maintainers reserve the right to modify the marshal format |
| 26 |
in backward incompatible ways should the need arise. |
| 27 |
The format of code objects is not compatible between Python versions, |
| 28 |
even if the version of the format is the same. |
| 29 |
De-serializing a code object in the incorrect Python version has undefined behavior. |
| 30 |
If you're serializing and |
| 31 |
de-serializing Python objects, use the :mod:`pickle` module instead -- the |
| 32 |
performance is comparable, version independence is guaranteed, and pickle |
| 33 |
supports a substantially wider range of objects than marshal. |
| 34 |
|
| 35 |
.. warning:: |
| 36 |
|
| 37 |
The :mod:`marshal` module is not intended to be secure against erroneous or |
| 38 |
maliciously constructed data. Never unmarshal data received from an |
| 39 |
untrusted or unauthenticated source. |
| 40 |
|
| 41 |
.. index:: object; code, code object |
| 42 |
|
| 43 |
Not all Python object types are supported; in general, only objects whose value |
| 44 |
is independent from a particular invocation of Python can be written and read by |
| 45 |
this module. The following types are supported: booleans, integers, floating-point |
| 46 |
numbers, complex numbers, strings, bytes, bytearrays, tuples, lists, sets, |
| 47 |
frozensets, dictionaries, and code objects (if *allow_code* is true), |
| 48 |
where it should be understood that |
| 49 |
tuples, lists, sets, frozensets and dictionaries are only supported as long as |
| 50 |
the values contained therein are themselves supported. The |
| 51 |
singletons :const:`None`, :const:`Ellipsis` and :exc:`StopIteration` can also be |
| 52 |
marshalled and unmarshalled. |
| 53 |
For format *version* lower than 3, recursive lists, sets and dictionaries cannot |
| 54 |
be written (see below). |
| 55 |
|
| 56 |
There are functions that read/write files as well as functions operating on |
| 57 |
bytes-like objects. |
| 58 |
|
| 59 |
The module defines these functions: |
| 60 |
|
| 61 |
|
| 62 |
.. function:: dump(value, file, version=version, /, *, allow_code=True) |
| 63 |
|
| 64 |
Write the value on the open file. The value must be a supported type. The |
| 65 |
file must be a writeable :term:`binary file`. |
| 66 |
|
| 67 |
If the value has (or contains an object that has) an unsupported type, a |
| 68 |
:exc:`ValueError` exception is raised --- but garbage data will also be written |
| 69 |
to the file. The object will not be properly read back by :func:`load`. |
| 70 |
:ref:`Code objects <code-objects>` are only supported if *allow_code* is true. |
| 71 |
|
| 72 |
The *version* argument indicates the data format that ``dump`` should use |
| 73 |
(see below). |
| 74 |
|
| 75 |
.. audit-event:: marshal.dumps value,version marshal.dump |
| 76 |
|
| 77 |
.. versionchanged:: 3.13 |
| 78 |
Added the *allow_code* parameter. |
| 79 |
|
| 80 |
|
| 81 |
.. function:: load(file, /, *, allow_code=True) |
| 82 |
|
| 83 |
Read one value from the open file and return it. If no valid value is read |
| 84 |
(e.g. because the data has a different Python version's incompatible marshal |
| 85 |
format), raise :exc:`EOFError`, :exc:`ValueError` or :exc:`TypeError`. |
| 86 |
:ref:`Code objects <code-objects>` are only supported if *allow_code* is true. |
| 87 |
The file must be a readable :term:`binary file`. |
| 88 |
|
| 89 |
.. audit-event:: marshal.load "" marshal.load |
| 90 |
|
| 91 |
.. note:: |
| 92 |
|
| 93 |
If an object containing an unsupported type was marshalled with :func:`dump`, |
| 94 |
:func:`load` will substitute ``None`` for the unmarshallable type. |
| 95 |
|
| 96 |
.. versionchanged:: 3.10 |
| 97 |
|
| 98 |
This call used to raise a ``code.__new__`` audit event for each code object. Now |
| 99 |
it raises a single ``marshal.load`` event for the entire load operation. |
| 100 |
|
| 101 |
.. versionchanged:: 3.13 |
| 102 |
Added the *allow_code* parameter. |
| 103 |
|
| 104 |
|
| 105 |
.. function:: dumps(value, version=version, /, *, allow_code=True) |
| 106 |
|
| 107 |
Return the bytes object that would be written to a file by ``dump(value, file)``. The |
| 108 |
value must be a supported type. Raise a :exc:`ValueError` exception if value |
| 109 |
has (or contains an object that has) an unsupported type. |
| 110 |
:ref:`Code objects <code-objects>` are only supported if *allow_code* is true. |
| 111 |
|
| 112 |
The *version* argument indicates the data format that ``dumps`` should use |
| 113 |
(see below). |
| 114 |
|
| 115 |
.. audit-event:: marshal.dumps value,version marshal.dump |
| 116 |
|
| 117 |
.. versionchanged:: 3.13 |
| 118 |
Added the *allow_code* parameter. |
| 119 |
|
| 120 |
|
| 121 |
.. function:: loads(bytes, /, *, allow_code=True) |
| 122 |
|
| 123 |
Convert the :term:`bytes-like object` to a value. If no valid value is found, raise |
| 124 |
:exc:`EOFError`, :exc:`ValueError` or :exc:`TypeError`. |
| 125 |
:ref:`Code objects <code-objects>` are only supported if *allow_code* is true. |
| 126 |
Extra bytes in the input are ignored. |
| 127 |
|
| 128 |
.. audit-event:: marshal.loads bytes marshal.load |
| 129 |
|
| 130 |
.. versionchanged:: 3.10 |
| 131 |
|
| 132 |
This call used to raise a ``code.__new__`` audit event for each code object. Now |
| 133 |
it raises a single ``marshal.loads`` event for the entire load operation. |
| 134 |
|
| 135 |
.. versionchanged:: 3.13 |
| 136 |
Added the *allow_code* parameter. |
| 137 |
|
| 138 |
|
| 139 |
In addition, the following constants are defined: |
| 140 |
|
| 141 |
.. data:: version |
| 142 |
|
| 143 |
Indicates the format that the module uses. Version 0 is the historical |
| 144 |
format, version 1 shares interned strings and version 2 uses a binary format |
| 145 |
for floating-point numbers. |
| 146 |
Version 3 adds support for object instancing and recursion. |
| 147 |
The current version is 4. |
| 148 |
|
| 149 |
|
| 150 |
.. rubric:: Footnotes |
| 151 |
|
| 152 |
.. [#] The name of this module stems from a bit of terminology used by the designers of |
| 153 |
Modula-3 (amongst others), who use the term "marshalling" for shipping of data |
| 154 |
around in a self-contained form. Strictly speaking, "to marshal" means to |
| 155 |
convert some data from internal to external form (in an RPC buffer for instance) |
| 156 |
and "unmarshalling" for the reverse process. |
| 157 |
|