| 1 |
:mod:`!marshal` --- Internal Python object serialization |
| 2 |
======================================================== |
| 3 |
|
| 4 |
.. module:: marshal |
| 5 |
:synopsis: Convert Python objects to streams of bytes and back (with different |
| 6 |
constraints). |
| 7 |
|
| 8 |
-------------- |
| 9 |
|
| 10 |
This module contains functions that can read and write Python values in a binary |
| 11 |
format. The format is specific to Python, but independent of machine |
| 12 |
architecture issues (e.g., you can write a Python value to a file on a PC, |
| 13 |
transport the file to a Mac, and read it back there). Details of the format are |
| 14 |
undocumented on purpose; it may change between Python versions (although it |
| 15 |
rarely does). [#]_ |
| 16 |
|
| 17 |
.. index:: |
| 18 |
pair: module; pickle |
| 19 |
pair: module; shelve |
| 20 |
|
| 21 |
This is not a general "persistence" module. For general persistence and |
| 22 |
transfer of Python objects through RPC calls, see the modules :mod:`pickle` and |
| 23 |
:mod:`shelve`. The :mod:`!marshal` module exists mainly to support reading and |
| 24 |
writing the "pseudo-compiled" code for Python modules of :file:`.pyc` files. |
| 25 |
Therefore, the Python maintainers reserve the right to modify the marshal format |
| 26 |
in backward incompatible ways should the need arise. |
| 27 |
The format of code objects is not compatible between Python versions, |
| 28 |
even if the version of the format is the same. |
| 29 |
De-serializing a code object in the incorrect Python version has undefined behavior. |
| 30 |
If you're serializing and |
| 31 |
de-serializing Python objects, use the :mod:`pickle` module instead -- the |
| 32 |
performance is comparable, version independence is guaranteed, and pickle |
| 33 |
supports a substantially wider range of objects than marshal. |
| 34 |
|
| 35 |
.. warning:: |
| 36 |
|
| 37 |
The :mod:`!marshal` module is not intended to be secure against erroneous or |
| 38 |
maliciously constructed data. Never unmarshal data received from an |
| 39 |
untrusted or unauthenticated source. |
| 40 |
|
| 41 |
There are functions that read/write files as well as functions operating on |
| 42 |
bytes-like objects. |
| 43 |
|
| 44 |
.. index:: object; code, code object |
| 45 |
|
| 46 |
Not all Python object types are supported; in general, only objects whose value |
| 47 |
is independent from a particular invocation of Python can be written and read by |
| 48 |
this module. The following types are supported: |
| 49 |
|
| 50 |
* Numeric types: :class:`int`, :class:`bool`, :class:`float`, :class:`complex`. |
| 51 |
* Strings (:class:`str`) and :class:`bytes`. |
| 52 |
:term:`Bytes-like objects <bytes-like object>` like :class:`bytearray` are |
| 53 |
marshalled as :class:`!bytes`. |
| 54 |
* Containers: :class:`tuple`, :class:`list`, :class:`set`, :class:`frozenset`, |
| 55 |
and (since :data:`version` 5), :class:`slice`. |
| 56 |
It should be understood that these are supported only if the values contained |
| 57 |
therein are themselves supported. |
| 58 |
Recursive containers are supported since :data:`version` 3. |
| 59 |
* The singletons :const:`None`, :const:`Ellipsis` and :exc:`StopIteration`. |
| 60 |
* :class:`code` objects, if *allow_code* is true. See note above about |
| 61 |
version dependence. |
| 62 |
|
| 63 |
.. versionchanged:: 3.4 |
| 64 |
|
| 65 |
* Added format version 3, which supports marshalling recursive lists, sets |
| 66 |
and dictionaries. |
| 67 |
* Added format version 4, which supports efficient representations |
| 68 |
of short strings. |
| 69 |
|
| 70 |
.. versionchanged:: 3.14 |
| 71 |
|
| 72 |
Added format version 5, which allows marshalling slices. |
| 73 |
|
| 74 |
|
| 75 |
The module defines these functions: |
| 76 |
|
| 77 |
|
| 78 |
.. function:: dump(value, file, version=version, /, *, allow_code=True) |
| 79 |
|
| 80 |
Write the value on the open file. The value must be a supported type. The |
| 81 |
file must be a writeable :term:`binary file`. |
| 82 |
|
| 83 |
If the value has (or contains an object that has) an unsupported type, a |
| 84 |
:exc:`ValueError` exception is raised --- but garbage data will also be written |
| 85 |
to the file. The object will not be properly read back by :func:`load`. |
| 86 |
:ref:`Code objects <code-objects>` are only supported if *allow_code* is true. |
| 87 |
|
| 88 |
The *version* argument indicates the data format that ``dump`` should use |
| 89 |
(see below). |
| 90 |
|
| 91 |
.. audit-event:: marshal.dumps value,version marshal.dump |
| 92 |
|
| 93 |
.. versionchanged:: 3.13 |
| 94 |
Added the *allow_code* parameter. |
| 95 |
|
| 96 |
|
| 97 |
.. function:: load(file, /, *, allow_code=True) |
| 98 |
|
| 99 |
Read one value from the open file and return it. If no valid value is read |
| 100 |
(e.g. because the data has a different Python version's incompatible marshal |
| 101 |
format), raise :exc:`EOFError`, :exc:`ValueError` or :exc:`TypeError`. |
| 102 |
:ref:`Code objects <code-objects>` are only supported if *allow_code* is true. |
| 103 |
The file must be a readable :term:`binary file`. |
| 104 |
|
| 105 |
.. audit-event:: marshal.load "" marshal.load |
| 106 |
|
| 107 |
.. note:: |
| 108 |
|
| 109 |
If an object containing an unsupported type was marshalled with :func:`dump`, |
| 110 |
:func:`load` will substitute ``None`` for the unmarshallable type. |
| 111 |
|
| 112 |
.. versionchanged:: 3.10 |
| 113 |
|
| 114 |
This call used to raise a ``code.__new__`` audit event for each code object. Now |
| 115 |
it raises a single ``marshal.load`` event for the entire load operation. |
| 116 |
|
| 117 |
.. versionchanged:: 3.13 |
| 118 |
Added the *allow_code* parameter. |
| 119 |
|
| 120 |
|
| 121 |
.. function:: dumps(value, version=version, /, *, allow_code=True) |
| 122 |
|
| 123 |
Return the bytes object that would be written to a file by ``dump(value, file)``. The |
| 124 |
value must be a supported type. Raise a :exc:`ValueError` exception if value |
| 125 |
has (or contains an object that has) an unsupported type. |
| 126 |
:ref:`Code objects <code-objects>` are only supported if *allow_code* is true. |
| 127 |
|
| 128 |
The *version* argument indicates the data format that ``dumps`` should use |
| 129 |
(see below). |
| 130 |
|
| 131 |
.. audit-event:: marshal.dumps value,version marshal.dump |
| 132 |
|
| 133 |
.. versionchanged:: 3.13 |
| 134 |
Added the *allow_code* parameter. |
| 135 |
|
| 136 |
|
| 137 |
.. function:: loads(bytes, /, *, allow_code=True) |
| 138 |
|
| 139 |
Convert the :term:`bytes-like object` to a value. If no valid value is found, raise |
| 140 |
:exc:`EOFError`, :exc:`ValueError` or :exc:`TypeError`. |
| 141 |
:ref:`Code objects <code-objects>` are only supported if *allow_code* is true. |
| 142 |
Extra bytes in the input are ignored. |
| 143 |
|
| 144 |
.. audit-event:: marshal.loads bytes marshal.load |
| 145 |
|
| 146 |
.. versionchanged:: 3.10 |
| 147 |
|
| 148 |
This call used to raise a ``code.__new__`` audit event for each code object. Now |
| 149 |
it raises a single ``marshal.loads`` event for the entire load operation. |
| 150 |
|
| 151 |
.. versionchanged:: 3.13 |
| 152 |
Added the *allow_code* parameter. |
| 153 |
|
| 154 |
|
| 155 |
In addition, the following constants are defined: |
| 156 |
|
| 157 |
.. data:: version |
| 158 |
|
| 159 |
Indicates the format that the module uses. |
| 160 |
Version 0 is the historical first version; subsequent versions |
| 161 |
add new features. |
| 162 |
Generally, a new version becomes the default when it is introduced. |
| 163 |
|
| 164 |
======= =============== ==================================================== |
| 165 |
Version Available since New features |
| 166 |
======= =============== ==================================================== |
| 167 |
1 Python 2.4 Sharing interned strings |
| 168 |
------- --------------- ---------------------------------------------------- |
| 169 |
2 Python 2.5 Binary representation of floats |
| 170 |
------- --------------- ---------------------------------------------------- |
| 171 |
3 Python 3.4 Support for object instancing and recursion |
| 172 |
------- --------------- ---------------------------------------------------- |
| 173 |
4 Python 3.4 Efficient representation of short strings |
| 174 |
------- --------------- ---------------------------------------------------- |
| 175 |
5 Python 3.14 Support for :class:`slice` objects |
| 176 |
======= =============== ==================================================== |
| 177 |
|
| 178 |
|
| 179 |
.. rubric:: Footnotes |
| 180 |
|
| 181 |
.. [#] The name of this module stems from a bit of terminology used by the designers of |
| 182 |
Modula-3 (amongst others), who use the term "marshalling" for shipping of data |
| 183 |
around in a self-contained form. Strictly speaking, "to marshal" means to |
| 184 |
convert some data from internal to external form (in an RPC buffer for instance) |
| 185 |
and "unmarshalling" for the reverse process. |