Debian Codegraph @

Doc/library/marshal.rst

raw
1 :mod:`!marshal` --- Internal Python object serialization
2 ========================================================
3
4 .. module:: marshal
5 :synopsis: Convert Python objects to streams of bytes and back (with different
6 constraints).
7
8 --------------
9
10 This module contains functions that can read and write Python values in a binary
11 format. The format is specific to Python, but independent of machine
12 architecture issues (e.g., you can write a Python value to a file on a PC,
13 transport the file to a Mac, and read it back there). Details of the format are
14 undocumented on purpose; it may change between Python versions (although it
15 rarely does). [#]_
16
17 .. index::
18 pair: module; pickle
19 pair: module; shelve
20
21 This is not a general "persistence" module. For general persistence and
22 transfer of Python objects through RPC calls, see the modules :mod:`pickle` and
23 :mod:`shelve`. The :mod:`!marshal` module exists mainly to support reading and
24 writing the "pseudo-compiled" code for Python modules of :file:`.pyc` files.
25 Therefore, the Python maintainers reserve the right to modify the marshal format
26 in backward incompatible ways should the need arise.
27 The format of code objects is not compatible between Python versions,
28 even if the version of the format is the same.
29 De-serializing a code object in the incorrect Python version has undefined behavior.
30 If you're serializing and
31 de-serializing Python objects, use the :mod:`pickle` module instead -- the
32 performance is comparable, version independence is guaranteed, and pickle
33 supports a substantially wider range of objects than marshal.
34
35 .. warning::
36
37 The :mod:`!marshal` module is not intended to be secure against erroneous or
38 maliciously constructed data. Never unmarshal data received from an
39 untrusted or unauthenticated source.
40
41 There are functions that read/write files as well as functions operating on
42 bytes-like objects.
43
44 .. index:: object; code, code object
45
46 Not all Python object types are supported; in general, only objects whose value
47 is independent from a particular invocation of Python can be written and read by
48 this module. The following types are supported:
49
50 * Numeric types: :class:`int`, :class:`bool`, :class:`float`, :class:`complex`.
51 * Strings (:class:`str`) and :class:`bytes`.
52 :term:`Bytes-like objects <bytes-like object>` like :class:`bytearray` are
53 marshalled as :class:`!bytes`.
54 * Containers: :class:`tuple`, :class:`list`, :class:`set`, :class:`frozenset`,
55 and (since :data:`version` 5), :class:`slice`.
56 It should be understood that these are supported only if the values contained
57 therein are themselves supported.
58 Recursive containers are supported since :data:`version` 3.
59 * The singletons :const:`None`, :const:`Ellipsis` and :exc:`StopIteration`.
60 * :class:`code` objects, if *allow_code* is true. See note above about
61 version dependence.
62
63 .. versionchanged:: 3.4
64
65 * Added format version 3, which supports marshalling recursive lists, sets
66 and dictionaries.
67 * Added format version 4, which supports efficient representations
68 of short strings.
69
70 .. versionchanged:: 3.14
71
72 Added format version 5, which allows marshalling slices.
73
74
75 The module defines these functions:
76
77
78 .. function:: dump(value, file, version=version, /, *, allow_code=True)
79
80 Write the value on the open file. The value must be a supported type. The
81 file must be a writeable :term:`binary file`.
82
83 If the value has (or contains an object that has) an unsupported type, a
84 :exc:`ValueError` exception is raised --- but garbage data will also be written
85 to the file. The object will not be properly read back by :func:`load`.
86 :ref:`Code objects <code-objects>` are only supported if *allow_code* is true.
87
88 The *version* argument indicates the data format that ``dump`` should use
89 (see below).
90
91 .. audit-event:: marshal.dumps value,version marshal.dump
92
93 .. versionchanged:: 3.13
94 Added the *allow_code* parameter.
95
96
97 .. function:: load(file, /, *, allow_code=True)
98
99 Read one value from the open file and return it. If no valid value is read
100 (e.g. because the data has a different Python version's incompatible marshal
101 format), raise :exc:`EOFError`, :exc:`ValueError` or :exc:`TypeError`.
102 :ref:`Code objects <code-objects>` are only supported if *allow_code* is true.
103 The file must be a readable :term:`binary file`.
104
105 .. audit-event:: marshal.load "" marshal.load
106
107 .. note::
108
109 If an object containing an unsupported type was marshalled with :func:`dump`,
110 :func:`load` will substitute ``None`` for the unmarshallable type.
111
112 .. versionchanged:: 3.10
113
114 This call used to raise a ``code.__new__`` audit event for each code object. Now
115 it raises a single ``marshal.load`` event for the entire load operation.
116
117 .. versionchanged:: 3.13
118 Added the *allow_code* parameter.
119
120
121 .. function:: dumps(value, version=version, /, *, allow_code=True)
122
123 Return the bytes object that would be written to a file by ``dump(value, file)``. The
124 value must be a supported type. Raise a :exc:`ValueError` exception if value
125 has (or contains an object that has) an unsupported type.
126 :ref:`Code objects <code-objects>` are only supported if *allow_code* is true.
127
128 The *version* argument indicates the data format that ``dumps`` should use
129 (see below).
130
131 .. audit-event:: marshal.dumps value,version marshal.dump
132
133 .. versionchanged:: 3.13
134 Added the *allow_code* parameter.
135
136
137 .. function:: loads(bytes, /, *, allow_code=True)
138
139 Convert the :term:`bytes-like object` to a value. If no valid value is found, raise
140 :exc:`EOFError`, :exc:`ValueError` or :exc:`TypeError`.
141 :ref:`Code objects <code-objects>` are only supported if *allow_code* is true.
142 Extra bytes in the input are ignored.
143
144 .. audit-event:: marshal.loads bytes marshal.load
145
146 .. versionchanged:: 3.10
147
148 This call used to raise a ``code.__new__`` audit event for each code object. Now
149 it raises a single ``marshal.loads`` event for the entire load operation.
150
151 .. versionchanged:: 3.13
152 Added the *allow_code* parameter.
153
154
155 In addition, the following constants are defined:
156
157 .. data:: version
158
159 Indicates the format that the module uses.
160 Version 0 is the historical first version; subsequent versions
161 add new features.
162 Generally, a new version becomes the default when it is introduced.
163
164 ======= =============== ====================================================
165 Version Available since New features
166 ======= =============== ====================================================
167 1 Python 2.4 Sharing interned strings
168 ------- --------------- ----------------------------------------------------
169 2 Python 2.5 Binary representation of floats
170 ------- --------------- ----------------------------------------------------
171 3 Python 3.4 Support for object instancing and recursion
172 ------- --------------- ----------------------------------------------------
173 4 Python 3.4 Efficient representation of short strings
174 ------- --------------- ----------------------------------------------------
175 5 Python 3.14 Support for :class:`slice` objects
176 ======= =============== ====================================================
177
178
179 .. rubric:: Footnotes
180
181 .. [#] The name of this module stems from a bit of terminology used by the designers of
182 Modula-3 (amongst others), who use the term "marshalling" for shipping of data
183 around in a self-contained form. Strictly speaking, "to marshal" means to
184 convert some data from internal to external form (in an RPC buffer for instance)
185 and "unmarshalling" for the reverse process.