| 1 |
:mod:`!struct` --- Interpret bytes as packed binary data |
| 2 |
======================================================== |
| 3 |
|
| 4 |
.. testsetup:: * |
| 5 |
|
| 6 |
from struct import * |
| 7 |
|
| 8 |
.. module:: struct |
| 9 |
:synopsis: Interpret bytes as packed binary data. |
| 10 |
|
| 11 |
**Source code:** :source:`Lib/struct.py` |
| 12 |
|
| 13 |
.. index:: |
| 14 |
pair: C; structures |
| 15 |
triple: packing; binary; data |
| 16 |
|
| 17 |
-------------- |
| 18 |
|
| 19 |
This module converts between Python values and C structs represented |
| 20 |
as Python :class:`bytes` objects. Compact :ref:`format strings <struct-format-strings>` |
| 21 |
describe the intended conversions to/from Python values. |
| 22 |
The module's functions and objects can be used for two largely |
| 23 |
distinct applications, data exchange with external sources (files or |
| 24 |
network connections), or data transfer between the Python application |
| 25 |
and the C layer. |
| 26 |
|
| 27 |
.. note:: |
| 28 |
|
| 29 |
When no prefix character is given, native mode is the default. It |
| 30 |
packs or unpacks data based on the platform and compiler on which |
| 31 |
the Python interpreter was built. |
| 32 |
The result of packing a given C struct includes pad bytes which |
| 33 |
maintain proper alignment for the C types involved; similarly, |
| 34 |
alignment is taken into account when unpacking. In contrast, when |
| 35 |
communicating data between external sources, the programmer is |
| 36 |
responsible for defining byte ordering and padding between elements. |
| 37 |
See :ref:`struct-alignment` for details. |
| 38 |
|
| 39 |
Several :mod:`!struct` functions (and methods of :class:`Struct`) take a *buffer* |
| 40 |
argument. This refers to objects that implement the :ref:`bufferobjects` and |
| 41 |
provide either a readable or read-writable buffer. The most common types used |
| 42 |
for that purpose are :class:`bytes` and :class:`bytearray`, but many other types |
| 43 |
that can be viewed as an array of bytes implement the buffer protocol, so that |
| 44 |
they can be read/filled without additional copying from a :class:`bytes` object. |
| 45 |
|
| 46 |
|
| 47 |
Functions and Exceptions |
| 48 |
------------------------ |
| 49 |
|
| 50 |
The module defines the following exception and functions: |
| 51 |
|
| 52 |
|
| 53 |
.. exception:: error |
| 54 |
|
| 55 |
Exception raised on various occasions; argument is a string describing what |
| 56 |
is wrong. |
| 57 |
|
| 58 |
|
| 59 |
.. function:: pack(format, v1, v2, ...) |
| 60 |
|
| 61 |
Return a bytes object containing the values *v1*, *v2*, ... packed according |
| 62 |
to the format string *format*. The arguments must match the values required by |
| 63 |
the format exactly. |
| 64 |
|
| 65 |
|
| 66 |
.. function:: pack_into(format, buffer, offset, v1, v2, ...) |
| 67 |
|
| 68 |
Pack the values *v1*, *v2*, ... according to the format string *format* and |
| 69 |
write the packed bytes into the writable buffer *buffer* starting at |
| 70 |
position *offset*. Note that *offset* is a required argument. |
| 71 |
|
| 72 |
|
| 73 |
.. function:: unpack(format, buffer) |
| 74 |
|
| 75 |
Unpack from the buffer *buffer* (presumably packed by ``pack(format, ...)``) |
| 76 |
according to the format string *format*. The result is a tuple even if it |
| 77 |
contains exactly one item. The buffer's size in bytes must match the |
| 78 |
size required by the format, as reflected by :func:`calcsize`. |
| 79 |
|
| 80 |
|
| 81 |
.. function:: unpack_from(format, /, buffer, offset=0) |
| 82 |
|
| 83 |
Unpack from *buffer* starting at position *offset*, according to the format |
| 84 |
string *format*. The result is a tuple even if it contains exactly one |
| 85 |
item. The buffer's size in bytes, starting at position *offset*, must be at |
| 86 |
least the size required by the format, as reflected by :func:`calcsize`. |
| 87 |
|
| 88 |
|
| 89 |
.. function:: iter_unpack(format, buffer) |
| 90 |
|
| 91 |
Iteratively unpack from the buffer *buffer* according to the format |
| 92 |
string *format*. This function returns an iterator which will read |
| 93 |
equally sized chunks from the buffer until all its contents have been |
| 94 |
consumed. The buffer's size in bytes must be a multiple of the size |
| 95 |
required by the format, as reflected by :func:`calcsize`. |
| 96 |
|
| 97 |
Each iteration yields a tuple as specified by the format string. |
| 98 |
|
| 99 |
.. versionadded:: 3.4 |
| 100 |
|
| 101 |
|
| 102 |
.. function:: calcsize(format) |
| 103 |
|
| 104 |
Return the size of the struct (and hence of the bytes object produced by |
| 105 |
``pack(format, ...)``) corresponding to the format string *format*. |
| 106 |
|
| 107 |
|
| 108 |
.. _struct-format-strings: |
| 109 |
|
| 110 |
Format Strings |
| 111 |
-------------- |
| 112 |
|
| 113 |
Format strings describe the data layout when |
| 114 |
packing and unpacking data. They are built up from :ref:`format characters<format-characters>`, |
| 115 |
which specify the type of data being packed/unpacked. In addition, |
| 116 |
special characters control the :ref:`byte order, size and alignment<struct-alignment>`. |
| 117 |
Each format string consists of an optional prefix character which |
| 118 |
describes the overall properties of the data and one or more format |
| 119 |
characters which describe the actual data values and padding. |
| 120 |
|
| 121 |
|
| 122 |
.. _struct-alignment: |
| 123 |
|
| 124 |
Byte Order, Size, and Alignment |
| 125 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 126 |
|
| 127 |
By default, C types are represented in the machine's native format and byte |
| 128 |
order, and properly aligned by skipping pad bytes if necessary (according to the |
| 129 |
rules used by the C compiler). |
| 130 |
This behavior is chosen so |
| 131 |
that the bytes of a packed struct correspond exactly to the memory layout |
| 132 |
of the corresponding C struct. |
| 133 |
Whether to use native byte ordering |
| 134 |
and padding or standard formats depends on the application. |
| 135 |
|
| 136 |
.. index:: |
| 137 |
single: @ (at); in struct format strings |
| 138 |
single: = (equals); in struct format strings |
| 139 |
single: < (less); in struct format strings |
| 140 |
single: > (greater); in struct format strings |
| 141 |
single: ! (exclamation); in struct format strings |
| 142 |
|
| 143 |
Alternatively, the first character of the format string can be used to indicate |
| 144 |
the byte order, size and alignment of the packed data, according to the |
| 145 |
following table: |
| 146 |
|
| 147 |
+-----------+------------------------+----------+-----------+ |
| 148 |
| Character | Byte order | Size | Alignment | |
| 149 |
+===========+========================+==========+===========+ |
| 150 |
| ``@`` | native | native | native | |
| 151 |
+-----------+------------------------+----------+-----------+ |
| 152 |
| ``=`` | native | standard | none | |
| 153 |
+-----------+------------------------+----------+-----------+ |
| 154 |
| ``<`` | little-endian | standard | none | |
| 155 |
+-----------+------------------------+----------+-----------+ |
| 156 |
| ``>`` | big-endian | standard | none | |
| 157 |
+-----------+------------------------+----------+-----------+ |
| 158 |
| ``!`` | network (= big-endian) | standard | none | |
| 159 |
+-----------+------------------------+----------+-----------+ |
| 160 |
|
| 161 |
If the first character is not one of these, ``'@'`` is assumed. |
| 162 |
|
| 163 |
.. note:: |
| 164 |
|
| 165 |
The number 1023 (``0x3ff`` in hexadecimal) has the following byte representations: |
| 166 |
|
| 167 |
* ``03 ff`` in big-endian (``>``) |
| 168 |
* ``ff 03`` in little-endian (``<``) |
| 169 |
|
| 170 |
Python example: |
| 171 |
|
| 172 |
>>> import struct |
| 173 |
>>> struct.pack('>h', 1023) |
| 174 |
b'\x03\xff' |
| 175 |
>>> struct.pack('<h', 1023) |
| 176 |
b'\xff\x03' |
| 177 |
|
| 178 |
Native byte order is big-endian or little-endian, depending on the |
| 179 |
host system. For example, Intel x86, AMD64 (x86-64), and Apple M1 are |
| 180 |
little-endian; IBM z and many legacy architectures are big-endian. |
| 181 |
Use :data:`sys.byteorder` to check the endianness of your system. |
| 182 |
|
| 183 |
Native size and alignment are determined using the C compiler's |
| 184 |
``sizeof`` expression. This is always combined with native byte order. |
| 185 |
|
| 186 |
Standard size depends only on the format character; see the table in |
| 187 |
the :ref:`format-characters` section. |
| 188 |
|
| 189 |
Note the difference between ``'@'`` and ``'='``: both use native byte order, but |
| 190 |
the size and alignment of the latter is standardized. |
| 191 |
|
| 192 |
The form ``'!'`` represents the network byte order which is always big-endian |
| 193 |
as defined in `IETF RFC 1700 <IETF RFC 1700_>`_. |
| 194 |
|
| 195 |
There is no way to indicate non-native byte order (force byte-swapping); use the |
| 196 |
appropriate choice of ``'<'`` or ``'>'``. |
| 197 |
|
| 198 |
Notes: |
| 199 |
|
| 200 |
(1) Padding is only automatically added between successive structure members. |
| 201 |
No padding is added at the beginning or the end of the encoded struct. |
| 202 |
|
| 203 |
(2) No padding is added when using non-native size and alignment, e.g. |
| 204 |
with '<', '>', '=', and '!'. |
| 205 |
|
| 206 |
(3) To align the end of a structure to the alignment requirement of a |
| 207 |
particular type, end the format with the code for that type with a repeat |
| 208 |
count of zero. See :ref:`struct-examples`. |
| 209 |
|
| 210 |
|
| 211 |
.. _format-characters: |
| 212 |
|
| 213 |
Format Characters |
| 214 |
^^^^^^^^^^^^^^^^^ |
| 215 |
|
| 216 |
Format characters have the following meaning; the conversion between C and |
| 217 |
Python values should be obvious given their types. The 'Standard size' column |
| 218 |
refers to the size of the packed value in bytes when using standard size; that |
| 219 |
is, when the format string starts with one of ``'<'``, ``'>'``, ``'!'`` or |
| 220 |
``'='``. When using native size, the size of the packed value is |
| 221 |
platform-dependent. |
| 222 |
|
| 223 |
+--------+--------------------------+--------------------+----------------+------------+ |
| 224 |
| Format | C Type | Python type | Standard size | Notes | |
| 225 |
+========+==========================+====================+================+============+ |
| 226 |
| ``x`` | pad byte | no value | | \(7) | |
| 227 |
+--------+--------------------------+--------------------+----------------+------------+ |
| 228 |
| ``c`` | :c:expr:`char` | bytes of length 1 | 1 | | |
| 229 |
+--------+--------------------------+--------------------+----------------+------------+ |
| 230 |
| ``b`` | :c:expr:`signed char` | int | 1 | \(2) | |
| 231 |
+--------+--------------------------+--------------------+----------------+------------+ |
| 232 |
| ``B`` | :c:expr:`unsigned char` | int | 1 | \(2) | |
| 233 |
+--------+--------------------------+--------------------+----------------+------------+ |
| 234 |
| ``?`` | :c:expr:`_Bool` | bool | 1 | \(1) | |
| 235 |
+--------+--------------------------+--------------------+----------------+------------+ |
| 236 |
| ``h`` | :c:expr:`short` | int | 2 | \(2) | |
| 237 |
+--------+--------------------------+--------------------+----------------+------------+ |
| 238 |
| ``H`` | :c:expr:`unsigned short` | int | 2 | \(2) | |
| 239 |
+--------+--------------------------+--------------------+----------------+------------+ |
| 240 |
| ``i`` | :c:expr:`int` | int | 4 | \(2) | |
| 241 |
+--------+--------------------------+--------------------+----------------+------------+ |
| 242 |
| ``I`` | :c:expr:`unsigned int` | int | 4 | \(2) | |
| 243 |
+--------+--------------------------+--------------------+----------------+------------+ |
| 244 |
| ``l`` | :c:expr:`long` | int | 4 | \(2) | |
| 245 |
+--------+--------------------------+--------------------+----------------+------------+ |
| 246 |
| ``L`` | :c:expr:`unsigned long` | int | 4 | \(2) | |
| 247 |
+--------+--------------------------+--------------------+----------------+------------+ |
| 248 |
| ``q`` | :c:expr:`long long` | int | 8 | \(2) | |
| 249 |
+--------+--------------------------+--------------------+----------------+------------+ |
| 250 |
| ``Q`` | :c:expr:`unsigned long | int | 8 | \(2) | |
| 251 |
| | long` | | | | |
| 252 |
+--------+--------------------------+--------------------+----------------+------------+ |
| 253 |
| ``n`` | :c:type:`ssize_t` | int | | \(2), \(3) | |
| 254 |
+--------+--------------------------+--------------------+----------------+------------+ |
| 255 |
| ``N`` | :c:type:`size_t` | int | | \(2), \(3) | |
| 256 |
+--------+--------------------------+--------------------+----------------+------------+ |
| 257 |
| ``e`` | :c:expr:`_Float16` | float | 2 | \(4), \(6) | |
| 258 |
+--------+--------------------------+--------------------+----------------+------------+ |
| 259 |
| ``f`` | :c:expr:`float` | float | 4 | \(4) | |
| 260 |
+--------+--------------------------+--------------------+----------------+------------+ |
| 261 |
| ``d`` | :c:expr:`double` | float | 8 | \(4) | |
| 262 |
+--------+--------------------------+--------------------+----------------+------------+ |
| 263 |
| ``F`` | :c:expr:`float complex` | complex | 8 | \(10) | |
| 264 |
+--------+--------------------------+--------------------+----------------+------------+ |
| 265 |
| ``D`` | :c:expr:`double complex` | complex | 16 | \(10) | |
| 266 |
+--------+--------------------------+--------------------+----------------+------------+ |
| 267 |
| ``s`` | :c:expr:`char[]` | bytes | | \(9) | |
| 268 |
+--------+--------------------------+--------------------+----------------+------------+ |
| 269 |
| ``p`` | :c:expr:`char[]` | bytes | | \(8) | |
| 270 |
+--------+--------------------------+--------------------+----------------+------------+ |
| 271 |
| ``P`` | :c:expr:`void \*` | int | | \(2), \(5) | |
| 272 |
+--------+--------------------------+--------------------+----------------+------------+ |
| 273 |
|
| 274 |
.. versionchanged:: 3.3 |
| 275 |
Added support for the ``'n'`` and ``'N'`` formats. |
| 276 |
|
| 277 |
.. versionchanged:: 3.6 |
| 278 |
Added support for the ``'e'`` format. |
| 279 |
|
| 280 |
.. versionchanged:: 3.14 |
| 281 |
Added support for the ``'F'`` and ``'D'`` formats. |
| 282 |
|
| 283 |
.. seealso:: |
| 284 |
|
| 285 |
The :mod:`array` and :ref:`ctypes <ctypes-fundamental-data-types>` modules, |
| 286 |
as well as third-party modules like `numpy <https://numpy.org/doc/stable/reference/arrays.interface.html#object.__array_interface__>`__, |
| 287 |
use similar -- but slightly different -- type codes. |
| 288 |
|
| 289 |
|
| 290 |
Notes: |
| 291 |
|
| 292 |
(1) |
| 293 |
.. index:: single: ? (question mark); in struct format strings |
| 294 |
|
| 295 |
The ``'?'`` conversion code corresponds to the :c:expr:`_Bool` type |
| 296 |
defined by C standards since C99. In standard mode, it is |
| 297 |
represented by one byte. |
| 298 |
|
| 299 |
(2) |
| 300 |
When attempting to pack a non-integer using any of the integer conversion |
| 301 |
codes, if the non-integer has a :meth:`~object.__index__` method then that method is |
| 302 |
called to convert the argument to an integer before packing. |
| 303 |
|
| 304 |
.. versionchanged:: 3.2 |
| 305 |
Added use of the :meth:`~object.__index__` method for non-integers. |
| 306 |
|
| 307 |
(3) |
| 308 |
The ``'n'`` and ``'N'`` conversion codes are only available for the native |
| 309 |
size (selected as the default or with the ``'@'`` byte order character). |
| 310 |
For the standard size, you can use whichever of the other integer formats |
| 311 |
fits your application. |
| 312 |
|
| 313 |
(4) |
| 314 |
For the ``'f'``, ``'d'`` and ``'e'`` conversion codes, the packed |
| 315 |
representation uses the IEEE 754 binary32, binary64 or binary16 format (for |
| 316 |
``'f'``, ``'d'`` or ``'e'`` respectively), regardless of the floating-point |
| 317 |
format used by the platform. |
| 318 |
|
| 319 |
(5) |
| 320 |
The ``'P'`` format character is only available for the native byte ordering |
| 321 |
(selected as the default or with the ``'@'`` byte order character). The byte |
| 322 |
order character ``'='`` chooses to use little- or big-endian ordering based |
| 323 |
on the host system. The struct module does not interpret this as native |
| 324 |
ordering, so the ``'P'`` format is not available. |
| 325 |
|
| 326 |
(6) |
| 327 |
The IEEE 754 binary16 "half precision" type was introduced in the 2008 |
| 328 |
revision of the `IEEE 754 standard <ieee 754 standard_>`_. It has a sign |
| 329 |
bit, a 5-bit exponent and 11-bit precision (with 10 bits explicitly stored), |
| 330 |
and can represent numbers between approximately ``6.1e-05`` and ``6.5e+04`` |
| 331 |
at full precision. This type is not widely supported by C compilers: |
| 332 |
it's available as :c:expr:`_Float16` type, if the compiler supports the Annex H |
| 333 |
of the C23 standard. On a |
| 334 |
typical machine, an unsigned short can be used for storage, but not for math |
| 335 |
operations. See the Wikipedia page on the `half-precision floating-point |
| 336 |
format <half precision format_>`_ for more information. |
| 337 |
|
| 338 |
(7) |
| 339 |
When packing, ``'x'`` inserts one NUL byte. |
| 340 |
|
| 341 |
(8) |
| 342 |
The ``'p'`` format character encodes a "Pascal string", meaning a short |
| 343 |
variable-length string stored in a *fixed number of bytes*, given by the count. |
| 344 |
The first byte stored is the length of the string, or 255, whichever is |
| 345 |
smaller. The bytes of the string follow. If the byte string passed in to |
| 346 |
:func:`pack` is too long (longer than the count minus 1), only the leading |
| 347 |
``count-1`` bytes of the string are stored. If the byte string is shorter than |
| 348 |
``count-1``, it is padded with null bytes so that exactly count bytes in all |
| 349 |
are used. Note that for :func:`unpack`, the ``'p'`` format character consumes |
| 350 |
``count`` bytes, but that the :class:`!bytes` object returned can never contain more than 255 |
| 351 |
bytes. |
| 352 |
When packing, arguments of types :class:`bytes` and :class:`bytearray` |
| 353 |
are accepted. |
| 354 |
|
| 355 |
(9) |
| 356 |
For the ``'s'`` format character, the count is interpreted as the length of the |
| 357 |
byte string, not a repeat count like for the other format characters; for example, |
| 358 |
``'10s'`` means a single 10-byte string mapping to or from a single |
| 359 |
Python byte string, while ``'10c'`` means 10 |
| 360 |
separate one byte character elements (e.g., ``cccccccccc``) mapping |
| 361 |
to or from ten different Python byte objects. (See :ref:`struct-examples` |
| 362 |
for a concrete demonstration of the difference.) |
| 363 |
If a count is not given, it defaults to 1. For packing, the byte string is |
| 364 |
truncated or padded with null bytes as appropriate to make it fit. For |
| 365 |
unpacking, the resulting :class:`!bytes` object always has exactly the specified number |
| 366 |
of bytes. As a special case, ``'0s'`` means a single, empty byte string (while |
| 367 |
``'0c'`` means 0 characters). |
| 368 |
When packing, arguments of types :class:`bytes` and :class:`bytearray` |
| 369 |
are accepted. |
| 370 |
|
| 371 |
(10) |
| 372 |
For the ``'F'`` and ``'D'`` format characters, the packed representation uses |
| 373 |
the IEEE 754 binary32 and binary64 format for components of the complex |
| 374 |
number, regardless of the floating-point format used by the platform. |
| 375 |
Note that complex types (``F`` and ``D``) are available unconditionally, |
| 376 |
despite complex types being an optional feature in C. |
| 377 |
As specified in the C11 standard, each complex type is represented by a |
| 378 |
two-element C array containing, respectively, the real and imaginary parts. |
| 379 |
|
| 380 |
|
| 381 |
A format character may be preceded by an integral repeat count. For example, |
| 382 |
the format string ``'4h'`` means exactly the same as ``'hhhh'``. |
| 383 |
|
| 384 |
Whitespace characters between formats are ignored; a count and its format must |
| 385 |
not contain whitespace though. |
| 386 |
|
| 387 |
When packing a value ``x`` using one of the integer formats (``'b'``, |
| 388 |
``'B'``, ``'h'``, ``'H'``, ``'i'``, ``'I'``, ``'l'``, ``'L'``, |
| 389 |
``'q'``, ``'Q'``), if ``x`` is outside the valid range for that format |
| 390 |
then :exc:`struct.error` is raised. |
| 391 |
|
| 392 |
.. versionchanged:: 3.1 |
| 393 |
Previously, some of the integer formats wrapped out-of-range values and |
| 394 |
raised :exc:`DeprecationWarning` instead of :exc:`struct.error`. |
| 395 |
|
| 396 |
.. index:: single: ? (question mark); in struct format strings |
| 397 |
|
| 398 |
For the ``'?'`` format character, the return value is either :const:`True` or |
| 399 |
:const:`False`. When packing, the truth value of the argument object is used. |
| 400 |
Either 0 or 1 in the native or standard bool representation will be packed, and |
| 401 |
any non-zero value will be ``True`` when unpacking. |
| 402 |
|
| 403 |
|
| 404 |
|
| 405 |
.. _struct-examples: |
| 406 |
|
| 407 |
Examples |
| 408 |
^^^^^^^^ |
| 409 |
|
| 410 |
.. note:: |
| 411 |
Native byte order examples (designated by the ``'@'`` format prefix or |
| 412 |
lack of any prefix character) may not match what the reader's |
| 413 |
machine produces as |
| 414 |
that depends on the platform and compiler. |
| 415 |
|
| 416 |
Pack and unpack integers of three different sizes, using big endian |
| 417 |
ordering:: |
| 418 |
|
| 419 |
>>> from struct import * |
| 420 |
>>> pack(">bhl", 1, 2, 3) |
| 421 |
b'\x01\x00\x02\x00\x00\x00\x03' |
| 422 |
>>> unpack('>bhl', b'\x01\x00\x02\x00\x00\x00\x03') |
| 423 |
(1, 2, 3) |
| 424 |
>>> calcsize('>bhl') |
| 425 |
7 |
| 426 |
|
| 427 |
Attempt to pack an integer which is too large for the defined field:: |
| 428 |
|
| 429 |
>>> pack(">h", 99999) |
| 430 |
Traceback (most recent call last): |
| 431 |
File "<stdin>", line 1, in <module> |
| 432 |
struct.error: 'h' format requires -32768 <= number <= 32767 |
| 433 |
|
| 434 |
Demonstrate the difference between ``'s'`` and ``'c'`` format |
| 435 |
characters:: |
| 436 |
|
| 437 |
>>> pack("@ccc", b'1', b'2', b'3') |
| 438 |
b'123' |
| 439 |
>>> pack("@3s", b'123') |
| 440 |
b'123' |
| 441 |
|
| 442 |
Unpacked fields can be named by assigning them to variables or by wrapping |
| 443 |
the result in a named tuple:: |
| 444 |
|
| 445 |
>>> record = b'raymond \x32\x12\x08\x01\x08' |
| 446 |
>>> name, serialnum, school, gradelevel = unpack('<10sHHb', record) |
| 447 |
|
| 448 |
>>> from collections import namedtuple |
| 449 |
>>> Student = namedtuple('Student', 'name serialnum school gradelevel') |
| 450 |
>>> Student._make(unpack('<10sHHb', record)) |
| 451 |
Student(name=b'raymond ', serialnum=4658, school=264, gradelevel=8) |
| 452 |
|
| 453 |
The ordering of format characters may have an impact on size in native |
| 454 |
mode since padding is implicit. In standard mode, the user is |
| 455 |
responsible for inserting any desired padding. |
| 456 |
Note in |
| 457 |
the first ``pack`` call below that three NUL bytes were added after the |
| 458 |
packed ``'#'`` to align the following integer on a four-byte boundary. |
| 459 |
In this example, the output was produced on a little endian machine:: |
| 460 |
|
| 461 |
>>> pack('@ci', b'#', 0x12131415) |
| 462 |
b'#\x00\x00\x00\x15\x14\x13\x12' |
| 463 |
>>> pack('@ic', 0x12131415, b'#') |
| 464 |
b'\x15\x14\x13\x12#' |
| 465 |
>>> calcsize('@ci') |
| 466 |
8 |
| 467 |
>>> calcsize('@ic') |
| 468 |
5 |
| 469 |
|
| 470 |
The following format ``'llh0l'`` results in two pad bytes being added |
| 471 |
at the end, assuming the platform's longs are aligned on 4-byte boundaries:: |
| 472 |
|
| 473 |
>>> pack('@llh0l', 1, 2, 3) |
| 474 |
b'\x00\x00\x00\x01\x00\x00\x00\x02\x00\x03\x00\x00' |
| 475 |
|
| 476 |
|
| 477 |
.. seealso:: |
| 478 |
|
| 479 |
Module :mod:`array` |
| 480 |
Packed binary storage of homogeneous data. |
| 481 |
|
| 482 |
Module :mod:`json` |
| 483 |
JSON encoder and decoder. |
| 484 |
|
| 485 |
Module :mod:`pickle` |
| 486 |
Python object serialization. |
| 487 |
|
| 488 |
|
| 489 |
.. _applications: |
| 490 |
|
| 491 |
Applications |
| 492 |
------------ |
| 493 |
|
| 494 |
Two main applications for the :mod:`!struct` module exist, data |
| 495 |
interchange between Python and C code within an application or another |
| 496 |
application compiled using the same compiler (:ref:`native formats<struct-native-formats>`), and |
| 497 |
data interchange between applications using agreed upon data layout |
| 498 |
(:ref:`standard formats<struct-standard-formats>`). Generally speaking, the format strings |
| 499 |
constructed for these two domains are distinct. |
| 500 |
|
| 501 |
|
| 502 |
.. _struct-native-formats: |
| 503 |
|
| 504 |
Native Formats |
| 505 |
^^^^^^^^^^^^^^ |
| 506 |
|
| 507 |
When constructing format strings which mimic native layouts, the |
| 508 |
compiler and machine architecture determine byte ordering and padding. |
| 509 |
In such cases, the ``@`` format character should be used to specify |
| 510 |
native byte ordering and data sizes. Internal pad bytes are normally inserted |
| 511 |
automatically. It is possible that a zero-repeat format code will be |
| 512 |
needed at the end of a format string to round up to the correct |
| 513 |
byte boundary for proper alignment of consecutive chunks of data. |
| 514 |
|
| 515 |
Consider these two simple examples (on a 64-bit, little-endian |
| 516 |
machine):: |
| 517 |
|
| 518 |
>>> calcsize('@lhl') |
| 519 |
24 |
| 520 |
>>> calcsize('@llh') |
| 521 |
18 |
| 522 |
|
| 523 |
Data is not padded to an 8-byte boundary at the end of the second |
| 524 |
format string without the use of extra padding. A zero-repeat format |
| 525 |
code solves that problem:: |
| 526 |
|
| 527 |
>>> calcsize('@llh0l') |
| 528 |
24 |
| 529 |
|
| 530 |
The ``'x'`` format code can be used to specify the repeat, but for |
| 531 |
native formats it is better to use a zero-repeat format like ``'0l'``. |
| 532 |
|
| 533 |
By default, native byte ordering and alignment is used, but it is |
| 534 |
better to be explicit and use the ``'@'`` prefix character. |
| 535 |
|
| 536 |
|
| 537 |
.. _struct-standard-formats: |
| 538 |
|
| 539 |
Standard Formats |
| 540 |
^^^^^^^^^^^^^^^^ |
| 541 |
|
| 542 |
When exchanging data beyond your process such as networking or storage, |
| 543 |
be precise. Specify the exact byte order, size, and alignment. Do |
| 544 |
not assume they match the native order of a particular machine. |
| 545 |
For example, network byte order is big-endian, while many popular CPUs |
| 546 |
are little-endian. By defining this explicitly, the user need not |
| 547 |
care about the specifics of the platform their code is running on. |
| 548 |
The first character should typically be ``<`` or ``>`` |
| 549 |
(or ``!``). Padding is the responsibility of the programmer. The |
| 550 |
zero-repeat format character won't work. Instead, the user must |
| 551 |
explicitly add ``'x'`` pad bytes where needed. Revisiting the |
| 552 |
examples from the previous section, we have:: |
| 553 |
|
| 554 |
>>> calcsize('<qh6xq') |
| 555 |
24 |
| 556 |
>>> pack('<qh6xq', 1, 2, 3) == pack('@lhl', 1, 2, 3) |
| 557 |
True |
| 558 |
>>> calcsize('@llh') |
| 559 |
18 |
| 560 |
>>> pack('@llh', 1, 2, 3) == pack('<qqh', 1, 2, 3) |
| 561 |
True |
| 562 |
>>> calcsize('<qqh6x') |
| 563 |
24 |
| 564 |
>>> calcsize('@llh0l') |
| 565 |
24 |
| 566 |
>>> pack('@llh0l', 1, 2, 3) == pack('<qqh6x', 1, 2, 3) |
| 567 |
True |
| 568 |
|
| 569 |
The above results (executed on a 64-bit machine) aren't guaranteed to |
| 570 |
match when executed on different machines. For example, the examples |
| 571 |
below were executed on a 32-bit machine:: |
| 572 |
|
| 573 |
>>> calcsize('<qqh6x') |
| 574 |
24 |
| 575 |
>>> calcsize('@llh0l') |
| 576 |
12 |
| 577 |
>>> pack('@llh0l', 1, 2, 3) == pack('<qqh6x', 1, 2, 3) |
| 578 |
False |
| 579 |
|
| 580 |
|
| 581 |
.. _struct-objects: |
| 582 |
|
| 583 |
Classes |
| 584 |
------- |
| 585 |
|
| 586 |
The :mod:`!struct` module also defines the following type: |
| 587 |
|
| 588 |
|
| 589 |
.. class:: Struct(format) |
| 590 |
|
| 591 |
Return a new Struct object which writes and reads binary data according to |
| 592 |
the format string *format*. Creating a ``Struct`` object once and calling its |
| 593 |
methods is more efficient than calling module-level functions with the |
| 594 |
same format since the format string is only compiled once. |
| 595 |
|
| 596 |
.. note:: |
| 597 |
|
| 598 |
The compiled versions of the most recent format strings passed to |
| 599 |
the module-level functions are cached, so programs that use only a few |
| 600 |
format strings needn't worry about reusing a single :class:`Struct` |
| 601 |
instance. |
| 602 |
|
| 603 |
Compiled Struct objects support the following methods and attributes: |
| 604 |
|
| 605 |
.. method:: pack(v1, v2, ...) |
| 606 |
|
| 607 |
Identical to the :func:`pack` function, using the compiled format. |
| 608 |
(``len(result)`` will equal :attr:`size`.) |
| 609 |
|
| 610 |
|
| 611 |
.. method:: pack_into(buffer, offset, v1, v2, ...) |
| 612 |
|
| 613 |
Identical to the :func:`pack_into` function, using the compiled format. |
| 614 |
|
| 615 |
|
| 616 |
.. method:: unpack(buffer) |
| 617 |
|
| 618 |
Identical to the :func:`unpack` function, using the compiled format. |
| 619 |
The buffer's size in bytes must equal :attr:`size`. |
| 620 |
|
| 621 |
|
| 622 |
.. method:: unpack_from(buffer, offset=0) |
| 623 |
|
| 624 |
Identical to the :func:`unpack_from` function, using the compiled format. |
| 625 |
The buffer's size in bytes, starting at position *offset*, must be at least |
| 626 |
:attr:`size`. |
| 627 |
|
| 628 |
|
| 629 |
.. method:: iter_unpack(buffer) |
| 630 |
|
| 631 |
Identical to the :func:`iter_unpack` function, using the compiled format. |
| 632 |
The buffer's size in bytes must be a multiple of :attr:`size`. |
| 633 |
|
| 634 |
.. versionadded:: 3.4 |
| 635 |
|
| 636 |
.. attribute:: format |
| 637 |
|
| 638 |
The format string used to construct this Struct object. |
| 639 |
|
| 640 |
.. versionchanged:: 3.7 |
| 641 |
The format string type is now :class:`str` instead of :class:`bytes`. |
| 642 |
|
| 643 |
.. attribute:: size |
| 644 |
|
| 645 |
The calculated size of the struct (and hence of the bytes object produced |
| 646 |
by the :meth:`pack` method) corresponding to :attr:`format`. |
| 647 |
|
| 648 |
.. versionchanged:: 3.13 The *repr()* of structs has changed. It |
| 649 |
is now: |
| 650 |
|
| 651 |
>>> Struct('i') |
| 652 |
Struct('i') |
| 653 |
|
| 654 |
.. _half precision format: https://en.wikipedia.org/wiki/Half-precision_floating-point_format |
| 655 |
|
| 656 |
.. _ieee 754 standard: https://en.wikipedia.org/wiki/IEEE_754-2008_revision |
| 657 |
|
| 658 |
.. _IETF RFC 1700: https://datatracker.ietf.org/doc/html/rfc1700 |