| 1 |
:mod:`!io` --- Core tools for working with streams |
| 2 |
================================================== |
| 3 |
|
| 4 |
.. module:: io |
| 5 |
:synopsis: Core tools for working with streams. |
| 6 |
|
| 7 |
.. moduleauthor:: Guido van Rossum <[email protected]> |
| 8 |
.. moduleauthor:: Mike Verdone <[email protected]> |
| 9 |
.. moduleauthor:: Mark Russell <[email protected]> |
| 10 |
.. moduleauthor:: Antoine Pitrou <[email protected]> |
| 11 |
.. moduleauthor:: Amaury Forgeot d'Arc <[email protected]> |
| 12 |
.. moduleauthor:: Benjamin Peterson <[email protected]> |
| 13 |
.. sectionauthor:: Benjamin Peterson <[email protected]> |
| 14 |
|
| 15 |
**Source code:** :source:`Lib/io.py` |
| 16 |
|
| 17 |
-------------- |
| 18 |
|
| 19 |
.. _io-overview: |
| 20 |
|
| 21 |
Overview |
| 22 |
-------- |
| 23 |
|
| 24 |
.. index:: |
| 25 |
single: file object; io module |
| 26 |
|
| 27 |
The :mod:`!io` module provides Python's main facilities for dealing with various |
| 28 |
types of I/O. There are three main types of I/O: *text I/O*, *binary I/O* |
| 29 |
and *raw I/O*. These are generic categories, and various backing stores can |
| 30 |
be used for each of them. A concrete object belonging to any of these |
| 31 |
categories is called a :term:`file object`. Other common terms are *stream* |
| 32 |
and *file-like object*. |
| 33 |
|
| 34 |
Independent of its category, each concrete stream object will also have |
| 35 |
various capabilities: it can be read-only, write-only, or read-write. It can |
| 36 |
also allow arbitrary random access (seeking forwards or backwards to any |
| 37 |
location), or only sequential access (for example in the case of a socket or |
| 38 |
pipe). |
| 39 |
|
| 40 |
All streams are careful about the type of data you give to them. For example |
| 41 |
giving a :class:`str` object to the :meth:`!write` method of a binary stream |
| 42 |
will raise a :exc:`TypeError`. So will giving a :class:`bytes` object to the |
| 43 |
:meth:`!write` method of a text stream. |
| 44 |
|
| 45 |
.. versionchanged:: 3.3 |
| 46 |
Operations that used to raise :exc:`IOError` now raise :exc:`OSError`, since |
| 47 |
:exc:`IOError` is now an alias of :exc:`OSError`. |
| 48 |
|
| 49 |
.. _text-io: |
| 50 |
|
| 51 |
Text I/O |
| 52 |
^^^^^^^^ |
| 53 |
|
| 54 |
Text I/O expects and produces :class:`str` objects. This means that whenever |
| 55 |
the backing store is natively made of bytes (such as in the case of a file), |
| 56 |
encoding and decoding of data is made transparently as well as optional |
| 57 |
translation of platform-specific newline characters. |
| 58 |
|
| 59 |
The easiest way to create a text stream is with :meth:`open`, optionally |
| 60 |
specifying an encoding:: |
| 61 |
|
| 62 |
f = open("myfile.txt", "r", encoding="utf-8") |
| 63 |
|
| 64 |
In-memory text streams are also available as :class:`StringIO` objects:: |
| 65 |
|
| 66 |
f = io.StringIO("some initial text data") |
| 67 |
|
| 68 |
.. note:: |
| 69 |
|
| 70 |
When working with a non-blocking stream, be aware that read operations on text I/O objects |
| 71 |
might raise a :exc:`BlockingIOError` if the stream cannot perform the operation |
| 72 |
immediately. |
| 73 |
|
| 74 |
The text stream API is described in detail in the documentation of |
| 75 |
:class:`TextIOBase`. |
| 76 |
|
| 77 |
.. _binary-io: |
| 78 |
|
| 79 |
Binary I/O |
| 80 |
^^^^^^^^^^ |
| 81 |
|
| 82 |
Binary I/O (also called *buffered I/O*) expects |
| 83 |
:term:`bytes-like objects <bytes-like object>` and produces :class:`bytes` |
| 84 |
objects. No encoding, decoding, or newline translation is performed. This |
| 85 |
category of streams can be used for all kinds of non-text data, and also when |
| 86 |
manual control over the handling of text data is desired. |
| 87 |
|
| 88 |
The easiest way to create a binary stream is with :meth:`open` with ``'b'`` in |
| 89 |
the mode string:: |
| 90 |
|
| 91 |
f = open("myfile.jpg", "rb") |
| 92 |
|
| 93 |
In-memory binary streams are also available as :class:`BytesIO` objects:: |
| 94 |
|
| 95 |
f = io.BytesIO(b"some initial binary data: \x00\x01") |
| 96 |
|
| 97 |
The binary stream API is described in detail in the docs of |
| 98 |
:class:`BufferedIOBase`. |
| 99 |
|
| 100 |
Other library modules may provide additional ways to create text or binary |
| 101 |
streams. See :meth:`socket.socket.makefile` for example. |
| 102 |
|
| 103 |
|
| 104 |
Raw I/O |
| 105 |
^^^^^^^ |
| 106 |
|
| 107 |
Raw I/O (also called *unbuffered I/O*) is generally used as a low-level |
| 108 |
building-block for binary and text streams; it is rarely useful to directly |
| 109 |
manipulate a raw stream from user code. Nevertheless, you can create a raw |
| 110 |
stream by opening a file in binary mode with buffering disabled:: |
| 111 |
|
| 112 |
f = open("myfile.jpg", "rb", buffering=0) |
| 113 |
|
| 114 |
The raw stream API is described in detail in the docs of :class:`RawIOBase`. |
| 115 |
|
| 116 |
.. warning:: |
| 117 |
Raw I/O is a low-level interface and methods generally must have their return |
| 118 |
values checked and be explicitly retried to ensure an operation completes. |
| 119 |
For instance :meth:`~RawIOBase.write` returns the number of bytes written |
| 120 |
which may be less than the number of bytes provided (a partial write). |
| 121 |
High-level I/O objects like :ref:`binary-io` and :ref:`text-io` implement |
| 122 |
retry behavior. |
| 123 |
|
| 124 |
.. _io-text-encoding: |
| 125 |
|
| 126 |
Text Encoding |
| 127 |
------------- |
| 128 |
|
| 129 |
The default encoding of :class:`TextIOWrapper` and :func:`open` is |
| 130 |
locale-specific (:func:`locale.getencoding`). |
| 131 |
|
| 132 |
However, many developers forget to specify the encoding when opening text files |
| 133 |
encoded in UTF-8 (e.g. JSON, TOML, Markdown, etc...) since most Unix |
| 134 |
platforms use UTF-8 locale by default. This causes bugs because the locale |
| 135 |
encoding is not UTF-8 for most Windows users. For example:: |
| 136 |
|
| 137 |
# May not work on Windows when non-ASCII characters in the file. |
| 138 |
with open("README.md") as f: |
| 139 |
long_description = f.read() |
| 140 |
|
| 141 |
Accordingly, it is highly recommended that you specify the encoding |
| 142 |
explicitly when opening text files. If you want to use UTF-8, pass |
| 143 |
``encoding="utf-8"``. To use the current locale encoding, |
| 144 |
``encoding="locale"`` is supported since Python 3.10. |
| 145 |
|
| 146 |
.. seealso:: |
| 147 |
|
| 148 |
:ref:`utf8-mode` |
| 149 |
Python UTF-8 Mode can be used to change the default encoding to |
| 150 |
UTF-8 from locale-specific encoding. |
| 151 |
|
| 152 |
:pep:`686` |
| 153 |
Python 3.15 will make :ref:`utf8-mode` default. |
| 154 |
|
| 155 |
.. _io-encoding-warning: |
| 156 |
|
| 157 |
Opt-in EncodingWarning |
| 158 |
^^^^^^^^^^^^^^^^^^^^^^ |
| 159 |
|
| 160 |
.. versionadded:: 3.10 |
| 161 |
See :pep:`597` for more details. |
| 162 |
|
| 163 |
To find where the default locale encoding is used, you can enable |
| 164 |
the :option:`-X warn_default_encoding <-X>` command line option or set the |
| 165 |
:envvar:`PYTHONWARNDEFAULTENCODING` environment variable, which will |
| 166 |
emit an :exc:`EncodingWarning` when the default encoding is used. |
| 167 |
|
| 168 |
If you are providing an API that uses :func:`open` or |
| 169 |
:class:`TextIOWrapper` and passes ``encoding=None`` as a parameter, you |
| 170 |
can use :func:`text_encoding` so that callers of the API will emit an |
| 171 |
:exc:`EncodingWarning` if they don't pass an ``encoding``. However, |
| 172 |
please consider using UTF-8 by default (i.e. ``encoding="utf-8"``) for |
| 173 |
new APIs. |
| 174 |
|
| 175 |
|
| 176 |
High-level Module Interface |
| 177 |
--------------------------- |
| 178 |
|
| 179 |
.. data:: DEFAULT_BUFFER_SIZE |
| 180 |
|
| 181 |
An int containing the default buffer size used by the module's buffered I/O |
| 182 |
classes. :func:`open` uses the file's blksize (as obtained by |
| 183 |
:func:`os.stat`) if possible. |
| 184 |
|
| 185 |
|
| 186 |
.. function:: open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None) |
| 187 |
|
| 188 |
This is an alias for the builtin :func:`open` function. |
| 189 |
|
| 190 |
.. audit-event:: open path,mode,flags io.open |
| 191 |
|
| 192 |
This function raises an :ref:`auditing event <auditing>` ``open`` with |
| 193 |
arguments *path*, *mode* and *flags*. The *mode* and *flags* |
| 194 |
arguments may have been modified or inferred from the original call. |
| 195 |
|
| 196 |
|
| 197 |
.. function:: open_code(path) |
| 198 |
|
| 199 |
Opens the provided file with mode ``'rb'``. This function should be used |
| 200 |
when the intent is to treat the contents as executable code. |
| 201 |
|
| 202 |
*path* should be a :class:`str` and an absolute path. |
| 203 |
|
| 204 |
The behavior of this function may be overridden by an earlier call to the |
| 205 |
:c:func:`PyFile_SetOpenCodeHook`. However, assuming that *path* is a |
| 206 |
:class:`str` and an absolute path, ``open_code(path)`` should always behave |
| 207 |
the same as ``open(path, 'rb')``. Overriding the behavior is intended for |
| 208 |
additional validation or preprocessing of the file. |
| 209 |
|
| 210 |
.. versionadded:: 3.8 |
| 211 |
|
| 212 |
|
| 213 |
.. function:: text_encoding(encoding, stacklevel=2, /) |
| 214 |
|
| 215 |
This is a helper function for callables that use :func:`open` or |
| 216 |
:class:`TextIOWrapper` and have an ``encoding=None`` parameter. |
| 217 |
|
| 218 |
This function returns *encoding* if it is not ``None``. |
| 219 |
Otherwise, it returns ``"locale"`` or ``"utf-8"`` depending on |
| 220 |
:ref:`UTF-8 Mode <utf8-mode>`. |
| 221 |
|
| 222 |
This function emits an :class:`EncodingWarning` if |
| 223 |
:data:`sys.flags.warn_default_encoding <sys.flags>` is true and *encoding* |
| 224 |
is ``None``. *stacklevel* specifies where the warning is emitted. |
| 225 |
For example:: |
| 226 |
|
| 227 |
def read_text(path, encoding=None): |
| 228 |
encoding = io.text_encoding(encoding) # stacklevel=2 |
| 229 |
with open(path, encoding) as f: |
| 230 |
return f.read() |
| 231 |
|
| 232 |
In this example, an :class:`EncodingWarning` is emitted for the caller of |
| 233 |
``read_text()``. |
| 234 |
|
| 235 |
See :ref:`io-text-encoding` for more information. |
| 236 |
|
| 237 |
.. versionadded:: 3.10 |
| 238 |
|
| 239 |
.. versionchanged:: 3.11 |
| 240 |
:func:`text_encoding` returns "utf-8" when UTF-8 mode is enabled and |
| 241 |
*encoding* is ``None``. |
| 242 |
|
| 243 |
|
| 244 |
.. exception:: BlockingIOError |
| 245 |
|
| 246 |
This is a compatibility alias for the builtin :exc:`BlockingIOError` |
| 247 |
exception. |
| 248 |
|
| 249 |
|
| 250 |
.. exception:: UnsupportedOperation |
| 251 |
|
| 252 |
An exception inheriting :exc:`OSError` and :exc:`ValueError` that is raised |
| 253 |
when an unsupported operation is called on a stream. |
| 254 |
|
| 255 |
|
| 256 |
.. seealso:: |
| 257 |
|
| 258 |
:mod:`sys` |
| 259 |
contains the standard IO streams: :data:`sys.stdin`, :data:`sys.stdout`, |
| 260 |
and :data:`sys.stderr`. |
| 261 |
|
| 262 |
|
| 263 |
Class hierarchy |
| 264 |
--------------- |
| 265 |
|
| 266 |
The implementation of I/O streams is organized as a hierarchy of classes. First |
| 267 |
:term:`abstract base classes <abstract base class>` (ABCs), which are used to |
| 268 |
specify the various categories of streams, then concrete classes providing the |
| 269 |
standard stream implementations. |
| 270 |
|
| 271 |
.. note:: |
| 272 |
|
| 273 |
The abstract base classes also provide default implementations of some |
| 274 |
methods in order to help implementation of concrete stream classes. For |
| 275 |
example, :class:`BufferedIOBase` provides unoptimized implementations of |
| 276 |
:meth:`!readinto` and :meth:`!readline`. |
| 277 |
|
| 278 |
At the top of the I/O hierarchy is the abstract base class :class:`IOBase`. It |
| 279 |
defines the basic interface to a stream. Note, however, that there is no |
| 280 |
separation between reading and writing to streams; implementations are allowed |
| 281 |
to raise :exc:`UnsupportedOperation` if they do not support a given operation. |
| 282 |
|
| 283 |
The :class:`RawIOBase` ABC extends :class:`IOBase`. It deals with the reading |
| 284 |
and writing of bytes to a stream. :class:`FileIO` subclasses :class:`RawIOBase` |
| 285 |
to provide an interface to files in the machine's file system. |
| 286 |
|
| 287 |
The :class:`BufferedIOBase` ABC extends :class:`IOBase`. It deals with |
| 288 |
buffering on a raw binary stream (:class:`RawIOBase`). Its subclasses, |
| 289 |
:class:`BufferedWriter`, :class:`BufferedReader`, and :class:`BufferedRWPair` |
| 290 |
buffer raw binary streams that are writable, readable, and both readable and writable, |
| 291 |
respectively. :class:`BufferedRandom` provides a buffered interface to seekable streams. |
| 292 |
Another :class:`BufferedIOBase` subclass, :class:`BytesIO`, is a stream of |
| 293 |
in-memory bytes. |
| 294 |
|
| 295 |
The :class:`TextIOBase` ABC extends :class:`IOBase`. It deals with |
| 296 |
streams whose bytes represent text, and handles encoding and decoding to and |
| 297 |
from strings. :class:`TextIOWrapper`, which extends :class:`TextIOBase`, is a buffered text |
| 298 |
interface to a buffered raw stream (:class:`BufferedIOBase`). Finally, |
| 299 |
:class:`StringIO` is an in-memory stream for text. |
| 300 |
|
| 301 |
Argument names are not part of the specification, and only the arguments of |
| 302 |
:func:`open` are intended to be used as keyword arguments. |
| 303 |
|
| 304 |
The following table summarizes the ABCs provided by the :mod:`!io` module: |
| 305 |
|
| 306 |
.. tabularcolumns:: |l|l|L|L| |
| 307 |
|
| 308 |
========================= ================== ======================== ================================================== |
| 309 |
ABC Inherits Stub Methods Mixin Methods and Properties |
| 310 |
========================= ================== ======================== ================================================== |
| 311 |
:class:`IOBase` ``fileno``, ``seek``, ``close``, ``closed``, ``__enter__``, |
| 312 |
and ``truncate`` ``__exit__``, ``flush``, ``isatty``, ``__iter__``, |
| 313 |
``__next__``, ``readable``, ``readline``, |
| 314 |
``readlines``, ``seekable``, ``tell``, |
| 315 |
``writable``, and ``writelines`` |
| 316 |
:class:`RawIOBase` :class:`IOBase` ``readinto`` and Inherited :class:`IOBase` methods, ``read``, |
| 317 |
``write`` and ``readall`` |
| 318 |
:class:`BufferedIOBase` :class:`IOBase` ``detach``, ``read``, Inherited :class:`IOBase` methods, ``readinto``, |
| 319 |
``read1``, and ``write`` and ``readinto1`` |
| 320 |
:class:`TextIOBase` :class:`IOBase` ``detach``, ``read``, Inherited :class:`IOBase` methods, ``encoding``, |
| 321 |
``readline``, and ``errors``, and ``newlines`` |
| 322 |
``write`` |
| 323 |
========================= ================== ======================== ================================================== |
| 324 |
|
| 325 |
|
| 326 |
I/O Base Classes |
| 327 |
^^^^^^^^^^^^^^^^ |
| 328 |
|
| 329 |
.. class:: IOBase |
| 330 |
|
| 331 |
The abstract base class for all I/O classes. |
| 332 |
|
| 333 |
This class provides empty abstract implementations for many methods |
| 334 |
that derived classes can override selectively; the default |
| 335 |
implementations represent a file that cannot be read, written or |
| 336 |
seeked. |
| 337 |
|
| 338 |
Even though :class:`IOBase` does not declare :meth:`!read` |
| 339 |
or :meth:`!write` because their signatures will vary, implementations and |
| 340 |
clients should consider those methods part of the interface. Also, |
| 341 |
implementations may raise a :exc:`ValueError` (or :exc:`UnsupportedOperation`) |
| 342 |
when operations they do not support are called. |
| 343 |
|
| 344 |
The basic type used for binary data read from or written to a file is |
| 345 |
:class:`bytes`. Other :term:`bytes-like objects <bytes-like object>` are |
| 346 |
accepted as method arguments too. Text I/O classes work with :class:`str` data. |
| 347 |
|
| 348 |
Note that calling any method (even inquiries) on a closed stream is |
| 349 |
undefined. Implementations may raise :exc:`ValueError` in this case. |
| 350 |
|
| 351 |
:class:`IOBase` (and its subclasses) supports the iterator protocol, meaning |
| 352 |
that an :class:`IOBase` object can be iterated over yielding the lines in a |
| 353 |
stream. Lines are defined slightly differently depending on whether the |
| 354 |
stream is a binary stream (yielding bytes), or a text stream (yielding |
| 355 |
character strings). See :meth:`~IOBase.readline` below. |
| 356 |
|
| 357 |
:class:`IOBase` is also a context manager and therefore supports the |
| 358 |
:keyword:`with` statement. In this example, *file* is closed after the |
| 359 |
:keyword:`!with` statement's suite is finished---even if an exception occurs:: |
| 360 |
|
| 361 |
with open('spam.txt', 'w') as file: |
| 362 |
file.write('Spam and eggs!') |
| 363 |
|
| 364 |
:class:`IOBase` provides these data attributes and methods: |
| 365 |
|
| 366 |
.. method:: close() |
| 367 |
|
| 368 |
Flush and close this stream. This method has no effect if the file is |
| 369 |
already closed. Once the file is closed, any operation on the file |
| 370 |
(e.g. reading or writing) will raise a :exc:`ValueError`. |
| 371 |
|
| 372 |
As a convenience, it is allowed to call this method more than once; |
| 373 |
only the first call, however, will have an effect. |
| 374 |
|
| 375 |
.. attribute:: closed |
| 376 |
|
| 377 |
``True`` if the stream is closed. |
| 378 |
|
| 379 |
.. method:: fileno() |
| 380 |
|
| 381 |
Return the underlying file descriptor (an integer) of the stream if it |
| 382 |
exists. An :exc:`OSError` is raised if the IO object does not use a file |
| 383 |
descriptor. |
| 384 |
|
| 385 |
.. method:: flush() |
| 386 |
|
| 387 |
Flush the write buffers of the stream if applicable. This does nothing |
| 388 |
for read-only and non-blocking streams. |
| 389 |
|
| 390 |
.. method:: isatty() |
| 391 |
|
| 392 |
Return ``True`` if the stream is interactive (i.e., connected to |
| 393 |
a terminal/tty device). |
| 394 |
|
| 395 |
.. method:: readable() |
| 396 |
|
| 397 |
Return ``True`` if the stream can be read from. |
| 398 |
If ``False``, :meth:`!read` will raise :exc:`OSError`. |
| 399 |
|
| 400 |
.. method:: readline(size=-1, /) |
| 401 |
|
| 402 |
Read and return one line from the stream. If *size* is specified, at |
| 403 |
most *size* bytes will be read. |
| 404 |
|
| 405 |
The line terminator is always ``b'\n'`` for binary files; for text files, |
| 406 |
the *newline* argument to :func:`open` can be used to select the line |
| 407 |
terminator(s) recognized. |
| 408 |
|
| 409 |
.. method:: readlines(hint=-1, /) |
| 410 |
|
| 411 |
Read and return a list of lines from the stream. *hint* can be specified |
| 412 |
to control the number of lines read: no more lines will be read if the |
| 413 |
total size (in bytes/characters) of all lines so far exceeds *hint*. |
| 414 |
|
| 415 |
*hint* values of ``0`` or less, as well as ``None``, are treated as no |
| 416 |
hint. |
| 417 |
|
| 418 |
Note that it's already possible to iterate on file objects using ``for |
| 419 |
line in file: ...`` without calling :meth:`!file.readlines`. |
| 420 |
|
| 421 |
.. method:: seek(offset, whence=os.SEEK_SET, /) |
| 422 |
|
| 423 |
Change the stream position to the given byte *offset*, |
| 424 |
interpreted relative to the position indicated by *whence*, |
| 425 |
and return the new absolute position. |
| 426 |
Values for *whence* are: |
| 427 |
|
| 428 |
* :data:`os.SEEK_SET` or ``0`` -- start of the stream (the default); |
| 429 |
*offset* should be zero or positive |
| 430 |
* :data:`os.SEEK_CUR` or ``1`` -- current stream position; |
| 431 |
*offset* may be negative |
| 432 |
* :data:`os.SEEK_END` or ``2`` -- end of the stream; |
| 433 |
*offset* is usually negative |
| 434 |
|
| 435 |
.. versionadded:: 3.1 |
| 436 |
The :data:`!SEEK_*` constants. |
| 437 |
|
| 438 |
.. versionadded:: 3.3 |
| 439 |
Some operating systems could support additional values, like |
| 440 |
:const:`os.SEEK_HOLE` or :const:`os.SEEK_DATA`. The valid values |
| 441 |
for a file could depend on it being open in text or binary mode. |
| 442 |
|
| 443 |
.. method:: seekable() |
| 444 |
|
| 445 |
Return ``True`` if the stream supports random access. If ``False``, |
| 446 |
:meth:`seek`, :meth:`tell` and :meth:`truncate` will raise :exc:`OSError`. |
| 447 |
|
| 448 |
.. method:: tell() |
| 449 |
|
| 450 |
Return the current stream position. |
| 451 |
|
| 452 |
.. method:: truncate(size=None, /) |
| 453 |
|
| 454 |
Resize the stream to the given *size* in bytes (or the current position |
| 455 |
if *size* is not specified). The current stream position isn't changed. |
| 456 |
This resizing can extend or reduce the current file size. In case of |
| 457 |
extension, the contents of the new file area depend on the platform |
| 458 |
(on most systems, additional bytes are zero-filled). The new file size |
| 459 |
is returned. |
| 460 |
|
| 461 |
.. versionchanged:: 3.5 |
| 462 |
Windows will now zero-fill files when extending. |
| 463 |
|
| 464 |
.. method:: writable() |
| 465 |
|
| 466 |
Return ``True`` if the stream supports writing. If ``False``, |
| 467 |
:meth:`!write` and :meth:`truncate` will raise :exc:`OSError`. |
| 468 |
|
| 469 |
.. method:: writelines(lines, /) |
| 470 |
|
| 471 |
Write a list of lines to the stream. Line separators are not added, so it |
| 472 |
is usual for each of the lines provided to have a line separator at the |
| 473 |
end. |
| 474 |
|
| 475 |
.. method:: __del__() |
| 476 |
|
| 477 |
Prepare for object destruction. :class:`IOBase` provides a default |
| 478 |
implementation of this method that calls the instance's |
| 479 |
:meth:`~IOBase.close` method. |
| 480 |
|
| 481 |
|
| 482 |
.. class:: RawIOBase |
| 483 |
|
| 484 |
Base class for raw binary streams. It inherits from :class:`IOBase`. |
| 485 |
|
| 486 |
Raw binary streams typically provide low-level access to an underlying OS |
| 487 |
device or API, and do not try to encapsulate it in high-level primitives |
| 488 |
(this functionality is done at a higher-level in buffered binary streams and text streams, described later |
| 489 |
in this page). |
| 490 |
|
| 491 |
:class:`RawIOBase` provides these methods in addition to those from |
| 492 |
:class:`IOBase`: |
| 493 |
|
| 494 |
.. method:: read(size=-1, /) |
| 495 |
|
| 496 |
Read up to *size* bytes from the object and return them. As a convenience, |
| 497 |
if *size* is unspecified or -1, all bytes until EOF are returned. |
| 498 |
|
| 499 |
Attempts to make only one system call but will retry if interrupted and |
| 500 |
the signal handler does not raise an exception (see :pep:`475` for the |
| 501 |
rationale). This means fewer than *size* bytes may be returned if the |
| 502 |
operating system call returns fewer than *size* bytes. |
| 503 |
|
| 504 |
If 0 bytes are returned, and *size* was not 0, this indicates end of file. |
| 505 |
If the object is in non-blocking mode and no bytes are available, |
| 506 |
``None`` is returned. |
| 507 |
|
| 508 |
The default implementation defers to :meth:`readall` and |
| 509 |
:meth:`readinto`. |
| 510 |
|
| 511 |
.. method:: readall() |
| 512 |
|
| 513 |
Read and return all the bytes from the stream until EOF, using multiple |
| 514 |
calls to the stream if necessary. |
| 515 |
|
| 516 |
If ``0`` bytes are returned this indicates end of file. If the object is in |
| 517 |
non-blocking mode and the underlying :meth:`read` returns ``None`` |
| 518 |
indicating no bytes are available, ``None`` is returned. |
| 519 |
|
| 520 |
.. method:: readinto(b, /) |
| 521 |
|
| 522 |
Read bytes into a pre-allocated, writable |
| 523 |
:term:`bytes-like object` *b*, and return the |
| 524 |
number of bytes read. For example, *b* might be a :class:`bytearray`. |
| 525 |
|
| 526 |
If ``0`` is returned and ``len(b)`` is not ``0``, this indicates end of file. If |
| 527 |
the object is in non-blocking mode and no bytes are available, ``None`` is |
| 528 |
returned. |
| 529 |
|
| 530 |
.. method:: write(b, /) |
| 531 |
|
| 532 |
Write the given :term:`bytes-like object`, *b*, to the |
| 533 |
underlying raw stream, and return the number of |
| 534 |
bytes written. This can be less than the length of *b* in |
| 535 |
bytes, depending on specifics of the underlying raw |
| 536 |
stream, and especially if it is in non-blocking mode. ``None`` is |
| 537 |
returned if the raw stream is set not to block and no single byte could |
| 538 |
be readily written to it. The caller may release or mutate *b* after |
| 539 |
this method returns, so the implementation should only access *b* |
| 540 |
during the method call. |
| 541 |
|
| 542 |
.. warning:: |
| 543 |
|
| 544 |
This function does not ensure all bytes are written or an exception is |
| 545 |
thrown. Callers may implement that behavior by checking the return |
| 546 |
value and, if it is less than the length of *b*, looping with additional |
| 547 |
write calls until all unwritten bytes are written. High-level I/O |
| 548 |
objects like :ref:`binary-io` and :ref:`text-io` implement retry behavior. |
| 549 |
|
| 550 |
.. class:: BufferedIOBase |
| 551 |
|
| 552 |
Base class for binary streams that support some kind of buffering. |
| 553 |
It inherits from :class:`IOBase`. |
| 554 |
|
| 555 |
The main difference with :class:`RawIOBase` is that methods :meth:`read`, |
| 556 |
:meth:`readinto` and :meth:`write` will try (respectively) to read |
| 557 |
as much input as requested or to emit all provided data. |
| 558 |
|
| 559 |
In addition, if the underlying raw stream is in non-blocking mode, when the |
| 560 |
system returns would block :meth:`write` will raise :exc:`BlockingIOError` |
| 561 |
with :attr:`BlockingIOError.characters_written` and :meth:`read` will return |
| 562 |
data read so far or ``None`` if no data is available. |
| 563 |
|
| 564 |
Besides, the :meth:`read` method does not have a default |
| 565 |
implementation that defers to :meth:`readinto`. |
| 566 |
|
| 567 |
A typical :class:`BufferedIOBase` implementation should not inherit from a |
| 568 |
:class:`RawIOBase` implementation, but wrap one, like |
| 569 |
:class:`BufferedWriter` and :class:`BufferedReader` do. |
| 570 |
|
| 571 |
:class:`BufferedIOBase` provides or overrides these data attributes and |
| 572 |
methods in addition to those from :class:`IOBase`: |
| 573 |
|
| 574 |
.. attribute:: raw |
| 575 |
|
| 576 |
The underlying raw stream (a :class:`RawIOBase` instance) that |
| 577 |
:class:`BufferedIOBase` deals with. This is not part of the |
| 578 |
:class:`BufferedIOBase` API and may not exist on some implementations. |
| 579 |
|
| 580 |
.. method:: detach() |
| 581 |
|
| 582 |
Separate the underlying raw stream from the buffer and return it. |
| 583 |
|
| 584 |
After the raw stream has been detached, the buffer is in an unusable |
| 585 |
state. |
| 586 |
|
| 587 |
Some buffers, like :class:`BytesIO`, do not have the concept of a single |
| 588 |
raw stream to return from this method. They raise |
| 589 |
:exc:`UnsupportedOperation`. |
| 590 |
|
| 591 |
.. versionadded:: 3.1 |
| 592 |
|
| 593 |
.. method:: read(size=-1, /) |
| 594 |
|
| 595 |
Read and return up to *size* bytes. If the argument is omitted, ``None``, |
| 596 |
or negative read as much as possible. |
| 597 |
|
| 598 |
Fewer bytes may be returned than requested. An empty :class:`bytes` object |
| 599 |
is returned if the stream is already at EOF. More than one read may be |
| 600 |
made and calls may be retried if specific errors are encountered, see |
| 601 |
:meth:`os.read` and :pep:`475` for more details. Less than size bytes |
| 602 |
being returned does not imply that EOF is imminent. |
| 603 |
|
| 604 |
When reading as much as possible the default implementation will use |
| 605 |
``raw.readall`` if available (which should implement |
| 606 |
:meth:`RawIOBase.readall`), otherwise will read in a loop until read |
| 607 |
returns ``None``, an empty :class:`bytes`, or a non-retryable error. For |
| 608 |
most streams this is to EOF, but for non-blocking streams more data may |
| 609 |
become available. |
| 610 |
|
| 611 |
.. note:: |
| 612 |
|
| 613 |
When the underlying raw stream is non-blocking, implementations may |
| 614 |
either raise :exc:`BlockingIOError` or return ``None`` if no data is |
| 615 |
available. :mod:`!io` implementations return ``None``. |
| 616 |
|
| 617 |
.. method:: read1(size=-1, /) |
| 618 |
|
| 619 |
Read and return up to *size* bytes, calling :meth:`~RawIOBase.readinto` |
| 620 |
which may retry if :py:const:`~errno.EINTR` is encountered per |
| 621 |
:pep:`475`. If *size* is ``-1`` or not provided, the implementation will |
| 622 |
choose an arbitrary value for *size*. |
| 623 |
|
| 624 |
.. note:: |
| 625 |
|
| 626 |
When the underlying raw stream is non-blocking, implementations may |
| 627 |
either raise :exc:`BlockingIOError` or return ``None`` if no data is |
| 628 |
available. :mod:`!io` implementations return ``None``. |
| 629 |
|
| 630 |
.. method:: readinto(b, /) |
| 631 |
|
| 632 |
Read bytes into a pre-allocated, writable |
| 633 |
:term:`bytes-like object` *b* and return the number of bytes read. |
| 634 |
For example, *b* might be a :class:`bytearray`. |
| 635 |
|
| 636 |
Like :meth:`read`, multiple reads may be issued to the underlying raw |
| 637 |
stream, unless the latter is interactive. |
| 638 |
|
| 639 |
A :exc:`BlockingIOError` is raised if the underlying raw stream is in non |
| 640 |
blocking-mode, and has no data available at the moment. |
| 641 |
|
| 642 |
.. method:: readinto1(b, /) |
| 643 |
|
| 644 |
Read bytes into a pre-allocated, writable |
| 645 |
:term:`bytes-like object` *b*, using at most one call to |
| 646 |
the underlying raw stream's :meth:`~RawIOBase.read` (or |
| 647 |
:meth:`~RawIOBase.readinto`) method. Return the number of bytes read. |
| 648 |
|
| 649 |
A :exc:`BlockingIOError` is raised if the underlying raw stream is in non |
| 650 |
blocking-mode, and has no data available at the moment. |
| 651 |
|
| 652 |
.. versionadded:: 3.5 |
| 653 |
|
| 654 |
.. method:: write(b, /) |
| 655 |
|
| 656 |
Write the given :term:`bytes-like object`, *b*, and return the number |
| 657 |
of bytes written (always equal to the length of *b* in bytes, since if |
| 658 |
the write fails an :exc:`OSError` will be raised). Depending on the |
| 659 |
actual implementation, these bytes may be readily written to the |
| 660 |
underlying stream, or held in a buffer for performance and latency |
| 661 |
reasons. |
| 662 |
|
| 663 |
When in non-blocking mode, a :exc:`BlockingIOError` is raised if the |
| 664 |
data needed to be written to the raw stream but it couldn't accept |
| 665 |
all the data without blocking. |
| 666 |
|
| 667 |
The caller may release or mutate *b* after this method returns, |
| 668 |
so the implementation should only access *b* during the method call. |
| 669 |
|
| 670 |
|
| 671 |
Raw File I/O |
| 672 |
^^^^^^^^^^^^ |
| 673 |
|
| 674 |
.. class:: FileIO(name, mode='r', closefd=True, opener=None) |
| 675 |
|
| 676 |
A raw binary stream representing an OS-level file containing bytes data. It |
| 677 |
inherits from :class:`RawIOBase` and implements its low-level access design. |
| 678 |
This means :meth:`~RawIOBase.write` does not guarantee all bytes are written |
| 679 |
and :meth:`~RawIOBase.read` may read less bytes than requested even when more |
| 680 |
bytes may be present in the underlying file. To get "write all" and |
| 681 |
"read at least" behavior, use :ref:`binary-io`. |
| 682 |
|
| 683 |
The *name* can be one of two things: |
| 684 |
|
| 685 |
* a character string or :class:`bytes` object representing the path to the |
| 686 |
file which will be opened. In this case closefd must be ``True`` (the default) |
| 687 |
otherwise an error will be raised. |
| 688 |
* an integer representing the number of an existing OS-level file descriptor |
| 689 |
to which the resulting :class:`FileIO` object will give access. When the |
| 690 |
FileIO object is closed this fd will be closed as well, unless *closefd* |
| 691 |
is set to ``False``. |
| 692 |
|
| 693 |
The *mode* can be ``'r'``, ``'w'``, ``'x'`` or ``'a'`` for reading |
| 694 |
(default), writing, exclusive creation or appending. The file will be |
| 695 |
created if it doesn't exist when opened for writing or appending; it will be |
| 696 |
truncated when opened for writing. :exc:`FileExistsError` will be raised if |
| 697 |
it already exists when opened for creating. Opening a file for creating |
| 698 |
implies writing, so this mode behaves in a similar way to ``'w'``. Add a |
| 699 |
``'+'`` to the mode to allow simultaneous reading and writing. |
| 700 |
|
| 701 |
A custom opener can be used by passing a callable as *opener*. The underlying |
| 702 |
file descriptor for the file object is then obtained by calling *opener* with |
| 703 |
(*name*, *flags*). *opener* must return an open file descriptor (passing |
| 704 |
:mod:`os.open` as *opener* results in functionality similar to passing |
| 705 |
``None``). |
| 706 |
|
| 707 |
The newly created file is :ref:`non-inheritable <fd_inheritance>`. |
| 708 |
|
| 709 |
See the :func:`open` built-in function for examples on using the *opener* |
| 710 |
parameter. |
| 711 |
|
| 712 |
.. warning:: |
| 713 |
:class:`FileIO` is a low-level I/O object and members, such as |
| 714 |
:meth:`~RawIOBase.read` and :meth:`~RawIOBase.write`, need to have their |
| 715 |
return values checked explicitly in a retry loop to implement "write all" |
| 716 |
and "read at least" behavior. High-level I/O objects :ref:`binary-io` and |
| 717 |
:ref:`text-io` implement retry behavior. |
| 718 |
|
| 719 |
.. versionchanged:: 3.3 |
| 720 |
The *opener* parameter was added. |
| 721 |
The ``'x'`` mode was added. |
| 722 |
|
| 723 |
.. versionchanged:: 3.4 |
| 724 |
The file is now non-inheritable. |
| 725 |
|
| 726 |
:class:`FileIO` provides these data attributes in addition to those from |
| 727 |
:class:`RawIOBase` and :class:`IOBase`: |
| 728 |
|
| 729 |
.. attribute:: mode |
| 730 |
|
| 731 |
The mode as given in the constructor. |
| 732 |
|
| 733 |
.. attribute:: name |
| 734 |
|
| 735 |
The file name. This is the file descriptor of the file when no name is |
| 736 |
given in the constructor. |
| 737 |
|
| 738 |
|
| 739 |
Buffered Streams |
| 740 |
^^^^^^^^^^^^^^^^ |
| 741 |
|
| 742 |
Buffered I/O streams provide a higher-level interface to an I/O device |
| 743 |
than raw I/O does. |
| 744 |
|
| 745 |
.. class:: BytesIO(initial_bytes=b'') |
| 746 |
|
| 747 |
A binary stream using an in-memory bytes buffer. It inherits from |
| 748 |
:class:`BufferedIOBase`. The buffer is discarded when the |
| 749 |
:meth:`~IOBase.close` method is called. |
| 750 |
|
| 751 |
The optional argument *initial_bytes* is a :term:`bytes-like object` that |
| 752 |
contains initial data. |
| 753 |
|
| 754 |
:class:`BytesIO` provides or overrides these methods in addition to those |
| 755 |
from :class:`BufferedIOBase` and :class:`IOBase`: |
| 756 |
|
| 757 |
.. method:: getbuffer() |
| 758 |
|
| 759 |
Return a readable and writable view over the contents of the buffer |
| 760 |
without copying them. Also, mutating the view will transparently |
| 761 |
update the contents of the buffer:: |
| 762 |
|
| 763 |
>>> b = io.BytesIO(b"abcdef") |
| 764 |
>>> view = b.getbuffer() |
| 765 |
>>> view[2:4] = b"56" |
| 766 |
>>> b.getvalue() |
| 767 |
b'ab56ef' |
| 768 |
|
| 769 |
.. note:: |
| 770 |
As long as the view exists, the :class:`BytesIO` object cannot be |
| 771 |
resized or closed. |
| 772 |
|
| 773 |
.. versionadded:: 3.2 |
| 774 |
|
| 775 |
.. method:: getvalue() |
| 776 |
|
| 777 |
Return :class:`bytes` containing the entire contents of the buffer. |
| 778 |
|
| 779 |
|
| 780 |
.. method:: read1(size=-1, /) |
| 781 |
|
| 782 |
In :class:`BytesIO`, this is the same as :meth:`~BufferedIOBase.read`. |
| 783 |
|
| 784 |
.. versionchanged:: 3.7 |
| 785 |
The *size* argument is now optional. |
| 786 |
|
| 787 |
.. method:: readinto1(b, /) |
| 788 |
|
| 789 |
In :class:`BytesIO`, this is the same as :meth:`~BufferedIOBase.readinto`. |
| 790 |
|
| 791 |
.. versionadded:: 3.5 |
| 792 |
|
| 793 |
.. class:: BufferedReader(raw, buffer_size=DEFAULT_BUFFER_SIZE) |
| 794 |
|
| 795 |
A buffered binary stream providing higher-level access to a readable, non |
| 796 |
seekable :class:`RawIOBase` raw binary stream. It inherits from |
| 797 |
:class:`BufferedIOBase`. |
| 798 |
|
| 799 |
When reading data from this object, a larger amount of data may be |
| 800 |
requested from the underlying raw stream, and kept in an internal buffer. |
| 801 |
The buffered data can then be returned directly on subsequent reads. |
| 802 |
|
| 803 |
The constructor creates a :class:`BufferedReader` for the given readable |
| 804 |
*raw* stream and *buffer_size*. If *buffer_size* is omitted, |
| 805 |
:data:`DEFAULT_BUFFER_SIZE` is used. |
| 806 |
|
| 807 |
:class:`BufferedReader` provides or overrides these methods in addition to |
| 808 |
those from :class:`BufferedIOBase` and :class:`IOBase`: |
| 809 |
|
| 810 |
.. method:: peek(size=0, /) |
| 811 |
|
| 812 |
Return bytes from the stream without advancing the position. The number of |
| 813 |
bytes returned may be less or more than requested. If the underlying raw |
| 814 |
stream is non-blocking and the operation would block, returns empty bytes. |
| 815 |
|
| 816 |
.. method:: read(size=-1, /) |
| 817 |
|
| 818 |
In :class:`BufferedReader` this is the same as :meth:`io.BufferedIOBase.read` |
| 819 |
|
| 820 |
.. method:: read1(size=-1, /) |
| 821 |
|
| 822 |
In :class:`BufferedReader` this is the same as :meth:`io.BufferedIOBase.read1` |
| 823 |
|
| 824 |
.. versionchanged:: 3.7 |
| 825 |
The *size* argument is now optional. |
| 826 |
|
| 827 |
.. class:: BufferedWriter(raw, buffer_size=DEFAULT_BUFFER_SIZE) |
| 828 |
|
| 829 |
A buffered binary stream providing higher-level access to a writeable, non |
| 830 |
seekable :class:`RawIOBase` raw binary stream. It inherits from |
| 831 |
:class:`BufferedIOBase`. |
| 832 |
|
| 833 |
When writing to this object, data is normally placed into an internal |
| 834 |
buffer. The buffer will be written out to the underlying :class:`RawIOBase` |
| 835 |
object under various conditions, including: |
| 836 |
|
| 837 |
* when the buffer gets too small for all pending data; |
| 838 |
* when :meth:`flush` is called; |
| 839 |
* when a :meth:`~IOBase.seek` is requested (for :class:`BufferedRandom` objects); |
| 840 |
* when the :class:`BufferedWriter` object is closed or destroyed. |
| 841 |
|
| 842 |
The constructor creates a :class:`BufferedWriter` for the given writeable |
| 843 |
*raw* stream. If the *buffer_size* is not given, it defaults to |
| 844 |
:data:`DEFAULT_BUFFER_SIZE`. |
| 845 |
|
| 846 |
:class:`BufferedWriter` provides or overrides these methods in addition to |
| 847 |
those from :class:`BufferedIOBase` and :class:`IOBase`: |
| 848 |
|
| 849 |
.. method:: flush() |
| 850 |
|
| 851 |
Force bytes held in the buffer into the raw stream. A |
| 852 |
:exc:`BlockingIOError` should be raised if the raw stream blocks. |
| 853 |
|
| 854 |
.. method:: write(b, /) |
| 855 |
|
| 856 |
Write the :term:`bytes-like object`, *b*, and return the |
| 857 |
number of bytes written. When in non-blocking mode, a |
| 858 |
:exc:`BlockingIOError` with :attr:`BlockingIOError.characters_written` set |
| 859 |
is raised if the buffer needs to be written out but the raw stream blocks. |
| 860 |
|
| 861 |
|
| 862 |
.. class:: BufferedRandom(raw, buffer_size=DEFAULT_BUFFER_SIZE) |
| 863 |
|
| 864 |
A buffered binary stream implementing :class:`BufferedIOBase` interfaces |
| 865 |
providing higher-level access to a seekable :class:`RawIOBase` raw binary |
| 866 |
stream. |
| 867 |
|
| 868 |
The constructor creates a reader and writer for a seekable raw stream, given |
| 869 |
in the first argument. If the *buffer_size* is omitted it defaults to |
| 870 |
:data:`DEFAULT_BUFFER_SIZE`. |
| 871 |
|
| 872 |
:class:`BufferedRandom` is capable of anything :class:`BufferedReader` or |
| 873 |
:class:`BufferedWriter` can do. In addition, :meth:`~IOBase.seek` and |
| 874 |
:meth:`~IOBase.tell` are guaranteed to be implemented. |
| 875 |
|
| 876 |
|
| 877 |
.. class:: BufferedRWPair(reader, writer, buffer_size=DEFAULT_BUFFER_SIZE, /) |
| 878 |
|
| 879 |
A buffered binary stream providing higher-level access to two non seekable |
| 880 |
:class:`RawIOBase` raw binary streams---one readable, the other writeable. |
| 881 |
It inherits from :class:`BufferedIOBase`. |
| 882 |
|
| 883 |
*reader* and *writer* are :class:`RawIOBase` objects that are readable and |
| 884 |
writeable respectively. If the *buffer_size* is omitted it defaults to |
| 885 |
:data:`DEFAULT_BUFFER_SIZE`. |
| 886 |
|
| 887 |
:class:`BufferedRWPair` implements all of :class:`BufferedIOBase`\'s methods |
| 888 |
except for :meth:`~BufferedIOBase.detach`, which raises |
| 889 |
:exc:`UnsupportedOperation`. |
| 890 |
|
| 891 |
.. warning:: |
| 892 |
|
| 893 |
:class:`BufferedRWPair` does not attempt to synchronize accesses to |
| 894 |
its underlying raw streams. You should not pass it the same object |
| 895 |
as reader and writer; use :class:`BufferedRandom` instead. |
| 896 |
|
| 897 |
|
| 898 |
Text I/O |
| 899 |
^^^^^^^^ |
| 900 |
|
| 901 |
.. class:: TextIOBase |
| 902 |
|
| 903 |
Base class for text streams. This class provides a character and line based |
| 904 |
interface to stream I/O. It inherits from :class:`IOBase`. |
| 905 |
|
| 906 |
:class:`TextIOBase` provides or overrides these data attributes and |
| 907 |
methods in addition to those from :class:`IOBase`: |
| 908 |
|
| 909 |
.. attribute:: encoding |
| 910 |
|
| 911 |
The name of the encoding used to decode the stream's bytes into |
| 912 |
strings, and to encode strings into bytes. |
| 913 |
|
| 914 |
.. attribute:: errors |
| 915 |
|
| 916 |
The error setting of the decoder or encoder. |
| 917 |
|
| 918 |
.. attribute:: newlines |
| 919 |
|
| 920 |
A string, a tuple of strings, or ``None``, indicating the newlines |
| 921 |
translated so far. Depending on the implementation and the initial |
| 922 |
constructor flags, this may not be available. |
| 923 |
|
| 924 |
.. attribute:: buffer |
| 925 |
|
| 926 |
The underlying binary buffer (a :class:`BufferedIOBase` |
| 927 |
or :class:`RawIOBase` instance) that :class:`TextIOBase` deals with. |
| 928 |
This is not part of the :class:`TextIOBase` API and may not exist |
| 929 |
in some implementations. |
| 930 |
|
| 931 |
.. method:: detach() |
| 932 |
|
| 933 |
Separate the underlying binary buffer from the :class:`TextIOBase` and |
| 934 |
return it. |
| 935 |
|
| 936 |
After the underlying buffer has been detached, the :class:`TextIOBase` is |
| 937 |
in an unusable state. |
| 938 |
|
| 939 |
Some :class:`TextIOBase` implementations, like :class:`StringIO`, may not |
| 940 |
have the concept of an underlying buffer and calling this method will |
| 941 |
raise :exc:`UnsupportedOperation`. |
| 942 |
|
| 943 |
.. versionadded:: 3.1 |
| 944 |
|
| 945 |
.. method:: read(size=-1, /) |
| 946 |
|
| 947 |
Read and return at most *size* characters from the stream as a single |
| 948 |
:class:`str`. If *size* is negative or ``None``, reads until EOF. |
| 949 |
|
| 950 |
.. method:: readline(size=-1, /) |
| 951 |
|
| 952 |
Read until newline or EOF and return a single :class:`str`. If the stream is |
| 953 |
already at EOF, an empty string is returned. |
| 954 |
|
| 955 |
If *size* is specified, at most *size* characters will be read. |
| 956 |
|
| 957 |
.. method:: seek(offset, whence=SEEK_SET, /) |
| 958 |
|
| 959 |
Change the stream position to the given *offset*. Behaviour depends on |
| 960 |
the *whence* parameter. The default value for *whence* is |
| 961 |
:data:`!SEEK_SET`. |
| 962 |
|
| 963 |
* :data:`!SEEK_SET` or ``0``: seek from the start of the stream |
| 964 |
(the default); *offset* must either be a number returned by |
| 965 |
:meth:`TextIOBase.tell`, or zero. Any other *offset* value |
| 966 |
produces undefined behaviour. |
| 967 |
* :data:`!SEEK_CUR` or ``1``: "seek" to the current position; |
| 968 |
*offset* must be zero, which is a no-operation (all other values |
| 969 |
are unsupported). |
| 970 |
* :data:`!SEEK_END` or ``2``: seek to the end of the stream; |
| 971 |
*offset* must be zero (all other values are unsupported). |
| 972 |
|
| 973 |
Return the new absolute position as an opaque number. |
| 974 |
|
| 975 |
.. versionadded:: 3.1 |
| 976 |
The :data:`!SEEK_*` constants. |
| 977 |
|
| 978 |
.. method:: tell() |
| 979 |
|
| 980 |
Return the current stream position as an opaque number. The number |
| 981 |
does not usually represent a number of bytes in the underlying |
| 982 |
binary storage. |
| 983 |
|
| 984 |
.. method:: write(s, /) |
| 985 |
|
| 986 |
Write the string *s* to the stream and return the number of characters |
| 987 |
written. |
| 988 |
|
| 989 |
|
| 990 |
.. class:: TextIOWrapper(buffer, encoding=None, errors=None, newline=None, \ |
| 991 |
line_buffering=False, write_through=False) |
| 992 |
|
| 993 |
A buffered text stream providing higher-level access to a |
| 994 |
:class:`BufferedIOBase` buffered binary stream. It inherits from |
| 995 |
:class:`TextIOBase`. |
| 996 |
|
| 997 |
*encoding* gives the name of the encoding that the stream will be decoded or |
| 998 |
encoded with. In :ref:`UTF-8 Mode <utf8-mode>`, this defaults to UTF-8. |
| 999 |
Otherwise, it defaults to :func:`locale.getencoding`. |
| 1000 |
``encoding="locale"`` can be used to specify the current locale's encoding |
| 1001 |
explicitly. See :ref:`io-text-encoding` for more information. |
| 1002 |
|
| 1003 |
*errors* is an optional string that specifies how encoding and decoding |
| 1004 |
errors are to be handled. Pass ``'strict'`` to raise a :exc:`ValueError` |
| 1005 |
exception if there is an encoding error (the default of ``None`` has the same |
| 1006 |
effect), or pass ``'ignore'`` to ignore errors. (Note that ignoring encoding |
| 1007 |
errors can lead to data loss.) ``'replace'`` causes a replacement marker |
| 1008 |
(such as ``'?'``) to be inserted where there is malformed data. |
| 1009 |
``'backslashreplace'`` causes malformed data to be replaced by a |
| 1010 |
backslashed escape sequence. When writing, ``'xmlcharrefreplace'`` |
| 1011 |
(replace with the appropriate XML character reference) or ``'namereplace'`` |
| 1012 |
(replace with ``\N{...}`` escape sequences) can be used. Any other error |
| 1013 |
handling name that has been registered with |
| 1014 |
:func:`codecs.register_error` is also valid. |
| 1015 |
|
| 1016 |
.. index:: |
| 1017 |
single: universal newlines; io.TextIOWrapper class |
| 1018 |
|
| 1019 |
*newline* controls how line endings are handled. It can be ``None``, |
| 1020 |
``''``, ``'\n'``, ``'\r'``, and ``'\r\n'``. It works as follows: |
| 1021 |
|
| 1022 |
* When reading input from the stream, if *newline* is ``None``, |
| 1023 |
:term:`universal newlines` mode is enabled. Lines in the input can end in |
| 1024 |
``'\n'``, ``'\r'``, or ``'\r\n'``, and these are translated into ``'\n'`` |
| 1025 |
before being returned to the caller. If *newline* is ``''``, universal |
| 1026 |
newlines mode is enabled, but line endings are returned to the caller |
| 1027 |
untranslated. If *newline* has any of the other legal values, input lines |
| 1028 |
are only terminated by the given string, and the line ending is returned to |
| 1029 |
the caller untranslated. |
| 1030 |
|
| 1031 |
* When writing output to the stream, if *newline* is ``None``, any ``'\n'`` |
| 1032 |
characters written are translated to the system default line separator, |
| 1033 |
:data:`os.linesep`. If *newline* is ``''`` or ``'\n'``, no translation |
| 1034 |
takes place. If *newline* is any of the other legal values, any ``'\n'`` |
| 1035 |
characters written are translated to the given string. |
| 1036 |
|
| 1037 |
If *line_buffering* is ``True``, :meth:`~IOBase.flush` is implied when a call to |
| 1038 |
write contains a newline character or a carriage return. |
| 1039 |
|
| 1040 |
If *write_through* is ``True``, calls to :meth:`~BufferedIOBase.write` are guaranteed |
| 1041 |
not to be buffered: any data written on the :class:`TextIOWrapper` |
| 1042 |
object is immediately handled to its underlying binary *buffer*. |
| 1043 |
|
| 1044 |
.. versionchanged:: 3.3 |
| 1045 |
The *write_through* argument has been added. |
| 1046 |
|
| 1047 |
.. versionchanged:: 3.3 |
| 1048 |
The default *encoding* is now ``locale.getpreferredencoding(False)`` |
| 1049 |
instead of ``locale.getpreferredencoding()``. Don't change temporary the |
| 1050 |
locale encoding using :func:`locale.setlocale`, use the current locale |
| 1051 |
encoding instead of the user preferred encoding. |
| 1052 |
|
| 1053 |
.. versionchanged:: 3.10 |
| 1054 |
The *encoding* argument now supports the ``"locale"`` dummy encoding name. |
| 1055 |
|
| 1056 |
.. note:: |
| 1057 |
|
| 1058 |
When the underlying raw stream is non-blocking, a :exc:`BlockingIOError` |
| 1059 |
may be raised if a read operation cannot be completed immediately. |
| 1060 |
|
| 1061 |
:class:`TextIOWrapper` provides these data attributes and methods in |
| 1062 |
addition to those from :class:`TextIOBase` and :class:`IOBase`: |
| 1063 |
|
| 1064 |
.. attribute:: line_buffering |
| 1065 |
|
| 1066 |
Whether line buffering is enabled. |
| 1067 |
|
| 1068 |
.. attribute:: write_through |
| 1069 |
|
| 1070 |
Whether writes are passed immediately to the underlying binary |
| 1071 |
buffer. |
| 1072 |
|
| 1073 |
.. versionadded:: 3.7 |
| 1074 |
|
| 1075 |
.. method:: reconfigure(*, encoding=None, errors=None, newline=None, \ |
| 1076 |
line_buffering=None, write_through=None) |
| 1077 |
|
| 1078 |
Reconfigure this text stream using new settings for *encoding*, |
| 1079 |
*errors*, *newline*, *line_buffering* and *write_through*. |
| 1080 |
|
| 1081 |
Parameters not specified keep current settings, except |
| 1082 |
``errors='strict'`` is used when *encoding* is specified but |
| 1083 |
*errors* is not specified. |
| 1084 |
|
| 1085 |
It is not possible to change the encoding or newline if some data |
| 1086 |
has already been read from the stream. On the other hand, changing |
| 1087 |
encoding after write is possible. |
| 1088 |
|
| 1089 |
This method does an implicit stream flush before setting the |
| 1090 |
new parameters. |
| 1091 |
|
| 1092 |
.. versionadded:: 3.7 |
| 1093 |
|
| 1094 |
.. versionchanged:: 3.11 |
| 1095 |
The method supports ``encoding="locale"`` option. |
| 1096 |
|
| 1097 |
.. method:: seek(cookie, whence=os.SEEK_SET, /) |
| 1098 |
|
| 1099 |
Set the stream position. |
| 1100 |
Return the new stream position as an :class:`int`. |
| 1101 |
|
| 1102 |
Four operations are supported, |
| 1103 |
given by the following argument combinations: |
| 1104 |
|
| 1105 |
* ``seek(0, SEEK_SET)``: Rewind to the start of the stream. |
| 1106 |
* ``seek(cookie, SEEK_SET)``: Restore a previous position; |
| 1107 |
*cookie* **must be** a number returned by :meth:`tell`. |
| 1108 |
* ``seek(0, SEEK_END)``: Fast-forward to the end of the stream. |
| 1109 |
* ``seek(0, SEEK_CUR)``: Leave the current stream position unchanged. |
| 1110 |
|
| 1111 |
Any other argument combinations are invalid, |
| 1112 |
and may raise exceptions. |
| 1113 |
|
| 1114 |
.. seealso:: |
| 1115 |
|
| 1116 |
:data:`os.SEEK_SET`, :data:`os.SEEK_CUR`, and :data:`os.SEEK_END`. |
| 1117 |
|
| 1118 |
.. method:: tell() |
| 1119 |
|
| 1120 |
Return the stream position as an opaque number. |
| 1121 |
The return value of :meth:`!tell` can be given as input to :meth:`seek`, |
| 1122 |
to restore a previous stream position. |
| 1123 |
|
| 1124 |
|
| 1125 |
.. class:: StringIO(initial_value='', newline='\n') |
| 1126 |
|
| 1127 |
A text stream using an in-memory text buffer. It inherits from |
| 1128 |
:class:`TextIOBase`. |
| 1129 |
|
| 1130 |
The text buffer is discarded when the :meth:`~IOBase.close` method is |
| 1131 |
called. |
| 1132 |
|
| 1133 |
The initial value of the buffer can be set by providing *initial_value*. |
| 1134 |
If newline translation is enabled, newlines will be encoded as if by |
| 1135 |
:meth:`~TextIOBase.write`. The stream is positioned at the start of the |
| 1136 |
buffer which emulates opening an existing file in a ``w+`` mode, making it |
| 1137 |
ready for an immediate write from the beginning or for a write that |
| 1138 |
would overwrite the initial value. To emulate opening a file in an ``a+`` |
| 1139 |
mode ready for appending, use ``f.seek(0, io.SEEK_END)`` to reposition the |
| 1140 |
stream at the end of the buffer. |
| 1141 |
|
| 1142 |
The *newline* argument works like that of :class:`TextIOWrapper`, |
| 1143 |
except that when writing output to the stream, if *newline* is ``None``, |
| 1144 |
newlines are written as ``\n`` on all platforms. |
| 1145 |
|
| 1146 |
:class:`StringIO` provides this method in addition to those from |
| 1147 |
:class:`TextIOBase` and :class:`IOBase`: |
| 1148 |
|
| 1149 |
.. method:: getvalue() |
| 1150 |
|
| 1151 |
Return a :class:`str` containing the entire contents of the buffer. |
| 1152 |
Newlines are decoded as if by :meth:`~TextIOBase.read`, although |
| 1153 |
the stream position is not changed. |
| 1154 |
|
| 1155 |
Example usage:: |
| 1156 |
|
| 1157 |
import io |
| 1158 |
|
| 1159 |
output = io.StringIO() |
| 1160 |
output.write('First line.\n') |
| 1161 |
print('Second line.', file=output) |
| 1162 |
|
| 1163 |
# Retrieve file contents -- this will be |
| 1164 |
# 'First line.\nSecond line.\n' |
| 1165 |
contents = output.getvalue() |
| 1166 |
|
| 1167 |
# Close object and discard memory buffer -- |
| 1168 |
# .getvalue() will now raise an exception. |
| 1169 |
output.close() |
| 1170 |
|
| 1171 |
|
| 1172 |
.. index:: |
| 1173 |
single: universal newlines; io.IncrementalNewlineDecoder class |
| 1174 |
|
| 1175 |
.. class:: IncrementalNewlineDecoder |
| 1176 |
|
| 1177 |
A helper codec that decodes newlines for :term:`universal newlines` mode. |
| 1178 |
It inherits from :class:`codecs.IncrementalDecoder`. |
| 1179 |
|
| 1180 |
|
| 1181 |
Static Typing |
| 1182 |
------------- |
| 1183 |
|
| 1184 |
The following protocols can be used for annotating function and method |
| 1185 |
arguments for simple stream reading or writing operations. They are decorated |
| 1186 |
with :deco:`typing.runtime_checkable`. |
| 1187 |
|
| 1188 |
.. class:: Reader[T] |
| 1189 |
|
| 1190 |
Generic protocol for reading from a file or other input stream. ``T`` will |
| 1191 |
usually be :class:`str` or :class:`bytes`, but can be any type that is |
| 1192 |
read from the stream. |
| 1193 |
|
| 1194 |
.. versionadded:: 3.14 |
| 1195 |
|
| 1196 |
.. method:: read() |
| 1197 |
read(size, /) |
| 1198 |
|
| 1199 |
Read data from the input stream and return it. If *size* is |
| 1200 |
specified, it should be an integer, and at most *size* items |
| 1201 |
(bytes/characters) will be read. |
| 1202 |
|
| 1203 |
For example:: |
| 1204 |
|
| 1205 |
def read_it(reader: Reader[str]): |
| 1206 |
data = reader.read(11) |
| 1207 |
assert isinstance(data, str) |
| 1208 |
|
| 1209 |
.. class:: Writer[T] |
| 1210 |
|
| 1211 |
Generic protocol for writing to a file or other output stream. ``T`` will |
| 1212 |
usually be :class:`str` or :class:`bytes`, but can be any type that can be |
| 1213 |
written to the stream. |
| 1214 |
|
| 1215 |
.. versionadded:: 3.14 |
| 1216 |
|
| 1217 |
.. method:: write(data, /) |
| 1218 |
|
| 1219 |
Write *data* to the output stream and return the number of items |
| 1220 |
(bytes/characters) written. |
| 1221 |
|
| 1222 |
For example:: |
| 1223 |
|
| 1224 |
def write_binary(writer: Writer[bytes]): |
| 1225 |
writer.write(b"Hello world!\n") |
| 1226 |
|
| 1227 |
See :ref:`typing-io` for other I/O related protocols and classes that can be |
| 1228 |
used for static type checking. |
| 1229 |
|
| 1230 |
Performance |
| 1231 |
----------- |
| 1232 |
|
| 1233 |
This section discusses the performance of the provided concrete I/O |
| 1234 |
implementations. |
| 1235 |
|
| 1236 |
Binary I/O |
| 1237 |
^^^^^^^^^^ |
| 1238 |
|
| 1239 |
By reading and writing only large chunks of data even when the user asks for a |
| 1240 |
single byte, buffered I/O hides any inefficiency in calling and executing the |
| 1241 |
operating system's unbuffered I/O routines. The gain depends on the OS and the |
| 1242 |
kind of I/O which is performed. For example, on some modern OSes such as Linux, |
| 1243 |
unbuffered disk I/O can be as fast as buffered I/O. The bottom line, however, |
| 1244 |
is that buffered I/O offers predictable performance regardless of the platform |
| 1245 |
and the backing device. Therefore, it is almost always preferable to use |
| 1246 |
buffered I/O rather than unbuffered I/O for binary data. |
| 1247 |
|
| 1248 |
Text I/O |
| 1249 |
^^^^^^^^ |
| 1250 |
|
| 1251 |
Text I/O over a binary storage (such as a file) is significantly slower than |
| 1252 |
binary I/O over the same storage, because it requires conversions between |
| 1253 |
unicode and binary data using a character codec. This can become noticeable |
| 1254 |
handling huge amounts of text data like large log files. Also, |
| 1255 |
:meth:`~TextIOBase.tell` and :meth:`~TextIOBase.seek` are both quite slow |
| 1256 |
due to the reconstruction algorithm used. |
| 1257 |
|
| 1258 |
:class:`StringIO`, however, is a native in-memory unicode container and will |
| 1259 |
exhibit similar speed to :class:`BytesIO`. |
| 1260 |
|
| 1261 |
Multi-threading |
| 1262 |
^^^^^^^^^^^^^^^ |
| 1263 |
|
| 1264 |
:class:`FileIO` objects are thread-safe to the extent that the operating system |
| 1265 |
calls (such as :manpage:`read(2)` under Unix) they wrap are thread-safe too. |
| 1266 |
|
| 1267 |
Binary buffered objects (instances of :class:`BufferedReader`, |
| 1268 |
:class:`BufferedWriter`, :class:`BufferedRandom` and :class:`BufferedRWPair`) |
| 1269 |
protect their internal structures using a lock; it is therefore safe to call |
| 1270 |
them from multiple threads at once. |
| 1271 |
|
| 1272 |
:class:`TextIOWrapper` objects are not thread-safe. |
| 1273 |
|
| 1274 |
Reentrancy |
| 1275 |
^^^^^^^^^^ |
| 1276 |
|
| 1277 |
Binary buffered objects (instances of :class:`BufferedReader`, |
| 1278 |
:class:`BufferedWriter`, :class:`BufferedRandom` and :class:`BufferedRWPair`) |
| 1279 |
are not reentrant. While reentrant calls will not happen in normal situations, |
| 1280 |
they can arise from doing I/O in a :mod:`signal` handler. If a thread tries to |
| 1281 |
re-enter a buffered object which it is already accessing, a :exc:`RuntimeError` |
| 1282 |
is raised. Note this doesn't prohibit a different thread from entering the |
| 1283 |
buffered object. |
| 1284 |
|
| 1285 |
The above implicitly extends to text files, since the :func:`open` function |
| 1286 |
will wrap a buffered object inside a :class:`TextIOWrapper`. This includes |
| 1287 |
standard streams and therefore affects the built-in :func:`print` function as |
| 1288 |
well. |