Debian Codegraph @

Doc/tutorial/inputoutput.rst

raw
1 .. _tut-io:
2
3 ****************
4 Input and Output
5 ****************
6
7 There are several ways to present the output of a program; data can be printed
8 in a human-readable form, or written to a file for future use. This chapter will
9 discuss some of the possibilities.
10
11
12 .. _tut-formatting:
13
14 Fancier Output Formatting
15 =========================
16
17 So far we've encountered two ways of writing values: *expression statements* and
18 the :func:`print` function. (A third way is using the :meth:`~io.TextIOBase.write` method
19 of file objects; the standard output file can be referenced as ``sys.stdout``.
20 See the Library Reference for more information on this.)
21
22 Often you'll want more control over the formatting of your output than simply
23 printing space-separated values. There are several ways to format output.
24
25 * To use :ref:`formatted string literals <tut-f-strings>`, begin a string
26 with ``f`` or ``F`` before the opening quotation mark or triple quotation mark.
27 Inside this string, you can write a Python expression between ``{`` and ``}``
28 characters that can refer to variables or literal values.
29
30 ::
31
32 >>> year = 2016
33 >>> event = 'Referendum'
34 >>> f'Results of the {year} {event}'
35 'Results of the 2016 Referendum'
36
37 * The :meth:`str.format` method of strings requires more manual
38 effort. You'll still use ``{`` and ``}`` to mark where a variable
39 will be substituted and can provide detailed formatting directives,
40 but you'll also need to provide the information to be formatted. In the following code
41 block there are two examples of how to format variables:
42
43
44 ::
45
46 >>> yes_votes = 42_572_654
47 >>> total_votes = 85_705_149
48 >>> percentage = yes_votes / total_votes
49 >>> '{:-9} YES votes {:2.2%}'.format(yes_votes, percentage)
50 ' 42572654 YES votes 49.67%'
51
52 Notice how the ``yes_votes`` are padded with spaces and a negative sign only for negative numbers.
53 The example also prints ``percentage`` multiplied by 100, with 2 decimal
54 places and followed by a percent sign (see :ref:`formatspec` for details).
55
56
57 * Finally, you can do all the string handling yourself by using string slicing and
58 concatenation operations to create any layout you can imagine. The
59 string type has some methods that perform useful operations for padding
60 strings to a given column width.
61
62 When you don't need fancy output but just want a quick display of some
63 variables for debugging purposes, you can convert any value to a string with
64 the :func:`repr` or :func:`str` functions.
65
66 The :func:`str` function is meant to return representations of values which are
67 fairly human-readable, while :func:`repr` is meant to generate representations
68 which can be read by the interpreter (or will force a :exc:`SyntaxError` if
69 there is no equivalent syntax). For objects which don't have a particular
70 representation for human consumption, :func:`str` will return the same value as
71 :func:`repr`. Many values, such as numbers or structures like lists and
72 dictionaries, have the same representation using either function. Strings, in
73 particular, have two distinct representations.
74
75 Some examples::
76
77 >>> s = 'Hello, world.'
78 >>> str(s)
79 'Hello, world.'
80 >>> repr(s)
81 "'Hello, world.'"
82 >>> str(1/7)
83 '0.14285714285714285'
84 >>> x = 10 * 3.25
85 >>> y = 200 * 200
86 >>> s = 'The value of x is ' + repr(x) + ', and y is ' + repr(y) + '...'
87 >>> print(s)
88 The value of x is 32.5, and y is 40000...
89 >>> # The repr() of a string adds string quotes and backslashes:
90 >>> hello = 'hello, world\n'
91 >>> hellos = repr(hello)
92 >>> print(hellos)
93 'hello, world\n'
94 >>> # The argument to repr() may be any Python object:
95 >>> repr((x, y, ('spam', 'eggs')))
96 "(32.5, 40000, ('spam', 'eggs'))"
97
98 The :mod:`string` module contains a :class:`~string.Template` class that offers
99 yet another way to substitute values into strings, using placeholders like
100 ``$x`` and replacing them with values from a dictionary, but offers much less
101 control of the formatting.
102
103 .. index::
104 single: formatted string literal
105 single: interpolated string literal
106 single: string; formatted literal
107 single: string; interpolated literal
108 single: f-string
109 single: fstring
110
111 .. _tut-f-strings:
112
113 Formatted String Literals
114 -------------------------
115
116 :ref:`Formatted string literals <f-strings>` (also called f-strings for
117 short) let you include the value of Python expressions inside a string by
118 prefixing the string with ``f`` or ``F`` and writing expressions as
119 ``{expression}``.
120
121 An optional format specifier can follow the expression. This allows greater
122 control over how the value is formatted. The following example rounds pi to
123 three places after the decimal::
124
125 >>> import math
126 >>> print(f'The value of pi is approximately {math.pi:.3f}.')
127 The value of pi is approximately 3.142.
128
129 Passing an integer after the ``':'`` will cause that field to be a minimum
130 number of characters wide. This is useful for making columns line up. ::
131
132 >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 7678}
133 >>> for name, phone in table.items():
134 ... print(f'{name:10} ==> {phone:10d}')
135 ...
136 Sjoerd ==> 4127
137 Jack ==> 4098
138 Dcab ==> 7678
139
140 Other modifiers can be used to convert the value before it is formatted.
141 ``'!a'`` applies :func:`ascii`, ``'!s'`` applies :func:`str`, and ``'!r'``
142 applies :func:`repr`::
143
144 >>> animals = 'eels'
145 >>> print(f'My hovercraft is full of {animals}.')
146 My hovercraft is full of eels.
147 >>> print(f'My hovercraft is full of {animals!r}.')
148 My hovercraft is full of 'eels'.
149
150 The ``=`` specifier can be used to expand an expression to the text of the
151 expression, an equal sign, then the representation of the evaluated expression:
152
153 >>> bugs = 'roaches'
154 >>> count = 13
155 >>> area = 'living room'
156 >>> print(f'Debugging {bugs=} {count=} {area=}')
157 Debugging bugs='roaches' count=13 area='living room'
158
159 See :ref:`self-documenting expressions <bpo-36817-whatsnew>` for more information
160 on the ``=`` specifier. For a reference on these format specifications, see
161 the reference guide for the :ref:`formatspec`.
162
163 .. _tut-string-format:
164
165 The String format() Method
166 --------------------------
167
168 Basic usage of the :meth:`str.format` method looks like this::
169
170 >>> print('We are the {} who say "{}!"'.format('knights', 'Ni'))
171 We are the knights who say "Ni!"
172
173 The brackets and characters within them (called format fields) are replaced with
174 the objects passed into the :meth:`str.format` method. A number in the
175 brackets can be used to refer to the position of the object passed into the
176 :meth:`str.format` method. ::
177
178 >>> print('{0} and {1}'.format('spam', 'eggs'))
179 spam and eggs
180 >>> print('{1} and {0}'.format('spam', 'eggs'))
181 eggs and spam
182
183 If keyword arguments are used in the :meth:`str.format` method, their values
184 are referred to by using the name of the argument. ::
185
186 >>> print('This {food} is {adjective}.'.format(
187 ... food='spam', adjective='absolutely horrible'))
188 This spam is absolutely horrible.
189
190 Positional and keyword arguments can be arbitrarily combined::
191
192 >>> print('The story of {0}, {1}, and {other}.'.format('Bill', 'Manfred',
193 ... other='Georg'))
194 The story of Bill, Manfred, and Georg.
195
196 If you have a really long format string that you don't want to split up, it
197 would be nice if you could reference the variables to be formatted by name
198 instead of by position. This can be done by simply passing the dict and using
199 square brackets ``'[]'`` to access the keys. ::
200
201 >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 8637678}
202 >>> print('Jack: {0[Jack]:d}; Sjoerd: {0[Sjoerd]:d}; '
203 ... 'Dcab: {0[Dcab]:d}'.format(table))
204 Jack: 4098; Sjoerd: 4127; Dcab: 8637678
205
206 This could also be done by passing the ``table`` dictionary as keyword arguments with the ``**``
207 notation. ::
208
209 >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 8637678}
210 >>> print('Jack: {Jack:d}; Sjoerd: {Sjoerd:d}; Dcab: {Dcab:d}'.format(**table))
211 Jack: 4098; Sjoerd: 4127; Dcab: 8637678
212
213 This is particularly useful in combination with the built-in function
214 :func:`vars`, which returns a dictionary containing all local variables::
215
216 >>> table = {k: str(v) for k, v in vars().items()}
217 >>> message = " ".join([f'{k}: ' + '{' + k +'};' for k in table.keys()])
218 >>> print(message.format(**table))
219 __name__: __main__; __doc__: None; __package__: None; __loader__: ...
220
221 As an example, the following lines produce a tidily aligned
222 set of columns giving integers and their squares and cubes::
223
224 >>> for x in range(1, 11):
225 ... print('{0:2d} {1:3d} {2:4d}'.format(x, x*x, x*x*x))
226 ...
227 1 1 1
228 2 4 8
229 3 9 27
230 4 16 64
231 5 25 125
232 6 36 216
233 7 49 343
234 8 64 512
235 9 81 729
236 10 100 1000
237
238 For a complete overview of string formatting with :meth:`str.format`, see
239 :ref:`formatstrings`.
240
241
242 Manual String Formatting
243 ------------------------
244
245 Here's the same table of squares and cubes, formatted manually::
246
247 >>> for x in range(1, 11):
248 ... print(repr(x).rjust(2), repr(x*x).rjust(3), end=' ')
249 ... # Note use of 'end' on previous line
250 ... print(repr(x*x*x).rjust(4))
251 ...
252 1 1 1
253 2 4 8
254 3 9 27
255 4 16 64
256 5 25 125
257 6 36 216
258 7 49 343
259 8 64 512
260 9 81 729
261 10 100 1000
262
263 (Note that the one space between each column was added by the
264 way :func:`print` works: it always adds spaces between its arguments.)
265
266 The :meth:`str.rjust` method of string objects right-justifies a string in a
267 field of a given width by padding it with spaces on the left. There are
268 similar methods :meth:`str.ljust` and :meth:`str.center`. These methods do
269 not write anything, they just return a new string. If the input string is too
270 long, they don't truncate it, but return it unchanged; this will mess up your
271 column lay-out but that's usually better than the alternative, which would be
272 lying about a value. (If you really want truncation you can always add a
273 slice operation, as in ``x.ljust(n)[:n]``.)
274
275 There is another method, :meth:`str.zfill`, which pads a numeric string on the
276 left with zeros. It understands about plus and minus signs::
277
278 >>> '12'.zfill(5)
279 '00012'
280 >>> '-3.14'.zfill(7)
281 '-003.14'
282 >>> '3.14159265359'.zfill(5)
283 '3.14159265359'
284
285
286 Old string formatting
287 ---------------------
288
289 The % operator (modulo) can also be used for string formatting.
290 Given ``format % values`` (where *format* is a string),
291 ``%`` conversion specifications in *format* are replaced with
292 zero or more elements of *values*.
293 This operation is commonly known as string
294 interpolation. For example::
295
296 >>> import math
297 >>> print('The value of pi is approximately %5.3f.' % math.pi)
298 The value of pi is approximately 3.142.
299
300 More information can be found in the :ref:`old-string-formatting` section.
301
302
303 .. _tut-files:
304
305 Reading and Writing Files
306 =========================
307
308 .. index::
309 pair: built-in function; open
310 pair: object; file
311
312 :func:`open` returns a :term:`file object`, and is most commonly used with
313 two positional arguments and one keyword argument:
314 ``open(filename, mode, encoding=None)``
315
316 ::
317
318 >>> f = open('workfile', 'w', encoding="utf-8")
319
320 .. XXX str(f) is <io.TextIOWrapper object at 0x82e8dc4>
321
322 >>> print(f)
323 <open file 'workfile', mode 'w' at 80a0960>
324
325 The first argument is a string containing the filename. The second argument is
326 another string containing a few characters describing the way in which the file
327 will be used. *mode* can be ``'r'`` when the file will only be read, ``'w'``
328 for only writing (an existing file with the same name will be erased), and
329 ``'a'`` opens the file for appending; any data written to the file is
330 automatically added to the end. ``'r+'`` opens the file for both reading and
331 writing. The *mode* argument is optional; ``'r'`` will be assumed if it's
332 omitted.
333
334 Normally, files are opened in :dfn:`text mode`, that means, you read and write
335 strings from and to the file, which are encoded in a specific *encoding*.
336 If *encoding* is not specified, the default is platform dependent
337 (see :func:`open`).
338 Because UTF-8 is the modern de-facto standard, ``encoding="utf-8"`` is
339 recommended unless you know that you need to use a different encoding.
340 Appending a ``'b'`` to the mode opens the file in :dfn:`binary mode`.
341 Binary mode data is read and written as :class:`bytes` objects.
342 You can not specify *encoding* when opening file in binary mode.
343
344 In text mode, the default when reading is to convert platform-specific line
345 endings (``\n`` on Unix, ``\r\n`` on Windows) to just ``\n``. When writing in
346 text mode, the default is to convert occurrences of ``\n`` back to
347 platform-specific line endings. This behind-the-scenes modification
348 to file data is fine for text files, but will corrupt binary data like that in
349 :file:`JPEG` or :file:`EXE` files. Be very careful to use binary mode when
350 reading and writing such files.
351
352 It is good practice to use the :keyword:`with` keyword when dealing
353 with file objects. The advantage is that the file is properly closed
354 after its suite finishes, even if an exception is raised at some
355 point. Using :keyword:`!with` is also much shorter than writing
356 equivalent :keyword:`try`\ -\ :keyword:`finally` blocks::
357
358 >>> with open('workfile', encoding="utf-8") as f:
359 ... read_data = f.read()
360
361 >>> # We can check that the file has been automatically closed.
362 >>> f.closed
363 True
364
365 If you're not using the :keyword:`with` keyword, then you should call
366 ``f.close()`` to close the file and immediately free up any system
367 resources used by it.
368
369 .. warning::
370 Calling ``f.write()`` without using the :keyword:`!with` keyword or calling
371 ``f.close()`` **might** result in the arguments
372 of ``f.write()`` not being completely written to the disk, even if the
373 program exits successfully.
374
375 ..
376 See also https://bugs.python.org/issue17852
377
378 After a file object is closed, either by a :keyword:`with` statement
379 or by calling ``f.close()``, attempts to use the file object will
380 automatically fail. ::
381
382 >>> f.close()
383 >>> f.read()
384 Traceback (most recent call last):
385 File "<stdin>", line 1, in <module>
386 ValueError: I/O operation on closed file.
387
388
389 .. _tut-filemethods:
390
391 Methods of File Objects
392 -----------------------
393
394 The rest of the examples in this section will assume that a file object called
395 ``f`` has already been created.
396
397 To read a file's contents, call ``f.read(size)``, which reads some quantity of
398 data and returns it as a string (in text mode) or bytes object (in binary mode).
399 *size* is an optional numeric argument. When *size* is omitted or negative, the
400 entire contents of the file will be read and returned; it's your problem if the
401 file is twice as large as your machine's memory. Otherwise, at most *size*
402 characters (in text mode) or *size* bytes (in binary mode) are read and returned.
403 If the end of the file has been reached, ``f.read()`` will return an empty
404 string (``''``). ::
405
406 >>> f.read()
407 'This is the entire file.\n'
408 >>> f.read()
409 ''
410
411 ``f.readline()`` reads a single line from the file; a newline character (``\n``)
412 is left at the end of the string, and is only omitted on the last line of the
413 file if the file doesn't end in a newline. This makes the return value
414 unambiguous; if ``f.readline()`` returns an empty string, the end of the file
415 has been reached, while a blank line is represented by ``'\n'``, a string
416 containing only a single newline. ::
417
418 >>> f.readline()
419 'This is the first line of the file.\n'
420 >>> f.readline()
421 'Second line of the file\n'
422 >>> f.readline()
423 ''
424
425 For reading lines from a file, you can loop over the file object. This is memory
426 efficient, fast, and leads to simple code::
427
428 >>> for line in f:
429 ... print(line, end='')
430 ...
431 This is the first line of the file.
432 Second line of the file
433
434 If you want to read all the lines of a file in a list you can also use
435 ``list(f)`` or ``f.readlines()``.
436
437 ``f.write(string)`` writes the contents of *string* to the file, returning
438 the number of characters written. ::
439
440 >>> f.write('This is a test\n')
441 15
442
443 Other types of objects need to be converted -- either to a string (in text mode)
444 or a bytes object (in binary mode) -- before writing them::
445
446 >>> value = ('the answer', 42)
447 >>> s = str(value) # convert the tuple to string
448 >>> f.write(s)
449 18
450
451 ``f.tell()`` returns an integer giving the file object's current position in the file
452 represented as number of bytes from the beginning of the file when in binary mode and
453 an opaque number when in text mode.
454
455 To change the file object's position, use ``f.seek(offset, whence)``. The position is computed
456 from adding *offset* to a reference point; the reference point is selected by
457 the *whence* argument. A *whence* value of 0 measures from the beginning
458 of the file, 1 uses the current file position, and 2 uses the end of the file as
459 the reference point. *whence* can be omitted and defaults to 0, using the
460 beginning of the file as the reference point. ::
461
462 >>> f = open('workfile', 'rb+')
463 >>> f.write(b'0123456789abcdef')
464 16
465 >>> f.seek(5) # Go to the 6th byte in the file
466 5
467 >>> f.read(1)
468 b'5'
469 >>> f.seek(-3, 2) # Go to the 3rd byte before the end
470 13
471 >>> f.read(1)
472 b'd'
473
474 In text files (those opened without a ``b`` in the mode string), only seeks
475 relative to the beginning of the file are allowed (the exception being seeking
476 to the very file end with ``seek(0, 2)``) and the only valid *offset* values are
477 those returned from the ``f.tell()``, or zero. Any other *offset* value produces
478 undefined behaviour.
479
480 File objects have some additional methods, such as :meth:`~io.IOBase.isatty` and
481 :meth:`~io.IOBase.truncate` which are less frequently used; consult the Library
482 Reference for a complete guide to file objects.
483
484
485 .. _tut-json:
486
487 Saving structured data with :mod:`json`
488 ---------------------------------------
489
490 .. index:: pair: module; json
491
492 Strings can easily be written to and read from a file. Numbers take a bit more
493 effort, since the :meth:`~io.TextIOBase.read` method only returns strings, which will have to
494 be passed to a function like :func:`int`, which takes a string like ``'123'``
495 and returns its numeric value 123. When you want to save more complex data
496 types like nested lists and dictionaries, parsing and serializing by hand
497 becomes complicated.
498
499 Rather than having users constantly writing and debugging code to save
500 complicated data types to files, Python allows you to use the popular data
501 interchange format called `JSON (JavaScript Object Notation)
502 <https://json.org>`_. The standard module called :mod:`json` can take Python
503 data hierarchies, and convert them to string representations; this process is
504 called :dfn:`serializing`. Reconstructing the data from the string representation
505 is called :dfn:`deserializing`. Between serializing and deserializing, the
506 string representing the object may have been stored in a file or data, or
507 sent over a network connection to some distant machine.
508
509 .. note::
510 The JSON format is commonly used by modern applications to allow for data
511 exchange. Many programmers are already familiar with it, which makes
512 it a good choice for interoperability.
513
514 If you have an object ``x``, you can view its JSON string representation with a
515 simple line of code::
516
517 >>> import json
518 >>> x = [1, 'simple', 'list']
519 >>> json.dumps(x)
520 '[1, "simple", "list"]'
521
522 Another variant of the :func:`~json.dumps` function, called :func:`~json.dump`,
523 simply serializes the object to a :term:`text file`. So if ``f`` is a
524 :term:`text file` object opened for writing, we can do this::
525
526 json.dump(x, f)
527
528 To decode the object again, if ``f`` is a :term:`binary file` or
529 :term:`text file` object which has been opened for reading::
530
531 x = json.load(f)
532
533 .. note::
534 JSON files must be encoded in UTF-8. Use ``encoding="utf-8"`` when opening
535 JSON file as a :term:`text file` for both of reading and writing.
536
537 This simple serialization technique can handle lists and dictionaries, but
538 serializing arbitrary class instances in JSON requires a bit of extra effort.
539 The reference for the :mod:`json` module contains an explanation of this.
540
541 .. seealso::
542
543 :mod:`pickle` - the pickle module
544
545 Contrary to :ref:`JSON <tut-json>`, *pickle* is a protocol which allows
546 the serialization of arbitrarily complex Python objects. As such, it is
547 specific to Python and cannot be used to communicate with applications
548 written in other languages. It is also insecure by default:
549 deserializing pickle data coming from an untrusted source can execute
550 arbitrary code, if the data was crafted by a skilled attacker.