Debian Codegraph @

Doc/tutorial/inputoutput.rst

raw
1 .. _tut-io:
2
3 ****************
4 Input and Output
5 ****************
6
7 There are several ways to present the output of a program; data can be printed
8 in a human-readable form, or written to a file for future use. This chapter will
9 discuss some of the possibilities.
10
11
12 .. _tut-formatting:
13
14 Fancier Output Formatting
15 =========================
16
17 So far we've encountered two ways of writing values: *expression statements* and
18 the :func:`print` function. (A third way is using the :meth:`~io.TextIOBase.write` method
19 of file objects; the standard output file can be referenced as ``sys.stdout``.
20 See the Library Reference for more information on this.)
21
22 Often you'll want more control over the formatting of your output than simply
23 printing space-separated values. There are several ways to format output.
24
25 * To use :ref:`formatted string literals <tut-f-strings>`, begin a string
26 with ``f`` or ``F`` before the opening quotation mark or triple quotation mark.
27 Inside this string, you can write a Python expression between ``{`` and ``}``
28 characters that can refer to variables or literal values.
29
30 ::
31
32 >>> year = 2016
33 >>> event = 'Referendum'
34 >>> f'Results of the {year} {event}'
35 'Results of the 2016 Referendum'
36
37 * The :meth:`str.format` method of strings requires more manual
38 effort. You'll still use ``{`` and ``}`` to mark where a variable
39 will be substituted and can provide detailed formatting directives,
40 but you'll also need to provide the information to be formatted. In the following code
41 block there are two examples of how to format variables:
42
43
44 ::
45
46 >>> yes_votes = 42_572_654
47 >>> total_votes = 85_705_149
48 >>> percentage = yes_votes / total_votes
49 >>> '{:-9} YES votes {:2.2%}'.format(yes_votes, percentage)
50 ' 42572654 YES votes 49.67%'
51
52 Notice how the ``yes_votes`` are padded with spaces and a negative sign only for negative numbers.
53 The example also prints ``percentage`` multiplied by 100, with 2 decimal
54 places and followed by a percent sign (see :ref:`formatspec` for details).
55
56
57 * Finally, you can do all the string handling yourself by using string slicing and
58 concatenation operations to create any layout you can imagine. The
59 string type has some methods that perform useful operations for padding
60 strings to a given column width.
61
62 When you don't need fancy output but just want a quick display of some
63 variables for debugging purposes, you can convert any value to a string with
64 the :func:`repr` or :func:`str` functions.
65
66 The :func:`str` function is meant to return representations of values which are
67 fairly human-readable, while :func:`repr` is meant to generate representations
68 which can be read by the interpreter (or will force a :exc:`SyntaxError` if
69 there is no equivalent syntax). For objects which don't have a particular
70 representation for human consumption, :func:`str` will return the same value as
71 :func:`repr`. Many values, such as numbers or structures like lists and
72 dictionaries, have the same representation using either function. Strings, in
73 particular, have two distinct representations.
74
75 Some examples::
76
77 >>> s = 'Hello, world.'
78 >>> str(s)
79 'Hello, world.'
80 >>> repr(s)
81 "'Hello, world.'"
82 >>> str(1/7)
83 '0.14285714285714285'
84 >>> x = 10 * 3.25
85 >>> y = 200 * 200
86 >>> s = 'The value of x is ' + repr(x) + ', and y is ' + repr(y) + '...'
87 >>> print(s)
88 The value of x is 32.5, and y is 40000...
89 >>> # The repr() of a string adds string quotes and backslashes:
90 >>> hello = 'hello, world\n'
91 >>> hellos = repr(hello)
92 >>> print(hellos)
93 'hello, world\n'
94 >>> # The argument to repr() may be any Python object:
95 >>> repr((x, y, ('spam', 'eggs')))
96 "(32.5, 40000, ('spam', 'eggs'))"
97
98 The :mod:`string` module contains support for a simple templating approach
99 based upon regular expressions, via :class:`string.Template`.
100 This offers yet another way to substitute values into strings,
101 using placeholders like ``$x`` and replacing them with values from a dictionary.
102 This syntax is easy to use, although it offers much less control for formatting.
103
104 .. index::
105 single: formatted string literal
106 single: interpolated string literal
107 single: string; formatted literal
108 single: string; interpolated literal
109 single: f-string
110 single: fstring
111
112 .. _tut-f-strings:
113
114 Formatted String Literals
115 -------------------------
116
117 :ref:`Formatted string literals <f-strings>` (also called f-strings for
118 short) let you include the value of Python expressions inside a string by
119 prefixing the string with ``f`` or ``F`` and writing expressions as
120 ``{expression}``.
121
122 An optional format specifier can follow the expression. This allows greater
123 control over how the value is formatted. The following example rounds pi to
124 three places after the decimal::
125
126 >>> import math
127 >>> print(f'The value of pi is approximately {math.pi:.3f}.')
128 The value of pi is approximately 3.142.
129
130 Passing an integer after the ``':'`` will cause that field to be a minimum
131 number of characters wide. This is useful for making columns line up. ::
132
133 >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 7678}
134 >>> for name, phone in table.items():
135 ... print(f'{name:10} ==> {phone:10d}')
136 ...
137 Sjoerd ==> 4127
138 Jack ==> 4098
139 Dcab ==> 7678
140
141 Other modifiers can be used to convert the value before it is formatted.
142 ``'!a'`` applies :func:`ascii`, ``'!s'`` applies :func:`str`, and ``'!r'``
143 applies :func:`repr`::
144
145 >>> animals = 'eels'
146 >>> print(f'My hovercraft is full of {animals}.')
147 My hovercraft is full of eels.
148 >>> print(f'My hovercraft is full of {animals!r}.')
149 My hovercraft is full of 'eels'.
150
151 The ``=`` specifier can be used to expand an expression to the text of the
152 expression, an equal sign, then the representation of the evaluated expression:
153
154 >>> bugs = 'roaches'
155 >>> count = 13
156 >>> area = 'living room'
157 >>> print(f'Debugging {bugs=} {count=} {area=}')
158 Debugging bugs='roaches' count=13 area='living room'
159
160 See :ref:`self-documenting expressions <bpo-36817-whatsnew>` for more information
161 on the ``=`` specifier. For a reference on these format specifications, see
162 the reference guide for the :ref:`formatspec`.
163
164 .. _tut-string-format:
165
166 The String format() Method
167 --------------------------
168
169 Basic usage of the :meth:`str.format` method looks like this::
170
171 >>> print('We are the {} who say "{}!"'.format('knights', 'Ni'))
172 We are the knights who say "Ni!"
173
174 The brackets and characters within them (called format fields) are replaced with
175 the objects passed into the :meth:`str.format` method. A number in the
176 brackets can be used to refer to the position of the object passed into the
177 :meth:`str.format` method. ::
178
179 >>> print('{0} and {1}'.format('spam', 'eggs'))
180 spam and eggs
181 >>> print('{1} and {0}'.format('spam', 'eggs'))
182 eggs and spam
183
184 If keyword arguments are used in the :meth:`str.format` method, their values
185 are referred to by using the name of the argument. ::
186
187 >>> print('This {food} is {adjective}.'.format(
188 ... food='spam', adjective='absolutely horrible'))
189 This spam is absolutely horrible.
190
191 Positional and keyword arguments can be arbitrarily combined::
192
193 >>> print('The story of {0}, {1}, and {other}.'.format('Bill', 'Manfred',
194 ... other='Georg'))
195 The story of Bill, Manfred, and Georg.
196
197 If you have a really long format string that you don't want to split up, it
198 would be nice if you could reference the variables to be formatted by name
199 instead of by position. This can be done by simply passing the dict and using
200 square brackets ``'[]'`` to access the keys. ::
201
202 >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 8637678}
203 >>> print('Jack: {0[Jack]:d}; Sjoerd: {0[Sjoerd]:d}; '
204 ... 'Dcab: {0[Dcab]:d}'.format(table))
205 Jack: 4098; Sjoerd: 4127; Dcab: 8637678
206
207 This could also be done by passing the ``table`` dictionary as keyword arguments with the ``**``
208 notation. ::
209
210 >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 8637678}
211 >>> print('Jack: {Jack:d}; Sjoerd: {Sjoerd:d}; Dcab: {Dcab:d}'.format(**table))
212 Jack: 4098; Sjoerd: 4127; Dcab: 8637678
213
214 This is particularly useful in combination with the built-in function
215 :func:`vars`, which returns a dictionary containing all local variables::
216
217 >>> table = {k: str(v) for k, v in vars().items()}
218 >>> message = " ".join([f'{k}: ' + '{' + k +'};' for k in table.keys()])
219 >>> print(message.format(**table))
220 __name__: __main__; __doc__: None; __package__: None; __loader__: ...
221
222 As an example, the following lines produce a tidily aligned
223 set of columns giving integers and their squares and cubes::
224
225 >>> for x in range(1, 11):
226 ... print('{0:2d} {1:3d} {2:4d}'.format(x, x*x, x*x*x))
227 ...
228 1 1 1
229 2 4 8
230 3 9 27
231 4 16 64
232 5 25 125
233 6 36 216
234 7 49 343
235 8 64 512
236 9 81 729
237 10 100 1000
238
239 For a complete overview of string formatting with :meth:`str.format`, see
240 :ref:`formatstrings`.
241
242
243 Manual String Formatting
244 ------------------------
245
246 Here's the same table of squares and cubes, formatted manually::
247
248 >>> for x in range(1, 11):
249 ... print(repr(x).rjust(2), repr(x*x).rjust(3), end=' ')
250 ... # Note use of 'end' on previous line
251 ... print(repr(x*x*x).rjust(4))
252 ...
253 1 1 1
254 2 4 8
255 3 9 27
256 4 16 64
257 5 25 125
258 6 36 216
259 7 49 343
260 8 64 512
261 9 81 729
262 10 100 1000
263
264 (Note that the one space between each column was added by the
265 way :func:`print` works: it always adds spaces between its arguments.)
266
267 The :meth:`str.rjust` method of string objects right-justifies a string in a
268 field of a given width by padding it with spaces on the left. There are
269 similar methods :meth:`str.ljust` and :meth:`str.center`. These methods do
270 not write anything, they just return a new string. If the input string is too
271 long, they don't truncate it, but return it unchanged; this will mess up your
272 column lay-out but that's usually better than the alternative, which would be
273 lying about a value. (If you really want truncation you can always add a
274 slice operation, as in ``x.ljust(n)[:n]``.)
275
276 There is another method, :meth:`str.zfill`, which pads a numeric string on the
277 left with zeros. It understands about plus and minus signs::
278
279 >>> '12'.zfill(5)
280 '00012'
281 >>> '-3.14'.zfill(7)
282 '-003.14'
283 >>> '3.14159265359'.zfill(5)
284 '3.14159265359'
285
286
287 Old string formatting
288 ---------------------
289
290 The % operator (modulo) can also be used for string formatting.
291 Given ``format % values`` (where *format* is a string),
292 ``%`` conversion specifications in *format* are replaced with
293 zero or more elements of *values*.
294 This operation is commonly known as string
295 interpolation. For example::
296
297 >>> import math
298 >>> print('The value of pi is approximately %5.3f.' % math.pi)
299 The value of pi is approximately 3.142.
300
301 More information can be found in the :ref:`old-string-formatting` section.
302
303
304 .. _tut-files:
305
306 Reading and Writing Files
307 =========================
308
309 .. index::
310 pair: built-in function; open
311 pair: object; file
312
313 :func:`open` returns a :term:`file object`, and is most commonly used with
314 two positional arguments and one keyword argument:
315 ``open(filename, mode, encoding=None)``
316
317 ::
318
319 >>> f = open('workfile', 'w', encoding="utf-8")
320
321 .. XXX str(f) is <io.TextIOWrapper object at 0x82e8dc4>
322
323 >>> print(f)
324 <open file 'workfile', mode 'w' at 80a0960>
325
326 The first argument is a string containing the filename. The second argument is
327 another string containing a few characters describing the way in which the file
328 will be used. *mode* can be ``'r'`` when the file will only be read, ``'w'``
329 for only writing (an existing file with the same name will be erased), and
330 ``'a'`` opens the file for appending; any data written to the file is
331 automatically added to the end. ``'r+'`` opens the file for both reading and
332 writing. The *mode* argument is optional; ``'r'`` will be assumed if it's
333 omitted.
334
335 Normally, files are opened in :dfn:`text mode`, that means, you read and write
336 strings from and to the file, which are encoded in a specific *encoding*.
337 If *encoding* is not specified, the default is platform dependent
338 (see :func:`open`).
339 Because UTF-8 is the modern de-facto standard, ``encoding="utf-8"`` is
340 recommended unless you know that you need to use a different encoding.
341 Appending a ``'b'`` to the mode opens the file in :dfn:`binary mode`.
342 Binary mode data is read and written as :class:`bytes` objects.
343 You can not specify *encoding* when opening file in binary mode.
344
345 In text mode, the default when reading is to convert platform-specific line
346 endings (``\n`` on Unix, ``\r\n`` on Windows) to just ``\n``. When writing in
347 text mode, the default is to convert occurrences of ``\n`` back to
348 platform-specific line endings. This behind-the-scenes modification
349 to file data is fine for text files, but will corrupt binary data like that in
350 :file:`JPEG` or :file:`EXE` files. Be very careful to use binary mode when
351 reading and writing such files.
352
353 It is good practice to use the :keyword:`with` keyword when dealing
354 with file objects. The advantage is that the file is properly closed
355 after its suite finishes, even if an exception is raised at some
356 point. Using :keyword:`!with` is also much shorter than writing
357 equivalent :keyword:`try`\ -\ :keyword:`finally` blocks::
358
359 >>> with open('workfile', encoding="utf-8") as f:
360 ... read_data = f.read()
361
362 >>> # We can check that the file has been automatically closed.
363 >>> f.closed
364 True
365
366 If you're not using the :keyword:`with` keyword, then you should call
367 ``f.close()`` to close the file and immediately free up any system
368 resources used by it.
369
370 .. warning::
371 Calling ``f.write()`` without using the :keyword:`!with` keyword or calling
372 ``f.close()`` **might** result in the arguments
373 of ``f.write()`` not being completely written to the disk, even if the
374 program exits successfully.
375
376 ..
377 See also https://bugs.python.org/issue17852
378
379 After a file object is closed, either by a :keyword:`with` statement
380 or by calling ``f.close()``, attempts to use the file object will
381 automatically fail. ::
382
383 >>> f.close()
384 >>> f.read()
385 Traceback (most recent call last):
386 File "<stdin>", line 1, in <module>
387 ValueError: I/O operation on closed file.
388
389
390 .. _tut-filemethods:
391
392 Methods of File Objects
393 -----------------------
394
395 The rest of the examples in this section will assume that a file object called
396 ``f`` has already been created.
397
398 To read a file's contents, call ``f.read(size)``, which reads some quantity of
399 data and returns it as a string (in text mode) or bytes object (in binary mode).
400 *size* is an optional numeric argument. When *size* is omitted or negative, the
401 entire contents of the file will be read and returned; it's your problem if the
402 file is twice as large as your machine's memory. Otherwise, at most *size*
403 characters (in text mode) or *size* bytes (in binary mode) are read and returned.
404 If the end of the file has been reached, ``f.read()`` will return an empty
405 string (``''``). ::
406
407 >>> f.read()
408 'This is the entire file.\n'
409 >>> f.read()
410 ''
411
412 ``f.readline()`` reads a single line from the file; a newline character (``\n``)
413 is left at the end of the string, and is only omitted on the last line of the
414 file if the file doesn't end in a newline. This makes the return value
415 unambiguous; if ``f.readline()`` returns an empty string, the end of the file
416 has been reached, while a blank line is represented by ``'\n'``, a string
417 containing only a single newline. ::
418
419 >>> f.readline()
420 'This is the first line of the file.\n'
421 >>> f.readline()
422 'Second line of the file\n'
423 >>> f.readline()
424 ''
425
426 For reading lines from a file, you can loop over the file object. This is memory
427 efficient, fast, and leads to simple code::
428
429 >>> for line in f:
430 ... print(line, end='')
431 ...
432 This is the first line of the file.
433 Second line of the file
434
435 If you want to read all the lines of a file in a list you can also use
436 ``list(f)`` or ``f.readlines()``.
437
438 ``f.write(string)`` writes the contents of *string* to the file, returning
439 the number of characters written. ::
440
441 >>> f.write('This is a test\n')
442 15
443
444 Other types of objects need to be converted -- either to a string (in text mode)
445 or a bytes object (in binary mode) -- before writing them::
446
447 >>> value = ('the answer', 42)
448 >>> s = str(value) # convert the tuple to string
449 >>> f.write(s)
450 18
451
452 ``f.tell()`` returns an integer giving the file object's current position in the file
453 represented as number of bytes from the beginning of the file when in binary mode and
454 an opaque number when in text mode.
455
456 To change the file object's position, use ``f.seek(offset, whence)``. The position is computed
457 from adding *offset* to a reference point; the reference point is selected by
458 the *whence* argument. A *whence* value of 0 measures from the beginning
459 of the file, 1 uses the current file position, and 2 uses the end of the file as
460 the reference point. *whence* can be omitted and defaults to 0, using the
461 beginning of the file as the reference point. ::
462
463 >>> f = open('workfile', 'rb+')
464 >>> f.write(b'0123456789abcdef')
465 16
466 >>> f.seek(5) # Go to the 6th byte in the file
467 5
468 >>> f.read(1)
469 b'5'
470 >>> f.seek(-3, 2) # Go to the 3rd byte before the end
471 13
472 >>> f.read(1)
473 b'd'
474
475 In text files (those opened without a ``b`` in the mode string), only seeks
476 relative to the beginning of the file are allowed (the exception being seeking
477 to the very file end with ``seek(0, 2)``) and the only valid *offset* values are
478 those returned from the ``f.tell()``, or zero. Any other *offset* value produces
479 undefined behaviour.
480
481 File objects have some additional methods, such as :meth:`~io.IOBase.isatty` and
482 :meth:`~io.IOBase.truncate` which are less frequently used; consult the Library
483 Reference for a complete guide to file objects.
484
485
486 .. _tut-json:
487
488 Saving structured data with :mod:`json`
489 ---------------------------------------
490
491 .. index:: pair: module; json
492
493 Strings can easily be written to and read from a file. Numbers take a bit more
494 effort, since the :meth:`~io.TextIOBase.read` method only returns strings, which will have to
495 be passed to a function like :func:`int`, which takes a string like ``'123'``
496 and returns its numeric value 123. When you want to save more complex data
497 types like nested lists and dictionaries, parsing and serializing by hand
498 becomes complicated.
499
500 Rather than having users constantly writing and debugging code to save
501 complicated data types to files, Python allows you to use the popular data
502 interchange format called `JSON (JavaScript Object Notation)
503 <https://json.org>`_. The standard module called :mod:`json` can take Python
504 data hierarchies, and convert them to string representations; this process is
505 called :dfn:`serializing`. Reconstructing the data from the string representation
506 is called :dfn:`deserializing`. Between serializing and deserializing, the
507 string representing the object may have been stored in a file or data, or
508 sent over a network connection to some distant machine.
509
510 .. note::
511 The JSON format is commonly used by modern applications to allow for data
512 exchange. Many programmers are already familiar with it, which makes
513 it a good choice for interoperability.
514
515 If you have an object ``x``, you can view its JSON string representation with a
516 simple line of code::
517
518 >>> import json
519 >>> x = [1, 'simple', 'list']
520 >>> json.dumps(x)
521 '[1, "simple", "list"]'
522
523 Another variant of the :func:`~json.dumps` function, called :func:`~json.dump`,
524 simply serializes the object to a :term:`text file`. So if ``f`` is a
525 :term:`text file` object opened for writing, we can do this::
526
527 json.dump(x, f)
528
529 To decode the object again, if ``f`` is a :term:`binary file` or
530 :term:`text file` object which has been opened for reading::
531
532 x = json.load(f)
533
534 .. note::
535 JSON files must be encoded in UTF-8. Use ``encoding="utf-8"`` when opening
536 JSON file as a :term:`text file` for both of reading and writing.
537
538 This simple serialization technique can handle lists and dictionaries, but
539 serializing arbitrary class instances in JSON requires a bit of extra effort.
540 The reference for the :mod:`json` module contains an explanation of this.
541
542 .. seealso::
543
544 :mod:`pickle` - the pickle module
545
546 Contrary to :ref:`JSON <tut-json>`, *pickle* is a protocol which allows
547 the serialization of arbitrarily complex Python objects. As such, it is
548 specific to Python and cannot be used to communicate with applications
549 written in other languages. It is also insecure by default:
550 deserializing pickle data coming from an untrusted source can execute
551 arbitrary code, if the data was crafted by a skilled attacker.