Debian Codegraph @

Doc/library/csv.rst

raw
1 :mod:`!csv` --- CSV File Reading and Writing
2 ============================================
3
4 .. module:: csv
5 :synopsis: Write and read tabular data to and from delimited files.
6
7 .. sectionauthor:: Skip Montanaro <[email protected]>
8
9 **Source code:** :source:`Lib/csv.py`
10
11 .. index::
12 single: csv
13 pair: data; tabular
14
15 --------------
16
17 The so-called CSV (Comma Separated Values) format is the most common import and
18 export format for spreadsheets and databases. CSV format was used for many
19 years prior to attempts to describe the format in a standardized way in
20 :rfc:`4180`. The lack of a well-defined standard means that subtle differences
21 often exist in the data produced and consumed by different applications. These
22 differences can make it annoying to process CSV files from multiple sources.
23 Still, while the delimiters and quoting characters vary, the overall format is
24 similar enough that it is possible to write a single module which can
25 efficiently manipulate such data, hiding the details of reading and writing the
26 data from the programmer.
27
28 The :mod:`!csv` module implements classes to read and write tabular data in CSV
29 format. It allows programmers to say, "write this data in the format preferred
30 by Excel," or "read data from this file which was generated by Excel," without
31 knowing the precise details of the CSV format used by Excel. Programmers can
32 also describe the CSV formats understood by other applications or define their
33 own special-purpose CSV formats.
34
35 The :mod:`!csv` module's :class:`reader` and :class:`writer` objects read and
36 write sequences. Programmers can also read and write data in dictionary form
37 using the :class:`DictReader` and :class:`DictWriter` classes.
38
39 .. seealso::
40
41 :pep:`305` - CSV File API
42 The Python Enhancement Proposal which proposed this addition to Python.
43
44
45 .. _csv-contents:
46
47 Module Contents
48 ---------------
49
50 The :mod:`!csv` module defines the following functions:
51
52
53 .. index::
54 single: universal newlines; csv.reader function
55
56 .. function:: reader(csvfile, /, dialect='excel', **fmtparams)
57
58 Return a :ref:`reader object <reader-objects>` that will process
59 lines from the given *csvfile*. A csvfile must be an iterable of
60 strings, each in the reader's defined csv format.
61 A csvfile is most commonly a file-like object or list.
62 If *csvfile* is a file object,
63 it should be opened with ``newline=''``. [1]_ An optional
64 *dialect* parameter can be given which is used to define a set of parameters
65 specific to a particular CSV dialect. It may be an instance of a subclass of
66 the :class:`Dialect` class or one of the strings returned by the
67 :func:`list_dialects` function. The other optional *fmtparams* keyword arguments
68 can be given to override individual formatting parameters in the current
69 dialect. For full details about the dialect and formatting parameters, see
70 section :ref:`csv-fmt-params`.
71
72 Each row read from the csv file is returned as a list of strings. No
73 automatic data type conversion is performed unless the :data:`QUOTE_NONNUMERIC` format
74 option is specified (in which case unquoted fields are transformed into floats).
75
76 A short usage example::
77
78 >>> import csv
79 >>> with open('eggs.csv', newline='') as csvfile:
80 ... spamreader = csv.reader(csvfile, delimiter=' ', quotechar='|')
81 ... for row in spamreader:
82 ... print(', '.join(row))
83 Spam, Spam, Spam, Spam, Spam, Baked Beans
84 Spam, Lovely Spam, Wonderful Spam
85
86
87 .. function:: writer(csvfile, /, dialect='excel', **fmtparams)
88
89 Return a writer object responsible for converting the user's data into delimited
90 strings on the given file-like object. *csvfile* can be any object with a
91 :meth:`~io.TextIOBase.write` method. If *csvfile* is a file object, it should be opened with
92 ``newline=''`` [1]_. An optional *dialect*
93 parameter can be given which is used to define a set of parameters specific to a
94 particular CSV dialect. It may be an instance of a subclass of the
95 :class:`Dialect` class or one of the strings returned by the
96 :func:`list_dialects` function. The other optional *fmtparams* keyword arguments
97 can be given to override individual formatting parameters in the current
98 dialect. For full details about dialects and formatting parameters, see
99 the :ref:`csv-fmt-params` section. To make it
100 as easy as possible to interface with modules which implement the DB API, the
101 value :const:`None` is written as the empty string. While this isn't a
102 reversible transformation, it makes it easier to dump SQL NULL data values to
103 CSV files without preprocessing the data returned from a ``cursor.fetch*`` call.
104 All other non-string data are stringified with :func:`str` before being written.
105
106 A short usage example::
107
108 import csv
109 with open('eggs.csv', 'w', newline='') as csvfile:
110 spamwriter = csv.writer(csvfile, delimiter=' ',
111 quotechar='|', quoting=csv.QUOTE_MINIMAL)
112 spamwriter.writerow(['Spam'] * 5 + ['Baked Beans'])
113 spamwriter.writerow(['Spam', 'Lovely Spam', 'Wonderful Spam'])
114
115
116 .. function:: register_dialect(name, /, dialect='excel', **fmtparams)
117
118 Associate *dialect* with *name*. *name* must be a string. The
119 dialect can be specified either by passing a sub-class of :class:`Dialect`, or
120 by *fmtparams* keyword arguments, or both, with keyword arguments overriding
121 parameters of the dialect. For full details about dialects and formatting
122 parameters, see section :ref:`csv-fmt-params`.
123
124
125 .. function:: unregister_dialect(name)
126
127 Delete the dialect associated with *name* from the dialect registry. An
128 :exc:`Error` is raised if *name* is not a registered dialect name.
129
130
131 .. function:: get_dialect(name)
132
133 Return the dialect associated with *name*. An :exc:`Error` is raised if
134 *name* is not a registered dialect name. This function returns an immutable
135 :class:`Dialect`.
136
137 .. function:: list_dialects()
138
139 Return the names of all registered dialects.
140
141
142 .. function:: field_size_limit()
143 field_size_limit(new_limit)
144
145 Returns the current maximum field size allowed by the parser. If *new_limit* is
146 given, this becomes the new limit.
147
148
149 The :mod:`!csv` module defines the following classes:
150
151 .. class:: DictReader(f, fieldnames=None, restkey=None, restval=None, \
152 dialect='excel', *args, **kwds)
153
154 Create an object that operates like a regular reader but maps the
155 information in each row to a :class:`dict` whose keys are given by the
156 optional *fieldnames* parameter.
157
158 The *fieldnames* parameter is a :term:`sequence`. If *fieldnames* is
159 omitted, the values in the first row of file *f* will be used as the
160 fieldnames and will be omitted from the results. If
161 *fieldnames* is provided, they will be used and the first row will be
162 included in the results. Regardless of how the fieldnames are determined,
163 the dictionary preserves their original ordering.
164
165 If a row has more fields than fieldnames, the remaining data is put in a
166 list and stored with the fieldname specified by *restkey* (which defaults
167 to ``None``). If a non-blank row has fewer fields than fieldnames, the
168 missing values are filled-in with the value of *restval* (which defaults
169 to ``None``).
170
171 All other optional or keyword arguments are passed to the underlying
172 :class:`reader` instance.
173
174 If the argument passed to *fieldnames* is an iterator, it will be coerced to a :class:`list`.
175
176 .. versionchanged:: 3.6
177 Returned rows are now of type :class:`OrderedDict`.
178
179 .. versionchanged:: 3.8
180 Returned rows are now of type :class:`dict`.
181
182 A short usage example::
183
184 >>> import csv
185 >>> with open('names.csv', newline='') as csvfile:
186 ... reader = csv.DictReader(csvfile)
187 ... for row in reader:
188 ... print(row['first_name'], row['last_name'])
189 ...
190 Eric Idle
191 John Cleese
192
193 >>> print(row)
194 {'first_name': 'John', 'last_name': 'Cleese'}
195
196
197 .. class:: DictWriter(f, fieldnames, restval='', extrasaction='raise', \
198 dialect='excel', *args, **kwds)
199
200 Create an object which operates like a regular writer but maps dictionaries
201 onto output rows. The *fieldnames* parameter is a :mod:`sequence
202 <collections.abc>` of keys that identify the order in which values in the
203 dictionary passed to the :meth:`~csvwriter.writerow` method are written to file
204 *f*. The optional *restval* parameter specifies the value to be
205 written if the dictionary is missing a key in *fieldnames*. If the
206 dictionary passed to the :meth:`~csvwriter.writerow` method contains a key not found in
207 *fieldnames*, the optional *extrasaction* parameter indicates what action to
208 take.
209 If it is set to ``'raise'``, the default value, a :exc:`ValueError`
210 is raised.
211 If it is set to ``'ignore'``, extra values in the dictionary are ignored.
212 Any other optional or keyword arguments are passed to the underlying
213 :class:`writer` instance.
214
215 Note that unlike the :class:`DictReader` class, the *fieldnames* parameter
216 of the :class:`DictWriter` class is not optional.
217
218 If the argument passed to *fieldnames* is an iterator, it will be coerced to a :class:`list`.
219
220 A short usage example::
221
222 import csv
223
224 with open('names.csv', 'w', newline='') as csvfile:
225 fieldnames = ['first_name', 'last_name']
226 writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
227
228 writer.writeheader()
229 writer.writerow({'first_name': 'Baked', 'last_name': 'Beans'})
230 writer.writerow({'first_name': 'Lovely', 'last_name': 'Spam'})
231 writer.writerow({'first_name': 'Wonderful', 'last_name': 'Spam'})
232
233
234 .. class:: Dialect
235
236 The :class:`Dialect` class is a container class whose attributes contain
237 information for how to handle doublequotes, whitespace, delimiters, etc.
238 Due to the lack of a strict CSV specification, different applications
239 produce subtly different CSV data. :class:`Dialect` instances define how
240 :class:`reader` and :class:`writer` instances behave.
241
242 All available :class:`Dialect` names are returned by :func:`list_dialects`,
243 and they can be registered with specific :class:`reader` and :class:`writer`
244 classes through their initializer (``__init__``) functions like this::
245
246 import csv
247
248 with open('students.csv', 'w', newline='') as csvfile:
249 writer = csv.writer(csvfile, dialect='unix')
250
251
252 .. class:: excel()
253
254 The :class:`excel` class defines the usual properties of an Excel-generated CSV
255 file. It is registered with the dialect name ``'excel'``.
256
257
258 .. class:: excel_tab()
259
260 The :class:`excel_tab` class defines the usual properties of an Excel-generated
261 TAB-delimited file. It is registered with the dialect name ``'excel-tab'``.
262
263
264 .. class:: unix_dialect()
265
266 The :class:`unix_dialect` class defines the usual properties of a CSV file
267 generated on UNIX systems, i.e. using ``'\n'`` as line terminator and quoting
268 all fields. It is registered with the dialect name ``'unix'``.
269
270 .. versionadded:: 3.2
271
272
273 .. class:: Sniffer()
274
275 The :class:`Sniffer` class is used to deduce the format of a CSV file.
276
277 The :class:`Sniffer` class provides two methods:
278
279 .. method:: sniff(sample, delimiters=None)
280
281 Analyze the given *sample* and return a :class:`Dialect` subclass
282 reflecting the parameters found. If the optional *delimiters* parameter
283 is given, it is interpreted as a string containing possible valid
284 delimiter characters.
285
286
287 .. method:: has_header(sample)
288
289 Analyze the sample text (presumed to be in CSV format) and return
290 :const:`True` if the first row appears to be a series of column headers.
291 Inspecting each column, one of two key criteria will be considered to
292 estimate if the sample contains a header:
293
294 - the second through n-th rows contain numeric values
295 - the second through n-th rows contain strings where at least one value's
296 length differs from that of the putative header of that column.
297
298 Twenty-one rows after the header are sampled; if more than half of the
299 columns + rows meet the criteria, :const:`True` is returned.
300
301 .. note::
302
303 This method is a rough heuristic and may produce both false positives and
304 negatives.
305
306 An example for :class:`Sniffer` use::
307
308 with open('example.csv', newline='') as csvfile:
309 dialect = csv.Sniffer().sniff(csvfile.read(1024))
310 csvfile.seek(0)
311 reader = csv.reader(csvfile, dialect)
312 # ... process CSV file contents here ...
313
314
315 .. _csv-constants:
316
317 The :mod:`!csv` module defines the following constants:
318
319 .. data:: QUOTE_ALL
320
321 Instructs :class:`writer` objects to quote all fields.
322
323
324 .. data:: QUOTE_MINIMAL
325
326 Instructs :class:`writer` objects to only quote those fields which contain
327 special characters such as *delimiter*, *quotechar*, ``'\r'``, ``'\n'``
328 or any of the characters in *lineterminator*.
329
330
331 .. data:: QUOTE_NONNUMERIC
332
333 Instructs :class:`writer` objects to quote all non-numeric fields.
334
335 Instructs :class:`reader` objects to convert all non-quoted fields to type :class:`float`.
336
337 .. note::
338 Some numeric types, such as :class:`bool`, :class:`~fractions.Fraction`,
339 or :class:`~enum.IntEnum`, have a string representation that cannot be
340 converted to :class:`float`.
341 They cannot be read in the :data:`QUOTE_NONNUMERIC` and
342 :data:`QUOTE_STRINGS` modes.
343
344 .. data:: QUOTE_NONE
345
346 Instructs :class:`writer` objects to never quote fields.
347 When the current *delimiter*, *quotechar*, *escapechar*, ``'\r'``, ``'\n'``
348 or any of the characters in *lineterminator* occurs in output data
349 it is preceded by the current *escapechar* character.
350 If *escapechar* is not set, the writer will raise :exc:`Error` if
351 any characters that require escaping are encountered.
352 Set *quotechar* to ``None`` to prevent its escaping.
353
354 Instructs :class:`reader` objects to perform no special processing of quote characters.
355
356 .. data:: QUOTE_NOTNULL
357
358 Instructs :class:`writer` objects to quote all fields which are not
359 ``None``. This is similar to :data:`QUOTE_ALL`, except that if a
360 field value is ``None`` an empty (unquoted) string is written.
361
362 Instructs :class:`reader` objects to interpret an empty (unquoted) field
363 as ``None`` and to otherwise behave as :data:`QUOTE_ALL`.
364
365 .. versionadded:: 3.12
366
367 .. data:: QUOTE_STRINGS
368
369 Instructs :class:`writer` objects to always place quotes around fields
370 which are strings. This is similar to :data:`QUOTE_NONNUMERIC`, except that if a
371 field value is ``None`` an empty (unquoted) string is written.
372
373 Instructs :class:`reader` objects to interpret an empty (unquoted) string as ``None`` and
374 to otherwise behave as :data:`QUOTE_NONNUMERIC`.
375
376 .. versionadded:: 3.12
377
378 The :mod:`!csv` module defines the following exception:
379
380
381 .. exception:: Error
382
383 Raised by any of the functions when an error is detected.
384
385 .. _csv-fmt-params:
386
387 Dialects and Formatting Parameters
388 ----------------------------------
389
390 To make it easier to specify the format of input and output records, specific
391 formatting parameters are grouped together into dialects. A dialect is a
392 subclass of the :class:`Dialect` class containing various attributes
393 describing the format of the CSV file. When creating :class:`reader` or
394 :class:`writer` objects, the programmer can specify a string or a subclass of
395 the :class:`Dialect` class as the dialect parameter. In addition to, or instead
396 of, the *dialect* parameter, the programmer can also specify individual
397 formatting parameters, which have the same names as the attributes defined below
398 for the :class:`Dialect` class.
399
400 Dialects support the following attributes:
401
402
403 .. attribute:: Dialect.delimiter
404
405 A one-character string used to separate fields. It defaults to ``','``.
406
407
408 .. attribute:: Dialect.doublequote
409
410 Controls how instances of *quotechar* appearing inside a field should
411 themselves be quoted. When :const:`True`, the character is doubled. When
412 :const:`False`, the *escapechar* is used as a prefix to the *quotechar*. It
413 defaults to :const:`True`.
414
415 On output, if *doublequote* is :const:`False` and no *escapechar* is set,
416 :exc:`Error` is raised if a *quotechar* is found in a field.
417
418
419 .. attribute:: Dialect.escapechar
420
421 A one-character string used by the writer to escape characters that
422 require escaping:
423
424 * the *delimiter*, the *quotechar*, ``'\r'``, ``'\n'`` and any of the
425 characters in *lineterminator* are escaped if *quoting* is set to
426 :const:`QUOTE_NONE`;
427 * the *quotechar* is escaped if *doublequote* is :const:`False`;
428 * the *escapechar* itself.
429
430 On reading, the *escapechar* removes any special meaning from
431 the following character. It defaults to :const:`None`, which disables escaping.
432
433 .. versionchanged:: 3.11
434 An empty *escapechar* is not allowed.
435
436 .. attribute:: Dialect.lineterminator
437
438 The string used to terminate lines produced by the :class:`writer`. It defaults
439 to ``'\r\n'``.
440
441 .. note::
442
443 The :class:`reader` is hard-coded to recognise either ``'\r'`` or ``'\n'`` as
444 end-of-line, and ignores *lineterminator*. This behavior may change in the
445 future.
446
447
448 .. attribute:: Dialect.quotechar
449
450 A one-character string used to quote fields containing special characters,
451 such as the *delimiter* or the *quotechar*, or which contain new-line
452 characters (``'\r'``, ``'\n'`` or any of the characters in *lineterminator*).
453 It defaults to ``'"'``.
454 Can be set to ``None`` to prevent escaping ``'"'`` if *quoting* is set
455 to :const:`QUOTE_NONE`.
456
457 .. versionchanged:: 3.11
458 An empty *quotechar* is not allowed.
459
460 .. attribute:: Dialect.quoting
461
462 Controls when quotes should be generated by the writer and recognised by the
463 reader. It can take on any of the :ref:`QUOTE_\* constants <csv-constants>`
464 and defaults to :const:`QUOTE_MINIMAL` if *quotechar* is not ``None``,
465 and :const:`QUOTE_NONE` otherwise.
466
467
468 .. attribute:: Dialect.skipinitialspace
469
470 When :const:`True`, spaces immediately following the *delimiter* are ignored.
471 The default is :const:`False`. When combining ``delimiter=' '`` with
472 ``skipinitialspace=True``, unquoted empty fields are not allowed.
473
474
475 .. attribute:: Dialect.strict
476
477 When ``True``, raise exception :exc:`Error` on bad CSV input.
478 The default is ``False``.
479
480 .. _reader-objects:
481
482 Reader Objects
483 --------------
484
485 Reader objects (:class:`DictReader` instances and objects returned by the
486 :func:`reader` function) have the following public methods:
487
488 .. method:: csvreader.__next__()
489
490 Return the next row of the reader's iterable object as a list (if the object
491 was returned from :func:`reader`) or a dict (if it is a :class:`DictReader`
492 instance), parsed according to the current :class:`Dialect`. Usually you
493 should call this as ``next(reader)``.
494
495
496 Reader objects have the following public attributes:
497
498 .. attribute:: csvreader.dialect
499
500 A read-only description of the dialect in use by the parser.
501
502
503 .. attribute:: csvreader.line_num
504
505 The number of lines read from the source iterator. This is not the same as the
506 number of records returned, as records can span multiple lines.
507
508
509 DictReader objects have the following public attribute:
510
511 .. attribute:: DictReader.fieldnames
512
513 If not passed as a parameter when creating the object, this attribute is
514 initialized upon first access or when the first record is read from the
515 file.
516
517
518
519 Writer Objects
520 --------------
521
522 :class:`writer` objects (:class:`DictWriter` instances and objects returned by
523 the :func:`writer` function) have the following public methods. A *row* must be
524 an iterable of strings or numbers for :class:`writer` objects and a dictionary
525 mapping fieldnames to strings or numbers (by passing them through :func:`str`
526 first) for :class:`DictWriter` objects. Note that complex numbers are written
527 out surrounded by parens. This may cause some problems for other programs which
528 read CSV files (assuming they support complex numbers at all).
529
530
531 .. method:: csvwriter.writerow(row, /)
532
533 Write the *row* parameter to the writer's file object, formatted according
534 to the current :class:`Dialect`. Return the return value of the call to the
535 *write* method of the underlying file object.
536
537 .. versionchanged:: 3.5
538 Added support of arbitrary iterables.
539
540 .. method:: csvwriter.writerows(rows, /)
541
542 Write all elements in *rows* (an iterable of *row* objects as described
543 above) to the writer's file object, formatted according to the current
544 dialect.
545
546 Writer objects have the following public attribute:
547
548
549 .. attribute:: csvwriter.dialect
550
551 A read-only description of the dialect in use by the writer.
552
553
554 DictWriter objects have the following public method:
555
556
557 .. method:: DictWriter.writeheader()
558
559 Write a row with the field names (as specified in the constructor) to
560 the writer's file object, formatted according to the current dialect. Return
561 the return value of the :meth:`csvwriter.writerow` call used internally.
562
563 .. versionadded:: 3.2
564 .. versionchanged:: 3.8
565 :meth:`writeheader` now also returns the value returned by
566 the :meth:`csvwriter.writerow` method it uses internally.
567
568
569 .. _csv-examples:
570
571 Examples
572 --------
573
574 The simplest example of reading a CSV file::
575
576 import csv
577 with open('some.csv', newline='') as f:
578 reader = csv.reader(f)
579 for row in reader:
580 print(row)
581
582 Reading a file with an alternate format::
583
584 import csv
585 with open('passwd', newline='') as f:
586 reader = csv.reader(f, delimiter=':', quoting=csv.QUOTE_NONE)
587 for row in reader:
588 print(row)
589
590 The corresponding simplest possible writing example is::
591
592 import csv
593 with open('some.csv', 'w', newline='') as f:
594 writer = csv.writer(f)
595 writer.writerows(someiterable)
596
597 Since :func:`open` is used to open a CSV file for reading, the file
598 will by default be decoded into unicode using the system default
599 encoding (see :func:`locale.getencoding`). To decode a file
600 using a different encoding, use the ``encoding`` argument of open::
601
602 import csv
603 with open('some.csv', newline='', encoding='utf-8') as f:
604 reader = csv.reader(f)
605 for row in reader:
606 print(row)
607
608 The same applies to writing in something other than the system default
609 encoding: specify the encoding argument when opening the output file.
610
611 Registering a new dialect::
612
613 import csv
614 csv.register_dialect('unixpwd', delimiter=':', quoting=csv.QUOTE_NONE)
615 with open('passwd', newline='') as f:
616 reader = csv.reader(f, 'unixpwd')
617
618 A slightly more advanced use of the reader --- catching and reporting errors::
619
620 import csv, sys
621 filename = 'some.csv'
622 with open(filename, newline='') as f:
623 reader = csv.reader(f)
624 try:
625 for row in reader:
626 print(row)
627 except csv.Error as e:
628 sys.exit(f'file {filename}, line {reader.line_num}: {e}')
629
630 And while the module doesn't directly support parsing strings, it can easily be
631 done::
632
633 import csv
634 for row in csv.reader(['one,two,three']):
635 print(row)
636
637
638 .. rubric:: Footnotes
639
640 .. [1] If ``newline=''`` is not specified, newlines embedded inside quoted fields
641 will not be interpreted correctly, and on platforms that use ``\r\n`` line endings
642 on write an extra ``\r`` will be added. It should always be safe to specify
643 ``newline=''``, since the csv module does its own
644 (:term:`universal <universal newlines>`) newline handling.