Debian Codegraph @

Doc/library/gettext.rst

raw
1 :mod:`!gettext` --- Multilingual internationalization services
2 ==============================================================
3
4 .. module:: gettext
5 :synopsis: Multilingual internationalization services.
6
7 .. moduleauthor:: Barry A. Warsaw <[email protected]>
8 .. sectionauthor:: Barry A. Warsaw <[email protected]>
9
10 **Source code:** :source:`Lib/gettext.py`
11
12 --------------
13
14 The :mod:`!gettext` module provides internationalization (I18N) and localization
15 (L10N) services for your Python modules and applications. It supports both the
16 GNU :program:`gettext` message catalog API and a higher level, class-based API that may
17 be more appropriate for Python files. The interface described below allows you
18 to write your module and application messages in one natural language, and
19 provide a catalog of translated messages for running under different natural
20 languages.
21
22 Some hints on localizing your Python modules and applications are also given.
23
24
25 GNU :program:`gettext` API
26 --------------------------
27
28 The :mod:`!gettext` module defines the following API, which is very similar to
29 the GNU :program:`gettext` API. If you use this API you will affect the
30 translation of your entire application globally. Often this is what you want if
31 your application is monolingual, with the choice of language dependent on the
32 locale of your user. If you are localizing a Python module, or if your
33 application needs to switch languages on the fly, you probably want to use the
34 class-based API instead.
35
36
37 .. function:: bindtextdomain(domain, localedir=None)
38
39 Bind the *domain* to the locale directory *localedir*. More concretely,
40 :mod:`!gettext` will look for binary :file:`.mo` files for the given domain using
41 the path (on Unix): :file:`{localedir}/{language}/LC_MESSAGES/{domain}.mo`, where
42 *language* is searched for in the environment variables :envvar:`LANGUAGE`,
43 :envvar:`LC_ALL`, :envvar:`LC_MESSAGES`, and :envvar:`LANG` respectively.
44
45 If *localedir* is omitted or ``None``, then the current binding for *domain* is
46 returned. [#]_
47
48
49 .. function:: textdomain(domain=None)
50
51 Change or query the current global domain. If *domain* is ``None``, then the
52 current global domain is returned, otherwise the global domain is set to
53 *domain*, which is returned.
54
55
56 .. index:: single: _ (underscore); gettext
57 .. function:: gettext(message)
58
59 Return the localized translation of *message*, based on the current global
60 domain, language, and locale directory. This function is usually aliased as
61 :func:`!_` in the local namespace (see examples below).
62
63
64 .. function:: dgettext(domain, message)
65
66 Like :func:`.gettext`, but look the message up in the specified *domain*.
67
68
69 .. function:: ngettext(singular, plural, n)
70
71 Like :func:`.gettext`, but consider plural forms. If a translation is found,
72 apply the plural formula to *n*, and return the resulting message (some
73 languages have more than two plural forms). If no translation is found, return
74 *singular* if *n* is 1; return *plural* otherwise.
75
76 The Plural formula is taken from the catalog header. It is a C or Python
77 expression that has a free variable *n*; the expression evaluates to the index
78 of the plural in the catalog. See
79 `the GNU gettext documentation <https://www.gnu.org/software/gettext/manual/gettext.html>`__
80 for the precise syntax to be used in :file:`.po` files and the
81 formulas for a variety of languages.
82
83
84 .. function:: dngettext(domain, singular, plural, n)
85
86 Like :func:`ngettext`, but look the message up in the specified *domain*.
87
88
89 .. function:: pgettext(context, message)
90 .. function:: dpgettext(domain, context, message)
91 .. function:: npgettext(context, singular, plural, n)
92 .. function:: dnpgettext(domain, context, singular, plural, n)
93
94 Similar to the corresponding functions without the ``p`` in the prefix (that
95 is, :func:`gettext`, :func:`dgettext`, :func:`ngettext`, :func:`dngettext`),
96 but the translation is restricted to the given message *context*.
97
98 .. versionadded:: 3.8
99
100
101 Note that GNU :program:`gettext` also defines a :func:`!dcgettext` method, but
102 this was deemed not useful and so it is currently unimplemented.
103
104 Here's an example of typical usage for this API::
105
106 import gettext
107 gettext.bindtextdomain('myapplication', '/path/to/my/language/directory')
108 gettext.textdomain('myapplication')
109 _ = gettext.gettext
110 # ...
111 print(_('This is a translatable string.'))
112
113
114 Class-based API
115 ---------------
116
117 The class-based API of the :mod:`!gettext` module gives you more flexibility and
118 greater convenience than the GNU :program:`gettext` API. It is the recommended
119 way of localizing your Python applications and modules. :mod:`!gettext` defines
120 a :class:`GNUTranslations` class which implements the parsing of GNU :file:`.mo` format
121 files, and has methods for returning strings. Instances of this class can also
122 install themselves in the built-in namespace as the function :func:`!_`.
123
124
125 .. function:: find(domain, localedir=None, languages=None, all=False)
126
127 This function implements the standard :file:`.mo` file search algorithm. It
128 takes a *domain*, identical to what :func:`textdomain` takes. Optional
129 *localedir* is as in :func:`bindtextdomain`. Optional *languages* is a list of
130 strings, where each string is a language code.
131
132 If *localedir* is not given, then the default system locale directory is used.
133 [#]_ If *languages* is not given, then the following environment variables are
134 searched: :envvar:`LANGUAGE`, :envvar:`LC_ALL`, :envvar:`LC_MESSAGES`, and
135 :envvar:`LANG`. The first one returning a non-empty value is used for the
136 *languages* variable. The environment variables should contain a colon separated
137 list of languages, which will be split on the colon to produce the expected list
138 of language code strings.
139
140 :func:`find` then expands and normalizes the languages, and then iterates
141 through them, searching for an existing file built of these components:
142
143 :file:`{localedir}/{language}/LC_MESSAGES/{domain}.mo`
144
145 The first such file name that exists is returned by :func:`find`. If no such
146 file is found, then ``None`` is returned. If *all* is given, it returns a list
147 of all file names, in the order in which they appear in the languages list or
148 the environment variables.
149
150
151 .. function:: translation(domain, localedir=None, languages=None, class_=None, fallback=False)
152
153 Return a ``*Translations`` instance based on the *domain*, *localedir*,
154 and *languages*, which are first passed to :func:`find` to get a list of the
155 associated :file:`.mo` file paths. Instances with identical :file:`.mo` file
156 names are cached. The actual class instantiated is *class_* if
157 provided, otherwise :class:`GNUTranslations`. The class's constructor must
158 take a single :term:`file object` argument.
159
160 If multiple files are found, later files are used as fallbacks for earlier ones.
161 To allow setting the fallback, :func:`copy.copy` is used to clone each
162 translation object from the cache; the actual instance data is still shared with
163 the cache.
164
165 If no :file:`.mo` file is found, this function raises :exc:`OSError` if
166 *fallback* is false (which is the default), and returns a
167 :class:`NullTranslations` instance if *fallback* is true.
168
169 .. versionchanged:: 3.3
170 :exc:`IOError` used to be raised, it is now an alias of :exc:`OSError`.
171
172 .. versionchanged:: 3.11
173 *codeset* parameter is removed.
174
175 .. function:: install(domain, localedir=None, *, names=None)
176
177 This installs the function :func:`!_` in Python's builtins namespace, based on
178 *domain* and *localedir* which are passed to the function :func:`translation`.
179
180 For the *names* parameter, please see the description of the translation
181 object's :meth:`~NullTranslations.install` method.
182
183 As seen below, you usually mark the strings in your application that are
184 candidates for translation, by wrapping them in a call to the :func:`!_`
185 function, like this::
186
187 print(_('This string will be translated.'))
188
189 For convenience, you want the :func:`!_` function to be installed in Python's
190 builtins namespace, so it is easily accessible in all modules of your
191 application.
192
193 .. versionchanged:: 3.11
194 *names* is now a keyword-only parameter.
195
196 The :class:`NullTranslations` class
197 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
198
199 Translation classes are what actually implement the translation of original
200 source file message strings to translated message strings. The base class used
201 by all translation classes is :class:`NullTranslations`; this provides the basic
202 interface you can use to write your own specialized translation classes. Here
203 are the methods of :class:`!NullTranslations`:
204
205
206 .. class:: NullTranslations(fp=None)
207
208 Takes an optional :term:`file object` *fp*, which is ignored by the base class.
209 Initializes "protected" instance variables *_info* and *_charset* which are set
210 by derived classes, as well as *_fallback*, which is set through
211 :meth:`add_fallback`. It then calls ``self._parse(fp)`` if *fp* is not
212 ``None``.
213
214 .. method:: _parse(fp)
215
216 No-op in the base class, this method takes file object *fp*, and reads
217 the data from the file, initializing its message catalog. If you have an
218 unsupported message catalog file format, you should override this method
219 to parse your format.
220
221
222 .. method:: add_fallback(fallback)
223
224 Add *fallback* as the fallback object for the current translation object.
225 A translation object should consult the fallback if it cannot provide a
226 translation for a given message.
227
228
229 .. method:: gettext(message)
230
231 If a fallback has been set, forward :meth:`!gettext` to the fallback.
232 Otherwise, return *message*. Overridden in derived classes.
233
234
235 .. method:: ngettext(singular, plural, n)
236
237 If a fallback has been set, forward :meth:`!ngettext` to the fallback.
238 Otherwise, return *singular* if *n* is 1; return *plural* otherwise.
239 Overridden in derived classes.
240
241
242 .. method:: pgettext(context, message)
243
244 If a fallback has been set, forward :meth:`pgettext` to the fallback.
245 Otherwise, return the translated message. Overridden in derived classes.
246
247 .. versionadded:: 3.8
248
249
250 .. method:: npgettext(context, singular, plural, n)
251
252 If a fallback has been set, forward :meth:`npgettext` to the fallback.
253 Otherwise, return the translated message. Overridden in derived classes.
254
255 .. versionadded:: 3.8
256
257
258 .. method:: info()
259
260 Return a dictionary containing
261 the metadata found in the message catalog file.
262
263
264 .. method:: charset()
265
266 Return the encoding of the message catalog file.
267
268
269 .. method:: install(names=None)
270
271 This method installs :meth:`.gettext` into the built-in namespace,
272 binding it to ``_``.
273
274 If the *names* parameter is given, it must be a sequence containing the
275 names of functions you want to install in the builtins namespace in
276 addition to :func:`!_`. Supported names are ``'gettext'``, ``'ngettext'``,
277 ``'pgettext'``, and ``'npgettext'``.
278
279 Note that this is only one way, albeit the most convenient way, to make
280 the :func:`!_` function available to your application. Because it affects
281 the entire application globally, and specifically the built-in namespace,
282 localized modules should never install :func:`!_`. Instead, they should use
283 this code to make :func:`!_` available to their module::
284
285 import gettext
286 t = gettext.translation('mymodule', ...)
287 _ = t.gettext
288
289 This puts :func:`!_` only in the module's global namespace and so only
290 affects calls within this module.
291
292 .. versionchanged:: 3.8
293 Added ``'pgettext'`` and ``'npgettext'``.
294
295
296 The :class:`GNUTranslations` class
297 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
298
299 The :mod:`!gettext` module provides one additional class derived from
300 :class:`NullTranslations`: :class:`GNUTranslations`. This class overrides
301 :meth:`!_parse` to enable reading GNU :program:`gettext` format :file:`.mo` files
302 in both big-endian and little-endian format.
303
304 :class:`GNUTranslations` parses optional metadata out of the translation
305 catalog. It is convention with GNU :program:`gettext` to include metadata as
306 the translation for the empty string. This metadata is in :rfc:`822`\ -style
307 ``key: value`` pairs, and should contain the ``Project-Id-Version`` key. If the
308 key ``Content-Type`` is found, then the ``charset`` property is used to
309 initialize the "protected" :attr:`!_charset` instance variable, defaulting to
310 ``None`` if not found. If the charset encoding is specified, then all message
311 ids and message strings read from the catalog are converted to Unicode using
312 this encoding, else ASCII is assumed.
313
314 Since message ids are read as Unicode strings too, all ``*gettext()`` methods
315 will assume message ids as Unicode strings, not byte strings.
316
317 The entire set of key/value pairs are placed into a dictionary and set as the
318 "protected" :attr:`!_info` instance variable.
319
320 If the :file:`.mo` file's magic number is invalid, the major version number is
321 unexpected, or if other problems occur while reading the file, instantiating a
322 :class:`GNUTranslations` class can raise :exc:`OSError`.
323
324 .. class:: GNUTranslations
325
326 The following methods are overridden from the base class implementation:
327
328 .. method:: gettext(message)
329
330 Look up the *message* id in the catalog and return the corresponding message
331 string, as a Unicode string. If there is no entry in the catalog for the
332 *message* id, and a fallback has been set, the look up is forwarded to the
333 fallback's :meth:`~NullTranslations.gettext` method. Otherwise, the
334 *message* id is returned.
335
336
337 .. method:: ngettext(singular, plural, n)
338
339 Do a plural-forms lookup of a message id. *singular* is used as the message id
340 for purposes of lookup in the catalog, while *n* is used to determine which
341 plural form to use. The returned message string is a Unicode string.
342
343 If the message id is not found in the catalog, and a fallback is specified,
344 the request is forwarded to the fallback's :meth:`~NullTranslations.ngettext`
345 method. Otherwise, when *n* is 1 *singular* is returned, and *plural* is
346 returned in all other cases.
347
348 Here is an example::
349
350 n = len(os.listdir('.'))
351 cat = GNUTranslations(somefile)
352 message = cat.ngettext(
353 'There is %(num)d file in this directory',
354 'There are %(num)d files in this directory',
355 n) % {'num': n}
356
357
358 .. method:: pgettext(context, message)
359
360 Look up the *context* and *message* id in the catalog and return the
361 corresponding message string, as a Unicode string. If there is no
362 entry in the catalog for the *message* id and *context*, and a fallback
363 has been set, the look up is forwarded to the fallback's
364 :meth:`pgettext` method. Otherwise, the *message* id is returned.
365
366 .. versionadded:: 3.8
367
368
369 .. method:: npgettext(context, singular, plural, n)
370
371 Do a plural-forms lookup of a message id. *singular* is used as the
372 message id for purposes of lookup in the catalog, while *n* is used to
373 determine which plural form to use.
374
375 If the message id for *context* is not found in the catalog, and a
376 fallback is specified, the request is forwarded to the fallback's
377 :meth:`npgettext` method. Otherwise, when *n* is 1 *singular* is
378 returned, and *plural* is returned in all other cases.
379
380 .. versionadded:: 3.8
381
382
383 Solaris message catalog support
384 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
385
386 The Solaris operating system defines its own binary :file:`.mo` file format, but
387 since no documentation can be found on this format, it is not supported at this
388 time.
389
390
391 The Catalog constructor
392 ^^^^^^^^^^^^^^^^^^^^^^^
393
394 .. index:: single: GNOME
395
396 GNOME uses a version of the :mod:`!gettext` module by James Henstridge, but this
397 version has a slightly different API. Its documented usage was::
398
399 import gettext
400 cat = gettext.Catalog(domain, localedir)
401 _ = cat.gettext
402 print(_('hello world'))
403
404 For compatibility with this older module, the function :func:`!Catalog` is an
405 alias for the :func:`translation` function described above.
406
407 One difference between this module and Henstridge's: his catalog objects
408 supported access through a mapping API, but this appears to be unused and so is
409 not currently supported.
410
411 .. _i18n-howto:
412
413 Internationalizing your programs and modules
414 --------------------------------------------
415
416 Internationalization (I18N) refers to the operation by which a program is made
417 aware of multiple languages. Localization (L10N) refers to the adaptation of
418 your program, once internationalized, to the local language and cultural habits.
419 In order to provide multilingual messages for your Python programs, you need to
420 take the following steps:
421
422 #. prepare your program or module by specially marking translatable strings
423
424 #. run a suite of tools over your marked files to generate raw messages catalogs
425
426 #. create language-specific translations of the message catalogs
427
428 #. use the :mod:`!gettext` module so that message strings are properly translated
429
430 In order to prepare your code for I18N, you need to look at all the strings in
431 your files. Any string that needs to be translated should be marked by wrapping
432 it in ``_('...')`` --- that is, a call to the function :func:`_ <gettext>`. For example::
433
434 filename = 'mylog.txt'
435 message = _('writing a log message')
436 with open(filename, 'w') as fp:
437 fp.write(message)
438
439 In this example, the string ``'writing a log message'`` is marked as a candidate
440 for translation, while the strings ``'mylog.txt'`` and ``'w'`` are not.
441
442 There are a few tools to extract the strings meant for translation.
443 The original GNU :program:`gettext` only supported C or C++ source
444 code but its extended version :program:`xgettext` scans code written
445 in a number of languages, including Python, to find strings marked as
446 translatable. `Babel <https://babel.pocoo.org/>`__ is a Python
447 internationalization library that includes a :file:`pybabel` script to
448 extract and compile message catalogs. François Pinard's program
449 called :program:`xpot` does a similar job and is available as part of
450 his `po-utils package <https://github.com/pinard/po-utils>`__.
451
452 (Python also includes pure-Python versions of these programs, called
453 :program:`pygettext.py` and :program:`msgfmt.py`; some Python distributions
454 will install them for you. :program:`pygettext.py` is similar to
455 :program:`xgettext`, but only understands Python source code and
456 cannot handle other programming languages such as C or C++.
457 :program:`pygettext.py` supports a command-line interface similar to
458 :program:`xgettext`; for details on its use, run ``pygettext.py
459 --help``. :program:`msgfmt.py` is binary compatible with GNU
460 :program:`msgfmt`. With these two programs, you may not need the GNU
461 :program:`gettext` package to internationalize your Python
462 applications.)
463
464 :program:`xgettext`, :program:`pygettext`, and similar tools generate
465 :file:`.po` files that are message catalogs. They are structured
466 human-readable files that contain every marked string in the source
467 code, along with a placeholder for the translated versions of these
468 strings.
469
470 Copies of these :file:`.po` files are then handed over to the
471 individual human translators who write translations for every
472 supported natural language. They send back the completed
473 language-specific versions as a :file:`<language-name>.po` file that's
474 compiled into a machine-readable :file:`.mo` binary catalog file using
475 the :program:`msgfmt` program. The :file:`.mo` files are used by the
476 :mod:`!gettext` module for the actual translation processing at
477 run-time.
478
479 How you use the :mod:`!gettext` module in your code depends on whether you are
480 internationalizing a single module or your entire application. The next two
481 sections will discuss each case.
482
483
484 Localizing your module
485 ^^^^^^^^^^^^^^^^^^^^^^
486
487 If you are localizing your module, you must take care not to make global
488 changes, e.g. to the built-in namespace. You should not use the GNU :program:`gettext`
489 API but instead the class-based API.
490
491 Let's say your module is called "spam" and the module's various natural language
492 translation :file:`.mo` files reside in :file:`/usr/share/locale` in GNU
493 :program:`gettext` format. Here's what you would put at the top of your
494 module::
495
496 import gettext
497 t = gettext.translation('spam', '/usr/share/locale')
498 _ = t.gettext
499
500
501 Localizing your application
502 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
503
504 If you are localizing your application, you can install the :func:`!_` function
505 globally into the built-in namespace, usually in the main driver file of your
506 application. This will let all your application-specific files just use
507 ``_('...')`` without having to explicitly install it in each file.
508
509 In the simple case then, you need only add the following bit of code to the main
510 driver file of your application::
511
512 import gettext
513 gettext.install('myapplication')
514
515 If you need to set the locale directory, you can pass it into the
516 :func:`install` function::
517
518 import gettext
519 gettext.install('myapplication', '/usr/share/locale')
520
521
522 Changing languages on the fly
523 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
524
525 If your program needs to support many languages at the same time, you may want
526 to create multiple translation instances and then switch between them
527 explicitly, like so::
528
529 import gettext
530
531 lang1 = gettext.translation('myapplication', languages=['en'])
532 lang2 = gettext.translation('myapplication', languages=['fr'])
533 lang3 = gettext.translation('myapplication', languages=['de'])
534
535 # start by using language1
536 lang1.install()
537
538 # ... time goes by, user selects language 2
539 lang2.install()
540
541 # ... more time goes by, user selects language 3
542 lang3.install()
543
544
545 Deferred translations
546 ^^^^^^^^^^^^^^^^^^^^^
547
548 In most coding situations, strings are translated where they are coded.
549 Occasionally however, you need to mark strings for translation, but defer actual
550 translation until later. A classic example is::
551
552 animals = ['mollusk',
553 'albatross',
554 'rat',
555 'penguin',
556 'python', ]
557 # ...
558 for a in animals:
559 print(a)
560
561 Here, you want to mark the strings in the ``animals`` list as being
562 translatable, but you don't actually want to translate them until they are
563 printed.
564
565 Here is one way you can handle this situation::
566
567 def _(message): return message
568
569 animals = [_('mollusk'),
570 _('albatross'),
571 _('rat'),
572 _('penguin'),
573 _('python'), ]
574
575 del _
576
577 # ...
578 for a in animals:
579 print(_(a))
580
581 This works because the dummy definition of :func:`!_` simply returns the string
582 unchanged. And this dummy definition will temporarily override any definition
583 of :func:`!_` in the built-in namespace (until the :keyword:`del` command). Take
584 care, though if you have a previous definition of :func:`!_` in the local
585 namespace.
586
587 Note that the second use of :func:`!_` will not identify "a" as being
588 translatable to the :program:`gettext` program, because the parameter
589 is not a string literal.
590
591 Another way to handle this is with the following example::
592
593 def N_(message): return message
594
595 animals = [N_('mollusk'),
596 N_('albatross'),
597 N_('rat'),
598 N_('penguin'),
599 N_('python'), ]
600
601 # ...
602 for a in animals:
603 print(_(a))
604
605 In this case, you are marking translatable strings with the function
606 :func:`!N_`, which won't conflict with any definition of :func:`!_`.
607 However, you will need to teach your message extraction program to
608 look for translatable strings marked with :func:`!N_`. :program:`xgettext`,
609 :program:`pygettext`, ``pybabel extract``, and :program:`xpot` all
610 support this through the use of the :option:`!-k` command-line switch.
611 The choice of :func:`!N_` here is totally arbitrary; it could have just
612 as easily been :func:`!MarkThisStringForTranslation`.
613
614
615 Acknowledgements
616 ----------------
617
618 The following people contributed code, feedback, design suggestions, previous
619 implementations, and valuable experience to the creation of this module:
620
621 * Peter Funk
622
623 * James Henstridge
624
625 * Juan David Ibáñez Palomar
626
627 * Marc-André Lemburg
628
629 * Martin von Löwis
630
631 * François Pinard
632
633 * Barry Warsaw
634
635 * Gustavo Niemeyer
636
637 .. rubric:: Footnotes
638
639 .. [#] The default locale directory is system dependent; for example, on Red Hat Linux
640 it is :file:`/usr/share/locale`, but on Solaris it is :file:`/usr/lib/locale`.
641 The :mod:`!gettext` module does not try to support these system dependent
642 defaults; instead its default is :file:`{sys.base_prefix}/share/locale` (see
643 :data:`sys.base_prefix`). For this reason, it is always best to call
644 :func:`bindtextdomain` with an explicit absolute path at the start of your
645 application.
646
647 .. [#] See the footnote for :func:`bindtextdomain` above.