| 1 |
:mod:`!unicodedata` --- Unicode Database |
| 2 |
======================================== |
| 3 |
|
| 4 |
.. module:: unicodedata |
| 5 |
:synopsis: Access the Unicode Database. |
| 6 |
|
| 7 |
.. moduleauthor:: Marc-André Lemburg <[email protected]> |
| 8 |
.. sectionauthor:: Marc-André Lemburg <[email protected]> |
| 9 |
.. sectionauthor:: Martin v. Löwis <[email protected]> |
| 10 |
|
| 11 |
.. index:: |
| 12 |
single: Unicode |
| 13 |
single: character |
| 14 |
pair: Unicode; database |
| 15 |
|
| 16 |
-------------- |
| 17 |
|
| 18 |
This module provides access to the Unicode Character Database (UCD) which |
| 19 |
defines character properties for all Unicode characters. The data contained in |
| 20 |
this database is compiled from the `UCD version 16.0.0 |
| 21 |
<https://www.unicode.org/Public/16.0.0/ucd>`_. |
| 22 |
|
| 23 |
The module uses the same names and symbols as defined by Unicode |
| 24 |
Standard Annex #44, `"Unicode Character Database" |
| 25 |
<https://www.unicode.org/reports/tr44/>`_. It defines the |
| 26 |
following functions: |
| 27 |
|
| 28 |
.. seealso:: |
| 29 |
|
| 30 |
The :ref:`unicode-howto` for more information about Unicode and how to use |
| 31 |
this module. |
| 32 |
|
| 33 |
|
| 34 |
.. function:: lookup(name) |
| 35 |
|
| 36 |
Look up character by name. If a character with the given name is found, return |
| 37 |
the corresponding character. If not found, :exc:`KeyError` is raised. |
| 38 |
For example:: |
| 39 |
|
| 40 |
>>> unicodedata.lookup('LEFT CURLY BRACKET') |
| 41 |
'{' |
| 42 |
|
| 43 |
The characters returned by this function are the same as those produced by |
| 44 |
``\N`` escape sequence in string literals. For example:: |
| 45 |
|
| 46 |
>>> unicodedata.lookup('MIDDLE DOT') == '\N{MIDDLE DOT}' |
| 47 |
True |
| 48 |
|
| 49 |
.. versionchanged:: 3.3 |
| 50 |
Support for name aliases [#]_ and named sequences [#]_ has been added. |
| 51 |
|
| 52 |
|
| 53 |
.. function:: name(chr, default=None, /) |
| 54 |
|
| 55 |
Returns the name assigned to the character *chr* as a string. If no |
| 56 |
name is defined, *default* is returned, or, if not given, :exc:`ValueError` is |
| 57 |
raised. For example:: |
| 58 |
|
| 59 |
>>> unicodedata.name('½') |
| 60 |
'VULGAR FRACTION ONE HALF' |
| 61 |
>>> unicodedata.name('\uFFFF', 'fallback') |
| 62 |
'fallback' |
| 63 |
|
| 64 |
|
| 65 |
.. function:: decimal(chr, default=None, /) |
| 66 |
|
| 67 |
Returns the decimal value assigned to the character *chr* as integer. |
| 68 |
If no such value is defined, *default* is returned, or, if not given, |
| 69 |
:exc:`ValueError` is raised. For example:: |
| 70 |
|
| 71 |
>>> unicodedata.decimal('\N{ARABIC-INDIC DIGIT NINE}') |
| 72 |
9 |
| 73 |
>>> unicodedata.decimal('\N{SUPERSCRIPT NINE}', -1) |
| 74 |
-1 |
| 75 |
|
| 76 |
|
| 77 |
.. function:: digit(chr, default=None, /) |
| 78 |
|
| 79 |
Returns the digit value assigned to the character *chr* as integer. |
| 80 |
If no such value is defined, *default* is returned, or, if not given, |
| 81 |
:exc:`ValueError` is raised:: |
| 82 |
|
| 83 |
>>> unicodedata.digit('\N{SUPERSCRIPT NINE}') |
| 84 |
9 |
| 85 |
|
| 86 |
|
| 87 |
.. function:: numeric(chr, default=None, /) |
| 88 |
|
| 89 |
Returns the numeric value assigned to the character *chr* as float. |
| 90 |
If no such value is defined, *default* is returned, or, if not given, |
| 91 |
:exc:`ValueError` is raised:: |
| 92 |
|
| 93 |
>>> unicodedata.numeric('½') |
| 94 |
0.5 |
| 95 |
|
| 96 |
|
| 97 |
.. function:: category(chr) |
| 98 |
|
| 99 |
Returns the general category assigned to the character *chr* as |
| 100 |
string. General category names consist of two letters. |
| 101 |
See the `General Category Values section of the Unicode Character |
| 102 |
Database documentation <https://www.unicode.org/reports/tr44/tr44-34.html#General_Category_Values>`_ |
| 103 |
for a list of category codes. For example:: |
| 104 |
|
| 105 |
>>> unicodedata.category('A') # 'L'etter, 'u'ppercase |
| 106 |
'Lu' |
| 107 |
|
| 108 |
|
| 109 |
.. function:: bidirectional(chr) |
| 110 |
|
| 111 |
Returns the bidirectional class assigned to the character *chr* as |
| 112 |
string. If no such value is defined, an empty string is returned. |
| 113 |
See the `Bidirectional Class Values section of the Unicode Character |
| 114 |
Database <https://www.unicode.org/reports/tr44/tr44-34.html#Bidi_Class_Values>`_ |
| 115 |
documentation for a list of bidirectional codes. For example:: |
| 116 |
|
| 117 |
>>> unicodedata.bidirectional('\N{ARABIC-INDIC DIGIT SEVEN}') # 'A'rabic, 'N'umber |
| 118 |
'AN' |
| 119 |
|
| 120 |
|
| 121 |
.. function:: combining(chr) |
| 122 |
|
| 123 |
Returns the canonical combining class assigned to the character *chr* |
| 124 |
as integer. Returns ``0`` if no combining class is defined. |
| 125 |
See the `Canonical Combining Class Values section of the Unicode Character |
| 126 |
Database <https://www.unicode.org/reports/tr44/tr44-34.html#Canonical_Combining_Class_Values>`_ |
| 127 |
for more information. |
| 128 |
|
| 129 |
|
| 130 |
.. function:: east_asian_width(chr) |
| 131 |
|
| 132 |
Returns the east asian width assigned to the character *chr* as |
| 133 |
string. For a list of widths and or more information, see the |
| 134 |
`Unicode Standard Annex #11 <https://www.unicode.org/reports/tr11/tr11-43.html>`_. |
| 135 |
|
| 136 |
|
| 137 |
.. function:: mirrored(chr) |
| 138 |
|
| 139 |
Returns the mirrored property assigned to the character *chr* as |
| 140 |
integer. Returns ``1`` if the character has been identified as a "mirrored" |
| 141 |
character in bidirectional text, ``0`` otherwise. For example:: |
| 142 |
|
| 143 |
>>> unicodedata.mirrored('>') |
| 144 |
1 |
| 145 |
|
| 146 |
|
| 147 |
.. function:: decomposition(chr) |
| 148 |
|
| 149 |
Returns the character decomposition mapping assigned to the character |
| 150 |
*chr* as string. An empty string is returned in case no such mapping is |
| 151 |
defined. For example:: |
| 152 |
|
| 153 |
>>> unicodedata.decomposition('Ã') |
| 154 |
'0041 0303' |
| 155 |
|
| 156 |
|
| 157 |
.. function:: normalize(form, unistr) |
| 158 |
|
| 159 |
Return the normal form *form* for the Unicode string *unistr*. Valid values for |
| 160 |
*form* are 'NFC', 'NFKC', 'NFD', and 'NFKD'. |
| 161 |
|
| 162 |
The Unicode standard defines various normalization forms of a Unicode string, |
| 163 |
based on the definition of canonical equivalence and compatibility equivalence. |
| 164 |
In Unicode, several characters can be expressed in various way. For example, the |
| 165 |
character U+00C7 (LATIN CAPITAL LETTER C WITH CEDILLA) can also be expressed as |
| 166 |
the sequence U+0043 (LATIN CAPITAL LETTER C) U+0327 (COMBINING CEDILLA). |
| 167 |
|
| 168 |
For each character, there are two normal forms: normal form C and normal form D. |
| 169 |
Normal form D (NFD) is also known as canonical decomposition, and translates |
| 170 |
each character into its decomposed form. Normal form C (NFC) first applies a |
| 171 |
canonical decomposition, then composes pre-combined characters again. |
| 172 |
|
| 173 |
In addition to these two forms, there are two additional normal forms based on |
| 174 |
compatibility equivalence. In Unicode, certain characters are supported which |
| 175 |
normally would be unified with other characters. For example, U+2160 (ROMAN |
| 176 |
NUMERAL ONE) is really the same thing as U+0049 (LATIN CAPITAL LETTER I). |
| 177 |
However, it is supported in Unicode for compatibility with existing character |
| 178 |
sets (for example, gb2312). |
| 179 |
|
| 180 |
The normal form KD (NFKD) will apply the compatibility decomposition, that is, |
| 181 |
replace all compatibility characters with their equivalents. The normal form KC |
| 182 |
(NFKC) first applies the compatibility decomposition, followed by the canonical |
| 183 |
composition. |
| 184 |
|
| 185 |
Even if two unicode strings are normalized and look the same to |
| 186 |
a human reader, if one has combining characters and the other |
| 187 |
doesn't, they may not compare equal. |
| 188 |
|
| 189 |
|
| 190 |
.. function:: is_normalized(form, unistr) |
| 191 |
|
| 192 |
Return whether the Unicode string *unistr* is in the normal form *form*. Valid |
| 193 |
values for *form* are 'NFC', 'NFKC', 'NFD', and 'NFKD'. |
| 194 |
|
| 195 |
.. versionadded:: 3.8 |
| 196 |
|
| 197 |
|
| 198 |
In addition, the module exposes the following constant: |
| 199 |
|
| 200 |
.. data:: unidata_version |
| 201 |
|
| 202 |
The version of the Unicode database used in this module. |
| 203 |
|
| 204 |
|
| 205 |
.. data:: ucd_3_2_0 |
| 206 |
|
| 207 |
This is an object that has the same methods as the entire module, but uses the |
| 208 |
Unicode database version 3.2 instead, for applications that require this |
| 209 |
specific version of the Unicode database (such as IDNA). |
| 210 |
|
| 211 |
|
| 212 |
.. rubric:: Footnotes |
| 213 |
|
| 214 |
.. [#] https://www.unicode.org/Public/16.0.0/ucd/NameAliases.txt |
| 215 |
|
| 216 |
.. [#] https://www.unicode.org/Public/16.0.0/ucd/NamedSequences.txt |