| 1 |
.\" -*- mode: troff; coding: utf-8 -*- |
| 2 |
.\" Automatically generated by Pod::Man 5.0102 (Pod::Simple 3.45) |
| 3 |
.\" |
| 4 |
.\" Standard preamble: |
| 5 |
.\" ======================================================================== |
| 6 |
.de Sp \" Vertical space (when we can't use .PP) |
| 7 |
.if t .sp .5v |
| 8 |
.if n .sp |
| 9 |
.. |
| 10 |
.de Vb \" Begin verbatim text |
| 11 |
.ft CW |
| 12 |
.nf |
| 13 |
.ne \\$1 |
| 14 |
.. |
| 15 |
.de Ve \" End verbatim text |
| 16 |
.ft R |
| 17 |
.fi |
| 18 |
.. |
| 19 |
.\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>. |
| 20 |
.ie n \{\ |
| 21 |
. ds C` "" |
| 22 |
. ds C' "" |
| 23 |
'br\} |
| 24 |
.el\{\ |
| 25 |
. ds C` |
| 26 |
. ds C' |
| 27 |
'br\} |
| 28 |
.\" |
| 29 |
.\" Escape single quotes in literal strings from groff's Unicode transform. |
| 30 |
.ie \n(.g .ds Aq \(aq |
| 31 |
.el .ds Aq ' |
| 32 |
.\" |
| 33 |
.\" If the F register is >0, we'll generate index entries on stderr for |
| 34 |
.\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index |
| 35 |
.\" entries marked with X<> in POD. Of course, you'll have to process the |
| 36 |
.\" output yourself in some meaningful fashion. |
| 37 |
.\" |
| 38 |
.\" Avoid warning from groff about undefined register 'F'. |
| 39 |
.de IX |
| 40 |
.. |
| 41 |
.nr rF 0 |
| 42 |
.if \n(.g .if rF .nr rF 1 |
| 43 |
.if (\n(rF:(\n(.g==0)) \{\ |
| 44 |
. if \nF \{\ |
| 45 |
. de IX |
| 46 |
. tm Index:\\$1\t\\n%\t"\\$2" |
| 47 |
.. |
| 48 |
. if !\nF==2 \{\ |
| 49 |
. nr % 0 |
| 50 |
. nr F 2 |
| 51 |
. \} |
| 52 |
. \} |
| 53 |
.\} |
| 54 |
.rr rF |
| 55 |
.\" ======================================================================== |
| 56 |
.\" |
| 57 |
.IX Title "Encode::Supported 3perl" |
| 58 |
.TH Encode::Supported 3perl 2025-11-16 "perl v5.40.1" "Perl Programmers Reference Guide" |
| 59 |
.\" For nroff, turn off justification. Always turn off hyphenation; it makes |
| 60 |
.\" way too many mistakes in technical documents. |
| 61 |
.if n .ad l |
| 62 |
.nh |
| 63 |
.SH NAME |
| 64 |
Encode::Supported \-\- Encodings supported by Encode |
| 65 |
.SH DESCRIPTION |
| 66 |
.IX Header "DESCRIPTION" |
| 67 |
.SS "Encoding Names" |
| 68 |
.IX Subsection "Encoding Names" |
| 69 |
Encoding names are case insensitive. White space in names |
| 70 |
is ignored. In addition, an encoding may have aliases. |
| 71 |
Each encoding has one "canonical" name. The "canonical" |
| 72 |
name is chosen from the names of the encoding by picking |
| 73 |
the first in the following sequence (with a few exceptions). |
| 74 |
.IP \(bu 2 |
| 75 |
The name used by the Perl community. That includes 'utf8' and 'ascii'. |
| 76 |
Unlike aliases, canonical names directly reach the method so such |
| 77 |
frequently used words like 'utf8' don't need to do alias lookups. |
| 78 |
.IP \(bu 2 |
| 79 |
The MIME name as defined in IETF RFCs. This includes all "iso\-"s. |
| 80 |
.IP \(bu 2 |
| 81 |
The name in the IANA registry. |
| 82 |
.IP \(bu 2 |
| 83 |
The name used by the organization that defined it. |
| 84 |
.PP |
| 85 |
In case \fIde jure\fR canonical names differ from that of the Encode |
| 86 |
module, they are always aliased if it ever be implemented. So you can |
| 87 |
safely tell if a given encoding is implemented or not just by passing |
| 88 |
the canonical name. |
| 89 |
.PP |
| 90 |
Because of all the alias issues, and because in the general case |
| 91 |
encodings have state, "Encode" uses an encoding object internally |
| 92 |
once an operation is in progress. |
| 93 |
.SH "Supported Encodings" |
| 94 |
.IX Header "Supported Encodings" |
| 95 |
As of Perl 5.8.0, at least the following encodings are recognized. |
| 96 |
Note that unless otherwise specified, they are all case insensitive |
| 97 |
(via alias) and all occurrence of spaces are replaced with '\-'. |
| 98 |
In other words, "ISO 8859 1" and "iso\-8859\-1" are identical. |
| 99 |
.PP |
| 100 |
Encodings are categorized and implemented in several different modules |
| 101 |
but you don't have to \f(CW\*(C`use Encode::XX\*(C'\fR to make them available for |
| 102 |
most cases. Encode.pm will automatically load those modules on demand. |
| 103 |
.SS "Built-in Encodings" |
| 104 |
.IX Subsection "Built-in Encodings" |
| 105 |
The following encodings are always available. |
| 106 |
.PP |
| 107 |
.Vb 8 |
| 108 |
\& Canonical Aliases Comments & References |
| 109 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 110 |
\& ascii US\-ascii ISO\-646\-US [ECMA] |
| 111 |
\& ascii\-ctrl Special Encoding |
| 112 |
\& iso\-8859\-1 latin1 [ISO] |
| 113 |
\& null Special Encoding |
| 114 |
\& utf8 UTF\-8 [RFC2279] |
| 115 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 116 |
.Ve |
| 117 |
.PP |
| 118 |
\&\fInull\fR and \fIascii-ctrl\fR are special. "null" fails for all character |
| 119 |
so when you set fallback mode to PERLQQ, HTMLCREF or XMLCREF, ALL |
| 120 |
CHARACTERS will fall back to character references. Ditto for |
| 121 |
"ascii-ctrl" except for control characters. For fallback modes, see |
| 122 |
Encode. |
| 123 |
.SS "Encode::Unicode \-\- other Unicode encodings" |
| 124 |
.IX Subsection "Encode::Unicode -- other Unicode encodings" |
| 125 |
Unicode coding schemes other than native utf8 are supported by |
| 126 |
Encode::Unicode, which will be autoloaded on demand. |
| 127 |
.PP |
| 128 |
.Vb 11 |
| 129 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 130 |
\& UCS\-2BE UCS\-2, iso\-10646\-1 [IANA, UC] |
| 131 |
\& UCS\-2LE [UC] |
| 132 |
\& UTF\-16 [UC] |
| 133 |
\& UTF\-16BE [UC] |
| 134 |
\& UTF\-16LE [UC] |
| 135 |
\& UTF\-32 [UC] |
| 136 |
\& UTF\-32BE UCS\-4 [UC] |
| 137 |
\& UTF\-32LE [UC] |
| 138 |
\& UTF\-7 [RFC2152] |
| 139 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 140 |
.Ve |
| 141 |
.PP |
| 142 |
To find how (UCS\-2|UTF\-(16|32))(LE|BE)? differ from one another, |
| 143 |
see Encode::Unicode. |
| 144 |
.PP |
| 145 |
UTF\-7 is a special encoding which "re-encodes" UTF\-16BE into a 7\-bit |
| 146 |
encoding. It is implemented separately by Encode::Unicode::UTF7. |
| 147 |
.SS "Encode::Byte \-\- Extended ASCII" |
| 148 |
.IX Subsection "Encode::Byte -- Extended ASCII" |
| 149 |
Encode::Byte implements most single-byte encodings except for |
| 150 |
Symbols and EBCDIC. The following encodings are based on single-byte |
| 151 |
encodings implemented as extended ASCII. Most of them map |
| 152 |
\&\ex80\-\exff (upper half) to non-ASCII characters. |
| 153 |
.IP "ISO\-8859 and corresponding vendor mappings" 2 |
| 154 |
.IX Item "ISO-8859 and corresponding vendor mappings" |
| 155 |
Since there are so many, they are presented in table format with |
| 156 |
languages and corresponding encoding names by vendors. Note that |
| 157 |
the table is sorted in order of ISO\-8859 and the corresponding vendor |
| 158 |
mappings are slightly different from that of ISO. See |
| 159 |
<http://czyborra.com/charsets/iso8859.html> for details. |
| 160 |
.Sp |
| 161 |
.Vb 10 |
| 162 |
\& Lang/Regions ISO/Other Std. DOS Windows Macintosh Others |
| 163 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 164 |
\& N. America (ASCII) cp437 AdobeStandardEncoding |
| 165 |
\& cp863 (DOSCanadaF) |
| 166 |
\& W. Europe iso\-8859\-1 cp850 cp1252 MacRoman nextstep |
| 167 |
\& hp\-roman8 |
| 168 |
\& cp860 (DOSPortuguese) |
| 169 |
\& Cntrl. Europe iso\-8859\-2 cp852 cp1250 MacCentralEurRoman |
| 170 |
\& MacCroatian |
| 171 |
\& MacRomanian |
| 172 |
\& MacRumanian |
| 173 |
\& Latin3[1] iso\-8859\-3 |
| 174 |
\& Latin4[2] iso\-8859\-4 |
| 175 |
\& Cyrillics iso\-8859\-5 cp855 cp1251 MacCyrillic |
| 176 |
\& (See also next section) cp866 MacUkrainian |
| 177 |
\& Arabic iso\-8859\-6 cp864 cp1256 MacArabic |
| 178 |
\& cp1006 MacFarsi |
| 179 |
\& Greek iso\-8859\-7 cp737 cp1253 MacGreek |
| 180 |
\& cp869 (DOSGreek2) |
| 181 |
\& Hebrew iso\-8859\-8 cp862 cp1255 MacHebrew |
| 182 |
\& Turkish iso\-8859\-9 cp857 cp1254 MacTurkish |
| 183 |
\& Nordics iso\-8859\-10 cp865 |
| 184 |
\& cp861 MacIcelandic |
| 185 |
\& MacSami |
| 186 |
\& Thai iso\-8859\-11[3] cp874 MacThai |
| 187 |
\& (iso\-8859\-12 is nonexistent. Reserved for Indics?) |
| 188 |
\& Baltics iso\-8859\-13 cp775 cp1257 |
| 189 |
\& Celtics iso\-8859\-14 |
| 190 |
\& Latin9 [4] iso\-8859\-15 |
| 191 |
\& Latin10 iso\-8859\-16 |
| 192 |
\& Vietnamese viscii cp1258 MacVietnamese |
| 193 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 194 |
\& |
| 195 |
\& [1] Esperanto, Maltese, and Turkish. Turkish is now on 8859\-9. |
| 196 |
\& [2] Baltics. Now on 8859\-10, except for Latvian. |
| 197 |
\& [3] TIS 620 + Non\-Breaking Space (0xA0 / U+00A0) |
| 198 |
\& [4] Nicknamed Latin0; the Euro sign as well as French and Finnish |
| 199 |
\& letters that are missing from 8859\-1 were added. |
| 200 |
.Ve |
| 201 |
.Sp |
| 202 |
All cp* are also available as ibm\-*, ms\-*, and windows\-* . See also |
| 203 |
<http://czyborra.com/charsets/codepages.html>. |
| 204 |
.Sp |
| 205 |
Macintosh encodings don't seem to be registered in such entities as |
| 206 |
IANA. "Canonical" names in Encode are based upon Apple's Tech Note |
| 207 |
1150. See <http://developer.apple.com/technotes/tn/tn1150.html> |
| 208 |
for details. |
| 209 |
.IP "KOI8 \- De Facto Standard for the Cyrillic world" 2 |
| 210 |
.IX Item "KOI8 - De Facto Standard for the Cyrillic world" |
| 211 |
Though ISO\-8859 does have ISO\-8859\-5, the KOI8 series is far more |
| 212 |
popular in the Net. Encode comes with the following KOI charsets. |
| 213 |
For gory details, see <http://czyborra.com/charsets/cyrillic.html> |
| 214 |
.Sp |
| 215 |
.Vb 5 |
| 216 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 217 |
\& koi8\-f |
| 218 |
\& koi8\-r cp878 [RFC1489] |
| 219 |
\& koi8\-u [RFC2319] |
| 220 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 221 |
.Ve |
| 222 |
.SS "gsm0338 \- Hentai Latin 1" |
| 223 |
.IX Subsection "gsm0338 - Hentai Latin 1" |
| 224 |
GSM0338 is for GSM handsets. Though it shares alphanumerals with |
| 225 |
ASCII, control character ranges and other parts are mapped very |
| 226 |
differently, mainly to store Greek characters. There are also escape |
| 227 |
sequences (starting with 0x1B) to cover e.g. the Euro sign. |
| 228 |
.PP |
| 229 |
This was once handled by Encode::Bytes but because of all those |
| 230 |
unusual specifications, Encode 2.20 has relocated the support to |
| 231 |
Encode::GSM0338. See Encode::GSM0338 for details. |
| 232 |
.IP "gsm0338 support before 2.19" 2 |
| 233 |
.IX Item "gsm0338 support before 2.19" |
| 234 |
Some special cases like a trailing 0x00 byte or a lone 0x1B byte are not |
| 235 |
well-defined and \fBdecode()\fR will return an empty string for them. |
| 236 |
One possible workaround is |
| 237 |
.Sp |
| 238 |
.Vb 3 |
| 239 |
\& $gsm =~ s/\ex00\ez/\ex00\ex00/; |
| 240 |
\& $uni = decode("gsm0338", $gsm); |
| 241 |
\& $uni .= "\exA0" if $gsm =~ /\ex1B\ez/; |
| 242 |
.Ve |
| 243 |
.Sp |
| 244 |
Note that the Encode implementation of GSM0338 does not implement the |
| 245 |
reuse of Latin capital letters as Greek capital letters (for example, |
| 246 |
the 0x5A is U+005A (LATIN CAPITAL LETTER Z), not U+0396 (GREEK CAPITAL |
| 247 |
LETTER ZETA). |
| 248 |
.Sp |
| 249 |
The GSM0338 is also covered in Encode::Byte even though it is not |
| 250 |
an "extended ASCII" encoding. |
| 251 |
.SS "CJK: Chinese, Japanese, Korean (Multibyte)" |
| 252 |
.IX Subsection "CJK: Chinese, Japanese, Korean (Multibyte)" |
| 253 |
Note that Vietnamese is listed above. Also read "Encoding vs Charset" |
| 254 |
below. Also note that these are implemented in distinct modules by |
| 255 |
countries, due to the size concerns (simplified Chinese is mapped |
| 256 |
to 'CN', continental China, while traditional Chinese is mapped to |
| 257 |
\&'TW', Taiwan). Please refer to their respective documentation pages. |
| 258 |
.IP "Encode::CN \-\- Continental China" 2 |
| 259 |
.IX Item "Encode::CN -- Continental China" |
| 260 |
.Vb 9 |
| 261 |
\& Standard DOS/Win Macintosh Comment/Reference |
| 262 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 263 |
\& euc\-cn [1] MacChineseSimp |
| 264 |
\& (gbk) cp936 [2] |
| 265 |
\& gb12345\-raw { GB12345 without CES } |
| 266 |
\& gb2312\-raw { GB2312 without CES } |
| 267 |
\& hz |
| 268 |
\& iso\-ir\-165 |
| 269 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 270 |
\& |
| 271 |
\& [1] GB2312 is aliased to this. See L<Microsoft\-related naming mess> |
| 272 |
\& [2] gbk is aliased to this. See L<Microsoft\-related naming mess> |
| 273 |
.Ve |
| 274 |
.IP "Encode::JP \-\- Japan" 2 |
| 275 |
.IX Item "Encode::JP -- Japan" |
| 276 |
.Vb 11 |
| 277 |
\& Standard DOS/Win Macintosh Comment/Reference |
| 278 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 279 |
\& euc\-jp |
| 280 |
\& shiftjis cp932 macJapanese |
| 281 |
\& 7bit\-jis |
| 282 |
\& iso\-2022\-jp [RFC1468] |
| 283 |
\& iso\-2022\-jp\-1 [RFC2237] |
| 284 |
\& jis0201\-raw { JIS X 0201 (roman + halfwidth kana) without CES } |
| 285 |
\& jis0208\-raw { JIS X 0208 (Kanji + fullwidth kana) without CES } |
| 286 |
\& jis0212\-raw { JIS X 0212 (Extended Kanji) without CES } |
| 287 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 288 |
.Ve |
| 289 |
.IP "Encode::KR \-\- Korea" 2 |
| 290 |
.IX Item "Encode::KR -- Korea" |
| 291 |
.Vb 8 |
| 292 |
\& Standard DOS/Win Macintosh Comment/Reference |
| 293 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 294 |
\& euc\-kr MacKorean [RFC1557] |
| 295 |
\& cp949 [1] |
| 296 |
\& iso\-2022\-kr [RFC1557] |
| 297 |
\& johab [KS X 1001:1998, Annex 3] |
| 298 |
\& ksc5601\-raw { KSC5601 without CES } |
| 299 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 300 |
\& |
| 301 |
\& [1] ks_c_5601\-1987, (x\-)?windows\-949, and uhc are aliased to this. |
| 302 |
\& See below. |
| 303 |
.Ve |
| 304 |
.IP "Encode::TW \-\- Taiwan" 2 |
| 305 |
.IX Item "Encode::TW -- Taiwan" |
| 306 |
.Vb 5 |
| 307 |
\& Standard DOS/Win Macintosh Comment/Reference |
| 308 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 309 |
\& big5\-eten cp950 MacChineseTrad {big5 aliased to big5\-eten} |
| 310 |
\& big5\-hkscs |
| 311 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 312 |
.Ve |
| 313 |
.IP "Encode::HanExtra \-\- More Chinese via CPAN" 2 |
| 314 |
.IX Item "Encode::HanExtra -- More Chinese via CPAN" |
| 315 |
Due to the size concerns, additional Chinese encodings below are |
| 316 |
distributed separately on CPAN, under the name Encode::HanExtra. |
| 317 |
.Sp |
| 318 |
.Vb 8 |
| 319 |
\& Standard DOS/Win Macintosh Comment/Reference |
| 320 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 321 |
\& big5ext CMEX\*(Aqs Big5e Extension |
| 322 |
\& big5plus CMEX\*(Aqs Big5+ Extension |
| 323 |
\& cccii Chinese Character Code for Information Interchange |
| 324 |
\& euc\-tw EUC (Extended Unix Character) |
| 325 |
\& gb18030 GBK with Traditional Characters |
| 326 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 327 |
.Ve |
| 328 |
.IP "Encode::JIS2K \-\- JIS X 0213 encodings via CPAN" 2 |
| 329 |
.IX Item "Encode::JIS2K -- JIS X 0213 encodings via CPAN" |
| 330 |
Due to size concerns, additional Japanese encodings below are |
| 331 |
distributed separately on CPAN, under the name Encode::JIS2K. |
| 332 |
.Sp |
| 333 |
.Vb 8 |
| 334 |
\& Standard DOS/Win Macintosh Comment/Reference |
| 335 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 336 |
\& euc\-jisx0213 |
| 337 |
\& shiftjisx0123 |
| 338 |
\& iso\-2022\-jp\-3 |
| 339 |
\& jis0213\-1\-raw |
| 340 |
\& jis0213\-2\-raw |
| 341 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 342 |
.Ve |
| 343 |
.SS "Miscellaneous encodings" |
| 344 |
.IX Subsection "Miscellaneous encodings" |
| 345 |
.IP Encode::EBCDIC 2 |
| 346 |
.IX Item "Encode::EBCDIC" |
| 347 |
See perlebcdic for details. |
| 348 |
.Sp |
| 349 |
.Vb 8 |
| 350 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 351 |
\& cp37 |
| 352 |
\& cp500 |
| 353 |
\& cp875 |
| 354 |
\& cp1026 |
| 355 |
\& cp1047 |
| 356 |
\& posix\-bc |
| 357 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 358 |
.Ve |
| 359 |
.IP Encode::Symbol 2 |
| 360 |
.IX Item "Encode::Symbol" |
| 361 |
For symbols and dingbats. |
| 362 |
.Sp |
| 363 |
.Vb 7 |
| 364 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 365 |
\& symbol |
| 366 |
\& dingbats |
| 367 |
\& MacDingbats |
| 368 |
\& AdobeZdingbat |
| 369 |
\& AdobeSymbol |
| 370 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 371 |
.Ve |
| 372 |
.IP Encode::MIME::Header 2 |
| 373 |
.IX Item "Encode::MIME::Header" |
| 374 |
Strictly speaking, MIME header encoding documented in RFC 2047 is more |
| 375 |
of encapsulation than encoding. However, their support in modern |
| 376 |
world is imperative so they are supported. |
| 377 |
.Sp |
| 378 |
.Vb 5 |
| 379 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 380 |
\& MIME\-Header [RFC2047] |
| 381 |
\& MIME\-B [RFC2047] |
| 382 |
\& MIME\-Q [RFC2047] |
| 383 |
\& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- |
| 384 |
.Ve |
| 385 |
.IP Encode::Guess 2 |
| 386 |
.IX Item "Encode::Guess" |
| 387 |
This one is not a name of encoding but a utility that lets you pick up |
| 388 |
the most appropriate encoding for a data out of given \fIsuspects\fR. See |
| 389 |
Encode::Guess for details. |
| 390 |
.SH "Unsupported encodings" |
| 391 |
.IX Header "Unsupported encodings" |
| 392 |
The following encodings are not supported as yet; some because they |
| 393 |
are rarely used, some because of technical difficulties. They may |
| 394 |
be supported by external modules via CPAN in the future, however. |
| 395 |
.IP "ISO\-2022\-JP\-2 [RFC1554]" 2 |
| 396 |
.IX Item "ISO-2022-JP-2 [RFC1554]" |
| 397 |
Not very popular yet. Needs Unicode Database or equivalent to |
| 398 |
implement \fBencode()\fR (because it includes JIS X 0208/0212, KSC5601, and |
| 399 |
GB2312 simultaneously, whose code points in Unicode overlap. So you |
| 400 |
need to lookup the database to determine to what character set a given |
| 401 |
Unicode character should belong). |
| 402 |
.IP "ISO\-2022\-CN [RFC1922]" 2 |
| 403 |
.IX Item "ISO-2022-CN [RFC1922]" |
| 404 |
Not very popular. Needs CNS 11643\-1 and \-2 which are not available in |
| 405 |
this module. CNS 11643 is supported (via euc-tw) in Encode::HanExtra. |
| 406 |
Audrey Tang may add support for this encoding in her module in future. |
| 407 |
.IP "Various HP-UX encodings" 2 |
| 408 |
.IX Item "Various HP-UX encodings" |
| 409 |
The following are unsupported due to the lack of mapping data. |
| 410 |
.Sp |
| 411 |
.Vb 2 |
| 412 |
\& \*(Aq8\*(Aq \- arabic8, greek8, hebrew8, kana8, thai8, and turkish8 |
| 413 |
\& \*(Aq15\*(Aq \- japanese15, korean15, and roi15 |
| 414 |
.Ve |
| 415 |
.IP "Cyrillic encoding ISO\-IR\-111" 2 |
| 416 |
.IX Item "Cyrillic encoding ISO-IR-111" |
| 417 |
Anton Tagunov doubts its usefulness. |
| 418 |
.IP "ISO\-8859\-8\-1 [Hebrew]" 2 |
| 419 |
.IX Item "ISO-8859-8-1 [Hebrew]" |
| 420 |
None of the Encode team knows Hebrew enough (ISO\-8859\-8, cp1255 and |
| 421 |
MacHebrew are supported because and just because there were mappings |
| 422 |
available at <http://www.unicode.org/>). Contributions welcome. |
| 423 |
.IP "ISIRI 3342, Iran System, ISIRI 2900 [Farsi]" 2 |
| 424 |
.IX Item "ISIRI 3342, Iran System, ISIRI 2900 [Farsi]" |
| 425 |
Ditto. |
| 426 |
.IP "Thai encoding TCVN" 2 |
| 427 |
.IX Item "Thai encoding TCVN" |
| 428 |
Ditto. |
| 429 |
.IP "Vietnamese encodings VPS" 2 |
| 430 |
.IX Item "Vietnamese encodings VPS" |
| 431 |
Though Jungshik Shin has reported that Mozilla supports this encoding, |
| 432 |
it was too late before 5.8.0 for us to add it. In the future, it |
| 433 |
may be available via a separate module. See |
| 434 |
<http://lxr.mozilla.org/seamonkey/source/intl/uconv/ucvlatin/vps.uf> |
| 435 |
and |
| 436 |
<http://lxr.mozilla.org/seamonkey/source/intl/uconv/ucvlatin/vps.ut> |
| 437 |
if you are interested in helping us. |
| 438 |
.IP "Various Mac encodings" 2 |
| 439 |
.IX Item "Various Mac encodings" |
| 440 |
The following are unsupported due to the lack of mapping data. |
| 441 |
.Sp |
| 442 |
.Vb 5 |
| 443 |
\& MacArmenian, MacBengali, MacBurmese, MacEthiopic |
| 444 |
\& MacExtArabic, MacGeorgian, MacKannada, MacKhmer |
| 445 |
\& MacLaotian, MacMalayalam, MacMongolian, MacOriya |
| 446 |
\& MacSinhalese, MacTamil, MacTelugu, MacTibetan |
| 447 |
\& MacVietnamese |
| 448 |
.Ve |
| 449 |
.Sp |
| 450 |
The rest which are already available are based upon the vendor mappings |
| 451 |
at <http://www.unicode.org/Public/MAPPINGS/VENDORS/APPLE/> . |
| 452 |
.IP "(Mac) Indic encodings" 2 |
| 453 |
.IX Item "(Mac) Indic encodings" |
| 454 |
The maps for the following are available at <http://www.unicode.org/> |
| 455 |
but remain unsupported because those encodings need an algorithmical |
| 456 |
approach, currently unsupported by \fIenc2xs\fR: |
| 457 |
.Sp |
| 458 |
.Vb 3 |
| 459 |
\& MacDevanagari |
| 460 |
\& MacGurmukhi |
| 461 |
\& MacGujarati |
| 462 |
.Ve |
| 463 |
.Sp |
| 464 |
For details, please see \f(CW\*(C`Unicode mapping issues and notes:\*(C'\fR at |
| 465 |
<http://www.unicode.org/Public/MAPPINGS/VENDORS/APPLE/DEVANAGA.TXT> . |
| 466 |
.Sp |
| 467 |
I believe this issue is prevalent not only for Mac Indics but also in |
| 468 |
other Indic encodings, but the above were the only Indic encodings |
| 469 |
maps that I could find at <http://www.unicode.org/> . |
| 470 |
.SH "Encoding vs. Charset \-\- terminology" |
| 471 |
.IX Header "Encoding vs. Charset -- terminology" |
| 472 |
We are used to using the term (character) \fIencoding\fR and \fIcharacter |
| 473 |
set\fR interchangeably. But just as confusing the terms byte and |
| 474 |
character is dangerous and the terms should be differentiated when |
| 475 |
needed, we need to differentiate \fIencoding\fR and \fIcharacter set\fR. |
| 476 |
.PP |
| 477 |
To understand that, here is a description of how we make computers |
| 478 |
grok our characters. |
| 479 |
.IP \(bu 2 |
| 480 |
First we start with which characters to include. We call this |
| 481 |
collection of characters \fIcharacter repertoire\fR. |
| 482 |
.IP \(bu 2 |
| 483 |
Then we have to give each character a unique ID so your computer can |
| 484 |
tell the difference between 'a' and 'A'. This itemized character |
| 485 |
repertoire is now a \fIcharacter set\fR. |
| 486 |
.IP \(bu 2 |
| 487 |
If your computer can grow the character set without further |
| 488 |
processing, you can go ahead and use it. This is called a \fIcoded |
| 489 |
character set\fR (CCS) or \fIraw character encoding\fR. ASCII is used this |
| 490 |
way for most cases. |
| 491 |
.IP \(bu 2 |
| 492 |
But in many cases, especially multi-byte CJK encodings, you have to |
| 493 |
tweak a little more. Your network connection may not accept any data |
| 494 |
with the Most Significant Bit set, and your computer may not be able to |
| 495 |
tell if a given byte is a whole character or just half of it. So you |
| 496 |
have to \fIencode\fR the character set to use it. |
| 497 |
.Sp |
| 498 |
A \fIcharacter encoding scheme\fR (CES) determines how to encode a given |
| 499 |
character set, or a set of multiple character sets. 7bit ISO\-2022 is |
| 500 |
an example of a CES. You switch between character sets via \fIescape |
| 501 |
sequences\fR. |
| 502 |
.PP |
| 503 |
Technically, or mathematically, speaking, a character set encoded in |
| 504 |
such a CES that maps character by character may form a CCS. EUC is such |
| 505 |
an example. The CES of EUC is as follows: |
| 506 |
.IP \(bu 2 |
| 507 |
Map ASCII unchanged. |
| 508 |
.IP \(bu 2 |
| 509 |
Map such a character set that consists of 94 or 96 powered by N |
| 510 |
members by adding 0x80 to each byte. |
| 511 |
.IP \(bu 2 |
| 512 |
You can also use 0x8e and 0x8f to indicate that the following sequence of |
| 513 |
characters belongs to yet another character set. To each following byte |
| 514 |
is added the value 0x80. |
| 515 |
.PP |
| 516 |
By carefully looking at the encoded byte sequence, you can find that the |
| 517 |
byte sequence conforms a unique number. In that sense, EUC is a CCS |
| 518 |
generated by a CES above from up to four CCS (complicated?). UTF\-8 |
| 519 |
falls into this category. See "UTF\-8" in perlUnicode to find out how |
| 520 |
UTF\-8 maps Unicode to a byte sequence. |
| 521 |
.PP |
| 522 |
You may also have found out by now why 7bit ISO\-2022 cannot comprise |
| 523 |
a CCS. If you look at a byte sequence \ex21\ex21, you can't tell if |
| 524 |
it is two !'s or IDEOGRAPHIC SPACE. EUC maps the latter to \exA1\exA1 |
| 525 |
so you have no trouble differentiating between "!!". and "\ \ ". |
| 526 |
.SH "Encoding Classification (by Anton Tagunov and Dan Kogai)" |
| 527 |
.IX Header "Encoding Classification (by Anton Tagunov and Dan Kogai)" |
| 528 |
This section tries to classify the supported encodings by their |
| 529 |
applicability for information exchange over the Internet and to |
| 530 |
choose the most suitable aliases to name them in the context of |
| 531 |
such communication. |
| 532 |
.IP \(bu 2 |
| 533 |
To (en|de)code encodings marked by \f(CW\*(C`(**)\*(C'\fR, you need |
| 534 |
\&\f(CW\*(C`Encode::HanExtra\*(C'\fR, available from CPAN. |
| 535 |
.PP |
| 536 |
Encoding names |
| 537 |
.PP |
| 538 |
.Vb 3 |
| 539 |
\& US\-ASCII UTF\-8 ISO\-8859\-* KOI8\-R |
| 540 |
\& Shift_JIS EUC\-JP ISO\-2022\-JP ISO\-2022\-JP\-1 |
| 541 |
\& EUC\-KR Big5 GB2312 |
| 542 |
.Ve |
| 543 |
.PP |
| 544 |
are registered with IANA as preferred MIME names and may |
| 545 |
be used over the Internet. |
| 546 |
.PP |
| 547 |
\&\f(CW\*(C`Shift_JIS\*(C'\fR has been officialized by JIS X 0208:1997. |
| 548 |
"Microsoft-related naming mess" gives details. |
| 549 |
.PP |
| 550 |
\&\f(CW\*(C`GB2312\*(C'\fR is the IANA name for \f(CW\*(C`EUC\-CN\*(C'\fR. |
| 551 |
See "Microsoft-related naming mess" for details. |
| 552 |
.PP |
| 553 |
\&\f(CW\*(C`GB_2312\-80\*(C'\fR \fIraw\fR encoding is available as \f(CW\*(C`gb2312\-raw\*(C'\fR |
| 554 |
with Encode. See Encode::CN for details. |
| 555 |
.PP |
| 556 |
.Vb 2 |
| 557 |
\& EUC\-CN |
| 558 |
\& KOI8\-U [RFC2319] |
| 559 |
.Ve |
| 560 |
.PP |
| 561 |
have not been registered with IANA (as of March 2002) but |
| 562 |
seem to be supported by major web browsers. |
| 563 |
The IANA name for \f(CW\*(C`EUC\-CN\*(C'\fR is \f(CW\*(C`GB2312\*(C'\fR. |
| 564 |
.PP |
| 565 |
.Vb 1 |
| 566 |
\& KS_C_5601\-1987 |
| 567 |
.Ve |
| 568 |
.PP |
| 569 |
is heavily misused. |
| 570 |
See "Microsoft-related naming mess" for details. |
| 571 |
.PP |
| 572 |
\&\f(CW\*(C`KS_C_5601\-1987\*(C'\fR \fIraw\fR encoding is available as \f(CW\*(C`kcs5601\-raw\*(C'\fR |
| 573 |
with Encode. See Encode::KR for details. |
| 574 |
.PP |
| 575 |
.Vb 1 |
| 576 |
\& UTF\-16 UTF\-16BE UTF\-16LE |
| 577 |
.Ve |
| 578 |
.PP |
| 579 |
are IANA-registered \f(CW\*(C`charset\*(C'\fRs. See [RFC 2781] for details. |
| 580 |
Jungshik Shin reports that UTF\-16 with a BOM is well accepted |
| 581 |
by MS IE 5/6 and NS 4/6. Beware however that |
| 582 |
.IP \(bu 2 |
| 583 |
\&\f(CW\*(C`UTF\-16\*(C'\fR support in any software you're going to be |
| 584 |
using/interoperating with has probably been less tested |
| 585 |
then \f(CW\*(C`UTF\-8\*(C'\fR support |
| 586 |
.IP \(bu 2 |
| 587 |
\&\f(CW\*(C`UTF\-8\*(C'\fR coded data seamlessly passes traditional |
| 588 |
command piping (\f(CW\*(C`cat\*(C'\fR, \f(CW\*(C`more\*(C'\fR, etc.) while \f(CW\*(C`UTF\-16\*(C'\fR coded |
| 589 |
data is likely to cause confusion (with its zero bytes, |
| 590 |
for example) |
| 591 |
.IP \(bu 2 |
| 592 |
it is beyond the power of words to describe the way HTML browsers |
| 593 |
encode non\-\f(CW\*(C`ASCII\*(C'\fR form data. To get a general impression, visit |
| 594 |
<http://www.alanflavell.org.uk/charset/form\-i18n.html>. |
| 595 |
While encoding of form data has stabilized for \f(CW\*(C`UTF\-8\*(C'\fR encoded pages |
| 596 |
(at least IE 5/6, NS 6, and Opera 6 behave consistently), be sure to |
| 597 |
expect fun (and cross-browser discrepancies) with \f(CW\*(C`UTF\-16\*(C'\fR encoded |
| 598 |
pages! |
| 599 |
.PP |
| 600 |
The rule of thumb is to use \f(CW\*(C`UTF\-8\*(C'\fR unless you know what |
| 601 |
you're doing and unless you really benefit from using \f(CW\*(C`UTF\-16\*(C'\fR. |
| 602 |
.PP |
| 603 |
.Vb 5 |
| 604 |
\& ISO\-IR\-165 [RFC1345] |
| 605 |
\& VISCII |
| 606 |
\& GB 12345 |
| 607 |
\& GB 18030 (**) (see links below) |
| 608 |
\& EUC\-TW (**) |
| 609 |
.Ve |
| 610 |
.PP |
| 611 |
are totally valid encodings but not registered at IANA. |
| 612 |
The names under which they are listed here are probably the |
| 613 |
most widely-known names for these encodings and are recommended |
| 614 |
names. |
| 615 |
.PP |
| 616 |
.Vb 1 |
| 617 |
\& BIG5PLUS (**) |
| 618 |
.Ve |
| 619 |
.PP |
| 620 |
is a proprietary name. |
| 621 |
.SS "Microsoft-related naming mess" |
| 622 |
.IX Subsection "Microsoft-related naming mess" |
| 623 |
Microsoft products misuse the following names: |
| 624 |
.IP KS_C_5601\-1987 2 |
| 625 |
.IX Item "KS_C_5601-1987" |
| 626 |
Microsoft extension to \f(CW\*(C`EUC\-KR\*(C'\fR. |
| 627 |
.Sp |
| 628 |
Proper names: \f(CW\*(C`CP949\*(C'\fR, \f(CW\*(C`UHC\*(C'\fR, \f(CW\*(C`x\-windows\-949\*(C'\fR (as used by Mozilla). |
| 629 |
.Sp |
| 630 |
See <http://lists.w3.org/Archives/Public/ietf\-charsets/2001AprJun/0033.html> |
| 631 |
for details. |
| 632 |
.Sp |
| 633 |
Encode aliases \f(CW\*(C`KS_C_5601\-1987\*(C'\fR to \f(CW\*(C`cp949\*(C'\fR to reflect this common |
| 634 |
misusage. \fIRaw\fR \f(CW\*(C`KS_C_5601\-1987\*(C'\fR encoding is available as |
| 635 |
\&\f(CW\*(C`kcs5601\-raw\*(C'\fR. |
| 636 |
.Sp |
| 637 |
See Encode::KR for details. |
| 638 |
.IP GB2312 2 |
| 639 |
.IX Item "GB2312" |
| 640 |
Microsoft extension to \f(CW\*(C`EUC\-CN\*(C'\fR. |
| 641 |
.Sp |
| 642 |
Proper names: \f(CW\*(C`CP936\*(C'\fR, \f(CW\*(C`GBK\*(C'\fR. |
| 643 |
.Sp |
| 644 |
\&\f(CW\*(C`GB2312\*(C'\fR has been registered in the \f(CW\*(C`EUC\-CN\*(C'\fR meaning at |
| 645 |
IANA. This has partially repaired the situation: Microsoft's |
| 646 |
\&\f(CW\*(C`GB2312\*(C'\fR has become a superset of the official \f(CW\*(C`GB2312\*(C'\fR. |
| 647 |
.Sp |
| 648 |
Encode aliases \f(CW\*(C`GB2312\*(C'\fR to \f(CW\*(C`euc\-cn\*(C'\fR in full agreement with |
| 649 |
IANA registration. \f(CW\*(C`cp936\*(C'\fR is supported separately. |
| 650 |
\&\fIRaw\fR \f(CW\*(C`GB_2312\-80\*(C'\fR encoding is available as \f(CW\*(C`gb2312\-raw\*(C'\fR. |
| 651 |
.Sp |
| 652 |
See Encode::CN for details. |
| 653 |
.IP Big5 2 |
| 654 |
.IX Item "Big5" |
| 655 |
Microsoft extension to \f(CW\*(C`Big5\*(C'\fR. |
| 656 |
.Sp |
| 657 |
Proper name: \f(CW\*(C`CP950\*(C'\fR. |
| 658 |
.Sp |
| 659 |
Encode separately supports \f(CW\*(C`Big5\*(C'\fR and \f(CW\*(C`cp950\*(C'\fR. |
| 660 |
.IP Shift_JIS 2 |
| 661 |
.IX Item "Shift_JIS" |
| 662 |
Microsoft's understanding of \f(CW\*(C`Shift_JIS\*(C'\fR. |
| 663 |
.Sp |
| 664 |
JIS has not endorsed the full Microsoft standard however. |
| 665 |
The official \f(CW\*(C`Shift_JIS\*(C'\fR includes only JIS X 0201 and JIS X 0208 |
| 666 |
character sets, while Microsoft has always used \f(CW\*(C`Shift_JIS\*(C'\fR |
| 667 |
to encode a wider character repertoire. See \f(CW\*(C`IANA\*(C'\fR registration for |
| 668 |
\&\f(CW\*(C`Windows\-31J\*(C'\fR. |
| 669 |
.Sp |
| 670 |
As a historical predecessor, Microsoft's variant |
| 671 |
probably has more rights for the name, though it may be objected |
| 672 |
that Microsoft shouldn't have used JIS as part of the name |
| 673 |
in the first place. |
| 674 |
.Sp |
| 675 |
Unambiguous name: \f(CW\*(C`CP932\*(C'\fR. \f(CW\*(C`IANA\*(C'\fR name (also used by Mozilla, and |
| 676 |
provided as an alias by Encode): \f(CW\*(C`Windows\-31J\*(C'\fR. |
| 677 |
.Sp |
| 678 |
Encode separately supports \f(CW\*(C`Shift_JIS\*(C'\fR and \f(CW\*(C`cp932\*(C'\fR. |
| 679 |
.SH Glossary |
| 680 |
.IX Header "Glossary" |
| 681 |
.IP "character repertoire" 2 |
| 682 |
.IX Item "character repertoire" |
| 683 |
A collection of unique characters. A \fIcharacter\fR set in the strictest |
| 684 |
sense. At this stage, characters are not numbered. |
| 685 |
.IP "coded character set (CCS)" 2 |
| 686 |
.IX Item "coded character set (CCS)" |
| 687 |
A character set that is mapped in a way computers can use directly. |
| 688 |
Many character encodings, including EUC, fall in this category. |
| 689 |
.IP "character encoding scheme (CES)" 2 |
| 690 |
.IX Item "character encoding scheme (CES)" |
| 691 |
An algorithm to map a character set to a byte sequence. You don't |
| 692 |
have to be able to tell which character set a given byte sequence |
| 693 |
belongs. 7\-bit ISO\-2022 is a CES but it cannot be a CCS. EUC is an |
| 694 |
example of being both a CCS and CES. |
| 695 |
.IP "charset (in MIME context)" 2 |
| 696 |
.IX Item "charset (in MIME context)" |
| 697 |
has long been used in the meaning of \f(CW\*(C`encoding\*(C'\fR, CES. |
| 698 |
.Sp |
| 699 |
While the word combination \f(CW\*(C`character set\*(C'\fR has lost this meaning |
| 700 |
in MIME context since [RFC 2130], the \f(CW\*(C`charset\*(C'\fR abbreviation has |
| 701 |
retained it. This is how [RFC 2277] and [RFC 2278] bless \f(CW\*(C`charset\*(C'\fR: |
| 702 |
.Sp |
| 703 |
.Vb 7 |
| 704 |
\& This document uses the term "charset" to mean a set of rules for |
| 705 |
\& mapping from a sequence of octets to a sequence of characters, such |
| 706 |
\& as the combination of a coded character set and a character encoding |
| 707 |
\& scheme; this is also what is used as an identifier in MIME "charset=" |
| 708 |
\& parameters, and registered in the IANA charset registry ... (Note |
| 709 |
\& that this is NOT a term used by other standards bodies, such as ISO). |
| 710 |
\& [RFC 2277] |
| 711 |
.Ve |
| 712 |
.IP EUC 2 |
| 713 |
.IX Item "EUC" |
| 714 |
Extended Unix Character. See ISO\-2022. |
| 715 |
.IP ISO\-2022 2 |
| 716 |
.IX Item "ISO-2022" |
| 717 |
A CES that was carefully designed to coexist with ASCII. There are a 7 |
| 718 |
bit version and an 8 bit version. |
| 719 |
.Sp |
| 720 |
The 7 bit version switches character set via escape sequence so it |
| 721 |
cannot form a CCS. Since this is more difficult to handle in programs |
| 722 |
than the 8 bit version, the 7 bit version is not very popular except for |
| 723 |
iso\-2022\-jp, the \fIde facto\fR standard CES for e\-mails. |
| 724 |
.Sp |
| 725 |
The 8 bit version can form a CCS. EUC and ISO\-8859 are two examples |
| 726 |
thereof. Pre\-5.6 perl could use them as string literals. |
| 727 |
.IP UCS 2 |
| 728 |
.IX Item "UCS" |
| 729 |
Short for \fIUniversal Character Set\fR. When you say just UCS, it means |
| 730 |
\&\fIUnicode\fR. |
| 731 |
.IP UCS\-2 2 |
| 732 |
.IX Item "UCS-2" |
| 733 |
ISO/IEC 10646 encoding form: Universal Character Set coded in two |
| 734 |
octets. |
| 735 |
.IP Unicode 2 |
| 736 |
.IX Item "Unicode" |
| 737 |
A character set that aims to include all character repertoires of the |
| 738 |
world. Many character sets in various national as well as industrial |
| 739 |
standards have become, in a way, just subsets of Unicode. |
| 740 |
.IP UTF 2 |
| 741 |
.IX Item "UTF" |
| 742 |
Short for \fIUnicode Transformation Format\fR. Determines how to map a |
| 743 |
Unicode character into a byte sequence. |
| 744 |
.IP UTF\-16 2 |
| 745 |
.IX Item "UTF-16" |
| 746 |
A UTF in 16\-bit encoding. Can either be in big endian or little |
| 747 |
endian. The big endian version is called UTF\-16BE (equal to UCS\-2 + |
| 748 |
surrogate support) and the little endian version is called UTF\-16LE. |
| 749 |
.SH "See Also" |
| 750 |
.IX Header "See Also" |
| 751 |
Encode, |
| 752 |
Encode::Byte, |
| 753 |
Encode::CN, Encode::JP, Encode::KR, Encode::TW, |
| 754 |
Encode::EBCDIC, Encode::Symbol |
| 755 |
Encode::MIME::Header, Encode::Guess |
| 756 |
.SH References |
| 757 |
.IX Header "References" |
| 758 |
.IP ECMA 2 |
| 759 |
.IX Item "ECMA" |
| 760 |
European Computer Manufacturers Association |
| 761 |
<http://www.ecma.ch> |
| 762 |
.RS 2 |
| 763 |
.ie n .IP "ECMA\-035 (eq ""ISO\-2022"")" 2 |
| 764 |
.el .IP "ECMA\-035 (eq \f(CWISO\-2022\fR)" 2 |
| 765 |
.IX Item "ECMA-035 (eq ISO-2022)" |
| 766 |
<http://www.ecma.ch/ecma1/STAND/ECMA\-035.HTM> |
| 767 |
.Sp |
| 768 |
The specification of ISO\-2022 is available from the link above. |
| 769 |
.RE |
| 770 |
.RS 2 |
| 771 |
.RE |
| 772 |
.IP IANA 2 |
| 773 |
.IX Item "IANA" |
| 774 |
Internet Assigned Numbers Authority |
| 775 |
<http://www.iana.org/> |
| 776 |
.RS 2 |
| 777 |
.IP "Assigned Charset Names by IANA" 2 |
| 778 |
.IX Item "Assigned Charset Names by IANA" |
| 779 |
<http://www.iana.org/assignments/character\-sets> |
| 780 |
.Sp |
| 781 |
Most of the \f(CW\*(C`canonical names\*(C'\fR in Encode derive from this list |
| 782 |
so you can directly apply the string you have extracted from MIME |
| 783 |
header of mails and web pages. |
| 784 |
.RE |
| 785 |
.RS 2 |
| 786 |
.RE |
| 787 |
.IP ISO 2 |
| 788 |
.IX Item "ISO" |
| 789 |
International Organization for Standardization |
| 790 |
<http://www.iso.ch/> |
| 791 |
.IP RFC 2 |
| 792 |
.IX Item "RFC" |
| 793 |
Request For Comments \-\- need I say more? |
| 794 |
<http://www.rfc\-editor.org/>, <http://www.ietf.org/rfc.html>, |
| 795 |
<http://www.faqs.org/rfcs/> |
| 796 |
.IP UC 2 |
| 797 |
.IX Item "UC" |
| 798 |
Unicode Consortium |
| 799 |
<http://www.unicode.org/> |
| 800 |
.RS 2 |
| 801 |
.IP "Unicode Glossary" 2 |
| 802 |
.IX Item "Unicode Glossary" |
| 803 |
<http://www.unicode.org/glossary/> |
| 804 |
.Sp |
| 805 |
The glossary of this document is based upon this site. |
| 806 |
.RE |
| 807 |
.RS 2 |
| 808 |
.RE |
| 809 |
.SS "Other Notable Sites" |
| 810 |
.IX Subsection "Other Notable Sites" |
| 811 |
.IP czyborra.com 2 |
| 812 |
.IX Item "czyborra.com" |
| 813 |
<http://czyborra.com/> |
| 814 |
.Sp |
| 815 |
Contains a lot of useful information, especially gory details of ISO |
| 816 |
vs. vendor mappings. |
| 817 |
.IP CJK.inf 2 |
| 818 |
.IX Item "CJK.inf" |
| 819 |
<http://examples.oreilly.com/cjkvinfo/doc/cjk.inf> |
| 820 |
.Sp |
| 821 |
Somewhat obsolete (last update in 1996), but still useful. Also try |
| 822 |
.Sp |
| 823 |
<ftp://ftp.oreilly.com/pub/examples/nutshell/cjkv/pdf/GB18030_Summary.pdf> |
| 824 |
.Sp |
| 825 |
You will find brief info on \f(CW\*(C`EUC\-CN\*(C'\fR, \f(CW\*(C`GBK\*(C'\fR and mostly on \f(CW\*(C`GB 18030\*(C'\fR. |
| 826 |
.IP "Jungshik Shin's Hangul FAQ" 2 |
| 827 |
.IX Item "Jungshik Shin's Hangul FAQ" |
| 828 |
<http://jshin.net/faq> |
| 829 |
.Sp |
| 830 |
And especially its subject 8. |
| 831 |
.Sp |
| 832 |
<http://jshin.net/faq/qa8.html> |
| 833 |
.Sp |
| 834 |
A comprehensive overview of the Korean (\f(CW\*(C`KS *\*(C'\fR) standards. |
| 835 |
.IP "debian.org: ""Introduction to i18n""" 2 |
| 836 |
.IX Item "debian.org: ""Introduction to i18n""" |
| 837 |
A brief description for most of the mentioned CJK encodings is |
| 838 |
contained in |
| 839 |
<http://www.debian.org/doc/manuals/intro\-i18n/ch\-codes.en.html> |
| 840 |
.SS "Offline sources" |
| 841 |
.IX Subsection "Offline sources" |
| 842 |
.ie n .IP """CJKV Information Processing"" by Ken Lunde" 2 |
| 843 |
.el .IP "\f(CWCJKV Information Processing\fR by Ken Lunde" 2 |
| 844 |
.IX Item "CJKV Information Processing by Ken Lunde" |
| 845 |
CJKV Information Processing |
| 846 |
1999 O'Reilly & Associates, ISBN : 1\-56592\-224\-7 |
| 847 |
.Sp |
| 848 |
The modern successor of \f(CW\*(C`CJK.inf\*(C'\fR. |
| 849 |
.Sp |
| 850 |
Features a comprehensive coverage of CJKV character sets and |
| 851 |
encodings along with many other issues faced by anyone trying |
| 852 |
to better support CJKV languages/scripts in all the areas of |
| 853 |
information processing. |
| 854 |
.Sp |
| 855 |
To purchase this book, visit |
| 856 |
<http://oreilly.com/catalog/9780596514471/> |
| 857 |
or your favourite bookstore. |