Debian Codegraph @

upstream/debian-unstable/man3/Unicode::UCD.3perl

raw
1 .\" -*- mode: troff; coding: utf-8 -*-
2 .\" Automatically generated by Pod::Man 5.0102 (Pod::Simple 3.45)
3 .\"
4 .\" Standard preamble:
5 .\" ========================================================================
6 .de Sp \" Vertical space (when we can't use .PP)
7 .if t .sp .5v
8 .if n .sp
9 ..
10 .de Vb \" Begin verbatim text
11 .ft CW
12 .nf
13 .ne \\$1
14 ..
15 .de Ve \" End verbatim text
16 .ft R
17 .fi
18 ..
19 .\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>.
20 .ie n \{\
21 . ds C` ""
22 . ds C' ""
23 'br\}
24 .el\{\
25 . ds C`
26 . ds C'
27 'br\}
28 .\"
29 .\" Escape single quotes in literal strings from groff's Unicode transform.
30 .ie \n(.g .ds Aq \(aq
31 .el .ds Aq '
32 .\"
33 .\" If the F register is >0, we'll generate index entries on stderr for
34 .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
35 .\" entries marked with X<> in POD. Of course, you'll have to process the
36 .\" output yourself in some meaningful fashion.
37 .\"
38 .\" Avoid warning from groff about undefined register 'F'.
39 .de IX
40 ..
41 .nr rF 0
42 .if \n(.g .if rF .nr rF 1
43 .if (\n(rF:(\n(.g==0)) \{\
44 . if \nF \{\
45 . de IX
46 . tm Index:\\$1\t\\n%\t"\\$2"
47 ..
48 . if !\nF==2 \{\
49 . nr % 0
50 . nr F 2
51 . \}
52 . \}
53 .\}
54 .rr rF
55 .\" ========================================================================
56 .\"
57 .IX Title "Unicode::UCD 3perl"
58 .TH Unicode::UCD 3perl 2025-11-16 "perl v5.40.1" "Perl Programmers Reference Guide"
59 .\" For nroff, turn off justification. Always turn off hyphenation; it makes
60 .\" way too many mistakes in technical documents.
61 .if n .ad l
62 .nh
63 .SH NAME
64 Unicode::UCD \- Unicode character database
65 .SH SYNOPSIS
66 .IX Header "SYNOPSIS"
67 .Vb 2
68 \& use Unicode::UCD \*(Aqcharinfo\*(Aq;
69 \& my $charinfo = charinfo($codepoint);
70 \&
71 \& use Unicode::UCD \*(Aqcharprop\*(Aq;
72 \& my $value = charprop($codepoint, $property);
73 \&
74 \& use Unicode::UCD \*(Aqcharprops_all\*(Aq;
75 \& my $all_values_hash_ref = charprops_all($codepoint);
76 \&
77 \& use Unicode::UCD \*(Aqcasefold\*(Aq;
78 \& my $casefold = casefold($codepoint);
79 \&
80 \& use Unicode::UCD \*(Aqall_casefolds\*(Aq;
81 \& my $all_casefolds_ref = all_casefolds();
82 \&
83 \& use Unicode::UCD \*(Aqcasespec\*(Aq;
84 \& my $casespec = casespec($codepoint);
85 \&
86 \& use Unicode::UCD \*(Aqcharblock\*(Aq;
87 \& my $charblock = charblock($codepoint);
88 \&
89 \& use Unicode::UCD \*(Aqcharscript\*(Aq;
90 \& my $charscript = charscript($codepoint);
91 \&
92 \& use Unicode::UCD \*(Aqcharblocks\*(Aq;
93 \& my $charblocks = charblocks();
94 \&
95 \& use Unicode::UCD \*(Aqcharscripts\*(Aq;
96 \& my $charscripts = charscripts();
97 \&
98 \& use Unicode::UCD qw(charscript charinrange);
99 \& my $range = charscript($script);
100 \& print "looks like $script\en" if charinrange($range, $codepoint);
101 \&
102 \& use Unicode::UCD qw(general_categories bidi_types);
103 \& my $categories = general_categories();
104 \& my $types = bidi_types();
105 \&
106 \& use Unicode::UCD \*(Aqprop_aliases\*(Aq;
107 \& my @space_names = prop_aliases("space");
108 \&
109 \& use Unicode::UCD \*(Aqprop_value_aliases\*(Aq;
110 \& my @gc_punct_names = prop_value_aliases("Gc", "Punct");
111 \&
112 \& use Unicode::UCD \*(Aqprop_values\*(Aq;
113 \& my @all_EA_short_names = prop_values("East_Asian_Width");
114 \&
115 \& use Unicode::UCD \*(Aqprop_invlist\*(Aq;
116 \& my @puncts = prop_invlist("gc=punctuation");
117 \&
118 \& use Unicode::UCD \*(Aqprop_invmap\*(Aq;
119 \& my ($list_ref, $map_ref, $format, $missing)
120 \& = prop_invmap("General Category");
121 \&
122 \& use Unicode::UCD \*(Aqsearch_invlist\*(Aq;
123 \& my $index = search_invlist(\e@invlist, $code_point);
124 \&
125 \& # The following function should be used only internally in
126 \& # implementations of the Unicode Normalization Algorithm, and there
127 \& # are better choices than it.
128 \& use Unicode::UCD \*(Aqcompexcl\*(Aq;
129 \& my $compexcl = compexcl($codepoint);
130 \&
131 \& use Unicode::UCD \*(Aqnamedseq\*(Aq;
132 \& my $namedseq = namedseq($named_sequence_name);
133 \&
134 \& my $unicode_version = Unicode::UCD::UnicodeVersion();
135 \&
136 \& my $convert_to_numeric =
137 \& Unicode::UCD::num("\eN{RUMI DIGIT ONE}\eN{RUMI DIGIT TWO}");
138 .Ve
139 .SH DESCRIPTION
140 .IX Header "DESCRIPTION"
141 The Unicode::UCD module offers a series of functions that
142 provide a simple interface to the Unicode
143 Character Database.
144 .SS "code point argument"
145 .IX Subsection "code point argument"
146 Some of the functions are called with a \fIcode point argument\fR, which is either
147 a decimal or a hexadecimal scalar designating a code point in the platform's
148 native character set (extended to Unicode), or a string containing \f(CW\*(C`U+\*(C'\fR
149 followed by hexadecimals
150 designating a Unicode code point. A leading 0 will force a hexadecimal
151 interpretation, as will a hexadecimal digit that isn't a decimal digit.
152 .PP
153 Examples:
154 .PP
155 .Vb 6
156 \& 223 # Decimal 223 in native character set
157 \& 0223 # Hexadecimal 223, native (= 547 decimal)
158 \& 0xDF # Hexadecimal DF, native (= 223 decimal)
159 \& \*(Aq0xDF\*(Aq # String form of hexadecimal (= 223 decimal)
160 \& \*(AqU+DF\*(Aq # Hexadecimal DF, in Unicode\*(Aqs character set
161 \& (= LATIN SMALL LETTER SHARP S)
162 .Ve
163 .PP
164 Note that the largest code point in Unicode is U+10FFFF.
165 .SS \fBcharinfo()\fP
166 .IX Subsection "charinfo()"
167 .Vb 1
168 \& use Unicode::UCD \*(Aqcharinfo\*(Aq;
169 \&
170 \& my $charinfo = charinfo(0x41);
171 .Ve
172 .PP
173 This returns information about the input "code point argument"
174 as a reference to a hash of fields as defined by the Unicode
175 standard. If the "code point argument" is not assigned in the standard
176 (i.e., has the general category \f(CW\*(C`Cn\*(C'\fR meaning \f(CW\*(C`Unassigned\*(C'\fR)
177 or is a non-character (meaning it is guaranteed to never be assigned in
178 the standard),
179 \&\f(CW\*(C`undef\*(C'\fR is returned.
180 .PP
181 Fields that aren't applicable to the particular code point argument exist in the
182 returned hash, and are empty.
183 .PP
184 For results that are less "raw" than this function returns, or to get the values for
185 any property, not just the few covered by this function, use the
186 "\fBcharprop()\fR" function.
187 .PP
188 The keys in the hash with the meanings of their values are:
189 .IP \fBcode\fR 4
190 .IX Item "code"
191 the input native "code point argument" expressed in hexadecimal, with
192 leading zeros
193 added if necessary to make it contain at least four hexdigits
194 .IP \fBname\fR 4
195 .IX Item "name"
196 name of \fIcode\fR, all IN UPPER CASE.
197 Some control-type code points do not have names.
198 This field will be empty for \f(CW\*(C`Surrogate\*(C'\fR and \f(CW\*(C`Private Use\*(C'\fR code points,
199 and for the others without a name,
200 it will contain a description enclosed in angle brackets, like
201 \&\f(CW\*(C`<control>\*(C'\fR.
202 .IP \fBcategory\fR 4
203 .IX Item "category"
204 The short name of the general category of \fIcode\fR.
205 This will match one of the keys in the hash returned by "\fBgeneral_categories()\fR".
206 .Sp
207 The "\fBprop_value_aliases()\fR" function can be used to get all the synonyms
208 of the category name.
209 .IP \fBcombining\fR 4
210 .IX Item "combining"
211 the combining class number for \fIcode\fR used in the Canonical Ordering Algorithm.
212 For Unicode 5.1, this is described in Section 3.11 \f(CW\*(C`Canonical Ordering Behavior\*(C'\fR
213 available at
214 <http://www.unicode.org/versions/Unicode5.1.0/>
215 .Sp
216 The "\fBprop_value_aliases()\fR" function can be used to get all the synonyms
217 of the combining class number.
218 .IP \fBbidi\fR 4
219 .IX Item "bidi"
220 bidirectional type of \fIcode\fR.
221 This will match one of the keys in the hash returned by "\fBbidi_types()\fR".
222 .Sp
223 The "\fBprop_value_aliases()\fR" function can be used to get all the synonyms
224 of the bidi type name.
225 .IP \fBdecomposition\fR 4
226 .IX Item "decomposition"
227 is empty if \fIcode\fR has no decomposition; or is one or more codes
228 (separated by spaces) that, taken in order, represent a decomposition for
229 \&\fIcode\fR. Each has at least four hexdigits.
230 The codes may be preceded by a word enclosed in angle brackets, then a space,
231 like \f(CW\*(C`<compat> \*(C'\fR, giving the type of decomposition
232 .Sp
233 This decomposition may be an intermediate one whose components are also
234 decomposable. Use Unicode::Normalize to get the final decomposition in one
235 step.
236 .IP \fBdecimal\fR 4
237 .IX Item "decimal"
238 if \fIcode\fR represents a decimal digit this is its integer numeric value
239 .IP \fBdigit\fR 4
240 .IX Item "digit"
241 if \fIcode\fR represents some other digit-like number, this is its integer
242 numeric value
243 .IP \fBnumeric\fR 4
244 .IX Item "numeric"
245 if \fIcode\fR represents a whole or rational number, this is its numeric value.
246 Rational values are expressed as a string like \f(CW\*(C`1/4\*(C'\fR.
247 .IP \fBmirrored\fR 4
248 .IX Item "mirrored"
249 \&\f(CW\*(C`Y\*(C'\fR or \f(CW\*(C`N\*(C'\fR designating if \fIcode\fR is mirrored in bidirectional text
250 .IP \fBunicode10\fR 4
251 .IX Item "unicode10"
252 name of \fIcode\fR in the Unicode 1.0 standard if one
253 existed for this code point and is different from the current name
254 .IP \fBcomment\fR 4
255 .IX Item "comment"
256 As of Unicode 6.0, this is always empty.
257 .IP \fBupper\fR 4
258 .IX Item "upper"
259 is, if non-empty, the uppercase mapping for \fIcode\fR expressed as at least four
260 hexdigits. This indicates that the full uppercase mapping is a single
261 character, and is identical to the simple (single-character only) mapping.
262 When this field is empty, it means that the simple uppercase mapping is
263 \&\fIcode\fR itself; you'll need some other means, (like "\fBcharprop()\fR" or
264 "\fBcasespec()\fR" to get the full mapping.
265 .IP \fBlower\fR 4
266 .IX Item "lower"
267 is, if non-empty, the lowercase mapping for \fIcode\fR expressed as at least four
268 hexdigits. This indicates that the full lowercase mapping is a single
269 character, and is identical to the simple (single-character only) mapping.
270 When this field is empty, it means that the simple lowercase mapping is
271 \&\fIcode\fR itself; you'll need some other means, (like "\fBcharprop()\fR" or
272 "\fBcasespec()\fR" to get the full mapping.
273 .IP \fBtitle\fR 4
274 .IX Item "title"
275 is, if non-empty, the titlecase mapping for \fIcode\fR expressed as at least four
276 hexdigits. This indicates that the full titlecase mapping is a single
277 character, and is identical to the simple (single-character only) mapping.
278 When this field is empty, it means that the simple titlecase mapping is
279 \&\fIcode\fR itself; you'll need some other means, (like "\fBcharprop()\fR" or
280 "\fBcasespec()\fR" to get the full mapping.
281 .IP \fBblock\fR 4
282 .IX Item "block"
283 the block \fIcode\fR belongs to (used in \f(CW\*(C`\ep{Blk=...}\*(C'\fR).
284 The "\fBprop_value_aliases()\fR" function can be used to get all the synonyms
285 of the block name.
286 .Sp
287 See "Blocks versus Scripts".
288 .IP \fBscript\fR 4
289 .IX Item "script"
290 the script \fIcode\fR belongs to.
291 The "\fBprop_value_aliases()\fR" function can be used to get all the synonyms
292 of the script name. Note that this is the older "Script" property value, and
293 not the improved "Script_Extensions" value.
294 .Sp
295 See "Blocks versus Scripts".
296 .PP
297 Note that you cannot do (de)composition and casing based solely on the
298 \&\fIdecomposition\fR, \fIcombining\fR, \fIlower\fR, \fIupper\fR, and \fItitle\fR fields; you
299 will need also the "\fBcasespec()\fR" function and the \f(CW\*(C`Composition_Exclusion\*(C'\fR
300 property. (Or you could just use the \fBlc()\fR,
301 \&\fBuc()\fR, and \fBucfirst()\fR functions, and the
302 Unicode::Normalize module.)
303 .SS \fBcharprop()\fP
304 .IX Subsection "charprop()"
305 .Vb 1
306 \& use Unicode::UCD \*(Aqcharprop\*(Aq;
307 \&
308 \& print charprop(0x41, "Gc"), "\en";
309 \& print charprop(0x61, "General_Category"), "\en";
310 \&
311 \& prints
312 \& Lu
313 \& Ll
314 .Ve
315 .PP
316 This returns the value of the Unicode property given by the second parameter
317 for the "code point argument" given by the first.
318 .PP
319 The passed-in property may be specified as any of the synonyms returned by
320 "\fBprop_aliases()\fR".
321 .PP
322 The return value is always a scalar, either a string or a number. For
323 properties where there are synonyms for the values, the synonym returned by
324 this function is the longest, most descriptive form, the one returned by
325 "\fBprop_value_aliases()\fR" when called in a scalar context. Of course, you can
326 call "\fBprop_value_aliases()\fR" on the result to get other synonyms.
327 .PP
328 The return values are more "cooked" than the "\fBcharinfo()\fR" ones. For
329 example, the \f(CW"uc"\fR property value is the actual string containing the full
330 uppercase mapping of the input code point. You have to go to extra trouble
331 with \f(CW\*(C`charinfo\*(C'\fR to get this value from its \f(CW\*(C`upper\*(C'\fR hash element when the
332 full mapping differs from the simple one.
333 .PP
334 Special note should be made of the return values for a few properties:
335 .IP Block 4
336 .IX Item "Block"
337 The value returned is the new-style (see "Old-style versus new-style block
338 names").
339 .IP Decomposition_Mapping 4
340 .IX Item "Decomposition_Mapping"
341 Like "\fBcharinfo()\fR", the result may be an intermediate decomposition whose
342 components are also decomposable. Use Unicode::Normalize to get the final
343 decomposition in one step.
344 .Sp
345 Unlike "\fBcharinfo()\fR", this does not include the decomposition type. Use the
346 \&\f(CW\*(C`Decomposition_Type\*(C'\fR property to get that.
347 .IP Name_Alias 4
348 .IX Item "Name_Alias"
349 If the input code point's name has more than one synonym, they are returned
350 joined into a single comma-separated string.
351 .IP Numeric_Value 4
352 .IX Item "Numeric_Value"
353 If the result is a fraction, it is converted into a floating point number to
354 the accuracy of your platform.
355 .IP Script_Extensions 4
356 .IX Item "Script_Extensions"
357 If the result is multiple script names, they are returned joined into a single
358 comma-separated string.
359 .PP
360 When called with a property that is a Perl extension that isn't expressible in
361 a compound form, this function currently returns \f(CW\*(C`undef\*(C'\fR, as the only two
362 possible values are \fItrue\fR or \fIfalse\fR (1 or 0 I suppose). This behavior may
363 change in the future, so don't write code that relies on it. \f(CW\*(C`Present_In\*(C'\fR is
364 a Perl extension that is expressible in a bipartite or compound form (for
365 example, \f(CW\*(C`\ep{Present_In=4.0}\*(C'\fR), so \f(CW\*(C`charprop\*(C'\fR accepts it. But \f(CW\*(C`Any\*(C'\fR is a
366 Perl extension that isn't expressible that way, so \f(CW\*(C`charprop\*(C'\fR returns
367 \&\f(CW\*(C`undef\*(C'\fR for it. Also \f(CW\*(C`charprop\*(C'\fR returns \f(CW\*(C`undef\*(C'\fR for all Perl extensions
368 that are internal-only.
369 .SS \fBcharprops_all()\fP
370 .IX Subsection "charprops_all()"
371 .Vb 1
372 \& use Unicode::UCD \*(Aqcharprops_all\*(Aq;
373 \&
374 \& my $%properties_of_A_hash_ref = charprops_all("U+41");
375 .Ve
376 .PP
377 This returns a reference to a hash whose keys are all the distinct Unicode (no
378 Perl extension) properties, and whose values are the respective values for
379 those properties for the input "code point argument".
380 .PP
381 Each key is the property name in its longest, most descriptive form. The
382 values are what "\fBcharprop()\fR" would return.
383 .PP
384 This function is expensive in time and memory.
385 .SS \fBcharblock()\fP
386 .IX Subsection "charblock()"
387 .Vb 1
388 \& use Unicode::UCD \*(Aqcharblock\*(Aq;
389 \&
390 \& my $charblock = charblock(0x41);
391 \& my $charblock = charblock(1234);
392 \& my $charblock = charblock(0x263a);
393 \& my $charblock = charblock("U+263a");
394 \&
395 \& my $range = charblock(\*(AqArmenian\*(Aq);
396 .Ve
397 .PP
398 With a "code point argument" \f(CWcharblock()\fR returns the \fIblock\fR the code point
399 belongs to, e.g. \f(CW\*(C`Basic Latin\*(C'\fR. The old-style block name is returned (see
400 "Old-style versus new-style block names").
401 The "\fBprop_value_aliases()\fR" function can be used to get all the synonyms
402 of the block name.
403 .PP
404 If the code point is unassigned, this returns the block it would belong to if
405 it were assigned. (If the Unicode version being used is so early as to not
406 have blocks, all code points are considered to be in \f(CW\*(C`No_Block\*(C'\fR.)
407 .PP
408 See also "Blocks versus Scripts".
409 .PP
410 If supplied with an argument that can't be a code point, \f(CWcharblock()\fR tries to
411 do the opposite and interpret the argument as an old-style block name. On an
412 ASCII platform, the return value is a \fIrange set\fR with one range: an
413 anonymous array with a single element that consists of another anonymous array
414 whose first element is the first code point in the block, and whose second
415 element is the final code point in the block. On an EBCDIC
416 platform, the first two Unicode blocks are not contiguous. Their range sets
417 are lists containing \fIstart-of-range\fR, \fIend-of-range\fR code point pairs. You
418 can test whether a code point is in a range set using the "\fBcharinrange()\fR"
419 function. (To be precise, each \fIrange set\fR contains a third array element,
420 after the range boundary ones: the old_style block name.)
421 .PP
422 If the argument to \f(CWcharblock()\fR is not a known block, \f(CW\*(C`undef\*(C'\fR is
423 returned.
424 .SS \fBcharscript()\fP
425 .IX Subsection "charscript()"
426 .Vb 1
427 \& use Unicode::UCD \*(Aqcharscript\*(Aq;
428 \&
429 \& my $charscript = charscript(0x41);
430 \& my $charscript = charscript(1234);
431 \& my $charscript = charscript("U+263a");
432 \&
433 \& my $range = charscript(\*(AqThai\*(Aq);
434 .Ve
435 .PP
436 With a "code point argument", \f(CWcharscript()\fR returns the \fIscript\fR the
437 code point belongs to, e.g., \f(CW\*(C`Latin\*(C'\fR, \f(CW\*(C`Greek\*(C'\fR, \f(CW\*(C`Han\*(C'\fR.
438 If the code point is unassigned or the Unicode version being used is so early
439 that it doesn't have scripts, this function returns \f(CW"Unknown"\fR.
440 The "\fBprop_value_aliases()\fR" function can be used to get all the synonyms
441 of the script name.
442 .PP
443 Note that the Script_Extensions property is an improved version of the Script
444 property, and you should probably be using that instead, with the
445 "\fBcharprop()\fR" function.
446 .PP
447 If supplied with an argument that can't be a code point, \fBcharscript()\fR tries
448 to do the opposite and interpret the argument as a script name. The
449 return value is a \fIrange set\fR: an anonymous array of arrays that contain
450 \&\fIstart-of-range\fR, \fIend-of-range\fR code point pairs. You can test whether a
451 code point is in a range set using the "\fBcharinrange()\fR" function.
452 (To be precise, each \fIrange set\fR contains a third array element,
453 after the range boundary ones: the script name.)
454 .PP
455 If the \f(CWcharscript()\fR argument is not a known script, \f(CW\*(C`undef\*(C'\fR is returned.
456 .PP
457 See also "Blocks versus Scripts".
458 .SS \fBcharblocks()\fP
459 .IX Subsection "charblocks()"
460 .Vb 1
461 \& use Unicode::UCD \*(Aqcharblocks\*(Aq;
462 \&
463 \& my $charblocks = charblocks();
464 .Ve
465 .PP
466 \&\f(CWcharblocks()\fR returns a reference to a hash with the known block names
467 as the keys, and the code point ranges (see "\fBcharblock()\fR") as the values.
468 .PP
469 The names are in the old-style (see "Old-style versus new-style block
470 names").
471 .PP
472 prop_invmap("block") can be used to get this same data in a
473 different type of data structure.
474 .PP
475 prop_values("Block") can be used to get all
476 the known new-style block names as a list, without the code point ranges.
477 .PP
478 See also "Blocks versus Scripts".
479 .SS \fBcharscripts()\fP
480 .IX Subsection "charscripts()"
481 .Vb 1
482 \& use Unicode::UCD \*(Aqcharscripts\*(Aq;
483 \&
484 \& my $charscripts = charscripts();
485 .Ve
486 .PP
487 \&\f(CWcharscripts()\fR returns a reference to a hash with the known script
488 names as the keys, and the code point ranges (see "\fBcharscript()\fR") as
489 the values.
490 .PP
491 prop_invmap("script") can be used to get this same data in a
492 different type of data structure. Since the Script_Extensions property is an
493 improved version of the Script property, you should instead use
494 prop_invmap("scx").
495 .PP
496 \&\f(CWprop_values("Script")\fR can be used to get all
497 the known script names as a list, without the code point ranges.
498 .PP
499 See also "Blocks versus Scripts".
500 .SS \fBcharinrange()\fP
501 .IX Subsection "charinrange()"
502 In addition to using the \f(CW\*(C`\ep{Blk=...}\*(C'\fR and \f(CW\*(C`\eP{Blk=...}\*(C'\fR constructs, you
503 can also test whether a code point is in the \fIrange\fR as returned by
504 "\fBcharblock()\fR" and "\fBcharscript()\fR" or as the values of the hash returned
505 by "\fBcharblocks()\fR" and "\fBcharscripts()\fR" by using \f(CWcharinrange()\fR:
506 .PP
507 .Vb 1
508 \& use Unicode::UCD qw(charscript charinrange);
509 \&
510 \& $range = charscript(\*(AqHiragana\*(Aq);
511 \& print "looks like hiragana\en" if charinrange($range, $codepoint);
512 .Ve
513 .SS \fBgeneral_categories()\fP
514 .IX Subsection "general_categories()"
515 .Vb 1
516 \& use Unicode::UCD \*(Aqgeneral_categories\*(Aq;
517 \&
518 \& my $categories = general_categories();
519 .Ve
520 .PP
521 This returns a reference to a hash which has short
522 general category names (such as \f(CW\*(C`Lu\*(C'\fR, \f(CW\*(C`Nd\*(C'\fR, \f(CW\*(C`Zs\*(C'\fR, \f(CW\*(C`S\*(C'\fR) as keys and long
523 names (such as \f(CW\*(C`UppercaseLetter\*(C'\fR, \f(CW\*(C`DecimalNumber\*(C'\fR, \f(CW\*(C`SpaceSeparator\*(C'\fR,
524 \&\f(CW\*(C`Symbol\*(C'\fR) as values. The hash is reversible in case you need to go
525 from the long names to the short names. The general category is the
526 one returned from
527 "\fBcharinfo()\fR" under the \f(CW\*(C`category\*(C'\fR key.
528 .PP
529 The "\fBprop_values()\fR" and "\fBprop_value_aliases()\fR" functions can be used as an
530 alternative to this function; the first returning a simple list of the short
531 category names; and the second gets all the synonyms of a given category name.
532 .SS \fBbidi_types()\fP
533 .IX Subsection "bidi_types()"
534 .Vb 1
535 \& use Unicode::UCD \*(Aqbidi_types\*(Aq;
536 \&
537 \& my $categories = bidi_types();
538 .Ve
539 .PP
540 This returns a reference to a hash which has the short
541 bidi (bidirectional) type names (such as \f(CW\*(C`L\*(C'\fR, \f(CW\*(C`R\*(C'\fR) as keys and long
542 names (such as \f(CW\*(C`Left\-to\-Right\*(C'\fR, \f(CW\*(C`Right\-to\-Left\*(C'\fR) as values. The
543 hash is reversible in case you need to go from the long names to the
544 short names. The bidi type is the one returned from
545 "\fBcharinfo()\fR"
546 under the \f(CW\*(C`bidi\*(C'\fR key. For the exact meaning of the various bidi classes
547 the Unicode TR9 is recommended reading:
548 <http://www.unicode.org/reports/tr9/>
549 (as of Unicode 5.0.0)
550 .PP
551 The "\fBprop_values()\fR" and "\fBprop_value_aliases()\fR" functions can be used as an
552 alternative to this function; the first returning a simple list of the short
553 bidi type names; and the second gets all the synonyms of a given bidi type
554 name.
555 .SS \fBcompexcl()\fP
556 .IX Subsection "compexcl()"
557 WARNING: Unicode discourages the use of this function or any of the
558 alternative mechanisms listed in this section (the documentation of
559 \&\f(CWcompexcl()\fR), except internally in implementations of the Unicode
560 Normalization Algorithm. You should be using Unicode::Normalize directly
561 instead of these. Using these will likely lead to half-baked results.
562 .PP
563 .Vb 1
564 \& use Unicode::UCD \*(Aqcompexcl\*(Aq;
565 \&
566 \& my $compexcl = compexcl(0x09dc);
567 .Ve
568 .PP
569 This routine returns \f(CW\*(C`undef\*(C'\fR if the Unicode version being used is so early
570 that it doesn't have this property.
571 .PP
572 \&\f(CWcompexcl()\fR is included for backwards
573 compatibility, but as of Perl 5.12 and more modern Unicode versions, for
574 most purposes it is probably more convenient to use one of the following
575 instead:
576 .PP
577 .Vb 2
578 \& my $compexcl = chr(0x09dc) =~ /\ep{Comp_Ex};
579 \& my $compexcl = chr(0x09dc) =~ /\ep{Full_Composition_Exclusion};
580 .Ve
581 .PP
582 or even
583 .PP
584 .Vb 2
585 \& my $compexcl = chr(0x09dc) =~ /\ep{CE};
586 \& my $compexcl = chr(0x09dc) =~ /\ep{Composition_Exclusion};
587 .Ve
588 .PP
589 The first two forms return \fBtrue\fR if the "code point argument" should not
590 be produced by composition normalization. For the final two forms to return
591 \&\fBtrue\fR, it is additionally required that this fact not otherwise be
592 determinable from the Unicode data base.
593 .PP
594 This routine behaves identically to the final two forms. That is,
595 it does not return \fBtrue\fR if the code point has a decomposition
596 consisting of another single code point, nor if its decomposition starts
597 with a code point whose combining class is non-zero. Code points that meet
598 either of these conditions should also not be produced by composition
599 normalization, which is probably why you should use the
600 \&\f(CW\*(C`Full_Composition_Exclusion\*(C'\fR property instead, as shown above.
601 .PP
602 The routine returns \fBfalse\fR otherwise.
603 .SS \fBcasefold()\fP
604 .IX Subsection "casefold()"
605 .Vb 1
606 \& use Unicode::UCD \*(Aqcasefold\*(Aq;
607 \&
608 \& my $casefold = casefold(0xDF);
609 \& if (defined $casefold) {
610 \& my @full_fold_hex = split / /, $casefold\->{\*(Aqfull\*(Aq};
611 \& my $full_fold_string =
612 \& join "", map {chr(hex($_))} @full_fold_hex;
613 \& my @turkic_fold_hex =
614 \& split / /, ($casefold\->{\*(Aqturkic\*(Aq} ne "")
615 \& ? $casefold\->{\*(Aqturkic\*(Aq}
616 \& : $casefold\->{\*(Aqfull\*(Aq};
617 \& my $turkic_fold_string =
618 \& join "", map {chr(hex($_))} @turkic_fold_hex;
619 \& }
620 \& if (defined $casefold && $casefold\->{\*(Aqsimple\*(Aq} ne "") {
621 \& my $simple_fold_hex = $casefold\->{\*(Aqsimple\*(Aq};
622 \& my $simple_fold_string = chr(hex($simple_fold_hex));
623 \& }
624 .Ve
625 .PP
626 This returns the (almost) locale-independent case folding of the
627 character specified by the "code point argument". (Starting in Perl v5.16,
628 the core function \f(CWfc()\fR returns the \f(CW\*(C`full\*(C'\fR mapping (described below)
629 faster than this does, and for entire strings.)
630 .PP
631 If there is no case folding for the input code point, \f(CW\*(C`undef\*(C'\fR is returned.
632 .PP
633 If there is a case folding for that code point, a reference to a hash
634 with the following fields is returned:
635 .IP \fBcode\fR 4
636 .IX Item "code"
637 the input native "code point argument" expressed in hexadecimal, with
638 leading zeros
639 added if necessary to make it contain at least four hexdigits
640 .IP \fBfull\fR 4
641 .IX Item "full"
642 one or more codes (separated by spaces) that, taken in order, give the
643 code points for the case folding for \fIcode\fR.
644 Each has at least four hexdigits.
645 .IP \fBsimple\fR 4
646 .IX Item "simple"
647 is empty, or is exactly one code with at least four hexdigits which can be used
648 as an alternative case folding when the calling program cannot cope with the
649 fold being a sequence of multiple code points. If \fIfull\fR is just one code
650 point, then \fIsimple\fR equals \fIfull\fR. If there is no single code point folding
651 defined for \fIcode\fR, then \fIsimple\fR is the empty string. Otherwise, it is an
652 inferior, but still better-than-nothing alternative folding to \fIfull\fR.
653 .IP \fBmapping\fR 4
654 .IX Item "mapping"
655 is the same as \fIsimple\fR if \fIsimple\fR is not empty, and it is the same as \fIfull\fR
656 otherwise. It can be considered to be the simplest possible folding for
657 \&\fIcode\fR. It is defined primarily for backwards compatibility.
658 .IP \fBstatus\fR 4
659 .IX Item "status"
660 is \f(CW\*(C`C\*(C'\fR (for \f(CW\*(C`common\*(C'\fR) if the best possible fold is a single code point
661 (\fIsimple\fR equals \fIfull\fR equals \fImapping\fR). It is \f(CW\*(C`S\*(C'\fR if there are distinct
662 folds, \fIsimple\fR and \fIfull\fR (\fImapping\fR equals \fIsimple\fR). And it is \f(CW\*(C`F\*(C'\fR if
663 there is only a \fIfull\fR fold (\fImapping\fR equals \fIfull\fR; \fIsimple\fR is empty).
664 Note that this
665 describes the contents of \fImapping\fR. It is defined primarily for backwards
666 compatibility.
667 .Sp
668 For Unicode versions between 3.1 and 3.1.1 inclusive, \fIstatus\fR can also be
669 \&\f(CW\*(C`I\*(C'\fR which is the same as \f(CW\*(C`C\*(C'\fR but is a special case for dotted uppercase I and
670 dotless lowercase i:
671 .RS 4
672 .ie n .IP "\fB*\fR If you use this ""I"" mapping" 4
673 .el .IP "\fB*\fR If you use this \f(CWI\fR mapping" 4
674 .IX Item "* If you use this I mapping"
675 the result is case-insensitive,
676 but dotless and dotted I's are not distinguished
677 .ie n .IP "\fB*\fR If you exclude this ""I"" mapping" 4
678 .el .IP "\fB*\fR If you exclude this \f(CWI\fR mapping" 4
679 .IX Item "* If you exclude this I mapping"
680 the result is not fully case-insensitive, but
681 dotless and dotted I's are distinguished
682 .RE
683 .RS 4
684 .RE
685 .IP \fBturkic\fR 4
686 .IX Item "turkic"
687 contains any special folding for Turkic languages. For versions of Unicode
688 starting with 3.2, this field is empty unless \fIcode\fR has a different folding
689 in Turkic languages, in which case it is one or more codes (separated by
690 spaces) that, taken in order, give the code points for the case folding for
691 \&\fIcode\fR in those languages.
692 Each code has at least four hexdigits.
693 Note that this folding does not maintain canonical equivalence without
694 additional processing.
695 .Sp
696 For Unicode versions between 3.1 and 3.1.1 inclusive, this field is empty unless
697 there is a
698 special folding for Turkic languages, in which case \fIstatus\fR is \f(CW\*(C`I\*(C'\fR, and
699 \&\fImapping\fR, \fIfull\fR, \fIsimple\fR, and \fIturkic\fR are all equal.
700 .PP
701 Programs that want complete generality and the best folding results should use
702 the folding contained in the \fIfull\fR field. But note that the fold for some
703 code points will be a sequence of multiple code points.
704 .PP
705 Programs that can't cope with the fold mapping being multiple code points can
706 use the folding contained in the \fIsimple\fR field, with the loss of some
707 generality. In Unicode 5.1, about 7% of the defined foldings have no single
708 code point folding.
709 .PP
710 The \fImapping\fR and \fIstatus\fR fields are provided for backwards compatibility for
711 existing programs. They contain the same values as in previous versions of
712 this function.
713 .PP
714 Locale is not completely independent. The \fIturkic\fR field contains results to
715 use when the locale is a Turkic language.
716 .PP
717 For more information about case mappings see
718 <http://www.unicode.org/reports/tr21>
719 .SS \fBall_casefolds()\fP
720 .IX Subsection "all_casefolds()"
721 .Vb 1
722 \& use Unicode::UCD \*(Aqall_casefolds\*(Aq;
723 \&
724 \& my $all_folds_ref = all_casefolds();
725 \& foreach my $char_with_casefold (sort { $a <=> $b }
726 \& keys %$all_folds_ref)
727 \& {
728 \& printf "%04X:", $char_with_casefold;
729 \& my $casefold = $all_folds_ref\->{$char_with_casefold};
730 \&
731 \& # Get folds for $char_with_casefold
732 \&
733 \& my @full_fold_hex = split / /, $casefold\->{\*(Aqfull\*(Aq};
734 \& my $full_fold_string =
735 \& join "", map {chr(hex($_))} @full_fold_hex;
736 \& print " full=", join " ", @full_fold_hex;
737 \& my @turkic_fold_hex =
738 \& split / /, ($casefold\->{\*(Aqturkic\*(Aq} ne "")
739 \& ? $casefold\->{\*(Aqturkic\*(Aq}
740 \& : $casefold\->{\*(Aqfull\*(Aq};
741 \& my $turkic_fold_string =
742 \& join "", map {chr(hex($_))} @turkic_fold_hex;
743 \& print "; turkic=", join " ", @turkic_fold_hex;
744 \& if (defined $casefold && $casefold\->{\*(Aqsimple\*(Aq} ne "") {
745 \& my $simple_fold_hex = $casefold\->{\*(Aqsimple\*(Aq};
746 \& my $simple_fold_string = chr(hex($simple_fold_hex));
747 \& print "; simple=$simple_fold_hex";
748 \& }
749 \& print "\en";
750 \& }
751 .Ve
752 .PP
753 This returns all the case foldings in the current version of Unicode in the
754 form of a reference to a hash. Each key to the hash is the decimal
755 representation of a Unicode character that has a casefold to other than
756 itself. The casefold of a semi-colon is itself, so it isn't in the hash;
757 likewise for a lowercase "a", but there is an entry for a capital "A". The
758 hash value for each key is another hash, identical to what is returned by
759 "\fBcasefold()\fR" if called with that code point as its argument. So the value
760 \&\f(CW\*(C`all_casefolds()\->{ord("A")}\*(Aq\*(C'\fR is equivalent to \f(CW\*(C`casefold(ord("A"))\*(C'\fR;
761 .SS \fBcasespec()\fP
762 .IX Subsection "casespec()"
763 .Vb 1
764 \& use Unicode::UCD \*(Aqcasespec\*(Aq;
765 \&
766 \& my $casespec = casespec(0xFB00);
767 .Ve
768 .PP
769 This returns the potentially locale-dependent case mappings of the "code point
770 argument". The mappings may be longer than a single code point (which the basic
771 Unicode case mappings as returned by "\fBcharinfo()\fR" never are).
772 .PP
773 If there are no case mappings for the "code point argument", or if all three
774 possible mappings (\fIlower\fR, \fItitle\fR and \fIupper\fR) result in single code
775 points and are locale independent and unconditional, \f(CW\*(C`undef\*(C'\fR is returned
776 (which means that the case mappings, if any, for the code point are those
777 returned by "\fBcharinfo()\fR").
778 .PP
779 Otherwise, a reference to a hash giving the mappings (or a reference to a hash
780 of such hashes, explained below) is returned with the following keys and their
781 meanings:
782 .PP
783 The keys in the bottom layer hash with the meanings of their values are:
784 .IP \fBcode\fR 4
785 .IX Item "code"
786 the input native "code point argument" expressed in hexadecimal, with
787 leading zeros
788 added if necessary to make it contain at least four hexdigits
789 .IP \fBlower\fR 4
790 .IX Item "lower"
791 one or more codes (separated by spaces) that, taken in order, give the
792 code points for the lower case of \fIcode\fR.
793 Each has at least four hexdigits.
794 .IP \fBtitle\fR 4
795 .IX Item "title"
796 one or more codes (separated by spaces) that, taken in order, give the
797 code points for the title case of \fIcode\fR.
798 Each has at least four hexdigits.
799 .IP \fBupper\fR 4
800 .IX Item "upper"
801 one or more codes (separated by spaces) that, taken in order, give the
802 code points for the upper case of \fIcode\fR.
803 Each has at least four hexdigits.
804 .IP \fBcondition\fR 4
805 .IX Item "condition"
806 the conditions for the mappings to be valid.
807 If \f(CW\*(C`undef\*(C'\fR, the mappings are always valid.
808 When defined, this field is a list of conditions,
809 all of which must be true for the mappings to be valid.
810 The list consists of one or more
811 \&\fIlocales\fR (see below)
812 and/or \fIcontexts\fR (explained in the next paragraph),
813 separated by spaces.
814 (Other than as used to separate elements, spaces are to be ignored.)
815 Case distinctions in the condition list are not significant.
816 Conditions preceded by "NON_" represent the negation of the condition.
817 .Sp
818 A \fIcontext\fR is one of those defined in the Unicode standard.
819 For Unicode 5.1, they are defined in Section 3.13 \f(CW\*(C`Default Case Operations\*(C'\fR
820 available at
821 <http://www.unicode.org/versions/Unicode5.1.0/>.
822 These are for context-sensitive casing.
823 .PP
824 The hash described above is returned for locale-independent casing, where
825 at least one of the mappings has length longer than one. If \f(CW\*(C`undef\*(C'\fR is
826 returned, the code point may have mappings, but if so, all are length one,
827 and are returned by "\fBcharinfo()\fR".
828 Note that when this function does return a value, it will be for the complete
829 set of mappings for a code point, even those whose length is one.
830 .PP
831 If there are additional casing rules that apply only in certain locales,
832 an additional key for each will be defined in the returned hash. Each such key
833 will be its locale name, defined as a 2\-letter ISO 3166 country code, possibly
834 followed by a "_" and a 2\-letter ISO language code (possibly followed by a "_"
835 and a variant code). You can find the lists of all possible locales, see
836 Locale::Country and Locale::Language.
837 (In Unicode 6.0, the only locales returned by this function
838 are \f(CW\*(C`lt\*(C'\fR, \f(CW\*(C`tr\*(C'\fR, and \f(CW\*(C`az\*(C'\fR.)
839 .PP
840 Each locale key is a reference to a hash that has the form above, and gives
841 the casing rules for that particular locale, which take precedence over the
842 locale-independent ones when in that locale.
843 .PP
844 If the only casing for a code point is locale-dependent, then the returned
845 hash will not have any of the base keys, like \f(CW\*(C`code\*(C'\fR, \f(CW\*(C`upper\*(C'\fR, etc., but
846 will contain only locale keys.
847 .PP
848 For more information about case mappings see
849 <http://www.unicode.org/reports/tr21/>
850 .SS \fBnamedseq()\fP
851 .IX Subsection "namedseq()"
852 .Vb 1
853 \& use Unicode::UCD \*(Aqnamedseq\*(Aq;
854 \&
855 \& my $namedseq = namedseq("KATAKANA LETTER AINU P");
856 \& my @namedseq = namedseq("KATAKANA LETTER AINU P");
857 \& my %namedseq = namedseq();
858 .Ve
859 .PP
860 If used with a single argument in a scalar context, returns the string
861 consisting of the code points of the named sequence, or \f(CW\*(C`undef\*(C'\fR if no
862 named sequence by that name exists. If used with a single argument in
863 a list context, it returns the list of the ordinals of the code points.
864 .PP
865 If used with no
866 arguments in a list context, it returns a hash with the names of all the
867 named sequences as the keys and their sequences as strings as
868 the values. Otherwise, it returns \f(CW\*(C`undef\*(C'\fR or an empty list depending
869 on the context.
870 .PP
871 This function only operates on officially approved (not provisional) named
872 sequences.
873 .PP
874 Note that as of Perl 5.14, \f(CW\*(C`\eN{KATAKANA LETTER AINU P}\*(C'\fR will insert the named
875 sequence into double-quoted strings, and \f(CW\*(C`charnames::string_vianame("KATAKANA
876 LETTER AINU P")\*(C'\fR will return the same string this function does, but will also
877 operate on character names that aren't named sequences, without you having to
878 know which are which. See charnames.
879 .SS \fBnum()\fP
880 .IX Subsection "num()"
881 .Vb 1
882 \& use Unicode::UCD \*(Aqnum\*(Aq;
883 \&
884 \& my $val = num("123");
885 \& my $one_quarter = num("\eN{VULGAR FRACTION ONE QUARTER}");
886 \& my $val = num("12a", \e$valid_length); # $valid_length contains 2
887 .Ve
888 .PP
889 \&\f(CWnum()\fR returns the numeric value of the input Unicode string; or \f(CW\*(C`undef\*(C'\fR if it
890 doesn't think the entire string has a completely valid, safe numeric value.
891 If called with an optional second parameter, a reference to a scalar, \f(CWnum()\fR
892 will set the scalar to the length of any valid initial substring; or to 0 if none.
893 .PP
894 If the string is just one character in length, the Unicode numeric value
895 is returned if it has one, or \f(CW\*(C`undef\*(C'\fR otherwise. If the optional scalar ref
896 is passed, it would be set to 1 if the return is valid; or 0 if the return is
897 \&\f(CW\*(C`undef\*(C'\fR. Note that the numeric value returned need not be a whole number.
898 \&\f(CW\*(C`num("\eN{TIBETAN DIGIT HALF ZERO}")\*(C'\fR, for example returns \-0.5.
899 .PP
900 If the string is more than one character, \f(CW\*(C`undef\*(C'\fR is returned unless
901 all its characters are decimal digits (that is, they would match \f(CW\*(C`\ed+\*(C'\fR),
902 from the same script. For example if you have an ASCII '0' and a Bengali
903 \&'3', mixed together, they aren't considered a valid number, and \f(CW\*(C`undef\*(C'\fR
904 is returned. A further restriction is that the digits all have to be of
905 the same form. A half-width digit mixed with a full-width one will
906 return \f(CW\*(C`undef\*(C'\fR. The Arabic script has two sets of digits; \f(CW\*(C`num\*(C'\fR will
907 return \f(CW\*(C`undef\*(C'\fR unless all the digits in the string come from the same
908 set. In all cases, the optional scalar ref parameter is set to how
909 long any valid initial substring of digits is; hence it will be set to the
910 entire string length if the main return value is not \f(CW\*(C`undef\*(C'\fR.
911 .PP
912 \&\f(CW\*(C`num\*(C'\fR errs on the side of safety, and there may be valid strings of
913 decimal digits that it doesn't recognize. Note that Unicode defines
914 a number of "digit" characters that aren't "decimal digit" characters.
915 "Decimal digits" have the property that they have a positional value, i.e.,
916 there is a units position, a 10's position, a 100's, etc, AND they are
917 arranged in Unicode in blocks of 10 contiguous code points. The Chinese
918 digits, for example, are not in such a contiguous block, and so Unicode
919 doesn't view them as decimal digits, but merely digits, and so \f(CW\*(C`\ed\*(C'\fR will not
920 match them. A single-character string containing one of these digits will
921 have its decimal value returned by \f(CW\*(C`num\*(C'\fR, but any longer string containing
922 only these digits will return \f(CW\*(C`undef\*(C'\fR.
923 .PP
924 Strings of multiple sub\- and superscripts are not recognized as numbers. You
925 can use either of the compatibility decompositions in Unicode::Normalize to
926 change these into digits, and then call \f(CW\*(C`num\*(C'\fR on the result.
927 .SS \fBprop_aliases()\fP
928 .IX Subsection "prop_aliases()"
929 .Vb 1
930 \& use Unicode::UCD \*(Aqprop_aliases\*(Aq;
931 \&
932 \& my ($short_name, $full_name, @other_names) = prop_aliases("space");
933 \& my $same_full_name = prop_aliases("Space"); # Scalar context
934 \& my ($same_short_name) = prop_aliases("Space"); # gets 0th element
935 \& print "The full name is $full_name\en";
936 \& print "The short name is $short_name\en";
937 \& print "The other aliases are: ", join(", ", @other_names), "\en";
938 \&
939 \& prints:
940 \& The full name is White_Space
941 \& The short name is WSpace
942 \& The other aliases are: Space
943 .Ve
944 .PP
945 Most Unicode properties have several synonymous names. Typically, there is at
946 least a short name, convenient to type, and a long name that more fully
947 describes the property, and hence is more easily understood.
948 .PP
949 If you know one name for a Unicode property, you can use \f(CW\*(C`prop_aliases\*(C'\fR to find
950 either the long name (when called in scalar context), or a list of all of the
951 names, somewhat ordered so that the short name is in the 0th element, the long
952 name in the next element, and any other synonyms are in the remaining
953 elements, in no particular order.
954 .PP
955 The long name is returned in a form nicely capitalized, suitable for printing.
956 .PP
957 The input parameter name is loosely matched, which means that white space,
958 hyphens, and underscores are ignored (except for the trailing underscore in
959 the old_form grandfathered-in \f(CW"L_"\fR, which is better written as \f(CW"LC"\fR, and
960 both of which mean \f(CW\*(C`General_Category=Cased Letter\*(C'\fR).
961 .PP
962 If the name is unknown, \f(CW\*(C`undef\*(C'\fR is returned (or an empty list in list
963 context). Note that Perl typically recognizes property names in regular
964 expressions with an optional \f(CW\*(C`"Is_\*(C'\fR" (with or without the underscore)
965 prefixed to them, such as \f(CW\*(C`\ep{isgc=punct}\*(C'\fR. This function does not recognize
966 those in the input, returning \f(CW\*(C`undef\*(C'\fR. Nor are they included in the output
967 as possible synonyms.
968 .PP
969 \&\f(CW\*(C`prop_aliases\*(C'\fR does know about the Perl extensions to Unicode properties,
970 such as \f(CW\*(C`Any\*(C'\fR and \f(CW\*(C`XPosixAlpha\*(C'\fR, and the single form equivalents to Unicode
971 properties such as \f(CW\*(C`XDigit\*(C'\fR, \f(CW\*(C`Greek\*(C'\fR, \f(CW\*(C`In_Greek\*(C'\fR, and \f(CW\*(C`Is_Greek\*(C'\fR. The
972 final example demonstrates that the \f(CW"Is_"\fR prefix is recognized for these
973 extensions; it is needed to resolve ambiguities. For example,
974 \&\f(CWprop_aliases(\*(Aqlc\*(Aq)\fR returns the list \f(CW\*(C`(lc, Lowercase_Mapping)\*(C'\fR, but
975 \&\f(CWprop_aliases(\*(Aqislc\*(Aq)\fR returns \f(CW\*(C`(Is_LC, Cased_Letter)\*(C'\fR. This is
976 because \f(CW\*(C`islc\*(C'\fR is a Perl extension which is short for
977 \&\f(CW\*(C`General_Category=Cased Letter\*(C'\fR. The lists returned for the Perl extensions
978 will not include the \f(CW"Is_"\fR prefix (whether or not the input had it) unless
979 needed to resolve ambiguities, as shown in the \f(CW"islc"\fR example, where the
980 returned list had one element containing \f(CW"Is_"\fR, and the other without.
981 .PP
982 It is also possible for the reverse to happen: \f(CWprop_aliases(\*(Aqisc\*(Aq)\fR returns
983 the list \f(CW\*(C`(isc, ISO_Comment)\*(C'\fR; whereas \f(CWprop_aliases(\*(Aqc\*(Aq)\fR returns
984 \&\f(CW\*(C`(C, Other)\*(C'\fR (the latter being a Perl extension meaning
985 \&\f(CW\*(C`General_Category=Other\*(C'\fR.
986 "Properties accessible through Unicode::UCD" in perluniprops lists the available
987 forms, including which ones are discouraged from use.
988 .PP
989 Those discouraged forms are accepted as input to \f(CW\*(C`prop_aliases\*(C'\fR, but are not
990 returned in the lists. \f(CWprop_aliases(\*(AqisL&\*(Aq)\fR and \f(CWprop_aliases(\*(AqisL_\*(Aq)\fR,
991 which are old synonyms for \f(CW"Is_LC"\fR and should not be used in new code, are
992 examples of this. These both return \f(CW\*(C`(Is_LC, Cased_Letter)\*(C'\fR. Thus this
993 function allows you to take a discouraged form, and find its acceptable
994 alternatives. The same goes with single-form Block property equivalences.
995 Only the forms that begin with \f(CW"In_"\fR are not discouraged; if you pass
996 \&\f(CW\*(C`prop_aliases\*(C'\fR a discouraged form, you will get back the equivalent ones that
997 begin with \f(CW"In_"\fR. It will otherwise look like a new-style block name (see.
998 "Old-style versus new-style block names").
999 .PP
1000 \&\f(CW\*(C`prop_aliases\*(C'\fR does not know about any user-defined properties, and will
1001 return \f(CW\*(C`undef\*(C'\fR if called with one of those. Likewise for Perl internal
1002 properties, with the exception of "Perl_Decimal_Digit" which it does know
1003 about (and which is documented below in "\fBprop_invmap()\fR").
1004 .SS \fBprop_values()\fP
1005 .IX Subsection "prop_values()"
1006 .Vb 1
1007 \& use Unicode::UCD \*(Aqprop_values\*(Aq;
1008 \&
1009 \& print "AHex values are: ", join(", ", prop_values("AHex")),
1010 \& "\en";
1011 \& prints:
1012 \& AHex values are: N, Y
1013 .Ve
1014 .PP
1015 Some Unicode properties have a restricted set of legal values. For example,
1016 all binary properties are restricted to just \f(CW\*(C`true\*(C'\fR or \f(CW\*(C`false\*(C'\fR; and there
1017 are only a few dozen possible General Categories. Use \f(CW\*(C`prop_values\*(C'\fR
1018 to find out if a given property is one such, and if so, to get a list of the
1019 values:
1020 .PP
1021 .Vb 3
1022 \& print join ", ", prop_values("NFC_Quick_Check");
1023 \& prints:
1024 \& M, N, Y
1025 .Ve
1026 .PP
1027 If the property doesn't have such a restricted set, \f(CW\*(C`undef\*(C'\fR is returned.
1028 .PP
1029 There are usually several synonyms for each possible value. Use
1030 "\fBprop_value_aliases()\fR" to access those.
1031 .PP
1032 Case, white space, hyphens, and underscores are ignored in the input property
1033 name (except for the trailing underscore in the old-form grandfathered-in
1034 general category property value \f(CW"L_"\fR, which is better written as \f(CW"LC"\fR).
1035 .PP
1036 If the property name is unknown, \f(CW\*(C`undef\*(C'\fR is returned. Note that Perl typically
1037 recognizes property names in regular expressions with an optional \f(CW\*(C`"Is_\*(C'\fR"
1038 (with or without the underscore) prefixed to them, such as \f(CW\*(C`\ep{isgc=punct}\*(C'\fR.
1039 This function does not recognize those in the property parameter, returning
1040 \&\f(CW\*(C`undef\*(C'\fR.
1041 .PP
1042 For the block property, new-style block names are returned (see
1043 "Old-style versus new-style block names").
1044 .PP
1045 \&\f(CW\*(C`prop_values\*(C'\fR does not know about any user-defined properties, and
1046 will return \f(CW\*(C`undef\*(C'\fR if called with one of those.
1047 .SS \fBprop_value_aliases()\fP
1048 .IX Subsection "prop_value_aliases()"
1049 .Vb 1
1050 \& use Unicode::UCD \*(Aqprop_value_aliases\*(Aq;
1051 \&
1052 \& my ($short_name, $full_name, @other_names)
1053 \& = prop_value_aliases("Gc", "Punct");
1054 \& my $same_full_name = prop_value_aliases("Gc", "P"); # Scalar cntxt
1055 \& my ($same_short_name) = prop_value_aliases("Gc", "P"); # gets 0th
1056 \& # element
1057 \& print "The full name is $full_name\en";
1058 \& print "The short name is $short_name\en";
1059 \& print "The other aliases are: ", join(", ", @other_names), "\en";
1060 \&
1061 \& prints:
1062 \& The full name is Punctuation
1063 \& The short name is P
1064 \& The other aliases are: Punct
1065 .Ve
1066 .PP
1067 Some Unicode properties have a restricted set of legal values. For example,
1068 all binary properties are restricted to just \f(CW\*(C`true\*(C'\fR or \f(CW\*(C`false\*(C'\fR; and there
1069 are only a few dozen possible General Categories.
1070 .PP
1071 You can use "\fBprop_values()\fR" to find out if a given property is one which has
1072 a restricted set of values, and if so, what those values are. But usually
1073 each value actually has several synonyms. For example, in Unicode binary
1074 properties, \fItruth\fR can be represented by any of the strings "Y", "Yes", "T",
1075 or "True"; and the General Category "Punctuation" by that string, or "Punct",
1076 or simply "P".
1077 .PP
1078 Like property names, there is typically at least a short name for each such
1079 property-value, and a long name. If you know any name of the property-value
1080 (which you can get by "\fBprop_values()\fR", you can use \f(CW\*(C`prop_value_aliases\*(C'\fR()
1081 to get the long name (when called in scalar context), or a list of all the
1082 names, with the short name in the 0th element, the long name in the next
1083 element, and any other synonyms in the remaining elements, in no particular
1084 order, except that any all-numeric synonyms will be last.
1085 .PP
1086 The long name is returned in a form nicely capitalized, suitable for printing.
1087 .PP
1088 Case, white space, hyphens, and underscores are ignored in the input parameters
1089 (except for the trailing underscore in the old-form grandfathered-in general
1090 category property value \f(CW"L_"\fR, which is better written as \f(CW"LC"\fR).
1091 .PP
1092 If either name is unknown, \f(CW\*(C`undef\*(C'\fR is returned. Note that Perl typically
1093 recognizes property names in regular expressions with an optional \f(CW\*(C`"Is_\*(C'\fR"
1094 (with or without the underscore) prefixed to them, such as \f(CW\*(C`\ep{isgc=punct}\*(C'\fR.
1095 This function does not recognize those in the property parameter, returning
1096 \&\f(CW\*(C`undef\*(C'\fR.
1097 .PP
1098 If called with a property that doesn't have synonyms for its values, it
1099 returns the input value, possibly normalized with capitalization and
1100 underscores, but not necessarily checking that the input value is valid.
1101 .PP
1102 For the block property, new-style block names are returned (see
1103 "Old-style versus new-style block names").
1104 .PP
1105 To find the synonyms for single-forms, such as \f(CW\*(C`\ep{Any}\*(C'\fR, use
1106 "\fBprop_aliases()\fR" instead.
1107 .PP
1108 \&\f(CW\*(C`prop_value_aliases\*(C'\fR does not know about any user-defined properties, and
1109 will return \f(CW\*(C`undef\*(C'\fR if called with one of those.
1110 .SS \fBprop_invlist()\fP
1111 .IX Subsection "prop_invlist()"
1112 \&\f(CW\*(C`prop_invlist\*(C'\fR returns an inversion list (described below) that defines all the
1113 code points for the binary Unicode property (or "property=value" pair) given
1114 by the input parameter string:
1115 .PP
1116 .Vb 3
1117 \& use feature \*(Aqsay\*(Aq;
1118 \& use Unicode::UCD \*(Aqprop_invlist\*(Aq;
1119 \& say join ", ", prop_invlist("Any");
1120 \&
1121 \& prints:
1122 \& 0, 1114112
1123 .Ve
1124 .PP
1125 If the input is unknown \f(CW\*(C`undef\*(C'\fR is returned in scalar context; an empty-list
1126 in list context. If the input is known, the number of elements in
1127 the list is returned if called in scalar context.
1128 .PP
1129 perluniprops gives
1130 the list of properties that this function accepts, as well as all the possible
1131 forms for them (including with the optional "Is_" prefixes). (Except this
1132 function doesn't accept any Perl-internal properties, some of which are listed
1133 there.) This function uses the same loose or tighter matching rules for
1134 resolving the input property's name as is done for regular expressions. These
1135 are also specified in perluniprops. Examples of using the "property=value" form are:
1136 .PP
1137 .Vb 1
1138 \& say join ", ", prop_invlist("Script_Extensions=Shavian");
1139 \&
1140 \& prints:
1141 \& 66640, 66688
1142 \&
1143 \& say join ", ", prop_invlist("ASCII_Hex_Digit=No");
1144 \&
1145 \& prints:
1146 \& 0, 48, 58, 65, 71, 97, 103
1147 \&
1148 \& say join ", ", prop_invlist("ASCII_Hex_Digit=Yes");
1149 \&
1150 \& prints:
1151 \& 48, 58, 65, 71, 97, 103
1152 .Ve
1153 .PP
1154 Inversion lists are a compact way of specifying Unicode property-value
1155 definitions. The 0th item in the list is the lowest code point that has the
1156 property-value. The next item (item [1]) is the lowest code point beyond that
1157 one that does NOT have the property-value. And the next item beyond that
1158 ([2]) is the lowest code point beyond that one that does have the
1159 property-value, and so on. Put another way, each element in the list gives
1160 the beginning of a range that has the property-value (for even numbered
1161 elements), or doesn't have the property-value (for odd numbered elements).
1162 The name for this data structure stems from the fact that each element in the
1163 list toggles (or inverts) whether the corresponding range is or isn't on the
1164 list.
1165 .PP
1166 In the final example above, the first ASCII Hex digit is code point 48, the
1167 character "0", and all code points from it through 57 (a "9") are ASCII hex
1168 digits. Code points 58 through 64 aren't, but 65 (an "A") through 70 (an "F")
1169 are, as are 97 ("a") through 102 ("f"). 103 starts a range of code points
1170 that aren't ASCII hex digits. That range extends to infinity, which on your
1171 computer can be found in the variable \f(CW$Unicode::UCD::MAX_CP\fR. (This
1172 variable is as close to infinity as Perl can get on your platform, and may be
1173 too high for some operations to work; you may wish to use a smaller number for
1174 your purposes.)
1175 .PP
1176 Note that the inversion lists returned by this function can possibly include
1177 non-Unicode code points, that is anything above 0x10FFFF. Unicode properties
1178 are not defined on such code points. You might wish to change the output to
1179 not include these. Simply add 0x110000 at the end of the non-empty returned
1180 list if it isn't already that value; and pop that value if it is; like:
1181 .PP
1182 .Vb 9
1183 \& my @list = prop_invlist("foo");
1184 \& if (@list) {
1185 \& if ($list[\-1] == 0x110000) {
1186 \& pop @list; # Defeat the turning on for above Unicode
1187 \& }
1188 \& else {
1189 \& push @list, 0x110000; # Turn off for above Unicode
1190 \& }
1191 \& }
1192 .Ve
1193 .PP
1194 It is a simple matter to expand out an inversion list to a full list of all
1195 code points that have the property-value:
1196 .PP
1197 .Vb 11
1198 \& my @invlist = prop_invlist($property_name);
1199 \& die "empty" unless @invlist;
1200 \& my @full_list;
1201 \& for (my $i = 0; $i < @invlist; $i += 2) {
1202 \& my $upper = ($i + 1) < @invlist
1203 \& ? $invlist[$i+1] \- 1 # In range
1204 \& : $Unicode::UCD::MAX_CP; # To infinity.
1205 \& for my $j ($invlist[$i] .. $upper) {
1206 \& push @full_list, $j;
1207 \& }
1208 \& }
1209 .Ve
1210 .PP
1211 \&\f(CW\*(C`prop_invlist\*(C'\fR does not know about any user-defined nor Perl internal-only
1212 properties, and will return \f(CW\*(C`undef\*(C'\fR if called with one of those.
1213 .PP
1214 The "\fBsearch_invlist()\fR" function is provided for finding a code point within
1215 an inversion list.
1216 .SS \fBprop_invmap()\fP
1217 .IX Subsection "prop_invmap()"
1218 .Vb 3
1219 \& use Unicode::UCD \*(Aqprop_invmap\*(Aq;
1220 \& my ($list_ref, $map_ref, $format, $default)
1221 \& = prop_invmap("General Category");
1222 .Ve
1223 .PP
1224 \&\f(CW\*(C`prop_invmap\*(C'\fR is used to get the complete mapping definition for a property,
1225 in the form of an inversion map. An inversion map consists of two parallel
1226 arrays. One is an ordered list of code points that mark range beginnings, and
1227 the other gives the value (or mapping) that all code points in the
1228 corresponding range have.
1229 .PP
1230 \&\f(CW\*(C`prop_invmap\*(C'\fR is called with the name of the desired property. The name is
1231 loosely matched, meaning that differences in case, white-space, hyphens, and
1232 underscores are not meaningful (except for the trailing underscore in the
1233 old-form grandfathered-in property \f(CW"L_"\fR, which is better written as \f(CW"LC"\fR,
1234 or even better, \f(CW"Gc=LC"\fR).
1235 .PP
1236 Many Unicode properties have more than one name (or alias). \f(CW\*(C`prop_invmap\*(C'\fR
1237 understands all of these, including Perl extensions to them. Ambiguities are
1238 resolved as described above for "\fBprop_aliases()\fR" (except if a property has
1239 both a complete mapping, and a binary \f(CW\*(C`Y\*(C'\fR/\f(CW\*(C`N\*(C'\fR mapping, then specifying the
1240 property name prefixed by \f(CW"is"\fR causes the binary one to be returned). The
1241 Perl internal property "Perl_Decimal_Digit, described below, is also accepted.
1242 An empty list is returned if the property name is unknown.
1243 See "Properties accessible through Unicode::UCD" in perluniprops for the
1244 properties acceptable as inputs to this function.
1245 .PP
1246 It is a fatal error to call this function except in list context.
1247 .PP
1248 In addition to the two arrays that form the inversion map, \f(CW\*(C`prop_invmap\*(C'\fR
1249 returns two other values; one is a scalar that gives some details as to the
1250 format of the entries of the map array; the other is a default value, useful
1251 in maps whose format name begins with the letter \f(CW"a"\fR, as described
1252 below in its subsection; and for specialized purposes, such as
1253 converting to another data structure, described at the end of this main
1254 section.
1255 .PP
1256 This means that \f(CW\*(C`prop_invmap\*(C'\fR returns a 4 element list. For example,
1257 .PP
1258 .Vb 2
1259 \& my ($blocks_ranges_ref, $blocks_maps_ref, $format, $default)
1260 \& = prop_invmap("Block");
1261 .Ve
1262 .PP
1263 In this call, the two arrays will be populated as shown below (for Unicode
1264 6.0):
1265 .PP
1266 .Vb 10
1267 \& Index @blocks_ranges @blocks_maps
1268 \& 0 0x0000 Basic Latin
1269 \& 1 0x0080 Latin\-1 Supplement
1270 \& 2 0x0100 Latin Extended\-A
1271 \& 3 0x0180 Latin Extended\-B
1272 \& 4 0x0250 IPA Extensions
1273 \& 5 0x02B0 Spacing Modifier Letters
1274 \& 6 0x0300 Combining Diacritical Marks
1275 \& 7 0x0370 Greek and Coptic
1276 \& 8 0x0400 Cyrillic
1277 \& ...
1278 \& 233 0x2B820 No_Block
1279 \& 234 0x2F800 CJK Compatibility Ideographs Supplement
1280 \& 235 0x2FA20 No_Block
1281 \& 236 0xE0000 Tags
1282 \& 237 0xE0080 No_Block
1283 \& 238 0xE0100 Variation Selectors Supplement
1284 \& 239 0xE01F0 No_Block
1285 \& 240 0xF0000 Supplementary Private Use Area\-A
1286 \& 241 0x100000 Supplementary Private Use Area\-B
1287 \& 242 0x110000 No_Block
1288 .Ve
1289 .PP
1290 The first line (with Index [0]) means that the value for code point 0 is "Basic
1291 Latin". The entry "0x0080" in the \f(CW@blocks_ranges\fR column in the second line
1292 means that the value from the first line, "Basic Latin", extends to all code
1293 points in the range from 0 up to but not including 0x0080, that is, through
1294 127. In other words, the code points from 0 to 127 are all in the "Basic
1295 Latin" block. Similarly, all code points in the range from 0x0080 up to (but
1296 not including) 0x0100 are in the block named "Latin\-1 Supplement", etc.
1297 (Notice that the return is the old-style block names; see "Old-style versus
1298 new-style block names").
1299 .PP
1300 The final line (with Index [242]) means that the value for all code points above
1301 the legal Unicode maximum code point have the value "No_Block", which is the
1302 term Unicode uses for a non-existing block.
1303 .PP
1304 The arrays completely specify the mappings for all possible code points.
1305 The final element in an inversion map returned by this function will always be
1306 for the range that consists of all the code points that aren't legal Unicode,
1307 but that are expressible on the platform. (That is, it starts with code point
1308 0x110000, the first code point above the legal Unicode maximum, and extends to
1309 infinity.) The value for that range will be the same that any typical
1310 unassigned code point has for the specified property. (Certain unassigned
1311 code points are not "typical"; for example the non-character code points, or
1312 those in blocks that are to be written right-to-left. The above-Unicode
1313 range's value is not based on these atypical code points.) It could be argued
1314 that, instead of treating these as unassigned Unicode code points, the value
1315 for this range should be \f(CW\*(C`undef\*(C'\fR. If you wish, you can change the returned
1316 arrays accordingly.
1317 .PP
1318 The maps for almost all properties are simple scalars that should be
1319 interpreted as-is.
1320 These values are those given in the Unicode-supplied data files, which may be
1321 inconsistent as to capitalization and as to which synonym for a property-value
1322 is given. The results may be normalized by using the "\fBprop_value_aliases()\fR"
1323 function.
1324 .PP
1325 There are exceptions to the simple scalar maps. Some properties have some
1326 elements in their map list that are themselves lists of scalars; and some
1327 special strings are returned that are not to be interpreted as-is. Element
1328 [2] (placed into \f(CW$format\fR in the example above) of the returned four element
1329 list tells you if the map has any of these special elements or not, as follows:
1330 .ie n .IP "\fR\fB""s""\fR\fB\fR" 4
1331 .el .IP \fR\f(CBs\fR\fB\fR 4
1332 .IX Item "s"
1333 means all the elements of the map array are simple scalars, with no special
1334 elements. Almost all properties are like this, like the \f(CW\*(C`block\*(C'\fR example
1335 above.
1336 .ie n .IP "\fR\fB""sl""\fR\fB\fR" 4
1337 .el .IP \fR\f(CBsl\fR\fB\fR 4
1338 .IX Item "sl"
1339 means that some of the map array elements have the form given by \f(CW"s"\fR, and
1340 the rest are lists of scalars. For example, here is a portion of the output
1341 of calling \f(CW\*(C`prop_invmap\*(C'\fR() with the "Script Extensions" property:
1342 .Sp
1343 .Vb 6
1344 \& @scripts_ranges @scripts_maps
1345 \& ...
1346 \& 0x0953 Devanagari
1347 \& 0x0964 [ Bengali, Devanagari, Gurumukhi, Oriya ]
1348 \& 0x0966 Devanagari
1349 \& 0x0970 Common
1350 .Ve
1351 .Sp
1352 Here, the code points 0x964 and 0x965 are both used in Bengali,
1353 Devanagari, Gurmukhi, and Oriya, but no other scripts.
1354 .Sp
1355 The Name_Alias property is also of this form. But each scalar consists of two
1356 components: 1) the name, and 2) the type of alias this is. They are
1357 separated by a colon and a space. In Unicode 6.1, there are several alias types:
1358 .RS 4
1359 .ie n .IP """correction""" 4
1360 .el .IP \f(CWcorrection\fR 4
1361 .IX Item "correction"
1362 indicates that the name is a corrected form for the
1363 original name (which remains valid) for the same code point.
1364 .ie n .IP """control""" 4
1365 .el .IP \f(CWcontrol\fR 4
1366 .IX Item "control"
1367 adds a new name for a control character.
1368 .ie n .IP """alternate""" 4
1369 .el .IP \f(CWalternate\fR 4
1370 .IX Item "alternate"
1371 is an alternate name for a character
1372 .ie n .IP """figment""" 4
1373 .el .IP \f(CWfigment\fR 4
1374 .IX Item "figment"
1375 is a name for a character that has been documented but was never in any
1376 actual standard.
1377 .ie n .IP """abbreviation""" 4
1378 .el .IP \f(CWabbreviation\fR 4
1379 .IX Item "abbreviation"
1380 is a common abbreviation for a character
1381 .RE
1382 .RS 4
1383 .Sp
1384 The lists are ordered (roughly) so the most preferred names come before less
1385 preferred ones.
1386 .Sp
1387 For example,
1388 .Sp
1389 .Vb 10
1390 \& @aliases_ranges @alias_maps
1391 \& ...
1392 \& 0x009E [ \*(AqPRIVACY MESSAGE: control\*(Aq, \*(AqPM: abbreviation\*(Aq ]
1393 \& 0x009F [ \*(AqAPPLICATION PROGRAM COMMAND: control\*(Aq,
1394 \& \*(AqAPC: abbreviation\*(Aq
1395 \& ]
1396 \& 0x00A0 \*(AqNBSP: abbreviation\*(Aq
1397 \& 0x00A1 ""
1398 \& 0x00AD \*(AqSHY: abbreviation\*(Aq
1399 \& 0x00AE ""
1400 \& 0x01A2 \*(AqLATIN CAPITAL LETTER GHA: correction\*(Aq
1401 \& 0x01A3 \*(AqLATIN SMALL LETTER GHA: correction\*(Aq
1402 \& 0x01A4 ""
1403 \& ...
1404 .Ve
1405 .Sp
1406 A map to the empty string means that there is no alias defined for the code
1407 point.
1408 .RE
1409 .ie n .IP "\fR\fB""a""\fR\fB\fR" 4
1410 .el .IP \fR\f(CBa\fR\fB\fR 4
1411 .IX Item "a"
1412 is like \f(CW"s"\fR in that all the map array elements are scalars, but here they are
1413 restricted to all being integers, and some have to be adjusted (hence the name
1414 \&\f(CW"a"\fR) to get the correct result. For example, in:
1415 .Sp
1416 .Vb 2
1417 \& my ($uppers_ranges_ref, $uppers_maps_ref, $format, $default)
1418 \& = prop_invmap("Simple_Uppercase_Mapping");
1419 .Ve
1420 .Sp
1421 the returned arrays look like this:
1422 .Sp
1423 .Vb 7
1424 \& @$uppers_ranges_ref @$uppers_maps_ref Note
1425 \& 0 0
1426 \& 97 65 \*(Aqa\*(Aq maps to \*(AqA\*(Aq, b => B ...
1427 \& 123 0
1428 \& 181 924 MICRO SIGN => Greek Cap MU
1429 \& 182 0
1430 \& ...
1431 .Ve
1432 .Sp
1433 and \f(CW$default\fR is 0.
1434 .Sp
1435 Let's start with the second line. It says that the uppercase of code point 97
1436 is 65; or \f(CWuc("a")\fR == "A". But the line is for the entire range of code
1437 points 97 through 122. To get the mapping for any code point in this range,
1438 you take the offset it has from the beginning code point of the range, and add
1439 that to the mapping for that first code point. So, the mapping for 122 ("z")
1440 is derived by taking the offset of 122 from 97 (=25) and adding that to 65,
1441 yielding 90 ("Z"). Likewise for everything in between.
1442 .Sp
1443 Requiring this simple adjustment allows the returned arrays to be
1444 significantly smaller than otherwise, up to a factor of 10, speeding up
1445 searching through them.
1446 .Sp
1447 Ranges that map to \f(CW$default\fR, \f(CW"0"\fR, behave somewhat differently. For
1448 these, each code point maps to itself. So, in the first line in the example,
1449 \&\f(CW\*(C`ord(uc(chr(0)))\*(C'\fR is 0, \f(CW\*(C`ord(uc(chr(1)))\*(C'\fR is 1, ..
1450 \&\f(CW\*(C`ord(uc(chr(96)))\*(C'\fR is 96.
1451 .ie n .IP "\fR\fB""al""\fR\fB\fR" 4
1452 .el .IP \fR\f(CBal\fR\fB\fR 4
1453 .IX Item "al"
1454 means that some of the map array elements have the form given by \f(CW"a"\fR, and
1455 the rest are ordered lists of code points.
1456 For example, in:
1457 .Sp
1458 .Vb 2
1459 \& my ($uppers_ranges_ref, $uppers_maps_ref, $format, $default)
1460 \& = prop_invmap("Uppercase_Mapping");
1461 .Ve
1462 .Sp
1463 the returned arrays look like this:
1464 .Sp
1465 .Vb 11
1466 \& @$uppers_ranges_ref @$uppers_maps_ref
1467 \& 0 0
1468 \& 97 65
1469 \& 123 0
1470 \& 181 924
1471 \& 182 0
1472 \& ...
1473 \& 0x0149 [ 0x02BC 0x004E ]
1474 \& 0x014A 0
1475 \& 0x014B 330
1476 \& ...
1477 .Ve
1478 .Sp
1479 This is the full Uppercase_Mapping property (as opposed to the
1480 Simple_Uppercase_Mapping given in the example for format \f(CW"a"\fR). The only
1481 difference between the two in the ranges shown is that the code point at
1482 0x0149 (LATIN SMALL LETTER N PRECEDED BY APOSTROPHE) maps to a string of two
1483 characters, 0x02BC (MODIFIER LETTER APOSTROPHE) followed by 0x004E (LATIN
1484 CAPITAL LETTER N).
1485 .Sp
1486 No adjustments are needed to entries that are references to arrays; each such
1487 entry will have exactly one element in its range, so the offset is always 0.
1488 .Sp
1489 The fourth (index [3]) element (\f(CW$default\fR) in the list returned for this
1490 format is 0.
1491 .ie n .IP "\fR\fB""ae""\fR\fB\fR" 4
1492 .el .IP \fR\f(CBae\fR\fB\fR 4
1493 .IX Item "ae"
1494 This is like \f(CW"a"\fR, but some elements are the empty string, and should not be
1495 adjusted.
1496 The one internal Perl property accessible by \f(CW\*(C`prop_invmap\*(C'\fR is of this type:
1497 "Perl_Decimal_Digit" returns an inversion map which gives the numeric values
1498 that are represented by the Unicode decimal digit characters. Characters that
1499 don't represent decimal digits map to the empty string, like so:
1500 .Sp
1501 .Vb 12
1502 \& @digits @values
1503 \& 0x0000 ""
1504 \& 0x0030 0
1505 \& 0x003A: ""
1506 \& 0x0660: 0
1507 \& 0x066A: ""
1508 \& 0x06F0: 0
1509 \& 0x06FA: ""
1510 \& 0x07C0: 0
1511 \& 0x07CA: ""
1512 \& 0x0966: 0
1513 \& ...
1514 .Ve
1515 .Sp
1516 This means that the code points from 0 to 0x2F do not represent decimal digits;
1517 the code point 0x30 (DIGIT ZERO) represents 0; code point 0x31, (DIGIT ONE),
1518 represents 0+1\-0 = 1; ... code point 0x39, (DIGIT NINE), represents 0+9\-0 = 9;
1519 \&... code points 0x3A through 0x65F do not represent decimal digits; 0x660
1520 (ARABIC-INDIC DIGIT ZERO), represents 0; ... 0x07C1 (NKO DIGIT ONE),
1521 represents 0+1\-0 = 1 ...
1522 .Sp
1523 The fourth (index [3]) element (\f(CW$default\fR) in the list returned for this
1524 format is the empty string.
1525 .ie n .IP "\fR\fB""ale""\fR\fB\fR" 4
1526 .el .IP \fR\f(CBale\fR\fB\fR 4
1527 .IX Item "ale"
1528 is a combination of the \f(CW"al"\fR type and the \f(CW"ae"\fR type. Some of
1529 the map array elements have the forms given by \f(CW"al"\fR, and
1530 the rest are the empty string. The property \f(CW\*(C`NFKC_Casefold\*(C'\fR has this form.
1531 An example slice is:
1532 .Sp
1533 .Vb 9
1534 \& @$ranges_ref @$maps_ref Note
1535 \& ...
1536 \& 0x00AA 97 FEMININE ORDINAL INDICATOR => \*(Aqa\*(Aq
1537 \& 0x00AB 0
1538 \& 0x00AD SOFT HYPHEN => ""
1539 \& 0x00AE 0
1540 \& 0x00AF [ 0x0020, 0x0304 ] MACRON => SPACE . COMBINING MACRON
1541 \& 0x00B0 0
1542 \& ...
1543 .Ve
1544 .Sp
1545 The fourth (index [3]) element (\f(CW$default\fR) in the list returned for this
1546 format is 0.
1547 .ie n .IP "\fR\fB""ar""\fR\fB\fR" 4
1548 .el .IP \fR\f(CBar\fR\fB\fR 4
1549 .IX Item "ar"
1550 means that all the elements of the map array are either rational numbers or
1551 the string \f(CW"NaN"\fR, meaning "Not a Number". A rational number is either an
1552 integer, or two integers separated by a solidus (\f(CW"/"\fR). The second integer
1553 represents the denominator of the division implied by the solidus, and is
1554 actually always positive, so it is guaranteed not to be 0 and to not be
1555 signed. When the element is a plain integer (without the
1556 solidus), it may need to be adjusted to get the correct value by adding the
1557 offset, just as other \f(CW"a"\fR properties. No adjustment is needed for
1558 fractions, as the range is guaranteed to have just a single element, and so
1559 the offset is always 0.
1560 .Sp
1561 If you want to convert the returned map to entirely scalar numbers, you
1562 can use something like this:
1563 .Sp
1564 .Vb 4
1565 \& my ($invlist_ref, $invmap_ref, $format) = prop_invmap($property);
1566 \& if ($format && $format eq "ar") {
1567 \& map { $_ = eval $_ if $_ ne \*(AqNaN\*(Aq } @$map_ref;
1568 \& }
1569 .Ve
1570 .Sp
1571 Here's some entries from the output of the property "Nv", which has format
1572 \&\f(CW"ar"\fR.
1573 .Sp
1574 .Vb 10
1575 \& @numerics_ranges @numerics_maps Note
1576 \& 0x00 "NaN"
1577 \& 0x30 0 DIGIT 0 .. DIGIT 9
1578 \& 0x3A "NaN"
1579 \& 0xB2 2 SUPERSCRIPTs 2 and 3
1580 \& 0xB4 "NaN"
1581 \& 0xB9 1 SUPERSCRIPT 1
1582 \& 0xBA "NaN"
1583 \& 0xBC 1/4 VULGAR FRACTION 1/4
1584 \& 0xBD 1/2 VULGAR FRACTION 1/2
1585 \& 0xBE 3/4 VULGAR FRACTION 3/4
1586 \& 0xBF "NaN"
1587 \& 0x660 0 ARABIC\-INDIC DIGIT ZERO .. NINE
1588 \& 0x66A "NaN"
1589 .Ve
1590 .Sp
1591 The fourth (index [3]) element (\f(CW$default\fR) in the list returned for this
1592 format is \f(CW"NaN"\fR.
1593 .ie n .IP "\fR\fB""n""\fR\fB\fR" 4
1594 .el .IP \fR\f(CBn\fR\fB\fR 4
1595 .IX Item "n"
1596 means the Name property. All the elements of the map array are simple
1597 scalars, but some of them contain special strings that require more work to
1598 get the actual name.
1599 .Sp
1600 Entries such as:
1601 .Sp
1602 .Vb 1
1603 \& CJK UNIFIED IDEOGRAPH\-<code point>
1604 .Ve
1605 .Sp
1606 mean that the name for the code point is "CJK UNIFIED IDEOGRAPH\-"
1607 with the code point (expressed in hexadecimal) appended to it, like "CJK
1608 UNIFIED IDEOGRAPH\-3403" (similarly for \f(CW\*(C`CJK\ COMPATIBILITY\ IDEOGRAPH\-<code\ point>\*(C'\fR).
1609 .Sp
1610 Also, entries like
1611 .Sp
1612 .Vb 1
1613 \& <hangul syllable>
1614 .Ve
1615 .Sp
1616 means that the name is algorithmically calculated. This is easily done by
1617 the function "charnames::viacode(code)" in charnames.
1618 .Sp
1619 Note that for control characters (\f(CW\*(C`Gc=cc\*(C'\fR), Unicode's data files have the
1620 string "\f(CW\*(C`<control>\*(C'\fR", but the real name of each of these characters is the empty
1621 string. This function returns that real name, the empty string. (There are
1622 names for these characters, but they are considered aliases, not the Name
1623 property name, and are contained in the \f(CW\*(C`Name_Alias\*(C'\fR property.)
1624 .ie n .IP "\fR\fB""ad""\fR\fB\fR" 4
1625 .el .IP \fR\f(CBad\fR\fB\fR 4
1626 .IX Item "ad"
1627 means the Decomposition_Mapping property. This property is like \f(CW"al"\fR
1628 properties, except that one of the scalar elements is of the form:
1629 .Sp
1630 .Vb 1
1631 \& <hangul syllable>
1632 .Ve
1633 .Sp
1634 This signifies that this entry should be replaced by the decompositions for
1635 all the code points whose decomposition is algorithmically calculated. (All
1636 of them are currently in one range and no others outside the range are likely
1637 to ever be added to Unicode; the \f(CW"n"\fR format
1638 has this same entry.) These can be generated via the function
1639 \&\fBUnicode::Normalize::NFD()\fR.
1640 .Sp
1641 Note that the mapping is the one that is specified in the Unicode data files,
1642 and to get the final decomposition, it may need to be applied recursively.
1643 Unicode in fact discourages use of this property except internally in
1644 implementations of the Unicode Normalization Algorithm.
1645 .Sp
1646 The fourth (index [3]) element (\f(CW$default\fR) in the list returned for this
1647 format is 0.
1648 .PP
1649 Note that a format begins with the letter "a" if and only the property it is
1650 for requires adjustments by adding the offsets in multi-element ranges. For
1651 all these properties, an entry should be adjusted only if the map is a scalar
1652 which is an integer. That is, it must match the regular expression:
1653 .PP
1654 .Vb 1
1655 \& / ^ \-? \ed+ $ /xa
1656 .Ve
1657 .PP
1658 Further, the first element in a range never needs adjustment, as the
1659 adjustment would be just adding 0.
1660 .PP
1661 A binary search such as that provided by "\fBsearch_invlist()\fR", can be used to
1662 quickly find a code point in the inversion list, and hence its corresponding
1663 mapping.
1664 .PP
1665 The final, fourth element (index [3], assigned to \f(CW$default\fR in the "block"
1666 example) in the four element list returned by this function is used with the
1667 \&\f(CW"a"\fR format types; it may also be useful for applications
1668 that wish to convert the returned inversion map data structure into some
1669 other, such as a hash. It gives the mapping that most code points map to
1670 under the property. If you establish the convention that any code point not
1671 explicitly listed in your data structure maps to this value, you can
1672 potentially make your data structure much smaller. As you construct your data
1673 structure from the one returned by this function, simply ignore those ranges
1674 that map to this value. For example, to
1675 convert to the data structure searchable by "\fBcharinrange()\fR", you can follow
1676 this recipe for properties that don't require adjustments:
1677 .PP
1678 .Vb 2
1679 \& my ($list_ref, $map_ref, $format, $default) = prop_invmap($property);
1680 \& my @range_list;
1681 \&
1682 \& # Look at each element in the list, but the \-2 is needed because we
1683 \& # look at $i+1 in the loop, and the final element is guaranteed to map
1684 \& # to $default by prop_invmap(), so we would skip it anyway.
1685 \& for my $i (0 .. @$list_ref \- 2) {
1686 \& next if $map_ref\->[$i] eq $default;
1687 \& push @range_list, [ $list_ref\->[$i],
1688 \& $list_ref\->[$i+1],
1689 \& $map_ref\->[$i]
1690 \& ];
1691 \& }
1692 \&
1693 \& print charinrange(\e@range_list, $code_point), "\en";
1694 .Ve
1695 .PP
1696 With this, \f(CWcharinrange()\fR will return \f(CW\*(C`undef\*(C'\fR if its input code point maps
1697 to \f(CW$default\fR. You can avoid this by omitting the \f(CW\*(C`next\*(C'\fR statement, and adding
1698 a line after the loop to handle the final element of the inversion map.
1699 .PP
1700 Similarly, this recipe can be used for properties that do require adjustments:
1701 .PP
1702 .Vb 2
1703 \& for my $i (0 .. @$list_ref \- 2) {
1704 \& next if $map_ref\->[$i] eq $default;
1705 \&
1706 \& # prop_invmap() guarantees that if the mapping is to an array, the
1707 \& # range has just one element, so no need to worry about adjustments.
1708 \& if (ref $map_ref\->[$i]) {
1709 \& push @range_list,
1710 \& [ $list_ref\->[$i], $list_ref\->[$i], $map_ref\->[$i] ];
1711 \& }
1712 \& else { # Otherwise each element is actually mapped to a separate
1713 \& # value, so the range has to be split into single code point
1714 \& # ranges.
1715 \&
1716 \& my $adjustment = 0;
1717 \&
1718 \& # For each code point that gets mapped to something...
1719 \& for my $j ($list_ref\->[$i] .. $list_ref\->[$i+1] \-1 ) {
1720 \&
1721 \& # ... add a range consisting of just it mapping to the
1722 \& # original plus the adjustment, which is incremented for the
1723 \& # next time through the loop, as the offset increases by 1
1724 \& # for each element in the range
1725 \& push @range_list,
1726 \& [ $j, $j, $map_ref\->[$i] + $adjustment++ ];
1727 \& }
1728 \& }
1729 \& }
1730 .Ve
1731 .PP
1732 Note that the inversion maps returned for the \f(CW\*(C`Case_Folding\*(C'\fR and
1733 \&\f(CW\*(C`Simple_Case_Folding\*(C'\fR properties do not include the Turkic-locale mappings.
1734 Use "\fBcasefold()\fR" for these.
1735 .PP
1736 \&\f(CW\*(C`prop_invmap\*(C'\fR does not know about any user-defined properties, and will
1737 return \f(CW\*(C`undef\*(C'\fR if called with one of those.
1738 .PP
1739 The returned values for the Perl extension properties, such as \f(CW\*(C`Any\*(C'\fR and
1740 \&\f(CW\*(C`Greek\*(C'\fR are somewhat misleading. The values are either \f(CW"Y"\fR or \f(CW\*(C`"N\*(C'\fR".
1741 All Unicode properties are bipartite, so you can actually use the \f(CW"Y"\fR or
1742 \&\f(CW\*(C`"N\*(C'\fR" in a Perl regular expression for these, like \f(CW\*(C`qr/\ep{ID_Start=Y/}\*(C'\fR or
1743 \&\f(CW\*(C`qr/\ep{Upper=N/}\*(C'\fR. But the Perl extensions aren't specified this way, only
1744 like \f(CW\*(C`/qr/\ep{Any}\*(C'\fR, \fIetc\fR. You can't actually use the \f(CW"Y"\fR and \f(CW\*(C`"N\*(C'\fR" in
1745 them.
1746 .PP
1747 \fIGetting every available name\fR
1748 .IX Subsection "Getting every available name"
1749 .PP
1750 Instead of reading the Unicode Database directly from files, as you were able
1751 to do for a long time, you are encouraged to use the supplied functions. So,
1752 instead of reading \f(CW\*(C`Name.pl\*(C'\fR directly, which changed formats in 5.32, and may
1753 do so again without notice in the future or even disappear, you ought to use
1754 "\fBprop_invmap()\fR" like this:
1755 .PP
1756 .Vb 10
1757 \& my (%name, %cp, %cps, $n);
1758 \& # All codepoints
1759 \& foreach my $cat (qw( Name Name_Alias )) {
1760 \& my ($codepoints, $names, $format, $default) = prop_invmap($cat);
1761 \& # $format => "n", $default => ""
1762 \& foreach my $i (0 .. @$codepoints \- 2) {
1763 \& my ($cp, $n) = ($codepoints\->[$i], $names\->[$i]);
1764 \& # If $n is a ref, the same codepoint has multiple names
1765 \& foreach my $name (ref $n ? @$n : $n) {
1766 \& $name{$cp} //= $name;
1767 \& $cp{$name} //= $cp;
1768 \& }
1769 \& }
1770 \& }
1771 \& # Named sequences
1772 \& { my %ns = namedseq();
1773 \& foreach my $name (sort { $ns{$a} cmp $ns{$b} } keys %ns) {
1774 \& $cp{$name} //= [ map { ord } split "" => $ns{$name} ];
1775 \& }
1776 \& }
1777 .Ve
1778 .SS \fBsearch_invlist()\fP
1779 .IX Subsection "search_invlist()"
1780 .Vb 2
1781 \& use Unicode::UCD qw(prop_invmap prop_invlist);
1782 \& use Unicode::UCD \*(Aqsearch_invlist\*(Aq;
1783 \&
1784 \& my @invlist = prop_invlist($property_name);
1785 \& print $code_point, ((search_invlist(\e@invlist, $code_point) // \-1) % 2)
1786 \& ? " isn\*(Aqt"
1787 \& : " is",
1788 \& " in $property_name\en";
1789 \&
1790 \& my ($blocks_ranges_ref, $blocks_map_ref) = prop_invmap("Block");
1791 \& my $index = search_invlist($blocks_ranges_ref, $code_point);
1792 \& print "$code_point is in block ", $blocks_map_ref\->[$index], "\en";
1793 .Ve
1794 .PP
1795 \&\f(CW\*(C`search_invlist\*(C'\fR is used to search an inversion list returned by
1796 \&\f(CW\*(C`prop_invlist\*(C'\fR or \f(CW\*(C`prop_invmap\*(C'\fR for a particular "code point argument".
1797 \&\f(CW\*(C`undef\*(C'\fR is returned if the code point is not found in the inversion list
1798 (this happens only when it is not a legal "code point argument", or is less
1799 than the list's first element). A warning is raised in the first instance.
1800 .PP
1801 Otherwise, it returns the index into the list of the range that contains the
1802 code point.; that is, find \f(CW\*(C`i\*(C'\fR such that
1803 .PP
1804 .Vb 1
1805 \& list[i]<= code_point < list[i+1].
1806 .Ve
1807 .PP
1808 As explained in "\fBprop_invlist()\fR", whether a code point is in the list or not
1809 depends on if the index is even (in) or odd (not in). And as explained in
1810 "\fBprop_invmap()\fR", the index is used with the returned parallel array to find
1811 the mapping.
1812 .SS Unicode::UCD::UnicodeVersion
1813 .IX Subsection "Unicode::UCD::UnicodeVersion"
1814 This returns the version of the Unicode Character Database, in other words, the
1815 version of the Unicode standard the database implements. The version is a
1816 string of numbers delimited by dots (\f(CW\*(Aq.\*(Aq\fR).
1817 .SS "\fBBlocks versus Scripts\fP"
1818 .IX Subsection "Blocks versus Scripts"
1819 The difference between a block and a script is that scripts are closer
1820 to the linguistic notion of a set of code points required to represent
1821 languages, while block is more of an artifact of the Unicode code point
1822 numbering and separation into blocks of consecutive code points (so far the
1823 size of a block is some multiple of 16, like 128 or 256).
1824 .PP
1825 For example the Latin \fBscript\fR is spread over several \fBblocks\fR, such
1826 as \f(CW\*(C`Basic Latin\*(C'\fR, \f(CW\*(C`Latin 1 Supplement\*(C'\fR, \f(CW\*(C`Latin Extended\-A\*(C'\fR, and
1827 \&\f(CW\*(C`Latin Extended\-B\*(C'\fR. On the other hand, the Latin script does not
1828 contain all the characters of the \f(CW\*(C`Basic Latin\*(C'\fR block (also known as
1829 ASCII): it includes only the letters, and not, for example, the digits
1830 nor the punctuation.
1831 .PP
1832 For blocks see <http://www.unicode.org/Public/UNIDATA/Blocks.txt>
1833 .PP
1834 For scripts see UTR #24: <http://www.unicode.org/reports/tr24/>
1835 .SS "\fBMatching Scripts and Blocks\fP"
1836 .IX Subsection "Matching Scripts and Blocks"
1837 Scripts are matched with the regular-expression construct
1838 \&\f(CW\*(C`\ep{...}\*(C'\fR (e.g. \f(CW\*(C`\ep{Tibetan}\*(C'\fR matches characters of the Tibetan script),
1839 while \f(CW\*(C`\ep{Blk=...}\*(C'\fR is used for blocks (e.g. \f(CW\*(C`\ep{Blk=Tibetan}\*(C'\fR matches
1840 any of the 256 code points in the Tibetan block).
1841 .SS "Old-style versus new-style block names"
1842 .IX Subsection "Old-style versus new-style block names"
1843 Unicode publishes the names of blocks in two different styles, though the two
1844 are equivalent under Unicode's loose matching rules.
1845 .PP
1846 The original style uses blanks and hyphens in the block names (except for
1847 \&\f(CW\*(C`No_Block\*(C'\fR), like so:
1848 .PP
1849 .Vb 1
1850 \& Miscellaneous Mathematical Symbols\-B
1851 .Ve
1852 .PP
1853 The newer style replaces these with underscores, like this:
1854 .PP
1855 .Vb 1
1856 \& Miscellaneous_Mathematical_Symbols_B
1857 .Ve
1858 .PP
1859 This newer style is consistent with the values of other Unicode properties.
1860 To preserve backward compatibility, all the functions in Unicode::UCD that
1861 return block names (except as noted) return the old-style ones.
1862 "\fBprop_value_aliases()\fR" returns the new-style and can be used to convert from
1863 old-style to new-style:
1864 .PP
1865 .Vb 1
1866 \& my $new_style = prop_values_aliases("block", $old_style);
1867 .Ve
1868 .PP
1869 Perl also has single-form extensions that refer to blocks, \f(CW\*(C`In_Cyrillic\*(C'\fR,
1870 meaning \f(CW\*(C`Block=Cyrillic\*(C'\fR. These have always been written in the new style.
1871 .PP
1872 To convert from new-style to old-style, follow this recipe:
1873 .PP
1874 .Vb 1
1875 \& $old_style = charblock((prop_invlist("block=$new_style"))[0]);
1876 .Ve
1877 .PP
1878 (which finds the range of code points in the block using \f(CW\*(C`prop_invlist\*(C'\fR,
1879 gets the lower end of the range (0th element) and then looks up the old name
1880 for its block using \f(CW\*(C`charblock\*(C'\fR).
1881 .PP
1882 Note that starting in Unicode 6.1, many of the block names have shorter
1883 synonyms. These are always given in the new style.
1884 .SS "Use with older Unicode versions"
1885 .IX Subsection "Use with older Unicode versions"
1886 The functions in this module work as well as can be expected when
1887 used on earlier Unicode versions. But, obviously, they use the available data
1888 from that Unicode version. For example, if the Unicode version predates the
1889 definition of the script property (Unicode 3.1), then any function that deals
1890 with scripts is going to return \f(CW\*(C`undef\*(C'\fR for the script portion of the return
1891 value.
1892 .SH AUTHOR
1893 .IX Header "AUTHOR"
1894 Jarkko Hietaniemi. Now maintained by perl5 porters.