Debian Codegraph @

upstream/opensuse-tumbleweed/man3/Unicode::Normalize.3pm

raw
1 .\" -*- mode: troff; coding: utf-8 -*-
2 .\" Automatically generated by Pod::Man v6.0.2 (Pod::Simple 3.45)
3 .\"
4 .\" Standard preamble:
5 .\" ========================================================================
6 .de Sp \" Vertical space (when we can't use .PP)
7 .if t .sp .5v
8 .if n .sp
9 ..
10 .de Vb \" Begin verbatim text
11 .ft CW
12 .nf
13 .ne \\$1
14 ..
15 .de Ve \" End verbatim text
16 .ft R
17 .fi
18 ..
19 .\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>.
20 .ie n \{\
21 . ds C` ""
22 . ds C' ""
23 'br\}
24 .el\{\
25 . ds C`
26 . ds C'
27 'br\}
28 .\"
29 .\" Escape single quotes in literal strings from groff's Unicode transform.
30 .ie \n(.g .ds Aq \(aq
31 .el .ds Aq '
32 .\"
33 .\" If the F register is >0, we'll generate index entries on stderr for
34 .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
35 .\" entries marked with X<> in POD. Of course, you'll have to process the
36 .\" output yourself in some meaningful fashion.
37 .\"
38 .\" Avoid warning from groff about undefined register 'F'.
39 .de IX
40 ..
41 .nr rF 0
42 .if \n(.g .if rF .nr rF 1
43 .if (\n(rF:(\n(.g==0)) \{\
44 . if \nF \{\
45 . de IX
46 . tm Index:\\$1\t\\n%\t"\\$2"
47 ..
48 . if !\nF==2 \{\
49 . nr % 0
50 . nr F 2
51 . \}
52 . \}
53 .\}
54 .rr rF
55 .\"
56 .\" Required to disable full justification in groff 1.23.0.
57 .if n .ds AD l
58 .\" ========================================================================
59 .\"
60 .IX Title "Unicode::Normalize 3pm"
61 .TH Unicode::Normalize 3pm 2026-03-09 "perl v5.42.1" "Perl Programmers Reference Guide"
62 .\" For nroff, turn off justification. Always turn off hyphenation; it makes
63 .\" way too many mistakes in technical documents.
64 .if n .ad l
65 .nh
66 .SH NAME
67 Unicode::Normalize \- Unicode Normalization Forms
68 .SH SYNOPSIS
69 .IX Header "SYNOPSIS"
70 (1) using function names exported by default:
71 .PP
72 .Vb 1
73 \& use Unicode::Normalize;
74 \&
75 \& $NFD_string = NFD($string); # Normalization Form D
76 \& $NFC_string = NFC($string); # Normalization Form C
77 \& $NFKD_string = NFKD($string); # Normalization Form KD
78 \& $NFKC_string = NFKC($string); # Normalization Form KC
79 .Ve
80 .PP
81 (2) using function names exported on request:
82 .PP
83 .Vb 1
84 \& use Unicode::Normalize \*(Aqnormalize\*(Aq;
85 \&
86 \& $NFD_string = normalize(\*(AqD\*(Aq, $string); # Normalization Form D
87 \& $NFC_string = normalize(\*(AqC\*(Aq, $string); # Normalization Form C
88 \& $NFKD_string = normalize(\*(AqKD\*(Aq, $string); # Normalization Form KD
89 \& $NFKC_string = normalize(\*(AqKC\*(Aq, $string); # Normalization Form KC
90 .Ve
91 .SH DESCRIPTION
92 .IX Header "DESCRIPTION"
93 Parameters:
94 .PP
95 \&\f(CW$string\fR is used as a string under character semantics (see perlunicode).
96 .PP
97 \&\f(CW$code_point\fR should be an unsigned integer representing a Unicode code point.
98 .PP
99 Note: Between XSUB and pure Perl, there is an incompatibility
100 about the interpretation of \f(CW$code_point\fR as a decimal number.
101 XSUB converts \f(CW$code_point\fR to an unsigned integer, but pure Perl does not.
102 Do not use a floating point nor a negative sign in \f(CW$code_point\fR.
103 .SS "Normalization Forms"
104 .IX Subsection "Normalization Forms"
105 .ie n .IP """$NFD_string = NFD($string)""" 4
106 .el .IP "\f(CW$NFD_string = NFD($string)\fR" 4
107 .IX Item "$NFD_string = NFD($string)"
108 It returns the Normalization Form D (formed by canonical decomposition).
109 .ie n .IP """$NFC_string = NFC($string)""" 4
110 .el .IP "\f(CW$NFC_string = NFC($string)\fR" 4
111 .IX Item "$NFC_string = NFC($string)"
112 It returns the Normalization Form C (formed by canonical decomposition
113 followed by canonical composition).
114 .ie n .IP """$NFKD_string = NFKD($string)""" 4
115 .el .IP "\f(CW$NFKD_string = NFKD($string)\fR" 4
116 .IX Item "$NFKD_string = NFKD($string)"
117 It returns the Normalization Form KD (formed by compatibility decomposition).
118 .ie n .IP """$NFKC_string = NFKC($string)""" 4
119 .el .IP "\f(CW$NFKC_string = NFKC($string)\fR" 4
120 .IX Item "$NFKC_string = NFKC($string)"
121 It returns the Normalization Form KC (formed by compatibility decomposition
122 followed by \fBcanonical\fR composition).
123 .ie n .IP """$FCD_string = FCD($string)""" 4
124 .el .IP "\f(CW$FCD_string = FCD($string)\fR" 4
125 .IX Item "$FCD_string = FCD($string)"
126 If the given string is in FCD ("Fast C or D" form; cf. UTN #5),
127 it returns the string without modification; otherwise it returns an FCD string.
128 .Sp
129 Note: FCD is not always unique, then plural forms may be equivalent
130 each other. \f(CWFCD()\fR will return one of these equivalent forms.
131 .ie n .IP """$FCC_string = FCC($string)""" 4
132 .el .IP "\f(CW$FCC_string = FCC($string)\fR" 4
133 .IX Item "$FCC_string = FCC($string)"
134 It returns the FCC form ("Fast C Contiguous"; cf. UTN #5).
135 .Sp
136 Note: FCC is unique, as well as four normalization forms (NF*).
137 .ie n .IP """$normalized_string = normalize($form_name, $string)""" 4
138 .el .IP "\f(CW$normalized_string = normalize($form_name, $string)\fR" 4
139 .IX Item "$normalized_string = normalize($form_name, $string)"
140 It returns the normalization form of \f(CW$form_name\fR.
141 .Sp
142 As \f(CW$form_name\fR, one of the following names must be given.
143 .Sp
144 .Vb 4
145 \& \*(AqC\*(Aq or \*(AqNFC\*(Aq for Normalization Form C (UAX #15)
146 \& \*(AqD\*(Aq or \*(AqNFD\*(Aq for Normalization Form D (UAX #15)
147 \& \*(AqKC\*(Aq or \*(AqNFKC\*(Aq for Normalization Form KC (UAX #15)
148 \& \*(AqKD\*(Aq or \*(AqNFKD\*(Aq for Normalization Form KD (UAX #15)
149 \&
150 \& \*(AqFCD\*(Aq for "Fast C or D" Form (UTN #5)
151 \& \*(AqFCC\*(Aq for "Fast C Contiguous" (UTN #5)
152 .Ve
153 .SS "Decomposition and Composition"
154 .IX Subsection "Decomposition and Composition"
155 .ie n .IP """$decomposed_string = decompose($string [, $useCompatMapping])""" 4
156 .el .IP "\f(CW$decomposed_string = decompose($string [, $useCompatMapping])\fR" 4
157 .IX Item "$decomposed_string = decompose($string [, $useCompatMapping])"
158 It returns the concatenation of the decomposition of each character
159 in the string.
160 .Sp
161 If the second parameter (a boolean) is omitted or false,
162 the decomposition is canonical decomposition;
163 if the second parameter (a boolean) is true,
164 the decomposition is compatibility decomposition.
165 .Sp
166 The string returned is not always in NFD/NFKD. Reordering may be required.
167 .Sp
168 .Vb 2
169 \& $NFD_string = reorder(decompose($string)); # eq. to NFD()
170 \& $NFKD_string = reorder(decompose($string, TRUE)); # eq. to NFKD()
171 .Ve
172 .ie n .IP """$reordered_string = reorder($string)""" 4
173 .el .IP "\f(CW$reordered_string = reorder($string)\fR" 4
174 .IX Item "$reordered_string = reorder($string)"
175 It returns the result of reordering the combining characters
176 according to Canonical Ordering Behavior.
177 .Sp
178 For example, when you have a list of NFD/NFKD strings,
179 you can get the concatenated NFD/NFKD string from them, by saying
180 .Sp
181 .Vb 2
182 \& $concat_NFD = reorder(join \*(Aq\*(Aq, @NFD_strings);
183 \& $concat_NFKD = reorder(join \*(Aq\*(Aq, @NFKD_strings);
184 .Ve
185 .ie n .IP """$composed_string = compose($string)""" 4
186 .el .IP "\f(CW$composed_string = compose($string)\fR" 4
187 .IX Item "$composed_string = compose($string)"
188 It returns the result of canonical composition
189 without applying any decomposition.
190 .Sp
191 For example, when you have a NFD/NFKD string,
192 you can get its NFC/NFKC string, by saying
193 .Sp
194 .Vb 2
195 \& $NFC_string = compose($NFD_string);
196 \& $NFKC_string = compose($NFKD_string);
197 .Ve
198 .ie n .IP """($processed, $unprocessed) = splitOnLastStarter($normalized)""" 4
199 .el .IP "\f(CW($processed, $unprocessed) = splitOnLastStarter($normalized)\fR" 4
200 .IX Item "($processed, $unprocessed) = splitOnLastStarter($normalized)"
201 It returns two strings: the first one, \f(CW$processed\fR, is a part
202 before the last starter, and the second one, \f(CW$unprocessed\fR is
203 another part after the first part. A starter is a character having
204 a combining class of zero (see UAX #15).
205 .Sp
206 Note that \f(CW$processed\fR may be empty (when \f(CW$normalized\fR contains no
207 starter or starts with the last starter), and then \f(CW$unprocessed\fR
208 should be equal to the entire \f(CW$normalized\fR.
209 .Sp
210 When you have a \f(CW$normalized\fR string and an \f(CW$unnormalized\fR string
211 following it, a simple concatenation is wrong:
212 .Sp
213 .Vb 1
214 \& $concat = $normalized . normalize($form, $unnormalized); # wrong!
215 .Ve
216 .Sp
217 Instead of it, do like this:
218 .Sp
219 .Vb 2
220 \& ($processed, $unprocessed) = splitOnLastStarter($normalized);
221 \& $concat = $processed . normalize($form,$unprocessed.$unnormalized);
222 .Ve
223 .Sp
224 \&\f(CWsplitOnLastStarter()\fR should be called with a pre\-normalized parameter
225 \&\f(CW$normalized\fR, that is in the same form as \f(CW$form\fR you want.
226 .Sp
227 If you have an array of \f(CW@string\fR that should be concatenated and then
228 normalized, you can do like this:
229 .Sp
230 .Vb 11
231 \& my $result = "";
232 \& my $unproc = "";
233 \& foreach my $str (@string) {
234 \& $unproc .= $str;
235 \& my $n = normalize($form, $unproc);
236 \& my($p, $u) = splitOnLastStarter($n);
237 \& $result .= $p;
238 \& $unproc = $u;
239 \& }
240 \& $result .= $unproc;
241 \& # instead of normalize($form, join(\*(Aq\*(Aq, @string))
242 .Ve
243 .ie n .IP """$processed = normalize_partial($form, $unprocessed)""" 4
244 .el .IP "\f(CW$processed = normalize_partial($form, $unprocessed)\fR" 4
245 .IX Item "$processed = normalize_partial($form, $unprocessed)"
246 A wrapper for the combination of \f(CWnormalize()\fR and \f(CWsplitOnLastStarter()\fR.
247 Note that \f(CW$unprocessed\fR will be modified as a side\-effect.
248 .Sp
249 If you have an array of \f(CW@string\fR that should be concatenated and then
250 normalized, you can do like this:
251 .Sp
252 .Vb 8
253 \& my $result = "";
254 \& my $unproc = "";
255 \& foreach my $str (@string) {
256 \& $unproc .= $str;
257 \& $result .= normalize_partial($form, $unproc);
258 \& }
259 \& $result .= $unproc;
260 \& # instead of normalize($form, join(\*(Aq\*(Aq, @string))
261 .Ve
262 .ie n .IP """$processed = NFD_partial($unprocessed)""" 4
263 .el .IP "\f(CW$processed = NFD_partial($unprocessed)\fR" 4
264 .IX Item "$processed = NFD_partial($unprocessed)"
265 It does like \f(CW\*(C`normalize_partial(\*(AqNFD\*(Aq, $unprocessed)\*(C'\fR.
266 Note that \f(CW$unprocessed\fR will be modified as a side\-effect.
267 .ie n .IP """$processed = NFC_partial($unprocessed)""" 4
268 .el .IP "\f(CW$processed = NFC_partial($unprocessed)\fR" 4
269 .IX Item "$processed = NFC_partial($unprocessed)"
270 It does like \f(CW\*(C`normalize_partial(\*(AqNFC\*(Aq, $unprocessed)\*(C'\fR.
271 Note that \f(CW$unprocessed\fR will be modified as a side\-effect.
272 .ie n .IP """$processed = NFKD_partial($unprocessed)""" 4
273 .el .IP "\f(CW$processed = NFKD_partial($unprocessed)\fR" 4
274 .IX Item "$processed = NFKD_partial($unprocessed)"
275 It does like \f(CW\*(C`normalize_partial(\*(AqNFKD\*(Aq, $unprocessed)\*(C'\fR.
276 Note that \f(CW$unprocessed\fR will be modified as a side\-effect.
277 .ie n .IP """$processed = NFKC_partial($unprocessed)""" 4
278 .el .IP "\f(CW$processed = NFKC_partial($unprocessed)\fR" 4
279 .IX Item "$processed = NFKC_partial($unprocessed)"
280 It does like \f(CW\*(C`normalize_partial(\*(AqNFKC\*(Aq, $unprocessed)\*(C'\fR.
281 Note that \f(CW$unprocessed\fR will be modified as a side\-effect.
282 .SS "Quick Check"
283 .IX Subsection "Quick Check"
284 (see Annex 8, UAX #15; and \fIlib/unicore/DerivedNormalizationProps.txt\fR)
285 .PP
286 The following functions check whether the string is in that normalization form.
287 .PP
288 The result returned will be one of the following:
289 .PP
290 .Vb 3
291 \& YES The string is in that normalization form.
292 \& NO The string is not in that normalization form.
293 \& MAYBE Dubious. Maybe yes, maybe no.
294 .Ve
295 .ie n .IP """$result = checkNFD($string)""" 4
296 .el .IP "\f(CW$result = checkNFD($string)\fR" 4
297 .IX Item "$result = checkNFD($string)"
298 It returns true (\f(CW1\fR) if \f(CW\*(C`YES\*(C'\fR; false (\f(CW\*(C`empty string\*(C'\fR) if \f(CW\*(C`NO\*(C'\fR.
299 .ie n .IP """$result = checkNFC($string)""" 4
300 .el .IP "\f(CW$result = checkNFC($string)\fR" 4
301 .IX Item "$result = checkNFC($string)"
302 It returns true (\f(CW1\fR) if \f(CW\*(C`YES\*(C'\fR; false (\f(CW\*(C`empty string\*(C'\fR) if \f(CW\*(C`NO\*(C'\fR;
303 \&\f(CW\*(C`undef\*(C'\fR if \f(CW\*(C`MAYBE\*(C'\fR.
304 .ie n .IP """$result = checkNFKD($string)""" 4
305 .el .IP "\f(CW$result = checkNFKD($string)\fR" 4
306 .IX Item "$result = checkNFKD($string)"
307 It returns true (\f(CW1\fR) if \f(CW\*(C`YES\*(C'\fR; false (\f(CW\*(C`empty string\*(C'\fR) if \f(CW\*(C`NO\*(C'\fR.
308 .ie n .IP """$result = checkNFKC($string)""" 4
309 .el .IP "\f(CW$result = checkNFKC($string)\fR" 4
310 .IX Item "$result = checkNFKC($string)"
311 It returns true (\f(CW1\fR) if \f(CW\*(C`YES\*(C'\fR; false (\f(CW\*(C`empty string\*(C'\fR) if \f(CW\*(C`NO\*(C'\fR;
312 \&\f(CW\*(C`undef\*(C'\fR if \f(CW\*(C`MAYBE\*(C'\fR.
313 .ie n .IP """$result = checkFCD($string)""" 4
314 .el .IP "\f(CW$result = checkFCD($string)\fR" 4
315 .IX Item "$result = checkFCD($string)"
316 It returns true (\f(CW1\fR) if \f(CW\*(C`YES\*(C'\fR; false (\f(CW\*(C`empty string\*(C'\fR) if \f(CW\*(C`NO\*(C'\fR.
317 .ie n .IP """$result = checkFCC($string)""" 4
318 .el .IP "\f(CW$result = checkFCC($string)\fR" 4
319 .IX Item "$result = checkFCC($string)"
320 It returns true (\f(CW1\fR) if \f(CW\*(C`YES\*(C'\fR; false (\f(CW\*(C`empty string\*(C'\fR) if \f(CW\*(C`NO\*(C'\fR;
321 \&\f(CW\*(C`undef\*(C'\fR if \f(CW\*(C`MAYBE\*(C'\fR.
322 .Sp
323 Note: If a string is not in FCD, it must not be in FCC.
324 So \f(CWcheckFCC($not_FCD_string)\fR should return \f(CW\*(C`NO\*(C'\fR.
325 .ie n .IP """$result = check($form_name, $string)""" 4
326 .el .IP "\f(CW$result = check($form_name, $string)\fR" 4
327 .IX Item "$result = check($form_name, $string)"
328 It returns true (\f(CW1\fR) if \f(CW\*(C`YES\*(C'\fR; false (\f(CW\*(C`empty string\*(C'\fR) if \f(CW\*(C`NO\*(C'\fR;
329 \&\f(CW\*(C`undef\*(C'\fR if \f(CW\*(C`MAYBE\*(C'\fR.
330 .Sp
331 As \f(CW$form_name\fR, one of the following names must be given.
332 .Sp
333 .Vb 4
334 \& \*(AqC\*(Aq or \*(AqNFC\*(Aq for Normalization Form C (UAX #15)
335 \& \*(AqD\*(Aq or \*(AqNFD\*(Aq for Normalization Form D (UAX #15)
336 \& \*(AqKC\*(Aq or \*(AqNFKC\*(Aq for Normalization Form KC (UAX #15)
337 \& \*(AqKD\*(Aq or \*(AqNFKD\*(Aq for Normalization Form KD (UAX #15)
338 \&
339 \& \*(AqFCD\*(Aq for "Fast C or D" Form (UTN #5)
340 \& \*(AqFCC\*(Aq for "Fast C Contiguous" (UTN #5)
341 .Ve
342 .PP
343 \&\fBNote\fR
344 .PP
345 In the cases of NFD, NFKD, and FCD, the answer must be
346 either \f(CW\*(C`YES\*(C'\fR or \f(CW\*(C`NO\*(C'\fR. The answer \f(CW\*(C`MAYBE\*(C'\fR may be returned
347 in the cases of NFC, NFKC, and FCC.
348 .PP
349 A \f(CW\*(C`MAYBE\*(C'\fR string should contain at least one combining character
350 or the like. For example, \f(CW\*(C`COMBINING ACUTE ACCENT\*(C'\fR has
351 the MAYBE_NFC/MAYBE_NFKC property.
352 .PP
353 Both \f(CW\*(C`checkNFC("A\eN{COMBINING ACUTE ACCENT}")\*(C'\fR
354 and \f(CW\*(C`checkNFC("B\eN{COMBINING ACUTE ACCENT}")\*(C'\fR will return \f(CW\*(C`MAYBE\*(C'\fR.
355 \&\f(CW"A\eN{COMBINING ACUTE ACCENT}"\fR is not in NFC
356 (its NFC is \f(CW"\eN{LATIN CAPITAL LETTER A WITH ACUTE}"\fR),
357 while \f(CW"B\eN{COMBINING ACUTE ACCENT}"\fR is in NFC.
358 .PP
359 If you want to check exactly, compare the string with its NFC/NFKC/FCC.
360 .PP
361 .Vb 5
362 \& if ($string eq NFC($string)) {
363 \& # $string is exactly normalized in NFC;
364 \& } else {
365 \& # $string is not normalized in NFC;
366 \& }
367 \&
368 \& if ($string eq NFKC($string)) {
369 \& # $string is exactly normalized in NFKC;
370 \& } else {
371 \& # $string is not normalized in NFKC;
372 \& }
373 .Ve
374 .SS "Character Data"
375 .IX Subsection "Character Data"
376 These functions are interface of character data used internally.
377 If you want only to get Unicode normalization forms, you don\*(Aqt need
378 call them yourself.
379 .ie n .IP """$canonical_decomposition = getCanon($code_point)""" 4
380 .el .IP "\f(CW$canonical_decomposition = getCanon($code_point)\fR" 4
381 .IX Item "$canonical_decomposition = getCanon($code_point)"
382 If the character is canonically decomposable (including Hangul Syllables),
383 it returns the (full) canonical decomposition as a string.
384 Otherwise it returns \f(CW\*(C`undef\*(C'\fR.
385 .Sp
386 \&\fBNote:\fR According to the Unicode standard, the canonical decomposition
387 of the character that is not canonically decomposable is same as
388 the character itself.
389 .ie n .IP """$compatibility_decomposition = getCompat($code_point)""" 4
390 .el .IP "\f(CW$compatibility_decomposition = getCompat($code_point)\fR" 4
391 .IX Item "$compatibility_decomposition = getCompat($code_point)"
392 If the character is compatibility decomposable (including Hangul Syllables),
393 it returns the (full) compatibility decomposition as a string.
394 Otherwise it returns \f(CW\*(C`undef\*(C'\fR.
395 .Sp
396 \&\fBNote:\fR According to the Unicode standard, the compatibility decomposition
397 of the character that is not compatibility decomposable is same as
398 the character itself.
399 .ie n .IP """$code_point_composite = getComposite($code_point_here, $code_point_next)""" 4
400 .el .IP "\f(CW$code_point_composite = getComposite($code_point_here, $code_point_next)\fR" 4
401 .IX Item "$code_point_composite = getComposite($code_point_here, $code_point_next)"
402 If two characters here and next (as code points) are composable
403 (including Hangul Jamo/Syllables and Composition Exclusions),
404 it returns the code point of the composite.
405 .Sp
406 If they are not composable, it returns \f(CW\*(C`undef\*(C'\fR.
407 .ie n .IP """$combining_class = getCombinClass($code_point)""" 4
408 .el .IP "\f(CW$combining_class = getCombinClass($code_point)\fR" 4
409 .IX Item "$combining_class = getCombinClass($code_point)"
410 It returns the combining class (as an integer) of the character.
411 .ie n .IP """$may_be_composed_with_prev_char = isComp2nd($code_point)""" 4
412 .el .IP "\f(CW$may_be_composed_with_prev_char = isComp2nd($code_point)\fR" 4
413 .IX Item "$may_be_composed_with_prev_char = isComp2nd($code_point)"
414 It returns a boolean whether the character of the specified codepoint
415 may be composed with the previous one in a certain composition
416 (including Hangul Compositions, but excluding
417 Composition Exclusions and Non\-Starter Decompositions).
418 .ie n .IP """$is_exclusion = isExclusion($code_point)""" 4
419 .el .IP "\f(CW$is_exclusion = isExclusion($code_point)\fR" 4
420 .IX Item "$is_exclusion = isExclusion($code_point)"
421 It returns a boolean whether the code point is a composition exclusion.
422 .ie n .IP """$is_singleton = isSingleton($code_point)""" 4
423 .el .IP "\f(CW$is_singleton = isSingleton($code_point)\fR" 4
424 .IX Item "$is_singleton = isSingleton($code_point)"
425 It returns a boolean whether the code point is a singleton
426 .ie n .IP """$is_non_starter_decomposition = isNonStDecomp($code_point)""" 4
427 .el .IP "\f(CW$is_non_starter_decomposition = isNonStDecomp($code_point)\fR" 4
428 .IX Item "$is_non_starter_decomposition = isNonStDecomp($code_point)"
429 It returns a boolean whether the code point has Non\-Starter Decomposition.
430 .ie n .IP """$is_Full_Composition_Exclusion = isComp_Ex($code_point)""" 4
431 .el .IP "\f(CW$is_Full_Composition_Exclusion = isComp_Ex($code_point)\fR" 4
432 .IX Item "$is_Full_Composition_Exclusion = isComp_Ex($code_point)"
433 It returns a boolean of the derived property Comp_Ex
434 (Full_Composition_Exclusion). This property is generated from
435 Composition Exclusions + Singletons + Non\-Starter Decompositions.
436 .ie n .IP """$NFD_is_NO = isNFD_NO($code_point)""" 4
437 .el .IP "\f(CW$NFD_is_NO = isNFD_NO($code_point)\fR" 4
438 .IX Item "$NFD_is_NO = isNFD_NO($code_point)"
439 It returns a boolean of the derived property NFD_NO
440 (NFD_Quick_Check=No).
441 .ie n .IP """$NFC_is_NO = isNFC_NO($code_point)""" 4
442 .el .IP "\f(CW$NFC_is_NO = isNFC_NO($code_point)\fR" 4
443 .IX Item "$NFC_is_NO = isNFC_NO($code_point)"
444 It returns a boolean of the derived property NFC_NO
445 (NFC_Quick_Check=No).
446 .ie n .IP """$NFC_is_MAYBE = isNFC_MAYBE($code_point)""" 4
447 .el .IP "\f(CW$NFC_is_MAYBE = isNFC_MAYBE($code_point)\fR" 4
448 .IX Item "$NFC_is_MAYBE = isNFC_MAYBE($code_point)"
449 It returns a boolean of the derived property NFC_MAYBE
450 (NFC_Quick_Check=Maybe).
451 .ie n .IP """$NFKD_is_NO = isNFKD_NO($code_point)""" 4
452 .el .IP "\f(CW$NFKD_is_NO = isNFKD_NO($code_point)\fR" 4
453 .IX Item "$NFKD_is_NO = isNFKD_NO($code_point)"
454 It returns a boolean of the derived property NFKD_NO
455 (NFKD_Quick_Check=No).
456 .ie n .IP """$NFKC_is_NO = isNFKC_NO($code_point)""" 4
457 .el .IP "\f(CW$NFKC_is_NO = isNFKC_NO($code_point)\fR" 4
458 .IX Item "$NFKC_is_NO = isNFKC_NO($code_point)"
459 It returns a boolean of the derived property NFKC_NO
460 (NFKC_Quick_Check=No).
461 .ie n .IP """$NFKC_is_MAYBE = isNFKC_MAYBE($code_point)""" 4
462 .el .IP "\f(CW$NFKC_is_MAYBE = isNFKC_MAYBE($code_point)\fR" 4
463 .IX Item "$NFKC_is_MAYBE = isNFKC_MAYBE($code_point)"
464 It returns a boolean of the derived property NFKC_MAYBE
465 (NFKC_Quick_Check=Maybe).
466 .SH EXPORT
467 .IX Header "EXPORT"
468 \&\f(CW\*(C`NFC\*(C'\fR, \f(CW\*(C`NFD\*(C'\fR, \f(CW\*(C`NFKC\*(C'\fR, \f(CW\*(C`NFKD\*(C'\fR: by default.
469 .PP
470 \&\f(CW\*(C`normalize\*(C'\fR and other some functions: on request.
471 .SH CAVEATS
472 .IX Header "CAVEATS"
473 .IP "Perl\*(Aqs version vs. Unicode version" 4
474 .IX Item "Perl's version vs. Unicode version"
475 Since this module refers to perl core\*(Aqs Unicode database in the directory
476 \&\fI/lib/unicore\fR (or formerly \fI/lib/unicode\fR), the Unicode version of
477 normalization implemented by this module depends on what has been
478 compiled into your perl. The following table lists the default Unicode
479 version that comes with various perl versions. (It is possible to change
480 the Unicode version in any perl version to be any earlier Unicode version,
481 so one could cause Unicode 3.2 to be used in any perl version starting with
482 5.8.0. Read \fR\f(CI$Config{privlib}\fR\fI/unicore/README.perl\fR for details.
483 .Sp
484 .Vb 10
485 \& perl\*(Aqs version implemented Unicode version
486 \& 5.6.1 3.0.1
487 \& 5.7.2 3.1.0
488 \& 5.7.3 3.1.1 (normalization is same as 3.1.0)
489 \& 5.8.0 3.2.0
490 \& 5.8.1\-5.8.3 4.0.0
491 \& 5.8.4\-5.8.6 4.0.1 (normalization is same as 4.0.0)
492 \& 5.8.7\-5.8.8 4.1.0
493 \& 5.10.0 5.0.0
494 \& 5.8.9, 5.10.1 5.1.0
495 \& 5.12.x 5.2.0
496 \& 5.14.x 6.0.0
497 \& 5.16.x 6.1.0
498 \& 5.18.x 6.2.0
499 \& 5.20.x 6.3.0
500 \& 5.22.x 7.0.0
501 .Ve
502 .IP "Correction of decomposition mapping" 4
503 .IX Item "Correction of decomposition mapping"
504 In older Unicode versions, a small number of characters (all of which are
505 CJK compatibility ideographs as far as they have been found) may have
506 an erroneous decomposition mapping (see
507 \&\fIlib/unicore/NormalizationCorrections.txt\fR).
508 Anyhow, this module will neither refer to
509 \&\fIlib/unicore/NormalizationCorrections.txt\fR
510 nor provide any specific version of normalization. Therefore this module
511 running on an older perl with an older Unicode database may use
512 the erroneous decomposition mapping blindly conforming to the Unicode database.
513 .IP "Revised definition of canonical composition" 4
514 .IX Item "Revised definition of canonical composition"
515 In Unicode 4.1.0, the definition D2 of canonical composition (which
516 affects NFC and NFKC) has been changed (see Public Review Issue #29
517 and recent UAX #15). This module has used the newer definition
518 since the version 0.07 (Oct 31, 2001).
519 This module will not support the normalization according to the older
520 definition, even if the Unicode version implemented by perl is
521 lower than 4.1.0.
522 .SH AUTHOR
523 .IX Header "AUTHOR"
524 SADAHIRO Tomoyuki <[email protected]>
525 .PP
526 Currently maintained by <perl5\[email protected]>
527 .PP
528 Copyright(C) 2001\-2012, SADAHIRO Tomoyuki. Japan. All rights reserved.
529 .SH LICENSE
530 .IX Header "LICENSE"
531 This module is free software; you can redistribute it
532 and/or modify it under the same terms as Perl itself.
533 .SH "SEE ALSO"
534 .IX Header "SEE ALSO"
535 .IP <http://www.unicode.org/reports/tr15/> 4
536 .IX Item "<http://www.unicode.org/reports/tr15/>"
537 Unicode Normalization Forms \- UAX #15
538 .IP <http://www.unicode.org/Public/UNIDATA/CompositionExclusions.txt> 4
539 .IX Item "<http://www.unicode.org/Public/UNIDATA/CompositionExclusions.txt>"
540 Composition Exclusion Table
541 .IP <http://www.unicode.org/Public/UNIDATA/DerivedNormalizationProps.txt> 4
542 .IX Item "<http://www.unicode.org/Public/UNIDATA/DerivedNormalizationProps.txt>"
543 Derived Normalization Properties
544 .IP <http://www.unicode.org/Public/UNIDATA/NormalizationCorrections.txt> 4
545 .IX Item "<http://www.unicode.org/Public/UNIDATA/NormalizationCorrections.txt>"
546 Normalization Corrections
547 .IP <http://www.unicode.org/review/pr\-29.html> 4
548 .IX Item "<http://www.unicode.org/review/pr-29.html>"
549 Public Review Issue #29: Normalization Issue
550 .IP <http://www.unicode.org/notes/tn5/> 4
551 .IX Item "<http://www.unicode.org/notes/tn5/>"
552 Canonical Equivalence in Applications \- UTN #5