Debian Codegraph @

upstream/mageia-cauldron/man3pm/encoding::warnings.3pm

raw
1 .\" -*- mode: troff; coding: utf-8 -*-
2 .\" Automatically generated by Pod::Man v6.0.2 (Pod::Simple 3.45)
3 .\"
4 .\" Standard preamble:
5 .\" ========================================================================
6 .de Sp \" Vertical space (when we can't use .PP)
7 .if t .sp .5v
8 .if n .sp
9 ..
10 .de Vb \" Begin verbatim text
11 .ft CW
12 .nf
13 .ne \\$1
14 ..
15 .de Ve \" End verbatim text
16 .ft R
17 .fi
18 ..
19 .\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>.
20 .ie n \{\
21 . ds C` ""
22 . ds C' ""
23 'br\}
24 .el\{\
25 . ds C`
26 . ds C'
27 'br\}
28 .\"
29 .\" Escape single quotes in literal strings from groff's Unicode transform.
30 .ie \n(.g .ds Aq \(aq
31 .el .ds Aq '
32 .\"
33 .\" If the F register is >0, we'll generate index entries on stderr for
34 .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
35 .\" entries marked with X<> in POD. Of course, you'll have to process the
36 .\" output yourself in some meaningful fashion.
37 .\"
38 .\" Avoid warning from groff about undefined register 'F'.
39 .de IX
40 ..
41 .nr rF 0
42 .if \n(.g .if rF .nr rF 1
43 .if (\n(rF:(\n(.g==0)) \{\
44 . if \nF \{\
45 . de IX
46 . tm Index:\\$1\t\\n%\t"\\$2"
47 ..
48 . if !\nF==2 \{\
49 . nr % 0
50 . nr F 2
51 . \}
52 . \}
53 .\}
54 .rr rF
55 .\"
56 .\" Required to disable full justification in groff 1.23.0.
57 .if n .ds AD l
58 .\" ========================================================================
59 .\"
60 .IX Title "encoding::warnings 3pm"
61 .TH encoding::warnings 3pm 2023-12-22 "perl v5.42.0" "Perl Programmers Reference Guide"
62 .\" For nroff, turn off justification. Always turn off hyphenation; it makes
63 .\" way too many mistakes in technical documents.
64 .if n .ad l
65 .nh
66 .SH NAME
67 encoding::warnings \- Warn on implicit encoding conversions
68 .SH VERSION
69 .IX Header "VERSION"
70 This document describes version 0.13 of encoding::warnings, released
71 June 20, 2016.
72 .SH NOTICE
73 .IX Header "NOTICE"
74 As of Perl 5.26.0, this module has no effect. The internal Perl feature
75 that was used to implement this module has been removed. In recent years,
76 much work has been done on the Perl core to eliminate discrepancies in the
77 treatment of upgraded versus downgraded strings. In addition, the
78 encoding pragma, which caused many of the problems, is no longer
79 supported. Thus, the warnings this module produced are no longer
80 necessary.
81 .PP
82 Hence, if you load this module on Perl 5.26.0, you will get one warning
83 that the module is no longer supported; and the module will do nothing
84 thereafter.
85 .SH SYNOPSIS
86 .IX Header "SYNOPSIS"
87 .Vb 1
88 \& use encoding::warnings; # or \*(AqFATAL\*(Aq to raise fatal exceptions
89 \&
90 \& utf8::encode($a = chr(20000)); # a byte\-string (raw bytes)
91 \& $b = chr(20000); # a unicode\-string (wide characters)
92 \&
93 \& # "Bytes implicitly upgraded into wide characters as iso\-8859\-1"
94 \& $c = $a . $b;
95 .Ve
96 .SH DESCRIPTION
97 .IX Header "DESCRIPTION"
98 .SS "Overview of the problem"
99 .IX Subsection "Overview of the problem"
100 By default, there is a fundamental asymmetry in Perl\*(Aqs unicode model:
101 implicit upgrading from byte\-strings to unicode\-strings assumes that
102 they were encoded in \fIISO 8859\-1 (Latin\-1)\fR, but unicode\-strings are
103 downgraded with UTF\-8 encoding. This happens because the first 256
104 codepoints in Unicode happens to agree with Latin\-1.
105 .PP
106 However, this silent upgrading can easily cause problems, if you happen
107 to mix unicode strings with non\-Latin1 data \-\- i.e. byte\-strings encoded
108 in UTF\-8 or other encodings. The error will not manifest until the
109 combined string is written to output, at which time it would be impossible
110 to see where did the silent upgrading occur.
111 .SS "Detecting the problem"
112 .IX Subsection "Detecting the problem"
113 This module simplifies the process of diagnosing such problems. Just put
114 this line on top of your main program:
115 .PP
116 .Vb 1
117 \& use encoding::warnings;
118 .Ve
119 .PP
120 Afterwards, implicit upgrading of high\-bit bytes will raise a warning.
121 Ex.: \f(CW\*(C`Bytes implicitly upgraded into wide characters as iso\-8859\-1 at
122 \&\- line 7\*(C'\fR.
123 .PP
124 However, strings composed purely of ASCII code points (\f(CW0x00\fR..\f(CW0x7F\fR)
125 will \fInot\fR trigger this warning.
126 .PP
127 You can also make the warnings fatal by importing this module as:
128 .PP
129 .Vb 1
130 \& use encoding::warnings \*(AqFATAL\*(Aq;
131 .Ve
132 .SS "Solving the problem"
133 .IX Subsection "Solving the problem"
134 Most of the time, this warning occurs when a byte\-string is concatenated
135 with a unicode\-string. There are a number of ways to solve it:
136 .IP \(bu 4
137 Upgrade both sides to unicode\-strings
138 .Sp
139 If your program does not need compatibility for Perl 5.6 and earlier,
140 the recommended approach is to apply appropriate IO disciplines, so all
141 data in your program become unicode\-strings. See encoding, open and
142 "binmode" in perlfunc for how.
143 .IP \(bu 4
144 Downgrade both sides to byte\-strings
145 .Sp
146 The other way works too, especially if you are sure that all your data
147 are under the same encoding, or if compatibility with older versions
148 of Perl is desired.
149 .Sp
150 You may downgrade strings with \f(CW\*(C`Encode::encode\*(C'\fR and \f(CW\*(C`utf8::encode\*(C'\fR.
151 See Encode and utf8 for details.
152 .IP \(bu 4
153 Specify the encoding for implicit byte\-string upgrading
154 .Sp
155 If you are confident that all byte\-strings will be in a specific
156 encoding like UTF\-8, \fIand\fR need not support older versions of Perl,
157 use the \f(CW\*(C`encoding\*(C'\fR pragma:
158 .Sp
159 .Vb 1
160 \& use encoding \*(Aqutf8\*(Aq;
161 .Ve
162 .Sp
163 Similarly, this will silence warnings from this module, and preserve the
164 default behaviour:
165 .Sp
166 .Vb 1
167 \& use encoding \*(Aqiso\-8859\-1\*(Aq;
168 .Ve
169 .Sp
170 However, note that \f(CW\*(C`use encoding\*(C'\fR actually had three distinct effects:
171 .RS 4
172 .IP \(bu 4
173 PerlIO layers for \fBSTDIN\fR and \fBSTDOUT\fR
174 .Sp
175 This is similar to what open pragma does.
176 .IP \(bu 4
177 Literal conversions
178 .Sp
179 This turns \fIall\fR literal string in your program into unicode\-strings
180 (equivalent to a \f(CW\*(C`use utf8\*(C'\fR), by decoding them using the specified
181 encoding.
182 .IP \(bu 4
183 Implicit upgrading for byte\-strings
184 .Sp
185 This will silence warnings from this module, as shown above.
186 .RE
187 .RS 4
188 .Sp
189 Because literal conversions also work on empty strings, it may surprise
190 some people:
191 .Sp
192 .Vb 1
193 \& use encoding \*(Aqbig5\*(Aq;
194 \&
195 \& my $byte_string = pack("C*", 0xA4, 0x40);
196 \& print length $a; # 2 here.
197 \& $a .= ""; # concatenating with a unicode string...
198 \& print length $a; # 1 here!
199 .Ve
200 .Sp
201 In other words, do not \f(CW\*(C`use encoding\*(C'\fR unless you are certain that the
202 program will not deal with any raw, 8\-bit binary data at all.
203 .Sp
204 However, the \f(CW\*(C`Filter => 1\*(C'\fR flavor of \f(CW\*(C`use encoding\*(C'\fR will \fInot\fR
205 affect implicit upgrading for byte\-strings, and is thus incapable of
206 silencing warnings from this module. See encoding for more details.
207 .RE
208 .SH CAVEATS
209 .IX Header "CAVEATS"
210 For Perl 5.9.4 or later, this module\*(Aqs effect is lexical.
211 .PP
212 For Perl versions prior to 5.9.4, this module affects the whole script,
213 instead of inside its lexical block.
214 .SH "SEE ALSO"
215 .IX Header "SEE ALSO"
216 perlunicode, perluniintro
217 .PP
218 open, utf8, encoding, Encode
219 .SH AUTHORS
220 .IX Header "AUTHORS"
221 Audrey Tang
222 .SH COPYRIGHT
223 .IX Header "COPYRIGHT"
224 Copyright 2004, 2005, 2006, 2007 by Audrey Tang <[email protected]>.
225 .PP
226 This program is free software; you can redistribute it and/or modify it
227 under the same terms as Perl itself.
228 .PP
229 See <http://www.perl.com/perl/misc/Artistic.html>