Debian Codegraph @

upstream/debian-unstable/man3/encoding::warnings.3perl

raw
1 .\" -*- mode: troff; coding: utf-8 -*-
2 .\" Automatically generated by Pod::Man 5.0102 (Pod::Simple 3.45)
3 .\"
4 .\" Standard preamble:
5 .\" ========================================================================
6 .de Sp \" Vertical space (when we can't use .PP)
7 .if t .sp .5v
8 .if n .sp
9 ..
10 .de Vb \" Begin verbatim text
11 .ft CW
12 .nf
13 .ne \\$1
14 ..
15 .de Ve \" End verbatim text
16 .ft R
17 .fi
18 ..
19 .\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>.
20 .ie n \{\
21 . ds C` ""
22 . ds C' ""
23 'br\}
24 .el\{\
25 . ds C`
26 . ds C'
27 'br\}
28 .\"
29 .\" Escape single quotes in literal strings from groff's Unicode transform.
30 .ie \n(.g .ds Aq \(aq
31 .el .ds Aq '
32 .\"
33 .\" If the F register is >0, we'll generate index entries on stderr for
34 .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
35 .\" entries marked with X<> in POD. Of course, you'll have to process the
36 .\" output yourself in some meaningful fashion.
37 .\"
38 .\" Avoid warning from groff about undefined register 'F'.
39 .de IX
40 ..
41 .nr rF 0
42 .if \n(.g .if rF .nr rF 1
43 .if (\n(rF:(\n(.g==0)) \{\
44 . if \nF \{\
45 . de IX
46 . tm Index:\\$1\t\\n%\t"\\$2"
47 ..
48 . if !\nF==2 \{\
49 . nr % 0
50 . nr F 2
51 . \}
52 . \}
53 .\}
54 .rr rF
55 .\" ========================================================================
56 .\"
57 .IX Title "encoding::warnings 3perl"
58 .TH encoding::warnings 3perl 2025-11-16 "perl v5.40.1" "Perl Programmers Reference Guide"
59 .\" For nroff, turn off justification. Always turn off hyphenation; it makes
60 .\" way too many mistakes in technical documents.
61 .if n .ad l
62 .nh
63 .SH NAME
64 encoding::warnings \- Warn on implicit encoding conversions
65 .SH VERSION
66 .IX Header "VERSION"
67 This document describes version 0.13 of encoding::warnings, released
68 June 20, 2016.
69 .SH NOTICE
70 .IX Header "NOTICE"
71 As of Perl 5.26.0, this module has no effect. The internal Perl feature
72 that was used to implement this module has been removed. In recent years,
73 much work has been done on the Perl core to eliminate discrepancies in the
74 treatment of upgraded versus downgraded strings. In addition, the
75 encoding pragma, which caused many of the problems, is no longer
76 supported. Thus, the warnings this module produced are no longer
77 necessary.
78 .PP
79 Hence, if you load this module on Perl 5.26.0, you will get one warning
80 that the module is no longer supported; and the module will do nothing
81 thereafter.
82 .SH SYNOPSIS
83 .IX Header "SYNOPSIS"
84 .Vb 1
85 \& use encoding::warnings; # or \*(AqFATAL\*(Aq to raise fatal exceptions
86 \&
87 \& utf8::encode($a = chr(20000)); # a byte\-string (raw bytes)
88 \& $b = chr(20000); # a unicode\-string (wide characters)
89 \&
90 \& # "Bytes implicitly upgraded into wide characters as iso\-8859\-1"
91 \& $c = $a . $b;
92 .Ve
93 .SH DESCRIPTION
94 .IX Header "DESCRIPTION"
95 .SS "Overview of the problem"
96 .IX Subsection "Overview of the problem"
97 By default, there is a fundamental asymmetry in Perl's unicode model:
98 implicit upgrading from byte-strings to unicode-strings assumes that
99 they were encoded in \fIISO 8859\-1 (Latin\-1)\fR, but unicode-strings are
100 downgraded with UTF\-8 encoding. This happens because the first 256
101 codepoints in Unicode happens to agree with Latin\-1.
102 .PP
103 However, this silent upgrading can easily cause problems, if you happen
104 to mix unicode strings with non\-Latin1 data \-\- i.e. byte-strings encoded
105 in UTF\-8 or other encodings. The error will not manifest until the
106 combined string is written to output, at which time it would be impossible
107 to see where did the silent upgrading occur.
108 .SS "Detecting the problem"
109 .IX Subsection "Detecting the problem"
110 This module simplifies the process of diagnosing such problems. Just put
111 this line on top of your main program:
112 .PP
113 .Vb 1
114 \& use encoding::warnings;
115 .Ve
116 .PP
117 Afterwards, implicit upgrading of high-bit bytes will raise a warning.
118 Ex.: \f(CW\*(C`Bytes implicitly upgraded into wide characters as iso\-8859\-1 at
119 \&\- line 7\*(C'\fR.
120 .PP
121 However, strings composed purely of ASCII code points (\f(CW0x00\fR..\f(CW0x7F\fR)
122 will \fInot\fR trigger this warning.
123 .PP
124 You can also make the warnings fatal by importing this module as:
125 .PP
126 .Vb 1
127 \& use encoding::warnings \*(AqFATAL\*(Aq;
128 .Ve
129 .SS "Solving the problem"
130 .IX Subsection "Solving the problem"
131 Most of the time, this warning occurs when a byte-string is concatenated
132 with a unicode-string. There are a number of ways to solve it:
133 .IP \(bu 4
134 Upgrade both sides to unicode-strings
135 .Sp
136 If your program does not need compatibility for Perl 5.6 and earlier,
137 the recommended approach is to apply appropriate IO disciplines, so all
138 data in your program become unicode-strings. See encoding, open and
139 "binmode" in perlfunc for how.
140 .IP \(bu 4
141 Downgrade both sides to byte-strings
142 .Sp
143 The other way works too, especially if you are sure that all your data
144 are under the same encoding, or if compatibility with older versions
145 of Perl is desired.
146 .Sp
147 You may downgrade strings with \f(CW\*(C`Encode::encode\*(C'\fR and \f(CW\*(C`utf8::encode\*(C'\fR.
148 See Encode and utf8 for details.
149 .IP \(bu 4
150 Specify the encoding for implicit byte-string upgrading
151 .Sp
152 If you are confident that all byte-strings will be in a specific
153 encoding like UTF\-8, \fIand\fR need not support older versions of Perl,
154 use the \f(CW\*(C`encoding\*(C'\fR pragma:
155 .Sp
156 .Vb 1
157 \& use encoding \*(Aqutf8\*(Aq;
158 .Ve
159 .Sp
160 Similarly, this will silence warnings from this module, and preserve the
161 default behaviour:
162 .Sp
163 .Vb 1
164 \& use encoding \*(Aqiso\-8859\-1\*(Aq;
165 .Ve
166 .Sp
167 However, note that \f(CW\*(C`use encoding\*(C'\fR actually had three distinct effects:
168 .RS 4
169 .IP \(bu 4
170 PerlIO layers for \fBSTDIN\fR and \fBSTDOUT\fR
171 .Sp
172 This is similar to what open pragma does.
173 .IP \(bu 4
174 Literal conversions
175 .Sp
176 This turns \fIall\fR literal string in your program into unicode-strings
177 (equivalent to a \f(CW\*(C`use utf8\*(C'\fR), by decoding them using the specified
178 encoding.
179 .IP \(bu 4
180 Implicit upgrading for byte-strings
181 .Sp
182 This will silence warnings from this module, as shown above.
183 .RE
184 .RS 4
185 .Sp
186 Because literal conversions also work on empty strings, it may surprise
187 some people:
188 .Sp
189 .Vb 1
190 \& use encoding \*(Aqbig5\*(Aq;
191 \&
192 \& my $byte_string = pack("C*", 0xA4, 0x40);
193 \& print length $a; # 2 here.
194 \& $a .= ""; # concatenating with a unicode string...
195 \& print length $a; # 1 here!
196 .Ve
197 .Sp
198 In other words, do not \f(CW\*(C`use encoding\*(C'\fR unless you are certain that the
199 program will not deal with any raw, 8\-bit binary data at all.
200 .Sp
201 However, the \f(CW\*(C`Filter => 1\*(C'\fR flavor of \f(CW\*(C`use encoding\*(C'\fR will \fInot\fR
202 affect implicit upgrading for byte-strings, and is thus incapable of
203 silencing warnings from this module. See encoding for more details.
204 .RE
205 .SH CAVEATS
206 .IX Header "CAVEATS"
207 For Perl 5.9.4 or later, this module's effect is lexical.
208 .PP
209 For Perl versions prior to 5.9.4, this module affects the whole script,
210 instead of inside its lexical block.
211 .SH "SEE ALSO"
212 .IX Header "SEE ALSO"
213 perlunicode, perluniintro
214 .PP
215 open, utf8, encoding, Encode
216 .SH AUTHORS
217 .IX Header "AUTHORS"
218 Audrey Tang
219 .SH COPYRIGHT
220 .IX Header "COPYRIGHT"
221 Copyright 2004, 2005, 2006, 2007 by Audrey Tang <[email protected]>.
222 .PP
223 This program is free software; you can redistribute it and/or modify it
224 under the same terms as Perl itself.
225 .PP
226 See <http://www.perl.com/perl/misc/Artistic.html>