Debian Codegraph @

upstream/mageia-cauldron/man3pm/Encode::Guess.3pm

raw
1 .\" -*- mode: troff; coding: utf-8 -*-
2 .\" Automatically generated by Pod::Man v6.0.2 (Pod::Simple 3.45)
3 .\"
4 .\" Standard preamble:
5 .\" ========================================================================
6 .de Sp \" Vertical space (when we can't use .PP)
7 .if t .sp .5v
8 .if n .sp
9 ..
10 .de Vb \" Begin verbatim text
11 .ft CW
12 .nf
13 .ne \\$1
14 ..
15 .de Ve \" End verbatim text
16 .ft R
17 .fi
18 ..
19 .\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>.
20 .ie n \{\
21 . ds C` ""
22 . ds C' ""
23 'br\}
24 .el\{\
25 . ds C`
26 . ds C'
27 'br\}
28 .\"
29 .\" Escape single quotes in literal strings from groff's Unicode transform.
30 .ie \n(.g .ds Aq \(aq
31 .el .ds Aq '
32 .\"
33 .\" If the F register is >0, we'll generate index entries on stderr for
34 .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
35 .\" entries marked with X<> in POD. Of course, you'll have to process the
36 .\" output yourself in some meaningful fashion.
37 .\"
38 .\" Avoid warning from groff about undefined register 'F'.
39 .de IX
40 ..
41 .nr rF 0
42 .if \n(.g .if rF .nr rF 1
43 .if (\n(rF:(\n(.g==0)) \{\
44 . if \nF \{\
45 . de IX
46 . tm Index:\\$1\t\\n%\t"\\$2"
47 ..
48 . if !\nF==2 \{\
49 . nr % 0
50 . nr F 2
51 . \}
52 . \}
53 .\}
54 .rr rF
55 .\"
56 .\" Required to disable full justification in groff 1.23.0.
57 .if n .ds AD l
58 .\" ========================================================================
59 .\"
60 .IX Title "Encode::Guess 3pm"
61 .TH Encode::Guess 3pm 2023-12-22 "perl v5.42.0" "Perl Programmers Reference Guide"
62 .\" For nroff, turn off justification. Always turn off hyphenation; it makes
63 .\" way too many mistakes in technical documents.
64 .if n .ad l
65 .nh
66 .SH NAME
67 Encode::Guess \-\- Guesses encoding from data
68 .SH SYNOPSIS
69 .IX Header "SYNOPSIS"
70 .Vb 1
71 \& # if you are sure $data won\*(Aqt contain anything bogus
72 \&
73 \& use Encode;
74 \& use Encode::Guess qw/euc\-jp shiftjis 7bit\-jis/;
75 \& my $utf8 = decode("Guess", $data);
76 \& my $data = encode("Guess", $utf8); # this doesn\*(Aqt work!
77 \&
78 \& # more elaborate way
79 \& use Encode::Guess;
80 \& my $enc = guess_encoding($data, qw/euc\-jp shiftjis 7bit\-jis/);
81 \& ref($enc) or die "Can\*(Aqt guess: $enc"; # trap error this way
82 \& $utf8 = $enc\->decode($data);
83 \& # or
84 \& $utf8 = decode($enc\->name, $data)
85 .Ve
86 .SH ABSTRACT
87 .IX Header "ABSTRACT"
88 Encode::Guess enables you to guess in what encoding a given data is
89 encoded, or at least tries to.
90 .SH DESCRIPTION
91 .IX Header "DESCRIPTION"
92 By default, it checks only ascii, utf8 and UTF\-16/32 with BOM.
93 .PP
94 .Vb 1
95 \& use Encode::Guess; # ascii/utf8/BOMed UTF
96 .Ve
97 .PP
98 To use it more practically, you have to give the names of encodings to
99 check (\fIsuspects\fR as follows). The name of suspects can either be
100 canonical names or aliases.
101 .PP
102 CAVEAT: Unlike UTF\-(16|32), BOM in utf8 is NOT AUTOMATICALLY STRIPPED.
103 .PP
104 .Vb 2
105 \& # tries all major Japanese Encodings as well
106 \& use Encode::Guess qw/euc\-jp shiftjis 7bit\-jis/;
107 .Ve
108 .PP
109 If the \f(CW$Encode::Guess::NoUTFAutoGuess\fR variable is set to a true
110 value, no heuristics will be applied to UTF8/16/32, and the result
111 will be limited to the suspects and \f(CW\*(C`ascii\*(C'\fR.
112 .IP Encode::Guess\->set_suspects 4
113 .IX Item "Encode::Guess->set_suspects"
114 You can also change the internal suspects list via \f(CW\*(C`set_suspects\*(C'\fR
115 method.
116 .Sp
117 .Vb 2
118 \& use Encode::Guess;
119 \& Encode::Guess\->set_suspects(qw/euc\-jp shiftjis 7bit\-jis/);
120 .Ve
121 .IP Encode::Guess\->add_suspects 4
122 .IX Item "Encode::Guess->add_suspects"
123 Or you can use \f(CW\*(C`add_suspects\*(C'\fR method. The difference is that
124 \&\f(CW\*(C`set_suspects\*(C'\fR flushes the current suspects list while
125 \&\f(CW\*(C`add_suspects\*(C'\fR adds.
126 .Sp
127 .Vb 5
128 \& use Encode::Guess;
129 \& Encode::Guess\->add_suspects(qw/euc\-jp shiftjis 7bit\-jis/);
130 \& # now the suspects are euc\-jp,shiftjis,7bit\-jis, AND
131 \& # euc\-kr,euc\-cn, and big5\-eten
132 \& Encode::Guess\->add_suspects(qw/euc\-kr euc\-cn big5\-eten/);
133 .Ve
134 .IP "Encode::decode(""Guess"" ...)" 4
135 .IX Item "Encode::decode(""Guess"" ...)"
136 When you are content with suspects list, you can now
137 .Sp
138 .Vb 1
139 \& my $utf8 = Encode::decode("Guess", $data);
140 .Ve
141 .IP Encode::Guess\->guess($data) 4
142 .IX Item "Encode::Guess->guess($data)"
143 But it will croak if:
144 .RS 4
145 .IP \(bu 4
146 Two or more suspects remain
147 .IP \(bu 4
148 No suspects left
149 .RE
150 .RS 4
151 .Sp
152 So you should instead try this;
153 .Sp
154 .Vb 1
155 \& my $decoder = Encode::Guess\->guess($data);
156 .Ve
157 .Sp
158 On success, \f(CW$decoder\fR is an object that is documented in
159 Encode::Encoding. So you can now do this;
160 .Sp
161 .Vb 1
162 \& my $utf8 = $decoder\->decode($data);
163 .Ve
164 .Sp
165 On failure, \f(CW$decoder\fR now contains an error message so the whole thing
166 would be as follows;
167 .Sp
168 .Vb 3
169 \& my $decoder = Encode::Guess\->guess($data);
170 \& die $decoder unless ref($decoder);
171 \& my $utf8 = $decoder\->decode($data);
172 .Ve
173 .RE
174 .IP "guess_encoding($data, [, \fIlist of suspects\fR])" 4
175 .IX Item "guess_encoding($data, [, list of suspects])"
176 You can also try \f(CW\*(C`guess_encoding\*(C'\fR function which is exported by
177 default. It takes \f(CW$data\fR to check and it also takes the list of
178 suspects by option. The optional suspect list is \fInot reflected\fR to
179 the internal suspects list.
180 .Sp
181 .Vb 5
182 \& my $decoder = guess_encoding($data, qw/euc\-jp euc\-kr euc\-cn/);
183 \& die $decoder unless ref($decoder);
184 \& my $utf8 = $decoder\->decode($data);
185 \& # check only ascii, utf8 and UTF\-(16|32) with BOM
186 \& my $decoder = guess_encoding($data);
187 .Ve
188 .SH CAVEATS
189 .IX Header "CAVEATS"
190 .IP \(bu 4
191 Because of the algorithm used, ISO\-8859 series and other single\-byte
192 encodings do not work well unless either one of ISO\-8859 is the only
193 one suspect (besides ascii and utf8).
194 .Sp
195 .Vb 5
196 \& use Encode::Guess;
197 \& # perhaps ok
198 \& my $decoder = guess_encoding($data, \*(Aqlatin1\*(Aq);
199 \& # definitely NOT ok
200 \& my $decoder = guess_encoding($data, qw/latin1 greek/);
201 .Ve
202 .Sp
203 The reason is that Encode::Guess guesses encoding by trial and error.
204 It first splits \f(CW$data\fR into lines and tries to decode the line for each
205 suspect. It keeps it going until all but one encoding is eliminated
206 out of suspects list. ISO\-8859 series is just too successful for most
207 cases (because it fills almost all code points in \ex00\-\exff).
208 .IP \(bu 4
209 Do not mix national standard encodings and the corresponding vendor
210 encodings.
211 .Sp
212 .Vb 3
213 \& # a very bad idea
214 \& my $decoder
215 \& = guess_encoding($data, qw/shiftjis MacJapanese cp932/);
216 .Ve
217 .Sp
218 The reason is that vendor encoding is usually a superset of national
219 standard so it becomes too ambiguous for most cases.
220 .IP \(bu 4
221 On the other hand, mixing various national standard encodings
222 automagically works unless \f(CW$data\fR is too short to allow for guessing.
223 .Sp
224 .Vb 6
225 \& # This is ok if $data is long enough
226 \& my $decoder =
227 \& guess_encoding($data, qw/euc\-cn
228 \& euc\-jp shiftjis 7bit\-jis
229 \& euc\-kr
230 \& big5\-eten/);
231 .Ve
232 .IP \(bu 4
233 DO NOT PUT TOO MANY SUSPECTS! Don\*(Aqt you try something like this!
234 .Sp
235 .Vb 2
236 \& my $decoder = guess_encoding($data,
237 \& Encode\->encodings(":all"));
238 .Ve
239 .PP
240 It is, after all, just a guess. You should alway be explicit when it
241 comes to encodings. But there are some, especially Japanese,
242 environment that guess\-coding is a must. Use this module with care.
243 .SH "TO DO"
244 .IX Header "TO DO"
245 Encode::Guess does not work on EBCDIC platforms.
246 .SH "SEE ALSO"
247 .IX Header "SEE ALSO"
248 Encode, Encode::Encoding