Debian Codegraph @

upstream/debian-unstable/man3/Encode::Guess.3perl

raw
1 .\" -*- mode: troff; coding: utf-8 -*-
2 .\" Automatically generated by Pod::Man 5.0102 (Pod::Simple 3.45)
3 .\"
4 .\" Standard preamble:
5 .\" ========================================================================
6 .de Sp \" Vertical space (when we can't use .PP)
7 .if t .sp .5v
8 .if n .sp
9 ..
10 .de Vb \" Begin verbatim text
11 .ft CW
12 .nf
13 .ne \\$1
14 ..
15 .de Ve \" End verbatim text
16 .ft R
17 .fi
18 ..
19 .\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>.
20 .ie n \{\
21 . ds C` ""
22 . ds C' ""
23 'br\}
24 .el\{\
25 . ds C`
26 . ds C'
27 'br\}
28 .\"
29 .\" Escape single quotes in literal strings from groff's Unicode transform.
30 .ie \n(.g .ds Aq \(aq
31 .el .ds Aq '
32 .\"
33 .\" If the F register is >0, we'll generate index entries on stderr for
34 .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
35 .\" entries marked with X<> in POD. Of course, you'll have to process the
36 .\" output yourself in some meaningful fashion.
37 .\"
38 .\" Avoid warning from groff about undefined register 'F'.
39 .de IX
40 ..
41 .nr rF 0
42 .if \n(.g .if rF .nr rF 1
43 .if (\n(rF:(\n(.g==0)) \{\
44 . if \nF \{\
45 . de IX
46 . tm Index:\\$1\t\\n%\t"\\$2"
47 ..
48 . if !\nF==2 \{\
49 . nr % 0
50 . nr F 2
51 . \}
52 . \}
53 .\}
54 .rr rF
55 .\" ========================================================================
56 .\"
57 .IX Title "Encode::Guess 3perl"
58 .TH Encode::Guess 3perl 2025-11-16 "perl v5.40.1" "Perl Programmers Reference Guide"
59 .\" For nroff, turn off justification. Always turn off hyphenation; it makes
60 .\" way too many mistakes in technical documents.
61 .if n .ad l
62 .nh
63 .SH NAME
64 Encode::Guess \-\- Guesses encoding from data
65 .SH SYNOPSIS
66 .IX Header "SYNOPSIS"
67 .Vb 1
68 \& # if you are sure $data won\*(Aqt contain anything bogus
69 \&
70 \& use Encode;
71 \& use Encode::Guess qw/euc\-jp shiftjis 7bit\-jis/;
72 \& my $utf8 = decode("Guess", $data);
73 \& my $data = encode("Guess", $utf8); # this doesn\*(Aqt work!
74 \&
75 \& # more elaborate way
76 \& use Encode::Guess;
77 \& my $enc = guess_encoding($data, qw/euc\-jp shiftjis 7bit\-jis/);
78 \& ref($enc) or die "Can\*(Aqt guess: $enc"; # trap error this way
79 \& $utf8 = $enc\->decode($data);
80 \& # or
81 \& $utf8 = decode($enc\->name, $data)
82 .Ve
83 .SH ABSTRACT
84 .IX Header "ABSTRACT"
85 Encode::Guess enables you to guess in what encoding a given data is
86 encoded, or at least tries to.
87 .SH DESCRIPTION
88 .IX Header "DESCRIPTION"
89 By default, it checks only ascii, utf8 and UTF\-16/32 with BOM.
90 .PP
91 .Vb 1
92 \& use Encode::Guess; # ascii/utf8/BOMed UTF
93 .Ve
94 .PP
95 To use it more practically, you have to give the names of encodings to
96 check (\fIsuspects\fR as follows). The name of suspects can either be
97 canonical names or aliases.
98 .PP
99 CAVEAT: Unlike UTF\-(16|32), BOM in utf8 is NOT AUTOMATICALLY STRIPPED.
100 .PP
101 .Vb 2
102 \& # tries all major Japanese Encodings as well
103 \& use Encode::Guess qw/euc\-jp shiftjis 7bit\-jis/;
104 .Ve
105 .PP
106 If the \f(CW$Encode::Guess::NoUTFAutoGuess\fR variable is set to a true
107 value, no heuristics will be applied to UTF8/16/32, and the result
108 will be limited to the suspects and \f(CW\*(C`ascii\*(C'\fR.
109 .IP Encode::Guess\->set_suspects 4
110 .IX Item "Encode::Guess->set_suspects"
111 You can also change the internal suspects list via \f(CW\*(C`set_suspects\*(C'\fR
112 method.
113 .Sp
114 .Vb 2
115 \& use Encode::Guess;
116 \& Encode::Guess\->set_suspects(qw/euc\-jp shiftjis 7bit\-jis/);
117 .Ve
118 .IP Encode::Guess\->add_suspects 4
119 .IX Item "Encode::Guess->add_suspects"
120 Or you can use \f(CW\*(C`add_suspects\*(C'\fR method. The difference is that
121 \&\f(CW\*(C`set_suspects\*(C'\fR flushes the current suspects list while
122 \&\f(CW\*(C`add_suspects\*(C'\fR adds.
123 .Sp
124 .Vb 5
125 \& use Encode::Guess;
126 \& Encode::Guess\->add_suspects(qw/euc\-jp shiftjis 7bit\-jis/);
127 \& # now the suspects are euc\-jp,shiftjis,7bit\-jis, AND
128 \& # euc\-kr,euc\-cn, and big5\-eten
129 \& Encode::Guess\->add_suspects(qw/euc\-kr euc\-cn big5\-eten/);
130 .Ve
131 .IP "Encode::decode(""Guess"" ...)" 4
132 .IX Item "Encode::decode(""Guess"" ...)"
133 When you are content with suspects list, you can now
134 .Sp
135 .Vb 1
136 \& my $utf8 = Encode::decode("Guess", $data);
137 .Ve
138 .IP Encode::Guess\->guess($data) 4
139 .IX Item "Encode::Guess->guess($data)"
140 But it will croak if:
141 .RS 4
142 .IP \(bu 4
143 Two or more suspects remain
144 .IP \(bu 4
145 No suspects left
146 .RE
147 .RS 4
148 .Sp
149 So you should instead try this;
150 .Sp
151 .Vb 1
152 \& my $decoder = Encode::Guess\->guess($data);
153 .Ve
154 .Sp
155 On success, \f(CW$decoder\fR is an object that is documented in
156 Encode::Encoding. So you can now do this;
157 .Sp
158 .Vb 1
159 \& my $utf8 = $decoder\->decode($data);
160 .Ve
161 .Sp
162 On failure, \f(CW$decoder\fR now contains an error message so the whole thing
163 would be as follows;
164 .Sp
165 .Vb 3
166 \& my $decoder = Encode::Guess\->guess($data);
167 \& die $decoder unless ref($decoder);
168 \& my $utf8 = $decoder\->decode($data);
169 .Ve
170 .RE
171 .IP "guess_encoding($data, [, \fIlist of suspects\fR])" 4
172 .IX Item "guess_encoding($data, [, list of suspects])"
173 You can also try \f(CW\*(C`guess_encoding\*(C'\fR function which is exported by
174 default. It takes \f(CW$data\fR to check and it also takes the list of
175 suspects by option. The optional suspect list is \fInot reflected\fR to
176 the internal suspects list.
177 .Sp
178 .Vb 5
179 \& my $decoder = guess_encoding($data, qw/euc\-jp euc\-kr euc\-cn/);
180 \& die $decoder unless ref($decoder);
181 \& my $utf8 = $decoder\->decode($data);
182 \& # check only ascii, utf8 and UTF\-(16|32) with BOM
183 \& my $decoder = guess_encoding($data);
184 .Ve
185 .SH CAVEATS
186 .IX Header "CAVEATS"
187 .IP \(bu 4
188 Because of the algorithm used, ISO\-8859 series and other single-byte
189 encodings do not work well unless either one of ISO\-8859 is the only
190 one suspect (besides ascii and utf8).
191 .Sp
192 .Vb 5
193 \& use Encode::Guess;
194 \& # perhaps ok
195 \& my $decoder = guess_encoding($data, \*(Aqlatin1\*(Aq);
196 \& # definitely NOT ok
197 \& my $decoder = guess_encoding($data, qw/latin1 greek/);
198 .Ve
199 .Sp
200 The reason is that Encode::Guess guesses encoding by trial and error.
201 It first splits \f(CW$data\fR into lines and tries to decode the line for each
202 suspect. It keeps it going until all but one encoding is eliminated
203 out of suspects list. ISO\-8859 series is just too successful for most
204 cases (because it fills almost all code points in \ex00\-\exff).
205 .IP \(bu 4
206 Do not mix national standard encodings and the corresponding vendor
207 encodings.
208 .Sp
209 .Vb 3
210 \& # a very bad idea
211 \& my $decoder
212 \& = guess_encoding($data, qw/shiftjis MacJapanese cp932/);
213 .Ve
214 .Sp
215 The reason is that vendor encoding is usually a superset of national
216 standard so it becomes too ambiguous for most cases.
217 .IP \(bu 4
218 On the other hand, mixing various national standard encodings
219 automagically works unless \f(CW$data\fR is too short to allow for guessing.
220 .Sp
221 .Vb 6
222 \& # This is ok if $data is long enough
223 \& my $decoder =
224 \& guess_encoding($data, qw/euc\-cn
225 \& euc\-jp shiftjis 7bit\-jis
226 \& euc\-kr
227 \& big5\-eten/);
228 .Ve
229 .IP \(bu 4
230 DO NOT PUT TOO MANY SUSPECTS! Don't you try something like this!
231 .Sp
232 .Vb 2
233 \& my $decoder = guess_encoding($data,
234 \& Encode\->encodings(":all"));
235 .Ve
236 .PP
237 It is, after all, just a guess. You should alway be explicit when it
238 comes to encodings. But there are some, especially Japanese,
239 environment that guess-coding is a must. Use this module with care.
240 .SH "TO DO"
241 .IX Header "TO DO"
242 Encode::Guess does not work on EBCDIC platforms.
243 .SH "SEE ALSO"
244 .IX Header "SEE ALSO"
245 Encode, Encode::Encoding