Debian Codegraph @

upstream/debian-trixie/man1/preconv.1

raw
1 .TH preconv 1 "3 June 2025" "groff 1.23.0"
2 .SH Name
3 preconv \- prepare files for typesetting with
4 .I groff
5 .
6 .
7 .\" ====================================================================
8 .\" Legal Terms
9 .\" ====================================================================
10 .\"
11 .\" Copyright (C) 2006-2020 Free Software Foundation, Inc.
12 .\"
13 .\" Permission is granted to make and distribute verbatim copies of this
14 .\" manual provided the copyright notice and this permission notice are
15 .\" preserved on all copies.
16 .\"
17 .\" Permission is granted to copy and distribute modified versions of
18 .\" this manual under the conditions for verbatim copying, provided that
19 .\" the entire resulting derived work is distributed under the terms of
20 .\" a permission notice identical to this one.
21 .\"
22 .\" Permission is granted to copy and distribute translations of this
23 .\" manual into another language, under the above conditions for
24 .\" modified versions, except that this permission notice may be
25 .\" included in translations approved by the Free Software Foundation
26 .\" instead of in the original English.
27 .
28 .
29 .\" Save and disable compatibility mode (for, e.g., Solaris 10/11).
30 .do nr *groff_preconv_1_man_C \n[.cp]
31 .cp 0
32 .
33 .\" Define fallback for groff 1.23's MR macro if the system lacks it.
34 .nr do-fallback 0
35 .if !\n(.f .nr do-fallback 1 \" mandoc
36 .if \n(.g .if !d MR .nr do-fallback 1 \" older groff
37 .if !\n(.g .nr do-fallback 1 \" non-groff *roff
38 .if \n[do-fallback] \{\
39 . de MR
40 . ie \\n(.$=1 \
41 . I \%\\$1
42 . el \
43 . IR \%\\$1 (\\$2)\\$3
44 . .
45 .\}
46 .rr do-fallback
47 .
48 .
49 .\" ====================================================================
50 .SH Synopsis
51 .\" ====================================================================
52 .
53 .SY preconv
54 .RB [ \-dr ]
55 .RB [ \-D\~\c
56 .IR fallback-encoding ]
57 .RB [ \-e\~\c
58 .IR encoding ]
59 .RI [ file\~ .\|.\|.]
60 .YS
61 .
62 .
63 .SY preconv
64 .B \-h
65 .
66 .SY preconv
67 .B \-\-help
68 .YS
69 .
70 .
71 .SY preconv
72 .B \-v
73 .
74 .SY preconv
75 .B \-\-version
76 .YS
77 .
78 .
79 .\" ====================================================================
80 .SH Description
81 .\" ====================================================================
82 .
83 .I preconv
84 reads each
85 .IR file ,
86 converts its encoded characters to a form
87 .MR \%troff 1
88 can interpret,
89 and sends the result to the standard output stream.
90 .
91 Currently,
92 this means that code points in the range 0\[en]127
93 (in US-ASCII,
94 ISO\~8859,
95 or Unicode)
96 remain as-is and the remainder are converted to the
97 .I groff
98 special character form
99 .RB \[lq] \[rs][\c
100 .BI u XXXX ]\c
101 \[rq],
102 where
103 .I XXXX
104 is a hexadecimal number of four to six digits corresponding to a Unicode
105 code point.
106 .
107 By default,
108 .I preconv
109 also inserts a
110 .I roff
111 .B .lf
112 request at the beginning of each
113 .IR file ,
114 identifying it for the benefit of later processing
115 (including diagnostic messages);
116 the
117 .B \-r
118 option suppresses this behavior.
119 .
120 .
121 .PP
122 In typical usage scenarios,
123 .I preconv
124 need not be run directly;
125 instead it should be invoked with the
126 .B \-k
127 or
128 .B \-K
129 options of
130 .IR groff .
131 .
132 If no
133 .I file
134 operands are given on the command line,
135 or if
136 .I file
137 is
138 .RB \[lq] \- \[rq],
139 the standard input stream is read.
140 .
141 .
142 .PP
143 .I preconv
144 tries to find the input encoding with the following algorithm,
145 stopping at the first success.
146 .
147 .
148 .IP 1. 4n
149 If the input encoding has been explicitly specified with option
150 .BR \-e ,
151 use it.
152 .
153 .
154 .IP 2.
155 If the input starts with a Unicode Byte Order Mark,
156 determine the encoding as UTF-8,
157 UTF-16,
158 or UTF-32 accordingly.
159 .
160 .
161 .IP 3.
162 If the input stream is seekable,
163 check the first and second input lines for a recognized GNU\~Emacs
164 file-local variable identifying the character encoding,
165 here referred to as the \[lq]coding tag\[rq] for brevity.
166 .
167 If found,
168 use it.
169 .
170 .
171 .IP 4.
172 If the input stream is seekable,
173 and if the
174 .I uchardet
175 library is available on the system,
176 use it to try to infer the encoding of the file.
177 .
178 .
179 .IP 5.
180 If the
181 .B \-D
182 option specifies an encoding,
183 use it.
184 .
185 .
186 .IP 6.
187 Use the encoding specified by the current locale
188 .RI ( LC_CTYPE ),
189 unless the locale is
190 \[lq]C\[rq],
191 \[lq]POSIX\[rq],
192 or empty,
193 in which case assume Latin-1
194 (ISO\~8859-1).
195 .
196 .
197 .PP
198 The coding tag and
199 .I uchardet
200 methods in the above procedure rely upon a seekable input stream;
201 when
202 .I preconv
203 reads from a pipe,
204 the stream is not seekable,
205 and these detection methods are skipped.
206 .
207 If character encoding detection of your input files is unreliable,
208 arrange for one of the other methods to succeed by using
209 .IR preconv 's
210 .B \-D
211 or
212 .B \-e
213 options,
214 or by configuring your locale appropriately.
215 .
216 .I groff
217 also supports a
218 .I \%GROFF_ENCODING
219 environment variable,
220 which can be overridden by its
221 .B \-K
222 option.
223 .
224 Valid values for
225 (or parameters to)
226 all of these are enumerated in the lists of recognized coding tags in
227 the next subsection,
228 and are further influenced by
229 .I iconv
230 library support.
231 .
232 .
233 .\" ====================================================================
234 .SS "Coding tags"
235 .\" ====================================================================
236 .
237 Text editors that support more than a single character encoding need
238 tags within the input files to mark the file's encoding.
239 .
240 While it is possible to guess the right input encoding with the help of
241 heuristics that are reliable for a preponderance of natural language
242 texts,
243 they are not absolutely reliable.
244 .
245 Heuristics can fail on inputs that are too short or don't represent a
246 natural language.
247 .
248 .
249 .PP
250 Consequently,
251 .I preconv
252 supports the coding tag convention used by GNU\~Emacs
253 (with some restrictions).
254 .
255 This notation appears in specially marked regions of an input file
256 designated for \[lq]file-local variables\[rq].
257 .
258 .
259 .PP
260 .I preconv
261 interprets the following syntax if it occurs in a
262 .I roff
263 comment
264 in the first or second line of the input file.
265 .
266 Both \[lq]\[rs]"\[rq] and \[lq]\[rs]#\[rq] comment forms are recognized,
267 but the control
268 (or no-break control)
269 character must be the default and must begin the line.
270 .
271 Similarly,
272 the escape character must be the default.
273 .
274 .
275 .RS
276 .EX
277 .B \-*\- \c
278 .RB [.\|.\|. ; ]\~\c
279 .B coding: \c
280 .I encoding\c
281 .RB [ ;\~ .\|.\|.\&]\~\c
282 .B \-*\-
283 .EE
284 .RE
285 .
286 .
287 .PP
288 The only variable
289 .I preconv
290 interprets is \[lq]coding\[rq],
291 which can take the values listed below.
292 .
293 .
294 .PP
295 The following list comprises all MIME \[lq]charset\[rq] parameter values
296 recognized,
297 case-insensitively,
298 by
299 .IR preconv .
300 .
301 .RS
302 \%big5,
303 \%cp1047,
304 \%euc\-jp,
305 \%euc\-kr,
306 \%gb2312,
307 \%iso\-8859\-1,
308 \%iso\-8859\-2,
309 \%iso\-8859\-5,
310 \%iso\-8859\-7,
311 \%iso\-8859\-9,
312 \%iso\-8859\-13,
313 \%iso\-8859\-15,
314 \%koi8\-r,
315 \%us\-ascii,
316 \%utf\-8,
317 \%utf\-16,
318 \%utf\-16be,
319 \%utf\-16le
320 .RE
321 .
322 .
323 .PP
324 In addition,
325 the following list of other coding tags is recognized,
326 each of which is mapped to an appropriate value from the list above.
327 .
328 .RS
329 \%ascii,
330 \%chinese\-big5,
331 \%chinese\-euc,
332 \%chinese\-iso\-8bit,
333 \%cn\-big5,
334 \%cn\-gb,
335 \%cn\-gb\-2312,
336 \%cp878,
337 \%csascii,
338 \%csisolatin1,
339 \%cyrillic\-iso\-8bit,
340 \%cyrillic\-koi8,
341 \%euc\-china,
342 \%euc\-cn,
343 \%euc\-japan,
344 \%euc\-japan\-1990,
345 \%euc\-korea,
346 \%greek\-iso\-8bit,
347 \%iso\-10646/utf8,
348 \%iso\-10646/utf\-8,
349 \%iso\-latin\-1,
350 \%iso\-latin\-2,
351 \%iso\-latin\-5,
352 \%iso\-latin\-7,
353 \%iso\-latin\-9,
354 \%japanese\-euc,
355 \%japanese\-iso\-8bit,
356 \%jis8,
357 \%koi8,
358 \%korean\-euc,
359 \%korean\-iso\-8bit,
360 \%latin\-0,
361 \%latin1,
362 \%latin\-1,
363 \%latin\-2,
364 \%latin\-5,
365 \%latin\-7,
366 \%latin\-9,
367 \%mule\-utf\-8,
368 \%mule\-utf\-16,
369 \%mule\-utf\-16be,
370 \%mule\-utf\-16\-be,
371 \%mule\-utf\-16be\-with\-signature,
372 \%mule\-utf\-16le,
373 \%mule\-utf\-16\-le,
374 \%mule\-utf\-16le\-with\-signature,
375 \%utf8,
376 \%utf\-16\-be,
377 \%utf\-16\-be\-with\-signature,
378 \%utf\-16be\-with\-signature,
379 \%utf\-16\-le,
380 \%utf\-16\-le\-with\-signature,
381 \%utf\-16le\-with\-signature
382 .RE
383 .
384 .
385 .PP
386 Trailing
387 \[lq]\-dos\[rq],
388 \[lq]\-unix\[rq],
389 and
390 \[lq]\-mac\[rq]
391 suffixes on coding tags
392 (which indicate the end-of-line convention used in the file)
393 are disregarded for the purpose of comparison with the above tags.
394 .
395 .
396 .\" ====================================================================
397 .SS "\f[I]iconv\f[] support"
398 .\" ====================================================================
399 .
400 While
401 .I preconv
402 recognizes all of the coding tags listed above,
403 it is capable on its own of interpreting only three encodings:
404 Latin-1,
405 code page 1047,
406 and UTF-8.
407 .
408 If
409 .I iconv
410 support is configured at compile time and available at run time,
411 all others are passed to
412 .I iconv
413 library functions,
414 which may recognize many additional encoding strings.
415 .
416 The command
417 .RB \[lq] preconv\~\-v \[rq]
418 discloses whether
419 .I iconv
420 support is configured.
421 .
422 .
423 .PP
424 The use of
425 .I iconv
426 means that characters in the input that encode invalid code points for
427 that encoding may be dropped from the output stream or mapped to the
428 Unicode replacement character
429 (U+FFFD).
430 .
431 Compare the following examples using the input \[lq]caf\['e]\[rq]
432 (note the \[lq]e\[rq] with an acute accent),
433 which due to its short length challenges inference of the encoding used.
434 .
435 .RS
436 .EX
437 printf \[aq]caf\[rs]351\[rs]n\[aq] | LC_ALL=en_US.UTF\-8 preconv
438 printf \[aq]caf\[rs]351\[rs]n\[aq] | preconv \-e us\-ascii
439 printf \[aq]caf\[rs]351\[rs]n\[aq] | preconv \-e latin\-1
440 .EE
441 .RE
442 .
443 The fate of the accented \[lq]e\[rq] differs in each case.
444 .
445 In the first,
446 .I uchardet
447 fails to detect an encoding
448 (though the library on your system may behave differently)
449 and
450 .I preconv
451 falls back to the locale settings,
452 where octal 351 starts an incomplete UTF-8 sequence and results in the
453 Unicode replacement character.
454 .
455 In the second,
456 it is not a representable character in the declared input encoding of
457 US-ASCII and is discarded by
458 .IR iconv .
459 .
460 In the last,
461 it is correctly detected and mapped.
462 .
463 .
464 .\" ====================================================================
465 .SS Limitations
466 .\" ====================================================================
467 .
468 .I preconv
469 cannot perform any transformation on input that it cannot see.
470 .
471 Examples include files that are interpolated by preprocessors that run
472 subsequently,
473 including
474 .MR \%soelim 1 ;
475 files included by
476 .I \%troff
477 itself through
478 .RB \[lq] so \[rq]
479 and similar requests;
480 and string definitions passed to
481 .I \%troff
482 through its
483 .B \-d
484 command-line option.
485 .
486 .
487 .P
488 .I preconv
489 assumes that its input uses the default escape character,
490 a backslash
491 .BR \[rs] ,
492 and writes special character escape sequences accordingly.
493 .
494 .
495 .\" ====================================================================
496 .SH Options
497 .\" ====================================================================
498 .
499 .B \-h
500 and
501 .B \-\-help
502 display a usage message,
503 while
504 .B \-v
505 and
506 .B \-\-version
507 show version information;
508 all exit afterward.
509 .
510 .
511 .TP
512 .B \-d
513 Emit debugging messages to the standard error stream.
514 .
515 .
516 .TP
517 .BI \-D\~ fallback-encoding
518 Report
519 .I fallback-encoding
520 if all detection methods fail.
521 .
522 .
523 .TP
524 .BI \-e\~ encoding
525 Skip detection and assume
526 .IR encoding ;
527 see
528 .IR groff 's
529 .B \-K
530 option.
531 .
532 .
533 .TP
534 .B \-r
535 Write files \[lq]raw\[rq];
536 do not add
537 .B .lf
538 requests.
539 .
540 .
541 .\" ====================================================================
542 .SH "See also"
543 .\" ====================================================================
544 .
545 .MR groff 1 ,
546 .MR iconv 3 ,
547 .MR locale 7
548 .
549 .
550 .\" Restore compatibility mode (for, e.g., Solaris 10/11).
551 .cp \n[*groff_preconv_1_man_C]
552 .do rr *groff_preconv_1_man_C
553 .
554 .
555 .\" Local Variables:
556 .\" fill-column: 72
557 .\" mode: nroff
558 .\" End:
559 .\" vim: set filetype=groff textwidth=72: