Debian Codegraph @

upstream/debian-unstable/man1/preconv.1

raw
1 .TH preconv 1 "2026-04-22" "groff 1.24.1"
2 .SH Name
3 preconv \- prepare files for typesetting with
4 .I groff
5 .
6 .
7 .\" ====================================================================
8 .\" Legal Terms
9 .\" ====================================================================
10 .\"
11 .\" Copyright (C) 2006-2024 Free Software Foundation, Inc.
12 .\"
13 .\" Permission is granted to make and distribute verbatim copies of this
14 .\" manual provided the copyright notice and this permission notice are
15 .\" preserved on all copies.
16 .\"
17 .\" Permission is granted to copy and distribute modified versions of
18 .\" this manual under the conditions for verbatim copying, provided that
19 .\" the entire resulting derived work is distributed under the terms of
20 .\" a permission notice identical to this one.
21 .\"
22 .\" Permission is granted to copy and distribute translations of this
23 .\" manual into another language, under the above conditions for
24 .\" modified versions, except that this permission notice may be
25 .\" included in translations approved by the Free Software Foundation
26 .\" instead of in the original English.
27 .
28 .
29 .\" Save and disable compatibility mode (for, e.g., Solaris 10/11).
30 .do nr *groff_preconv_1_man_C \n[.cp]
31 .cp 0
32 .
33 .\" Define fallback for groff 1.23's MR macro if the system lacks it.
34 .nr do-fallback 0
35 .if !\n(.f .nr do-fallback 1 \" mandoc
36 .if \n(.g .if !d MR .nr do-fallback 1 \" older groff
37 .if !\n(.g .nr do-fallback 1 \" non-groff *roff
38 .if \n[do-fallback] \{\
39 . de MR
40 . ie \\n(.$=1 \
41 . I \%\\$1
42 . el \
43 . IR \%\\$1 (\\$2)\\$3
44 . .
45 .\}
46 .rr do-fallback
47 .
48 .
49 .\" ====================================================================
50 .SH Synopsis
51 .\" ====================================================================
52 .
53 .SY preconv
54 .RB [ \-dr ]
55 .RB [ \-D\~\c
56 .IR fallback-encoding ]
57 .RB [ \-e\~\c
58 .IR encoding ]
59 .RI [ file\~ .\|.\|.]
60 .YS
61 .
62 .
63 .P
64 .SY preconv
65 .B \-h
66 .YS
67 .
68 .SY preconv
69 .B \-\-help
70 .YS
71 .
72 .
73 .P
74 .SY preconv
75 .B \-v
76 .YS
77 .
78 .SY preconv
79 .B \%\-\-version
80 .YS
81 .
82 .
83 .\" ====================================================================
84 .SH Description
85 .\" ====================================================================
86 .
87 .I \%preconv
88 reads each
89 .IR file ,
90 converts its encoded characters to a form
91 .MR \%troff 1
92 can interpret,
93 and sends the result to the standard output stream.
94 .
95 Currently,
96 this means that code points in the range 0\[en]127
97 in
98 ISO 646:1991 IRV (US-ASCII),
99 ISO\~8859,
100 or Unicode
101 remain as-is and the remainder are converted to the
102 .I groff
103 special character form
104 .RB \[lq] \[rs][\c
105 .BI u XXXX ]\c
106 \[rq],
107 where
108 .I XXXX
109 is a hexadecimal number of four to six digits corresponding to a Unicode
110 code point.
111 .
112 By default,
113 .I \%preconv
114 also inserts a
115 .I roff
116 .B .lf
117 request at the beginning of each
118 .IR file ,
119 identifying it for the benefit of later processing
120 (including diagnostic messages);
121 the
122 .B \-r
123 option suppresses this behavior.
124 .
125 .
126 .PP
127 In typical usage scenarios,
128 .I \%preconv
129 need not be run directly;
130 instead it should be invoked with the
131 .B \-k
132 or
133 .B \-K
134 options of
135 .IR groff .
136 .
137 If no
138 .I file
139 operands are present,
140 or if
141 .I file
142 is
143 .RB \[lq] \- \[rq],
144 .I \%preconv
145 reads the standard input stream.
146 .
147 .
148 .PP
149 .I \%preconv
150 selects an input encoding with the following algorithm,
151 stopping at the first success.
152 .
153 .
154 .IP 1. 4n \" "1." + 2n
155 If the input encoding has been explicitly specified with option
156 .BR \-e ,
157 use it.
158 .
159 .
160 .IP 2.
161 If the input starts with a Unicode Byte Order Mark,
162 select UTF-8,
163 UTF-16,
164 or UTF-32 accordingly.
165 .
166 .
167 .IP 3.
168 If the input stream is seekable,
169 check the first two input lines for a GNU\~Emacs file-local variable
170 identifying the character encoding,
171 here referred to as the \[lq]coding tag\[rq].
172 .
173 If found,
174 use it.
175 .
176 .
177 .IP 4.
178 If the input stream is seekable,
179 and if the
180 .I uchardet
181 library is available on the system,
182 use it to try to infer the encoding of the file.
183 .
184 .
185 .IP 5.
186 If the
187 .B \-D
188 option specifies an encoding,
189 use it.
190 .
191 .
192 .IP 6.
193 Use the encoding specified by the current locale
194 .RI ( LC_CTYPE ),
195 unless the locale is
196 \[lq]C\[rq],
197 \[lq]POSIX\[rq],
198 or empty,
199 in which case assume ISO\~Latin-1
200 (8859-1).
201 .
202 .
203 .PP
204 The coding tag and
205 .I uchardet
206 methods in the above procedure rely upon a seekable input stream;
207 when
208 .I \%preconv
209 reads from a pipe,
210 the stream is not seekable,
211 and these detection methods are skipped.
212 .
213 If character encoding detection of your input is unreliable,
214 arrange for one of the other methods to succeed by using
215 .IR \%preconv 's
216 .B \-D
217 or
218 .B \-e
219 options,
220 or by configuring your locale appropriately.
221 .
222 .I groff
223 also supports a
224 .I \%GROFF_ENCODING
225 environment variable,
226 which can be overridden by its
227 .B \-K
228 option.
229 .
230 Valid values for
231 (or parameters to)
232 all of these are enumerated in the lists of recognized coding tags in
233 the next subsection,
234 and are further influenced by
235 .I iconv
236 library support.
237 .
238 .
239 .\" ====================================================================
240 .SS "Coding tags"
241 .\" ====================================================================
242 .
243 Text editors that support more than a single character encoding need
244 tags within the input files to mark the file's encoding.
245 .
246 While it is possible to guess the right input encoding with the help of
247 heuristics that produce good results for a preponderance of natural
248 language texts,
249 they are not absolutely reliable.
250 .
251 Heuristics can fail on inputs that are too short or don't represent a
252 natural language.
253 .
254 .
255 .PP
256 Consequently,
257 .I \%preconv
258 supports the coding tag convention used by GNU\~Emacs
259 (with some restrictions).
260 .
261 This notation appears in specially marked regions of an input file
262 designated for \[lq]file-local variables\[rq].
263 .
264 .
265 .br
266 .ne 4v
267 .P
268 .I \%preconv
269 interprets the following syntax if it occurs in a
270 .I roff
271 comment
272 in the first or second line of the input.
273 .
274 Both \[lq]\^\[rs]"\^\[rq] and \[lq]\^\[rs]#\^\[rq] comment forms are
275 recognized,
276 but the control
277 (or no-break control)
278 character must be the default and must begin the line.
279 .
280 Similarly,
281 the escape character must be the default.
282 .
283 .
284 .RS
285 .EX
286 .B \-*\- \c
287 .RB [.\|.\|. ; ]\~\c
288 .B coding: \c
289 .I encoding\c
290 .RB [ ;\~ .\|.\|.\&]\~\c
291 .B \-*\-
292 .EE
293 .RE
294 .
295 .
296 .PP
297 The only variable
298 .I \%preconv
299 interprets is \[lq]coding\[rq],
300 which can take the values listed below.
301 .
302 .
303 .PP
304 The following list comprises all MIME \[lq]charset\[rq] parameter values
305 recognized,
306 case-insensitively,
307 by
308 .IR \%preconv .
309 .
310 .RS
311 \%big5,
312 \%cp1047,
313 \%euc\-jp,
314 \%euc\-kr,
315 \%gb2312,
316 \%iso\-8859\-1,
317 \%iso\-8859\-2,
318 \%iso\-8859\-5,
319 \%iso\-8859\-7,
320 \%iso\-8859\-9,
321 \%iso\-8859\-13,
322 \%iso\-8859\-15,
323 \%koi8\-r,
324 \%us\-ascii,
325 \%utf\-8,
326 \%utf\-16,
327 \%utf\-16be,
328 \%utf\-16le
329 .RE
330 .
331 .
332 .PP
333 In addition,
334 the following list of other coding tags is recognized,
335 each of which is mapped to an appropriate value from the list above.
336 .
337 .RS
338 \%ascii,
339 \%chinese\-big5,
340 \%chinese\-euc,
341 \%chinese\-iso\-8bit,
342 \%cn\-big5,
343 \%cn\-gb,
344 \%cn\-gb\-2312,
345 \%cp878,
346 \%csascii,
347 \%csisolatin1,
348 \%cyrillic\-iso\-8bit,
349 \%cyrillic\-koi8,
350 \%euc\-china,
351 \%euc\-cn,
352 \%euc\-japan,
353 \%euc\-japan\-1990,
354 \%euc\-korea,
355 \%greek\-iso\-8bit,
356 \%iso\-10646/utf8,
357 \%iso\-10646/utf\-8,
358 \%iso\-latin\-1,
359 \%iso\-latin\-2,
360 \%iso\-latin\-5,
361 \%iso\-latin\-7,
362 \%iso\-latin\-9,
363 \%japanese\-euc,
364 \%japanese\-iso\-8bit,
365 \%jis8,
366 \%koi8,
367 \%korean\-euc,
368 \%korean\-iso\-8bit,
369 \%latin\-0,
370 \%latin1,
371 \%latin\-1,
372 \%latin\-2,
373 \%latin\-5,
374 \%latin\-7,
375 \%latin\-9,
376 \%mule\-utf\-8,
377 \%mule\-utf\-16,
378 \%mule\-utf\-16be,
379 \%mule\-utf\-16\-be,
380 \%mule\-utf\-16be\-with\-signature,
381 \%mule\-utf\-16le,
382 \%mule\-utf\-16\-le,
383 \%mule\-utf\-16le\-with\-signature,
384 \%utf8,
385 \%utf\-16\-be,
386 \%utf\-16\-be\-with\-signature,
387 \%utf\-16be\-with\-signature,
388 \%utf\-16\-le,
389 \%utf\-16\-le\-with\-signature,
390 \%utf\-16le\-with\-signature
391 .RE
392 .
393 .
394 .PP
395 Any
396 \[lq]\-dos\[rq],
397 \[lq]\-unix\[rq],
398 or
399 \[lq]\-mac\[rq]
400 suffix on a coding tag
401 (which indicates the end-of-line convention used in the file)
402 is ignored during comparison with the above tags.
403 .
404 .
405 .\" ====================================================================
406 .SS "\f[I]iconv\f[] support"
407 .\" ====================================================================
408 .
409 While
410 .I \%preconv
411 recognizes all of the coding tags listed above,
412 it is capable on its own of interpreting only two encodings:
413 ISO\~Latin-1 and and UTF-8.
414 .
415 ISO\~646:1991\~IRV (US-ASCII)
416 is a proper subset of both these encodings.
417 .
418 If
419 .MR iconv 3
420 support is configured at compile time and available at run time,
421 all others are passed to
422 .I iconv
423 library functions,
424 which may recognize many additional encoding strings.
425 .
426 The command
427 .RB \[lq] \%preconv\~\-v \[rq]
428 discloses whether
429 .I iconv
430 support is configured.
431 .
432 .
433 .PP
434 The use of
435 .I iconv
436 means that characters in the input that encode invalid code points for
437 that encoding may be dropped from the output stream or mapped to the
438 Unicode replacement character
439 (U+FFFD).
440 .
441 Compare the following examples using the input \[lq]caf\['e]\[rq]
442 (note the \[lq]e\[rq] with an acute accent),
443 which due to its short length challenges inference of the encoding used.
444 .
445 .RS
446 .EX
447 printf \[aq]caf\[rs]351\[rs]n\[aq] | LC_ALL=en_US.UTF\-8 preconv
448 printf \[aq]caf\[rs]351\[rs]n\[aq] | preconv \-e us\-ascii
449 printf \[aq]caf\[rs]351\[rs]n\[aq] | preconv \-e latin\-1
450 .EE
451 .RE
452 .
453 The fate of the accented \[lq]e\[rq] differs in each case.
454 .
455 In the first,
456 .I uchardet
457 fails to detect an encoding
458 (though the library on your system may behave differently)
459 and
460 .I \%preconv
461 falls back to the locale settings,
462 where octal 351 starts an incomplete UTF-8 sequence and results in the
463 Unicode replacement character.
464 .
465 In the second,
466 it is not a representable character in the declared input encoding of
467 US-ASCII and is discarded by
468 .IR iconv .
469 .
470 In the last,
471 it is correctly detected and mapped.
472 .
473 .
474 .\" ====================================================================
475 .SS Limitations
476 .\" ====================================================================
477 .
478 .I \%preconv
479 cannot perform any transformation on input that it cannot see.
480 .
481 Examples include files that are interpolated by preprocessors that run
482 subsequently,
483 including
484 .MR \%soelim 1 ;
485 files included by
486 .I \%troff
487 itself through
488 .RB \[lq] so \[rq]
489 and similar requests;
490 and string definitions passed to
491 .I \%troff
492 through its
493 .B \-d
494 command-line option.
495 .
496 .
497 .P
498 .I \%preconv
499 assumes that its input uses the default escape character,
500 a backslash
501 .BR \[rs] ,
502 and writes special character escape sequences accordingly.
503 .
504 .
505 .\" ====================================================================
506 .SH Options
507 .\" ====================================================================
508 .
509 .B \-h
510 and
511 .B \-\-help
512 display a usage message,
513 while
514 .B \-v
515 and
516 .B \%\-\-version
517 show version and configuration information;
518 all exit afterward.
519 .
520 .
521 .TP 10n \" "-D fbenc" + 2n
522 .B \-d
523 Emit debugging messages to the standard error stream.
524 .
525 .
526 .TP
527 .BI \-D\~ fbenc
528 Select the fallback encoding
529 .I fbenc
530 if all detection methods fail.
531 .
532 .
533 .TP
534 .BI \-e\~ enc
535 Skip detection and select encoding
536 .IR enc ;
537 see
538 .IR groff 's
539 .B \-K
540 option.
541 .
542 .
543 .TP
544 .B \-r
545 Write files \[lq]raw\[rq];
546 do not add
547 .B .lf
548 requests.
549 .
550 .
551 .\" ====================================================================
552 .SH "Exit status"
553 .\" ====================================================================
554 .
555 .I \%preconv
556 exits with
557 .RB status\~ 0
558 on successful operation,
559 .RB status\~ 2
560 if the program cannot interpret its command-line arguments,
561 and
562 .RB status\~ 1
563 if it encounters an error during operation.
564 .
565 .
566 .\" ====================================================================
567 .SH "See also"
568 .\" ====================================================================
569 .
570 .MR groff 1 ,
571 .MR iconv 3 ,
572 .MR locale 7
573 .
574 .
575 .\" Restore compatibility mode (for, e.g., Solaris 10/11).
576 .cp \n[*groff_preconv_1_man_C]
577 .do rr *groff_preconv_1_man_C
578 .
579 .
580 .\" Local Variables:
581 .\" fill-column: 72
582 .\" mode: nroff
583 .\" End:
584 .\" vim: set filetype=groff textwidth=72: