| 1 |
NAME |
| 2 |
|
| 3 |
utf8::all - turn on Unicode - all of it |
| 4 |
|
| 5 |
VERSION |
| 6 |
|
| 7 |
version 0.026 |
| 8 |
|
| 9 |
SYNOPSIS |
| 10 |
|
| 11 |
use utf8::all; # Turn on UTF-8, all of it. |
| 12 |
|
| 13 |
open my $in, '<', 'contains-utf8'; # UTF-8 already turned on here |
| 14 |
print length 'føø bār'; # 7 UTF-8 characters |
| 15 |
my $utf8_arg = shift @ARGV; # @ARGV is UTF-8 too (only for main) |
| 16 |
|
| 17 |
DESCRIPTION |
| 18 |
|
| 19 |
The use utf8 pragma tells the Perl parser to allow UTF-8 in the program |
| 20 |
text in the current lexical scope. This also means that you can now use |
| 21 |
literal Unicode characters as part of strings, variable names, and |
| 22 |
regular expressions. |
| 23 |
|
| 24 |
utf8::all goes further: |
| 25 |
|
| 26 |
* charnames are imported so \N{...} sequences can be used to compile |
| 27 |
Unicode characters based on names. |
| 28 |
|
| 29 |
* On Perl v5.11.0 or higher, the use feature 'unicode_strings' is |
| 30 |
enabled. |
| 31 |
|
| 32 |
* use feature fc and use feature unicode_eval are enabled on Perl |
| 33 |
5.16.0 and higher. |
| 34 |
|
| 35 |
* Filehandles are opened with UTF-8 encoding turned on by default |
| 36 |
(including STDIN, STDOUT, and STDERR when utf8::all is used from the |
| 37 |
main package). Meaning that they automatically convert UTF-8 octets |
| 38 |
to characters and vice versa. If you don't want UTF-8 for a |
| 39 |
particular filehandle, you'll have to set binmode $filehandle. |
| 40 |
|
| 41 |
* @ARGV gets converted from UTF-8 octets to Unicode characters (when |
| 42 |
utf8::all is used from the main package). This is similar to the |
| 43 |
behaviour of the -CA perl command-line switch (see perlrun). |
| 44 |
|
| 45 |
* readdir, readlink, readpipe (including the qx// and backtick |
| 46 |
operators), and glob (including the <> operator) now all work with |
| 47 |
and return Unicode characters instead of (UTF-8) octets (again only |
| 48 |
when utf8::all is used from the main package). |
| 49 |
|
| 50 |
Lexical Scope |
| 51 |
|
| 52 |
The pragma is lexically-scoped, so you can do the following if you had |
| 53 |
some reason to: |
| 54 |
|
| 55 |
{ |
| 56 |
use utf8::all; |
| 57 |
open my $out, '>', 'outfile'; |
| 58 |
my $utf8_str = 'føø bār'; |
| 59 |
print length $utf8_str, "\n"; # 7 |
| 60 |
print $out $utf8_str; # out as utf8 |
| 61 |
} |
| 62 |
open my $in, '<', 'outfile'; # in as raw |
| 63 |
my $text = do { local $/; <$in>}; |
| 64 |
print length $text, "\n"; # 10, not 7! |
| 65 |
|
| 66 |
Instead of lexical scoping, you can also use no utf8::all to turn off |
| 67 |
the effects. |
| 68 |
|
| 69 |
Note that the effect on @ARGV and the STDIN, STDOUT, and STDERR file |
| 70 |
handles is always global and can not be undone! |
| 71 |
|
| 72 |
Enabling/Disabling Global Features |
| 73 |
|
| 74 |
As described above, the default behaviour of utf8::all is to convert |
| 75 |
@ARGV and to open the STDIN, STDOUT, and STDERR file handles with UTF-8 |
| 76 |
encoding, and override the readlink and readdir functions and glob |
| 77 |
operators when utf8::all is used from the main package. |
| 78 |
|
| 79 |
If you want to disable these features even when utf8::all is used from |
| 80 |
the main package, add the option NO-GLOBAL (or LEXICAL-ONLY) to the use |
| 81 |
line. E.g.: |
| 82 |
|
| 83 |
use utf8::all 'NO-GLOBAL'; |
| 84 |
|
| 85 |
If on the other hand you want to enable these global effects even when |
| 86 |
utf8::all was used from another package than main, use the option |
| 87 |
GLOBAL on the use line: |
| 88 |
|
| 89 |
use utf8::all 'GLOBAL'; |
| 90 |
|
| 91 |
UTF-8 Errors |
| 92 |
|
| 93 |
utf8::all will handle invalid code points (i.e., utf-8 that does not |
| 94 |
map to a valid unicode "character"), as a fatal error. |
| 95 |
|
| 96 |
For glob, readdir, and readlink, one can change this behaviour by |
| 97 |
setting the attribute "$utf8::all::UTF8_CHECK". |
| 98 |
|
| 99 |
ATTRIBUTES |
| 100 |
|
| 101 |
$utf8::all::UTF8_CHECK |
| 102 |
|
| 103 |
By default utf8::all marks decoding errors as fatal (default value for |
| 104 |
this setting is Encode::FB_CROAK). If you want, you can change this by |
| 105 |
setting $utf8::all::UTF8_CHECK. The value Encode::FB_WARN reports the |
| 106 |
encoding errors as warnings, and Encode::FB_DEFAULT will completely |
| 107 |
ignore them. Please see Encode for details. Note: Encode::LEAVE_SRC is |
| 108 |
always enforced. |
| 109 |
|
| 110 |
Important: Only controls the handling of decoding errors in glob, |
| 111 |
readdir, and readlink. |
| 112 |
|
| 113 |
INTERACTION WITH AUTODIE |
| 114 |
|
| 115 |
If you use autodie, which is a great idea, you need to use at least |
| 116 |
version 2.12, released on June 26, 2012 |
| 117 |
<https://metacpan.org/source/PJF/autodie-2.12/Changes#L3>. Otherwise, |
| 118 |
autodie obliterates the IO layers set by the open pragma. See RT #54777 |
| 119 |
<https://rt.cpan.org/Ticket/Display.html?id=54777> and GH #7 |
| 120 |
<https://github.com/doherty/utf8-all/issues/7>. |
| 121 |
|
| 122 |
BUGS |
| 123 |
|
| 124 |
Please report any bugs or feature requests on the bugtracker website |
| 125 |
<https://github.com/doherty/utf8-all/issues>. |
| 126 |
|
| 127 |
When submitting a bug or request, please include a test-file or a patch |
| 128 |
to an existing test-file that illustrates the bug or desired feature. |
| 129 |
|
| 130 |
COMPATIBILITY |
| 131 |
|
| 132 |
The filesystems of Dos, Windows, and OS/2 do not (fully) support UTF-8. |
| 133 |
The readlink and readdir functions and glob operators will therefore |
| 134 |
not be replaced on these systems. |
| 135 |
|
| 136 |
SEE ALSO |
| 137 |
|
| 138 |
* File::Find::utf8 for fully utf-8 aware File::Find functions. |
| 139 |
|
| 140 |
* Cwd::utf8 for fully utf-8 aware Cwd functions. |
| 141 |
|
| 142 |
AUTHORS |
| 143 |
|
| 144 |
* Michael Schwern <[email protected]> |
| 145 |
|
| 146 |
* Mike Doherty <[email protected]> |
| 147 |
|
| 148 |
* Hayo Baan <[email protected]> |
| 149 |
|
| 150 |
COPYRIGHT AND LICENSE |
| 151 |
|
| 152 |
This software is copyright (c) 2009 by Michael Schwern |
| 153 |
<[email protected]>; he originated it. |
| 154 |
|
| 155 |
This is free software; you can redistribute it and/or modify it under |
| 156 |
the same terms as the Perl 5 programming language system itself. |
| 157 |
|