| 1 |
|
| 2 |
# Time-stamp: "2004-01-11 18:35:34 AST" |
| 3 |
|
| 4 |
=head1 NAME |
| 5 |
|
| 6 |
Locale::Maketext - framework for localization |
| 7 |
|
| 8 |
=head1 SYNOPSIS |
| 9 |
|
| 10 |
package MyProgram; |
| 11 |
use strict; |
| 12 |
use MyProgram::L10N; |
| 13 |
# ...which inherits from Locale::Maketext |
| 14 |
my $lh = MyProgram::L10N->get_handle() || die "What language?"; |
| 15 |
... |
| 16 |
# And then any messages your program emits, like: |
| 17 |
warn $lh->maketext( "Can't open file [_1]: [_2]\n", $f, $! ); |
| 18 |
... |
| 19 |
|
| 20 |
=head1 DESCRIPTION |
| 21 |
|
| 22 |
It is a common feature of applications (whether run directly, |
| 23 |
or via the Web) for them to be "localized" -- i.e., for them |
| 24 |
to a present an English interface to an English-speaker, a German |
| 25 |
interface to a German-speaker, and so on for all languages it's |
| 26 |
programmed with. Locale::Maketext |
| 27 |
is a framework for software localization; it provides you with the |
| 28 |
tools for organizing and accessing the bits of text and text-processing |
| 29 |
code that you need for producing localized applications. |
| 30 |
|
| 31 |
In order to make sense of Maketext and how all its |
| 32 |
components fit together, you should probably |
| 33 |
go read L<Locale::Maketext::TPJ13|Locale::Maketext::TPJ13>, and |
| 34 |
I<then> read the following documentation. |
| 35 |
|
| 36 |
You may also want to read over the source for C<File::Findgrep> |
| 37 |
and its constituent modules -- they are a complete (if small) |
| 38 |
example application that uses Maketext. |
| 39 |
|
| 40 |
=head1 QUICK OVERVIEW |
| 41 |
|
| 42 |
The basic design of Locale::Maketext is object-oriented, and |
| 43 |
Locale::Maketext is an abstract base class, from which you |
| 44 |
derive a "project class". |
| 45 |
The project class (with a name like "TkBocciBall::Localize", |
| 46 |
which you then use in your module) is in turn the base class |
| 47 |
for all the "language classes" for your project |
| 48 |
(with names "TkBocciBall::Localize::it", |
| 49 |
"TkBocciBall::Localize::en", |
| 50 |
"TkBocciBall::Localize::fr", etc.). |
| 51 |
|
| 52 |
A language class is |
| 53 |
a class containing a lexicon of phrases as class data, |
| 54 |
and possibly also some methods that are of use in interpreting |
| 55 |
phrases in the lexicon, or otherwise dealing with text in that |
| 56 |
language. |
| 57 |
|
| 58 |
An object belonging to a language class is called a "language |
| 59 |
handle"; it's typically a flyweight object. |
| 60 |
|
| 61 |
The normal course of action is to call: |
| 62 |
|
| 63 |
use TkBocciBall::Localize; # the localization project class |
| 64 |
$lh = TkBocciBall::Localize->get_handle(); |
| 65 |
# Depending on the user's locale, etc., this will |
| 66 |
# make a language handle from among the classes available, |
| 67 |
# and any defaults that you declare. |
| 68 |
die "Couldn't make a language handle??" unless $lh; |
| 69 |
|
| 70 |
From then on, you use the C<maketext> function to access |
| 71 |
entries in whatever lexicon(s) belong to the language handle |
| 72 |
you got. So, this: |
| 73 |
|
| 74 |
print $lh->maketext("You won!"), "\n"; |
| 75 |
|
| 76 |
...emits the right text for this language. If the object |
| 77 |
in C<$lh> belongs to class "TkBocciBall::Localize::fr" and |
| 78 |
%TkBocciBall::Localize::fr::Lexicon contains C<("You won!" |
| 79 |
=E<gt> "Tu as gagnE<eacute>!")>, then the above |
| 80 |
code happily tells the user "Tu as gagnE<eacute>!". |
| 81 |
|
| 82 |
=head1 METHODS |
| 83 |
|
| 84 |
Locale::Maketext offers a variety of methods, which fall |
| 85 |
into three categories: |
| 86 |
|
| 87 |
=over |
| 88 |
|
| 89 |
=item * |
| 90 |
|
| 91 |
Methods to do with constructing language handles. |
| 92 |
|
| 93 |
=item * |
| 94 |
|
| 95 |
C<maketext> and other methods to do with accessing %Lexicon data |
| 96 |
for a given language handle. |
| 97 |
|
| 98 |
=item * |
| 99 |
|
| 100 |
Methods that you may find it handy to use, from routines of |
| 101 |
yours that you put in %Lexicon entries. |
| 102 |
|
| 103 |
=back |
| 104 |
|
| 105 |
These are covered in the following section. |
| 106 |
|
| 107 |
=head2 Construction Methods |
| 108 |
|
| 109 |
These are to do with constructing a language handle: |
| 110 |
|
| 111 |
=over |
| 112 |
|
| 113 |
=item * |
| 114 |
|
| 115 |
$lh = YourProjClass->get_handle( ...langtags... ) || die "lg-handle?"; |
| 116 |
|
| 117 |
This tries loading classes based on the language-tags you give (like |
| 118 |
C<("en-US", "sk", "kon", "es-MX", "ja", "i-klingon")>, and for the first class |
| 119 |
that succeeds, returns YourProjClass::I<language>->new(). |
| 120 |
|
| 121 |
If it runs thru the entire given list of language-tags, and finds no classes |
| 122 |
for those exact terms, it then tries "superordinate" language classes. |
| 123 |
So if no "en-US" class (i.e., YourProjClass::en_us) |
| 124 |
was found, nor classes for anything else in that list, we then try |
| 125 |
its superordinate, "en" (i.e., YourProjClass::en), and so on thru |
| 126 |
the other language-tags in the given list: "es". |
| 127 |
(The other language-tags in our example list: |
| 128 |
happen to have no superordinates.) |
| 129 |
|
| 130 |
If none of those language-tags leads to loadable classes, we then |
| 131 |
try classes derived from YourProjClass->fallback_languages() and |
| 132 |
then if nothing comes of that, we use classes named by |
| 133 |
YourProjClass->fallback_language_classes(). Then in the (probably |
| 134 |
quite unlikely) event that that fails, we just return undef. |
| 135 |
|
| 136 |
=item * |
| 137 |
|
| 138 |
$lh = YourProjClass->get_handleB<()> || die "lg-handle?"; |
| 139 |
|
| 140 |
When C<get_handle> is called with an empty parameter list, magic happens: |
| 141 |
|
| 142 |
If C<get_handle> senses that it's running in program that was |
| 143 |
invoked as a CGI, then it tries to get language-tags out of the |
| 144 |
environment variable "HTTP_ACCEPT_LANGUAGE", and it pretends that |
| 145 |
those were the languages passed as parameters to C<get_handle>. |
| 146 |
|
| 147 |
Otherwise (i.e., if not a CGI), this tries various OS-specific ways |
| 148 |
to get the language-tags for the current locale/language, and then |
| 149 |
pretends that those were the value(s) passed to C<get_handle>. |
| 150 |
|
| 151 |
Currently this OS-specific stuff consists of looking in the environment |
| 152 |
variables "LANG" and "LANGUAGE"; and on MSWin machines (where those |
| 153 |
variables are typically unused), this also tries using |
| 154 |
the module Win32::Locale to get a language-tag for whatever language/locale |
| 155 |
is currently selected in the "Regional Settings" (or "International"?) |
| 156 |
Control Panel. I welcome further |
| 157 |
suggestions for making this do the Right Thing under other operating |
| 158 |
systems that support localization. |
| 159 |
|
| 160 |
If you're using localization in an application that keeps a configuration |
| 161 |
file, you might consider something like this in your project class: |
| 162 |
|
| 163 |
sub get_handle_via_config { |
| 164 |
my $class = $_[0]; |
| 165 |
my $chosen_language = $Config_settings{'language'}; |
| 166 |
my $lh; |
| 167 |
if($chosen_language) { |
| 168 |
$lh = $class->get_handle($chosen_language) |
| 169 |
|| die "No language handle for \"$chosen_language\"" |
| 170 |
. " or the like"; |
| 171 |
} else { |
| 172 |
# Config file missing, maybe? |
| 173 |
$lh = $class->get_handle() |
| 174 |
|| die "Can't get a language handle"; |
| 175 |
} |
| 176 |
return $lh; |
| 177 |
} |
| 178 |
|
| 179 |
=item * |
| 180 |
|
| 181 |
$lh = YourProjClass::langname->new(); |
| 182 |
|
| 183 |
This constructs a language handle. You usually B<don't> call this |
| 184 |
directly, but instead let C<get_handle> find a language class to C<use> |
| 185 |
and to then call ->new on. |
| 186 |
|
| 187 |
=item * |
| 188 |
|
| 189 |
$lh->init(); |
| 190 |
|
| 191 |
This is called by ->new to initialize newly-constructed language handles. |
| 192 |
If you define an init method in your class, remember that it's usually |
| 193 |
considered a good idea to call $lh->SUPER::init in it (presumably at the |
| 194 |
beginning), so that all classes get a chance to initialize a new object |
| 195 |
however they see fit. |
| 196 |
|
| 197 |
=item * |
| 198 |
|
| 199 |
YourProjClass->fallback_languages() |
| 200 |
|
| 201 |
C<get_handle> appends the return value of this to the end of |
| 202 |
whatever list of languages you pass C<get_handle>. Unless |
| 203 |
you override this method, your project class |
| 204 |
will inherit Locale::Maketext's C<fallback_languages>, which |
| 205 |
currently returns C<('i-default', 'en', 'en-US')>. |
| 206 |
("i-default" is defined in RFC 2277). |
| 207 |
|
| 208 |
This method (by having it return the name |
| 209 |
of a language-tag that has an existing language class) |
| 210 |
can be used for making sure that |
| 211 |
C<get_handle> will always manage to construct a language |
| 212 |
handle (assuming your language classes are in an appropriate |
| 213 |
@INC directory). Or you can use the next method: |
| 214 |
|
| 215 |
=item * |
| 216 |
|
| 217 |
YourProjClass->fallback_language_classes() |
| 218 |
|
| 219 |
C<get_handle> appends the return value of this to the end |
| 220 |
of the list of classes it will try using. Unless |
| 221 |
you override this method, your project class |
| 222 |
will inherit Locale::Maketext's C<fallback_language_classes>, |
| 223 |
which currently returns an empty list, C<()>. |
| 224 |
By setting this to some value (namely, the name of a loadable |
| 225 |
language class), you can be sure that |
| 226 |
C<get_handle> will always manage to construct a language |
| 227 |
handle. |
| 228 |
|
| 229 |
=back |
| 230 |
|
| 231 |
=head2 The "maketext" Method |
| 232 |
|
| 233 |
This is the most important method in Locale::Maketext: |
| 234 |
|
| 235 |
$text = $lh->maketext(I<key>, ...parameters for this phrase...); |
| 236 |
|
| 237 |
This looks in the %Lexicon of the language handle |
| 238 |
$lh and all its superclasses, looking |
| 239 |
for an entry whose key is the string I<key>. Assuming such |
| 240 |
an entry is found, various things then happen, depending on the |
| 241 |
value found: |
| 242 |
|
| 243 |
If the value is a scalarref, the scalar is dereferenced and returned |
| 244 |
(and any parameters are ignored). |
| 245 |
|
| 246 |
If the value is a coderef, we return &$value($lh, ...parameters...). |
| 247 |
|
| 248 |
If the value is a string that I<doesn't> look like it's in Bracket Notation, |
| 249 |
we return it (after replacing it with a scalarref, in its %Lexicon). |
| 250 |
|
| 251 |
If the value I<does> look like it's in Bracket Notation, then we compile |
| 252 |
it into a sub, replace the string in the %Lexicon with the new coderef, |
| 253 |
and then we return &$new_sub($lh, ...parameters...). |
| 254 |
|
| 255 |
Bracket Notation is discussed in a later section. Note |
| 256 |
that trying to compile a string into Bracket Notation can throw |
| 257 |
an exception if the string is not syntactically valid (say, by not |
| 258 |
balancing brackets right.) |
| 259 |
|
| 260 |
Also, calling &$coderef($lh, ...parameters...) can throw any sort of |
| 261 |
exception (if, say, code in that sub tries to divide by zero). But |
| 262 |
a very common exception occurs when you have Bracket |
| 263 |
Notation text that says to call a method "foo", but there is no such |
| 264 |
method. (E.g., "You have [quaB<tn>,_1,ball]." will throw an exception |
| 265 |
on trying to call $lh->quaB<tn>($_[1],'ball') -- you presumably meant |
| 266 |
"quant".) C<maketext> catches these exceptions, but only to make the |
| 267 |
error message more readable, at which point it rethrows the exception. |
| 268 |
|
| 269 |
An exception I<may> be thrown if I<key> is not found in any |
| 270 |
of $lh's %Lexicon hashes. What happens if a key is not found, |
| 271 |
is discussed in a later section, "Controlling Lookup Failure". |
| 272 |
|
| 273 |
Note that you might find it useful in some cases to override |
| 274 |
the C<maketext> method with an "after method", if you want to |
| 275 |
translate encodings, or even scripts: |
| 276 |
|
| 277 |
package YrProj::zh_cn; # Chinese with PRC-style glyphs |
| 278 |
use base ('YrProj::zh_tw'); # Taiwan-style |
| 279 |
sub maketext { |
| 280 |
my $self = shift(@_); |
| 281 |
my $value = $self->maketext(@_); |
| 282 |
return Chineeze::taiwan2mainland($value); |
| 283 |
} |
| 284 |
|
| 285 |
Or you may want to override it with something that traps |
| 286 |
any exceptions, if that's critical to your program: |
| 287 |
|
| 288 |
sub maketext { |
| 289 |
my($lh, @stuff) = @_; |
| 290 |
my $out; |
| 291 |
eval { $out = $lh->SUPER::maketext(@stuff) }; |
| 292 |
return $out unless $@; |
| 293 |
...otherwise deal with the exception... |
| 294 |
} |
| 295 |
|
| 296 |
Other than those two situations, I don't imagine that |
| 297 |
it's useful to override the C<maketext> method. (If |
| 298 |
you run into a situation where it is useful, I'd be |
| 299 |
interested in hearing about it.) |
| 300 |
|
| 301 |
=over |
| 302 |
|
| 303 |
=item $lh->fail_with I<or> $lh->fail_with(I<PARAM>) |
| 304 |
|
| 305 |
=item $lh->failure_handler_auto |
| 306 |
|
| 307 |
These two methods are discussed in the section "Controlling |
| 308 |
Lookup Failure". |
| 309 |
|
| 310 |
=item $lh->denylist(@list) <or> $lh->blacklist(@list) |
| 311 |
|
| 312 |
=item $lh->allowlist(@list) <or> $lh->whitelist(@list) |
| 313 |
|
| 314 |
These methods are discussed in the section "Bracket Notation |
| 315 |
Security". |
| 316 |
|
| 317 |
=back |
| 318 |
|
| 319 |
=head2 Utility Methods |
| 320 |
|
| 321 |
These are methods that you may find it handy to use, generally |
| 322 |
from %Lexicon routines of yours (whether expressed as |
| 323 |
Bracket Notation or not). |
| 324 |
|
| 325 |
=over |
| 326 |
|
| 327 |
=item $language->quant($number, $singular) |
| 328 |
|
| 329 |
=item $language->quant($number, $singular, $plural) |
| 330 |
|
| 331 |
=item $language->quant($number, $singular, $plural, $negative) |
| 332 |
|
| 333 |
This is generally meant to be called from inside Bracket Notation |
| 334 |
(which is discussed later), as in |
| 335 |
|
| 336 |
"Your search matched [quant,_1,document]!" |
| 337 |
|
| 338 |
It's for I<quantifying> a noun (i.e., saying how much of it there is, |
| 339 |
while giving the correct form of it). The behavior of this method is |
| 340 |
handy for English and a few other Western European languages, and you |
| 341 |
should override it for languages where it's not suitable. You can feel |
| 342 |
free to read the source, but the current implementation is basically |
| 343 |
as this pseudocode describes: |
| 344 |
|
| 345 |
if $number is 0 and there's a $negative, |
| 346 |
return $negative; |
| 347 |
elsif $number is 1, |
| 348 |
return "1 $singular"; |
| 349 |
elsif there's a $plural, |
| 350 |
return "$number $plural"; |
| 351 |
else |
| 352 |
return "$number " . $singular . "s"; |
| 353 |
# |
| 354 |
# ...except that we actually call numf to |
| 355 |
# stringify $number before returning it. |
| 356 |
|
| 357 |
So for English (with Bracket Notation) |
| 358 |
C<"...[quant,_1,file]..."> is fine (for 0 it returns "0 files", |
| 359 |
for 1 it returns "1 file", and for more it returns "2 files", etc.) |
| 360 |
|
| 361 |
But for "directory", you'd want C<"[quant,_1,directory,directories]"> |
| 362 |
so that our elementary C<quant> method doesn't think that the |
| 363 |
plural of "directory" is "directorys". And you might find that the |
| 364 |
output may sound better if you specify a negative form, as in: |
| 365 |
|
| 366 |
"[quant,_1,file,files,No files] matched your query.\n" |
| 367 |
|
| 368 |
Remember to keep in mind verb agreement (or adjectives too, in |
| 369 |
other languages), as in: |
| 370 |
|
| 371 |
"[quant,_1,document] were matched.\n" |
| 372 |
|
| 373 |
Because if _1 is one, you get "1 document B<were> matched". |
| 374 |
An acceptable hack here is to do something like this: |
| 375 |
|
| 376 |
"[quant,_1,document was, documents were] matched.\n" |
| 377 |
|
| 378 |
=item $language->numf($number) |
| 379 |
|
| 380 |
This returns the given number formatted nicely according to |
| 381 |
this language's conventions. Maketext's default method is |
| 382 |
mostly to just take the normal string form of the number |
| 383 |
(applying sprintf "%G" for only very large numbers), and then |
| 384 |
to add commas as necessary. (Except that |
| 385 |
we apply C<tr/,./.,/> if $language->{'numf_comma'} is true; |
| 386 |
that's a bit of a hack that's useful for languages that express |
| 387 |
two million as "2.000.000" and not as "2,000,000"). |
| 388 |
|
| 389 |
If you want anything fancier, consider overriding this with something |
| 390 |
that uses L<Number::Format|Number::Format>, or does something else |
| 391 |
entirely. |
| 392 |
|
| 393 |
Note that numf is called by quant for stringifying all quantifying |
| 394 |
numbers. |
| 395 |
|
| 396 |
=item $language->numerate($number, $singular, $plural, $negative) |
| 397 |
|
| 398 |
This returns the given noun form which is appropriate for the quantity |
| 399 |
C<$number> according to this language's conventions. C<numerate> is |
| 400 |
used internally by C<quant> to quantify nouns. Use it directly -- |
| 401 |
usually from bracket notation -- to avoid C<quant>'s implicit call to |
| 402 |
C<numf> and output of a numeric quantity. |
| 403 |
|
| 404 |
=item $language->sprintf($format, @items) |
| 405 |
|
| 406 |
This is just a wrapper around Perl's normal C<sprintf> function. |
| 407 |
It's provided so that you can use "sprintf" in Bracket Notation: |
| 408 |
|
| 409 |
"Couldn't access datanode [sprintf,%10x=~[%s~],_1,_2]!\n" |
| 410 |
|
| 411 |
returning... |
| 412 |
|
| 413 |
Couldn't access datanode Stuff=[thangamabob]! |
| 414 |
|
| 415 |
=item $language->language_tag() |
| 416 |
|
| 417 |
Currently this just takes the last bit of C<ref($language)>, turns |
| 418 |
underscores to dashes, and returns it. So if $language is |
| 419 |
an object of class Hee::HOO::Haw::en_us, $language->language_tag() |
| 420 |
returns "en-us". (Yes, the usual representation for that language |
| 421 |
tag is "en-US", but case is I<never> considered meaningful in |
| 422 |
language-tag comparison.) |
| 423 |
|
| 424 |
You may override this as you like; Maketext doesn't use it for |
| 425 |
anything. |
| 426 |
|
| 427 |
=item $language->encoding() |
| 428 |
|
| 429 |
Currently this isn't used for anything, but it's provided |
| 430 |
(with default value of |
| 431 |
C<(ref($language) && $language-E<gt>{'encoding'})) or "iso-8859-1"> |
| 432 |
) as a sort of suggestion that it may be useful/necessary to |
| 433 |
associate encodings with your language handles (whether on a |
| 434 |
per-class or even per-handle basis.) |
| 435 |
|
| 436 |
=back |
| 437 |
|
| 438 |
=head2 Language Handle Attributes and Internals |
| 439 |
|
| 440 |
A language handle is a flyweight object -- i.e., it doesn't (necessarily) |
| 441 |
carry any data of interest, other than just being a member of |
| 442 |
whatever class it belongs to. |
| 443 |
|
| 444 |
A language handle is implemented as a blessed hash. Subclasses of yours |
| 445 |
can store whatever data you want in the hash. Currently the only hash |
| 446 |
entry used by any crucial Maketext method is "fail", so feel free to |
| 447 |
use anything else as you like. |
| 448 |
|
| 449 |
B<Remember: Don't be afraid to read the Maketext source if there's |
| 450 |
any point on which this documentation is unclear.> This documentation |
| 451 |
is vastly longer than the module source itself. |
| 452 |
|
| 453 |
=head1 LANGUAGE CLASS HIERARCHIES |
| 454 |
|
| 455 |
These are Locale::Maketext's assumptions about the class |
| 456 |
hierarchy formed by all your language classes: |
| 457 |
|
| 458 |
=over |
| 459 |
|
| 460 |
=item * |
| 461 |
|
| 462 |
You must have a project base class, which you load, and |
| 463 |
which you then use as the first argument in |
| 464 |
the call to YourProjClass->get_handle(...). It should derive |
| 465 |
(whether directly or indirectly) from Locale::Maketext. |
| 466 |
It B<doesn't matter> how you name this class, although assuming this |
| 467 |
is the localization component of your Super Mega Program, |
| 468 |
good names for your project class might be |
| 469 |
SuperMegaProgram::Localization, SuperMegaProgram::L10N, |
| 470 |
SuperMegaProgram::I18N, SuperMegaProgram::International, |
| 471 |
or even SuperMegaProgram::Languages or SuperMegaProgram::Messages. |
| 472 |
|
| 473 |
=item * |
| 474 |
|
| 475 |
Language classes are what YourProjClass->get_handle will try to load. |
| 476 |
It will look for them by taking each language-tag (B<skipping> it |
| 477 |
if it doesn't look like a language-tag or locale-tag!), turning it to |
| 478 |
all lowercase, turning dashes to underscores, and appending it |
| 479 |
to YourProjClass . "::". So this: |
| 480 |
|
| 481 |
$lh = YourProjClass->get_handle( |
| 482 |
'en-US', 'fr', 'kon', 'i-klingon', 'i-klingon-romanized' |
| 483 |
); |
| 484 |
|
| 485 |
will try loading the classes |
| 486 |
YourProjClass::en_us (note lowercase!), YourProjClass::fr, |
| 487 |
YourProjClass::kon, |
| 488 |
YourProjClass::i_klingon |
| 489 |
and YourProjClass::i_klingon_romanized. (And it'll stop at the |
| 490 |
first one that actually loads.) |
| 491 |
|
| 492 |
=item * |
| 493 |
|
| 494 |
I assume that each language class derives (directly or indirectly) |
| 495 |
from your project class, and also defines its @ISA, its %Lexicon, |
| 496 |
or both. But I anticipate no dire consequences if these assumptions |
| 497 |
do not hold. |
| 498 |
|
| 499 |
=item * |
| 500 |
|
| 501 |
Language classes may derive from other language classes (although they |
| 502 |
should have "use I<Thatclassname>" or "use base qw(I<...classes...>)"). |
| 503 |
They may derive from the project |
| 504 |
class. They may derive from some other class altogether. Or via |
| 505 |
multiple inheritance, it may derive from any mixture of these. |
| 506 |
|
| 507 |
=item * |
| 508 |
|
| 509 |
I foresee no problems with having multiple inheritance in |
| 510 |
your hierarchy of language classes. (As usual, however, Perl will |
| 511 |
complain bitterly if you have a cycle in the hierarchy: i.e., if |
| 512 |
any class is its own ancestor.) |
| 513 |
|
| 514 |
=back |
| 515 |
|
| 516 |
=head1 ENTRIES IN EACH LEXICON |
| 517 |
|
| 518 |
A typical %Lexicon entry is meant to signify a phrase, |
| 519 |
taking some number (0 or more) of parameters. An entry |
| 520 |
is meant to be accessed by via |
| 521 |
a string I<key> in $lh->maketext(I<key>, ...parameters...), |
| 522 |
which should return a string that is generally meant for |
| 523 |
be used for "output" to the user -- regardless of whether |
| 524 |
this actually means printing to STDOUT, writing to a file, |
| 525 |
or putting into a GUI widget. |
| 526 |
|
| 527 |
While the key must be a string value (since that's a basic |
| 528 |
restriction that Perl places on hash keys), the value in |
| 529 |
the lexicon can currently be of several types: |
| 530 |
a defined scalar, scalarref, or coderef. The use of these is |
| 531 |
explained above, in the section 'The "maketext" Method', and |
| 532 |
Bracket Notation for strings is discussed in the next section. |
| 533 |
|
| 534 |
While you can use arbitrary unique IDs for lexicon keys |
| 535 |
(like "_min_larger_max_error"), it is often |
| 536 |
useful for if an entry's key is itself a valid value, like |
| 537 |
this example error message: |
| 538 |
|
| 539 |
"Minimum ([_1]) is larger than maximum ([_2])!\n", |
| 540 |
|
| 541 |
Compare this code that uses an arbitrary ID... |
| 542 |
|
| 543 |
die $lh->maketext( "_min_larger_max_error", $min, $max ) |
| 544 |
if $min > $max; |
| 545 |
|
| 546 |
...to this code that uses a key-as-value: |
| 547 |
|
| 548 |
die $lh->maketext( |
| 549 |
"Minimum ([_1]) is larger than maximum ([_2])!\n", |
| 550 |
$min, $max |
| 551 |
) if $min > $max; |
| 552 |
|
| 553 |
The second is, in short, more readable. In particular, it's obvious |
| 554 |
that the number of parameters you're feeding to that phrase (two) is |
| 555 |
the number of parameters that it I<wants> to be fed. (Since you see |
| 556 |
_1 and a _2 being used in the key there.) |
| 557 |
|
| 558 |
Also, once a project is otherwise |
| 559 |
complete and you start to localize it, you can scrape together |
| 560 |
all the various keys you use, and pass it to a translator; and then |
| 561 |
the translator's work will go faster if what he's presented is this: |
| 562 |
|
| 563 |
"Minimum ([_1]) is larger than maximum ([_2])!\n", |
| 564 |
=> "", # fill in something here, Jacques! |
| 565 |
|
| 566 |
rather than this more cryptic mess: |
| 567 |
|
| 568 |
"_min_larger_max_error" |
| 569 |
=> "", # fill in something here, Jacques |
| 570 |
|
| 571 |
I think that keys as lexicon values makes the completed lexicon |
| 572 |
entries more readable: |
| 573 |
|
| 574 |
"Minimum ([_1]) is larger than maximum ([_2])!\n", |
| 575 |
=> "Le minimum ([_1]) est plus grand que le maximum ([_2])!\n", |
| 576 |
|
| 577 |
Also, having valid values as keys becomes very useful if you set |
| 578 |
up an _AUTO lexicon. _AUTO lexicons are discussed in a later |
| 579 |
section. |
| 580 |
|
| 581 |
I almost always use keys that are themselves |
| 582 |
valid lexicon values. One notable exception is when the value is |
| 583 |
quite long. For example, to get the screenful of data that |
| 584 |
a command-line program might return when given an unknown switch, |
| 585 |
I often just use a brief, self-explanatory key such as "_USAGE_MESSAGE". At that point I then go |
| 586 |
and immediately to define that lexicon entry in the |
| 587 |
ProjectClass::L10N::en lexicon (since English is always my "project |
| 588 |
language"): |
| 589 |
|
| 590 |
'_USAGE_MESSAGE' => <<'EOSTUFF', |
| 591 |
...long long message... |
| 592 |
EOSTUFF |
| 593 |
|
| 594 |
and then I can use it as: |
| 595 |
|
| 596 |
getopt('oDI', \%opts) or die $lh->maketext('_USAGE_MESSAGE'); |
| 597 |
|
| 598 |
Incidentally, |
| 599 |
note that each class's C<%Lexicon> inherits-and-extends |
| 600 |
the lexicons in its superclasses. This is not because these are |
| 601 |
special hashes I<per se>, but because you access them via the |
| 602 |
C<maketext> method, which looks for entries across all the |
| 603 |
C<%Lexicon> hashes in a language class I<and> all its ancestor classes. |
| 604 |
(This is because the idea of "class data" isn't directly implemented |
| 605 |
in Perl, but is instead left to individual class-systems to implement |
| 606 |
as they see fit..) |
| 607 |
|
| 608 |
Note that you may have things stored in a lexicon |
| 609 |
besides just phrases for output: for example, if your program |
| 610 |
takes input from the keyboard, asking a "(Y/N)" question, |
| 611 |
you probably need to know what the equivalent of "Y[es]/N[o]" is |
| 612 |
in whatever language. You probably also need to know what |
| 613 |
the equivalents of the answers "y" and "n" are. You can |
| 614 |
store that information in the lexicon (say, under the keys |
| 615 |
"~answer_y" and "~answer_n", and the long forms as |
| 616 |
"~answer_yes" and "~answer_no", where "~" is just an ad-hoc |
| 617 |
character meant to indicate to programmers/translators that |
| 618 |
these are not phrases for output). |
| 619 |
|
| 620 |
Or instead of storing this in the language class's lexicon, |
| 621 |
you can (and, in some cases, really should) represent the same bit |
| 622 |
of knowledge as code in a method in the language class. (That |
| 623 |
leaves a tidy distinction between the lexicon as the things we |
| 624 |
know how to I<say>, and the rest of the things in the lexicon class |
| 625 |
as things that we know how to I<do>.) Consider |
| 626 |
this example of a processor for responses to French "oui/non" |
| 627 |
questions: |
| 628 |
|
| 629 |
sub y_or_n { |
| 630 |
return undef unless defined $_[1] and length $_[1]; |
| 631 |
my $answer = lc $_[1]; # smash case |
| 632 |
return 1 if $answer eq 'o' or $answer eq 'oui'; |
| 633 |
return 0 if $answer eq 'n' or $answer eq 'non'; |
| 634 |
return undef; |
| 635 |
} |
| 636 |
|
| 637 |
...which you'd then call in a construct like this: |
| 638 |
|
| 639 |
my $response; |
| 640 |
until(defined $response) { |
| 641 |
print $lh->maketext("Open the pod bay door (y/n)? "); |
| 642 |
$response = $lh->y_or_n( get_input_from_keyboard_somehow() ); |
| 643 |
} |
| 644 |
if($response) { $pod_bay_door->open() } |
| 645 |
else { $pod_bay_door->leave_closed() } |
| 646 |
|
| 647 |
Other data worth storing in a lexicon might be things like |
| 648 |
filenames for language-targetted resources: |
| 649 |
|
| 650 |
... |
| 651 |
"_main_splash_png" |
| 652 |
=> "/styles/en_us/main_splash.png", |
| 653 |
"_main_splash_imagemap" |
| 654 |
=> "/styles/en_us/main_splash.incl", |
| 655 |
"_general_graphics_path" |
| 656 |
=> "/styles/en_us/", |
| 657 |
"_alert_sound" |
| 658 |
=> "/styles/en_us/hey_there.wav", |
| 659 |
"_forward_icon" |
| 660 |
=> "left_arrow.png", |
| 661 |
"_backward_icon" |
| 662 |
=> "right_arrow.png", |
| 663 |
# In some other languages, left equals |
| 664 |
# BACKwards, and right is FOREwards. |
| 665 |
... |
| 666 |
|
| 667 |
You might want to do the same thing for expressing key bindings |
| 668 |
or the like (since hardwiring "q" as the binding for the function |
| 669 |
that quits a screen/menu/program is useful only if your language |
| 670 |
happens to associate "q" with "quit"!) |
| 671 |
|
| 672 |
=head1 BRACKET NOTATION |
| 673 |
|
| 674 |
Bracket Notation is a crucial feature of Locale::Maketext. I mean |
| 675 |
Bracket Notation to provide a replacement for the use of sprintf formatting. |
| 676 |
Everything you do with Bracket Notation could be done with a sub block, |
| 677 |
but bracket notation is meant to be much more concise. |
| 678 |
|
| 679 |
Bracket Notation is a like a miniature "template" system (in the sense |
| 680 |
of L<Text::Template|Text::Template>, not in the sense of C++ templates), |
| 681 |
where normal text is passed thru basically as is, but text in special |
| 682 |
regions is specially interpreted. In Bracket Notation, you use square brackets ("[...]"), |
| 683 |
not curly braces ("{...}") to note sections that are specially interpreted. |
| 684 |
|
| 685 |
For example, here all the areas that are taken literally are underlined with |
| 686 |
a "^", and all the in-bracket special regions are underlined with an X: |
| 687 |
|
| 688 |
"Minimum ([_1]) is larger than maximum ([_2])!\n", |
| 689 |
^^^^^^^^^ XX ^^^^^^^^^^^^^^^^^^^^^^^^^^ XX ^^^^ |
| 690 |
|
| 691 |
When that string is compiled from bracket notation into a real Perl sub, |
| 692 |
it's basically turned into: |
| 693 |
|
| 694 |
sub { |
| 695 |
my $lh = $_[0]; |
| 696 |
my @params = @_; |
| 697 |
return join '', |
| 698 |
"Minimum (", |
| 699 |
...some code here... |
| 700 |
") is larger than maximum (", |
| 701 |
...some code here... |
| 702 |
")!\n", |
| 703 |
} |
| 704 |
# to be called by $lh->maketext(KEY, params...) |
| 705 |
|
| 706 |
In other words, text outside bracket groups is turned into string |
| 707 |
literals. Text in brackets is rather more complex, and currently follows |
| 708 |
these rules: |
| 709 |
|
| 710 |
=over |
| 711 |
|
| 712 |
=item * |
| 713 |
|
| 714 |
Bracket groups that are empty, or which consist only of whitespace, |
| 715 |
are ignored. (Examples: "[]", "[ ]", or a [ and a ] with returns |
| 716 |
and/or tabs and/or spaces between them. |
| 717 |
|
| 718 |
Otherwise, each group is taken to be a comma-separated group of items, |
| 719 |
and each item is interpreted as follows: |
| 720 |
|
| 721 |
=item * |
| 722 |
|
| 723 |
An item that is "_I<digits>" or "_-I<digits>" is interpreted as |
| 724 |
$_[I<value>]. I.e., "_1" becomes with $_[1], and "_-3" is interpreted |
| 725 |
as $_[-3] (in which case @_ should have at least three elements in it). |
| 726 |
Note that $_[0] is the language handle, and is typically not named |
| 727 |
directly. |
| 728 |
|
| 729 |
=item * |
| 730 |
|
| 731 |
An item "_*" is interpreted to mean "all of @_ except $_[0]". |
| 732 |
I.e., C<@_[1..$#_]>. Note that this is an empty list in the case |
| 733 |
of calls like $lh->maketext(I<key>) where there are no |
| 734 |
parameters (except $_[0], the language handle). |
| 735 |
|
| 736 |
=item * |
| 737 |
|
| 738 |
Otherwise, each item is interpreted as a string literal. |
| 739 |
|
| 740 |
=back |
| 741 |
|
| 742 |
The group as a whole is interpreted as follows: |
| 743 |
|
| 744 |
=over |
| 745 |
|
| 746 |
=item * |
| 747 |
|
| 748 |
If the first item in a bracket group looks like a method name, |
| 749 |
then that group is interpreted like this: |
| 750 |
|
| 751 |
$lh->that_method_name( |
| 752 |
...rest of items in this group... |
| 753 |
), |
| 754 |
|
| 755 |
=item * |
| 756 |
|
| 757 |
If the first item in a bracket group is "*", it's taken as shorthand |
| 758 |
for the so commonly called "quant" method. Similarly, if the first |
| 759 |
item in a bracket group is "#", it's taken to be shorthand for |
| 760 |
"numf". |
| 761 |
|
| 762 |
=item * |
| 763 |
|
| 764 |
If the first item in a bracket group is the empty-string, or "_*" |
| 765 |
or "_I<digits>" or "_-I<digits>", then that group is interpreted |
| 766 |
as just the interpolation of all its items: |
| 767 |
|
| 768 |
join('', |
| 769 |
...rest of items in this group... |
| 770 |
), |
| 771 |
|
| 772 |
Examples: "[_1]" and "[,_1]", which are synonymous; and |
| 773 |
"C<[,ID-(,_4,-,_2,)]>", which compiles as |
| 774 |
C<join "", "ID-(", $_[4], "-", $_[2], ")">. |
| 775 |
|
| 776 |
=item * |
| 777 |
|
| 778 |
Otherwise this bracket group is invalid. For example, in the group |
| 779 |
"[!@#,whatever]", the first item C<"!@#"> is neither the empty-string, |
| 780 |
"_I<number>", "_-I<number>", "_*", nor a valid method name; and so |
| 781 |
Locale::Maketext will throw an exception of you try compiling an |
| 782 |
expression containing this bracket group. |
| 783 |
|
| 784 |
=back |
| 785 |
|
| 786 |
Note, incidentally, that items in each group are comma-separated, |
| 787 |
not C</\s*,\s*/>-separated. That is, you might expect that this |
| 788 |
bracket group: |
| 789 |
|
| 790 |
"Hoohah [foo, _1 , bar ,baz]!" |
| 791 |
|
| 792 |
would compile to this: |
| 793 |
|
| 794 |
sub { |
| 795 |
my $lh = $_[0]; |
| 796 |
return join '', |
| 797 |
"Hoohah ", |
| 798 |
$lh->foo( $_[1], "bar", "baz"), |
| 799 |
"!", |
| 800 |
} |
| 801 |
|
| 802 |
But it actually compiles as this: |
| 803 |
|
| 804 |
sub { |
| 805 |
my $lh = $_[0]; |
| 806 |
return join '', |
| 807 |
"Hoohah ", |
| 808 |
$lh->foo(" _1 ", " bar ", "baz"), # note the <space> in " bar " |
| 809 |
"!", |
| 810 |
} |
| 811 |
|
| 812 |
In the notation discussed so far, the characters "[" and "]" are given |
| 813 |
special meaning, for opening and closing bracket groups, and "," has |
| 814 |
a special meaning inside bracket groups, where it separates items in the |
| 815 |
group. This begs the question of how you'd express a literal "[" or |
| 816 |
"]" in a Bracket Notation string, and how you'd express a literal |
| 817 |
comma inside a bracket group. For this purpose I've adopted "~" (tilde) |
| 818 |
as an escape character: "~[" means a literal '[' character anywhere |
| 819 |
in Bracket Notation (i.e., regardless of whether you're in a bracket |
| 820 |
group or not), and ditto for "~]" meaning a literal ']', and "~," meaning |
| 821 |
a literal comma. (Altho "," means a literal comma outside of |
| 822 |
bracket groups -- it's only inside bracket groups that commas are special.) |
| 823 |
|
| 824 |
And on the off chance you need a literal tilde in a bracket expression, |
| 825 |
you get it with "~~". |
| 826 |
|
| 827 |
Currently, an unescaped "~" before a character |
| 828 |
other than a bracket or a comma is taken to mean just a "~" and that |
| 829 |
character. I.e., "~X" means the same as "~~X" -- i.e., one literal tilde, |
| 830 |
and then one literal "X". However, by using "~X", you are assuming that |
| 831 |
no future version of Maketext will use "~X" as a magic escape sequence. |
| 832 |
In practice this is not a great problem, since first off you can just |
| 833 |
write "~~X" and not worry about it; second off, I doubt I'll add lots |
| 834 |
of new magic characters to bracket notation; and third off, you |
| 835 |
aren't likely to want literal "~" characters in your messages anyway, |
| 836 |
since it's not a character with wide use in natural language text. |
| 837 |
|
| 838 |
Brackets must be balanced -- every openbracket must have |
| 839 |
one matching closebracket, and vice versa. So these are all B<invalid>: |
| 840 |
|
| 841 |
"I ate [quant,_1,rhubarb pie." |
| 842 |
"I ate [quant,_1,rhubarb pie[." |
| 843 |
"I ate quant,_1,rhubarb pie]." |
| 844 |
"I ate quant,_1,rhubarb pie[." |
| 845 |
|
| 846 |
Currently, bracket groups do not nest. That is, you B<cannot> say: |
| 847 |
|
| 848 |
"Foo [bar,baz,[quux,quuux]]\n"; |
| 849 |
|
| 850 |
If you need a notation that's that powerful, use normal Perl: |
| 851 |
|
| 852 |
%Lexicon = ( |
| 853 |
... |
| 854 |
"some_key" => sub { |
| 855 |
my $lh = $_[0]; |
| 856 |
join '', |
| 857 |
"Foo ", |
| 858 |
$lh->bar('baz', $lh->quux('quuux')), |
| 859 |
"\n", |
| 860 |
}, |
| 861 |
... |
| 862 |
); |
| 863 |
|
| 864 |
Or write the "bar" method so you don't need to pass it the |
| 865 |
output from calling quux. |
| 866 |
|
| 867 |
I do not anticipate that you will need (or particularly want) |
| 868 |
to nest bracket groups, but you are welcome to email me with |
| 869 |
convincing (real-life) arguments to the contrary. |
| 870 |
|
| 871 |
=head1 BRACKET NOTATION SECURITY |
| 872 |
|
| 873 |
Locale::Maketext does not use any special syntax to differentiate |
| 874 |
bracket notation methods from normal class or object methods. This |
| 875 |
design makes it vulnerable to format string attacks whenever it is |
| 876 |
used to process strings provided by untrusted users. |
| 877 |
|
| 878 |
Locale::Maketext does support denylist and allowlist functionality |
| 879 |
to limit which methods may be called as bracket notation methods. |
| 880 |
|
| 881 |
By default, Locale::Maketext denies all methods in the |
| 882 |
Locale::Maketext namespace that begin with the '_' character, |
| 883 |
and all methods which include Perl's namespace separator characters. |
| 884 |
|
| 885 |
The default denylist for Locale::Maketext also prevents use of the |
| 886 |
following methods in bracket notation: |
| 887 |
|
| 888 |
denylist |
| 889 |
encoding |
| 890 |
fail_with |
| 891 |
failure_handler_auto |
| 892 |
fallback_language_classes |
| 893 |
fallback_languages |
| 894 |
get_handle |
| 895 |
init |
| 896 |
language_tag |
| 897 |
maketext |
| 898 |
new |
| 899 |
allowlist |
| 900 |
whitelist |
| 901 |
blacklist |
| 902 |
|
| 903 |
This list can be extended by either deny-listing additional "known bad" |
| 904 |
methods, or allow-listing only "known good" methods. |
| 905 |
|
| 906 |
To prevent specific methods from being called in bracket notation, use |
| 907 |
the denylist() method: |
| 908 |
|
| 909 |
my $lh = MyProgram::L10N->get_handle(); |
| 910 |
$lh->denylist(qw{my_internal_method my_other_method}); |
| 911 |
$lh->maketext('[my_internal_method]'); # dies |
| 912 |
|
| 913 |
To limit the allowed bracked notation methods to a specific list, use the |
| 914 |
allowlist() method: |
| 915 |
|
| 916 |
my $lh = MyProgram::L10N->get_handle(); |
| 917 |
$lh->allowlist('numerate', 'numf'); |
| 918 |
$lh->maketext('[_1] [numerate, _1,shoe,shoes]', 12); # works |
| 919 |
$lh->maketext('[my_internal_method]'); # dies |
| 920 |
|
| 921 |
The denylist() and allowlist() methods extend their internal lists |
| 922 |
whenever they are called. To reset the denylist or allowlist, create |
| 923 |
a new maketext object. |
| 924 |
|
| 925 |
my $lh = MyProgram::L10N->get_handle(); |
| 926 |
$lh->denylist('numerate'); |
| 927 |
$lh->denylist('numf'); |
| 928 |
$lh->maketext('[_1] [numerate,_1,shoe,shoes]', 12); # dies |
| 929 |
|
| 930 |
For lexicons that use an internal cache, translations which have already |
| 931 |
been cached in their compiled form are not affected by subsequent changes |
| 932 |
to the allowlist or denylist settings. Lexicons that use an external |
| 933 |
cache will have their cache cleared whenever the allowlist or denylist |
| 934 |
settings change. The difference between the two types of caching is explained |
| 935 |
in the "Readonly Lexicons" section. |
| 936 |
|
| 937 |
Methods disallowed by the denylist cannot be permitted by the |
| 938 |
allowlist. |
| 939 |
|
| 940 |
NOTE: denylist() is the preferred method name to use instead of the |
| 941 |
historical and non-inclusive method blacklist(). blacklist() may be |
| 942 |
removed in a future release of this package and so it's use should be |
| 943 |
removed from usage. |
| 944 |
|
| 945 |
NOTE: allowlist() is the preferred method name to use instead of the |
| 946 |
historical and non-inclusive method whitelist(). whitelist() may be |
| 947 |
removed in a future release of this package and so it's use should be |
| 948 |
removed from usage. |
| 949 |
|
| 950 |
=head1 AUTO LEXICONS |
| 951 |
|
| 952 |
If maketext goes to look in an individual %Lexicon for an entry |
| 953 |
for I<key> (where I<key> does not start with an underscore), and |
| 954 |
sees none, B<but does see> an entry of "_AUTO" => I<some_true_value>, |
| 955 |
then we actually define $Lexicon{I<key>} = I<key> right then and there, |
| 956 |
and then use that value as if it had been there all |
| 957 |
along. This happens before we even look in any superclass %Lexicons! |
| 958 |
|
| 959 |
(This is meant to be somewhat like the AUTOLOAD mechanism in |
| 960 |
Perl's function call system -- or, looked at another way, |
| 961 |
like the L<AutoLoader|AutoLoader> module.) |
| 962 |
|
| 963 |
I can picture all sorts of circumstances where you just |
| 964 |
do not want lookup to be able to fail (since failing |
| 965 |
normally means that maketext throws a C<die>, although |
| 966 |
see the next section for greater control over that). But |
| 967 |
here's one circumstance where _AUTO lexicons are meant to |
| 968 |
be I<especially> useful: |
| 969 |
|
| 970 |
As you're writing an application, you decide as you go what messages |
| 971 |
you need to emit. Normally you'd go to write this: |
| 972 |
|
| 973 |
if(-e $filename) { |
| 974 |
go_process_file($filename) |
| 975 |
} else { |
| 976 |
print qq{Couldn't find file "$filename"!\n}; |
| 977 |
} |
| 978 |
|
| 979 |
but since you anticipate localizing this, you write: |
| 980 |
|
| 981 |
use ThisProject::I18N; |
| 982 |
my $lh = ThisProject::I18N->get_handle(); |
| 983 |
# For the moment, assume that things are set up so |
| 984 |
# that we load class ThisProject::I18N::en |
| 985 |
# and that that's the class that $lh belongs to. |
| 986 |
... |
| 987 |
if(-e $filename) { |
| 988 |
go_process_file($filename) |
| 989 |
} else { |
| 990 |
print $lh->maketext( |
| 991 |
qq{Couldn't find file "[_1]"!\n}, $filename |
| 992 |
); |
| 993 |
} |
| 994 |
|
| 995 |
Now, right after you've just written the above lines, you'd |
| 996 |
normally have to go open the file |
| 997 |
ThisProject/I18N/en.pm, and immediately add an entry: |
| 998 |
|
| 999 |
"Couldn't find file \"[_1]\"!\n" |
| 1000 |
=> "Couldn't find file \"[_1]\"!\n", |
| 1001 |
|
| 1002 |
But I consider that somewhat of a distraction from the work |
| 1003 |
of getting the main code working -- to say nothing of the fact |
| 1004 |
that I often have to play with the program a few times before |
| 1005 |
I can decide exactly what wording I want in the messages (which |
| 1006 |
in this case would require me to go changing three lines of code: |
| 1007 |
the call to maketext with that key, and then the two lines in |
| 1008 |
ThisProject/I18N/en.pm). |
| 1009 |
|
| 1010 |
However, if you set "_AUTO => 1" in the %Lexicon in, |
| 1011 |
ThisProject/I18N/en.pm (assuming that English (en) is |
| 1012 |
the language that all your programmers will be using for this |
| 1013 |
project's internal message keys), then you don't ever have to |
| 1014 |
go adding lines like this |
| 1015 |
|
| 1016 |
"Couldn't find file \"[_1]\"!\n" |
| 1017 |
=> "Couldn't find file \"[_1]\"!\n", |
| 1018 |
|
| 1019 |
to ThisProject/I18N/en.pm, because if _AUTO is true there, |
| 1020 |
then just looking for an entry with the key "Couldn't find |
| 1021 |
file \"[_1]\"!\n" in that lexicon will cause it to be added, |
| 1022 |
with that value! |
| 1023 |
|
| 1024 |
Note that the reason that keys that start with "_" |
| 1025 |
are immune to _AUTO isn't anything generally magical about |
| 1026 |
the underscore character -- I just wanted a way to have most |
| 1027 |
lexicon keys be autoable, except for possibly a few, and I |
| 1028 |
arbitrarily decided to use a leading underscore as a signal |
| 1029 |
to distinguish those few. |
| 1030 |
|
| 1031 |
=head1 READONLY LEXICONS |
| 1032 |
|
| 1033 |
If your lexicon is a tied hash the simple act of caching the compiled value can be fatal. |
| 1034 |
|
| 1035 |
For example a L<GDBM_File> GDBM_READER tied hash will die with something like: |
| 1036 |
|
| 1037 |
gdbm store returned -1, errno 2, key "..." at ... |
| 1038 |
|
| 1039 |
All you need to do is turn on caching outside of the lexicon hash itself like so: |
| 1040 |
|
| 1041 |
sub init { |
| 1042 |
my ($lh) = @_; |
| 1043 |
... |
| 1044 |
$lh->{'use_external_lex_cache'} = 1; |
| 1045 |
... |
| 1046 |
} |
| 1047 |
|
| 1048 |
And then instead of storing the compiled value in the lexicon hash it will store it in $lh->{'_external_lex_cache'} |
| 1049 |
|
| 1050 |
=head1 CONTROLLING LOOKUP FAILURE |
| 1051 |
|
| 1052 |
If you call $lh->maketext(I<key>, ...parameters...), |
| 1053 |
and there's no entry I<key> in $lh's class's %Lexicon, nor |
| 1054 |
in the superclass %Lexicon hash, I<and> if we can't auto-make |
| 1055 |
I<key> (because either it starts with a "_", or because none |
| 1056 |
of its lexicons have C<_AUTO =E<gt> 1,>), then we have |
| 1057 |
failed to find a normal way to maketext I<key>. What then |
| 1058 |
happens in these failure conditions, depends on the $lh object's |
| 1059 |
"fail" attribute. |
| 1060 |
|
| 1061 |
If the language handle has no "fail" attribute, maketext |
| 1062 |
will simply throw an exception (i.e., it calls C<die>, mentioning |
| 1063 |
the I<key> whose lookup failed, and naming the line number where |
| 1064 |
the calling $lh->maketext(I<key>,...) was. |
| 1065 |
|
| 1066 |
If the language handle has a "fail" attribute whose value is a |
| 1067 |
coderef, then $lh->maketext(I<key>,...params...) gives up and calls: |
| 1068 |
|
| 1069 |
return $that_subref->($lh, $key, @params); |
| 1070 |
|
| 1071 |
Otherwise, the "fail" attribute's value should be a string denoting |
| 1072 |
a method name, so that $lh->maketext(I<key>,...params...) can |
| 1073 |
give up with: |
| 1074 |
|
| 1075 |
return $lh->$that_method_name($phrase, @params); |
| 1076 |
|
| 1077 |
The "fail" attribute can be accessed with the C<fail_with> method: |
| 1078 |
|
| 1079 |
# Set to a coderef: |
| 1080 |
$lh->fail_with( \&failure_handler ); |
| 1081 |
|
| 1082 |
# Set to a method name: |
| 1083 |
$lh->fail_with( 'failure_method' ); |
| 1084 |
|
| 1085 |
# Set to nothing (i.e., so failure throws a plain exception) |
| 1086 |
$lh->fail_with( undef ); |
| 1087 |
|
| 1088 |
# Get the current value |
| 1089 |
$handler = $lh->fail_with(); |
| 1090 |
|
| 1091 |
Now, as to what you may want to do with these handlers: Maybe you'd |
| 1092 |
want to log what key failed for what class, and then die. Maybe |
| 1093 |
you don't like C<die> and instead you want to send the error message |
| 1094 |
to STDOUT (or wherever) and then merely C<exit()>. |
| 1095 |
|
| 1096 |
Or maybe you don't want to C<die> at all! Maybe you could use a |
| 1097 |
handler like this: |
| 1098 |
|
| 1099 |
# Make all lookups fall back onto an English value, |
| 1100 |
# but only after we log it for later fingerpointing. |
| 1101 |
my $lh_backup = ThisProject->get_handle('en'); |
| 1102 |
open(LEX_FAIL_LOG, ">>wherever/lex.log") || die "GNAARGH $!"; |
| 1103 |
sub lex_fail { |
| 1104 |
my($failing_lh, $key, $params) = @_; |
| 1105 |
print LEX_FAIL_LOG scalar(localtime), "\t", |
| 1106 |
ref($failing_lh), "\t", $key, "\n"; |
| 1107 |
return $lh_backup->maketext($key,@params); |
| 1108 |
} |
| 1109 |
|
| 1110 |
Some users have expressed that they think this whole mechanism of |
| 1111 |
having a "fail" attribute at all, seems a rather pointless complication. |
| 1112 |
But I want Locale::Maketext to be usable for software projects of I<any> |
| 1113 |
scale and type; and different software projects have different ideas |
| 1114 |
of what the right thing is to do in failure conditions. I could simply |
| 1115 |
say that failure always throws an exception, and that if you want to be |
| 1116 |
careful, you'll just have to wrap every call to $lh->maketext in an |
| 1117 |
S<eval { }>. However, I want programmers to reserve the right (via |
| 1118 |
the "fail" attribute) to treat lookup failure as something other than |
| 1119 |
an exception of the same level of severity as a config file being |
| 1120 |
unreadable, or some essential resource being inaccessible. |
| 1121 |
|
| 1122 |
One possibly useful value for the "fail" attribute is the method name |
| 1123 |
"failure_handler_auto". This is a method defined in the class |
| 1124 |
Locale::Maketext itself. You set it with: |
| 1125 |
|
| 1126 |
$lh->fail_with('failure_handler_auto'); |
| 1127 |
|
| 1128 |
Then when you call $lh->maketext(I<key>, ...parameters...) and |
| 1129 |
there's no I<key> in any of those lexicons, maketext gives up with |
| 1130 |
|
| 1131 |
return $lh->failure_handler_auto($key, @params); |
| 1132 |
|
| 1133 |
But failure_handler_auto, instead of dying or anything, compiles |
| 1134 |
$key, caching it in |
| 1135 |
|
| 1136 |
$lh->{'failure_lex'}{$key} = $compiled |
| 1137 |
|
| 1138 |
and then calls the compiled value, and returns that. (I.e., if |
| 1139 |
$key looks like bracket notation, $compiled is a sub, and we return |
| 1140 |
&{$compiled}(@params); but if $key is just a plain string, we just |
| 1141 |
return that.) |
| 1142 |
|
| 1143 |
The effect of using "failure_auto_handler" |
| 1144 |
is like an AUTO lexicon, except that it 1) compiles $key even if |
| 1145 |
it starts with "_", and 2) you have a record in the new hashref |
| 1146 |
$lh->{'failure_lex'} of all the keys that have failed for |
| 1147 |
this object. This should avoid your program dying -- as long |
| 1148 |
as your keys aren't actually invalid as bracket code, and as |
| 1149 |
long as they don't try calling methods that don't exist. |
| 1150 |
|
| 1151 |
"failure_auto_handler" may not be exactly what you want, but I |
| 1152 |
hope it at least shows you that maketext failure can be mitigated |
| 1153 |
in any number of very flexible ways. If you can formalize exactly |
| 1154 |
what you want, you should be able to express that as a failure |
| 1155 |
handler. You can even make it default for every object of a given |
| 1156 |
class, by setting it in that class's init: |
| 1157 |
|
| 1158 |
sub init { |
| 1159 |
my $lh = $_[0]; # a newborn handle |
| 1160 |
$lh->SUPER::init(); |
| 1161 |
$lh->fail_with('my_clever_failure_handler'); |
| 1162 |
return; |
| 1163 |
} |
| 1164 |
sub my_clever_failure_handler { |
| 1165 |
...you clever things here... |
| 1166 |
} |
| 1167 |
|
| 1168 |
=head1 HOW TO USE MAKETEXT |
| 1169 |
|
| 1170 |
Here is a brief checklist on how to use Maketext to localize |
| 1171 |
applications: |
| 1172 |
|
| 1173 |
=over |
| 1174 |
|
| 1175 |
=item * |
| 1176 |
|
| 1177 |
Decide what system you'll use for lexicon keys. If you insist, |
| 1178 |
you can use opaque IDs (if you're nostalgic for C<catgets>), |
| 1179 |
but I have better suggestions in the |
| 1180 |
section "Entries in Each Lexicon", above. Assuming you opt for |
| 1181 |
meaningful keys that double as values (like "Minimum ([_1]) is |
| 1182 |
larger than maximum ([_2])!\n"), you'll have to settle on what |
| 1183 |
language those should be in. For the sake of argument, I'll |
| 1184 |
call this English, specifically American English, "en-US". |
| 1185 |
|
| 1186 |
=item * |
| 1187 |
|
| 1188 |
Create a class for your localization project. This is |
| 1189 |
the name of the class that you'll use in the idiom: |
| 1190 |
|
| 1191 |
use Projname::L10N; |
| 1192 |
my $lh = Projname::L10N->get_handle(...) || die "Language?"; |
| 1193 |
|
| 1194 |
Assuming you call your class Projname::L10N, create a class |
| 1195 |
consisting minimally of: |
| 1196 |
|
| 1197 |
package Projname::L10N; |
| 1198 |
use base qw(Locale::Maketext); |
| 1199 |
...any methods you might want all your languages to share... |
| 1200 |
|
| 1201 |
# And, assuming you want the base class to be an _AUTO lexicon, |
| 1202 |
# as is discussed a few sections up: |
| 1203 |
|
| 1204 |
1; |
| 1205 |
|
| 1206 |
=item * |
| 1207 |
|
| 1208 |
Create a class for the language your internal keys are in. Name |
| 1209 |
the class after the language-tag for that language, in lowercase, |
| 1210 |
with dashes changed to underscores. Assuming your project's first |
| 1211 |
language is US English, you should call this Projname::L10N::en_us. |
| 1212 |
It should consist minimally of: |
| 1213 |
|
| 1214 |
package Projname::L10N::en_us; |
| 1215 |
use base qw(Projname::L10N); |
| 1216 |
%Lexicon = ( |
| 1217 |
'_AUTO' => 1, |
| 1218 |
); |
| 1219 |
1; |
| 1220 |
|
| 1221 |
(For the rest of this section, I'll assume that this "first |
| 1222 |
language class" of Projname::L10N::en_us has |
| 1223 |
_AUTO lexicon.) |
| 1224 |
|
| 1225 |
=item * |
| 1226 |
|
| 1227 |
Go and write your program. Everywhere in your program where |
| 1228 |
you would say: |
| 1229 |
|
| 1230 |
print "Foobar $thing stuff\n"; |
| 1231 |
|
| 1232 |
instead do it thru maketext, using no variable interpolation in |
| 1233 |
the key: |
| 1234 |
|
| 1235 |
print $lh->maketext("Foobar [_1] stuff\n", $thing); |
| 1236 |
|
| 1237 |
If you get tired of constantly saying C<print $lh-E<gt>maketext>, |
| 1238 |
consider making a functional wrapper for it, like so: |
| 1239 |
|
| 1240 |
use Projname::L10N; |
| 1241 |
our $lh; |
| 1242 |
$lh = Projname::L10N->get_handle(...) || die "Language?"; |
| 1243 |
sub pmt (@) { print( $lh->maketext(@_)) } |
| 1244 |
# "pmt" is short for "Print MakeText" |
| 1245 |
$Carp::Verbose = 1; |
| 1246 |
# so if maketext fails, we see made the call to pmt |
| 1247 |
|
| 1248 |
Besides whole phrases meant for output, anything language-dependent |
| 1249 |
should be put into the class Projname::L10N::en_us, |
| 1250 |
whether as methods, or as lexicon entries -- this is discussed |
| 1251 |
in the section "Entries in Each Lexicon", above. |
| 1252 |
|
| 1253 |
=item * |
| 1254 |
|
| 1255 |
Once the program is otherwise done, and once its localization for |
| 1256 |
the first language works right (via the data and methods in |
| 1257 |
Projname::L10N::en_us), you can get together the data for translation. |
| 1258 |
If your first language lexicon isn't an _AUTO lexicon, then you already |
| 1259 |
have all the messages explicitly in the lexicon (or else you'd be |
| 1260 |
getting exceptions thrown when you call $lh->maketext to get |
| 1261 |
messages that aren't in there). But if you were (advisedly) lazy and are |
| 1262 |
using an _AUTO lexicon, then you've got to make a list of all the phrases |
| 1263 |
that you've so far been letting _AUTO generate for you. There are very |
| 1264 |
many ways to assemble such a list. The most straightforward is to simply |
| 1265 |
grep the source for every occurrence of "maketext" (or calls |
| 1266 |
to wrappers around it, like the above C<pmt> function), and to log the |
| 1267 |
following phrase. |
| 1268 |
|
| 1269 |
=item * |
| 1270 |
|
| 1271 |
You may at this point want to consider whether your base class |
| 1272 |
(Projname::L10N), from which all lexicons inherit from (Projname::L10N::en, |
| 1273 |
Projname::L10N::es, etc.), should be an _AUTO lexicon. It may be true |
| 1274 |
that in theory, all needed messages will be in each language class; |
| 1275 |
but in the presumably unlikely or "impossible" case of lookup failure, |
| 1276 |
you should consider whether your program should throw an exception, |
| 1277 |
emit text in English (or whatever your project's first language is), |
| 1278 |
or some more complex solution as described in the section |
| 1279 |
"Controlling Lookup Failure", above. |
| 1280 |
|
| 1281 |
=item * |
| 1282 |
|
| 1283 |
Submit all messages/phrases/etc. to translators. |
| 1284 |
|
| 1285 |
(You may, in fact, want to start with localizing to I<one> other language |
| 1286 |
at first, if you're not sure that you've properly abstracted the |
| 1287 |
language-dependent parts of your code.) |
| 1288 |
|
| 1289 |
Translators may request clarification of the situation in which a |
| 1290 |
particular phrase is found. For example, in English we are entirely happy |
| 1291 |
saying "I<n> files found", regardless of whether we mean "I looked for files, |
| 1292 |
and found I<n> of them" or the rather distinct situation of "I looked for |
| 1293 |
something else (like lines in files), and along the way I saw I<n> |
| 1294 |
files." This may involve rethinking things that you thought quite clear: |
| 1295 |
should "Edit" on a toolbar be a noun ("editing") or a verb ("to edit")? Is |
| 1296 |
there already a conventionalized way to express that menu option, separate |
| 1297 |
from the target language's normal word for "to edit"? |
| 1298 |
|
| 1299 |
In all cases where the very common phenomenon of quantification |
| 1300 |
(saying "I<N> files", for B<any> value of N) |
| 1301 |
is involved, each translator should make clear what dependencies the |
| 1302 |
number causes in the sentence. In many cases, dependency is |
| 1303 |
limited to words adjacent to the number, in places where you might |
| 1304 |
expect them ("I found the-?PLURAL I<N> |
| 1305 |
empty-?PLURAL directory-?PLURAL"), but in some cases there are |
| 1306 |
unexpected dependencies ("I found-?PLURAL ..."!) as well as long-distance |
| 1307 |
dependencies "The I<N> directory-?PLURAL could not be deleted-?PLURAL"!). |
| 1308 |
|
| 1309 |
Remind the translators to consider the case where N is 0: |
| 1310 |
"0 files found" isn't exactly natural-sounding in any language, but it |
| 1311 |
may be unacceptable in many -- or it may condition special |
| 1312 |
kinds of agreement (similar to English "I didN'T find ANY files"). |
| 1313 |
|
| 1314 |
Remember to ask your translators about numeral formatting in their |
| 1315 |
language, so that you can override the C<numf> method as |
| 1316 |
appropriate. Typical variables in number formatting are: what to |
| 1317 |
use as a decimal point (comma? period?); what to use as a thousands |
| 1318 |
separator (space? nonbreaking space? comma? period? small |
| 1319 |
middot? prime? apostrophe?); and even whether the so-called "thousands |
| 1320 |
separator" is actually for every third digit -- I've heard reports of |
| 1321 |
two hundred thousand being expressible as "2,00,000" for some Indian |
| 1322 |
(Subcontinental) languages, besides the less surprising "S<200 000>", |
| 1323 |
"200.000", "200,000", and "200'000". Also, using a set of numeral |
| 1324 |
glyphs other than the usual ASCII "0"-"9" might be appreciated, as via |
| 1325 |
C<tr/0-9/\x{0966}-\x{096F}/> for getting digits in Devanagari script |
| 1326 |
(for Hindi, Konkani, others). |
| 1327 |
|
| 1328 |
The basic C<quant> method that Locale::Maketext provides should be |
| 1329 |
good for many languages. For some languages, it might be useful |
| 1330 |
to modify it (or its constituent C<numerate> method) |
| 1331 |
to take a plural form in the two-argument call to C<quant> |
| 1332 |
(as in "[quant,_1,files]") if |
| 1333 |
it's all-around easier to infer the singular form from the plural, than |
| 1334 |
to infer the plural form from the singular. |
| 1335 |
|
| 1336 |
But for other languages (as is discussed at length |
| 1337 |
in L<Locale::Maketext::TPJ13|Locale::Maketext::TPJ13>), simple |
| 1338 |
C<quant>/C<numf> is not enough. For the particularly problematic |
| 1339 |
Slavic languages, what you may need is a method which you provide |
| 1340 |
with the number, the citation form of the noun to quantify, and |
| 1341 |
the case and gender that the sentence's syntax projects onto that |
| 1342 |
noun slot. The method would then be responsible for determining |
| 1343 |
what grammatical number that numeral projects onto its noun phrase, |
| 1344 |
and what case and gender it may override the normal case and gender |
| 1345 |
with; and then it would look up the noun in a lexicon providing |
| 1346 |
all needed inflected forms. |
| 1347 |
|
| 1348 |
=item * |
| 1349 |
|
| 1350 |
You may also wish to discuss with the translators the question of |
| 1351 |
how to relate different subforms of the same language tag, |
| 1352 |
considering how this reacts with C<get_handle>'s treatment of |
| 1353 |
these. For example, if a user accepts interfaces in "en, fr", and |
| 1354 |
you have interfaces available in "en-US" and "fr", what should |
| 1355 |
they get? You may wish to resolve this by establishing that "en" |
| 1356 |
and "en-US" are effectively synonymous, by having one class |
| 1357 |
zero-derive from the other. |
| 1358 |
|
| 1359 |
For some languages this issue may never come up (Danish is rarely |
| 1360 |
expressed as "da-DK", but instead is just "da"). And for other |
| 1361 |
languages, the whole concept of a "generic" form may verge on |
| 1362 |
being uselessly vague, particularly for interfaces involving voice |
| 1363 |
media in forms of Arabic or Chinese. |
| 1364 |
|
| 1365 |
=item * |
| 1366 |
|
| 1367 |
Once you've localized your program/site/etc. for all desired |
| 1368 |
languages, be sure to show the result (whether live, or via |
| 1369 |
screenshots) to the translators. Once they approve, make every |
| 1370 |
effort to have it then checked by at least one other speaker of |
| 1371 |
that language. This holds true even when (or especially when) the |
| 1372 |
translation is done by one of your own programmers. Some |
| 1373 |
kinds of systems may be harder to find testers for than others, |
| 1374 |
depending on the amount of domain-specific jargon and concepts |
| 1375 |
involved -- it's easier to find people who can tell you whether |
| 1376 |
they approve of your translation for "delete this message" in an |
| 1377 |
email-via-Web interface, than to find people who can give you |
| 1378 |
an informed opinion on your translation for "attribute value" |
| 1379 |
in an XML query tool's interface. |
| 1380 |
|
| 1381 |
=back |
| 1382 |
|
| 1383 |
=head1 SEE ALSO |
| 1384 |
|
| 1385 |
I recommend reading all of these: |
| 1386 |
|
| 1387 |
L<Locale::Maketext::TPJ13|Locale::Maketext::TPJ13> -- my I<The Perl |
| 1388 |
Journal> article about Maketext. It explains many important concepts |
| 1389 |
underlying Locale::Maketext's design, and some insight into why |
| 1390 |
Maketext is better than the plain old approach of having |
| 1391 |
message catalogs that are just databases of sprintf formats. |
| 1392 |
|
| 1393 |
L<File::Findgrep|File::Findgrep> is a sample application/module |
| 1394 |
that uses Locale::Maketext to localize its messages. For a larger |
| 1395 |
internationalized system, see also L<Apache::MP3>. |
| 1396 |
|
| 1397 |
L<I18N::LangTags|I18N::LangTags>. |
| 1398 |
|
| 1399 |
L<Win32::Locale|Win32::Locale>. |
| 1400 |
|
| 1401 |
RFC 3066, I<Tags for the Identification of Languages>, |
| 1402 |
as at L<http://sunsite.dk/RFC/rfc/rfc3066.html> |
| 1403 |
|
| 1404 |
RFC 2277, I<IETF Policy on Character Sets and Languages> |
| 1405 |
is at L<http://sunsite.dk/RFC/rfc/rfc2277.html> -- much of it is |
| 1406 |
just things of interest to protocol designers, but it explains |
| 1407 |
some basic concepts, like the distinction between locales and |
| 1408 |
language-tags. |
| 1409 |
|
| 1410 |
The manual for GNU C<gettext>. The gettext dist is available in |
| 1411 |
C<L<ftp://prep.ai.mit.edu/pub/gnu/>> -- get |
| 1412 |
a recent gettext tarball and look in its "doc/" directory, there's |
| 1413 |
an easily browsable HTML version in there. The |
| 1414 |
gettext documentation asks lots of questions worth thinking |
| 1415 |
about, even if some of their answers are sometimes wonky, |
| 1416 |
particularly where they start talking about pluralization. |
| 1417 |
|
| 1418 |
The Locale/Maketext.pm source. Observe that the module is much |
| 1419 |
shorter than its documentation! |
| 1420 |
|
| 1421 |
=head1 COPYRIGHT AND DISCLAIMER |
| 1422 |
|
| 1423 |
Copyright (c) 1999-2004 Sean M. Burke. All rights reserved. |
| 1424 |
|
| 1425 |
This library is free software; you can redistribute it and/or modify |
| 1426 |
it under the same terms as Perl itself. |
| 1427 |
|
| 1428 |
This program is distributed in the hope that it will be useful, but |
| 1429 |
without any warranty; without even the implied warranty of |
| 1430 |
merchantability or fitness for a particular purpose. |
| 1431 |
|
| 1432 |
=head1 AUTHOR |
| 1433 |
|
| 1434 |
Sean M. Burke C<[email protected]> |
| 1435 |
|
| 1436 |
=cut |