Debian Codegraph @

upstream/debian-unstable/man1/perlfaq4.1

raw
1 .\" -*- mode: troff; coding: utf-8 -*-
2 .\" Automatically generated by Pod::Man 5.0102 (Pod::Simple 3.45)
3 .\"
4 .\" Standard preamble:
5 .\" ========================================================================
6 .de Sp \" Vertical space (when we can't use .PP)
7 .if t .sp .5v
8 .if n .sp
9 ..
10 .de Vb \" Begin verbatim text
11 .ft CW
12 .nf
13 .ne \\$1
14 ..
15 .de Ve \" End verbatim text
16 .ft R
17 .fi
18 ..
19 .\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>.
20 .ie n \{\
21 . ds C` ""
22 . ds C' ""
23 'br\}
24 .el\{\
25 . ds C`
26 . ds C'
27 'br\}
28 .\"
29 .\" Escape single quotes in literal strings from groff's Unicode transform.
30 .ie \n(.g .ds Aq \(aq
31 .el .ds Aq '
32 .\"
33 .\" If the F register is >0, we'll generate index entries on stderr for
34 .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
35 .\" entries marked with X<> in POD. Of course, you'll have to process the
36 .\" output yourself in some meaningful fashion.
37 .\"
38 .\" Avoid warning from groff about undefined register 'F'.
39 .de IX
40 ..
41 .nr rF 0
42 .if \n(.g .if rF .nr rF 1
43 .if (\n(rF:(\n(.g==0)) \{\
44 . if \nF \{\
45 . de IX
46 . tm Index:\\$1\t\\n%\t"\\$2"
47 ..
48 . if !\nF==2 \{\
49 . nr % 0
50 . nr F 2
51 . \}
52 . \}
53 .\}
54 .rr rF
55 .\" ========================================================================
56 .\"
57 .IX Title "PERLFAQ4 1"
58 .TH PERLFAQ4 1 2025-11-16 "perl v5.40.1" "Perl Programmers Reference Guide"
59 .\" For nroff, turn off justification. Always turn off hyphenation; it makes
60 .\" way too many mistakes in technical documents.
61 .if n .ad l
62 .nh
63 .SH NAME
64 perlfaq4 \- Data Manipulation
65 .SH VERSION
66 .IX Header "VERSION"
67 version 5.20240218
68 .SH DESCRIPTION
69 .IX Header "DESCRIPTION"
70 This section of the FAQ answers questions related to manipulating
71 numbers, dates, strings, arrays, hashes, and miscellaneous data issues.
72 .SH "Data: Numbers"
73 .IX Header "Data: Numbers"
74 .SS "Why am I getting long decimals (eg, 19.9499999999999) instead of the numbers I should be getting (eg, 19.95)?"
75 .IX Subsection "Why am I getting long decimals (eg, 19.9499999999999) instead of the numbers I should be getting (eg, 19.95)?"
76 For the long explanation, see David Goldberg's "What Every Computer
77 Scientist Should Know About Floating-Point Arithmetic"
78 (<http://web.cse.msu.edu/~cse320/Documents/FloatingPoint.pdf>).
79 .PP
80 Internally, your computer represents floating-point numbers in binary.
81 Digital (as in powers of two) computers cannot store all numbers
82 exactly. Some real numbers lose precision in the process. This is a
83 problem with how computers store numbers and affects all computer
84 languages, not just Perl.
85 .PP
86 perlnumber shows the gory details of number representations and
87 conversions.
88 .PP
89 To limit the number of decimal places in your numbers, you can use the
90 \&\f(CW\*(C`printf\*(C'\fR or \f(CW\*(C`sprintf\*(C'\fR function. See
91 "Floating-point Arithmetic" in perlop for more details.
92 .PP
93 .Vb 1
94 \& printf "%.2f", 10/3;
95 \&
96 \& my $number = sprintf "%.2f", 10/3;
97 .Ve
98 .SS "Why is \fBint()\fP broken?"
99 .IX Subsection "Why is int() broken?"
100 Your \f(CWint()\fR is most probably working just fine. It's the numbers that
101 aren't quite what you think.
102 .PP
103 First, see the answer to "Why am I getting long decimals
104 (eg, 19.9499999999999) instead of the numbers I should be getting
105 (eg, 19.95)?".
106 .PP
107 For example, this
108 .PP
109 .Vb 1
110 \& print int(0.6/0.2\-2), "\en";
111 .Ve
112 .PP
113 will in most computers print 0, not 1, because even such simple
114 numbers as 0.6 and 0.2 cannot be presented exactly by floating-point
115 numbers. What you think in the above as 'three' is really more like
116 2.9999999999999995559.
117 .SS "Why isn't my octal data interpreted correctly?"
118 .IX Subsection "Why isn't my octal data interpreted correctly?"
119 (contributed by brian d foy)
120 .PP
121 You're probably trying to convert a string to a number, which Perl only
122 converts as a decimal number. When Perl converts a string to a number, it
123 ignores leading spaces and zeroes, then assumes the rest of the digits
124 are in base 10:
125 .PP
126 .Vb 1
127 \& my $string = \*(Aq0644\*(Aq;
128 \&
129 \& print $string + 0; # prints 644
130 \&
131 \& print $string + 44; # prints 688, certainly not octal!
132 .Ve
133 .PP
134 This problem usually involves one of the Perl built-ins that has the
135 same name a Unix command that uses octal numbers as arguments on the
136 command line. In this example, \f(CW\*(C`chmod\*(C'\fR on the command line knows that
137 its first argument is octal because that's what it does:
138 .PP
139 .Vb 1
140 \& %prompt> chmod 644 file
141 .Ve
142 .PP
143 If you want to use the same literal digits (644) in Perl, you have to tell
144 Perl to treat them as octal numbers either by prefixing the digits with
145 a \f(CW0\fR or using \f(CW\*(C`oct\*(C'\fR:
146 .PP
147 .Vb 2
148 \& chmod( 0644, $filename ); # right, has leading zero
149 \& chmod( oct(644), $filename ); # also correct
150 .Ve
151 .PP
152 The problem comes in when you take your numbers from something that Perl
153 thinks is a string, such as a command line argument in \f(CW@ARGV\fR:
154 .PP
155 .Vb 1
156 \& chmod( $ARGV[0], $filename ); # wrong, even if "0644"
157 \&
158 \& chmod( oct($ARGV[0]), $filename ); # correct, treat string as octal
159 .Ve
160 .PP
161 You can always check the value you're using by printing it in octal
162 notation to ensure it matches what you think it should be. Print it
163 in octal and decimal format:
164 .PP
165 .Vb 1
166 \& printf "0%o %d", $number, $number;
167 .Ve
168 .SS "Does Perl have a \fBround()\fP function? What about \fBceil()\fP and \fBfloor()\fP? Trig functions?"
169 .IX Subsection "Does Perl have a round() function? What about ceil() and floor()? Trig functions?"
170 Remember that \f(CWint()\fR merely truncates toward 0. For rounding to a
171 certain number of digits, \f(CWsprintf()\fR or \f(CWprintf()\fR is usually the
172 easiest route.
173 .PP
174 .Vb 1
175 \& printf("%.3f", 3.1415926535); # prints 3.142
176 .Ve
177 .PP
178 The POSIX module (part of the standard Perl distribution)
179 implements \f(CWceil()\fR, \f(CWfloor()\fR, and a number of other mathematical
180 and trigonometric functions.
181 .PP
182 .Vb 3
183 \& use POSIX;
184 \& my $ceil = ceil(3.5); # 4
185 \& my $floor = floor(3.5); # 3
186 .Ve
187 .PP
188 In 5.000 to 5.003 perls, trigonometry was done in the Math::Complex
189 module. With 5.004, the Math::Trig module (part of the standard Perl
190 distribution) implements the trigonometric functions. Internally it
191 uses the Math::Complex module and some functions can break out from
192 the real axis into the complex plane, for example the inverse sine of
193 2.
194 .PP
195 Rounding in financial applications can have serious implications, and
196 the rounding method used should be specified precisely. In these
197 cases, it probably pays not to trust whichever system of rounding is
198 being used by Perl, but instead to implement the rounding function you
199 need yourself.
200 .PP
201 To see why, notice how you'll still have an issue on half-way-point
202 alternation:
203 .PP
204 .Vb 1
205 \& for (my $i = \-5; $i <= 5; $i += 0.5) { printf "%.0f ",$i }
206 \&
207 \& \-5 \-4 \-4 \-4 \-3 \-2 \-2 \-2 \-1 \-0 0 0 1 2 2 2 3 4 4 4 5
208 .Ve
209 .PP
210 Don't blame Perl. It's the same as in C. IEEE says we have to do
211 this. Perl numbers whose absolute values are integers under 2**31 (on
212 32\-bit machines) will work pretty much like mathematical integers.
213 Other numbers are not guaranteed.
214 .SS "How do I convert between numeric representations/bases/radixes?"
215 .IX Subsection "How do I convert between numeric representations/bases/radixes?"
216 As always with Perl there is more than one way to do it. Below are a
217 few examples of approaches to making common conversions between number
218 representations. This is intended to be representational rather than
219 exhaustive.
220 .PP
221 Some of the examples later in perlfaq4 use the Bit::Vector
222 module from CPAN. The reason you might choose Bit::Vector over the
223 perl built-in functions is that it works with numbers of ANY size,
224 that it is optimized for speed on some operations, and for at least
225 some programmers the notation might be familiar.
226 .IP "How do I convert hexadecimal into decimal" 4
227 .IX Item "How do I convert hexadecimal into decimal"
228 Using perl's built in conversion of \f(CW\*(C`0x\*(C'\fR notation:
229 .Sp
230 .Vb 1
231 \& my $dec = 0xDEADBEEF;
232 .Ve
233 .Sp
234 Using the \f(CW\*(C`hex\*(C'\fR function:
235 .Sp
236 .Vb 1
237 \& my $dec = hex("DEADBEEF");
238 .Ve
239 .Sp
240 Using \f(CW\*(C`pack\*(C'\fR:
241 .Sp
242 .Vb 1
243 \& my $dec = unpack("N", pack("H8", substr("0" x 8 . "DEADBEEF", \-8)));
244 .Ve
245 .Sp
246 Using the CPAN module \f(CW\*(C`Bit::Vector\*(C'\fR:
247 .Sp
248 .Vb 3
249 \& use Bit::Vector;
250 \& my $vec = Bit::Vector\->new_Hex(32, "DEADBEEF");
251 \& my $dec = $vec\->to_Dec();
252 .Ve
253 .IP "How do I convert from decimal to hexadecimal" 4
254 .IX Item "How do I convert from decimal to hexadecimal"
255 Using \f(CW\*(C`sprintf\*(C'\fR:
256 .Sp
257 .Vb 2
258 \& my $hex = sprintf("%X", 3735928559); # upper case A\-F
259 \& my $hex = sprintf("%x", 3735928559); # lower case a\-f
260 .Ve
261 .Sp
262 Using \f(CW\*(C`unpack\*(C'\fR:
263 .Sp
264 .Vb 1
265 \& my $hex = unpack("H*", pack("N", 3735928559));
266 .Ve
267 .Sp
268 Using Bit::Vector:
269 .Sp
270 .Vb 3
271 \& use Bit::Vector;
272 \& my $vec = Bit::Vector\->new_Dec(32, \-559038737);
273 \& my $hex = $vec\->to_Hex();
274 .Ve
275 .Sp
276 And Bit::Vector supports odd bit counts:
277 .Sp
278 .Vb 4
279 \& use Bit::Vector;
280 \& my $vec = Bit::Vector\->new_Dec(33, 3735928559);
281 \& $vec\->Resize(32); # suppress leading 0 if unwanted
282 \& my $hex = $vec\->to_Hex();
283 .Ve
284 .IP "How do I convert from octal to decimal" 4
285 .IX Item "How do I convert from octal to decimal"
286 Using Perl's built in conversion of numbers with leading zeros:
287 .Sp
288 .Vb 1
289 \& my $dec = 033653337357; # note the leading 0!
290 .Ve
291 .Sp
292 Using the \f(CW\*(C`oct\*(C'\fR function:
293 .Sp
294 .Vb 1
295 \& my $dec = oct("33653337357");
296 .Ve
297 .Sp
298 Using Bit::Vector:
299 .Sp
300 .Vb 4
301 \& use Bit::Vector;
302 \& my $vec = Bit::Vector\->new(32);
303 \& $vec\->Chunk_List_Store(3, split(//, reverse "33653337357"));
304 \& my $dec = $vec\->to_Dec();
305 .Ve
306 .IP "How do I convert from decimal to octal" 4
307 .IX Item "How do I convert from decimal to octal"
308 Using \f(CW\*(C`sprintf\*(C'\fR:
309 .Sp
310 .Vb 1
311 \& my $oct = sprintf("%o", 3735928559);
312 .Ve
313 .Sp
314 Using Bit::Vector:
315 .Sp
316 .Vb 3
317 \& use Bit::Vector;
318 \& my $vec = Bit::Vector\->new_Dec(32, \-559038737);
319 \& my $oct = reverse join(\*(Aq\*(Aq, $vec\->Chunk_List_Read(3));
320 .Ve
321 .IP "How do I convert from binary to decimal" 4
322 .IX Item "How do I convert from binary to decimal"
323 Perl 5.6 lets you write binary numbers directly with
324 the \f(CW\*(C`0b\*(C'\fR notation:
325 .Sp
326 .Vb 1
327 \& my $number = 0b10110110;
328 .Ve
329 .Sp
330 Using \f(CW\*(C`oct\*(C'\fR:
331 .Sp
332 .Vb 2
333 \& my $input = "10110110";
334 \& my $decimal = oct( "0b$input" );
335 .Ve
336 .Sp
337 Using \f(CW\*(C`pack\*(C'\fR and \f(CW\*(C`ord\*(C'\fR:
338 .Sp
339 .Vb 1
340 \& my $decimal = ord(pack(\*(AqB8\*(Aq, \*(Aq10110110\*(Aq));
341 .Ve
342 .Sp
343 Using \f(CW\*(C`pack\*(C'\fR and \f(CW\*(C`unpack\*(C'\fR for larger strings:
344 .Sp
345 .Vb 3
346 \& my $int = unpack("N", pack("B32",
347 \& substr("0" x 32 . "11110101011011011111011101111", \-32)));
348 \& my $dec = sprintf("%d", $int);
349 \&
350 \& # substr() is used to left\-pad a 32\-character string with zeros.
351 .Ve
352 .Sp
353 Using Bit::Vector:
354 .Sp
355 .Vb 2
356 \& my $vec = Bit::Vector\->new_Bin(32, "11011110101011011011111011101111");
357 \& my $dec = $vec\->to_Dec();
358 .Ve
359 .IP "How do I convert from decimal to binary" 4
360 .IX Item "How do I convert from decimal to binary"
361 Using \f(CW\*(C`sprintf\*(C'\fR (perl 5.6+):
362 .Sp
363 .Vb 1
364 \& my $bin = sprintf("%b", 3735928559);
365 .Ve
366 .Sp
367 Using \f(CW\*(C`unpack\*(C'\fR:
368 .Sp
369 .Vb 1
370 \& my $bin = unpack("B*", pack("N", 3735928559));
371 .Ve
372 .Sp
373 Using Bit::Vector:
374 .Sp
375 .Vb 3
376 \& use Bit::Vector;
377 \& my $vec = Bit::Vector\->new_Dec(32, \-559038737);
378 \& my $bin = $vec\->to_Bin();
379 .Ve
380 .Sp
381 The remaining transformations (e.g. hex \-> oct, bin \-> hex, etc.)
382 are left as an exercise to the inclined reader.
383 .SS "Why doesn't & work the way I want it to?"
384 .IX Subsection "Why doesn't & work the way I want it to?"
385 Perl's \f(CW\*(C`&\*(C'\fR bitwise operator works on both numbers and strings,
386 sometimes producing surprising results when you expected a number
387 but received a string. You probably expected perl to automatically
388 convert the operands to numbers like the mathematical operators would.
389 Instead, perl treats string operands as bitvectors.
390 .PP
391 Consider the bitwise difference between the number 3 and the bitvector
392 represented by "3". A number has the bit pattern for its magnitude. The
393 number 3 is 0b11 (a 2 and a 1). The bitvector has the bit pattern that
394 is the ordinal value for each octet, and that value is unrelated to any
395 numeric value that the digit represents. The character "3" is the
396 bitvector 0b0011_0011.
397 .PP
398 These operations have different results even though you might think they
399 look like the same "number":
400 .PP
401 .Vb 4
402 \& 11 & 3; # 0b0000_1011 & 0b0000_0011
403 \& # \-> 0b0000_0011 (number 3)
404 \& "11" & "3"; # 0b0011_0001_0011_0001 & 0b0011_0011
405 \& # \-> 0b0011_0001 (ASCII char "1")
406 .Ve
407 .PP
408 Note that if any operand has a numeric value, perl uses numeric
409 semantics (although you should not count on this):
410 .PP
411 .Vb 3
412 \& my($i, $j) = ( 11, 3 ); # $i & $j # 11 & 3 \-> 3
413 \& my($i, $j) = ("11", 3 ); # $i & $j # 11 & 3 \-> 3
414 \& my($i, $j) = ("11", "3"); # $i & $j # "11" & "3" \-> 1
415 .Ve
416 .PP
417 Remember that a perl scalar can have both string and numeric values at
418 the same time. A value that starts as a string and has never encountered
419 a numeric operation has no numeric value yet. Perl does this to save
420 time and work so it doesn't have to decide a numeric value for a scalar
421 it might never use as a number. In that case, string semantics wins. But,
422 if there is a numeric value already, numeric semantics win. Force perl
423 to compute the numeric value by adding 0:
424 .PP
425 .Vb 1
426 \& my($i, $j) = ("11", "3"); $j += 0 # $i & $j # "11" & 3 \-> 3
427 .Ve
428 .PP
429 However, this is not a documented feature, or as perlop says, it "is not
430 well defined". One way to fix ensure numeric semantics is to explicitly
431 convert both of values to numbers:
432 .PP
433 .Vb 1
434 \& (0+$i) & (0+$j)
435 .Ve
436 .PP
437 To fix this annoyance, Perl v5.22 separated the string and number
438 behavior. The \f(CW\*(C`bitwise\*(C'\fR feature introduced four new operators that
439 would work with only string semantics: \f(CW\*(C`&.\*(C'\fR, \f(CW\*(C`|.\*(C'\fR, \f(CW\*(C`^.\*(C'\fR, and \f(CW\*(C`~.\*(C'\fR.
440 The original operators, \f(CW\*(C`&\*(C'\fR, \f(CW\*(C`|\*(C'\fR, \f(CW\*(C`^\*(C'\fR, and \f(CW\*(C`~\*(C'\fR, would then apply
441 only numeric semantics.
442 .PP
443 Enable this feature explicitly with feature:
444 .PP
445 .Vb 1
446 \& use feature qw(bitwise);
447 .Ve
448 .PP
449 Or, as of v5.28, require the minimum version of perl with \f(CW\*(C`use\*(C'\fR:
450 .PP
451 .Vb 1
452 \& use v5.28; # bitwise feature for free
453 .Ve
454 .SS "How do I multiply matrices?"
455 .IX Subsection "How do I multiply matrices?"
456 Use the Math::Matrix or Math::MatrixReal modules (available from CPAN)
457 or the PDL extension (also available from CPAN).
458 .SS "How do I perform an operation on a series of integers?"
459 .IX Subsection "How do I perform an operation on a series of integers?"
460 To call a function on each element in an array, and collect the
461 results, use:
462 .PP
463 .Vb 1
464 \& my @results = map { my_func($_) } @array;
465 .Ve
466 .PP
467 For example:
468 .PP
469 .Vb 1
470 \& my @triple = map { 3 * $_ } @single;
471 .Ve
472 .PP
473 To call a function on each element of an array, but ignore the
474 results:
475 .PP
476 .Vb 3
477 \& foreach my $iterator (@array) {
478 \& some_func($iterator);
479 \& }
480 .Ve
481 .PP
482 To call a function on each integer in a (small) range, you \fBcan\fR use:
483 .PP
484 .Vb 1
485 \& my @results = map { some_func($_) } (5 .. 25);
486 .Ve
487 .PP
488 but you should be aware that in this form, the \f(CW\*(C`..\*(C'\fR operator
489 creates a list of all integers in the range, which can take a lot of
490 memory for large ranges. However, the problem does not occur when
491 using \f(CW\*(C`..\*(C'\fR within a \f(CW\*(C`for\*(C'\fR loop, because in that case the range
492 operator is optimized to \fIiterate\fR over the range, without creating
493 the entire list. So
494 .PP
495 .Vb 4
496 \& my @results = ();
497 \& for my $i (5 .. 500_005) {
498 \& push(@results, some_func($i));
499 \& }
500 .Ve
501 .PP
502 or even
503 .PP
504 .Vb 1
505 \& push(@results, some_func($_)) for 5 .. 500_005;
506 .Ve
507 .PP
508 will not create an intermediate list of 500,000 integers.
509 .SS "How can I output Roman numerals?"
510 .IX Subsection "How can I output Roman numerals?"
511 Get the <http://www.cpan.org/modules/by\-module/Roman> module.
512 .SS "Why aren't my random numbers random?"
513 .IX Subsection "Why aren't my random numbers random?"
514 If you're using a version of Perl before 5.004, you must call \f(CW\*(C`srand\*(C'\fR
515 once at the start of your program to seed the random number generator.
516 .PP
517 .Vb 1
518 \& BEGIN { srand() if $] < 5.004 }
519 .Ve
520 .PP
521 5.004 and later automatically call \f(CW\*(C`srand\*(C'\fR at the beginning. Don't
522 call \f(CW\*(C`srand\*(C'\fR more than once\-\-you make your numbers less random,
523 rather than more.
524 .PP
525 Computers are good at being predictable and bad at being random
526 (despite appearances caused by bugs in your programs :\-). The
527 \&\fIrandom\fR article in the "Far More Than You Ever Wanted To Know"
528 collection in <http://www.cpan.org/misc/olddoc/FMTEYEWTK.tgz>, courtesy
529 of Tom Phoenix, talks more about this. John von Neumann said, "Anyone
530 who attempts to generate random numbers by deterministic means is, of
531 course, living in a state of sin."
532 .PP
533 Perl relies on the underlying system for the implementation of
534 \&\f(CW\*(C`rand\*(C'\fR and \f(CW\*(C`srand\*(C'\fR; on some systems, the generated numbers are
535 not random enough (especially on Windows : see
536 <http://www.perlmonks.org/?node_id=803632>).
537 Several CPAN modules in the \f(CW\*(C`Math\*(C'\fR namespace implement better
538 pseudorandom generators; see for example
539 Math::Random::MT ("Mersenne Twister", fast), or
540 Math::TrulyRandom (uses the imperfections in the system's
541 timer to generate random numbers, which is rather slow).
542 More algorithms for random numbers are described in
543 "Numerical Recipes in C" at <http://www.nr.com/>
544 .SS "How do I get a random number between X and Y?"
545 .IX Subsection "How do I get a random number between X and Y?"
546 To get a random number between two values, you can use the \f(CWrand()\fR
547 built-in to get a random number between 0 and 1. From there, you shift
548 that into the range that you want.
549 .PP
550 \&\f(CWrand($x)\fR returns a number such that \f(CW\*(C`0 <= rand($x) < $x\*(C'\fR. Thus
551 what you want to have perl figure out is a random number in the range
552 from 0 to the difference between your \fIX\fR and \fIY\fR.
553 .PP
554 That is, to get a number between 10 and 15, inclusive, you want a
555 random number between 0 and 5 that you can then add to 10.
556 .PP
557 .Vb 1
558 \& my $number = 10 + int rand( 15\-10+1 ); # ( 10,11,12,13,14, or 15 )
559 .Ve
560 .PP
561 Hence you derive the following simple function to abstract
562 that. It selects a random integer between the two given
563 integers (inclusive). For example: \f(CW\*(C`random_int_between(50,120)\*(C'\fR.
564 .PP
565 .Vb 7
566 \& sub random_int_between {
567 \& my($min, $max) = @_;
568 \& # Assumes that the two arguments are integers themselves!
569 \& return $min if $min == $max;
570 \& ($min, $max) = ($max, $min) if $min > $max;
571 \& return $min + int rand(1 + $max \- $min);
572 \& }
573 .Ve
574 .SH "Data: Dates"
575 .IX Header "Data: Dates"
576 .SS "How do I find the day or week of the year?"
577 .IX Subsection "How do I find the day or week of the year?"
578 The day of the year is in the list returned
579 by the \f(CW\*(C`localtime\*(C'\fR function. Without an
580 argument \f(CW\*(C`localtime\*(C'\fR uses the current time.
581 .PP
582 .Vb 1
583 \& my $day_of_year = (localtime)[7];
584 .Ve
585 .PP
586 The POSIX module can also format a date as the day of the year or
587 week of the year.
588 .PP
589 .Vb 3
590 \& use POSIX qw/strftime/;
591 \& my $day_of_year = strftime "%j", localtime;
592 \& my $week_of_year = strftime "%W", localtime;
593 .Ve
594 .PP
595 To get the day of year for any date, use POSIX's \f(CW\*(C`mktime\*(C'\fR to get
596 a time in epoch seconds for the argument to \f(CW\*(C`localtime\*(C'\fR.
597 .PP
598 .Vb 3
599 \& use POSIX qw/mktime strftime/;
600 \& my $week_of_year = strftime "%j",
601 \& localtime( mktime( 0, 0, 0, 18, 11, 87 ) );
602 .Ve
603 .PP
604 You can also use Time::Piece, which comes with Perl and provides a
605 \&\f(CW\*(C`localtime\*(C'\fR that returns an object:
606 .PP
607 .Vb 3
608 \& use Time::Piece;
609 \& my $day_of_year = localtime\->yday;
610 \& my $week_of_year = localtime\->week;
611 .Ve
612 .PP
613 The Date::Calc module provides two functions to calculate these, too:
614 .PP
615 .Vb 3
616 \& use Date::Calc;
617 \& my $day_of_year = Day_of_Year( 1987, 12, 18 );
618 \& my $week_of_year = Week_of_Year( 1987, 12, 18 );
619 .Ve
620 .SS "How do I find the current century or millennium?"
621 .IX Subsection "How do I find the current century or millennium?"
622 Use the following simple functions:
623 .PP
624 .Vb 3
625 \& sub get_century {
626 \& return int((((localtime(shift || time))[5] + 1999))/100);
627 \& }
628 \&
629 \& sub get_millennium {
630 \& return 1+int((((localtime(shift || time))[5] + 1899))/1000);
631 \& }
632 .Ve
633 .PP
634 On some systems, the POSIX module's \f(CWstrftime()\fR function has been
635 extended in a non-standard way to use a \f(CW%C\fR format, which they
636 sometimes claim is the "century". It isn't, because on most such
637 systems, this is only the first two digits of the four-digit year, and
638 thus cannot be used to determine reliably the current century or
639 millennium.
640 .SS "How can I compare two dates and find the difference?"
641 .IX Subsection "How can I compare two dates and find the difference?"
642 (contributed by brian d foy)
643 .PP
644 You could just store all your dates as a number and then subtract.
645 Life isn't always that simple though.
646 .PP
647 The Time::Piece module, which comes with Perl, replaces localtime
648 with a version that returns an object. It also overloads the comparison
649 operators so you can compare them directly:
650 .PP
651 .Vb 3
652 \& use Time::Piece;
653 \& my $date1 = localtime( $some_time );
654 \& my $date2 = localtime( $some_other_time );
655 \&
656 \& if( $date1 < $date2 ) {
657 \& print "The date was in the past\en";
658 \& }
659 .Ve
660 .PP
661 You can also get differences with a subtraction, which returns a
662 Time::Seconds object:
663 .PP
664 .Vb 2
665 \& my $date_diff = $date1 \- $date2;
666 \& print "The difference is ", $date_diff\->days, " days\en";
667 .Ve
668 .PP
669 If you want to work with formatted dates, the Date::Manip,
670 Date::Calc, or DateTime modules can help you.
671 .SS "How can I take a string and turn it into epoch seconds?"
672 .IX Subsection "How can I take a string and turn it into epoch seconds?"
673 If it's a regular enough string that it always has the same format,
674 you can split it up and pass the parts to \f(CW\*(C`timelocal\*(C'\fR in the standard
675 Time::Local module. Otherwise, you should look into the Date::Calc,
676 Date::Parse, and Date::Manip modules from CPAN.
677 .SS "How can I find the Julian Day?"
678 .IX Subsection "How can I find the Julian Day?"
679 (contributed by brian d foy and Dave Cross)
680 .PP
681 You can use the Time::Piece module, part of the Standard Library,
682 which can convert a date/time to a Julian Day:
683 .PP
684 .Vb 2
685 \& $ perl \-MTime::Piece \-le \*(Aqprint localtime\->julian_day\*(Aq
686 \& 2455607.7959375
687 .Ve
688 .PP
689 Or the modified Julian Day:
690 .PP
691 .Vb 2
692 \& $ perl \-MTime::Piece \-le \*(Aqprint localtime\->mjd\*(Aq
693 \& 55607.2961226851
694 .Ve
695 .PP
696 Or even the day of the year (which is what some people think of as a
697 Julian day):
698 .PP
699 .Vb 2
700 \& $ perl \-MTime::Piece \-le \*(Aqprint localtime\->yday\*(Aq
701 \& 45
702 .Ve
703 .PP
704 You can also do the same things with the DateTime module:
705 .PP
706 .Vb 6
707 \& $ perl \-MDateTime \-le\*(Aqprint DateTime\->today\->jd\*(Aq
708 \& 2453401.5
709 \& $ perl \-MDateTime \-le\*(Aqprint DateTime\->today\->mjd\*(Aq
710 \& 53401
711 \& $ perl \-MDateTime \-le\*(Aqprint DateTime\->today\->doy\*(Aq
712 \& 31
713 .Ve
714 .PP
715 You can use the Time::JulianDay module available on CPAN. Ensure
716 that you really want to find a Julian day, though, as many people have
717 different ideas about Julian days (see <http://www.hermetic.ch/cal_stud/jdn.htm>
718 for instance):
719 .PP
720 .Vb 2
721 \& $ perl \-MTime::JulianDay \-le \*(Aqprint local_julian_day( time )\*(Aq
722 \& 55608
723 .Ve
724 .SS "How do I find yesterday's date?"
725 .IX Xref "date yesterday DateTime Date::Calc Time::Local daylight saving time day Today_and_Now localtime timelocal"
726 .IX Subsection "How do I find yesterday's date?"
727 (contributed by brian d foy)
728 .PP
729 To do it correctly, you can use one of the \f(CW\*(C`Date\*(C'\fR modules since they
730 work with calendars instead of times. The DateTime module makes it
731 simple, and give you the same time of day, only the day before,
732 despite daylight saving time changes:
733 .PP
734 .Vb 1
735 \& use DateTime;
736 \&
737 \& my $yesterday = DateTime\->now\->subtract( days => 1 );
738 \&
739 \& print "Yesterday was $yesterday\en";
740 .Ve
741 .PP
742 You can also use the Date::Calc module using its \f(CW\*(C`Today_and_Now\*(C'\fR
743 function.
744 .PP
745 .Vb 1
746 \& use Date::Calc qw( Today_and_Now Add_Delta_DHMS );
747 \&
748 \& my @date_time = Add_Delta_DHMS( Today_and_Now(), \-1, 0, 0, 0 );
749 \&
750 \& print "@date_time\en";
751 .Ve
752 .PP
753 Most people try to use the time rather than the calendar to figure out
754 dates, but that assumes that days are twenty-four hours each. For
755 most people, there are two days a year when they aren't: the switch to
756 and from summer time throws this off. For example, the rest of the
757 suggestions will be wrong sometimes:
758 .PP
759 Starting with Perl 5.10, Time::Piece and Time::Seconds are part
760 of the standard distribution, so you might think that you could do
761 something like this:
762 .PP
763 .Vb 2
764 \& use Time::Piece;
765 \& use Time::Seconds;
766 \&
767 \& my $yesterday = localtime() \- ONE_DAY; # WRONG
768 \& print "Yesterday was $yesterday\en";
769 .Ve
770 .PP
771 The Time::Piece module exports a new \f(CW\*(C`localtime\*(C'\fR that returns an
772 object, and Time::Seconds exports the \f(CW\*(C`ONE_DAY\*(C'\fR constant that is a
773 set number of seconds. This means that it always gives the time 24
774 hours ago, which is not always yesterday. This can cause problems
775 around the end of daylight saving time when there's one day that is 25
776 hours long.
777 .PP
778 You have the same problem with Time::Local, which will give the wrong
779 answer for those same special cases:
780 .PP
781 .Vb 5
782 \& # contributed by Gunnar Hjalmarsson
783 \& use Time::Local;
784 \& my $today = timelocal 0, 0, 12, ( localtime )[3..5];
785 \& my ($d, $m, $y) = ( localtime $today\-86400 )[3..5]; # WRONG
786 \& printf "Yesterday: %d\-%02d\-%02d\en", $y+1900, $m+1, $d;
787 .Ve
788 .SS "Does Perl have a Year 2000 or 2038 problem? Is Perl Y2K compliant?"
789 .IX Subsection "Does Perl have a Year 2000 or 2038 problem? Is Perl Y2K compliant?"
790 (contributed by brian d foy)
791 .PP
792 Perl itself never had a Y2K problem, although that never stopped people
793 from creating Y2K problems on their own. See the documentation for
794 \&\f(CW\*(C`localtime\*(C'\fR for its proper use.
795 .PP
796 Starting with Perl 5.12, \f(CW\*(C`localtime\*(C'\fR and \f(CW\*(C`gmtime\*(C'\fR can handle dates past
797 03:14:08 January 19, 2038, when a 32\-bit based time would overflow. You
798 still might get a warning on a 32\-bit \f(CW\*(C`perl\*(C'\fR:
799 .PP
800 .Vb 3
801 \& % perl5.12 \-E \*(Aqsay scalar localtime( 0x9FFF_FFFFFFFF )\*(Aq
802 \& Integer overflow in hexadecimal number at \-e line 1.
803 \& Wed Nov 1 19:42:39 5576711
804 .Ve
805 .PP
806 On a 64\-bit \f(CW\*(C`perl\*(C'\fR, you can get even larger dates for those really long
807 running projects:
808 .PP
809 .Vb 2
810 \& % perl5.12 \-E \*(Aqsay scalar gmtime( 0x9FFF_FFFFFFFF )\*(Aq
811 \& Thu Nov 2 00:42:39 5576711
812 .Ve
813 .PP
814 You're still out of luck if you need to keep track of decaying protons
815 though.
816 .SH "Data: Strings"
817 .IX Header "Data: Strings"
818 .SS "How do I validate input?"
819 .IX Subsection "How do I validate input?"
820 (contributed by brian d foy)
821 .PP
822 There are many ways to ensure that values are what you expect or
823 want to accept. Besides the specific examples that we cover in the
824 perlfaq, you can also look at the modules with "Assert" and "Validate"
825 in their names, along with other modules such as Regexp::Common.
826 .PP
827 Some modules have validation for particular types of input, such
828 as Business::ISBN, Business::CreditCard, Email::Valid,
829 and Data::Validate::IP.
830 .SS "How do I unescape a string?"
831 .IX Subsection "How do I unescape a string?"
832 It depends just what you mean by "escape". URL escapes are dealt
833 with in perlfaq9. Shell escapes with the backslash (\f(CW\*(C`\e\*(C'\fR)
834 character are removed with
835 .PP
836 .Vb 1
837 \& s/\e\e(.)/$1/g;
838 .Ve
839 .PP
840 This won't expand \f(CW"\en"\fR or \f(CW"\et"\fR or any other special escapes.
841 .SS "How do I remove consecutive pairs of characters?"
842 .IX Subsection "How do I remove consecutive pairs of characters?"
843 (contributed by brian d foy)
844 .PP
845 You can use the substitution operator to find pairs of characters (or
846 runs of characters) and replace them with a single instance. In this
847 substitution, we find a character in \f(CW\*(C`(.)\*(C'\fR. The memory parentheses
848 store the matched character in the back-reference \f(CW\*(C`\eg1\*(C'\fR and we use
849 that to require that the same thing immediately follow it. We replace
850 that part of the string with the character in \f(CW$1\fR.
851 .PP
852 .Vb 1
853 \& s/(.)\eg1/$1/g;
854 .Ve
855 .PP
856 We can also use the transliteration operator, \f(CW\*(C`tr///\*(C'\fR. In this
857 example, the search list side of our \f(CW\*(C`tr///\*(C'\fR contains nothing, but
858 the \f(CW\*(C`c\*(C'\fR option complements that so it contains everything. The
859 replacement list also contains nothing, so the transliteration is
860 almost a no-op since it won't do any replacements (or more exactly,
861 replace the character with itself). However, the \f(CW\*(C`s\*(C'\fR option squashes
862 duplicated and consecutive characters in the string so a character
863 does not show up next to itself
864 .PP
865 .Vb 2
866 \& my $str = \*(AqHaarlem\*(Aq; # in the Netherlands
867 \& $str =~ tr///cs; # Now Harlem, like in New York
868 .Ve
869 .SS "How do I expand function calls in a string?"
870 .IX Subsection "How do I expand function calls in a string?"
871 (contributed by brian d foy)
872 .PP
873 This is documented in perlref, and although it's not the easiest
874 thing to read, it does work. In each of these examples, we call the
875 function inside the braces used to dereference a reference. If we
876 have more than one return value, we can construct and dereference an
877 anonymous array. In this case, we call the function in list context.
878 .PP
879 .Vb 1
880 \& print "The time values are @{ [localtime] }.\en";
881 .Ve
882 .PP
883 If we want to call the function in scalar context, we have to do a bit
884 more work. We can really have any code we like inside the braces, so
885 we simply have to end with the scalar reference, although how you do
886 that is up to you, and you can use code inside the braces. Note that
887 the use of parens creates a list context, so we need \f(CW\*(C`scalar\*(C'\fR to
888 force the scalar context on the function:
889 .PP
890 .Vb 1
891 \& print "The time is ${\e(scalar localtime)}.\en"
892 \&
893 \& print "The time is ${ my $x = localtime; \e$x }.\en";
894 .Ve
895 .PP
896 If your function already returns a reference, you don't need to create
897 the reference yourself.
898 .PP
899 .Vb 1
900 \& sub timestamp { my $t = localtime; \e$t }
901 \&
902 \& print "The time is ${ timestamp() }.\en";
903 .Ve
904 .PP
905 The \f(CW\*(C`Interpolation\*(C'\fR module can also do a lot of magic for you. You can
906 specify a variable name, in this case \f(CW\*(C`E\*(C'\fR, to set up a tied hash that
907 does the interpolation for you. It has several other methods to do this
908 as well.
909 .PP
910 .Vb 2
911 \& use Interpolation E => \*(Aqeval\*(Aq;
912 \& print "The time values are $E{localtime()}.\en";
913 .Ve
914 .PP
915 In most cases, it is probably easier to simply use string concatenation,
916 which also forces scalar context.
917 .PP
918 .Vb 1
919 \& print "The time is " . localtime() . ".\en";
920 .Ve
921 .SS "How do I find matching/nesting anything?"
922 .IX Subsection "How do I find matching/nesting anything?"
923 To find something between two single
924 characters, a pattern like \f(CW\*(C`/x([^x]*)x/\*(C'\fR will get the intervening
925 bits in \f(CW$1\fR. For multiple ones, then something more like
926 \&\f(CW\*(C`/alpha(.*?)omega/\*(C'\fR would be needed. For nested patterns
927 and/or balanced expressions, see the so-called
928 (?PARNO)
929 construct (available since perl 5.10).
930 The CPAN module Regexp::Common can help to build such
931 regular expressions (see in particular
932 Regexp::Common::balanced and Regexp::Common::delimited).
933 .PP
934 More complex cases will require to write a parser, probably
935 using a parsing module from CPAN, like
936 Regexp::Grammars, Parse::RecDescent, Parse::Yapp,
937 Text::Balanced, or Marpa::R2.
938 .SS "How do I reverse a string?"
939 .IX Subsection "How do I reverse a string?"
940 Use \f(CWreverse()\fR in scalar context, as documented in
941 "reverse" in perlfunc.
942 .PP
943 .Vb 1
944 \& my $reversed = reverse $string;
945 .Ve
946 .SS "How do I expand tabs in a string?"
947 .IX Subsection "How do I expand tabs in a string?"
948 You can do it yourself:
949 .PP
950 .Vb 1
951 \& 1 while $string =~ s/\et+/\*(Aq \*(Aq x (length($&) * 8 \- length($\`) % 8)/e;
952 .Ve
953 .PP
954 Or you can just use the Text::Tabs module (part of the standard Perl
955 distribution).
956 .PP
957 .Vb 2
958 \& use Text::Tabs;
959 \& my @expanded_lines = expand(@lines_with_tabs);
960 .Ve
961 .SS "How do I reformat a paragraph?"
962 .IX Subsection "How do I reformat a paragraph?"
963 Use Text::Wrap (part of the standard Perl distribution):
964 .PP
965 .Vb 2
966 \& use Text::Wrap;
967 \& print wrap("\et", \*(Aq \*(Aq, @paragraphs);
968 .Ve
969 .PP
970 The paragraphs you give to Text::Wrap should not contain embedded
971 newlines. Text::Wrap doesn't justify the lines (flush-right).
972 .PP
973 Or use the CPAN module Text::Autoformat. Formatting files can be
974 easily done by making a shell alias, like so:
975 .PP
976 .Vb 2
977 \& alias fmt="perl \-i \-MText::Autoformat \-n0777 \e
978 \& \-e \*(Aqprint autoformat $_, {all=>1}\*(Aq $*"
979 .Ve
980 .PP
981 See the documentation for Text::Autoformat to appreciate its many
982 capabilities.
983 .SS "How can I access or change N characters of a string?"
984 .IX Subsection "How can I access or change N characters of a string?"
985 You can access the first characters of a string with \fBsubstr()\fR.
986 To get the first character, for example, start at position 0
987 and grab the string of length 1.
988 .PP
989 .Vb 2
990 \& my $string = "Just another Perl Hacker";
991 \& my $first_char = substr( $string, 0, 1 ); # \*(AqJ\*(Aq
992 .Ve
993 .PP
994 To change part of a string, you can use the optional fourth
995 argument which is the replacement string.
996 .PP
997 .Vb 1
998 \& substr( $string, 13, 4, "Perl 5.8.0" );
999 .Ve
1000 .PP
1001 You can also use \fBsubstr()\fR as an lvalue.
1002 .PP
1003 .Vb 1
1004 \& substr( $string, 13, 4 ) = "Perl 5.8.0";
1005 .Ve
1006 .SS "How do I change the Nth occurrence of something?"
1007 .IX Subsection "How do I change the Nth occurrence of something?"
1008 You have to keep track of N yourself. For example, let's say you want
1009 to change the fifth occurrence of \f(CW"whoever"\fR or \f(CW"whomever"\fR into
1010 \&\f(CW"whosoever"\fR or \f(CW"whomsoever"\fR, case insensitively. These
1011 all assume that \f(CW$_\fR contains the string to be altered.
1012 .PP
1013 .Vb 6
1014 \& $count = 0;
1015 \& s{((whom?)ever)}{
1016 \& ++$count == 5 # is it the 5th?
1017 \& ? "${2}soever" # yes, swap
1018 \& : $1 # renege and leave it there
1019 \& }ige;
1020 .Ve
1021 .PP
1022 In the more general case, you can use the \f(CW\*(C`/g\*(C'\fR modifier in a \f(CW\*(C`while\*(C'\fR
1023 loop, keeping count of matches.
1024 .PP
1025 .Vb 8
1026 \& $WANT = 3;
1027 \& $count = 0;
1028 \& $_ = "One fish two fish red fish blue fish";
1029 \& while (/(\ew+)\es+fish\eb/gi) {
1030 \& if (++$count == $WANT) {
1031 \& print "The third fish is a $1 one.\en";
1032 \& }
1033 \& }
1034 .Ve
1035 .PP
1036 That prints out: \f(CW"The third fish is a red one."\fR You can also use a
1037 repetition count and repeated pattern like this:
1038 .PP
1039 .Vb 1
1040 \& /(?:\ew+\es+fish\es+){2}(\ew+)\es+fish/i;
1041 .Ve
1042 .SS "How can I count the number of occurrences of a substring within a string?"
1043 .IX Subsection "How can I count the number of occurrences of a substring within a string?"
1044 There are a number of ways, with varying efficiency. If you want a
1045 count of a certain single character (X) within a string, you can use the
1046 \&\f(CW\*(C`tr///\*(C'\fR function like so:
1047 .PP
1048 .Vb 3
1049 \& my $string = "ThisXlineXhasXsomeXx\*(AqsXinXit";
1050 \& my $count = ($string =~ tr/X//);
1051 \& print "There are $count X characters in the string";
1052 .Ve
1053 .PP
1054 This is fine if you are just looking for a single character. However,
1055 if you are trying to count multiple character substrings within a
1056 larger string, \f(CW\*(C`tr///\*(C'\fR won't work. What you can do is wrap a \fBwhile()\fR
1057 loop around a global pattern match. For example, let's count negative
1058 integers:
1059 .PP
1060 .Vb 4
1061 \& my $string = "\-9 55 48 \-2 23 \-76 4 14 \-44";
1062 \& my $count = 0;
1063 \& while ($string =~ /\-\ed+/g) { $count++ }
1064 \& print "There are $count negative numbers in the string";
1065 .Ve
1066 .PP
1067 Another version uses a global match in list context, then assigns the
1068 result to a scalar, producing a count of the number of matches.
1069 .PP
1070 .Vb 1
1071 \& my $count = () = $string =~ /\-\ed+/g;
1072 .Ve
1073 .SS "How do I capitalize all the words on one line?"
1074 .IX Xref "Text::Autoformat capitalize case, title case, sentence"
1075 .IX Subsection "How do I capitalize all the words on one line?"
1076 (contributed by brian d foy)
1077 .PP
1078 Damian Conway's Text::Autoformat handles all of the thinking
1079 for you:
1080 .PP
1081 .Vb 2
1082 \& use Text::Autoformat;
1083 \& my $s = "Going to the desert with a camel";
1084 \&
1085 \& printf "%\-10s %s\en", "original:", $s;
1086 \& for my $style (qw( sentence title highlight )) {
1087 \& printf "%\-10s %s\en",
1088 \& "$style:",
1089 \& autoformat($s, { case => $style }) =~ s/\eR+\ez//r;
1090 \& }
1091 .Ve
1092 .PP
1093 Each style has a different idea of what if should do with small words:
1094 .PP
1095 .Vb 4
1096 \& original: Going to the desert with a camel
1097 \& sentence: Going to the desert with a camel
1098 \& title: Going To The Desert With A Camel
1099 \& highlight: Going to the Desert with a Camel
1100 .Ve
1101 .PP
1102 Trying this yourself in a simple regex has more than a few pitfalls.
1103 Perl "words" are groups of \f(CW\*(C`\ew+\*(C'\fR, but that's not what you want to
1104 capitalize. Some of those characters aren't even letters. What if you
1105 used a word boundary, \f(CW\*(C`\eb\*(C'\fR?
1106 .PP
1107 .Vb 2
1108 \& my $string = "A camel\*(Aqs journey";
1109 \& $string =~ s/\eb(\ew)/\eU$1/g; # A Camel\*(AqS Journey
1110 .Ve
1111 .PP
1112 How is Perl supposed to know not to capitalize that \f(CW\*(C`s\*(C'\fR? You could work
1113 harder to look for preceding whitespace or the beginning of the string:
1114 .PP
1115 .Vb 1
1116 \& $string =~ s/( (?:^|\es) \ew ) /\eU$1/xg; # A Camel\*(Aqs Journey
1117 .Ve
1118 .PP
1119 Or maybe you match the entire group, lowercase it all (\f(CW\*(C`L\*(C'\fR), then
1120 uppercase the first letter (\f(CW\*(C`\eu\*(C'\fR):
1121 .PP
1122 .Vb 1
1123 \& $string =~ s/([\ew\*(Aq]+)/\eu\eL$1/g;
1124 .Ve
1125 .PP
1126 Each of these have their annoying edge cases. Perl v5.22 added fancier
1127 Unicode word and sentence boundaries to fix this sort of problem. The
1128 \&\f(CW\*(C`\eb{wb}\*(C'\fR word boundary ignores some things that it doesn't think start
1129 or end "human" words:
1130 .PP
1131 .Vb 3
1132 \& use v5.22;
1133 \& my $string = "A camel\*(Aqs journey";
1134 \& $string =~ s/\eb{wb}(\ew)/\eU$1/g; # A Camel\*(Aqs Journey
1135 .Ve
1136 .PP
1137 The details are in Unicode Technical Report #29
1138 (<https://unicode.org/reports/tr29/>), and you should make sure that
1139 its idea of a word boundary corresponds to yours.
1140 .PP
1141 Now, what if you don't want to capitalize that "and" or "the"? Just use
1142 Text::Autoformat and get on with the next problem. :)
1143 .SS "How can I split a [character]\-delimited string except when inside [character]?"
1144 .IX Subsection "How can I split a [character]-delimited string except when inside [character]?"
1145 Several modules can handle this sort of parsing\-\-Text::Balanced,
1146 Text::CSV, Text::CSV_XS, and Text::ParseWords, among others.
1147 .PP
1148 Take the example case of trying to split a string that is
1149 comma-separated into its different fields. You can't use \f(CW\*(C`split(/,/)\*(C'\fR
1150 because you shouldn't split if the comma is inside quotes. For
1151 example, take a data line like this:
1152 .PP
1153 .Vb 1
1154 \& SAR001,"","Cimetrix, Inc","Bob Smith","CAM",N,8,1,0,7,"Error, Core Dumped"
1155 .Ve
1156 .PP
1157 Due to the restriction of the quotes, this is a fairly complex
1158 problem. Thankfully, we have Jeffrey Friedl, author of
1159 \&\fIMastering Regular Expressions\fR, to handle these for us. He
1160 suggests (assuming your string is contained in \f(CW$text\fR):
1161 .PP
1162 .Vb 7
1163 \& my @new = ();
1164 \& push(@new, $+) while $text =~ m{
1165 \& "([^\e"\e\e]*(?:\e\e.[^\e"\e\e]*)*)",? # groups the phrase inside the quotes
1166 \& | ([^,]+),?
1167 \& | ,
1168 \& }gx;
1169 \& push(@new, undef) if substr($text,\-1,1) eq \*(Aq,\*(Aq;
1170 .Ve
1171 .PP
1172 If you want to represent quotation marks inside a
1173 quotation-mark-delimited field, escape them with backslashes (eg,
1174 \&\f(CW"like \e"this\e""\fR.
1175 .PP
1176 Alternatively, the Text::ParseWords module (part of the standard
1177 Perl distribution) lets you say:
1178 .PP
1179 .Vb 2
1180 \& use Text::ParseWords;
1181 \& @new = quotewords(",", 0, $text);
1182 .Ve
1183 .PP
1184 For parsing or generating CSV, though, using Text::CSV rather than
1185 implementing it yourself is highly recommended; you'll save yourself odd bugs
1186 popping up later by just using code which has already been tried and tested in
1187 production for years.
1188 .SS "How do I strip blank space from the beginning/end of a string?"
1189 .IX Subsection "How do I strip blank space from the beginning/end of a string?"
1190 (contributed by brian d foy)
1191 .PP
1192 A substitution can do this for you. For a single line, you want to
1193 replace all the leading or trailing whitespace with nothing. You
1194 can do that with a pair of substitutions:
1195 .PP
1196 .Vb 2
1197 \& s/^\es+//;
1198 \& s/\es+$//;
1199 .Ve
1200 .PP
1201 You can also write that as a single substitution, although it turns
1202 out the combined statement is slower than the separate ones. That
1203 might not matter to you, though:
1204 .PP
1205 .Vb 1
1206 \& s/^\es+|\es+$//g;
1207 .Ve
1208 .PP
1209 In this regular expression, the alternation matches either at the
1210 beginning or the end of the string since the anchors have a lower
1211 precedence than the alternation. With the \f(CW\*(C`/g\*(C'\fR flag, the substitution
1212 makes all possible matches, so it gets both. Remember, the trailing
1213 newline matches the \f(CW\*(C`\es+\*(C'\fR, and the \f(CW\*(C`$\*(C'\fR anchor can match to the
1214 absolute end of the string, so the newline disappears too. Just add
1215 the newline to the output, which has the added benefit of preserving
1216 "blank" (consisting entirely of whitespace) lines which the \f(CW\*(C`^\es+\*(C'\fR
1217 would remove all by itself:
1218 .PP
1219 .Vb 4
1220 \& while( <> ) {
1221 \& s/^\es+|\es+$//g;
1222 \& print "$_\en";
1223 \& }
1224 .Ve
1225 .PP
1226 For a multi-line string, you can apply the regular expression to each
1227 logical line in the string by adding the \f(CW\*(C`/m\*(C'\fR flag (for
1228 "multi-line"). With the \f(CW\*(C`/m\*(C'\fR flag, the \f(CW\*(C`$\*(C'\fR matches \fIbefore\fR an
1229 embedded newline, so it doesn't remove it. This pattern still removes
1230 the newline at the end of the string:
1231 .PP
1232 .Vb 1
1233 \& $string =~ s/^\es+|\es+$//gm;
1234 .Ve
1235 .PP
1236 Remember that lines consisting entirely of whitespace will disappear,
1237 since the first part of the alternation can match the entire string
1238 and replace it with nothing. If you need to keep embedded blank lines,
1239 you have to do a little more work. Instead of matching any whitespace
1240 (since that includes a newline), just match the other whitespace:
1241 .PP
1242 .Vb 1
1243 \& $string =~ s/^[\et\ef ]+|[\et\ef ]+$//mg;
1244 .Ve
1245 .SS "How do I pad a string with blanks or pad a number with zeroes?"
1246 .IX Subsection "How do I pad a string with blanks or pad a number with zeroes?"
1247 In the following examples, \f(CW$pad_len\fR is the length to which you wish
1248 to pad the string, \f(CW$text\fR or \f(CW$num\fR contains the string to be padded,
1249 and \f(CW$pad_char\fR contains the padding character. You can use a single
1250 character string constant instead of the \f(CW$pad_char\fR variable if you
1251 know what it is in advance. And in the same way you can use an integer in
1252 place of \f(CW$pad_len\fR if you know the pad length in advance.
1253 .PP
1254 The simplest method uses the \f(CW\*(C`sprintf\*(C'\fR function. It can pad on the left
1255 or right with blanks and on the left with zeroes and it will not
1256 truncate the result. The \f(CW\*(C`pack\*(C'\fR function can only pad strings on the
1257 right with blanks and it will truncate the result to a maximum length of
1258 \&\f(CW$pad_len\fR.
1259 .PP
1260 .Vb 3
1261 \& # Left padding a string with blanks (no truncation):
1262 \& my $padded = sprintf("%${pad_len}s", $text);
1263 \& my $padded = sprintf("%*s", $pad_len, $text); # same thing
1264 \&
1265 \& # Right padding a string with blanks (no truncation):
1266 \& my $padded = sprintf("%\-${pad_len}s", $text);
1267 \& my $padded = sprintf("%\-*s", $pad_len, $text); # same thing
1268 \&
1269 \& # Left padding a number with 0 (no truncation):
1270 \& my $padded = sprintf("%0${pad_len}d", $num);
1271 \& my $padded = sprintf("%0*d", $pad_len, $num); # same thing
1272 \&
1273 \& # Right padding a string with blanks using pack (will truncate):
1274 \& my $padded = pack("A$pad_len",$text);
1275 .Ve
1276 .PP
1277 If you need to pad with a character other than blank or zero you can use
1278 one of the following methods. They all generate a pad string with the
1279 \&\f(CW\*(C`x\*(C'\fR operator and combine that with \f(CW$text\fR. These methods do
1280 not truncate \f(CW$text\fR.
1281 .PP
1282 Left and right padding with any character, creating a new string:
1283 .PP
1284 .Vb 2
1285 \& my $padded = $pad_char x ( $pad_len \- length( $text ) ) . $text;
1286 \& my $padded = $text . $pad_char x ( $pad_len \- length( $text ) );
1287 .Ve
1288 .PP
1289 Left and right padding with any character, modifying \f(CW$text\fR directly:
1290 .PP
1291 .Vb 2
1292 \& substr( $text, 0, 0 ) = $pad_char x ( $pad_len \- length( $text ) );
1293 \& $text .= $pad_char x ( $pad_len \- length( $text ) );
1294 .Ve
1295 .SS "How do I extract selected columns from a string?"
1296 .IX Subsection "How do I extract selected columns from a string?"
1297 (contributed by brian d foy)
1298 .PP
1299 If you know the columns that contain the data, you can
1300 use \f(CW\*(C`substr\*(C'\fR to extract a single column.
1301 .PP
1302 .Vb 1
1303 \& my $column = substr( $line, $start_column, $length );
1304 .Ve
1305 .PP
1306 You can use \f(CW\*(C`split\*(C'\fR if the columns are separated by whitespace or
1307 some other delimiter, as long as whitespace or the delimiter cannot
1308 appear as part of the data.
1309 .PP
1310 .Vb 3
1311 \& my $line = \*(Aq fred barney betty \*(Aq;
1312 \& my @columns = split /\es+/, $line;
1313 \& # ( \*(Aq\*(Aq, \*(Aqfred\*(Aq, \*(Aqbarney\*(Aq, \*(Aqbetty\*(Aq );
1314 \&
1315 \& my $line = \*(Aqfred||barney||betty\*(Aq;
1316 \& my @columns = split /\e|/, $line;
1317 \& # ( \*(Aqfred\*(Aq, \*(Aq\*(Aq, \*(Aqbarney\*(Aq, \*(Aq\*(Aq, \*(Aqbetty\*(Aq );
1318 .Ve
1319 .PP
1320 If you want to work with comma-separated values, don't do this since
1321 that format is a bit more complicated. Use one of the modules that
1322 handle that format, such as Text::CSV, Text::CSV_XS, or
1323 Text::CSV_PP.
1324 .PP
1325 If you want to break apart an entire line of fixed columns, you can use
1326 \&\f(CW\*(C`unpack\*(C'\fR with the A (ASCII) format. By using a number after the format
1327 specifier, you can denote the column width. See the \f(CW\*(C`pack\*(C'\fR and \f(CW\*(C`unpack\*(C'\fR
1328 entries in perlfunc for more details.
1329 .PP
1330 .Vb 1
1331 \& my @fields = unpack( $line, "A8 A8 A8 A16 A4" );
1332 .Ve
1333 .PP
1334 Note that spaces in the format argument to \f(CW\*(C`unpack\*(C'\fR do not denote literal
1335 spaces. If you have space separated data, you may want \f(CW\*(C`split\*(C'\fR instead.
1336 .SS "How do I find the soundex value of a string?"
1337 .IX Subsection "How do I find the soundex value of a string?"
1338 (contributed by brian d foy)
1339 .PP
1340 You can use the \f(CW\*(C`Text::Soundex\*(C'\fR module. If you want to do fuzzy or close
1341 matching, you might also try the String::Approx, and
1342 Text::Metaphone, and Text::DoubleMetaphone modules.
1343 .SS "How can I expand variables in text strings?"
1344 .IX Subsection "How can I expand variables in text strings?"
1345 (contributed by brian d foy)
1346 .PP
1347 If you can avoid it, don't, or if you can use a templating system,
1348 such as Text::Template or Template Toolkit, do that instead. You
1349 might even be able to get the job done with \f(CW\*(C`sprintf\*(C'\fR or \f(CW\*(C`printf\*(C'\fR:
1350 .PP
1351 .Vb 1
1352 \& my $string = sprintf \*(AqSay hello to %s and %s\*(Aq, $foo, $bar;
1353 .Ve
1354 .PP
1355 However, for the one-off simple case where I don't want to pull out a
1356 full templating system, I'll use a string that has two Perl scalar
1357 variables in it. In this example, I want to expand \f(CW$foo\fR and \f(CW$bar\fR
1358 to their variable's values:
1359 .PP
1360 .Vb 3
1361 \& my $foo = \*(AqFred\*(Aq;
1362 \& my $bar = \*(AqBarney\*(Aq;
1363 \& $string = \*(AqSay hello to $foo and $bar\*(Aq;
1364 .Ve
1365 .PP
1366 One way I can do this involves the substitution operator and a double
1367 \&\f(CW\*(C`/e\*(C'\fR flag. The first \f(CW\*(C`/e\*(C'\fR evaluates \f(CW$1\fR on the replacement side and
1368 turns it into \f(CW$foo\fR. The second /e starts with \f(CW$foo\fR and replaces
1369 it with its value. \f(CW$foo\fR, then, turns into 'Fred', and that's finally
1370 what's left in the string:
1371 .PP
1372 .Vb 1
1373 \& $string =~ s/(\e$\ew+)/$1/eeg; # \*(AqSay hello to Fred and Barney\*(Aq
1374 .Ve
1375 .PP
1376 The \f(CW\*(C`/e\*(C'\fR will also silently ignore violations of strict, replacing
1377 undefined variable names with the empty string. Since I'm using the
1378 \&\f(CW\*(C`/e\*(C'\fR flag (twice even!), I have all of the same security problems I
1379 have with \f(CW\*(C`eval\*(C'\fR in its string form. If there's something odd in
1380 \&\f(CW$foo\fR, perhaps something like \f(CW\*(C`@{[ system "rm \-rf /" ]}\*(C'\fR, then
1381 I could get myself in trouble.
1382 .PP
1383 To get around the security problem, I could also pull the values from
1384 a hash instead of evaluating variable names. Using a single \f(CW\*(C`/e\*(C'\fR, I
1385 can check the hash to ensure the value exists, and if it doesn't, I
1386 can replace the missing value with a marker, in this case \f(CW\*(C`???\*(C'\fR to
1387 signal that I missed something:
1388 .PP
1389 .Vb 1
1390 \& my $string = \*(AqThis has $foo and $bar\*(Aq;
1391 \&
1392 \& my %Replacements = (
1393 \& foo => \*(AqFred\*(Aq,
1394 \& );
1395 \&
1396 \& # $string =~ s/\e$(\ew+)/$Replacements{$1}/g;
1397 \& $string =~ s/\e$(\ew+)/
1398 \& exists $Replacements{$1} ? $Replacements{$1} : \*(Aq???\*(Aq
1399 \& /eg;
1400 \&
1401 \& print $string;
1402 .Ve
1403 .SS "Does Perl have anything like Ruby's #{} or Python's f string?"
1404 .IX Subsection "Does Perl have anything like Ruby's #{} or Python's f string?"
1405 Unlike the others, Perl allows you to embed a variable naked in a double
1406 quoted string, e.g. \f(CW"variable $variable"\fR. When there isn't whitespace or
1407 other non-word characters following the variable name, you can add braces
1408 (e.g. \f(CW"foo ${foo}bar"\fR) to ensure correct parsing.
1409 .PP
1410 An array can also be embedded directly in a string, and will be expanded
1411 by default with spaces between the elements. The default
1412 LIST_SEPARATOR can be changed by assigning a
1413 different string to the special variable \f(CW$"\fR, such as \f(CW\*(C`local $" = \*(Aq, \*(Aq;\*(C'\fR.
1414 .PP
1415 Perl also supports references within a string providing the equivalent of
1416 the features in the other two languages.
1417 .PP
1418 \&\f(CW\*(C`${\e ... }\*(C'\fR embedded within a string will work for most simple statements
1419 such as an object\->method call. More complex code can be wrapped in a do
1420 block \f(CW\*(C`${\e do{...} }\*(C'\fR.
1421 .PP
1422 When you want a list to be expanded per \f(CW$"\fR, use \f(CW\*(C`@{[ ... ]}\*(C'\fR.
1423 .PP
1424 .Vb 4
1425 \& use Time::Piece;
1426 \& use Time::Seconds;
1427 \& my $scalar = \*(AqSTRING\*(Aq;
1428 \& my @array = ( \*(Aqzorro\*(Aq, \*(Aqa\*(Aq, 1, \*(AqB\*(Aq, 3 );
1429 \&
1430 \& # Print the current date and time and then Tommorrow
1431 \& my $t = Time::Piece\->new;
1432 \& say "Now is: ${\e $t\->cdate() }";
1433 \& say "Tomorrow: ${\e do{ my $T=Time::Piece\->new + ONE_DAY ; $T\->fullday }}";
1434 \&
1435 \& # some variables in strings
1436 \& say "This is some scalar I have $scalar, this is an array @array.";
1437 \& say "You can also write it like this ${scalar} @{array}.";
1438 \&
1439 \& # Change the $LIST_SEPARATOR
1440 \& local $" = \*(Aq:\*(Aq;
1441 \& say "Set \e$\e" to delimit with \*(Aq:\*(Aq and sort the Array @{[ sort @array ]}";
1442 .Ve
1443 .PP
1444 You may also want to look at the module
1445 Quote::Code, and templating tools such as Template::Toolkit and
1446 Mojo::Template.
1447 .PP
1448 See also: "How can I expand variables in text strings?" and
1449 "How do I expand function calls in a string?" in this FAQ.
1450 .SS "What's wrong with always quoting ""$vars""?"
1451 .IX Subsection "What's wrong with always quoting ""$vars""?"
1452 The problem is that those double-quotes force
1453 stringification\-\-coercing numbers and references into strings\-\-even
1454 when you don't want them to be strings. Think of it this way:
1455 double-quote expansion is used to produce new strings. If you already
1456 have a string, why do you need more?
1457 .PP
1458 If you get used to writing odd things like these:
1459 .PP
1460 .Vb 3
1461 \& print "$var"; # BAD
1462 \& my $new = "$old"; # BAD
1463 \& somefunc("$var"); # BAD
1464 .Ve
1465 .PP
1466 You'll be in trouble. Those should (in 99.8% of the cases) be
1467 the simpler and more direct:
1468 .PP
1469 .Vb 3
1470 \& print $var;
1471 \& my $new = $old;
1472 \& somefunc($var);
1473 .Ve
1474 .PP
1475 Otherwise, besides slowing you down, you're going to break code when
1476 the thing in the scalar is actually neither a string nor a number, but
1477 a reference:
1478 .PP
1479 .Vb 5
1480 \& func(\e@array);
1481 \& sub func {
1482 \& my $aref = shift;
1483 \& my $oref = "$aref"; # WRONG
1484 \& }
1485 .Ve
1486 .PP
1487 You can also get into subtle problems on those few operations in Perl
1488 that actually do care about the difference between a string and a
1489 number, such as the magical \f(CW\*(C`++\*(C'\fR autoincrement operator or the
1490 \&\fBsyscall()\fR function.
1491 .PP
1492 Stringification also destroys arrays.
1493 .PP
1494 .Vb 3
1495 \& my @lines = \`command\`;
1496 \& print "@lines"; # WRONG \- extra blanks
1497 \& print @lines; # right
1498 .Ve
1499 .SS "Why don't my <<HERE documents work?"
1500 .IX Subsection "Why don't my <<HERE documents work?"
1501 Here documents are found in perlop. Check for these three things:
1502 .IP "There must be no space after the << part." 4
1503 .IX Item "There must be no space after the << part."
1504 .PD 0
1505 .IP "There (probably) should be a semicolon at the end of the opening token" 4
1506 .IX Item "There (probably) should be a semicolon at the end of the opening token"
1507 .IP "You can't (easily) have any space in front of the tag." 4
1508 .IX Item "You can't (easily) have any space in front of the tag."
1509 .IP "There needs to be at least a line separator after the end token." 4
1510 .IX Item "There needs to be at least a line separator after the end token."
1511 .PD
1512 .PP
1513 If you want to indent the text in the here document, you
1514 can do this:
1515 .PP
1516 .Vb 5
1517 \& # all in one
1518 \& (my $VAR = <<HERE_TARGET) =~ s/^\es+//gm;
1519 \& your text
1520 \& goes here
1521 \& HERE_TARGET
1522 .Ve
1523 .PP
1524 But the HERE_TARGET must still be flush against the margin.
1525 If you want that indented also, you'll have to quote
1526 in the indentation.
1527 .PP
1528 .Vb 7
1529 \& (my $quote = <<\*(Aq FINIS\*(Aq) =~ s/^\es+//gm;
1530 \& ...we will have peace, when you and all your works have
1531 \& perished\-\-and the works of your dark master to whom you
1532 \& would deliver us. You are a liar, Saruman, and a corrupter
1533 \& of men\*(Aqs hearts. \-\-Theoden in /usr/src/perl/taint.c
1534 \& FINIS
1535 \& $quote =~ s/\es+\-\-/\en\-\-/;
1536 .Ve
1537 .PP
1538 A nice general-purpose fixer-upper function for indented here documents
1539 follows. It expects to be called with a here document as its argument.
1540 It looks to see whether each line begins with a common substring, and
1541 if so, strips that substring off. Otherwise, it takes the amount of leading
1542 whitespace found on the first line and removes that much off each
1543 subsequent line.
1544 .PP
1545 .Vb 11
1546 \& sub fix {
1547 \& local $_ = shift;
1548 \& my ($white, $leader); # common whitespace and common leading string
1549 \& if (/^\es*(?:([^\ew\es]+)(\es*).*\en)(?:\es*\eg1\eg2?.*\en)+$/) {
1550 \& ($white, $leader) = ($2, quotemeta($1));
1551 \& } else {
1552 \& ($white, $leader) = (/^(\es+)/, \*(Aq\*(Aq);
1553 \& }
1554 \& s/^\es*?$leader(?:$white)?//gm;
1555 \& return $_;
1556 \& }
1557 .Ve
1558 .PP
1559 This works with leading special strings, dynamically determined:
1560 .PP
1561 .Vb 10
1562 \& my $remember_the_main = fix<<\*(Aq MAIN_INTERPRETER_LOOP\*(Aq;
1563 \& @@@ int
1564 \& @@@ runops() {
1565 \& @@@ SAVEI32(runlevel);
1566 \& @@@ runlevel++;
1567 \& @@@ while ( op = (*op\->op_ppaddr)() );
1568 \& @@@ TAINT_NOT;
1569 \& @@@ return 0;
1570 \& @@@ }
1571 \& MAIN_INTERPRETER_LOOP
1572 .Ve
1573 .PP
1574 Or with a fixed amount of leading whitespace, with remaining
1575 indentation correctly preserved:
1576 .PP
1577 .Vb 9
1578 \& my $poem = fix<<EVER_ON_AND_ON;
1579 \& Now far ahead the Road has gone,
1580 \& And I must follow, if I can,
1581 \& Pursuing it with eager feet,
1582 \& Until it joins some larger way
1583 \& Where many paths and errands meet.
1584 \& And whither then? I cannot say.
1585 \& \-\-Bilbo in /usr/src/perl/pp_ctl.c
1586 \& EVER_ON_AND_ON
1587 .Ve
1588 .PP
1589 Beginning with Perl version 5.26, a much simpler and cleaner way to
1590 write indented here documents has been added to the language: the
1591 tilde (~) modifier. See "Indented Here-docs" in perlop for details.
1592 .SH "Data: Arrays"
1593 .IX Header "Data: Arrays"
1594 .SS "What is the difference between a list and an array?"
1595 .IX Subsection "What is the difference between a list and an array?"
1596 (contributed by brian d foy)
1597 .PP
1598 A list is a fixed collection of scalars. An array is a variable that
1599 holds a variable collection of scalars. An array can supply its collection
1600 for list operations, so list operations also work on arrays:
1601 .PP
1602 .Vb 3
1603 \& # slices
1604 \& ( \*(Aqdog\*(Aq, \*(Aqcat\*(Aq, \*(Aqbird\*(Aq )[2,3];
1605 \& @animals[2,3];
1606 \&
1607 \& # iteration
1608 \& foreach ( qw( dog cat bird ) ) { ... }
1609 \& foreach ( @animals ) { ... }
1610 \&
1611 \& my @three = grep { length == 3 } qw( dog cat bird );
1612 \& my @three = grep { length == 3 } @animals;
1613 \&
1614 \& # supply an argument list
1615 \& wash_animals( qw( dog cat bird ) );
1616 \& wash_animals( @animals );
1617 .Ve
1618 .PP
1619 Array operations, which change the scalars, rearrange them, or add
1620 or subtract some scalars, only work on arrays. These can't work on a
1621 list, which is fixed. Array operations include \f(CW\*(C`shift\*(C'\fR, \f(CW\*(C`unshift\*(C'\fR,
1622 \&\f(CW\*(C`push\*(C'\fR, \f(CW\*(C`pop\*(C'\fR, and \f(CW\*(C`splice\*(C'\fR.
1623 .PP
1624 An array can also change its length:
1625 .PP
1626 .Vb 2
1627 \& $#animals = 1; # truncate to two elements
1628 \& $#animals = 10000; # pre\-extend to 10,001 elements
1629 .Ve
1630 .PP
1631 You can change an array element, but you can't change a list element:
1632 .PP
1633 .Vb 2
1634 \& $animals[0] = \*(AqRottweiler\*(Aq;
1635 \& qw( dog cat bird )[0] = \*(AqRottweiler\*(Aq; # syntax error!
1636 \&
1637 \& foreach ( @animals ) {
1638 \& s/^d/fr/; # works fine
1639 \& }
1640 \&
1641 \& foreach ( qw( dog cat bird ) ) {
1642 \& s/^d/fr/; # Error! Modification of read only value!
1643 \& }
1644 .Ve
1645 .PP
1646 However, if the list element is itself a variable, it appears that you
1647 can change a list element. However, the list element is the variable, not
1648 the data. You're not changing the list element, but something the list
1649 element refers to. The list element itself doesn't change: it's still
1650 the same variable.
1651 .PP
1652 You also have to be careful about context. You can assign an array to
1653 a scalar to get the number of elements in the array. This only works
1654 for arrays, though:
1655 .PP
1656 .Vb 1
1657 \& my $count = @animals; # only works with arrays
1658 .Ve
1659 .PP
1660 If you try to do the same thing with what you think is a list, you
1661 get a quite different result. Although it looks like you have a list
1662 on the righthand side, Perl actually sees a bunch of scalars separated
1663 by a comma:
1664 .PP
1665 .Vb 1
1666 \& my $scalar = ( \*(Aqdog\*(Aq, \*(Aqcat\*(Aq, \*(Aqbird\*(Aq ); # $scalar gets bird
1667 .Ve
1668 .PP
1669 Since you're assigning to a scalar, the righthand side is in scalar
1670 context. The comma operator (yes, it's an operator!) in scalar
1671 context evaluates its lefthand side, throws away the result, and
1672 evaluates its righthand side and returns the result. In effect,
1673 that list-lookalike assigns to \f(CW$scalar\fR its rightmost value. Many
1674 people mess this up because they choose a list-lookalike whose
1675 last element is also the count they expect:
1676 .PP
1677 .Vb 1
1678 \& my $scalar = ( 1, 2, 3 ); # $scalar gets 3, accidentally
1679 .Ve
1680 .ie n .SS "What is the difference between $array[1] and @array[1]?"
1681 .el .SS "What is the difference between \f(CW$array\fP[1] and \f(CW@array\fP[1]?"
1682 .IX Subsection "What is the difference between $array[1] and @array[1]?"
1683 (contributed by brian d foy)
1684 .PP
1685 The difference is the sigil, that special character in front of the
1686 array name. The \f(CW\*(C`$\*(C'\fR sigil means "exactly one item", while the \f(CW\*(C`@\*(C'\fR
1687 sigil means "zero or more items". The \f(CW\*(C`$\*(C'\fR gets you a single scalar,
1688 while the \f(CW\*(C`@\*(C'\fR gets you a list.
1689 .PP
1690 The confusion arises because people incorrectly assume that the sigil
1691 denotes the variable type.
1692 .PP
1693 The \f(CW$array[1]\fR is a single-element access to the array. It's going
1694 to return the item in index 1 (or undef if there is no item there).
1695 If you intend to get exactly one element from the array, this is the
1696 form you should use.
1697 .PP
1698 The \f(CW@array[1]\fR is an array slice, although it has only one index.
1699 You can pull out multiple elements simultaneously by specifying
1700 additional indices as a list, like \f(CW@array[1,4,3,0]\fR.
1701 .PP
1702 Using a slice on the lefthand side of the assignment supplies list
1703 context to the righthand side. This can lead to unexpected results.
1704 For instance, if you want to read a single line from a filehandle,
1705 assigning to a scalar value is fine:
1706 .PP
1707 .Vb 1
1708 \& $array[1] = <STDIN>;
1709 .Ve
1710 .PP
1711 However, in list context, the line input operator returns all of the
1712 lines as a list. The first line goes into \f(CW@array[1]\fR and the rest
1713 of the lines mysteriously disappear:
1714 .PP
1715 .Vb 1
1716 \& @array[1] = <STDIN>; # most likely not what you want
1717 .Ve
1718 .PP
1719 Either the \f(CW\*(C`use warnings\*(C'\fR pragma or the \fB\-w\fR flag will warn you when
1720 you use an array slice with a single index.
1721 .SS "How can I remove duplicate elements from a list or array?"
1722 .IX Subsection "How can I remove duplicate elements from a list or array?"
1723 (contributed by brian d foy)
1724 .PP
1725 Use a hash. When you think the words "unique" or "duplicated", think
1726 "hash keys".
1727 .PP
1728 If you don't care about the order of the elements, you could just
1729 create the hash then extract the keys. It's not important how you
1730 create that hash: just that you use \f(CW\*(C`keys\*(C'\fR to get the unique
1731 elements.
1732 .PP
1733 .Vb 3
1734 \& my %hash = map { $_, 1 } @array;
1735 \& # or a hash slice: @hash{ @array } = ();
1736 \& # or a foreach: $hash{$_} = 1 foreach ( @array );
1737 \&
1738 \& my @unique = keys %hash;
1739 .Ve
1740 .PP
1741 If you want to use a module, try the \f(CW\*(C`uniq\*(C'\fR function from
1742 List::Util. In list context it returns the unique elements,
1743 preserving their order in the list. In scalar context, it returns the
1744 number of unique elements.
1745 .PP
1746 .Vb 1
1747 \& use List::Util qw(uniq);
1748 \&
1749 \& my @unique = uniq( 1, 2, 3, 4, 4, 5, 6, 5, 8 ); # 1,2,3,4,5,6,8
1750 \& my $unique = uniq( 1, 2, 3, 4, 4, 5, 6, 5, 8 ); # 7
1751 .Ve
1752 .PP
1753 You can also go through each element and skip the ones you've seen
1754 before. Use a hash to keep track. The first time the loop sees an
1755 element, that element has no key in \f(CW%seen\fR. The \f(CW\*(C`next\*(C'\fR statement
1756 creates the key and immediately uses its value, which is \f(CW\*(C`undef\*(C'\fR, so
1757 the loop continues to the \f(CW\*(C`push\*(C'\fR and increments the value for that
1758 key. The next time the loop sees that same element, its key exists in
1759 the hash \fIand\fR the value for that key is true (since it's not 0 or
1760 \&\f(CW\*(C`undef\*(C'\fR), so the next skips that iteration and the loop goes to the
1761 next element.
1762 .PP
1763 .Vb 2
1764 \& my @unique = ();
1765 \& my %seen = ();
1766 \&
1767 \& foreach my $elem ( @array ) {
1768 \& next if $seen{ $elem }++;
1769 \& push @unique, $elem;
1770 \& }
1771 .Ve
1772 .PP
1773 You can write this more briefly using a grep, which does the
1774 same thing.
1775 .PP
1776 .Vb 2
1777 \& my %seen = ();
1778 \& my @unique = grep { ! $seen{ $_ }++ } @array;
1779 .Ve
1780 .SS "How can I tell whether a certain element is contained in a list or array?"
1781 .IX Subsection "How can I tell whether a certain element is contained in a list or array?"
1782 (portions of this answer contributed by Anno Siegel and brian d foy)
1783 .PP
1784 Hearing the word "in" is an \fIin\fRdication that you probably should have
1785 used a hash, not a list or array, to store your data. Hashes are
1786 designed to answer this question quickly and efficiently. Arrays aren't.
1787 .PP
1788 That being said, there are several ways to approach this. If you
1789 are going to make this query many times over arbitrary string values,
1790 the fastest way is probably to invert the original array and maintain a
1791 hash whose keys are the first array's values:
1792 .PP
1793 .Vb 3
1794 \& my @blues = qw/azure cerulean teal turquoise lapis\-lazuli/;
1795 \& my %is_blue = ();
1796 \& for (@blues) { $is_blue{$_} = 1 }
1797 .Ve
1798 .PP
1799 Now you can check whether \f(CW$is_blue{$some_color}\fR. It might have
1800 been a good idea to keep the blues all in a hash in the first place.
1801 .PP
1802 If the values are all small integers, you could use a simple indexed
1803 array. This kind of an array will take up less space:
1804 .PP
1805 .Vb 4
1806 \& my @primes = (2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31);
1807 \& my @is_tiny_prime = ();
1808 \& for (@primes) { $is_tiny_prime[$_] = 1 }
1809 \& # or simply @istiny_prime[@primes] = (1) x @primes;
1810 .Ve
1811 .PP
1812 Now you check whether \f(CW$is_tiny_prime\fR[$some_number].
1813 .PP
1814 If the values in question are integers instead of strings, you can save
1815 quite a lot of space by using bit strings instead:
1816 .PP
1817 .Vb 3
1818 \& my @articles = ( 1..10, 150..2000, 2017 );
1819 \& undef $read;
1820 \& for (@articles) { vec($read,$_,1) = 1 }
1821 .Ve
1822 .PP
1823 Now check whether \f(CW\*(C`vec($read,$n,1)\*(C'\fR is true for some \f(CW$n\fR.
1824 .PP
1825 These methods guarantee fast individual tests but require a re-organization
1826 of the original list or array. They only pay off if you have to test
1827 multiple values against the same array.
1828 .PP
1829 If you are testing only once, the standard module List::Util exports
1830 the function \f(CW\*(C`any\*(C'\fR for this purpose. It works by stopping once it
1831 finds the element. It's written in C for speed, and its Perl equivalent
1832 looks like this subroutine:
1833 .PP
1834 .Vb 7
1835 \& sub any (&@) {
1836 \& my $code = shift;
1837 \& foreach (@_) {
1838 \& return 1 if $code\->();
1839 \& }
1840 \& return 0;
1841 \& }
1842 .Ve
1843 .PP
1844 If speed is of little concern, the common idiom uses grep in scalar context
1845 (which returns the number of items that passed its condition) to traverse the
1846 entire list. This does have the benefit of telling you how many matches it
1847 found, though.
1848 .PP
1849 .Vb 1
1850 \& my $is_there = grep $_ eq $whatever, @array;
1851 .Ve
1852 .PP
1853 If you want to actually extract the matching elements, simply use grep in
1854 list context.
1855 .PP
1856 .Vb 1
1857 \& my @matches = grep $_ eq $whatever, @array;
1858 .Ve
1859 .SS "How do I compute the difference of two arrays? How do I compute the intersection of two arrays?"
1860 .IX Subsection "How do I compute the difference of two arrays? How do I compute the intersection of two arrays?"
1861 Use a hash. Here's code to do both and more. It assumes that each
1862 element is unique in a given array:
1863 .PP
1864 .Vb 7
1865 \& my (@union, @intersection, @difference);
1866 \& my %count = ();
1867 \& foreach my $element (@array1, @array2) { $count{$element}++ }
1868 \& foreach my $element (keys %count) {
1869 \& push @union, $element;
1870 \& push @{ $count{$element} > 1 ? \e@intersection : \e@difference }, $element;
1871 \& }
1872 .Ve
1873 .PP
1874 Note that this is the \fIsymmetric difference\fR, that is, all elements
1875 in either A or in B but not in both. Think of it as an xor operation.
1876 .SS "How do I test whether two arrays or hashes are equal?"
1877 .IX Subsection "How do I test whether two arrays or hashes are equal?"
1878 The following code works for single-level arrays. It uses a
1879 stringwise comparison, and does not distinguish defined versus
1880 undefined empty strings. Modify if you have other needs.
1881 .PP
1882 .Vb 1
1883 \& $are_equal = compare_arrays(\e@frogs, \e@toads);
1884 \&
1885 \& sub compare_arrays {
1886 \& my ($first, $second) = @_;
1887 \& no warnings; # silence spurious \-w undef complaints
1888 \& return 0 unless @$first == @$second;
1889 \& for (my $i = 0; $i < @$first; $i++) {
1890 \& return 0 if $first\->[$i] ne $second\->[$i];
1891 \& }
1892 \& return 1;
1893 \& }
1894 .Ve
1895 .PP
1896 For multilevel structures, you may wish to use an approach more
1897 like this one. It uses the CPAN module FreezeThaw:
1898 .PP
1899 .Vb 2
1900 \& use FreezeThaw qw(cmpStr);
1901 \& my @a = my @b = ( "this", "that", [ "more", "stuff" ] );
1902 \&
1903 \& printf "a and b contain %s arrays\en",
1904 \& cmpStr(\e@a, \e@b) == 0
1905 \& ? "the same"
1906 \& : "different";
1907 .Ve
1908 .PP
1909 This approach also works for comparing hashes. Here we'll demonstrate
1910 two different answers:
1911 .PP
1912 .Vb 1
1913 \& use FreezeThaw qw(cmpStr cmpStrHard);
1914 \&
1915 \& my %a = my %b = ( "this" => "that", "extra" => [ "more", "stuff" ] );
1916 \& $a{EXTRA} = \e%b;
1917 \& $b{EXTRA} = \e%a;
1918 \&
1919 \& printf "a and b contain %s hashes\en",
1920 \& cmpStr(\e%a, \e%b) == 0 ? "the same" : "different";
1921 \&
1922 \& printf "a and b contain %s hashes\en",
1923 \& cmpStrHard(\e%a, \e%b) == 0 ? "the same" : "different";
1924 .Ve
1925 .PP
1926 The first reports that both those the hashes contain the same data,
1927 while the second reports that they do not. Which you prefer is left as
1928 an exercise to the reader.
1929 .SS "How do I find the first array element for which a condition is true?"
1930 .IX Subsection "How do I find the first array element for which a condition is true?"
1931 To find the first array element which satisfies a condition, you can
1932 use the \f(CWfirst()\fR function in the List::Util module, which comes
1933 with Perl 5.8. This example finds the first element that contains
1934 "Perl".
1935 .PP
1936 .Vb 1
1937 \& use List::Util qw(first);
1938 \&
1939 \& my $element = first { /Perl/ } @array;
1940 .Ve
1941 .PP
1942 If you cannot use List::Util, you can make your own loop to do the
1943 same thing. Once you find the element, you stop the loop with last.
1944 .PP
1945 .Vb 4
1946 \& my $found;
1947 \& foreach ( @array ) {
1948 \& if( /Perl/ ) { $found = $_; last }
1949 \& }
1950 .Ve
1951 .PP
1952 If you want the array index, use the \f(CWfirstidx()\fR function from
1953 \&\f(CW\*(C`List::MoreUtils\*(C'\fR:
1954 .PP
1955 .Vb 2
1956 \& use List::MoreUtils qw(firstidx);
1957 \& my $index = firstidx { /Perl/ } @array;
1958 .Ve
1959 .PP
1960 Or write it yourself, iterating through the indices
1961 and checking the array element at each index until you find one
1962 that satisfies the condition:
1963 .PP
1964 .Vb 8
1965 \& my( $found, $index ) = ( undef, \-1 );
1966 \& for( $i = 0; $i < @array; $i++ ) {
1967 \& if( $array[$i] =~ /Perl/ ) {
1968 \& $found = $array[$i];
1969 \& $index = $i;
1970 \& last;
1971 \& }
1972 \& }
1973 .Ve
1974 .SS "How do I handle linked lists?"
1975 .IX Subsection "How do I handle linked lists?"
1976 (contributed by brian d foy)
1977 .PP
1978 Perl's arrays do not have a fixed size, so you don't need linked lists
1979 if you just want to add or remove items. You can use array operations
1980 such as \f(CW\*(C`push\*(C'\fR, \f(CW\*(C`pop\*(C'\fR, \f(CW\*(C`shift\*(C'\fR, \f(CW\*(C`unshift\*(C'\fR, or \f(CW\*(C`splice\*(C'\fR to do
1981 that.
1982 .PP
1983 Sometimes, however, linked lists can be useful in situations where you
1984 want to "shard" an array so you have many small arrays instead of
1985 a single big array. You can keep arrays longer than Perl's largest
1986 array index, lock smaller arrays separately in threaded programs,
1987 reallocate less memory, or quickly insert elements in the middle of
1988 the chain.
1989 .PP
1990 Steve Lembark goes through the details in his YAPC::NA 2009 talk "Perly
1991 Linked Lists" ( <http://www.slideshare.net/lembark/perly\-linked\-lists> ),
1992 although you can just use his LinkedList::Single module.
1993 .SS "How do I handle circular lists?"
1994 .IX Xref "circular array Tie::Cycle Array::Iterator::Circular cycle modulus"
1995 .IX Subsection "How do I handle circular lists?"
1996 (contributed by brian d foy)
1997 .PP
1998 If you want to cycle through an array endlessly, you can increment the
1999 index modulo the number of elements in the array:
2000 .PP
2001 .Vb 2
2002 \& my @array = qw( a b c );
2003 \& my $i = 0;
2004 \&
2005 \& while( 1 ) {
2006 \& print $array[ $i++ % @array ], "\en";
2007 \& last if $i > 20;
2008 \& }
2009 .Ve
2010 .PP
2011 You can also use Tie::Cycle to use a scalar that always has the
2012 next element of the circular array:
2013 .PP
2014 .Vb 1
2015 \& use Tie::Cycle;
2016 \&
2017 \& tie my $cycle, \*(AqTie::Cycle\*(Aq, [ qw( FFFFFF 000000 FFFF00 ) ];
2018 \&
2019 \& print $cycle; # FFFFFF
2020 \& print $cycle; # 000000
2021 \& print $cycle; # FFFF00
2022 .Ve
2023 .PP
2024 The Array::Iterator::Circular creates an iterator object for
2025 circular arrays:
2026 .PP
2027 .Vb 1
2028 \& use Array::Iterator::Circular;
2029 \&
2030 \& my $color_iterator = Array::Iterator::Circular\->new(
2031 \& qw(red green blue orange)
2032 \& );
2033 \&
2034 \& foreach ( 1 .. 20 ) {
2035 \& print $color_iterator\->next, "\en";
2036 \& }
2037 .Ve
2038 .SS "How do I shuffle an array randomly?"
2039 .IX Subsection "How do I shuffle an array randomly?"
2040 If you either have Perl 5.8.0 or later installed, or if you have
2041 Scalar-List-Utils 1.03 or later installed, you can say:
2042 .PP
2043 .Vb 1
2044 \& use List::Util \*(Aqshuffle\*(Aq;
2045 \&
2046 \& @shuffled = shuffle(@list);
2047 .Ve
2048 .PP
2049 If not, you can use a Fisher-Yates shuffle.
2050 .PP
2051 .Vb 3
2052 \& sub fisher_yates_shuffle {
2053 \& my $deck = shift; # $deck is a reference to an array
2054 \& return unless @$deck; # must not be empty!
2055 \&
2056 \& my $i = @$deck;
2057 \& while (\-\-$i) {
2058 \& my $j = int rand ($i+1);
2059 \& @$deck[$i,$j] = @$deck[$j,$i];
2060 \& }
2061 \& }
2062 \&
2063 \& # shuffle my mpeg collection
2064 \& #
2065 \& my @mpeg = <audio/*/*.mp3>;
2066 \& fisher_yates_shuffle( \e@mpeg ); # randomize @mpeg in place
2067 \& print @mpeg;
2068 .Ve
2069 .PP
2070 Note that the above implementation shuffles an array in place,
2071 unlike the \f(CWList::Util::shuffle()\fR which takes a list and returns
2072 a new shuffled list.
2073 .PP
2074 You've probably seen shuffling algorithms that work using splice,
2075 randomly picking another element to swap the current element with
2076 .PP
2077 .Vb 6
2078 \& srand;
2079 \& @new = ();
2080 \& @old = 1 .. 10; # just a demo
2081 \& while (@old) {
2082 \& push(@new, splice(@old, rand @old, 1));
2083 \& }
2084 .Ve
2085 .PP
2086 This is bad because splice is already O(N), and since you do it N
2087 times, you just invented a quadratic algorithm; that is, O(N**2).
2088 This does not scale, although Perl is so efficient that you probably
2089 won't notice this until you have rather largish arrays.
2090 .SS "How do I process/modify each element of an array?"
2091 .IX Subsection "How do I process/modify each element of an array?"
2092 Use \f(CW\*(C`for\*(C'\fR/\f(CW\*(C`foreach\*(C'\fR:
2093 .PP
2094 .Vb 4
2095 \& for (@lines) {
2096 \& s/foo/bar/; # change that word
2097 \& tr/XZ/ZX/; # swap those letters
2098 \& }
2099 .Ve
2100 .PP
2101 Here's another; let's compute spherical volumes:
2102 .PP
2103 .Vb 5
2104 \& my @volumes = @radii;
2105 \& for (@volumes) { # @volumes has changed parts
2106 \& $_ **= 3;
2107 \& $_ *= (4/3) * 3.14159; # this will be constant folded
2108 \& }
2109 .Ve
2110 .PP
2111 which can also be done with \f(CWmap()\fR which is made to transform
2112 one list into another:
2113 .PP
2114 .Vb 1
2115 \& my @volumes = map {$_ ** 3 * (4/3) * 3.14159} @radii;
2116 .Ve
2117 .PP
2118 If you want to do the same thing to modify the values of the
2119 hash, you can use the \f(CW\*(C`values\*(C'\fR function. As of Perl 5.6
2120 the values are not copied, so if you modify \f(CW$orbit\fR (in this
2121 case), you modify the value.
2122 .PP
2123 .Vb 3
2124 \& for my $orbit ( values %orbits ) {
2125 \& ($orbit **= 3) *= (4/3) * 3.14159;
2126 \& }
2127 .Ve
2128 .PP
2129 Prior to perl 5.6 \f(CW\*(C`values\*(C'\fR returned copies of the values,
2130 so older perl code often contains constructions such as
2131 \&\f(CW@orbits{keys %orbits}\fR instead of \f(CW\*(C`values %orbits\*(C'\fR where
2132 the hash is to be modified.
2133 .SS "How do I select a random element from an array?"
2134 .IX Subsection "How do I select a random element from an array?"
2135 Use the \f(CWrand()\fR function (see "rand" in perlfunc):
2136 .PP
2137 .Vb 2
2138 \& my $index = rand @array;
2139 \& my $element = $array[$index];
2140 .Ve
2141 .PP
2142 Or, simply:
2143 .PP
2144 .Vb 1
2145 \& my $element = $array[ rand @array ];
2146 .Ve
2147 .SS "How do I permute N elements of a list?"
2148 .IX Xref "List::Permutor permute Algorithm::Loops Knuth The Art of Computer Programming Fischer-Krause"
2149 .IX Subsection "How do I permute N elements of a list?"
2150 Use the List::Permutor module on CPAN. If the list is actually an
2151 array, try the Algorithm::Permute module (also on CPAN). It's
2152 written in XS code and is very efficient:
2153 .PP
2154 .Vb 1
2155 \& use Algorithm::Permute;
2156 \&
2157 \& my @array = \*(Aqa\*(Aq..\*(Aqd\*(Aq;
2158 \& my $p_iterator = Algorithm::Permute\->new ( \e@array );
2159 \&
2160 \& while (my @perm = $p_iterator\->next) {
2161 \& print "next permutation: (@perm)\en";
2162 \& }
2163 .Ve
2164 .PP
2165 For even faster execution, you could do:
2166 .PP
2167 .Vb 1
2168 \& use Algorithm::Permute;
2169 \&
2170 \& my @array = \*(Aqa\*(Aq..\*(Aqd\*(Aq;
2171 \&
2172 \& Algorithm::Permute::permute {
2173 \& print "next permutation: (@array)\en";
2174 \& } @array;
2175 .Ve
2176 .PP
2177 Here's a little program that generates all permutations of all the
2178 words on each line of input. The algorithm embodied in the
2179 \&\f(CWpermute()\fR function is discussed in Volume 4 (still unpublished) of
2180 Knuth's \fIThe Art of Computer Programming\fR and will work on any list:
2181 .PP
2182 .Vb 2
2183 \& #!/usr/bin/perl \-n
2184 \& # Fischer\-Krause ordered permutation generator
2185 \&
2186 \& sub permute (&@) {
2187 \& my $code = shift;
2188 \& my @idx = 0..$#_;
2189 \& while ( $code\->(@_[@idx]) ) {
2190 \& my $p = $#idx;
2191 \& \-\-$p while $idx[$p\-1] > $idx[$p];
2192 \& my $q = $p or return;
2193 \& push @idx, reverse splice @idx, $p;
2194 \& ++$q while $idx[$p\-1] > $idx[$q];
2195 \& @idx[$p\-1,$q]=@idx[$q,$p\-1];
2196 \& }
2197 \& }
2198 \&
2199 \& permute { print "@_\en" } split;
2200 .Ve
2201 .PP
2202 The Algorithm::Loops module also provides the \f(CW\*(C`NextPermute\*(C'\fR and
2203 \&\f(CW\*(C`NextPermuteNum\*(C'\fR functions which efficiently find all unique permutations
2204 of an array, even if it contains duplicate values, modifying it in-place:
2205 if its elements are in reverse-sorted order then the array is reversed,
2206 making it sorted, and it returns false; otherwise the next
2207 permutation is returned.
2208 .PP
2209 \&\f(CW\*(C`NextPermute\*(C'\fR uses string order and \f(CW\*(C`NextPermuteNum\*(C'\fR numeric order, so
2210 you can enumerate all the permutations of \f(CW0..9\fR like this:
2211 .PP
2212 .Vb 1
2213 \& use Algorithm::Loops qw(NextPermuteNum);
2214 \&
2215 \& my @list= 0..9;
2216 \& do { print "@list\en" } while NextPermuteNum @list;
2217 .Ve
2218 .SS "How do I sort an array by (anything)?"
2219 .IX Subsection "How do I sort an array by (anything)?"
2220 Supply a comparison function to \fBsort()\fR (described in "sort" in perlfunc):
2221 .PP
2222 .Vb 1
2223 \& @list = sort { $a <=> $b } @list;
2224 .Ve
2225 .PP
2226 The default sort function is cmp, string comparison, which would
2227 sort \f(CW\*(C`(1, 2, 10)\*(C'\fR into \f(CW\*(C`(1, 10, 2)\*(C'\fR. \f(CW\*(C`<=>\*(C'\fR, used above, is
2228 the numerical comparison operator.
2229 .PP
2230 If you have a complicated function needed to pull out the part you
2231 want to sort on, then don't do it inside the sort function. Pull it
2232 out first, because the sort BLOCK can be called many times for the
2233 same element. Here's an example of how to pull out the first word
2234 after the first number on each item, and then sort those words
2235 case-insensitively.
2236 .PP
2237 .Vb 7
2238 \& my @idx;
2239 \& for (@data) {
2240 \& my $item;
2241 \& ($item) = /\ed+\es*(\eS+)/;
2242 \& push @idx, uc($item);
2243 \& }
2244 \& my @sorted = @data[ sort { $idx[$a] cmp $idx[$b] } 0 .. $#idx ];
2245 .Ve
2246 .PP
2247 which could also be written this way, using a trick
2248 that's come to be known as the Schwartzian Transform:
2249 .PP
2250 .Vb 3
2251 \& my @sorted = map { $_\->[0] }
2252 \& sort { $a\->[1] cmp $b\->[1] }
2253 \& map { [ $_, uc( (/\ed+\es*(\eS+)/)[0]) ] } @data;
2254 .Ve
2255 .PP
2256 If you need to sort on several fields, the following paradigm is useful.
2257 .PP
2258 .Vb 5
2259 \& my @sorted = sort {
2260 \& field1($a) <=> field1($b) ||
2261 \& field2($a) cmp field2($b) ||
2262 \& field3($a) cmp field3($b)
2263 \& } @data;
2264 .Ve
2265 .PP
2266 This can be conveniently combined with precalculation of keys as given
2267 above.
2268 .PP
2269 See the \fIsort\fR article in the "Far More Than You Ever Wanted
2270 To Know" collection in <http://www.cpan.org/misc/olddoc/FMTEYEWTK.tgz> for
2271 more about this approach.
2272 .PP
2273 See also the question later in perlfaq4 on sorting hashes.
2274 .SS "How do I manipulate arrays of bits?"
2275 .IX Subsection "How do I manipulate arrays of bits?"
2276 Use \f(CWpack()\fR and \f(CWunpack()\fR, or else \f(CWvec()\fR and the bitwise
2277 operations.
2278 .PP
2279 For example, you don't have to store individual bits in an array
2280 (which would mean that you're wasting a lot of space). To convert an
2281 array of bits to a string, use \f(CWvec()\fR to set the right bits. This
2282 sets \f(CW$vec\fR to have bit N set only if \f(CW$ints[N]\fR was set:
2283 .PP
2284 .Vb 5
2285 \& my @ints = (...); # array of bits, e.g. ( 1, 0, 0, 1, 1, 0 ... )
2286 \& my $vec = \*(Aq\*(Aq;
2287 \& foreach( 0 .. $#ints ) {
2288 \& vec($vec,$_,1) = 1 if $ints[$_];
2289 \& }
2290 .Ve
2291 .PP
2292 The string \f(CW$vec\fR only takes up as many bits as it needs. For
2293 instance, if you had 16 entries in \f(CW@ints\fR, \f(CW$vec\fR only needs two
2294 bytes to store them (not counting the scalar variable overhead).
2295 .PP
2296 Here's how, given a vector in \f(CW$vec\fR, you can get those bits into
2297 your \f(CW@ints\fR array:
2298 .PP
2299 .Vb 7
2300 \& sub bitvec_to_list {
2301 \& my $vec = shift;
2302 \& my @ints;
2303 \& # Find null\-byte density then select best algorithm
2304 \& if ($vec =~ tr/\e0// / length $vec > 0.95) {
2305 \& use integer;
2306 \& my $i;
2307 \&
2308 \& # This method is faster with mostly null\-bytes
2309 \& while($vec =~ /[^\e0]/g ) {
2310 \& $i = \-9 + 8 * pos $vec;
2311 \& push @ints, $i if vec($vec, ++$i, 1);
2312 \& push @ints, $i if vec($vec, ++$i, 1);
2313 \& push @ints, $i if vec($vec, ++$i, 1);
2314 \& push @ints, $i if vec($vec, ++$i, 1);
2315 \& push @ints, $i if vec($vec, ++$i, 1);
2316 \& push @ints, $i if vec($vec, ++$i, 1);
2317 \& push @ints, $i if vec($vec, ++$i, 1);
2318 \& push @ints, $i if vec($vec, ++$i, 1);
2319 \& }
2320 \& }
2321 \& else {
2322 \& # This method is a fast general algorithm
2323 \& use integer;
2324 \& my $bits = unpack "b*", $vec;
2325 \& push @ints, 0 if $bits =~ s/^(\ed)// && $1;
2326 \& push @ints, pos $bits while($bits =~ /1/g);
2327 \& }
2328 \&
2329 \& return \e@ints;
2330 \& }
2331 .Ve
2332 .PP
2333 This method gets faster the more sparse the bit vector is.
2334 (Courtesy of Tim Bunce and Winfried Koenig.)
2335 .PP
2336 You can make the while loop a lot shorter with this suggestion
2337 from Benjamin Goldberg:
2338 .PP
2339 .Vb 3
2340 \& while($vec =~ /[^\e0]+/g ) {
2341 \& push @ints, grep vec($vec, $_, 1), $\-[0] * 8 .. $+[0] * 8;
2342 \& }
2343 .Ve
2344 .PP
2345 Or use the CPAN module Bit::Vector:
2346 .PP
2347 .Vb 3
2348 \& my $vector = Bit::Vector\->new($num_of_bits);
2349 \& $vector\->Index_List_Store(@ints);
2350 \& my @ints = $vector\->Index_List_Read();
2351 .Ve
2352 .PP
2353 Bit::Vector provides efficient methods for bit vector, sets of
2354 small integers and "big int" math.
2355 .PP
2356 Here's a more extensive illustration using \fBvec()\fR:
2357 .PP
2358 .Vb 7
2359 \& # vec demo
2360 \& my $vector = "\exff\ex0f\exef\exfe";
2361 \& print "Ilya\*(Aqs string \e\exff\e\ex0f\e\exef\e\exfe represents the number ",
2362 \& unpack("N", $vector), "\en";
2363 \& my $is_set = vec($vector, 23, 1);
2364 \& print "Its 23rd bit is ", $is_set ? "set" : "clear", ".\en";
2365 \& pvec($vector);
2366 \&
2367 \& set_vec(1,1,1);
2368 \& set_vec(3,1,1);
2369 \& set_vec(23,1,1);
2370 \&
2371 \& set_vec(3,1,3);
2372 \& set_vec(3,2,3);
2373 \& set_vec(3,4,3);
2374 \& set_vec(3,4,7);
2375 \& set_vec(3,8,3);
2376 \& set_vec(3,8,7);
2377 \&
2378 \& set_vec(0,32,17);
2379 \& set_vec(1,32,17);
2380 \&
2381 \& sub set_vec {
2382 \& my ($offset, $width, $value) = @_;
2383 \& my $vector = \*(Aq\*(Aq;
2384 \& vec($vector, $offset, $width) = $value;
2385 \& print "offset=$offset width=$width value=$value\en";
2386 \& pvec($vector);
2387 \& }
2388 \&
2389 \& sub pvec {
2390 \& my $vector = shift;
2391 \& my $bits = unpack("b*", $vector);
2392 \& my $i = 0;
2393 \& my $BASE = 8;
2394 \&
2395 \& print "vector length in bytes: ", length($vector), "\en";
2396 \& @bytes = unpack("A8" x length($vector), $bits);
2397 \& print "bits are: @bytes\en\en";
2398 \& }
2399 .Ve
2400 .SS "Why does \fBdefined()\fP return true on empty arrays and hashes?"
2401 .IX Subsection "Why does defined() return true on empty arrays and hashes?"
2402 The short story is that you should probably only use defined on scalars or
2403 functions, not on aggregates (arrays and hashes). See "defined" in perlfunc
2404 in the 5.004 release or later of Perl for more detail.
2405 .SH "Data: Hashes (Associative Arrays)"
2406 .IX Header "Data: Hashes (Associative Arrays)"
2407 .SS "How do I process an entire hash?"
2408 .IX Subsection "How do I process an entire hash?"
2409 (contributed by brian d foy)
2410 .PP
2411 There are a couple of ways that you can process an entire hash. You
2412 can get a list of keys, then go through each key, or grab a one
2413 key-value pair at a time.
2414 .PP
2415 To go through all of the keys, use the \f(CW\*(C`keys\*(C'\fR function. This extracts
2416 all of the keys of the hash and gives them back to you as a list. You
2417 can then get the value through the particular key you're processing:
2418 .PP
2419 .Vb 4
2420 \& foreach my $key ( keys %hash ) {
2421 \& my $value = $hash{$key}
2422 \& ...
2423 \& }
2424 .Ve
2425 .PP
2426 Once you have the list of keys, you can process that list before you
2427 process the hash elements. For instance, you can sort the keys so you
2428 can process them in lexical order:
2429 .PP
2430 .Vb 4
2431 \& foreach my $key ( sort keys %hash ) {
2432 \& my $value = $hash{$key}
2433 \& ...
2434 \& }
2435 .Ve
2436 .PP
2437 Or, you might want to only process some of the items. If you only want
2438 to deal with the keys that start with \f(CW\*(C`text:\*(C'\fR, you can select just
2439 those using \f(CW\*(C`grep\*(C'\fR:
2440 .PP
2441 .Vb 4
2442 \& foreach my $key ( grep /^text:/, keys %hash ) {
2443 \& my $value = $hash{$key}
2444 \& ...
2445 \& }
2446 .Ve
2447 .PP
2448 If the hash is very large, you might not want to create a long list of
2449 keys. To save some memory, you can grab one key-value pair at a time using
2450 \&\f(CWeach()\fR, which returns a pair you haven't seen yet:
2451 .PP
2452 .Vb 3
2453 \& while( my( $key, $value ) = each( %hash ) ) {
2454 \& ...
2455 \& }
2456 .Ve
2457 .PP
2458 The \f(CW\*(C`each\*(C'\fR operator returns the pairs in apparently random order, so if
2459 ordering matters to you, you'll have to stick with the \f(CW\*(C`keys\*(C'\fR method.
2460 .PP
2461 The \f(CWeach()\fR operator can be a bit tricky though. You can't add or
2462 delete keys of the hash while you're using it without possibly
2463 skipping or re-processing some pairs after Perl internally rehashes
2464 all of the elements. Additionally, a hash has only one iterator, so if
2465 you mix \f(CW\*(C`keys\*(C'\fR, \f(CW\*(C`values\*(C'\fR, or \f(CW\*(C`each\*(C'\fR on the same hash, you risk resetting
2466 the iterator and messing up your processing. See the \f(CW\*(C`each\*(C'\fR entry in
2467 perlfunc for more details.
2468 .SS "How do I merge two hashes?"
2469 .IX Xref "hash merge slice, hash"
2470 .IX Subsection "How do I merge two hashes?"
2471 (contributed by brian d foy)
2472 .PP
2473 Before you decide to merge two hashes, you have to decide what to do
2474 if both hashes contain keys that are the same and if you want to leave
2475 the original hashes as they were.
2476 .PP
2477 If you want to preserve the original hashes, copy one hash (\f(CW%hash1\fR)
2478 to a new hash (\f(CW%new_hash\fR), then add the keys from the other hash
2479 (\f(CW%hash2\fR to the new hash. Checking that the key already exists in
2480 \&\f(CW%new_hash\fR gives you a chance to decide what to do with the
2481 duplicates:
2482 .PP
2483 .Vb 1
2484 \& my %new_hash = %hash1; # make a copy; leave %hash1 alone
2485 \&
2486 \& foreach my $key2 ( keys %hash2 ) {
2487 \& if( exists $new_hash{$key2} ) {
2488 \& warn "Key [$key2] is in both hashes!";
2489 \& # handle the duplicate (perhaps only warning)
2490 \& ...
2491 \& next;
2492 \& }
2493 \& else {
2494 \& $new_hash{$key2} = $hash2{$key2};
2495 \& }
2496 \& }
2497 .Ve
2498 .PP
2499 If you don't want to create a new hash, you can still use this looping
2500 technique; just change the \f(CW%new_hash\fR to \f(CW%hash1\fR.
2501 .PP
2502 .Vb 11
2503 \& foreach my $key2 ( keys %hash2 ) {
2504 \& if( exists $hash1{$key2} ) {
2505 \& warn "Key [$key2] is in both hashes!";
2506 \& # handle the duplicate (perhaps only warning)
2507 \& ...
2508 \& next;
2509 \& }
2510 \& else {
2511 \& $hash1{$key2} = $hash2{$key2};
2512 \& }
2513 \& }
2514 .Ve
2515 .PP
2516 If you don't care that one hash overwrites keys and values from the other, you
2517 could just use a hash slice to add one hash to another. In this case, values
2518 from \f(CW%hash2\fR replace values from \f(CW%hash1\fR when they have keys in common:
2519 .PP
2520 .Vb 1
2521 \& @hash1{ keys %hash2 } = values %hash2;
2522 .Ve
2523 .SS "What happens if I add or remove keys from a hash while iterating over it?"
2524 .IX Subsection "What happens if I add or remove keys from a hash while iterating over it?"
2525 (contributed by brian d foy)
2526 .PP
2527 The easy answer is "Don't do that!"
2528 .PP
2529 If you iterate through the hash with \fBeach()\fR, you can delete the key
2530 most recently returned without worrying about it. If you delete or add
2531 other keys, the iterator may skip or double up on them since perl
2532 may rearrange the hash table. See the
2533 entry for \f(CWeach()\fR in perlfunc.
2534 .SS "How do I look up a hash element by value?"
2535 .IX Subsection "How do I look up a hash element by value?"
2536 Create a reverse hash:
2537 .PP
2538 .Vb 2
2539 \& my %by_value = reverse %by_key;
2540 \& my $key = $by_value{$value};
2541 .Ve
2542 .PP
2543 That's not particularly efficient. It would be more space-efficient
2544 to use:
2545 .PP
2546 .Vb 3
2547 \& while (my ($key, $value) = each %by_key) {
2548 \& $by_value{$value} = $key;
2549 \& }
2550 .Ve
2551 .PP
2552 If your hash could have repeated values, the methods above will only find
2553 one of the associated keys. This may or may not worry you. If it does
2554 worry you, you can always reverse the hash into a hash of arrays instead:
2555 .PP
2556 .Vb 3
2557 \& while (my ($key, $value) = each %by_key) {
2558 \& push @{$key_list_by_value{$value}}, $key;
2559 \& }
2560 .Ve
2561 .SS "How can I know how many entries are in a hash?"
2562 .IX Subsection "How can I know how many entries are in a hash?"
2563 (contributed by brian d foy)
2564 .PP
2565 This is very similar to "How do I process an entire hash?", also in
2566 perlfaq4, but a bit simpler in the common cases.
2567 .PP
2568 You can use the \f(CWkeys()\fR built-in function in scalar context to find out
2569 how many entries you have in a hash:
2570 .PP
2571 .Vb 1
2572 \& my $key_count = keys %hash; # must be scalar context!
2573 .Ve
2574 .PP
2575 If you want to find out how many entries have a defined value, that's
2576 a bit different. You have to check each value. A \f(CW\*(C`grep\*(C'\fR is handy:
2577 .PP
2578 .Vb 1
2579 \& my $defined_value_count = grep { defined } values %hash;
2580 .Ve
2581 .PP
2582 You can use that same structure to count the entries any way that
2583 you like. If you want the count of the keys with vowels in them,
2584 you just test for that instead:
2585 .PP
2586 .Vb 1
2587 \& my $vowel_count = grep { /[aeiou]/ } keys %hash;
2588 .Ve
2589 .PP
2590 The \f(CW\*(C`grep\*(C'\fR in scalar context returns the count. If you want the list
2591 of matching items, just use it in list context instead:
2592 .PP
2593 .Vb 1
2594 \& my @defined_values = grep { defined } values %hash;
2595 .Ve
2596 .PP
2597 The \f(CWkeys()\fR function also resets the iterator, which means that you may
2598 see strange results if you use this between uses of other hash operators
2599 such as \f(CWeach()\fR.
2600 .SS "How do I sort a hash (optionally by value instead of key)?"
2601 .IX Subsection "How do I sort a hash (optionally by value instead of key)?"
2602 (contributed by brian d foy)
2603 .PP
2604 To sort a hash, start with the keys. In this example, we give the list of
2605 keys to the sort function which then compares them as strings. The output list
2606 has the keys in string order. Once we have the keys, we can go through them to
2607 create a report which lists the keys in string order:
2608 .PP
2609 .Vb 1
2610 \& my @keys = sort { $a cmp $b } keys %hash;
2611 \&
2612 \& foreach my $key ( @keys ) {
2613 \& printf "%\-20s %6d\en", $key, $hash{$key};
2614 \& }
2615 .Ve
2616 .PP
2617 We could get more fancy in the \f(CWsort()\fR block though. Instead of
2618 comparing the keys, we can compute a value with them and use that
2619 value as the comparison.
2620 .PP
2621 For instance, to make our report order case-insensitive, we use
2622 \&\f(CW\*(C`fc\*(C'\fR to safely lowercase the keys before comparing them:
2623 .PP
2624 .Vb 2
2625 \& use v5.16;
2626 \& my @keys = sort { fc($a) cmp fc($b) } keys %hash;
2627 .Ve
2628 .PP
2629 Earlier versions of this answer used \f(CW\*(C`lc\*(C'\fR, but that could give
2630 unexpected results with some Unicode strings. See "fc" in perlfunc
2631 for the details. The Unicode::UCD module does the same thing for
2632 earlier perls.
2633 .PP
2634 Note: if the computation is expensive or the hash has many elements,
2635 you may want to look at the Schwartzian Transform to cache the
2636 computation results.
2637 .PP
2638 If we want to sort by the hash value instead, we use the hash key
2639 to look it up. We still get out a list of keys, but this time they
2640 are ordered by their value:
2641 .PP
2642 .Vb 1
2643 \& my @keys = sort { $hash{$a} <=> $hash{$b} } keys %hash;
2644 .Ve
2645 .PP
2646 From there we can get more complex. If the hash values are the same,
2647 we can provide a secondary sort on the hash key:
2648 .PP
2649 .Vb 6
2650 \& use v5.16;
2651 \& my @keys = sort {
2652 \& $hash{$a} <=> $hash{$b}
2653 \& or
2654 \& fc($a) cmp fc($b)
2655 \& } keys %hash;
2656 .Ve
2657 .SS "How can I always keep my hash sorted?"
2658 .IX Xref "hash tie sort DB_File Tie::IxHash"
2659 .IX Subsection "How can I always keep my hash sorted?"
2660 You can look into using the \f(CW\*(C`DB_File\*(C'\fR module and \f(CWtie()\fR using the
2661 \&\f(CW$DB_BTREE\fR hash bindings as documented in "In Memory
2662 Databases" in DB_File. The Tie::IxHash module from CPAN might also be
2663 instructive. Although this does keep your hash sorted, you might not
2664 like the slowdown you suffer from the tie interface. Are you sure you
2665 need to do this? :)
2666 .SS "What's the difference between ""delete"" and ""undef"" with hashes?"
2667 .IX Subsection "What's the difference between ""delete"" and ""undef"" with hashes?"
2668 Hashes contain pairs of scalars: the first is the key, the
2669 second is the value. The key will be coerced to a string,
2670 although the value can be any kind of scalar: string,
2671 number, or reference. If a key \f(CW$key\fR is present in
2672 \&\f(CW%hash\fR, \f(CWexists($hash{$key})\fR will return true. The value
2673 for a given key can be \f(CW\*(C`undef\*(C'\fR, in which case
2674 \&\f(CW$hash{$key}\fR will be \f(CW\*(C`undef\*(C'\fR while \f(CW\*(C`exists $hash{$key}\*(C'\fR
2675 will return true. This corresponds to (\f(CW$key\fR, \f(CW\*(C`undef\*(C'\fR)
2676 being in the hash.
2677 .PP
2678 Pictures help... Here's the \f(CW%hash\fR table:
2679 .PP
2680 .Vb 7
2681 \& keys values
2682 \& +\-\-\-\-\-\-+\-\-\-\-\-\-+
2683 \& | a | 3 |
2684 \& | x | 7 |
2685 \& | d | 0 |
2686 \& | e | 2 |
2687 \& +\-\-\-\-\-\-+\-\-\-\-\-\-+
2688 .Ve
2689 .PP
2690 And these conditions hold
2691 .PP
2692 .Vb 6
2693 \& $hash{\*(Aqa\*(Aq} is true
2694 \& $hash{\*(Aqd\*(Aq} is false
2695 \& defined $hash{\*(Aqd\*(Aq} is true
2696 \& defined $hash{\*(Aqa\*(Aq} is true
2697 \& exists $hash{\*(Aqa\*(Aq} is true (Perl 5 only)
2698 \& grep ($_ eq \*(Aqa\*(Aq, keys %hash) is true
2699 .Ve
2700 .PP
2701 If you now say
2702 .PP
2703 .Vb 1
2704 \& undef $hash{\*(Aqa\*(Aq}
2705 .Ve
2706 .PP
2707 your table now reads:
2708 .PP
2709 .Vb 7
2710 \& keys values
2711 \& +\-\-\-\-\-\-+\-\-\-\-\-\-+
2712 \& | a | undef|
2713 \& | x | 7 |
2714 \& | d | 0 |
2715 \& | e | 2 |
2716 \& +\-\-\-\-\-\-+\-\-\-\-\-\-+
2717 .Ve
2718 .PP
2719 and these conditions now hold; changes in caps:
2720 .PP
2721 .Vb 6
2722 \& $hash{\*(Aqa\*(Aq} is FALSE
2723 \& $hash{\*(Aqd\*(Aq} is false
2724 \& defined $hash{\*(Aqd\*(Aq} is true
2725 \& defined $hash{\*(Aqa\*(Aq} is FALSE
2726 \& exists $hash{\*(Aqa\*(Aq} is true (Perl 5 only)
2727 \& grep ($_ eq \*(Aqa\*(Aq, keys %hash) is true
2728 .Ve
2729 .PP
2730 Notice the last two: you have an undef value, but a defined key!
2731 .PP
2732 Now, consider this:
2733 .PP
2734 .Vb 1
2735 \& delete $hash{\*(Aqa\*(Aq}
2736 .Ve
2737 .PP
2738 your table now reads:
2739 .PP
2740 .Vb 6
2741 \& keys values
2742 \& +\-\-\-\-\-\-+\-\-\-\-\-\-+
2743 \& | x | 7 |
2744 \& | d | 0 |
2745 \& | e | 2 |
2746 \& +\-\-\-\-\-\-+\-\-\-\-\-\-+
2747 .Ve
2748 .PP
2749 and these conditions now hold; changes in caps:
2750 .PP
2751 .Vb 6
2752 \& $hash{\*(Aqa\*(Aq} is false
2753 \& $hash{\*(Aqd\*(Aq} is false
2754 \& defined $hash{\*(Aqd\*(Aq} is true
2755 \& defined $hash{\*(Aqa\*(Aq} is false
2756 \& exists $hash{\*(Aqa\*(Aq} is FALSE (Perl 5 only)
2757 \& grep ($_ eq \*(Aqa\*(Aq, keys %hash) is FALSE
2758 .Ve
2759 .PP
2760 See, the whole entry is gone!
2761 .SS "Why don't my tied hashes make the defined/exists distinction?"
2762 .IX Subsection "Why don't my tied hashes make the defined/exists distinction?"
2763 This depends on the tied hash's implementation of \fBEXISTS()\fR.
2764 For example, there isn't the concept of undef with hashes
2765 that are tied to DBM* files. It also means that \fBexists()\fR and
2766 \&\fBdefined()\fR do the same thing with a DBM* file, and what they
2767 end up doing is not what they do with ordinary hashes.
2768 .SS "How do I reset an \fBeach()\fP operation part-way through?"
2769 .IX Subsection "How do I reset an each() operation part-way through?"
2770 (contributed by brian d foy)
2771 .PP
2772 You can use the \f(CW\*(C`keys\*(C'\fR or \f(CW\*(C`values\*(C'\fR functions to reset \f(CW\*(C`each\*(C'\fR. To
2773 simply reset the iterator used by \f(CW\*(C`each\*(C'\fR without doing anything else,
2774 use one of them in void context:
2775 .PP
2776 .Vb 2
2777 \& keys %hash; # resets iterator, nothing else.
2778 \& values %hash; # resets iterator, nothing else.
2779 .Ve
2780 .PP
2781 See the documentation for \f(CW\*(C`each\*(C'\fR in perlfunc.
2782 .SS "How can I get the unique keys from two hashes?"
2783 .IX Subsection "How can I get the unique keys from two hashes?"
2784 First you extract the keys from the hashes into lists, then solve
2785 the "removing duplicates" problem described above. For example:
2786 .PP
2787 .Vb 5
2788 \& my %seen = ();
2789 \& for my $element (keys(%foo), keys(%bar)) {
2790 \& $seen{$element}++;
2791 \& }
2792 \& my @uniq = keys %seen;
2793 .Ve
2794 .PP
2795 Or more succinctly:
2796 .PP
2797 .Vb 1
2798 \& my @uniq = keys %{{%foo,%bar}};
2799 .Ve
2800 .PP
2801 Or if you really want to save space:
2802 .PP
2803 .Vb 8
2804 \& my %seen = ();
2805 \& while (defined ($key = each %foo)) {
2806 \& $seen{$key}++;
2807 \& }
2808 \& while (defined ($key = each %bar)) {
2809 \& $seen{$key}++;
2810 \& }
2811 \& my @uniq = keys %seen;
2812 .Ve
2813 .SS "How can I store a multidimensional array in a DBM file?"
2814 .IX Subsection "How can I store a multidimensional array in a DBM file?"
2815 Either stringify the structure yourself (no fun), or else
2816 get the MLDBM (which uses Data::Dumper) module from CPAN and layer
2817 it on top of either DB_File or GDBM_File. You might also try DBM::Deep, but
2818 it can be a bit slow.
2819 .SS "How can I make my hash remember the order I put elements into it?"
2820 .IX Subsection "How can I make my hash remember the order I put elements into it?"
2821 Use the Tie::IxHash from CPAN.
2822 .PP
2823 .Vb 1
2824 \& use Tie::IxHash;
2825 \&
2826 \& tie my %myhash, \*(AqTie::IxHash\*(Aq;
2827 \&
2828 \& for (my $i=0; $i<20; $i++) {
2829 \& $myhash{$i} = 2*$i;
2830 \& }
2831 \&
2832 \& my @keys = keys %myhash;
2833 \& # @keys = (0,1,2,3,...)
2834 .Ve
2835 .SS "Why does passing a subroutine an undefined element in a hash create it?"
2836 .IX Subsection "Why does passing a subroutine an undefined element in a hash create it?"
2837 (contributed by brian d foy)
2838 .PP
2839 Are you using a really old version of Perl?
2840 .PP
2841 Normally, accessing a hash key's value for a nonexistent key will
2842 \&\fInot\fR create the key.
2843 .PP
2844 .Vb 3
2845 \& my %hash = ();
2846 \& my $value = $hash{ \*(Aqfoo\*(Aq };
2847 \& print "This won\*(Aqt print\en" if exists $hash{ \*(Aqfoo\*(Aq };
2848 .Ve
2849 .PP
2850 Passing \f(CW$hash{ \*(Aqfoo\*(Aq }\fR to a subroutine used to be a special case, though.
2851 Since you could assign directly to \f(CW$_[0]\fR, Perl had to be ready to
2852 make that assignment so it created the hash key ahead of time:
2853 .PP
2854 .Vb 2
2855 \& my_sub( $hash{ \*(Aqfoo\*(Aq } );
2856 \& print "This will print before 5.004\en" if exists $hash{ \*(Aqfoo\*(Aq };
2857 \&
2858 \& sub my_sub {
2859 \& # $_[0] = \*(Aqbar\*(Aq; # create hash key in case you do this
2860 \& 1;
2861 \& }
2862 .Ve
2863 .PP
2864 Since Perl 5.004, however, this situation is a special case and Perl
2865 creates the hash key only when you make the assignment:
2866 .PP
2867 .Vb 2
2868 \& my_sub( $hash{ \*(Aqfoo\*(Aq } );
2869 \& print "This will print, even after 5.004\en" if exists $hash{ \*(Aqfoo\*(Aq };
2870 \&
2871 \& sub my_sub {
2872 \& $_[0] = \*(Aqbar\*(Aq;
2873 \& }
2874 .Ve
2875 .PP
2876 However, if you want the old behavior (and think carefully about that
2877 because it's a weird side effect), you can pass a hash slice instead.
2878 Perl 5.004 didn't make this a special case:
2879 .PP
2880 .Vb 1
2881 \& my_sub( @hash{ qw/foo/ } );
2882 .Ve
2883 .SS "How can I make the Perl equivalent of a C structure/C++ class/hash or array of hashes or arrays?"
2884 .IX Subsection "How can I make the Perl equivalent of a C structure/C++ class/hash or array of hashes or arrays?"
2885 Usually a hash ref, perhaps like this:
2886 .PP
2887 .Vb 8
2888 \& $record = {
2889 \& NAME => "Jason",
2890 \& EMPNO => 132,
2891 \& TITLE => "deputy peon",
2892 \& AGE => 23,
2893 \& SALARY => 37_000,
2894 \& PALS => [ "Norbert", "Rhys", "Phineas"],
2895 \& };
2896 .Ve
2897 .PP
2898 References are documented in perlref and perlreftut.
2899 Examples of complex data structures are given in perldsc and
2900 perllol. Examples of structures and object-oriented classes are
2901 in perlootut.
2902 .SS "How can I use a reference as a hash key?"
2903 .IX Subsection "How can I use a reference as a hash key?"
2904 (contributed by brian d foy and Ben Morrow)
2905 .PP
2906 Hash keys are strings, so you can't really use a reference as the key.
2907 When you try to do that, perl turns the reference into its stringified
2908 form (for instance, \f(CWHASH(0xDEADBEEF)\fR). From there you can't get
2909 back the reference from the stringified form, at least without doing
2910 some extra work on your own.
2911 .PP
2912 Remember that the entry in the hash will still be there even if
2913 the referenced variable goes out of scope, and that it is entirely
2914 possible for Perl to subsequently allocate a different variable at
2915 the same address. This will mean a new variable might accidentally
2916 be associated with the value for an old.
2917 .PP
2918 If you have Perl 5.10 or later, and you just want to store a value
2919 against the reference for lookup later, you can use the core
2920 Hash::Util::Fieldhash module. This will also handle renaming the
2921 keys if you use multiple threads (which causes all variables to be
2922 reallocated at new addresses, changing their stringification), and
2923 garbage-collecting the entries when the referenced variable goes out
2924 of scope.
2925 .PP
2926 If you actually need to be able to get a real reference back from
2927 each hash entry, you can use the Tie::RefHash module, which does the
2928 required work for you.
2929 .SS "How can I check if a key exists in a multilevel hash?"
2930 .IX Subsection "How can I check if a key exists in a multilevel hash?"
2931 (contributed by brian d foy)
2932 .PP
2933 The trick to this problem is avoiding accidental autovivification. If
2934 you want to check three keys deep, you might naïvely try this:
2935 .PP
2936 .Vb 4
2937 \& my %hash;
2938 \& if( exists $hash{key1}{key2}{key3} ) {
2939 \& ...;
2940 \& }
2941 .Ve
2942 .PP
2943 Even though you started with a completely empty hash, after that call to
2944 \&\f(CW\*(C`exists\*(C'\fR you've created the structure you needed to check for \f(CW\*(C`key3\*(C'\fR:
2945 .PP
2946 .Vb 5
2947 \& %hash = (
2948 \& \*(Aqkey1\*(Aq => {
2949 \& \*(Aqkey2\*(Aq => {}
2950 \& }
2951 \& );
2952 .Ve
2953 .PP
2954 That's autovivification. You can get around this in a few ways. The
2955 easiest way is to just turn it off. The lexical \f(CW\*(C`autovivification\*(C'\fR
2956 pragma is available on CPAN. Now you don't add to the hash:
2957 .PP
2958 .Vb 7
2959 \& {
2960 \& no autovivification;
2961 \& my %hash;
2962 \& if( exists $hash{key1}{key2}{key3} ) {
2963 \& ...;
2964 \& }
2965 \& }
2966 .Ve
2967 .PP
2968 The Data::Diver module on CPAN can do it for you too. Its \f(CW\*(C`Dive\*(C'\fR
2969 subroutine can tell you not only if the keys exist but also get the
2970 value:
2971 .PP
2972 .Vb 1
2973 \& use Data::Diver qw(Dive);
2974 \&
2975 \& my @exists = Dive( \e%hash, qw(key1 key2 key3) );
2976 \& if( ! @exists ) {
2977 \& ...; # keys do not exist
2978 \& }
2979 \& elsif( ! defined $exists[0] ) {
2980 \& ...; # keys exist but value is undef
2981 \& }
2982 .Ve
2983 .PP
2984 You can easily do this yourself too by checking each level of the hash
2985 before you move onto the next level. This is essentially what
2986 Data::Diver does for you:
2987 .PP
2988 .Vb 3
2989 \& if( check_hash( \e%hash, qw(key1 key2 key3) ) ) {
2990 \& ...;
2991 \& }
2992 \&
2993 \& sub check_hash {
2994 \& my( $hash, @keys ) = @_;
2995 \&
2996 \& return unless @keys;
2997 \&
2998 \& foreach my $key ( @keys ) {
2999 \& return unless eval { exists $hash\->{$key} };
3000 \& $hash = $hash\->{$key};
3001 \& }
3002 \&
3003 \& return 1;
3004 \& }
3005 .Ve
3006 .SS "How can I prevent addition of unwanted keys into a hash?"
3007 .IX Subsection "How can I prevent addition of unwanted keys into a hash?"
3008 Since version 5.8.0, hashes can be \fIrestricted\fR to a fixed number
3009 of given keys. Methods for creating and dealing with restricted hashes
3010 are exported by the Hash::Util module.
3011 .SH "Data: Misc"
3012 .IX Header "Data: Misc"
3013 .SS "How do I handle binary data correctly?"
3014 .IX Subsection "How do I handle binary data correctly?"
3015 Perl is binary-clean, so it can handle binary data just fine.
3016 On Windows or DOS, however, you have to use \f(CW\*(C`binmode\*(C'\fR for binary
3017 files to avoid conversions for line endings. In general, you should
3018 use \f(CW\*(C`binmode\*(C'\fR any time you want to work with binary data.
3019 .PP
3020 Also see "binmode" in perlfunc or perlopentut.
3021 .PP
3022 If you're concerned about 8\-bit textual data then see perllocale.
3023 If you want to deal with multibyte characters, however, there are
3024 some gotchas. See the section on Regular Expressions.
3025 .SS "How do I determine whether a scalar is a number/whole/integer/float?"
3026 .IX Subsection "How do I determine whether a scalar is a number/whole/integer/float?"
3027 Assuming that you don't care about IEEE notations like "NaN" or
3028 "Infinity", you probably just want to use a regular expression (see also
3029 perlretut and perlre):
3030 .PP
3031 .Vb 1
3032 \& use 5.010;
3033 \&
3034 \& if ( /\eD/ )
3035 \& { say "\ethas nondigits"; }
3036 \& if ( /^\ed+\ez/ )
3037 \& { say "\etis a whole number"; }
3038 \& if ( /^\-?\ed+\ez/ )
3039 \& { say "\etis an integer"; }
3040 \& if ( /^[+\-]?\ed+\ez/ )
3041 \& { say "\etis a +/\- integer"; }
3042 \& if ( /^\-?(?:\ed+\e.?|\e.\ed)\ed*\ez/ )
3043 \& { say "\etis a real number"; }
3044 \& if ( /^[+\-]?(?=\e.?\ed)\ed*\e.?\ed*(?:e[+\-]?\ed+)?\ez/i )
3045 \& { say "\etis a C float" }
3046 .Ve
3047 .PP
3048 There are also some commonly used modules for the task.
3049 Scalar::Util (distributed with 5.8) provides access to perl's
3050 internal function \f(CW\*(C`looks_like_number\*(C'\fR for determining whether a
3051 variable looks like a number. Data::Types exports functions that
3052 validate data types using both the above and other regular
3053 expressions. Thirdly, there is Regexp::Common which has regular
3054 expressions to match various types of numbers. Those three modules are
3055 available from the CPAN.
3056 .PP
3057 If you're on a POSIX system, Perl supports the \f(CW\*(C`POSIX::strtod\*(C'\fR
3058 function for converting strings to doubles (and also \f(CW\*(C`POSIX::strtol\*(C'\fR
3059 for longs). Its semantics are somewhat cumbersome, so here's a
3060 \&\f(CW\*(C`getnum\*(C'\fR wrapper function for more convenient access. This function
3061 takes a string and returns the number it found, or \f(CW\*(C`undef\*(C'\fR for input
3062 that isn't a C float. The \f(CW\*(C`is_numeric\*(C'\fR function is a front end to
3063 \&\f(CW\*(C`getnum\*(C'\fR if you just want to say, "Is this a float?"
3064 .PP
3065 .Vb 10
3066 \& sub getnum {
3067 \& use POSIX qw(strtod);
3068 \& my $str = shift;
3069 \& $str =~ s/^\es+//;
3070 \& $str =~ s/\es+$//;
3071 \& $! = 0;
3072 \& my($num, $unparsed) = strtod($str);
3073 \& if (($str eq \*(Aq\*(Aq) || ($unparsed != 0) || $!) {
3074 \& return undef;
3075 \& }
3076 \& else {
3077 \& return $num;
3078 \& }
3079 \& }
3080 \&
3081 \& sub is_numeric { defined getnum($_[0]) }
3082 .Ve
3083 .PP
3084 Or you could check out the String::Scanf module on the CPAN
3085 instead.
3086 .SS "How do I keep persistent data across program calls?"
3087 .IX Subsection "How do I keep persistent data across program calls?"
3088 For some specific applications, you can use one of the DBM modules.
3089 See AnyDBM_File. More generically, you should consult the FreezeThaw
3090 or Storable modules from CPAN. Starting from Perl 5.8, Storable is part
3091 of the standard distribution. Here's one example using Storable's \f(CW\*(C`store\*(C'\fR
3092 and \f(CW\*(C`retrieve\*(C'\fR functions:
3093 .PP
3094 .Vb 2
3095 \& use Storable;
3096 \& store(\e%hash, "filename");
3097 \&
3098 \& # later on...
3099 \& $href = retrieve("filename"); # by ref
3100 \& %hash = %{ retrieve("filename") }; # direct to hash
3101 .Ve
3102 .SS "How do I print out or copy a recursive data structure?"
3103 .IX Subsection "How do I print out or copy a recursive data structure?"
3104 The Data::Dumper module on CPAN (or the 5.005 release of Perl) is great
3105 for printing out data structures. The Storable module on CPAN (or the
3106 5.8 release of Perl), provides a function called \f(CW\*(C`dclone\*(C'\fR that recursively
3107 copies its argument.
3108 .PP
3109 .Vb 2
3110 \& use Storable qw(dclone);
3111 \& $r2 = dclone($r1);
3112 .Ve
3113 .PP
3114 Where \f(CW$r1\fR can be a reference to any kind of data structure you'd like.
3115 It will be deeply copied. Because \f(CW\*(C`dclone\*(C'\fR takes and returns references,
3116 you'd have to add extra punctuation if you had a hash of arrays that
3117 you wanted to copy.
3118 .PP
3119 .Vb 1
3120 \& %newhash = %{ dclone(\e%oldhash) };
3121 .Ve
3122 .SS "How do I define methods for every class/object?"
3123 .IX Subsection "How do I define methods for every class/object?"
3124 (contributed by Ben Morrow)
3125 .PP
3126 You can use the \f(CW\*(C`UNIVERSAL\*(C'\fR class (see UNIVERSAL). However, please
3127 be very careful to consider the consequences of doing this: adding
3128 methods to every object is very likely to have unintended
3129 consequences. If possible, it would be better to have all your object
3130 inherit from some common base class, or to use an object system like
3131 Moose that supports roles.
3132 .SS "How do I verify a credit card checksum?"
3133 .IX Subsection "How do I verify a credit card checksum?"
3134 Get the Business::CreditCard module from CPAN.
3135 .SS "How do I pack arrays of doubles or floats for XS code?"
3136 .IX Subsection "How do I pack arrays of doubles or floats for XS code?"
3137 The arrays.h/arrays.c code in the PGPLOT module on CPAN does just this.
3138 If you're doing a lot of float or double processing, consider using
3139 the PDL module from CPAN instead\-\-it makes number-crunching easy.
3140 .PP
3141 See <https://metacpan.org/release/PGPLOT> for the code.
3142 .SH "AUTHOR AND COPYRIGHT"
3143 .IX Header "AUTHOR AND COPYRIGHT"
3144 Copyright (c) 1997\-2010 Tom Christiansen, Nathan Torkington, and
3145 other authors as noted. All rights reserved.
3146 .PP
3147 This documentation is free; you can redistribute it and/or modify it
3148 under the same terms as Perl itself.
3149 .PP
3150 Irrespective of its distribution, all code examples in this file
3151 are hereby placed into the public domain. You are permitted and
3152 encouraged to use this code in your own programs for fun
3153 or for profit as you see fit. A simple comment in the code giving
3154 credit would be courteous but is not required.