Debian Codegraph @

upstream/mageia-cauldron/man3pm/Storable.3pm

raw
1 .\" -*- mode: troff; coding: utf-8 -*-
2 .\" Automatically generated by Pod::Man v6.0.2 (Pod::Simple 3.45)
3 .\"
4 .\" Standard preamble:
5 .\" ========================================================================
6 .de Sp \" Vertical space (when we can't use .PP)
7 .if t .sp .5v
8 .if n .sp
9 ..
10 .de Vb \" Begin verbatim text
11 .ft CW
12 .nf
13 .ne \\$1
14 ..
15 .de Ve \" End verbatim text
16 .ft R
17 .fi
18 ..
19 .\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>.
20 .ie n \{\
21 . ds C` ""
22 . ds C' ""
23 'br\}
24 .el\{\
25 . ds C`
26 . ds C'
27 'br\}
28 .\"
29 .\" Escape single quotes in literal strings from groff's Unicode transform.
30 .ie \n(.g .ds Aq \(aq
31 .el .ds Aq '
32 .\"
33 .\" If the F register is >0, we'll generate index entries on stderr for
34 .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
35 .\" entries marked with X<> in POD. Of course, you'll have to process the
36 .\" output yourself in some meaningful fashion.
37 .\"
38 .\" Avoid warning from groff about undefined register 'F'.
39 .de IX
40 ..
41 .nr rF 0
42 .if \n(.g .if rF .nr rF 1
43 .if (\n(rF:(\n(.g==0)) \{\
44 . if \nF \{\
45 . de IX
46 . tm Index:\\$1\t\\n%\t"\\$2"
47 ..
48 . if !\nF==2 \{\
49 . nr % 0
50 . nr F 2
51 . \}
52 . \}
53 .\}
54 .rr rF
55 .\"
56 .\" Required to disable full justification in groff 1.23.0.
57 .if n .ds AD l
58 .\" ========================================================================
59 .\"
60 .IX Title "Storable 3pm"
61 .TH Storable 3pm 2025-06-24 "perl v5.42.0" "Perl Programmers Reference Guide"
62 .\" For nroff, turn off justification. Always turn off hyphenation; it makes
63 .\" way too many mistakes in technical documents.
64 .if n .ad l
65 .nh
66 .SH NAME
67 Storable \- persistence for Perl data structures
68 .SH SYNOPSIS
69 .IX Header "SYNOPSIS"
70 .Vb 3
71 \& use Storable;
72 \& store \e%table, \*(Aqfile\*(Aq;
73 \& $hashref = retrieve(\*(Aqfile\*(Aq);
74 \&
75 \& use Storable qw(nstore store_fd nstore_fd freeze thaw dclone);
76 \&
77 \& # Network order
78 \& nstore \e%table, \*(Aqfile\*(Aq;
79 \& $hashref = retrieve(\*(Aqfile\*(Aq); # There is NO nretrieve()
80 \&
81 \& # Storing to and retrieving from an already opened file
82 \& store_fd \e@array, \e*STDOUT;
83 \& nstore_fd \e%table, \e*STDOUT;
84 \& $aryref = fd_retrieve(\e*SOCKET);
85 \& $hashref = fd_retrieve(\e*SOCKET);
86 \&
87 \& # Serializing to memory
88 \& $serialized = freeze \e%table;
89 \& %table_clone = %{ thaw($serialized) };
90 \&
91 \& # Deep (recursive) cloning
92 \& $cloneref = dclone($ref);
93 \&
94 \& # Advisory locking
95 \& use Storable qw(lock_store lock_nstore lock_retrieve)
96 \& lock_store \e%table, \*(Aqfile\*(Aq;
97 \& lock_nstore \e%table, \*(Aqfile\*(Aq;
98 \& $hashref = lock_retrieve(\*(Aqfile\*(Aq);
99 .Ve
100 .SH DESCRIPTION
101 .IX Header "DESCRIPTION"
102 The Storable package brings persistence to your Perl data structures
103 containing SCALAR, ARRAY, HASH or REF objects, i.e. anything that can be
104 conveniently stored to disk and retrieved at a later time.
105 .PP
106 It can be used in the regular procedural way by calling \f(CW\*(C`store\*(C'\fR with
107 a reference to the object to be stored, along with the file name where
108 the image should be written.
109 .PP
110 The routine returns \f(CW\*(C`undef\*(C'\fR for I/O problems or other internal error,
111 a true value otherwise. Serious errors are propagated as a \f(CW\*(C`die\*(C'\fR exception.
112 .PP
113 To retrieve data stored to disk, use \f(CW\*(C`retrieve\*(C'\fR with a file name.
114 The objects stored into that file are recreated into memory for you,
115 and a \fIreference\fR to the root object is returned. In case an I/O error
116 occurs while reading, \f(CW\*(C`undef\*(C'\fR is returned instead. Other serious
117 errors are propagated via \f(CW\*(C`die\*(C'\fR.
118 .PP
119 Since storage is performed recursively, you might want to stuff references
120 to objects that share a lot of common data into a single array or hash
121 table, and then store that object. That way, when you retrieve back the
122 whole thing, the objects will continue to share what they originally shared.
123 .PP
124 At the cost of a slight header overhead, you may store to an already
125 opened file descriptor using the \f(CW\*(C`store_fd\*(C'\fR routine, and retrieve
126 from a file via \f(CW\*(C`fd_retrieve\*(C'\fR. Those names aren\*(Aqt imported by default,
127 so you will have to do that explicitly if you need those routines.
128 The file descriptor you supply must be already opened, for read
129 if you\*(Aqre going to retrieve and for write if you wish to store.
130 .PP
131 .Vb 2
132 \& store_fd(\e%table, *STDOUT) || die "can\*(Aqt store to stdout\en";
133 \& $hashref = fd_retrieve(*STDIN);
134 .Ve
135 .PP
136 You can also store data in network order to allow easy sharing across
137 multiple platforms, or when storing on a socket known to be remotely
138 connected. The routines to call have an initial \f(CW\*(C`n\*(C'\fR prefix for \fInetwork\fR,
139 as in \f(CW\*(C`nstore\*(C'\fR and \f(CW\*(C`nstore_fd\*(C'\fR. At retrieval time, your data will be
140 correctly restored so you don\*(Aqt have to know whether you\*(Aqre restoring
141 from native or network ordered data. Double values are stored stringified
142 to ensure portability as well, at the slight risk of loosing some precision
143 in the last decimals.
144 .PP
145 When using \f(CW\*(C`fd_retrieve\*(C'\fR, objects are retrieved in sequence, one
146 object (i.e. one recursive tree) per associated \f(CW\*(C`store_fd\*(C'\fR.
147 .PP
148 If you\*(Aqre more from the object\-oriented camp, you can inherit from
149 Storable and directly store your objects by invoking \f(CW\*(C`store\*(C'\fR as
150 a method. The fact that the root of the to\-be\-stored tree is a
151 blessed reference (i.e. an object) is special\-cased so that the
152 retrieve does not provide a reference to that object but rather the
153 blessed object reference itself. (Otherwise, you\*(Aqd get a reference
154 to that blessed object).
155 .SH "MEMORY STORE"
156 .IX Header "MEMORY STORE"
157 The Storable engine can also store data into a Perl scalar instead, to
158 later retrieve them. This is mainly used to freeze a complex structure in
159 some safe compact memory place (where it can possibly be sent to another
160 process via some IPC, since freezing the structure also serializes it in
161 effect). Later on, and maybe somewhere else, you can thaw the Perl scalar
162 out and recreate the original complex structure in memory.
163 .PP
164 Surprisingly, the routines to be called are named \f(CW\*(C`freeze\*(C'\fR and \f(CW\*(C`thaw\*(C'\fR.
165 If you wish to send out the frozen scalar to another machine, use
166 \&\f(CW\*(C`nfreeze\*(C'\fR instead to get a portable image.
167 .PP
168 Note that freezing an object structure and immediately thawing it
169 actually achieves a deep cloning of that structure:
170 .PP
171 .Vb 1
172 \& dclone(.) = thaw(freeze(.))
173 .Ve
174 .PP
175 Storable provides you with a \f(CW\*(C`dclone\*(C'\fR interface which does not create
176 that intermediary scalar but instead freezes the structure in some
177 internal memory space and then immediately thaws it out.
178 .SH "ADVISORY LOCKING"
179 .IX Header "ADVISORY LOCKING"
180 The \f(CW\*(C`lock_store\*(C'\fR and \f(CW\*(C`lock_nstore\*(C'\fR routine are equivalent to
181 \&\f(CW\*(C`store\*(C'\fR and \f(CW\*(C`nstore\*(C'\fR, except that they get an exclusive lock on
182 the file before writing. Likewise, \f(CW\*(C`lock_retrieve\*(C'\fR does the same
183 as \f(CW\*(C`retrieve\*(C'\fR, but also gets a shared lock on the file before reading.
184 .PP
185 As with any advisory locking scheme, the protection only works if you
186 systematically use \f(CW\*(C`lock_store\*(C'\fR and \f(CW\*(C`lock_retrieve\*(C'\fR. If one side of
187 your application uses \f(CW\*(C`store\*(C'\fR whilst the other uses \f(CW\*(C`lock_retrieve\*(C'\fR,
188 you will get no protection at all.
189 .PP
190 The internal advisory locking is implemented using Perl\*(Aqs \fBflock()\fR
191 routine. If your system does not support any form of \fBflock()\fR, or if
192 you share your files across NFS, you might wish to use other forms
193 of locking by using modules such as LockFile::Simple which lock a
194 file using a filesystem entry, instead of locking the file descriptor.
195 .SH SPEED
196 .IX Header "SPEED"
197 The heart of Storable is written in C for decent speed. Extra low\-level
198 optimizations have been made when manipulating perl internals, to
199 sacrifice encapsulation for the benefit of greater speed.
200 .SH "CANONICAL REPRESENTATION"
201 .IX Header "CANONICAL REPRESENTATION"
202 Normally, Storable stores elements of hashes in the order they are
203 stored internally by Perl, i.e. pseudo\-randomly. If you set
204 \&\f(CW$Storable::canonical\fR to some \f(CW\*(C`TRUE\*(C'\fR value, Storable will store
205 hashes with the elements sorted by their key. This allows you to
206 compare data structures by comparing their frozen representations (or
207 even the compressed frozen representations), which can be useful for
208 creating lookup tables for complicated queries.
209 .PP
210 Canonical order does not imply network order; those are two orthogonal
211 settings.
212 .SH "CODE REFERENCES"
213 .IX Header "CODE REFERENCES"
214 Since Storable version 2.05, CODE references may be serialized with
215 the help of B::Deparse. To enable this feature, set
216 \&\f(CW$Storable::Deparse\fR to a true value. To enable deserialization,
217 \&\f(CW$Storable::Eval\fR should be set to a true value. Be aware that
218 deserialization is done through \f(CW\*(C`eval\*(C'\fR, which is dangerous if the
219 Storable file contains malicious data. You can set \f(CW$Storable::Eval\fR
220 to a subroutine reference which would be used instead of \f(CW\*(C`eval\*(C'\fR. See
221 below for an example using a Safe compartment for deserialization
222 of CODE references.
223 .PP
224 If \f(CW$Storable::Deparse\fR and/or \f(CW$Storable::Eval\fR are set to false
225 values, then the value of \f(CW$Storable::forgive_me\fR (see below) is
226 respected while serializing and deserializing.
227 .SH "FORWARD COMPATIBILITY"
228 .IX Header "FORWARD COMPATIBILITY"
229 This release of Storable can be used on a newer version of Perl to
230 serialize data which is not supported by earlier Perls. By default,
231 Storable will attempt to do the right thing, by \f(CWcroak()\fRing if it
232 encounters data that it cannot deserialize. However, the defaults
233 can be changed as follows:
234 .IP "utf8 data" 4
235 .IX Item "utf8 data"
236 Perl 5.6 added support for Unicode characters with code points > 255,
237 and Perl 5.8 has full support for Unicode characters in hash keys.
238 Perl internally encodes strings with these characters using utf8, and
239 Storable serializes them as utf8. By default, if an older version of
240 Perl encounters a utf8 value it cannot represent, it will \f(CWcroak()\fR.
241 To change this behaviour so that Storable deserializes utf8 encoded
242 values as the string of bytes (effectively dropping the \fIis_utf8\fR flag)
243 set \f(CW$Storable::drop_utf8\fR to some \f(CW\*(C`TRUE\*(C'\fR value. This is a form of
244 data loss, because with \f(CW$drop_utf8\fR true, it becomes impossible to tell
245 whether the original data was the Unicode string, or a series of bytes
246 that happen to be valid utf8.
247 .IP "restricted hashes" 4
248 .IX Item "restricted hashes"
249 Perl 5.8 adds support for restricted hashes, which have keys
250 restricted to a given set, and can have values locked to be read only.
251 By default, when Storable encounters a restricted hash on a perl
252 that doesn\*(Aqt support them, it will deserialize it as a normal hash,
253 silently discarding any placeholder keys and leaving the keys and
254 all values unlocked. To make Storable \f(CWcroak()\fR instead, set
255 \&\f(CW$Storable::downgrade_restricted\fR to a \f(CW\*(C`FALSE\*(C'\fR value. To restore
256 the default set it back to some \f(CW\*(C`TRUE\*(C'\fR value.
257 .Sp
258 The cperl PERL_PERTURB_KEYS_TOP hash strategy has a known problem with
259 restricted hashes.
260 .IP "huge objects" 4
261 .IX Item "huge objects"
262 On 64bit systems some data structures may exceed the 2G (i.e. I32_MAX)
263 limit. On 32bit systems also strings between I32 and U32 (2G\-4G).
264 Since Storable 3.00 (not in perl5 core) we are able to store and
265 retrieve these objects, even if perl5 itself is not able to handle
266 them. These are strings longer then 4G, arrays with more then 2G
267 elements and hashes with more then 2G elements. cperl forbids hashes
268 with more than 2G elements, but this fail in cperl then. perl5 itself
269 at least until 5.26 allows it, but cannot iterate over them.
270 Note that creating those objects might cause out of memory
271 exceptions by the operating system before perl has a chance to abort.
272 .IP "files from future versions of Storable" 4
273 .IX Item "files from future versions of Storable"
274 Earlier versions of Storable would immediately croak if they encountered
275 a file with a higher internal version number than the reading Storable
276 knew about. Internal version numbers are increased each time new data
277 types (such as restricted hashes) are added to the vocabulary of the file
278 format. This meant that a newer Storable module had no way of writing a
279 file readable by an older Storable, even if the writer didn\*(Aqt store newer
280 data types.
281 .Sp
282 This version of Storable will defer croaking until it encounters a data
283 type in the file that it does not recognize. This means that it will
284 continue to read files generated by newer Storable modules which are careful
285 in what they write out, making it easier to upgrade Storable modules in a
286 mixed environment.
287 .Sp
288 The old behaviour of immediate croaking can be re\-instated by setting
289 \&\f(CW$Storable::accept_future_minor\fR to some \f(CW\*(C`FALSE\*(C'\fR value.
290 .PP
291 All these variables have no effect on a newer Perl which supports the
292 relevant feature.
293 .SH "ERROR REPORTING"
294 .IX Header "ERROR REPORTING"
295 Storable uses the "exception" paradigm, in that it does not try to
296 workaround failures: if something bad happens, an exception is
297 generated from the caller\*(Aqs perspective (see Carp and \f(CWcroak()\fR).
298 Use eval {} to trap those exceptions.
299 .PP
300 When Storable croaks, it tries to report the error via the \f(CWlogcroak()\fR
301 routine from the \f(CW\*(C`Log::Agent\*(C'\fR package, if it is available.
302 .PP
303 Normal errors are reported by having \fBstore()\fR or \fBretrieve()\fR return \f(CW\*(C`undef\*(C'\fR.
304 Such errors are usually I/O errors (or truncated stream errors at retrieval).
305 .PP
306 When Storable throws the "Max. recursion depth with nested structures
307 exceeded" error we are already out of stack space. Unfortunately on
308 some earlier perl versions cleaning up a recursive data structure
309 recurses into the free calls, which will lead to stack overflows in
310 the cleanup. This data structure is not properly cleaned up then, it
311 will only be destroyed during global destruction.
312 .SH "WIZARDS ONLY"
313 .IX Header "WIZARDS ONLY"
314 .SS Hooks
315 .IX Subsection "Hooks"
316 Any class may define hooks that will be called during the serialization
317 and deserialization process on objects that are instances of that class.
318 Those hooks can redefine the way serialization is performed (and therefore,
319 how the symmetrical deserialization should be conducted).
320 .PP
321 Since we said earlier:
322 .PP
323 .Vb 1
324 \& dclone(.) = thaw(freeze(.))
325 .Ve
326 .PP
327 everything we say about hooks should also hold for deep cloning. However,
328 hooks get to know whether the operation is a mere serialization, or a cloning.
329 .PP
330 Therefore, when serializing hooks are involved,
331 .PP
332 .Vb 1
333 \& dclone(.) <> thaw(freeze(.))
334 .Ve
335 .PP
336 Well, you could keep them in sync, but there\*(Aqs no guarantee it will always
337 hold on classes somebody else wrote. Besides, there is little to gain in
338 doing so: a serializing hook could keep only one attribute of an object,
339 which is probably not what should happen during a deep cloning of that
340 same object.
341 .PP
342 Here is the hooking interface:
343 .ie n .IP """STORABLE_freeze"" \fIobj\fR, \fIcloning\fR" 4
344 .el .IP "\f(CWSTORABLE_freeze\fR \fIobj\fR, \fIcloning\fR" 4
345 .IX Item "STORABLE_freeze obj, cloning"
346 The serializing hook, called on the object during serialization. It can be
347 inherited, or defined in the class itself, like any other method.
348 .Sp
349 Arguments: \fIobj\fR is the object to serialize, \fIcloning\fR is a flag indicating
350 whether we\*(Aqre in a \fBdclone()\fR or a regular serialization via \fBstore()\fR or \fBfreeze()\fR.
351 .Sp
352 Returned value: A LIST \f(CW\*(C`($serialized, $ref1, $ref2, ...)\*(C'\fR where \f(CW$serialized\fR
353 is the serialized form to be used, and the optional \f(CW$ref1\fR, \f(CW$ref2\fR, etc... are
354 extra references that you wish to let the Storable engine serialize.
355 .Sp
356 At deserialization time, you will be given back the same LIST, but all the
357 extra references will be pointing into the deserialized structure.
358 .Sp
359 The \fBfirst time\fR the hook is hit in a serialization flow, you may have it
360 return an empty list. That will signal the Storable engine to further
361 discard that hook for this class and to therefore revert to the default
362 serialization of the underlying Perl data. The hook will again be normally
363 processed in the next serialization.
364 .Sp
365 Unless you know better, serializing hook should always say:
366 .Sp
367 .Vb 5
368 \& sub STORABLE_freeze {
369 \& my ($self, $cloning) = @_;
370 \& return if $cloning; # Regular default serialization
371 \& ....
372 \& }
373 .Ve
374 .Sp
375 in order to keep reasonable \fBdclone()\fR semantics.
376 .ie n .IP """STORABLE_thaw"" \fIobj\fR, \fIcloning\fR, \fIserialized\fR, ..." 4
377 .el .IP "\f(CWSTORABLE_thaw\fR \fIobj\fR, \fIcloning\fR, \fIserialized\fR, ..." 4
378 .IX Item "STORABLE_thaw obj, cloning, serialized, ..."
379 The deserializing hook called on the object during deserialization.
380 But wait: if we\*(Aqre deserializing, there\*(Aqs no object yet... right?
381 .Sp
382 Wrong: the Storable engine creates an empty one for you. If you know Eiffel,
383 you can view \f(CW\*(C`STORABLE_thaw\*(C'\fR as an alternate creation routine.
384 .Sp
385 This means the hook can be inherited like any other method, and that
386 \&\fIobj\fR is your blessed reference for this particular instance.
387 .Sp
388 The other arguments should look familiar if you know \f(CW\*(C`STORABLE_freeze\*(C'\fR:
389 \&\fIcloning\fR is true when we\*(Aqre part of a deep clone operation, \fIserialized\fR
390 is the serialized string you returned to the engine in \f(CW\*(C`STORABLE_freeze\*(C'\fR,
391 and there may be an optional list of references, in the same order you gave
392 them at serialization time, pointing to the deserialized objects (which
393 have been processed courtesy of the Storable engine).
394 .Sp
395 When the Storable engine does not find any \f(CW\*(C`STORABLE_thaw\*(C'\fR hook routine,
396 it tries to load the class by requiring the package dynamically (using
397 the blessed package name), and then re\-attempts the lookup. If at that
398 time the hook cannot be located, the engine croaks. Note that this mechanism
399 will fail if you define several classes in the same file, but perlmod
400 warned you.
401 .Sp
402 It is up to you to use this information to populate \fIobj\fR the way you want.
403 .Sp
404 Returned value: none.
405 .ie n .IP """STORABLE_attach"" \fIclass\fR, \fIcloning\fR, \fIserialized\fR" 4
406 .el .IP "\f(CWSTORABLE_attach\fR \fIclass\fR, \fIcloning\fR, \fIserialized\fR" 4
407 .IX Item "STORABLE_attach class, cloning, serialized"
408 While \f(CW\*(C`STORABLE_freeze\*(C'\fR and \f(CW\*(C`STORABLE_thaw\*(C'\fR are useful for classes where
409 each instance is independent, this mechanism has difficulty (or is
410 incompatible) with objects that exist as common process\-level or
411 system\-level resources, such as singleton objects, database pools, caches
412 or memoized objects.
413 .Sp
414 The alternative \f(CW\*(C`STORABLE_attach\*(C'\fR method provides a solution for these
415 shared objects. Instead of \f(CW\*(C`STORABLE_freeze\*(C'\fR \-\-> \f(CW\*(C`STORABLE_thaw\*(C'\fR,
416 you implement \f(CW\*(C`STORABLE_freeze\*(C'\fR \-\-> \f(CW\*(C`STORABLE_attach\*(C'\fR instead.
417 .Sp
418 Arguments: \fIclass\fR is the class we are attaching to, \fIcloning\fR is a flag
419 indicating whether we\*(Aqre in a \fBdclone()\fR or a regular de\-serialization via
420 \&\fBthaw()\fR, and \fIserialized\fR is the stored string for the resource object.
421 .Sp
422 Because these resource objects are considered to be owned by the entire
423 process/system, and not the "property" of whatever is being serialized,
424 no references underneath the object should be included in the serialized
425 string. Thus, in any class that implements \f(CW\*(C`STORABLE_attach\*(C'\fR, the
426 \&\f(CW\*(C`STORABLE_freeze\*(C'\fR method cannot return any references, and \f(CW\*(C`Storable\*(C'\fR
427 will throw an error if \f(CW\*(C`STORABLE_freeze\*(C'\fR tries to return references.
428 .Sp
429 All information required to "attach" back to the shared resource object
430 \&\fBmust\fR be contained \fBonly\fR in the \f(CW\*(C`STORABLE_freeze\*(C'\fR return string.
431 Otherwise, \f(CW\*(C`STORABLE_freeze\*(C'\fR behaves as normal for \f(CW\*(C`STORABLE_attach\*(C'\fR
432 classes.
433 .Sp
434 Because \f(CW\*(C`STORABLE_attach\*(C'\fR is passed the class (rather than an object),
435 it also returns the object directly, rather than modifying the passed
436 object.
437 .Sp
438 Returned value: object of type \f(CW\*(C`class\*(C'\fR
439 .SS Predicates
440 .IX Subsection "Predicates"
441 Predicates are not exportable. They must be called by explicitly prefixing
442 them with the Storable package name.
443 .ie n .IP """Storable::last_op_in_netorder""" 4
444 .el .IP \f(CWStorable::last_op_in_netorder\fR 4
445 .IX Item "Storable::last_op_in_netorder"
446 The \f(CWStorable::last_op_in_netorder()\fR predicate will tell you whether
447 network order was used in the last store or retrieve operation. If you
448 don\*(Aqt know how to use this, just forget about it.
449 .ie n .IP """Storable::is_storing""" 4
450 .el .IP \f(CWStorable::is_storing\fR 4
451 .IX Item "Storable::is_storing"
452 Returns true if within a store operation (via STORABLE_freeze hook).
453 .ie n .IP """Storable::is_retrieving""" 4
454 .el .IP \f(CWStorable::is_retrieving\fR 4
455 .IX Item "Storable::is_retrieving"
456 Returns true if within a retrieve operation (via STORABLE_thaw hook).
457 .SS Recursion
458 .IX Subsection "Recursion"
459 With hooks comes the ability to recurse back to the Storable engine.
460 Indeed, hooks are regular Perl code, and Storable is convenient when
461 it comes to serializing and deserializing things, so why not use it
462 to handle the serialization string?
463 .PP
464 There are a few things you need to know, however:
465 .IP \(bu 4
466 From Storable 3.05 to 3.13 we probed for the stack recursion limit for references,
467 arrays and hashes to a maximal depth of ~1200\-35000, otherwise we might
468 fall into a stack\-overflow. On JSON::XS this limit is 512 btw. With
469 references not immediately referencing each other there\*(Aqs no such
470 limit yet, so you might fall into such a stack\-overflow segfault.
471 .Sp
472 This probing and the checks we performed have some limitations:
473 .RS 4
474 .IP \(bu 4
475 the stack size at build time might be different at run time, eg. the
476 stack size may have been modified with \fBulimit\fR\|(1). If it\*(Aqs larger at
477 run time Storable may fail the \fBfreeze()\fR or \fBthaw()\fR unnecessarily. If
478 it\*(Aqs larger at build time Storable may segmentation fault when
479 processing a deep structure at run time.
480 .IP \(bu 4
481 the stack size might be different in a thread.
482 .IP \(bu 4
483 array and hash recursion limits are checked separately against the
484 same recursion depth, a frozen structure with a large sequence of
485 nested arrays within many nested hashes may exhaust the processor
486 stack without triggering Storable\*(Aqs recursion protection.
487 .RE
488 .RS 4
489 .Sp
490 So these now have simple defaults rather than probing at build\-time.
491 .Sp
492 You can control the maximum array and hash recursion depths by
493 modifying \f(CW$Storable::recursion_limit\fR and
494 \&\f(CW$Storable::recursion_limit_hash\fR respectively. Either can be set to
495 \&\f(CW\-1\fR to prevent any depth checks, though this isn\*(Aqt recommended.
496 .Sp
497 If you want to test what the limits are, the \fIstacksize\fR tool is
498 included in the \f(CW\*(C`Storable\*(C'\fR distribution.
499 .RE
500 .IP \(bu 4
501 You can create endless loops if the things you serialize via \fBfreeze()\fR
502 (for instance) point back to the object we\*(Aqre trying to serialize in
503 the hook.
504 .IP \(bu 4
505 Shared references among objects will not stay shared: if we\*(Aqre serializing
506 the list of object [A, C] where both object A and C refer to the SAME object
507 B, and if there is a serializing hook in A that says freeze(B), then when
508 deserializing, we\*(Aqll get [A\*(Aq, C\*(Aq] where A\*(Aq refers to B\*(Aq, but C\*(Aq refers to D,
509 a deep clone of B\*(Aq. The topology was not preserved.
510 .IP \(bu 4
511 The maximal stack recursion limit for your system is returned by
512 \&\f(CWstack_depth()\fR and \f(CWstack_depth_hash()\fR. The hash limit is usually
513 half the size of the array and ref limit, as the Perl hash API is not optimal.
514 .PP
515 That\*(Aqs why \f(CW\*(C`STORABLE_freeze\*(C'\fR lets you provide a list of references
516 to serialize. The engine guarantees that those will be serialized in the
517 same context as the other objects, and therefore that shared objects will
518 stay shared.
519 .PP
520 In the above [A, C] example, the \f(CW\*(C`STORABLE_freeze\*(C'\fR hook could return:
521 .PP
522 .Vb 1
523 \& ("something", $self\->{B})
524 .Ve
525 .PP
526 and the B part would be serialized by the engine. In \f(CW\*(C`STORABLE_thaw\*(C'\fR, you
527 would get back the reference to the B\*(Aq object, deserialized for you.
528 .PP
529 Therefore, recursion should normally be avoided, but is nonetheless supported.
530 .SS "Deep Cloning"
531 .IX Subsection "Deep Cloning"
532 There is a Clone module available on CPAN which implements deep cloning
533 natively, i.e. without freezing to memory and thawing the result. It is
534 aimed to replace Storable\*(Aqs \fBdclone()\fR some day. However, it does not currently
535 support Storable hooks to redefine the way deep cloning is performed.
536 .SH "Storable magic"
537 .IX Header "Storable magic"
538 Yes, there\*(Aqs a lot of that :\-) But more precisely, in UNIX systems
539 there\*(Aqs a utility called \f(CW\*(C`file\*(C'\fR, which recognizes data files based on
540 their contents (usually their first few bytes). For this to work,
541 a certain file called \fImagic\fR needs to taught about the \fIsignature\fR
542 of the data. Where that configuration file lives depends on the UNIX
543 flavour; often it\*(Aqs something like \fI/usr/share/misc/magic\fR or
544 \&\fI/etc/magic\fR. Your system administrator needs to do the updating of
545 the \fImagic\fR file. The necessary signature information is output to
546 STDOUT by invoking \fBStorable::show_file_magic()\fR. Note that the GNU
547 implementation of the \f(CW\*(C`file\*(C'\fR utility, version 3.38 or later,
548 is expected to contain support for recognising Storable files
549 out\-of\-the\-box, in addition to other kinds of Perl files.
550 .PP
551 You can also use the following functions to extract the file header
552 information from Storable images:
553 .ie n .IP "$info = Storable::file_magic( $filename )" 4
554 .el .IP "\f(CW$info\fR = Storable::file_magic( \f(CW$filename\fR )" 4
555 .IX Item "$info = Storable::file_magic( $filename )"
556 If the given file is a Storable image return a hash describing it. If
557 the file is readable, but not a Storable image return \f(CW\*(C`undef\*(C'\fR. If
558 the file does not exist or is unreadable then croak.
559 .Sp
560 The hash returned has the following elements:
561 .RS 4
562 .ie n .IP """version""" 4
563 .el .IP \f(CWversion\fR 4
564 .IX Item "version"
565 This returns the file format version. It is a string like "2.7".
566 .Sp
567 Note that this version number is not the same as the version number of
568 the Storable module itself. For instance Storable v0.7 create files
569 in format v2.0 and Storable v2.15 create files in format v2.7. The
570 file format version number only increment when additional features
571 that would confuse older versions of the module are added.
572 .Sp
573 Files older than v2.0 will have the one of the version numbers "\-1",
574 "0" or "1". No minor number was used at that time.
575 .ie n .IP """version_nv""" 4
576 .el .IP \f(CWversion_nv\fR 4
577 .IX Item "version_nv"
578 This returns the file format version as number. It is a string like
579 "2.007". This value is suitable for numeric comparisons.
580 .Sp
581 The constant function \f(CW\*(C`Storable::BIN_VERSION_NV\*(C'\fR returns a comparable
582 number that represents the highest file version number that this
583 version of Storable fully supports (but see discussion of
584 \&\f(CW$Storable::accept_future_minor\fR above). The constant
585 \&\f(CW\*(C`Storable::BIN_WRITE_VERSION_NV\*(C'\fR function returns what file version
586 is written and might be less than \f(CW\*(C`Storable::BIN_VERSION_NV\*(C'\fR in some
587 configurations.
588 .ie n .IP """major"", ""minor""" 4
589 .el .IP "\f(CWmajor\fR, \f(CWminor\fR" 4
590 .IX Item "major, minor"
591 This also returns the file format version. If the version is "2.7"
592 then major would be 2 and minor would be 7. The minor element is
593 missing for when major is less than 2.
594 .ie n .IP """hdrsize""" 4
595 .el .IP \f(CWhdrsize\fR 4
596 .IX Item "hdrsize"
597 The is the number of bytes that the Storable header occupies.
598 .ie n .IP """netorder""" 4
599 .el .IP \f(CWnetorder\fR 4
600 .IX Item "netorder"
601 This is TRUE if the image store data in network order. This means
602 that it was created with \fBnstore()\fR or similar.
603 .ie n .IP """byteorder""" 4
604 .el .IP \f(CWbyteorder\fR 4
605 .IX Item "byteorder"
606 This is only present when \f(CW\*(C`netorder\*(C'\fR is FALSE. It is the
607 \&\f(CW$Config\fR{byteorder} string of the perl that created this image. It is
608 a string like "1234" (32 bit little endian) or "87654321" (64 bit big
609 endian). This must match the current perl for the image to be
610 readable by Storable.
611 .ie n .IP """intsize"", ""longsize"", ""ptrsize"", ""nvsize""" 4
612 .el .IP "\f(CWintsize\fR, \f(CWlongsize\fR, \f(CWptrsize\fR, \f(CWnvsize\fR" 4
613 .IX Item "intsize, longsize, ptrsize, nvsize"
614 These are only present when \f(CW\*(C`netorder\*(C'\fR is FALSE. These are the sizes of
615 various C datatypes of the perl that created this image. These must
616 match the current perl for the image to be readable by Storable.
617 .Sp
618 The \f(CW\*(C`nvsize\*(C'\fR element is only present for file format v2.2 and
619 higher.
620 .ie n .IP """file""" 4
621 .el .IP \f(CWfile\fR 4
622 .IX Item "file"
623 The name of the file.
624 .RE
625 .RS 4
626 .RE
627 .ie n .IP "$info = Storable::read_magic( $buffer )" 4
628 .el .IP "\f(CW$info\fR = Storable::read_magic( \f(CW$buffer\fR )" 4
629 .IX Item "$info = Storable::read_magic( $buffer )"
630 .PD 0
631 .ie n .IP "$info = Storable::read_magic( $buffer, $must_be_file )" 4
632 .el .IP "\f(CW$info\fR = Storable::read_magic( \f(CW$buffer\fR, \f(CW$must_be_file\fR )" 4
633 .IX Item "$info = Storable::read_magic( $buffer, $must_be_file )"
634 .PD
635 The \f(CW$buffer\fR should be a Storable image or the first few bytes of it.
636 If \f(CW$buffer\fR starts with a Storable header, then a hash describing the
637 image is returned, otherwise \f(CW\*(C`undef\*(C'\fR is returned.
638 .Sp
639 The hash has the same structure as the one returned by
640 \&\fBStorable::file_magic()\fR. The \f(CW\*(C`file\*(C'\fR element is true if the image is a
641 file image.
642 .Sp
643 If the \f(CW$must_be_file\fR argument is provided and is TRUE, then return
644 \&\f(CW\*(C`undef\*(C'\fR unless the image looks like it belongs to a file dump.
645 .Sp
646 The maximum size of a Storable header is currently 21 bytes. If the
647 provided \f(CW$buffer\fR is only the first part of a Storable image it should
648 at least be this long to ensure that \fBread_magic()\fR will recognize it as
649 such.
650 .SH EXAMPLES
651 .IX Header "EXAMPLES"
652 Here are some code samples showing a possible usage of Storable:
653 .PP
654 .Vb 1
655 \& use Storable qw(store retrieve freeze thaw dclone);
656 \&
657 \& %color = (\*(AqBlue\*(Aq => 0.1, \*(AqRed\*(Aq => 0.8, \*(AqBlack\*(Aq => 0, \*(AqWhite\*(Aq => 1);
658 \&
659 \& store(\e%color, \*(Aqmycolors\*(Aq) or die "Can\*(Aqt store %a in mycolors!\en";
660 \&
661 \& $colref = retrieve(\*(Aqmycolors\*(Aq);
662 \& die "Unable to retrieve from mycolors!\en" unless defined $colref;
663 \& printf "Blue is still %lf\en", $colref\->{\*(AqBlue\*(Aq};
664 \&
665 \& $colref2 = dclone(\e%color);
666 \&
667 \& $str = freeze(\e%color);
668 \& printf "Serialization of %%color is %d bytes long.\en", length($str);
669 \& $colref3 = thaw($str);
670 .Ve
671 .PP
672 which prints (on my machine):
673 .PP
674 .Vb 2
675 \& Blue is still 0.100000
676 \& Serialization of %color is 102 bytes long.
677 .Ve
678 .PP
679 Serialization of CODE references and deserialization in a safe
680 compartment:
681 .PP
682 .Vb 11
683 \& use Storable qw(freeze thaw);
684 \& use Safe;
685 \& use strict;
686 \& my $safe = Safe\->new;
687 \& # because of opcodes used in "use strict":
688 \& $safe\->permit(qw(:default require));
689 \& local $Storable::Deparse = 1;
690 \& local $Storable::Eval = sub { $safe\->reval($_[0]) };
691 \& my $serialized = freeze(sub { 42 });
692 \& my $code = thaw($serialized);
693 \& $code\->() == 42;
694 .Ve
695 .SH "SECURITY WARNING"
696 .IX Header "SECURITY WARNING"
697 \&\fBDo not accept Storable documents from untrusted sources!\fR There is
698 \&\fBno\fR way to configure Storable so that it can be used safely to process
699 untrusted data. While there \fIare\fR various options that can be used to
700 mitigate specific security issues these options do \fInot\fR comprise a
701 complete safety net for the user, and processing untrusted data may
702 result in segmentation faults, remote code execution, or privilege
703 escalation. The following lists some known features which represent
704 security issues that should be considered by users of this module.
705 .PP
706 Most obviously, the optional (off by default) CODE reference
707 serialization feature allows transfer of code to the deserializing
708 process. Furthermore, any serialized object will cause Storable to
709 helpfully load the module corresponding to the class of the object in
710 the deserializing module. For manipulated module names, this can load
711 almost arbitrary code. Finally, the deserialized object\*(Aqs destructors
712 will be invoked when the objects get destroyed in the deserializing
713 process. Maliciously crafted Storable documents may put such objects
714 in the value of a hash key that is overridden by another key/value
715 pair in the same hash, thus causing immediate destructor execution.
716 .PP
717 To disable blessing objects while thawing/retrieving remove the flag
718 \&\f(CW\*(C`BLESS_OK\*(C'\fR = 2 from \f(CW$Storable::flags\fR or set the 2nd argument for
719 thaw/retrieve to 0.
720 .PP
721 To disable tieing data while thawing/retrieving remove the flag \f(CW\*(C`TIE_OK\*(C'\fR
722 = 4 from \f(CW$Storable::flags\fR or set the 2nd argument for thaw/retrieve
723 to 0.
724 .PP
725 With the default setting of \f(CW$Storable::flags\fR = 6, creating or destroying
726 random objects, even renamed objects can be controlled by an attacker.
727 See CVE\-2015\-1592 and its metasploit module.
728 .PP
729 If your application requires accepting data from untrusted sources, you
730 are best off with a less powerful and more\-likely safe serialization
731 format and implementation. If your data is sufficiently simple,
732 Cpanel::JSON::XS or Data::MessagePack are fine alternatives. For
733 more complex data structures containing various Perl specific data types
734 like regular expressions or aliased data Sereal is the best
735 alternative and offers maximum interoperability. Note that Sereal is
736 unsafe by default, but you can configure
737 the encoder and decoder to mitigate any security issues.
738 .SH WARNING
739 .IX Header "WARNING"
740 If you\*(Aqre using references as keys within your hash tables, you\*(Aqre bound
741 to be disappointed when retrieving your data. Indeed, Perl stringifies
742 references used as hash table keys. If you later wish to access the
743 items via another reference stringification (i.e. using the same
744 reference that was used for the key originally to record the value into
745 the hash table), it will work because both references stringify to the
746 same string.
747 .PP
748 It won\*(Aqt work across a sequence of \f(CW\*(C`store\*(C'\fR and \f(CW\*(C`retrieve\*(C'\fR operations,
749 however, because the addresses in the retrieved objects, which are
750 part of the stringified references, will probably differ from the
751 original addresses. The topology of your structure is preserved,
752 but not hidden semantics like those.
753 .PP
754 On platforms where it matters, be sure to call \f(CWbinmode()\fR on the
755 descriptors that you pass to Storable functions.
756 .PP
757 Storing data canonically that contains large hashes can be
758 significantly slower than storing the same data normally, as
759 temporary arrays to hold the keys for each hash have to be allocated,
760 populated, sorted and freed. Some tests have shown a halving of the
761 speed of storing \-\- the exact penalty will depend on the complexity of
762 your data. There is no slowdown on retrieval.
763 .SH "REGULAR EXPRESSIONS"
764 .IX Header "REGULAR EXPRESSIONS"
765 Storable now has experimental support for storing regular expressions,
766 but there are significant limitations:
767 .IP \(bu 4
768 perl 5.8 or later is required.
769 .IP \(bu 4
770 regular expressions with code blocks, ie \f(CW\*(C`/(?{ ... })/\*(C'\fR or \f(CW\*(C`/(??{
771 \&... })/\*(C'\fR will throw an exception when thawed.
772 .IP \(bu 4
773 regular expression syntax and flags have changed over the history of
774 perl, so a regular expression that you freeze in one version of perl
775 may fail to thaw or behave differently in another version of perl.
776 .IP \(bu 4
777 depending on the version of perl, regular expressions can change in
778 behaviour depending on the context, but later perls will bake that
779 behaviour into the regexp.
780 .PP
781 Storable will throw an exception if a frozen regular expression cannot
782 be thawed.
783 .SH BUGS
784 .IX Header "BUGS"
785 You can\*(Aqt store GLOB, FORMLINE, etc.... If you can define semantics
786 for those operations, feel free to enhance Storable so that it can
787 deal with them.
788 .PP
789 The store functions will \f(CW\*(C`croak\*(C'\fR if they run into such references
790 unless you set \f(CW$Storable::forgive_me\fR to some \f(CW\*(C`TRUE\*(C'\fR value. In that
791 case, the fatal message is converted to a warning and some meaningless
792 string is stored instead.
793 .PP
794 Setting \f(CW$Storable::canonical\fR may not yield frozen strings that
795 compare equal due to possible stringification of numbers. When the
796 string version of a scalar exists, it is the form stored; therefore,
797 if you happen to use your numbers as strings between two freezing
798 operations on the same data structures, you will get different
799 results.
800 .PP
801 When storing doubles in network order, their value is stored as text.
802 However, you should also not expect non\-numeric floating\-point values
803 such as infinity and "not a number" to pass successfully through a
804 \&\fBnstore()\fR/\fBretrieve()\fR pair.
805 .PP
806 As Storable neither knows nor cares about character sets (although it
807 does know that characters may be more than eight bits wide), any difference
808 in the interpretation of character codes between a host and a target
809 system is your problem. In particular, if host and target use different
810 code points to represent the characters used in the text representation
811 of floating\-point numbers, you will not be able be able to exchange
812 floating\-point data, even with \fBnstore()\fR.
813 .PP
814 \&\f(CW\*(C`Storable::drop_utf8\*(C'\fR is a blunt tool. There is no facility either to
815 return \fBall\fR strings as utf8 sequences, or to attempt to convert utf8
816 data back to 8 bit and \f(CWcroak()\fR if the conversion fails.
817 .PP
818 Prior to Storable 2.01, no distinction was made between signed and
819 unsigned integers on storing. By default Storable prefers to store a
820 scalars string representation (if it has one) so this would only cause
821 problems when storing large unsigned integers that had never been converted
822 to string or floating point. In other words values that had been generated
823 by integer operations such as logic ops and then not used in any string or
824 arithmetic context before storing.
825 .SS "64 bit data in perl 5.6.0 and 5.6.1"
826 .IX Subsection "64 bit data in perl 5.6.0 and 5.6.1"
827 This section only applies to you if you have existing data written out
828 by Storable 2.02 or earlier on perl 5.6.0 or 5.6.1 on Unix or Linux which
829 has been configured with 64 bit integer support (not the default)
830 If you got a precompiled perl, rather than running Configure to build
831 your own perl from source, then it almost certainly does not affect you,
832 and you can stop reading now (unless you\*(Aqre curious). If you\*(Aqre using perl
833 on Windows it does not affect you.
834 .PP
835 Storable writes a file header which contains the sizes of various C
836 language types for the C compiler that built Storable (when not writing in
837 network order), and will refuse to load files written by a Storable not
838 on the same (or compatible) architecture. This check and a check on
839 machine byteorder is needed because the size of various fields in the file
840 are given by the sizes of the C language types, and so files written on
841 different architectures are incompatible. This is done for increased speed.
842 (When writing in network order, all fields are written out as standard
843 lengths, which allows full interworking, but takes longer to read and write)
844 .PP
845 Perl 5.6.x introduced the ability to optional configure the perl interpreter
846 to use C\*(Aqs \f(CW\*(C`long long\*(C'\fR type to allow scalars to store 64 bit integers on 32
847 bit systems. However, due to the way the Perl configuration system
848 generated the C configuration files on non\-Windows platforms, and the way
849 Storable generates its header, nothing in the Storable file header reflected
850 whether the perl writing was using 32 or 64 bit integers, despite the fact
851 that Storable was storing some data differently in the file. Hence Storable
852 running on perl with 64 bit integers will read the header from a file
853 written by a 32 bit perl, not realise that the data is actually in a subtly
854 incompatible format, and then go horribly wrong (possibly crashing) if it
855 encountered a stored integer. This is a design failure.
856 .PP
857 Storable has now been changed to write out and read in a file header with
858 information about the size of integers. It\*(Aqs impossible to detect whether
859 an old file being read in was written with 32 or 64 bit integers (they have
860 the same header) so it\*(Aqs impossible to automatically switch to a correct
861 backwards compatibility mode. Hence this Storable defaults to the new,
862 correct behaviour.
863 .PP
864 What this means is that if you have data written by Storable 1.x running
865 on perl 5.6.0 or 5.6.1 configured with 64 bit integers on Unix or Linux
866 then by default this Storable will refuse to read it, giving the error
867 \&\fIByte order is not compatible\fR. If you have such data then you
868 should set \f(CW$Storable::interwork_56_64bit\fR to a true value to make this
869 Storable read and write files with the old header. You should also
870 migrate your data, or any older perl you are communicating with, to this
871 current version of Storable.
872 .PP
873 If you don\*(Aqt have data written with specific configuration of perl described
874 above, then you do not and should not do anything. Don\*(Aqt set the flag \-
875 not only will Storable on an identically configured perl refuse to load them,
876 but Storable a differently configured perl will load them believing them
877 to be correct for it, and then may well fail or crash part way through
878 reading them.
879 .SH CREDITS
880 .IX Header "CREDITS"
881 Thank you to (in chronological order):
882 .PP
883 .Vb 10
884 \& Jarkko Hietaniemi <[email protected]>
885 \& Ulrich Pfeifer <[email protected]\-dortmund.de>
886 \& Benjamin A. Holzman <[email protected]>
887 \& Andrew Ford <A.Ford@ford\-mason.co.uk>
888 \& Gisle Aas <[email protected]>
889 \& Jeff Gresham <[email protected]>
890 \& Murray Nesbitt <[email protected]>
891 \& Marc Lehmann <[email protected]>
892 \& Justin Banks <[email protected]>
893 \& Jarkko Hietaniemi <[email protected]> (AGAIN, as perl 5.7.0 Pumpkin!)
894 \& Salvador Ortiz Garcia <[email protected]>
895 \& Dominic Dunlop <[email protected]>
896 \& Erik Haugan <[email protected]>
897 \& Benjamin A. Holzman <[email protected]>
898 \& Reini Urban <[email protected]>
899 \& Todd Rinaldo <[email protected]>
900 \& Aaron Crane <[email protected]>
901 .Ve
902 .PP
903 for their bug reports, suggestions and contributions.
904 .PP
905 Benjamin Holzman contributed the tied variable support, Andrew Ford
906 contributed the canonical order for hashes, and Gisle Aas fixed
907 a few misunderstandings of mine regarding the perl internals,
908 and optimized the emission of "tags" in the output streams by
909 simply counting the objects instead of tagging them (leading to
910 a binary incompatibility for the Storable image starting at version
911 0.6\-\-older images are, of course, still properly understood).
912 Murray Nesbitt made Storable thread\-safe. Marc Lehmann added overloading
913 and references to tied items support. Benjamin Holzman added a performance
914 improvement for overloaded classes; thanks to Grant Street Group for footing
915 the bill.
916 Reini Urban took over maintenance from p5p, and added security fixes
917 and huge object support.
918 .SH AUTHOR
919 .IX Header "AUTHOR"
920 Storable was written by Raphael Manfredi
921 \&\fI<[email protected]>\fR
922 Maintenance is now done by cperl <http://perl11.org/cperl>
923 .PP
924 Please e\-mail us with problems, bug fixes, comments and complaints,
925 although if you have compliments you should send them to Raphael.
926 Please don\*(Aqt e\-mail Raphael with problems, as he no longer works on
927 Storable, and your message will be delayed while he forwards it to us.
928 .SH "SEE ALSO"
929 .IX Header "SEE ALSO"
930 Clone.