Debian Codegraph @

lang/perl/BerkeleyDB/BerkeleyDB.pod

raw
1 =head1 NAME
2
3 BerkeleyDB - Perl extension for Berkeley DB version 2, 3, 4 or 5
4
5 =head1 SYNOPSIS
6
7 use BerkeleyDB;
8
9 $env = new BerkeleyDB::Env [OPTIONS] ;
10
11 $db = tie %hash, 'BerkeleyDB::Hash', [OPTIONS] ;
12 $db = new BerkeleyDB::Hash [OPTIONS] ;
13
14 $db = tie %hash, 'BerkeleyDB::Btree', [OPTIONS] ;
15 $db = new BerkeleyDB::Btree [OPTIONS] ;
16
17 $db = tie @array, 'BerkeleyDB::Recno', [OPTIONS] ;
18 $db = new BerkeleyDB::Recno [OPTIONS] ;
19
20 $db = tie @array, 'BerkeleyDB::Queue', [OPTIONS] ;
21 $db = new BerkeleyDB::Queue [OPTIONS] ;
22
23 $db = new BerkeleyDB::Heap [OPTIONS] ;
24
25 $db = new BerkeleyDB::Unknown [OPTIONS] ;
26
27 $status = BerkeleyDB::db_remove [OPTIONS]
28 $status = BerkeleyDB::db_rename [OPTIONS]
29 $status = BerkeleyDB::db_verify [OPTIONS]
30
31 $hash{$key} = $value ;
32 $value = $hash{$key} ;
33 each %hash ;
34 keys %hash ;
35 values %hash ;
36
37 $env = $db->Env()
38 $status = $db->db_get()
39 $status = $db->db_exists() ;
40 $status = $db->db_put() ;
41 $status = $db->db_del() ;
42 $status = $db->db_sync() ;
43 $status = $db->db_close() ;
44 $status = $db->db_pget()
45 $hash_ref = $db->db_stat() ;
46 $status = $db->db_key_range();
47 $type = $db->type() ;
48 $status = $db->status() ;
49 $boolean = $db->byteswapped() ;
50 $status = $db->truncate($count) ;
51 $status = $db->compact($start, $stop, $c_data, $flags, $end);
52
53 $bool = $env->cds_enabled();
54 $bool = $db->cds_enabled();
55 $lock = $db->cds_lock();
56 $lock->cds_unlock();
57
58 ($flag, $old_offset, $old_length) = $db->partial_set($offset, $length) ;
59 ($flag, $old_offset, $old_length) = $db->partial_clear() ;
60
61 $cursor = $db->db_cursor([$flags]) ;
62 $newcursor = $cursor->c_dup([$flags]);
63 $status = $cursor->c_get() ;
64 $status = $cursor->c_put() ;
65 $status = $cursor->c_del() ;
66 $status = $cursor->c_count() ;
67 $status = $cursor->c_pget() ;
68 $status = $cursor->status() ;
69 $status = $cursor->c_close() ;
70
71 $cursor = $db->db_join() ;
72 $status = $cursor->c_get() ;
73 $status = $cursor->c_close() ;
74
75 $status = $env->txn_checkpoint()
76 $hash_ref = $env->txn_stat()
77 $status = $env->set_mutexlocks()
78 $status = $env->set_flags()
79 $status = $env->set_timeout()
80 $status = $env->lock_detect()
81 $status = $env->lsn_reset()
82
83 $txn = $env->txn_begin() ;
84 $db->Txn($txn);
85 $txn->Txn($db1, $db2,...);
86 $status = $txn->txn_prepare()
87 $status = $txn->txn_commit()
88 $status = $txn->txn_abort()
89 $status = $txn->txn_id()
90 $status = $txn->txn_discard()
91 $status = $txn->set_timeout()
92
93 $status = $env->set_lg_dir();
94 $status = $env->set_lg_bsize();
95 $status = $env->set_lg_max();
96
97 $status = $env->set_data_dir() ;
98 $status = $env->set_tmp_dir() ;
99 $status = $env->set_verbose() ;
100 $db_env_ptr = $env->DB_ENV() ;
101
102 $BerkeleyDB::Error
103 $BerkeleyDB::db_version
104
105 # DBM Filters
106 $old_filter = $db->filter_store_key ( sub { ... } ) ;
107 $old_filter = $db->filter_store_value( sub { ... } ) ;
108 $old_filter = $db->filter_fetch_key ( sub { ... } ) ;
109 $old_filter = $db->filter_fetch_value( sub { ... } ) ;
110
111 # deprecated, but supported
112 $txn_mgr = $env->TxnMgr();
113 $status = $txn_mgr->txn_checkpoint()
114 $hash_ref = $txn_mgr->txn_stat()
115 $txn = $txn_mgr->txn_begin() ;
116
117 =head1 DESCRIPTION
118
119 B<NOTE: This document is still under construction. Expect it to be
120 incomplete in places.>
121
122 This Perl module provides an interface to most of the functionality
123 available in Berkeley DB versions 2, 3 and 4. In general it is safe to assume
124 that the interface provided here to be identical to the Berkeley DB
125 interface. The main changes have been to make the Berkeley DB API work
126 in a Perl way. Note that if you are using Berkeley DB 2.x, the new
127 features available in Berkeley DB 3.x or DB 4.x are not available via
128 this module.
129
130 The reader is expected to be familiar with the Berkeley DB
131 documentation. Where the interface provided here is identical to the
132 Berkeley DB library and the... TODO
133
134 The B<db_appinit>, B<db_cursor>, B<db_open> and B<db_txn> man pages are
135 particularly relevant.
136
137 The interface to Berkeley DB is implemented with a number of Perl
138 classes.
139
140 =head1 The BerkeleyDB::Env Class
141
142 The B<BerkeleyDB::Env> class provides an interface to the Berkeley DB
143 function B<db_appinit> in Berkeley DB 2.x or B<db_env_create> and
144 B<DBENV-E<gt>open> in Berkeley DB 3.x/4.x. Its purpose is to initialise a
145 number of sub-systems that can then be used in a consistent way in all
146 the databases you make use of in the environment.
147
148 If you don't intend using transactions, locking or logging, then you
149 shouldn't need to make use of B<BerkeleyDB::Env>.
150
151 Note that an environment consists of a number of files that Berkeley DB
152 manages behind the scenes for you. When you first use an environment, it
153 needs to be explicitly created. This is done by including C<DB_CREATE>
154 with the C<Flags> parameter, described below.
155
156 =head2 Synopsis
157
158 $env = new BerkeleyDB::Env
159 [ -Home => $path, ]
160 [ -Server => $name, ]
161 [ -CacheSize => $number, ]
162 [ -Config => { name => value, name => value }, ]
163 [ -ErrFile => filename, ]
164 [ -MsgFile => filename, ]
165 [ -ErrPrefix => "string", ]
166 [ -Flags => number, ]
167 [ -SetFlags => bitmask, ]
168 [ -LockDetect => number, ]
169 [ -TxMax => number, ]
170 [ -LogConfig => number, ]
171 [ -MaxLockers => number, ]
172 [ -MaxLocks => number, ]
173 [ -MaxObjects => number, ]
174 [ -SharedMemKey => number, ]
175 [ -Verbose => boolean, ]
176 [ -Encrypt => { Password => "string",
177 Flags => number }, ]
178
179 All the parameters to the BerkeleyDB::Env constructor are optional.
180
181 =over 5
182
183 =item -Home
184
185 If present, this parameter should point to an existing directory. Any
186 files that I<aren't> specified with an absolute path in the sub-systems
187 that are initialised by the BerkeleyDB::Env class will be assumed to
188 live in the B<Home> directory.
189
190 For example, in the code fragment below the database "fred.db" will be
191 opened in the directory "/home/databases" because it was specified as a
192 relative path, but "joe.db" will be opened in "/other" because it was
193 part of an absolute path.
194
195 $env = new BerkeleyDB::Env
196 -Home => "/home/databases"
197 ...
198
199 $db1 = new BerkeleyDB::Hash
200 -Filename => "fred.db",
201 -Env => $env
202 ...
203
204 $db2 = new BerkeleyDB::Hash
205 -Filename => "/other/joe.db",
206 -Env => $env
207 ...
208
209 =item -Server
210
211 If present, this parameter should be the hostname of a server that is running
212 the Berkeley DB RPC server. All databases will be accessed via the RPC server.
213
214 =item -Encrypt
215
216 If present, this parameter will enable encryption of all data before
217 it is written to the database. This parameters must be given a hash
218 reference. The format is shown below.
219
220 -Encrypt => { -Password => "abc", Flags => DB_ENCRYPT_AES }
221
222 Valid values for the Flags are 0 or C<DB_ENCRYPT_AES>.
223
224 This option requires Berkeley DB 4.1 or better.
225
226 =item -Cachesize
227
228 If present, this parameter sets the size of the environments shared memory
229 buffer pool.
230
231 =item -TxMax
232
233 If present, this parameter sets the number of simultaneous
234 transactions that are allowed. Default 100. This default is
235 definitely too low for programs using the MVCC capabilities.
236
237 =item -LogConfig
238
239 If present, this parameter is used to configure log options.
240
241 =item -MaxLockers
242
243 If present, this parameter is used to configure the maximum number of
244 processes doing locking on the database. Default 1000.
245
246 =item -MaxLocks
247
248 If present, this parameter is used to configure the maximum number of
249 locks on the database. Default 1000. This is often lower than required.
250
251 =item -MaxObjects
252
253 If present, this parameter is used to configure the maximum number of
254 locked objects. Default 1000. This is often lower than required.
255
256 =item -SharedMemKey
257
258 If present, this parameter sets the base segment ID for the shared memory
259 region used by Berkeley DB.
260
261 This option requires Berkeley DB 3.1 or better.
262
263 Use C<$env-E<gt>get_shm_key($id)> to find out the base segment ID used
264 once the environment is open.
265
266 =item -ThreadCount
267
268 If present, this parameter declares the approximate number of threads that
269 will be used in the database environment. This parameter is only necessary
270 when the $env->failchk method will be used. It does not actually set the
271 maximum number of threads but rather is used to determine memory sizing.
272
273 This option requires Berkeley DB 4.4 or better. It is only supported on
274 Unix/Linux.
275
276 =item -Config
277
278 This is a variation on the C<-Home> parameter, but it allows finer
279 control of where specific types of files will be stored.
280
281 The parameter expects a reference to a hash. Valid keys are:
282 B<DB_DATA_DIR>, B<DB_LOG_DIR> and B<DB_TMP_DIR>
283
284 The code below shows an example of how it can be used.
285
286 $env = new BerkeleyDB::Env
287 -Config => { DB_DATA_DIR => "/home/databases",
288 DB_LOG_DIR => "/home/logs",
289 DB_TMP_DIR => "/home/tmp"
290 }
291 ...
292
293 =item -ErrFile
294
295 Expects a filename or filenhandle. Any errors generated internally by
296 Berkeley DB will be logged to this file. A useful debug setting is to
297 open environments with either
298
299 -ErrFile => *STDOUT
300
301 or
302
303 -ErrFile => *STDERR
304
305 =item -ErrPrefix
306
307 Allows a prefix to be added to the error messages before they are sent
308 to B<-ErrFile>.
309
310 =item -Flags
311
312 The B<Flags> parameter specifies both which sub-systems to initialise,
313 as well as a number of environment-wide options.
314 See the Berkeley DB documentation for more details of these options.
315
316 Any of the following can be specified by OR'ing them:
317
318 B<DB_CREATE>
319
320 If any of the files specified do not already exist, create them.
321
322 B<DB_INIT_CDB>
323
324 Initialise the Concurrent Access Methods
325
326 B<DB_INIT_LOCK>
327
328 Initialise the Locking sub-system.
329
330 B<DB_INIT_LOG>
331
332 Initialise the Logging sub-system.
333
334 B<DB_INIT_MPOOL>
335
336 Initialize the shared memory buffer pool subsystem. This subsystem should be used whenever an application is using any Berkeley DB access method.
337
338 B<DB_INIT_TXN>
339
340 Initialize the transaction subsystem. This subsystem should be used when recovery and atomicity of multiple operations are important. The DB_INIT_TXN flag implies the DB_INIT_LOG flag.
341
342
343 B<DB_MPOOL_PRIVATE>
344
345 Create a private memory pool; see memp_open. Ignored unless DB_INIT_MPOOL is also specified.
346
347
348 B<DB_INIT_MPOOL> is also specified.
349
350
351 B<DB_NOMMAP>
352
353 Do not map this database into process memory.
354
355
356 B<DB_RECOVER>
357
358 Run normal recovery on this environment before opening it for normal use. If this flag is set, the DB_CREATE flag must also be set since the regions will be removed and recreated.
359
360 The db_appinit function returns successfully if DB_RECOVER is specified and no log files exist, so it is necessary to ensure all necessary log files are present before running recovery.
361
362
363 B<DB_PRIVATE>
364
365 B<DB_RECOVER_FATAL>
366
367 Run catastrophic recovery on this environment before opening it for normal use. If this flag is set, the DB_CREATE flag must also be set since the regions will be removed and recreated.
368
369 The db_appinit function returns successfully if DB_RECOVER_FATAL is specified and no log files exist, so it is necessary to ensure all necessary log files are present before running recovery.
370
371 B<DB_THREAD>
372
373 Ensure that handles returned by the Berkeley DB subsystems are useable by multiple threads within a single process, i.e., that the system is free-threaded.
374
375 B<DB_TXN_NOSYNC>
376
377 On transaction commit, do not synchronously flush the log; see txn_open. Ignored unless DB_INIT_TXN is also specified.
378
379 B<DB_USE_ENVIRON>
380
381 The Berkeley DB process' environment may be permitted to specify information to be used when naming files; see Berkeley DB File Naming. As permitting users to specify which files are used can create security problems, environment information will be used in file naming for all users only if the DB_USE_ENVIRON flag is set.
382
383 B<DB_USE_ENVIRON_ROOT>
384
385 The Berkeley DB process' environment may be permitted to specify information to be used when naming files; see Berkeley DB File Naming. As permitting users to specify which files are used can create security problems, if the DB_USE_ENVIRON_ROOT flag is set, environment information will be used for file naming only for users with a user-ID matching that of the superuser (specifically, users for whom the getuid(2) system call returns the user-ID 0).
386
387 =item -SetFlags
388
389 Calls ENV->set_flags with the supplied bitmask. Use this when you need to make
390 use of DB_ENV->set_flags before DB_ENV->open is called.
391
392 Only valid when Berkeley DB 3.x or better is used.
393
394 =item -LockDetect
395
396 Specifies what to do when a lock conflict occurs. The value should be one of
397
398 B<DB_LOCK_DEFAULT>
399
400 Use the default policy as specified by db_deadlock.
401
402 B<DB_LOCK_OLDEST>
403
404 Abort the oldest transaction.
405
406 B<DB_LOCK_RANDOM>
407
408 Abort a random transaction involved in the deadlock.
409
410 B<DB_LOCK_YOUNGEST>
411
412 Abort the youngest transaction.
413
414
415 =item -Verbose
416
417 Add extra debugging information to the messages sent to B<-ErrFile>.
418
419 =back
420
421 =head2 Methods
422
423 The environment class has the following methods:
424
425 =over 5
426
427 =item $env->errPrefix("string") ;
428
429 This method is identical to the B<-ErrPrefix> flag. It allows the
430 error prefix string to be changed dynamically.
431
432 =item $env->set_flags(bitmask, 1|0);
433
434 =item $txn = $env->TxnMgr()
435
436 Constructor for creating a B<TxnMgr> object.
437 See L<"TRANSACTIONS"> for more details of using transactions.
438
439 This method is deprecated. Access the transaction methods using the B<txn_>
440 methods below from the environment object directly.
441
442 =item $env->txn_begin()
443
444 TODO
445
446 =item $env->txn_stat()
447
448 TODO
449
450 =item $env->txn_checkpoint()
451
452 TODO
453
454 =item $env->status()
455
456 Returns the status of the last BerkeleyDB::Env method.
457
458
459 =item $env->DB_ENV()
460
461 Returns a pointer to the underlying DB_ENV data structure that Berkeley
462 DB uses.
463
464 =item $env->get_shm_key($id)
465
466 Writes the base segment ID for the shared memory region used by the
467 Berkeley DB environment into C<$id>. Returns 0 on success.
468
469 This option requires Berkeley DB 4.2 or better.
470
471 Use the C<-SharedMemKey> option when opening the environemt to set the
472 base segment ID.
473
474 =item $env->set_isalive()
475
476 Set the callback that determines if the thread of control, identified by
477 the pid and tid arguments, is still running. This method should only be
478 used in combination with $env->failchk.
479
480 This option requires Berkeley DB 4.4 or better.
481
482 =item $env->failchk($flags)
483
484 The $env->failchk method checks for threads of control (either a true
485 thread or a process) that have exited while manipulating Berkeley DB
486 library data structures, while holding a logical database lock, or with an
487 unresolved transaction (that is, a transaction that was never aborted or
488 committed).
489
490 If $env->failchk determines a thread of control exited while holding
491 database read locks, it will release those locks. If $env->failchk
492 determines a thread of control exited with an unresolved transaction, the
493 transaction will be aborted.
494
495 Applications calling the $env->failchk method must have already called the
496 $env->set_isalive method, on the same DB environement, and must have
497 configured their database environment using the -ThreadCount flag. The
498 ThreadCount flag cannot be used on an environment that wasn't previously
499 initialized with it.
500
501 This option requires Berkeley DB 4.4 or better.
502
503 =item $env->stat_print
504
505 Prints statistical information.
506
507 If the C<MsgFile> option is specified the output will be sent to the
508 file. Otherwise output is sent to standard output.
509
510 This option requires Berkeley DB 4.3 or better.
511
512 =item $env->lock_stat_print
513
514 Prints locking subsystem statistics.
515
516 If the C<MsgFile> option is specified the output will be sent to the
517 file. Otherwise output is sent to standard output.
518
519 This option requires Berkeley DB 4.3 or better.
520
521 =item $env->mutex_stat_print
522
523 Prints mutex subsystem statistics.
524
525 If the C<MsgFile> option is specified the output will be sent to the
526 file. Otherwise output is sent to standard output.
527
528 This option requires Berkeley DB 4.4 or better.
529
530
531 =item $env->set_timeout($timeout, $flags)
532
533 =item $env->status()
534
535 Returns the status of the last BerkeleyDB::Env method.
536
537 =back
538
539 =head2 Examples
540
541 TODO.
542
543 =head1 Global Classes
544
545 $status = BerkeleyDB::db_remove [OPTIONS]
546 $status = BerkeleyDB::db_rename [OPTIONS]
547 $status = BerkeleyDB::db_verify [OPTIONS]
548
549 =head1 THE DATABASE CLASSES
550
551 B<BerkeleyDB> supports the following database formats:
552
553 =over 5
554
555 =item B<BerkeleyDB::Hash>
556
557 This database type allows arbitrary key/value pairs to be stored in data
558 files. This is equivalent to the functionality provided by other
559 hashing packages like DBM, NDBM, ODBM, GDBM, and SDBM. Remember though,
560 the files created using B<BerkeleyDB::Hash> are not compatible with any
561 of the other packages mentioned.
562
563 A default hashing algorithm, which will be adequate for most applications,
564 is built into BerkeleyDB. If you do need to use your own hashing algorithm
565 it is possible to write your own in Perl and have B<BerkeleyDB> use
566 it instead.
567
568 =item B<BerkeleyDB::Btree>
569
570 The Btree format allows arbitrary key/value pairs to be stored in a
571 B+tree.
572
573 As with the B<BerkeleyDB::Hash> format, it is possible to provide a
574 user defined Perl routine to perform the comparison of keys. By default,
575 though, the keys are stored in lexical order.
576
577 =item B<BerkeleyDB::Recno>
578
579 TODO.
580
581
582 =item B<BerkeleyDB::Queue>
583
584 TODO.
585
586 =item B<BerkeleyDB::Heap>
587
588 TODO.
589
590 =item B<BerkeleyDB::Unknown>
591
592 This isn't a database format at all. It is used when you want to open an
593 existing Berkeley DB database without having to know what type is it.
594
595 =back
596
597
598 Each of the database formats described above is accessed via a
599 corresponding B<BerkeleyDB> class. These will be described in turn in
600 the next sections.
601
602 =head1 BerkeleyDB::Hash
603
604 Equivalent to calling B<db_open> with type B<DB_HASH> in Berkeley DB 2.x and
605 calling B<db_create> followed by B<DB-E<gt>open> with type B<DB_HASH> in
606 Berkeley DB 3.x or greater.
607
608 Two forms of constructor are supported:
609
610 $db = new BerkeleyDB::Hash
611 [ -Filename => "filename", ]
612 [ -Subname => "sub-database name", ]
613 [ -Flags => flags,]
614 [ -Property => flags,]
615 [ -Mode => number,]
616 [ -Cachesize => number,]
617 [ -Lorder => number,]
618 [ -Pagesize => number,]
619 [ -Env => $env,]
620 [ -Txn => $txn,]
621 [ -Encrypt => { Password => "string",
622 Flags => number }, ],
623 # BerkeleyDB::Hash specific
624 [ -Ffactor => number,]
625 [ -Nelem => number,]
626 [ -Hash => code reference,]
627 [ -DupCompare => code reference,]
628
629 and this
630
631 [$db =] tie %hash, 'BerkeleyDB::Hash',
632 [ -Filename => "filename", ]
633 [ -Subname => "sub-database name", ]
634 [ -Flags => flags,]
635 [ -Property => flags,]
636 [ -Mode => number,]
637 [ -Cachesize => number,]
638 [ -Lorder => number,]
639 [ -Pagesize => number,]
640 [ -Env => $env,]
641 [ -Txn => $txn,]
642 [ -Encrypt => { Password => "string",
643 Flags => number }, ],
644 # BerkeleyDB::Hash specific
645 [ -Ffactor => number,]
646 [ -Nelem => number,]
647 [ -Hash => code reference,]
648 [ -DupCompare => code reference,]
649
650
651 When the "tie" interface is used, reading from and writing to the database
652 is achieved via the tied hash. In this case the database operates like
653 a Perl associative array that happens to be stored on disk.
654
655 In addition to the high-level tied hash interface, it is possible to
656 make use of the underlying methods provided by Berkeley DB
657
658 =head2 Options
659
660 In addition to the standard set of options (see L<COMMON OPTIONS>)
661 B<BerkeleyDB::Hash> supports these options:
662
663 =over 5
664
665 =item -Property
666
667 Used to specify extra flags when opening a database. The following
668 flags may be specified by bitwise OR'ing together one or more of the
669 following values:
670
671 B<DB_DUP>
672
673 When creating a new database, this flag enables the storing of duplicate
674 keys in the database. If B<DB_DUPSORT> is not specified as well, the
675 duplicates are stored in the order they are created in the database.
676
677 B<DB_DUPSORT>
678
679 Enables the sorting of duplicate keys in the database. Ignored if
680 B<DB_DUP> isn't also specified.
681
682 =item -Ffactor
683
684 =item -Nelem
685
686 See the Berkeley DB documentation for details of these options.
687
688 =item -Hash
689
690 Allows you to provide a user defined hash function. If not specified,
691 a default hash function is used. Here is a template for a user-defined
692 hash function
693
694 sub hash
695 {
696 my ($data) = shift ;
697 ...
698 # return the hash value for $data
699 return $hash ;
700 }
701
702 tie %h, "BerkeleyDB::Hash",
703 -Filename => $filename,
704 -Hash => \&hash,
705 ...
706
707 See L<""> for an example.
708
709 =item -DupCompare
710
711 Used in conjunction with the B<DB_DUPOSRT> flag.
712
713 sub compare
714 {
715 my ($key, $key2) = @_ ;
716 ...
717 # return 0 if $key1 eq $key2
718 # -1 if $key1 lt $key2
719 # 1 if $key1 gt $key2
720 return (-1 , 0 or 1) ;
721 }
722
723 tie %h, "BerkeleyDB::Hash",
724 -Filename => $filename,
725 -Property => DB_DUP|DB_DUPSORT,
726 -DupCompare => \&compare,
727 ...
728
729 =back
730
731
732 =head2 Methods
733
734 B<BerkeleyDB::Hash> only supports the standard database methods.
735 See L<COMMON DATABASE METHODS>.
736
737 =head2 A Simple Tied Hash Example
738
739 use strict ;
740 use BerkeleyDB ;
741 use vars qw( %h $k $v ) ;
742
743 my $filename = "fruit" ;
744 unlink $filename ;
745 tie %h, "BerkeleyDB::Hash",
746 -Filename => $filename,
747 -Flags => DB_CREATE
748 or die "Cannot open file $filename: $! $BerkeleyDB::Error\n" ;
749
750 # Add a few key/value pairs to the file
751 $h{"apple"} = "red" ;
752 $h{"orange"} = "orange" ;
753 $h{"banana"} = "yellow" ;
754 $h{"tomato"} = "red" ;
755
756 # Check for existence of a key
757 print "Banana Exists\n\n" if $h{"banana"} ;
758
759 # Delete a key/value pair.
760 delete $h{"apple"} ;
761
762 # print the contents of the file
763 while (($k, $v) = each %h)
764 { print "$k -> $v\n" }
765
766 untie %h ;
767
768 here is the output:
769
770 Banana Exists
771
772 orange -> orange
773 tomato -> red
774 banana -> yellow
775
776 Note that the like ordinary associative arrays, the order of the keys
777 retrieved from a Hash database are in an apparently random order.
778
779 =head2 Another Simple Hash Example
780
781 Do the same as the previous example but not using tie.
782
783 use strict ;
784 use BerkeleyDB ;
785
786 my $filename = "fruit" ;
787 unlink $filename ;
788 my $db = new BerkeleyDB::Hash
789 -Filename => $filename,
790 -Flags => DB_CREATE
791 or die "Cannot open file $filename: $! $BerkeleyDB::Error\n" ;
792
793 # Add a few key/value pairs to the file
794 $db->db_put("apple", "red") ;
795 $db->db_put("orange", "orange") ;
796 $db->db_put("banana", "yellow") ;
797 $db->db_put("tomato", "red") ;
798
799 # Check for existence of a key
800 print "Banana Exists\n\n" if $db->db_get("banana", $v) == 0;
801
802 # Delete a key/value pair.
803 $db->db_del("apple") ;
804
805 # print the contents of the file
806 my ($k, $v) = ("", "") ;
807 my $cursor = $db->db_cursor() ;
808 while ($cursor->c_get($k, $v, DB_NEXT) == 0)
809 { print "$k -> $v\n" }
810
811 undef $cursor ;
812 undef $db ;
813
814 =head2 Duplicate keys
815
816 The code below is a variation on the examples above. This time the hash has
817 been inverted. The key this time is colour and the value is the fruit name.
818 The B<DB_DUP> flag has been specified to allow duplicates.
819
820 use strict ;
821 use BerkeleyDB ;
822
823 my $filename = "fruit" ;
824 unlink $filename ;
825 my $db = new BerkeleyDB::Hash
826 -Filename => $filename,
827 -Flags => DB_CREATE,
828 -Property => DB_DUP
829 or die "Cannot open file $filename: $! $BerkeleyDB::Error\n" ;
830
831 # Add a few key/value pairs to the file
832 $db->db_put("red", "apple") ;
833 $db->db_put("orange", "orange") ;
834 $db->db_put("green", "banana") ;
835 $db->db_put("yellow", "banana") ;
836 $db->db_put("red", "tomato") ;
837 $db->db_put("green", "apple") ;
838
839 # print the contents of the file
840 my ($k, $v) = ("", "") ;
841 my $cursor = $db->db_cursor() ;
842 while ($cursor->c_get($k, $v, DB_NEXT) == 0)
843 { print "$k -> $v\n" }
844
845 undef $cursor ;
846 undef $db ;
847
848 here is the output:
849
850 orange -> orange
851 yellow -> banana
852 red -> apple
853 red -> tomato
854 green -> banana
855 green -> apple
856
857 =head2 Sorting Duplicate Keys
858
859 In the previous example, when there were duplicate keys, the values are
860 sorted in the order they are stored in. The code below is
861 identical to the previous example except the B<DB_DUPSORT> flag is
862 specified.
863
864 use strict ;
865 use BerkeleyDB ;
866
867 my $filename = "fruit" ;
868 unlink $filename ;
869 my $db = new BerkeleyDB::Hash
870 -Filename => $filename,
871 -Flags => DB_CREATE,
872 -Property => DB_DUP | DB_DUPSORT
873 or die "Cannot open file $filename: $! $BerkeleyDB::Error\n" ;
874
875 # Add a few key/value pairs to the file
876 $db->db_put("red", "apple") ;
877 $db->db_put("orange", "orange") ;
878 $db->db_put("green", "banana") ;
879 $db->db_put("yellow", "banana") ;
880 $db->db_put("red", "tomato") ;
881 $db->db_put("green", "apple") ;
882
883 # print the contents of the file
884 my ($k, $v) = ("", "") ;
885 my $cursor = $db->db_cursor() ;
886 while ($cursor->c_get($k, $v, DB_NEXT) == 0)
887 { print "$k -> $v\n" }
888
889 undef $cursor ;
890 undef $db ;
891
892 Notice that in the output below the duplicate values are sorted.
893
894 orange -> orange
895 yellow -> banana
896 red -> apple
897 red -> tomato
898 green -> apple
899 green -> banana
900
901 =head2 Custom Sorting Duplicate Keys
902
903 Another variation
904
905 TODO
906
907 =head2 Changing the hash
908
909 TODO
910
911 =head2 Using db_stat
912
913 TODO
914
915 =head1 BerkeleyDB::Btree
916
917 Equivalent to calling B<db_open> with type B<DB_BTREE> in Berkeley DB 2.x and
918 calling B<db_create> followed by B<DB-E<gt>open> with type B<DB_BTREE> in
919 Berkeley DB 3.x or greater.
920
921 Two forms of constructor are supported:
922
923
924 $db = new BerkeleyDB::Btree
925 [ -Filename => "filename", ]
926 [ -Subname => "sub-database name", ]
927 [ -Flags => flags,]
928 [ -Property => flags,]
929 [ -Mode => number,]
930 [ -Cachesize => number,]
931 [ -Lorder => number,]
932 [ -Pagesize => number,]
933 [ -Env => $env,]
934 [ -Txn => $txn,]
935 [ -Encrypt => { Password => "string",
936 Flags => number }, ],
937 # BerkeleyDB::Btree specific
938 [ -Minkey => number,]
939 [ -Compare => code reference,]
940 [ -DupCompare => code reference,]
941 [ -Prefix => code reference,]
942
943 and this
944
945 [$db =] tie %hash, 'BerkeleyDB::Btree',
946 [ -Filename => "filename", ]
947 [ -Subname => "sub-database name", ]
948 [ -Flags => flags,]
949 [ -Property => flags,]
950 [ -Mode => number,]
951 [ -Cachesize => number,]
952 [ -Lorder => number,]
953 [ -Pagesize => number,]
954 [ -Env => $env,]
955 [ -Txn => $txn,]
956 [ -Encrypt => { Password => "string",
957 Flags => number }, ],
958 # BerkeleyDB::Btree specific
959 [ -Minkey => number,]
960 [ -Compare => code reference,]
961 [ -DupCompare => code reference,]
962 [ -Prefix => code reference,]
963
964 =head2 Options
965
966 In addition to the standard set of options (see L<COMMON OPTIONS>)
967 B<BerkeleyDB::Btree> supports these options:
968
969 =over 5
970
971 =item -Property
972
973 Used to specify extra flags when opening a database. The following
974 flags may be specified by bitwise OR'ing together one or more of the
975 following values:
976
977 B<DB_DUP>
978
979 When creating a new database, this flag enables the storing of duplicate
980 keys in the database. If B<DB_DUPSORT> is not specified as well, the
981 duplicates are stored in the order they are created in the database.
982
983 B<DB_DUPSORT>
984
985 Enables the sorting of duplicate keys in the database. Ignored if
986 B<DB_DUP> isn't also specified.
987
988 =item Minkey
989
990 TODO
991
992 =item Compare
993
994 Allow you to override the default sort order used in the database. See
995 L<"Changing the sort order"> for an example.
996
997 sub compare
998 {
999 my ($key, $key2) = @_ ;
1000 ...
1001 # return 0 if $key1 eq $key2
1002 # -1 if $key1 lt $key2
1003 # 1 if $key1 gt $key2
1004 return (-1 , 0 or 1) ;
1005 }
1006
1007 tie %h, "BerkeleyDB::Hash",
1008 -Filename => $filename,
1009 -Compare => \&compare,
1010 ...
1011
1012 =item Prefix
1013
1014 sub prefix
1015 {
1016 my ($key, $key2) = @_ ;
1017 ...
1018 # return number of bytes of $key2 which are
1019 # necessary to determine that it is greater than $key1
1020 return $bytes ;
1021 }
1022
1023 tie %h, "BerkeleyDB::Hash",
1024 -Filename => $filename,
1025 -Prefix => \&prefix,
1026 ...
1027 =item DupCompare
1028
1029 sub compare
1030 {
1031 my ($key, $key2) = @_ ;
1032 ...
1033 # return 0 if $key1 eq $key2
1034 # -1 if $key1 lt $key2
1035 # 1 if $key1 gt $key2
1036 return (-1 , 0 or 1) ;
1037 }
1038
1039 tie %h, "BerkeleyDB::Hash",
1040 -Filename => $filename,
1041 -DupCompare => \&compare,
1042 ...
1043
1044 =item set_bt_compress
1045
1046 Enabled compression of the btree data. The callback interface is not
1047 supported at present. Need Berkeley DB 4.8 or better.
1048
1049 =back
1050
1051 =head2 Methods
1052
1053 B<BerkeleyDB::Btree> supports the following database methods.
1054 See also L<COMMON DATABASE METHODS>.
1055
1056 All the methods below return 0 to indicate success.
1057
1058 =over 5
1059
1060 =item $status = $db->db_key_range($key, $less, $equal, $greater [, $flags])
1061
1062 Given a key, C<$key>, this method returns the proportion of keys less than
1063 C<$key> in C<$less>, the proportion equal to C<$key> in C<$equal> and the
1064 proportion greater than C<$key> in C<$greater>.
1065
1066 The proportion is returned as a double in the range 0.0 to 1.0.
1067
1068 =back
1069
1070 =head2 A Simple Btree Example
1071
1072 The code below is a simple example of using a btree database.
1073
1074 use strict ;
1075 use BerkeleyDB ;
1076
1077 my $filename = "tree" ;
1078 unlink $filename ;
1079 my %h ;
1080 tie %h, 'BerkeleyDB::Btree',
1081 -Filename => $filename,
1082 -Flags => DB_CREATE
1083 or die "Cannot open $filename: $! $BerkeleyDB::Error\n" ;
1084
1085 # Add a key/value pair to the file
1086 $h{'Wall'} = 'Larry' ;
1087 $h{'Smith'} = 'John' ;
1088 $h{'mouse'} = 'mickey' ;
1089 $h{'duck'} = 'donald' ;
1090
1091 # Delete
1092 delete $h{"duck"} ;
1093
1094 # Cycle through the keys printing them in order.
1095 # Note it is not necessary to sort the keys as
1096 # the btree will have kept them in order automatically.
1097 foreach (keys %h)
1098 { print "$_\n" }
1099
1100 untie %h ;
1101
1102 Here is the output from the code above. The keys have been sorted using
1103 Berkeley DB's default sorting algorithm.
1104
1105 Smith
1106 Wall
1107 mouse
1108
1109
1110 =head2 Changing the sort order
1111
1112 It is possible to supply your own sorting algorithm if the one that Berkeley
1113 DB used isn't suitable. The code below is identical to the previous example
1114 except for the case insensitive compare function.
1115
1116 use strict ;
1117 use BerkeleyDB ;
1118
1119 my $filename = "tree" ;
1120 unlink $filename ;
1121 my %h ;
1122 tie %h, 'BerkeleyDB::Btree',
1123 -Filename => $filename,
1124 -Flags => DB_CREATE,
1125 -Compare => sub { lc $_[0] cmp lc $_[1] }
1126 or die "Cannot open $filename: $!\n" ;
1127
1128 # Add a key/value pair to the file
1129 $h{'Wall'} = 'Larry' ;
1130 $h{'Smith'} = 'John' ;
1131 $h{'mouse'} = 'mickey' ;
1132 $h{'duck'} = 'donald' ;
1133
1134 # Delete
1135 delete $h{"duck"} ;
1136
1137 # Cycle through the keys printing them in order.
1138 # Note it is not necessary to sort the keys as
1139 # the btree will have kept them in order automatically.
1140 foreach (keys %h)
1141 { print "$_\n" }
1142
1143 untie %h ;
1144
1145 Here is the output from the code above.
1146
1147 mouse
1148 Smith
1149 Wall
1150
1151 There are a few point to bear in mind if you want to change the
1152 ordering in a BTREE database:
1153
1154 =over 5
1155
1156 =item 1.
1157
1158 The new compare function must be specified when you create the database.
1159
1160 =item 2.
1161
1162 You cannot change the ordering once the database has been created. Thus
1163 you must use the same compare function every time you access the
1164 database.
1165
1166 =back
1167
1168 =head2 Using db_stat
1169
1170 TODO
1171
1172 =head1 BerkeleyDB::Recno
1173
1174 Equivalent to calling B<db_open> with type B<DB_RECNO> in Berkeley DB 2.x and
1175 calling B<db_create> followed by B<DB-E<gt>open> with type B<DB_RECNO> in
1176 Berkeley DB 3.x or greater.
1177
1178 Two forms of constructor are supported:
1179
1180 $db = new BerkeleyDB::Recno
1181 [ -Filename => "filename", ]
1182 [ -Subname => "sub-database name", ]
1183 [ -Flags => flags,]
1184 [ -Property => flags,]
1185 [ -Mode => number,]
1186 [ -Cachesize => number,]
1187 [ -Lorder => number,]
1188 [ -Pagesize => number,]
1189 [ -Env => $env,]
1190 [ -Txn => $txn,]
1191 [ -Encrypt => { Password => "string",
1192 Flags => number }, ],
1193 # BerkeleyDB::Recno specific
1194 [ -Delim => byte,]
1195 [ -Len => number,]
1196 [ -Pad => byte,]
1197 [ -Source => filename,]
1198
1199 and this
1200
1201 [$db =] tie @arry, 'BerkeleyDB::Recno',
1202 [ -Filename => "filename", ]
1203 [ -Subname => "sub-database name", ]
1204 [ -Flags => flags,]
1205 [ -Property => flags,]
1206 [ -Mode => number,]
1207 [ -Cachesize => number,]
1208 [ -Lorder => number,]
1209 [ -Pagesize => number,]
1210 [ -Env => $env,]
1211 [ -Txn => $txn,]
1212 [ -Encrypt => { Password => "string",
1213 Flags => number }, ],
1214 # BerkeleyDB::Recno specific
1215 [ -Delim => byte,]
1216 [ -Len => number,]
1217 [ -Pad => byte,]
1218 [ -Source => filename,]
1219
1220 =head2 A Recno Example
1221
1222 Here is a simple example that uses RECNO (if you are using a version
1223 of Perl earlier than 5.004_57 this example won't work -- see
1224 L<Extra RECNO Methods> for a workaround).
1225
1226 use strict ;
1227 use BerkeleyDB ;
1228
1229 my $filename = "text" ;
1230 unlink $filename ;
1231
1232 my @h ;
1233 tie @h, 'BerkeleyDB::Recno',
1234 -Filename => $filename,
1235 -Flags => DB_CREATE,
1236 -Property => DB_RENUMBER
1237 or die "Cannot open $filename: $!\n" ;
1238
1239 # Add a few key/value pairs to the file
1240 $h[0] = "orange" ;
1241 $h[1] = "blue" ;
1242 $h[2] = "yellow" ;
1243
1244 push @h, "green", "black" ;
1245
1246 my $elements = scalar @h ;
1247 print "The array contains $elements entries\n" ;
1248
1249 my $last = pop @h ;
1250 print "popped $last\n" ;
1251
1252 unshift @h, "white" ;
1253 my $first = shift @h ;
1254 print "shifted $first\n" ;
1255
1256 # Check for existence of a key
1257 print "Element 1 Exists with value $h[1]\n" if $h[1] ;
1258
1259 untie @h ;
1260
1261 Here is the output from the script:
1262
1263 The array contains 5 entries
1264 popped black
1265 shifted white
1266 Element 1 Exists with value blue
1267 The last element is green
1268 The 2nd last element is yellow
1269
1270 =head1 BerkeleyDB::Queue
1271
1272 Equivalent to calling B<db_create> followed by B<DB-E<gt>open> with
1273 type B<DB_QUEUE> in Berkeley DB 3.x or greater. This database format
1274 isn't available if you use Berkeley DB 2.x.
1275
1276 Two forms of constructor are supported:
1277
1278 $db = new BerkeleyDB::Queue
1279 [ -Filename => "filename", ]
1280 [ -Subname => "sub-database name", ]
1281 [ -Flags => flags,]
1282 [ -Property => flags,]
1283 [ -Mode => number,]
1284 [ -Cachesize => number,]
1285 [ -Lorder => number,]
1286 [ -Pagesize => number,]
1287 [ -Env => $env,]
1288 [ -Txn => $txn,]
1289 [ -Encrypt => { Password => "string",
1290 Flags => number }, ],
1291 # BerkeleyDB::Queue specific
1292 [ -Len => number,]
1293 [ -Pad => byte,]
1294 [ -ExtentSize => number, ]
1295
1296 and this
1297
1298 [$db =] tie @arry, 'BerkeleyDB::Queue',
1299 [ -Filename => "filename", ]
1300 [ -Subname => "sub-database name", ]
1301 [ -Flags => flags,]
1302 [ -Property => flags,]
1303 [ -Mode => number,]
1304 [ -Cachesize => number,]
1305 [ -Lorder => number,]
1306 [ -Pagesize => number,]
1307 [ -Env => $env,]
1308 [ -Txn => $txn,]
1309 [ -Encrypt => { Password => "string",
1310 Flags => number }, ],
1311 # BerkeleyDB::Queue specific
1312 [ -Len => number,]
1313 [ -Pad => byte,]
1314
1315
1316 =head1 BerkeleyDB::Heap
1317
1318 Equivalent to calling B<db_create> followed by B<DB-E<gt>open> with
1319 type B<DB_HEAP> in Berkeley DB 5.2.x or greater. This database format
1320 isn't available if you use an older version of Berkeley DB.
1321
1322 One form of constructor is supported:
1323
1324 $db = new BerkeleyDB::Heap
1325 [ -Filename => "filename", ]
1326 [ -Subname => "sub-database name", ]
1327 [ -Flags => flags,]
1328 [ -Property => flags,]
1329 [ -Mode => number,]
1330 [ -Cachesize => number,]
1331 [ -Lorder => number,]
1332 [ -Pagesize => number,]
1333 [ -Env => $env,]
1334 [ -Txn => $txn,]
1335 [ -Encrypt => { Password => "string",
1336 Flags => number }, ],
1337 # BerkeleyDB::Heap specific
1338 [ -HeapSize => number, ]
1339 [ -HeapSizeGb => number, ]
1340
1341 =head1 BerkeleyDB::Unknown
1342
1343 This class is used to open an existing database.
1344
1345 Equivalent to calling B<db_open> with type B<DB_UNKNOWN> in Berkeley DB 2.x and
1346 calling B<db_create> followed by B<DB-E<gt>open> with type B<DB_UNKNOWN> in
1347 Berkeley DB 3.x or greater.
1348
1349 The constructor looks like this:
1350
1351 $db = new BerkeleyDB::Unknown
1352 [ -Filename => "filename", ]
1353 [ -Subname => "sub-database name", ]
1354 [ -Flags => flags,]
1355 [ -Property => flags,]
1356 [ -Mode => number,]
1357 [ -Cachesize => number,]
1358 [ -Lorder => number,]
1359 [ -Pagesize => number,]
1360 [ -Env => $env,]
1361 [ -Txn => $txn,]
1362 [ -Encrypt => { Password => "string",
1363 Flags => number }, ],
1364
1365
1366 =head2 An example
1367
1368 =head1 COMMON OPTIONS
1369
1370 All database access class constructors support the common set of
1371 options defined below. All are optional.
1372
1373 =over 5
1374
1375 =item -Filename
1376
1377 The database filename. If no filename is specified, a temporary file will
1378 be created and removed once the program terminates.
1379
1380 =item -Subname
1381
1382 Specifies the name of the sub-database to open.
1383 This option is only valid if you are using Berkeley DB 3.x or greater.
1384
1385 =item -Flags
1386
1387 Specify how the database will be opened/created. The valid flags are:
1388
1389 B<DB_CREATE>
1390
1391 Create any underlying files, as necessary. If the files do not already
1392 exist and the B<DB_CREATE> flag is not specified, the call will fail.
1393
1394 B<DB_NOMMAP>
1395
1396 Not supported by BerkeleyDB.
1397
1398 B<DB_RDONLY>
1399
1400 Opens the database in read-only mode.
1401
1402 B<DB_THREAD>
1403
1404 Not supported by BerkeleyDB.
1405
1406 B<DB_TRUNCATE>
1407
1408 If the database file already exists, remove all the data before
1409 opening it.
1410
1411 =item -Mode
1412
1413 Determines the file protection when the database is created. Defaults
1414 to 0666.
1415
1416 =item -Cachesize
1417
1418 =item -Lorder
1419
1420 =item -Pagesize
1421
1422 =item -Env
1423
1424 When working under a Berkeley DB environment, this parameter
1425
1426 Defaults to no environment.
1427
1428 =item -Encrypt
1429
1430 If present, this parameter will enable encryption of all data before
1431 it is written to the database. This parameters must be given a hash
1432 reference. The format is shown below.
1433
1434 -Encrypt => { -Password => "abc", Flags => DB_ENCRYPT_AES }
1435
1436 Valid values for the Flags are 0 or C<DB_ENCRYPT_AES>.
1437
1438 This option requires Berkeley DB 4.1 or better.
1439
1440 =item -Txn
1441
1442 TODO.
1443
1444 =back
1445
1446 =head1 COMMON DATABASE METHODS
1447
1448 All the database interfaces support the common set of methods defined
1449 below.
1450
1451 All the methods below return 0 to indicate success.
1452
1453 =head2 $env = $db->Env();
1454
1455 Returns the environment object the database is associated with or C<undef>
1456 when no environment was used when opening the database.
1457
1458 =head2 $status = $db->db_get($key, $value [, $flags])
1459
1460 Given a key (C<$key>) this method reads the value associated with it
1461 from the database. If it exists, the value read from the database is
1462 returned in the C<$value> parameter.
1463
1464 The B<$flags> parameter is optional. If present, it must be set to B<one>
1465 of the following values:
1466
1467 =over 5
1468
1469 =item B<DB_GET_BOTH>
1470
1471 When the B<DB_GET_BOTH> flag is specified, B<db_get> checks for the
1472 existence of B<both> the C<$key> B<and> C<$value> in the database.
1473
1474 =item B<DB_SET_RECNO>
1475
1476 TODO.
1477
1478 =back
1479
1480 In addition, the following value may be set by bitwise OR'ing it into
1481 the B<$flags> parameter:
1482
1483 =over 5
1484
1485 =item B<DB_RMW>
1486
1487 TODO
1488
1489 =back
1490
1491 The variant C<db_pget> allows you to query a secondary database:
1492
1493 $status = $sdb->db_pget($skey, $pkey, $value);
1494
1495 using the key C<$skey> in the secondary db to lookup C<$pkey> and C<$value>
1496 from the primary db.
1497
1498 =head2 $status = $db->db_exists($key [, $flags])
1499
1500 This method checks for the existence of the given key (C<$key>), but
1501 does not read the value. If the key is not found, B<db_exists> will
1502 return B<DB_NOTFOUND>. Requires BDB 4.6 or better.
1503
1504 =head2 $status = $db->db_put($key, $value [, $flags])
1505
1506 Stores a key/value pair in the database.
1507
1508 The B<$flags> parameter is optional. If present it must be set to B<one>
1509 of the following values:
1510
1511 =over 5
1512
1513 =item B<DB_APPEND>
1514
1515 This flag is only applicable when accessing a B<BerkeleyDB::Recno>
1516 database.
1517
1518 TODO.
1519
1520
1521 =item B<DB_NOOVERWRITE>
1522
1523 If this flag is specified and C<$key> already exists in the database,
1524 the call to B<db_put> will return B<DB_KEYEXIST>.
1525
1526 =back
1527
1528 =head2 $status = $db->db_del($key [, $flags])
1529
1530 Deletes a key/value pair in the database associated with C<$key>.
1531 If duplicate keys are enabled in the database, B<db_del> will delete
1532 B<all> key/value pairs with key C<$key>.
1533
1534 The B<$flags> parameter is optional and is currently unused.
1535
1536 =head2 $status = $env->stat_print([$flags])
1537
1538 Prints statistical information.
1539
1540 If the C<MsgFile> option is specified the output will be sent to the
1541 file. Otherwise output is sent to standard output.
1542
1543 This option requires Berkeley DB 4.3 or better.
1544
1545 =head2 $status = $db->db_sync()
1546
1547 If any parts of the database are in memory, write them to the database.
1548
1549 =head2 $cursor = $db->db_cursor([$flags])
1550
1551 Creates a cursor object. This is used to access the contents of the
1552 database sequentially. See L<CURSORS> for details of the methods
1553 available when working with cursors.
1554
1555 The B<$flags> parameter is optional. If present it must be set to B<one>
1556 of the following values:
1557
1558 =over 5
1559
1560 =item B<DB_RMW>
1561
1562 TODO.
1563
1564 =back
1565
1566 =head2 ($flag, $old_offset, $old_length) = $db->partial_set($offset, $length) ;
1567
1568 TODO
1569
1570 =head2 ($flag, $old_offset, $old_length) = $db->partial_clear() ;
1571
1572 TODO
1573
1574 =head2 $db->byteswapped()
1575
1576 TODO
1577
1578 =head2 $db->type()
1579
1580 Returns the type of the database. The possible return code are B<DB_HASH>
1581 for a B<BerkeleyDB::Hash> database, B<DB_BTREE> for a B<BerkeleyDB::Btree>
1582 database and B<DB_RECNO> for a B<BerkeleyDB::Recno> database. This method
1583 is typically used when a database has been opened with
1584 B<BerkeleyDB::Unknown>.
1585
1586 =head2 $bool = $env->cds_enabled();
1587
1588 Returns true if the Berkeley DB environment C<$env> has been opened on
1589 CDS mode.
1590
1591 =head2 $bool = $db->cds_enabled();
1592
1593 Returns true if the database C<$db> has been opened on CDS mode.
1594
1595 =head2 $lock = $db->cds_lock();
1596
1597 Creates a CDS write lock object C<$lock>.
1598
1599 It is a fatal error to attempt to create a cds_lock if the Berkeley DB
1600 environment has not been opened in CDS mode.
1601
1602 =head2 $lock->cds_unlock();
1603
1604 Removes a CDS lock. The destruction of the CDS lock object automatically
1605 calls this method.
1606
1607 Note that if multiple CDS lock objects are created, the underlying write
1608 lock will not be released until all CDS lock objects are either explictly
1609 unlocked with this method, or the CDS lock objects have been destroyed.
1610
1611 =head2 $ref = $db->db_stat()
1612
1613 Returns a reference to an associative array containing information about
1614 the database. The keys of the associative array correspond directly to the
1615 names of the fields defined in the Berkeley DB documentation. For example,
1616 in the DB documentation, the field B<bt_version> stores the version of the
1617 Btree database. Assuming you called B<db_stat> on a Btree database the
1618 equivalent field would be accessed as follows:
1619
1620 $version = $ref->{'bt_version'} ;
1621
1622 If you are using Berkeley DB 3.x or better, this method will work will
1623 all database formats. When DB 2.x is used, it only works with
1624 B<BerkeleyDB::Btree>.
1625
1626 =head2 $status = $db->status()
1627
1628 Returns the status of the last C<$db> method called.
1629
1630 =head2 $status = $db->truncate($count)
1631
1632 Truncates the datatabase and returns the number or records deleted
1633 in C<$count>.
1634
1635 =head2 $status = $db->compact($start, $stop, $c_data, $flags, $end);
1636
1637 Compacts the database C<$db>.
1638
1639 All the parameters are optional - if only want to make use of some of them,
1640 use C<undef> for those you don't want. Trailing unusused parameters can be
1641 omitted. For example, if you only want to use the C<$c_data> parameter to
1642 set the C<compact_fillpercent>, write you code like this
1643
1644 my %hash;
1645 $hash{compact_fillpercent} = 50;
1646 $db->compact(undef, undef, \%hash);
1647
1648 The parameters operate identically to the C equivalent of this method.
1649 The C<$c_data> needs a bit of explanation - it must be a hash reference.
1650 The values of the following keys can be set before calling C<compact> and
1651 will affect the operation of the compaction.
1652
1653 =over 5
1654
1655 =item * compact_fillpercent
1656
1657 =item * compact_timeout
1658
1659 =back
1660
1661 The following keys, along with associated values, will be created in the
1662 hash reference if the C<compact> operation was successful.
1663
1664 =over 5
1665
1666 =item * compact_deadlock
1667
1668 =item * compact_levels
1669
1670 =item * compact_pages_free
1671
1672 =item * compact_pages_examine
1673
1674 =item * compact_pages_truncated
1675
1676 =back
1677
1678 You need to be running Berkeley DB 4.4 or better if you want to make use of
1679 C<compact>.
1680
1681 =head2 $status = $db->associate($secondary, \&key_callback)
1682
1683 Associate C<$db> with the secondary DB C<$secondary>
1684
1685 New key/value pairs inserted to the database will be passed to the callback
1686 which must set its third argument to the secondary key to allow lookup. If
1687 an array reference is set multiple keys secondary keys will be associated
1688 with the primary database entry.
1689
1690 Data may be retrieved fro the secondary database using C<db_pget> to also
1691 obtain the primary key.
1692
1693 Secondary databased are maintained automatically.
1694
1695 =head2 $status = $db->associate_foreign($secondary, callback, $flags)
1696
1697 Associate a foreign key database C<$db> with the secondary DB
1698 C<$secondary>.
1699
1700 The second parameter must be a reference to a sub or C<undef>.
1701
1702 The C<$flags> parameter must be either C<DB_FOREIGN_CASCADE>,
1703 C<DB_FOREIGN_ABORT> or C<DB_FOREIGN_NULLIFY>.
1704
1705 When the flags parameter is C<DB_FOREIGN_NULLIFY> the second parameter is a
1706 reference to a sub of the form
1707
1708 sub foreign_cb
1709 {
1710 my $key = \$_[0];
1711 my $value = \$_[1];
1712 my $foreignkey = \$_[2];
1713 my $changed = \$_[3] ;
1714
1715 # for ... set $$value and set $$changed to 1
1716
1717 return 0;
1718 }
1719
1720 $foreign_db->associate_foreign($secondary, \&foreign_cb, DB_FOREIGN_NULLIFY);
1721
1722 =head1 CURSORS
1723
1724 A cursor is used whenever you want to access the contents of a database
1725 in sequential order.
1726 A cursor object is created with the C<db_cursor>
1727
1728 A cursor object has the following methods available:
1729
1730 =head2 $newcursor = $cursor->c_dup($flags)
1731
1732 Creates a duplicate of C<$cursor>. This method needs Berkeley DB 3.0.x or better.
1733
1734 The C<$flags> parameter is optional and can take the following value:
1735
1736 =over 5
1737
1738 =item DB_POSITION
1739
1740 When present this flag will position the new cursor at the same place as the
1741 existing cursor.
1742
1743 =back
1744
1745 =head2 $status = $cursor->c_get($key, $value, $flags)
1746
1747 Reads a key/value pair from the database, returning the data in C<$key>
1748 and C<$value>. The key/value pair actually read is controlled by the
1749 C<$flags> parameter, which can take B<one> of the following values:
1750
1751 =over 5
1752
1753 =item B<DB_FIRST>
1754
1755 Set the cursor to point to the first key/value pair in the
1756 database. Return the key/value pair in C<$key> and C<$value>.
1757
1758 =item B<DB_LAST>
1759
1760 Set the cursor to point to the last key/value pair in the database. Return
1761 the key/value pair in C<$key> and C<$value>.
1762
1763 =item B<DB_NEXT>
1764
1765 If the cursor is already pointing to a key/value pair, it will be
1766 incremented to point to the next key/value pair and return its contents.
1767
1768 If the cursor isn't initialised, B<DB_NEXT> works just like B<DB_FIRST>.
1769
1770 If the cursor is already positioned at the last key/value pair, B<c_get>
1771 will return B<DB_NOTFOUND>.
1772
1773 =item B<DB_NEXT_DUP>
1774
1775 This flag is only valid when duplicate keys have been enabled in
1776 a database.
1777 If the cursor is already pointing to a key/value pair and the key of
1778 the next key/value pair is identical, the cursor will be incremented to
1779 point to it and their contents returned.
1780
1781 =item B<DB_PREV>
1782
1783 If the cursor is already pointing to a key/value pair, it will be
1784 decremented to point to the previous key/value pair and return its
1785 contents.
1786
1787 If the cursor isn't initialised, B<DB_PREV> works just like B<DB_LAST>.
1788
1789 If the cursor is already positioned at the first key/value pair, B<c_get>
1790 will return B<DB_NOTFOUND>.
1791
1792 =item B<DB_CURRENT>
1793
1794 If the cursor has been set to point to a key/value pair, return their
1795 contents.
1796 If the key/value pair referenced by the cursor has been deleted, B<c_get>
1797 will return B<DB_KEYEMPTY>.
1798
1799 =item B<DB_SET>
1800
1801 Set the cursor to point to the key/value pair referenced by B<$key>
1802 and return the value in B<$value>.
1803
1804 =item B<DB_SET_RANGE>
1805
1806 This flag is a variation on the B<DB_SET> flag. As well as returning
1807 the value, it also returns the key, via B<$key>.
1808 When used with a B<BerkeleyDB::Btree> database the key matched by B<c_get>
1809 will be the shortest key (in length) which is greater than or equal to
1810 the key supplied, via B<$key>. This allows partial key searches.
1811 See ??? for an example of how to use this flag.
1812
1813 =item B<DB_GET_BOTH>
1814
1815 Another variation on B<DB_SET>. This one returns both the key and
1816 the value.
1817
1818 =item B<DB_SET_RECNO>
1819
1820 TODO.
1821
1822 =item B<DB_GET_RECNO>
1823
1824 TODO.
1825
1826 =back
1827
1828 In addition, the following value may be set by bitwise OR'ing it into
1829 the B<$flags> parameter:
1830
1831 =over 5
1832
1833 =item B<DB_RMW>
1834
1835 TODO.
1836
1837 =back
1838
1839 =head2 $status = $cursor->c_put($key, $value, $flags)
1840
1841 Stores the key/value pair in the database. The position that the data is
1842 stored in the database is controlled by the C<$flags> parameter, which
1843 must take B<one> of the following values:
1844
1845 =over 5
1846
1847 =item B<DB_AFTER>
1848
1849 When used with a Btree or Hash database, a duplicate of the key referenced
1850 by the current cursor position will be created and the contents of
1851 B<$value> will be associated with it - B<$key> is ignored.
1852 The new key/value pair will be stored immediately after the current
1853 cursor position.
1854 Obviously the database has to have been opened with B<DB_DUP>.
1855
1856 When used with a Recno ... TODO
1857
1858
1859 =item B<DB_BEFORE>
1860
1861 When used with a Btree or Hash database, a duplicate of the key referenced
1862 by the current cursor position will be created and the contents of
1863 B<$value> will be associated with it - B<$key> is ignored.
1864 The new key/value pair will be stored immediately before the current
1865 cursor position.
1866 Obviously the database has to have been opened with B<DB_DUP>.
1867
1868 When used with a Recno ... TODO
1869
1870 =item B<DB_CURRENT>
1871
1872 If the cursor has been initialised, replace the value of the key/value
1873 pair stored in the database with the contents of B<$value>.
1874
1875 =item B<DB_KEYFIRST>
1876
1877 Only valid with a Btree or Hash database. This flag is only really
1878 used when duplicates are enabled in the database and sorted duplicates
1879 haven't been specified.
1880 In this case the key/value pair will be inserted as the first entry in
1881 the duplicates for the particular key.
1882
1883 =item B<DB_KEYLAST>
1884
1885 Only valid with a Btree or Hash database. This flag is only really
1886 used when duplicates are enabled in the database and sorted duplicates
1887 haven't been specified.
1888 In this case the key/value pair will be inserted as the last entry in
1889 the duplicates for the particular key.
1890
1891 =back
1892
1893 =head2 $status = $cursor->c_del([$flags])
1894
1895 This method deletes the key/value pair associated with the current cursor
1896 position. The cursor position will not be changed by this operation, so
1897 any subsequent cursor operation must first initialise the cursor to
1898 point to a valid key/value pair.
1899
1900 If the key/value pair associated with the cursor have already been
1901 deleted, B<c_del> will return B<DB_KEYEMPTY>.
1902
1903 The B<$flags> parameter is not used at present.
1904
1905 =head2 $status = $cursor->c_count($cnt [, $flags])
1906
1907 Stores the number of duplicates at the current cursor position in B<$cnt>.
1908
1909 The B<$flags> parameter is not used at present. This method needs
1910 Berkeley DB 3.1 or better.
1911
1912 =head2 $status = $cursor->status()
1913
1914 Returns the status of the last cursor method as a dual type.
1915
1916 =head2 $status = $cursor->c_pget() ;
1917
1918 See C<db_pget>
1919
1920 =head2 $status = $cursor->c_close()
1921
1922 Closes the cursor B<$cursor>.
1923
1924 =head2 Cursor Examples
1925
1926 TODO
1927
1928 Iterating from first to last, then in reverse.
1929
1930 examples of each of the flags.
1931
1932 =head1 JOIN
1933
1934 Join support for BerkeleyDB is in progress. Watch this space.
1935
1936 TODO
1937
1938 =head1 TRANSACTIONS
1939
1940 Transactions are created using the C<txn_begin> method on L<BerkeleyDB::Env>:
1941
1942 my $txn = $env->txn_begin;
1943
1944 If this is a nested transaction, supply the parent transaction as an
1945 argument:
1946
1947 my $child_txn = $env->txn_begin($parent_txn);
1948
1949 Then in order to work with the transaction, you must set it as the current
1950 transaction on the database handles you want to work with:
1951
1952 $db->Txn($txn);
1953
1954 Or for multiple handles:
1955
1956 $txn->Txn(@handles);
1957
1958 The current transaction is given by BerkeleyDB each time to the various BDB
1959 operations. In the C api it is required explicitly as an argument to every
1960 operation.
1961
1962 To commit a transaction call the C<commit> method on it:
1963
1964 $txn->txn_commit;
1965
1966 and to roll back call abort:
1967
1968 $txn->txn_abort
1969
1970 After committing or aborting a child transaction you need to set the active
1971 transaction again using C<Txn>.
1972
1973
1974 =head1 Berkeley DB Concurrent Data Store (CDS)
1975
1976 The Berkeley DB I<Concurrent Data Store> (CDS) is a lightweight locking
1977 mechanism that is useful in scenarios where transactions are overkill.
1978
1979 =head2 What is CDS?
1980
1981 The Berkeley DB CDS interface is a simple lightweight locking mechanism
1982 that allows safe concurrent access to Berkeley DB databases. Your
1983 application can have multiple reader and write processes, but Berkeley DB
1984 will arrange it so that only one process can have a write lock against the
1985 database at a time, i.e. multiple processes can read from a database
1986 concurrently, but all write processes will be serialised.
1987
1988 =head2 Should I use it?
1989
1990 Whilst this simple locking model is perfectly adequate for some
1991 applications, it will be too restrictive for others. Before deciding on
1992 using CDS mode, you need to be sure that it is suitable for the expected
1993 behaviour of your application.
1994
1995 The key features of this model are
1996
1997 =over 5
1998
1999 =item *
2000
2001 All writes operations are serialised.
2002
2003 =item *
2004
2005 A write operation will block until all reads have finished.
2006
2007 =back
2008
2009 There are a few of the attributes of your application that you need to be
2010 aware of before choosing to use CDS.
2011
2012 Firstly, if you application needs either recoverability or transaction
2013 support, then CDS will not be suitable.
2014
2015 Next what is the ratio of read operation to write operations will your
2016 application have?
2017
2018 If it is carrying out mostly read operations, and very few writes, then CDS
2019 may be appropriate.
2020
2021 What is the expected throughput of reads/writes in your application?
2022
2023 If you application does 90% writes and 10% reads, but on average you only
2024 have a transaction every 5 seconds, then the fact that all writes are
2025 serialised will not matter, because there will hardly ever be multiple
2026 writes processes blocking.
2027
2028 In summary CDS mode may be appropriate for your application if it performs
2029 mostly reads and very few writes or there is a low throughput. Also, if
2030 you do not need to be able to roll back a series of database operations if
2031 an error occurs, then CDS is ok.
2032
2033 If any of these is not the case you will need to use Berkeley DB
2034 transactions. That is outside the scope of this document.
2035
2036 =head2 Locking Used
2037
2038 Berkeley DB implements CDS mode using two kinds of lock behind the scenes -
2039 namely read locks and write locks. A read lock allows multiple processes to
2040 access the database for reading at the same time. A write lock will only
2041 get access to the database when there are no read or write locks active.
2042 The write lock will block until the process holding the lock releases it.
2043
2044 Multiple processes with read locks can all access the database at the same
2045 time as long as no process has a write lock. A process with a write lock
2046 can only access the database if there are no other active read or write
2047 locks.
2048
2049 The majority of the time the Berkeley DB CDS mode will handle all locking
2050 without your application having to do anything. There are a couple of
2051 exceptions you need to be aware of though - these will be discussed in
2052 L<Safely Updating Records> and L<Implicit Cursors> below.
2053
2054 A Berkeley DB Cursor (created with C<< $db->db_cursor >>) will by hold a
2055 lock on the database until it is either explicitly closed or destroyed.
2056 This means the lock has the potential to be long lived.
2057
2058 By default Berkeley DB cursors create a read lock, but it is possible to
2059 create a cursor that holds a write lock, thus
2060
2061 $cursor = $db->db_cursor(DB_WRITECURSOR);
2062
2063
2064 Whilst either a read or write cursor is active, it will block any other
2065 processes that wants to write to the database.
2066
2067 To avoid blocking problems, only keep cursors open as long as they are
2068 needed. The same is true when you use the C<cursor> method or the
2069 C<cds_lock> method.
2070
2071 For full information on CDS see the "Berkeley DB Concurrent Data Store
2072 applications" section in the Berkeley DB Reference Guide.
2073
2074
2075 =head2 Opening a database for CDS
2076
2077 Here is the typical signature that is used when opening a database in CDS
2078 mode.
2079
2080 use BerkeleyDB ;
2081
2082 my $env = new BerkeleyDB::Env
2083 -Home => "./home" ,
2084 -Flags => DB_CREATE| DB_INIT_CDB | DB_INIT_MPOOL
2085 or die "cannot open environment: $BerkeleyDB::Error\n";
2086
2087 my $db = new BerkeleyDB::Hash
2088 -Filename => 'test1.db',
2089 -Flags => DB_CREATE,
2090 -Env => $env
2091 or die "cannot open database: $BerkeleyDB::Error\n";
2092
2093 or this, if you use the tied interface
2094
2095 tie %hash, "BerkeleyDB::Hash",
2096 -Filename => 'test2.db',
2097 -Flags => DB_CREATE,
2098 -Env => $env
2099 or die "cannot open database: $BerkeleyDB::Error\n";
2100
2101 The first thing to note is that you B<MUST> always use a Berkeley DB
2102 environment if you want to use locking with Berkeley DB.
2103
2104 Remember, that apart from the actual database files you explicitly create
2105 yourself, Berkeley DB will create a few behind the scenes to handle locking
2106 - they usually have names like "__db.001". It is therefore a good idea to
2107 use the C<-Home> option, unless you are happy for all these files to be
2108 written in the current directory.
2109
2110 Next, remember to include the C<DB_CREATE> flag when opening the
2111 environment for the first time. A common mistake is to forget to add this
2112 option and then wonder why the application doesn't work.
2113
2114 Finally, it is vital that all processes that are going to access the
2115 database files use the same Berkeley DB environment.
2116
2117
2118 =head2 Safely Updating a Record
2119
2120 One of the main gotchas when using CDS is if you want to update a record in
2121 a database, i.e. you want to retrieve a record from a database, modify it
2122 in some way and put it back in the database.
2123
2124 For example, say you are writing a web application and you want to keep a
2125 record of the number of times your site is accessed in a Berkeley DB
2126 database. So your code will have a line of code like this (assume, of
2127 course, that C<%hash> has been tied to a Berkeley DB database):
2128
2129 $hash{Counter} ++ ;
2130
2131 That may look innocent enough, but there is a race condition lurking in
2132 there. If I rewrite the line of code using the low-level Berkeley DB API,
2133 which is what will actually be executed, the race condition may be more
2134 apparent:
2135
2136 $db->db_get("Counter", $value);
2137 ++ $value ;
2138 $db->db_put("Counter", $value);
2139
2140 Consider what happens behind the scenes when you execute the commands
2141 above. Firstly, the existing value for the key "Counter" is fetched from
2142 the database using C<db_get>. A read lock will be used for this part of the
2143 update. The value is then incremented, and the new value is written back
2144 to the database using C<db_put>. This time a write lock will be used.
2145
2146 Here's the problem - there is nothing to stop two (or more) processes
2147 executing the read part at the same time. Remember multiple processes can
2148 hold a read lock on the database at the same time. So both will fetch the
2149 same value, let's say 7, from the database. Both increment the value to 8
2150 and attempt to write it to the database. Berkeley DB will ensure that only
2151 one of the processes gets a write lock, while the other will be blocked. So
2152 the process that happened to get the write lock will store the value 8 to
2153 the database and release the write lock. Now the other process will be
2154 unblocked, and it too will write the value 8 to the database. The result,
2155 in this example, is we have missed a hit in the counter.
2156
2157 To deal with this kind of scenario, you need to make the update atomic. A
2158 convenience method, called C<cds_lock>, is supplied with the BerkeleyDB
2159 module for this purpose. Using C<cds_lock>, the counter update code can now
2160 be rewritten thus:
2161
2162 my $lk = $dbh->cds_lock() ;
2163 $hash{Counter} ++ ;
2164 $lk->cds_unlock;
2165
2166 or this, where scoping is used to limit the lifetime of the lock object
2167
2168 {
2169 my $lk = $dbh->cds_lock() ;
2170 $hash{Counter} ++ ;
2171 }
2172
2173 Similarly, C<cds_lock> can be used with the native Berkeley DB API
2174
2175 my $lk = $dbh->cds_lock() ;
2176 $db->db_get("Counter", $value);
2177 ++ $value ;
2178 $db->db_put("Counter", $value);
2179 $lk->unlock;
2180
2181
2182 The C<cds_lock> method will ensure that the current process has exclusive
2183 access to the database until the lock is either explicitly released, via
2184 the C<< $lk->cds_unlock() >> or by the lock object being destroyed.
2185
2186 If you are interested, all that C<cds_lock> does is open a "write" cursor.
2187 This has the useful side-effect of holding a write-lock on the database
2188 until the cursor is deleted. This is how you create a write-cursor
2189
2190 $cursor = $db->db_cursor(DB_WRITECURSOR);
2191
2192 If you have instantiated multiple C<cds_lock> objects for one database
2193 within a single process, that process will hold a write-lock on the
2194 database until I<ALL> C<cds_lock> objects have been destroyed.
2195
2196 As with all write-cursors, you should try to limit the scope of the
2197 C<cds_lock> to as short a time as possible. Remember the complete database
2198 will be locked to other process whilst the write lock is in place.
2199
2200 =head2 Cannot write with a read cursor while a write cursor is active
2201
2202 This issue is easier to demonstrate with an example, so consider the code
2203 below. The intention of the code is to increment the values of all the
2204 elements in a database by one.
2205
2206 # Assume $db is a database opened in a CDS environment.
2207
2208 # Create a write-lock
2209 my $lock = $db->db_cursor(DB_WRITECURSOR);
2210 # or
2211 # my $lock = $db->cds_lock();
2212
2213
2214 my $cursor = $db->db_cursor();
2215
2216 # Now loop through the database, and increment
2217 # each value using c_put.
2218 while ($cursor->c_get($key, $value, DB_NEXT) == 0)
2219 {
2220 $cursor->c_put($key, $value+1, DB_CURRENT) == 0
2221 or die "$BerkeleyDB::Error\n";
2222 }
2223
2224
2225 When this code is run, it will fail on the C<c_put> line with this error
2226
2227 Write attempted on read-only cursor
2228
2229 The read cursor has automatically disallowed a write operation to prevent a
2230 deadlock.
2231
2232
2233 So the rule is -- you B<CANNOT> carry out a write operation using a
2234 read-only cursor (i.e. you cannot use C<c_put> or C<c_del>) whilst another
2235 write-cursor is already active.
2236
2237 The workaround for this issue is to just use C<db_put> instead of C<c_put>,
2238 like this
2239
2240 # Assume $db is a database opened in a CDS environment.
2241
2242 # Create a write-lock
2243 my $lock = $db->db_cursor(DB_WRITECURSOR);
2244 # or
2245 # my $lock = $db->cds_lock();
2246
2247
2248 my $cursor = $db->db_cursor();
2249
2250 # Now loop through the database, and increment
2251 # each value using c_put.
2252 while ($cursor->c_get($key, $value, DB_NEXT) == 0)
2253 {
2254 $db->db_put($key, $value+1) == 0
2255 or die "$BerkeleyDB::Error\n";
2256 }
2257
2258
2259
2260 =head2 Implicit Cursors
2261
2262 All Berkeley DB cursors will hold either a read lock or a write lock on the
2263 database for the existence of the cursor. In order to prevent blocking of
2264 other processes you need to make sure that they are not long lived.
2265
2266 There are a number of instances where the Perl interface to Berkeley DB
2267 will create a cursor behind the scenes without you being aware of it. Most
2268 of these are very short-lived and will not affect the running of your
2269 script, but there are a few notable exceptions.
2270
2271 Consider this snippet of code
2272
2273 while (my ($k, $v) = each %hash)
2274 {
2275 # do something
2276 }
2277
2278
2279 To implement the "each" functionality, a read cursor will be created behind
2280 the scenes to allow you to iterate through the tied hash, C<%hash>. While
2281 that cursor is still active, a read lock will obviously be held against the
2282 database. If your application has any other writing processes, these will
2283 be blocked until the read cursor is closed. That won't happen until the
2284 loop terminates.
2285
2286 To avoid blocking problems, only keep cursors open as long as they are
2287 needed. The same is true when you use the C<cursor> method or the
2288 C<cds_lock> method.
2289
2290
2291 The locking behaviour of the C<values> or C<keys> functions, shown below,
2292 is subtly different.
2293
2294 foreach my $k (keys %hash)
2295 {
2296 # do something
2297 }
2298
2299 foreach my $v (values %hash)
2300 {
2301 # do something
2302 }
2303
2304
2305 Just as in the C<each> function, a read cursor will be created to iterate
2306 over the database in both of these cases. Where C<keys> and C<values>
2307 differ is the place where the cursor carries out the iteration through the
2308 database. Whilst C<each> carried out a single iteration every time it was
2309 invoked, the C<keys> and C<values> functions will iterate through the
2310 entire database in one go -- the complete database will be read into memory
2311 before the first iteration of the loop.
2312
2313 Apart from the fact that a read lock will be held for the amount of time
2314 required to iterate through the database, the use of C<keys> and C<values>
2315 is B<not> recommended because it will result in the complete database being
2316 read into memory.
2317
2318
2319 =head2 Avoiding Deadlock with multiple databases
2320
2321 If your CDS application uses multiple database files, and you need to write
2322 to more than one of them, you need to be careful you don't create a
2323 deadlock.
2324
2325 For example, say you have two databases, D1 and D2, and two processes, P1
2326 and P2. Assume you want to write a record to each database. If P1 writes
2327 the records to the databases in the order D1, D2 while process P2 writes
2328 the records in the order D2, D1, there is the potential for a deadlock to
2329 occur.
2330
2331 This scenario can be avoided by either always acquiring the write locks in
2332 exactly the same order in your application code, or by using the
2333 C<DB_CDB_ALLDB> flag when opening the environment. This flag will make a
2334 write-lock apply to all the databases in the environment.
2335
2336 Add example here
2337
2338 =head1 DBM Filters
2339
2340 A DBM Filter is a piece of code that is be used when you I<always>
2341 want to make the same transformation to all keys and/or values in a DBM
2342 database. All of the database classes (BerkeleyDB::Hash,
2343 BerkeleyDB::Btree and BerkeleyDB::Recno) support DBM Filters.
2344
2345 There are four methods associated with DBM Filters. All work
2346 identically, and each is used to install (or uninstall) a single DBM
2347 Filter. Each expects a single parameter, namely a reference to a sub.
2348 The only difference between them is the place that the filter is
2349 installed.
2350
2351 To summarise:
2352
2353 =over 5
2354
2355 =item B<filter_store_key>
2356
2357 If a filter has been installed with this method, it will be invoked
2358 every time you write a key to a DBM database.
2359
2360 =item B<filter_store_value>
2361
2362 If a filter has been installed with this method, it will be invoked
2363 every time you write a value to a DBM database.
2364
2365
2366 =item B<filter_fetch_key>
2367
2368 If a filter has been installed with this method, it will be invoked
2369 every time you read a key from a DBM database.
2370
2371 =item B<filter_fetch_value>
2372
2373 If a filter has been installed with this method, it will be invoked
2374 every time you read a value from a DBM database.
2375
2376 =back
2377
2378 You can use any combination of the methods, from none, to all four.
2379
2380 All filter methods return the existing filter, if present, or C<undef>
2381 in not.
2382
2383 To delete a filter pass C<undef> to it.
2384
2385 =head2 The Filter
2386
2387 When each filter is called by Perl, a local copy of C<$_> will contain
2388 the key or value to be filtered. Filtering is achieved by modifying
2389 the contents of C<$_>. The return code from the filter is ignored.
2390
2391 =head2 An Example -- the NULL termination problem.
2392
2393 Consider the following scenario. You have a DBM database that you need
2394 to share with a third-party C application. The C application assumes
2395 that I<all> keys and values are NULL terminated. Unfortunately when
2396 Perl writes to DBM databases it doesn't use NULL termination, so your
2397 Perl application will have to manage NULL termination itself. When you
2398 write to the database you will have to use something like this:
2399
2400 $hash{"$key\0"} = "$value\0" ;
2401
2402 Similarly the NULL needs to be taken into account when you are considering
2403 the length of existing keys/values.
2404
2405 It would be much better if you could ignore the NULL terminations issue
2406 in the main application code and have a mechanism that automatically
2407 added the terminating NULL to all keys and values whenever you write to
2408 the database and have them removed when you read from the database. As I'm
2409 sure you have already guessed, this is a problem that DBM Filters can
2410 fix very easily.
2411
2412 use strict ;
2413 use BerkeleyDB ;
2414
2415 my %hash ;
2416 my $filename = "filt.db" ;
2417 unlink $filename ;
2418
2419 my $db = tie %hash, 'BerkeleyDB::Hash',
2420 -Filename => $filename,
2421 -Flags => DB_CREATE
2422 or die "Cannot open $filename: $!\n" ;
2423
2424 # Install DBM Filters
2425 $db->filter_fetch_key ( sub { s/\0$// } ) ;
2426 $db->filter_store_key ( sub { $_ .= "\0" } ) ;
2427 $db->filter_fetch_value( sub { s/\0$// } ) ;
2428 $db->filter_store_value( sub { $_ .= "\0" } ) ;
2429
2430 $hash{"abc"} = "def" ;
2431 my $a = $hash{"ABC"} ;
2432 # ...
2433 undef $db ;
2434 untie %hash ;
2435
2436 Hopefully the contents of each of the filters should be
2437 self-explanatory. Both "fetch" filters remove the terminating NULL,
2438 and both "store" filters add a terminating NULL.
2439
2440
2441 =head2 Another Example -- Key is a C int.
2442
2443 Here is another real-life example. By default, whenever Perl writes to
2444 a DBM database it always writes the key and value as strings. So when
2445 you use this:
2446
2447 $hash{12345} = "something" ;
2448
2449 the key 12345 will get stored in the DBM database as the 5 byte string
2450 "12345". If you actually want the key to be stored in the DBM database
2451 as a C int, you will have to use C<pack> when writing, and C<unpack>
2452 when reading.
2453
2454 Here is a DBM Filter that does it:
2455
2456 use strict ;
2457 use BerkeleyDB ;
2458 my %hash ;
2459 my $filename = "filt.db" ;
2460 unlink $filename ;
2461
2462
2463 my $db = tie %hash, 'BerkeleyDB::Btree',
2464 -Filename => $filename,
2465 -Flags => DB_CREATE
2466 or die "Cannot open $filename: $!\n" ;
2467
2468 $db->filter_fetch_key ( sub { $_ = unpack("i", $_) } ) ;
2469 $db->filter_store_key ( sub { $_ = pack ("i", $_) } ) ;
2470 $hash{123} = "def" ;
2471 # ...
2472 undef $db ;
2473 untie %hash ;
2474
2475 This time only two filters have been used -- we only need to manipulate
2476 the contents of the key, so it wasn't necessary to install any value
2477 filters.
2478
2479 =head1 Using BerkeleyDB with MLDBM
2480
2481 Both BerkeleyDB::Hash and BerkeleyDB::Btree can be used with the MLDBM
2482 module. The code fragment below shows how to open associate MLDBM with
2483 BerkeleyDB::Btree. To use BerkeleyDB::Hash just replace
2484 BerkeleyDB::Btree with BerkeleyDB::Hash.
2485
2486 use strict ;
2487 use BerkeleyDB ;
2488 use MLDBM qw(BerkeleyDB::Btree) ;
2489 use Data::Dumper;
2490
2491 my $filename = 'testmldbm' ;
2492 my %o ;
2493
2494 unlink $filename ;
2495 tie %o, 'MLDBM', -Filename => $filename,
2496 -Flags => DB_CREATE
2497 or die "Cannot open database '$filename: $!\n";
2498
2499 See the MLDBM documentation for information on how to use the module
2500 and for details of its limitations.
2501
2502 =head1 EXAMPLES
2503
2504 TODO.
2505
2506 =head1 HINTS & TIPS
2507
2508 =head2 Sharing Databases With C Applications
2509
2510 There is no technical reason why a Berkeley DB database cannot be
2511 shared by both a Perl and a C application.
2512
2513 The vast majority of problems that are reported in this area boil down
2514 to the fact that C strings are NULL terminated, whilst Perl strings
2515 are not. See L<An Example -- the NULL termination problem.> in the DBM
2516 FILTERS section for a generic way to work around this problem.
2517
2518
2519 =head2 The untie Gotcha
2520
2521 TODO
2522
2523 =head1 COMMON QUESTIONS
2524
2525 This section attempts to answer some of the more common questions that
2526 I get asked.
2527
2528
2529 =head2 Relationship with DB_File
2530
2531 Before Berkeley DB 2.x was written there was only one Perl module that
2532 interfaced to Berkeley DB. That module is called B<DB_File>. Although
2533 B<DB_File> can be build with Berkeley DB 1.x, 2.x, 3.x or 4.x, it only
2534 provides an interface to the functionality available in Berkeley DB
2535 1.x. That means that it doesn't support transactions, locking or any of
2536 the other new features available in DB 2.x or better.
2537
2538 =head2 How do I store Perl data structures with BerkeleyDB?
2539
2540 See L<Using BerkeleyDB with MLDBM>.
2541
2542 =head1 HISTORY
2543
2544 See the Changes file.
2545
2546 =head1 AVAILABILITY
2547
2548 The most recent version of B<BerkeleyDB> can always be found
2549 on CPAN (see L<perlmod/CPAN> for details), in the directory
2550 F<modules/by-module/BerkeleyDB>.
2551
2552 The official web site for Berkeley DB is F<http://www.oracle.com/technology/products/berkeley-db/db/index.html>.
2553
2554 =head1 COPYRIGHT
2555
2556 Copyright (c) 1997-2004 Paul Marquess. All rights reserved. This program
2557 is free software; you can redistribute it and/or modify it under the
2558 same terms as Perl itself.
2559
2560 Although B<BerkeleyDB> is covered by the Perl license, the library it
2561 makes use of, namely Berkeley DB, is not. Berkeley DB has its own
2562 copyright and its own license. Please take the time to read it.
2563
2564 Here are few words taken from the Berkeley DB FAQ (at
2565 F<http://www.oracle.com/technology/products/berkeley-db/db/index.html>) regarding the license:
2566
2567 Do I have to license DB to use it in Perl scripts?
2568
2569 No. The Berkeley DB license requires that software that uses
2570 Berkeley DB be freely redistributable. In the case of Perl, that
2571 software is Perl, and not your scripts. Any Perl scripts that you
2572 write are your property, including scripts that make use of Berkeley
2573 DB. Neither the Perl license nor the Berkeley DB license
2574 place any restriction on what you may do with them.
2575
2576 If you are in any doubt about the license situation, contact either the
2577 Berkeley DB authors or the author of BerkeleyDB.
2578 See L<"AUTHOR"> for details.
2579
2580
2581 =head1 AUTHOR
2582
2583 Paul Marquess E<lt>[email protected]<gt>.
2584
2585
2586 =head1 SEE ALSO
2587
2588 perl(1), DB_File, Berkeley DB.
2589
2590 =cut