Debian Codegraph @

manuals/en/utility/progs.tex

raw
1 \chapter{Volume Utility Tools}
2 \label{blb:TheUtilityChapter}
3 \index[general]{Volume Utility Tools}
4 \index[general]{Tools!Volume Utility}
5
6 This document describes the utility programs written to aid \bacula{} users and
7 developers in dealing with Volumes external to \bacula{}.
8
9 \section{Specifying the Configuration File}
10 \index[general]{Specifying the Configuration File}
11
12 Each of the following programs requires a valid
13 Storage daemon configuration file (actually, the only part of the
14 configuration file that these programs need is the \textbf{Device} resource
15 definitions). This permits the programs to find the configuration parameters
16 for your archive device (generally a tape drive). By default, they read \bfilename{bacula-sd.conf} in the current directory, but you may specify a different
17 configuration file using the \textbf{-c} option.
18
19
20 \section{Specifying a Device Name For a Tape}
21 \index[general]{Tape!Specifying a Device Name For a}
22 \index[general]{Specifying a Device Name For a Tape}
23
24 Each of these programs require a \textbf{device-name} where the Volume can be
25 found. In the case of a tape, this is the physical device name such as \bfilename{/dev/nst0} or \bfilename{/dev/rmt/0ubn} depending on your system. For the program to
26 work, it must find the identical name in the Device resource of the
27 configuration file. See below for specifying Volume names.
28
29 Please note that if you have \bacula{} running and you ant to use
30 one of these programs, you will either need to stop the Storage daemon, or
31 \btool{unmount} any tape drive you want to use, otherwise the drive
32 will \textbf{busy} because \bacula{} is using it.
33
34
35 \section{Specifying a Device Name For a File}
36 \index[general]{File!Specifying a Device Name For a}
37 \index[general]{Specifying a Device Name For a File}
38
39 If you are attempting to read or write an archive file rather than a tape, the
40 \textbf{device-name} should be the full path to the archive location including
41 the filename. The filename (last part of the specification) will be stripped
42 and used as the Volume name, and the path (first part before the filename)
43 must have the same entry in the configuration file. So, the path is equivalent
44 to the archive device name, and the filename is equivalent to the volume name.
45
46
47 \section{Specifying Volumes}
48 \index[general]{Volumes!Specifying}
49 \index[general]{Specifying Volumes}
50
51 In general, you must specify the Volume name to each of the programs below
52 (with the exception of \btool{btape}). The best method to do so is to specify a
53 \textbf{bootstrap} file on the command line with the \textbf{-b} option. As part of
54 the bootstrap file, you will then specify the Volume name or Volume names if
55 more than one volume is needed. For example, suppose you want to read tapes
56 \textbf{tape1} and \textbf{tape2}. First construct a \textbf{bootstrap} file named say,
57 \bfilename{list.bsr} which contains:
58
59 \begin{bVerbatim}
60 Volume=test1|test2
61 \end{bVerbatim}
62
63 where each Volume is separated by a vertical bar. Then simply use:
64
65 \begin{bVerbatim}
66 ./bls -b list.bsr /dev/nst0
67 \end{bVerbatim}
68
69 In the case of \bacula{} Volumes that are on files, you may simply append volumes
70 as follows:
71
72 \begin{bVerbatim}
73 ./bls /tmp/test1\|test2
74 \end{bVerbatim}
75
76 where the backslash (\textbackslash{}) was necessary as a shell escape to
77 permit entering the vertical bar (|).
78
79 And finally, if you feel that specifying a Volume name is a bit complicated
80 with a bootstrap file, you can use the \textbf{-V} option (on all programs except
81 \btool{bcopy}) to specify one or more Volume names separated by the vertical bar
82 (|). For example,
83
84 \begin{bVerbatim}
85 ./bls -V Vol001 /dev/nst0
86 \end{bVerbatim}
87
88 You may also specify an asterisk (*) to indicate that the program should
89 accept any volume. For example:
90
91 \begin{bVerbatim}
92 ./bls -V* /dev/nst0
93 \end{bVerbatim}
94
95 \section{bls}
96 \label{blb:bls}
97 \index[general]{bls}
98 \index[general]{program!bls}
99
100 \btool{bls} can be used to do an \btool{ls} type listing of a \textbf{\bacula{}} tape or
101 file. It is called:
102
103 \begin{bVerbatim}
104 Usage: bls [options] <device-name>
105 -b <file> specify a bootstrap file
106 -c <file> specify a config file
107 -d <level> specify debug level
108 -e <file> exclude list
109 -i <file> include list
110 -j list jobs
111 -k list blocks
112 (no j or k option) list saved files
113 -L dump label
114 -p proceed inspite of errors
115 -v be verbose
116 -V specify Volume names (separated by |)
117 -E Check records to detect errors
118 -? print this message
119 \end{bVerbatim}
120
121 For example, to list the contents of a tape:
122
123 \begin{bVerbatim}
124 ./bls -V Volume-name /dev/nst0
125 \end{bVerbatim}
126
127 Or to list the contents of a file:
128
129 \begin{bVerbatim}
130 ./bls /tmp/Volume-name
131 \end{bVerbatim}
132 or
133 \begin{bVerbatim}
134 ./bls -V Volume-name /tmp
135 \end{bVerbatim}
136
137 Note that, in the case of a file, the Volume name becomes the filename, so in
138 the above example, you will replace the \textbf{Volume-name} with the name of the volume
139 (file) you wrote.
140
141 Normally if no options are specified, \btool{bls} will produce the equivalent
142 output to the \btool{ls -l} command for each file on the tape. Using other
143 options listed above, it is possible to display only the Job records, only the
144 tape blocks, etc. For example:
145
146 \begin{bVerbatim}
147 ./bls /tmp/File002
148 bls: butil.c:148 Using device: /tmp
149 drwxrwxr-x 3 k k 4096 02-10-19 21:08 /home/kern/bacula/k/src/dird/
150 drwxrwxr-x 2 k k 4096 02-10-10 18:59 /home/kern/bacula/k/src/dird/CVS/
151 -rw-rw-r-- 1 k k 54 02-07-06 18:02 /home/kern/bacula/k/src/dird/CVS/Root
152 -rw-rw-r-- 1 k k 16 02-07-06 18:02 /home/kern/bacula/k/src/dird/CVS/Repository
153 -rw-rw-r-- 1 k k 1783 02-10-10 18:59 /home/kern/bacula/k/src/dird/CVS/Entries
154 -rw-rw-r-- 1 k k 97506 02-10-18 21:07 /home/kern/bacula/k/src/dird/Makefile
155 -rw-r--r-- 1 k k 3513 02-10-18 21:02 /home/kern/bacula/k/src/dird/Makefile.in
156 -rw-rw-r-- 1 k k 4669 02-07-06 18:02 /home/kern/bacula/k/src/dird/README-config
157 -rw-r--r-- 1 k k 4391 02-09-14 16:51 /home/kern/bacula/k/src/dird/authenticate.c
158 -rw-r--r-- 1 k k 3609 02-07-07 16:41 /home/kern/bacula/k/src/dird/autoprune.c
159 -rw-rw-r-- 1 k k 4418 02-10-18 21:03 /home/kern/bacula/k/src/dird/bacula-dir.conf
160 ...
161 -rw-rw-r-- 1 k k 83 02-08-31 19:19 /home/kern/bacula/k/src/dird/.cvsignore
162 bls: Got EOF on device /tmp
163 84 files found.
164 \end{bVerbatim}
165
166 \subsection{Listing Jobs}
167 \index[general]{Listing Jobs with bls}
168 \index[general]{bls!Listing Jobs}
169
170 If you are listing a Volume to determine what Jobs to restore, normally the
171 \textbf{-j} option provides you with most of what you will need as long as you
172 don't have multiple clients. For example,
173
174 \begin{bVerbatim}
175 ./bls -j -V Test1 -c stored.conf DDS-4
176 bls: butil.c:258 Using device: "DDS-4" for reading.
177 11-Jul 11:54 bls: Ready to read from volume "Test1" on device "DDS-4" (/dev/nst0).
178 Volume Record: File:blk=0:1 SessId=4 SessTime=1121074625 JobId=0 DataLen=165
179 Begin Job Session Record: File:blk=0:2 SessId=4 SessTime=1121074625 JobId=1 Level=F Type=B
180 Begin Job Session Record: File:blk=0:3 SessId=5 SessTime=1121074625 JobId=5 Level=F Type=B
181 Begin Job Session Record: File:blk=0:6 SessId=3 SessTime=1121074625 JobId=2 Level=F Type=B
182 Begin Job Session Record: File:blk=0:13 SessId=2 SessTime=1121074625 JobId=4 Level=F Type=B
183 End Job Session Record: File:blk=0:99 SessId=3 SessTime=1121074625 JobId=2 Level=F Type=B
184 Files=168 Bytes=1,732,978 Errors=0 Status=T
185 End Job Session Record: File:blk=0:101 SessId=2 SessTime=1121074625 JobId=4 Level=F Type=B
186 Files=168 Bytes=1,732,978 Errors=0 Status=T
187 End Job Session Record: File:blk=0:108 SessId=5 SessTime=1121074625 JobId=5 Level=F Type=B
188 Files=168 Bytes=1,732,978 Errors=0 Status=T
189 End Job Session Record: File:blk=0:109 SessId=4 SessTime=1121074625 JobId=1 Level=F Type=B
190 Files=168 Bytes=1,732,978 Errors=0 Status=T
191 11-Jul 11:54 bls: End of Volume at file 1 on device "DDS-4" (/dev/nst0), Volume "Test1"
192 11-Jul 11:54 bls: End of all volumes.
193 \end{bVerbatim}
194
195 shows a full save followed by two incremental saves.
196
197 Adding the \textbf{-v} option will display virtually all information that is
198 available for each record:
199
200 \subsection{Listing Blocks}
201 \index[general]{Listing Blocks with bls}
202 \index[general]{bls!Listing Blocks}
203
204 Normally, except for debugging purposes, you will not need to list \bacula{}
205 blocks (the ``primitive'' unit of \bacula{} data on the Volume). However, you can
206 do so with:
207
208 \begin{bVerbatim}
209 ./bls -k /tmp/File002
210 bls: butil.c:148 Using device: /tmp
211 Block: 1 size=64512
212 Block: 2 size=64512
213 ...
214 Block: 65 size=64512
215 Block: 66 size=19195
216 bls: Got EOF on device /tmp
217 End of File on device
218 \end{bVerbatim}
219
220 By adding the \textbf{-v} option, you can get more information, which can be
221 useful in knowing what sessions were written to the volume:
222
223 \begin{bVerbatim}
224 ./bls -k -v /tmp/File002
225 Volume Label:
226 Id : Bacula 0.9 mortal
227 VerNo : 10
228 VolName : File002
229 PrevVolName :
230 VolFile : 0
231 LabelType : VOL_LABEL
232 LabelSize : 147
233 PoolName : Default
234 MediaType : File
235 PoolType : Backup
236 HostName :
237 Date label written: 2002-10-19 at 21:16
238 Block: 1 blen=64512 First rec FI=VOL_LABEL SessId=1 SessTim=1035062102 Strm=0 rlen=147
239 Block: 2 blen=64512 First rec FI=6 SessId=1 SessTim=1035062102 Strm=DATA rlen=4087
240 Block: 3 blen=64512 First rec FI=12 SessId=1 SessTim=1035062102 Strm=DATA rlen=5902
241 Block: 4 blen=64512 First rec FI=19 SessId=1 SessTim=1035062102 Strm=DATA rlen=28382
242 ...
243 Block: 65 blen=64512 First rec FI=83 SessId=1 SessTim=1035062102 Strm=DATA rlen=1873
244 Block: 66 blen=19195 First rec FI=83 SessId=1 SessTim=1035062102 Strm=DATA rlen=2973
245 bls: Got EOF on device /tmp
246 End of File on device
247 \end{bVerbatim}
248
249 Armed with the SessionId and the SessionTime, you can extract just about
250 anything.
251
252 If you want to know even more, add a second \textbf{-v} to the command line to
253 get a dump of every record in every block.
254
255 \begin{bVerbatim}
256 ./bls -k -v -v /tmp/File002
257 bls: block.c:79 Dump block 80f8ad0: size=64512 BlkNum=1
258 Hdrcksum=b1bdfd6d cksum=b1bdfd6d
259 bls: block.c:92 Rec: VId=1 VT=1035062102 FI=VOL_LABEL Strm=0 len=147 p=80f8b40
260 bls: block.c:92 Rec: VId=1 VT=1035062102 FI=SOS_LABEL Strm=-7 len=122 p=80f8be7
261 bls: block.c:92 Rec: VId=1 VT=1035062102 FI=1 Strm=UATTR len=86 p=80f8c75
262 bls: block.c:92 Rec: VId=1 VT=1035062102 FI=2 Strm=UATTR len=90 p=80f8cdf
263 bls: block.c:92 Rec: VId=1 VT=1035062102 FI=3 Strm=UATTR len=92 p=80f8d4d
264 bls: block.c:92 Rec: VId=1 VT=1035062102 FI=3 Strm=DATA len=54 p=80f8dbd
265 bls: block.c:92 Rec: VId=1 VT=1035062102 FI=3 Strm=MD5 len=16 p=80f8e07
266 bls: block.c:92 Rec: VId=1 VT=1035062102 FI=4 Strm=UATTR len=98 p=80f8e2b
267 bls: block.c:92 Rec: VId=1 VT=1035062102 FI=4 Strm=DATA len=16 p=80f8ea1
268 bls: block.c:92 Rec: VId=1 VT=1035062102 FI=4 Strm=MD5 len=16 p=80f8ec5
269 bls: block.c:92 Rec: VId=1 VT=1035062102 FI=5 Strm=UATTR len=96 p=80f8ee9
270 bls: block.c:92 Rec: VId=1 VT=1035062102 FI=5 Strm=DATA len=1783 p=80f8f5d
271 bls: block.c:92 Rec: VId=1 VT=1035062102 FI=5 Strm=MD5 len=16 p=80f9668
272 bls: block.c:92 Rec: VId=1 VT=1035062102 FI=6 Strm=UATTR len=95 p=80f968c
273 bls: block.c:92 Rec: VId=1 VT=1035062102 FI=6 Strm=DATA len=32768 p=80f96ff
274 bls: block.c:92 Rec: VId=1 VT=1035062102 FI=6 Strm=DATA len=32768 p=8101713
275 bls: block.c:79 Dump block 80f8ad0: size=64512 BlkNum=2
276 Hdrcksum=9acc1e7f cksum=9acc1e7f
277 bls: block.c:92 Rec: VId=1 VT=1035062102 FI=6 Strm=contDATA len=4087 p=80f8b40
278 bls: block.c:92 Rec: VId=1 VT=1035062102 FI=6 Strm=DATA len=31970 p=80f9b4b
279 bls: block.c:92 Rec: VId=1 VT=1035062102 FI=6 Strm=MD5 len=16 p=8101841
280 ...
281 \end{bVerbatim}
282
283 Normally, except for deep debugging purposes, you will not need to verify \bacula{}
284 checksum blocks. However, you can do so with:
285
286 \begin{bVerbatim}
287 ./bls -j -E -V File002 FileStorage
288 bls: butil.c:297-0 Using device: "FileChgr1-Dev2" for reading.
289 14-Feb 09:43 bls JobId 0: Ready to read from volume "TestVolume001" on file device "FileChgr1-Dev2" (/tmp/regress/tmp).
290 Volume Record: File:blk=0:215 SessId=1 SessTime=1423731064 JobId=0 DataLen=180
291 14-Feb 09:43 bls JobId 0: Error: block_util.c:451 Volume data error at 0:215!
292 Block checksum mismatch in block=1 len=64512: calc=34589ccc blk=8d8c5921
293 \end{bVerbatim}
294
295 \section{bextract}
296 \label{blb:bextract}
297 \index[general]{Bextract}
298 \index[general]{program!bextract}
299
300 If you find yourself using \btool{bextract}, you probably have done
301 something wrong. For example, if you are trying to recover a file
302 but are having problems, please see the \bxlink{Restoring When Things Go Wrong}{blb:database_restore}{main}{section} of the \bmainman{}.
303
304 Normally, you will restore files by running a \textbf{Restore} Job from the
305 \textbf{Console} program. However, \btool{bextract} can be used to extract a single file
306 or a list of files from a \bacula{} tape or file. In fact, \btool{bextract} can be
307 a useful tool to restore files to an empty system assuming you are able to
308 boot, you have statically linked \btool{bextract} and you have an appropriate
309 \textbf{bootstrap} file.
310
311 Please note that some of the current limitations of bextract are:
312
313 \begin{benumerate}
314 \item It cannot restore access control lists (\acs{ACL}) that have been
315 backed up along with the file data.
316 \item It cannot restore encrypted files.
317 \item The command line length is relatively limited,
318 which means that you cannot enter a huge number of volumes. If you need to
319 enter more volumes than the command line supports, please use a bootstrap
320 file (see below).
321 \end{benumerate}
322
323
324 It is called:
325
326 \begin{bVerbatim}
327 Usage: bextract [-d debug_level] <device-name> <directory-to-store-files>
328 -b <file> specify a bootstrap file
329 -dnn set debug level to nn
330 -dt print timestamp in debug output
331 -T send debug traces to trace file
332 -e <file> exclude list
333 -i <file> include list
334 -p proceed inspite of I/O errors
335 -t read data from volume, do not write anything
336 -v verbose
337 -V specify Volume names (separated by |)
338 -? print this message
339 \end{bVerbatim}
340
341 where \textbf{device-name} is the Archive Device (raw device name or full
342 filename) of the device to be read, and \textbf{directory-to-store-files} is a
343 path prefix to prepend to all the files restored.
344
345 NOTE: On Windows systems, if you specify a prefix of say \bdirectoryname{d:/tmp}, any file that
346 would have been restored to \bdirectoryname{c:/My Documents} will be restored to
347 \bdirectoryname{d:/tmp/My Documents}. That is, the original drive specification will be
348 stripped. If no prefix is specified, the file will be restored to the original
349 drive.
350
351 \subsection{Extracting with Include or Exclude Lists}
352 \index[general]{Lists!Extracting with Include or Exclude}
353 \index[general]{Extracting with Include or Exclude Lists}
354
355 Using the \textbf{-e} option, you can specify a file containing a list of files
356 to be excluded. Wildcards can be used in the exclusion list. This option will
357 normally be used in conjunction with the \textbf{-i} option (see below). Both the
358 \textbf{-e} and the \textbf{-i} options may be specified at the same time as the
359 \textbf{-b} option. The bootstrap filters will be applied first, then the include
360 list, then the exclude list.
361
362 Likewise, and probably more importantly, with the \textbf{-i} option, you can
363 specify a file that contains a list (one file per line) of files and
364 directories to include to be restored. The list must contain the full filename
365 with the path. If you specify a path name only, all files and subdirectories
366 of that path will be restored. If you specify a line containing only the
367 filename (e.g. \bfilename{my-file.txt}) it probably will not be extracted because
368 you have not specified the full path.
369
370 For example, if the file \bfilename{include-list} contains:
371
372 \begin{bVerbatim}
373 /home/kern/bacula
374 /usr/local/bin
375 \end{bVerbatim}
376
377 Then the command:
378
379 \begin{bVerbatim}
380 ./bextract -i include-list -V Volume /dev/nst0 /tmp
381 \end{bVerbatim}
382
383 will restore from the \bacula{} archive \bfilename{/dev/nst0} all files and directories
384 in the backup from \bdirectoryname{/home/kern/bacula} and from \bdirectoryname{/usr/local/bin}. The
385 restored files will be placed in a file of the original name under the
386 directory \bdirectoryname{/tmp} (i.e. \bdirectoryname{/tmp/home/kern/bacula/...} and
387 \bdirectoryname{/tmp/usr/local/bin/...}).
388
389 \subsection{Extracting With a Bootstrap File}
390 \index[general]{File!Extracting With a Bootstrap}
391 \index[general]{Extracting With a Bootstrap File}
392
393 The \textbf{-b} option is used to specify a \textbf{bootstrap} file containing the
394 information needed to restore precisely the files you want. Specifying a
395 \textbf{bootstrap} file is optional but recommended because it gives you the most
396 control over which files will be restored. For more details on the
397 \textbf{bootstrap} file, please see
398 \bilink{Restoring Files with the Bootstrap File}{blb:BootstrapChapter}
399 chapter of this document. Note, you may also use a bootstrap file produced by
400 the \bcommandname{restore} command. For example:
401
402 \begin{bVerbatim}
403 ./bextract -b bootstrap-file /dev/nst0 /tmp
404 \end{bVerbatim}
405
406 The bootstrap file allows detailed specification of what files you want
407 restored (extracted). You may specify a bootstrap file and include and/or
408 exclude files at the same time. The bootstrap conditions will first be
409 applied, and then each file record seen will be compared to the include and
410 exclude lists.
411
412 \subsection{Extracting From Multiple Volumes}
413 \index[general]{Volumes!Extracting From Multiple}
414 \index[general]{Extracting From Multiple Volumes}
415
416 If you wish to extract files that span several Volumes, you can specify the
417 Volume names in the bootstrap file or you may specify the Volume names on the
418 command line by separating them with a vertical bar. See the section above
419 under the \btool{bls} program entitled \textbf{Listing Multiple Volumes} for more
420 information. The same techniques apply equally well to the \btool{bextract}
421 program or read the \bilink{Bootstrap}{blb:BootstrapChapter}
422 chapter of this document.
423
424 \subsection{Test Extraction}
425 \index[general]{Volumes!Test Extraction}
426
427 If you wish to ensure that all blocks of a volume are valid, you can specify
428 the \textbf{-t} flag. A bootstrap might be used with this option. It is also
429 possible to setup and use a Verify Job.
430
431 \section{bscan}
432 \label{blb:bscan}
433 \index[general]{blb:bscan}
434 \index[general]{program!bscan}
435
436 If you find yourself using this program, you have probably done something
437 wrong. For example, the best way to recover a lost or damaged \bacula{}
438 database is to reload the database by using the bootstrap file that
439 was written when you saved it (default bacula-dir.conf file).
440
441 The \btool{bscan} program can be used to re-create a database (catalog)
442 records from the backup information written to one or more Volumes. This
443 is normally needed only if one or more Volumes have been pruned or purged
444 from your catalog so that the records on the Volume are no longer in the
445 catalog, or for Volumes that you have archived. Note, if you scan in
446 Volumes that were previously purged, you will be able to do restores from
447 those Volumes. However, unless you modify the Job and File retention times
448 for the Jobs that were added by scanning, the next time you run any backup Job
449 with the same name, the records will be pruned again. Since it takes a
450 long time to scan Volumes this can be very frustrating.
451
452 With some care, \btool{bscan} can also be used to synchronize your existing
453 catalog with a Volume. Although we have never seen a case of \btool{bscan}
454 damaging a catalog, since \btool{bscan} modifies your catalog, we recommend that
455 you do a simple ASCII backup of your database before running \btool{bscan}
456 just to be sure. See \bilink{Compacting Your Database}{blb:CompactingMySQL} for
457 the details of making a copy of your database.
458
459 \btool{bscan} can also be useful in a disaster recovery situation, after the
460 loss of a hard disk, if you do not have a valid \textbf{bootstrap} file for
461 reloading your system, or if a Volume has been recycled but not overwritten,
462 you can use \btool{bscan} to re-create your database, which can then be used to
463 \textbf{restore} your system or a file to its previous state.
464
465 \btool{bscan} Is also able to re-create / list records such as Restore Object and Plugin Objects.
466
467 It is called:
468
469 \begin{bVerbatim}
470 Usage: bscan [options] <bacula-archive>
471 -b bootstrap specify a bootstrap file
472 -c <file> specify configuration file
473 -d <nn> set debug level to nn
474 -m update media info in database
475 -n <name> specify the database name (default bacula)
476 -u <user> specify database user name (default bacula)
477 -P <password> specify database password (default none)
478 -h <host> specify database host (default NULL)
479 -p proceed inspite of I/O errors
480 -r list records
481 -s synchronize or store in database
482 -v verbose
483 -V <Volumes> specify Volume names (separated by |)
484 -w <dir> specify working directory (default from conf file)
485 -? print this message
486 \end{bVerbatim}
487
488 If you are using \mysql{} or \postgresql{}, there is no need to supply a working
489 directory since in that case, \btool{bscan} knows where the databases are. However, if
490 you have provided security on your database, you may need to supply either the
491 database name (\textbf{-b} option), the user name (\textbf{-u} option), and/or the
492 password (\textbf{-p}) options.
493
494 NOTE: before \btool{bscan} can work, it needs at least a bare bones valid
495 database. If your database exists but some records are missing because
496 they were pruned, then you are all set. If your database was lost or
497 destroyed, then you must first ensure that you have the SQL program running
498 (\mysql{} or \postgresql{}), then you must create the \bacula{} database (normally
499 named bacula), and you must create the \bacula{} tables using the scripts in
500 the \bdirectoryname{cats} directory. This is explained in the
501 \bilink{Installation}{blb:CreateDatabase} chapter of the manual. Finally, before
502 scanning into an empty database, you must start and stop the Director with
503 the appropriate \bfilename{bacula-dir.conf} file so that it can create the Client and
504 Storage records which are not stored on the Volumes. Without these
505 records, scanning is unable to connect the Job records to the proper
506 client.
507
508 Forgetting for the moment the extra complications of a full rebuild of
509 your catalog, let's suppose that you did a backup to Volumes ``Vol001''
510 and ``Vol002'', then sometime later all records of one or both those
511 Volumes were pruned or purged from the
512 database. By using \btool{bscan} you can recreate the catalog entries for
513 those Volumes and then use the \bcommandname{restore} command in the Console to restore
514 whatever you want. A command something like:
515
516 \begin{bVerbatim}
517 bscan -c bacula-sd.conf -v -V Vol001\|Vol002 /dev/nst0
518 \end{bVerbatim}
519
520 will give you an idea of what is going to happen without changing
521 your catalog. Of course, you may need to change the path to the Storage
522 daemon's conf file, the Volume name, and your tape (or disk) device name. This
523 command must read the entire tape, so if it has a lot of data, it may take a
524 long time, and thus you might want to immediately use the command listed
525 below. Note, if you are writing to a disk file, replace the device name with
526 the path to the directory that contains the Volumes. This must correspond to
527 the Archive Device in the conf file.
528
529 Then to actually write or store the records in the catalog, add the \textbf{-s}
530 option as follows:
531
532 \begin{bVerbatim}
533 bscan -s -m -c bacula-sd.conf -v -V Vol001\|Vol002 /dev/nst0
534 \end{bVerbatim}
535
536 When writing to the database, if \btool{bscan} finds existing records, it will
537 generally either update them if something is wrong or leave them alone. Thus
538 if the Volumes you are scanning are all or partially in the catalog already, no
539 harm will be done to that existing data. Any missing data will simply be
540 added.
541
542 If you have multiple tapes, you should scan them with:
543
544 \begin{bVerbatim}
545 bscan -s -m -c bacula-sd.conf -v -V Vol001\|Vol002\|Vol003 /dev/nst0
546 \end{bVerbatim}
547
548 Since there is a limit on the command line length (511 bytes) accepted
549 by \btool{bscan}, if you have too many Volumes, you will need to manually
550 create a bootstrap file.
551 See the \bxlink{Restoring Files with the Bootstrap File}{blb:BootstrapChapter}{main}{chapter} of the \bmainman{}, in particular the section
552 entitled \bxdlink{Bootstrap for bscan}{blb:bscanBootstrap}{main}{section}. Basically, the \bfilename{.bsr} file for the above example might look like:
553
554 \begin{bVerbatim}
555 Volume=Vol001
556 Volume=Vol002
557 Volume=Vol003
558 \end{bVerbatim}
559
560 Note: \btool{bscan} does not support supplying Volume names on the
561 command line and at the same time in a bootstrap file. Please
562 use only one or the other.
563
564 You should, always try to specify the tapes in the order they are written.
565 If you do not, any Jobs that span a volume may not be fully or properly
566 restored. However, \btool{bscan} can handle scanning tapes that are not sequential. Any
567 incomplete records at the end of the tape will simply be ignored in that
568 case. If you are simply repairing an existing catalog, this may be OK, but
569 if you are creating a new catalog from scratch, it will leave your database
570 in an incorrect state. If you do not specify all necessary Volumes on a
571 single \btool{bscan} command, \btool{bscan} will not be able to correctly restore the
572 records that span two volumes. In other words, it is much better to
573 specify two or three volumes on a single \btool{bscan} command (or in a \bfilename{.bsr} file)
574 rather than run \btool{bscan} two or three times, each with a single volume.
575
576 Note, the restoration process using \btool{bscan} is not identical to the original
577 creation of the catalog data. This is because certain data such as Client
578 records and other non-essential data such
579 as volume reads, volume mounts, etc is not stored on the Volume, and thus is
580 not restored by \btool{bscan}. The results of bscanning are, however, perfectly valid,
581 and will permit restoration of any or all the files in the catalog using the
582 normal \bacula{} console commands. If you are starting with an empty catalog
583 and expecting \btool{bscan} to reconstruct it, you may be a bit disappointed, but
584 at a minimum, you must ensure that your \bfilename{bacula-dir.conf} file is the same
585 as what it previously was -- that is, it must contain all the appropriate
586 Client resources so that they will be recreated in your new database
587 \textbf{before} running \btool{bscan}. Normally when the Director starts, it will recreate
588 any missing Client records in the catalog. Another problem you will have
589 is that even if the Volumes (Media records) are recreated in the database,
590 they will not have their autochanger status and slots properly set. As a
591 result, you will need to repair that by using the \bcommandname{update slots}
592 command. There may be other considerations as well. Rather than
593 bscanning, you should always attempt to recover you previous catalog
594 backup.
595
596
597 \subsection{Using \btool{bscan} to Compare a Volume to an existing Catalog}
598 \index[general]{Catalog!Using \btool{bscan} to Compare a Volume to an existing}
599 \index[general]{Using \btool{bscan} to Compare a Volume to an existing Catalog}
600
601 If you wish to compare the contents of a Volume to an existing catalog
602 without changing the catalog, you can safely do so if and only if you do
603 {\bf not} specify either the {\bf -m} or the {\bf -s} options in order to
604 avoid modifying the catalog. However, not every record that is stored in
605 the catalog is stored on the volumes, so we don't particularly
606 recommend using {\bf bscan} other than for testing. Probably the best
607 way to compare what is on a Volume to what is in the Catalog is to use
608 one of the Verify Job options.
609
610 \subsection{Using \btool{bscan} to Recreate a Catalog from a Volume}
611 \index[general]{Volume!Using \btool{bscan} to Recreate a Catalog from a Volume}
612 \index[general]{Using \btool{bscan} to Recreate a Catalog from a Volume}
613
614 This is the mode for which \btool{bscan} is most useful. You can either
615 \btool{bscan} into a freshly created catalog, or directly into your existing catalog
616 (after having made an ASCII copy as described above). Normally, you should
617 start with a freshly created catalog that contains no data.
618
619 Starting with a single Volume named \textbf{TestVolume1}, you run a command such
620 as:
621
622 \begin{bVerbatim}
623 ./bscan -V TestVolume1 -v -s -m -c bacula-sd.conf /dev/nst0
624 \end{bVerbatim}
625
626 If there is more than one volume, simply append it to the first one separating
627 it with a vertical bar. You may need to precede the vertical bar with a
628 forward slash escape the shell -- e.g.
629 \textbf{TestVolume1\textbackslash{}|TestVolume2}. The \textbf{-v} option was added for
630 verbose output (this can be omitted if desired). The \textbf{-s} option that
631 tells \btool{bscan} to store information in the database. The physical device
632 name \bfilename{/dev/nst0} is specified after all the options.
633
634 \textbf{} For example, after having done a full backup of a directory, then two
635 incrementals, I reinitialized the SQLite database as described above, and
636 using the \bfilename{bootstrap.bsr} file noted above, I entered the following command:
637
638 \begin{bVerbatim}
639 ./bscan -b bootstrap.bsr -v -s -c bacula-sd.conf /dev/nst0
640 \end{bVerbatim}
641
642 which produced the following output:
643
644 \begin{bVerbatim}
645 bscan: bscan.c:182 Using Database: bacula, User: bacula
646 bscan: bscan.c:673 Created Pool record for Pool: Default
647 bscan: bscan.c:271 Pool type "Backup" is OK.
648 bscan: bscan.c:632 Created Media record for Volume: TestVolume1
649 bscan: bscan.c:298 Media type "DDS-4" is OK.
650 bscan: bscan.c:307 VOL_LABEL: OK for Volume: TestVolume1
651 bscan: bscan.c:693 Created Client record for Client: Rufus
652 bscan: bscan.c:769 Created new JobId=1 record for original JobId=2
653 bscan: bscan.c:717 Created FileSet record "Kerns Files"
654 bscan: bscan.c:819 Updated Job termination record for new JobId=1
655 bscan: bscan.c:905 Created JobMedia record JobId 1, MediaId 1
656 bscan: Got EOF on device /dev/nst0
657 bscan: bscan.c:693 Created Client record for Client: Rufus
658 bscan: bscan.c:769 Created new JobId=2 record for original JobId=3
659 bscan: bscan.c:708 Fileset "Kerns Files" already exists.
660 bscan: bscan.c:819 Updated Job termination record for new JobId=2
661 bscan: bscan.c:905 Created JobMedia record JobId 2, MediaId 1
662 bscan: Got EOF on device /dev/nst0
663 bscan: bscan.c:693 Created Client record for Client: Rufus
664 bscan: bscan.c:769 Created new JobId=3 record for original JobId=4
665 bscan: bscan.c:708 Fileset "Kerns Files" already exists.
666 bscan: bscan.c:819 Updated Job termination record for new JobId=3
667 bscan: bscan.c:905 Created JobMedia record JobId 3, MediaId 1
668 bscan: Got EOF on device /dev/nst0
669 bscan: bscan.c:652 Updated Media record at end of Volume: TestVolume1
670 bscan: bscan.c:428 End of Volume. VolFiles=3 VolBlocks=57 VolBytes=10,027,437
671 \end{bVerbatim}
672
673 The key points to note are that \btool{bscan} prints a line when each major
674 record is created. Due to the volume of output, it does not print a line for
675 each file record unless you supply the \textbf{-v} option twice or more on the
676 command line.
677
678 In the case of a Job record, the new JobId will not normally be the same as
679 the original Jobid. For example, for the first JobId above, the new JobId is
680 1, but the original JobId is 2. This is nothing to be concerned about as it is
681 the normal nature of databases. \btool{bscan} will keep everything straight.
682
683 Although \btool{bscan} claims that it created a Client record for Client: Rufus
684 three times, it was actually only created the first time. This is normal.
685
686 You will also notice that it read an end of file after each Job (Got \acs{EOF} on
687 device \ldots{}). Finally the last line gives the total statistics for the \btool{bscan}.
688
689 If you had added a second \textbf{-v} option to the command line, \bacula{} would
690 have been even more verbose, dumping virtually all the details of each Job
691 record it encountered.
692
693 Now if you start \bacula{} and enter a \textbf{list jobs} command to the console
694 program, you will get:
695
696 \begin{bVerbatim}
697 +-------+----------+------------------+------+-----+----------+----------+---------+
698 | JobId | Name | StartTime | Type | Lvl | JobFiles | JobBytes | JobStat |
699 +-------+----------+------------------+------+-----+----------+----------+---------+
700 | 1 | kernsave | 2002-10-07 14:59 | B | F | 84 | 4180207 | T |
701 | 2 | kernsave | 2002-10-07 15:00 | B | I | 15 | 2170314 | T |
702 | 3 | kernsave | 2002-10-07 15:01 | B | I | 33 | 3662184 | T |
703 +-------+----------+------------------+------+-----+----------+----------+---------+
704 \end{bVerbatim}
705
706 which corresponds virtually identically with what the database contained
707 before it was re-initialized and restored with \btool{bscan}. All the Jobs and Files
708 found on the tape are restored including most of the Media record. The Volume
709 (Media) records restored will be marked as \textbf{Full} so that they cannot be
710 rewritten without operator intervention.
711
712 It should be noted that \btool{bscan} cannot restore a database to the exact
713 condition it was in previously because a lot of the less important information
714 contained in the database is not saved to the tape. Nevertheless, the
715 reconstruction is sufficiently complete, that you can run \bcommandname{restore}
716 against it and get valid results.
717
718 An interesting aspect of restoring a catalog backup using \btool{bscan} is
719 that the backup was made while \bacula{} was running and writing to a tape. At
720 the point the backup of the catalog is made, the tape \bacula{} is writing to
721 will have say 10 files on it, but after the catalog backup is made, there
722 will be 11 files on the tape \bacula{} is writing. This there is a difference
723 between what is contained in the backed up catalog and what is actually on
724 the tape. If after restoring a catalog, you attempt to write on the same
725 tape that was used to backup the catalog, \bacula{} will detect the difference
726 in the number of files registered in the catalog compared to what is on the
727 tape, and will mark the tape in error.
728
729 There are two solutions to this problem. The first is possibly the simplest
730 and is to mark the volume as Used before doing any backups. The second is
731 to manually correct the number of files listed in the Media record of the
732 catalog. This procedure is documented elsewhere in the manual and involves
733 using the \bcommandname{update volume} command in \btool{bconsole}.
734
735 \subsection{Using \btool{bscan} to Correct the Volume File Count}
736 \index[general]{Using \btool{bscan} to Correct the Volume File Count}
737 \index[general]{Count!Using \btool{bscan} to Correct the Volume File Count}
738
739 If the Storage daemon crashes during a backup Job, the catalog will not be
740 properly updated for the Volume being used at the time of the crash. This
741 means that the Storage daemon will have written say 20 files on the tape, but
742 the catalog record for the Volume indicates only 19 files.
743
744 \bacula{} refuses to write on a tape that contains a different number of files
745 from what is in the catalog. To correct this situation, you may run a
746 \btool{bscan} with the \textbf{-m} option (but \textbf{without} the \textbf{-s} option) to
747 update only the final Media record for the Volumes read.
748
749 \subsection{After \btool{bscan}}
750 \index[general]{After \btool{bscan}}
751 \index[general]{Bscan!After}
752
753 If you use \btool{bscan} to enter the contents of the Volume into an existing
754 catalog, you should be aware that the records you entered may be immediately
755 pruned during the next job, particularly if the Volume is very old or had been
756 previously purged. To avoid this, after running \btool{bscan}, you can manually
757 set the volume status (VolStatus) to \textbf{Read-Only} by using the \bcommandname{update}
758 command in the catalog. This will allow you to restore from the volume without
759 having it immediately purged. When you have restored and backed up the data,
760 you can reset the VolStatus to \textbf{Used} and the Volume will be purged from
761 the catalog.
762
763 \section{bcopy}
764 \label{blb:bcopy}
765 \index[general]{Bcopy}
766 \index[general]{program!bcopy}
767
768 The \btool{bcopy} program can be used to copy one \textbf{\bacula{}} archive file to
769 another. For example, you may copy a tape to a file, a file to a tape, a file
770 to a file, or a tape to a tape. For tape to tape, you will need two tape
771 drives. (a later version is planned that will buffer it to disk). In the
772 process of making the copy, no record of the information written to the new
773 Volume is stored in the catalog. This means that the new Volume, though it
774 contains valid backup data, cannot be accessed directly from existing catalog
775 entries. If you wish to be able to use the Volume with the Console restore
776 command, for example, you must first \btool{bscan} the new Volume into the catalog.
777
778 \subsection{bcopy Command Options}
779 \index[general]{Options!bcopy Command}
780 \index[general]{Bcopy Command Options}
781
782 \begin{bVerbatim}
783 Usage: bcopy [-d debug_level] <input-archive> <output-archive>
784 -b bootstrap specify a bootstrap file
785 -c <file> specify configuration file
786 -dnn set debug level to nn
787 -i specify input Volume names (separated by |)
788 -o specify output Volume names (separated by |)
789 -p proceed inspite of I/O errors
790 -v verbose
791 -w dir specify working directory (default /tmp)
792 -? print this message
793 \end{bVerbatim}
794
795 By using a \textbf{bootstrap} file, you can copy parts of a \bacula{} archive file
796 to another archive.
797
798 One of the objectives of this program is to be able to recover as much data as
799 possible from a damaged tape. However, the current version does not yet have
800 this feature.
801
802 As this is a new program, any feedback on its use would be appreciated. In
803 addition, I only have a single tape drive, so I have never been able to test
804 this program with two tape drives.
805
806 \section{btape}
807 \label{blb:btape}
808 \index[general]{Btape}
809 \index[general]{program!btape}
810
811 This program permits a number of elementary tape operations via a tty command
812 interface. It works only with tapes and not with other kinds of \bacula{}
813 storage media (DVD, File, \ldots{}). The \textbf{test} command, described below,
814 can be very useful for testing older tape drive compatibility problems.
815 Aside from initial testing of tape drive compatibility with \textbf{\bacula{}},
816 \btool{btape} will be mostly used by developers writing new tape drivers.
817
818 \btool{btape} can be dangerous to use with existing \textbf{\bacula{}} tapes because
819 it will relabel a tape or write on the tape if so requested regardless that
820 the tape may contain valuable data, so please be careful and use it only on
821 blank tapes.
822
823 To work properly, \btool{btape} needs to read the Storage daemon's configuration
824 file. As a default, it will look for \bfilename{bacula-sd.conf} in the current
825 directory. If your configuration file is elsewhere, please use the \textbf{-c}
826 option to specify where.
827
828 The physical device name must be specified on the command line, and this
829 same device name must be present in the Storage daemon's configuration file
830 read by \btool{btape}
831
832 \begin{bVerbatim}
833 Usage: btape <options> <device_name>
834 -b <file> specify bootstrap file
835 -c <file> set configuration file to file
836 -d <nn> set debug level to nn
837 -p proceed inspite of I/O errors
838 -s turn off signals
839 -v be verbose
840 -? print this message.
841 \end{bVerbatim}
842
843 \subsection{Using btape to Verify your Tape Drive}
844 \index[general]{Using btape to Verify your Tape Drive}
845 \index[general]{Drive!Using btape to Verify your Tape}
846
847 An important reason for this program is to ensure that a Storage daemon
848 configuration file is defined so that \bacula{} will correctly read and write
849 tapes.
850
851 It is highly recommended that you run the \bcommandname{test} command before running
852 your first \bacula{} job to ensure that the parameters you have defined for your
853 storage device (tape drive) will permit \textbf{\bacula{}} to function properly. You
854 only need to mount a blank tape, enter the command, and the output should be
855 reasonably self explanatory. Please see the
856 \bxlink{Tape Testing}{blb:TapeTestingChapter}{problems}{chapter} of the \bproblemman{} for
857 the details.
858
859 \subsection{btape Commands}
860 \index[general]{Btape Commands}
861 \index[general]{Commands!btape}
862
863 The full list of commands are:
864
865 \begin{bVerbatim}
866 Command Description
867 ======= ===========
868 autochanger test autochanger
869 bsf backspace file
870 bsr backspace record
871 cap list device capabilities
872 clear clear tape errors
873 eod go to end of Bacula data for append
874 eom go to the physical end of medium
875 fill fill tape, write onto second volume
876 unfill read filled tape
877 fsf forward space a file
878 fsr forward space a record
879 help print this command
880 label write a Bacula label to the tape
881 load load a tape
882 quit quit btape
883 rawfill use write() to fill tape
884 readlabel read and print the Bacula tape label
885 rectest test record handling functions
886 rewind rewind the tape
887 scan read() tape block by block to EOT and report
888 scanblocks Bacula read block by block to EOT and report
889 speed report drive speed
890 status print tape status
891 test General test Bacula tape functions
892 weof write an EOF on the tape
893 wr write a single Bacula block
894 rr read a single record
895 qfill quick fill command
896 \end{bVerbatim}
897
898 The most useful commands are:
899
900 \begin{bdescription}
901 \item [ test ] test writing records and EOF marks and reading them back.
902 \item [ fill ] completely fill a volume with records, then write a few records
903 on a second volume, and finally, both volumes will be read back.
904 This command writes blocks containing random data, so your drive will
905 not be able to compress the data, and thus it is a good test of
906 the real physical capacity of your tapes.
907 \item [ readlabel ] read and dump the label on a Bacula tape.
908 \item [ cap ] list the device capabilities as defined in the configuration
909 file and as perceived by the Storage daemon.
910 \end{bdescription}
911
912 The \bcommandname{readlabel} command can be used to display the details of a Bacula
913 tape label. This can be useful if the physical tape label was lost or damaged.
914
915
916 In the event that you want to relabel a \textbf{\bacula{}}, you can simply use the
917 \bcommandname{label} command which will write over any existing label. However, please
918 note for labeling tapes, we recommend that you use the \bcommandname{label} command in
919 the \textbf{Console} program since it will never overwrite a valid \bacula{} tape.
920
921 \subsubsection*{Testing your Tape Drive}
922 \label{blb:sec:btapespeed}
923
924 To determine the best configuration of your tape drive, you can run the new
925 \bcommandname{speed} command available in the \btool{btape} program.
926
927 This command can have the following arguments:
928 \begin{bdescription}
929 \item [\texttt{file\_size=n}] Specify the Maximum File Size for this test
930 (between 1 and 5GB). This counter is in GB.
931 \item [\texttt{nb\_file=n}] Specify the number of file to be written. The amount
932 of data should be greater than your memory ($file\_size*nb\_file$).
933 \item [\texttt{skip\_zero}] This flag permits to skip tests with constant
934 data.
935 \item [\texttt{skip\_random}] This flag permits to skip tests with random
936 data.
937 \item [\texttt{skip\_raw}] This flag permits to skip tests with raw access.
938 \item [\texttt{skip\_block}] This flag permits to skip tests with \bacula{} block
939 access.
940 \end{bdescription}
941
942 \begin{bVerbatim}
943 *speed file_size=3 skip_raw
944 btape.c:1078 Test with zero data and bacula block structure.
945 btape.c:956 Begin writing 3 files of 3.221 GB with blocks of 129024 bytes.
946 ++++++++++++++++++++++++++++++++++++++++++
947 btape.c:604 Wrote 1 EOF to "Drive-0" (/dev/nst0)
948 btape.c:406 Volume bytes=3.221 GB. Write rate = 44.128 MB/s
949 ...
950 btape.c:383 Total Volume bytes=9.664 GB. Total Write rate = 43.531 MB/s
951
952 btape.c:1090 Test with random data, should give the minimum throughput.
953 btape.c:956 Begin writing 3 files of 3.221 GB with blocks of 129024 bytes.
954 +++++++++++++++++++++++++++++++++++++++++++
955 btape.c:604 Wrote 1 EOF to "Drive-0" (/dev/nst0)
956 btape.c:406 Volume bytes=3.221 GB. Write rate = 7.271 MB/s
957 +++++++++++++++++++++++++++++++++++++++++++
958 ...
959 btape.c:383 Total Volume bytes=9.664 GB. Total Write rate = 7.365 MB/s
960 \end{bVerbatim}
961
962 When using compression, the random test will give your the minimum throughput
963 of your drive . The test using constant string will give you the maximum speed
964 of your hardware chain. (cpu, memory, scsi card, cable, drive, tape).
965
966 You can change the block size in the Storage Daemon configuration file.
967
968 \section{Other Programs}
969 \index[general]{Programs!Other}
970 \index[general]{Other Programs}
971
972 The following programs are general utility programs and in general do not need
973 a configuration file nor a device name.
974
975 \section{bsmtp}
976 \label{blb:bsmtp}
977 \index[general]{Bsmtp}
978 \index[general]{program!bsmtp}
979
980 \btool{bsmtp} is a simple mail transport program that permits more flexibility
981 than the standard mail programs typically found on Unix systems. It can even
982 be used on Windows machines.
983
984 It is called:
985
986 \begin{bVerbatim}
987 Usage: bsmtp [-f from] [-h mailhost] [-s subject] [-c copy] [recipient ...]
988 -c set the Cc: field
989 -dnn set debug level to nn
990 -f set the From: field
991 -h use mailhost:port as the bsmtp server
992 -l limit the lines accepted to nn
993 -s set the Subject: field
994 -? print this message.
995 \end{bVerbatim}
996
997 If the \textbf{-f} option is not specified, \btool{bsmtp} will use your userid. If
998 the option \textbf{-h} is not specified \btool{bsmtp} will use the value in the environment
999 variable \textbf{bsmtpSERVER} or if there is none \textbf{localhost}. By default
1000 port 25 is used.
1001
1002 If a line count limit is set with the \textbf{-l} option, \btool{bsmtp} will
1003 not send an email with a body text exceeding that number of lines. This
1004 is especially useful for large restore job reports where the list of
1005 files restored might produce very long mails your mail-server would
1006 refuse or crash. However, be aware that you will probably suppress the
1007 job report and any error messages unless you check the log file written
1008 by the Director (see the messages resource in this manual for details).
1009
1010
1011 \textbf{recipients} is a space separated list of email recipients.
1012
1013 The body of the email message is read from standard input.
1014
1015 An example of the use of \btool{bsmtp} would be to put the following statement
1016 in the \textbf{Messages} resource of your \bfilename{bacula-dir.conf} file. Note, these
1017 commands should appear on a single line each.
1018
1019 \begin{bVerbatim}
1020 mailcommand = "/home/bacula/bin/bsmtp -h mail.domain.com -f \"\(Bacula\) %r\"
1021 -s \"Bacula: %t %e of %c %l\" %r"
1022 operatorcommand = "/home/bacula/bin/bsmtp -h mail.domain.com -f \"\(Bacula\) %r\"
1023 -s \"Bacula: Intervention needed for %j\" %r"
1024 \end{bVerbatim}
1025
1026 Where you replace \bdirectoryname{/home/bacula/bin} with the path to your \textbf{\bacula{}}
1027 binary directory, and you replace \url{mail.domain.com} with the fully
1028 qualified name of your \btool{bsmtp} (email) server, which normally listens on port
1029 \bvalue{25}. For more details on the substitution characters (e.g. \%r) used in the
1030 above line, please see the documentation of the \bxlink{Mail Command in the Messages Resource}{blb:mailcommand}{main}{chapter} of the \bmainman{}.
1031
1032 It is \bhighlight{highly} recommended that you test one or two cases by hand to make sure
1033 that the \textbf{mailhost} that you specified is correct and that it will accept
1034 your email requests. Since \btool{bsmtp} always uses a TCP connection rather
1035 than writing in the spool file, you may find that your \textbf{from} address is
1036 being rejected because it does not contain a valid domain, or because your
1037 message is caught in your spam filtering rules. Generally, you should specify
1038 a fully qualified domain name in the \textbf{from} field, and depending on
1039 whether your \btool{bsmtp} gateway is Exim or Sendmail, you may need to modify the
1040 syntax of the from part of the message. Please test.
1041
1042 When running \btool{bsmtp} by hand, you will need to terminate the message by
1043 entering a ctl-d in column 1 of the last line.
1044 % TODO: is "column" the correct terminology for this?
1045
1046 If you are getting incorrect dates (e.g. 1970) and you are
1047 running with a non-English language setting, you might try adding
1048 a LANG=''en\_US'' immediately before the \btool{bsmtp} call.
1049
1050 In general, \btool{bsmtp} attempts to cleanup email addresses that you
1051 specify in the from, copy, mailhost, and recipient fields, by adding
1052 the necessary \lt{} and \gt{} characters around the address part. However,
1053 if you include a \textbf{display-name} (see RFC 5332), some SMTP servers
1054 such as Exchange may not accept the message if the \textbf{display-name} is
1055 also included in \lt{} and \gt{}. As mentioned above, you must test, and
1056 if you run into this situation, you may manually add the \lt{} and \gt{}
1057 to the \bacula{} \textbf{mailcommand} or \textbf{operatorcommand} and when
1058 \btool{bsmtp} is formatting an address if it already contains a \lt{} or
1059 \gt{} character, it will leave the address unchanged.
1060
1061 \section{dbcheck}
1062 \label{blb:dbcheck}
1063 \index[general]{dbcheck}
1064 \index[general]{program!dbcheck}
1065 \btool{dbcheck} is a simple program that will search for logical
1066 inconsistencies in the \bacula{} tables in your database, and optionally fix them.
1067 It is a database maintenance routine, in the sense that it can
1068 detect and remove unused rows, but it is not a database repair
1069 routine. To repair a database, see the tools furnished by the
1070 database vendor. Normally \btool{dbcheck} should never need to be run,
1071 but if \bacula{} has crashed or you have a lot of Clients, Pools, or
1072 Jobs that you have removed, it could be useful.
1073
1074 The \btool{dbcheck} program can be found in
1075 the \bdirectoryname{/opt/bacula/bin} directory.
1076
1077 It is called:
1078
1079 \begin{bVerbatim}
1080 Usage: dbcheck [-c config ] [-B] [-C catalog name] [-d debug_level]
1081 <working-directory> <bacula-database> <user> <password> [<dbhost>] [<dbport>]
1082 -b batch mode
1083 -C catalog name in the director conf file
1084 -c Director conf filename
1085 -B print catalog configuration and exit
1086 -d <nn> set debug level to <nn>
1087 -dt print timestamp in debug output
1088 -f fix inconsistencies
1089 -v verbose
1090 -? print this message
1091 \end{bVerbatim}
1092
1093 If the \textbf{-B} option is specified, \btool{dbcheck} will print out catalog
1094 information in a simple text based format. This is useful to backup it in a
1095 secure way.
1096
1097 \begin{bVerbatim}
1098 $ dbcheck -B
1099 catalog=MyCatalog
1100 db_type=SQLite
1101 db_name=regress
1102 db_driver=
1103 db_user=regress
1104 db_password=
1105 db_address=
1106 db_port=0
1107 db_socket=
1108 \end{bVerbatim}
1109
1110 If the \textbf{-c} option is given with the Director's conf file, there is no
1111 need to enter any of the command line arguments, in particular the working
1112 directory as \btool{dbcheck} will read them from the file.
1113
1114 If the \textbf{-f} option is specified, \btool{dbcheck} will repair (\textbf{fix}) the
1115 inconsistencies it finds. Otherwise, it will report only.
1116
1117 If the \textbf{-b} option is specified, \btool{dbcheck} will run in batch mode, and
1118 it will proceed to examine and fix (if -f is set) all programmed inconsistency
1119 checks. If the \textbf{-b} option is not specified, \btool{dbcheck} will enter
1120 interactive mode and prompt with the following:
1121
1122 \begin{bVerbatim}
1123 Hello, this is the database check/correct program.
1124 Please select the function you want to perform.
1125 1) Toggle modify database flag
1126 2) Toggle verbose flag
1127 3) Repair bad Filename records
1128 4) Repair bad Path records
1129 5) Eliminate duplicate Filename records
1130 6) Eliminate duplicate Path records
1131 7) Eliminate orphaned Jobmedia records
1132 8) Eliminate orphaned File records
1133 9) Eliminate orphaned Path records
1134 10) Eliminate orphaned Filename records
1135 11) Eliminate orphaned FileSet records
1136 12) Eliminate orphaned Client records
1137 13) Eliminate orphaned Job records
1138 14) Eliminate all Admin records
1139 15) Eliminate all Restore records
1140 16) All (3-15)
1141 17) Quit
1142 Select function number:
1143 \end{bVerbatim}
1144
1145 By entering 1 or 2, you can toggle the modify database flag (-f option) and
1146 the verbose flag (-v). It can be helpful and reassuring to turn off the modify
1147 database flag, then select one or more of the consistency checks (items 3
1148 through 9) to see what will be done, then toggle the modify flag on and re-run
1149 the check.
1150
1151 The inconsistencies examined are the following:
1152
1153 \begin{bitemize}
1154 \item Duplicate filename records. This can happen if you accidentally run two
1155 copies of \bacula{} at the same time, and they are both adding filenames
1156 simultaneously. It is a rare occurrence, but will create an inconsistent
1157 database. If this is the case, you will receive error messages during Jobs
1158 warning of duplicate database records. If you are not getting these error
1159 messages, there is no reason to run this check.
1160 \item Repair bad Filename records. This checks and corrects filenames that
1161 have a trailing slash. They should not.
1162 \item Repair bad Path records. This checks and corrects path names that do
1163 not have a trailing slash. They should.
1164 \item Duplicate path records. This can happen if you accidentally run two
1165 copies of \bacula{} at the same time, and they are both adding filenames
1166 simultaneously. It is a rare occurrence, but will create an inconsistent
1167 database. See the item above for why this occurs and how you know it is
1168 happening.
1169 \item Orphaned JobMedia records. This happens when a Job record is deleted
1170 (perhaps by a user issued SQL statement), but the corresponding JobMedia
1171 record (one for each Volume used in the Job) was not deleted. Normally, this
1172 should not happen, and even if it does, these records generally do not take
1173 much space in your database. However, by running this check, you can
1174 eliminate any such orphans.
1175 \item Orphaned File records. This happens when a Job record is deleted
1176 (perhaps by a user issued SQL statement), but the corresponding File record
1177 (one for each Volume used in the Job) was not deleted. Note, searching for
1178 these records can be \textbf{very} time consuming (i.e. it may take hours) for a
1179 large database. Normally this should not happen as \bacula{} takes care to
1180 prevent it. Just the same, this check can remove any orphaned File records.
1181 It is recommended that you run this once a year since orphaned File records
1182 can take a large amount of space in your database. You might
1183 want to ensure that you have indexes on JobId, FilenameId, and
1184 PathId for the File table in your catalog before running this
1185 command.
1186 \item Orphaned Path records. This condition happens any time a directory is
1187 deleted from your system and all associated Job records have been purged.
1188 During standard purging (or pruning) of Job records, \bacula{} does not check
1189 for orphaned Path records. As a consequence, over a period of time, old
1190 unused Path records will tend to accumulate and use space in your database.
1191 This check will eliminate them. It is recommended that you run this
1192 check at least once a year.
1193 \item Orphaned Filename records. This condition happens any time a file is
1194 deleted from your system and all associated Job records have been purged.
1195 This can happen quite frequently as there are quite a large number of files
1196 that are created and then deleted. In addition, if you do a system update or
1197 delete an entire directory, there can be a very large number of Filename
1198 records that remain in the catalog but are no longer used.
1199
1200 During standard purging (or pruning) of Job records, \bacula{} does not check
1201 for orphaned Filename records. As a consequence, over a period of time, old
1202 unused Filename records will accumulate and use space in your database. This
1203 check will eliminate them. It is strongly recommended that you run this check
1204 at least once a year, and for large database (more than 200 Megabytes), it is
1205 probably better to run this once every 6 months.
1206 \item Orphaned Client records. These records can remain in the database long
1207 after you have removed a client.
1208 \item Orphaned Job records. If no client is defined for a job or you do not
1209 run a job for a long time, you can accumulate old job records. This option
1210 allow you to remove jobs that are not attached to any client (and thus
1211 useless).
1212 \item All Admin records. This command will remove all Admin records,
1213 regardless of their age.
1214 \item All Restore records. This command will remove all Restore records,
1215 regardless of their age.
1216 \end{bitemize}
1217
1218
1219 If you are using \mysql{}, \btool{dbcheck} will ask you if you want to create temporary
1220 indexes to speed up orphaned Path and Filename elimination.
1221
1222 Mostly for \postgresql{} users, we provide a pure SQL script \btool{dbcheck} replacement
1223 in \bfilename{examples/database/dbcheck.sql} that works with global queries instead
1224 of many small queries like \btool{dbcheck}. Execution instructions are at the top of
1225 the script and you will need to type \texttt{COMMIT} at the end to validate
1226 modifications.
1227
1228 If you are using bweb or brestore, don't eliminate orphaned Path, else you will
1229 have to rebuild \texttt{brestore\_pathvisibility} and
1230 \texttt{brestore\_pathhierarchy} indexes.
1231
1232 By the way, I personally run \btool{dbcheck} only where I have messed up
1233 my database due to a bug in developing \bacula{} code, so normally
1234 you should never need to run \btool{dbcheck} in spite of the
1235 recommendations given above, which are given so that users don't
1236 waste their time running \btool{dbcheck} too often.
1237
1238 \section{bregex}
1239 \label{blb:bregex}
1240 \index[general]{bregex}
1241 \index[general]{program!bregex}
1242
1243 \btool{bregex} is a simple program that will allow you to test
1244 regular expressions against a file of data. This can be useful
1245 because the regex libraries on most systems differ, and in
1246 addition, regex expressions can be complicated.
1247
1248 \btool{bregex} is found in the \bdirectoryname{src/tools} directory and it is
1249 normally installed with your system binaries. To run it, use:
1250
1251 \begin{bVerbatim}
1252 Usage: bregex [-d debug_level] -f <data-file>
1253 -f specify file of data to be matched
1254 -l suppress line numbers
1255 -n print lines that do not match
1256 -? print this message.
1257 \end{bVerbatim}
1258
1259 The \bbracket{data-file} is a filename that contains lines
1260 of data to be matched (or not) against one or more patterns.
1261 When the program is run, it will prompt you for a regular
1262 expression pattern, then apply it one line at a time against
1263 the data in the file. Each line that matches will be printed
1264 preceded by its line number. You will then be prompted again
1265 for another pattern.
1266
1267 Enter an empty line for a pattern to terminate the program. You
1268 can print only lines that do not match by using the -n option,
1269 and you can suppress printing of line numbers with the -l option.
1270
1271 This program can be useful for testing regex expressions to be
1272 applied against a list of filenames.
1273
1274 \section{bwild}
1275 \label{blb:bwild}
1276 \index[general]{bwild}
1277 \index[general]{program!bwild}
1278
1279 \btool{bwild} is a simple program that will allow you to test
1280 wild-card expressions against a file of data.
1281
1282 \btool{bwild} is found in the \bdirectoryname{src/tools} directory and it is
1283 normally installed with your system binaries. To run it, use:
1284
1285 \begin{bVerbatim}
1286 Usage: bwild [-d debug_level] -f <data-file>
1287 -f specify file of data to be matched
1288 -l suppress line numbers
1289 -n print lines that do not match
1290 -? print this message.
1291 \end{bVerbatim}
1292
1293 The \bbracket{data-file} is a filename that contains lines
1294 of data to be matched (or not) against one or more patterns.
1295 When the program is run, it will prompt you for a wild-card
1296 pattern, then apply it one line at a time against
1297 the data in the file. Each line that matches will be printed
1298 preceded by its line number. You will then be prompted again
1299 for another pattern.
1300
1301 Enter an empty line for a pattern to terminate the program. You
1302 can print only lines that do not match by using the -n option,
1303 and you can suppress printing of line numbers with the -l option.
1304
1305 This program can be useful for testing wild expressions to be
1306 applied against a list of filenames.
1307
1308 \section{testfind}
1309 \label{blb:testfind}
1310 \index[general]{Testfind}
1311 \index[general]{program!testfind}
1312
1313 \btool{testfind} permits listing of files using the same search engine that is
1314 used for the \textbf{Include} resource in Job resources. Note, much of the
1315 functionality of this program (listing of files to be included) is present in
1316 the
1317 \bilink{\bcommandname{estimate} command}{blb:estimate} in the Console program.
1318
1319 The original use of testfind was to ensure that \bacula{}'s file search engine
1320 was correct and to print some statistics on file name and path length.
1321 However, you may find it useful to see what bacula would do with a given
1322 \textbf{Include} resource. The \btool{testfind} program can be found in the
1323 \bdirectoryname{\bbracket{bacula-source}/src/tools} directory of the source distribution.
1324 Though it is built with the make process, it is not normally ``installed''.
1325
1326 It is called:
1327
1328 \begin{bVerbatim}
1329 Usage: testfind [-d debug_level] [-] [pattern1 ...]
1330 -a print extended attributes (Win32 debug)
1331 -dnn set debug level to nn
1332 - read pattern(s) from stdin
1333 -? print this message.
1334 Patterns are used for file inclusion -- normally directories.
1335 Debug level>= 1 prints each file found.
1336 Debug level>= 10 prints path/file for catalog.
1337 Errors are always printed.
1338 Files/paths truncated is a number with len> 255.
1339 Truncation is only in the catalog.
1340 \end{bVerbatim}
1341
1342 Where a pattern is any filename specification that is valid within an
1343 \textbf{Include} resource definition. If none is specified, \textbf{/} (the root
1344 directory) is assumed. For example:
1345
1346 \begin{bVerbatim}
1347 ./testfind /bin
1348 \end{bVerbatim}
1349
1350 Would print the following:
1351
1352 \begin{bVerbatim}
1353 Dir: /bin
1354 Reg: /bin/bash
1355 Lnk: /bin/bash2 -> bash
1356 Lnk: /bin/sh -> bash
1357 Reg: /bin/cpio
1358 Reg: /bin/ed
1359 Lnk: /bin/red -> ed
1360 Reg: /bin/chgrp
1361 ...
1362 Reg: /bin/ipcalc
1363 Reg: /bin/usleep
1364 Reg: /bin/aumix-minimal
1365 Reg: /bin/mt
1366 Lnka: /bin/gawk-3.1.0 -> /bin/gawk
1367 Reg: /bin/pgawk
1368 Total files : 85
1369 Max file length: 13
1370 Max path length: 5
1371 Files truncated: 0
1372 Paths truncated: 0
1373 \end{bVerbatim}
1374
1375 Even though \btool{testfind} uses the same search engine as \textbf{\bacula{}}, each
1376 directory to be listed, must be entered as a separate command line entry or
1377 entered one line at a time to standard input if the \textbf{-} option was
1378 specified.
1379
1380 Specifying a debug level of one (i.e. \textbf{-d1}) on the command line will
1381 cause \btool{testfind} to print the raw filenames without showing the \bacula{}
1382 internal file type, or the link (if any). Debug levels of 10 or greater cause
1383 the filename and the path to be separated using the same algorithm that is
1384 used when putting filenames into the Catalog database.
1385
1386 \input{bimagemgr-chapter}