Debian Codegraph @

manuals/en/main/newfeatures.tex

raw
1 \chapter{New Features in 15.0.0}
2
3 \subsection{Security Enhancements}
4
5 The password information that might be present in some specific plugin FileSets is now hidden
6 from the \texttt{status client} console output.
7
8 \smallskip
9
10 New security events are generated after incorrect network connections.
11
12 \smallskip
13
14 The restricted console have now a better control over various bconsole commands such
15 last \textbf{list}, \textbf{restore}, ...
16
17 \subsection{Storage Daemon Encryption}
18
19 The Bacula Storage Daemon can now encrypt the data at the volume level to
20 enhance security of data at rest. The volumes cannot be read by a system that doesn't
21 have the correct encryption keys.
22
23 More information can be found in the Security chapter of the documentation
24 \vref{volumeencryption}.
25
26 \subsection{Malware Detection}
27
28 Bacula allows you to configure your jobs to detect known Malware. The detection
29 can be done at the end of the Backup job and/or with a Verify job. The Malware
30 database can be downloaded from different providers, the default is set to
31 abuse.ch.
32 If a Backup job detects a malware in the backup content, an error is reported
33 and the Job status is adapted.
34
35 \begin{bVerbatim}
36 20-Sep 12:26 zog8-dir JobId 9: Start Backup JobId 9, Job=backup.2022-09-20_12.26.30_13
37 ...
38 20-Sep 12:26 zog8-dir JobId 9: [DI0002] Checking file metadata for Malwares
39 20-Sep 12:26 zog8-dir JobId 9: Error: [DE0007] Found Malware(s) on JobIds 9
40 Build OS: x86_64-pc-linux-gnu archlinux
41 JobId: 9
42 Job: backup.2022-09-20_12.26.30_13
43 Backup Level: Full
44 ...
45 Last Volume Bytes: 659,912,644 (659.9 MB)
46 Non-fatal FD errors: 1
47 SD Errors: 0
48 FD termination status: OK
49 SD termination status: OK
50 Termination: Backup OK -- with warnings
51 \end{bVerbatim}
52
53 The list of the Malware detected in a given Job can be displayed with the \texttt{list files type=malware}
54 command.
55
56 \begin{bVerbatim}
57
58 *list files type=malware jobid=1
59 +-------+-----------------------------+---------------+----------+
60 | jobid | filename | description | source |
61 +-------+-----------------------------+---------------+----------+
62 | 1 | /tmp/regress/build/po/fr.po | Malware found | abuse.ch |
63 +-------+-----------------------------+---------------+----------+
64
65 \end{bVerbatim}
66
67 See the \vref{MalwareDetection} section of this manual for more information.
68
69
70 \subsection{TOTP Console Authentication Plugin}
71
72 The TOTP (Time based One Time Password) Authentication Plugin is compatible
73 with the RFC 6238. Many smartphone apps are available to store the keys and
74 compute the TOTP code.
75
76 The standard password and encryption mechanisms will still be used to accept an
77 incoming console connection. Once accepted, the Console will prompt for a
78 second level of authentication with a TOTP secret key generated from a shared
79 token.
80
81 To enable this feature, you needed to install the
82 \texttt{bacula-totp-dir-plugin} package on your Director system, then to
83 set the \texttt{PluginDirectory} directive of the \texttt{Director} resource
84 and configure the \texttt{AuthenticationPlugin} directive of a given restricted
85 Console in the Director configuration file.
86
87 \begin{bVerbatim}
88 # in bacula-dir.conf
89 Director {
90 Name = myname-dir
91 ...
92 Plugin Directory = /opt/bacula/plugins
93 }
94
95 Console {
96 Name = "totpconsole"
97 Password = "xxx"
98
99 Authentication Plugin = "totp"
100
101 CommandACL = *all*
102 JobACL = *all*
103 ClientAcl = *all*
104 PoolACL = *all*
105 DirectoryACL = *all*
106 WhereACL = *all*
107 FileSetACL = *all*
108 StorageACL = *all*
109 CatalogACL = *all*
110 ScheduleACL = *all*
111 }
112 \end{bVerbatim}
113
114 The matching Console configuration in \texttt{bconsole.conf} has no
115 extra settings compared to a standard restricted Console.
116
117 \begin{bVerbatim}
118 # in bconsole.conf
119 Console {
120 Name = totpconsole
121 Password = "xxx" # Same as in bacula-dir.conf/Console
122 }
123 Director {
124 Name = mydir-dir
125 Address = localhost
126 Password = notused
127 }
128 \end{bVerbatim}
129
130 At the first console connection, if the TLS link is correctly setup
131 (using the shared secret key), the plugin will generate a specific
132 random key for the console and display a QR code in the console
133 output. The user must then scan the QR code with his smartphone using
134 an app such as Aegis (Opensource) or Google Authenticator. The plugin
135 can also be configured to send the QR code via an extranal program.
136
137 More information can be found in the main manual on \vref{TOTPPlugin}.
138
139 \subsection{Storage Group Enhancements}
140
141 The StorageGroup feature is now compatible with Copy and Migration jobs.
142
143 \subsubsection{FreeSpace Storage Policy}
144
145 This policy queries each Storage Daemon in the list for its FreeSpace (as a sum
146 of devices specified in the Storage Daemon config) and sort the list by
147 FreeSpace returned.
148
149 \subsubsection{LastBackedUpTo Storage Policy}
150
151 This policy ensures that job is backed up to the storage where the same job (of
152 the same level, i.e. Full or Incremental) has been backed up the longest time
153 ago. The goal is to split the jobs to improve redundancy.
154
155 \subsubsection{LastBackupedToStore Storage Policy}
156
157 This policy ensures that job is backed up on the storage where the same job
158 (with same level i.e. Full or Incremental) has been backed up to the longest
159 time ago. The goal is to spread the jobs to improve redundancy.
160
161 \subsubsection{FreeSpaceLeastUsed Storage Policy}
162
163 This policy ensures that a job is backed up to the storage with the most free
164 space and least running jobs. Within the candidates storages, the least used
165 one will be selected. Candidate storages are determined by the
166 \texttt{StorageGroupPolicyThreshold} Job directive. If MaxFreeSpace is the largest
167 amount of free space for all storages in the group, a storage will be a
168 candidate if its free space is above $$MaxFreeSpace - StorageGroupPolicyThreshold$$.
169
170 For example:
171 \begin{bVerbatim}
172
173 with StorageGroupPolicyThreshold=100MB
174
175 and storages free space being:
176 Storage1 = 500GB free
177 Storage2 = 200GB free
178 Storage3 = 400GB free
179 storage4 = 500GB free
180
181 In this case MaxFreeSpace=500GB.
182
183 Storage 1, 4 and 3 are candidates.
184
185 If 5 jobs are running on Storage1, 2 on Storage4, and 3 on Storage3 then Storage4 will be the selected storage.
186 \end{bVerbatim}
187
188 Storage Groups can be used as follows (as part of Job and Pool configuration):
189
190 \begin{bVerbatim}
191 Job {
192 ...
193 Storage = File1, File2, File3
194 ...
195 }
196 Pool {
197 ...
198 Storage = File4, File5, File6
199 StorageGroupPolicy = FreeSpaceLeastUsed
200 StorageGroupPolicyThreshold = 200 MB
201 ...
202 }
203 \end{bVerbatim}
204
205 \subsection{FileDaemon Security Enhancements}
206 \subsubsection{Allowed Backup and Restore Directories}
207
208 New FileDaemon directives lets the deamon control which client directories are
209 allowed for backup on a per-director basis. Directives can be specified as
210 comma-separated list of directories. The simplest version of the
211 \texttt{AllowedBackupDirectories} and \texttt{ExcludedBackupDirectories}
212 directives may look as follows:
213
214 \begin{bVerbatim}
215 # in bacula-fd.conf
216 Director {
217 Name = myname-dir
218 ...
219 AllowedBackupDirectories = "/path/to/allowed/directory"
220 }
221 Director {
222 Name = my-other-dir
223 ...
224 ExcludedBackupDirectories = "/path/to/excluded/directory"
225
226 }
227 \end{bVerbatim}
228
229 This directive works on the FD side, and is fully independent of the include/exclude part of the Fileset
230 defined in the Director's config file. Nothing is backed up if none of the files defined in
231 the Fileset is inside FD's allowed directory.
232
233 \subsubsection{Allowed Restore Directories}
234 This new directive lets the deamon control which directories are allowed
235 to be used as a restore destination on a per-director basis. Directive can be specified as a list of directories.
236 The simplest version of the AllowedRestoreDirectories directive may look as follows:
237 \begin{bVerbatim}
238 # in bacula-fd.conf
239 Director {
240 Name = myname-dir
241 ...
242 AllowedRestoreDirectories = "/path/to/directory"
243
244 }
245 \end{bVerbatim}
246
247 \subsubsection{Allowed Script Directories}
248 New FileDaemon directive lets the deamon control directories which from the Director can execute client's scripts and programs
249 (e.g. using the Runscript feature or with the Fileset's 'File=' directive).
250 Directive can be specified as a list of directories.
251 The simplest version of the AllowedScriptDirectories directive may look as follows:
252 \begin{bVerbatim}
253 # in bacula-fd.conf
254 Director {
255 Name = myname-dir
256 ...
257 AllowedScriptDirectories = "/path/to/directory"
258
259 }
260 \end{bVerbatim}
261
262 When this directive is set, Bacula is also checking programs to be run against set of not-allowed characters.
263 When following resource:
264 \begin{bVerbatim}
265 FileSet {
266 Name = "Fileset_1"
267 Include {
268 File = "\\|/path/to/binary &"
269 }
270 }
271 \end{bVerbatim}
272 is defined inside the Director's config file, Bacula won't backup any file for such Fileset.
273 It's because of the '\&' character, which is not allowed when the 'Allowed Script Directories' is used on the Client's side.
274 Full list of not-allowed characters:
275 \begin{bVerbatim}
276 $ ! ; \ & < > ` ( )
277 \end{bVerbatim}
278 % $
279
280 To disable all command sent by the Director, it is possible to use the following configuration in your FileDaemon:
281 \begin{bVerbatim}
282 AllowedScriptDirectories = none
283 \end{bVerbatim}
284
285 \subsection{Antivirus Plugin}
286
287 The FileDaemon Antivirus plugin provides integration between the ClamAV Antivirus daemon
288 and Bacula Verify Jobs, allowing post-backup virus detection within Bacula Enterprise.
289
290 More information can be found in the Antivirus Plugin user's guide.
291
292 \subsection{Volume Protection}
293 \paragraph{Warning}: This feature is only for file-based devices.
294
295 This feature can only be used if Bacula is run as a \btool{systemd} service because only then, with proper capabilities set for
296 the daemon, it's allowed to manage Volume Files' attributes.
297 The `show storage` bconsole command informs if Bacula has needed capabilities.
298
299 For File-based volumes Bacula will set the \bvalue{Append Only} attribute during the first backup job that uses the new volume.
300 Note that the flag is set when Bacula actually uses volume for the first time, not when the volume is labelled (either automatically
301 or using the `label` command).
302 This will prevent Volumes to lost data by being ovewritten.
303
304 The \bvalue{Append Only} file attribute is cleared when the volume is being relabeled.
305
306 Bacula is now also able to set the \bvalue{Immutable} file attribute on a file volume which is marked as Full.
307 Note that as of now, Immutable flag is set only for Full volumes, setting it for other statuses (e.g. 'Used' volumes)
308 may come in the future releases.
309
310 When a volume is Full and has the \bvalue{Immutable} flag set, it cannot be relabeled and reused until the expiration
311 period elapses. This helps to protect volumes from being reused too early, according to the protection period set.
312
313 If Volume's filesystem does not support the \bvalue{Append only} or \bvalue{Immutable} flags, a proper warning message is printed in the job log and
314 Bacula proceeds with the usual backup workflow.
315
316 There are three new directives available to set on a per-device basis to control the the Volume Protection behavior:
317 \begin{bdescription}
318 \item [SetVolumeAppendOnly]: Determines if Bacula should set the \bvalue{Append\_Only} attribute when writing on the volume for the rist time.
319 \item [SetVolumeImmutable]: Determines if Bacula should set the \bvalue{Immutable} attribute when marking volume as 'Full'.
320 \item [Minimum Volume Protection Time]: Specifies how much time has to elapse before Bacula is able to clear the attribute
321 \end{bdescription}
322
323 Note:
324 Both Append Only and Immutable flags set for volumes cannot be modified using catalog-related commands,
325 e.g. purging a volume won't clear the Immutable flag, that can only be done when MinimumVolumeProtection time expires.
326
327 If administrator wants to manually remove file attributes, \btool{chattr} must be used:
328 \begin{itemize}
329 \item [chattr -i path-to-vol/VolumeName] - removes the Immutable flag
330 \item [chattr -a path-to-vol/VolumeName] - removes the Append Only flag
331 \end{itemize}
332 This should not be needed, and it should be used only in a state of emergency.
333 chattr tool must be used with user with proper capabilities to set/clear file attributes (e.g. root)
334
335
336 In some cases, for example when the status of the Volume is changed by the
337 Director via the \texttt{update volume} command, the Storage Daemon will not be
338 able to change the permission on the Volume. Some Volumes may have the Full/Used
339 status without the proper protection.
340
341 The command \texttt{update volumeprotect} is designed to determine the list of the
342 volumes that are not protected and connect the Storage Daemon to update the
343 permissions. It can be executed in an Admin job once a day.
344
345 \begin{bVerbatim}
346 *update
347 Update choice:
348 1: Volume parameters
349 2: Pool from resource
350 3: Slots from autochanger
351 4: Long term statistics
352 5: Snapshot parameters
353 6: Volume protection attributes on Storage Daemon
354 Choose catalog item to update (1-6): 6
355 Found 1 volumes with status Used/Full that must be protected
356 Connected to Storage "File2" at zog8:8103 with TLS
357 3000 Marking volume "Vol-0009" as read-only.
358
359 \end{bVerbatim}
360
361 or via \texttt{update volumeprotect}
362
363 \begin{bVerbatim}
364 *update volumeprotect
365 Found 1 volumes with status Used/Full that must be protected
366 Connected to Storage "File2" at zog8:8103 with TLS
367 3000 Marking volume "Vol-0009" as read-only.
368
369 \end{bVerbatim}
370
371 The command can be scheduled in an Admin job
372
373 \begin{bVerbatim}
374 Job {
375 Name = adm-update-protected
376 Type = Admin
377 Runscript {
378 Console = "update volumeprotect"
379 RunsOnClient = no
380 RunsWhen = Before
381 }
382 JobDefs = DefaultJob
383 }
384
385 \end{bVerbatim}
386
387 \subsection{ZStandard FileSet Compression Option}
388
389 The ZSTD compression algorithm is now available in the FileSet option directive
390 \texttt{Compression}. It is possible to configure ZSTD level 1 \texttt{zstd1},
391 level 10 \texttt{zstd10} and level 19 \texttt{zstd19}. The default \texttt{zstd}
392 compression is 10.
393
394 \subsection{Kubernetes Plugin}
395
396 The Bacula Kubernetes plugin will save all the important Kubernetes resources
397 which make up the application or service. The plugin can now use the CSI
398 Snapshot method to backup persistent volumes.
399
400 \subsection{Amazon Cloud Driver}
401
402 A new Amazon Cloud Driver is available for beta testing. In the long term, it
403 will enhance and replace the existing S3 cloud driver. The \texttt{aws} tool
404 provided by Amazon is needed to use this cloud driver. The Amazon cloud
405 driver is available within the bacula-cloud-storage-s3 package and can be
406 enabled by using ``Driver = Amazon'' in the Cloud Storage Daemon resource.
407
408 \subsection{Misc}
409
410 \subsubsection{New Storage Daemon Disk Volume Format}
411
412 Bacula version 15.0 uses a new volume version named \texttt{BB03}. The new format adds
413 the support for Volume Encryption, and the previous 32bits CRC32 checksum was
414 replaced by the faster 64bits \texttt{XXH64}.
415
416 Volumes written with the BB03 format can only be read by Bacula
417 version 15 or later. Old BB02 volumes can still be restored, and Volumes
418 may start with BB02 blocks, and continue with BB03 blocks.
419
420 It is not possible to use the \texttt{Volume Encryption=yes} directive on a volume
421 that was labeled using the BB02 format. In that case, the volume will be
422 automatically marked as Used.
423
424
425 \subsubsection{Progress Status for Copy/Migration Jobs}
426
427 The \textbf{status director} command can now report the progress of Copy and Migration Jobs.
428
429 \subsubsection{Scheduled Job List}
430
431 The \textbf{status director} bconsole command has been updated to limit the number
432 of scheduled jobs listed by default to 50.
433
434 The keyword \textbf{limit} can be used to choose how many lines will be printed.
435
436
437 \begin{bVerbatim}
438
439 * status director limit=5
440 127.0.0.1-dir Version: 14.1.5 (26 October 2022) x86_64-pc-linux-gnu archlinux
441 Daemon started 31-Oct-22 17:27, conf reloaded 31-Oct-2022 17:27:55
442 Jobs: run=2, running=0 max=4 mode=1,2010
443 Crypto: fips=no crypto=OpenSSL 1.0.2u 20 Dec 2019
444 Heap: heap=675,840 smbytes=774,259 max_bytes=2,058,994 bufs=2,470 max_bufs=4,024
445 Res: njobs=24 nclients=1 nstores=3 npools=1 ncats=1 nfsets=25 nscheds=3
446 Plugin: ldap totp
447
448 Scheduled Jobs (5/1440):
449 Level Type Pri Scheduled Job Name Volume
450 ===================================================================================
451 Full Backup 10 31-Oct-22 17:29 NightlySave0 TestVolume001
452 Full Backup 10 31-Oct-22 17:30 NightlySave1 TestVolume001
453 Full Backup 10 31-Oct-22 17:31 NightlySave2 TestVolume001
454 Full Backup 10 31-Oct-22 17:32 NightlySave3 TestVolume001
455 Full Backup 10 31-Oct-22 17:33 NightlySave4 TestVolume001
456
457 5 scheduled Jobs over 1440 are displayed. Use the limit parameter to display more Jobs.
458 ====
459
460 Running Jobs:
461 Console connected using TLS at 31-Oct-22 17:27
462 No Jobs running.
463 ====
464
465 Terminated Jobs:
466 JobId Level Files Bytes Status Finished Name
467 ====================================================================
468 1 Full 35 5.070 M OK 31-Oct-22 15:50 NightlySave
469 2 Full 35 5.070 M OK 31-Oct-22 17:27 NightlySave
470 3 Full 35 5.070 M OK 31-Oct-22 17:28 NightlySave
471
472 ====
473
474 \end{bVerbatim}
475
476
477 The APIv2 json output support has been added for the \textbf{status dir scheduled}
478 command.
479
480 \subsubsection{Catalog Enhancements}
481
482 \subsubsection*{FileSet Content Overview}
483
484 Now, the Catalog stores an overview of the FileSet definition in the **Content**
485 field. If the FileSet handles files and directories, the Content field will be
486 set to **files**. If any plugins are used, each plugin will be inserted into
487 the Content field.
488
489 \begin{bVerbatim}
490
491 *sql
492 SELECT Content FROM FileSet;
493 +------------------------------+
494 | content |
495 +------------------------------+
496 | bpipe |
497 +------------------------------+
498
499 \end{bVerbatim}
500
501 \subsubsection*{New Job Attributes}
502
503 New SQL attributes have been added to the Job table such as isVirtualFull,
504 Encrypted, LastReadStorageId, WriteStorageId, Rate, CompressRatio,
505 StatusInfo, and so on.
506
507 \subsubsection*{New Media Attributes}
508
509 New SQL attributes have been added to the Media table such as **UseProtect**, **Protected**,
510 **VolEncrypted**
511
512
513 \subsubsection{Search Function}
514
515 Bconsole has a new \textbf{.search} command to search into the catalog
516 accross client, job and volumes.
517
518 \begin{bVerbatim}
519 * .search text=sometext
520 volume=
521 client=sometext-fd
522 job=
523 \end{bVerbatim}
524
525 The text to be searched must have 4 characters at the minimum.
526
527 \subsubsection{Advanced Job Queue Control}
528
529 The Job RunScript feature has been enhanced to control the start of a Job
530 inside the Run Queue. When a Job is starting, the Director controls that
531 resources are available for the Job to start properly, if these resources are
532 not available, the Job will stay in the queue, waiting to acquire them.
533
534 It is now possible to execute a script to control any kind of external and
535 custom resources and decide when a Job should start. For example, a script
536 might control the load average of a server before to start a Job to find
537 the best execution time.
538
539 More information can be found in the Director documentation \vref{Director:Job:Runscript}.
540
541
542 \subsubsection{Director's PID File timestamp}
543
544 The Director PID file timestamp is now updated after a successful \textbf{reload}
545 command. External tools can use this information to perform certain actions
546 such as to clear the cache.
547
548 \subsubsection{RunScript Enhancements}
549 \index[general]{RunScript Enhancements}
550 A new Director RunScript {\bf RunsWhen} keyword of {\bf AtJobCompletion} has been
551 implemented, which runs the command after at the end of job and can update the
552 job status if the command fails.
553
554 \begin{bVerbatim}
555 Job {
556 ...
557 Runscript {
558 RunsOnClient = no
559 RunsWhen = AtJobCompletion
560 Command = "mail command"
561 AbortJobOnError = yes
562 }
563 }
564 \end{bVerbatim}
565 It has been added because the \textbf{RunsWhen} keyword \textbf{After} was not
566 designed to update the job status if the command fails.
567
568
569 \subsubsection{JSON Output}
570
571 The console has been improved to support a JSON output to list catalog objects
572 and various daemon output. The new ``.jlist'' command is a shortcut of the
573 standard ``list'' command and will display the results in a JSON table. All
574 options and filters of the ``list'' command can be used in the ``.jlist''
575 command. Only catalog objects are listed with the ``list'' or ``.jlist'' commands.
576 Resources such as Schedule, FileSets, etc... are not handled by the ``list'' command.
577
578 See the ``help list'' bconsole output for more information about the ``list''
579 command. The Bacula configuration can be displayed in JSON format with the
580 standard ``bdirjson'', ``bsdjson'', ``bfdjson'' and ``bbconsjson'' tools.
581
582 \begin{bVerbatim}
583 *.jlist jobs
584 {"type": "jobs", "data":[{"jobid": 1,"job": "CopyJobSave.2021-10-04_18.35.55_03",...
585 \end{bVerbatim}
586
587 \begin{bVerbatim}
588 *.api 2 api_opts=j
589 *.status dir header
590 {"header":{"name":"127.0.0.1-dir","version":"12.8.2 (09 September 2021)"...
591 \end{bVerbatim}
592
593
594 \chapter{New Features in 13.0.0}
595
596 \subsection{Kubernetes Plugin}
597
598 The Bacula Kubernetes plugin will save all the important Kubernetes resources
599 which make up the application or service. The plugin has the following features:
600
601 \begin{bitemize}
602 \item Kubernetes cluster resources configuration backup
603 \item Ability to restore single Kubernetes configuration resource
604 \item Ability to restore Kubernetes resource configuration to local directory
605 \item Kubernetes Persistent Volumes data backup and restore
606 \item Ability to restore Kubernetes Persistent Volumes data to local directory
607 \item Ability to use new Kubernetes CSI driver snapshot features to perform Persistent Volume data backup using the clone method
608 \item Ability to execute user defined commands on required Pod containers to prepare and clean data backup
609 \item Configure Kubernetes workload backup requirements directly with Pod annotations
610 \end{bitemize}
611
612 More information about the Kubernetes plugin can be found on \vref{kubernetesplugin}.
613
614 \subsection{Storage Group}
615 It is now possible to configure more than one Storage resource for each Job/Pool.
616 Storage can be specified as a comma separated list of Storage resources to use.
617
618 Along with specifying a Storage list it is now possible to specify a Storage Group Policy which
619 will be used for determining which Storage will be used for the Job.
620 If no policy is specified, Bacula tries to use the first available Storage from the list.
621
622 If the first few storage daemons are unavailable due to network problems; broken or unreachable for some other reason,
623 Bacula will use the first one from the list (sorted according to the policy used) which is network reachable and healthy.
624
625 Currently supported policies are:
626 \begin{itemize}
627 \item [ListedOrder] - This is the default policy. It uses the first available storage from the list provided
628 \item [LeastUsed] - This policy scans all storage daemons from the list and chooses the one with the least
629 number of Jobs currently running
630 \end{itemize}
631
632 Storage Groups may be used as follows (as a part of Job and Pool resources):
633 \begin{bVerbatim}
634 Job {
635 ...
636 Storage = File1, File2, File3
637 ...
638 }
639
640 Pool {
641 ...
642 Storage = File4, File5, File6
643 StorageGroupPolicy = LeastUsed
644 ...
645 }
646 \end{bVerbatim}
647
648 When a Job or Pool with a Storage Group is used, the user will observe some messages related to the
649 choice of Storage such as:
650
651 \begin{bVerbatim}
652 messages
653 31-maj 19:23 VBox-dir JobId 1: Start Backup JobId 1, Job=StoreGroupJob.2021-05-31_19.23.36_03
654 31-maj 19:23 VBox-dir JobId 1: Possible storage choices: "File1, File2"
655 31-maj 19:23 VBox-dir JobId 1: Storage daemon "File1" didn't accept Device "FileChgr1-Dev1" because: 3924 Device "FileChgr1-Dev1" not in SD Device resources or no matching Media Type or is disabled.
656 31-maj 19:23 VBox-dir JobId 1: Selected storage: File2, device: FileChgr2-Dev1, StorageGroupPolicy: "ListedOrder"
657 \end{bVerbatim}
658
659
660 \subsection{New Accurate Option to Save Only File's Metadata}
661
662 The new 'o' Accurate directive option for a Fileset has been added to
663 save only the metadata of a file when possible.
664
665 \smallskip{}
666
667 The new 'o' option should be used in conjunction with one of the signature
668 checking options (1, 2, 3, or 5). When the 'o' option is specified, the signature is
669 computed only for files that have one of the other accurate options specified
670 triggering a backup of the file (for example an inode change, a permission
671 change, etc...).
672
673 \smallskip{}
674
675 In cases where only the file's metadata has changed (ie: the signature is
676 identical), only the file's attributes will be backed up. If the file's data
677 has been changed (hence a different singature), the file will be backed up in
678 the usual way (attributes as well as the file's contents will be saved on the
679 volume).
680
681 \smallskip{}
682
683 For example:
684 \begin{bVerbatim}
685 Job {
686 Name = JobTest
687 JobDefs = DefaultJob
688 FileSet = TestFS
689 Accurate = yes
690 }
691
692 FileSet {
693 Name = TestFS
694 Options {
695 Signature = MD5
696 Accurate = pino5
697 }
698 File = /
699 }
700 \end{bVerbatim}
701
702 The backup job will compare permission bits, inodes, number of links and,
703 if any of it changes, it will also compute file's signature to verify if only
704 the metadata must be backed up or if the full file must be saved.
705
706 \subsection{External LDAP Console Authentication}
707
708 The new Bacula Plugable Authentication Module (BPAM) API framework introduced
709 in Bacula 13.0 comes with the first plugin which handles user authentication
710 against any LDAP Directory Server (including OpenLDAP and Active Directory). To
711 enable the build of the LDAP Console Authentication plugin, add the \texttt{ldap}
712 options to your \texttt{./configure} command line as shown below:
713
714 \begin{bVerbatim}
715 ./configure --with-ldap --enable-ldap-bpam [... other options]
716 \end{bVerbatim}
717
718
719 \begin{bVerbatim}
720 # bconsole
721 *status director
722 ...
723 Plugin: ldap-dir.so
724 ...
725 \end{bVerbatim}
726
727 When the LDAP plugin is loaded you can configure a named console resource to use
728 LDAP to authenticate users. BConsole will prompt for a User and Password and it
729 will be verified by the Director. TLS PSK (activated by default) is
730 recommended. To use this plugin, you have to specify the \texttt{PluginDirectory}
731 Director resource directive, and add a new console resource directive
732 \texttt{Authentication Plugin} as shown below:
733
734 \begin{bVerbatim}
735 Director {
736 ...
737 Plugin Directory = /opt/bacula/plugins
738 }
739
740 Console {
741 Name = "ldapconsole"
742 Password = "xxx"
743
744 # New directive
745 Authentication Plugin = "ldap:<parameters>"
746 ...
747 }
748 \end{bVerbatim}
749
750 where \texttt{parameters} are the space separated list of one or more plugin parameters:
751
752 \begin{itemize}
753 \item [url] - LDAP Directory service location, i.e. "url=ldap://10.0.0.1/"
754 \item [binddn] - DN used to connect to LDAP Directory service to perform required \texttt{query}
755 \item [bindpass] - DN password used to connect to LDAP Directory service to perform required \texttt{query}
756 \item [query] - A query performed on the LDAP Directory serice to find user for authentication
757 The \texttt{query} string is composed as \texttt{<basedn>/<filter>}. Where `\texttt{<basedn>} is a DN search starting
758 point and \texttt{<filter>} is a standard LDAP search object filter which support dynamic string
759 substitution: \texttt{\%u} will be replaced by credential's username and \texttt{\%p} by credential's
760 password, i.e. \texttt{query=dc=bacula,dc=com/(cn=\%u)}.
761 \item [starttls] - Instruct the BPAM LDAP Plugin to use the \texttt{StartTLS} extension if the LDAP Directory service
762 supports it and fallback to no TLS if this extension is not available.
763 \item [starttlsforce] - Does the same as the `starttls` setting does but reports error on fallback.
764 \end{itemize}
765
766 Working configuration examples:
767
768 \smallskip
769 bacula-dir.conf - Console resource configuration for BPAM LDAP Plugin with OpenLDAP authentication example.
770
771 \begin{bVerbatim}
772 Console {
773 Name = "bacula_ldap_console"
774 Password = "xxx"
775
776 # New directive (on a single line)
777 Authentication Plugin = "ldap: url=ldap://ldapsrv/ binddn=cn=root,dc=bacula,dc=com
778 bindpass=secret query=dc=bacula,dc=com/(cn=%u) starttls"
779 ...
780 }
781 \end{bVerbatim}
782
783
784 bacula-dir.conf - Console resource configuration for BPAM LDAP Plugin with Active Directory authentication example.
785
786 \begin{bVerbatim}
787 Console {
788 Name = "bacula_ad_console"
789 Password = "xxx"
790
791 # New directive (on a single line)
792 Authentication Plugin = "ldap: url=ldaps://ldapsrv/
793 binddn=cn=bacula,ou=Users,dc=bacula,dc=com bindpass=secret
794 query=dc=bacula,dc=com/(&(objectCategory=person)(objectClass=user)(sAMAccountName=%u))"
795 ...
796 }
797 \end{bVerbatim}
798
799 \subsection{New Baculum features}
800
801 The following features were introduced in version 11.0.6
802
803 \subsubsection{New Advanced Schedule Settings}
804
805 The schedule function has been reworked to support all possible schedule
806 settings. Now it can be configured in five different ways: hourly, daily,
807 weekly, monthly, and also custom for various uncommon settings.
808
809 \subsubsection{New Copy and Migrate Job Wizards}
810
811 Using the new wizards it is possible to easily create new copy and migrate jobs.
812 The Wizard's steps are prepared and optimized to make the process as easy as possible.
813
814 \subsubsection{New Director Page}
815
816 This new page provides two major functions: The Director status and the
817 Director configuration. The first one is for displaying in a graphical and
818 textual way the Director status. The second one is for configuring all Director
819 resources (Jobs, Clients, Pools, Storage ...etc.). From one place users
820 can configure various Director resources.
821
822 \subsubsection{Bulk Delete Volumes Action}
823
824 This feature is available on the media page. Apart from prune and purge bulk
825 actions, now users are also able to bulk delete volumes. It is possible by selecting
826 volumes in the volume table and using this new delete volumes action.
827
828 \subsubsection{Loading Pages Optimization}
829
830 Pages have been optimized to faster loading. The entire Baculum Web interface
831 works faster.
832
833 \subsubsection{Directive Documentation}
834
835 In all Bacula component configurations, for almost all directives, official Bacula documentation has been added. It is available directly within the web interface.
836
837 \subsection{Tag Support}
838
839 It is now possible to assign custom Tags to various catalog records in Bacula such as:
840 \begin{itemize}
841 \item Volume
842 \item Client
843 \item Job
844 \end{itemize}
845
846 \begin{bVerbatim}
847 *tag
848 Automatically selected Catalog: MyCatalog
849 Using Catalog "MyCatalog"
850 Available Tag operations:
851 1: Add
852 2: Delete
853 3: List
854 Select Tag operation (1-3): 1
855 Available Tag target:
856 1: Client
857 2: Job
858 3: Volume
859 Select Tag target (1-3): 1
860 The defined Client resources are:
861 1: 127.0.0.1-fd
862 2: test1-fd
863 3: test2-fd
864 4: test-rst-fd
865 5: test-bkp-fd
866 Select Client (File daemon) resource (1-5): 1
867 Enter the Tag value: test1
868 1000 Tag added
869
870 *tag add client=127.0.0.1-fd name=#important"
871 1000 Tag added
872
873 *tag list client
874 +--------------+----------+--------------+
875 | tag | clientid | client |
876 +--------------+----------+--------------+
877 | #tagviamenu3 | 1 | 127.0.0.1-fd |
878 | test1 | 1 | 127.0.0.1-fd |
879 | #tagviamenu2 | 1 | 127.0.0.1-fd |
880 | #tagviamenu1 | 1 | 127.0.0.1-fd |
881 | #important | 1 | 127.0.0.1-fd |
882 +--------------+----------+--------------+
883
884 *tag list client name=#important
885 +----------+--------------+
886 | clientid | client |
887 +----------+--------------+
888 | 1 | 127.0.0.1-fd |
889 +----------+--------------+
890 \end{bVerbatim}
891
892 It is possible to assign Tags to a Job record with the new 'Tag' directive in a Job resource.
893 \begin{bVerbatim}
894 Job {
895 Name = backup
896 ...
897 Tag = "#important", "#production"
898 }
899 \end{bVerbatim}
900
901 \begin{bVerbatim}
902 *tag list job
903 +--------------+-------+-------------+
904 | tag | jobid | job |
905 +--------------+-------+-------------+
906 | #important | 2 | backup |
907 | #production | 2 | backup |
908 +--------------+-------+-------------+
909 \end{bVerbatim}
910
911 The Tags are also accessible from various BWeb Management Suite screens. (See \vref{tagbweb}).
912
913 \subsection{Support for SHA256 and SHA512 in FileSet}
914
915 The support for strong signature algorithms SHA256 and SHA512 has been added
916 to Verify Jobs. It is now possible to check if data generated by a Job that uses
917 an SHA256 or SHA512 signature is valid.
918
919 \begin{bVerbatim}
920 FileSet {
921 Name = Linux-Etc
922 Options {
923 Signature = SHA512
924 Verify = pins3
925 }
926 File = /etc
927 }
928 \end{bVerbatim}
929
930 In the \texttt{FileSet} \bdirectivename{Verify} option directive, the following
931 code has been added:
932 \begin{itemize}
933 \item [3] for SHA512
934 \item [2] for SHA256
935 \end{itemize}
936
937 \subsection{Windows Installer Silent Mode Enhancement}
938
939 The following command line options may be used to control the regular Bacula
940 installer values in silent mode:
941 \begin{itemize}
942 \item -ConfigClientName
943 \item -ConfigClientPort
944 \item -ConfigClientPassword
945 \item -ConfigClientMaxJobs
946 \item -ConfigClientInstallService
947 \item -ConfigClientStartService
948 \item -ConfigStorageName
949 \item -ConfigStoragePort
950 \item -ConfigStorageMaxJobs
951 \item -ConfigStoragePassword
952 \item -ConfigStorageInstallService
953 \item -ConfigStorageStartService
954 \item -ConfigDirectorName
955 \item -ConfigDirectorPort
956 \item -ConfigDirectorMaxJobs
957 \item -ConfigDirectorPassword
958 \item -ConfigDirectorDB
959 \item -ConfigDirectorInstallService
960 \item -ConfigDirectorStartService
961 \item -ConfigMonitorName
962 \item -ConfigMonitorPassword
963 \end{itemize}
964
965 The following options control the components that will be installed:
966 \begin{itemize}
967 \item -ComponentFile
968 \item -ComponentStorage
969 \item -ComponentTextConsole
970 \item -ComponentBatConsole
971 \item -ComponentTrayMonitor
972 \item -ComponentAllDrivesPlugin
973 \item -ComponentWinBMRPlugin
974 \item -ComponentCDPPlugin
975 \end{itemize}
976
977 \subsubsection*{Example}
978
979 \begin{bVerbatim}
980 bacula-win64-13.0.0.exe /S -ComponentFile \
981 -ConfigClientName foo -ConfigClientPassword bar
982 \end{bVerbatim}
983
984 Will install only file deamon with bacula-fd.conf configured.
985
986 \begin{bVerbatim}
987 bacula-win64-13.0.0.exe /S -ComponentStorage \
988 -ComponentFile-ConfigClientName foo \
989 -ConfigClientPassword bar -ConfigStorageName foo2 \
990 -ConfigStoragePassword bar2
991 \end{bVerbatim}
992
993 Will install the Storage Deamon plus File Deamon with bacula-sd.conf and bacula-fd.conf configured.
994
995 \subsection{New Message Identification Format}
996 We are starting to add unique message indentifiers to each message (other
997 than debug and the Job report) that Bacula prints. At the current time
998 only two files in the Storage Daemon have these message identifiers and
999 over time with subsequent releases we will modify all messages.
1000
1001 \smallskip
1002 The message identifier will be kept unique for each message and once
1003 assigned to a message it will not change even if the text of the message
1004 changes. This means that the message identifier will be the same no matter
1005 what language the text is displayed in, and more importantly, it will allow
1006 us to make listing of the messages with in some cases, an additional
1007 explanation or instructions on how to correct the problem. All this will
1008 take several years since it is a lot of work and requires some new programs
1009 that are not yet written to manage these message identifiers.
1010
1011 \smallskip
1012 The format of the message identifier is:
1013
1014 \begin{bVerbatim}
1015 [AAnnnn]
1016 \end{bVerbatim}
1017 where A is an upper case character and nnnn is a four digit number, where
1018 the first character indicates the software component (daemon); the second
1019 letter indicates the severity, and the number is unique for a given
1020 component and severity.
1021
1022 \smallskip
1023 For example:
1024
1025 \begin{bVerbatim}
1026 [SF0001]
1027 \end{bVerbatim}
1028
1029 The first character representing the component at the current time one of
1030 the following:
1031
1032 \begin{bVerbatim}
1033 S Storage daemon
1034 D Director
1035 F File daemon
1036 \end{bVerbatim}
1037
1038 \smallskip
1039 The second character representing the severity or level can be:
1040
1041 \begin{bVerbatim}
1042 A Abort
1043 F Fatal
1044 E Error
1045 W Warning
1046 S Security
1047 I Info
1048 D Debug
1049 O OK (i.e. operation completed normally)
1050 \end{bVerbatim}
1051
1052 So in the example above [SF0001] indicates it is a message id, because of
1053 the brackets and because it is at the beginning of the message, and that
1054 it was generated by the Storage Daemon as a fatal error.
1055 \smallskip
1056 As mentioned above it will take some time to implement these message ids
1057 everywhere, and over time we may add more component letters and more
1058 severity levels as needed.
1059
1060 \subsection{Misc}
1061
1062 \subsubsection{Plugin Object Status Support}
1063
1064 The Object table has now a ObjectStatus field that can be used by plugins
1065 to report more precise information about the backup of an Object (generated
1066 by plugins).
1067
1068 \subsubsection{Network Buffer Management}
1069
1070 The new \texttt{SDPacketCheck} FileDaemon directive can be used to control the
1071 network flow in some specific use cases.
1072
1073 See \vref{FileDaemon:FileDaemon:SDPacketCheck} for more information.
1074
1075 \subsubsection{IBM Lintape Driver (BETA)}
1076
1077 The new \texttt{Use Lintape} Storage Daemon directive has been added to support
1078 the Lintape Kernel driver.
1079
1080 See \vref{Storage:Device:UseLintape} for more information.
1081
1082 \chapter{New Features in 11.0.0}
1083
1084 \section{Catalog Performance Improvements}
1085 There is a new \bacula{} database format (schema) in this version
1086 of \bacula{} that eliminates the FileName table by placing the
1087 Filename into the File record of the File table.
1088 This substantiallly improves performance,
1089 particularly for large databases.
1090
1091 The \btool{update\_xxx\_catalog} script will automatically update the \bacula{}
1092 database format, but you should realize that for very large databases (greater
1093 than 50GB), it may take some time and it will double the size of the database on disk
1094 during the migration.
1095
1096 \smallskip
1097
1098 This database format change can provide very significant improvements in
1099 the speed of metadata insertion into the database, and in some cases
1100 (backup of large email servers) can significantly reduce the size of the
1101 database.
1102
1103 \section{Automatic TLS Encryption}
1104
1105 Starting with Bacula 11.0, all daemons and consoles are now using TLS
1106 automatically for all network communications. It is no longer required to setup
1107 TLS keys in advance. It is possible to turn off automatic TLS PSK encryption using
1108 the \bdirectivename{TLS PSK Enable} directive.
1109
1110 \section{Client Behind NAT Support with the \bdirectivename{Connect To Director} Directive}
1111
1112 A Client can now initiate a connection to the Director (permanently or
1113 scheduled) to allow the Director to communicate to the Client when a new Job is started or a
1114 bconsole command such as \bcommandname{status client} or
1115 \bcommandname{estimate} is issued.
1116 \smallskip{}
1117
1118 This new network configuration option is particularly useful for Clients that are not
1119 directly reachable by the Director.
1120 \bimageH%[0.9\linewidth]
1121 {client-behind-nat}{Client Behind NAT Example}{fig:clientbehindnat}
1122
1123 \begin{bVerbatim}
1124 # cat /opt/bacula/etc/bacula-fd.conf
1125 Director {
1126 Name = bac-dir
1127 Password = aigh3wu7oothieb4geeph3noo # Password used to connect
1128
1129 # New directives
1130 Address = bac-dir.mycompany.com # Director address to connect
1131 Connect To Director = yes # FD will call the Director
1132 }
1133
1134
1135 # cat /opt/bacula/etc/bacula-dir.conf
1136 Client {
1137 Name = bac-fd
1138 Password = aigh3wu7oothieb4geeph3noo
1139
1140 # New directive
1141 Allow FD Connections = yes
1142 }
1143 \end{bVerbatim}
1144
1145 \smallskip{}
1146
1147 It is possible to schedule the Client connenction at certain periods of the day:
1148 \begin{bVerbatim}
1149 # cat /opt/bacula/etc/bacula-fd.conf
1150 Director {
1151 Name = bac-dir
1152 Password = aigh3wu7oothieb4geeph3noo # Password used to connect
1153
1154 # New directives
1155 Address = bac-dir.mycompany.com # Director address to connect
1156 Connect To Director = yes # FD will call the Director
1157 Schedule = WorkingHours
1158 }
1159
1160 Schedule {
1161 Name = WorkingHours
1162 # Connect the Director between 12:00 and 14:00
1163 Connect = MaxConnectTime=2h on mon-fri at 12:00
1164 }
1165 \end{bVerbatim}
1166
1167 Note that in the current version, if the File Daemon is started after 12:00, the next
1168 connection to the Director will occur at 12:00 the next day.
1169
1170 A Job can be scheduled in the Director around 12:00, and if the Client is connected, the
1171 Job will be executed as if the Client was reachable from the Director.
1172
1173
1174 \section{Continuous Data Protection Plugin}
1175
1176 Continuous Data Protection (CDP), also called continuous backup or real-time
1177 backup, refers to backup of Client data by automatically saving a copy of
1178 every change made to that data, essentially capturing every version of the data
1179 that the user saves. It allows the user or administrator to restore data to any
1180 point in time.
1181
1182
1183 \bimageH%[0.9\linewidth]
1184 {cdp}{CDP Example}{fig:cdp}
1185
1186 The Bacula CDP feature is composed of two components: An
1187 application (\texttt{cdp-client} or \texttt{tray-monitor}) that will monitor a
1188 set of directories configured by the user, and a \bacula{} FileDaemon plugin
1189 responsible to secure the data using \bacula{} infrastructure.
1190
1191 The user application (\texttt{cdp-client} or \texttt{tray-monitor}) is
1192 responsible for monitoring files and directories. When a modification is detected,
1193 the new data is copied into a \textsl{spool} directory. At a regular
1194 interval, a \bacula{} backup job will contact the FileDaemon and will save all
1195 the files archived by the \texttt{cdp-client}. The locally copied data can be restored
1196 at any time without a network connection to the Director.
1197
1198 See the CDP (Continious Data Protection) chapter \vref{blb:cdp} for more information.
1199
1200 \subsubsection{Global \bdirectivename{Autoprune} Control Directive}
1201
1202 The \bDirector{} \bdirectivename{Autoprune} directive can now globally control
1203 the Autoprune feature. This directive will take precedence over
1204 \bresourcename{Pool} or \bresourcename{Client} \bdirectivename{Autoprune}
1205 directives.
1206
1207 \begin{bVerbatim}
1208 Director {
1209 Name = mydir-dir
1210 ...
1211 AutoPrune = no # switch off Autoprune globally
1212 }
1213 \end{bVerbatim}
1214
1215
1216 \section{Event and Auditing}
1217
1218 The Director daemon can now record events such as:
1219 \begin{bitemize}
1220 \item Console connection/disconnection
1221 \item Daemon startup/shutdown
1222 \item Command execution
1223 \item ...
1224 \end{bitemize}
1225
1226 The events may be stored in a new catalog table, to disk, or sent via syslog.
1227
1228 \begin{bVerbatim}
1229 Messages {
1230 Name = Standard
1231 catalog = all, events
1232 append = /opt/bacula/working/bacula.log = all, !skipped
1233 append = /opt/bacula/working/audit.log = events, !events.bweb
1234 }
1235
1236 Messages {
1237 Name = Daemon
1238 catalog = all, events
1239 append = /opt/bacula/working/bacula.log = all, !skipped
1240 append = /opt/bacula/working/audit.log = events, !events.bweb
1241 append = /opt/bacula/working/bweb.log = events.bweb
1242 }
1243 \end{bVerbatim}
1244
1245 The new message category ``events'' is not included in the default configuration files by default.
1246
1247 It is possible to filter out some events using "!events." form. It is possible to specify
1248 10 custom events per Messages resource.
1249
1250 All event types are recorded by default.
1251
1252 \smallskip
1253
1254 When stored in the catalog, the events can be listed with the ``list events'' command.
1255
1256 \begin{bVerbatim}
1257 * list events [type=<str> | limit=<int> | order=<asc|desc> | days=<int> |
1258 start=<time-specification> | end=<time-specification>]
1259 +---------------------+------------+-----------+--------------------------------+
1260 | time | type | source | event |
1261 +---------------------+------------+-----------+--------------------------------+
1262 | 2020-04-24 17:04:07 | daemon | *Daemon* | Director startup |
1263 | 2020-04-24 17:04:12 | connection | *Console* | Connection from 127.0.0.1:8101 |
1264 | 2020-04-24 17:04:20 | command | *Console* | purge jobid=1 |
1265 +---------------------+------------+-----------+--------------------------------+
1266 \end{bVerbatim}
1267
1268 The \texttt{.events} command can be used to record an external event. The source recorded
1269 will be recorded as ``**source**''. The events type can have a custom name.
1270
1271 \begin{bVerbatim}
1272 * .events type=baculum source=joe text="User login"
1273 \end{bVerbatim}
1274
1275
1276 \section{New Prune Command Option}
1277
1278 The \bcommandname{prune jobs all} command will query the catalog to find all
1279 combinations of Client/Pool, and will run the pruning algorithm on each of
1280 them. At the end, all files and jobs not needed for restore that have passed the
1281 relevant retention times will be pruned.
1282 \smallskip
1283
1284 The \bcommandname{prune} command \bcommandname{prune jobs all yes} can be scheduled in a RunScript
1285 to prune the catalog once per day for example. All Clients and Pools will be analyzed automatically.
1286
1287 \begin{bVerbatim}
1288 Job {
1289 ...
1290 RunScript {
1291 Console = "prune jobs all yes"
1292 RunsWhen = Before
1293 failjobonerror = no
1294 runsonclient = no
1295 }
1296 }
1297 \end{bVerbatim}
1298
1299 \subsubsection{Dynamic Client Address Directive}
1300
1301 It is now possible to use a script to determine the address of a Client when
1302 dynamic DNS option is not a viable solution:
1303 \begin{bVerbatim}
1304 Client {
1305 Name = my-fd
1306 ...
1307 Address = "|/opt/bacula/bin/compute-ip my-fd"
1308 }
1309 \end{bVerbatim}
1310
1311 The command used to generate the address should return one single line with
1312 a valid address and end with the exit code~0. An example would be
1313 \begin{bVerbatim}
1314 Address = "|echo 127.0.0.1"
1315 \end{bVerbatim}
1316
1317 This option might be useful in some complex cluster environments.
1318
1319 \section{Volume Retention Enhancements}
1320
1321 The Pool/Volume parameter \texttt{Volume Retention} can now be disabled to
1322 never prune a volume based on the \texttt{Volume Retention}
1323 time. When \texttt{Volume Retention} is disabled, only the \texttt{Job
1324 Retention} time will be used to prune jobs.
1325
1326 \begin{bVerbatim}
1327 Pool {
1328 Volume Retention = 0
1329 ...
1330 }
1331 \end{bVerbatim}
1332
1333 \section{Windows Enhancements}
1334
1335 \begin{bitemize}
1336 \item Support for Windows files with non-UTF16 names.
1337 \item Snapshot management has been improved, and a backup job now relies
1338 exclusively on the snapshot tree structure.
1339 \item Support for the \texttt{system.cifs\_acl} extended attribute backup with
1340 Linux \acs{CIFS} has been added. It can be used to backup Windows security
1341 attributes from a CIFS share mounted on a Linux system. Note that only recent
1342 Linux kernels can handle the \texttt{system.cifs\_acl} feature correctly. The
1343 FileSet must use the \texttt{XATTR Support=yes} option, and the CIFS share
1344 must be mounted with the \texttt{cifsacl} options. See mount.cifs(8) for more
1345 information.
1346 \end{bitemize}
1347
1348 \section{GPFS ACL Support}
1349
1350 The new Bacula FileDaemon supports the GPFS filesystem specific
1351 ACL. The GPFS libraries must be installed in the standard location. To
1352 know if the GPFS support is available on your system, the following commands
1353 can be used.
1354
1355 \begin{bVerbatim}
1356 *setdebug level=1 client=stretch-amd64-fd
1357 Connecting to Client stretch-amd64-fd at stretch-amd64:9102
1358 2000 OK setdebug=1 trace=0 hangup=0 blowup=0 options= tags=
1359
1360 *st client=stretch-amd64-fd
1361 Connecting to Client stretch-amd64-fd at stretch-amd64:9102
1362
1363 stretch-amd64-fd Version: 11.0.0 (01 Dec 2020) x86_64-pc-linux-gnu-bacula debian 9.11
1364 Daemon started 21-Jul-20 14:42. Jobs: run=0 running=0.
1365 Ulimits: nofile=1024 memlock=65536 status=ok
1366 Heap: heap=135,168 smbytes=199,993 max_bytes=200,010 bufs=104 max_bufs=105
1367 Sizes: boffset_t=8 size_t=8 debug=1 trace=0 mode=0,2010 bwlimit=0kB/s
1368 Crypto: fips=no crypto=OpenSSL 1.0.2u 20 Dec 2019
1369 APIs: GPFS
1370 Plugin: bpipe-fd.so(2)
1371 \end{bVerbatim}
1372
1373 The \texttt{APIs} line will indicate if the \texttt{/usr/lpp/mmfs/libgpfs.so}
1374 was loaded at the start of the Bacula FD service or not.
1375
1376 The standard \texttt{ACL Support} (cf \vref{blb:ACLSupport}) directive can be
1377 used to enable automatically the support for the GPFS ACL backup.
1378
1379 \section{New Baculum Features}
1380
1381 \subsection{Multi-user interface improvements}
1382
1383 There have been added new functions and improvements to the multi-user
1384 interface and restricted access.
1385
1386 The Security page has new tabs:
1387
1388 \begin{bitemize}
1389 \item Console ACLs
1390 \item OAuth2 clients
1391 \item API hosts
1392 \end{bitemize}
1393
1394 These new tabs help to configure OAuth2 accounts, create restricted Bacula
1395 Console for users and create API hosts. They ease the process of creating
1396 users with a restricted Bacula resources access.
1397
1398 \subsection{Add searching jobs by filename in the restore wizard}
1399
1400 In the restore wizard now is possible to select job to restore by
1401 filename of file stored in backups. There is also possible to limit
1402 results to specific path.
1403
1404 \subsection{Show more detailed job file list}
1405
1406 The job file list now displays file details like: file attributes,
1407 UID, GID, size, mtime and information if the file record for saved or
1408 deleted file.
1409
1410 \subsection{Add graphs to job view page}
1411
1412 On the job view page, new pie and bar graphs for selected job are available.
1413
1414 \subsection{Implement graphical status storage}
1415
1416 On the storage status page are available two new types of the status (raw
1417 and graphical). The graphical status page is modern and refreshed
1418 asynchronously.
1419
1420 \subsection{Add Russian translations}
1421
1422 \subsection{Global messages log window}
1423
1424 There has been added new window to browse Bacula logs in a friendly way.
1425
1426 \subsection{Job status weather}
1427 Add the job status weather on job list page to express current job condition.
1428
1429 \subsection{Restore wizard improvements}
1430 In the restore wizard has been added listing and browsing names encoded in
1431 non-UTF encoding.
1432
1433 \subsection{New API endpoints}
1434
1435 \begin{bitemize}
1436 \item /oauth2/clients
1437 \item /oauth2/clients/{client\_id}
1438 \item /jobs/files
1439 \end{bitemize}
1440
1441 \subsection{New parameters in API endpoints}
1442
1443 \begin{bitemize}
1444 \item /jobs/{jobid}/files - 'details' parameter
1445 \item /storages/show - 'output' parameter
1446 \item /storages/{storageid}/show - 'output' parameter
1447 \end{bitemize}
1448
1449 \chapter{New Features in 9.6.0}
1450
1451 \subsection{Building 9.6.4 and later}
1452 Version 9.6.4 is a major security and bug fix release. We suggest everyone
1453 to upgrade as soon as possible.
1454 <p>
1455 One significant improvement in this version is for the AWS S3 cloud driver.
1456 First the code base has been brought much closer to the Enterprise version
1457 (still a long ways to go). Second major change is that the community code now uses the
1458 latest version of libs3 as maintained by Bacula Systems. The libs3 code is
1459 available as a tar file for Bacula version 9.6.4 at:
1460 <p>
1461
1462 \elink{http://www.bacula.org/downloads/libs3-20200523.tar.gz}{http://www.bacula.org/downloads/libs3-20200523.tar.gz}
1463
1464 <p>
1465 Note: Version 9.6.4 must be compiled with the above libs3 version or later.
1466 To build libs3:
1467 \begin{bVerbatim}
1468 - Remove any libs3 package loaded by your OS
1469 - Download above link
1470 - tar xvfz libs3-20200523.tar.gz
1471 - cd libs3-20200523
1472 - make # should have no errors
1473 - sudo make install
1474 \end{bVerbatim}
1475 <p>
1476 Then when you do your Bacula ./configure <args> it should automatically
1477 detect and use the libs3. The output from the ./configure will show
1478 whether or not libs3 was found during the configuration. E.g.
1479 <p>
1480 \begin{bVerbatim}
1481 S3 support: yes
1482 \end{bVerbatim}
1483 <p>
1484 in the output from ./configure.
1485
1486
1487 \subsection{Docker Plugin}
1488
1489 Containers is a relatively new system level virtualization concept
1490 that has less overhead than traditional virtualation. This is true
1491 because Container use the underlying operating system to provide
1492 all the needed services thus eliminating the need for multiple operating
1493 systems.
1494
1495 Docker containers rely on sophisticated file system level data abstraction
1496 with a number of read-only images to create templates used for container
1497 initialization.
1498
1499 With its Docker Plugin, the Bacula will save the full container
1500 image including all read-only and writable layers into a single image archive.
1501
1502 It is not necessary to install a Bacula File daemon in each container, so
1503 each container can be backed up from a common image repository.
1504
1505 The Bacula Docker Plugin will contact the Docker service to read and save
1506 the contents of any system image or container image using
1507 snapshots (default behavior) and dump them using the Docker~API.
1508
1509 The Docker Plugin whitepaper provides more detailed information.
1510
1511 \subsection{Real-Time Statistics Monitoring}
1512 \label{CollectCommand}
1513
1514 All Bacula daemons can now collect internal performance statistics
1515 periodically and provide mechanisms to store the values to a CSV file or to
1516 send the values to a Graphite daemon via the network. Graphite is an
1517 enterprise-ready monitoring tool (\url{https://graphiteapp.org}).
1518
1519
1520 \begin{figure}[htbp]
1521 \centering
1522 \includegraphics[width=0.47\linewidth]{graphite1}
1523 \includegraphics[width=0.47\linewidth]{graphite2}
1524 \caption{Graphite Examples}
1525 \label{fig:graphite}
1526 \end{figure}
1527 \smallskip
1528
1529 To activate the statistic collector feature, simply define a
1530 \textbf{Statistics} resource in the daemon of your choice:
1531
1532 \begin{bVerbatim}
1533 Statistics {
1534 Name = "Graphite"
1535 Type = Graphite
1536
1537 # Graphite host information
1538 Host = "localhost"
1539 Port = 2003
1540 }
1541 \end{bVerbatim}
1542
1543 It is possible to change the interval that is used to collect the statistics
1544 with the \textbf{Interval} directive (\textbf{5 mins} by default), and use the
1545 \textbf{Metrics} directive to select the data to collect (\textbf{all} by default).
1546 \smallskip
1547
1548 If the Graphite daemon cannot be reached, the statistics data are spooled on
1549 disk and are sent automatically when the Graphite daemon is available again.
1550 \smallskip
1551
1552 The bconsole \textbf{statistics} command can be used to display the current
1553 statistics in various formats (text or json for now).
1554
1555 \begin{bVerbatim}
1556 *statistics
1557 Statistics available for:
1558 1: Director
1559 2: Storage
1560 3: Client
1561 Select daemon type for statistics (1-3): 1
1562 bacula.dir.config.clients=1
1563 bacula.dir.config.jobs=3
1564 bacula.dir.config.filesets=2
1565 bacula.dir.config.pools=3
1566 bacula.dir.config.schedules=2
1567 ...
1568 *statistics storage
1569 ...
1570 bacula.storage.bac-sd.device.File1.readbytes=214
1571 bacula.storage.bac-sd.device.File1.readtime=12
1572 bacula.storage.bac-sd.device.File1.readspeed=0.000000
1573 bacula.storage.bac-sd.device.File1.writespeed=0.000000
1574 bacula.storage.bac-sd.device.File1.status=1
1575 bacula.storage.bac-sd.device.File1.writebytes=83013529
1576 bacula.storage.bac-sd.device.File1.writetime=20356
1577 ...
1578 \end{bVerbatim}
1579
1580 The \textbf{statistics} bconsole command can accept parameters to be scripted,
1581 for example it is possible to export the data in JSON, or to select which
1582 metrics to display.
1583
1584 \begin{bVerbatim}
1585 *statistics bacula.dir.config.clients bacula.dir.config.jobs json
1586 [
1587 {
1588 "name": "bacula.dir.config.clients",
1589 "value": 1,
1590 "type": "Integer",
1591 "unit": "Clients",
1592 "description": "The number of defined clients in the Director."
1593 },
1594 {
1595 "name": "bacula.dir.config.jobs",
1596 "value": 3,
1597 "type": "Integer",
1598 "unit": "Jobs",
1599 "description": "The number of defined jobs in the Director."
1600 }
1601 ]
1602 \end{bVerbatim}
1603
1604 The \textbf{.status statistics} command can be used to query the status
1605 of the Statistic collector thread.
1606
1607 \begin{bVerbatim}
1608 *.status dir statistics
1609 Statistics backend: Graphite is running
1610 type=2 lasttimestamp=12-Sep-18 09:45
1611 interval=300 secs
1612 spooling=in progress
1613 lasterror=Could not connect to localhost:2003 Err=Connection refused
1614
1615 Update Statistics: running interval=300 secs lastupdate=12-Sep-18 09:45
1616 *
1617 \end{bVerbatim}
1618
1619
1620 \chapter{New Features in 9.4.0}
1621 \subsubsection{Cloud Backup}
1622 A major problem of Cloud backup is that data transmission to and from the
1623 Cloud is very slow compared to traditional backup to disk or tape.
1624 The Bacula Cloud drivers provide a means to quickly finish the backups
1625 and then to transfer the data from the local cache to the Cloud in the
1626 background. This is done by first splitting the data Volumes into small
1627 parts that are cached locally then uploading those parts to the Cloud
1628 storage service in the background, either while the job continues to run or
1629 after the backup Job has terminated. Once the parts are written to the
1630 Cloud, they may either be left in the local cache for quick restores or
1631 they can be removed (truncate cache).
1632
1633 \subsubsection{Cloud Volume Architecture}
1634
1635 \begin{figure}[htbp]
1636 \centering
1637 \includegraphics[width=0.8\linewidth]{nativeCloudStorage-diagram}
1638 \caption{Bacula Cloud Architecture}
1639 \label{fig:cloud0}
1640 \end{figure}
1641
1642
1643 The picture shown above shows two Volumes (Volume0001 and Volume0002) with
1644 their parts in the cache. Below the cache, one can see that Volume0002 has
1645 been uploaded or synchronized with the Cloud.
1646
1647 \smallskip
1648 Note: Regular Bacula disk Volumes are implemented as standard files that reside in
1649 the user defined \textbf{Archive Directory}. On the other hand, Bacula
1650 Cloud Volumes are directories that reside in the user defined
1651 \textbf{Archive Directory}. Each Cloud Volume's directory contains the cloud Volume parts
1652 which are implemented as numbered files (part.1, part.2, ...).
1653
1654
1655 \subsubsection{Cloud Restore}
1656 During a restore, if the needed parts are in the local cache, they will be
1657 immediately used, otherwise, they will be downloaded from the Cloud as
1658 needed. The restore starts with parts already in the local cache but
1659 will wait in turn for any part that needs to be downloaded. The Cloud part
1660 downloads proceed while the restore is running.
1661
1662 \smallskip
1663 With most Cloud providers, uploads are usually free of charge, but
1664 downloads of data from the Cloud are billed. By using local cache and
1665 multiple small parts, you can configure Bacula to substantially reduce
1666 download costs.
1667
1668 \smallskip
1669
1670 The \texttt{MaximumFileSize} Device directive is still valid within the Storage
1671 Daemon and defines the granularity of a restore chunk. In order to limit volume
1672 parts to download during restore (specially when restoring single files), it
1673 might be useful to set the \texttt{MaximumFileSize} to a value smaller than or
1674 equal to the \texttt{MaximumPartSize}.
1675
1676 \subsubsection{Compatibility}
1677 Since a Cloud Volume contains the same data as an
1678 ordinary Bacula Volume, all existing types of Bacula data may be stored in
1679 the cloud -- that is client encrypted, compressed data, plugin data, etc.
1680 All existing Bacula functionality, with the exception of deduplication,
1681 is available with the Bacula Cloud drivers.
1682
1683 \subsubsection{Deduplication and the Cloud}
1684 At the current time, Bacula Global Endpoint Backup does not support
1685 writing to the cloud because the cloud would be too slow to support large
1686 hashed and indexed containers of deduplication data.
1687
1688 \subsubsection{Virtual Autochangers and Disk Autochangers}
1689 If you use a Bacula Virtual Autochanger you will find it compatible with
1690 the new Bacula Cloud drivers. However, if you use a third party disk
1691 autochanger script such as \textbf{Vchanger}, unless or until it is
1692 modified to handle Volume directories, it may not be compatible with Bacula
1693 Cloud drivers.
1694
1695 \subsubsection{Security}
1696 All data that is sent to and received from the cloud by default uses the
1697 HTTPS protocol, so your data is encrypted while being transmitted
1698 and received. However, data that resides in the Cloud is not encrypted by
1699 default. If you wish extra security of your data while it resides in the
1700 cloud, you should consider using Bacula's PKI data encryption feature
1701 during the backup.
1702
1703 \subsubsection{Cache and Pruning}
1704
1705 The Cache is treated much like a normal Disk based backup, so that in
1706 configuring Cloud the administrator should take care to set "Archive Device" in
1707 the Device resource to a directory where he/she would normally start data
1708 backed up to disk. Obviously, unless he/she uses the truncate/prune cache
1709 commands, the Archive Device will continue to fill.
1710
1711 \smallskip
1712
1713 The cache retention can be controlled per Volume with the ``CacheRetention''
1714 attribute. The default value is 0, meaning that the pruning of the cache is
1715 disabled.
1716
1717 \smallskip
1718
1719 The ``CacheRetention'' value for a volume can be modified with the ``update''
1720 command or via the Pool directive ``CacheRetention'' for newly created volumes.
1721
1722 \subsection{New Commands, Resource, and Directives for Cloud}
1723 To support Cloud, there are new \textbf{bconsole} commands, new Storage Daemon
1724 directives and a new Cloud resource that is specified in the Storage Daemon's
1725 Device resource.
1726
1727 \subsubsection{New Cloud Bconsole Commands}
1728 \begin{itemize}
1729 \item \textbf{Cloud} The new cloud \textbf{bconsole} command allows you
1730 to do a number of things with cloud volumes. The options are the
1731 following:
1732 \begin{itemize}
1733 \item None. If you specify no arguments to the command, bconsole will
1734 prompt with:
1735 \begin{bVerbatim}
1736 Cloud choice:
1737 1: List Cloud Volumes in the Cloud
1738 2: Upload a Volume to the Cloud
1739 3: Prune the Cloud Cache
1740 4: Truncate a Volume Cache
1741 5: Done
1742 Select action to perform on Cloud (1-5):
1743 \end{bVerbatim}
1744 The different choices should be rather obvious.
1745
1746 \item \textbf{Truncate} This command will attempt to truncate the
1747 local cache for the specified Volume. Bacula will prompt you for the
1748 information needed to determine the Volume name or names. To avoid
1749 the prompts, the following additional command line options may be
1750 specified:
1751 \begin{itemize}
1752 \item \textbf{Storage=xxx}
1753 \item \textbf{Volume=xxx}
1754 \item \textbf{AllPools}
1755 \item \textbf{AllFromPool}
1756 \item \textbf{Pool=xxx}
1757 \item \textbf{MediaType=xxx}
1758 \item \textbf{Drive=xxx}
1759 \item \textbf{Slots=nnn}
1760 \end{itemize}
1761 \item \textbf{Prune} This command will attempt to prune the local cache for
1762 the specified Volume. Bacula will respect the CacheRetention volume
1763 attribute to determine if the cache can be truncated or not. Only parts
1764 that are uploaded to the cloud will be deleted from the cache. Bacula
1765 will prompt you for the information needed to determine the Volume name or
1766 names. To avoid the prompts, the following additional command line
1767 options may be specified:
1768 \begin{itemize}
1769 \item \textbf{Storage=xxx}
1770 \item \textbf{Volume=xxx}
1771 \item \textbf{AllPools}
1772 \item \textbf{AllFromPool}
1773 \item \textbf{Pool=xxx}
1774 \item \textbf{MediaType=xxx}
1775 \item \textbf{Drive=xxx}
1776 \item \textbf{Slots=nnn}
1777 \end{itemize}
1778 \item \textbf{Upload} This command will attempt to upload the specified
1779 Volumes. It will prompt you for the information needed to determine
1780 the Volume name or names. To avoid the prompts, you may specify any
1781 of the following additional command line options:
1782 \begin{itemize}
1783 \item \textbf{Storage=xxx}
1784 \item \textbf{Volume=xxx}
1785 \item \textbf{AllPools}
1786 \item \textbf{AllFromPool}
1787 \item \textbf{Pool=xxx}
1788 \item \textbf{MediaType=xxx}
1789 \item \textbf{Drive=xxx}
1790 \item \textbf{Slots=nnn}
1791 \end{itemize}
1792 \item \textbf{List} This command will list volumes stored in the Cloud. If a
1793 volume name is specified, the command will list all parts for the given
1794 volume.
1795 To avoid the prompts, you may specify any
1796 of the following additional command line options:
1797 \begin{itemize}
1798 \item \textbf{Storage=xxx}
1799 \item \textbf{Volume=xxx}
1800 \item \textbf{Storage=xxx}
1801 \end{itemize}
1802 \end{itemize}
1803 \end{itemize}
1804
1805 \subsubsection{Cloud Additions to the DIR Pool Resource}
1806 Within the \textbf{bacula-dir.conf} file each \textbf{Pool} resource
1807 there is an additional keyword \textbf{CacheRetention} that can be specified.
1808
1809 \subsubsection{Cloud Additions to the SD Device Resource}
1810 Within the \textbf{bacula-sd.conf} file each \textbf{Device} resource
1811 there is an additional keyword \textbf{Cloud} that must be specified on
1812 the \textbf{Device Type} directive, and two new directives \textbf{Maximum
1813 Part Size} and \textbf{Cloud}.
1814
1815
1816 \subsubsection{New Cloud SD Device Directives}
1817 \begin{itemize}
1818 \item \textbf{Device Type} The Device Type has been extended to include
1819 the new keyword \textbf{Cloud} to specify that the device supports
1820 cloud Volumes. Example:
1821 \begin{bVerbatim}
1822 Device Type = Cloud
1823 \end{bVerbatim}
1824 \item \textbf{Cloud} The new Cloud directive permits specification of
1825 a new Cloud Resource. As with other Bacula resource specifications,
1826 one specifies the name of the Cloud resource. Example:
1827 \begin{bVerbatim}
1828 Cloud = S3Cloud
1829 \end{bVerbatim}
1830 \item \textbf{Maximum Part Size} This directive allows one to specify
1831 the maximum size for each part. Smaller part sizes will reduce restore
1832 costs, but may require a small additional overhead to handle multiple
1833 parts. The maximum number of parts permitted in a Cloud Volume is
1834 524,288. The maximum size of any given part is approximately 17.5TB.
1835 \end{itemize}
1836
1837 \subsubsection{Example Cloud Device Specification}
1838 An example of a Cloud Device Resource might be:
1839
1840 \begin{bVerbatim}
1841 Device {
1842 Name = CloudStorage
1843 Device Type = Cloud
1844 Cloud = S3Cloud
1845 Archive Device = /opt/bacula/backups
1846 Maximum Part Size = 10000000
1847 Media Type = CloudType
1848 LabelMedia = yes
1849 Random Access = Yes;
1850 AutomaticMount = yes
1851 RemovableMedia = no
1852 AlwaysOpen = no
1853 }
1854 \end{bVerbatim}
1855
1856 As you can see from the above, the \textbf{Cloud} directive in the
1857 \textbf{Device} resource contains the name (\textbf{S3Cloud}) of the
1858 \textbf{Cloud} resource that is shown below.
1859 \smallskip
1860 Note also the \textbf{Archive Device} is specified in the same manner
1861 as one would use for a File device. However, in place of containing
1862 files with Volume names, the archive device for the Cloud drivers will
1863 contain the local cache, which consists of directories with the Volume
1864 name; and these directories contain the parts associated with the
1865 particular Volume. So with the above Device resource, and the
1866 two cache Volumes shown in figure \vref{fig:cloud0ay2} above would have
1867 the following layout on disk:
1868
1869 \begin{bVerbatim}
1870 /opt/bacula/backups
1871 /opt/bacula/backups/Volume0001
1872 /opt/bacula/backups/Volume0001/part.1
1873 /opt/bacula/backups/Volume0001/part.2
1874 /opt/bacula/backups/Volume0001/part.3
1875 /opt/bacula/backups/Volume0001/part.4
1876 /opt/bacula/backups/Volume0002
1877 /opt/bacula/backups/Volume0002/part.1
1878 /opt/bacula/backups/Volume0002/part.2
1879 /opt/bacula/backups/Volume0002/part.3
1880 \end{bVerbatim}
1881
1882 % A good part of this section is also in storedconf.tex and the whitepaper
1883 % Please, keep everything synchronized
1884 \subsubsection{The Cloud Resource}
1885 The \textbf{Cloud} resource has a number of directives that may
1886 be specified as exemplified in the following example:
1887
1888 default east USA location:
1889 \begin{bVerbatim}
1890 Cloud {
1891 Name = S3Cloud
1892 Driver = "S3"
1893 HostName = "s3.amazonaws.com"
1894 BucketName = "BaculaVolumes"
1895 AccessKey = "BZIXAIS39DP9YNER5DFZ"
1896 SecretKey = "beesheeg7iTe0Gaexee7aedie4aWohfuewohGaa0"
1897 Protocol = HTTPS
1898 UriStyle = VirtualHost
1899 Truncate Cache = No
1900 Upload = EachPart
1901 Region = "us-east-1"
1902 MaximumUploadBandwidth = 5MB/s
1903 }
1904 \end{bVerbatim}
1905
1906 central europe location:
1907 \begin{bVerbatim}
1908 Cloud {
1909 Name = S3Cloud
1910 Driver = "S3"
1911 HostName = "s3-eu-central-1.amazonaws.com"
1912 BucketName = "BaculaVolumes"
1913 AccessKey = "BZIXAIS39DP9YNER5DFZ"
1914 SecretKey = "beesheeg7iTe0Gaexee7aedie4aWohfuewohGaa0"
1915 Protocol = HTTPS
1916 UriStyle = VirtualHost
1917 Truncate Cache = No
1918 Upload = EachPart
1919 Region = "eu-central-1"
1920 MaximumUploadBandwidth = 4MB/s
1921 }\end{bVerbatim}
1922
1923 For Amazon Cloud, refer to \url{http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region}
1924 to get a complete list of regions and corresponding endpoints and use them respectively
1925 as Region and HostName directive.
1926
1927 For CEPH S3 interface:
1928 \begin{bVerbatim}
1929 Cloud {
1930 Name = CEPH_S3
1931 Driver = "S3"
1932 HostName = ceph.mydomain.lan
1933 BucketName = "CEPHBucket"
1934 AccessKey = "xxxXXXxxxx"
1935 SecretKey = "xxheeg7iTe0Gaexee7aedie4aWohfuewohxx0"
1936 Protocol = HTTPS
1937 Upload = EachPart
1938
1939 UriStyle = Path # Must be set for CEPH
1940 }
1941 \end{bVerbatim}
1942
1943
1944 The directives of the above Cloud resource for the S3 driver are defined as
1945 follows:
1946
1947 \begin{description}
1948 \label{Storage:Cloud:Name}
1949 \item [Name = \lt{}Device-Name\gt{}]
1950 \index[sd]{Name}
1951 \index[sd]{Directive!Name}
1952
1953 The name of the Cloud resource. This is the logical Cloud name, and may be
1954 any string up to 127 characters in length. Shown as \textbf{S3Cloud} above.
1955
1956 \label{Storage:Cloud:Description}
1957 \item [Description = \lt{}Text\gt{}]
1958
1959 The description is used for display purposes as is the case with all
1960 resource.
1961
1962 \label{Storage:Cloud:Driver}
1963 \item [Driver = \lt{}DriverName\gt{}]
1964 \index[sd]{Driver}
1965
1966 This defines which driver to use. It can be \textbf{S3}.
1967 There is also a \textbf{File} driver, which is used mostly for testing.
1968
1969 \label{Storage:Cloud:HostName}
1970 \item [Host Name = \lt{}Name\gt{}]
1971 \index[sd]{HostName}
1972 This directive specifies the hostname to be used in the URL. Each Cloud
1973 service provider has a different and unique hostname. The maximum size is
1974 255 characters and may contain a tcp port specification.
1975
1976 \label{Storage:Cloud:BucketName}
1977 \item [Bucket Name = \lt{}Name\gt{}]
1978 \index[sd]{BucketName}
1979
1980 This directive specifies the bucket name that you wish to use on the Cloud
1981 service. This name is normally a unique name name that identifies where
1982 you want to place your Cloud Volume parts.
1983 With Amazon S3, the bucket must be created previously on the Cloud service.
1984 The maximum bucket name size is 255 characters.
1985
1986 \label{Storage:Cloud:AccessKey}
1987 \item [Access Key = \lt{}String\gt{}]
1988 \index[sd]{AccessKey}
1989
1990 The access key is your unique user identifier given to you by your cloud
1991 service provider.
1992
1993 \label{Storage:Cloud:SecretKey}
1994 \item [Secret Key = \lt{}String\gt{}]
1995 \index[sd]{SecretKey}
1996
1997 The secret key is the security key that was given to you by your cloud
1998 service provider. It is equivalent to a password.
1999
2000 \label{Storage:Cloud:Protocol}
2001 \item [Protocol = \lt{}HTTP | HTTPS\gt{}]
2002 \index[sd]{Protocol}
2003
2004 The protocol defines the communications protocol to use with the cloud
2005 service provider. The two protocols currently supported are: \textbf{HTTPS}
2006 and \textbf{HTTP}. The default is \textbf{HTTPS}.
2007
2008 \label{Storage:Cloud:UriStyle}
2009 \item [Uri Style = \lt{}VirtualHost | Path\gt{}]
2010 \index[sd]{UriStyle}
2011
2012 This directive specifies the URI style to use to communicate with the cloud
2013 service provider. The two Uri Styles currently supported are:
2014 \textbf{VirtualHost} and \textbf{Path}. The default is \textbf{VirtualHost}.
2015
2016 \label{Storage:Cloud:TruncateCache}
2017 \item [Truncate Cache = \lt{}Truncate-kw\gt{}]
2018 \index[sd]{TruncateCache}
2019
2020 This directive specifies when Bacula should automatically remove (truncate)
2021 the local cache parts. Local cache parts can only be removed if they have
2022 been uploaded to the cloud. The currently implemented values are:
2023
2024 \begin{itemize}
2025 \item \textbf{No} Do not remove cache. With this option you must manually
2026 delete the cache parts with a \textbf{bconsole} \textbf{Truncate Cache}
2027 command, or do so with an \textbf{Admin} Job that runs an
2028 \textbf{Truncate Cache} command. This is the default.
2029 \item \textbf{AfterUpload} Each part will be removed just after it is
2030 uploaded. Note, if this option is specified, all restores will require
2031 a download from the Cloud. {\textbf Note: Not yet implemented.}
2032 \item \textbf{AtEndOfJob} With this option, at the end of the Job, every part that has
2033 been uploaded to the Cloud will be removed (truncated).
2034 {\textbf Note: Not yet implemented.}
2035 \end{itemize}
2036
2037 \label{Storage:Cloud:Upload}
2038 \item [Upload = \lt{}Upload-kw\gt{}]
2039
2040 This directive specifies when local cache parts
2041 will be uploaded to the Cloud. The options are:
2042
2043 \begin{itemize}
2044 \item \textbf{No} Do not upload cache parts. With this option you must
2045 manually upload the cache parts with a \textbf{bconsole}
2046 \textbf{Upload} command, or do so with an \textbf{Admin} Job that runs
2047 an \textbf{Upload} command. This is the default.
2048 \item \textbf{EachPart} With this option, each part will be uploaded
2049 when it is complete i.e. when the next part is created or at the
2050 end of the Job.
2051 \item \textbf{AtEndOfJob} With this option all parts that have not been
2052 previously uploaded will be uploaded at the end of the Job.
2053 {\textbf Note: Not yet implemented.}
2054 \end{itemize}
2055
2056 \label{Storage:Cloud:MaximumUploadBandwidth}
2057 \item [Maximum Upload Bandwidth = \lt{}speed\gt{}]
2058
2059 The default is unlimited, but by using this directive, you may limit the
2060 upload bandwidth used globally by all devices referencing this Cloud
2061 resource.
2062
2063 \label{Storage:Cloud:MaximumDownloadBandwidth}
2064 \item [Maximum Download Bandwidth = \lt{}speed\gt{}]
2065
2066 The default is unlimited, but by using this directive, you may limit the
2067 download bandwidth used globally by all devices referencing this Cloud
2068 resource.
2069
2070 \label{Storage:Cloud:Region}
2071 \item [Region = \lt{}String\lt{}]
2072
2073 The Cloud resource can be configured to use a specific endpoint within a
2074 region. This directive is required for AWS-V4 regions. ex:
2075 \texttt{Region="eu-central-1"}
2076
2077 \end{description}
2078
2079 \subsubsection{File Driver for the Cloud}
2080 As mentioned above, one may specify the keyword \textbf{File} on the
2081 \textbf{Driver} directive of the Cloud resource. Instead of writing to
2082 the Cloud, Bacula will instead create a Cloud Volume but write it to disk.
2083 The rest of this section applies to the Cloud resource directives when
2084 the File driver is specified.
2085
2086 \smallskip
2087 The following Cloud directives are ignored: Bucket Name, Access Key, Secret
2088 Key, Protocol, Uri Style. The directives {\bf Truncate Cache} and
2089 {\bf Upload} work on the local cache in the same manner as they
2090 do for the S3 driver. \smallskip The main difference to note is that the
2091 Host Name, specifies the destination directory for the Cloud Volume files,
2092 and this Host Name must be different from the Archive Device name, or there
2093 will be a conflict between the local cache (in the Archive Device
2094 directory) and the destination Cloud Volumes (in the Host Name directory).
2095 \smallskip As noted above, the File driver is mostly used for testing
2096 purposes, and we do not particularly recommend using it. However, if you
2097 have a particularly slow backup device you might want to stage your backup
2098 data into an SSD or disk using the local cache feature of the Cloud device,
2099 and have your Volumes transferred in the background to a slow File device.
2100
2101 \subsection{WORM Tape Support}
2102
2103 Automatic WORM (Write Once Read Multiple) tapes detection has been added in
2104 10.2.
2105
2106 \smallskip{}
2107
2108 When a WORM tape is detected, the catalog volume entry is changed
2109 automatically to set \texttt{Recycle=no}. It will prevent the volume from being
2110 automatically recycled by Bacula.
2111
2112 \smallskip{}
2113
2114 There is no change in how the Job and File records are pruned from the catalog
2115 as that is a separate issue that is currently adequately implemented in Bacula.
2116
2117 \smallskip{}
2118
2119 When a WORM tape is detected, the SD will show WORM on the device state output
2120 (must have debug greater or equal to 6) otherwise the status shows as
2121 \texttt{!WORM}
2122
2123 \begin{bVerbatim}
2124 Device state:
2125 OPENED !TAPE LABEL APPEND !READ !EOT !WEOT !EOF WORM !SHORT !MOUNTED ...
2126 \end{bVerbatim}
2127
2128 \smallskip{}
2129
2130 The output of the used volume status has been modified to include the worm
2131 state. It shows \texttt{worm=1} for a worm cassette and \texttt{worm=0}
2132 otherwise. Example:
2133
2134 \begin{bVerbatim}
2135 Used Volume status:
2136 Reserved volume: TestVolume001 on Tape device "nst0" (/dev/nst0)
2137 Reader=0 writers=0 reserves=0 volinuse=0 worm=1
2138 \end{bVerbatim}
2139
2140 \smallskip{}
2141
2142 The following programs are needed for the WORM tape detection:
2143 \begin{itemize}
2144 \item sdparm
2145 \item tapeinfo
2146 \end{itemize}
2147
2148 \smallskip{}
2149
2150 The new Storage Device directive \texttt{Worm Command} must be configured as
2151 well as the \texttt{Control Device} directive (used with the Tape Alert
2152 feature).
2153
2154 \begin{bVerbatim}
2155 Device {
2156 Name = "LTO-0"
2157 Archive Device = "/dev/nst0"
2158 Control Device = "/dev/sg0" # from lsscsi -g
2159 Worm Command = "/opt/bacula/scripts/isworm %l"
2160 ...
2161 }
2162 \end{bVerbatim}
2163
2164
2165 \chapter{New Features in 9.2.0}
2166 This chapter describes new features that have been added to the
2167 current version of Bacula in version 9.2.0
2168
2169 \smallskip
2170 In general, this is a fairly substantial release because it contains
2171 a very large number of bug fixes backported from the Bacula Enterprise
2172 version. There are also a few new features backported from Bacula
2173 Enterprise.
2174
2175 \subsection{Enhanced Autochanger Support}
2176 Note: this feature was actually backported into version 9.0.0, but
2177 the documentation was added much after the 9.0.0 release. To call
2178 your attention to this new feature, we have also included the
2179 documentation here.
2180
2181 To make Bacula function properly with multiple Autochanger definitions, in
2182 the Director's configuration, you must adapt your {\bf bacula-dir.conf}
2183 {\bf Storage} directives.
2184
2185 \smallskip
2186 Each autochanger that you have defined in an {\bf Autochanger}
2187 resource in the Storage daemon's {\bf bacula-sd.conf} file,
2188 must have a corresponding {\bf Autochanger} resource defined
2189 in the Director's {\bf bacula-dir.conf} file. Normally you will
2190 already have a {\bf Storage} resource that points to the
2191 Storage daemon's {\bf Autochanger} resource. Thus you need
2192 only to change the name of the {\bf Storage} resource to
2193 {\bf Autochanger}. In addition the {\bf Autochanger = yes}
2194 directive is not needed in the Director's {\bf Autochanger}
2195 resource, since the resource name is {\bf Autochanger}, the
2196 Director already knows that it represents an autochanger.
2197
2198 \smallskip
2199 In addition to the above change ({\bf Storage} to {\bf Autochanger}),
2200 you must modify any additional {\bf Storage} resources that correspond
2201 to devices that are part of the {\bf Autochanger} device.
2202 Instead of the previous {\bf Autochanger = yes} directive they
2203 should be modified to be {\bf Autochanger = xxx} where you
2204 replace the {\bf xxx} with the name of the Autochanger.
2205
2206 \smallskip
2207 For example, in the bacula-dir.conf file:
2208
2209 \begin{bVerbatim}
2210 Autochanger { # New resource
2211 Name = Changer-1
2212 Address = cibou.company.com
2213 SDPort = 9103
2214 Password = "xxxxxxxxxx"
2215 Device = LTO-Changer-1
2216 Media Type = LTO-4
2217 Maximum Concurrent Jobs = 50
2218 }
2219
2220 Storage {
2221 Name = Changer-1-Drive0
2222 Address = cibou.company.com
2223 SDPort = 9103
2224 Password = "xxxxxxxxxx"
2225 Device = LTO4_1_Drive0
2226 Media Type = LTO-4
2227 Maximum Concurrent Jobs = 5
2228 Autochanger = Changer-1 # New directive
2229 }
2230
2231 Storage {
2232 Name = Changer-1-Drive1
2233 Address = cibou.company.com
2234 SDPort = 9103
2235 Password = "xxxxxxxxxx"
2236 Device = LTO4_1_Drive1
2237 Media Type = LTO-4
2238 Maximum Concurrent Jobs = 5
2239 Autochanger = Changer-1 # New directive
2240 }
2241
2242 ...
2243 \end{bVerbatim}
2244
2245 Note that Storage resources {\bf Changer-1-Drive0} and
2246 {\bf Changer-1-Drive1} are not required since they make
2247 up part of an autochanger, and normally, Jobs refer only
2248 to the Autochanger resource.
2249 However, by referring to those
2250 Storage definitions in a Job, you will use only
2251 the indicated drive. This is not normally what
2252 you want to do, but it is very useful and often used
2253 for reserving a drive for restores. See the Storage daemon
2254 example .conf below and the use of {\bf AutoSelect = no}.
2255
2256 So, in summary, the changes are:
2257 \begin{itemize}
2258 \item Change {\bf Storage} to {\bf Autochanger} in the LTO4 resource.
2259 \item Remove the {\bf Autochanger = yes} from the {\bf Autochanger}
2260 LTO4 resource.
2261 \item Change the {\bf Autochanger = yes} in each of the {\bf Storage}
2262 device that belong to the {\bf Autochanger} to point to the
2263 {\bf Autochanger} resource with for the example above the
2264 directive {\bf Autochanger = LTO4}.
2265
2266 \smallskip
2267 Please note that if you define two different autochangers, you must
2268 give a unique Media Type to the Volumes in each autochanger. More
2269 specifically, you may have multiple Media Types, but you cannot have
2270 Volumes with the same Media Type in two different autochangers. If you
2271 attempt to do so, Bacula will most likely reference the wrong autochanger
2272 (Storage) and not find the correct Volume.
2273 \end{itemize}
2274
2275 \subsection{New Prune Command Option}
2276 The bconsole ``prune'' command can now run the pruning algorithm on all
2277 volumes from a Pool or on all Pools.
2278
2279 \begin{bVerbatim}
2280 * prune allfrompool pool=Default yes
2281 * prune allfrompool allpools yes
2282 \end{bVerbatim}
2283
2284 \subsection{BConsole Features}
2285
2286 \subsubsection{Delete a Client}
2287 The \texttt{delete client} bconsole command delete
2288 the database record of a client that is no longer defined in the
2289 configuration file. It also removes all other records (Jobs, Files, ...)
2290 associated with the client that is deleted.
2291
2292 \subsubsection{Status Schedule Enhancements}
2293 The \texttt{status schedule} command can now accept multiple
2294 \texttt{client} or \texttt{job} keywords on the command line. The
2295 \texttt{limit} parameter is disabled when the \texttt{days} parameter is
2296 used. The output is now ordered by day.
2297
2298 \subsubsection{Restore option ``noautoparent''}
2299 During a \texttt{bconsole} restore session, parent directories are
2300 automatically selected to avoid issues with permissions. It is possible to
2301 disable this feature with the \texttt{noautoparent} command line parameter.
2302
2303 \subsection{Tray Monitor Restore Screen}
2304 It is now possible to restore files from the Tray Monitor GUI program.
2305
2306 \begin{figure}[htbp]
2307 \centering
2308 \includegraphics[width=0.8\linewidth]{tray-monitor-restore}
2309 \caption{Tray Monitor Restore Wizard}
2310 \label{fig:tray-monitor-restore}
2311 \end{figure}
2312
2313
2314 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%5
2315
2316 \chapter{New Features in 9.0.0}
2317 This chapter describes new features that have been added to the
2318 current version of Bacula in version 9.0.0
2319
2320 \subsection{Enhanced Autochanger Support}
2321 To make Bacula function properly with multiple Autochanger definitions, in
2322 the Director's configuration, you must adapt your {\bf bacula-dir.conf}
2323 {\bf Storage} directives.
2324
2325 \smallskip
2326 Each autochanger that you have defined in an {\bf Autochanger}
2327 resource in the Storage daemon's {\bf bacula-sd.conf} file,
2328 must have a corresponding {\bf Autochanger} resource defined
2329 in the Director's {\bf bacula-dir.conf} file. Normally you will
2330 already have a {\bf Storage} resource that points to the
2331 Storage daemon's {\bf Autochanger} resource. Thus you need
2332 only to change the name of the {\bf Storage} resource to
2333 {\bf Autochanger}. In addition the {\bf Autochanger = yes}
2334 directive is not needed in the Director's {\bf Autochanger}
2335 resource, since the resource name is {\bf Autochanger}, the
2336 Director already knows that it represents an autochanger.
2337
2338 \smallskip
2339 In addition to the above change ({\bf Storage} to {\bf Autochanger}),
2340 you must modify any additional {\bf Storage} resources that correspond
2341 to devices that are part of the {\bf Autochanger} device.
2342 Instead of the previous {\bf Autochanger = yes} directive they
2343 should be modified to be {\bf Autochanger = xxx} where you
2344 replace the {\bf xxx} with the name of the Autochanger.
2345
2346 \smallskip
2347 For example, in the bacula-dir.conf file:
2348
2349 \begin{bVerbatim}
2350 Autochanger { # New resource
2351 Name = Changer-1
2352 Address = cibou.company.com
2353 SDPort = 9103
2354 Password = "xxxxxxxxxx"
2355 Device = LTO-Changer-1
2356 Media Type = LTO-4
2357 Maximum Concurrent Jobs = 50
2358 }
2359
2360 Storage {
2361 Name = Changer-1-Drive0
2362 Address = cibou.company.com
2363 SDPort = 9103
2364 Password = "xxxxxxxxxx"
2365 Device = LTO4_1_Drive0
2366 Media Type = LTO-4
2367 Maximum Concurrent Jobs = 5
2368 Autochanger = Changer-1 # New directive
2369 }
2370
2371 Storage {
2372 Name = Changer-1-Drive1
2373 Address = cibou.company.com
2374 SDPort = 9103
2375 Password = "xxxxxxxxxx"
2376 Device = LTO4_1_Drive1
2377 Media Type = LTO-4
2378 Maximum Concurrent Jobs = 5
2379 Autochanger = Changer-1 # New directive
2380 }
2381
2382 ...
2383 \end{bVerbatim}
2384
2385 Note that Storage resources {\bf Changer-1-Drive0} and
2386 {\bf Changer-1-Drive1} are not required since they make
2387 up part of an autochanger, and normally, Jobs refer only
2388 to the Autochanger resource.
2389 However, by referring to those
2390 Storage definitions in a Job, you will use only
2391 the indicated drive. This is not normally what
2392 you want to do, but it is very useful and often used
2393 for reserving a drive for restores. See the Storage daemon
2394 example .conf below and the use of {\bf AutoSelect = no}.
2395
2396 So, in summary, the changes are:
2397 \begin{itemize}
2398 \item Change {\bf Storage} to {\bf Autochanger} in the LTO4 resource.
2399 \item Remove the {\bf Autochanger = yes} from the {\bf Autochanger}
2400 LTO4 resource.
2401 \item Change the {\bf Autochanger = yes} in each of the {\bf Storage}
2402 device that belong to the {\bf Autochanger} to point to the
2403 {\bf Autochanger} resource with for the example above the
2404 directive {\bf Autochanger = LTO4}.
2405 \end{itemize}
2406
2407
2408 \subsection{Source Code for Windows}
2409 With this version of Bacula, we have included the old source code for
2410 Windows and also updated it to contain the code from the latest Bacula
2411 Enterprise version. The project is also directly distributing binaries for
2412 Windows rather than relying on Bacula Systems to supply them.
2413
2414 \subsection{Maximum Virtual Full Interval Option}
2415 Two new director directives have been added:
2416
2417 \begin{bVerbatim}
2418 Max Virtual Full Interval
2419 and
2420 Virtual Full Backup Pool
2421 \end{bVerbatim}
2422
2423 The {\bf Max Virtual Full Interval} directive should behave similar to the
2424 {\bf Max Full Interval}, but for Virtual Full jobs. If Bacula sees that
2425 there has not been a Full backup in Max Virtual Full Interval time then it
2426 will upgrade the job to Virtual Full. If you have both {\bf Max Full
2427 Interval} and {\bf Max Virtual Full Interval} set then Max Full Interval
2428 should take precedence.
2429
2430 The {\bf Virtual Full Backup Pool} directive allows one to change the pool
2431 as well. You probably want to use these two directives in
2432 conjunction with each other but that may depend on the specifics of one's
2433 setup. If you set the {\bf Max Full Interval} without setting {\bf Max
2434 Virtual Full Interval} then Bacula will use whatever the "default" pool is
2435 set to which is the same behavior as with the Max Full Interval.
2436
2437 \subsection{Progressive Virtual Full}
2438
2439 In Bacula version 9.0.0, we have added a new Directive named {\bf Backups To Keep} that
2440 permits you to implement Progressive Virtual Fulls within Bacula. Sometimes
2441 this feature is known as Incremental Forever with Consolidation.
2442
2443 \smallskip
2444
2445 \begin{figure}[htbp]
2446 \centering
2447 \includegraphics[width=.8\linewidth]{pvf-slidingbackups}
2448 \caption{Backup Sequence Slides Forward One Day, Each Day}
2449 \label{fig:slidingbackups}
2450 \end{figure}
2451
2452 To implement the Progressive Virtual Full feature, simply add the
2453 {\bf Backups To Keep} directive to your Virtual Full backup Job resource.
2454 The value specified on the directive indicates the number of backup jobs
2455 that should not be merged into the Virtual Full (i.e. the number of backup
2456 jobs that should remain after the Virtual Full has completed. The default
2457 is zero, which reverts to a standard Virtual Full than consolidates all the
2458 backup jobs that it finds.
2459
2460 \subsubsection{Backups To Keep Directive}
2461 The new {\bf BackupsToKeep} directive is specified in the Job Resource and
2462 has the form:
2463
2464 \begin{bVerbatim}
2465 Backups To Keep = 30
2466 \end{bVerbatim}
2467
2468 where the value (30 in the above figure and example) is the number of
2469 backups to retain. When this directive is present during a Virtual Full
2470 (it is ignored for other Job types), it will look for the most recent Full
2471 backup that has more subsequent backups than the value specified. In the
2472 above example the Job will simply terminate unless there is a Full back
2473 followed by at least 31 backups of either level Differential or
2474 Incremental.
2475
2476 \smallskip
2477 Assuming that the last Full backup is followed by 32 Incremental backups, a
2478 Virtual Full will be run that consolidates the Full with the first two
2479 Incrementals that were run after the Full. The result is that you will end
2480 up with a Full followed by 30 Incremental backups. The Job Resource
2481 in {\bf bacula-dir.conf} to accomplish this would be:
2482
2483 \begin{bVerbatim}
2484 Job {
2485 Name = "VFull"
2486 Type = Backup
2487 Level = VirtualFull
2488 Client = "my-fd"
2489 File Set = "FullSet"
2490 Accurate = Yes
2491 Backups To Keep = 10
2492 }
2493 \end{bVerbatim}
2494
2495 \subsubsection{Delete Consolidated Jobs}
2496 The new directive {\bf Delete Consolidated Jobs} expects a {\bf yes}
2497 or {\bf no} value that if set to {\bf yes} will cause any old Job that is
2498 consolidated during a Virtual Full to be deleted. In the example above
2499 we saw that a Full plus one other job (either an Incremental or
2500 Differential) were consolidated into a new Full backup. The original Full
2501 plus the other Job consolidated will be deleted. The default value is
2502 {\bf no}.
2503
2504 \subsubsection{Virtual Full Compatibility}
2505 Virtual Full as well as Progressive Virtual Full works with any
2506 standard backup Job.
2507
2508 \smallskip
2509 However, it should be noted that Virtual Full jobs are not compatible with
2510 any plugins that you may be using.
2511
2512 \subsection{TapeAlert Enhancements}
2513 There are some significant enhancements to the TapeAlert feature of Bacula.
2514 Several directives are used slightly differently, which unfortunately
2515 causes a compatibility problem with the old TapeAlert implementation.
2516 Consequently, if you are already using TapeAlert, you must modify your
2517 {\bf bacula-sd.conf} in order for Tape Alerts to work. See below
2518 for the details ...
2519
2520 \subsubsection{What is New}
2521 First, you must define a \textbf{Alert Command} directive in the Device
2522 resource that calls the new \textbf{tapealert} script that is installed in
2523 the scripts directory (normally: /opt/bacula/scripts). It is defined as
2524 follows:
2525
2526 \begin{bVerbatim}
2527 Device {
2528 Name = ...
2529 Archive Device = /dev/nst0
2530 Alert Command = "/opt/bacula/scripts/tapealert %l"
2531 Control Device = /dev/sg1 # must be SCSI ctl for /dev/nst0
2532 ...
2533 }
2534 \end{bVerbatim}
2535
2536 In addition the \textbf{Control Device} directive in the Storage Daemon's
2537 conf file must be specified in each Device resource to permit Bacula to
2538 detect tape alerts on a specific devices (normally only tape devices).
2539
2540 Once the above mentioned two directives (Alert Command and Control Device)
2541 are in place in each of your Device resources, Bacula will check for tape
2542 alerts at two points:
2543
2544 \begin{itemize}
2545 \item After the Drive is used and it becomes idle.
2546 \item After each read or write error on the drive.
2547 \end{itemize}
2548
2549 At each of the above times, Bacula will call the new \textbf{tapealert}
2550 script, which uses the \textbf{tapeinfo} program. The tapeinfo utility is
2551 part of the apt sg3-utils and rpm sg3\_utils packages that must be
2552 installed on your systems. Then after each alert that Bacula finds for
2553 that drive, Bacula will emit a Job message that is either INFO, WARNING, or
2554 FATAL depending on the designation in the Tape Alert published by the T10
2555 Technical Committee on SCSI Storage Interfaces (www.t10.org). For the
2556 specification, please see: www.t10.org/ftp/t10/document.02/02-142r0.pdf
2557
2558 \smallskip
2559 As a somewhat extreme example, if tape alerts 3, 5, and 39 are set, you
2560 will get the following output in your backup job.
2561
2562 {\small
2563 \begin{bVerbatim}
2564 17-Nov 13:37 rufus-sd JobId 1: Error: block.c:287
2565 Write error at 0:17 on device "tape"
2566 (/home/kern/bacula/k/regress/working/ach/drive0)
2567 Vol=TestVolume001. ERR=Input/output error.
2568
2569 17-Nov 13:37 rufus-sd JobId 1: Fatal error: Alert:
2570 Volume="TestVolume001" alert=3: ERR=The operation has stopped because
2571 an error has occurred while reading or writing data which the drive
2572 cannot correct. The drive had a hard read or write error
2573
2574 17-Nov 13:37 rufus-sd JobId 1: Fatal error: Alert:
2575 Volume="TestVolume001" alert=5: ERR=The tape is damaged or the drive
2576 is faulty. Call the tape drive supplier helpline. The drive can no
2577 longer read data from the tape
2578
2579 17-Nov 13:37 rufus-sd JobId 1: Warning: Disabled Device "tape"
2580 (/home/kern/bacula/k/regress/working/ach/drive0) due to tape alert=39.
2581
2582 17-Nov 13:37 rufus-sd JobId 1: Warning: Alert: Volume="TestVolume001"
2583 alert=39: ERR=The tape drive may have a fault. Check for availability
2584 of diagnostic information and run extended diagnostics if applicable.
2585 The drive may have had a failure which may be identified by stored
2586 diagnostic information or by running extended diagnostics (eg Send
2587 Diagnostic). Check the tape drive users manual for instructions on
2588 running extended diagnostic tests and retrieving diagnostic data.
2589
2590 \end{bVerbatim}
2591 }
2592
2593 Without the tape alert feature enabled, you would only get the first error
2594 message above, which is the error return Bacula received when it gets the
2595 error. Notice also, that in the above output the alert number 5 is a
2596 critical error, which causes two things to happen. First the tape drive is
2597 disabled, and second the Job is failed.
2598
2599 \smallskip
2600 If you attempt to run another Job using the Device that has been disabled,
2601 you will get a message similar to the following:
2602
2603 \begin{bVerbatim}
2604 17-Nov 15:08 rufus-sd JobId 2: Warning:
2605 Device "tape" requested by DIR is disabled.
2606 \end{bVerbatim}
2607
2608 and the Job may be failed if no other drive can be found.
2609
2610 \smallskip
2611 Once the problem with the tape drive has been corrected, you can
2612 clear the tape alerts and re-enable the device with the Bacula bconsole
2613 command such as the following:
2614
2615 \begin{bVerbatim}
2616 enable Storage=Tape
2617 \end{bVerbatim}
2618
2619 Note, when you enable the device, the list of prior tape alerts for that
2620 drive will be discarded.
2621
2622 \smallskip
2623 Since is is possible to miss tape alerts, Bacula maintains a temporary list
2624 of the last 8 alerts, and each time Bacula calls the \textbf{tapealert}
2625 script, it will keep up to 10 alert status codes. Normally there will only
2626 be one or two alert errors for each call to the tapealert script.
2627
2628 \smallskip
2629 Once a drive has one or more tape alerts, you can see them by using the
2630 bconsole status command as follows:
2631 \begin{bVerbatim}
2632 status storage=Tape
2633 \end{bVerbatim}
2634 which produces the following output:
2635 \begin{bVerbatim}
2636 Device Vtape is "tape" (/home/kern/bacula/k/regress/working/ach/drive0)
2637 mounted with:
2638 Volume: TestVolume001
2639 Pool: Default
2640 Media type: tape
2641 Device is disabled. User command.
2642 Total Bytes Read=0 Blocks Read=1 Bytes/block=0
2643 Positioned at File=1 Block=0
2644 Critical Alert: at 17-Nov-2016 15:08:01 Volume="TestVolume001"
2645 alert=Hard Error
2646 Critical Alert: at 17-Nov-2016 15:08:01 Volume="TestVolume001"
2647 alert=Read Failure
2648 Warning Alert: at 17-Nov-2016 15:08:01 Volume="TestVolume001"
2649 alert=Diagnostics Required
2650 \end{bVerbatim}
2651 if you want to see the long message associated with each of the alerts,
2652 simply set the debug level to 10 or more and re-issue the status command:
2653 \begin{bVerbatim}
2654 setdebug storage=Tape level=10
2655 status storage=Tape
2656 \end{bVerbatim}
2657 \begin{bVerbatim}
2658 ...
2659 Critical Alert: at 17-Nov-2016 15:08:01 Volume="TestVolume001"
2660 flags=0x0 alert=The operation has stopped because an error has occurred
2661 while reading or writing data which the drive cannot correct. The drive had
2662 a hard read or write error
2663 Critical Alert: at 17-Nov-2016 15:08:01 Volume="TestVolume001"
2664 flags=0x0 alert=The tape is damaged or the drive is faulty. Call the tape
2665 drive supplier helpline. The drive can no longer read data from the tape
2666 Warning Alert: at 17-Nov-2016 15:08:01 Volume="TestVolume001" flags=0x1
2667 alert=The tape drive may have a fault. Check for availability of diagnostic
2668 information and run extended diagnostics if applicable. The drive may
2669 have had a failure which may be identified by stored diagnostic information
2670 or by running extended diagnostics (eg Send Diagnostic). Check the tape
2671 drive users manual for instructions on running extended diagnostic tests
2672 and retrieving diagnostic data.
2673 ...
2674 \end{bVerbatim}
2675 The next time you \textbf{enable} the Device by either using
2676 \textbf{bconsole} or you restart the Storage Daemon, all the saved alert
2677 messages will be discarded.
2678
2679 \subsubsection{Handling of Alerts}
2680 Tape Alerts numbered 7,8,13,14,20,22,52,53, and 54 will cause Bacula to
2681 disable the current Volume.
2682
2683 \smallskip
2684 Tape Alerts numbered 14,20,29,30,31,38, and 39 will cause Bacula to disable
2685 the drive.
2686
2687 \smallskip
2688 Please note certain tape alerts such as 14 have multiple effects (disable
2689 the Volume and disable the drive).
2690
2691 \subsection{New Console ACL Directives}
2692 By default, if a Console ACL directive is not set, Bacula will assume that the
2693 ACL list is empty. If the current Bacula Director configuration uses restricted
2694 Consoles and allows restore jobs, it is mandatory to configure the new
2695 directives.
2696
2697 \subsubsection{DirectoryACL}
2698 \index[dir]{Directive!DirectoryACL}
2699
2700 This directive is used to specify a list of directories that can be
2701 accessed by a restore session. Without this directive, a restricted
2702 console cannot restore any file. Multiple directories names may be
2703 specified by separating them with commas, and/or by specifying multiple
2704 DirectoryACL directives. For example, the directive may be specified as:
2705
2706 \footnotesize
2707 \begin{bVerbatim}
2708 DirectoryACL = /home/bacula/, "/etc/", "/home/test/*"
2709 \end{bVerbatim}
2710 \normalsize
2711
2712 With the above specification, the console can access the following
2713 directories:
2714 \begin{itemize}
2715 \item \texttt{/etc/password}
2716 \item \texttt{/etc/group}
2717 \item \texttt{/home/bacula/.bashrc}
2718 \item \texttt{/home/test/.ssh/config}
2719 \item \texttt{/home/test/Desktop/Images/something.png}
2720 \end{itemize}
2721
2722 But not to the following files or directories:
2723 \begin{itemize}
2724 \item \texttt{/etc/security/limits.conf}
2725 \item \texttt{/home/bacula/.ssh/id\_dsa.pub}
2726 \item \texttt{/home/guest/something}
2727 \item \texttt{/usr/bin/make}
2728 \end{itemize}
2729
2730 If a directory starts with a Windows pattern (ex: c:/), Bacula will
2731 automatically ignore the case when checking directory names.
2732
2733 \subsection{New Bconsole ``list'' Command Behavior}
2734
2735 The bconsole \texttt{list} commands can now be used safely from a
2736 restricted bconsole session. The information displayed will respect the
2737 ACL configured for the Console session. For example, if a restricted
2738 Console has access to JobA, JobB and JobC, information about JobD will not
2739 appear in the \texttt{list jobs} command.
2740
2741 \subsection{New Console ACL Directives}
2742 \index[dir]{Directive!BackupClientACL}
2743 It is now possible to configure a restricted Console to distinguish Backup
2744 and Restore job permissions. The \texttt{BackupClientACL} can restrict
2745 backup jobs on a specific set of clients, while the
2746 \texttt{RestoreClientACL} can restrict restore jobs.
2747
2748 {\small
2749 \begin{bVerbatim}
2750 # cat /opt/bacula/etc/bacula-dir.conf
2751 ...
2752
2753 Console {
2754 Name = fd-cons # Name of the FD Console
2755 Password = yyy
2756 ...
2757 ClientACL = localhost-fd # everything allowed
2758 RestoreClientACL = test-fd # restore only
2759 BackupClientACL = production-fd # backup only
2760 }
2761 \end{bVerbatim}
2762 }
2763
2764 The \texttt{ClientACL} directive takes precedence over the
2765 \texttt{RestoreClientACL} and the \texttt{BackupClientACL}. In the Console
2766 resource resource above, it means that the bconsole linked to the Console{}
2767 named "fd-cons" will be able to run:
2768
2769 \begin{itemize}
2770 \item backup and restore for ``localhost-fd''
2771 \item backup for ``production-fd''
2772 \item restore for ``test-fd''
2773 \end{itemize}
2774
2775 At the restore time, jobs for client ``localhost-fd'', ``test-fd'' and
2776 ``production-fd'' will be available.
2777
2778 If \texttt{*all*} is set for \texttt{ClientACL}, backup and restore will be
2779 allowed for all clients, despite the use of \texttt{RestoreClientACL} or
2780 \texttt{"BackupClientACL}.
2781
2782 \subsection{Client Initiated Backup}
2783 \label{sec:featurecib}
2784 A console program such as the new \texttt{tray-monitor} or
2785 \texttt{bconsole} can now be configured to connect a File Daemon. There
2786 are many new features available (see the New Tray Monitor section below),
2787 but probably the most important is the ability for the user to initiate a
2788 backup of his own machine. The connection established by the FD to the
2789 Director for the backup will be used by the Director for the backup, thus
2790 not only can clients (users) initiate backups, but a File Daemon that is
2791 NATed (cannot be reached by the Director) can now be backed up without
2792 using advanced tunneling techniques providing that the File Daemon can
2793 connect to the Director.
2794
2795 \smallskip
2796 The flow of information is shown in the picture below:
2797 \bimageH{nat}{Client Initiated Backup Network Flow}{fig:nat3}
2798
2799 \newpage
2800 \subsection{Configuring Client Initiated Backup}
2801 \smallskip
2802 In order to ensure security, there are a number of new directives
2803 that must be enabled in the new \texttt{tray-monitor}, the File
2804 Daemon and in the Director.
2805 A typical configuration might look like the following:
2806
2807 {\small
2808 \begin{bVerbatim}
2809 # cat /opt/bacula/etc/bacula-dir.conf
2810 ...
2811
2812 Console {
2813 Name = fd-cons # Name of the FD Console
2814 Password = yyy
2815
2816 # These commands are used by the tray-monitor, it is possible to restrict
2817 CommandACL = run, restore, wait, .status, .jobs, .clients
2818 CommandACL = .storages, .pools, .filesets, .defaults, .estimate
2819
2820 # Adapt for your needs
2821 jobacl = *all*
2822 poolacl = *all*
2823 clientacl = *all*
2824 storageacl = *all*
2825 catalogacl = *all*
2826 filesetacl = *all*
2827 }
2828 \end{bVerbatim}
2829 }
2830
2831 {\small
2832 \begin{bVerbatim}
2833 # cat /opt/bacula/etc/bacula-fd.conf
2834 ...
2835
2836 Console { # Console to connect the Director
2837 Name = fd-cons
2838 DIRPort = 9101
2839 address = localhost
2840 Password = "yyy"
2841 }
2842
2843 Director {
2844 Name = remote-cons # Name of the tray monitor/bconsole
2845 Password = "xxx" # Password of the tray monitor/bconsole
2846 Remote = yes # Allow to use send commands to the Console defined
2847 }
2848 \end{bVerbatim}
2849 }
2850
2851 {\small
2852 \begin{bVerbatim}
2853 cat /opt/bacula/etc/bconsole-remote.conf
2854 ....
2855
2856 Director {
2857 Name = localhost-fd
2858 address = localhost # Specify the FD address
2859 DIRport = 9102 # Specify the FD Port
2860 Password = "notused"
2861 }
2862
2863 Console {
2864 Name = remote-cons # Name used in the auth process
2865 Password = "xxx"
2866 }
2867 \end{bVerbatim}
2868 }
2869
2870 {\small
2871 \begin{bVerbatim}
2872 cat ~/.bacula-tray-monitor.conf
2873 Monitor {
2874 Name = remote-cons
2875 }
2876
2877 Client {
2878 Name = localhost-fd
2879 address = localhost # Specify the FD address
2880 Port = 9102 # Specify the FD Port
2881 Password = "xxx"
2882 Remote = yes
2883 }
2884 \end{bVerbatim}
2885 }
2886
2887 \bimageH{conf-nat}{Relation Between Resources (bconsole)}{fig:nat}
2888 \bimageH{conf-nat2}{Relation Between Resources (tray-monitor)}{fig:nat2}
2889
2890 \medskip
2891 A more detailed description with complete examples is available in
2892 chapter~\ref{TrayMonitorChapter}.
2893
2894 \subsection{New Tray Monitor}
2895
2896 A new tray monitor has been added to the 9.0 release, the tray monitor offers
2897 the following features:
2898
2899 \begin{itemize}
2900 \item Director, File and Storage Daemon status page
2901 \item Support for the Client Initiated Backup protocol (See
2902 \vref{sec:featurecib}). To use the Client Initiated Backup option from the
2903 tray monitor, the Client option ``Remote'' should be checked in the
2904 configuration (Fig \vref{fig:tray2}).
2905 \item Wizard to run new job (Fig \vref{fig:tray4})
2906 \item Display an estimation of the number of files and the size of the next
2907 backup job (Fig \vref{fig:tray4})
2908 \item Ability to configure the tray monitor configuration file directly from
2909 the GUI (Fig \vref{fig:tray2})
2910 \item Ability to monitor a component and adapt the tray monitor task bar icon
2911 if a jobs are running.
2912 \item TLS Support
2913 \item Better network connection handling
2914 \item Default configuration file is stored under \texttt{\$HOME/.bacula-tray-monitor.conf}
2915 \item Ability to ``schedule'' jobs
2916 \item Available on Linux and Windows platforms
2917 \end{itemize}
2918
2919 % \medskip
2920 % Please see chapter \ref{TrayMonitorChapter} for more details about this new
2921 % functionality.
2922
2923
2924 \begin{figure}[htbp]
2925 \centering
2926 \includegraphics[width=0.8\linewidth]{tray-monitor-status}
2927 \caption{Tray Monitor Status}
2928 \label{fig:tray0}
2929 \end{figure}
2930
2931 \begin{figure}[htbp]
2932 \centering
2933 \includegraphics[width=0.9\linewidth]{tray-monitor-conf-fd}
2934 \caption{Tray Monitor Client Configuration}
2935 \label{fig:tray2}
2936 \end{figure}
2937
2938 \begin{figure}[htbp]
2939 \centering
2940 \includegraphics[width=0.8\linewidth]{tray-monitor-run1}
2941 \smallskip
2942 \includegraphics[width=0.8\linewidth]{tray-monitor-run2}
2943 \caption{Tray Monitor Run a Job}
2944 \label{fig:tray4}
2945 \end{figure}
2946
2947 \subsection{Schedule Jobs via the Tray Monitor}
2948
2949 The Tray Monitor can scan periodically a specific directory ``Command
2950 Directory'' and process ``*.bcmd'' files to find jobs to run.
2951
2952 The format of the ``file.bcmd'' command file is the following:
2953 \begin{bVerbatim}
2954 <component name>:<run command>
2955 <component name>:<run command>
2956 ...
2957
2958 <component name> = string
2959 <run command> = string (bconsole command line)
2960 \end{bVerbatim}
2961
2962 For example:
2963 \begin{bVerbatim}
2964 localhost-fd: run job=backup-localhost-fd level=full
2965 localhost-dir: run job=BackupCatalog
2966 \end{bVerbatim}
2967
2968 The command file should contain at least one command. The component specified
2969 in the first part of the command line should be defined in the tray
2970 monitor. Once the command file is detected by the tray monitor, a popup is
2971 displayed to the user and it is possible for the user to cancel the job directly.
2972
2973 \smallskip{}
2974
2975 The file can be created with tools such as ``cron'' or the ``task scheduler''
2976 on Windows. It is possible to verify the network connection at that time to
2977 avoid network errors.
2978
2979 \begin{bVerbatim}
2980 #!/bin/sh
2981 if ping -c 1 director &> /dev/null
2982 then
2983 echo "my-dir: run job=backup" > /path/to/commands/backup.bcmd
2984 fi
2985 \end{bVerbatim}
2986
2987 %\bimageH{tray-monitor-status}{Tray Monitor Status}{fig:tray0}
2988 %\bimageH{tray-monitor1}{Tray Monitor Configuration}{fig:tray1}
2989 %\bimageH{tray-monitor-conf-fd}{Tray Monitor Client Configuration}{fig:tray2}
2990 %\bimageH{tray-monitor-conf-dir}{Tray Monitor Director Configuration}{fig:tray3}
2991 %\bimageH{tray-monitor-run1}{Tray Monitor Run new Job}{fig:tray4}
2992 % find a way to group them together
2993 %\bimageH{tray-monitor-run2}{Tray Monitor Setup new Job}{fig:tray5}
2994
2995
2996 \subsection{Accurate Option for Verify ``Volume Data'' Job}
2997
2998 Since Bacula version 8.4.1, it has been possible to have a Verify Job
2999 configured with \texttt{level=Data} that will reread all records from a job
3000 and optionally check the size and the checksum of all files. Starting with
3001
3002 \smallskip
3003 Bacula version 9.0, it is now possible to use the \texttt{accurate} option to check
3004 catalog records at the same time. When using a Verify job with
3005 \texttt{level=Data} and \texttt{accurate=yes} can replace the
3006 \texttt{level=VolumeToCatalog} option.
3007
3008 For more information on how to setup a Verify Data job, see
3009 \vref{blb:verifyvolumedata}.
3010
3011 To run a Verify Job with the \texttt{accurate} option, it is possible to set
3012 the option in the Job definition or set use the \texttt{accurate=yes} on the
3013 command line.
3014
3015 \begin{bVerbatim}
3016 * run job=VerifyData jobid=10 accurate=yes
3017 \end{bVerbatim}
3018
3019 \subsection{FileDaemon Saved Messages Resource Destination}
3020
3021 It is now possible to send the list of all saved files to a Messages
3022 resource with the \texttt{saved} message type. It is not recommended to
3023 send this flow of information to the director and/or the catalog when the
3024 client FileSet is pretty large. To avoid side effects, the \texttt{all}
3025 keyword doesn't include the \texttt{saved} message type. The
3026 \texttt{saved} message type should be explicitly set.
3027
3028 \begin{bVerbatim}
3029 # cat /opt/bacula/etc/bacula-fd.conf
3030 ...
3031 Messages {
3032 Name = Standard
3033 director = mydirector-dir = all, !terminate, !restored, !saved
3034 append = /opt/bacula/working/bacula-fd.log = all, saved, restored
3035 }
3036 \end{bVerbatim}
3037
3038 \subsection{Minor Enhancements}
3039
3040 \subsubsection{New Bconsole ".estimate" Command}
3041
3042 The new \texttt{.estimate} command can be used to get statistics about a
3043 job to run. The command uses the database to approximate the size and the
3044 number of files of the next job. On a PostgreSQL database, the command
3045 uses regression slope to compute values. On MySQL, where these
3046 statistical functions are not available, the command uses a simple
3047 ``average'' estimation. The correlation number is given for each value.
3048
3049 {\small
3050 \begin{bVerbatim}
3051 *.estimate job=backup
3052 level=I
3053 nbjob=0
3054 corrbytes=0
3055 jobbytes=0
3056 corrfiles=0
3057 jobfiles=0
3058 duration=0
3059 job=backup
3060
3061 *.estimate job=backup level=F
3062 level=F
3063 nbjob=1
3064 corrbytes=0
3065 jobbytes=210937774
3066 corrfiles=0
3067 jobfiles=2545
3068 duration=0
3069 job=backup
3070 \end{bVerbatim}
3071 }
3072
3073 \subsubsection{Traceback and Lockdump}
3074
3075 After the reception of a signal, \texttt{traceback} and \texttt{lockdump}
3076 information are now stored in the same file.
3077
3078 \subsection{Bconsole ``list jobs'' command options}
3079
3080 The \texttt{list jobs} bconsole command now accepts new command line options:
3081
3082 \begin{itemize}
3083 \item \textbf{joberrors} Display jobs with JobErrors
3084 \item \textbf{jobstatus=T} Display jobs with the specified status code
3085 \item \textbf{client=cli} Display jobs for a specified client
3086 \item \textbf{order=asc/desc} Change the output format of the job list. The
3087 jobs are sorted by start time and JobId, the sort can use ascendant (asc) or
3088 descendant (desc) (default) value.
3089 \end{itemize}
3090
3091 \subsection{Minor Enhancements}
3092
3093 \subsubsection{New Bconsole "Tee All" Command}
3094
3095 The ``@tall'' command allows logging all input/output from a console session.
3096
3097 \begin{bVerbatim}
3098 *@tall /tmp/log
3099 *st dir
3100 ...
3101 \end{bVerbatim}
3102
3103 \subsection{Bconsole ``list jobs'' command options}
3104
3105 The \texttt{list jobs} bconsole command now accepts new command line options:
3106
3107 \begin{itemize}
3108 \item \textbf{joberrors} Display jobs with JobErrors
3109 \item \textbf{jobstatus=T} Display jobs with the specified status code
3110 \item \textbf{client=cli} Display jobs for a specified client
3111 \item \textbf{order=asc/desc} Change the output format of the job list. The
3112 jobs are sorted by start time and JobId, the sort can use ascendant (asc) or
3113 descendant (desc) (default) value.
3114 \end{itemize}
3115
3116 \subsection{New Bconsole "Tee All" Command}
3117
3118 The ``@tall'' command allows logging all input/output from a console session.
3119
3120 \begin{bVerbatim}
3121 *@tall /tmp/log
3122 *st dir
3123 ...
3124 \end{bVerbatim}
3125
3126 \subsection{New Job Edit Codes \%I}
3127 In various places such as RunScripts, you have now access to \%I to get the
3128 JobId of the copy or migration job started by a migrate job.
3129
3130 \begin{bVerbatim}
3131 Job {
3132 Name = Migrate-Job
3133 Type = Migrate
3134 ...
3135 RunAfter = "echo New JobId is %I"
3136 }
3137 \end{bVerbatim}
3138
3139
3140 \subsection*{.api version 2}
3141
3142 In Bacula version 9.0 and later, we introduced a new .api version
3143 to help external tools to parse various Bacula bconsole output.
3144
3145 % waa - 20150317 - this section needs just a little more to explain what the "43" in "s43" mean. Perhaps
3146 % if it is not a good place to list the possibilities here, then list where a reference
3147 % is. Also, I think .api 2 ... Means "use API version 2" but that should be stated too
3148
3149 The \texttt{api\_opts} option can use the following arguments:
3150 \begin{itemize}
3151 \item [C] Clear current options
3152 \item [tn] Use a specific time format (1 ISO format, 2 Unix Timestamp, 3 Default Bacula time format)
3153 \item [sn] Use a specific separator between items (new line by default).
3154 \item [Sn] Use a specific separator between objects (new line by default).
3155 \item [o] Convert all keywords to lowercase and convert all non \textsl{isalpha} characters to \_
3156 \end{itemize}
3157
3158 % waa - 20150317 - I think there should either be more output listed here to give a better feeling
3159 % or, perhaps another output listing for different .status commands
3160
3161 \begin{bVerbatim}
3162 .api 2 api_opts=t1s43S35
3163 .status dir running
3164 ==================================
3165 jobid=10
3166 job=AJob
3167 ...
3168 \end{bVerbatim}
3169
3170 \subsection*{New Debug Options}
3171
3172 In Bacula version 9.0 and later, we introduced a new \texttt{options} parameter for
3173 the \texttt{setdebug} bconsole command.
3174
3175 \smallskip{}
3176
3177 The following arguments to the new \texttt{option} parameter are available to control debug functions.
3178
3179 \begin{itemize}
3180 \item [0] Clear debug flags
3181 \item [i] Turn off, ignore bwrite() errors on restore on File Daemon
3182 \item [d] Turn off decomp of BackupRead() streams on File Daemon
3183 \item [t] Turn on timestamps in traces
3184 \item [T] Turn off timestamps in traces
3185
3186 % waa - 20150306 - does this "c" item mean to say "Truncate trace file if one exists, otherwise append to it" ???
3187 \item [c] Truncate trace file if trace file is activated
3188
3189 \item [l] Turn on recording events on P() and V()
3190 \item [p] Turn on the display of the event ring when doing a backtrace
3191 \end{itemize}
3192
3193 \smallskip{}
3194
3195 The following command will enable debugging for the File Daemon, truncate an existing trace file,
3196 and turn on timestamps when writing to the trace file.
3197
3198 \begin{bVerbatim}
3199 * setdebug level=10 trace=1 options=ct fd
3200 \end{bVerbatim}
3201
3202 \smallskip{}
3203
3204 It is now possible to use a \textsl{class} of debug messages called \texttt{tags}
3205 to control the debug output of Bacula daemons.
3206
3207 \begin{itemize}
3208 \item [all] Display all debug messages
3209 \item [bvfs] Display BVFS debug messages
3210 \item [sql] Display SQL related debug messages
3211 \item [memory] Display memory and poolmem allocation messages
3212 \item [scheduler] Display scheduler related debug messages
3213 \end{itemize}
3214
3215 \begin{bVerbatim}
3216 * setdebug level=10 tags=bvfs,sql,memory
3217 * setdebug level=10 tags=!bvfs
3218
3219 # bacula-dir -t -d 200,bvfs,sql
3220 \end{bVerbatim}
3221
3222 The \texttt{tags} option is composed of a list of tags. Tags are separated by
3223 ``,'' or ``+'' or ``-'' or ``!''. To disable a specific tag, use ``-'' or ``!''
3224 in front of the tag. Note that more tags are planned for future versions.
3225
3226 %\LTXtable{\linewidth}{table_debugtags}
3227
3228 \subsection{Communication Line Compression}
3229 Bacula version 9.0.0 and later now includes communication
3230 line compression. It is turned on by default, and if the
3231 two Bacula components (Dir, FD, SD, bconsole) are both
3232 version 6.6.0 or greater, communication line compression)
3233 will be enabled, by default. If for some reason, you do not want
3234 communication line compression, you may disable it with the
3235 following directive:
3236
3237 \begin{bVerbatim}
3238 Comm Compression = no
3239 \end{bVerbatim}
3240
3241 This directive can appear in the following resources:
3242 \begin{bVerbatim}
3243 bacula-dir.conf: Director resource
3244 bacula-fd.conf Client (or FileDaemon) resource
3245 bacula-sd.conf: Storage resource
3246 bconsole.conf: Console resource
3247 bat.conf: Console resource
3248 \end{bVerbatim}
3249
3250 \smallskip
3251 In many cases, the volume of data transmitted across the
3252 communications line can be reduced by a factor of three when
3253 this directive is enabled (default) In the case that the compression is not
3254 effective, Bacula turns it off on a. record by record basis.
3255
3256 \smallskip
3257 If you are backing up data that is already compressed the comm line
3258 compression will not be effective, and you are likely
3259 to end up with an average compression ratio that is very small.
3260 In this case, Bacula reports {\bf None} in the Job report.
3261
3262 \subsection{Deduplication Optimized Volumes}
3263 This version of Bacula includes a new alternative (or additional)
3264 volume format that optimizes the placement of files so
3265 that an underlying deduplicating filesystem such as ZFS
3266 can optimally deduplicate the backup data that is written
3267 by Bacula. These are called Deduplication Optimized Volumes
3268 or Aligned Volumes for short. The details of how to use this
3269 feature and its considerations are in the
3270 Deduplication Optimized Volumes whitepaper.
3271
3272 \smallskip
3273 This feature is available if you have Bacula Community produced binaries
3274 and the Aligned Volumes plugin.
3275
3276 \subsection{baculabackupreport}
3277 I have added a new script called {\bf baculabackupreport} to the scripts
3278 directory. This script was written by Bill Arlofski. It prints a
3279 backup summary of the backups that occurred in the prior number of
3280 hours specified on the command line. You need to edit
3281 the first few lines of the file to ensure that your email address is
3282 correct and the database type you are using is correct (default is
3283 PostgreSQL). Once you do that, you can manually implement it with:
3284
3285 \begin{bVerbatim}
3286 /opt/bacula/scripts/baculabackupreport 24
3287 \end{bVerbatim}
3288
3289 I have put the above line in my {\bf scripts/delete\_catalog\_backup} script
3290 so that it will be mailed to me nightly.
3291
3292 \subsection{New Message Identification Format}
3293 We are starting to add unique message identifiers to each message (other
3294 than debug and the Job report) that Bacula prints. At the current time
3295 only two files in the Storage Daemon have these message identifiers and
3296 over time with subsequent releases we will modify all messages.
3297
3298 \smallskip
3299 The message identifier will be kept unique for each message and once
3300 assigned to a message it will not change even if the text of the message
3301 changes. This means that the message identifier will be the same no matter
3302 what language the text is displayed in, and more importantly, it will allow
3303 us to make listing of the messages with in some cases, additional
3304 explanation or instructions on how to correct the problem. All this will
3305 take several years since it is a lot of work and requires some new programs
3306 that are not yet written to manage these message identifiers.
3307
3308 \smallskip
3309 The format of the message identifier is:
3310
3311 \begin{bVerbatim}
3312 [AAnnnn]
3313 \end{bVerbatim}
3314 where A is an upper case character and nnnn is a four digit number, where
3315 the first character indicates the software component (daemon); the second
3316 letter indicates the severity, and the number is unique for a given
3317 component and severity.
3318
3319 \smallskip
3320 For example:
3321
3322 \begin{bVerbatim}
3323 [SF0001]
3324 \end{bVerbatim}
3325
3326 The first character representing the component at the current time one of
3327 the following:
3328
3329 \begin{bVerbatim}
3330 S Storage daemon
3331 D Director
3332 F File daemon
3333 \end{bVerbatim}
3334
3335 \smallskip
3336 The second character representing the severity or level can be:
3337
3338 \begin{bVerbatim}
3339 A Abort
3340 F Fatal
3341 E Error
3342 W Warning
3343 S Security
3344 I Info
3345 D Debug
3346 O OK (i.e. operation completed normally)
3347 \end{bVerbatim}
3348
3349 So in the example above [SF0001] indicates it is a message id, because of
3350 the brackets and because it is at the beginning of the message, and that
3351 it was generated by the Storage daemon as a fatal error.
3352 \smallskip
3353 As mentioned above it will take some time to implement these message ids
3354 everywhere, and over time we may add more component letters and more
3355 severity levels as needed.
3356
3357
3358 \chapter{New Features in 7.4.0}
3359 This chapter presents the new features that have been added to
3360 the various versions of Bacula.
3361
3362 \section{New Features in 7.4.3}
3363 \subsection{RunScripts}
3364 There are two new RunScript short cut directives implemented in
3365 the Director. They are:
3366
3367 \begin{bVerbatim}
3368 Job {
3369 ...
3370 ConsoleRunBeforeJob = "console-command"
3371 ...
3372 }
3373 \end{bVerbatim}
3374
3375 \begin{bVerbatim}
3376 Job {
3377 ...
3378 ConsoleRunAfterJob = "console-command"
3379 ...
3380 }
3381 \end{bVerbatim}
3382
3383 As with other RunScript commands, you may have multiple copies
3384 of either the {\bf ConsoleRunBeforeJob} or the {\bf ConsoleRunAfterJob}
3385 in the same Job resource definition.
3386 \smallskip
3387 Please note that not all console commands are permitted, and that
3388 if you run a console command that requires a response, the results
3389 are not determined (i.e. it will probably fail).
3390
3391
3392
3393 \section{New Features in 7.4.0}
3394 \subsection{Verify Volume Data}
3395
3396 It is now possible to have a Verify Job configured with \texttt{level=Data} to
3397 reread all records from a job and optionally check the size and the checksum
3398 of all files.
3399
3400 \begin{bVerbatim}
3401 # Verify Job definition
3402 Job {
3403 Name = VerifyData
3404 Level = Data
3405 Client = 127.0.0.1-fd # Use local file daemon
3406 FileSet = Dummy # Will be adapted during the job
3407 Storage = File # Should be the right one
3408 Messages = Standard
3409 Pool = Default
3410 }
3411
3412 # Backup Job definition
3413 Job {
3414 Name = MyBackupJob
3415 Type = Backup
3416 Client = windows1
3417 FileSet = MyFileSet
3418 Pool = 1Month
3419 Storage = File
3420 }
3421
3422 FileSet {
3423 Name = MyFileSet
3424 Include {
3425 Options {
3426 Verify = s5
3427 Signature = MD5
3428 }
3429 File = /
3430 }
3431 \end{bVerbatim}
3432
3433 To run the Verify job, it is possible to use the ``jobid'' parameter of the ``run'' command.
3434
3435 \begin{bVerbatim}
3436 *run job=VerifyData jobid=10
3437 Run Verify Job
3438 JobName: VerifyData
3439 Level: Data
3440 Client: 127.0.0.1-fd
3441 FileSet: Dummy
3442 Pool: Default (From Job resource)
3443 Storage: File (From Job resource)
3444 Verify Job: MyBackupJob.2015-11-11_09.41.55_03
3445 Verify List: /opt/bacula/working/working/VerifyVol.bsr
3446 When: 2015-11-11 09:47:38
3447 Priority: 10
3448 OK to run? (yes/mod/no): yes
3449 Job queued. JobId=14
3450
3451 ...
3452
3453 11-Nov 09:46 my-dir JobId 13: Bacula 7.4.0 (13Nov15):
3454 Build OS: x86_64-unknown-linux-gnu archlinux
3455 JobId: 14
3456 Job: VerifyData.2015-11-11_09.46.29_03
3457 FileSet: MyFileSet
3458 Verify Level: Data
3459 Client: 127.0.0.1-fd
3460 Verify JobId: 10
3461 Verify Job:q
3462 Start time: 11-Nov-2015 09:46:31
3463 End time: 11-Nov-2015 09:46:32
3464 Files Expected: 1,116
3465 Files Examined: 1,116
3466 Non-fatal FD errors: 0
3467 SD Errors: 0
3468 FD termination status: Verify differences
3469 SD termination status: OK
3470 Termination: Verify Differences
3471 \end{bVerbatim}
3472
3473 The current Verify Data implementation requires specifying the correct Storage
3474 resource in the Verify job. The Storage resource can be changed with the bconsole
3475 command line and with the menu.
3476
3477 \subsection{Bconsole ``list jobs'' command options}
3478
3479 The \texttt{list jobs} bconsole command now accepts new command line options:
3480
3481 \begin{itemize}
3482 \item \textbf{joberrors} Display jobs with JobErrors
3483 \item \textbf{jobstatus=T} Display jobs with the specified status code
3484 \item \textbf{client=cli} Display jobs for a specified client
3485 \item \textbf{order=asc/desc} Change the output format of the job list. The
3486 jobs are sorted by start time and JobId, the sort can use ascendant (asc) or
3487 descendant (desc) (default) value.
3488 \end{itemize}
3489
3490 \subsection{Minor Enhancements}
3491
3492 \subsubsection{New Bconsole "Tee All" Command}
3493
3494 The ``@tall'' command allows logging all input/output from a console session.
3495
3496 \begin{bVerbatim}
3497 *@tall /tmp/log
3498 *st dir
3499 ...
3500 \end{bVerbatim}
3501
3502 \subsection{Windows Encrypted File System (EFS) Support}
3503
3504 The Bacula Enterprise Windows File Daemon for the community version
3505 7.4.0 now automatically supports files and
3506 directories that are encrypted on Windows filesystem.
3507
3508 \subsection{SSL Connections to MySQL}
3509
3510 There are five new Directives for the Catalog resource in the
3511 {\bf bacula-dir.conf} file that you can use to encrypt the
3512 communications between Bacula and MySQL for additional
3513 security.
3514
3515 \begin{description}
3516 \item [dbsslkey] takes a string variable that specifies the filename of an
3517 SSL key file.
3518 \item [dbsslcert] takes a string variable that specifies the filename of an
3519 SSL certificate file.
3520 \item [dbsslca] takes a string variable that specifies the filename of a
3521 SSL CA (certificate authority) certificate.
3522 \item [dbsslcipher] takes a string variable that specifies the cipher
3523 to be used.
3524 \end{description}
3525
3526 \subsection{Max Virtual Full Interval}
3527 This is a new Job resource directive that specifies the time in seconds
3528 that is a maximum time between Virtual Full jobs. It is much like the
3529 Max Full Interval directive but applies to Virtual Full jobs rather
3530 that Full jobs.
3531
3532 \subsection{New List Volumes Output}
3533 The {\bf list} and {\bf llist} commands have been modified so that when
3534 listing Volumes a new pseudo field {\bf expiresin} will be printed. This
3535 field is the number of seconds in which the retention period will expire.
3536 If the retention period has already expired the value will be zero. Any
3537 non-zero value means that the retention period is still in effect.
3538
3539 An example with many columns shorted for display purpose is:
3540
3541 \begin{bVerbatim}
3542 *list volumes
3543 Pool: Default
3544 *list volumes
3545 Pool: Default
3546 +----+---------------+-----------+---------+-------------+-----------+
3547 | id | volumename | volstatus | enabled | volbytes | expiresin |
3548 +----+---------------+-----------+---------+-------------+-----------+
3549 | 1 | TestVolume001 | Full | 1 | 249,940,696 | 0 |
3550 | 2 | TestVolume002 | Full | 1 | 249,961,704 | 1 |
3551 | 3 | TestVolume003 | Full | 1 | 249,961,704 | 2 |
3552 | 4 | TestVolume004 | Append | 1 | 127,367,896 | 3 |
3553 +----+---------------+-----------+---------+-------------+-----------+
3554 \end{bVerbatim}
3555
3556 %%
3557 %%
3558 \chapter{New Features in 7.2.0}
3559 This chapter presents the new features that have been added to
3560 the various versions of Bacula.
3561
3562 \section{New Features in 7.2.0}
3563
3564 \subsection{New Job Edit Codes \%E \%R}
3565 In various places such as RunScripts, you have now access to \%E to get the
3566 number of non-fatal errors for the current Job and \%R to get the number of
3567 bytes read from disk or from the network during a job.
3568
3569 \subsection{Enable/Disable commands}
3570 The \textbf{bconsole} \textbf{enable} and \textbf{disable} commands have
3571 been extended from enabling/disabling Jobs to include Clients, Schedule,
3572 and Storage devices. Examples:
3573
3574 \begin{bVerbatim}
3575 disable Job=NightlyBackup Client=Windows-fd
3576 \end{bVerbatim}
3577
3578 will disable the Job named \textbf{NightlyBackup} as well as the
3579 client named \textbf{Windows-fd}.
3580
3581 \begin{bVerbatim}
3582 disable Storage=LTO-changer Drive=1
3583 \end{bVerbatim}
3584
3585 will disable the first drive in the autochanger named \textbf{LTO-changer}.
3586
3587 Please note that doing a \textbf{reload} command will set any values
3588 changed by the enable/disable commands back to the values in the
3589 bacula-dir.conf file.
3590
3591 The Client and Schedule resources in the bacula-dir.conf file now permit
3592 the directive Enable = yes or Enable = no.
3593
3594
3595 \section{Bacula 7.2}
3596
3597 \subsection{Snapshot Management}
3598
3599 Bacula 7.2 is now able to handle Snapshots on Linux/Unix
3600 systems. Snapshots can be automatically created and used to backup files. It is
3601 also possible to manage Snapshots from Bacula's \texttt{bconsole} tool through a
3602 unique interface.
3603
3604 \subsubsection{Snapshot Backends}
3605
3606 The following Snapshot backends are supported:
3607
3608 \begin{itemize}
3609 \item BTRFS
3610 \item ZFS
3611 \item LVM\footnote{Some restrictions described in \vref{LVMBackend} applies to
3612 the LVM backend}
3613 \end{itemize}
3614
3615 By default, Snapshots are mounted (or directly available) under
3616 \textbf{.snapshots} directory on the root filesystem. (On ZFS, the default
3617 is \textbf{.zfs/snapshots}).
3618
3619 \smallskip{}
3620
3621 The Snapshot backend program is called \textbf{bsnapshot} and is available in
3622 the \textbf{bacula-enterprise-snapshot} package. In order to use the Snapshot
3623 Management feature, the package must be installed on the Client.
3624
3625 \smallskip{}
3626 \label{bsnapshotconf}
3627 The \textbf{bsnapshot} program can be configured using
3628 \texttt{/opt/bacula/etc/bsnapshot.conf} file. The following parameters can
3629 be adjusted in the configuration file:
3630
3631 \begin{itemize}
3632 \item \texttt{trace=<file>} Specify a trace file
3633 \item \texttt{debug=<num>} Specify a debug level
3634 \item \texttt{sudo=<yes/no>} Use sudo to run commands
3635 \item \texttt{disabled=<yes/no>} Disable snapshot support
3636 \item \texttt{retry=<num>} Configure the number of retries for some operations
3637 \item \texttt{snapshot\_dir=<dirname>} Use a custom name for the Snapshot directory. (\textbf{.SNAPSHOT}, \textbf{.snapdir}, etc...)
3638 \item \texttt{lvm\_snapshot\_size=<lvpath:size>} Specify a custom snapshot size for a given LVM volume
3639 \end{itemize}
3640
3641 \begin{bVerbatim}
3642 # cat /opt/bacula/etc/bsnapshot.conf
3643 trace=/tmp/snap.log
3644 debug=10
3645 lvm_snapshot_size=/dev/ubuntu-vg/root:5%
3646 \end{bVerbatim}
3647
3648
3649 \subsubsection{Application Quiescing}
3650
3651 When using Snapshots, it is very important to quiesce applications that are
3652 running on the system. The simplest way to quiesce an application is to stop
3653 it. Usually, taking the Snapshot is very fast, and the downtime is only about a
3654 couple of seconds. If downtime is not possible and/or the application provides
3655 a way to quiesce, a more advanced script can be used. An example is
3656 described on \vref{SnapRunScriptExample}.
3657
3658 \subsubsection{New Director Directives}
3659
3660 The use of the Snapshot Engine on the FileDaemon is determined by the
3661 new \textbf{Enable Snapshot} FileSet directive. The default is \textbf{no}.
3662
3663 \begin{bVerbatim}
3664 FileSet {
3665 Name = LinuxHome
3666
3667 Enable Snapshot = yes
3668
3669 Include {
3670 Options = { Compression = LZO }
3671 File = /home
3672 }
3673 }
3674 \end{bVerbatim}
3675
3676 By default, Snapshots are deleted from the Client at the end of the backup. To
3677 keep Snapshots on the Client and record them in the Catalog for a determined
3678 period, it is possible to use the \textbf{Snapshot Retention} directive in the
3679 Client or in the Job resource. The default value is 0 secconds. If, for a given Job,
3680 both Client and Job \textbf{Snapshot Retention} directives are set, the Job
3681 directive will be used.
3682
3683 \begin{bVerbatim}
3684 Client {
3685 Name = linux1
3686 ...
3687
3688 Snapshot Retention = 5 days
3689 }
3690 \end{bVerbatim}
3691
3692 To automatically prune Snapshots, it is possible to use the following RunScript
3693 command:
3694
3695 \begin{bVerbatim}
3696 Job {
3697 ...
3698 Client = linux1
3699 ...
3700 RunScript {
3701 RunsOnClient = no
3702 Console = "prune snapshot client=%c yes"
3703 RunsAfter = yes
3704 }
3705 }
3706 \end{bVerbatim}
3707
3708
3709 \smallskip{}
3710
3711 In RunScripts, the \texttt{AfterSnapshot} keyword for the \texttt{RunsWhen} directive will
3712 allow a command to be run just after the Snapshot creation. \texttt{AfterSnapshot} is a
3713 synonym for the \texttt{AfterVSS} keyword.
3714
3715 \label{SnapRunScriptExample}
3716 \begin{bVerbatim}
3717 Job {
3718 ...
3719 RunScript {
3720 Command = "/etc/init.d/mysql start"
3721 RunsWhen = AfterSnapshot
3722 RunsOnClient = yes
3723 }
3724 RunScript {
3725 Command = "/etc/init.d/mysql stop"
3726 RunsWhen = Before
3727 RunsOnClient = yes
3728 }
3729 }
3730 \end{bVerbatim}
3731
3732 \subsubsection{Job Output Information}
3733
3734 Information about Snapshots are displayed in the Job output. The list of all
3735 devices used by the Snapshot Engine is displayed, and the Job summary
3736 indicates if Snapshots were available.
3737
3738 \begin{bVerbatim}
3739 JobId 3: Create Snapshot of /home/build
3740 JobId 3: Create Snapshot of /home/build/subvol
3741 JobId 3: Delete snapshot of /home/build
3742 JobId 3: Delete snapshot of /home/build/subvol
3743 ...
3744 JobId 3: Bacula 127.0.0.1-dir 7.2.0 (23Jul15):
3745 Build OS: x86_64-unknown-linux-gnu archlinux
3746 JobId: 3
3747 Job: Incremental.2015-02-24_11.20.27_08
3748 Backup Level: Full
3749 ...
3750 Snapshot/VSS: yes
3751 ...
3752 Termination: Backup OK
3753 \end{bVerbatim}
3754
3755
3756 \subsubsection{New ``snapshot'' Bconsole Commands}
3757
3758 The new \textbf{snapshot} command will display by default the following menu:
3759 \begin{bVerbatim}
3760 *snapshot
3761 Snapshot choice:
3762 1: List snapshots in Catalog
3763 2: List snapshots on Client
3764 3: Prune snapshots
3765 4: Delete snapshot
3766 5: Update snapshot parameters
3767 6: Update catalog with Client snapshots
3768 7: Done
3769 Select action to perform on Snapshot Engine (1-7):
3770 \end{bVerbatim}
3771
3772 The \textbf{snapshot} command can also have the following parameters:
3773 \begin{bVerbatim}
3774 [client=<client-name> | job=<job-name> | jobid=<jobid>]
3775 [delete | list | listclient | prune | sync | update]
3776 \end{bVerbatim}
3777
3778 It is also possible to use traditional \texttt{list}, \texttt{llist},
3779 \texttt{update}, \texttt{prune} or \texttt{delete} commands on Snapshots.
3780
3781 \begin{bVerbatim}
3782 *llist snapshot jobid=5
3783 snapshotid: 1
3784 name: NightlySave.2015-02-24_12.01.00_04
3785 createdate: 2015-02-24 12:01:03
3786 client: 127.0.0.1-fd
3787 fileset: Full Set
3788 jobid: 5
3789 volume: /home/.snapshots/NightlySave.2015-02-24_12.01.00_04
3790 device: /home/btrfs
3791 type: btrfs
3792 retention: 30
3793 comment:
3794 \end{bVerbatim}
3795
3796 \begin{bVerbatim}
3797 * snapshot listclient
3798 Automatically selected Client: 127.0.0.1-fd
3799 Connecting to Client 127.0.0.1-fd at 127.0.0.1:8102
3800 Snapshot NightlySave.2015-02-24_12.01.00_04:
3801 Volume: /home/.snapshots/NightlySave.2015-02-24_12.01.00_04
3802 Device: /home
3803 CreateDate: 2015-02-24 12:01:03
3804 Type: btrfs
3805 Status: OK
3806 Error:
3807 \end{bVerbatim}
3808
3809 \smallskip{}
3810
3811 With the \textsl{Update catalog with Client snapshots} option (or
3812 \textbf{snapshot sync}), the Director contacts the FileDaemon, lists snapshots
3813 of the system and creates catalog records of the Snapshots.
3814
3815 \begin{bVerbatim}
3816 *snapshot sync
3817 Automatically selected Client: 127.0.0.1-fd
3818 Connecting to Client 127.0.0.1-fd at 127.0.0.1:8102
3819 Snapshot NightlySave.2015-02-24_12.35.47_06:
3820 Volume: /home/.snapshots/NightlySave.2015-02-24_12.35.47_06
3821 Device: /home
3822 CreateDate: 2015-02-24 12:35:47
3823 Type: btrfs
3824 Status: OK
3825 Error:
3826 Snapshot added in Catalog
3827
3828 *llist snapshot
3829 snapshotid: 13
3830 name: NightlySave.2015-02-24_12.35.47_06
3831 createdate: 2015-02-24 12:35:47
3832 client: 127.0.0.1-fd
3833 fileset:
3834 jobid: 0
3835 volume: /home/.snapshots/NightlySave.2015-02-24_12.35.47_06
3836 device: /home
3837 type: btrfs
3838 retention: 0
3839 comment:
3840 \end{bVerbatim}
3841
3842 % list
3843 % llist
3844 % prune
3845 % delete
3846 % update snapshot
3847 % sync
3848
3849 \subsubsection{LVM Backend Restrictions}
3850 \label{LVMBackend}
3851
3852 LVM Snapshots are quite primitive compared to ZFS, BTRFS, NetApp and other
3853 systems. For example, it is not possible to use Snapshots if the Volume Group
3854 (VG) is full. The administrator must keep some free space in the VG
3855 to create Snapshots. The amount of free space required depends on the activity of the
3856 Logical Volume (LV). \textbf{bsnapshot} uses 10\% of the LV by
3857 default. This number can be configured per LV in the
3858 \textbf{bsnapshot.conf} file.
3859
3860 \begin{bVerbatim}
3861 [root@system1]# vgdisplay
3862 --- Volume group ---
3863 VG Name vg_ssd
3864 System ID
3865 Format lvm2
3866 ...
3867 VG Size 29,81 GiB
3868 PE Size 4,00 MiB
3869 Total PE 7632
3870 Alloc PE / Size 125 / 500,00 MiB
3871 Free PE / Size 7507 / 29,32 GiB
3872 ...
3873 \end{bVerbatim}
3874
3875 It is also not advisable to leave snapshots on the LVM backend. Having multiple
3876 snapshots of the same LV on LVM will slow down the system.
3877
3878 \subsubsection{Debug Options}
3879
3880 To get low level information about the Snapshot Engine, the debug tag ``snapshot''
3881 should be used in the \textbf{setdebug} command.
3882
3883 \begin{bVerbatim}
3884 * setdebug level=10 tags=snapshot client
3885 * setdebug level=10 tags=snapshot dir
3886 \end{bVerbatim}
3887
3888 \subsection{Minor Enhancements}
3889 \subsubsection{Storage Daemon Reports Disk Usage}
3890
3891 The \texttt{status storage} command now reports the space available on disk devices:
3892 \begin{bVerbatim}
3893 ...
3894 Device status:
3895
3896 Device file: "FileStorage" (/bacula/arch1) is not open.
3897 Available Space=5.762 GB
3898 ==
3899
3900 Device file: "FileStorage1" (/bacula/arch2) is not open.
3901 Available Space=5.862 GB
3902 \end{bVerbatim}
3903
3904 \subsection{Data Encryption Cipher Configuration}
3905 Bacula Enterprise version 8.0 and later now allows configuration of the data
3906 encryption cipher and the digest algorithm. Previously, the cipher was forced to AES 128,
3907 but it is now possible to choose between the following ciphers:
3908
3909 \begin{itemize}
3910 \item AES128 (default)
3911 \item AES192
3912 \item AES256
3913 \item blowfish
3914 \end{itemize}
3915
3916 The digest algorithm was set to SHA1 or SHA256 depending on the local OpenSSL
3917 options. We advise you to not modify the PkiDigest default setting. Please,
3918 refer to the OpenSSL documentation to understand the pros and cons regarding these options.
3919
3920 \begin{bVerbatim}
3921 FileDaemon {
3922 ...
3923 PkiCipher = AES256
3924 }
3925 \end{bVerbatim}
3926
3927 \subsubsection*{New Option Letter ``M'' for Accurate Directive in FileSet}
3928
3929 % waa - 20150317 - is 8.0.5 correct here?
3930 Added in version 8.0.5, the new ``M'' option letter for the Accurate directive
3931 in the FileSet Options block, which allows comparing the modification time and/or
3932 creation time against the last backup timestamp. This is in contrast to the
3933 existing options letters ``m'' and/or ``c'', mtime and ctime, which are checked
3934 against the stored catalog values, which can vary accross different machines
3935 when using the BaseJob feature.
3936
3937 The advantage of the new ``M'' option letter for Jobs that refer to BaseJobs is
3938 that it will instruct Bacula to backup files based on the last backup time, which
3939 is more useful because the mtime/ctime timestamps may differ on various Clients,
3940 causing files to be needlessly backed up.
3941
3942 \smallskip{}
3943
3944 \begin{bVerbatim}
3945 Job {
3946 Name = USR
3947 Level = Base
3948 FileSet = BaseFS
3949 ...
3950 }
3951
3952 Job {
3953 Name = Full
3954 FileSet = FullFS
3955 Base = USR
3956 ...
3957 }
3958
3959 FileSet {
3960 Name = BaseFS
3961 Include {
3962 Options {
3963 Signature = MD5
3964 }
3965 File = /usr
3966 }
3967 }
3968
3969 FileSet {
3970 Name = FullFS
3971 Include {
3972 Options {
3973 Accurate = Ms # check for mtime/ctime of last backup timestamp and Size
3974 Signature = MD5
3975 }
3976 File = /home
3977 File = /usr
3978 }
3979 }
3980 \end{bVerbatim}
3981
3982 \subsection{Read Only Storage Devices}
3983 This version of Bacula allows you to define a Storage deamon device
3984 to be read-only. If the {\bf Read Only} directive is specified and
3985 enabled, the drive can only be used for read operations.
3986 The {\bf Read Only} directive can be defined in any bacula-sd.conf
3987 Device resource, and is most useful for reserving one or more
3988 drives for restores. An example is:
3989
3990 \begin{bVerbatim}
3991 Read Only = yes
3992 \end{bVerbatim}
3993
3994 \subsection{New Resume Command}
3995 The new \texttt{resume} command does exactly the same thing as a
3996 {\bf restart} command, but for some users the
3997 name may be more logical because in general the
3998 {\bf restart} command is used to resume running
3999 a Job that was incomplete.
4000
4001 \subsection{New Prune ``Expired'' Volume Command}
4002 In Bacula Enterprise 6.4, it is now possible to prune all volumes
4003 (from a pool, or globally) that are ``expired''. This option can be
4004 scheduled after or before the backup of the catalog and can be
4005 combined with the \texttt{Truncate On Purge} option. The \texttt{prune expired volme} command may
4006 be used instead of the \texttt{manual\_prune.pl} script.
4007
4008 \begin{bVerbatim}
4009 * prune expired volume
4010
4011 * prune expired volume pool=FullPool
4012 \end{bVerbatim}
4013
4014 To schedule this option automatically, it can be added to the Catalog backup job
4015 definition.
4016
4017 \begin{bVerbatim}
4018 Job {
4019 Name = CatalogBackup
4020 ...
4021 RunScript {
4022 Console = "prune expired volume yes"
4023 RunsWhen = Before
4024 }
4025 }
4026 \end{bVerbatim}
4027
4028
4029 \subsection{New Job Edit Codes \%P \%C}
4030 In various places such as RunScripts, you have now access to \%P to get the
4031 current Bacula process ID (PID) and \%C to know if the current job is a
4032 cloned job.
4033
4034 \subsection{Enhanced Status and Error Messages}
4035 We have enhanced the Storage daemon status output to be more
4036 readable. This is important when there are a large number of
4037 devices. In addition to formatting changes, it also includes more
4038 details on which devices are reading and writing.
4039
4040 A number of error messages have been enhanced to have more specific
4041 data on what went wrong.
4042
4043 If a file changes size while being backed up the old and new size
4044 are reported.
4045
4046 \subsection{Miscellaneous New Features}
4047 \begin{itemize}
4048 \item Allow unlimited line lengths in .conf files (previously limited
4049 to 2000 characters).
4050
4051 \item Allow /dev/null in ChangerCommand to indicated a Virtual Autochanger.
4052
4053 \item Add a --fileprune option to the manual\_prune.pl script.
4054
4055 \item Add a -m option to make\_catalog\_backup.pl to do maintenance
4056 on the catalog.
4057
4058 \item Safer code that cleans up the working directory when starting
4059 the daemons. It limits what files can be deleted, hence enhances
4060 security.
4061
4062 \item Added a new .ls command in bconsole to permit browsing a client's
4063 filesystem.
4064
4065 \item Fixed a number of bugs, includes some obscure seg faults, and a
4066 race condition that occurred infrequently when running Copy, Migration,
4067 or Virtual Full backups.
4068
4069 \item Upgraded to a newer version of Qt4 for bat. All indications
4070 are that this will improve bat's stability on Windows machines.
4071
4072 \item The Windows installers now detect and refuse to install on
4073 an OS that does not match the 32/64 bit value of the installer.
4074 \end{itemize}
4075
4076 \subsection{FD Storage Address}
4077
4078 When the Director is behind a NAT, in a WAN area, to connect to
4079 % the FileDaemon or
4080 the StorageDaemon, the Director uses an ``external'' ip address,
4081 and the FileDaemon should use an ``internal'' IP address to contact the
4082 StorageDaemon.
4083
4084 The normal way to handle this situation is to use a canonical name such as
4085 ``storage-server'' that will be resolved on the Director side as the WAN
4086 address and on the Client side as the LAN address. This is now possible to
4087 configure this parameter using the new directive \texttt{FDStorageAddress} in
4088 the Storage or Client resource.
4089
4090
4091 %%\bimageH{BackupOverWan1}{Backup Over WAN}{figbs6:fdstorageaddress}
4092 % \label{fig:fdstorageaddress}
4093
4094 \begin{bVerbatim}
4095 Storage {
4096 Name = storage1
4097 Address = 65.1.1.1
4098 FD Storage Address = 10.0.0.1
4099 SD Port = 9103
4100 ...
4101 }
4102 \end{bVerbatim}
4103
4104 % # or in the Client resouce
4105 %
4106
4107 \begin{bVerbatim}
4108 Client {
4109 Name = client1
4110 Address = 65.1.1.2
4111 FD Storage Address = 10.0.0.1
4112 FD Port = 9102
4113 ...
4114 }
4115 \end{bVerbatim}
4116
4117 Note that using the Client \texttt{FDStorageAddress} directive will not allow
4118 to use multiple Storage Daemon, all Backup or Restore requests will be sent to
4119 the specified \texttt{FDStorageAddress}.
4120
4121 \subsection{Maximum Concurrent Read Jobs}
4122 This is a new directive that can be used in the {\bf bacula-dir.conf} file
4123 in the Storage resource. The main purpose is to limit the number
4124 of concurrent Copy, Migration, and VirtualFull jobs so that
4125 they don't monopolize all the Storage drives causing a deadlock situation
4126 where all the drives are allocated for reading but none remain for
4127 writing. This deadlock situation can occur when running multiple
4128 simultaneous Copy, Migration, and VirtualFull jobs.
4129
4130 \smallskip
4131 The default value is set to 0 (zero), which means there is no
4132 limit on the number of read jobs. Note, limiting the read jobs
4133 does not apply to Restore jobs, which are normally started by
4134 hand. A reasonable value for this directive is one half the number
4135 of drives that the Storage resource has rounded down. Doing so,
4136 will leave the same number of drives for writing and will generally
4137 avoid over committing drives and a deadlock.
4138
4139 \subsection{Incomplete Jobs}
4140 During a backup, if the Storage daemon experiences disconnection
4141 with the File daemon during backup (normally a comm line problem
4142 or possibly an FD failure), under conditions that the SD determines
4143 to be safe it will make the failed job as Incomplete rather than
4144 failed. This is done only if there is sufficient valid backup
4145 data that was written to the Volume. The advantage of an Incomplete
4146 job is that it can be restarted by the new bconsole {\bf restart}
4147 command from the point where it left off rather than from the
4148 beginning of the jobs as is the case with a cancel.
4149
4150 \subsection{The Stop Command}
4151 Bacula has been enhanced to provide a {\bf stop} command,
4152 very similar to the {\bf cancel} command with the main difference
4153 that the Job that is stopped is marked as Incomplete so that
4154 it can be restarted later by the {\bf restart} command where
4155 it left off (see below). The {\bf stop} command with no
4156 arguments, will like the cancel command, prompt you with the
4157 list of running jobs allowing you to select one, which might
4158 look like the following:
4159
4160 \begin{bVerbatim}
4161 *stop
4162 Select Job:
4163 1: JobId=3 Job=Incremental.2012-03-26_12.04.26_07
4164 2: JobId=4 Job=Incremental.2012-03-26_12.04.30_08
4165 3: JobId=5 Job=Incremental.2012-03-26_12.04.36_09
4166 Choose Job to stop (1-3): 2
4167 2001 Job "Incremental.2012-03-26_12.04.30_08" marked to be stopped.
4168 3000 JobId=4 Job="Incremental.2012-03-26_12.04.30_08" marked to be stopped.
4169 \end{bVerbatim}
4170
4171 \subsection{The Restart Command}
4172 The new {\bf Restart command} allows console users to restart
4173 a canceled, failed, or incomplete Job. For canceled and failed
4174 Jobs, the Job will restart from the beginning. For incomplete
4175 Jobs the Job will restart at the point that it was stopped either
4176 by a stop command or by some recoverable failure.
4177
4178 \smallskip
4179 If you enter the {\bf restart} command in bconsole, you will get the
4180 following prompts:
4181
4182 \begin{bVerbatim}
4183 *restart
4184 You have the following choices:
4185 1: Incomplete
4186 2: Canceled
4187 3: Failed
4188 4: All
4189 Select termination code: (1-4):
4190 \end{bVerbatim}
4191
4192 If you select the {\bf All} option, you may see something like:
4193
4194 \begin{bVerbatim}
4195 Select termination code: (1-4): 4
4196 +-------+-------------+---------------------+------+-------+----------+-----------+-----------+
4197 | jobid | name | starttime | type | level | jobfiles |
4198 jobbytes | jobstatus |
4199 +-------+-------------+---------------------+------+-------+----------+-----------+-----------+
4200 | 1 | Incremental | 2012-03-26 12:15:21 | B | F | 0 |
4201 0 | A |
4202 | 2 | Incremental | 2012-03-26 12:18:14 | B | F | 350 |
4203 4,013,397 | I |
4204 | 3 | Incremental | 2012-03-26 12:18:30 | B | F | 0 |
4205 0 | A |
4206 | 4 | Incremental | 2012-03-26 12:18:38 | B | F | 331 |
4207 3,548,058 | I |
4208 +-------+-------------+---------------------+------+-------+----------+-----------+-----------+
4209 Enter the JobId list to select:
4210 \end{bVerbatim}
4211
4212 Then you may enter one or more JobIds to be restarted, which may
4213 take the form of a list of JobIds separated by commas, and/or JobId
4214 ranges such as {\bf 1-4}, which indicates you want to restart JobIds
4215 1 through 4, inclusive.
4216
4217 \subsection{Job Bandwidth Limitation}
4218
4219 The new {\bf Job Bandwidth Limitation} directive may be added to the File
4220 daemon's and/or Director's configuration to limit the bandwidth used by a
4221 Job on a Client. It can be set in the File daemon's conf file for all Jobs
4222 run in that File daemon, or it can be set for each Job in the Director's
4223 conf file. The speed is always specified in bytes per second.
4224
4225 For example:
4226 \begin{bVerbatim}
4227 FileDaemon {
4228 Name = localhost-fd
4229 Working Directory = /some/path
4230 Pid Directory = /some/path
4231 ...
4232 Maximum Bandwidth Per Job = 5Mb/s
4233 }
4234 \end{bVerbatim}
4235
4236 The above example would cause any jobs running with the FileDaemon to not
4237 exceed 5 megabytes per second of throughput when sending data to the
4238 Storage Daemon. Note, the speed is always specified in bytes per second
4239 (not in bits per second), and the case (upper/lower) of the specification
4240 characters is ignored (i.e. 1MB/s = 1Mb/s).
4241
4242 You may specify the following speed parameter modifiers:
4243 k/s (1,000 bytes per second), kb/s (1,024 bytes per second),
4244 m/s (1,000,000 bytes per second), or mb/s (1,048,576 bytes per second).
4245
4246 For example:
4247 \begin{bVerbatim}
4248 Job {
4249 Name = locahost-data
4250 FileSet = FS_localhost
4251 Accurate = yes
4252 ...
4253 Maximum Bandwidth = 5Mb/s
4254 ...
4255 }
4256 \end{bVerbatim}
4257
4258 The above example would cause Job \texttt{localhost-data} to not exceed 5MB/s
4259 of throughput when sending data from the File daemon to the Storage daemon.
4260
4261 A new console command \texttt{setbandwidth} permits to set dynamically the
4262 maximum throughput of a running Job or for future jobs of a Client.
4263
4264 \begin{bVerbatim}
4265 * setbandwidth limit=1000 jobid=10
4266 \end{bVerbatim}
4267
4268 Please note that the value specified for the \texttt{limit} command
4269 line parameter is always in units of 1024 bytes (i.e. the number
4270 is multiplied by 1024 to give the number of bytes per second). As
4271 a consequence, the above limit of 1000 will be interpreted as a
4272 limit of 1000 * 1024 = 1,024,000 bytes per second.
4273
4274 \subsection{Always Backup a File}
4275
4276 When the Accurate mode is turned on, you can decide to always backup a file
4277 by using then new {\bf A} Accurate option in your FileSet. For example:
4278
4279 \begin{bVerbatim}
4280 Job {
4281 Name = ...
4282 FileSet = FS_Example
4283 Accurate = yes
4284 ...
4285 }
4286
4287 FileSet {
4288 Name = FS_Example
4289 Include {
4290 Options {
4291 Accurate = A
4292 }
4293 File = /file
4294 File = /file2
4295 }
4296 ...
4297 }
4298 \end{bVerbatim}
4299
4300 This project was funded by Bacula Systems based on an idea of James Harper and
4301 is available with the Bacula Enterprise Edition.
4302
4303 \subsection{Setting Accurate Mode at Runtime}
4304
4305 You are now able to specify the Accurate mode on the \texttt{run} command and
4306 in the Schedule resource.
4307
4308 \begin{bVerbatim}
4309 * run accurate=yes job=Test
4310 \end{bVerbatim}
4311
4312 \begin{bVerbatim}
4313 Schedule {
4314 Name = WeeklyCycle
4315 Run = Full 1st sun at 23:05
4316 Run = Differential accurate=yes 2nd-5th sun at 23:05
4317 Run = Incremental accurate=no mon-sat at 23:05
4318 }
4319 \end{bVerbatim}
4320
4321 It can allow you to save memory and and CPU resources on the catalog server in
4322 some cases.
4323
4324 \medskip
4325 These advanced tuning options are available with the Bacula Enterprise Edition.
4326
4327 % Common with community
4328 \subsection{Additions to RunScript variables}
4329 You can have access to JobBytes, JobFiles and Director name using \%b, \%F and \%D
4330 in your runscript command. The Client address is now available through \%h.
4331
4332 \begin{bVerbatim}
4333 RunAfterJob = "/bin/echo Job=%j JobBytes=%b JobFiles=%F ClientAddress=%h Dir=%D"
4334 \end{bVerbatim}
4335
4336 \subsection{LZO Compression}
4337
4338 LZO compression was added in the Unix File Daemon. From the user point of view,
4339 it works like the GZIP compression (just replace {\bf compression=GZIP} with
4340 {\bf compression=LZO}).
4341
4342 For example:
4343 \begin{bVerbatim}
4344 Include {
4345 Options { compression=LZO }
4346 File = /home
4347 File = /data
4348 }
4349 \end{bVerbatim}
4350
4351 LZO provides much faster compression and decompression speed but lower
4352 compression ratio than GZIP. It is a good option when you backup to disk. For
4353 tape, the built-in compression may be a better option.
4354
4355 LZO is a good alternative for GZIP1 when you don't want to slow down your
4356 backup. On a modern CPU it should be able to run almost as fast as:
4357
4358 \begin{itemize}
4359 \item your client can read data from disk. Unless you have very fast disks like
4360 SSD or large/fast RAID array.
4361 \item the data transfers between the file daemon and the storage daemon even on
4362 a 1Gb/s link.
4363 \end{itemize}
4364
4365 Note that bacula only use one compression level LZO1X-1.
4366
4367 \medskip
4368 The code for this feature was contributed by Laurent Papier.
4369
4370 \subsection{Purge Migration Job}
4371
4372 The new {\bf Purge Migration Job} directive may be added to the Migration
4373 Job definition in the Director's configuration file. When it is enabled
4374 the Job that was migrated during a migration will be purged at
4375 the end of the migration job.
4376
4377 For example:
4378 \begin{bVerbatim}
4379 Job {
4380 Name = "migrate-job"
4381 Type = Migrate
4382 Level = Full
4383 Client = localhost-fd
4384 FileSet = "Full Set"
4385 Messages = Standard
4386 Storage = DiskChanger
4387 Pool = Default
4388 Selection Type = Job
4389 Selection Pattern = ".*Save"
4390 ...
4391 Purge Migration Job = yes
4392 }
4393 \end{bVerbatim}
4394
4395 \medskip
4396
4397 This project was submitted by Dunlap Blake; testing and documentation was funded
4398 by Bacula Systems.
4399
4400 \subsection{Changes in the Pruning Algorithm}
4401
4402 We rewrote the job pruning algorithm in this version. Previously, in some
4403 users reported that the pruning process at the end of jobs was very long.
4404 It should not be longer the case. Now, Bacula won't prune automatically a
4405 Job if this particular Job is needed to restore data. Example:
4406
4407 \begin{bVerbatim}
4408 JobId: 1 Level: Full
4409 JobId: 2 Level: Incremental
4410 JobId: 3 Level: Incremental
4411 JobId: 4 Level: Differential
4412 .. Other incrementals up to now
4413 \end{bVerbatim}
4414
4415 In this example, if the Job Retention defined in the Pool or in the Client
4416 resource causes that Jobs with Jobid in 1,2,3,4 can be pruned, Bacula will
4417 detect that JobId 1 and 4 are essential to restore data at the current state
4418 and will prune only JobId 2 and 3.
4419
4420 \texttt{Important}, this change affect only the automatic pruning step
4421 after a Job and the \texttt{prune jobs} Bconsole command. If a volume
4422 expires after the \texttt{VolumeRetention} period, important jobs can be
4423 pruned.
4424
4425 \subsection{Ability to Verify any specified Job}
4426 You now have the ability to tell Bacula which Job should verify instead of
4427 automatically verify just the last one.
4428
4429 This feature can be used with VolumeToCatalog, DiskToCatalog and Catalog level.
4430
4431 To verify a given job, just specify the Job jobid in argument when starting the
4432 job.
4433 \begin{bVerbatim}
4434 *run job=VerifyVolume jobid=1 level=VolumeToCatalog
4435 Run Verify job
4436 JobName: VerifyVolume
4437 Level: VolumeToCatalog
4438 Client: 127.0.0.1-fd
4439 FileSet: Full Set
4440 Pool: Default (From Job resource)
4441 Storage: File (From Job resource)
4442 Verify Job: VerifyVol.2010-09-08_14.17.17_03
4443 Verify List: /tmp/regress/working/VerifyVol.bsr
4444 When: 2010-09-08 14:17:31
4445 Priority: 10
4446 OK to run? (yes/mod/no):
4447 \end{bVerbatim}
4448
4449
4450
4451 \chapter{New Features in 7.0.0}
4452 This chapter presents the new features that have been added to
4453 the various versions of Bacula.
4454
4455 \section{New Features in 7.0.0}
4456
4457 \subsection{Storage daemon to Storage daemon}
4458 Bacula version 7.0 permits SD to SD transfer of Copy and Migration
4459 Jobs. This permits what is commonly referred to as replication or
4460 off-site transfer of Bacula backups. It occurs automatically, if
4461 the source SD and destination SD of a Copy or Migration job are
4462 different. The following picture shows how this works.
4463
4464 \includegraphics[width=0.8\linewidth]{sd-to-sd}
4465
4466 \subsection{SD Calls Client}
4467 If the {\bf SD Calls Client} directive is set to true in a Client resource
4468 any Backup, Restore, Verify, Copy, or Migration Job where the client
4469 is involved, the client will wait for the Storage daemon to contact it.
4470 By default this directive is set to false, and the Client will call
4471 the Storage daemon. This directive can be useful if your Storage daemon
4472 is behind a firewall that permits outgoing connections but not incoming
4473 one. The following picture shows the communications connection paths in
4474 both cases.
4475
4476 \includegraphics[width=0.8\linewidth]{sd-calls-client}
4477
4478 \subsection{Next Pool}
4479 In previous versions of Bacula the Next Pool directive could be
4480 specified in the Pool resource for use with Migration and Copy Jobs.
4481 The Next Pool concept has been
4482 extended in Bacula version 7.0.0 to allow you to specify the
4483 Next Pool directive in the Job resource as well. If specified in
4484 the Job resource, it will override any value specified in the Pool
4485 resource.
4486
4487 In addition to being permitted in the Job resource, the
4488 {\bf nextpool=xxx} specification can be specified as a run
4489 override in the {\bf run} directive of a Schedule resource.
4490 Any {\bf nextpool} specification in a {\bf run}
4491 directive will override any other specification in either
4492 the Job or the Pool.
4493
4494 In general, more information is displayed in the Job log
4495 on exactly which Next Pool specification is ultimately used.
4496
4497 \subsection{status storage}
4498 The bconsole {\bf status storage} has been modified to attempt to eliminate
4499 duplicate storage resources and only show one that references any given
4500 storage daemon. This might be confusing at first, but tends to make a
4501 much more compact list of storage resource from which to select if there
4502 are multiple storage devices in the same storage daemon.
4503
4504 If you want the old behavior (always display all storage resources) simply
4505 add the keyword {\bf select} to the command -- i.e. use
4506 {\bf status select storage}.
4507
4508
4509
4510
4511
4512 \subsection{status schedule}
4513 A new status command option called {\bf scheduled} has been implemented
4514 in bconsole. By default it will display 20 lines of the next scheduled
4515 jobs. For example, with the default bacula-dir.conf configuration file,
4516 a bconsole command {\bf status scheduled} produces:
4517
4518 \begin{bVerbatim}
4519 Scheduled Jobs:
4520 Level Type Pri Scheduled Job Name Schedule
4521 ======================================================================
4522 Differential Backup 10 Sun 30-Mar 23:05 BackupClient1 WeeklyCycle
4523 Incremental Backup 10 Mon 24-Mar 23:05 BackupClient1 WeeklyCycle
4524 Incremental Backup 10 Tue 25-Mar 23:05 BackupClient1 WeeklyCycle
4525 ...
4526 Full Backup 11 Mon 24-Mar 23:10 BackupCatalog WeeklyCycleAfterBackup
4527 Full Backup 11 Wed 26-Mar 23:10 BackupCatalog WeeklyCycleAfterBackup
4528 ...
4529 ====
4530 \end{bVerbatim}
4531
4532 Note, the output is listed by the Jobs found, and is not sorted
4533 chronologically.
4534
4535 \smallskip
4536 This command has a number of options, most of which act as filters:
4537 \begin{itemize}
4538 \item {\bf days=nn} This specifies the number of days to list. The default is
4539 10 but can be set from 0 to 500.
4540 \item {\bf limit=nn} This specifies the limit to the number of lines to print.
4541 The default is 100 but can be any number in the range 0 to 2000.
4542 \item {\bf time="YYYY-MM-DD HH:MM:SS"} Sets the start time for listing the
4543 scheduled jobs. The default is to use the current time. Note, the
4544 time value must be specified inside double quotes and must be in
4545 the exact form shown above.
4546 \item {\bf schedule=schedule-name} This option restricts the output to
4547 the named schedule.
4548 \item {\bf job=job-name} This option restricts the output to the specified
4549 Job name.
4550 \end{itemize}
4551
4552 \subsection{Data Encryption Cipher Configuration}
4553 Bacula version 7.0 and later now allows to configure the data
4554 encryption cipher and the digest algorithm. The cipher was forced to AES
4555 128, and it is now possible to choose between the following ciphers:
4556
4557 \begin{itemize}
4558 \item AES128 (default)
4559 \item AES192
4560 \item AES256
4561 \item blowfish
4562 \end{itemize}
4563
4564 The digest algorithm was set to SHA1 or SHA256 depending on the local
4565 OpenSSL
4566 options. We advise you to not modify the PkiDigest default setting. Please,
4567 refer to OpenSSL documentation to know about pro and cons on these options.
4568
4569 \begin{bVerbatim}
4570 FileDaemon {
4571 ...
4572 PkiCipher = AES256
4573 }
4574 \end{bVerbatim}
4575
4576 \subsection{New Truncate Command}
4577 We have added a new truncate command to bconsole, which
4578 will truncate a Volume if the Volume is purged and if
4579 the Volume is also marked {\bf Action On Purge = Truncate}.
4580 This feature was originally added in Bacula version 5.0.1,
4581 but the mechanism for actually doing the truncate required
4582 the user to enter a command such as:
4583
4584 \begin{bVerbatim}
4585 purge volume action=truncate storage=File pool=Default
4586 \end{bVerbatim}
4587
4588 The above command is now simplified to be:
4589
4590 \begin{bVerbatim}
4591 truncate storage=File pool=Default
4592 \end{bVerbatim}
4593
4594 \subsection{Migration/Copy/VirtualFull Performance Enhancements}
4595 The Bacula Storage daemon now permits multiple jobs to simultaneously read
4596 the same disk Volume, which gives substantial performance enhancements when
4597 running Migration, Copy, or VirtualFull jobs that read disk Volumes. Our
4598 testing shows that when running multiple simultaneous jobs, the jobs can
4599 finish up to ten times faster with this version of Bacula. This is
4600 built-in to the Storage daemon, so it happens automatically and
4601 transparently.
4602
4603 \subsection{VirtualFull Backup Consolidation Enhancements}
4604 By default Bacula selects jobs automatically for a VirtualFull,
4605 however, you may want to create the Virtual backup based on a
4606 particular backup (point in time) that exists.
4607
4608 For example, if you have the following backup Jobs in your catalog:
4609 \begin{bVerbatim}
4610 +-------+---------+-------+----------+----------+-----------+
4611 | JobId | Name | Level | JobFiles | JobBytes | JobStatus |
4612 +-------+---------+-------+----------+----------+-----------+
4613 | 1 | Vbackup | F | 1754 | 50118554 | T |
4614 | 2 | Vbackup | I | 1 | 4 | T |
4615 | 3 | Vbackup | I | 1 | 4 | T |
4616 | 4 | Vbackup | D | 2 | 8 | T |
4617 | 5 | Vbackup | I | 1 | 6 | T |
4618 | 6 | Vbackup | I | 10 | 60 | T |
4619 | 7 | Vbackup | I | 11 | 65 | T |
4620 | 8 | Save | F | 1758 | 50118564 | T |
4621 +-------+---------+-------+----------+----------+-----------+
4622 \end{bVerbatim}
4623
4624 and you want to consolidate only the first 3 jobs and create a
4625 virtual backup equivalent to Job 1 + Job 2 + Job 3, you will use
4626 \texttt{jobid=3} in the \texttt{run} command, then Bacula will select the
4627 previous Full backup, the previous Differential (if any) and all subsequent
4628 Incremental jobs.
4629
4630 \begin{bVerbatim}
4631 run job=Vbackup jobid=3 level=VirtualFull
4632 \end{bVerbatim}
4633
4634 If you want to consolidate a specific job list, you must specify the exact
4635 list of jobs to merge in the run command line. For example, to consolidate
4636 the last Differential and all subsequent Incremental, you will use
4637 \texttt{jobid=4,5,6,7} or \texttt{jobid=4-7} on the run command line. As one
4638 of the Job in the list is a Differential backup, Bacula will set the new job
4639 level to Differential. If the list is composed only with Incremental jobs,
4640 the new job will have a level set to Incremental.
4641
4642 \begin{bVerbatim}
4643 run job=Vbackup jobid=4-7 level=VirtualFull
4644 \end{bVerbatim}
4645
4646 When using this feature, Bacula will automatically discard jobs that are
4647 not related to the current Job. For example, specifying
4648 \texttt{jobid=7,8}, Bacula will discard JobId 8 because it is not
4649 part of the same backup Job.
4650
4651 We do not recommend it, but really want to consolidate jobs that have
4652 different names (so probably different clients, filesets, etc...), you must
4653 use \texttt{alljobid=} keyword instead of \texttt{jobid=}.
4654
4655 \begin{bVerbatim}
4656 run job=Vbackup alljobid=1-3,6-8 level=VirtualFull
4657 \end{bVerbatim}
4658
4659
4660 \subsection{FD Storage Address}
4661
4662 When the Director is behind a NAT, in a WAN area, to connect to
4663 % the FileDaemon or
4664 the StorageDaemon, the Director uses an ``external'' ip address,
4665 and the FileDaemon should use an ``internal'' IP address to contact the
4666 StorageDaemon.
4667
4668 The normal way to handle this situation is to use a canonical name such as
4669 ``storage-server'' that will be resolved on the Director side as the WAN
4670 address and on the Client side as the LAN address. This is now possible to
4671 configure this parameter using the new directive \texttt{FDStorageAddress} in
4672 the Storage or Client resource.
4673
4674
4675 \includegraphics[width=0.8\linewidth]{BackupOverWan1}
4676 \label{fig:fdstorageaddress}
4677
4678 \begin{bVerbatim}
4679 Storage {
4680 Name = storage1
4681 Address = 65.1.1.1
4682 FD Storage Address = 10.0.0.1
4683 SD Port = 9103
4684 ...
4685 }
4686 \end{bVerbatim}
4687
4688 % # or in the Client resouce
4689 %
4690
4691 \begin{bVerbatim}
4692 Client {
4693 Name = client1
4694 Address = 65.1.1.2
4695 FD Storage Address = 10.0.0.1
4696 FD Port = 9102
4697 ...
4698 }
4699 \end{bVerbatim}
4700
4701 Note that using the Client \texttt{FDStorageAddress} directive will not allow
4702 to use multiple Storage Daemon, all Backup or Restore requests will be sent to
4703 the specified \texttt{FDStorageAddress}.
4704
4705 \subsection{Job Bandwidth Limitation}
4706
4707 The new {\bf Job Bandwidth Limitation} directive may be added to the File
4708 daemon's and/or Director's configuration to limit the bandwidth used by a
4709 Job on a Client. It can be set in the File daemon's conf file for all Jobs
4710 run in that File daemon, or it can be set for each Job in the Director's
4711 conf file. The speed is always specified in bytes per second.
4712
4713 For example:
4714 \begin{bVerbatim}
4715 FileDaemon {
4716 Name = localhost-fd
4717 Working Directory = /some/path
4718 Pid Directory = /some/path
4719 ...
4720 Maximum Bandwidth Per Job = 5Mb/s
4721 }
4722 \end{bVerbatim}
4723
4724 The above example would cause any jobs running with the FileDaemon to not
4725 exceed 5 megabytes per second of throughput when sending data to the
4726 Storage Daemon. Note, the speed is always specified in bytes per second
4727 (not in bits per second), and the case (upper/lower) of the specification
4728 characters is ignored (i.e. 1MB/s = 1Mb/s).
4729
4730 You may specify the following speed parameter modifiers:
4731 k/s (1,000 bytes per second), kb/s (1,024 bytes per second),
4732 m/s (1,000,000 bytes per second), or mb/s (1,048,576 bytes per second).
4733
4734 For example:
4735 \begin{bVerbatim}
4736 Job {
4737 Name = locahost-data
4738 FileSet = FS_localhost
4739 Accurate = yes
4740 ...
4741 Maximum Bandwidth = 5Mb/s
4742 ...
4743 }
4744 \end{bVerbatim}
4745
4746 The above example would cause Job \texttt{localhost-data} to not exceed 5MB/s
4747 of throughput when sending data from the File daemon to the Storage daemon.
4748
4749 A new console command \texttt{setbandwidth} permits to set dynamically the
4750 maximum throughput of a running Job or for future jobs of a Client.
4751
4752 \begin{bVerbatim}
4753 * setbandwidth limit=1000 jobid=10
4754 \end{bVerbatim}
4755
4756 Please note that the value specified for the \texttt{limit} command
4757 line parameter is always in units of 1024 bytes (i.e. the number
4758 is multiplied by 1024 to give the number of bytes per second). As
4759 a consequence, the above limit of 1000 will be interpreted as a
4760 limit of 1000 * 1024 = 1,024,000 bytes per second.
4761
4762 \medskip
4763 This project was funded by Bacula Systems.
4764
4765
4766 \subsection{Maximum Concurrent Read Jobs}
4767 This is a new directive that can be used in the {\bf bacula-dir.conf} file
4768 in the Storage resource. The main purpose is to limit the number
4769 of concurrent Copy, Migration, and VirtualFull jobs so that
4770 they don't monopolize all the Storage drives causing a deadlock situation
4771 where all the drives are allocated for reading but none remain for
4772 writing. This deadlock situation can occur when running multiple
4773 simultaneous Copy, Migration, and VirtualFull jobs.
4774
4775 \smallskip
4776 The default value is set to 0 (zero), which means there is no
4777 limit on the number of read jobs. Note, limiting the read jobs
4778 does not apply to Restore jobs, which are normally started by
4779 hand. A reasonable value for this directive is one half the number
4780 of drives that the Storage resource has rounded down. Doing so,
4781 will leave the same number of drives for writing and will generally
4782 avoid over committing drives and a deadlock.
4783
4784
4785 \subsection{Director job Codes in Message Resource Commands}
4786 Before submitting the specified mail command to the operating system, Bacula
4787 performs character substitution like in Runscript commands. Bacula will now
4788 perform also specific Director character substitution.
4789
4790 \smallskip{}
4791 The code for this feature was contributed by Bastian Friedrich.
4792
4793 \subsection{Additions to RunScript variables}
4794 The following variables are now available in runscripts:
4795 \begin{itemize}
4796 \item current PID using \%P
4797 \item if the job is a clone job using \%C
4798 \end{itemize}
4799
4800 \begin{bVerbatim}
4801 RunAfterJob = "/bin/echo Pid=%P isCloned=%C"
4802
4803 \end{bVerbatim}
4804
4805 \subsection{Read Only Storage Devices}
4806 This version of Bacula permits defining a Storage daemon device
4807 to be read-only. That is if the {\bf ReadOnly} directive is specified and
4808 enabled, the drive can only be used for read operations.
4809 The the {\bf ReadOnly} directive can be defined in any bacula-sd.conf
4810 Device resource, and is most useful to reserve one or more
4811 drives for restores. An example is:
4812
4813 \begin{bVerbatim}
4814 Read Only = yes
4815 \end{bVerbatim}
4816
4817 \subsection{New Prune ``Expired'' Volume Command}
4818 It is now possible to prune all volumes
4819 (from a pool, or globally) that are ``expired''. This option can be
4820 scheduled after or before the backup of the Catalog and can be
4821 combined with the Truncate On Purge option. The Expired Prune option can
4822 be used instead of the \texttt{manual\_prune.pl} script.
4823
4824 \begin{bVerbatim}
4825 * prune expired volumes
4826
4827 * prune expired volumes pool=FullPool
4828 \end{bVerbatim}
4829
4830 To schedule this option automatically, it can be added to the BackupCatalog job
4831 definition.
4832
4833 \begin{bVerbatim}
4834 Job {
4835 Name = CatalogBackup
4836 ...
4837 RunScript {
4838 Console = "prune expired volume yes"
4839 RunsWhen = Before
4840 }
4841 }
4842 \end{bVerbatim}
4843
4844 \subsection{Hardlink Performance Enhancements}
4845 If you use a program such as Cyrus IMAP that creates very large numbers
4846 of hardlinks, the time to build the interactive restore tree can be
4847 excessively long. This version of Bacula has a new feature that
4848 automatically keeps the hardlinks associated with the restore tree
4849 in memory, which consumes a bit more memory but vastly speeds up
4850 building the tree. If the memory usage is too big for your system, you
4851 can reduce the amount of memory used during the restore command by
4852 adding the option {\bf optimizespeed=false} on the bconsole run
4853 command line.
4854
4855 This feature was developed by Josip Almasi, and enhanced to be runtime
4856 dynamic by Kern Sibbald.
4857
4858 \subsection{DisableCommand Directive}
4859 There is a new Directive named {\bf Disable Command} that
4860 can be put in the File daemon Client or Director resource.
4861 If it is in the Client, it applies globally, otherwise the
4862 directive applies only to the Director in which it is found.
4863 The Disable Command adds security to your File daemon by
4864 disabling certain commands. The commands that can be
4865 disabled are:
4866
4867 \begin{bVerbatim}
4868 backup
4869 cancel
4870 setdebug=
4871 setbandwidth=
4872 estimate
4873 fileset
4874 JobId=
4875 level =
4876 restore
4877 endrestore
4878 session
4879 status
4880 .status
4881 storage
4882 verify
4883 RunBeforeNow
4884 RunBeforeJob
4885 RunAfterJob
4886 Run
4887 accurate
4888 \end{bVerbatim}
4889
4890 On or more of these command keywords can be placed in quotes and separated
4891 by spaces on the Disable Command directive line. Note: the commands must
4892 be written exactly as they appear above.
4893
4894 \subsection{Multiple Console Directors}
4895 Support for multiple bconsole and bat Directors in the bconsole.conf and
4896 bat.conf files has been implemented and/or improved.
4897
4898 \subsection{Restricted Consoles}
4899 Better support for Restricted consoles has been implement for bconsole and
4900 bat.
4901
4902 \subsection{Configuration Files}
4903 In previous versions of Bacula the configuration files for each component
4904 were limited to a maximum of 499 bytes per configuration file line. This
4905 version of Bacula permits unlimited input line lengths. This can be
4906 especially useful for specifying more complicated Migration/Copy SQL
4907 statements and in creating long restricted console ACL lists.
4908
4909 \subsection{Maximum Spawned Jobs}
4910 The Job resource now permits specifying a number of {\bf Maximum Spawned
4911 Jobs}. The default is 600. This directive can be useful if you have
4912 big hardware and you do a lot of Migration/Copy jobs which start
4913 at the same time. In prior versions of Bacula, Migration/Copy
4914 was limited to spawning a maximum of 100 jobs at a time.
4915
4916 \subsection{Progress Meter}
4917 The new File daemon has been enhanced to send its progress (files
4918 processed and bytes written) to the Director every 30 seconds. These
4919 figures can then be displayed with a bconsole {\bf status dir}
4920 command.
4921
4922 \subsection{Scheduling a 6th Week}
4923 Prior version of Bacula permits specifying 1st through 5th week of
4924 a month (first through fifth) as a keyword on the {\bf run}
4925 directive of a Schedule resource. This version of Bacula also permits
4926 specifying the 6th week of a month with the keyword {\bf sixth} or
4927 {\bf 6th}.
4928
4929 \subsection{Scheduling the Last Day of a Month}
4930 This version of Bacula now permits specifying the {\bf lastday}
4931 keyword in the {\bf run} directive of a Schedule resource.
4932 If {\bf lastday} is specified, it will apply only to those months
4933 specified on the {\bf run} directive. Note: by default all months
4934 are specified.
4935
4936 \subsection{Improvements to Cancel and Restart bconsole Commands}
4937 The Restart bconsole command now allow selection of either
4938 canceled or failed jobs to be restarted. In addition both the
4939 {\bf cancel} and {\bf restart} bconsole commands permit entering
4940 a number of JobIds separated by commas or a range of JobIds indicated
4941 by a dash between the begin and end range (e.g. 3-10). Finally the
4942 two commands also allow one to enter the special keyword {\bf all}
4943 to select all the appropriate Jobs.
4944
4945 \subsection{bconsole Performance Improvements}
4946 In previous versions of Bacula certain bconsole commands could wait a long
4947 time due to catalog lock contention. This was especially noticeable
4948 when a large number of jobs were running and putting their attributes
4949 into the catalog. This version uses a separate catalog connection that
4950 should significantly enhance performance.
4951
4952 \subsection{New .bvfs\_decode\_lstat Command}
4953 There is a new bconsole command, which is
4954 {\bf .bvfs\_decode\_lstat} it requires one argument, which
4955 is {\bf lstat="lstat value to decode"}. An example command
4956 in bconsole and the output might be:
4957
4958 \small
4959 \begin{bVerbatim}
4960 .bvfs_decode_lstat lstat="A A EHt B A A A JP BAA B BTL/A7 BTL/A7 BTL/A7 A A C"
4961
4962 st_nlink=1
4963 st_mode=16877
4964 st_uid=0
4965 st_gid=0
4966 st_size=591
4967 st_blocks=1
4968 st_ino=0
4969 st_ctime=1395650619
4970 st_mtime=1395650619
4971 st_mtime=1395650619
4972 st_dev=0
4973 LinkFI=0
4974 \end{bVerbatim}
4975 \normalsize
4976
4977
4978 \subsection*{New Debug Options}
4979
4980 In Bacula Enterprise version 8.0 and later, we introduced new options to
4981 the \texttt{setdebug} command.
4982
4983 \smallskip{}
4984
4985 If the \texttt{options} parameter is set, the following arguments can be
4986 used to control debug functions.
4987
4988 \begin{itemize}
4989 \item [0] clear debug flags
4990 \item [i] Turn off, ignore bwrite() errors on restore on File Daemon
4991 \item [d] Turn off decomp of BackupRead() streams on File Daemon
4992 \item [t] Turn on timestamp in traces
4993 \item [T] Turn off timestamp in traces
4994 \item [c] Truncate trace file if trace file is activated
4995 \item [l] Turn on recoding events on P() and V()
4996 \item [p] Turn on the display of the event ring when doing a bactrace
4997 \end{itemize}
4998
4999 \smallskip{}
5000
5001 The following command will truncate the trace file and will turn on timestamps
5002 in the trace file.
5003
5004 \begin{bVerbatim}
5005 * setdebug level=10 trace=1 options=ct fd
5006 \end{bVerbatim}
5007
5008 \smallskip{}
5009
5010 It is now possible to use \textsl{class} of debug messages called \texttt{tags}
5011 to control the debug output of Bacula daemons.
5012
5013 \begin{itemize}
5014 \item [all] Display all debug messages
5015 \item [bvfs] Display BVFS debug messages
5016 \item [sql] Display SQL related debug messages
5017 \item [memory] Display memory and poolmem allocation messages
5018 \item [scheduler] Display scheduler related debug messages
5019 \end{itemize}
5020
5021 \begin{bVerbatim}
5022 * setdebug level=10 tags=bvfs,sql,memory
5023 * setdebug level=10 tags=!bvfs
5024
5025 # bacula-dir -t -d 200,bvfs,sql
5026 \end{bVerbatim}
5027
5028 The \texttt{tags} option is composed of a list of tags, tags are separated by
5029 ``,'' or ``+'' or ``-'' or ``!''. To disable a specific tag, use ``-'' or ``!''
5030 in front of the tag. Note that more tags will come in future versions.
5031
5032 %\LTXtable{\linewidth}{table_debugtags}
5033
5034
5035 \chapter{New Features in 5.2.13}
5036 This chapter presents the new features that have been added to the current
5037 Community version of Bacula that is now released.
5038
5039 \subsection{Additions to RunScript variables}
5040 You can have access to Director name using \%D in your runscript
5041 command.
5042
5043 \begin{bVerbatim}
5044 RunAfterJob = "/bin/echo Director=%D
5045 \end{bVerbatim}
5046
5047 \section{New Features in 5.2.1}
5048 This chapter presents the new features were added in the
5049 Community release version 5.2.1.
5050
5051 There are additional features (plugins) available in the Enterprise version
5052 that are described in another chapter. A subscription to Bacula Systems
5053 is required for the Enterprise version.
5054
5055 \subsection{LZO Compression}
5056
5057 LZO compression has been to the File daemon. From the user's point of view,
5058 it works like the GZIP compression (just replace {\bf compression=GZIP} with
5059 {\bf compression=LZO}).
5060
5061 For example:
5062 \begin{bVerbatim}
5063 Include {
5064 Options {compression=LZO }
5065 File = /home
5066 File = /data
5067 }
5068 \end{bVerbatim}
5069
5070 LZO provides a much faster compression and decompression speed but lower
5071 compression ratio than GZIP. It is a good option when you backup to disk. For
5072 tape, the hardware compression is almost always a better option.
5073
5074 LZO is a good alternative for GZIP1 when you don't want to slow down your
5075 backup. With a modern CPU it should be able to run almost as fast as:
5076
5077 \begin{itemize}
5078 \item your client can read data from disk. Unless you have very fast disks like
5079 SSD or large/fast RAID array.
5080 \item the data transfers between the file daemon and the storage daemon even on
5081 a 1Gb/s link.
5082 \end{itemize}
5083
5084 Note, Bacula uses compression level LZO1X-1.
5085
5086 \medskip
5087 The code for this feature was contributed by Laurent Papier.
5088
5089 \subsection{New Tray Monitor}
5090
5091 Since the old integrated Windows tray monitor doesn't work with
5092 recent Windows versions, we have written a new Qt Tray Monitor that is available
5093 for both Linux and Windows. In addition to all the previous features,
5094 this new version allows you to run Backups from
5095 the tray monitor menu.
5096
5097 \begin{figure}[htbp]
5098 \centering
5099 \includegraphics[width=0.8\linewidth]{tray-monitor}
5100 \label{fig:traymonitor}
5101 \caption{New tray monitor}
5102 \end{figure}
5103
5104 \begin{figure}[htbp]
5105 \centering
5106 \includegraphics[width=0.8\linewidth]{tray-monitor1}
5107 \label{fig:traymonitor1}
5108 \caption{Run a Job through the new tray monitor}
5109 \end{figure}
5110
5111
5112 To be able to run a job from the tray monitor, you need to
5113 allow specific commands in the Director monitor console:
5114 \begin{bVerbatim}
5115 Console {
5116 Name = win2003-mon
5117 Password = "xxx"
5118 CommandACL = status, .clients, .jobs, .pools, .storage, .filesets, .messages, run
5119 ClientACL = *all* # you can restrict to a specific host
5120 CatalogACL = *all*
5121 JobACL = *all*
5122 StorageACL = *all*
5123 ScheduleACL = *all*
5124 PoolACL = *all*
5125 FileSetACL = *all*
5126 WhereACL = *all*
5127 }
5128 \end{bVerbatim}
5129
5130 \medskip
5131 This project was funded by Bacula Systems and is available with Bacula
5132 the Enterprise Edition and the Community Edition.
5133
5134 \subsection{Purge Migration Job}
5135
5136 The new {\bf Purge Migration Job} directive may be added to the Migration
5137 Job definition in the Director's configuration file. When it is enabled
5138 the Job that was migrated during a migration will be purged at
5139 the end of the migration job.
5140
5141 For example:
5142 \begin{bVerbatim}
5143 Job {
5144 Name = "migrate-job"
5145 Type = Migrate
5146 Level = Full
5147 Client = localhost-fd
5148 FileSet = "Full Set"
5149 Messages = Standard
5150 Storage = DiskChanger
5151 Pool = Default
5152 Selection Type = Job
5153 Selection Pattern = ".*Save"
5154 ...
5155 Purge Migration Job = yes
5156 }
5157 \end{bVerbatim}
5158
5159 \medskip
5160
5161 This project was submitted by Dunlap Blake; testing and documentation was funded
5162 by Bacula Systems.
5163
5164 \subsection{Changes in Bvfs (Bacula Virtual FileSystem)}
5165
5166 Bat has now a bRestore panel that uses Bvfs to display files and
5167 directories.
5168
5169 \begin{figure}[htbp]
5170 \centering
5171 \includegraphics[width=0.8\linewidth]{bat-brestore}
5172 \label{fig:batbrestore}
5173 \caption{Bat Brestore Panel}
5174 \end{figure}
5175
5176 the Bvfs module works correctly with BaseJobs, Copy and Migration jobs.
5177
5178 \medskip
5179 This project was funded by Bacula Systems.
5180
5181 \subsubsection*{General notes}
5182
5183 \begin{itemize}
5184 \item All fields are separated by a tab
5185 \item You can specify \texttt{limit=} and \texttt{offset=} to list smoothly
5186 records in very big directories
5187 \item All operations (except cache creation) are designed to run instantly
5188 \item At this time, Bvfs works faster on PostgreSQL than MySQL catalog. If you
5189 can contribute new faster SQL queries we will be happy, else don't complain
5190 about speed.
5191 \item The cache creation is dependent of the number of directories. As Bvfs
5192 shares information across jobs, the first creation can be slow
5193 \item All fields are separated by a tab
5194 \item Due to potential encoding problem, it's advised to always use pathid in
5195 queries.
5196 \end{itemize}
5197
5198 \subsubsection*{Get dependent jobs from a given JobId}
5199
5200 Bvfs allows you to query the catalog against any combination of jobs. You
5201 can combine all Jobs and all FileSet for a Client in a single session.
5202
5203 To get all JobId needed to restore a particular job, you can use the
5204 \texttt{.bvfs\_get\_jobids} command.
5205
5206 \begin{bVerbatim}
5207 .bvfs_get_jobids jobid=num [all]
5208 \end{bVerbatim}
5209
5210 \begin{bVerbatim}
5211 .bvfs_get_jobids jobid=10
5212 1,2,5,10
5213 .bvfs_get_jobids jobid=10 all
5214 1,2,3,5,10
5215 \end{bVerbatim}
5216
5217 In this example, a normal restore will need to use JobIds 1,2,5,10 to
5218 compute a complete restore of the system.
5219
5220 With the \texttt{all} option, the Director will use all defined FileSet for
5221 this client.
5222
5223 \subsubsection*{Generating Bvfs cache}
5224
5225 The \texttt{.bvfs\_update} command computes the directory cache for jobs
5226 specified in argument, or for all jobs if unspecified.
5227
5228 \begin{bVerbatim}
5229 .bvfs_update [jobid=numlist]
5230 \end{bVerbatim}
5231
5232 Example:
5233 \begin{bVerbatim}
5234 .bvfs_update jobid=1,2,3
5235 \end{bVerbatim}
5236
5237 You can run the cache update process in a RunScript after the catalog backup.
5238
5239 \subsubsection*{Get all versions of a specific file}
5240
5241 Bvfs allows you to find all versions of a specific file for a given Client with
5242 the \texttt{.bvfs\_version} command. To avoid problems with encoding, this
5243 function uses only PathId and FilenameId. The jobid argument is mandatory but
5244 unused.
5245
5246 \begin{bVerbatim}
5247 .bvfs_versions client=filedaemon pathid=num filenameid=num jobid=1
5248 PathId FilenameId FileId JobId LStat Md5 VolName Inchanger
5249 PathId FilenameId FileId JobId LStat Md5 VolName Inchanger
5250 ...
5251 \end{bVerbatim}
5252
5253 Example:
5254
5255 \begin{bVerbatim}
5256 .bvfs_versions client=localhost-fd pathid=1 fnid=47 jobid=1
5257 1 47 52 12 gD HRid IGk D Po Po A P BAA I A /uPgWaxMgKZlnMti7LChyA Vol1 1
5258 \end{bVerbatim}
5259
5260 \subsubsection*{List directories}
5261
5262 Bvfs allows you to list directories in a specific path.
5263 \begin{bVerbatim}
5264 .bvfs_lsdirs pathid=num path=/apath jobid=numlist limit=num offset=num
5265 PathId FilenameId FileId JobId LStat Path
5266 PathId FilenameId FileId JobId LStat Path
5267 PathId FilenameId FileId JobId LStat Path
5268 ...
5269 \end{bVerbatim}
5270
5271 You need to \texttt{pathid} or \texttt{path}. Using \texttt{path=""} will list
5272 ``/'' on Unix and all drives on Windows. If FilenameId is 0, the record
5273 listed is a directory.
5274
5275 \begin{bVerbatim}
5276 .bvfs_lsdirs pathid=4 jobid=1,11,12
5277 4 0 0 0 A A A A A A A A A A A A A A .
5278 5 0 0 0 A A A A A A A A A A A A A A ..
5279 3 0 0 0 A A A A A A A A A A A A A A regress/
5280 \end{bVerbatim}
5281
5282 In this example, to list directories present in \texttt{regress/}, you can use
5283 \begin{bVerbatim}
5284 .bvfs_lsdirs pathid=3 jobid=1,11,12
5285 3 0 0 0 A A A A A A A A A A A A A A .
5286 4 0 0 0 A A A A A A A A A A A A A A ..
5287 2 0 0 0 A A A A A A A A A A A A A A tmp/
5288 \end{bVerbatim}
5289
5290 \subsubsection*{List files}
5291
5292 Bvfs allows you to list files in a specific path.
5293 \begin{bVerbatim}
5294 .bvfs_lsfiles pathid=num path=/apath jobid=numlist limit=num offset=num
5295 PathId FilenameId FileId JobId LStat Path
5296 PathId FilenameId FileId JobId LStat Path
5297 PathId FilenameId FileId JobId LStat Path
5298 ...
5299 \end{bVerbatim}
5300
5301 You need to \texttt{pathid} or \texttt{path}. Using \texttt{path=""} will list
5302 ``/'' on Unix and all drives on Windows. If FilenameId is 0, the record listed
5303 is a directory.
5304
5305 \begin{bVerbatim}
5306 .bvfs_lsfiles pathid=4 jobid=1,11,12
5307 4 0 0 0 A A A A A A A A A A A A A A .
5308 5 0 0 0 A A A A A A A A A A A A A A ..
5309 1 0 0 0 A A A A A A A A A A A A A A regress/
5310 \end{bVerbatim}
5311
5312 In this example, to list files present in \texttt{regress/}, you can use
5313 \begin{bVerbatim}
5314 .bvfs_lsfiles pathid=1 jobid=1,11,12
5315 1 47 52 12 gD HRid IGk BAA I BMqcPH BMqcPE BMqe+t A titi
5316 1 49 53 12 gD HRid IGk BAA I BMqe/K BMqcPE BMqe+t B toto
5317 1 48 54 12 gD HRie IGk BAA I BMqcPH BMqcPE BMqe+3 A tutu
5318 1 45 55 12 gD HRid IGk BAA I BMqe/K BMqcPE BMqe+t B ficheriro1.txt
5319 1 46 56 12 gD HRie IGk BAA I BMqe/K BMqcPE BMqe+3 D ficheriro2.txt
5320 \end{bVerbatim}
5321
5322 \subsubsection*{Restore set of files}
5323
5324 Bvfs allows you to create a SQL table that contains files that you want to
5325 restore. This table can be provided to a restore command with the file option.
5326
5327 \begin{bVerbatim}
5328 .bvfs_restore fileid=numlist dirid=numlist hardlink=numlist path=b2num
5329 OK
5330 restore file=?b2num ...
5331 \end{bVerbatim}
5332
5333 To include a directory (with \texttt{dirid}), Bvfs needs to run a query to
5334 select all files. This query could be time consuming.
5335
5336 \texttt{hardlink} list is always composed of a series of two numbers (jobid,
5337 fileindex). This information can be found in the LinkFI field of the LStat
5338 packet.
5339
5340 The \texttt{path} argument represents the name of the table that Bvfs will
5341 store results. The format of this table is \texttt{b2[0-9]+}. (Should start by
5342 b2 and followed by digits).
5343
5344 Example:
5345
5346 \begin{bVerbatim}
5347 .bvfs_restore fileid=1,2,3,4 hardlink=10,15,10,20 jobid=10 path=b20001
5348 OK
5349 \end{bVerbatim}
5350
5351 \subsubsection*{Cleanup after Restore}
5352
5353 To drop the table used by the restore command, you can use the
5354 \texttt{.bvfs\_cleanup} command.
5355
5356 \begin{bVerbatim}
5357 .bvfs_cleanup path=b20001
5358 \end{bVerbatim}
5359
5360 \subsubsection*{Clearing the BVFS Cache}
5361
5362 To clear the BVFS cache, you can use the \texttt{.bvfs\_clear\_cache} command.
5363
5364 \begin{bVerbatim}
5365 .bvfs_clear_cache yes
5366 OK
5367 \end{bVerbatim}
5368
5369 \subsection{Changes in the Pruning Algorithm}
5370
5371 We rewrote the job pruning algorithm in this version. Previously, in some users
5372 reported that the pruning process at the end of jobs was very long. It should
5373 not be longer the case. Now, Bacula won't prune automatically a Job if this
5374 particular Job is needed to restore data. Example:
5375
5376 \begin{bVerbatim}
5377 JobId: 1 Level: Full
5378 JobId: 2 Level: Incremental
5379 JobId: 3 Level: Incremental
5380 JobId: 4 Level: Differential
5381 .. Other incrementals up to now
5382 \end{bVerbatim}
5383
5384 In this example, if the Job Retention defined in the Pool or in the Client
5385 resource causes that Jobs with Jobid in 1,2,3,4 can be pruned, Bacula will
5386 detect that JobId 1 and 4 are essential to restore data at the current state
5387 and will prune only JobId 2 and 3.
5388
5389 \texttt{Important}, this change affect only the automatic pruning step after a
5390 Job and the \texttt{prune jobs} Bconsole command. If a volume expires after the
5391 \texttt{VolumeRetention} period, important jobs can be pruned.
5392
5393 \subsection{Ability to Verify any specified Job}
5394 You now have the ability to tell Bacula which Job should verify instead of
5395 automatically verify just the last one.
5396
5397 This feature can be used with VolumeToCatalog, DiskToCatalog and Catalog level.
5398
5399 To verify a given job, just specify the Job jobid in argument when starting the
5400 job.
5401 \begin{bVerbatim}
5402 *run job=VerifyVolume jobid=1 level=VolumeToCatalog
5403 Run Verify job
5404 JobName: VerifyVolume
5405 Level: VolumeToCatalog
5406 Client: 127.0.0.1-fd
5407 FileSet: Full Set
5408 Pool: Default (From Job resource)
5409 Storage: File (From Job resource)
5410 Verify Job: VerifyVol.2010-09-08_14.17.17_03
5411 Verify List: /tmp/regress/working/VerifyVol.bsr
5412 When: 2010-09-08 14:17:31
5413 Priority: 10
5414 OK to run? (yes/mod/no):
5415 \end{bVerbatim}
5416
5417 \medskip
5418 This project was funded by Bacula Systems and is available with Bacula
5419 Enterprise Edition and Community Edition.
5420
5421 \subsection{Additions to RunScript variables}
5422 You can have access to JobBytes and JobFiles using \%b and \%F in your runscript
5423 command. The Client address is now available through \%h.
5424
5425 \begin{bVerbatim}
5426 RunAfterJob = "/bin/echo Job=%j JobBytes=%b JobFiles=%F ClientAddress=%h"
5427 \end{bVerbatim}
5428
5429 %\subsection{Changes in drivetype.exe}
5430 %
5431 %Now the \texttt{drivetype.exe} program allows you to list all local hard
5432 %drives. It can help to build dynamic FileSet on Windows.
5433 %
5434 %\begin{bVerbatim}
5435 %File = "\\|\"c:/program files/bacula/bin32/drivetype\" -l -a"
5436 %\end{bVerbatim}
5437 %
5438
5439 \subsection{Additions to the Plugin API}
5440 The bfuncs structure has been extended to include a number of
5441 new entrypoints.
5442
5443 \subsubsection{bfuncs}
5444 The bFuncs structure defines the callback entry points within Bacula
5445 that the plugin can use register events, get Bacula values, set
5446 Bacula values, and send messages to the Job output or debug output.
5447
5448 The exact definition as of this writing is:
5449 \begin{bVerbatim}
5450 typedef struct s_baculaFuncs {
5451 uint32_t size;
5452 uint32_t version;
5453 bRC (*registerBaculaEvents)(bpContext *ctx, ...);
5454 bRC (*getBaculaValue)(bpContext *ctx, bVariable var, void *value);
5455 bRC (*setBaculaValue)(bpContext *ctx, bVariable var, void *value);
5456 bRC (*JobMessage)(bpContext *ctx, const char *file, int line,
5457 int type, utime_t mtime, const char *fmt, ...);
5458 bRC (*DebugMessage)(bpContext *ctx, const char *file, int line,
5459 int level, const char *fmt, ...);
5460 void *(*baculaMalloc)(bpContext *ctx, const char *file, int line,
5461 size_t size);
5462 void (*baculaFree)(bpContext *ctx, const char *file, int line, void *mem);
5463
5464 /* New functions follow */
5465 bRC (*AddExclude)(bpContext *ctx, const char *file);
5466 bRC (*AddInclude)(bpContext *ctx, const char *file);
5467 bRC (*AddIncludeOptions)(bpContext *ctx, const char *opts);
5468 bRC (*AddRegex)(bpContext *ctx, const char *item, int type);
5469 bRC (*AddWild)(bpContext *ctx, const char *item, int type);
5470 bRC (*checkChanges)(bpContext *ctx, struct save_pkt *sp);
5471
5472 } bFuncs;
5473 \end{bVerbatim}
5474
5475 \begin{description}
5476 \item [AddExclude] can be called to exclude a file. The file
5477 string passed may include wildcards that will be interpreted by
5478 the {\bf fnmatch} subroutine. This function can be called
5479 multiple times, and each time the file specified will be added
5480 to the list of files to be excluded. Note, this function only
5481 permits adding excludes of specific file or directory names,
5482 or files matched by the rather simple fnmatch mechanism.
5483 See below for information on doing wild-card and regex excludes.
5484
5485 \item [NewPreInclude] can be called to create a new Include block. This
5486 block will be added after the current defined Include block. This
5487 function can be called multiple times, but each time, it will create
5488 a new Include section (not normally needed). This function should
5489 be called only if you want to add an entirely new Include block.
5490
5491 \item [NewInclude] can be called to create a new Include block. This
5492 block will be added before any user defined Include blocks. This
5493 function can be called multiple times, but each time, it will create
5494 a new Include section (not normally needed). This function should
5495 be called only if you want to add an entirely new Include block.
5496
5497 \item [AddInclude] can be called to add new files/directories to
5498 be included. They are added to the current Include block. If
5499 NewInclude has not been included, the current Include block is
5500 the last one that the user created. This function
5501 should be used only if you want to add totally new files/directories
5502 to be included in the backup.
5503
5504 \item [NewOptions] adds a new Options block to the current Include
5505 in front of any other Options blocks. This permits the plugin to
5506 add exclude directives (wild-cards and regexes) in front of the
5507 user Options, and thus prevent certain files from being backed up.
5508 This can be useful if the plugin backs up files, and they should
5509 not be also backed up by the main Bacula code. This function
5510 may be called multiple times, and each time, it creates a new
5511 prepended Options block. Note: normally you want to call this
5512 entry point prior to calling AddOptions, AddRegex, or AddWild.
5513
5514 \item [AddOptions] allows the plugin it set options in
5515 the current Options block, which is normally created with the
5516 NewOptions call just prior to adding Include Options.
5517 The permitted options are passed as a character string, where
5518 each character has a specific meaning as defined below:
5519
5520 \begin{description}
5521 \item [a] always replace files (default).
5522 \item [e] exclude rather than include.
5523 \item [h] no recursion into subdirectories.
5524 \item [H] do not handle hard links.
5525 \item [i] ignore case in wildcard and regex matches.
5526 \item [M] compute an MD5 sum.
5527 \item [p] use a portable data format on Windows (not recommended).
5528 \item [R] backup resource forks and Findr Info.
5529 \item [r] read from a fifo
5530 \item [S1] compute an SHA1 sum.
5531 \item [S2] compute an SHA256 sum.
5532 \item [S3] comput an SHA512 sum.
5533 \item [s] handle sparse files.
5534 \item [m] use st\_mtime only for file differences.
5535 \item [k] restore the st\_atime after accessing a file.
5536 \item [A] enable ACL backup.
5537 \item [Vxxx:] specify verify options. Must terminate with :
5538 \item [Cxxx:] specify accurate options. Must terminate with :
5539 \item [Jxxx:] specify base job Options. Must terminate with :
5540 \item [Pnnn:] specify integer nnn paths to strip. Must terminate with :
5541 \item [w] if newer
5542 \item [Zn] specify gzip compression level n.
5543 \item [K] do not use st\_atime in backup decision.
5544 \item [c] check if file changed during backup.
5545 \item [N] honor no dump flag.
5546 \item [X] enable backup of extended attributes.
5547 \end{description}
5548
5549 \item [AddRegex] adds a regex expression to the current Options block.
5550 The following options are permitted:
5551 \begin{description}
5552 \item [ ] (a blank) regex applies to whole path and filename.
5553 \item [F] regex applies only to the filename (directory or path stripped).
5554 \item [D] regex applies only to the directory (path) part of the name.
5555 \end{description}
5556
5557 \item [AddWild] adds a wildcard expression to the current Options block.
5558 The following options are permitted:
5559 \begin{description}
5560 \item [ ] (a blank) regex applies to whole path and filename.
5561 \item [F] regex applies only to the filename (directory or path stripped).
5562 \item [D] regex applies only to the directory (path) part of the name.
5563 \end{description}
5564
5565 \item [checkChanges] call the \texttt{check\_changes()} function in Bacula code
5566 that can use Accurate code to compare the file information in argument with
5567 the previous file information. The \texttt{delta\_seq} attribute of the
5568 \texttt{save\_pkt} will be updated, and the call will return
5569 \texttt{bRC\_Seen} if the core code wouldn't decide to backup it.
5570
5571 \end{description}
5572
5573
5574 \subsubsection{Bacula events}
5575 The list of events has been extended to include:
5576
5577 \begin{bVerbatim}
5578 typedef enum {
5579 bEventJobStart = 1,
5580 bEventJobEnd = 2,
5581 bEventStartBackupJob = 3,
5582 bEventEndBackupJob = 4,
5583 bEventStartRestoreJob = 5,
5584 bEventEndRestoreJob = 6,
5585 bEventStartVerifyJob = 7,
5586 bEventEndVerifyJob = 8,
5587 bEventBackupCommand = 9,
5588 bEventRestoreCommand = 10,
5589 bEventLevel = 11,
5590 bEventSince = 12,
5591
5592 /* New events */
5593 bEventCancelCommand = 13,
5594 bEventVssBackupAddComponents = 14,
5595 bEventVssRestoreLoadComponentMetadata = 15,
5596 bEventVssRestoreSetComponentsSelected = 16,
5597 bEventRestoreObject = 17,
5598 bEventEndFileSet = 18,
5599 bEventPluginCommand = 19,
5600 bEventVssBeforeCloseRestore = 20,
5601 bEventVssPrepareSnapshot = 21
5602
5603 } bEventType;
5604 \end{bVerbatim}
5605
5606 \begin{description}
5607 \item [bEventCancelCommand] is called whenever the currently
5608 running Job is canceled */
5609
5610 \item [bEventVssBackupAddComponents]
5611
5612 \item [bEventVssPrepareSnapshot] is called before creating VSS snapshots, it
5613 provides a char[27] table where the plugin can add Windows drives that will
5614 be used during the Job. You need to add them without duplicates, and you can
5615 use in \texttt{fd\_common.h} \texttt{add\_drive()} and \texttt{copy\_drives()}
5616 for this purpose.
5617 \end{description}
5618
5619 \subsection{ACL enhancements}
5620
5621 The following enhancements are made to the Bacula Filed with regards to
5622 Access Control Lists (ACLs)
5623
5624 \begin{itemize}
5625 \item Added support for AIX 5.3 and later new aclx\_get interface which supports
5626 POSIX and NFSv4 ACLs.
5627 \item Added support for new acl types on FreeBSD 8.1 and later which supports
5628 POSIX and NFSv4 ACLs.
5629 \item Some generic cleanups for internal ACL handling.
5630 \item Fix for acl storage on OSX
5631 \item Cleanup of configure checks for ACL detection, now configure only
5632 tests for a certain interface type based on the operating system
5633 this should give less false positives on detection. Also when ACLs
5634 are detected no other acl checks are performed anymore.
5635 \end{itemize}
5636
5637 \medskip
5638 This project was funded by Planets Communications B.V. and ELM Consultancy B.V.
5639 and is available with Bacula Enterprise Edition and Community Edition.
5640
5641 \subsection{XATTR enhancements}
5642
5643 The following enhancements are made to the Bacula Filed with regards to
5644 Extended Attributes (XATTRs)
5645
5646 \begin{itemize}
5647 \item Added support for IRIX extended attributes using the attr\_get interface.
5648 \item Added support for Tru64 (OSF1) extended attributes using the
5649 getproplist interface.
5650 \item Added support for AIX extended attributes available in AIX 6.x
5651 and higher using the listea/getea/setea interface.
5652 \item Added some debugging to generic xattr code so it easier to
5653 debug.
5654 \item Cleanup of configure checks for XATTR detection, now configure only
5655 tests for a certain interface type based on the operating system
5656 this should give less false positives on detection. Also when xattrs
5657 are detected no other xattr checks are performed anymore.
5658 \end{itemize}
5659
5660 \medskip
5661 This project was funded by Planets Communications B.V. and ELM Consultancy B.V.
5662 and is available with Bacula Enterprise Edition and Community Edition.
5663
5664 \subsection{Class Based Database Backend Drivers}
5665
5666 The main Bacula Director code is independent of the SQL backend
5667 in version 5.2.0 and greater. This means that the Bacula Director can be
5668 packaged by itself, then each of the different SQL backends supported can
5669 be packaged separately. It is possible to build all the DB backends at the
5670 same time by including multiple database options at the same time.
5671
5672 ./configure can be run with multiple database configure options.
5673 \begin{bVerbatim}
5674 --with-mysql
5675 --with-postgresql
5676 \end{bVerbatim}
5677
5678 Order of testing for databases is:
5679 \begin{itemize}
5680 \item postgresql
5681 \item mysql
5682 \end{itemize}
5683
5684 Each configured backend generates a file named:
5685 \verb+libbaccats-<sql_backend_name>-<version>.so+
5686 A dummy catalog library is created named libbaccats-version.so
5687
5688 At configure time the first detected backend is used as the so called
5689 default backend and at install time the dummy
5690 \verb+libbaccats-<version>.so+ is replaced with the default backend type.
5691
5692 If you configure all three backends you get three backend libraries and the
5693 postgresql gets installed as the default.
5694
5695 When you want to switch to another database, first save any old catalog you
5696 may have then you can copy one of the three backend libraries over the
5697 \verb+libbaccats-<version>.so+ e.g.
5698
5699 An actual command, depending on your Bacula version might be:
5700 \begin{bVerbatim}
5701 cp libbaccats-postgresql-5.2.2.so libbaccats-5.2.2.so
5702 \end{bVerbatim}
5703
5704 where the \verb+5.2.2+ must be replaced by the Bacula release
5705 version number.
5706
5707 Then you must update the default backend in the following files:
5708
5709 \begin{bVerbatim}
5710 create_bacula_database
5711 drop_bacula_database
5712 drop_bacula_tables
5713 grant_bacula_privileges
5714 make_bacula_tables
5715 make_catalog_backup
5716 update_bacula_tables
5717 \end{bVerbatim}
5718
5719 And re-run all the above scripts. Please note, this means
5720 you will have a new empty database and if you had a previous
5721 one it will be lost.
5722
5723 All current database backend drivers for catalog information are rewritten
5724 to use a set of multi inherited C++ classes which abstract the specific
5725 database specific internals and make sure we have a more stable generic
5726 interface with the rest of SQL code. From now on there is a strict
5727 boundary between the SQL code and the low-level database functions. This
5728 new interface should also make it easier to add a new backend for a
5729 currently unsupported database. An extra bonus of the new code is that you
5730 can configure multiple backends in the configure and build all backends in
5731 one compile session and select the correct database backend at install
5732 time. This should make it a lot easier for packages maintainers.
5733
5734
5735
5736 \medskip
5737 We also added cursor support for PostgreSQL backend, this improves memory
5738 usage for large installation.
5739
5740 \medskip
5741 This project was implemented by Planets Communications B.V. and ELM
5742 Consultancy B.V. and Bacula Systems and is available with both the Bacula
5743 Enterprise Edition and the Community Edition.
5744
5745 \subsection{Hash List Enhancements}
5746
5747 The htable hash table class has been extended with extra hash functions for
5748 handling next to char pointer hashes also 32 bits and 64 bits hash keys.
5749 Also the hash table initialization routines have been enhanced with
5750 support for passing a hint as to the number of initial pages to use
5751 for the size of the hash table. Until now the hash table always used
5752 a fixed value of 10 Mb. The private hash functions of the mountpoint entry
5753 cache have been rewritten to use the new htable class with a small memory
5754 footprint.
5755
5756 \medskip
5757 This project was funded by Planets Communications B.V. and ELM Consultancy B.V.
5758 and Bacula Systems and is available with Bacula Enterprise Edition and
5759 Community Edition.
5760
5761 %%
5762 %%
5763 %%% =====================================================================
5764 %%
5765 %%
5766
5767
5768 \section{Release Version 5.0.3}
5769
5770 There are no new features in version 5.0.2. This version simply fixes a
5771 number of bugs found in version 5.0.1 during the ongoing development
5772 process.
5773
5774 \section{Release Version 5.0.2}
5775
5776 There are no new features in version 5.0.2. This version simply fixes a
5777 number of bugs found in version 5.0.1 during the ongoing development
5778 process.
5779
5780 %%
5781 %%
5782
5783 \section{New Features in 5.0.1}
5784
5785 This chapter presents the new features that are in the released Bacula version
5786 5.0.1. This version mainly fixes a number of bugs found in version 5.0.0 during
5787 the ongoing development process.
5788
5789 \subsection{Truncate Volume after Purge}
5790 \label{sec:actiononpurge}
5791
5792 The Pool directive \textbf{ActionOnPurge=Truncate} instructs Bacula to truncate
5793 the volume when it is purged with the new command \texttt{purge volume
5794 action}. It is useful to prevent disk based volumes from consuming too much
5795 space.
5796
5797 \begin{bVerbatim}
5798 Pool {
5799 Name = Default
5800 Action On Purge = Truncate
5801 ...
5802 }
5803 \end{bVerbatim}
5804
5805 As usual you can also set this property with the \texttt{update volume} command
5806 \begin{bVerbatim}
5807 *update volume=xxx ActionOnPurge=Truncate
5808 *update volume=xxx actiononpurge=None
5809 \end{bVerbatim}
5810
5811 To ask Bacula to truncate your \texttt{Purged} volumes, you need to use the
5812 following command in interactive mode or in a RunScript as shown after:
5813 \begin{bVerbatim}
5814 *purge volume action=truncate storage=File allpools
5815 # or by default, action=all
5816 *purge volume action storage=File pool=Default
5817 \end{bVerbatim}
5818
5819 This is possible to specify the volume name, the media type, the pool, the
5820 storage, etc\dots (see \texttt{help purge}) Be sure that your storage device is
5821 idle when you decide to run this command.
5822
5823 \begin{bVerbatim}
5824 Job {
5825 Name = CatalogBackup
5826 ...
5827 RunScript {
5828 RunsWhen=After
5829 RunsOnClient=No
5830 Console = "purge volume action=all allpools storage=File"
5831 }
5832 }
5833 \end{bVerbatim}
5834
5835 \textbf{Important note}: This feature doesn't work as
5836 expected in version 5.0.0. Please do not use it before version 5.0.1.
5837
5838 \subsection{Allow Higher Duplicates}
5839 This directive did not work correctly and has been depreciated
5840 (disabled) in version 5.0.1. Please remove it from your bacula-dir.conf
5841 file as it will be removed in a future release.
5842
5843 \subsection{Cancel Lower Level Duplicates}
5844 This directive was added in Bacula version 5.0.1. It compares the
5845 level of a new backup job to old jobs of the same name, if any,
5846 and will kill the job which has a lower level than the other one.
5847 If the levels are the same (i.e. both are Full backups), then
5848 nothing is done and the other Cancel XXX Duplicate directives
5849 will be examined.
5850
5851 \section{New Features in 5.0.0}
5852
5853 \subsection{Maximum Concurrent Jobs for Devices}
5854 \label{sec:maximumconcurrentjobdevice}
5855
5856 {\bf Maximum Concurrent Jobs} is a new Device directive in the Storage
5857 Daemon configuration permits setting the maximum number of Jobs that can
5858 run concurrently on a specified Device. Using this directive, it is
5859 possible to have different Jobs using multiple drives, because when the
5860 Maximum Concurrent Jobs limit is reached, the Storage Daemon will start new
5861 Jobs on any other available compatible drive. This facilitates writing to
5862 multiple drives with multiple Jobs that all use the same Pool.
5863
5864 This project was funded by Bacula Systems.
5865
5866 \subsection{Restore from Multiple Storage Daemons}
5867 \index[general]{Restore}
5868
5869 Previously, you were able to restore from multiple devices in a single Storage
5870 Daemon. Now, Bacula is able to restore from multiple Storage Daemons. For
5871 example, if your full backup runs on a Storage Daemon with an autochanger, and
5872 your incremental jobs use another Storage Daemon with lots of disks, Bacula
5873 will switch automatically from one Storage Daemon to an other within the same
5874 Restore job.
5875
5876 You must upgrade your File Daemon to version 3.1.3 or greater to use this
5877 feature.
5878
5879 This project was funded by Bacula Systems with the help of Equiinet.
5880
5881 \subsection{File Deduplication using Base Jobs}
5882 A base job is sort of like a Full save except that you will want the FileSet to
5883 contain only files that are unlikely to change in the future (i.e. a snapshot
5884 of most of your system after installing it). After the base job has been run,
5885 when you are doing a Full save, you specify one or more Base jobs to be used.
5886 All files that have been backed up in the Base job/jobs but not modified will
5887 then be excluded from the backup. During a restore, the Base jobs will be
5888 automatically pulled in where necessary.
5889
5890 This is something none of the competition does, as far as we know (except
5891 perhaps BackupPC, which is a Perl program that saves to disk only). It is big
5892 win for the user, it makes Bacula stand out as offering a unique optimization
5893 that immediately saves time and money. Basically, imagine that you have 100
5894 nearly identical Windows or Linux machine containing the OS and user files.
5895 Now for the OS part, a Base job will be backed up once, and rather than making
5896 100 copies of the OS, there will be only one. If one or more of the systems
5897 have some files updated, no problem, they will be automatically restored.
5898
5899 See the \bilink{Base Job Chapter}{blb:basejobs} for more information.
5900
5901 This project was funded by Bacula Systems.
5902
5903 \subsection{AllowCompression = \lt{}yes\vb{}no\gt{}}
5904 \index[dir]{AllowCompression}
5905
5906 This new directive may be added to Storage resource within the Director's
5907 configuration to allow users to selectively disable the client compression for
5908 any job which writes to this storage resource.
5909
5910 For example:
5911 \begin{bVerbatim}
5912 Storage {
5913 Name = UltriumTape
5914 Address = ultrium-tape
5915 Password = storage_password # Password for Storage Daemon
5916 Device = Ultrium
5917 Media Type = LTO 3
5918 AllowCompression = No # Tape drive has hardware compression
5919 }
5920 \end{bVerbatim}
5921 The above example would cause any jobs running with the UltriumTape storage
5922 resource to run without compression from the client file daemons. This
5923 effectively overrides any compression settings defined at the FileSet level.
5924
5925 This feature is probably most useful if you have a tape drive which supports
5926 hardware compression. By setting the \texttt{AllowCompression = No} directive
5927 for your tape drive storage resource, you can avoid additional load on the file
5928 daemon and possibly speed up tape backups.
5929
5930 This project was funded by Collaborative Fusion, Inc.
5931
5932 \subsection{Accurate Fileset Options}
5933 \label{sec:accuratefileset}
5934
5935 In previous versions, the accurate code used the file creation and modification
5936 times to determine if a file was modified or not. Now you can specify which
5937 attributes to use (time, size, checksum, permission, owner, group, \dots),
5938 similar to the Verify options.
5939
5940 \begin{bVerbatim}
5941 FileSet {
5942 Name = Full
5943 Include = {
5944 Options {
5945 Accurate = mcs
5946 Verify = pin5
5947 }
5948 File = /
5949 }
5950 }
5951 \end{bVerbatim}
5952
5953 \begin{description}
5954 \item {\bf i} compare the inodes
5955 \item {\bf p} compare the permission bits
5956 \item {\bf n} compare the number of links
5957 \item {\bf u} compare the user id
5958 \item {\bf g} compare the group id
5959 \item {\bf s} compare the size
5960 \item {\bf a} compare the access time
5961 \item {\bf m} compare the modification time (st\_mtime)
5962 \item {\bf c} compare the change time (st\_ctime)
5963 \item {\bf d} report file size decreases
5964 \item {\bf 5} compare the MD5 signature
5965 \item {\bf 1} compare the SHA1 signature
5966 \end{description}
5967
5968 \textbf{Important note:} If you decide to use checksum in Accurate jobs,
5969 the File Daemon will have to read all files even if they normally would not
5970 be saved. This increases the I/O load, but also the accuracy of the
5971 deduplication. By default, Bacula will check modification/creation time
5972 and size.
5973
5974 This project was funded by Bacula Systems.
5975
5976 \subsection{Tab-completion for Bconsole}
5977 \label{sec:tabcompletion}
5978
5979 If you build \texttt{bconsole} with readline support, you will be able to use
5980 the new auto-completion mode. This mode supports all commands, gives help
5981 inside command, and lists resources when required. It works also in the restore
5982 mode.
5983
5984 To use this feature, you should have readline development package loaded on
5985 your system, and use the following option in configure.
5986 \begin{bVerbatim}
5987 ./configure --with-readline=/usr/include/readline --disable-conio ...
5988 \end{bVerbatim}
5989
5990 The new bconsole won't be able to tab-complete with older directors.
5991
5992 This project was funded by Bacula Systems.
5993
5994 \subsection{Pool File and Job Retention}
5995 \label{sec:poolfilejobretention}
5996
5997 We added two new Pool directives, \texttt{FileRetention} and
5998 \texttt{JobRetention}, that take precedence over Client directives of the same
5999 name. It allows you to control the Catalog pruning algorithm Pool by Pool. For
6000 example, you can decide to increase Retention times for Archive or OffSite Pool.
6001
6002 It seems obvious to us, but apparently not to some users, that given the
6003 definition above that the Pool File and Job Retention periods is a global
6004 override for the normal Client based pruning, which means that when the
6005 Job is pruned, the pruning will apply globally to that particular Job.
6006
6007 Currently, there is a bug in the implementation that causes any Pool
6008 retention periods specified to apply to {\bf all} Pools for that
6009 particular Client. Thus we suggest that you avoid using these two
6010 directives until this implementation problem is corrected.
6011
6012 \subsection{Read-only File Daemon using capabilities}
6013 \label{sec:fdreadonly}
6014 This feature implements support of keeping \textbf{ReadAll} capabilities after
6015 UID/GID switch, this allows FD to keep root read but drop write permission.
6016
6017 It introduces new \texttt{bacula-fd} option (\texttt{-k}) specifying that
6018 \textbf{ReadAll} capabilities should be kept after UID/GID switch.
6019
6020 \begin{bVerbatim}
6021 root@localhost:~# bacula-fd -k -u nobody -g nobody
6022 \end{bVerbatim}
6023
6024 The code for this feature was contributed by our friends at AltLinux.
6025
6026 \subsection{Bvfs API}
6027 \label{sec:bvfs}
6028
6029 To help developers of restore GUI interfaces, we have added new \textsl{dot
6030 commands} that permit browsing the catalog in a very simple way.
6031
6032 \begin{itemize}
6033 \item \texttt{.bvfs\_update [jobid=x,y,z]} This command is required to update
6034 the Bvfs cache in the catalog. You need to run it before any access to the
6035 Bvfs layer.
6036
6037 \item \texttt{.bvfs\_lsdirs jobid=x,y,z path=/path | pathid=101} This command
6038 will list all directories in the specified \texttt{path} or
6039 \texttt{pathid}. Using \texttt{pathid} avoids problems with character
6040 encoding of path/filenames.
6041
6042 \item \texttt{.bvfs\_lsfiles jobid=x,y,z path=/path | pathid=101} This command
6043 will list all files in the specified \texttt{path} or \texttt{pathid}. Using
6044 \texttt{pathid} avoids problems with character encoding.
6045 \end{itemize}
6046
6047 You can use \texttt{limit=xxx} and \texttt{offset=yyy} to limit the amount of
6048 data that will be displayed.
6049
6050 \begin{bVerbatim}
6051 * .bvfs_update jobid=1,2
6052 * .bvfs_update
6053 * .bvfs_lsdir path=/ jobid=1,2
6054 \end{bVerbatim}
6055
6056 This project was funded by Bacula Systems.
6057
6058 \subsection{Testing your Tape Drive}
6059 \label{sec:btapespeed}
6060
6061 To determine the best configuration of your tape drive, you can run the new
6062 \texttt{speed} command available in the \texttt{btape} program.
6063
6064 This command can have the following arguments:
6065 \begin{itemize}
6066 \item[\texttt{file\_size=n}] Specify the Maximum File Size for this test
6067 (between 1 and 5GB). This counter is in GB.
6068 \item[\texttt{nb\_file=n}] Specify the number of file to be written. The amount
6069 of data should be greater than your memory ($file\_size*nb\_file$).
6070 \item[\texttt{skip\_zero}] This flag permits to skip tests with constant
6071 data.
6072 \item[\texttt{skip\_random}] This flag permits to skip tests with random
6073 data.
6074 \item[\texttt{skip\_raw}] This flag permits to skip tests with raw access.
6075 \item[\texttt{skip\_block}] This flag permits to skip tests with Bacula block
6076 access.
6077 \end{itemize}
6078
6079 \begin{bVerbatim}
6080 *speed file_size=3 skip_raw
6081 btape.c:1078 Test with zero data and bacula block structure.
6082 btape.c:956 Begin writing 3 files of 3.221 GB with blocks of 129024 bytes.
6083 ++++++++++++++++++++++++++++++++++++++++++
6084 btape.c:604 Wrote 1 EOF to "Drive-0" (/dev/nst0)
6085 btape.c:406 Volume bytes=3.221 GB. Write rate = 44.128 MB/s
6086 ...
6087 btape.c:383 Total Volume bytes=9.664 GB. Total Write rate = 43.531 MB/s
6088
6089 btape.c:1090 Test with random data, should give the minimum throughput.
6090 btape.c:956 Begin writing 3 files of 3.221 GB with blocks of 129024 bytes.
6091 +++++++++++++++++++++++++++++++++++++++++++
6092 btape.c:604 Wrote 1 EOF to "Drive-0" (/dev/nst0)
6093 btape.c:406 Volume bytes=3.221 GB. Write rate = 7.271 MB/s
6094 +++++++++++++++++++++++++++++++++++++++++++
6095 ...
6096 btape.c:383 Total Volume bytes=9.664 GB. Total Write rate = 7.365 MB/s
6097
6098 \end{bVerbatim}
6099
6100 When using compression, the random test will give your the minimum throughput
6101 of your drive . The test using constant string will give you the maximum speed
6102 of your hardware chain. (CPU, memory, SCSI card, cable, drive, tape).
6103
6104 You can change the block size in the Storage Daemon configuration file.
6105
6106 \subsection{New {\bf Block Checksum} Device Directive}
6107 You may now turn off the Block Checksum (CRC32) code
6108 that Bacula uses when writing blocks to a Volume. This is
6109 done by adding:
6110
6111 \begin{bVerbatim}
6112 Block Checksum = no
6113 \end{bVerbatim}
6114
6115 doing so can reduce the Storage daemon CPU usage slightly. It
6116 will also permit Bacula to read a Volume that has corrupted data.
6117
6118 The default is {\bf yes} -- i.e. the checksum is computed on write
6119 and checked on read.
6120
6121 We do not recommend to turn this off particularly on older tape
6122 drives or for disk Volumes where doing so may allow corrupted data
6123 to go undetected.
6124
6125 \subsection{New Bat Features}
6126
6127 Those new features were funded by Bacula Systems.
6128
6129 \subsubsection{Media List View}
6130
6131 By clicking on ``Media'', you can see the list of all your volumes. You will be
6132 able to filter by Pool, Media Type, Location,\dots And sort the result directly
6133 in the table. The old ``Media'' view is now known as ``Pool''.
6134 \begin{figure}[htbp]
6135 \centering
6136 \includegraphics[width=0.8\linewidth]{bat-mediaview}
6137 \label{fig:mediaview}
6138 \end{figure}
6139
6140
6141 \subsubsection{Media Information View}
6142
6143 By double-clicking on a volume (on the Media list, in the Autochanger content
6144 or in the Job information panel), you can access a detailed overview of your
6145 Volume. (cf figure \vref{fig:mediainfo}.)
6146 \begin{figure}[htbp]
6147 \centering
6148 \includegraphics[width=0.8\linewidth]{bat11}
6149 \caption{Media information}
6150 \label{fig:mediainfo}
6151 \end{figure}
6152
6153 \subsubsection{Job Information View}
6154
6155 By double-clicking on a Job record (on the Job run list or in the Media
6156 information panel), you can access a detailed overview of your Job. (cf
6157 figure \vref{fig:jobinfo}.)
6158 \begin{figure}[htbp]
6159 \centering
6160 \includegraphics[width=0.8\linewidth]{bat12}
6161 \caption{Job information}
6162 \label{fig:jobinfo}
6163 \end{figure}
6164
6165 \subsubsection{Autochanger Content View}
6166
6167 By double-clicking on a Storage record (on the Storage list panel), you can
6168 access a detailed overview of your Autochanger. (cf figure \vref{fig:jobinfo}.)
6169 \begin{figure}[htbp]
6170 \centering
6171 \includegraphics[width=0.8\linewidth]{bat13}
6172 \caption{Autochanger content}
6173 \label{fig:achcontent}
6174 \end{figure}
6175
6176 To use this feature, you need to use the latest mtx-changer script
6177 version. (With new \texttt{listall} and \texttt{transfer} commands)
6178
6179 \subsection{Bat on Windows}
6180 We have ported {\bf bat} to Windows and it is now installed
6181 by default when the installer is run. It works quite well
6182 on Win32, but has not had a lot of testing there, so your
6183 feedback would be welcome. Unfortunately, even though it is
6184 installed by default, it does not yet work on 64 bit Windows
6185 operating systems.
6186
6187 \subsection{New Win32 Installer}
6188 The Win32 installer has been modified in several very important
6189 ways.
6190 \begin{itemize}
6191 \item You must deinstall any current version of the
6192 Win32 File daemon before upgrading to the new one.
6193 If you forget to do so, the new installation will fail.
6194 To correct this failure, you must manually shutdown
6195 and deinstall the old File daemon.
6196 \item All files (other than menu links) are installed
6197 in {\bf c:/Program Files/Bacula}.
6198 \item The installer no longer sets this
6199 file to require administrator privileges by default. If you want
6200 to do so, please do it manually using the {\bf cacls} program.
6201 For example:
6202 \begin{bVerbatim}
6203 cacls "C:\Program Files\Bacula" /T /G SYSTEM:F Administrators:F
6204 \end{bVerbatim}
6205 \item The server daemons (Director and Storage daemon) are
6206 no longer included in the Windows installer. If you want the
6207 Windows servers, you will either need to build them yourself (note
6208 they have not been ported to 64 bits), or you can contact
6209 Bacula Systems about this.
6210 \end{itemize}
6211
6212 \subsection{Win64 Installer}
6213 We have corrected a number of problems that required manual
6214 editing of the conf files. In most cases, it should now
6215 install and work. {\bf bat} is by default installed in
6216 {\bf c:/Program Files/Bacula/bin32} rather than
6217 {\bf c:/Program Files/Bacula} as is the case with the 32
6218 bit Windows installer.
6219
6220 \subsection{Linux Bare Metal Recovery USB Key}
6221 We have made a number of significant improvements in the
6222 Bare Metal Recovery USB key. Please see the README files
6223 it the {\bf rescue} release for more details.
6224
6225 We are working on an equivalent USB key for Windows bare
6226 metal recovery, but it will take some time to develop it (best
6227 estimate 3Q2010 or 4Q2010)
6228
6229
6230 \subsection{bconsole Timeout Option}
6231 You can now use the -u option of {\bf bconsole} to set a timeout in seconds
6232 for commands. This is useful with GUI programs that use {\bf bconsole}
6233 to interface to the Director.
6234
6235 \subsection{Important Changes}
6236 \label{sec:importantchanges}
6237
6238 \begin{itemize}
6239 \item You are now allowed to Migrate, Copy, and Virtual Full to read and write
6240 to the same Pool. The Storage daemon ensures that you do not read and
6241 write to the same Volume.
6242 \item The \texttt{Device Poll Interval} is now 5 minutes. (previously did not
6243 poll by default).
6244 \item Virtually all the features of {\bf mtx-changer} have
6245 now been parametrized, which allows you to configure
6246 mtx-changer without changing it. There is a new configuration file {\bf mtx-changer.conf}
6247 that contains variables that you can set to configure mtx-changer.
6248 This configuration file will not be overwritten during upgrades.
6249 We encourage you to submit any changes
6250 that are made to mtx-changer and to parametrize it all in
6251 mtx-changer.conf so that all configuration will be done by
6252 changing only mtx-changer.conf.
6253 \item The new \texttt{mtx-changer} script has two new options, \texttt{listall}
6254 and \texttt{transfer}. Please configure them as appropriate
6255 in mtx-changer.conf.
6256 \item To enhance security of the \texttt{BackupCatalog} job, we provide a new
6257 script (\texttt{make\_catalog\_backup.pl}) that does not expose your catalog
6258 password. If you want to use the new script, you will need to
6259 manually change the \texttt{BackupCatalog} Job definition.
6260 \item The \texttt{bconsole} \texttt{help} command now accepts
6261 an argument, which if provided produces information on that
6262 command (ex: \texttt{help run}).
6263 \end{itemize}
6264
6265
6266 \subsubsection*{Truncate volume after purge}
6267
6268 Note that the Truncate Volume after purge feature doesn't work as expected
6269 in 5.0.0 version. Please, don't use it before version 5.0.1.
6270
6271 \subsubsection{Custom Catalog queries}
6272
6273 If you wish to add specialized commands that list the contents of the catalog,
6274 you can do so by adding them to the \texttt{query.sql} file. This
6275 \texttt{query.sql} file is now empty by default. The file
6276 \texttt{examples/sample-query.sql} has an a number of sample commands
6277 you might find useful.
6278
6279 \subsubsection{Deprecated parts}
6280
6281 The following items have been \textbf{deprecated} for a long time, and are now
6282 removed from the code.
6283 \begin{itemize}
6284 \item Gnome console
6285 \item Support for SQLite. Note the source code still exists, but the
6286 project does not maintain it. Please do not use it.
6287 \end{itemize}
6288
6289 \subsection{Misc Changes}
6290 \label{sec:miscchanges}
6291
6292 \begin{itemize}
6293 \item Updated Nagios check\_bacula
6294 \item Updated man files
6295 \item Added OSX package generation script in platforms/darwin
6296 \item Added Spanish and Ukrainian Bacula translations
6297 \item Enable/disable command shows only Jobs that can change
6298 \item Added \texttt{show disabled} command to show disabled Jobs
6299 \item Many ACL improvements
6300 \item Added Level to FD status Job output
6301 \item Begin Ingres DB driver (not yet working)
6302 \item Split RedHat spec files into bacula, bat, mtx, and docs
6303 \item Reorganized the manuals (fewer separate manuals)
6304 \item Added lock/unlock order protection in lock manager
6305 \item Allow 64 bit sizes for a number of variables
6306 \item Fixed several deadlocks or potential race conditions in the SD
6307 \end{itemize}
6308
6309 \chapter{Released Version 3.0.3 and 3.0.3a}
6310
6311 There are no new features in version 3.0.3. This version simply fixes a
6312 number of bugs found in version 3.0.2 during the ongoing development
6313 process.
6314
6315 \section{New Features in Released Version 3.0.2}
6316
6317 This chapter presents the new features added to the
6318 Released Bacula Version 3.0.2.
6319
6320 \subsection{Full Restore from a Given JobId}
6321 \index[general]{Restore menu}
6322
6323 This feature allows selecting a single JobId and having Bacula
6324 automatically select all the other jobs that comprise a full backup up to
6325 and including the selected date (through JobId).
6326
6327 Assume we start with the following jobs:
6328 \begin{bVerbatim}
6329 +-------+--------------+---------------------+-------+----------+------------+
6330 | jobid | client | starttime | level | jobfiles | jobbytes |
6331 +-------+--------------+---------------------+-------+----------+------------
6332 | 6 | localhost-fd | 2009-07-15 11:45:49 | I | 2 | 0 |
6333 | 5 | localhost-fd | 2009-07-15 11:45:45 | I | 15 | 44143 |
6334 | 3 | localhost-fd | 2009-07-15 11:45:38 | I | 1 | 10 |
6335 | 1 | localhost-fd | 2009-07-15 11:45:30 | F | 1527 | 44143073 |
6336 +-------+--------------+---------------------+-------+----------+------------+
6337 \end{bVerbatim}
6338
6339 Below is an example of this new feature (which is number 12 in the
6340 menu).
6341
6342 \begin{bVerbatim}
6343 * restore
6344 To select the JobIds, you have the following choices:
6345 1: List last 20 Jobs run
6346 2: List Jobs where a given File is saved
6347 ...
6348 12: Select full restore to a specified Job date
6349 13: Cancel
6350
6351 Select item: (1-13): 12
6352 Enter JobId to get the state to restore: 5
6353 Selecting jobs to build the Full state at 2009-07-15 11:45:45
6354 You have selected the following JobIds: 1,3,5
6355
6356 Building directory tree for JobId(s) 1,3,5 ... +++++++++++++++++++
6357 1,444 files inserted into the tree.
6358 \end{bVerbatim}
6359
6360 This project was funded by Bacula Systems.
6361
6362 \subsection{Source Address}
6363 \index[general]{Source Address}
6364
6365 A feature has been added which allows the administrator to specify the address
6366 from which the Director and File daemons will establish connections. This
6367 may be used to simplify system configuration overhead when working in complex
6368 networks utilizing multi-homing and policy-routing.
6369
6370 To accomplish this, two new configuration directives have been implemented:
6371 \begin{bVerbatim}
6372 FileDaemon {
6373 FDSourceAddress=10.0.1.20 # Always initiate connections from this address
6374 }
6375
6376 Director {
6377 DirSourceAddress=10.0.1.10 # Always initiate connections from this address
6378 }
6379 \end{bVerbatim}
6380
6381 Simply adding specific host routes on the OS
6382 would have an undesirable side-effect: any
6383 application trying to contact the destination host would be forced to use the
6384 more specific route possibly diverting management traffic onto a backup VLAN.
6385 Instead of adding host routes for each client connected to a multi-homed backup
6386 server (for example where there are management and backup VLANs), one can
6387 use the new directives to specify a specific source address at the application
6388 level.
6389
6390 Additionally, this allows the simplification and abstraction of firewall rules
6391 when dealing with a Hot-Standby director or storage daemon configuration. The
6392 Hot-standby pair may share a CARP address, which connections must be sourced
6393 from, while system services listen and act from the unique interface addresses.
6394
6395 This project was funded by Collaborative Fusion, Inc.
6396
6397 \subsection{Show volume availability when doing restore}
6398
6399 When doing a restore the selection dialog ends by displaying this
6400 screen:
6401
6402 \begin{bVerbatim}
6403 The job will require the following
6404 Volume(s) Storage(s) SD Device(s)
6405 ===========================================================================
6406 *000741L3 LTO-4 LTO3
6407 *000866L3 LTO-4 LTO3
6408 *000765L3 LTO-4 LTO3
6409 *000764L3 LTO-4 LTO3
6410 *000756L3 LTO-4 LTO3
6411 *001759L3 LTO-4 LTO3
6412 *001763L3 LTO-4 LTO3
6413 001762L3 LTO-4 LTO3
6414 001767L3 LTO-4 LTO3
6415
6416 Volumes marked with ``*'' are online (in the autochanger).
6417 \end{bVerbatim}
6418
6419 This should help speed up large restores by minimizing the time spent
6420 waiting for the operator to discover that he must change tapes in the library.
6421
6422 This project was funded by Bacula Systems.
6423
6424 \subsection{Accurate estimate command}
6425
6426 The \texttt{estimate} command can now use the accurate code to detect changes
6427 and give a better estimation.
6428
6429 You can set the accurate behavior on the command line by using
6430 \texttt{accurate=yes\vb{}no} or use the Job setting as default value.
6431
6432 \begin{bVerbatim}
6433 * estimate listing accurate=yes level=incremental job=BackupJob
6434 \end{bVerbatim}
6435
6436 This project was funded by Bacula Systems.
6437
6438 \section{New Features in 3.0.0}
6439 \label{NewFeaturesChapter}
6440 \index[general]{New Features}
6441
6442 This chapter presents the new features added to the development 2.5.x
6443 versions to be released as Bacula version 3.0.0 sometime in April 2009.
6444
6445 \subsection{Accurate Backup}
6446 \index[general]{Accurate Backup}
6447
6448 As with most other backup programs, by default Bacula decides what files to
6449 backup for Incremental and Differential backup by comparing the change
6450 (st\_ctime) and modification (st\_mtime) times of the file to the time the last
6451 backup completed. If one of those two times is later than the last backup
6452 time, then the file will be backed up. This does not, however, permit tracking
6453 what files have been deleted and will miss any file with an old time that may
6454 have been restored to or moved onto the client filesystem.
6455
6456 \subsubsection{Accurate = \lt{}yes\vb{}no\gt{}}
6457 If the {\bf Accurate = \lt{}yes\vb{}no\gt{}} directive is enabled (default no) in
6458 the Job resource, the job will be run as an Accurate Job. For a {\bf Full}
6459 backup, there is no difference, but for {\bf Differential} and {\bf
6460 Incremental} backups, the Director will send a list of all previous files
6461 backed up, and the File daemon will use that list to determine if any new files
6462 have been added or or moved and if any files have been deleted. This allows
6463 Bacula to make an accurate backup of your system to that point in time so that
6464 if you do a restore, it will restore your system exactly.
6465
6466 One note of caution
6467 about using Accurate backup is that it requires more resources (CPU and memory)
6468 on both the Director and the Client machines to create the list of previous
6469 files backed up, to send that list to the File daemon, for the File daemon to
6470 keep the list (possibly very big) in memory, and for the File daemon to do
6471 comparisons between every file in the FileSet and the list. In particular,
6472 if your client has lots of files (more than a few million), you will need
6473 lots of memory on the client machine.
6474
6475 Accurate must not be enabled when backing up with a plugin that is not
6476 specially designed to work with Accurate. If you enable it, your restores
6477 will probably not work correctly.
6478
6479 This project was funded by Bacula Systems.
6480
6481
6482
6483 \subsection{Copy Jobs}
6484 \index[general]{Copy Jobs}
6485
6486 A new {\bf Copy} job type 'C' has been implemented. It is similar to the
6487 existing Migration feature with the exception that the Job that is copied is
6488 left unchanged. This essentially creates two identical copies of the same
6489 backup. However, the copy is treated as a copy rather than a backup job, and
6490 hence is not directly available for restore. The {\bf restore} command lists
6491 copy jobs and allows selection of copies by using \texttt{jobid=}
6492 option. If the keyword {\bf copies} is present on the command line, Bacula will
6493 display the list of all copies for selected jobs.
6494
6495 \begin{bVerbatim}
6496 * restore copies
6497 [...]
6498 These JobIds have copies as follows:
6499 +-------+------------------------------------+-----------+------------------+
6500 | JobId | Job | CopyJobId | MediaType |
6501 +-------+------------------------------------+-----------+------------------+
6502 | 2 | CopyJobSave.2009-02-17_16.31.00.11 | 7 | DiskChangerMedia |
6503 +-------+------------------------------------+-----------+------------------+
6504 +-------+-------+----------+----------+---------------------+------------------+
6505 | JobId | Level | JobFiles | JobBytes | StartTime | VolumeName |
6506 +-------+-------+----------+----------+---------------------+------------------+
6507 | 19 | F | 6274 | 76565018 | 2009-02-17 16:30:45 | ChangerVolume002 |
6508 | 2 | I | 1 | 5 | 2009-02-17 16:30:51 | FileVolume001 |
6509 +-------+-------+----------+----------+---------------------+------------------+
6510 You have selected the following JobIds: 19,2
6511
6512 Building directory tree for JobId(s) 19,2 ... ++++++++++++++++++++++++++++++++++++++++++++
6513 5,611 files inserted into the tree.
6514 ...
6515 \end{bVerbatim}
6516
6517
6518 The Copy Job runs without using the File daemon by copying the data from the
6519 old backup Volume to a different Volume in a different Pool. See the Migration
6520 documentation for additional details. For copy Jobs there is a new selection
6521 directive named {\bf PoolUncopiedJobs} which selects all Jobs that were
6522 not already copied to another Pool.
6523
6524 As with Migration, the Client, Volume, Job, or SQL query, are
6525 other possible ways of selecting the Jobs to be copied. Selection
6526 types like SmallestVolume, OldestVolume, PoolOccupancy and PoolTime also
6527 work, but are probably more suited for Migration Jobs.
6528
6529 If Bacula finds a Copy of a job record that is purged (deleted) from the catalog,
6530 it will promote the Copy to a \textsl{real} backup job and will make it available for
6531 automatic restore. If more than one Copy is available, it will promote the copy
6532 with the smallest JobId.
6533
6534 A nice solution which can be built with the new Copy feature is often
6535 called disk-to-disk-to-tape backup (DTDTT). A sample config could
6536 look something like the one below:
6537
6538 \begin{bVerbatim}
6539 Pool {
6540 Name = FullBackupsVirtualPool
6541 Pool Type = Backup
6542 Purge Oldest Volume = Yes
6543 Storage = vtl
6544 NextPool = FullBackupsTapePool
6545 }
6546
6547 Pool {
6548 Name = FullBackupsTapePool
6549 Pool Type = Backup
6550 Recycle = Yes
6551 AutoPrune = Yes
6552 Volume Retention = 365 days
6553 Storage = superloader
6554 }
6555
6556 #
6557 # Fake fileset for copy jobs
6558 #
6559 Fileset {
6560 Name = None
6561 Include {
6562 Options {
6563 signature = MD5
6564 }
6565 }
6566 }
6567
6568 #
6569 # Fake client for copy jobs
6570 #
6571 Client {
6572 Name = None
6573 Address = localhost
6574 Password = "NoNe"
6575 Catalog = MyCatalog
6576 }
6577
6578 #
6579 # Default template for a CopyDiskToTape Job
6580 #
6581 JobDefs {
6582 Name = CopyDiskToTape
6583 Type = Copy
6584 Messages = StandardCopy
6585 Client = None
6586 FileSet = None
6587 Selection Type = PoolUncopiedJobs
6588 Maximum Concurrent Jobs = 10
6589 SpoolData = No
6590 Allow Duplicate Jobs = Yes
6591 Cancel Queued Duplicates = No
6592 Cancel Running Duplicates = No
6593 Priority = 13
6594 }
6595
6596 Schedule {
6597 Name = DaySchedule7:00
6598 Run = Level=Full daily at 7:00
6599 }
6600
6601 Job {
6602 Name = CopyDiskToTapeFullBackups
6603 Enabled = Yes
6604 Schedule = DaySchedule7:00
6605 Pool = FullBackupsVirtualPool
6606 JobDefs = CopyDiskToTape
6607 }
6608 \end{bVerbatim}
6609
6610 The example above had 2 pool which are copied using the PoolUncopiedJobs
6611 selection criteria. Normal Full backups go to the Virtual pool and are copied
6612 to the Tape pool the next morning.
6613
6614 The command \texttt{list copies [jobid=x,y,z]} lists copies for a given
6615 \textbf{jobid}.
6616
6617 \begin{bVerbatim}
6618 *list copies
6619 +-------+------------------------------------+-----------+------------------+
6620 | JobId | Job | CopyJobId | MediaType |
6621 +-------+------------------------------------+-----------+------------------+
6622 | 9 | CopyJobSave.2008-12-20_22.26.49.05 | 11 | DiskChangerMedia |
6623 +-------+------------------------------------+-----------+------------------+
6624 \end{bVerbatim}
6625
6626 \subsection{ACL Updates}
6627 \index[general]{ACL Updates}
6628 The whole ACL code had been overhauled and in this version each platforms has
6629 different streams for each type of acl available on such an platform. As ACLs
6630 between platforms tend to be not that portable (most implement POSIX acls but
6631 some use an other draft or a completely different format) we currently only
6632 allow certain platform specific ACL streams to be decoded and restored on the
6633 same platform that they were created on. The old code allowed to restore ACL
6634 cross platform but the comments already mention that not being to wise. For
6635 backward compatibility the new code will accept the two old ACL streams and
6636 handle those with the platform specific handler. But for all new backups it
6637 will save the ACLs using the new streams.
6638
6639 Currently the following platforms support ACLs:
6640
6641 \begin{itemize}
6642 \item {\bf AIX}
6643 \item {\bf Darwin/OSX}
6644 \item {\bf FreeBSD}
6645 \item {\bf HPUX}
6646 \item {\bf IRIX}
6647 \item {\bf Linux}
6648 \item {\bf Tru64}
6649 \item {\bf Solaris}
6650 \end{itemize}
6651
6652 Currently we support the following ACL types (these ACL streams use a reserved
6653 part of the stream numbers):
6654
6655 \begin{itemize}
6656 \item {\bf STREAM\_ACL\_AIX\_TEXT} 1000 AIX specific string representation from
6657 acl\_get
6658 \item {\bf STREAM\_ACL\_DARWIN\_ACCESS\_ACL} 1001 Darwin (OSX) specific acl\_t
6659 string representation from acl\_to\_text (POSIX acl)
6660 \item {\bf STREAM\_ACL\_FREEBSD\_DEFAULT\_ACL} 1002 FreeBSD specific acl\_t
6661 string representation from acl\_to\_text (POSIX acl) for default acls.
6662 \item {\bf STREAM\_ACL\_FREEBSD\_ACCESS\_ACL} 1003 FreeBSD specific acl\_t
6663 string representation from acl\_to\_text (POSIX acl) for access acls.
6664 \item {\bf STREAM\_ACL\_HPUX\_ACL\_ENTRY} 1004 HPUX specific acl\_entry
6665 string representation from acltostr (POSIX acl)
6666 \item {\bf STREAM\_ACL\_IRIX\_DEFAULT\_ACL} 1005 IRIX specific acl\_t string
6667 representation from acl\_to\_text (POSIX acl) for default acls.
6668 \item {\bf STREAM\_ACL\_IRIX\_ACCESS\_ACL} 1006 IRIX specific acl\_t string
6669 representation from acl\_to\_text (POSIX acl) for access acls.
6670 \item {\bf STREAM\_ACL\_LINUX\_DEFAULT\_ACL} 1007 Linux specific acl\_t
6671 string representation from acl\_to\_text (POSIX acl) for default acls.
6672 \item {\bf STREAM\_ACL\_LINUX\_ACCESS\_ACL} 1008 Linux specific acl\_t string
6673 representation from acl\_to\_text (POSIX acl) for access acls.
6674 \item {\bf STREAM\_ACL\_TRU64\_DEFAULT\_ACL} 1009 Tru64 specific acl\_t
6675 string representation from acl\_to\_text (POSIX acl) for default acls.
6676 \item {\bf STREAM\_ACL\_TRU64\_DEFAULT\_DIR\_ACL} 1010 Tru64 specific acl\_t
6677 string representation from acl\_to\_text (POSIX acl) for default acls.
6678 \item {\bf STREAM\_ACL\_TRU64\_ACCESS\_ACL} 1011 Tru64 specific acl\_t string
6679 representation from acl\_to\_text (POSIX acl) for access acls.
6680 \item {\bf STREAM\_ACL\_SOLARIS\_ACLENT} 1012 Solaris specific aclent\_t
6681 string representation from acltotext or acl\_totext (POSIX acl)
6682 \item {\bf STREAM\_ACL\_SOLARIS\_ACE} 1013 Solaris specific ace\_t string
6683 representation from from acl\_totext (NFSv4 or ZFS acl)
6684 \end{itemize}
6685
6686 In future versions we might support conversion functions from one type of acl
6687 into an other for types that are either the same or easily convertible. For now
6688 the streams are separate and restoring them on a platform that doesn't
6689 recognize them will give you a warning.
6690
6691 \subsection{Extended Attributes}
6692 \index[general]{Extended Attributes}
6693 Something that was on the project list for some time is now implemented for
6694 platforms that support a similar kind of interface. Its the support for backup
6695 and restore of so called extended attributes. As extended attributes are so
6696 platform specific these attributes are saved in separate streams for each
6697 platform. Restores of the extended attributes can only be performed on the
6698 same platform the backup was done. There is support for all types of extended
6699 attributes, but restoring from one type of filesystem onto an other type of
6700 filesystem on the same platform may lead to surprises. As extended attributes
6701 can contain any type of data they are stored as a series of so called
6702 value-pairs. This data must be seen as mostly binary and is stored as such.
6703 As security labels from selinux are also extended attributes this option also
6704 stores those labels and no specific code is enabled for handling selinux
6705 security labels.
6706
6707 Currently the following platforms support extended attributes:
6708 \begin{itemize}
6709 \item {\bf Darwin/OSX}
6710 \item {\bf FreeBSD}
6711 \item {\bf Linux}
6712 \item {\bf NetBSD}
6713 \end{itemize}
6714
6715 On Linux acls are also extended attributes, as such when you enable ACLs on a
6716 Linux platform it will NOT save the same data twice e.g. it will save the ACLs
6717 and not the same extended attribute.
6718
6719 To enable the backup of extended attributes please add the following to your
6720 fileset definition.
6721 \begin{bVerbatim}
6722 FileSet {
6723 Name = "MyFileSet"
6724 Include {
6725 Options {
6726 signature = MD5
6727 xattrsupport = yes
6728 }
6729 File = ...
6730 }
6731 }
6732 \end{bVerbatim}
6733
6734 \subsection{Shared objects}
6735 \index[general]{Shared objects}
6736 A default build of Bacula will now create the libraries as shared objects
6737 (.so) rather than static libraries as was previously the case.
6738 The shared libraries are built using {\bf libtool} so it should be quite
6739 portable.
6740
6741 An important advantage of using shared objects is that on a machine with the
6742 Directory, File daemon, the Storage daemon, and a console, you will have only
6743 one copy of the code in memory rather than four copies. Also the total size of
6744 the binary release is smaller since the library code appears only once rather
6745 than once for every program that uses it; this results in significant reduction
6746 in the size of the binaries particularly for the utility tools.
6747
6748 In order for the system loader to find the shared objects when loading the
6749 Bacula binaries, the Bacula shared objects must either be in a shared object
6750 directory known to the loader (typically /usr/lib) or they must be in the
6751 directory that may be specified on the {\bf ./configure} line using the {\bf
6752 {-}{-}libdir} option as:
6753
6754 \begin{bVerbatim}
6755 ./configure --libdir=/full-path/dir
6756 \end{bVerbatim}
6757
6758 the default is /usr/lib. If {-}{-}libdir is specified, there should be
6759 no need to modify your loader configuration provided that
6760 the shared objects are installed in that directory (Bacula
6761 does this with the make install command). The shared objects
6762 that Bacula references are:
6763
6764 \begin{bVerbatim}
6765 libbaccfg.so
6766 libbacfind.so
6767 libbacpy.so
6768 libbac.so
6769 \end{bVerbatim}
6770
6771 These files are symbolically linked to the real shared object file,
6772 which has a version number to permit running multiple versions of
6773 the libraries if desired (not normally the case).
6774
6775 If you have problems with libtool or you wish to use the old
6776 way of building static libraries, or you want to build a static
6777 version of Bacula you may disable
6778 libtool on the configure command line with:
6779
6780 \begin{bVerbatim}
6781 ./configure --disable-libtool
6782 \end{bVerbatim}
6783
6784
6785 \subsection{Building Static versions of Bacula}
6786 \index[general]{Static linking}
6787 In order to build static versions of Bacula, in addition
6788 to configuration options that were needed you now must
6789 also add --disable-libtool. Example
6790
6791 \begin{bVerbatim}
6792 ./configure --enable-static-client-only --disable-libtool
6793 \end{bVerbatim}
6794
6795
6796 \subsection{Virtual Backup (Vbackup)}
6797 \index[general]{Virtual Backup}
6798 \index[general]{Vbackup}
6799
6800 Bacula's virtual backup feature is often called Synthetic Backup or
6801 Consolidation in other backup products. It permits you to consolidate the
6802 previous Full backup plus the most recent Differential backup and any
6803 subsequent Incremental backups into a new Full backup. This new Full
6804 backup will then be considered as the most recent Full for any future
6805 Incremental or Differential backups. The VirtualFull backup is
6806 accomplished without contacting the client by reading the previous backup
6807 data and writing it to a volume in a different pool.
6808
6809 In some respects the Vbackup feature works similar to a Migration job, in
6810 that Bacula normally reads the data from the pool specified in the
6811 Job resource, and writes it to the {\bf Next Pool} specified in the
6812 Job resource. Note, this means that usually the output from the Virtual
6813 Backup is written into a different pool from where your prior backups
6814 are saved. Doing it this way guarantees that you will not get a deadlock
6815 situation attempting to read and write to the same volume in the Storage
6816 daemon. If you then want to do subsequent backups, you may need to
6817 move the Virtual Full Volume back to your normal backup pool.
6818 Alternatively, you can set your {\bf Next Pool} to point to the current
6819 pool. This will cause Bacula to read and write to Volumes in the
6820 current pool. In general, this will work, because Bacula will
6821 not allow reading and writing on the same Volume. In any case, once
6822 a VirtualFull has been created, and a restore is done involving the
6823 most current Full, it will read the Volume or Volumes by the VirtualFull
6824 regardless of in which Pool the Volume is found.
6825
6826 The Vbackup is enabled on a Job by Job in the Job resource by specifying
6827 a level of {\bf VirtualFull}.
6828
6829 A typical Job resource definition might look like the following:
6830
6831 \begin{bVerbatim}
6832 Job {
6833 Name = "MyBackup"
6834 Type = Backup
6835 Client=localhost-fd
6836 FileSet = "Full Set"
6837 Storage = File
6838 Messages = Standard
6839 Pool = Default
6840 SpoolData = yes
6841 }
6842
6843 # Default pool definition
6844 Pool {
6845 Name = Default
6846 Pool Type = Backup
6847 Recycle = yes # Automatically recycle Volumes
6848 AutoPrune = yes # Prune expired volumes
6849 Volume Retention = 365d # one year
6850 NextPool = Full
6851 Storage = File
6852 }
6853
6854 Pool {
6855 Name = Full
6856 Pool Type = Backup
6857 Recycle = yes # Automatically recycle Volumes
6858 AutoPrune = yes # Prune expired volumes
6859 Volume Retention = 365d # one year
6860 Storage = DiskChanger
6861 }
6862
6863 # Definition of file storage device
6864 Storage {
6865 Name = File
6866 Address = localhost
6867 Password = "xxx"
6868 Device = FileStorage
6869 Media Type = File
6870 Maximum Concurrent Jobs = 5
6871 }
6872
6873 # Definition of DDS Virtual tape disk storage device
6874 Storage {
6875 Name = DiskChanger
6876 Address = localhost # N.B. Use a fully qualified name here
6877 Password = "yyy"
6878 Device = DiskChanger
6879 Media Type = DiskChangerMedia
6880 Maximum Concurrent Jobs = 4
6881 Autochanger = yes
6882 }
6883 \end{bVerbatim}
6884
6885 Then in bconsole or via a Run schedule, you would run the job as:
6886
6887 \begin{bVerbatim}
6888 run job=MyBackup level=Full
6889 run job=MyBackup level=Incremental
6890 run job=MyBackup level=Differential
6891 run job=MyBackup level=Incremental
6892 run job=MyBackup level=Incremental
6893 \end{bVerbatim}
6894
6895 So providing there were changes between each of those jobs, you would end up
6896 with a Full backup, a Differential, which includes the first Incremental
6897 backup, then two Incremental backups. All the above jobs would be written to
6898 the {\bf Default} pool.
6899
6900 To consolidate those backups into a new Full backup, you would run the
6901 following:
6902
6903 \begin{bVerbatim}
6904 run job=MyBackup level=VirtualFull
6905 \end{bVerbatim}
6906
6907 And it would produce a new Full backup without using the client, and the output
6908 would be written to the {\bf Full} Pool which uses the Diskchanger Storage.
6909
6910 If the Virtual Full is run, and there are no prior Jobs, the Virtual Full will
6911 fail with an error.
6912
6913 Note, the Start and End time of the Virtual Full backup is set to the
6914 values for the last job included in the Virtual Full (in the above example,
6915 it is an Increment). This is so that if another incremental is done, which
6916 will be based on the Virtual Full, it will backup all files from the
6917 last Job included in the Virtual Full rather than from the time the Virtual
6918 Full was actually run.
6919
6920
6921
6922 \subsection{Catalog Format}
6923 \index[general]{Catalog Format}
6924 Bacula 3.0 comes with some changes to the catalog format. The upgrade
6925 operation will convert the FileId field of the File table from 32 bits (max 4
6926 billion table entries) to 64 bits (very large number of items). The
6927 conversion process can take a bit of time and will likely DOUBLE THE SIZE of
6928 your catalog during the conversion. Also you won't be able to run jobs during
6929 this conversion period. For example, a 3 million file catalog will take 2
6930 minutes to upgrade on a normal machine. Please don't forget to make a valid
6931 backup of your database before executing the upgrade script. See the
6932 ReleaseNotes for additional details.
6933
6934 \subsection{64 bit Windows Client}
6935 \index[general]{Win64 Client}
6936 Unfortunately, Microsoft's implementation of Volume Shadown Copy (VSS) on
6937 their 64 bit OS versions is not compatible with a 32 bit Bacula Client.
6938 As a consequence, we are also releasing a 64 bit version of the Bacula
6939 Windows Client (win64bacula-3.0.0.exe) that does work with VSS.
6940 These binaries should only be installed on 64 bit Windows operating systems.
6941 What is important is not your hardware but whether or not you have
6942 a 64 bit version of the Windows OS.
6943
6944 Compared to the Win32 Bacula Client, the 64 bit release contains a few differences:
6945 \begin{enumerate}
6946 \item Before installing the Win64 Bacula Client, you must totally
6947 deinstall any prior 2.4.x Client installation using the
6948 Bacula deinstallation (see the menu item). You may want
6949 to save your .conf files first.
6950 \item Only the Client (File daemon) is ported to Win64, the Director
6951 and the Storage daemon are not in the 64 bit Windows installer.
6952 \item bwx-console is not yet ported.
6953 \item bconsole is ported but it has not been tested.
6954 \item The documentation is not included in the installer.
6955 \item Due to Vista security restrictions imposed on a default installation
6956 of Vista, before upgrading the Client, you must manually stop
6957 any prior version of Bacula from running, otherwise the install
6958 will fail.
6959 \item Due to Vista security restrictions imposed on a default installation
6960 of Vista, attempting to edit the conf files via the menu items
6961 will fail. You must directly edit the files with appropriate
6962 permissions. Generally double clicking on the appropriate .conf
6963 file will work providing you have sufficient permissions.
6964 \item All Bacula files are now installed in
6965 {\bf C:/Program Files/Bacula} except the main menu items,
6966 which are installed as before. This vastly simplifies the installation.
6967 \item If you are running on a foreign language version of Windows, most
6968 likely {\bf C:/Program Files} does not exist, so you should use the
6969 Custom installation and enter an appropriate location to install
6970 the files.
6971 \item The 3.0.0 Win32 Client continues to install files in the locations used
6972 by prior versions. For the next version we will convert it to use
6973 the same installation conventions as the Win64 version.
6974 \end{enumerate}
6975
6976 This project was funded by Bacula Systems.
6977
6978
6979 \subsection{Duplicate Job Control}
6980 \index[general]{Duplicate Jobs}
6981 The new version of Bacula provides four new directives that
6982 give additional control over what Bacula does if duplicate jobs
6983 are started. A duplicate job in the sense we use it here means
6984 a second or subsequent job with the same name starts. This
6985 happens most frequently when the first job runs longer than expected because no
6986 tapes are available.
6987
6988 The four directives each take as an argument a {\bf yes} or {\bf no} value and
6989 are specified in the Job resource.
6990
6991 They are:
6992
6993 \subsubsection{Allow Duplicate Jobs = \lt{}yes\vb{}no\gt{}}
6994 \index[general]{Allow Duplicate Jobs}
6995 If this directive is set to {\bf yes}, duplicate jobs will be run. If
6996 the directive is set to {\bf no} (default) then only one job of a given name
6997 may run at one time, and the action that Bacula takes to ensure only
6998 one job runs is determined by the other directives (see below).
6999
7000 If {\bf Allow Duplicate Jobs} is set to {\bf no} and two jobs
7001 are present and none of the three directives given below permit
7002 Canceling a job, then the current job (the second one started)
7003 will be canceled.
7004
7005 \subsubsection{Allow Higher Duplicates = \lt{}yes\vb{}no\gt{}}
7006 \index[general]{Allow Higher Duplicates}
7007 This directive was in version 5.0.0, but does not work as
7008 expected. If used, it should always be set to no. In later versions
7009 of Bacula the directive is disabled (disregarded).
7010
7011 \subsubsection{Cancel Running Duplicates = \lt{}yes\vb{}no\gt{}}
7012 \index[general]{Cancel Running Duplicates}
7013 If {\bf Allow Duplicate Jobs} is set to {\bf no} and
7014 if this directive is set to {\bf yes} any job that is already running
7015 will be canceled. The default is {\bf no}.
7016
7017 \subsubsection{Cancel Queued Duplicates = \lt{}yes\vb{}no\gt{}}
7018 \index[general]{Cancel Queued Duplicates}
7019 If {\bf Allow Duplicate Jobs} is set to {\bf no} and
7020 if this directive is set to {\bf yes} any job that is
7021 already queued to run but not yet running will be canceled.
7022 The default is {\bf no}.
7023
7024
7025 \subsection{TLS Authentication}
7026 \index[general]{TLS Authentication}
7027 In Bacula version 2.5.x and later, in addition to the normal Bacula
7028 CRAM-MD5 authentication that is used to authenticate each Bacula
7029 connection, you can specify that you want TLS Authentication as well,
7030 which will provide more secure authentication.
7031
7032 This new feature uses Bacula's existing TLS code (normally used for
7033 communications encryption) to do authentication. To use it, you must
7034 specify all the TLS directives normally used to enable communications
7035 encryption (TLS Enable, TLS Verify Peer, TLS Certificate, ...) and
7036 a new directive:
7037
7038 \subsubsection{TLS Authenticate = yes}
7039 \begin{bVerbatim}
7040 TLS Authenticate = yes
7041 \end{bVerbatim}
7042
7043 in the main daemon configuration resource (Director for the Director,
7044 Client for the File daemon, and Storage for the Storage daemon).
7045
7046 When {\bf TLS Authenticate} is enabled, after doing the CRAM-MD5
7047 authentication, Bacula will also do TLS authentication, then TLS
7048 encryption will be turned off, and the rest of the communication between
7049 the two Bacula daemons will be done without encryption.
7050
7051 If you want to encrypt communications data, use the normal TLS directives
7052 but do not turn on {\bf TLS Authenticate}.
7053
7054 \subsection{bextract non-portable Win32 data}
7055 \index[general]{bextract handles Win32 non-portable data}
7056 {\bf bextract} has been enhanced to be able to restore
7057 non-portable Win32 data to any OS. Previous versions were
7058 unable to restore non-portable Win32 data to machines that
7059 did not have the Win32 BackupRead and BackupWrite API calls.
7060
7061 \subsection{State File updated at Job Termination}
7062 \index[general]{State File}
7063 In previous versions of Bacula, the state file, which provides a
7064 summary of previous jobs run in the {\bf status} command output was
7065 updated only when Bacula terminated, thus if the daemon crashed, the
7066 state file might not contain all the run data. This version of
7067 the Bacula daemons updates the state file on each job termination.
7068
7069 \subsection{MaxFullInterval = \lt{}time-interval\gt{}}
7070 \index[general]{MaxFullInterval}
7071 The new Job resource directive {\bf Max Full Interval = \lt{}time-interval\gt{}}
7072 can be used to specify the maximum time interval between {\bf Full} backup
7073 jobs. When a job starts, if the time since the last Full backup is
7074 greater than the specified interval, and the job would normally be an
7075 {\bf Incremental} or {\bf Differential}, it will be automatically
7076 upgraded to a {\bf Full} backup.
7077
7078 \subsection{MaxDiffInterval = \lt{}time-interval\gt{}}
7079 \index[general]{MaxDiffInterval}
7080 The new Job resource directive {\bf Max Diff Interval = \lt{}time-interval\gt{}}
7081 can be used to specify the maximum time interval between {\bf Differential} backup
7082 jobs. When a job starts, if the time since the last Differential backup is
7083 greater than the specified interval, and the job would normally be an
7084 {\bf Incremental}, it will be automatically
7085 upgraded to a {\bf Differential} backup.
7086
7087 \subsection{Honor No Dump Flag = \lt{}yes\vb{}no\gt{}}
7088 \index[general]{MaxDiffInterval}
7089 On FreeBSD systems, each file has a {\bf no dump flag} that can be set
7090 by the user, and when it is set it is an indication to backup programs
7091 to not backup that particular file. This version of Bacula contains a
7092 new Options directive within a FileSet resource, which instructs Bacula to
7093 obey this flag. The new directive is:
7094
7095 \begin{bVerbatim}
7096 Honor No Dump Flag = yes\vb{}no
7097 \end{bVerbatim}
7098
7099 The default value is {\bf no}.
7100
7101
7102 \subsection{Exclude Dir Containing = \lt{}filename-string\gt{}}
7103 \index[general]{IgnoreDir}
7104 The {\bf ExcludeDirContaining = \lt{}filename\gt{}} is a new directive that
7105 can be added to the Include section of the FileSet resource. If the specified
7106 filename ({\bf filename-string}) is found on the Client in any directory to be
7107 backed up, the whole directory will be ignored (not backed up). For example:
7108
7109 \begin{bVerbatim}
7110 # List of files to be backed up
7111 FileSet {
7112 Name = "MyFileSet"
7113 Include {
7114 Options {
7115 signature = MD5
7116 }
7117 File = /home
7118 Exclude Dir Containing = .excludeme
7119 }
7120 }
7121 \end{bVerbatim}
7122
7123 But in /home, there may be hundreds of directories of users and some
7124 people want to indicate that they don't want to have certain
7125 directories backed up. For example, with the above FileSet, if
7126 the user or sysadmin creates a file named {\bf .excludeme} in
7127 specific directories, such as
7128
7129 \begin{bVerbatim}
7130 /home/user/www/cache/.excludeme
7131 /home/user/temp/.excludeme
7132 \end{bVerbatim}
7133
7134 then Bacula will not backup the two directories named:
7135
7136 \begin{bVerbatim}
7137 /home/user/www/cache
7138 /home/user/temp
7139 \end{bVerbatim}
7140
7141 NOTE: subdirectories will not be backed up. That is, the directive
7142 applies to the two directories in question and any children (be they
7143 files, directories, etc).
7144
7145
7146 \subsection{Bacula Plugins}
7147 \index[general]{Plugin}
7148 Support for shared object plugins has been implemented in the Linux, Unix
7149 and Win32 File daemons. The API will be documented separately in
7150 the Developer's Guide or in a new document. For the moment, there is
7151 a single plugin named {\bf bpipe} that allows an external program to
7152 get control to backup and restore a file.
7153
7154 Plugins are also planned (partially implemented) in the Director and the
7155 Storage daemon.
7156
7157 \subsubsection{Plugin Directory}
7158 \index[general]{Plugin Directory}
7159 Each daemon (DIR, FD, SD) has a new {\bf Plugin Directory} directive that may
7160 be added to the daemon definition resource. The directory takes a quoted
7161 string argument, which is the name of the directory in which the daemon can
7162 find the Bacula plugins. If this directive is not specified, Bacula will not
7163 load any plugins. Since each plugin has a distinctive name, all the daemons
7164 can share the same plugin directory.
7165
7166 \subsubsection{Plugin Options}
7167 \index[general]{Plugin Options}
7168 The {\bf Plugin Options} directive takes a quoted string
7169 argument (after the equal sign) and may be specified in the
7170 Job resource. The options specified will be passed to all plugins
7171 when they are run. This each plugin must know what it is looking
7172 for. The value defined in the Job resource can be modified
7173 by the user when he runs a Job via the {\bf bconsole} command line
7174 prompts.
7175
7176 Note: this directive may be specified, and there is code to modify
7177 the string in the run command, but the plugin options are not yet passed to
7178 the plugin (i.e. not fully implemented).
7179
7180 \subsubsection{Plugin Options ACL}
7181 \index[general]{Plugin Options ACL}
7182 The {\bf Plugin Options ACL} directive may be specified in the
7183 Director's Console resource. It functions as all the other ACL commands
7184 do by permitting users running restricted consoles to specify a
7185 {\bf Plugin Options} that overrides the one specified in the Job
7186 definition. Without this directive restricted consoles may not modify
7187 the Plugin Options.
7188
7189 \subsubsection{Plugin = \lt{}plugin-command-string\gt{}}
7190 \index[general]{Plugin}
7191 The {\bf Plugin} directive is specified in the Include section of
7192 a FileSet resource where you put your {\bf File = xxx} directives.
7193 For example:
7194
7195 \begin{bVerbatim}
7196 FileSet {
7197 Name = "MyFileSet"
7198 Include {
7199 Options {
7200 signature = MD5
7201 }
7202 File = /home
7203 Plugin = "bpipe:..."
7204 }
7205 }
7206 \end{bVerbatim}
7207
7208 In the above example, when the File daemon is processing the directives
7209 in the Include section, it will first backup all the files in {\bf /home}
7210 then it will load the plugin named {\bf bpipe} (actually bpipe-dir.so) from
7211 the Plugin Directory. The syntax and semantics of the Plugin directive
7212 require the first part of the string up to the colon (:) to be the name
7213 of the plugin. Everything after the first colon is ignored by the File daemon but
7214 is passed to the plugin. Thus the plugin writer may define the meaning of the
7215 rest of the string as he wishes.
7216
7217 Please see the next section for information about the {\bf bpipe} Bacula
7218 plugin.
7219
7220 \subsection{The bpipe Plugin}
7221 \index[general]{The bpipe Plugin}
7222 The {\bf bpipe} plugin is provided in the directory src/plugins/fd/bpipe-fd.c of
7223 the Bacula source distribution. When the plugin is compiled and linking into
7224 the resulting dynamic shared object (DSO), it will have the name {\bf bpipe-fd.so}.
7225 Please note that this is a very simple plugin that was written for
7226 demonstration and test purposes. It is and can be used in production, but
7227 that was never really intended.
7228
7229 The purpose of the plugin is to provide an interface to any system program for
7230 backup and restore. As specified above the {\bf bpipe} plugin is specified in
7231 the Include section of your Job's FileSet resource. The full syntax of the
7232 plugin directive as interpreted by the {\bf bpipe} plugin (each plugin is free
7233 to specify the sytax as it wishes) is:
7234
7235 \begin{bVerbatim}
7236 Plugin = "<field1>:<field2>:<field3>:<field4>"
7237 \end{bVerbatim}
7238
7239 where
7240 \begin{description}
7241 \item {\bf field1} is the name of the plugin with the trailing {\bf -fd.so}
7242 stripped off, so in this case, we would put {\bf bpipe} in this field.
7243
7244 \item {\bf field2} specifies the namespace, which for {\bf bpipe} is the
7245 pseudo path and filename under which the backup will be saved. This pseudo
7246 path and filename will be seen by the user in the restore file tree.
7247 For example, if the value is {\bf /MYSQL/regress.sql}, the data
7248 backed up by the plugin will be put under that "pseudo" path and filename.
7249 You must be careful to choose a naming convention that is unique to avoid
7250 a conflict with a path and filename that actually exists on your system.
7251
7252 \item {\bf field3} for the {\bf bpipe} plugin
7253 specifies the "reader" program that is called by the plugin during
7254 backup to read the data. {\bf bpipe} will call this program by doing a
7255 {\bf popen} on it.
7256
7257 \item {\bf field4} for the {\bf bpipe} plugin
7258 specifies the "writer" program that is called by the plugin during
7259 restore to write the data back to the filesystem.
7260 \end{description}
7261
7262 Please note that for two items above describing the "reader" and "writer"
7263 fields, these programs are "executed" by Bacula, which
7264 means there is no shell interpretation of any command line arguments
7265 you might use. If you want to use shell characters (redirection of input
7266 or output, ...), then we recommend that you put your command or commands
7267 in a shell script and execute the script. In addition if you backup a
7268 file with the reader program, when running the writer program during
7269 the restore, Bacula will not automatically create the path to the file.
7270 Either the path must exist, or you must explicitly do so with your command
7271 or in a shell script.
7272
7273 Putting it all together, the full plugin directive line might look
7274 like the following:
7275
7276 \begin{bVerbatim}
7277 Plugin = "bpipe:/MYSQL/regress.sql:mysqldump -f
7278 --opt --databases bacula:mysql"
7279 \end{bVerbatim}
7280
7281 The directive has been split into two lines, but within the {\bf bacula-dir.conf} file
7282 would be written on a single line.
7283
7284 This causes the File daemon to call the {\bf bpipe} plugin, which will write
7285 its data into the "pseudo" file {\bf /MYSQL/regress.sql} by calling the
7286 program {\bf mysqldump -f --opt --database bacula} to read the data during
7287 backup. The mysqldump command outputs all the data for the database named
7288 {\bf bacula}, which will be read by the plugin and stored in the backup.
7289 During restore, the data that was backed up will be sent to the program
7290 specified in the last field, which in this case is {\bf mysql}. When
7291 {\bf mysql} is called, it will read the data sent to it by the plugn
7292 then write it back to the same database from which it came ({\bf bacula}
7293 in this case).
7294
7295 The {\bf bpipe} plugin is a generic pipe program, that simply transmits
7296 the data from a specified program to Bacula for backup, and then from Bacula to
7297 a specified program for restore.
7298
7299 By using different command lines to {\bf bpipe},
7300 you can backup any kind of data (ASCII or binary) depending
7301 on the program called.
7302
7303 \subsection{Microsoft Exchange Server 2003/2007 Plugin}
7304 \index[general]{Microsoft Exchange Server 2003/2007 Plugin}
7305 \subsubsection{Background}
7306 The Exchange plugin was made possible by a funded development project
7307 between Equiinet Ltd -- www.equiinet.com (many thanks) and Bacula Systems.
7308 The code for the plugin was written by James Harper, and the Bacula core
7309 code by Kern Sibbald. All the code for this funded development has become
7310 part of the Bacula project. Thanks to everyone who made it happen.
7311
7312 \subsubsection{Concepts}
7313 Although it is possible to backup Exchange using Bacula VSS the Exchange
7314 plugin adds a good deal of functionality, because while Bacula VSS
7315 completes a full backup (snapshot) of Exchange, it does
7316 not support Incremental or Differential backups, restoring is more
7317 complicated, and a single database restore is not possible.
7318
7319 Microsoft Exchange organises its storage into Storage Groups with
7320 Databases inside them. A default installation of Exchange will have a
7321 single Storage Group called 'First Storage Group', with two Databases
7322 inside it, "Mailbox Store (SERVER NAME)" and
7323 "Public Folder Store (SERVER NAME)",
7324 which hold user email and public folders respectively.
7325
7326 In the default configuration, Exchange logs everything that happens to
7327 log files, such that if you have a backup, and all the log files since,
7328 you can restore to the present time. Each Storage Group has its own set
7329 of log files and operates independently of any other Storage Groups. At
7330 the Storage Group level, the logging can be turned off by enabling a
7331 function called "Enable circular logging". At this time the Exchange
7332 plugin will not function if this option is enabled.
7333
7334 The plugin allows backing up of entire storage groups, and the restoring
7335 of entire storage groups or individual databases. Backing up and
7336 restoring at the individual mailbox or email item is not supported but
7337 can be simulated by use of the "Recovery" Storage Group (see below).
7338
7339 \subsubsection{Installing}
7340 The Exchange plugin requires a DLL that is shipped with Microsoft
7341 Exchanger Server called {\bf esebcli2.dll}. Assuming Exchange is installed
7342 correctly the Exchange plugin should find this automatically and run
7343 without any additional installation.
7344
7345 If the DLL can not be found automatically it will need to be copied into
7346 the Bacula installation
7347 directory (eg C:\verb+\+Program Files\verb+\+Bacula\verb+\+bin). The Exchange API DLL is
7348 named esebcli2.dll and is found in C:\verb+\+Program Files\verb+\+Exchsrvr\verb+\+bin on a
7349 default Exchange installation.
7350
7351 \subsubsection{Backing Up}
7352 To back up an Exchange server the Fileset definition must contain at
7353 least {\bf Plugin = "exchange:/@EXCHANGE/Microsoft Information Store"} for
7354 the backup to work correctly. The 'exchange:' bit tells Bacula to look
7355 for the exchange plugin, the '@EXCHANGE' bit makes sure all the backed
7356 up files are prefixed with something that isn't going to share a name
7357 with something outside the plugin, and the 'Microsoft Information Store'
7358 bit is required also. It is also possible to add the name of a storage
7359 group to the "Plugin =" line, eg \\
7360 {\bf Plugin = "exchange:/@EXCHANGE/Microsoft Information Store/First Storage Group"} \\
7361 if you want only a single storage group backed up.
7362
7363 Additionally, you can suffix the 'Plugin =' directive with
7364 ":notrunconfull" which will tell the plugin not to truncate the Exchange
7365 database at the end of a full backup.
7366
7367 An Incremental or Differential backup will backup only the database logs
7368 for each Storage Group by inspecting the "modified date" on each
7369 physical log file. Because of the way the Exchange API works, the last
7370 logfile backed up on each backup will always be backed up by the next
7371 Incremental or Differential backup too. This adds 5MB to each
7372 Incremental or Differential backup size but otherwise does not cause any
7373 problems.
7374
7375 By default, a normal VSS fileset containing all the drive letters will
7376 also back up the Exchange databases using VSS. This will interfere with
7377 the plugin and Exchange's shared ideas of when the last full backup was
7378 done, and may also truncate log files incorrectly. It is important,
7379 therefore, that the Exchange database files be excluded from the backup,
7380 although the folders the files are in should be included, or they will
7381 have to be recreated manually if a bare metal restore is done.
7382
7383 \begin{bVerbatim}
7384 FileSet {
7385 Include {
7386 File = C:/Program Files/Exchsrvr/mdbdata
7387 Plugin = "exchange:..."
7388 }
7389 Exclude {
7390 File = C:/Program Files/Exchsrvr/mdbdata/E00.chk
7391 File = C:/Program Files/Exchsrvr/mdbdata/E00.log
7392 File = C:/Program Files/Exchsrvr/mdbdata/E000000F.log
7393 File = C:/Program Files/Exchsrvr/mdbdata/E0000010.log
7394 File = C:/Program Files/Exchsrvr/mdbdata/E0000011.log
7395 File = C:/Program Files/Exchsrvr/mdbdata/E00tmp.log
7396 File = C:/Program Files/Exchsrvr/mdbdata/priv1.edb
7397 }
7398 }
7399 \end{bVerbatim}
7400
7401 The advantage of excluding the above files is that you can significantly
7402 reduce the size of your backup since all the important Exchange files
7403 will be properly saved by the Plugin.
7404
7405
7406 \subsubsection{Restoring}
7407 The restore operation is much the same as a normal Bacula restore, with
7408 the following provisos:
7409
7410 \begin{itemize}
7411 \item The {\bf Where} restore option must not be specified
7412 \item Each Database directory must be marked as a whole. You cannot just
7413 select (say) the .edb file and not the others.
7414 \item If a Storage Group is restored, the directory of the Storage Group
7415 must be marked too.
7416 \item It is possible to restore only a subset of the available log files,
7417 but they {\bf must} be contiguous. Exchange will fail to restore correctly
7418 if a log file is missing from the sequence of log files
7419 \item Each database to be restored must be dismounted and marked as "Can be
7420 overwritten by restore"
7421 \item If an entire Storage Group is to be restored (eg all databases and
7422 logs in the Storage Group), then it is best to manually delete the
7423 database files from the server (eg C:\verb+\+Program Files\verb+\+Exchsrvr\verb+\+mdbdata\verb+\+*)
7424 as Exchange can get confused by stray log files lying around.
7425 \end{itemize}
7426
7427 \subsubsection{Restoring to the Recovery Storage Group}
7428 The concept of the Recovery Storage Group is
7429 \bulink{well documented by Microsoft}{http://support.microsoft.com/kb/824126},
7430 but to briefly summarize\ldots{}
7431
7432 Microsoft Exchange allows the creation of an additional Storage Group
7433 called the Recovery Storage Group, which is used to restore an older
7434 copy of a database (e.g. before a mailbox was deleted) into without
7435 messing with the current live data. This is required as the Standard and
7436 Small Business Server versions of Exchange can not ordinarily have more
7437 than one Storage Group.
7438
7439 To create the Recovery Storage Group, drill down to the Server in Exchange
7440 System Manager, right click, and select
7441 {\bf "New -> Recovery Storage Group..."}. Accept or change the file
7442 locations and click OK. On the Recovery Storage Group, right click and
7443 select {\bf "Add Database to Recover..."} and select the database you will
7444 be restoring.
7445
7446 Restore only the single database nominated as the database in the
7447 Recovery Storage Group. Exchange will redirect the restore to the
7448 Recovery Storage Group automatically.
7449 Then run the restore.
7450
7451 \subsubsection{Restoring on Microsoft Server 2007}
7452 Apparently the {\bf Exmerge} program no longer exists in Microsoft Server
7453 2007, and hence you use a new procedure for recovering a single mail box.
7454
7455 \bulink{This procedure is documented by Microsoft}{http://technet.microsoft.com/en-us/library/aa997694.aspx},
7456 and involves using the {\bf Restore-Mailbox} and {\bf Get-Mailbox Statistics} shell commands.
7457
7458 \subsubsection{Caveats}
7459 This plugin is still being developed, so you should consider it
7460 currently in BETA test, and thus use in a production environment
7461 should be done only after very careful testing.
7462
7463 When doing a full backup, the Exchange database logs are truncated by
7464 Exchange as soon as the plugin has completed the backup. If the data
7465 never makes it to the backup medium (eg because of spooling) then the
7466 logs will still be truncated, but they will also not have been backed
7467 up. A solution to this is being worked on. You will have to schedule a
7468 new Full backup to ensure that your next backups will be usable.
7469
7470 The "Enable Circular Logging" option cannot be enabled or the plugin
7471 will fail.
7472
7473 Exchange insists that a successful Full backup must have taken place if
7474 an Incremental or Differential backup is desired, and the plugin will
7475 fail if this is not the case. If a restore is done, Exchange will
7476 require that a Full backup be done before an Incremental or Differential
7477 backup is done.
7478
7479 The plugin will most likely not work well if another backup application
7480 (eg NTBACKUP) is backing up the Exchange database, especially if the
7481 other backup application is truncating the log files.
7482
7483 The Exchange plugin has not been tested with the {\bf Accurate} option, so
7484 we recommend either carefully testing or that you avoid this option for
7485 the current time.
7486
7487 The Exchange plugin is not called during processing the bconsole {\bf
7488 estimate} command, and so anything that would be backed up by the plugin
7489 will not be added to the estimate total that is displayed.
7490
7491
7492 \subsection{libdbi Framework}
7493 \index[general]{libdbi Framework}
7494 As a general guideline, Bacula has support for a few catalog database drivers
7495 (MySQL and PostgreSQL)
7496 coded natively by the Bacula team. With the libdbi implementation, which is a
7497 Bacula driver that uses libdbi to access the catalog, we have an open field to
7498 use many different kinds database engines following the needs of users.
7499
7500 The according to libdbi (http://libdbi.sourceforge.net/) project: libdbi
7501 implements a database-independent abstraction layer in C, similar to the
7502 DBI/DBD layer in Perl. Writing one generic set of code, programmers can
7503 leverage the power of multiple databases and multiple simultaneous database
7504 connections by using this framework.
7505
7506 Currently the libdbi driver in Bacula project only supports the same drivers
7507 natively coded in Bacula. However the libdbi project has support for many
7508 others database engines. You can view the list at
7509 http://libdbi-drivers.sourceforge.net/. In the future all those drivers can be
7510 supported by Bacula, however, they must be tested properly by the Bacula team.
7511
7512 Some of benefits of using libdbi are:
7513 \begin{itemize}
7514 \item The possibility to use proprietary databases engines in which your
7515 proprietary licenses prevent the Bacula team from developing the driver.
7516 \item The possibility to use the drivers written for the libdbi project.
7517 \item The possibility to use other database engines without recompiling Bacula
7518 to use them. Just change one line in bacula-dir.conf
7519 \item Abstract Database access, this is, unique point to code and profiling
7520 catalog database access.
7521 \end{itemize}
7522
7523 The following drivers have been tested:
7524 \begin{itemize}
7525 \item PostgreSQL, with and without batch insert
7526 \item Mysql, with and without batch insert
7527 \end{itemize}
7528
7529 In the future, we will test and approve to use others databases engines
7530 (proprietary or not) like DB2, Oracle, Microsoft SQL.
7531
7532 To compile Bacula to support libdbi we need to configure the code with the
7533 --with-dbi and --with-dbi-driver=[database] ./configure options, where
7534 [database] is the database engine to be used with Bacula (of course we can
7535 change the driver in file bacula-dir.conf, see below). We must configure the
7536 access port of the database engine with the option --with-db-port, because the
7537 libdbi framework doesn't know the default access port of each database.
7538
7539 The next phase is checking (or configuring) the bacula-dir.conf, example:
7540 \begin{bVerbatim}
7541 Catalog {
7542 Name = MyCatalog
7543 dbdriver = dbi:mysql; dbaddress = 127.0.0.1; dbport = 3306
7544 dbname = regress; user = regress; password = ""
7545 }
7546 \end{bVerbatim}
7547
7548 The parameter {\bf dbdriver} indicates that we will use the driver dbi with a
7549 mysql database. Currently the drivers supported by Bacula are: postgresql,
7550 amd mysql; these are the names that may be added to string "dbi:".
7551
7552 The following limitations apply when Bacula is set to use the libdbi framework:
7553 - Not tested on the Win32 platform
7554 - A little performance is lost if comparing with native database driver.
7555 The reason is bound with the database driver provided by libdbi and the
7556 simple fact that one more layer of code was added.
7557
7558 It is important to remember, when compiling Bacula with libdbi, the
7559 following packages are needed:
7560 \begin{itemize}
7561 \item libdbi version 1.0.0, http://libdbi.sourceforge.net/
7562 \item libdbi-drivers 1.0.0, http://libdbi-drivers.sourceforge.net/
7563 \end{itemize}
7564
7565 You can download them and compile them on your system or install the packages
7566 from your OS distribution.
7567
7568 \subsection{Console Command Additions and Enhancements}
7569 \index[general]{Console Additions}
7570
7571 \subsubsection{Display Autochanger Content}
7572 \index[general]{StatusSlots}
7573
7574 The {\bf status slots storage=\lt{}storage-name\gt{}} command displays
7575 autochanger content.
7576
7577 \footnotesize
7578 \begin{bVerbatim}
7579 Slot | Volume Name | Status | Media Type | Pool |
7580 ------+---------------+----------+-------------------+------------|
7581 1 | 00001 | Append | DiskChangerMedia | Default |
7582 2 | 00002 | Append | DiskChangerMedia | Default |
7583 3*| 00003 | Append | DiskChangerMedia | Scratch |
7584 4 | | | | |
7585 \end{bVerbatim}
7586 \normalsize
7587
7588 If you an asterisk ({\bf *}) appears after the slot number, you must run an
7589 {\bf update slots} command to synchronize autochanger content with your
7590 catalog.
7591
7592 \subsubsection{list joblog job=xxx or jobid=nnn}
7593 \index[general]{list joblog}
7594 A new list command has been added that allows you to list the contents
7595 of the Job Log stored in the catalog for either a Job Name (fully qualified)
7596 or for a particular JobId. The {\bf llist} command will include a line with
7597 the time and date of the entry.
7598
7599 Note for the catalog to have Job Log entries, you must have a directive
7600 such as:
7601
7602 \begin{bVerbatim}
7603 catalog = all
7604 \end{bVerbatim}
7605
7606 In your Director's {\bf Messages} resource.
7607
7608 \subsubsection{Use separator for multiple commands}
7609 \index[general]{Command Separator}
7610 When using bconsole with readline, you can set the command separator with
7611 \textbf{@separator} command to one
7612 of those characters to write commands who require multiple input in one line.
7613 \begin{bVerbatim}
7614 !$%&'()*+,-/:;<>?[]^`{|}~
7615 \end{bVerbatim}
7616
7617 \subsubsection{Deleting Volumes}
7618 The delete volume bconsole command has been modified to
7619 require an asterisk (*) in front of a MediaId otherwise the
7620 value you enter is a taken to be a Volume name. This is so that
7621 users may delete numeric Volume names. The previous Bacula versions
7622 assumed that all input that started with a number was a MediaId.
7623
7624 This new behavior is indicated in the prompt if you read it
7625 carefully.
7626
7627 \subsection{Bare Metal Recovery}
7628 The old bare metal recovery project is essentially dead. One
7629 of the main features of it was that it would build a recovery
7630 CD based on the kernel on your system. The problem was that
7631 every distribution has a different boot procedure and different
7632 scripts, and worse yet, the boot procedures and scripts change
7633 from one distribution to another. This meant that maintaining
7634 (keeping up with the changes) the rescue CD was too much work.
7635
7636 To replace it, a new bare metal recovery USB boot stick has been developed
7637 by Bacula Systems. This technology involves remastering a Ubuntu LiveCD to
7638 boot from a USB key.
7639
7640 Advantages:
7641 \begin{enumerate}
7642 \item Recovery can be done from within graphical environment.
7643 \item Recovery can be done in a shell.
7644 \item Ubuntu boots on a large number of Linux systems.
7645 \item The process of updating the system and adding new
7646 packages is not too difficult.
7647 \item The USB key can easily be upgraded to newer Ubuntu versions.
7648 \item The USB key has writable partitions for modifications to
7649 the OS and for modification to your home directory.
7650 \item You can add new files/directories to the USB key very easily.
7651 \item You can save the environment from multiple machines on
7652 one USB key.
7653 \item Bacula Systems is funding its ongoing development.
7654 \end{enumerate}
7655
7656 The disadvantages are:
7657 \begin{enumerate}
7658 \item The USB key is usable but currently under development.
7659 \item Not everyone may be familiar with Ubuntu (no worse
7660 than using Knoppix)
7661 \item Some older OSes cannot be booted from USB. This can
7662 be resolved by first booting a Ubuntu LiveCD then plugging
7663 in the USB key.
7664 \item Currently the documentation is sketchy and not yet added
7665 to the main manual. See below ...
7666 \end{enumerate}
7667
7668 The documentation and the code can be found in the {\bf rescue} package
7669 in the directory {\bf linux/usb}.
7670
7671 \subsection{Miscellaneous}
7672 \index[general]{Misc New Features}
7673
7674 \subsubsection{Allow Mixed Priority = \lt{}yes\vb{}no\gt{}}
7675 \index[general]{Allow Mixed Priority}
7676 This directive is only implemented in version 2.5 and later. When
7677 set to {\bf yes} (default {\bf no}), this job may run even if lower
7678 priority jobs are already running. This means a high priority job
7679 will not have to wait for other jobs to finish before starting.
7680 The scheduler will only mix priorities when all running jobs have
7681 this set to true.
7682
7683 Note that only higher priority jobs will start early. Suppose the
7684 director will allow two concurrent jobs, and that two jobs with
7685 priority 10 are running, with two more in the queue. If a job with
7686 priority 5 is added to the queue, it will be run as soon as one of
7687 the running jobs finishes. However, new priority 10 jobs will not
7688 be run until the priority 5 job has finished.
7689
7690 \subsubsection{Bootstrap File Directive -- FileRegex}
7691 \index[general]{Bootstrap File Directive}
7692 {\bf FileRegex} is a new command that can be added to the bootstrap
7693 (.bsr) file. The value is a regular expression. When specified, only
7694 matching filenames will be restored.
7695
7696 During a restore, if all File records are pruned from the catalog
7697 for a Job, normally Bacula can restore only all files saved. That
7698 is there is no way using the catalog to select individual files.
7699 With this new feature, Bacula will ask if you want to specify a Regex
7700 expression for extracting only a part of the full backup.
7701
7702 \begin{bVerbatim}
7703 Building directory tree for JobId(s) 1,3 ...
7704 There were no files inserted into the tree, so file selection
7705 is not possible.Most likely your retention policy pruned the files
7706
7707 Do you want to restore all the files? (yes\vb{}no): no
7708
7709 Regexp matching files to restore? (empty to abort): /tmp/regress/(bin|tests)/
7710 Bootstrap records written to /tmp/regress/working/zog4-dir.restore.1.bsr
7711 \end{bVerbatim}
7712
7713 \subsubsection{Bootstrap File Optimization Changes}
7714 In order to permit proper seeking on disk files, we have extended the bootstrap
7715 file format to include a {\bf VolStartAddr} and {\bf VolEndAddr} records. Each
7716 takes a 64 bit unsigned integer range (i.e. nnn-mmm) which defines the start
7717 address range and end address range respectively. These two directives replace
7718 the {\bf VolStartFile}, {\bf VolEndFile}, {\bf VolStartBlock} and {\bf
7719 VolEndBlock} directives. Bootstrap files containing the old directives will
7720 still work, but will not properly take advantage of proper disk seeking, and
7721 may read completely to the end of a disk volume during a restore. With the new
7722 format (automatically generated by the new Director), restores will seek
7723 properly and stop reading the volume when all the files have been restored.
7724
7725 \subsubsection{Solaris ZFS/NFSv4 ACLs}
7726 This is an upgrade of the previous Solaris ACL backup code
7727 to the new library format, which will backup both the old
7728 POSIX(UFS) ACLs as well as the ZFS ACLs.
7729
7730 The new code can also restore POSIX(UFS) ACLs to a ZFS filesystem
7731 (it will translate the POSIX(UFS)) ACL into a ZFS/NFSv4 one) it can also
7732 be used to transfer from UFS to ZFS filesystems.
7733
7734
7735 \subsubsection{Virtual Tape Emulation}
7736 \index[general]{Virtual Tape Emulation}
7737 We now have a Virtual Tape emulator that allows us to run though 99.9\% of
7738 the tape code but actually reading and writing to a disk file. Used with the
7739 \textbf{disk-changer} script, you can now emulate an autochanger with 10 drives
7740 and 700 slots. This feature is most useful in testing. It is enabled
7741 by using {\bf Device Type = vtape} in the Storage daemon's Device
7742 directive. This feature is only implemented on Linux machines and should not be
7743 used for production.
7744
7745 \subsubsection{Bat Enhancements}
7746 \index[general]{Bat Enhancements}
7747 Bat (the Bacula Administration Tool) GUI program has been significantly
7748 enhanced and stabilized. In particular, there are new table based status
7749 commands; it can now be easily localized using Qt4 Linguist.
7750
7751 The Bat communications protocol has been significantly enhanced to improve
7752 GUI handling. Note, you {\bf must} use a the bat that is distributed with
7753 the Director you are using otherwise the communications protocol will not
7754 work.
7755
7756 \subsubsection{RunScript Enhancements}
7757 \index[general]{RunScript Enhancements}
7758 The {\bf RunScript} resource has been enhanced to permit multiple
7759 commands per RunScript. Simply specify multiple {\bf Command} directives
7760 in your RunScript.
7761
7762 \begin{bVerbatim}
7763 Job {
7764 Name = aJob
7765 RunScript {
7766 Command = "/bin/echo test"
7767 Command = "/bin/echo an other test"
7768 Command = "/bin/echo 3 commands in the same runscript"
7769 RunsWhen = Before
7770 }
7771 ...
7772 }
7773 \end{bVerbatim}
7774
7775 A new Client RunScript {\bf RunsWhen} keyword of {\bf AfterVSS} has been
7776 implemented, which runs the command after the Volume Shadow Copy has been made.
7777
7778 Console commands can be specified within a RunScript by using:
7779 {\bf Console = \lt{}command\gt{}}, however, this command has not been
7780 carefully tested and debugged and is known to easily crash the Director.
7781 We would appreciate feedback. Due to the recursive nature of this command, we
7782 may remove it before the final release.
7783
7784 \subsubsection{Status Enhancements}
7785 \index[general]{Status Enhancements}
7786 The bconsole {\bf status dir} output has been enhanced to indicate
7787 Storage daemon job spooling and despooling activity.
7788
7789 \subsubsection{Connect Timeout}
7790 \index[general]{Connect Timeout}
7791 The default connect timeout to the File
7792 daemon has been set to 3 minutes. Previously it was 30 minutes.
7793
7794 \subsubsection{ftruncate for NFS Volumes}
7795 \index[general]{ftruncate for NFS Volumes}
7796 If you write to a Volume mounted by NFS (say on a local file server),
7797 in previous Bacula versions, when the Volume was recycled, it was not
7798 properly truncated because NFS does not implement ftruncate (file
7799 truncate). This is now corrected in the new version because we have
7800 written code (actually a kind user) that deletes and recreates the Volume,
7801 thus accomplishing the same thing as a truncate.
7802
7803 \subsubsection{Support for Ubuntu}
7804 The new version of Bacula now recognizes the Ubuntu (and Kubuntu)
7805 version of Linux, and thus now provides correct autostart routines.
7806 Since Ubuntu officially supports Bacula, you can also obtain any
7807 recent release of Bacula from the Ubuntu repositories.
7808
7809 \subsubsection{Recycle Pool = \lt{}pool-name\gt{}}
7810 \index[general]{Recycle Pool}
7811 The new \textbf{RecyclePool} directive defines to which pool the Volume will
7812 be placed (moved) when it is recycled. Without this directive, a Volume will
7813 remain in the same pool when it is recycled. With this directive, it can be
7814 moved automatically to any existing pool during a recycle. This directive is
7815 probably most useful when defined in the Scratch pool, so that volumes will
7816 be recycled back into the Scratch pool.
7817
7818 \subsubsection{FD Version}
7819 \index[general]{FD Version}
7820 The File daemon to Director protocol now includes a version
7821 number, which although there is no visible change for users,
7822 will help us in future versions automatically determine
7823 if a File daemon is not compatible.
7824
7825 \subsubsection{Max Run Sched Time = \lt{}time-period-in-seconds\gt{}}
7826 \index[general]{Max Run Sched Time}
7827 The time specifies the maximum allowed time that a job may run, counted from
7828 when the job was scheduled. This can be useful to prevent jobs from running
7829 during working hours. We can see it like \texttt{Max Start Delay + Max Run
7830 Time}.
7831
7832 \subsubsection{Max Wait Time = \lt{}time-period-in-seconds\gt{}}
7833 \index[general]{Max Wait Time}
7834 Previous \textbf{MaxWaitTime} directives aren't working as expected, instead
7835 of checking the maximum allowed time that a job may block for a resource,
7836 those directives worked like \textbf{MaxRunTime}. Some users are reporting to
7837 use \textbf{Incr/Diff/Full Max Wait Time} to control the maximum run time of
7838 their job depending on the level. Now, they have to use
7839 \textbf{Incr/Diff/Full Max Run Time}. \textbf{Incr/Diff/Full Max Wait Time}
7840 directives are now deprecated.
7841
7842 \subsubsection{Incremental|Differential Max Wait Time = \lt{}time-period-in-seconds\gt{}}
7843 \index[general]{Incremental Max Wait Time}
7844 \index[general]{Differential Max Wait Time}
7845
7846 These directives have been deprecated in favor of
7847 \texttt{Incremental|Differential Max Run Time}.
7848
7849 \subsubsection{Max Run Time directives}
7850 \index[general]{Max Run Time directives}
7851 Using \textbf{Full/Diff/Incr Max Run Time}, it's now possible to specify the
7852 maximum allowed time that a job can run depending on the level.
7853
7854 \addcontentsline{lof}{figure}{Job time control directives}
7855 \begin{center}
7856 \includegraphics[width=\linewidth]{different_time}
7857 \end{center}
7858
7859 \subsubsection{Statistics Enhancements}
7860 \index[general]{Statistics Enhancements}
7861 If you (or probably your boss) want to have statistics on your backups to
7862 provide some \textit{Service Level Agreement} indicators, you could use a few
7863 SQL queries on the Job table to report how many:
7864
7865 \begin{itemize}
7866 \item jobs have run
7867 \item jobs have been successful
7868 \item files have been backed up
7869 \item ...
7870 \end{itemize}
7871
7872 However, these statistics are accurate only if your job retention is greater
7873 than your statistics period. Ie, if jobs are purged from the catalog, you won't
7874 be able to use them.
7875
7876 Now, you can use the \textbf{update stats [days=num]} console command to fill
7877 the JobHistory table with new Job records. If you want to be sure to take in
7878 account only \textbf{good jobs}, ie if one of your important job has failed but
7879 you have fixed the problem and restarted it on time, you probably want to
7880 delete the first \textit{bad} job record and keep only the successful one. For
7881 that simply let your staff do the job, and update JobHistory table after two or
7882 three days depending on your organization using the \textbf{[days=num]} option.
7883
7884 These statistics records aren't used for restoring, but mainly for
7885 capacity planning, billings, etc.
7886
7887 The Bweb interface provides a statistics module that can use this feature. You
7888 can also use tools like Talend or extract information by yourself.
7889
7890 The \textbf{Statistics Retention = \lt{}time\gt{}} director directive defines
7891 the length of time that Bacula will keep statistics job records in the Catalog
7892 database after the Job End time. (In \texttt{JobHistory} table) When this time
7893 period expires, and if user runs \texttt{prune stats} command, Bacula will
7894 prune (remove) Job records that are older than the specified period.
7895
7896 You can use the following Job resource in your nightly \textbf{BackupCatalog}
7897 job to maintain statistics.
7898 \begin{bVerbatim}
7899 Job {
7900 Name = BackupCatalog
7901 ...
7902 RunScript {
7903 Console = "update stats days=3"
7904 Console = "prune stats yes"
7905 RunsWhen = After
7906 RunsOnClient = no
7907 }
7908 }
7909 \end{bVerbatim}
7910
7911 \subsubsection{ScratchPool = \lt{}pool-resource-name\gt{}}
7912 \index[general]{ScratchPool}
7913 This directive permits to specify a specific \textsl{Scratch} pool for the
7914 current pool. This is useful when using multiple storage sharing the same
7915 mediatype or when you want to dedicate volumes to a particular set of pool.
7916
7917 \subsubsection{Enhanced Attribute Despooling}
7918 \index[general]{Attribute Despooling}
7919 If the storage daemon and the Director are on the same machine, the spool file
7920 that contains attributes is read directly by the Director instead of being
7921 transmitted across the network. That should reduce load and speedup insertion.
7922
7923 \subsubsection{SpoolSize = \lt{}size-specification-in-bytes\gt{}}
7924 \index[general]{SpoolSize}
7925 A new Job directive permits to specify the spool size per job. This is used
7926 in advanced job tunning. {\bf SpoolSize={\it bytes}}
7927
7928 \subsubsection{MaximumConsoleConnections = \lt{}number\gt{}}
7929 \index[general]{MaximumConsoleConnections}
7930 A new director directive permits to specify the maximum number of Console
7931 Connections that could run concurrently. The default is set to 20, but you may
7932 set it to a larger number.
7933
7934 \subsubsection{VerId = \lt{}string\gt{}}
7935 \index[general]{VerId}
7936 A new director directive permits to specify a personnal identifier that will be
7937 displayed in the \texttt{version} command.
7938
7939 \subsubsection{dbcheck enhancements}
7940 \index[general]{dbcheck enhancements}
7941 If you are using Mysql, dbcheck will now ask you if you want to create
7942 temporary indexes to speed up orphaned Path and Filename elimination.
7943
7944 A new \texttt{-B} option allows you to print catalog information in a simple
7945 text based format. This is useful to backup it in a secure way.
7946
7947 \begin{bVerbatim}
7948 $ dbcheck -B
7949 catalog=MyCatalog
7950 db_type=MySQL
7951 db_name=regress
7952 db_driver=
7953 db_user=regress
7954 db_password=
7955 db_address=
7956 db_port=0
7957 db_socket=
7958 \end{bVerbatim}
7959
7960 You can now specify the database connection port in the command line.
7961
7962 \subsubsection{{-}{-}docdir configure option}
7963 \index[general]{{-}{-}docdir configure option}
7964 You can use {-}{-}docdir= on the ./configure command to
7965 specify the directory where you want Bacula to install the
7966 LICENSE, ReleaseNotes, ChangeLog, ... files. The default is
7967 {\bf /usr/share/doc/bacula}.
7968
7969 \subsubsection{{-}{-}htmldir configure option}
7970 \index[general]{{-}{-}htmldir configure option}
7971 You can use {-}{-}htmldir= on the ./configure command to
7972 specify the directory where you want Bacula to install the bat html help
7973 files. The default is {\bf /usr/share/doc/bacula/html}
7974
7975 \subsubsection{{-}{-}with-plugindir configure option}
7976 \index[general]{{-}{-}plugindir configure option}
7977 You can use {-}{-}plugindir= on the ./configure command to
7978 specify the directory where you want Bacula to install
7979 the plugins (currently only bpipe-fd). The default is
7980 /usr/lib.