| 1 |
\chapter{Data Spooling} |
| 2 |
\label{blb:SpoolingChapter} |
| 3 |
\index[general]{Data Spooling } |
| 4 |
\index[general]{Spooling!Data } |
| 5 |
|
| 6 |
\bacula{} allows you to specify that you want the Storage daemon to initially |
| 7 |
write your data to disk and then subsequently to tape. This serves several |
| 8 |
important purposes. |
| 9 |
|
| 10 |
\begin{bitemize} |
| 11 |
\item It takes a long time for data to come in from the File daemon during |
| 12 |
an Incremental backup. If it is directly written to tape, the tape will |
| 13 |
start and stop or shoe-shine as it is often called causing tape wear. |
| 14 |
By first writing the data to disk, then writing it to tape, the tape can |
| 15 |
be kept in continual motion. |
| 16 |
\item While the spooled data is being written to the tape, the despooling |
| 17 |
process has exclusive use of the tape. This means that you can spool |
| 18 |
multiple simultaneous jobs to disk, then have them very efficiently |
| 19 |
despooled one at a time without having the data blocks from several jobs |
| 20 |
intermingled, thus substantially improving the time needed to restore |
| 21 |
files. While despooling, all jobs spooling continue running. |
| 22 |
\item Writing to a tape can be slow. By first spooling your data to disk, |
| 23 |
you can often reduce the time the File daemon is running on a system, |
| 24 |
thus reducing downtime, and/or interference with users. Of course, if |
| 25 |
your spool device is not large enough to hold all the data from your |
| 26 |
File daemon, you may actually slow down the overall backup. |
| 27 |
\end{bitemize} |
| 28 |
|
| 29 |
Data spooling is exactly that ``spooling''. It is not a way to first write a |
| 30 |
``backup'' to a disk file and then to a tape. When the backup has only been |
| 31 |
spooled to disk, it is not complete yet and cannot be restored until it is |
| 32 |
written to tape. |
| 33 |
|
| 34 |
\bacula{} version 1.39.x and later supports writing a backup |
| 35 |
to disk then later \textbf{Migrating} or moving it to a tape (or any |
| 36 |
other medium). For |
| 37 |
details on this, please see the \bilink{Migration}{blb:MigrationChapter} chapter |
| 38 |
of this manual for more details. |
| 39 |
|
| 40 |
The remainder of this chapter explains the various directives that you can use |
| 41 |
in the spooling process. |
| 42 |
|
| 43 |
\label{blb:directives} |
| 44 |
\section{Data Spooling Directives} |
| 45 |
\index[general]{Directives!Data Spooling } |
| 46 |
\index[general]{Data Spooling Directives } |
| 47 |
|
| 48 |
The following directives can be used to control data spooling. |
| 49 |
|
| 50 |
\begin{bitemize} |
| 51 |
\item To turn data spooling on/off at the Job level in the Job resource in |
| 52 |
the Director's conf file (default \bdefaultvalue{no}). |
| 53 |
|
| 54 |
\textbf{SpoolData = \byesorno{}} |
| 55 |
|
| 56 |
\item To override the Job specification in a Schedule Run directive in the |
| 57 |
Director's conf file. |
| 58 |
|
| 59 |
\textbf{SpoolData = \byesorno{}} |
| 60 |
|
| 61 |
\item To override the Job specification in a bconsole session using the \bcommandname{run} |
| 62 |
command. Please note that this does \textbf{not} refer to a configuration |
| 63 |
statement, but to an argument for the \bcommandname{run} command. |
| 64 |
|
| 65 |
\textbf{SpoolData= \byesorno{}} |
| 66 |
|
| 67 |
\item To limit the the maximum spool file size for a particular job in the Job |
| 68 |
resource |
| 69 |
|
| 70 |
\textbf{Spool Size = size} |
| 71 |
Where size is a the maximum spool size for this job specified in \bhighlight{bytes}. |
| 72 |
|
| 73 |
\item To limit the maximum total size of the spooled data for a particular |
| 74 |
device. Specified in the Device resource of the Storage daemon's conf file |
| 75 |
(default \bdefaultvalue{unlimited}). |
| 76 |
|
| 77 |
\textbf{Maximum Spool Size = size} |
| 78 |
Where size is a the maximum spool size for all jobs specified in \bhighlight{bytes}. |
| 79 |
|
| 80 |
\item To limit the maximum total size of the spooled data for a particular |
| 81 |
device for a single job. Specified in the Device Resource of the Storage |
| 82 |
daemon's configuration file (default \bdefaultvalue{unlimited}). |
| 83 |
|
| 84 |
\textbf{Maximum Job Spool Size = size} |
| 85 |
Where size is the maximum spool file size for a single job specified in |
| 86 |
\bhighlight{bytes}. |
| 87 |
|
| 88 |
\item To specify the spool directory for a particular device. Specified in |
| 89 |
the Device Resource of the Storage daemon's configuration file (default, |
| 90 |
the \bdefaultvalue{working directory}). |
| 91 |
|
| 92 |
\textbf{Spool Directory = directory} |
| 93 |
\end{bitemize} |
| 94 |
|
| 95 |
\label{blb:warning} |
| 96 |
|
| 97 |
% TODO: fix this section name |
| 98 |
\section{FileSet consideration}%%% Philippe's idea for renaming this section. !!! MAJOR WARNING !!!} |
| 99 |
\index[general]{Spooling! Fileset} %%WARNING! MAJOR} |
| 100 |
\index[general]{Fileset and spooling}%%MAJOR WARNING} |
| 101 |
|
| 102 |
Please be very careful to \bhighlight{exclude} the spool directory from any backup, |
| 103 |
otherwise, your job will write enormous amounts of data to the Volume, and |
| 104 |
most probably terminate in error. This is because in attempting to backup the |
| 105 |
spool file, the backup data will be written a second time to the spool file, |
| 106 |
and so on \emph{ad infinitum}. |
| 107 |
|
| 108 |
Another advice is to always specify the maximum spool size so that your disk |
| 109 |
doesn't completely fill up. In principle, data spooling will properly detect a |
| 110 |
full disk, and despool data allowing the job to continue. However, attribute |
| 111 |
spooling is not so kind to the user. If the disk on which attributes are being |
| 112 |
spooled fills, the job will be canceled. In addition, if your working |
| 113 |
directory is on the same partition as the spool directory, then \bacula{} jobs |
| 114 |
will fail possibly in bizarre ways when the spool fills. |
| 115 |
|
| 116 |
\label{blb:points} |
| 117 |
\section{Other Points} |
| 118 |
\index[general]{Points!Other } |
| 119 |
\index[general]{Other Points } |
| 120 |
|
| 121 |
\begin{bitemize} |
| 122 |
\item When data spooling is enabled, \bacula{} automatically turns on attribute |
| 123 |
spooling. In other words, it also spools the catalog entries to disk. This is |
| 124 |
done so that in case the job fails, there will be no catalog entries |
| 125 |
pointing to non-existent tape backups. |
| 126 |
\item Attribute despooling occurs near the end of a job. The Storage daemon |
| 127 |
accumulates file attributes during the backup and sends them to the |
| 128 |
Director at the end of the job. The Director then inserts the file |
| 129 |
attributes into the catalog. During this insertion, the tape drive may |
| 130 |
be inactive. When the file attribute insertion is completed, the job |
| 131 |
terminates. |
| 132 |
\item Attribute spool files are always placed in the working directory of |
| 133 |
the Storage daemon. |
| 134 |
\item When \bacula{} begins despooling data spooled to disk, it takes exclusive |
| 135 |
use of the tape. This has the major advantage that in running multiple |
| 136 |
simultaneous jobs at the same time, the blocks of several jobs will not be |
| 137 |
intermingled. |
| 138 |
\item It probably does not make a lot of sense to enable data spooling if you |
| 139 |
are writing to disk files. |
| 140 |
\item It is probably best to provide as large a spool file as possible to |
| 141 |
avoid repeatedly spooling/despooling. Also, while a job is despooling to |
| 142 |
tape, the File daemon must wait (i.e. spooling stops for the job while it is |
| 143 |
despooling). |
| 144 |
\item If you are running multiple simultaneous jobs, \bacula{} will continue |
| 145 |
spooling other jobs while one is despooling to tape, provided there is |
| 146 |
sufficient spool file space. |
| 147 |
\end{bitemize} |