| 1 |
s\chapter{Automatic Volume Recycling} |
| 2 |
\label{blb:RecyclingChapter} |
| 3 |
\index[general]{Recycling!Automatic Volume } |
| 4 |
\index[general]{Automatic Volume Recycling } |
| 5 |
|
| 6 |
By default, once \bacula{} starts writing a Volume, it may append to the |
| 7 |
volume, but it will not overwrite the existing data thus destroying it. |
| 8 |
However when \bacula{} \textbf{recycles} a Volume, the Volume becomes available |
| 9 |
for being reused, and \bacula{} can at some later time overwrite the previous |
| 10 |
contents of that Volume. At that point all previous data on that Volume |
| 11 |
will be lost. If the Volume is a tape, the tape will be rewritten from the |
| 12 |
beginning. If the Volume is a disk file, the file will be truncated before |
| 13 |
being rewritten. |
| 14 |
|
| 15 |
You may not want Bacula to automatically recycle (reuse) Volumes. Doing so |
| 16 |
may require a large number of Volumes though. However, it is also possible |
| 17 |
to manually recycle Volumes so that they may be reused. For more on manual |
| 18 |
recycling, see the section entitled \bilink{Manually Recycling Volumes}{blb:manualrecycling} below in this chapter. |
| 19 |
|
| 20 |
Most people prefer to have a Pool of Volumes that are used for daily backups and |
| 21 |
recycled once a week, another Pool of Volumes that are used for Full backups |
| 22 |
once a week and recycled monthly, and finally a Pool of Volumes that are used |
| 23 |
once a month and recycled after a year or two. With a scheme like this, the |
| 24 |
number of Volumes in your pool or pools remains constant. |
| 25 |
|
| 26 |
By properly defining your Volume Pools with appropriate Retention periods, |
| 27 |
\bacula{} can manage the recycling (such as defined above) automatically. |
| 28 |
|
| 29 |
Automatic recycling of Volumes is controlled by four records in the \bresourcename{Pool} |
| 30 |
resource definition in the Director's configuration file. These four |
| 31 |
records are: |
| 32 |
|
| 33 |
\begin{bitemize} |
| 34 |
\item AutoPrune = yes |
| 35 |
\item VolumeRetention = \bbracket{time} |
| 36 |
\item Recycle = yes |
| 37 |
\item RecyclePool = \bbracket{APool} |
| 38 |
\item ScratchPool = \bbracket{APool} |
| 39 |
\end{bitemize} |
| 40 |
|
| 41 |
The above first three directives are all you need assuming that you fill |
| 42 |
each of your Volumes then wait the Volume Retention period before |
| 43 |
reusing them, providing there is some non-pruned Jobs or Files on the |
| 44 |
Volume. \bdirectivename{Recycle Pool} and \bdirectivename{Scratch Pool} directives are not required. If not |
| 45 |
defined, \bacula{} will recycle Volumes and keep them in the current Pool. |
| 46 |
|
| 47 |
If you want \bacula{} to stop using a Volume and recycle |
| 48 |
it before it is full, you will need to use one or more additional |
| 49 |
directives such as: |
| 50 |
|
| 51 |
\begin{bitemize} |
| 52 |
\item \bvalueddirective{Use Volume Once}{yes} |
| 53 |
\item \bvalueddirective{Volume Use Duration}{ttt} |
| 54 |
\item \bvalueddirective{Maximum Volume Jobs}{nnn} |
| 55 |
\item \bvalueddirective{Maximum Volume Bytes}{mmm} |
| 56 |
\end{bitemize} |
| 57 |
|
| 58 |
Please see below and |
| 59 |
the \bilink{Basic Volume Management}{blb:DiskChapter} chapter |
| 60 |
of this manual for more complete examples. |
| 61 |
|
| 62 |
Automatic recycling of Volumes is performed by \bacula{} only when it wants a |
| 63 |
new Volume and no appendable Volumes are available in the Pool. It will then |
| 64 |
search the Pool for any Volumes with the \bstatus{Recycle} flag set and the |
| 65 |
Volume Status is \bstatus{Purged}. At that point, it will choose the oldest |
| 66 |
purged volume and recycle it. |
| 67 |
|
| 68 |
If there are no volumes with status \bstatus{Purged}, then the recycling occurs |
| 69 |
in two steps: |
| 70 |
\smallskip |
| 71 |
|
| 72 |
The first is that the Catalog for a Volume must be pruned of all Jobs (i.e. |
| 73 |
Purged) and Files contained on that Volume. |
| 74 |
\smallskip |
| 75 |
|
| 76 |
The second step is the actual recycling of the Volume. Only Volumes marked |
| 77 |
\bstatus{Full} or \bstatus{Used} will be considered for pruning. The Volume will be |
| 78 |
purged, all Jobs and Files associated to this Volume are pruned from Catalog, |
| 79 |
if the \bdirectivename{VolumeRetention} period has expired. When a Volume is marked |
| 80 |
as \bstatus{Purged}, it means that no Catalog records for Jobs and Files reference |
| 81 |
that Volume, and the Volume can be recycled and reused. Please note a Volume |
| 82 |
can be reused even though the \bdirectivename{Volume Retention} period has not expired if the |
| 83 |
Jobs and Files associated to the Volume had been already pruned. Until |
| 84 |
recycling actually occurs, the Volume data remains intact. If no Volumes |
| 85 |
can be found for recycling for any of the reasons stated above, \bacula{} will |
| 86 |
request operator intervention (i.e. it will ask you to label a new volume). |
| 87 |
|
| 88 |
|
| 89 |
A key point mentioned above, that can be a source of frustration, is that |
| 90 |
\bacula{} will only recycle purged Volumes if there is no other appendable |
| 91 |
Volume available, otherwise, it will always write to an appendable Volume |
| 92 |
before recycling even if there are Volumes marked as \bstatus{Purged}. This preserves |
| 93 |
your data as long as possible. So, if you wish to ``force'' \bacula{} to use a |
| 94 |
purged Volume, you must first ensure that no other Volume in the Pool is |
| 95 |
marked \bstatus{Append}. If necessary, you can manually set a volume to \bstatus{Full}. |
| 96 |
The reason for this is that \bacula{} wants to preserve the data on |
| 97 |
your old Volumes (even though purged from the catalog) as long as absolutely |
| 98 |
possible before overwriting it. There are also a number of directives such |
| 99 |
as \bdirectivename{Volume Use Duration} that will automatically mark a volume as \bstatus{Used} |
| 100 |
and thus no longer appendable. |
| 101 |
|
| 102 |
\label{blb:AutoPruning} |
| 103 |
\section{Automatic Pruning} |
| 104 |
\index[general]{Automatic Pruning} |
| 105 |
\index[general]{Pruning!Automatic} |
| 106 |
|
| 107 |
As \bacula{} writes files to a Volume, it keeps a list of files, jobs, and volumes |
| 108 |
in a database called the catalog. Among other things, the database helps |
| 109 |
\bacula{} to decide which files to back up in an incremental or differential |
| 110 |
backup, and helps you locate files on past backups when you want to restore |
| 111 |
something. However, the catalog will grow larger and larger as time goes |
| 112 |
on, and eventually it can become unacceptably large. |
| 113 |
|
| 114 |
\bacula{}'s process for removing entries from the catalog is called Pruning. |
| 115 |
The default is Automatic Pruning, which means that once a Job record |
| 116 |
reaches a certain age (e.g. 30 days old) and a pruning occurs, it will be |
| 117 |
removed from the catalog. Note that Job records that are required for |
| 118 |
current restore won't be removed automatically, and File records are needed |
| 119 |
for VirtualFull and Accurate backups. Once a job has been pruned, you can |
| 120 |
still restore it from the backup Volume, provided that the Volume has not |
| 121 |
been recycled, but one additional step is required: scanning the volume |
| 122 |
with \bcommandname{bscan}. The alternative to Automatic Pruning is Manual Pruning, in |
| 123 |
which you explicitly tell \bacula{} to erase the catalog entries for a volume. |
| 124 |
You'd usually do this when you want to reuse a \bacula{} volume, because |
| 125 |
there's no point in keeping a list of files that USED TO BE on a volume. |
| 126 |
Or, if the catalog is starting to get too big, you could prune the oldest |
| 127 |
jobs to save space. Manual pruning is done with the |
| 128 |
\bxlink{prune}{blb:ManualPruning}{console}{command} in the \bconsoleman{} |
| 129 |
(thanks to Bryce Denney for the above explanation). |
| 130 |
|
| 131 |
\section{Pruning Directives} |
| 132 |
\index[general]{Pruning Directives } |
| 133 |
\index[general]{Directives!Pruning } |
| 134 |
There are three pruning durations. All apply to catalog database Jobs |
| 135 |
and Files records and not to the actual data in a Volume. The pruning |
| 136 |
(or retention) durations are for: Volumes (Jobs and Files records in the |
| 137 |
Volume), Jobs (Job records), and Files (File records). The durations |
| 138 |
inter-depend a bit because if \bacula{} prunes a Volume, it automatically |
| 139 |
removes all the Job records, and, consequently, all the File records. |
| 140 |
Also when a Job record is pruned, all the File records for that |
| 141 |
Job are also pruned (deleted) from the catalog. |
| 142 |
|
| 143 |
Having the File records in the database means that you can examine all the |
| 144 |
files backed up for a particular Job. They take the most space in the catalog |
| 145 |
(probably 90-95\% of the total). When the File records are pruned, the Job |
| 146 |
records can remain, and you can still examine what Jobs ran, but not the |
| 147 |
details of the Files backed up. In addition, without the File records, you |
| 148 |
cannot use the Console \bcommandname{restore} command to restore specific files. |
| 149 |
|
| 150 |
When a Job record is pruned, the Volume (Media record) for that Job can still |
| 151 |
remain in the database, and if you do a \bcommandname{list volumes}, you will see the |
| 152 |
volume information, but the Job records (and its File records) will no longer |
| 153 |
be available. |
| 154 |
|
| 155 |
In each case, pruning removes information about where older files are, but it |
| 156 |
also prevents the catalog from growing to be too large. You choose the |
| 157 |
retention periods in function of how many files you are backing up and the |
| 158 |
time periods you want to keep those records online, and the size of the |
| 159 |
database. You can always re-insert the records (with 98\% of the original data) |
| 160 |
by using \btool{bscan} to scan in a whole Volume or any part of the volume that |
| 161 |
you want. |
| 162 |
|
| 163 |
By setting \bdirectivename{AutoPrune} to \bvalue{yes} you will permit \bacula{} to |
| 164 |
automatically prune Volumes in the Pool when a Job needs a Volume. |
| 165 |
Volume pruning means removing Jobs and Files records from the catalog. |
| 166 |
It does not shrink the size of the Volume or affect the Volume data until |
| 167 |
the Volume gets overwritten. When a Job requests a volume and there are |
| 168 |
no Volumes with Volume Status \bstatus{Append}, \bstatus{Recycle} or \bstatus{Purged} |
| 169 |
available, \bacula{} will begin volume pruning. This means that all Jobs |
| 170 |
that are older than the \bdirectivename{VolumeRetention} period will be pruned from |
| 171 |
every Volume that has Volume Status \bstatus{Full} or \bstatus{Used} and has |
| 172 |
Recycle set to \bvalue{yes}. Pruning consists of deleting the corresponding |
| 173 |
Job, File, and JobMedia records from the catalog database. No |
| 174 |
change to the physical data on the Volume occurs during the pruning process. |
| 175 |
When all files are pruned from a Volume (i.e. no records in the catalog), the |
| 176 |
Volume will be marked as \bstatus{Purged} implying that no Jobs remain on the |
| 177 |
volume. The Pool records that control the pruning are described below. |
| 178 |
|
| 179 |
\begin{bdescription} |
| 180 |
|
| 181 |
\item [AutoPrune = \byesorno{}] |
| 182 |
\index[console]{AutoPrune } |
| 183 |
If AutoPrune is set to \bdefaultvalue{yes} (default), \bacula{} |
| 184 |
will automatically apply the Volume retention period when running a Job and |
| 185 |
it needs a new Volume but no appendable volumes are available. At that point, |
| 186 |
\bacula{} will prune all Volumes that can be pruned (i.e. AutoPrune set) in an |
| 187 |
attempt to find a usable volume. If during the autoprune, all files are |
| 188 |
pruned from the Volume, it will be marked with VolStatus \bstatus{Purged}. The |
| 189 |
default is \bdefaultvalue{yes}. Note, that although the File and Job records may be |
| 190 |
pruned from the catalog, a Volume will be marked Purged (and hence |
| 191 |
ready for recycling) if the Volume status is Append, Full, Used, or Error. |
| 192 |
If the Volume has another status, such as Archive, Read-Only, Disabled, |
| 193 |
Busy, or Cleaning, the Volume status will not be changed to Purged. |
| 194 |
|
| 195 |
\item [Volume Retention = \bbracket{time-period-specification}] |
| 196 |
\index[console]{Volume Retention} |
| 197 |
The Volume Retention record defines the length of time that \bacula{} will |
| 198 |
guarantee that the Volume is not reused (i.e. recycled). As long as |
| 199 |
a Volume has the status {\bf Append}, it will not be recycled even |
| 200 |
though the retention period may have expired, until the status |
| 201 |
is changed to some other status such as Full, Used, Purged, ... |
| 202 |
|
| 203 |
\smallskip |
| 204 |
The retention period when applied starts at the time of the last |
| 205 |
write to the Volume. For a Volume to recycled, the volume must not |
| 206 |
have a status of {\bf Append} and the retention period must have |
| 207 |
expired. Even in that case, the Volume will not be recycled if |
| 208 |
any other appendable Volume is available for the Job to complete. |
| 209 |
As noted above, the Volume status will be marked {\bf Purged} by |
| 210 |
Bacula prior to it being recycled. |
| 211 |
|
| 212 |
Note, when all the Job records that are on the Volume have been removed, |
| 213 |
the Volume will be marked \bstatus{Purged} (i.e. it has no more valid |
| 214 |
Jobs stored on it), and the Volume may be recycled even if the \bdirectivename{Volume Retention} |
| 215 |
period has not expired. |
| 216 |
|
| 217 |
When this time period expires, and if \bdirectivename{AutoPrune} is set to \bvalue{yes}, |
| 218 |
and a new Volume is needed, but no appendable Volume is available, |
| 219 |
\bacula{} will prune (remove) Job records that are older than the specified |
| 220 |
Volume Retention period even if the Job or File retention has not been |
| 221 |
reached. Normally this should not happen since the Volume Retention |
| 222 |
period should always be set greater than the Job Retention period, |
| 223 |
which should be greater than the File Retention period. |
| 224 |
|
| 225 |
The Volume Retention period takes precedence over any Job Retention |
| 226 |
period you have specified in the Client resource. It should also be |
| 227 |
noted, that the Volume Retention period is obtained by reading the |
| 228 |
Catalog Database Media record rather than the Pool resource record. |
| 229 |
This means that if you change the VolumeRetention in the Pool resource |
| 230 |
record (in bacula-dir.conf), you must ensure that the corresponding |
| 231 |
change is made in the catalog by using the \bcommandname{update pool} command. |
| 232 |
Doing so will insure that any new Volumes will be created with the |
| 233 |
changed Volume Retention period. Any existing Volumes will have their |
| 234 |
own copy of the Volume Retention period that can only be changed on a |
| 235 |
Volume by Volume basis using the \bcommandname{update volume} command. |
| 236 |
|
| 237 |
When all Job catalog entries are removed from the volume, its VolStatus is |
| 238 |
set to \bstatus{Purged}. The files remain physically on the Volume until the |
| 239 |
volume is overwritten. |
| 240 |
|
| 241 |
Retention periods are specified in seconds, minutes, hours, days, weeks, |
| 242 |
months, quarters, or years on the record. See the |
| 243 |
\bilink{Configuration chapter}{blb:Time} of this manual for |
| 244 |
additional details of time specification. |
| 245 |
|
| 246 |
The default Volume Retention period is 1 year. |
| 247 |
|
| 248 |
% TODO: if that is the format, should it be in quotes? decide on a style |
| 249 |
|
| 250 |
\item [Recycle = \byesorno{}] |
| 251 |
\index[fd]{Recycle } |
| 252 |
This statement tells \bacula{} whether or not the particular Volume can be |
| 253 |
recycled (i.e. rewritten). If Recycle is set to \bdefaultvalue{no} (the |
| 254 |
default), then even if \bacula{} prunes all the Jobs on the volume and it |
| 255 |
is marked \bstatus{Purged}, it will not consider the volume for recycling. If |
| 256 |
Recycle is set to \bvalue{yes} and all Jobs have been pruned, the volume |
| 257 |
status will be set to \bstatus{Purged} and the volume may then be reused |
| 258 |
when another volume is needed. If the volume is reused, it is relabeled |
| 259 |
with the same Volume Name, however all previous data will be lost. |
| 260 |
\end{bdescription} |
| 261 |
|
| 262 |
It is also possible to ``force'' pruning of all Volumes in the Pool |
| 263 |
associated with a Job by adding \bvalueddirective{Prune Files}{yes} to the Job resource. |
| 264 |
|
| 265 |
\label{blb:Recycling} |
| 266 |
\label{blb:RecyclingAlgorithm} |
| 267 |
\section{Recycling Algorithm} |
| 268 |
\index[general]{Algorithm!Recycling } |
| 269 |
\index[general]{Recycling Algorithm } |
| 270 |
|
| 271 |
When a Job needs a Volume and no appendable, recycled or purged is available, |
| 272 |
\bacula{} starts the recycling algorithm pruning Jobs and Files for the oldest |
| 273 |
Volume in the pool used by the job. After the Volume is pruned, and if |
| 274 |
the \bdirectivename{Recycle} flag is on (\bvalueddirective{Recycle}{yes}) for that Volume, \bacula{} will |
| 275 |
relabel it and write new data on it. |
| 276 |
|
| 277 |
As mentioned above, there are two key points for getting a Volume |
| 278 |
to be recycled. First, the Volume must no longer be marked Append (there |
| 279 |
are a number of directives to automatically make this change), and second |
| 280 |
since the last write on the Volume, one or more of the Retention periods |
| 281 |
must have expired so that there are no more catalog backup job records |
| 282 |
that reference that Volume. Once both those conditions are satisfied, |
| 283 |
the volume can be marked Purged and hence recycled. |
| 284 |
|
| 285 |
The full algorithm that \bacula{} uses when it needs a new Volume is: |
| 286 |
\index[general]{New Volume Algorithm} |
| 287 |
\index[general]{Algorithm!New Volume} |
| 288 |
|
| 289 |
The algorithm described below assumes that AutoPrune is enabled, |
| 290 |
that Recycling is turned on, and that you have defined |
| 291 |
appropriate Retention periods, or used the defaults for all these |
| 292 |
items. |
| 293 |
|
| 294 |
\begin{bitemize} |
| 295 |
\item If the request is for an Autochanger device, look only |
| 296 |
for Volumes in the Autochanger (i.e. with InChanger set and that have |
| 297 |
the correct Storage device). |
| 298 |
\item Search the Pool for a Volume with VolStatus=Append (if there is more |
| 299 |
than one, the Volume with the oldest date last written is chosen. If |
| 300 |
two have the same date then the one with the lowest MediaId is chosen). |
| 301 |
\item Search the Pool for a Volume with VolStatus=Recycle and the InChanger |
| 302 |
flag is set true (if there is more than one, the Volume with the oldest |
| 303 |
date last written is chosen. If two have the same date then the one |
| 304 |
with the lowest MediaId is chosen). |
| 305 |
\item Search the Pool for a Volume with VolStatus=Purged and try recycling any purged Volumes. |
| 306 |
\item Prune volumes applying Volume retention period (Volumes with VolStatus |
| 307 |
\bstatus{Full} or \bstatus{Used} are pruned). Note, when all the File and Job |
| 308 |
records are pruned from a Volume, the Volume will be marked \bstatus{Purged} and |
| 309 |
this may be prior to the expiration of the Volume retention period. |
| 310 |
\item Prune the oldest Volume if \bvalueddirective{RecycleOldestVolume}{yes} (the Volume with the |
| 311 |
oldest LastWritten date and VolStatus equal to \bstatus{Full} or \bstatus{Used} is chosen). |
| 312 |
This record ensures that all retention periods are properly respected. |
| 313 |
\item Purge the oldest Volume if PurgeOldestVolume=yes (the Volume with the |
| 314 |
oldest LastWritten date and VolStatus equal to \bstatus{Full} or \bstatus{Used} is chosen). |
| 315 |
We strongly recommend against the use of \bdirectivename{PurgeOldestVolume} as |
| 316 |
it can quite easily lead to loss of current backup data. |
| 317 |
\item Search for a Volume in the Scratch Pool (the default Scratch pool |
| 318 |
or another Scratch Pool defined for the pool) and if found |
| 319 |
move it to the current Pool for the Job and use it. Note, when |
| 320 |
the Scratch Volume is moved into the current Pool, the basic |
| 321 |
Pool defaults are applied as if it is a newly labeled Volume |
| 322 |
(equivalent to an \bcommandname{update volume from pool} command). |
| 323 |
\item If we were looking for Volumes in the Autochanger, go back to step 2 above, but this time, look for any Volume whether or not it is in the Autochanger. |
| 324 |
\item Attempt to create a new Volume if automatic labeling is enabled and the maximum number of Volumes is not reached. |
| 325 |
\item Give up and ask operator. |
| 326 |
\end{bitemize} |
| 327 |
|
| 328 |
The above occurs when \bacula{} has finished writing a Volume or when no Volume |
| 329 |
is present in the drive. |
| 330 |
|
| 331 |
On the other hand, if you have inserted a different Volume after the last |
| 332 |
job, and \bacula{} recognizes the Volume as valid, the Storage daemon will |
| 333 |
request authorization from the Director to use this Volume. In this case, |
| 334 |
if you have set \bvalueddirective{Recycle Current Volume = yes} and the Volume is marked |
| 335 |
as Used or Full, Bacula will prune the volume and if all jobs were removed |
| 336 |
during the pruning (respecting the retention periods), the Volume will be |
| 337 |
recycled and re-used. |
| 338 |
|
| 339 |
\smallskip |
| 340 |
|
| 341 |
If you want to strictly observe the \bdirectivename{Volume Retention Period} under all |
| 342 |
circumstances -- i.e. you have a Volume with critical data, you must set |
| 343 |
\bvalueddirective{Recycle}{no}. However, in doing so, you must manually do the |
| 344 |
recycling of the Volume for it to be used again. |
| 345 |
|
| 346 |
The recycling algorithm in this case is: |
| 347 |
\begin{bitemize} |
| 348 |
\item If the VolStatus is \bstatus{Append} or \bstatus{Recycle} |
| 349 |
is set, the volume will be used. |
| 350 |
\item If \bdirectivename{Recycle Current Volume} is set and the volume is marked \bstatus{Full} |
| 351 |
or \bstatus{Used}, \bacula{} will prune the volume (applying the retention |
| 352 |
period). If all Jobs are pruned from the volume, it will be recycled. |
| 353 |
\end{bitemize} |
| 354 |
|
| 355 |
This permits users to manually change the Volume every day and load volumes in |
| 356 |
an order different from what is in the catalog, and if the volume does not |
| 357 |
contain a current copy of your backup data, it will be used. |
| 358 |
|
| 359 |
A few points to keep in mind: |
| 360 |
|
| 361 |
\begin{benumerate} |
| 362 |
\item If a pool doesn't have maximum volumes defined then \bacula{} will prefer to |
| 363 |
demand new volumes over forcibly purging older volumes. |
| 364 |
|
| 365 |
\item If volumes become free through pruning, then they get marked as |
| 366 |
\bstatus{Purged} and are immediately available for recycling -- these will |
| 367 |
be used in preference to creating new volumes. |
| 368 |
|
| 369 |
\item If the Job, File, and Volume retention periods are different, then |
| 370 |
it's common to see a volume with no files or jobs listed in the database, |
| 371 |
but which is still not marked as \bstatus{Purged}, if \bdirectivename{Autoprune} is set to \bvalue{no} |
| 372 |
in the pool resource. |
| 373 |
\end{benumerate} |
| 374 |
|
| 375 |
|
| 376 |
\section{Recycle Status} |
| 377 |
\index[general]{Status!Recycle } |
| 378 |
\index[general]{Recycle Status } |
| 379 |
|
| 380 |
Each Volume inherits the Recycle status (yes or no) from the Pool resource |
| 381 |
record when the Media record is created (normally when the Volume is |
| 382 |
labeled). This Recycle status is stored in the Media record of the |
| 383 |
Catalog. Using the Console program, you may subsequently change the |
| 384 |
Recycle status for each Volume. |
| 385 |
For example in the following output from \bcommandname{list volumes}: |
| 386 |
|
| 387 |
\begin{bVerbatim} |
| 388 |
+----------+-------+--------+---------+------------+--------+-----+ |
| 389 |
| VolumeNa | Media | VolSta | VolByte | LastWritte | VolRet | Rec | |
| 390 |
+----------+-------+--------+---------+------------+--------+-----+ |
| 391 |
| File0001 | File | Full | 4190055 | 2002-05-25 | 14400 | 1 | |
| 392 |
| File0002 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 | |
| 393 |
| File0003 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 | |
| 394 |
| File0004 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 | |
| 395 |
| File0005 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 | |
| 396 |
| File0006 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 | |
| 397 |
| File0007 | File | Purged | 1896466 | 2002-05-26 | 14400 | 1 | |
| 398 |
+----------+-------+--------+---------+------------+--------+-----+ |
| 399 |
\end{bVerbatim} |
| 400 |
|
| 401 |
all the volumes are marked as recyclable, and the last Volume, |
| 402 |
\bvalue{File0007} has been purged, so it may be immediately recycled. The other |
| 403 |
volumes are all marked recyclable and when their Volume Retention period |
| 404 |
(14400 seconds or four hours) expires, they will be eligible for pruning, |
| 405 |
and possibly recycling. Even though Volume \bvalue{File0007} has been purged, |
| 406 |
all the data on the Volume is still recoverable. A purged Volume simply |
| 407 |
means that there are no entries in the Catalog. Even if the Volume Status |
| 408 |
is changed to \bstatus{Recycle}, the data on the Volume will be recoverable. |
| 409 |
The data is lost only when the Volume is re-labeled and re-written. |
| 410 |
|
| 411 |
To modify Volume \bvalue{File0001} so that it cannot be recycled, you use the |
| 412 |
\bcommandname{update volume pool=File} command in the console program, or simply |
| 413 |
\bcommandname{update} and \bacula{} will prompt you for the information. |
| 414 |
|
| 415 |
\begin{bVerbatim} |
| 416 |
+----------+------+-------+---------+-------------+-------+-----+ |
| 417 |
| VolumeNa | Media| VolSta| VolByte | LastWritten | VolRet| Rec | |
| 418 |
+----------+------+-------+---------+-------------+-------+-----+ |
| 419 |
| File0001 | File | Full | 4190055 | 2002-05-25 | 14400 | 0 | |
| 420 |
| File0002 | File | Full | 1897236 | 2002-05-26 | 14400 | 1 | |
| 421 |
| File0003 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 | |
| 422 |
| File0004 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 | |
| 423 |
| File0005 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 | |
| 424 |
| File0006 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 | |
| 425 |
| File0007 | File | Purged| 1896466 | 2002-05-26 | 14400 | 1 | |
| 426 |
+----------+------+-------+---------+-------------+-------+-----+ |
| 427 |
\end{bVerbatim} |
| 428 |
|
| 429 |
In this case, \textbf{File0001} will never be automatically recycled. The same |
| 430 |
effect can be achieved by setting the Volume Status to Read-Only. |
| 431 |
|
| 432 |
As you have noted, the Volume Status (VolStatus) column in the |
| 433 |
catalog database contains the current status of the Volume, which |
| 434 |
is normally maintained automatically by \bacula{}. To give you an |
| 435 |
idea of some of the values it can take during the life cycle of |
| 436 |
a Volume, here is a picture of the process. |
| 437 |
|
| 438 |
\begin{bVerbatim} |
| 439 |
A typical volume life cycle is like this: |
| 440 |
|
| 441 |
because job count or size limit exceeded |
| 442 |
Append ------------------------------------> Used/Full |
| 443 |
^ | |
| 444 |
| First Job writes to Retention time passed | |
| 445 |
| the volume and recycling takes | |
| 446 |
| place | |
| 447 |
| v |
| 448 |
Recycled <-------------------------------------- Purged |
| 449 |
Volume is selected for reuse |
| 450 |
\end{bVerbatim} |
| 451 |
|
| 452 |
|
| 453 |
\section{Making \bacula{} Use a Single Tape} |
| 454 |
\label{blb:singletape} |
| 455 |
\index[general]{Tape!Making \bacula{} Use a Single} |
| 456 |
\index[general]{Making \bacula{} Use a Single Tape} |
| 457 |
|
| 458 |
Most people will want \bacula{} to fill a tape and when it is full, a new tape |
| 459 |
will be mounted, and so on. However, as an extreme example, it is possible |
| 460 |
for \bacula{} to write on a single tape, and every night to rewrite it. To |
| 461 |
get this to work, you must do two things: first, set the \bdirectivename{VolumeRetention} to |
| 462 |
less than your save period (one day), and the second item is to make \bacula{} |
| 463 |
mark the tape as full after using it once. This is done using |
| 464 |
\bvalueddirective{UseVolumeOnce}{yes}. If this latter record is not used and the tape is |
| 465 |
not full after the first time it is written, \bacula{} will simply append to |
| 466 |
the tape and eventually request another volume. Using the tape only once, |
| 467 |
forces the tape to be marked |
| 468 |
\bstatus{Full} after each use, and the next time \bacula{} runs, it will |
| 469 |
recycle the tape. |
| 470 |
|
| 471 |
An example Pool resource that does this is: |
| 472 |
|
| 473 |
\begin{bVerbatim} |
| 474 |
Pool { |
| 475 |
Name = DDS-4 |
| 476 |
Use Volume Once = yes |
| 477 |
Pool Type = Backup |
| 478 |
AutoPrune = yes |
| 479 |
VolumeRetention = 12h # expire after 12 hours |
| 480 |
Recycle = yes |
| 481 |
} |
| 482 |
\end{bVerbatim} |
| 483 |
|
| 484 |
\section{Daily, Weekly, Monthly Tape Usage Example} |
| 485 |
\label{blb:usageexample} |
| 486 |
\index[general]{Daily, Weekly, Monthly Tape Usage Example } |
| 487 |
\index[general]{Example!Daily Weekly Monthly Tape Usage } |
| 488 |
|
| 489 |
This example is meant to show you how one could define a fixed set of volumes |
| 490 |
that \bacula{} will rotate through on a regular schedule. There are an infinite |
| 491 |
number of such schemes, all of which have various advantages and |
| 492 |
disadvantages. Note: these volumes may either be tape volumes or disk |
| 493 |
volumes. |
| 494 |
|
| 495 |
We start with the following assumptions: |
| 496 |
|
| 497 |
\begin{bitemize} |
| 498 |
\item A single tape has more than enough capacity to do a full save. |
| 499 |
\item There are ten tapes that are used on a daily basis for incremental |
| 500 |
backups. They are prelabeled Daily1 ... Daily10. |
| 501 |
\item There are four tapes that are used on a weekly basis for full backups. |
| 502 |
They are labeled Week1 ... Week4. |
| 503 |
\item There are 12 tapes that are used on a monthly basis for full backups. |
| 504 |
They are numbered Month1 ... Month12 |
| 505 |
\item A full backup is done every Saturday evening (tape inserted Friday |
| 506 |
evening before leaving work). |
| 507 |
\item No backups are done over the weekend (this is easy to change). |
| 508 |
\item The first Friday of each month, a Monthly tape is used for the Full |
| 509 |
backup. |
| 510 |
\item Incremental backups are done Monday - Friday (actually Tue-Fri |
| 511 |
mornings). |
| 512 |
% TODO: why this ``actually''? does this need to be explained? |
| 513 |
\end{bitemize} |
| 514 |
|
| 515 |
We start the system by doing a Full save to one of the weekly volumes or one |
| 516 |
of the monthly volumes. The next morning, we remove the tape and insert a |
| 517 |
Daily tape. Friday evening, we remove the Daily tape and insert the next tape |
| 518 |
in the Weekly series. Monday, we remove the Weekly tape and re-insert the |
| 519 |
Daily tape. On the first Friday of the next month, we insert the next Monthly |
| 520 |
tape in the series rather than a Weekly tape, then continue. When a Daily tape |
| 521 |
finally fills up, \bacula{} will request the next one in the series, and |
| 522 |
the next day when you notice the email message, you will mount it and |
| 523 |
\bacula{} will finish the unfinished incremental backup. |
| 524 |
|
| 525 |
What does this give? Well, at any point, you will have the last complete |
| 526 |
Full save plus several Incremental saves. For any given file you want to |
| 527 |
recover (or your whole system), you will have a copy of that file every day |
| 528 |
for at least the last 14 days. For older versions, you will have at least three |
| 529 |
and probably four Friday full saves of that file, and going back further, you |
| 530 |
will have a copy of that file made on the beginning of the month for at least |
| 531 |
a year. |
| 532 |
|
| 533 |
So you have copies of any file (or your whole system) for at least a year, but |
| 534 |
as you go back in time, the time between copies increases from daily to weekly |
| 535 |
to monthly. |
| 536 |
|
| 537 |
What would the \bacula{} configuration look like to implement such a scheme? |
| 538 |
|
| 539 |
\begin{bVerbatim} |
| 540 |
Schedule { |
| 541 |
Name = "NightlySave" |
| 542 |
Run = Level=Full Pool=Monthly 1st sat at 03:05 |
| 543 |
Run = Level=Full Pool=Weekly 2nd-5th sat at 03:05 |
| 544 |
Run = Level=Incremental Pool=Daily tue-fri at 03:05 |
| 545 |
} |
| 546 |
Job { |
| 547 |
Name = "NightlySave" |
| 548 |
Type = Backup |
| 549 |
Level = Full |
| 550 |
Client = LocalMachine |
| 551 |
FileSet = "File Set" |
| 552 |
Messages = Standard |
| 553 |
Storage = DDS-4 |
| 554 |
Pool = Daily |
| 555 |
Schedule = "NightlySave" |
| 556 |
} |
| 557 |
# Definition of file storage device |
| 558 |
Storage { |
| 559 |
Name = DDS-4 |
| 560 |
Address = localhost |
| 561 |
SDPort = 9103 |
| 562 |
Password = XXXXXXXXXXXXX |
| 563 |
Device = FileStorage |
| 564 |
Media Type = 8mm |
| 565 |
} |
| 566 |
FileSet { |
| 567 |
Name = "File Set" |
| 568 |
Include { |
| 569 |
Options { signature=MD5 } |
| 570 |
File = fffffffffffffffff |
| 571 |
} |
| 572 |
Exclude { File=*.o } |
| 573 |
} |
| 574 |
Pool { |
| 575 |
Name = Daily |
| 576 |
Pool Type = Backup |
| 577 |
AutoPrune = yes |
| 578 |
VolumeRetention = 10d # recycle in 10 days |
| 579 |
Maximum Volumes = 10 |
| 580 |
Recycle = yes |
| 581 |
} |
| 582 |
Pool { |
| 583 |
Name = Weekly |
| 584 |
Use Volume Once = yes |
| 585 |
Pool Type = Backup |
| 586 |
AutoPrune = yes |
| 587 |
VolumeRetention = 30d # recycle in 30 days (default) |
| 588 |
Recycle = yes |
| 589 |
} |
| 590 |
Pool { |
| 591 |
Name = Monthly |
| 592 |
Use Volume Once = yes |
| 593 |
Pool Type = Backup |
| 594 |
AutoPrune = yes |
| 595 |
VolumeRetention = 365d # recycle in 1 year |
| 596 |
Recycle = yes |
| 597 |
} |
| 598 |
\end{bVerbatim} |
| 599 |
|
| 600 |
\section{ Automatic Pruning and Recycling Example} |
| 601 |
\label{blb:PruningExample} |
| 602 |
\index[general]{Automatic Pruning and Recycling Example } |
| 603 |
\index[general]{Example!Automatic Pruning and Recycling } |
| 604 |
|
| 605 |
Perhaps the best way to understand the various resource records that come |
| 606 |
into play during automatic pruning and recycling is to run a Job that goes |
| 607 |
through the whole cycle. If you add the following resources to your |
| 608 |
Director's configuration file: |
| 609 |
|
| 610 |
\begin{bVerbatim} |
| 611 |
Schedule { |
| 612 |
Name = "30 minute cycle" |
| 613 |
Run = Level=Full Pool=File Messages=Standard Storage=File |
| 614 |
hourly at 0:05 |
| 615 |
Run = Level=Full Pool=File Messages=Standard Storage=File |
| 616 |
hourly at 0:35 |
| 617 |
} |
| 618 |
Job { |
| 619 |
Name = "Filetest" |
| 620 |
Type = Backup |
| 621 |
Level = Full |
| 622 |
Client=XXXXXXXXXX |
| 623 |
FileSet="Test Files" |
| 624 |
Messages = Standard |
| 625 |
Storage = File |
| 626 |
Pool = File |
| 627 |
Schedule = "30 minute cycle" |
| 628 |
} |
| 629 |
# Definition of file storage device |
| 630 |
Storage { |
| 631 |
Name = File |
| 632 |
Address = XXXXXXXXXXX |
| 633 |
SDPort = 9103 |
| 634 |
Password = XXXXXXXXXXXXX |
| 635 |
Device = FileStorage |
| 636 |
Media Type = File |
| 637 |
} |
| 638 |
FileSet { |
| 639 |
Name = "File Set" |
| 640 |
Include { |
| 641 |
Options { signature=MD5 } |
| 642 |
File = fffffffffffffffff |
| 643 |
} |
| 644 |
Exclude { File=*.o } |
| 645 |
} |
| 646 |
Pool { |
| 647 |
Name = File |
| 648 |
Use Volume Once = yes |
| 649 |
Pool Type = Backup |
| 650 |
LabelFormat = "File" |
| 651 |
AutoPrune = yes |
| 652 |
VolumeRetention = 4h |
| 653 |
Maximum Volumes = 12 |
| 654 |
Recycle = yes |
| 655 |
} |
| 656 |
\end{bVerbatim} |
| 657 |
|
| 658 |
Where you will need to replace the \textbf{ffffffffff}'s by the appropriate files |
| 659 |
to be saved for your configuration. For the FileSet Include, choose a |
| 660 |
directory that has one or two megabytes maximum since there will probably be |
| 661 |
approximately eight copies of the directory that \bacula{} will cycle through. |
| 662 |
|
| 663 |
In addition, you will need to add the following to your Storage daemon's |
| 664 |
configuration file: |
| 665 |
|
| 666 |
\begin{bVerbatim} |
| 667 |
Device { |
| 668 |
Name = FileStorage |
| 669 |
Media Type = File |
| 670 |
Archive Device = /tmp |
| 671 |
LabelMedia = yes; |
| 672 |
Random Access = Yes; |
| 673 |
AutomaticMount = yes; |
| 674 |
RemovableMedia = no; |
| 675 |
AlwaysOpen = no; |
| 676 |
} |
| 677 |
\end{bVerbatim} |
| 678 |
|
| 679 |
With the above resources, \bacula{} will start a Job every half hour that saves a |
| 680 |
copy of the directory you chose to /tmp/File0001 ... /tmp/File0012. After 4 |
| 681 |
hours, \bacula{} will start recycling the backup Volumes (/tmp/File0001 ...). You |
| 682 |
should see this happening in the output produced. \bacula{} will automatically |
| 683 |
create the Volumes (Files) the first time it uses them. |
| 684 |
|
| 685 |
To turn it off, either delete all the resources you've added, or simply |
| 686 |
comment out the \bresourcename{Schedule} record in the \bresourcename{Job} resource. |
| 687 |
|
| 688 |
\section{Manually Recycling Volumes} |
| 689 |
\label{blb:manualrecycling} |
| 690 |
\index[general]{Volumes!Manually Recycling } |
| 691 |
\index[general]{Manually Recycling Volumes } |
| 692 |
|
| 693 |
Although automatic recycling of Volumes is implemented in version 1.20 and |
| 694 |
later (see the |
| 695 |
\bilink{Automatic Recycling of Volumes}{blb:RecyclingChapter} chapter of |
| 696 |
this manual), you may want to manually force reuse (recycling) of a Volume. |
| 697 |
|
| 698 |
Assuming that you want to keep the Volume name, but you simply want to write |
| 699 |
new data on the tape, the steps to take are: |
| 700 |
|
| 701 |
\begin{bitemize} |
| 702 |
\item Use the \bcommandname{update volume} command in the Console to ensure that the |
| 703 |
\bstatus{Recycle} field is set to \bvalue{1} |
| 704 |
\item Use the \bcommandname{purge jobs volume} command in the Console to mark the |
| 705 |
Volume as \bstatus{Purged}. Check by using \bcommandname{list volumes}. |
| 706 |
\end{bitemize} |
| 707 |
|
| 708 |
Once the Volume is marked Purged, it will be recycled the next time a Volume |
| 709 |
is needed. |
| 710 |
|
| 711 |
If you wish to reuse the volume by giving it a new name, follow the following |
| 712 |
steps: |
| 713 |
|
| 714 |
\begin{bitemize} |
| 715 |
\item Use the \bcommandname{purge jobs volume} command in the Console to mark the |
| 716 |
Volume as \bstatus{Purged}. Check by using \bcommandname{list volumes}. |
| 717 |
\item Use the Console \bcommandname{relabel} |
| 718 |
command to relabel the Volume. |
| 719 |
\end{bitemize} |
| 720 |
|
| 721 |
Please note that the \bcommandname{relabel} command applies only to tape Volumes. |
| 722 |
|
| 723 |
For \bacula{} versions prior to 1.30 or to manually relabel the Volume, use the |
| 724 |
instructions below: |
| 725 |
|
| 726 |
\begin{bitemize} |
| 727 |
\item Use the \bcommandname{delete volume} command in the Console to delete the Volume |
| 728 |
from the Catalog. |
| 729 |
\item If a different tape is mounted, use the \bcommandname{unmount} command, |
| 730 |
remove the tape, and insert the tape to be renamed. |
| 731 |
\item Write an EOF mark in the tape using the following commands: |
| 732 |
|
| 733 |
\begin{bVerbatim} |
| 734 |
mt -f /dev/nst0 rewind |
| 735 |
mt -f /dev/nst0 weof |
| 736 |
\end{bVerbatim} |
| 737 |
|
| 738 |
where you replace \bfilename{/dev/nst0} with the appropriate device name on your |
| 739 |
system. |
| 740 |
\item Use the \bcommandname{label} command to write a new label to the tape and to |
| 741 |
enter it in the catalog. |
| 742 |
\end{bitemize} |
| 743 |
|
| 744 |
Please be aware that the \bcommandname{delete} command can be dangerous. Once it is |
| 745 |
done, to recover the File records, you must either restore your database as it |
| 746 |
was before the \bcommandname{delete} command, or use the \btool{bscan} utility program to |
| 747 |
scan the tape and recreate the database entries. |