Debian Codegraph @

manuals/en/main/recycling.tex

raw
1 s\chapter{Automatic Volume Recycling}
2 \label{blb:RecyclingChapter}
3 \index[general]{Recycling!Automatic Volume }
4 \index[general]{Automatic Volume Recycling }
5
6 By default, once \bacula{} starts writing a Volume, it may append to the
7 volume, but it will not overwrite the existing data thus destroying it.
8 However when \bacula{} \textbf{recycles} a Volume, the Volume becomes available
9 for being reused, and \bacula{} can at some later time overwrite the previous
10 contents of that Volume. At that point all previous data on that Volume
11 will be lost. If the Volume is a tape, the tape will be rewritten from the
12 beginning. If the Volume is a disk file, the file will be truncated before
13 being rewritten.
14
15 You may not want Bacula to automatically recycle (reuse) Volumes. Doing so
16 may require a large number of Volumes though. However, it is also possible
17 to manually recycle Volumes so that they may be reused. For more on manual
18 recycling, see the section entitled \bilink{Manually Recycling Volumes}{blb:manualrecycling} below in this chapter.
19
20 Most people prefer to have a Pool of Volumes that are used for daily backups and
21 recycled once a week, another Pool of Volumes that are used for Full backups
22 once a week and recycled monthly, and finally a Pool of Volumes that are used
23 once a month and recycled after a year or two. With a scheme like this, the
24 number of Volumes in your pool or pools remains constant.
25
26 By properly defining your Volume Pools with appropriate Retention periods,
27 \bacula{} can manage the recycling (such as defined above) automatically.
28
29 Automatic recycling of Volumes is controlled by four records in the \bresourcename{Pool}
30 resource definition in the Director's configuration file. These four
31 records are:
32
33 \begin{bitemize}
34 \item AutoPrune = yes
35 \item VolumeRetention = \bbracket{time}
36 \item Recycle = yes
37 \item RecyclePool = \bbracket{APool}
38 \item ScratchPool = \bbracket{APool}
39 \end{bitemize}
40
41 The above first three directives are all you need assuming that you fill
42 each of your Volumes then wait the Volume Retention period before
43 reusing them, providing there is some non-pruned Jobs or Files on the
44 Volume. \bdirectivename{Recycle Pool} and \bdirectivename{Scratch Pool} directives are not required. If not
45 defined, \bacula{} will recycle Volumes and keep them in the current Pool.
46
47 If you want \bacula{} to stop using a Volume and recycle
48 it before it is full, you will need to use one or more additional
49 directives such as:
50
51 \begin{bitemize}
52 \item \bvalueddirective{Use Volume Once}{yes}
53 \item \bvalueddirective{Volume Use Duration}{ttt}
54 \item \bvalueddirective{Maximum Volume Jobs}{nnn}
55 \item \bvalueddirective{Maximum Volume Bytes}{mmm}
56 \end{bitemize}
57
58 Please see below and
59 the \bilink{Basic Volume Management}{blb:DiskChapter} chapter
60 of this manual for more complete examples.
61
62 Automatic recycling of Volumes is performed by \bacula{} only when it wants a
63 new Volume and no appendable Volumes are available in the Pool. It will then
64 search the Pool for any Volumes with the \bstatus{Recycle} flag set and the
65 Volume Status is \bstatus{Purged}. At that point, it will choose the oldest
66 purged volume and recycle it.
67
68 If there are no volumes with status \bstatus{Purged}, then the recycling occurs
69 in two steps:
70 \smallskip
71
72 The first is that the Catalog for a Volume must be pruned of all Jobs (i.e.
73 Purged) and Files contained on that Volume.
74 \smallskip
75
76 The second step is the actual recycling of the Volume. Only Volumes marked
77 \bstatus{Full} or \bstatus{Used} will be considered for pruning. The Volume will be
78 purged, all Jobs and Files associated to this Volume are pruned from Catalog,
79 if the \bdirectivename{VolumeRetention} period has expired. When a Volume is marked
80 as \bstatus{Purged}, it means that no Catalog records for Jobs and Files reference
81 that Volume, and the Volume can be recycled and reused. Please note a Volume
82 can be reused even though the \bdirectivename{Volume Retention} period has not expired if the
83 Jobs and Files associated to the Volume had been already pruned. Until
84 recycling actually occurs, the Volume data remains intact. If no Volumes
85 can be found for recycling for any of the reasons stated above, \bacula{} will
86 request operator intervention (i.e. it will ask you to label a new volume).
87
88
89 A key point mentioned above, that can be a source of frustration, is that
90 \bacula{} will only recycle purged Volumes if there is no other appendable
91 Volume available, otherwise, it will always write to an appendable Volume
92 before recycling even if there are Volumes marked as \bstatus{Purged}. This preserves
93 your data as long as possible. So, if you wish to ``force'' \bacula{} to use a
94 purged Volume, you must first ensure that no other Volume in the Pool is
95 marked \bstatus{Append}. If necessary, you can manually set a volume to \bstatus{Full}.
96 The reason for this is that \bacula{} wants to preserve the data on
97 your old Volumes (even though purged from the catalog) as long as absolutely
98 possible before overwriting it. There are also a number of directives such
99 as \bdirectivename{Volume Use Duration} that will automatically mark a volume as \bstatus{Used}
100 and thus no longer appendable.
101
102 \label{blb:AutoPruning}
103 \section{Automatic Pruning}
104 \index[general]{Automatic Pruning}
105 \index[general]{Pruning!Automatic}
106
107 As \bacula{} writes files to a Volume, it keeps a list of files, jobs, and volumes
108 in a database called the catalog. Among other things, the database helps
109 \bacula{} to decide which files to back up in an incremental or differential
110 backup, and helps you locate files on past backups when you want to restore
111 something. However, the catalog will grow larger and larger as time goes
112 on, and eventually it can become unacceptably large.
113
114 \bacula{}'s process for removing entries from the catalog is called Pruning.
115 The default is Automatic Pruning, which means that once a Job record
116 reaches a certain age (e.g. 30 days old) and a pruning occurs, it will be
117 removed from the catalog. Note that Job records that are required for
118 current restore won't be removed automatically, and File records are needed
119 for VirtualFull and Accurate backups. Once a job has been pruned, you can
120 still restore it from the backup Volume, provided that the Volume has not
121 been recycled, but one additional step is required: scanning the volume
122 with \bcommandname{bscan}. The alternative to Automatic Pruning is Manual Pruning, in
123 which you explicitly tell \bacula{} to erase the catalog entries for a volume.
124 You'd usually do this when you want to reuse a \bacula{} volume, because
125 there's no point in keeping a list of files that USED TO BE on a volume.
126 Or, if the catalog is starting to get too big, you could prune the oldest
127 jobs to save space. Manual pruning is done with the
128 \bxlink{prune}{blb:ManualPruning}{console}{command} in the \bconsoleman{}
129 (thanks to Bryce Denney for the above explanation).
130
131 \section{Pruning Directives}
132 \index[general]{Pruning Directives }
133 \index[general]{Directives!Pruning }
134 There are three pruning durations. All apply to catalog database Jobs
135 and Files records and not to the actual data in a Volume. The pruning
136 (or retention) durations are for: Volumes (Jobs and Files records in the
137 Volume), Jobs (Job records), and Files (File records). The durations
138 inter-depend a bit because if \bacula{} prunes a Volume, it automatically
139 removes all the Job records, and, consequently, all the File records.
140 Also when a Job record is pruned, all the File records for that
141 Job are also pruned (deleted) from the catalog.
142
143 Having the File records in the database means that you can examine all the
144 files backed up for a particular Job. They take the most space in the catalog
145 (probably 90-95\% of the total). When the File records are pruned, the Job
146 records can remain, and you can still examine what Jobs ran, but not the
147 details of the Files backed up. In addition, without the File records, you
148 cannot use the Console \bcommandname{restore} command to restore specific files.
149
150 When a Job record is pruned, the Volume (Media record) for that Job can still
151 remain in the database, and if you do a \bcommandname{list volumes}, you will see the
152 volume information, but the Job records (and its File records) will no longer
153 be available.
154
155 In each case, pruning removes information about where older files are, but it
156 also prevents the catalog from growing to be too large. You choose the
157 retention periods in function of how many files you are backing up and the
158 time periods you want to keep those records online, and the size of the
159 database. You can always re-insert the records (with 98\% of the original data)
160 by using \btool{bscan} to scan in a whole Volume or any part of the volume that
161 you want.
162
163 By setting \bdirectivename{AutoPrune} to \bvalue{yes} you will permit \bacula{} to
164 automatically prune Volumes in the Pool when a Job needs a Volume.
165 Volume pruning means removing Jobs and Files records from the catalog.
166 It does not shrink the size of the Volume or affect the Volume data until
167 the Volume gets overwritten. When a Job requests a volume and there are
168 no Volumes with Volume Status \bstatus{Append}, \bstatus{Recycle} or \bstatus{Purged}
169 available, \bacula{} will begin volume pruning. This means that all Jobs
170 that are older than the \bdirectivename{VolumeRetention} period will be pruned from
171 every Volume that has Volume Status \bstatus{Full} or \bstatus{Used} and has
172 Recycle set to \bvalue{yes}. Pruning consists of deleting the corresponding
173 Job, File, and JobMedia records from the catalog database. No
174 change to the physical data on the Volume occurs during the pruning process.
175 When all files are pruned from a Volume (i.e. no records in the catalog), the
176 Volume will be marked as \bstatus{Purged} implying that no Jobs remain on the
177 volume. The Pool records that control the pruning are described below.
178
179 \begin{bdescription}
180
181 \item [AutoPrune = \byesorno{}]
182 \index[console]{AutoPrune }
183 If AutoPrune is set to \bdefaultvalue{yes} (default), \bacula{}
184 will automatically apply the Volume retention period when running a Job and
185 it needs a new Volume but no appendable volumes are available. At that point,
186 \bacula{} will prune all Volumes that can be pruned (i.e. AutoPrune set) in an
187 attempt to find a usable volume. If during the autoprune, all files are
188 pruned from the Volume, it will be marked with VolStatus \bstatus{Purged}. The
189 default is \bdefaultvalue{yes}. Note, that although the File and Job records may be
190 pruned from the catalog, a Volume will be marked Purged (and hence
191 ready for recycling) if the Volume status is Append, Full, Used, or Error.
192 If the Volume has another status, such as Archive, Read-Only, Disabled,
193 Busy, or Cleaning, the Volume status will not be changed to Purged.
194
195 \item [Volume Retention = \bbracket{time-period-specification}]
196 \index[console]{Volume Retention}
197 The Volume Retention record defines the length of time that \bacula{} will
198 guarantee that the Volume is not reused (i.e. recycled). As long as
199 a Volume has the status {\bf Append}, it will not be recycled even
200 though the retention period may have expired, until the status
201 is changed to some other status such as Full, Used, Purged, ...
202
203 \smallskip
204 The retention period when applied starts at the time of the last
205 write to the Volume. For a Volume to recycled, the volume must not
206 have a status of {\bf Append} and the retention period must have
207 expired. Even in that case, the Volume will not be recycled if
208 any other appendable Volume is available for the Job to complete.
209 As noted above, the Volume status will be marked {\bf Purged} by
210 Bacula prior to it being recycled.
211
212 Note, when all the Job records that are on the Volume have been removed,
213 the Volume will be marked \bstatus{Purged} (i.e. it has no more valid
214 Jobs stored on it), and the Volume may be recycled even if the \bdirectivename{Volume Retention}
215 period has not expired.
216
217 When this time period expires, and if \bdirectivename{AutoPrune} is set to \bvalue{yes},
218 and a new Volume is needed, but no appendable Volume is available,
219 \bacula{} will prune (remove) Job records that are older than the specified
220 Volume Retention period even if the Job or File retention has not been
221 reached. Normally this should not happen since the Volume Retention
222 period should always be set greater than the Job Retention period,
223 which should be greater than the File Retention period.
224
225 The Volume Retention period takes precedence over any Job Retention
226 period you have specified in the Client resource. It should also be
227 noted, that the Volume Retention period is obtained by reading the
228 Catalog Database Media record rather than the Pool resource record.
229 This means that if you change the VolumeRetention in the Pool resource
230 record (in bacula-dir.conf), you must ensure that the corresponding
231 change is made in the catalog by using the \bcommandname{update pool} command.
232 Doing so will insure that any new Volumes will be created with the
233 changed Volume Retention period. Any existing Volumes will have their
234 own copy of the Volume Retention period that can only be changed on a
235 Volume by Volume basis using the \bcommandname{update volume} command.
236
237 When all Job catalog entries are removed from the volume, its VolStatus is
238 set to \bstatus{Purged}. The files remain physically on the Volume until the
239 volume is overwritten.
240
241 Retention periods are specified in seconds, minutes, hours, days, weeks,
242 months, quarters, or years on the record. See the
243 \bilink{Configuration chapter}{blb:Time} of this manual for
244 additional details of time specification.
245
246 The default Volume Retention period is 1 year.
247
248 % TODO: if that is the format, should it be in quotes? decide on a style
249
250 \item [Recycle = \byesorno{}]
251 \index[fd]{Recycle }
252 This statement tells \bacula{} whether or not the particular Volume can be
253 recycled (i.e. rewritten). If Recycle is set to \bdefaultvalue{no} (the
254 default), then even if \bacula{} prunes all the Jobs on the volume and it
255 is marked \bstatus{Purged}, it will not consider the volume for recycling. If
256 Recycle is set to \bvalue{yes} and all Jobs have been pruned, the volume
257 status will be set to \bstatus{Purged} and the volume may then be reused
258 when another volume is needed. If the volume is reused, it is relabeled
259 with the same Volume Name, however all previous data will be lost.
260 \end{bdescription}
261
262 It is also possible to ``force'' pruning of all Volumes in the Pool
263 associated with a Job by adding \bvalueddirective{Prune Files}{yes} to the Job resource.
264
265 \label{blb:Recycling}
266 \label{blb:RecyclingAlgorithm}
267 \section{Recycling Algorithm}
268 \index[general]{Algorithm!Recycling }
269 \index[general]{Recycling Algorithm }
270
271 When a Job needs a Volume and no appendable, recycled or purged is available,
272 \bacula{} starts the recycling algorithm pruning Jobs and Files for the oldest
273 Volume in the pool used by the job. After the Volume is pruned, and if
274 the \bdirectivename{Recycle} flag is on (\bvalueddirective{Recycle}{yes}) for that Volume, \bacula{} will
275 relabel it and write new data on it.
276
277 As mentioned above, there are two key points for getting a Volume
278 to be recycled. First, the Volume must no longer be marked Append (there
279 are a number of directives to automatically make this change), and second
280 since the last write on the Volume, one or more of the Retention periods
281 must have expired so that there are no more catalog backup job records
282 that reference that Volume. Once both those conditions are satisfied,
283 the volume can be marked Purged and hence recycled.
284
285 The full algorithm that \bacula{} uses when it needs a new Volume is:
286 \index[general]{New Volume Algorithm}
287 \index[general]{Algorithm!New Volume}
288
289 The algorithm described below assumes that AutoPrune is enabled,
290 that Recycling is turned on, and that you have defined
291 appropriate Retention periods, or used the defaults for all these
292 items.
293
294 \begin{bitemize}
295 \item If the request is for an Autochanger device, look only
296 for Volumes in the Autochanger (i.e. with InChanger set and that have
297 the correct Storage device).
298 \item Search the Pool for a Volume with VolStatus=Append (if there is more
299 than one, the Volume with the oldest date last written is chosen. If
300 two have the same date then the one with the lowest MediaId is chosen).
301 \item Search the Pool for a Volume with VolStatus=Recycle and the InChanger
302 flag is set true (if there is more than one, the Volume with the oldest
303 date last written is chosen. If two have the same date then the one
304 with the lowest MediaId is chosen).
305 \item Search the Pool for a Volume with VolStatus=Purged and try recycling any purged Volumes.
306 \item Prune volumes applying Volume retention period (Volumes with VolStatus
307 \bstatus{Full} or \bstatus{Used} are pruned). Note, when all the File and Job
308 records are pruned from a Volume, the Volume will be marked \bstatus{Purged} and
309 this may be prior to the expiration of the Volume retention period.
310 \item Prune the oldest Volume if \bvalueddirective{RecycleOldestVolume}{yes} (the Volume with the
311 oldest LastWritten date and VolStatus equal to \bstatus{Full} or \bstatus{Used} is chosen).
312 This record ensures that all retention periods are properly respected.
313 \item Purge the oldest Volume if PurgeOldestVolume=yes (the Volume with the
314 oldest LastWritten date and VolStatus equal to \bstatus{Full} or \bstatus{Used} is chosen).
315 We strongly recommend against the use of \bdirectivename{PurgeOldestVolume} as
316 it can quite easily lead to loss of current backup data.
317 \item Search for a Volume in the Scratch Pool (the default Scratch pool
318 or another Scratch Pool defined for the pool) and if found
319 move it to the current Pool for the Job and use it. Note, when
320 the Scratch Volume is moved into the current Pool, the basic
321 Pool defaults are applied as if it is a newly labeled Volume
322 (equivalent to an \bcommandname{update volume from pool} command).
323 \item If we were looking for Volumes in the Autochanger, go back to step 2 above, but this time, look for any Volume whether or not it is in the Autochanger.
324 \item Attempt to create a new Volume if automatic labeling is enabled and the maximum number of Volumes is not reached.
325 \item Give up and ask operator.
326 \end{bitemize}
327
328 The above occurs when \bacula{} has finished writing a Volume or when no Volume
329 is present in the drive.
330
331 On the other hand, if you have inserted a different Volume after the last
332 job, and \bacula{} recognizes the Volume as valid, the Storage daemon will
333 request authorization from the Director to use this Volume. In this case,
334 if you have set \bvalueddirective{Recycle Current Volume = yes} and the Volume is marked
335 as Used or Full, Bacula will prune the volume and if all jobs were removed
336 during the pruning (respecting the retention periods), the Volume will be
337 recycled and re-used.
338
339 \smallskip
340
341 If you want to strictly observe the \bdirectivename{Volume Retention Period} under all
342 circumstances -- i.e. you have a Volume with critical data, you must set
343 \bvalueddirective{Recycle}{no}. However, in doing so, you must manually do the
344 recycling of the Volume for it to be used again.
345
346 The recycling algorithm in this case is:
347 \begin{bitemize}
348 \item If the VolStatus is \bstatus{Append} or \bstatus{Recycle}
349 is set, the volume will be used.
350 \item If \bdirectivename{Recycle Current Volume} is set and the volume is marked \bstatus{Full}
351 or \bstatus{Used}, \bacula{} will prune the volume (applying the retention
352 period). If all Jobs are pruned from the volume, it will be recycled.
353 \end{bitemize}
354
355 This permits users to manually change the Volume every day and load volumes in
356 an order different from what is in the catalog, and if the volume does not
357 contain a current copy of your backup data, it will be used.
358
359 A few points to keep in mind:
360
361 \begin{benumerate}
362 \item If a pool doesn't have maximum volumes defined then \bacula{} will prefer to
363 demand new volumes over forcibly purging older volumes.
364
365 \item If volumes become free through pruning, then they get marked as
366 \bstatus{Purged} and are immediately available for recycling -- these will
367 be used in preference to creating new volumes.
368
369 \item If the Job, File, and Volume retention periods are different, then
370 it's common to see a volume with no files or jobs listed in the database,
371 but which is still not marked as \bstatus{Purged}, if \bdirectivename{Autoprune} is set to \bvalue{no}
372 in the pool resource.
373 \end{benumerate}
374
375
376 \section{Recycle Status}
377 \index[general]{Status!Recycle }
378 \index[general]{Recycle Status }
379
380 Each Volume inherits the Recycle status (yes or no) from the Pool resource
381 record when the Media record is created (normally when the Volume is
382 labeled). This Recycle status is stored in the Media record of the
383 Catalog. Using the Console program, you may subsequently change the
384 Recycle status for each Volume.
385 For example in the following output from \bcommandname{list volumes}:
386
387 \begin{bVerbatim}
388 +----------+-------+--------+---------+------------+--------+-----+
389 | VolumeNa | Media | VolSta | VolByte | LastWritte | VolRet | Rec |
390 +----------+-------+--------+---------+------------+--------+-----+
391 | File0001 | File | Full | 4190055 | 2002-05-25 | 14400 | 1 |
392 | File0002 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 |
393 | File0003 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 |
394 | File0004 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 |
395 | File0005 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 |
396 | File0006 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 |
397 | File0007 | File | Purged | 1896466 | 2002-05-26 | 14400 | 1 |
398 +----------+-------+--------+---------+------------+--------+-----+
399 \end{bVerbatim}
400
401 all the volumes are marked as recyclable, and the last Volume,
402 \bvalue{File0007} has been purged, so it may be immediately recycled. The other
403 volumes are all marked recyclable and when their Volume Retention period
404 (14400 seconds or four hours) expires, they will be eligible for pruning,
405 and possibly recycling. Even though Volume \bvalue{File0007} has been purged,
406 all the data on the Volume is still recoverable. A purged Volume simply
407 means that there are no entries in the Catalog. Even if the Volume Status
408 is changed to \bstatus{Recycle}, the data on the Volume will be recoverable.
409 The data is lost only when the Volume is re-labeled and re-written.
410
411 To modify Volume \bvalue{File0001} so that it cannot be recycled, you use the
412 \bcommandname{update volume pool=File} command in the console program, or simply
413 \bcommandname{update} and \bacula{} will prompt you for the information.
414
415 \begin{bVerbatim}
416 +----------+------+-------+---------+-------------+-------+-----+
417 | VolumeNa | Media| VolSta| VolByte | LastWritten | VolRet| Rec |
418 +----------+------+-------+---------+-------------+-------+-----+
419 | File0001 | File | Full | 4190055 | 2002-05-25 | 14400 | 0 |
420 | File0002 | File | Full | 1897236 | 2002-05-26 | 14400 | 1 |
421 | File0003 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 |
422 | File0004 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 |
423 | File0005 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 |
424 | File0006 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 |
425 | File0007 | File | Purged| 1896466 | 2002-05-26 | 14400 | 1 |
426 +----------+------+-------+---------+-------------+-------+-----+
427 \end{bVerbatim}
428
429 In this case, \textbf{File0001} will never be automatically recycled. The same
430 effect can be achieved by setting the Volume Status to Read-Only.
431
432 As you have noted, the Volume Status (VolStatus) column in the
433 catalog database contains the current status of the Volume, which
434 is normally maintained automatically by \bacula{}. To give you an
435 idea of some of the values it can take during the life cycle of
436 a Volume, here is a picture of the process.
437
438 \begin{bVerbatim}
439 A typical volume life cycle is like this:
440
441 because job count or size limit exceeded
442 Append ------------------------------------> Used/Full
443 ^ |
444 | First Job writes to Retention time passed |
445 | the volume and recycling takes |
446 | place |
447 | v
448 Recycled <-------------------------------------- Purged
449 Volume is selected for reuse
450 \end{bVerbatim}
451
452
453 \section{Making \bacula{} Use a Single Tape}
454 \label{blb:singletape}
455 \index[general]{Tape!Making \bacula{} Use a Single}
456 \index[general]{Making \bacula{} Use a Single Tape}
457
458 Most people will want \bacula{} to fill a tape and when it is full, a new tape
459 will be mounted, and so on. However, as an extreme example, it is possible
460 for \bacula{} to write on a single tape, and every night to rewrite it. To
461 get this to work, you must do two things: first, set the \bdirectivename{VolumeRetention} to
462 less than your save period (one day), and the second item is to make \bacula{}
463 mark the tape as full after using it once. This is done using
464 \bvalueddirective{UseVolumeOnce}{yes}. If this latter record is not used and the tape is
465 not full after the first time it is written, \bacula{} will simply append to
466 the tape and eventually request another volume. Using the tape only once,
467 forces the tape to be marked
468 \bstatus{Full} after each use, and the next time \bacula{} runs, it will
469 recycle the tape.
470
471 An example Pool resource that does this is:
472
473 \begin{bVerbatim}
474 Pool {
475 Name = DDS-4
476 Use Volume Once = yes
477 Pool Type = Backup
478 AutoPrune = yes
479 VolumeRetention = 12h # expire after 12 hours
480 Recycle = yes
481 }
482 \end{bVerbatim}
483
484 \section{Daily, Weekly, Monthly Tape Usage Example}
485 \label{blb:usageexample}
486 \index[general]{Daily, Weekly, Monthly Tape Usage Example }
487 \index[general]{Example!Daily Weekly Monthly Tape Usage }
488
489 This example is meant to show you how one could define a fixed set of volumes
490 that \bacula{} will rotate through on a regular schedule. There are an infinite
491 number of such schemes, all of which have various advantages and
492 disadvantages. Note: these volumes may either be tape volumes or disk
493 volumes.
494
495 We start with the following assumptions:
496
497 \begin{bitemize}
498 \item A single tape has more than enough capacity to do a full save.
499 \item There are ten tapes that are used on a daily basis for incremental
500 backups. They are prelabeled Daily1 ... Daily10.
501 \item There are four tapes that are used on a weekly basis for full backups.
502 They are labeled Week1 ... Week4.
503 \item There are 12 tapes that are used on a monthly basis for full backups.
504 They are numbered Month1 ... Month12
505 \item A full backup is done every Saturday evening (tape inserted Friday
506 evening before leaving work).
507 \item No backups are done over the weekend (this is easy to change).
508 \item The first Friday of each month, a Monthly tape is used for the Full
509 backup.
510 \item Incremental backups are done Monday - Friday (actually Tue-Fri
511 mornings).
512 % TODO: why this ``actually''? does this need to be explained?
513 \end{bitemize}
514
515 We start the system by doing a Full save to one of the weekly volumes or one
516 of the monthly volumes. The next morning, we remove the tape and insert a
517 Daily tape. Friday evening, we remove the Daily tape and insert the next tape
518 in the Weekly series. Monday, we remove the Weekly tape and re-insert the
519 Daily tape. On the first Friday of the next month, we insert the next Monthly
520 tape in the series rather than a Weekly tape, then continue. When a Daily tape
521 finally fills up, \bacula{} will request the next one in the series, and
522 the next day when you notice the email message, you will mount it and
523 \bacula{} will finish the unfinished incremental backup.
524
525 What does this give? Well, at any point, you will have the last complete
526 Full save plus several Incremental saves. For any given file you want to
527 recover (or your whole system), you will have a copy of that file every day
528 for at least the last 14 days. For older versions, you will have at least three
529 and probably four Friday full saves of that file, and going back further, you
530 will have a copy of that file made on the beginning of the month for at least
531 a year.
532
533 So you have copies of any file (or your whole system) for at least a year, but
534 as you go back in time, the time between copies increases from daily to weekly
535 to monthly.
536
537 What would the \bacula{} configuration look like to implement such a scheme?
538
539 \begin{bVerbatim}
540 Schedule {
541 Name = "NightlySave"
542 Run = Level=Full Pool=Monthly 1st sat at 03:05
543 Run = Level=Full Pool=Weekly 2nd-5th sat at 03:05
544 Run = Level=Incremental Pool=Daily tue-fri at 03:05
545 }
546 Job {
547 Name = "NightlySave"
548 Type = Backup
549 Level = Full
550 Client = LocalMachine
551 FileSet = "File Set"
552 Messages = Standard
553 Storage = DDS-4
554 Pool = Daily
555 Schedule = "NightlySave"
556 }
557 # Definition of file storage device
558 Storage {
559 Name = DDS-4
560 Address = localhost
561 SDPort = 9103
562 Password = XXXXXXXXXXXXX
563 Device = FileStorage
564 Media Type = 8mm
565 }
566 FileSet {
567 Name = "File Set"
568 Include {
569 Options { signature=MD5 }
570 File = fffffffffffffffff
571 }
572 Exclude { File=*.o }
573 }
574 Pool {
575 Name = Daily
576 Pool Type = Backup
577 AutoPrune = yes
578 VolumeRetention = 10d # recycle in 10 days
579 Maximum Volumes = 10
580 Recycle = yes
581 }
582 Pool {
583 Name = Weekly
584 Use Volume Once = yes
585 Pool Type = Backup
586 AutoPrune = yes
587 VolumeRetention = 30d # recycle in 30 days (default)
588 Recycle = yes
589 }
590 Pool {
591 Name = Monthly
592 Use Volume Once = yes
593 Pool Type = Backup
594 AutoPrune = yes
595 VolumeRetention = 365d # recycle in 1 year
596 Recycle = yes
597 }
598 \end{bVerbatim}
599
600 \section{ Automatic Pruning and Recycling Example}
601 \label{blb:PruningExample}
602 \index[general]{Automatic Pruning and Recycling Example }
603 \index[general]{Example!Automatic Pruning and Recycling }
604
605 Perhaps the best way to understand the various resource records that come
606 into play during automatic pruning and recycling is to run a Job that goes
607 through the whole cycle. If you add the following resources to your
608 Director's configuration file:
609
610 \begin{bVerbatim}
611 Schedule {
612 Name = "30 minute cycle"
613 Run = Level=Full Pool=File Messages=Standard Storage=File
614 hourly at 0:05
615 Run = Level=Full Pool=File Messages=Standard Storage=File
616 hourly at 0:35
617 }
618 Job {
619 Name = "Filetest"
620 Type = Backup
621 Level = Full
622 Client=XXXXXXXXXX
623 FileSet="Test Files"
624 Messages = Standard
625 Storage = File
626 Pool = File
627 Schedule = "30 minute cycle"
628 }
629 # Definition of file storage device
630 Storage {
631 Name = File
632 Address = XXXXXXXXXXX
633 SDPort = 9103
634 Password = XXXXXXXXXXXXX
635 Device = FileStorage
636 Media Type = File
637 }
638 FileSet {
639 Name = "File Set"
640 Include {
641 Options { signature=MD5 }
642 File = fffffffffffffffff
643 }
644 Exclude { File=*.o }
645 }
646 Pool {
647 Name = File
648 Use Volume Once = yes
649 Pool Type = Backup
650 LabelFormat = "File"
651 AutoPrune = yes
652 VolumeRetention = 4h
653 Maximum Volumes = 12
654 Recycle = yes
655 }
656 \end{bVerbatim}
657
658 Where you will need to replace the \textbf{ffffffffff}'s by the appropriate files
659 to be saved for your configuration. For the FileSet Include, choose a
660 directory that has one or two megabytes maximum since there will probably be
661 approximately eight copies of the directory that \bacula{} will cycle through.
662
663 In addition, you will need to add the following to your Storage daemon's
664 configuration file:
665
666 \begin{bVerbatim}
667 Device {
668 Name = FileStorage
669 Media Type = File
670 Archive Device = /tmp
671 LabelMedia = yes;
672 Random Access = Yes;
673 AutomaticMount = yes;
674 RemovableMedia = no;
675 AlwaysOpen = no;
676 }
677 \end{bVerbatim}
678
679 With the above resources, \bacula{} will start a Job every half hour that saves a
680 copy of the directory you chose to /tmp/File0001 ... /tmp/File0012. After 4
681 hours, \bacula{} will start recycling the backup Volumes (/tmp/File0001 ...). You
682 should see this happening in the output produced. \bacula{} will automatically
683 create the Volumes (Files) the first time it uses them.
684
685 To turn it off, either delete all the resources you've added, or simply
686 comment out the \bresourcename{Schedule} record in the \bresourcename{Job} resource.
687
688 \section{Manually Recycling Volumes}
689 \label{blb:manualrecycling}
690 \index[general]{Volumes!Manually Recycling }
691 \index[general]{Manually Recycling Volumes }
692
693 Although automatic recycling of Volumes is implemented in version 1.20 and
694 later (see the
695 \bilink{Automatic Recycling of Volumes}{blb:RecyclingChapter} chapter of
696 this manual), you may want to manually force reuse (recycling) of a Volume.
697
698 Assuming that you want to keep the Volume name, but you simply want to write
699 new data on the tape, the steps to take are:
700
701 \begin{bitemize}
702 \item Use the \bcommandname{update volume} command in the Console to ensure that the
703 \bstatus{Recycle} field is set to \bvalue{1}
704 \item Use the \bcommandname{purge jobs volume} command in the Console to mark the
705 Volume as \bstatus{Purged}. Check by using \bcommandname{list volumes}.
706 \end{bitemize}
707
708 Once the Volume is marked Purged, it will be recycled the next time a Volume
709 is needed.
710
711 If you wish to reuse the volume by giving it a new name, follow the following
712 steps:
713
714 \begin{bitemize}
715 \item Use the \bcommandname{purge jobs volume} command in the Console to mark the
716 Volume as \bstatus{Purged}. Check by using \bcommandname{list volumes}.
717 \item Use the Console \bcommandname{relabel}
718 command to relabel the Volume.
719 \end{bitemize}
720
721 Please note that the \bcommandname{relabel} command applies only to tape Volumes.
722
723 For \bacula{} versions prior to 1.30 or to manually relabel the Volume, use the
724 instructions below:
725
726 \begin{bitemize}
727 \item Use the \bcommandname{delete volume} command in the Console to delete the Volume
728 from the Catalog.
729 \item If a different tape is mounted, use the \bcommandname{unmount} command,
730 remove the tape, and insert the tape to be renamed.
731 \item Write an EOF mark in the tape using the following commands:
732
733 \begin{bVerbatim}
734 mt -f /dev/nst0 rewind
735 mt -f /dev/nst0 weof
736 \end{bVerbatim}
737
738 where you replace \bfilename{/dev/nst0} with the appropriate device name on your
739 system.
740 \item Use the \bcommandname{label} command to write a new label to the tape and to
741 enter it in the catalog.
742 \end{bitemize}
743
744 Please be aware that the \bcommandname{delete} command can be dangerous. Once it is
745 done, to recover the File records, you must either restore your database as it
746 was before the \bcommandname{delete} command, or use the \btool{bscan} utility program to
747 scan the tape and recreate the database entries.