Debian Codegraph @

manuals/en/main/migration.tex

raw
1 \chapter{Migration and Copy}
2 \label{blb:MigrationChapter}
3 \index[general]{Migration}
4 \index[general]{Copy}
5
6 The term Migration, as used in the context of \bacula{}, means moving data from
7 one Volume to another. In particular it refers to a Job (similar to a backup
8 job) that reads data that was previously backed up to a Volume and writes
9 it to another Volume. As part of this process, the File catalog records
10 associated with the first backup job are purged. In other words, Migration
11 moves \bacula{} Job data from one Volume to another by reading the Job data
12 from the Volume it is stored on, writing it to a different Volume in a
13 different Pool, and then purging the database records for the first Job.
14
15 The Copy process is essentially identical to the Migration feature with the
16 exception that the Job that is copied is left unchanged. This essentially
17 creates two identical copies of the same backup. However, the copy is treated
18 as a copy rather than a backup job, and hence is not directly available for
19 restore. If bacula founds a copy when a job record is purged (deleted) from the
20 catalog, it will promote the copy as \textsl{real} backup and will make it
21 available for automatic restore. Note: in the text below, to simplify it,
22 we usually speak of a migration job. This, in fact, means either a
23 migration job or a copy job.
24
25 The Copy and the Migration jobs run without using the File daemon by copying
26 the data from the old backup Volume to a different Volume in a different
27 Pool. It is not possible to run commands on the defined Client via a RunScript
28 from within the Migration or Copy Job.
29
30 The selection process for which Job or Jobs are migrated
31 can be based on quite a number of different criteria such as:
32 \begin{bitemize}
33 \item a single previous Job
34 \item a Volume
35 \item a Client
36 \item a regular expression matching a Job, Volume, or Client name
37 \item the time a Job has been on a Volume
38 \item high and low water marks (usage or occupation) of a Pool
39 \item Volume size
40 \end{bitemize}
41
42 The details of these selection criteria will be defined below.
43
44 To run a Migration job, you must first define a Job resource very similar
45 to a Backup Job but with \textbf{Type = Migrate} instead of
46 \textbf{Type = Backup}. One of the key points to remember is that the Pool that is
47 specified for the migration job is the only pool from which jobs will
48 be migrated, with one exception noted below. In addition, the Pool to
49 which the selected Job or Jobs will be migrated is defined by the
50 \textbf{Next Pool = \ldots{}} in the Pool resource specified for the Migration Job.
51
52 \bacula{} permits Pools to contain Volumes with different Media Types.
53 However, when doing migration, this is a very undesirable condition. For
54 migration to work properly, you should use Pools containing only Volumes of
55 the same Media Type for all migration jobs.
56
57 The migration job normally is either manually started or starts
58 from a Schedule much like a backup job. It searches
59 for a previous backup Job or Jobs that match the parameters you have
60 specified in the migration Job resource, primarily a \textbf{Selection Type}
61 (detailed a bit later). Then for
62 each previous backup JobId found, the Migration Job will run a new Job which
63 copies the old Job data from the previous Volume to a new Volume in
64 the Migration Pool. It is possible that no prior Jobs are found for
65 migration, in which case, the Migration job will simply terminate having
66 done nothing, but normally at a minimum, three jobs are involved during a
67 migration:
68
69 \begin{bitemize}
70 \item The currently running Migration control Job. This is only
71 a control job for starting the migration child jobs.
72 \item The previous Backup Job (already run). The File records
73 for this Job are purged if the Migration job successfully
74 terminates. The original data remains on the Volume until
75 it is recycled and rewritten.
76 \item A new Migration Backup Job that moves the data from the
77 previous Backup job to the new Volume. If you subsequently
78 do a restore, the data will be read from this Job.
79 \end{bitemize}
80
81 If the Migration control job finds a number of JobIds to migrate (e.g.
82 it is asked to migrate one or more Volumes), it will start one new
83 migration backup job for each JobId found on the specified Volumes.
84 Please note that Migration doesn't scale too well since Migrations are
85 done on a Job by Job basis. This if you select a very large volume or
86 a number of volumes for migration, you may have a large number of
87 Jobs that start. Because each job must read the same Volume, they will
88 run consecutively (not simultaneously).
89
90 \section{Migration and Copy Job Resource Directives}
91
92 The following directives can appear in a Director's Job resource, and they
93 are used to define a Migration job.
94
95 \begin{bdescription}
96 \item [Pool = \bbracket{Pool-name}] The Pool specified in the Migration
97 control Job is not a new directive for the Job resource, but it is
98 particularly important because it determines what Pool will be examined
99 for finding JobIds to migrate. The exception to this is when
100 \textbf{Selection Type = SQLQuery}, and although a Pool directive must still be
101 specified, no Pool is used, unless you specifically include it in the
102 SQL query. Note, in any case, the Pool resource defined by the Pool
103 directive must contain a \textbf{Next Pool = \ldots{}} directive to define the
104 Pool to which the data will be migrated.
105
106 \item [Type = Migrate]
107 \textbf{Migrate} is a new type that defines the job that is run as being a
108 Migration Job. A Migration Job is a sort of control job and does not have
109 any Files associated with it, and in that sense they are more or less like
110 an Admin job. Migration jobs simply check to see if there is anything to
111 Migrate then possibly start and control new Backup jobs to migrate the data
112 from the specified Pool to another Pool. Note, any original JobId that
113 is migrated will be marked as having been migrated, and the original
114 JobId can nolonger be used for restores; all restores will be done from
115 the new migrated Job.
116
117
118 \item [Type = Copy]
119 \textbf{Copy} is a new type that defines the job that is run as being a
120 Copy Job. A Copy Job is a sort of control job and does not have
121 any Files associated with it, and in that sense they are more or less like
122 an Admin job. Copy jobs simply check to see if there is anything to
123 Copy then possibly start and control new Backup jobs to copy the data
124 from the specified Pool to another Pool. Note that when a copy is
125 made, the original JobIds are left unchanged. The new copies can not
126 be used for restoration unless you specifically choose them by JobId.
127 Also you subsequently delete a JobId that has a copy, the copy will be
128 automatically upgraded to a Backup rather than a Copy, and it will
129 subsequently be used for restoration.
130
131 \smallskip
132 Once again, a Copy Job cannot be used to restore unless you explicitly
133 specify the Copy JobId during the restore command. If the original
134 backup Job is deleted and there is a Copy of that backup Job,
135 the Copy JobIds will be changed to be backup Jobs that can then
136 be restored.
137
138
139 \bwebdoc{Director:Job:SelectionType}
140 \item [Selection Type = \bbracket{Selection-type-keyword}]
141 The \bbracket{Selection-type-keyword} determines how the migration job
142 will go about selecting what JobIds to migrate. In most cases, it is
143 used in conjunction with a \textbf{Selection Pattern} to give you fine
144 control over exactly what JobIds are selected. The possible values
145 for \bbracket{Selection-type-keyword} are:
146 \begin{bdescription}
147 \item [SmallestVolume] This selection keyword selects the volume with the
148 fewest bytes from the Pool to be migrated. The Pool to be migrated
149 is the Pool defined in the Migration Job resource. The migration
150 control job will then start and run one migration backup job for
151 each of the Jobs found on this Volume. The Selection Pattern, if
152 specified, is not used.
153
154 \item [OldestVolume] This selection keyword selects the volume with the
155 oldest last write time in the Pool to be migrated. The Pool to be
156 migrated is the Pool defined in the Migration Job resource. The
157 migration control job will then start and run one migration backup
158 job for each of the Jobs found on this Volume. The Selection
159 Pattern, if specified, is not used.
160
161 \item [Client] The Client selection type, first selects all the Clients
162 that have been backed up in the Pool specified by the Migration
163 Job resource, then it applies the \textbf{Selection Pattern} (defined
164 below) as a regular expression to the list of Client names, giving
165 a filtered Client name list. All jobs that were backed up for those
166 filtered (regexed) Clients will be migrated.
167 The migration control job will then start and run one migration
168 backup job for each of the JobIds found for those filtered Clients.
169
170 \item [Volume] The Volume selection type, first selects all the Volumes
171 that have been backed up in the Pool specified by the Migration
172 Job resource, then it applies the \textbf{Selection Pattern} (defined
173 below) as a regular expression to the list of Volume names, giving
174 a filtered Volume list. All JobIds that were backed up for those
175 filtered (regexed) Volumes will be migrated.
176 The migration control job will then start and run one migration
177 backup job for each of the JobIds found on those filtered Volumes.
178
179 Jobs on Volumes will be considered for Migration only if the Volume
180 is marked, Full, Used, or Error. Volumes that are still marked
181 Append will not be considered for migration. This prevents \bacula{}
182 from attempting to read the Volume at the same time it is writing it.
183 It also reduces other deadlock situations, as well as avoids the
184 problem that you migrate a Volume and later find new files appended
185 to that Volume.
186
187
188 \item [Job] The Job selection type, first selects all the Jobs (as
189 defined on the \textbf{Name} directive in a Job resource)
190 that have been backed up in the Pool specified by the Migration
191 Job resource, then it applies the \textbf{Selection Pattern} (defined
192 below) as a regular expression to the list of Job names, giving
193 a filtered Job name list. All JobIds that were run for those
194 filtered (regexed) Job names will be migrated. Note, for a given
195 Job named, they can be many jobs (JobIds) that ran.
196 The migration control job will then start and run one migration
197 backup job for each of the Jobs found.
198
199 \item [SQLQuery] The SQLQuery selection type, used the
200 \textbf{Selection Pattern} as an SQL query to obtain the JobIds to be migrated.
201 The Selection Pattern must be a valid SELECT SQL statement for your
202 SQL engine, and it must return the JobId as the first field
203 of the SELECT.
204
205 \item [PoolOccupancy] This selection type will cause the Migration job
206 to compute the total size of the specified pool for all Media Types
207 combined. If it exceeds the \textbf{Migration High Bytes} defined in
208 the Pool, the Migration job will migrate all JobIds beginning with
209 the oldest Volume in the pool (determined by Last Write time) until
210 the Pool bytes drop below the \textbf{Migration Low Bytes} defined in the
211 Pool. This calculation should be consider rather approximate because
212 it is made once by the Migration job before migration is begun, and
213 thus does not take into account additional data written into the Pool
214 during the migration. In addition, the calculation of the total Pool
215 byte size is based on the Volume bytes saved in the Volume (Media)
216 database entries. The bytes calculate for Migration is based on
217 the value stored in the Job records of the Jobs to be migrated.
218 These do not include the Storage daemon overhead as is in the total
219 Pool size. As a consequence, normally, the migration will migrate
220 more bytes than strictly necessary.
221
222 \item [PoolTime] The PoolTime selection type will cause the Migration job to
223 look at the time each JobId has been in the Pool since the job ended.
224 All Jobs in the Pool longer than the time specified on \textbf{Migration Time}
225 directive in the Pool resource will be migrated.
226
227 \item [PoolUncopiedJobs] This selection which copies all jobs from a pool
228 to an other pool which were not copied before is available only for copy Jobs.
229
230 \end{bdescription}
231
232 \bwebdoc{Director:Job:SelectionPattern}
233 \item [Selection Pattern = \bbracket{Quoted-string}]
234 The Selection Patterns permitted for each Selection-type-keyword are
235 described above.
236
237 For the OldestVolume and SmallestVolume, this
238 Selection pattern is not used (ignored).
239
240 \smallskip
241 For the Client, Volume, and Job
242 keywords, this pattern must be a valid regular expression that will filter
243 the appropriate item names found in the Pool.
244
245 \smallskip
246 For the SQLQuery keyword, this pattern must be a valid SELECT SQL statement
247 that returns JobIds.
248
249 \bwebdoc{Director:Job:PurgeMigrationJob}
250 \item [Purge Migration Job = \byesorno{}]
251 This directive may be added to the Migration Job definition in the Director
252 configuration file to purge the job migrated at the end of a migration.
253 \end{bdescription}
254
255 \section{Migration Pool Resource Directives}
256
257 The following directives can appear in a Director's Pool resource, and they
258 are used to define a Migration job.
259
260 \begin{bdescription}
261 \bwebdoc{Director:Pool:MigrationTime}
262 \item [Migration Time = \bbracket{time-specification}]
263 If a PoolTime migration is done, the time specified here in seconds (time
264 modifiers are permitted -- e.g. hours, \ldots{}) will be used. If the
265 previous Backup Job or Jobs selected have been in the Pool longer than
266 the specified PoolTime, then they will be migrated.
267
268 \bwebdoc{Director:Pool:MigrationHighBytes}
269 \item [Migration High Bytes = \bbracket{byte-specification}]
270 This directive specifies the number of bytes in the Pool which will
271 trigger a migration if a \textbf{PoolOccupancy} migration selection
272 type has been specified. The fact that the Pool
273 usage goes above this level does not automatically trigger a migration
274 job. However, if a migration job runs and has the PoolOccupancy selection
275 type set, the Migration High Bytes will be applied. \bacula{} does not
276 currently restrict a pool to have only a single Media Type, so you
277 must keep in mind that if you mix Media Types in a Pool, the results
278 may not be what you want, as the Pool count of all bytes will be
279 for all Media Types combined.
280
281 \bwebdoc{Director:Pool:MigrationLowBytes}
282 \item [Migration Low Bytes = \bbracket{byte-specification}]
283 This directive specifies the number of bytes in the Pool which will
284 stop a migration if a \textbf{PoolOccupancy} migration selection
285 type has been specified and triggered by more than Migration High
286 Bytes being in the pool. In other words, once a migration job
287 is started with \textbf{PoolOccupancy} migration selection and it
288 determines that there are more than Migration High Bytes, the
289 migration job will continue to run jobs until the number of
290 bytes in the Pool drop to or below Migration Low Bytes.
291
292 \bwebdoc{Director:Pool:NextPool}
293 \item [Next Pool = \bbracket{pool-specification}]
294 The Next Pool directive specifies the pool to which Jobs will be
295 migrated. This directive is required to define the Pool into which
296 the data will be migrated. Without this directive, the migration job
297 will terminate in error.
298
299 The Next Pool directive may also be specified
300 in the Job resource or on a Run directive in the Schedule resource.
301 Any Next Pool directive in the Job resource will take precedence over
302 the Pool definition, and any Next Pool specification on the Run
303 directive in a Schedule resource will take ultimate precedence.
304
305 \bwebdoc{Director:Pool:End}
306 \item [Storage = \bbracket{storage-specification}]
307 The Storage directive specifies what Storage resource will be used
308 for all Jobs that use this Pool. It takes precedence over any other
309 Storage specifications that may have been given such as in the
310 Schedule Run directive, or in the Job resource. We highly recommend
311 that you define the Storage resource to be used in the Pool rather
312 than elsewhere (job, schedule run, \ldots{}).
313 \end{bdescription}
314
315 \section{Important Migration Considerations}
316 \index[general]{Important Migration Considerations}
317 \begin{bitemize}
318 \item Each Pool into which you migrate Jobs or Volumes \textbf{must}
319 contain Volumes of only one Media Type.
320
321 \item Migration takes place on a JobId by JobId basis. That is
322 each JobId is migrated in its entirety and independently
323 of other JobIds. Once the Job is migrated, it will be
324 on the new medium in the new Pool, but for the most part,
325 aside from having a new JobId, it will appear with all the
326 same characteristics of the original job (start, end time, \ldots{}).
327 The column RealEndTime in the catalog Job table will contain the
328 time and date that the Migration terminated, and by comparing
329 it with the EndTime column you can tell whether or not the
330 job was migrated. The original job is purged of its File
331 records, and its Type field is changed from "B" to "M" to
332 indicate that the job was migrated.
333
334 \item As noted above, for the Migration High Bytes, the calculation
335 of the bytes to migrate is somewhat approximate.
336
337 \item If you keep Volumes of different Media Types in the same Pool,
338 it is not clear how well migration will work. We recommend only
339 one Media Type per pool.
340
341 \item It is possible to get into a resource deadlock where \bacula{} does
342 not find enough drives to simultaneously read and write all the
343 Volumes needed to do Migrations. For the moment, you must take
344 care as all the resource deadlock algorithms are not yet implemented.
345 You can also use the \bdirectivename{Maximum Concurrent Read Jobs} directive to
346 control the drive assignment behavior to reduce resource deadlocks.
347 The MaximumConcurrentReadJobs should have a maximum of one less
348 than one half the number of drives that are available for reading
349 and writing. Doing so, will assure that for each read drive there
350 is one that can write so that the Migration jobs will not stall
351 due to a deadlock.
352
353 \item Migration is done only when you run a Migration job. If you set a
354 Migration High Bytes and that number of bytes is exceeded in the Pool
355 no migration job will automatically start. You must schedule the
356 migration jobs, and they must run for any migration to take place.
357
358 \item If you migrate a number of Volumes, a very large number of Migration
359 jobs may start.
360
361 \item Figuring out what jobs will actually be migrated can be a bit complicated
362 due to the flexibility provided by the regex patterns and the number of
363 different options. Turning on a debug level of 100 or more will provide
364 a limited amount of debug information about the migration selection
365 process.
366
367 \item \bacula{} currently does only minimal Storage conflict resolution, so you
368 must take care to ensure that you don't try to read and write to the
369 same device or \bacula{} may block waiting to reserve a drive that it
370 will never find. In general, ensure that all your migration
371 pools contain only one Media Type, and that you always
372 migrate to pools with different Media Types.
373
374 \item The \textbf{Next Pool = \ldots{}} directive must be defined in the Pool
375 referenced in the Migration Job to define the Pool into which the
376 data will be migrated.
377
378 \item Pay particular attention to the fact that data is migrated on a Job
379 by Job basis, and for any particular tape Volume, only one Job can read
380 that Volume at a time (no simultaneous read), so migration jobs that
381 all reference the same Volume will run sequentially. This can be a
382 potential bottle neck and does not scale very well to large numbers
383 of jobs.
384 \smallskip
385 For disk Volumes, multiple simultaneous Jobs can read the same
386 Volume at the same time, so the above restriction does not apply.
387
388
389 \item Only migration of Selection Types of Job and Volume have
390 been carefully tested. All the other migration methods (time,
391 occupancy, smallest, oldest, \ldots{}) need additional testing.
392
393 \item Plugins are not compatible with Copy/Migration/VirtualFull jobs by
394 default. Please contact \bacula{} Systems support team to know if your strategy
395 will be possible with your setup.
396
397 \item Windows backups using \acs{VSS} writers (normally plugins) are not
398 compatible with Migration and Copy. The reason is that \acs{VSS} writers
399 require that certain \acs{XML} files generated at the end of the Backup,
400 but they must be presented to the \acs{VSS} writer at the beginning of a
401 restore Job. This is very difficult for a backup program such as
402 \bacula{}. Just the same \bacula{} accomplishes this by saving the \acs{VSS}
403 writer's \acs{XML} files in Restore Objects in the catalog. However,
404 during Migration and Copy Jobs these Restore Objects are not currently
405 copied to the new Job.
406 \end{bitemize}
407
408
409 \section{Example Migration Jobs}
410 \index[general]{Example Migration Jobs}
411
412 When you specify a Migration Job, you must specify all the standard
413 directives as for a Job. However, certain such as the Level, Client, and
414 FileSet, though they must be defined, are ignored by the Migration job
415 because the values from the original job used instead.
416
417 \smallskip
418 As an example, suppose you have the following Job that
419 you run every night. To note: there is no Storage directive in the
420 Job resource; there is a Storage directive in each of the Pool
421 resources; the Pool to be migrated (File) contains a Next Pool
422 directive that defines the output Pool (where the data is written
423 by the migration job).
424
425 \begin{bVerbatim}
426 # Define the backup Job
427 Job {
428 Name = "NightlySave"
429 Type = Backup
430 Level = Incremental # default
431 Client=rufus-fd
432 FileSet="Full Set"
433 Schedule = "WeeklyCycle"
434 Messages = Standard
435 Pool = Default
436 }
437
438 # Default pool definition
439 Pool {
440 Name = Default
441 Pool Type = Backup
442 AutoPrune = yes
443 Recycle = yes
444 Next Pool = Tape
445 Storage = File
446 LabelFormat = "File"
447 }
448
449 # Tape pool definition
450 Pool {
451 Name = Tape
452 Pool Type = Backup
453 AutoPrune = yes
454 Recycle = yes
455 Storage = DLTDrive
456 }
457
458 # Definition of File storage device
459 Storage {
460 Name = File
461 Address = rufus
462 Password = "ccV3lVTsQRsdIUGyab0N4sMDavui2hOBkmpBU0aQKOr9"
463 Device = "File" # same as Device in Storage daemon
464 Media Type = File # same as MediaType in Storage daemon
465 }
466
467 # Definition of DLT tape storage device
468 Storage {
469 Name = DLTDrive
470 Address = rufus
471 Password = "ccV3lVTsQRsdIUGyab0N4sMDavui2hOBkmpBU0aQKOr9"
472 Device = "HP DLT 80" # same as Device in Storage daemon
473 Media Type = DLT8000 # same as MediaType in Storage daemon
474 }
475
476 \end{bVerbatim}
477
478 Where we have included only the essential information -- i.e. the
479 Director, FileSet, Catalog, Client, Schedule, and Messages resources are
480 omitted.
481
482 As you can see, by running the NightlySave Job, the data will be backed up
483 to File storage using the Default pool to specify the Storage as File.
484
485 Now, if we add the following Job resource to this conf file.
486
487 \begin{bVerbatim}
488 Job {
489 Name = "migrate-volume"
490 Type = Migrate
491 Level = Full
492 Client = rufus-fd
493 FileSet = "Full Set"
494 Messages = Standard
495 Pool = Default
496 Maximum Concurrent Jobs = 4
497 Selection Type = Volume
498 Selection Pattern = "File"
499 }
500 \end{bVerbatim}
501
502 and then run the job named \textbf{migrate-volume}, all volumes in the Pool
503 named Default (as specified in the migrate-volume Job that match the
504 regular expression pattern \textbf{File} will be migrated to tape storage
505 DLTDrive because the \textbf{Next Pool} in the Default Pool specifies that
506 Migrations should go to the pool named \textbf{Tape}, which uses
507 Storage \textbf{DLTDrive}.
508
509 If instead, we use a Job resource as follows:
510
511 \begin{bVerbatim}
512 Job {
513 Name = "migrate"
514 Type = Migrate
515 Level = Full
516 Client = rufus-fd
517 FileSet="Full Set"
518 Messages = Standard
519 Pool = Default
520 Maximum Concurrent Jobs = 4
521 Selection Type = Job
522 Selection Pattern = ".*Save"
523 }
524 \end{bVerbatim}
525
526 All jobs ending with the name Save will be migrated from the File Default to
527 the Tape Pool, or from File storage to Tape storage.
528
529 \section{Virtual Backup Consolidation}
530 \label{blb:VirtualBackupChapter}
531
532 When the Job Level is set to VirtualFull, it permits you to consolidate the
533 previous Full backup plus the most recent Differential backup and any
534 subsequent Incremental backups into a new Full backup. This new Full backup
535 will then be considered as the most recent Full for any future Incremental or
536 Differential backups. The VirtualFull backup is accomplished without
537 contacting the client by reading the previous backup data and writing it to a
538 volume in a different pool.
539
540 \smallskip
541 \bacula{}'s virtual backup feature is often called Synthetic Backup or
542 Consolidation in other backup products.
543
544 \smallskip
545 In some respects the Virtual Backup feature works similar to a Migration job,
546 in that \bacula{} normally reads the data from the pool specified in the Job
547 resource, and writes it to the \textbf{Next Pool} specified in the Job
548 resource. Note, this means that usually the output from the Virtual Backup is
549 written into a different pool from where your prior backups are saved. Doing it
550 this way guarantees that you will not get a deadlock situation attempting to
551 read and write to the same volume in the Storage daemon. If you then want to do
552 subsequent backups, you may need to move the Virtual Full Volume back to your
553 normal backup pool. Alternatively, you can set your \textbf{Next Pool} to point
554 to the current pool. This will cause \bacula{} to read and write to Volumes in
555 the current pool. In general, this will work, because \bacula{} will not allow
556 reading and writing on the same Volume. In any case, once a VirtualFull has
557 been created, and a restore is done involving the most current Full, it will
558 read the Volume or Volumes by the VirtualFull regardless of in which Pool the
559 Volume is found.
560
561 \smallskip
562 A typical Job resource definition might look like the following:
563
564 \begin{bVerbatim}
565 Job {
566 Name = "MyBackup"
567 Type = Backup
568 Client=localhost-fd
569 FileSet = "Full Set"
570 Storage = File
571 Messages = Standard
572 Pool = Default
573 SpoolData = yes
574 }
575
576 # Default pool definition
577 Pool {
578 Name = Default
579 Pool Type = Backup
580 Volume Retention = 365d # one year
581 NextPool = Full
582 Storage = File
583 }
584
585 Pool {
586 Name = Full
587 Pool Type = Backup
588 Volume Retention = 365d # one year
589 Storage = DiskChanger
590 }
591
592 # Definition of file storage device
593 Storage {
594 Name = File
595 Address = localhost
596 Password = "xxx"
597 Device = FileStorage
598 Media Type = File
599 Maximum Concurrent Jobs = 5
600 }
601
602 # Definition of DDS Virtual tape disk storage device
603 Storage {
604 Name = DiskChanger
605 Address = localhost # N.B. Use a fully qualified name here
606 Password = "yyy"
607 Device = DiskChanger
608 Media Type = DiskChangerMedia
609 Maximum Concurrent Jobs = 4
610 Autochanger = yes
611 }
612 \end{bVerbatim}
613
614 Then in bconsole or via a Run schedule, you would run the job as:
615
616 \begin{bVerbatim}
617 run job=MyBackup level=Full
618 run job=MyBackup level=Incremental
619 run job=MyBackup level=Differential
620 run job=MyBackup level=Incremental
621 run job=MyBackup level=Incremental
622 \end{bVerbatim}
623
624 So providing there were changes between each of those jobs, you would end up
625 with a Full backup, a Differential, which includes the first Incremental
626 backup, then two Incremental backups. All the above jobs would be written to
627 the \textbf{Default} pool.
628
629 \smallskip
630 To consolidate those backups into a new Full backup, you would run the
631 following:
632
633 \begin{bVerbatim}
634 run job=MyBackup level=VirtualFull
635 \end{bVerbatim}
636
637 And it would produce a new Full backup without using the client, and the output
638 would be written to the \textbf{Full} Pool which uses the Diskchanger Storage.
639
640 \smallskip
641 If the Virtual Full is run, and there are no prior Jobs, the Virtual Full will
642 fail with an error.
643
644 Note, the Start and End time of the Virtual Full backup is set to the
645 values for the last job included in the Virtual Full (in the above example,
646 it is an Increment). This is so that if another incremental is done, which
647 will be based on the Virtual Full, it will backup all files from the
648 last Job included in the Virtual Full rather than from the time the Virtual
649 Full was actually run.
650
651 \subsection{Manually Specify Jobs to Consolidate}
652 By default \bacula{} is selecting jobs automatically, however, you may want to
653 choose any point in time to create the Virtual backup.
654
655 For example, if you have the following Jobs in your catalog:
656 \begin{bVerbatim}
657 +-------+---------+-------+----------+----------+-----------+
658 | JobId | Name | Level | JobFiles | JobBytes | JobStatus |
659 +-------+---------+-------+----------+----------+-----------+
660 | 1 | Vbackup | F | 1754 | 50118554 | T |
661 | 2 | Vbackup | I | 1 | 4 | T |
662 | 3 | Vbackup | I | 1 | 4 | T |
663 | 4 | Vbackup | D | 2 | 8 | T |
664 | 5 | Vbackup | I | 1 | 6 | T |
665 | 6 | Vbackup | I | 10 | 60 | T |
666 | 7 | Vbackup | I | 11 | 65 | T |
667 | 8 | Save | F | 1758 | 50118564 | T |
668 +-------+---------+-------+----------+----------+-----------+
669 \end{bVerbatim}
670
671 If you want to consolidate only the first 3 jobs and create a virtual backup
672 equivalent to Job 1 + Job 2 + Job 3, you will use \texttt{jobid=3} in the
673 \bcommandname{run} command, then \bacula{} will select the previous Full backup, the
674 previous Differential (if any) and all subsequent Incremental jobs.
675
676 \begin{bVerbatim}
677 run job=Vbackup jobid=3 level=VirtualFull
678 \end{bVerbatim}
679
680 If you want to consolidate a specific job list, you must specify the exact
681 list of jobs to merge in the \bcommandname{run} command line. For example, to consolidate
682 the last Differential and all subsequent Incremental, you will use
683 \texttt{jobid=4,5,6,7} or \texttt{jobid=4-7} in the \bcommandname{run} command line. As one
684 of the Job in the list is a Differential backup, \bacula{} will set the new job
685 level to Differential. If the list is composed only with Incremental jobs,
686 the new job will have a level set to Incremental.
687
688 \begin{bVerbatim}
689 run job=Vbackup jobid=4-7 level=VirtualFull
690 \end{bVerbatim}
691
692 When using this feature, \bacula{} will automatically discard jobs that are not
693 related to the current Job. For example, specifying \texttt{jobid=7,8},
694 \bacula{} will discard the jobid 8.
695
696 If you know what you are doing and still want to consolidate jobs that have
697 different names (so probably different clients, filesets, etc\ldots{}), you must
698 use \texttt{alljobid=} keyword instead of \texttt{jobid=}.
699
700 \begin{bVerbatim}
701 run job=Vbackup alljobid=1-3,6-8 level=VirtualFull
702 \end{bVerbatim}
703