Debian Codegraph @

docs/manual/recovering.rst

raw
1 .. _cedar-recovering:
2
3 Data Recovery
4 =============
5
6 .. _cedar-recovering-finding:
7
8 Finding your Data
9 -----------------
10
11 The first step in data recovery is finding the data that you want to
12 recover. You need to decide whether you are going to to restore off
13 backup media, or out of some existing staging data that has not yet been
14 purged. The only difference is, if you purge staging data less
15 frequently than once per week, you might have some data available in the
16 staging directories which would not be found on your backup media,
17 depending on how you rotate your media. (And of course, if your system
18 is trashed or stolen, you probably will not have access to your old
19 staging data in any case.)
20
21 Regardless of the data source you choose, you will find the data
22 organized in the same way. The remainder of these examples will work off
23 an example backup disc, but the contents of the staging directory will
24 look pretty much like the contents of the disc, with data organized
25 first by date and then by backup peer name.
26
27 This is the root directory of my example disc:
28
29 ::
30
31 root:/mnt/cdrw# ls -l
32 total 4
33 drwxr-x--- 3 backup backup 4096 Sep 01 06:30 2005/
34
35
36 In this root directory is one subdirectory for each year represented in
37 the backup. In this example, the backup represents data entirely from
38 the year 2005. If your configured backup week happens to span a year
39 boundary, there would be two subdirectories here (for example, one for
40 2005 and one for 2006).
41
42 Within each year directory is one subdirectory for each month
43 represented in the backup.
44
45 ::
46
47 root:/mnt/cdrw/2005# ls -l
48 total 2
49 dr-xr-xr-x 6 root root 2048 Sep 11 05:30 09/
50
51
52 In this example, the backup represents data entirely from the month of
53 September, 2005. If your configured backup week happens to span a month
54 boundary, there would be two subdirectories here (for example, one for
55 August 2005 and one for September 2005).
56
57 Within each month directory is one subdirectory for each day represented
58 in the backup.
59
60 ::
61
62 root:/mnt/cdrw/2005/09# ls -l
63 total 8
64 dr-xr-xr-x 5 root root 2048 Sep 7 05:30 07/
65 dr-xr-xr-x 5 root root 2048 Sep 8 05:30 08/
66 dr-xr-xr-x 5 root root 2048 Sep 9 05:30 09/
67 dr-xr-xr-x 5 root root 2048 Sep 11 05:30 11/
68
69
70 Depending on how far into the week your backup media is from, you might
71 have as few as one daily directory in here, or as many as seven.
72
73 Within each daily directory is a stage indicator (indicating when the
74 directory was staged) and one directory for each peer configured in the
75 backup:
76
77 ::
78
79 root:/mnt/cdrw/2005/09/07# ls -l
80 total 10
81 dr-xr-xr-x 2 root root 2048 Sep 7 02:31 host1/
82 -r--r--r-- 1 root root 0 Sep 7 03:27 cback.stage
83 dr-xr-xr-x 2 root root 4096 Sep 7 02:30 host2/
84 dr-xr-xr-x 2 root root 4096 Sep 7 03:23 host3/
85
86
87 In this case, you can see that my backup includes three machines, and
88 that the backup data was staged on September 7, 2005 at 03:27.
89
90 Within the directory for a given host are all of the files collected on
91 that host. This might just include tarfiles from a normal Cedar Backup
92 collect run, and might also include files “collected” from Cedar Backup
93 extensions or by other third-party processes on your system.
94
95 ::
96
97 root:/mnt/cdrw/2005/09/07/host1# ls -l
98 total 157976
99 -r--r--r-- 1 root root 11206159 Sep 7 02:30 boot.tar.bz2
100 -r--r--r-- 1 root root 0 Sep 7 02:30 cback.collect
101 -r--r--r-- 1 root root 3199 Sep 7 02:30 dpkg-selections.txt.bz2
102 -r--r--r-- 1 root root 908325 Sep 7 02:30 etc.tar.bz2
103 -r--r--r-- 1 root root 389 Sep 7 02:30 fdisk-l.txt.bz2
104 -r--r--r-- 1 root root 1003100 Sep 7 02:30 ls-laR.txt.bz2
105 -r--r--r-- 1 root root 19800 Sep 7 02:30 mysqldump.txt.bz2
106 -r--r--r-- 1 root root 4133372 Sep 7 02:30 opt-local.tar.bz2
107 -r--r--r-- 1 root root 44794124 Sep 8 23:34 opt-public.tar.bz2
108 -r--r--r-- 1 root root 30028057 Sep 7 02:30 root.tar.bz2
109 -r--r--r-- 1 root root 4747070 Sep 7 02:30 svndump-0:782-opt-svn-repo1.txt.bz2
110 -r--r--r-- 1 root root 603863 Sep 7 02:30 svndump-0:136-opt-svn-repo2.txt.bz2
111 -r--r--r-- 1 root root 113484 Sep 7 02:30 var-lib-jspwiki.tar.bz2
112 -r--r--r-- 1 root root 19556660 Sep 7 02:30 var-log.tar.bz2
113 -r--r--r-- 1 root root 14753855 Sep 7 02:30 var-mail.tar.bz2
114
115
116 As you can see, I back up variety of different things on host1. I run
117 the normal collect action, as well as the sysinfo, mysql and subversion
118 extensions. The resulting backup files are named in a way that makes it
119 easy to determine what they represent.
120
121 Files of the form ``*.tar.bz2`` represent directories backed up by the
122 collect action. The first part of the name (before “.tar.bz2”),
123 represents the path to the directory. For example, ``boot.tar.gz``
124 contains data from ``/boot``, and ``var-lib-jspwiki.tar.bz2`` contains
125 data from ``/var/lib/jspwiki``.
126
127 The ``fdisk-l.txt.bz2``, ``ls-laR.tar.bz2`` and
128 ``dpkg-selections.tar.bz2`` files are produced by the sysinfo extension.
129
130 The ``mysqldump.txt.bz2`` file is produced by the mysql extension. It
131 represents a system-wide database dump, because I use the “all” flag in
132 configuration. If I were to configure Cedar Backup to dump individual
133 datbases, then the filename would contain the database name (something
134 like ``mysqldump-bugs.txt.bz2``).
135
136 Finally, the files of the form ``svndump-*.txt.bz2`` are produced by the
137 subversion extension. There is one dump file for each configured
138 repository, and the dump file name represents the name of the repository
139 and the revisions in that dump. So, the file
140 ``svndump-0:782-opt-svn-repo1.txt.bz2`` represents revisions 0-782 of
141 the repository at ``/opt/svn/repo1``. You can tell that this file
142 contains a full backup of the repository to this point, because the
143 starting revision is zero. Later incremental backups would have a
144 non-zero starting revision, i.e. perhaps 783-785, followed by 786-800,
145 etc.
146
147 .. _cedar-recovering-filesystem:
148
149 Recovering Filesystem Data
150 --------------------------
151
152 Filesystem data is gathered by the standard Cedar Backup collect action.
153 This data is placed into files of the form ``*.tar``. The first part of
154 the name (before “.tar”), represents the path to the directory. For
155 example, ``boot.tar`` would contain data from ``/boot``, and
156 ``var-lib-jspwiki.tar`` would contain data from ``/var/lib/jspwiki``.
157 (As a special case, data from the root directory would be placed in
158 ``-.tar``). Remember that your tarfile might have a bzip2 (``.bz2``) or
159 gzip (``.gz``) extension, depending on what compression you specified in
160 configuration.
161
162 If you are using full backups every day, the latest backup data is
163 always within the latest daily directory stored on your backup media or
164 within your staging directory. If you have some or all of your
165 directories configured to do incremental backups, then the first day of
166 the week holds the full backups and the other days represent incremental
167 differences relative to that first day of the week.
168
169 If you are restoring a home directory or some other non-system directory
170 as part of a full restore, it is probably fine to extract the backup
171 directly into the filesystem.
172
173 If you are restoring a system directory like ``/etc`` as part of a full
174 restore, extracting directly into the filesystem is likely to break
175 things, especially if you re-installed a newer version of your operating
176 system than the one you originally backed up. It's better to extract
177 directories like this to a temporary location and pick out only the
178 files you find you need.
179
180 When doing a partial restore, I suggest *always* extracting to a
181 temporary location. Doing it this way gives you more control over what
182 you restore, and helps you avoid compounding your original problem with
183 another one (like overwriting the wrong file, oops).
184
185 .. _cedar-recovering-filesystem-full:
186
187 Full Restore
188 ------------
189
190 To do a full system restore, find the newest applicable full backup and
191 extract it. If you have some incremental backups, extract them into the
192 same place as the full backup, one by one starting from oldest to
193 newest. (This way, if a file changed every day you will always get the
194 latest one.)
195
196 All of the backed-up files are stored in the tar file in a relative
197 fashion, so you can extract from the tar file either directly into the
198 filesystem, or into a temporary location.
199
200 For example, to restore ``boot.tar.bz2`` directly into ``/boot``,
201 execute ``tar`` from your root directory (``/``):
202
203 ::
204
205 root:/# bzcat boot.tar.bz2 | tar xvf -
206
207
208 Of course, use ``zcat`` or just ``cat``, depending on what kind of
209 compression is in use.
210
211 If you want to extract ``boot.tar.gz`` into a temporary location like
212 ``/tmp/boot`` instead, just change directories first. In this case,
213 you'd execute the ``tar`` command from within ``/tmp`` instead of ``/``.
214
215 ::
216
217 root:/tmp# bzcat boot.tar.bz2 | tar xvf -
218
219
220 Again, use ``zcat`` or just ``cat`` as appropriate.
221
222 For more information, you might want to check out the manpage or GNU
223 info documentation for the ``tar`` command.
224
225 .. _cedar-recovering-filesystem-partial:
226
227 Partial Restore
228 ---------------
229
230 Most users will need to do a partial restore much more frequently than a
231 full restore. Perhaps you accidentally removed your home directory, or
232 forgot to check in some version of a file before deleting it. Or,
233 perhaps the person who packaged Apache for your system blew away your
234 web server configuration on upgrade (it happens). The solution to these
235 and other kinds of problems is a partial restore (assuming you've backed
236 up the proper things).
237
238 The procedure is similar to a full restore. The specific steps depend on
239 how much information you have about the file you are looking for. Where
240 with a full restore, you can confidently extract the full backup
241 followed by each of the incremental backups, this might not be what you
242 want when doing a partial restore. You may need to take more care in
243 finding the right version of a file --- since the same file, if
244 changed frequently, would appear in more than one backup.
245
246 Start by finding the backup media that contains the file you are looking
247 for. If you rotate your backup media, and your last known “contact” with
248 the file was a while ago, you may need to look on older media to find
249 it. This may take some effort if you are not sure when the change you
250 are trying to correct took place.
251
252 Once you have decided to look at a particular piece of backup media,
253 find the correct peer (host), and look for the file in the full backup:
254
255 ::
256
257 root:/tmp# bzcat boot.tar.bz2 | tar tvf - path/to/file
258
259
260 Of course, use ``zcat`` or just ``cat``, depending on what kind of
261 compression is in use.
262
263 The ``tvf`` tells ``tar`` to search for the file in question and just
264 list the results rather than extracting the file. Note that the filename
265 is relative (with no starting ``/``). Alternately, you can omit the
266 ``path/to/file`` and search through the output using ``more`` or
267 ``less``
268
269 If you haven't found what you are looking for, work your way through the
270 incremental files for the directory in question. One of them may also
271 have the file if it changed during the course of the backup. Or, move to
272 older or newer media and see if you can find the file there.
273
274 Once you have found your file, extract it using ``xvf``:
275
276 ::
277
278 root:/tmp# bzcat boot.tar.bz2 | tar xvf - path/to/file
279
280
281 Again, use ``zcat`` or just ``cat`` as appropriate.
282
283 Inspect the file and make sure it's what you're looking for. Again, you
284 may need to move to older or newer media to find the exact version of
285 your file.
286
287 For more information, you might want to check out the manpage or GNU
288 info documentation for the ``tar`` command.
289
290 .. _cedar-recovering-mysql:
291
292 Recovering MySQL Data
293 ---------------------
294
295 MySQL data is gathered by the Cedar Backup mysql extension. This
296 extension always creates a full backup each time it runs. This wastes
297 some space, but makes it easy to restore database data. The following
298 procedure describes how to restore your MySQL database from the backup.
299
300 |warning|
301
302 I am not a MySQL expert. I am providing this information for
303 reference. I have tested these procedures on my own MySQL
304 installation; however, I only have a single database for use by
305 Bugzilla, and I may have misunderstood something with regard to
306 restoring individual databases as a user other than root. If you have
307 any doubts, test the procedure below before relying on it!
308
309 MySQL experts and/or knowledgable Cedar Backup users: feel free to
310 write me and correct any part of this procedure.
311
312 First, find the backup you are interested in. If you have specified “all
313 databases” in configuration, you will have a single backup file, called
314 ``mysqldump.txt``. If you have specified individual databases in
315 configuration, then you will have files with names like
316 ``mysqldump-database.txt`` instead. In either case, your file might have
317 a ``.gz`` or ``.bz2`` extension depending on what kind of compression
318 you specified in configuration.
319
320 If you are restoring an “all databases” backup, make sure that you have
321 correctly created the root user and know its password. Then, execute:
322
323 ::
324
325 daystrom:/# bzcat mysqldump.txt.bz2 | mysql -p -u root
326
327
328 Of course, use ``zcat`` or just ``cat``, depending on what kind of
329 compression is in use.
330
331 Because the database backup includes ``CREATE DATABASE`` SQL statements, this
332 command should take care of creating all of the databases within the backup, as
333 well as populating them.
334
335 If you are restoring a backup for a specific database, you have two
336 choices. If you have a root login, you can use the same command as
337 above:
338
339 ::
340
341 daystrom:/# bzcat mysqldump-database.txt.bz2 | mysql -p -u root
342
343
344 Otherwise, you can create the database and its login first (or have
345 someone create it) and then use a database-specific login to execute the
346 restore:
347
348 ::
349
350 daystrom:/# bzcat mysqldump-database.txt.bz2 | mysql -p -u user database
351
352
353 Again, use ``zcat`` or just ``cat`` as appropriate.
354
355 For more information on using MySQL, see the documentation on the MySQL
356 web site, `<http://mysql.org/>`__, or the manpages for the ``mysql``
357 and ``mysqldump`` commands.
358
359 .. _cedar-recovering-subversion:
360
361 Recovering Subversion Data
362 --------------------------
363
364 Subversion data is gathered by the Cedar Backup subversion extension.
365 Cedar Backup will create either full or incremental backups, but the
366 procedure for restoring is the same for both. Subversion backups are
367 always taken on a per-repository basis. If you need to restore more than
368 one repository, follow the procedures below for each repository you are
369 interested in.
370
371 First, find the backup or backups you are interested in. Typically, you
372 will need the full backup from the first day of the week and each
373 incremental backup from the other days of the week.
374
375 The subversion extension creates files of the form ``svndump-*.txt``.
376 These files might have a ``.gz`` or ``.bz2`` extension depending on what
377 kind of compression you specified in configuration. There is one dump
378 file for each configured repository, and the dump file name represents
379 the name of the repository and the revisions in that dump. So, the file
380 ``svndump-0:782-opt-svn-repo1.txt.bz2`` represents revisions 0-782 of
381 the repository at ``/opt/svn/repo1``. You can tell that this file
382 contains a full backup of the repository to this point, because the
383 starting revision is zero. Later incremental backups would have a
384 non-zero starting revision, i.e. perhaps 783-785, followed by 786-800,
385 etc.
386
387 Next, if you still have the old Subversion repository around, you might
388 want to just move it off (rename the top-level directory) before
389 executing the restore. Or, you can restore into a temporary directory
390 and rename it later to its real name once you've checked it out. That is
391 what my example below will show.
392
393 Next, you need to create a new Subversion repository to hold the
394 restored data. This example shows an FSFS repository, but that is an
395 arbitrary choice. You can restore from an FSFS backup into a FSFS
396 repository or a BDB repository. The Subversion dump format is
397 “backend-agnostic”.
398
399 ::
400
401 root:/tmp# svnadmin create --fs-type=fsfs testrepo
402
403
404 Next, load the full backup into the repository:
405
406 ::
407
408 root:/tmp# bzcat svndump-0:782-opt-svn-repo1.txt.bz2 | svnadmin load testrepo
409
410
411 Of course, use ``zcat`` or just ``cat``, depending on what kind of
412 compression is in use.
413
414 Follow that with loads for each of the incremental backups:
415
416 ::
417
418 root:/tmp# bzcat svndump-783:785-opt-svn-repo1.txt.bz2 | svnadmin load testrepo
419 root:/tmp# bzcat svndump-786:800-opt-svn-repo1.txt.bz2 | svnadmin load testrepo
420
421
422 Again, use ``zcat`` or just ``cat`` as appropriate.
423
424 When this is done, your repository will be restored to the point of the
425 last commit indicated in the svndump file (in this case, to revision
426 800).
427
428 |note|
429
430 *Note:* don't be surprised if, when you test this, the restored
431 directory doesn't have exactly the same contents as the original
432 directory. I can't explain why this happens, but if you execute
433 ``svnadmin dump`` on both old and new repositories, the results are
434 identical. This means that the repositories do contain the same
435 content.
436
437 For more information on using Subversion, see the book Version Control
438 with Subversion (`<http://svnbook.red-bean.com/>`__) or the Subversion
439 FAQ (`<http://subversion.tigris.org/faq.html>`__).
440
441 .. _cedar-recovering-mbox:
442
443 Recovering Mailbox Data
444 -----------------------
445
446 Mailbox data is gathered by the Cedar Backup mbox extension. Cedar
447 Backup will create either full or incremental backups, but both kinds of
448 backups are treated identically when restoring.
449
450 Individual mbox files and mbox directories are treated a little
451 differently, since individual files are just compressed, but directories
452 are collected into a tar archive.
453
454 First, find the backup or backups you are interested in. Typically, you
455 will need the full backup from the first day of the week and each
456 incremental backup from the other days of the week.
457
458 The mbox extension creates files of the form ``mbox-*``. Backup files
459 for individual mbox files might have a ``.gz`` or ``.bz2`` extension
460 depending on what kind of compression you specified in configuration.
461 Backup files for mbox directories will have a ``.tar``, ``.tar.gz`` or
462 ``.tar.bz2`` extension, again depending on what kind of compression you
463 specified in configuration.
464
465 There is one backup file for each configured mbox file or directory. The
466 backup file name represents the name of the file or directory and the
467 date it was backed up. So, the file
468 ``mbox-20060624-home-user-mail-greylist`` represents the backup for
469 ``/home/user/mail/greylist`` run on 24 Jun 2006. Likewise,
470 ``mbox-20060624-home-user-mail.tar`` represents the backup for the
471 ``/home/user/mail`` directory run on that same date.
472
473 Once you have found the files you are looking for, the restoration
474 procedure is fairly simple. First, concatenate all of the backup files
475 together. Then, use grepmail to eliminate duplicate messages (if any).
476
477 Here is an example for a single backed-up file:
478
479 ::
480
481 root:/tmp# rm restore.mbox # make sure it's not left over
482 root:/tmp# cat mbox-20060624-home-user-mail-greylist >> restore.mbox
483 root:/tmp# cat mbox-20060625-home-user-mail-greylist >> restore.mbox
484 root:/tmp# cat mbox-20060626-home-user-mail-greylist >> restore.mbox
485 root:/tmp# grepmail -a -u restore.mbox > nodups.mbox
486
487
488 At this point, ``nodups.mbox`` contains all of the backed-up messages
489 from ``/home/user/mail/greylist``.
490
491 Of course, if your backups are compressed, you'll have to use ``zcat``
492 or ``bzcat`` rather than just ``cat``.
493
494 If you are backing up mbox directories rather than individual files, see
495 the filesystem instructions for notes on now to extract the individual
496 files from inside tar archives. Extract the files you are interested in,
497 and then concatenate them together just like shown above for the
498 individual case.
499
500 .. _cedar-recovering-split:
501
502 Recovering Data split by the Split Extension
503 --------------------------------------------
504
505 The Split extension takes large files and splits them up into smaller
506 files. Typically, it would be used in conjunction with the
507 ``cback3-span`` command.
508
509 The split up files are not difficult to work with. Simply find all of
510 the files --- which could be split between multiple discs --- and
511 concatenate them together.
512
513 ::
514
515 root:/tmp# rm usr-src-software.tar.gz # make sure it's not there
516 root:/tmp# cat usr-src-software.tar.gz_00001 >> usr-src-software.tar.gz
517 root:/tmp# cat usr-src-software.tar.gz_00002 >> usr-src-software.tar.gz
518 root:/tmp# cat usr-src-software.tar.gz_00003 >> usr-src-software.tar.gz
519
520
521 Then, use the resulting file like usual.
522
523 Remember, you need to have *all* of the files that the original large
524 file was split into before this will work. If you are missing a file,
525 the result of the concatenation step will be either a corrupt file or a
526 truncated file (depending on which chunks you did not include).
527
528 ----------
529
530 *Previous*: :doc:`depends` • *Next*: :doc:`securingssh`
531
532 .. |note| image:: images/note.png
533 .. |tip| image:: images/tip.png
534 .. |warning| image:: images/warning.png