\chapter{The Restore Command} \label{blb:RestoreChapter} \index[general]{Command!Console Restore} \index[general]{Console Restore Command} \section{General} \index[general]{General } Below, we will discuss restoring files with the Console \bcommandname{restore} command, which is the recommended way of doing restoring files. It is not possible to restore files by automatically starting a job as you do with Backup, Verify, \ldots{} jobs. However, in addition to the console \bcommandname{restore} command, there is a standalone program named \btool{bextract}, which also permits restoring files. For more information on this program, please see the \bxlink{bextract}{blb:bextract}{utility}{command} in the \butilityman{}. We don't particularly recommend the \btool{bextract} program because it lacks many of the features of the normal \bacula{} restore, such as the ability to restore Win32 files to Unix systems, and the ability to restore access control lists (ACL). As a consequence, we recommend, wherever possible to use \bacula{} itself for restores as described below. You may also want to look at the \btool{bls} program in the same chapter, which allows you to list the contents of your Volumes. Finally, if you have an old Volume that is no longer in the catalog, you can restore the catalog entries using the program named \btool{bscan}, documented in the same \bxlink{bscan}{blb:bscan}{utility}{command} in the \butilityman{}. In general, to restore a file or a set of files, you must run a \bcommandname{restore} job. That is a job with \textbf{Type = Restore}. As a consequence, you will need a predefined \bcommandname{restore} job in your \bfilename{bacula-dir.conf} (Director's config) file. The exact parameters (Client, FileSet, \ldots{}) that you define are not important as you can either modify them manually before running the job or if you use the \bcommandname{restore} command, explained below, \bacula{} will automatically set them for you. In fact, you can no longer simply run a restore job. You must use the \bcommandname{restore} command. Since \bacula{} is a network backup program, you must be aware that when you restore files, it is up to you to ensure that you or \bacula{} have selected the correct Client and the correct hard disk location for restoring those files. \textbf{\bacula{}} will quite willingly backup client A, and restore it by sending the files to a different directory on client B. Normally, you will want to avoid this, but assuming the operating systems are not too different in their file structures, this should work perfectly well, if so desired. By default, \bacula{} will restore data to the same Client that was backed up, and those data will be restored not to the original places but to \bdirectoryname{/tmp/bacula-restores}. You may modify any of these defaults when the \bcommandname{restore} command prompts you to run the job by selecting the \textbf{mod} option. \label{blb:Example1} \section{The Restore Command} \index[general]{Command!Restore} \index[general]{Restore Command} Since \bacula{} maintains a catalog of your files and on which Volumes (disk or tape), they are stored, it can do most of the bookkeeping work, allowing you simply to specify what kind of restore you want (current, before a particular date), and what files to restore. \bacula{} will then do the rest. This is accomplished using the \bcommandname{restore} command in the Console. First you select the kind of restore you want, then the JobIds are selected, the File records for those Jobs are placed in an internal \bacula{} directory tree, and the restore enters a file selection mode that allows you to interactively walk up and down the file tree selecting individual files to be restored. This mode is somewhat similar to the standard Unix \bcommandname{restore} program's interactive file selection mode. If a Job's file records have been pruned from the catalog, the \bcommandname{restore} command will be unable to find any files to restore. \bacula{} will ask if you want to restore all of them or if you want to use a regular expression to restore only a selection while reading media. See \bilink{FileRegex option}{blb:FileRegex} and below for more details on this. Within the Console program, after entering the \bcommandname{restore} command, you are presented with the following selection prompt: \begin{bVerbatim} First you select one or more JobIds that contain files to be restored. You will be presented several methods of specifying the JobIds. Then you will be allowed to select which files from those JobIds are to be restored. To select the JobIds, you have the following choices: 1: List last 20 Jobs run 2: List Jobs where a given File is saved 3: Enter list of comma separated JobIds to select 4: Enter SQL list command 5: Select the most recent backup for a client 6: Select backup for a client before a specified time 7: Enter a list of files to restore 8: Enter a list of files to restore before a specified time 9: Find the JobIds of the most recent backup for a client 10: Find the JobIds for a backup for a client before a specified time 11: Enter a list of directories to restore for found JobIds 12: Select full restore to a specified Job date 13: Select object to restore 14: Cancel Select item: (1-14): \end{bVerbatim} There are a lot of options, and as a point of reference, most people will want to slect item 5 (the most recent backup for a client). The details of the above options are: \begin{bitemize} \item Item 1 will list the last 20 jobs run. If you find the Job you want, you can then select item 3 and enter its JobId(s). \item Item 2 will list all the Jobs where a specified file is saved. If you find the Job you want, you can then select item 3 and enter the JobId. \item Item 3 allows you the enter a list of comma separated JobIds whose files will be put into the directory tree. You may then select which files from those JobIds to restore. Normally, you would use this option if you have a particular version of a file that you want to restore and you know its JobId. The most common options (5 and 6) will not select a job that did not terminate normally, so if you know a file is backed up by a Job that failed (possibly because of a system crash), you can access it through this option by specifying the JobId. \item Item 4 allows you to enter any arbitrary \acs{SQL} command. This is probably the most primitive way of finding the desired JobIds, but at the same time, the most flexible. Once you have found the JobId(s), you can select item 3 and enter them. \item Item 5 will automatically select the most recent Full backup and all subsequent incremental and differential backups for a specified Client. These are the Jobs and Files which, if reloaded, will restore your system to the most current saved state. It automatically enters the JobIds found into the directory tree in an optimal way such that only the most recent copy of any particular file found in the set of Jobs will be restored. This is probably the most convenient of all the above options to use if you wish to restore a selected Client to its most recent state. There are two important things to note. First, this automatic selection will never select a job that failed (terminated with an error status). If you have such a job and want to recover one or more files from it, you will need to explicitly enter the JobId in item 3, then choose the files to restore. If some of the Jobs that are needed to do the restore have had their File records pruned, the restore will be incomplete. \bacula{} currently does not correctly detect this condition. You can however, check for this by looking carefully at the list of Jobs that \bacula{} selects and prints. If you find Jobs with the JobFiles column set to zero, when files should have been backed up, then you should expect problems. If all the File records have been pruned, \bacula{} will realize that there are no file records in any of the JobIds chosen and will inform you. It will then propose doing a full restore (non-selective) of those JobIds. This is possible because \bacula{} still knows where the beginning of the Job data is on the Volumes, even if it does not know where particular files are located or what their names are. \item Item 6 allows you to specify a date and time, after which \bacula{} will automatically select the most recent Full backup and all subsequent incremental and differential backups that started before the specified date and time. \item Item 7 allows you to specify one or more filenames (complete path required) to be restored. Each filename is entered one at a time or if you prefix a filename with the less-than symbol (\lt{}) \bacula{} will read that file and assume it is a list of filenames to be restored. If you prefix the filename with a question mark (?), then the filename will be interpreted as an SQL table name, and \bacula{} will include the rows of that table in the list to be restored. The table must contain the JobId in the first column and the FileIndex in the second column. This table feature is intended for external programs that want to build their own list of files to be restored. The filename entry mode is terminated by entering a blank line. \item Item 8 allows you to specify a date and time before entering the filenames. See Item 7 above for more details. \item Item 9 allows you find the JobIds of the most recent backup for a client. This is much like option 5 (it uses the same code), but those JobIds are retained internally as if you had entered them manually. You may then select item 11 (see below) to restore one or more directories. \item Item 10 is the same as item 9, except that it allows you to enter a before date (as with item 6). These JobIds will then be retained internally. \index[general]{Restore Directories} \item Item 11 allows you to enter a list of JobIds from which you can select directories to be restored. The list of JobIds can have been previously created by using either item 9 or 10 on the menu. You may then enter a full path to a directory name or a filename preceded by a less than sign (\lt{}). The filename should contain a list of directories to be restored. All files in those directories will be restored, but if the directory contains subdirectories, nothing will be restored in the subdirectory unless you explicitly enter its name. \item Item 12 allows you to restore choosen state of the Job. \item Item 13 allows you to enter menu to select object to be restored. \item Item 14 allows you to cancel the \bcommandname{restore} command. \end{bitemize} As an example, suppose that we select item 5 (restore to most recent state). If you have not specified a \texttt{client=xxx} on the command line, it it will then ask for the desired Client, which on my system, will print all the Clients found in the database as follows: \begin{bVerbatim} Defined clients: 1: Rufus 2: Matou 3: Polymatou 4: Minimatou 5: Minou 6: MatouVerify 7: PmatouVerify 8: RufusVerify 9: Watchdog Select Client (File daemon) resource (1-9): \end{bVerbatim} You will probably have far fewer Clients than this example, and if you have only one Client, it will be automatically selected. In this case, I enter \textbf{Rufus} to select the Client. Then \bacula{} needs to know what FileSet is to be restored, so it prompts with: \begin{bVerbatim} The defined FileSet resources are: 1: Full Set 2: Other Files Select FileSet resource (1-2): \end{bVerbatim} If you have only one FileSet defined for the Client, it will be selected automatically. I choose item 1, which is my full backup. Normally, you will only have a single FileSet for each Job, and if your machines are similar (all Linux) you may only have one FileSet for all your Clients. At this point, \textbf{\bacula{}} has all the information it needs to find the most recent set of backups. It will then query the database, which may take a bit of time, and it will come up with something like the following. Note, some of the columns are truncated here for presentation: \begin{bVerbatim} +-------+------+----------+-------------+-------------+------+-------+------------+ | JobId | Levl | JobFiles | StartTime | VolumeName | File | SesId |VolSesTime | +-------+------+----------+-------------+-------------+------+-------+------------+ | 1,792 | F | 128,374 | 08-03 01:58 | DLT-19Jul02 | 67 | 18 | 1028042998 | | 1,792 | F | 128,374 | 08-03 01:58 | DLT-04Aug02 | 0 | 18 | 1028042998 | | 1,797 | I | 254 | 08-04 13:53 | DLT-04Aug02 | 5 | 23 | 1028042998 | | 1,798 | I | 15 | 08-05 01:05 | DLT-04Aug02 | 6 | 24 | 1028042998 | +-------+------+----------+-------------+-------------+------+-------+------------+ You have selected the following JobId: 1792,1792,1797 Building directory tree for JobId 1792 ... Building directory tree for JobId 1797 ... Building directory tree for JobId 1798 ... cwd is: / $ \end{bVerbatim} Depending on the number of \textbf{JobFiles} for each JobId, the \textbf{Building directory tree \ldots{}"} can take a bit of time. If you notice ath all the JobFiles are zero, your Files have probably been pruned and you will not be able to select any individual files -- it will be restore everything or nothing. In our example, \bacula{} found four Jobs that comprise the most recent backup of the specified Client and FileSet. Two of the Jobs have the same JobId because that Job wrote on two different Volumes. The third Job was an incremental backup to the previous Full backup, and it only saved 254 Files compared to 128,374 for the Full backup. The fourth Job was also an incremental backup that saved 15 files. Next \bacula{} entered those Jobs into the directory tree, with no files marked to be restored as a default, tells you how many files are in the tree, and tells you that the current working directory (\bcommandname{cwd}) is /. Finally, \bacula{} prompts with the dollar sign (\$) to indicate that you may enter commands to move around the directory tree and to select files. If you want all the files to automatically be marked when the directory tree is built, you could have entered the command \bcommandname{restore all}, or at the \$ prompt, you can simply enter \bcommandname{mark *}. Instead of choosing item 5 on the first menu (Select the most recent backup for a client), if we had chosen item 3 (Enter list of JobIds to select) and we had entered the JobIds \textbf{1792,1797,1798} we would have arrived at the same point. One point to note, if you are manually entering JobIds, is that you must enter them in the order they were run (generally in increasing JobId order). If you enter them out of order and the same file was saved in two or more of the Jobs, you may end up with an old version of that file (i.e. not the most recent). Directly entering the JobIds can also permit you to recover data from a Job that wrote files to tape but that terminated with an error status. While in file selection mode, you can enter \bcommandname{help} or a question mark (?) to produce a summary of the available commands: \begin{bVerbatim} Command Description ======= =========== cd change current directory count count marked files in and below the cd dir long list current directory, wildcards allowed done leave file selection mode du estimate estimate restore size exit same as done command find find files, wildcards allowed help print help ls list current directory, wildcards allowed lsmark list the marked files in and below the cd mark mark dir/file to be restored recursively in dirs markdir mark directory name to be restored (no files) pwd print current working directory unmark unmark dir/file to be restored recursively in dir unmarkdir unmark directory name only no recursion quit quit and do not do restore ? print help \end{bVerbatim} As a default no files have been selected for restore (unless you added \textbf{all} to the command line. If you want to restore everything, at this point, you should enter \bcommandname{mark *}, and then \bcommandname{done} and \textbf{\bacula{}} will write the bootstrap records to a file and request your approval to start a restore job. If you do not enter the above mentioned \bcommandname{mark *} command, you will start with an empty slate. Now you can simply start looking at the tree and \bcommandname{mark} particular files or directories you want restored. It is easy to make a mistake in specifying a file to mark or unmark, and \bacula{}'s error handling is not perfect, so please check your work by using the \bcommandname{ls} or \bcommandname{dir} commands to see what files are actually selected. Any selected file has its name preceded by an asterisk. To check what is marked or not marked, enter the \bcommandname{count} command, which displays: \begin{bVerbatim} 128401 total files. 128401 marked to be restored. \end{bVerbatim} Each of the above commands will be described in more detail in the next section. We continue with the above example, having accepted to restore all files as \bacula{} set by default. On entering the \bcommandname{done} command, \bacula{} prints: \begin{bVerbatim} Bootstrap records written to /home/kern/bacula/working/restore.bsr The job will require the following Volume(s) Storage(s) SD Device(s) =========================================================================== DLT-19Jul02 Tape DLT8000 DLT-04Aug02 Tape DLT8000 128401 files selected to restore. Run Restore job JobName: kernsrestore Bootstrap: /home/kern/bacula/working/restore.bsr Where: /tmp/bacula-restores Replace: always FileSet: Other Files Client: Rufus Storage: Tape When: 2006-12-11 18:20:33 Catalog: MyCatalog Priority: 10 OK to run? (yes/mod/no): \end{bVerbatim} Please examine each of the items very carefully to make sure that they are correct. In particular, look at \textbf{Where}, which tells you where in the directory structure the files will be restored, and \textbf{Client}, which tells you which client will receive the files. Note that by default the Client which will receive the files is the Client that was backed up. These items will not always be completed with the correct values depending on which of the restore options you chose. You can change any of these default items by entering \textbf{mod} and responding to the prompts. The above assumes that you have defined a \textbf{Restore} Job resource in your Director's configuration file. Normally, you will only need one Restore Job resource definition because by its nature, restoring is a manual operation, and using the Console interface, you will be able to modify the Restore Job to do what you want. An example Restore Job resource definition is given below. Returning to the above example, you should verify that the Client name is correct before running the Job. However, you may want to modify some of the parameters of the restore job. For example, in addition to checking the Client it is wise to check that the Storage device chosen by \bacula{} is indeed correct. Although the \textbf{FileSet} is shown, it will be ignored in restore. The restore will choose the files to be restored either by reading the \textbf{Bootstrap} file, or if not specified, it will restore all files associated with the specified backup \textbf{JobId} (i.e. the JobId of the Job that originally backed up the files). Finally before running the job, please note that the default location for restoring files is \textbf{not} their original locations, but rather the directory \bdirectoryname{/tmp/bacula-restores}. You can change this default by modifying your \bfilename{bacula-dir.conf} file, or you can modify it using the \textbf{mod} option. If you want to restore the files to their original location, you must have \textbf{Where} set to nothing or to the root, i.e. \bdirectoryname{/}. If you now enter \textbf{yes}, \bacula{} will run the restore Job. The Storage daemon will first request Volume \textbf{DLT-19Jul02} and after the appropriate files have been restored from that volume, it will request Volume \textbf{DLT-04Aug02}. \subsection{Restore a pruned job using a pattern} During a restore, if all File records are pruned from the catalog for a Job, normally \bacula{} can restore only all files saved. That is there is no way using the catalog to select individual files. With this new feature, \bacula{} will ask if you want to specify a Regex expression for extracting only a part of the full backup. \begin{bVerbatim} Building directory tree for JobId(s) 1,3 ... There were no files inserted into the tree, so file selection is not possible.Most likely your retention policy pruned the files Do you want to restore all the files? (yes|no): no Regexp matching files to restore? (empty to abort): /tmp/regress/(bin|tests)/ Bootstrap records written to /tmp/regress/working/zog4-dir.restore.1.bsr \end{bVerbatim} See also \bilink{FileRegex bsr option}{blb:FileRegex} for more information. \section{Selecting Files by Filename} \index[general]{Selecting Files by Filename } \index[general]{Filename!Selecting Files by } If you have a small number of files to restore, and you know the filenames, you can either put the list of filenames in a file to be read by \bacula{}, or you can enter the names one at a time. The filenames must include the full path and filename. No wild cards are used. To enter the files, after the \bcommandname{restore}, you select item number 7 from the prompt list: \begin{bVerbatim} To select the JobIds, you have the following choices: 1: List last 20 Jobs run 2: List Jobs where a given File is saved 3: Enter list of comma separated JobIds to select 4: Enter SQL list command 5: Select the most recent backup for a client 6: Select backup for a client before a specified time 7: Enter a list of files to restore 8: Enter a list of files to restore before a specified time 9: Find the JobIds of the most recent backup for a client 10: Find the JobIds for a backup for a client before a specified time 11: Enter a list of directories to restore for found JobIds 12: Cancel Select item: (1-12): \end{bVerbatim} which then prompts you for the client name: \begin{bVerbatim} Defined Clients: 1: Timmy 2: Tibs 3: Rufus Select the Client (1-3): 3 \end{bVerbatim} Of course, your client list will be different, and if you have only one client, it will be automatically selected. And finally, \bacula{} requests you to enter a filename: \begin{bVerbatim} Enter filename: \end{bVerbatim} At this point, you can enter the full path and filename \begin{bVerbatim} Enter filename: /home/kern/bacula/k/Makefile.in Enter filename: \end{bVerbatim} as you can see, it took the filename. If \bacula{} cannot find a copy of the file, it prints the following: \begin{bVerbatim} Enter filename: junk filename No database record found for: junk filename Enter filename: \end{bVerbatim} If you want \bacula{} to read the filenames from a file, you simply precede the filename with a less-than symbol (\lt{}). When you have entered all the filenames, you enter a blank line, and \bacula{} will write the bootstrap file, tells you what tapes will be used, and proposes a Restore job to be run: \begin{bVerbatim} Enter filename: Automatically selected Storage: DDS-4 Bootstrap records written to /home/kern/bacula/working/restore.bsr The restore job will require the following Volumes: test1 1 file selected to restore. Run Restore job JobName: kernsrestore Bootstrap: /home/kern/bacula/working/restore.bsr Where: /tmp/bacula-restores Replace: always FileSet: Other Files Client: Rufus Storage: DDS-4 When: 2003-09-11 10:20:53 Priority: 10 OK to run? (yes/mod/no): \end{bVerbatim} It is possible to automate the selection by file by putting your list of files in say \bdirectoryname{/tmp/file-list}, then using the following command: \begin{bVerbatim} restore client=Rufus file= = / ! ; % : , ~ # = & \end{bVerbatim} You can use several expressions separated by a commas. \subsection*{Examples} \LTXtable{\linewidth}{t-main-regexp} %\subsubsection{Using group} % %Like with Perl or Sed, you can make submatch with \texttt{()}, % %\subsubsection*{Examples} %\subsubsection{Options} % % i Do case-insensitive pattern matching. \section{Restoring Directory Attributes} \index[general]{Attributes!Restoring Directory } \index[general]{Restoring Directory Attributes } Depending how you do the restore, you may or may not get the directory entries back to their original state. Here are a few of the problems you can encounter, and for same machine restores, how to avoid them. \begin{bitemize} \item You backed up on one machine and are restoring to another that is either a different OS or doesn't have the same users/groups defined. \bacula{} does the best it can in these situations. Note, \bacula{} has saved the user/groups in numeric form, which means on a different machine, they may map to different user/group names. \item You are restoring into a directory that is already created and has file creation restrictions. \bacula{} tries to reset everything but without walking up the full chain of directories and modifying them all during the restore, which \bacula{} does and will not do, getting permissions back correctly in this situation depends to a large extent on your OS. \item You are doing a recursive restore of a directory tree. In this case \bacula{} will restore a file before restoring the file's parent directory entry. In the process of restoring the file \bacula{} will create the parent directory with open permissions and ownership of the file being restored. Then when \bacula{} tries to restore the parent directory \bacula{} sees that it already exists (Similar to the previous situation). If you had set the Restore job's "Replace" property to "never" then \bacula{} will not change the directory's permissions and ownerships to match what it backed up, you should also notice that the actual number of files restored is less then the expected number. If you had set the Restore job's "Replace" property to "always" then \bacula{} will change the Directory's ownership and permissions to match what it backed up, also the actual number of files restored should be equal to the expected number. \item You selected one or more files in a directory, but did not select the directory entry to be restored. In that case, if the directory is not on disk \bacula{} simply creates the directory with some default attributes which may not be the same as the original. If you do not select a directory and all its contents to be restored, you can still select items within the directory to be restored by individually marking those files, but in that case, you should individually use the \bcommandname{markdir} command to select all higher level directory entries (one at a time) to be restored if you want the directory entries properly restored. \item The \btool{bextract} program does not restore access control lists (ACLs) to Unix machines. \end{bitemize} \label{blb:Windows} \section{Restoring on Windows} \index[general]{Restoring on Windows } \index[general]{Windows!Restoring on } If you are restoring on WinNT/2K/XP systems, \bacula{} will restore the files with the original ownerships and permissions as would be expected. This is also true if you are restoring those files to an alternate directory (using the Where option in restore). However, if the alternate directory does not already exist, the \bacula{} File daemon (Client) will try to create it. In some cases, it may not create the directories, and if it does since the File daemon runs under the SYSTEM account, the directory will be created with SYSTEM ownership and permissions. In this case, you may have problems accessing the newly restored files. To avoid this problem, you should create any alternate directory before doing the restore. \bacula{} will not change the ownership and permissions of the directory if it is already created as long as it is not one of the directories being restored (i.e. written to tape). The default restore location is \bdirectoryname{/tmp/bacula-restores/} and if you are restoring from drive \bdirectoryname{E:}, the default will be \bdirectoryname{/tmp/bacula-restores/e/}, so you should ensure that this directory exists before doing the restore, or use the \textbf{mod} option to select a different \textbf{where} directory that does exist. Some users have experienced problems restoring files that participate in the Active Directory. They also report that changing the userid under which \bacula{} (bacula-fd.exe) runs, from SYSTEM to a Domain Admin userid, resolves the problem. \section{Restoring Files Can Be Slow} \index[general]{Slow!Restoring Files Can Be } \index[general]{Restoring Files Can Be Slow } Restoring files is generally \textbf{much} slower than backing them up for several reasons. The first is that during a backup the tape is normally already positioned and \bacula{} only needs to write. On the other hand, because restoring files is done so rarely, \bacula{} keeps only the start file and block on the tape for the whole job rather than on a file by file basis which would use quite a lot of space in the catalog. \bacula{} will forward space to the correct file mark on the tape for the Job, then forward space to the correct block, and finally sequentially read each record until it gets to the correct one(s) for the file or files you want to restore. Once the desired files are restored, \bacula{} will stop reading the tape. Finally, instead of just reading a file for backup, during the restore, \bacula{} must create the file, and the operating system must allocate disk space for the file as \bacula{} is restoring it. For all the above reasons the restore process is generally much slower than backing up (sometimes it takes three times as long). \section{Problems Restoring Files} \index[general]{Files!Problems Restoring } \index[general]{Problems Restoring Files } The most frequent problems users have restoring files are error messages such as: \begin{bVerbatim} 04-Jan 00:33 z217-sd: RestoreFiles.2005-01-04_00.31.04 Error: block.c:868 Volume data error at 20:0! Short block of 512 bytes on device /dev/tape discarded. \end{bVerbatim} or \begin{bVerbatim} 04-Jan 00:33 z217-sd: RestoreFiles.2005-01-04_00.31.04 Error: block.c:264 Volume data error at 20:0! Wanted ID: "BB02", got ".". Buffer discarded. \end{bVerbatim} Both these kinds of messages indicate that you were probably running your tape drive in fixed block mode rather than variable block mode. Fixed block mode will work with any program that reads tapes sequentially such as tar, but \bacula{} repositions the tape on a block basis when restoring files because this will speed up the restore by orders of magnitude when only a few files are being restored. There are several ways that you can attempt to recover from this unfortunate situation. Try the following things, each separately, and reset your Device resource to what it is now after each individual test: \begin{benumerate} \item Set "Block Positioning = no" in your Device resource and try the restore. This is a new directive and untested. \item Set "Minimum Block Size = 512" and "Maximum Block Size = 512" and try the restore. If you are able to determine the block size your drive was previously using, you should try that size if 512 does not work. This is a really horrible solution, and it is not at all recommended to continue backing up your data without correcting this condition. Please see the Tape Testing chapter for more on this. \item Try editing the restore.bsr file at the Run xxx yes/mod/no prompt before starting the restore job and remove all the VolBlock statements. These are what causes \bacula{} to reposition the tape, and where problems occur if you have a fixed block size set for your drive. The VolFile commands also cause repositioning, but this will work regardless of the block size. \item Use bextract to extract the files you want -- it reads the Volume sequentially if you use the include list feature, or if you use a .bsr file, but remove all the VolBlock statements after the .bsr file is created (at the Run yes/mod/no) prompt but before you start the restore. \end{benumerate} \section{Restore Errors} \index[general]{Errors!Restore} \index[general]{Restore Errors} There are a number of reasons why there may be restore errors or warning messages. Some of the more common ones are: \begin{bdescription} \item [file count mismatch] This can occur for the following reasons: \begin{bitemize} \item You requested \bacula{} not to overwrite existing or newer files. \item A \bacula{} miscount of files/directories. This is an on-going problem due to the complications of directories, soft/hard link, and such. Simply check that all the files you wanted were actually restored. \end{bitemize} \item [file size error] When \bacula{} restores files, it checks that the size of the restored file is the same as the file status data it saved when starting the backup of the file. If the sizes do not agree, \bacula{} will print an error message. This size mismatch most often occurs because the file was being written as \bacula{} backed up the file. In this case, the size that \bacula{} restored will be greater than the status size. This often happens with log files. If the restored size is smaller, then you should be concerned about a possible tape error and check the \bacula{} output as well as your system logs. \end{bdescription} \section{Example Restore Job Resource} \index[general]{Example Restore Job Resource } \index[general]{Resource!Example Restore Job } \begin{bVerbatim} Job { Name = "RestoreFiles" Type = Restore Client = Any-client FileSet = "Any-FileSet" Storage = Any-storage Where = /tmp/bacula-restores Messages = Standard Pool = Default } \end{bVerbatim} If \textbf{Where} is not specified, the default location for restoring files will be their original locations. \label{blb:Selection} \section{File Selection Commands} \index[general]{Commands!File Selection } \index[general]{File Selection Commands } After you have selected the Jobs to be restored and \bacula{} has created the in-memory directory tree, you will enter file selection mode as indicated by the dollar sign (\textbf{\$}) prompt. While in this mode, you may use the commands listed above. The basic idea is to move up and down the in memory directory structure with the \bcommandname{cd} command much as you normally do on the system. Once you are in a directory, you may select the files that you want restored. As a default no files are marked to be restored. If you wish to start with all files, simply enter: \bcommandname{cd /} and \bcommandname{mark *}. Otherwise proceed to select the files you wish to restore by marking them with the \bcommandname{mark} command. The available commands are: \begin{bdescription} \item [cd] The \bcommandname{cd} command changes the current directory to the argument specified. It operates much like the Unix \bcommandname{cd} command. Wildcard specifications are not permitted. Note, on Windows systems, the various drives (c:, d:, \ldots{}) are treated like a directory within the file tree while in the file selection mode. As a consequence, you must do a \bcommandname{cd c:} or possibly in some cases a \bcommandname{cd C:} (note upper case) to get down to the first directory. \item [dir] \index[dir]{dir } The \bcommandname{dir} command is similar to the \bcommandname{ls} command, except that it prints it in long format (all details). This command can be a bit slower than the \bcommandname{ls} command because it must access the catalog database for the detailed information for each file. \item [estimate] \index[dir]{estimate } The \bcommandname{estimate} command prints a summary of the total files in the tree, how many are marked to be restored, and an estimate of the number of bytes to be restored. This can be useful if you are short on disk space on the machine where the files will be restored. \item [find] \index[dir]{find} The \bcommandname{find} command accepts one or more arguments and displays all files in the tree that match that argument. The argument may have wildcards. It is somewhat similar to the Unix command \bcommandname{find / -name arg}. \item [ls] The \bcommandname{ls} command produces a listing of all the files contained in the current directory much like the Unix \bcommandname{ls} command. You may specify an argument containing wildcards, in which case only those files will be listed. Any file that is marked to be restored will have its name preceded by an asterisk (\textbf{*}). Directory names will be terminated with a forward slash (\textbf{/}) to distinguish them from filenames. \item [du] The \bcommandname{du} command is similar to the \bcommandname{ls} command but it also prints size of each file directory listed. For directory it is just a sum of all files sizes in all directories below. \item [lsmark] \index[fd]{lsmark} The \bcommandname{lsmark} command is the same as the \bcommandname{ls} except that it will print only those files marked for extraction. The other distinction is that it will recursively descend into any directory selected. \item [mark] \index[dir]{mark} The \bcommandname{mark} command allows you to mark files to be restored. It takes a single argument which is the filename or directory name in the current directory to be marked for extraction. The argument may be a wildcard specification, in which case all files that match in the current directory are marked to be restored. If the argument matches a directory rather than a file, then the directory and all the files contained in that directory (recursively) are marked to be restored. Any marked file will have its name preceded with an asterisk (\textbf{*}) in the output produced by the \bcommandname{ls} or \bcommandname{dir} commands. Note, supplying a full path on the mark command does not work as expected to select a file or directory in the current directory. Also, the \bcommandname{mark} command works on the current and lower directories but does not touch higher level directories. After executing the \bcommandname{mark} command, it will print a brief summary: \begin{bVerbatim} No files marked. \end{bVerbatim} If no files were marked, or: \begin{bVerbatim} nn files marked. \end{bVerbatim} if some files are marked. \item [unmark] \index[dir]{unmark } The \bcommandname{unmark} is identical to the \bcommandname{mark} command, except that it unmarks the specified file or files so that they will not be restored. Note: the \bcommandname{unmark} command works from the current directory, so it does not unmark any files at a higher level. First do a \bcommandname{cd /} before the \bcommandname{unmark *} command if you want to unmark everything. \item [pwd] \index[dir]{pwd } The \bcommandname{pwd} command prints the current working directory. It accepts no arguments. \item [count] \index[dir]{count } The \bcommandname{count} command prints the total files in the directory tree and the number of files marked to be restored. \item [done] \index[dir]{done } This command terminates file selection mode. \item [exit] \index[fd]{exit } This command terminates file selection mode (the same as done). \item [quit] \index[fd]{quit } This command terminates the file selection and does not run the restore job. \item [help] \index[fd]{help } This command prints a summary of the commands available. \item [?] This command is the same as the \bcommandname{help} command. \end{bdescription} If your filename contains some weird caracters, you can use \texttt{?}, \texttt{*} or \textbackslash{}\textbackslash{}. For example, if your filename contains a \textbackslash{}, you can use \textbackslash{}\textbackslash{}\textbackslash{}\textbackslash{}. \begin{bVerbatim} * mark weird_file\\\\with-backslash \end{bVerbatim} \label{blb:database_restore} \section{Restoring When Things Go Wrong} \index[general]{Restoring When Things Go Wrong } \index[general]{Restoring Your Database} \index[general]{Database!Restoring} This and the following sections will try to present a few of the kinds of problems that can come up making restoring more difficult. We will try to provide a few ideas how to get out of these problem situations. In addition to what is presented here, there is more specific information on restoring a \bilink{Client}{blb:restore_client} and your \bilink{Server}{blb:restore_server} in the \bilink{Disaster Recovery Using \bacula{}}{blb:RescueChapter} chapter of this manual. \begin{bdescription} \item[Problem] My database is broken. \item[Solution] For either \mysql{} or \postgresql{}, see the vendor's documentation. They have specific tools that check and repair databases, see the \bilink{database repair}{blb:DatabaseRepair} sections of this manual for links to vendor information. Assuming the above does not resolve the problem, you will need to restore or rebuild your catalog. Note, if it is a matter of some inconsistencies in the \bacula{} tables rather than a broken database, then running the \bxlink{dbcheck}{blb:dbcheck}{utility}{command}\footnote{\butilityman{}} might help, but you will need to ensure that your database indexes are properly setup. Please see the \bilink{Database Performance Issues}{blb:DatabasePerformance} sections of this manual for more details. \item[Problem] How do I restore my catalog? \item[Solution with a Catalog backup] If you have backed up your database nightly (as you should) and you have made a bootstrap file, you can immediately load back your database (or the ASCII SQL output). Make a copy of your current database, then re-initialize it, by running the following scripts: \begin{bVerbatim} ./drop_bacula_tables ./make_bacula_tables \end{bVerbatim} After re-initializing the database, you should be able to run \bacula{}. If you now try to use the \bcommandname{restore} command, it will not work because the database will be empty. However, you can manually run a restore job and specify your bootstrap file. You do so by entering the \bcommandname{run} command in the console and selecting the restore job. If you are using the default \bfilename{bacula-dir.conf}, this Job will be named \textbf{RestoreFiles}. Most likely it will prompt you with something such as: \begin{bVerbatim} Run Restore job JobName: RestoreFiles Bootstrap: /home/kern/bacula/working/restore.bsr Where: /tmp/bacula-restores Replace: always FileSet: Full Set Client: rufus-fd Storage: File When: 2005-07-10 17:33:40 Catalog: MyCatalog Priority: 10 OK to run? (yes/mod/no): \end{bVerbatim} A number of the items will be different in your case. What you want to do is: to use the mod option to change the Bootstrap to point to your saved bootstrap file; and to make sure all the other items such as Client, Storage, Catalog, and Where are correct. The FileSet is not used when you specify a bootstrap file. Once you have set all the correct values, run the Job and it will restore the backup of your database, which is most likely an ASCII dump. You will then need to follow the instructions for your database type to recreate the database from the ASCII backup file. See the \bilink{Catalog Maintenance}{blb:CatMaintenanceChapter} chapter of this manual for examples of the command needed to restore a database from an ASCII dump (they are shown in the Compacting Your XXX Database sections). Also, please note that after you restore your database from an ASCII backup, you do NOT want to do a \btool{make\_bacula\_tables} command, or you will probably erase your newly restored database tables. \item[Solution with a Job listing] If you did save your database but did not make a bootstrap file, then recovering the database is more difficult. You will probably need to use bextract to extract the backup copy. First you should locate the listing of the job report from the last catalog backup. It has important information that will allow you to quickly find your database file. For example, in the job report for the CatalogBackup shown below, the critical items are the Volume name(s), the Volume Session Id and the Volume Session Time. If you know those, you can easily restore your Catalog. \begin{bVerbatim} 22-Apr 10:22 HeadMan: Start Backup JobId 7510, Job=CatalogBackup.2005-04-22_01.10.0 22-Apr 10:23 HeadMan: \bacula{} 1.37.14 (21Apr05): 22-Apr-2005 10:23:06 JobId: 7510 Job: CatalogBackup.2005-04-22_01.10.00 Backup Level: Full Client: Polymatou FileSet: "CatalogFile" 2003-04-10 01:24:01 Pool: "Default" Storage: "DLTDrive" Start time: 22-Apr-2005 10:21:00 End time: 22-Apr-2005 10:23:06 FD Files Written: 1 SD Files Written: 1 FD Bytes Written: 210,739,395 SD Bytes Written: 210,739,521 Rate: 1672.5 KB/s Software Compression: None Volume name(s): DLT-22Apr05 Volume Session Id: 11 Volume Session Time: 1114075126 Last Volume Bytes: 1,428,240,465 Non-fatal FD errors: 0 SD Errors: 0 FD termination status: OK SD termination status: OK Termination: Backup OK \end{bVerbatim} From the above information, you can manually create a bootstrap file, and then follow the instructions given above for restoring your database. A reconstructed bootstrap file for the above backup Job would look like the following: \begin{bVerbatim} Volume="DLT-22Apr05" VolSessionId=11 VolSessionTime=1114075126 FileIndex=1-1 \end{bVerbatim} Where we have inserted the Volume name, Volume Session Id, and Volume Session Time that correspond to the values in the job report. We've also used a FileIndex of one, which will always be the case providing that there was only one file backed up in the job. The disadvantage of this bootstrap file compared to what is created when you ask for one to be written, is that there is no File and Block specified, so the restore code must search all data in the Volume to find the requested file. A fully specified bootstrap file would have the File and Blocks specified as follows: \begin{bVerbatim} Volume="DLT-22Apr05" VolSessionId=11 VolSessionTime=1114075126 VolFile=118-118 VolBlock=0-4053 FileIndex=1-1 \end{bVerbatim} Once you have restored the ASCII dump of the database, you will then to follow the instructions for your database type to recreate the database from the ASCII backup file. See the \bilink{Catalog Maintenance}{blb:CatMaintenanceChapter} chapter of this manual for examples of the command needed to restore a database from an ASCII dump (they are shown in the Compacting Your XXX Database sections). Also, please note that after you restore your database from an ASCII backup, you do NOT want to do a \btool{make\_bacula\_tables} command, or you will probably erase your newly restored database tables. \item [Solution without a Job Listing] If you do not have a job listing, then it is a bit more difficult. Either you use the \bxlink{bscan}{blb:bscan}{utility}{program} to scan the contents of your tape into a database, which can be very time consuming depending on the size of the tape, or you can use the \bxlink{bls}{blb:bls}{utility}{program} to list everything on the tape, and reconstruct a bootstrap file from the bls listing for the file or files you want following the instructions given above. There is a specific example of how to use \btool{bls} below. \item [Problem] I try to restore the last known good full backup by specifying item 3 on the restore menu then the JobId to restore. \bacula{} then reports: \begin{bVerbatim} 1 Job 0 Files \end{bVerbatim} and restores nothing. \item[Solution] Most likely the File records were pruned from the database either due to the File Retention period expiring or by explicitly purging the Job. By using the \bcommandname{llist jobid=nn} command, you can obtain all the important information about the job: \begin{bVerbatim} llist jobid=120 JobId: 120 Job: save.2005-12-05_18.27.33 Job.Name: save PurgedFiles: 0 Type: B Level: F Job.ClientId: 1 Client.Name: Rufus JobStatus: T SchedTime: 2005-12-05 18:27:32 StartTime: 2005-12-05 18:27:35 EndTime: 2005-12-05 18:27:37 JobTDate: 1133803657 VolSessionId: 1 VolSessionTime: 1133803624 JobFiles: 236 JobErrors: 0 JobMissingFiles: 0 Job.PoolId: 4 Pool.Name: Full Job.FileSetId: 1 FileSet.FileSet: BackupSet \end{bVerbatim} Then you can find the Volume(s) used by doing: \begin{bVerbatim} sql select VolumeName from JobMedia,Media where JobId=1 and JobMedia.MediaId=Media.MediaId; \end{bVerbatim} Finally, you can create a bootstrap file as described in the previous problem above using this information. If you are using \bacula{} version 1.38.0 or greater, when you select item 3 from the menu and enter the JobId, it will ask you if you would like to restore all the files in the job, and it will collect the above information and write the bootstrap file for you. \item [Problem] You don't have a bootstrap file, and you don't have the Job report for the backup of your database, but you did backup the database, and you know the Volume to which it was backed up. \item [Solution] Either bscan the tape (see below for bscanning), or better use \btool{bls} to find where it is on the tape, then use \btool{bextract} to restore the database. For example, \begin{bVerbatim} ./bls -j -V DLT-22Apr05 /dev/nst0 \end{bVerbatim} Might produce the following output: \begin{bVerbatim} bls: butil.c:258 Using device: "/dev/nst0" for reading. 21-Jul 18:34 bls: Ready to read from volume "DLT-22Apr05" on device "DLTDrive" (/dev/nst0). Volume Record: File:blk=0:0 SessId=11 SessTime=1114075126 JobId=0 DataLen=164 ... Begin Job Session Record: File:blk=118:0 SessId=11 SessTime=1114075126 JobId=7510 Job=CatalogBackup.2005-04-22_01.10.0 Date=22-Apr-2005 10:21:00 Level=F Type=B End Job Session Record: File:blk=118:4053 SessId=11 SessTime=1114075126 JobId=7510 Date=22-Apr-2005 10:23:06 Level=F Type=B Files=1 Bytes=210,739,395 Errors=0 Status=T ... 21-Jul 18:34 bls: End of Volume at file 201 on device "DLTDrive" (/dev/nst0), Volume "DLT-22Apr05" 21-Jul 18:34 bls: End of all volumes. \end{bVerbatim} Of course, there will be many more records printed, but we have indicated the essential lines of output. From the information on the Begin Job and End Job Session Records, you can reconstruct a bootstrap file such as the one shown above. \item[Problem] How can I find where a file is stored. \item[Solution] Normally, it is not necessary, you just use the \bcommandname{restore} command to restore the most recently saved version (menu option 5), or a version saved before a given date (menu option 8). If you know the JobId of the job in which it was saved, you can use menu option 3 to enter that JobId. If you would like to know the JobId where a file was saved, select restore menu option 2. You can also use the \bcommandname{query} command to find information such as: \begin{bVerbatim} *query Available queries: 1: List up to 20 places where a File is saved regardless of the directory 2: List where the most recent copies of a file are saved 3: List last 20 Full Backups for a Client 4: List all backups for a Client after a specified time 5: List all backups for a Client 6: List Volume Attributes for a selected Volume 7: List Volumes used by selected JobId 8: List Volumes to Restore All Files 9: List Pool Attributes for a selected Pool 10: List total files/bytes by Job 11: List total files/bytes by Volume 12: List Files for a selected JobId 13: List Jobs stored on a selected MediaId 14: List Jobs stored for a given Volume name 15: List Volumes \bacula{} thinks are in changer 16: List Volumes likely to need replacement from age or errors Choose a query (1-16): \end{bVerbatim} \item[Problem] I didn't backup my database. What do I do now? \item[Solution] This is probably the worst of all cases, and you will probably have to re-create your database from scratch and then bscan in all your Volumes, which is a very long, painful, and inexact process. There are basically three steps to take: \begin{benumerate} \item Ensure that your SQL server is running (\mysql{} or \postgresql{}) and that the \bacula{} database (normally bacula) exists. See the \bilink{Installation}{blb:CreateDatabase} chapter of the manual. \item Ensure that the \bacula{} databases are created. This is also described at the above link. \item Start and stop the \bacula{} Director using the propriate \bfilename{bacula-dir.conf} file so that it can create the Client and Storage records which are not stored on the Volumes. Without these records, scanning is unable to connect the Job records to the proper client. \end{benumerate} When the above is complete, you can begin bscanning your Volumes. Please see the \bxlink{bscan}{blb:bscan}{utility}{section} of the \butilityman{}. \end{bdescription}