| 1 |
\chapter{Storage Daemon Design} |
| 2 |
\label{blb:developers:storage:daemon} |
| 3 |
\label{blb:TheChapterStart3} |
| 4 |
\index{Storage Daemon Design } |
| 5 |
\index{Design!Storage Daemon } |
| 6 |
|
| 7 |
This chapter is intended to be a technical discussion of the Storage daemon |
| 8 |
services and as such is not targeted at end users but rather at developers and |
| 9 |
system administrators that want or need to know more of the working details of |
| 10 |
\textbf{\bacula{}}. |
| 11 |
|
| 12 |
This document is somewhat out of date. |
| 13 |
|
| 14 |
\section{SD Design Introduction} |
| 15 |
\index{Introduction!SD Design } |
| 16 |
\index{SD Design Introduction } |
| 17 |
|
| 18 |
The \bacula{} Storage daemon provides storage resources to a \bacula{} installation. |
| 19 |
An individual Storage daemon is associated with a physical permanent storage |
| 20 |
device (for example, a tape drive, CD writer, tape changer or jukebox, etc.), |
| 21 |
and may employ auxiliary storage resources (such as space on a hard disk file |
| 22 |
system) to increase performance and/or optimize use of the permanent storage |
| 23 |
medium. |
| 24 |
|
| 25 |
Any number of storage daemons may be run on a given machine; each associated |
| 26 |
with an individual storage device connected to it, and BACULA operations may |
| 27 |
employ storage daemons on any number of hosts connected by a network, local or |
| 28 |
remote. The ability to employ remote storage daemons (with appropriate |
| 29 |
security measures) permits automatic off-site backup, possibly to publicly |
| 30 |
available backup repositories. |
| 31 |
|
| 32 |
\section{SD Development Outline} |
| 33 |
\index{Outline!SD Development } |
| 34 |
\index{SD Development Outline } |
| 35 |
|
| 36 |
In order to provide a high performance backup and restore solution that scales |
| 37 |
to very large capacity devices and networks, the storage daemon must be able |
| 38 |
to extract as much performance from the storage device and network with which |
| 39 |
it interacts. In order to accomplish this, storage daemons will eventually |
| 40 |
have to sacrifice simplicity and painless portability in favor of techniques |
| 41 |
which improve performance. My goal in designing the storage daemon protocol |
| 42 |
and developing the initial prototype storage daemon is to provide for these |
| 43 |
additions in the future, while implementing an initial storage daemon which is |
| 44 |
very simple and portable to almost any \acs{POSIX}-like environment. This original |
| 45 |
storage daemon (and its evolved descendants) can serve as a portable solution |
| 46 |
for non-demanding backup requirements (such as single servers of modest size, |
| 47 |
individual machines, or small local networks), while serving as the starting |
| 48 |
point for development of higher performance configurable derivatives which use |
| 49 |
techniques such as \acs{POSIX} threads, shared memory, asynchronous I/O, buffering |
| 50 |
to high-speed intermediate media, and support for tape changers and jukeboxes. |
| 51 |
|
| 52 |
|
| 53 |
\section{SD Connections and Sessions} |
| 54 |
\index{Sessions!SD Connections and } |
| 55 |
\index{SD Connections and Sessions } |
| 56 |
|
| 57 |
A client connects to a storage server by initiating a conventional TCP |
| 58 |
connection. The storage server accepts the connection unless its maximum |
| 59 |
number of connections has been reached or the specified host is not granted |
| 60 |
access to the storage server. Once a connection has been opened, the client |
| 61 |
may make any number of Query requests, and/or initiate (if permitted), one or |
| 62 |
more Append sessions (which transmit data to be stored by the storage daemon) |
| 63 |
and/or Read sessions (which retrieve data from the storage daemon). |
| 64 |
|
| 65 |
Most requests and replies sent across the connection are simple ASCII strings, |
| 66 |
with status replies prefixed by a four digit status code for easier parsing. |
| 67 |
Binary data appear in blocks stored and retrieved from the storage. Any |
| 68 |
request may result in a single-line status reply of |
| 69 |
``\texttt{3201 Notification pending}'', which indicates the client must send a ``Query notification'' |
| 70 |
request to retrieve one or more notifications posted to it. Once the |
| 71 |
notifications have been returned, the client may then resubmit the request |
| 72 |
which resulted in the 3201 status. |
| 73 |
|
| 74 |
The following descriptions omit common error codes, yet to be defined, which |
| 75 |
can occur from most or many requests due to events like media errors, |
| 76 |
restarting of the storage daemon, etc. These details will be filled in, along |
| 77 |
with a comprehensive list of status codes along with which requests can |
| 78 |
produce them in an update to this document. |
| 79 |
|
| 80 |
\subsection{SD Append Requests} |
| 81 |
\index{Requests!SD Append } |
| 82 |
\index{SD Append Requests } |
| 83 |
|
| 84 |
\begin{bdescription} |
| 85 |
|
| 86 |
\item [append open session = \bbracket{JobId} [ \bbracket{Password} ]] |
| 87 |
\index{SPAN class } |
| 88 |
A data append session is opened with the Job ID given by \emph{JobId} with |
| 89 |
client password (if required) given by \emph{Password}. If the session is |
| 90 |
successfully opened, a status of \texttt{3000 OK} is returned with a |
| 91 |
``\texttt{ticket = }\emph{number}'' reply used to identify subsequent messages in the |
| 92 |
session. If too many sessions are open, or a conflicting session (for |
| 93 |
example, a read in progress when simultaneous read and append sessions are |
| 94 |
not permitted), a status of ``\texttt{3502 Volume busy}'' is returned. If no |
| 95 |
volume is mounted, or the volume mounted cannot be appended to, a status of |
| 96 |
``\texttt{3503 Volume not mounted}'' is returned. |
| 97 |
|
| 98 |
\item [append data = \bbracket{ticket-number} ] |
| 99 |
\index{SPAN class } |
| 100 |
If the append data is accepted, a status of \texttt{3000 OK data address = \bbracket{IPaddress} port = \bbracket{port}} |
| 101 |
is returned, where the \texttt{IPaddress} and \texttt{port} specify the IP address and port number of the data |
| 102 |
channel. Error status codes are \texttt{3504 Invalid ticket number} and |
| 103 |
\texttt{3505 Session aborted}, the latter of which indicates the entire append |
| 104 |
session has failed due to a daemon or media error. |
| 105 |
|
| 106 |
Once the File daemon has established the connection to the data channel |
| 107 |
opened by the Storage daemon, it will transfer a header packet followed by |
| 108 |
any number of data packets. The header packet is of the form: |
| 109 |
|
| 110 |
\texttt{\bbracket{file-index} \bbracket{stream-id} \bbracket{info}} |
| 111 |
|
| 112 |
The details are specified in the |
| 113 |
\bilink{Daemon Protocol}{blb:TheChapterStart2} section of this |
| 114 |
document. |
| 115 |
|
| 116 |
\item [*append abort session = \bbracket{ticket-number} ] |
| 117 |
\index{SPAN class } |
| 118 |
The open append session with ticket \emph{ticket-number} is aborted; any blocks |
| 119 |
not yet written to permanent media are discarded. Subsequent attempts to |
| 120 |
append data to the session will receive an error status of \texttt{3505 Session aborted}. |
| 121 |
|
| 122 |
\item [append end session = \bbracket{ticket-number} ] |
| 123 |
\index{SPAN class } |
| 124 |
The open append session with ticket \emph{ticket-number} is marked complete; no |
| 125 |
further blocks may be appended. The storage daemon will give priority to |
| 126 |
saving any buffered blocks from this session to permanent media as soon as |
| 127 |
possible. |
| 128 |
|
| 129 |
\item [append close session = \bbracket{ticket-number} ] |
| 130 |
\index{SPAN class } |
| 131 |
The append session with ticket \emph{ticket} is closed. This message does not |
| 132 |
receive an \texttt{3000 OK} reply until all of the content of the session are |
| 133 |
stored on permanent media, at which time said reply is given, followed by a |
| 134 |
list of volumes, from first to last, which contain blocks from the session, |
| 135 |
along with the first and last file and block on each containing session data |
| 136 |
and the volume session key identifying data from that session in lines with |
| 137 |
the following format: |
| 138 |
|
| 139 |
\texttt{Volume = \bbracket{Volume-id} \bbracket{start-file} \bbracket{start-block} \bbracket{end-file} \bbracket{end-block} \bbracket{volume-session-id}} |
| 140 |
where \emph{Volume-id} is the volume label, \emph{start-file} and \emph{start-block} are the file and block containing the first |
| 141 |
data from that session on the volume, \emph{end-file} and \emph{end-block} are |
| 142 |
the file and block with the last data from the session on the volume and \emph{volume-session-id} |
| 143 |
is the volume session ID for blocks from the session |
| 144 |
stored on that volume. |
| 145 |
\end{bdescription} |
| 146 |
|
| 147 |
\subsection{SD Read Requests} |
| 148 |
\index{SD Read Requests } |
| 149 |
\index{Requests!SD Read } |
| 150 |
|
| 151 |
\begin{bdescription} |
| 152 |
|
| 153 |
\item [Read open session = \bbracket{JobId} \bbracket{Volume-id} \bbracket{start-file} \bbracket{start-block} \bbracket{end-file} \bbracket{end-block} \bbracket{volume-session-id} \bbracket{password} ] |
| 154 |
\index{SPAN class } |
| 155 |
where \emph{Volume-id} is the volume label, \emph{start-file} and \emph{start-block} are the file and block containing the first data from that |
| 156 |
session on the volume, \emph{end-file} and \emph{end-block} are the file and |
| 157 |
block with the last data from the session on the volume and \emph{volume-session-id} is the volume session ID for blocks from the session |
| 158 |
stored on that volume. |
| 159 |
|
| 160 |
If the session is successfully opened, a status of |
| 161 |
|
| 162 |
\texttt{3100 OK Ticket = \emph{number}} |
| 163 |
|
| 164 |
is returned with a reply used to identify subsequent messages in the session. |
| 165 |
If too many sessions are open, or a conflicting session (for example, an |
| 166 |
append in progress when simultaneous read and append sessions are not |
| 167 |
permitted), a status of ``\texttt{3502 Volume busy}'' is returned. If no |
| 168 |
volume is mounted, or the volume mounted cannot be appended to, a status of |
| 169 |
``\texttt{3503 Volume not mounted}'' is returned. If no block with the given |
| 170 |
volume session ID and the correct client ID number appears in the given first |
| 171 |
file and block for the volume, a status of ``\texttt{3505 Session not found}'' is returned. |
| 172 |
|
| 173 |
\item [Read data = \bbracket{Ticket} \gt{} \bbracket{Block} ] |
| 174 |
\index{SPAN class } |
| 175 |
The specified Block of data from open read session with the specified Ticket |
| 176 |
number is returned, with a status of \texttt{3000 OK} followed by a |
| 177 |
``\texttt{Length = }\emph{size}'' line giving the length in bytes of the block data |
| 178 |
which immediately follows. Blocks must be retrieved in ascending order, but |
| 179 |
blocks may be skipped. If a block number greater than the largest stored on |
| 180 |
the volume is requested, a status of ``\texttt{3201 End of volume}'' is |
| 181 |
returned. If a block number greater than the largest in the file is |
| 182 |
requested, a status of ``\texttt{3401 End of file}'' is returned. |
| 183 |
|
| 184 |
\item [Read close session = \bbracket{Ticket} ] |
| 185 |
\index{SPAN class } |
| 186 |
The read session with Ticket number is closed. A read session may be closed |
| 187 |
at any time; you needn't read all its blocks before closing it. |
| 188 |
\end{bdescription} |
| 189 |
|
| 190 |
\emph{by \bref{http://www.fourmilab.ch/}{John Walker} January 30th, MM } |
| 191 |
|
| 192 |
\section{SD Data Structures} |
| 193 |
\index{SD Data Structures} |
| 194 |
|
| 195 |
In the Storage daemon, there is a Device resource (i.e. from conf file) |
| 196 |
that describes each physical device. When the physical device is used it |
| 197 |
is controled by the \bstructure{DEVICE} (defined in \bfilename{dev.h}), and typically |
| 198 |
refered to as dev in the \texttt{C++} code. Anyone writing or reading a physical |
| 199 |
device must ultimately get a lock on the \bstructure{DEVICE} -- this controls |
| 200 |
the device. However, multiple Jobs (defined by a \bstructure{JCR} \bfilename{src/jcr.h}) |
| 201 |
can be writing a physical \bstructurename{DEVICE} at the same time (of course they are |
| 202 |
sequenced by locking the \bstructure{DEVICE}). There are a lot of job |
| 203 |
dependent ``device'' variables that may be different for each Job such as |
| 204 |
spooling (one job may spool and another may not, and when a job is |
| 205 |
spooling, it must have an i/o packet open, each job has its own record and |
| 206 |
block structures, \ldots{}), so there is a device control record or \bstructurename{DCR} that is |
| 207 |
the primary way of interfacing to the physical device. The \bstructurename{DCR} contains |
| 208 |
all the job specific data as well as a pointer to the Device resource |
| 209 |
(\bstructure{DEVRES}) and the physical \bstructure{DEVICE}. |
| 210 |
|
| 211 |
Now if a job is writing to two devices (it could be writing two separate |
| 212 |
streams to the same device), it must have two \bstructurename{DCR}s. Today, the code only |
| 213 |
permits one. This won't be hard to change, but it is new code. |
| 214 |
|
| 215 |
Today three jobs (threads), two physical devices each job |
| 216 |
writes to only one device: |
| 217 |
|
| 218 |
\begin{bVerbatim} |
| 219 |
Job1 -> DCR1 -> DEVICE1 |
| 220 |
Job2 -> DCR2 -> DEVICE1 |
| 221 |
Job3 -> DCR3 -> DEVICE2 |
| 222 |
\end{bVerbatim} |
| 223 |
|
| 224 |
To be implemented three jobs, three physical devices, but |
| 225 |
job1 is writing simultaneously to three devices: |
| 226 |
|
| 227 |
\begin{bVerbatim} |
| 228 |
Job1 -> DCR1 -> DEVICE1 |
| 229 |
-> DCR4 -> DEVICE2 |
| 230 |
-> DCR5 -> DEVICE3 |
| 231 |
Job2 -> DCR2 -> DEVICE1 |
| 232 |
Job3 -> DCR3 -> DEVICE2 |
| 233 |
|
| 234 |
Job = job control record |
| 235 |
DCR = Job contorl data for a specific device |
| 236 |
DEVICE = Device only control data |
| 237 |
\end{bVerbatim} |