| 1 |
.. _cedar-config: |
| 2 |
|
| 3 |
Configuration |
| 4 |
============= |
| 5 |
|
| 6 |
.. _cedar-config-overview: |
| 7 |
|
| 8 |
Overview |
| 9 |
-------- |
| 10 |
|
| 11 |
Configuring Cedar Backup is unfortunately somewhat complicated. The good |
| 12 |
news is that once you get through the initial configuration process, |
| 13 |
you'll hardly ever have to change anything. Even better, the most |
| 14 |
typical changes (i.e. adding and removing directories from a backup) are |
| 15 |
easy. |
| 16 |
|
| 17 |
First, familiarize yourself with the concepts in :doc:`basic`. |
| 18 |
In particular, be sure that you understand the differences between a |
| 19 |
master and a client. (If you only have one machine, then your machine |
| 20 |
will act as both a master and a client, and we'll refer to your setup as |
| 21 |
a pool of one.) Then, install Cedar Backup per the instructions in |
| 22 |
:doc:`install`. |
| 23 |
|
| 24 |
Once everything has been installed, you are ready to begin configuring Cedar |
| 25 |
Backup. Look over :doc:`commandline` to become familiar with the command line |
| 26 |
interface. Then, look over *Configuration File Format* (below) and create a |
| 27 |
configuration file for each peer in your backup pool. To start with, create a |
| 28 |
very simple configuration file, then expand it later. Decide now whether you |
| 29 |
will store the configuration file in the standard place (``/etc/cback3.conf``) |
| 30 |
or in some other location. |
| 31 |
|
| 32 |
After you have all of the configuration files in place, configure each |
| 33 |
of your machines, following the instructions in the appropriate section |
| 34 |
below (for master, client or pool of one). Since the master and |
| 35 |
client(s) must communicate over the network, you won't be able to fully |
| 36 |
configure the master without configuring each client and vice-versa. The |
| 37 |
instructions are clear on what needs to be done. |
| 38 |
|
| 39 |
Cedar Backup has been designed for use on all UNIX-like systems. |
| 40 |
However, since it was developed on a Debian GNU/Linux system, and |
| 41 |
because I am a Debian developer, the packaging is prettier and the setup |
| 42 |
is somewhat simpler on a Debian system than on a system where you |
| 43 |
install from source. |
| 44 |
|
| 45 |
The configuration instructions below have been generalized so they |
| 46 |
should work well regardless of what platform you are running (i.e. |
| 47 |
RedHat, Gentoo, FreeBSD, etc.). If instructions vary for a particular |
| 48 |
platform, you will find a note related to that platform. |
| 49 |
|
| 50 |
I am always open to adding more platform-specific hints and notes, so |
| 51 |
write me if you find problems with these instructions. |
| 52 |
|
| 53 |
.. _cedar-config-configfile: |
| 54 |
|
| 55 |
Configuration File Format |
| 56 |
------------------------- |
| 57 |
|
| 58 |
Cedar Backup is configured through an XML [1]_ configuration file, |
| 59 |
usually called ``/etc/cback3.conf``. The configuration file contains the |
| 60 |
following sections: reference, options, collect, stage, store, purge and |
| 61 |
extensions. |
| 62 |
|
| 63 |
All configuration files must contain the two general configuration |
| 64 |
sections, the reference section and the options section. Besides that, |
| 65 |
administrators need only configure actions they intend to use. For |
| 66 |
instance, on a client machine, administrators will generally only |
| 67 |
configure the collect and purge sections, while on a master machine they |
| 68 |
will have to configure all four action-related sections. [2]_ The |
| 69 |
extensions section is always optional and can be omitted unless |
| 70 |
extensions are in use. |
| 71 |
|
| 72 |
|note| |
| 73 |
|
| 74 |
Even though the Mac OS X (darwin) filesystem is *not* case-sensitive, |
| 75 |
Cedar Backup configuration *is* generally case-sensitive on that |
| 76 |
platform, just like on all other platforms. For instance, even though |
| 77 |
the files “Ken” and “ken” might be the same on the Mac OS X |
| 78 |
filesystem, an exclusion in Cedar Backup configuration for “ken” will |
| 79 |
only match the file if it is actually on the filesystem with a |
| 80 |
lower-case “k” as its first letter. This won't surprise the typical |
| 81 |
UNIX user, but might surprise someone who's gotten into the “Mac |
| 82 |
Mindset”. |
| 83 |
|
| 84 |
.. _cedar-config-configfile-sample: |
| 85 |
|
| 86 |
Sample Configuration File |
| 87 |
~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 88 |
|
| 89 |
Both the Python source distribution and the Debian package come with a |
| 90 |
sample configuration file. The Debian package includes its sample in |
| 91 |
``/usr/share/doc/cedar-backup3/examples/cback3.conf.sample``. |
| 92 |
|
| 93 |
This is a sample configuration file similar to the one provided in the |
| 94 |
source package. Documentation below provides more information about each |
| 95 |
of the individual configuration sections. |
| 96 |
|
| 97 |
:: |
| 98 |
|
| 99 |
<?xml version="1.0"?> |
| 100 |
<cb_config> |
| 101 |
<reference> |
| 102 |
<author>Kenneth J. Pronovici</author> |
| 103 |
<revision>1.3</revision> |
| 104 |
<description>Sample</description> |
| 105 |
</reference> |
| 106 |
<options> |
| 107 |
<starting_day>tuesday</starting_day> |
| 108 |
<working_dir>/opt/backup/tmp</working_dir> |
| 109 |
<backup_user>backup</backup_user> |
| 110 |
<backup_group>group</backup_group> |
| 111 |
<rcp_command>/usr/bin/scp -O -B</rcp_command> |
| 112 |
</options> |
| 113 |
<peers> |
| 114 |
<peer> |
| 115 |
<name>debian</name> |
| 116 |
<type>local</type> |
| 117 |
<collect_dir>/opt/backup/collect</collect_dir> |
| 118 |
</peer> |
| 119 |
</peers> |
| 120 |
<collect> |
| 121 |
<collect_dir>/opt/backup/collect</collect_dir> |
| 122 |
<collect_mode>daily</collect_mode> |
| 123 |
<archive_mode>targz</archive_mode> |
| 124 |
<ignore_file>.cbignore</ignore_file> |
| 125 |
<dir> |
| 126 |
<abs_path>/etc</abs_path> |
| 127 |
<collect_mode>incr</collect_mode> |
| 128 |
</dir> |
| 129 |
<file> |
| 130 |
<abs_path>/home/root/.profile</abs_path> |
| 131 |
<collect_mode>weekly</collect_mode> |
| 132 |
</file> |
| 133 |
</collect> |
| 134 |
<stage> |
| 135 |
<staging_dir>/opt/backup/staging</staging_dir> |
| 136 |
</stage> |
| 137 |
<store> |
| 138 |
<source_dir>/opt/backup/staging</source_dir> |
| 139 |
<media_type>cdrw-74</media_type> |
| 140 |
<device_type>cdwriter</device_type> |
| 141 |
<target_device>/dev/cdrw</target_device> |
| 142 |
<target_scsi_id>0,0,0</target_scsi_id> |
| 143 |
<drive_speed>4</drive_speed> |
| 144 |
<check_data>Y</check_data> |
| 145 |
<check_media>Y</check_media> |
| 146 |
<warn_midnite>Y</warn_midnite> |
| 147 |
</store> |
| 148 |
<purge> |
| 149 |
<dir> |
| 150 |
<abs_path>/opt/backup/stage</abs_path> |
| 151 |
<retain_days>7</retain_days> |
| 152 |
</dir> |
| 153 |
<dir> |
| 154 |
<abs_path>/opt/backup/collect</abs_path> |
| 155 |
<retain_days>0</retain_days> |
| 156 |
</dir> |
| 157 |
</purge> |
| 158 |
</cb_config> |
| 159 |
|
| 160 |
|
| 161 |
.. _cedar-config-configfile-reference: |
| 162 |
|
| 163 |
Reference Configuration |
| 164 |
~~~~~~~~~~~~~~~~~~~~~~~ |
| 165 |
|
| 166 |
The reference configuration section contains free-text elements that |
| 167 |
exist only for reference.. The section itself is required, but the |
| 168 |
individual elements may be left blank if desired. |
| 169 |
|
| 170 |
This is an example reference configuration section: |
| 171 |
|
| 172 |
:: |
| 173 |
|
| 174 |
<reference> |
| 175 |
<author>Kenneth J. Pronovici</author> |
| 176 |
<revision>Revision 1.3</revision> |
| 177 |
<description>Sample</description> |
| 178 |
<generator>Yet to be Written Config Tool (tm)</description> |
| 179 |
</reference> |
| 180 |
|
| 181 |
|
| 182 |
The following elements are part of the reference configuration section: |
| 183 |
|
| 184 |
``author`` |
| 185 |
Author of the configuration file. |
| 186 |
|
| 187 |
*Restrictions:* None |
| 188 |
|
| 189 |
``revision`` |
| 190 |
Revision of the configuration file. |
| 191 |
|
| 192 |
*Restrictions:* None |
| 193 |
|
| 194 |
``description`` |
| 195 |
Description of the configuration file. |
| 196 |
|
| 197 |
*Restrictions:* None |
| 198 |
|
| 199 |
``generator`` |
| 200 |
Tool that generated the configuration file, if any. |
| 201 |
|
| 202 |
*Restrictions:* None |
| 203 |
|
| 204 |
.. _cedar-config-configfile-options: |
| 205 |
|
| 206 |
Options Configuration |
| 207 |
~~~~~~~~~~~~~~~~~~~~~ |
| 208 |
|
| 209 |
The options configuration section contains configuration options that |
| 210 |
are not specific to any one action. |
| 211 |
|
| 212 |
This is an example options configuration section: |
| 213 |
|
| 214 |
:: |
| 215 |
|
| 216 |
<options> |
| 217 |
<starting_day>tuesday</starting_day> |
| 218 |
<working_dir>/opt/backup/tmp</working_dir> |
| 219 |
<backup_user>backup</backup_user> |
| 220 |
<backup_group>backup</backup_group> |
| 221 |
<rcp_command>/usr/bin/scp -O -B</rcp_command> |
| 222 |
<rsh_command>/usr/bin/ssh</rsh_command> |
| 223 |
<cback_command>/usr/bin/cback</cback_command> |
| 224 |
<managed_actions>collect, purge</managed_actions> |
| 225 |
<override> |
| 226 |
<command>cdrecord</command> |
| 227 |
<abs_path>/opt/local/bin/cdrecord</abs_path> |
| 228 |
</override> |
| 229 |
<override> |
| 230 |
<command>mkisofs</command> |
| 231 |
<abs_path>/opt/local/bin/mkisofs</abs_path> |
| 232 |
</override> |
| 233 |
<pre_action_hook> |
| 234 |
<action>collect</action> |
| 235 |
<command>echo "I AM A PRE-ACTION HOOK RELATED TO COLLECT"</command> |
| 236 |
</pre_action_hook> |
| 237 |
<post_action_hook> |
| 238 |
<action>collect</action> |
| 239 |
<command>echo "I AM A POST-ACTION HOOK RELATED TO COLLECT"</command> |
| 240 |
</post_action_hook> |
| 241 |
</options> |
| 242 |
|
| 243 |
|
| 244 |
The following elements are part of the options configuration section: |
| 245 |
|
| 246 |
``starting_day`` |
| 247 |
Day that starts the week. |
| 248 |
|
| 249 |
Cedar Backup is built around the idea of weekly backups. The starting |
| 250 |
day of week is the day that media will be rebuilt from scratch and |
| 251 |
that incremental backup information will be cleared. |
| 252 |
|
| 253 |
*Restrictions:* Must be a day of the week in English, i.e. |
| 254 |
``monday``, ``tuesday``, etc. The validation is case-sensitive. |
| 255 |
|
| 256 |
``working_dir`` |
| 257 |
Working (temporary) directory to use for backups. |
| 258 |
|
| 259 |
This directory is used for writing temporary files, such as tar file |
| 260 |
or ISO filesystem images as they are being built. It is also used to |
| 261 |
store day-to-day information about incremental backups. |
| 262 |
|
| 263 |
The working directory should contain enough free space to hold |
| 264 |
temporary tar files (on a client) or to build an ISO filesystem image |
| 265 |
(on a master). |
| 266 |
|
| 267 |
*Restrictions:* Must be an absolute path |
| 268 |
|
| 269 |
``backup_user`` |
| 270 |
Effective user that backups should run as. |
| 271 |
|
| 272 |
This user must exist on the machine which is being configured and |
| 273 |
should not be root (although that restriction is not enforced). |
| 274 |
|
| 275 |
This value is also used as the default remote backup user for remote |
| 276 |
peers. |
| 277 |
|
| 278 |
*Restrictions:* Must be non-empty |
| 279 |
|
| 280 |
``backup_group`` |
| 281 |
Effective group that backups should run as. |
| 282 |
|
| 283 |
This group must exist on the machine which is being configured, and |
| 284 |
should not be root or some other “powerful” group (although that |
| 285 |
restriction is not enforced). |
| 286 |
|
| 287 |
*Restrictions:* Must be non-empty |
| 288 |
|
| 289 |
``rcp_command`` |
| 290 |
Default rcp-compatible copy command for staging. |
| 291 |
|
| 292 |
The rcp command should be the exact command used for remote copies, |
| 293 |
including any required options. If you are using ``scp``, you should |
| 294 |
pass it the ``-B`` option, so ``scp`` will not ask for any user input |
| 295 |
(which could hang the backup). A common example is something like |
| 296 |
``/usr/bin/scp -B``. If you are following the instructions in the |
| 297 |
appendix to restrict SSH access, then use ``/usr/bin/scp -O -B`` |
| 298 |
instead. |
| 299 |
|
| 300 |
This value is used as the default value for all remote peers. |
| 301 |
Technically, this value is not needed by clients, but we require it |
| 302 |
for all config files anyway. |
| 303 |
|
| 304 |
*Restrictions:* Must be non-empty |
| 305 |
|
| 306 |
``rsh_command`` |
| 307 |
Default rsh-compatible command to use for remote shells. |
| 308 |
|
| 309 |
The rsh command should be the exact command used for remote shells, |
| 310 |
including any required options. |
| 311 |
|
| 312 |
This value is used as the default value for all managed clients. It |
| 313 |
is optional, because it is only used when executing actions on |
| 314 |
managed clients. However, each managed client must either be able to |
| 315 |
read the value from options configuration or must set the value |
| 316 |
explicitly. |
| 317 |
|
| 318 |
*Restrictions:* Must be non-empty |
| 319 |
|
| 320 |
``cback_command`` |
| 321 |
Default cback-compatible command to use on managed remote clients. |
| 322 |
|
| 323 |
The cback command should be the exact command used for for executing |
| 324 |
``cback`` on a remote managed client, including any required |
| 325 |
command-line options. Do *not* list any actions in the command line, |
| 326 |
and do *not* include the ``--full`` command-line option. |
| 327 |
|
| 328 |
This value is used as the default value for all managed clients. It |
| 329 |
is optional, because it is only used when executing actions on |
| 330 |
managed clients. However, each managed client must either be able to |
| 331 |
read the value from options configuration or must set the value |
| 332 |
explicitly. |
| 333 |
|
| 334 |
*Note:* if this command-line is complicated, it is often better to |
| 335 |
create a simple shell script on the remote host to encapsulate all of |
| 336 |
the options. Then, just reference the shell script in configuration. |
| 337 |
|
| 338 |
*Restrictions:* Must be non-empty |
| 339 |
|
| 340 |
``managed_actions`` |
| 341 |
Default set of actions that are managed on remote clients. |
| 342 |
|
| 343 |
This is a comma-separated list of actions that the master will manage |
| 344 |
on behalf of remote clients. Typically, it would include only |
| 345 |
collect-like actions and purge. |
| 346 |
|
| 347 |
This value is used as the default value for all managed clients. It |
| 348 |
is optional, because it is only used when executing actions on |
| 349 |
managed clients. However, each managed client must either be able to |
| 350 |
read the value from options configuration or must set the value |
| 351 |
explicitly. |
| 352 |
|
| 353 |
*Restrictions:* Must be non-empty. |
| 354 |
|
| 355 |
``override`` |
| 356 |
Command to override with a customized path. |
| 357 |
|
| 358 |
This is a subsection which contains a command to override with a |
| 359 |
customized path. This functionality would be used if root's ``$PATH`` |
| 360 |
does not include a particular required command, or if there is a need |
| 361 |
to use a version of a command that is different than the one listed |
| 362 |
on the ``$PATH``. Most users will only use this section when directed |
| 363 |
to, in order to fix a problem. |
| 364 |
|
| 365 |
This section is optional, and can be repeated as many times as |
| 366 |
necessary. |
| 367 |
|
| 368 |
This subsection must contain the following two fields: |
| 369 |
|
| 370 |
``command`` |
| 371 |
Name of the command to be overridden, i.e. “cdrecord”. |
| 372 |
|
| 373 |
*Restrictions:* Must be a non-empty string. |
| 374 |
|
| 375 |
``abs_path`` |
| 376 |
The absolute path where the overridden command can be found. |
| 377 |
|
| 378 |
*Restrictions:* Must be an absolute path. |
| 379 |
|
| 380 |
``pre_action_hook`` |
| 381 |
Hook configuring a command to be executed before an action. |
| 382 |
|
| 383 |
This is a subsection which configures a command to be executed |
| 384 |
immediately before a named action. It provides a way for |
| 385 |
administrators to associate their own custom functionality with |
| 386 |
standard Cedar Backup actions or with arbitrary extensions. |
| 387 |
|
| 388 |
This section is optional, and can be repeated as many times as |
| 389 |
necessary. |
| 390 |
|
| 391 |
This subsection must contain the following two fields: |
| 392 |
|
| 393 |
``action`` |
| 394 |
Name of the Cedar Backup action that the hook is associated with. |
| 395 |
The action can be a standard backup action (collect, stage, etc.) |
| 396 |
or can be an extension action. No validation is done to ensure |
| 397 |
that the configured action actually exists. |
| 398 |
|
| 399 |
*Restrictions:* Must be a non-empty string. |
| 400 |
|
| 401 |
``command`` |
| 402 |
Name of the command to be executed. This item can either specify |
| 403 |
the path to a shell script of some sort (the recommended approach) |
| 404 |
or can include a complete shell command. |
| 405 |
|
| 406 |
*Note:* if you choose to provide a complete shell command rather |
| 407 |
than the path to a script, you need to be aware of some |
| 408 |
limitations of Cedar Backup's command-line parser. You cannot use |
| 409 |
a subshell (via the :literal:`\`command\`` or ``$(command)`` |
| 410 |
syntaxes) or any shell variable in your command line. |
| 411 |
Additionally, the command-line parser only recognizes the |
| 412 |
double-quote character (``"``) to delimit groupings or strings on |
| 413 |
the command-line. The bottom line is, you are probably best off |
| 414 |
writing a shell script of some sort for anything more |
| 415 |
sophisticated than very simple shell commands. |
| 416 |
|
| 417 |
*Restrictions:* Must be a non-empty string. |
| 418 |
|
| 419 |
``post_action_hook`` |
| 420 |
Hook configuring a command to be executed after an action. |
| 421 |
|
| 422 |
This is a subsection which configures a command to be executed |
| 423 |
immediately after a named action. It provides a way for |
| 424 |
administrators to associate their own custom functionality with |
| 425 |
standard Cedar Backup actions or with arbitrary extensions. |
| 426 |
|
| 427 |
This section is optional, and can be repeated as many times as |
| 428 |
necessary. |
| 429 |
|
| 430 |
This subsection must contain the following two fields: |
| 431 |
|
| 432 |
``action`` |
| 433 |
Name of the Cedar Backup action that the hook is associated with. |
| 434 |
The action can be a standard backup action (collect, stage, etc.) |
| 435 |
or can be an extension action. No validation is done to ensure |
| 436 |
that the configured action actually exists. |
| 437 |
|
| 438 |
*Restrictions:* Must be a non-empty string. |
| 439 |
|
| 440 |
``command`` |
| 441 |
Name of the command to be executed. This item can either specify |
| 442 |
the path to a shell script of some sort (the recommended approach) |
| 443 |
or can include a complete shell command. |
| 444 |
|
| 445 |
*Note:* if you choose to provide a complete shell command rather |
| 446 |
than the path to a script, you need to be aware of some |
| 447 |
limitations of Cedar Backup's command-line parser. You cannot use |
| 448 |
a subshell (via the :literal:`\`command\`` or ``$(command)`` |
| 449 |
syntaxes) or any shell variable in your command line. |
| 450 |
Additionally, the command-line parser only recognizes the |
| 451 |
double-quote character (``"``) to delimit groupings or strings on |
| 452 |
the command-line. The bottom line is, you are probably best off |
| 453 |
writing a shell script of some sort for anything more |
| 454 |
sophisticated than very simple shell commands. |
| 455 |
|
| 456 |
*Restrictions:* Must be a non-empty string. |
| 457 |
|
| 458 |
.. _cedar-config-configfile-peers: |
| 459 |
|
| 460 |
Peers Configuration |
| 461 |
~~~~~~~~~~~~~~~~~~~ |
| 462 |
|
| 463 |
The peers configuration section contains a list of the peers managed by |
| 464 |
a master. This section is only required on a master. |
| 465 |
|
| 466 |
This is an example peers configuration section: |
| 467 |
|
| 468 |
:: |
| 469 |
|
| 470 |
<peers> |
| 471 |
<peer> |
| 472 |
<name>machine1</name> |
| 473 |
<type>local</type> |
| 474 |
<collect_dir>/opt/backup/collect</collect_dir> |
| 475 |
</peer> |
| 476 |
<peer> |
| 477 |
<name>machine2</name> |
| 478 |
<type>remote</type> |
| 479 |
<backup_user>backup</backup_user> |
| 480 |
<collect_dir>/opt/backup/collect</collect_dir> |
| 481 |
<ignore_failures>all</ignore_failures> |
| 482 |
</peer> |
| 483 |
<peer> |
| 484 |
<name>machine3</name> |
| 485 |
<type>remote</type> |
| 486 |
<managed>Y</managed> |
| 487 |
<backup_user>backup</backup_user> |
| 488 |
<collect_dir>/opt/backup/collect</collect_dir> |
| 489 |
<rcp_command>/usr/bin/scp</rcp_command> |
| 490 |
<rsh_command>/usr/bin/ssh</rsh_command> |
| 491 |
<cback_command>/usr/bin/cback</cback_command> |
| 492 |
<managed_actions>collect, purge</managed_actions> |
| 493 |
</peer> |
| 494 |
</peers> |
| 495 |
|
| 496 |
|
| 497 |
The following elements are part of the peers configuration section: |
| 498 |
|
| 499 |
``peer`` (local version) |
| 500 |
Local client peer in a backup pool. |
| 501 |
|
| 502 |
This is a subsection which contains information about a specific |
| 503 |
local client peer managed by a master. |
| 504 |
|
| 505 |
This section can be repeated as many times as is necessary. At least |
| 506 |
one remote or local peer must be configured. |
| 507 |
|
| 508 |
The local peer subsection must contain the following fields: |
| 509 |
|
| 510 |
``name`` |
| 511 |
Name of the peer, typically a valid hostname. |
| 512 |
|
| 513 |
For local peers, this value is only used for reference. However, |
| 514 |
it is good practice to list the peer's hostname here, for |
| 515 |
consistency with remote peers. |
| 516 |
|
| 517 |
*Restrictions:* Must be non-empty, and unique among all peers. |
| 518 |
|
| 519 |
``type`` |
| 520 |
Type of this peer. |
| 521 |
|
| 522 |
This value identifies the type of the peer. For a local peer, it |
| 523 |
must always be ``local``. |
| 524 |
|
| 525 |
*Restrictions:* Must be ``local``. |
| 526 |
|
| 527 |
``collect_dir`` |
| 528 |
Collect directory to stage from for this peer. |
| 529 |
|
| 530 |
The master will copy all files in this directory into the |
| 531 |
appropriate staging directory. Since this is a local peer, the |
| 532 |
directory is assumed to be reachable via normal filesystem |
| 533 |
operations (i.e. ``cp``). |
| 534 |
|
| 535 |
*Restrictions:* Must be an absolute path. |
| 536 |
|
| 537 |
``ignore_failures`` |
| 538 |
Ignore failure mode for this peer |
| 539 |
|
| 540 |
The ignore failure mode indicates whether “not ready to be staged” |
| 541 |
errors should be ignored for this peer. This option is intended to |
| 542 |
be used for peers that are up only intermittently, to cut down on |
| 543 |
the number of error emails received by the Cedar Backup |
| 544 |
administrator. |
| 545 |
|
| 546 |
The "none" mode means that all errors will be reported. This is |
| 547 |
the default behavior. The "all" mode means to ignore all failures. |
| 548 |
The "weekly" mode means to ignore failures for a start-of-week or |
| 549 |
full backup. The "daily" mode means to ignore failures for any |
| 550 |
backup that is not either a full backup or a start-of-week backup. |
| 551 |
|
| 552 |
*Restrictions:* If set, must be one of "none", "all", "daily", or |
| 553 |
"weekly". |
| 554 |
|
| 555 |
``peer`` (remote version) |
| 556 |
Remote client peer in a backup pool. |
| 557 |
|
| 558 |
This is a subsection which contains information about a specific |
| 559 |
remote client peer managed by a master. A remote peer is one which |
| 560 |
can be reached via an rsh-based network call. |
| 561 |
|
| 562 |
This section can be repeated as many times as is necessary. At least |
| 563 |
one remote or local peer must be configured. |
| 564 |
|
| 565 |
The remote peer subsection must contain the following fields: |
| 566 |
|
| 567 |
``name`` |
| 568 |
Hostname of the peer. |
| 569 |
|
| 570 |
For remote peers, this must be a valid DNS hostname or IP address |
| 571 |
which can be resolved during an rsh-based network call. |
| 572 |
|
| 573 |
*Restrictions:* Must be non-empty, and unique among all peers. |
| 574 |
|
| 575 |
``type`` |
| 576 |
Type of this peer. |
| 577 |
|
| 578 |
This value identifies the type of the peer. For a remote peer, it |
| 579 |
must always be ``remote``. |
| 580 |
|
| 581 |
*Restrictions:* Must be ``remote``. |
| 582 |
|
| 583 |
``managed`` |
| 584 |
Indicates whether this peer is managed. |
| 585 |
|
| 586 |
A managed peer (or managed client) is a peer for which the master |
| 587 |
manages all of the backup activites via a remote shell. |
| 588 |
|
| 589 |
This field is optional. If it doesn't exist, then ``N`` will be |
| 590 |
assumed. |
| 591 |
|
| 592 |
*Restrictions:* Must be a boolean (``Y`` or ``N``). |
| 593 |
|
| 594 |
``collect_dir`` |
| 595 |
Collect directory to stage from for this peer. |
| 596 |
|
| 597 |
The master will copy all files in this directory into the |
| 598 |
appropriate staging directory. Since this is a remote peer, the |
| 599 |
directory is assumed to be reachable via rsh-based network |
| 600 |
operations (i.e. ``scp`` or the configured rcp command). |
| 601 |
|
| 602 |
*Restrictions:* Must be an absolute path. |
| 603 |
|
| 604 |
``ignore_failures`` |
| 605 |
Ignore failure mode for this peer |
| 606 |
|
| 607 |
The ignore failure mode indicates whether “not ready to be staged” |
| 608 |
errors should be ignored for this peer. This option is intended to |
| 609 |
be used for peers that are up only intermittently, to cut down on |
| 610 |
the number of error emails received by the Cedar Backup |
| 611 |
administrator. |
| 612 |
|
| 613 |
The "none" mode means that all errors will be reported. This is |
| 614 |
the default behavior. The "all" mode means to ignore all failures. |
| 615 |
The "weekly" mode means to ignore failures for a start-of-week or |
| 616 |
full backup. The "daily" mode means to ignore failures for any |
| 617 |
backup that is not either a full backup or a start-of-week backup. |
| 618 |
|
| 619 |
*Restrictions:* If set, must be one of "none", "all", "daily", or |
| 620 |
"weekly". |
| 621 |
|
| 622 |
``backup_user`` |
| 623 |
Name of backup user on the remote peer. |
| 624 |
|
| 625 |
This username will be used when copying files from the remote peer |
| 626 |
via an rsh-based network connection. |
| 627 |
|
| 628 |
This field is optional. if it doesn't exist, the backup will use |
| 629 |
the default backup user from the options section. |
| 630 |
|
| 631 |
*Restrictions:* Must be non-empty. |
| 632 |
|
| 633 |
``rcp_command`` |
| 634 |
The rcp-compatible copy command for this peer. |
| 635 |
|
| 636 |
The rcp command should be the exact command used for remote |
| 637 |
copies, including any required options. If you are using ``scp``, |
| 638 |
you should pass it the ``-B`` option, so ``scp`` will not ask for |
| 639 |
any user input (which could hang the backup). A common example is |
| 640 |
something like ``/usr/bin/scp -B``. If you are following the |
| 641 |
instructions in the appendix to restrict SSH access, then use |
| 642 |
``/usr/bin/scp -O -B`` instead. |
| 643 |
|
| 644 |
This field is optional. if it doesn't exist, the backup will use |
| 645 |
the default rcp command from the options section. |
| 646 |
|
| 647 |
*Restrictions:* Must be non-empty. |
| 648 |
|
| 649 |
``rsh_command`` |
| 650 |
The rsh-compatible command for this peer. |
| 651 |
|
| 652 |
The rsh command should be the exact command used for remote |
| 653 |
shells, including any required options. |
| 654 |
|
| 655 |
This value only applies if the peer is managed. |
| 656 |
|
| 657 |
This field is optional. if it doesn't exist, the backup will use |
| 658 |
the default rsh command from the options section. |
| 659 |
|
| 660 |
*Restrictions:* Must be non-empty |
| 661 |
|
| 662 |
``cback_command`` |
| 663 |
The cback-compatible command for this peer. |
| 664 |
|
| 665 |
The cback command should be the exact command used for for |
| 666 |
executing cback on the peer as part of a managed backup. This |
| 667 |
value must include any required command-line options. Do *not* |
| 668 |
list any actions in the command line, and do *not* include the |
| 669 |
``--full`` command-line option. |
| 670 |
|
| 671 |
This value only applies if the peer is managed. |
| 672 |
|
| 673 |
This field is optional. if it doesn't exist, the backup will use |
| 674 |
the default cback command from the options section. |
| 675 |
|
| 676 |
*Note:* if this command-line is complicated, it is often better to |
| 677 |
create a simple shell script on the remote host to encapsulate all |
| 678 |
of the options. Then, just reference the shell script in |
| 679 |
configuration. |
| 680 |
|
| 681 |
*Restrictions:* Must be non-empty |
| 682 |
|
| 683 |
``managed_actions`` |
| 684 |
Set of actions that are managed for this peer. |
| 685 |
|
| 686 |
This is a comma-separated list of actions that the master will |
| 687 |
manage on behalf this peer. Typically, it would include only |
| 688 |
collect-like actions and purge. |
| 689 |
|
| 690 |
This value only applies if the peer is managed. |
| 691 |
|
| 692 |
This field is optional. if it doesn't exist, the backup will use |
| 693 |
the default list of managed actions from the options section. |
| 694 |
|
| 695 |
*Restrictions:* Must be non-empty. |
| 696 |
|
| 697 |
.. _cedar-config-configfile-collect: |
| 698 |
|
| 699 |
Collect Configuration |
| 700 |
~~~~~~~~~~~~~~~~~~~~~ |
| 701 |
|
| 702 |
The collect configuration section contains configuration options related |
| 703 |
the the collect action. This section contains a variable number of |
| 704 |
elements, including an optional exclusion section and a repeating |
| 705 |
subsection used to specify which directories and/or files to collect. |
| 706 |
You can also configure an ignore indicator file, which lets users mark |
| 707 |
their own directories as not backed up. |
| 708 |
|
| 709 |
Sometimes, it's not very convenient to list directories one by one in |
| 710 |
the Cedar Backup configuration file. For instance, when backing up your |
| 711 |
home directory, you often exclude as many directories as you include. |
| 712 |
The ignore file mechanism can be of some help, but it still isn't very |
| 713 |
convenient if there are a lot of directories to ignore (or if new |
| 714 |
directories pop up all of the time). |
| 715 |
|
| 716 |
In this situation, one option is to use a link farm rather than listing |
| 717 |
all of the directories in configuration. A link farm is a directory that |
| 718 |
contains nothing but a set of soft links to other files and directories. |
| 719 |
Normally, Cedar Backup does not follow soft links, but you can override |
| 720 |
this behavior for individual directories using the ``link_depth`` and |
| 721 |
``dereference`` options (see below). |
| 722 |
|
| 723 |
When using a link farm, you still have to deal with each backed-up |
| 724 |
directory individually, but you don't have to modify configuration. Some |
| 725 |
users find that this works better for them. |
| 726 |
|
| 727 |
In order to actually execute the collect action, you must have |
| 728 |
configured at least one collect directory or one collect file. However, |
| 729 |
if you are only including collect configuration for use by an extension, |
| 730 |
then it's OK to leave out these sections. The validation will take place |
| 731 |
only when the collect action is executed. |
| 732 |
|
| 733 |
This is an example collect configuration section: |
| 734 |
|
| 735 |
:: |
| 736 |
|
| 737 |
<collect> |
| 738 |
<collect_dir>/opt/backup/collect</collect_dir> |
| 739 |
<collect_mode>daily</collect_mode> |
| 740 |
<archive_mode>targz</archive_mode> |
| 741 |
<ignore_file>.cbignore</ignore_file> |
| 742 |
<exclude> |
| 743 |
<abs_path>/etc</abs_path> |
| 744 |
<pattern>.*\.conf</pattern> |
| 745 |
</exclude> |
| 746 |
<file> |
| 747 |
<abs_path>/home/root/.profile</abs_path> |
| 748 |
</file> |
| 749 |
<dir> |
| 750 |
<abs_path>/etc</abs_path> |
| 751 |
</dir> |
| 752 |
<dir> |
| 753 |
<abs_path>/var/log</abs_path> |
| 754 |
<collect_mode>incr</collect_mode> |
| 755 |
</dir> |
| 756 |
<dir> |
| 757 |
<abs_path>/opt</abs_path> |
| 758 |
<collect_mode>weekly</collect_mode> |
| 759 |
<exclude> |
| 760 |
<abs_path>/opt/large</abs_path> |
| 761 |
<rel_path>backup</rel_path> |
| 762 |
<pattern>.*tmp</pattern> |
| 763 |
</exclude> |
| 764 |
</dir> |
| 765 |
</collect> |
| 766 |
|
| 767 |
|
| 768 |
The following elements are part of the collect configuration section: |
| 769 |
|
| 770 |
``collect_dir`` |
| 771 |
Directory to collect files into. |
| 772 |
|
| 773 |
On a client, this is the directory which tarfiles for individual |
| 774 |
collect directories are written into. The master then stages files |
| 775 |
from this directory into its own staging directory. |
| 776 |
|
| 777 |
This field is always required. It must contain enough free space to |
| 778 |
collect all of the backed-up files on the machine in a compressed |
| 779 |
form. |
| 780 |
|
| 781 |
*Restrictions:* Must be an absolute path |
| 782 |
|
| 783 |
``collect_mode`` |
| 784 |
Default collect mode. |
| 785 |
|
| 786 |
The collect mode describes how frequently a directory is backed up. |
| 787 |
See :doc:`basic` for more information. |
| 788 |
|
| 789 |
This value is the collect mode that will be used by default during |
| 790 |
the collect process. Individual collect directories (below) may |
| 791 |
override this value. If *all* individual directories provide their |
| 792 |
own value, then this default value may be omitted from configuration. |
| 793 |
|
| 794 |
*Note:* if your backup device does not suppport multisession discs, |
| 795 |
then you should probably use the ``daily`` collect mode to avoid |
| 796 |
losing data. |
| 797 |
|
| 798 |
*Restrictions:* Must be one of ``daily``, ``weekly`` or ``incr``. |
| 799 |
|
| 800 |
``archive_mode`` |
| 801 |
Default archive mode for collect files. |
| 802 |
|
| 803 |
The archive mode maps to the way that a backup file is stored. A |
| 804 |
value ``tar`` means just a tarfile (``file.tar``); a value ``targz`` |
| 805 |
means a gzipped tarfile (``file.tar.gz``); and a value ``tarbz2`` |
| 806 |
means a bzipped tarfile (``file.tar.bz2``) |
| 807 |
|
| 808 |
This value is the archive mode that will be used by default during |
| 809 |
the collect process. Individual collect directories (below) may |
| 810 |
override this value. If *all* individual directories provide their |
| 811 |
own value, then this default value may be omitted from configuration. |
| 812 |
|
| 813 |
*Restrictions:* Must be one of ``tar``, ``targz`` or ``tarbz2``. |
| 814 |
|
| 815 |
``ignore_file`` |
| 816 |
Default ignore file name. |
| 817 |
|
| 818 |
The ignore file is an indicator file. If it exists in a given |
| 819 |
directory, then that directory will be recursively excluded from the |
| 820 |
backup as if it were explicitly excluded in configuration. |
| 821 |
|
| 822 |
The ignore file provides a way for individual users (who might not |
| 823 |
have access to Cedar Backup configuration) to control which of their |
| 824 |
own directories get backed up. For instance, users with a ``~/tmp`` |
| 825 |
directory might not want it backed up. If they create an ignore file |
| 826 |
in their directory (e.g. ``~/tmp/.cbignore``), then Cedar Backup will |
| 827 |
ignore it. |
| 828 |
|
| 829 |
This value is the ignore file name that will be used by default |
| 830 |
during the collect process. Individual collect directories (below) |
| 831 |
may override this value. If *all* individual directories provide |
| 832 |
their own value, then this default value may be omitted from |
| 833 |
configuration. |
| 834 |
|
| 835 |
*Restrictions:* Must be non-empty |
| 836 |
|
| 837 |
``recursion_level`` |
| 838 |
Recursion level to use when collecting directories. |
| 839 |
|
| 840 |
This is an integer value that Cedar Backup will consider when |
| 841 |
generating archive files for a configured collect directory. |
| 842 |
|
| 843 |
Normally, Cedar Backup generates one archive file per collect |
| 844 |
directory. So, if you collect ``/etc`` you get ``etc.tar.gz``. Most |
| 845 |
of the time, this is what you want. However, you may sometimes wish |
| 846 |
to generate multiple archive files for a single collect directory. |
| 847 |
|
| 848 |
The most obvious example is for ``/home``. By default, Cedar Backup |
| 849 |
will generate ``home.tar.gz``. If instead, you want one archive file |
| 850 |
per home directory you can set a recursion level of ``1``. Cedar |
| 851 |
Backup will generate ``home-user1.tar.gz``, ``home-user2.tar.gz``, |
| 852 |
etc. |
| 853 |
|
| 854 |
Higher recursion levels (``2``, ``3``, etc.) are legal, and it |
| 855 |
doesn't matter if the configured recursion level is deeper than the |
| 856 |
directory tree that is being collected. You can use a negative |
| 857 |
recursion level (like ``-1``) to specify an infinite level of |
| 858 |
recursion. This will exhaust the tree in the same way as if the |
| 859 |
recursion level is set too high. |
| 860 |
|
| 861 |
This field is optional. if it doesn't exist, the backup will use the |
| 862 |
default recursion level of zero. |
| 863 |
|
| 864 |
*Restrictions:* Must be an integer. |
| 865 |
|
| 866 |
``exclude`` |
| 867 |
List of paths or patterns to exclude from the backup. |
| 868 |
|
| 869 |
This is a subsection which contains a set of absolute paths and |
| 870 |
patterns to be excluded across all configured directories. For a |
| 871 |
given directory, the set of absolute paths and patterns to exclude is |
| 872 |
built from this list and any list that exists on the directory |
| 873 |
itself. Directories *cannot* override or remove entries that are in |
| 874 |
this list, however. |
| 875 |
|
| 876 |
This section is optional, and if it exists can also be empty. |
| 877 |
|
| 878 |
The exclude subsection can contain one or more of each of the |
| 879 |
following fields: |
| 880 |
|
| 881 |
``abs_path`` |
| 882 |
An absolute path to be recursively excluded from the backup. |
| 883 |
|
| 884 |
If a directory is excluded, then all of its children are also |
| 885 |
recursively excluded. For instance, a value ``/var/log/apache`` |
| 886 |
would exclude any files within ``/var/log/apache`` as well as |
| 887 |
files within other directories under ``/var/log/apache``. |
| 888 |
|
| 889 |
This field can be repeated as many times as is necessary. |
| 890 |
|
| 891 |
*Restrictions:* Must be an absolute path. |
| 892 |
|
| 893 |
``pattern`` |
| 894 |
A pattern to be recursively excluded from the backup. |
| 895 |
|
| 896 |
The pattern must be a Python regular expression. [3]_ It is |
| 897 |
assumed to be bounded at front and back by the beginning and end |
| 898 |
of the string (i.e. it is treated as if it begins with ``^`` and |
| 899 |
ends with ``$``). |
| 900 |
|
| 901 |
If the pattern causes a directory to be excluded, then all of the |
| 902 |
children of that directory are also recursively excluded. For |
| 903 |
instance, a value ``.*apache.*`` might match the |
| 904 |
``/var/log/apache`` directory. This would exclude any files within |
| 905 |
``/var/log/apache`` as well as files within other directories |
| 906 |
under ``/var/log/apache``. |
| 907 |
|
| 908 |
This field can be repeated as many times as is necessary. |
| 909 |
|
| 910 |
*Restrictions:* Must be non-empty |
| 911 |
|
| 912 |
``file`` |
| 913 |
A file to be collected. |
| 914 |
|
| 915 |
This is a subsection which contains information about a specific file |
| 916 |
to be collected (backed up). |
| 917 |
|
| 918 |
This section can be repeated as many times as is necessary. At least |
| 919 |
one collect directory or collect file must be configured when the |
| 920 |
collect action is executed. |
| 921 |
|
| 922 |
The collect file subsection contains the following fields: |
| 923 |
|
| 924 |
``abs_path`` |
| 925 |
Absolute path of the file to collect. |
| 926 |
|
| 927 |
*Restrictions:* Must be an absolute path. |
| 928 |
|
| 929 |
``collect_mode`` |
| 930 |
Collect mode for this file |
| 931 |
|
| 932 |
The collect mode describes how frequently a file is backed up. See |
| 933 |
:doc:`basic` for more information. |
| 934 |
|
| 935 |
This field is optional. If it doesn't exist, the backup will use |
| 936 |
the default collect mode. |
| 937 |
|
| 938 |
*Note:* if your backup device does not suppport multisession discs, |
| 939 |
then you should probably confine yourself to the ``daily`` collect |
| 940 |
mode, to avoid losing data. |
| 941 |
|
| 942 |
*Restrictions:* Must be one of ``daily``, ``weekly`` or ``incr``. |
| 943 |
|
| 944 |
``archive_mode`` |
| 945 |
Archive mode for this file. |
| 946 |
|
| 947 |
The archive mode maps to the way that a backup file is stored. A |
| 948 |
value ``tar`` means just a tarfile (``file.tar``); a value |
| 949 |
``targz`` means a gzipped tarfile (``file.tar.gz``); and a value |
| 950 |
``tarbz2`` means a bzipped tarfile (``file.tar.bz2``) |
| 951 |
|
| 952 |
This field is optional. if it doesn't exist, the backup will use |
| 953 |
the default archive mode. |
| 954 |
|
| 955 |
*Restrictions:* Must be one of ``tar``, ``targz`` or ``tarbz2``. |
| 956 |
|
| 957 |
``dir`` |
| 958 |
A directory to be collected. |
| 959 |
|
| 960 |
This is a subsection which contains information about a specific |
| 961 |
directory to be collected (backed up). |
| 962 |
|
| 963 |
This section can be repeated as many times as is necessary. At least |
| 964 |
one collect directory or collect file must be configured when the |
| 965 |
collect action is executed. |
| 966 |
|
| 967 |
The collect directory subsection contains the following fields: |
| 968 |
|
| 969 |
``abs_path`` |
| 970 |
Absolute path of the directory to collect. |
| 971 |
|
| 972 |
The path may be either a directory, a soft link to a directory, or |
| 973 |
a hard link to a directory. All three are treated the same at this |
| 974 |
level. |
| 975 |
|
| 976 |
The contents of the directory will be recursively collected. The |
| 977 |
backup will contain all of the files in the directory, as well as |
| 978 |
the contents of all of the subdirectories within the directory, |
| 979 |
etc. |
| 980 |
|
| 981 |
Soft links *within* the directory are treated as files, i.e. they |
| 982 |
are copied verbatim (as a link) and their contents are not backed |
| 983 |
up. |
| 984 |
|
| 985 |
*Restrictions:* Must be an absolute path. |
| 986 |
|
| 987 |
``collect_mode`` |
| 988 |
Collect mode for this directory |
| 989 |
|
| 990 |
The collect mode describes how frequently a directory is backed |
| 991 |
up. See :doc:`basic` for more information. |
| 992 |
|
| 993 |
This field is optional. If it doesn't exist, the backup will use |
| 994 |
the default collect mode. |
| 995 |
|
| 996 |
*Note:* if your backup device does not suppport multisession discs, |
| 997 |
then you should probably confine yourself to the ``daily`` collect |
| 998 |
mode, to avoid losing data. |
| 999 |
|
| 1000 |
*Restrictions:* Must be one of ``daily``, ``weekly`` or ``incr``. |
| 1001 |
|
| 1002 |
``archive_mode`` |
| 1003 |
Archive mode for this directory. |
| 1004 |
|
| 1005 |
The archive mode maps to the way that a backup file is stored. A |
| 1006 |
value ``tar`` means just a tarfile (``file.tar``); a value |
| 1007 |
``targz`` means a gzipped tarfile (``file.tar.gz``); and a value |
| 1008 |
``tarbz2`` means a bzipped tarfile (``file.tar.bz2``) |
| 1009 |
|
| 1010 |
This field is optional. if it doesn't exist, the backup will use |
| 1011 |
the default archive mode. |
| 1012 |
|
| 1013 |
*Restrictions:* Must be one of ``tar``, ``targz`` or ``tarbz2``. |
| 1014 |
|
| 1015 |
``ignore_file`` |
| 1016 |
Ignore file name for this directory. |
| 1017 |
|
| 1018 |
The ignore file is an indicator file. If it exists in a given |
| 1019 |
directory, then that directory will be recursively excluded from |
| 1020 |
the backup as if it were explicitly excluded in configuration. |
| 1021 |
|
| 1022 |
The ignore file provides a way for individual users (who might not |
| 1023 |
have access to Cedar Backup configuration) to control which of |
| 1024 |
their own directories get backed up. For instance, users with a |
| 1025 |
``~/tmp`` directory might not want it backed up. If they create an |
| 1026 |
ignore file in their directory (e.g. ``~/tmp/.cbignore``), then |
| 1027 |
Cedar Backup will ignore it. |
| 1028 |
|
| 1029 |
This field is optional. If it doesn't exist, the backup will use |
| 1030 |
the default ignore file name. |
| 1031 |
|
| 1032 |
*Restrictions:* Must be non-empty |
| 1033 |
|
| 1034 |
``link_depth`` |
| 1035 |
Link depth value to use for this directory. |
| 1036 |
|
| 1037 |
The link depth is maximum depth of the tree at which soft links |
| 1038 |
should be followed. So, a depth of 0 does not follow any soft |
| 1039 |
links within the collect directory, a depth of 1 follows only |
| 1040 |
links immediately within the collect directory, a depth of 2 |
| 1041 |
follows the links at the next level down, etc. |
| 1042 |
|
| 1043 |
This field is optional. If it doesn't exist, the backup will |
| 1044 |
assume a value of zero, meaning that soft links within the collect |
| 1045 |
directory will never be followed. |
| 1046 |
|
| 1047 |
*Restrictions:* If set, must be an integer GE 0. |
| 1048 |
|
| 1049 |
``dereference`` |
| 1050 |
Whether to dereference soft links. |
| 1051 |
|
| 1052 |
If this flag is set, links that are being followed will be |
| 1053 |
dereferenced before being added to the backup. The link will be |
| 1054 |
added (as a link), and then the directory or file that the link |
| 1055 |
points at will be added as well. |
| 1056 |
|
| 1057 |
This value only applies to a directory where soft links are being |
| 1058 |
followed (per the ``link_depth`` configuration option). It never |
| 1059 |
applies to a configured collect directory itself, only to other |
| 1060 |
directories within the collect directory. |
| 1061 |
|
| 1062 |
This field is optional. If it doesn't exist, the backup will |
| 1063 |
assume that links should never be dereferenced. |
| 1064 |
|
| 1065 |
*Restrictions:* Must be a boolean (``Y`` or ``N``). |
| 1066 |
|
| 1067 |
``exclude`` |
| 1068 |
List of paths or patterns to exclude from the backup. |
| 1069 |
|
| 1070 |
This is a subsection which contains a set of paths and patterns to |
| 1071 |
be excluded within this collect directory. This list is combined |
| 1072 |
with the program-wide list to build a complete list for the |
| 1073 |
directory. |
| 1074 |
|
| 1075 |
This section is entirely optional, and if it exists can also be |
| 1076 |
empty. |
| 1077 |
|
| 1078 |
The exclude subsection can contain one or more of each of the |
| 1079 |
following fields: |
| 1080 |
|
| 1081 |
``abs_path`` |
| 1082 |
An absolute path to be recursively excluded from the backup. |
| 1083 |
|
| 1084 |
If a directory is excluded, then all of its children are also |
| 1085 |
recursively excluded. For instance, a value ``/var/log/apache`` |
| 1086 |
would exclude any files within ``/var/log/apache`` as well as |
| 1087 |
files within other directories under ``/var/log/apache``. |
| 1088 |
|
| 1089 |
This field can be repeated as many times as is necessary. |
| 1090 |
|
| 1091 |
*Restrictions:* Must be an absolute path. |
| 1092 |
|
| 1093 |
``rel_path`` |
| 1094 |
A relative path to be recursively excluded from the backup. |
| 1095 |
|
| 1096 |
The path is assumed to be relative to the collect directory |
| 1097 |
itself. For instance, if the configured directory is |
| 1098 |
``/opt/web`` a configured relative path of ``something/else`` |
| 1099 |
would exclude the path ``/opt/web/something/else``. |
| 1100 |
|
| 1101 |
If a directory is excluded, then all of its children are also |
| 1102 |
recursively excluded. For instance, a value ``something/else`` |
| 1103 |
would exclude any files within ``something/else`` as well as |
| 1104 |
files within other directories under ``something/else``. |
| 1105 |
|
| 1106 |
This field can be repeated as many times as is necessary. |
| 1107 |
|
| 1108 |
*Restrictions:* Must be non-empty. |
| 1109 |
|
| 1110 |
``pattern`` |
| 1111 |
A pattern to be excluded from the backup. |
| 1112 |
|
| 1113 |
The pattern must be a Python regular expression. It is assumed |
| 1114 |
to be bounded at front and back by the beginning and end of the |
| 1115 |
string (i.e. it is treated as if it begins with ``^`` and ends |
| 1116 |
with ``$``). |
| 1117 |
|
| 1118 |
If the pattern causes a directory to be excluded, then all of |
| 1119 |
the children of that directory are also recursively excluded. |
| 1120 |
For instance, a value ``.*apache.*`` might match the |
| 1121 |
``/var/log/apache`` directory. This would exclude any files |
| 1122 |
within ``/var/log/apache`` as well as files within other |
| 1123 |
directories under ``/var/log/apache``. |
| 1124 |
|
| 1125 |
This field can be repeated as many times as is necessary. |
| 1126 |
|
| 1127 |
*Restrictions:* Must be non-empty |
| 1128 |
|
| 1129 |
.. _cedar-config-configfile-stage: |
| 1130 |
|
| 1131 |
Stage Configuration |
| 1132 |
~~~~~~~~~~~~~~~~~~~ |
| 1133 |
|
| 1134 |
The stage configuration section contains configuration options related |
| 1135 |
the the stage action. The section indicates where date from peers can be |
| 1136 |
staged to. |
| 1137 |
|
| 1138 |
This section can also (optionally) override the list of peers so that |
| 1139 |
not all peers are staged. If you provide *any* peers in this section, |
| 1140 |
then the list of peers here completely replaces the list of peers in the |
| 1141 |
peers configuration section for the purposes of staging. |
| 1142 |
|
| 1143 |
This is an example stage configuration section for the simple case where |
| 1144 |
the list of peers is taken from peers configuration: |
| 1145 |
|
| 1146 |
:: |
| 1147 |
|
| 1148 |
<stage> |
| 1149 |
<staging_dir>/opt/backup/stage</staging_dir> |
| 1150 |
</stage> |
| 1151 |
|
| 1152 |
|
| 1153 |
This is an example stage configuration section that overrides the |
| 1154 |
default list of peers: |
| 1155 |
|
| 1156 |
:: |
| 1157 |
|
| 1158 |
<stage> |
| 1159 |
<staging_dir>/opt/backup/stage</staging_dir> |
| 1160 |
<peer> |
| 1161 |
<name>machine1</name> |
| 1162 |
<type>local</type> |
| 1163 |
<collect_dir>/opt/backup/collect</collect_dir> |
| 1164 |
</peer> |
| 1165 |
<peer> |
| 1166 |
<name>machine2</name> |
| 1167 |
<type>remote</type> |
| 1168 |
<backup_user>backup</backup_user> |
| 1169 |
<collect_dir>/opt/backup/collect</collect_dir> |
| 1170 |
</peer> |
| 1171 |
</stage> |
| 1172 |
|
| 1173 |
|
| 1174 |
The following elements are part of the stage configuration section: |
| 1175 |
|
| 1176 |
``staging_dir`` |
| 1177 |
Directory to stage files into. |
| 1178 |
|
| 1179 |
This is the directory into which the master stages collected data |
| 1180 |
from each of the clients. Within the staging directory, data is |
| 1181 |
staged into date-based directories by peer name. For instance, peer |
| 1182 |
“daystrom” backed up on 19 Feb 2005 would be staged into something |
| 1183 |
like ``2005/02/19/daystrom`` relative to the staging directory |
| 1184 |
itself. |
| 1185 |
|
| 1186 |
This field is always required. The directory must contain enough free |
| 1187 |
space to stage all of the files collected from all of the various |
| 1188 |
machines in a backup pool. Many administrators set up purging to keep |
| 1189 |
staging directories around for a week or more, which requires even |
| 1190 |
more space. |
| 1191 |
|
| 1192 |
*Restrictions:* Must be an absolute path |
| 1193 |
|
| 1194 |
``peer`` (local version) |
| 1195 |
Local client peer in a backup pool. |
| 1196 |
|
| 1197 |
This is a subsection which contains information about a specific |
| 1198 |
local client peer to be staged (backed up). A local peer is one whose |
| 1199 |
collect directory can be reached without requiring any rsh-based |
| 1200 |
network calls. It is possible that a remote peer might be staged as a |
| 1201 |
local peer if its collect directory is mounted to the master via NFS, |
| 1202 |
AFS or some other method. |
| 1203 |
|
| 1204 |
This section can be repeated as many times as is necessary. At least |
| 1205 |
one remote or local peer must be configured. |
| 1206 |
|
| 1207 |
*Remember*, if you provide *any* local or remote peer in staging |
| 1208 |
configuration, the global peer configuration is completely replaced |
| 1209 |
by the staging peer configuration. |
| 1210 |
|
| 1211 |
The local peer subsection must contain the following fields: |
| 1212 |
|
| 1213 |
``name`` |
| 1214 |
Name of the peer, typically a valid hostname. |
| 1215 |
|
| 1216 |
For local peers, this value is only used for reference. However, |
| 1217 |
it is good practice to list the peer's hostname here, for |
| 1218 |
consistency with remote peers. |
| 1219 |
|
| 1220 |
*Restrictions:* Must be non-empty, and unique among all peers. |
| 1221 |
|
| 1222 |
``type`` |
| 1223 |
Type of this peer. |
| 1224 |
|
| 1225 |
This value identifies the type of the peer. For a local peer, it |
| 1226 |
must always be ``local``. |
| 1227 |
|
| 1228 |
*Restrictions:* Must be ``local``. |
| 1229 |
|
| 1230 |
``collect_dir`` |
| 1231 |
Collect directory to stage from for this peer. |
| 1232 |
|
| 1233 |
The master will copy all files in this directory into the |
| 1234 |
appropriate staging directory. Since this is a local peer, the |
| 1235 |
directory is assumed to be reachable via normal filesystem |
| 1236 |
operations (i.e. ``cp``). |
| 1237 |
|
| 1238 |
*Restrictions:* Must be an absolute path. |
| 1239 |
|
| 1240 |
``peer`` (remote version) |
| 1241 |
Remote client peer in a backup pool. |
| 1242 |
|
| 1243 |
This is a subsection which contains information about a specific |
| 1244 |
remote client peer to be staged (backed up). A remote peer is one |
| 1245 |
whose collect directory can only be reached via an rsh-based network |
| 1246 |
call. |
| 1247 |
|
| 1248 |
This section can be repeated as many times as is necessary. At least |
| 1249 |
one remote or local peer must be configured. |
| 1250 |
|
| 1251 |
*Remember*, if you provide *any* local or remote peer in staging |
| 1252 |
configuration, the global peer configuration is completely replaced |
| 1253 |
by the staging peer configuration. |
| 1254 |
|
| 1255 |
The remote peer subsection must contain the following fields: |
| 1256 |
|
| 1257 |
``name`` |
| 1258 |
Hostname of the peer. |
| 1259 |
|
| 1260 |
For remote peers, this must be a valid DNS hostname or IP address |
| 1261 |
which can be resolved during an rsh-based network call. |
| 1262 |
|
| 1263 |
*Restrictions:* Must be non-empty, and unique among all peers. |
| 1264 |
|
| 1265 |
``type`` |
| 1266 |
Type of this peer. |
| 1267 |
|
| 1268 |
This value identifies the type of the peer. For a remote peer, it |
| 1269 |
must always be ``remote``. |
| 1270 |
|
| 1271 |
*Restrictions:* Must be ``remote``. |
| 1272 |
|
| 1273 |
``collect_dir`` |
| 1274 |
Collect directory to stage from for this peer. |
| 1275 |
|
| 1276 |
The master will copy all files in this directory into the |
| 1277 |
appropriate staging directory. Since this is a remote peer, the |
| 1278 |
directory is assumed to be reachable via rsh-based network |
| 1279 |
operations (i.e. ``scp`` or the configured rcp command). |
| 1280 |
|
| 1281 |
*Restrictions:* Must be an absolute path. |
| 1282 |
|
| 1283 |
``backup_user`` |
| 1284 |
Name of backup user on the remote peer. |
| 1285 |
|
| 1286 |
This username will be used when copying files from the remote peer |
| 1287 |
via an rsh-based network connection. |
| 1288 |
|
| 1289 |
This field is optional. if it doesn't exist, the backup will use |
| 1290 |
the default backup user from the options section. |
| 1291 |
|
| 1292 |
*Restrictions:* Must be non-empty. |
| 1293 |
|
| 1294 |
``rcp_command`` |
| 1295 |
The rcp-compatible copy command for this peer. |
| 1296 |
|
| 1297 |
The rcp command should be the exact command used for remote |
| 1298 |
copies, including any required options. If you are using ``scp``, |
| 1299 |
you should pass it the ``-B`` option, so ``scp`` will not ask for |
| 1300 |
any user input (which could hang the backup). A common example is |
| 1301 |
something like ``/usr/bin/scp -B``. If you are following the |
| 1302 |
instructions in the appendix to restrict SSH access, then use |
| 1303 |
``/usr/bin/scp -O -B`` instead. |
| 1304 |
|
| 1305 |
This field is optional. if it doesn't exist, the backup will use |
| 1306 |
the default rcp command from the options section. |
| 1307 |
|
| 1308 |
*Restrictions:* Must be non-empty. |
| 1309 |
|
| 1310 |
.. _cedar-config-configfile-store: |
| 1311 |
|
| 1312 |
Store Configuration |
| 1313 |
~~~~~~~~~~~~~~~~~~~ |
| 1314 |
|
| 1315 |
The store configuration section contains configuration options related |
| 1316 |
the the store action. This section contains several optional fields. |
| 1317 |
Most fields control the way media is written using the writer device. |
| 1318 |
|
| 1319 |
This is an example store configuration section: |
| 1320 |
|
| 1321 |
:: |
| 1322 |
|
| 1323 |
<store> |
| 1324 |
<source_dir>/opt/backup/stage</source_dir> |
| 1325 |
<media_type>cdrw-74</media_type> |
| 1326 |
<device_type>cdwriter</device_type> |
| 1327 |
<target_device>/dev/cdrw</target_device> |
| 1328 |
<target_scsi_id>0,0,0</target_scsi_id> |
| 1329 |
<drive_speed>4</drive_speed> |
| 1330 |
<check_data>Y</check_data> |
| 1331 |
<check_media>Y</check_media> |
| 1332 |
<warn_midnite>Y</warn_midnite> |
| 1333 |
<no_eject>N</no_eject> |
| 1334 |
<refresh_media_delay>15</refresh_media_delay> |
| 1335 |
<eject_delay>2</eject_delay> |
| 1336 |
<blank_behavior> |
| 1337 |
<mode>weekly</mode> |
| 1338 |
<factor>1.3</factor> |
| 1339 |
</blank_behavior> |
| 1340 |
</store> |
| 1341 |
|
| 1342 |
|
| 1343 |
The following elements are part of the store configuration section: |
| 1344 |
|
| 1345 |
``source_dir`` |
| 1346 |
Directory whose contents should be written to media. |
| 1347 |
|
| 1348 |
This directory *must* be a Cedar Backup staging directory, as |
| 1349 |
configured in the staging configuration section. Only certain data |
| 1350 |
from that directory (typically, data from the current day) will be |
| 1351 |
written to disc. |
| 1352 |
|
| 1353 |
*Restrictions:* Must be an absolute path |
| 1354 |
|
| 1355 |
``device_type`` |
| 1356 |
Type of the device used to write the media. |
| 1357 |
|
| 1358 |
This field controls which type of writer device will be used by Cedar |
| 1359 |
Backup. Currently, Cedar Backup supports CD writers (``cdwriter``) |
| 1360 |
and DVD writers (``dvdwriter``). |
| 1361 |
|
| 1362 |
This field is optional. If it doesn't exist, the ``cdwriter`` device |
| 1363 |
type is assumed. |
| 1364 |
|
| 1365 |
*Restrictions:* If set, must be either ``cdwriter`` or ``dvdwriter``. |
| 1366 |
|
| 1367 |
``media_type`` |
| 1368 |
Type of the media in the device. |
| 1369 |
|
| 1370 |
Unless you want to throw away a backup disc every week, you are |
| 1371 |
probably best off using rewritable media. |
| 1372 |
|
| 1373 |
You must choose a media type that is appropriate for the device type |
| 1374 |
you chose above. For more information on media types, see |
| 1375 |
:doc:`basic`. |
| 1376 |
|
| 1377 |
*Restrictions:* Must be one of ``cdr-74``, ``cdrw-74``, ``cdr-80`` or |
| 1378 |
``cdrw-80`` if device type is ``cdwriter``; or one of ``dvd+r`` or |
| 1379 |
``dvd+rw`` if device type is ``dvdwriter``. |
| 1380 |
|
| 1381 |
``target_device`` |
| 1382 |
Filesystem device name for writer device. |
| 1383 |
|
| 1384 |
This value is required for both CD writers and DVD writers. |
| 1385 |
|
| 1386 |
This is the UNIX device name for the writer drive, for instance |
| 1387 |
``/dev/scd0`` or a symlink like ``/dev/cdrw``. |
| 1388 |
|
| 1389 |
In some cases, this device name is used to directly write to media. |
| 1390 |
This is true all of the time for DVD writers, and is true for CD |
| 1391 |
writers when a SCSI id (see below) has not been specified. |
| 1392 |
|
| 1393 |
Besides this, the device name is also needed in order to do several |
| 1394 |
pre-write checks (such as whether the device might already be |
| 1395 |
mounted) as well as the post-write consistency check, if enabled. |
| 1396 |
|
| 1397 |
*Note:* some users have reported intermittent problems when using a |
| 1398 |
symlink as the target device on Linux, especially with DVD media. If |
| 1399 |
you experience problems, try using the real device name rather than |
| 1400 |
the symlink. |
| 1401 |
|
| 1402 |
*Restrictions:* Must be an absolute path. |
| 1403 |
|
| 1404 |
``target_scsi_id`` |
| 1405 |
SCSI id for the writer device. |
| 1406 |
|
| 1407 |
This value is optional for CD writers and is ignored for DVD writers. |
| 1408 |
|
| 1409 |
If you have configured your CD writer hardware to work through the |
| 1410 |
normal filesystem device path, then you can leave this parameter |
| 1411 |
unset. Cedar Backup will just use the target device (above) when |
| 1412 |
talking to ``cdrecord``. |
| 1413 |
|
| 1414 |
Otherwise, if you have SCSI CD writer hardware or you have configured |
| 1415 |
your non-SCSI hardware to operate like a SCSI device, then you need |
| 1416 |
to provide Cedar Backup with a SCSI id it can use when talking with |
| 1417 |
``cdrecord``. |
| 1418 |
|
| 1419 |
For the purposes of Cedar Backup, a valid SCSI identifier must either |
| 1420 |
be in the standard SCSI identifier form ``scsibus,target,lun`` or in |
| 1421 |
the specialized-method form ``<method>:scsibus,target,lun``. |
| 1422 |
|
| 1423 |
An example of a standard SCSI identifier is ``1,6,2``. Today, the two |
| 1424 |
most common examples of the specialized-method form are |
| 1425 |
``ATA:scsibus,target,lun`` and ``ATAPI:scsibus,target,lun``, but you |
| 1426 |
may occassionally see other values (like ``OLDATAPI`` in some forks |
| 1427 |
of ``cdrecord``). |
| 1428 |
|
| 1429 |
See *Configuring your Writer Device* for more information on writer devices |
| 1430 |
and how they are configured. |
| 1431 |
|
| 1432 |
*Restrictions:* If set, must be a valid SCSI identifier. |
| 1433 |
|
| 1434 |
``drive_speed`` |
| 1435 |
Speed of the drive, i.e. ``2`` for a 2x device. |
| 1436 |
|
| 1437 |
This field is optional. If it doesn't exist, the underlying |
| 1438 |
device-related functionality will use the default drive speed. |
| 1439 |
|
| 1440 |
For DVD writers, it is best to leave this value unset, so |
| 1441 |
``growisofs`` can pick an appropriate speed. For CD writers, since |
| 1442 |
media can be speed-sensitive, it is probably best to set a sensible |
| 1443 |
value based on your specific writer and media. |
| 1444 |
|
| 1445 |
*Restrictions:* If set, must be an integer GE 1. |
| 1446 |
|
| 1447 |
``check_data`` |
| 1448 |
Whether the media should be validated. |
| 1449 |
|
| 1450 |
This field indicates whether a resulting image on the media should be |
| 1451 |
validated after the write completes, by running a consistency check |
| 1452 |
against it. If this check is enabled, the contents of the staging |
| 1453 |
directory are directly compared to the media, and an error is |
| 1454 |
reported if there is a mismatch. |
| 1455 |
|
| 1456 |
Practice shows that some drives can encounter an error when writing a |
| 1457 |
multisession disc, but not report any problems. This consistency |
| 1458 |
check allows us to catch the problem. By default, the consistency |
| 1459 |
check is disabled, but most users should choose to enable it unless |
| 1460 |
they have a good reason not to. |
| 1461 |
|
| 1462 |
This field is optional. If it doesn't exist, then ``N`` will be |
| 1463 |
assumed. |
| 1464 |
|
| 1465 |
*Restrictions:* Must be a boolean (``Y`` or ``N``). |
| 1466 |
|
| 1467 |
``check_media`` |
| 1468 |
Whether the media should be checked before writing to it. |
| 1469 |
|
| 1470 |
By default, Cedar Backup does not check its media before writing to |
| 1471 |
it. It will write to any media in the backup device. If you set this |
| 1472 |
flag to Y, Cedar Backup will make sure that the media has been |
| 1473 |
initialized before writing to it. (Rewritable media is initialized |
| 1474 |
using the initialize action.) |
| 1475 |
|
| 1476 |
If the configured media is not rewritable (like CD-R), then this |
| 1477 |
behavior is modified slightly. For this kind of media, the check |
| 1478 |
passes either if the media has been initialized *or* if the media |
| 1479 |
appears unused. |
| 1480 |
|
| 1481 |
This field is optional. If it doesn't exist, then ``N`` will be |
| 1482 |
assumed. |
| 1483 |
|
| 1484 |
*Restrictions:* Must be a boolean (``Y`` or ``N``). |
| 1485 |
|
| 1486 |
``warn_midnite`` |
| 1487 |
Whether to generate warnings for crossing midnite. |
| 1488 |
|
| 1489 |
This field indicates whether warnings should be generated if the |
| 1490 |
store operation has to cross a midnite boundary in order to find data |
| 1491 |
to write to disc. For instance, a warning would be generated if valid |
| 1492 |
store data was only found in the day before or day after the current |
| 1493 |
day. |
| 1494 |
|
| 1495 |
Configuration for some users is such that the store operation will |
| 1496 |
always cross a midnite boundary, so they will not care about this |
| 1497 |
warning. Other users will expect to never cross a boundary, and want |
| 1498 |
to be notified that something “strange” might have happened. |
| 1499 |
|
| 1500 |
This field is optional. If it doesn't exist, then ``N`` will be |
| 1501 |
assumed. |
| 1502 |
|
| 1503 |
*Restrictions:* Must be a boolean (``Y`` or ``N``). |
| 1504 |
|
| 1505 |
``no_eject`` |
| 1506 |
Indicates that the writer device should not be ejected. |
| 1507 |
|
| 1508 |
Under some circumstances, Cedar Backup ejects (opens and closes) the |
| 1509 |
writer device. This is done because some writer devices need to |
| 1510 |
re-load the media before noticing a media state change (like a new |
| 1511 |
session). |
| 1512 |
|
| 1513 |
For most writer devices this is safe, because they have a tray that |
| 1514 |
can be opened and closed. If your writer device does not have a tray |
| 1515 |
*and* Cedar Backup does not properly detect this, then set this flag. |
| 1516 |
Cedar Backup will not ever issue an eject command to your writer. |
| 1517 |
|
| 1518 |
*Note:* this could cause problems with your backup. For instance, with |
| 1519 |
many writers, the check data step may fail if the media is not |
| 1520 |
reloaded first. If this happens to you, you may need to get a |
| 1521 |
different writer device. |
| 1522 |
|
| 1523 |
This field is optional. If it doesn't exist, then ``N`` will be |
| 1524 |
assumed. |
| 1525 |
|
| 1526 |
*Restrictions:* Must be a boolean (``Y`` or ``N``). |
| 1527 |
|
| 1528 |
``refresh_media_delay`` |
| 1529 |
Number of seconds to delay after refreshing media |
| 1530 |
|
| 1531 |
This field is optional. If it doesn't exist, no delay will occur. |
| 1532 |
|
| 1533 |
Some devices seem to take a little while to stablize after refreshing |
| 1534 |
the media (i.e. closing and opening the tray). During this period, |
| 1535 |
operations on the media may fail. If your device behaves like this, |
| 1536 |
you can try setting a delay of 10-15 seconds. |
| 1537 |
|
| 1538 |
*Restrictions:* If set, must be an integer GE 1. |
| 1539 |
|
| 1540 |
``eject_delay`` |
| 1541 |
Number of seconds to delay after ejecting the tray |
| 1542 |
|
| 1543 |
This field is optional. If it doesn't exist, no delay will occur. |
| 1544 |
|
| 1545 |
If your system seems to have problems opening and closing the tray, |
| 1546 |
one possibility is that the open/close sequence is happening too |
| 1547 |
quickly --- either the tray isn't fully open when Cedar Backup |
| 1548 |
tries to close it, or it doesn't report being open. To work around |
| 1549 |
that problem, set an eject delay of a few seconds. |
| 1550 |
|
| 1551 |
*Restrictions:* If set, must be an integer GE 1. |
| 1552 |
|
| 1553 |
``blank_behavior`` |
| 1554 |
Optimized blanking strategy. |
| 1555 |
|
| 1556 |
For more information about Cedar Backup's optimized blanking |
| 1557 |
strategy, see :doc:`config`. |
| 1558 |
|
| 1559 |
This entire configuration section is optional. However, if you choose |
| 1560 |
to provide it, you must configure both a blanking mode and a blanking |
| 1561 |
factor. |
| 1562 |
|
| 1563 |
``blank_mode`` |
| 1564 |
Blanking mode. |
| 1565 |
|
| 1566 |
*Restrictions:*\ Must be one of "daily" or "weekly". |
| 1567 |
|
| 1568 |
``blank_factor`` |
| 1569 |
Blanking factor. |
| 1570 |
|
| 1571 |
*Restrictions:*\ Must be a floating point number GE 0. |
| 1572 |
|
| 1573 |
.. _cedar-config-configfile-purge: |
| 1574 |
|
| 1575 |
Purge Configuration |
| 1576 |
~~~~~~~~~~~~~~~~~~~ |
| 1577 |
|
| 1578 |
The purge configuration section contains configuration options related |
| 1579 |
the the purge action. This section contains a set of directories to be |
| 1580 |
purged, along with information about the schedule at which they should |
| 1581 |
be purged. |
| 1582 |
|
| 1583 |
Typically, Cedar Backup should be configured to purge collect |
| 1584 |
directories daily (retain days of ``0``). |
| 1585 |
|
| 1586 |
If you are tight on space, staging directories can also be purged daily. |
| 1587 |
However, if you have space to spare, you should consider purging about |
| 1588 |
once per week. That way, if your backup media is damaged, you will be |
| 1589 |
able to recreate the week's backup using the rebuild action. |
| 1590 |
|
| 1591 |
You should also purge the working directory periodically, once every few |
| 1592 |
weeks or once per month. This way, if any unneeded files are left |
| 1593 |
around, perhaps because a backup was interrupted or because |
| 1594 |
configuration changed, they will eventually be removed. *The working |
| 1595 |
directory should not be purged any more frequently than once per week, |
| 1596 |
otherwise you will risk destroying data used for incremental backups.* |
| 1597 |
|
| 1598 |
This is an example purge configuration section: |
| 1599 |
|
| 1600 |
:: |
| 1601 |
|
| 1602 |
<purge> |
| 1603 |
<dir> |
| 1604 |
<abs_path>/opt/backup/stage</abs_path> |
| 1605 |
<retain_days>7</retain_days> |
| 1606 |
</dir> |
| 1607 |
<dir> |
| 1608 |
<abs_path>/opt/backup/collect</abs_path> |
| 1609 |
<retain_days>0</retain_days> |
| 1610 |
</dir> |
| 1611 |
</purge> |
| 1612 |
|
| 1613 |
|
| 1614 |
The following elements are part of the purge configuration section: |
| 1615 |
|
| 1616 |
``dir`` |
| 1617 |
A directory to purge within. |
| 1618 |
|
| 1619 |
This is a subsection which contains information about a specific |
| 1620 |
directory to purge within. |
| 1621 |
|
| 1622 |
This section can be repeated as many times as is necessary. At least |
| 1623 |
one purge directory must be configured. |
| 1624 |
|
| 1625 |
The purge directory subsection contains the following fields: |
| 1626 |
|
| 1627 |
``abs_path`` |
| 1628 |
Absolute path of the directory to purge within. |
| 1629 |
|
| 1630 |
The contents of the directory will be purged based on age. The |
| 1631 |
purge will remove any files that were last modified more than |
| 1632 |
“retain days” days ago. Empty directories will also eventually be |
| 1633 |
removed. The purge directory itself will never be removed. |
| 1634 |
|
| 1635 |
The path may be either a directory, a soft link to a directory, or |
| 1636 |
a hard link to a directory. Soft links *within* the directory (if |
| 1637 |
any) are treated as files. |
| 1638 |
|
| 1639 |
*Restrictions:* Must be an absolute path. |
| 1640 |
|
| 1641 |
``retain_days`` |
| 1642 |
Number of days to retain old files. |
| 1643 |
|
| 1644 |
Once it has been more than this many days since a file was last |
| 1645 |
modified, it is a candidate for removal. |
| 1646 |
|
| 1647 |
*Restrictions:* Must be an integer GE 0. |
| 1648 |
|
| 1649 |
.. _cedar-config-configfile-extensions: |
| 1650 |
|
| 1651 |
Extensions Configuration |
| 1652 |
~~~~~~~~~~~~~~~~~~~~~~~~ |
| 1653 |
|
| 1654 |
The extensions configuration section is used to configure third-party |
| 1655 |
extensions to Cedar Backup. If you don't intend to use any extensions, |
| 1656 |
or don't know what extensions are, then you can safely leave this |
| 1657 |
section out of your configuration file. It is optional. |
| 1658 |
|
| 1659 |
Extensions configuration is used to specify “extended actions” |
| 1660 |
implemented by code external to Cedar Backup. An administrator can use |
| 1661 |
this section to map command-line Cedar Backup actions to third-party |
| 1662 |
extension functions. |
| 1663 |
|
| 1664 |
Each extended action has a name, which is mapped to a Python function |
| 1665 |
within a particular module. Each action also has an index associated |
| 1666 |
with it. This index is used to properly order execution when more than |
| 1667 |
one action is specified on the command line. The standard actions have |
| 1668 |
predefined indexes, and extended actions are interleaved into the normal |
| 1669 |
order of execution using those indexes. The collect action has index |
| 1670 |
100, the stage index has action 200, the store action has index 300 and |
| 1671 |
the purge action has index 400. |
| 1672 |
|
| 1673 |
|warning| |
| 1674 |
|
| 1675 |
Extended actions should always be configured to run *before* the |
| 1676 |
standard action they are associated with. This is because of the way |
| 1677 |
indicator files are used in Cedar Backup. For instance, the staging |
| 1678 |
process considers the collect action to be complete for a peer if the |
| 1679 |
file ``cback.collect`` can be found in that peer's collect directory. |
| 1680 |
|
| 1681 |
If you were to run the standard collect action before your other |
| 1682 |
collect-like actions, the indicator file would be written after the |
| 1683 |
collect action completes but *before* all of the other actions even |
| 1684 |
run. Because of this, there's a chance the stage process might back |
| 1685 |
up the collect directory before the entire set of collect-like |
| 1686 |
actions have completed --- and you would get no warning about this |
| 1687 |
in your email! |
| 1688 |
|
| 1689 |
So, imagine that a third-party developer provided a Cedar Backup |
| 1690 |
extension to back up a certain kind of database repository, and you |
| 1691 |
wanted to map that extension to the “database” command-line action. You |
| 1692 |
have been told that this function is called “foo.bar()”. You think of |
| 1693 |
this backup as a “collect” kind of action, so you want it to be |
| 1694 |
performed immediately before the collect action. |
| 1695 |
|
| 1696 |
To configure this extension, you would list an action with a name |
| 1697 |
“database”, a module “foo”, a function name “bar” and an index of “99”. |
| 1698 |
|
| 1699 |
This is how the hypothetical action would be configured: |
| 1700 |
|
| 1701 |
:: |
| 1702 |
|
| 1703 |
<extensions> |
| 1704 |
<action> |
| 1705 |
<name>database</name> |
| 1706 |
<module>foo</module> |
| 1707 |
<function>bar</function> |
| 1708 |
<index>99</index> |
| 1709 |
</action> |
| 1710 |
</extensions> |
| 1711 |
|
| 1712 |
|
| 1713 |
The following elements are part of the extensions configuration section: |
| 1714 |
|
| 1715 |
``action`` |
| 1716 |
This is a subsection that contains configuration related to a single |
| 1717 |
extended action. |
| 1718 |
|
| 1719 |
This section can be repeated as many times as is necessary. |
| 1720 |
|
| 1721 |
The action subsection contains the following fields: |
| 1722 |
|
| 1723 |
``name`` |
| 1724 |
Name of the extended action. |
| 1725 |
|
| 1726 |
*Restrictions:* Must be a non-empty string consisting of only |
| 1727 |
lower-case letters and digits. |
| 1728 |
|
| 1729 |
``module`` |
| 1730 |
Name of the Python module associated with the extension function. |
| 1731 |
|
| 1732 |
*Restrictions:* Must be a non-empty string and a valid Python |
| 1733 |
identifier. |
| 1734 |
|
| 1735 |
``function`` |
| 1736 |
Name of the Python extension function within the module. |
| 1737 |
|
| 1738 |
*Restrictions:* Must be a non-empty string and a valid Python |
| 1739 |
identifier. |
| 1740 |
|
| 1741 |
``index`` |
| 1742 |
Index of action, for execution ordering. |
| 1743 |
|
| 1744 |
*Restrictions:* Must be an integer GE 0. |
| 1745 |
|
| 1746 |
.. _cedar-config-poolofone: |
| 1747 |
|
| 1748 |
Setting up a Pool of One |
| 1749 |
------------------------ |
| 1750 |
|
| 1751 |
Cedar Backup has been designed primarily for situations where there is a |
| 1752 |
single master and a set of other clients that the master interacts with. |
| 1753 |
However, it will just as easily work for a single machine (a backup pool |
| 1754 |
of one). |
| 1755 |
|
| 1756 |
Once you complete all of these configuration steps, your backups will |
| 1757 |
run as scheduled out of cron. Any errors that occur will be reported in |
| 1758 |
daily emails to your root user (or the user that receives root's email). |
| 1759 |
If you don't receive any emails, then you know your backup worked. |
| 1760 |
|
| 1761 |
*Note:* all of these configuration steps should be run as the root user, |
| 1762 |
unless otherwise indicated. |
| 1763 |
|
| 1764 |
|tip| |
| 1765 |
|
| 1766 |
This setup procedure discusses how to set up Cedar Backup in the |
| 1767 |
“normal case” for a pool of one. If you would like to modify the way |
| 1768 |
Cedar Backup works (for instance, by ignoring the store stage and |
| 1769 |
just letting your backup sit in a staging directory), you can do |
| 1770 |
that. You'll just have to modify the procedure below based on |
| 1771 |
information in the remainder of the manual. |
| 1772 |
|
| 1773 |
Step 1: Decide when you will run your backup. |
| 1774 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 1775 |
|
| 1776 |
There are four parts to a Cedar Backup run: collect, stage, store and |
| 1777 |
purge. The usual way of setting off these steps is through a set of cron |
| 1778 |
jobs. Although you won't create your cron jobs just yet, you should |
| 1779 |
decide now when you will run your backup so you are prepared for later. |
| 1780 |
|
| 1781 |
Backing up large directories and creating ISO filesystem images can be |
| 1782 |
intensive operations, and could slow your computer down significantly. |
| 1783 |
Choose a backup time that will not interfere with normal use of your |
| 1784 |
computer. Usually, you will want the backup to occur every day, but it |
| 1785 |
is possible to configure cron to execute the backup only one day per |
| 1786 |
week, three days per week, etc. |
| 1787 |
|
| 1788 |
|warning| |
| 1789 |
|
| 1790 |
Because of the way Cedar Backup works, you must ensure that your |
| 1791 |
backup *always* runs on the first day of your configured week. This |
| 1792 |
is because Cedar Backup will only clear incremental backup |
| 1793 |
information and re-initialize your media when running on the first |
| 1794 |
day of the week. If you skip running Cedar Backup on the first day of |
| 1795 |
the week, your backups will likely be “confused” until the next week |
| 1796 |
begins, or until you re-run the backup using the ``--full`` flag. |
| 1797 |
|
| 1798 |
Step 2: Make sure email works. |
| 1799 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 1800 |
|
| 1801 |
Cedar Backup relies on email for problem notification. This notification |
| 1802 |
works through the magic of cron. Cron will email any output from each |
| 1803 |
job it executes to the user associated with the job. Since by default |
| 1804 |
Cedar Backup only writes output to the terminal if errors occur, this |
| 1805 |
ensures that notification emails will only be sent out if errors occur. |
| 1806 |
|
| 1807 |
In order to receive problem notifications, you must make sure that email |
| 1808 |
works for the user which is running the Cedar Backup cron jobs |
| 1809 |
(typically root). Refer to your distribution's documentation for |
| 1810 |
information on how to configure email on your system. Note that you may |
| 1811 |
prefer to configure root's email to forward to some other user, so you |
| 1812 |
do not need to check the root user's mail in order to see Cedar Backup |
| 1813 |
errors. |
| 1814 |
|
| 1815 |
Step 3: Configure your writer device. |
| 1816 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 1817 |
|
| 1818 |
Before using Cedar Backup, your writer device must be properly |
| 1819 |
configured. If you have configured your CD/DVD writer hardware to work |
| 1820 |
through the normal filesystem device path, then you just need to know |
| 1821 |
the path to the device on disk (something like ``/dev/cdrw``). Cedar |
| 1822 |
Backup will use the this device path both when talking to a command like |
| 1823 |
``cdrecord`` and when doing filesystem operations like running media |
| 1824 |
validation. |
| 1825 |
|
| 1826 |
Your other option is to configure your CD writer hardware like a SCSI |
| 1827 |
device (either because it *is* a SCSI device or because you are using |
| 1828 |
some sort of interface that makes it look like one). In this case, Cedar |
| 1829 |
Backup will use the SCSI id when talking to ``cdrecord`` and the device |
| 1830 |
path when running filesystem operations. |
| 1831 |
|
| 1832 |
See :doc:`config` for more information on writer devices and how they are |
| 1833 |
configured. |
| 1834 |
|
| 1835 |
|note| |
| 1836 |
|
| 1837 |
There is no need to set up your CD/DVD device if you have decided not |
| 1838 |
to execute the store action. |
| 1839 |
|
| 1840 |
Due to the underlying utilities that Cedar Backup uses, the SCSI id |
| 1841 |
may only be used for CD writers, *not* DVD writers. |
| 1842 |
|
| 1843 |
Step 4: Configure your backup user. |
| 1844 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 1845 |
|
| 1846 |
Choose a user to be used for backups. Some platforms may come with a |
| 1847 |
“ready made” backup user. For other platforms, you may have to create a |
| 1848 |
user yourself. You may choose any id you like, but a descriptive name |
| 1849 |
such as ``backup`` or ``cback`` is a good choice. See your |
| 1850 |
distribution's documentation for information on how to add a user. |
| 1851 |
|
| 1852 |
|note| |
| 1853 |
|
| 1854 |
Standard Debian systems come with a user named ``backup``. You may |
| 1855 |
choose to stay with this user or create another one. |
| 1856 |
|
| 1857 |
Step 5: Create your backup tree. |
| 1858 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 1859 |
|
| 1860 |
Cedar Backup requires a backup directory tree on disk. This directory |
| 1861 |
tree must be roughly three times as big as the amount of data that will |
| 1862 |
be backed up on a nightly basis, to allow for the data to be collected, |
| 1863 |
staged, and then placed into an ISO filesystem image on disk. (This is |
| 1864 |
one disadvantage to using Cedar Backup in single-machine pools, but in |
| 1865 |
this day of really large hard drives, it might not be an issue.) Note |
| 1866 |
that if you elect not to purge the staging directory every night, you |
| 1867 |
will need even more space. |
| 1868 |
|
| 1869 |
You should create a collect directory, a staging directory and a working |
| 1870 |
(temporary) directory. One recommended layout is this: |
| 1871 |
|
| 1872 |
:: |
| 1873 |
|
| 1874 |
/opt/ |
| 1875 |
backup/ |
| 1876 |
collect/ |
| 1877 |
stage/ |
| 1878 |
tmp/ |
| 1879 |
|
| 1880 |
|
| 1881 |
If you will be backing up sensitive information (i.e. password files), |
| 1882 |
it is recommended that these directories be owned by the backup user |
| 1883 |
(whatever you named it), with permissions ``700``. |
| 1884 |
|
| 1885 |
|note| |
| 1886 |
|
| 1887 |
You don't have to use ``/opt`` as the root of your directory |
| 1888 |
structure. Use anything you would like. I use ``/opt`` because it is |
| 1889 |
my “dumping ground” for filesystems that Debian does not manage. |
| 1890 |
|
| 1891 |
Some users have requested that the Debian packages set up a more |
| 1892 |
“standard” location for backups right out-of-the-box. I have resisted |
| 1893 |
doing this because it's difficult to choose an appropriate backup |
| 1894 |
location from within the package. If you would prefer, you can create |
| 1895 |
the backup directory structure within some existing Debian directory |
| 1896 |
such as ``/var/backups`` or ``/var/tmp``. |
| 1897 |
|
| 1898 |
Step 6: Create the Cedar Backup configuration file. |
| 1899 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 1900 |
|
| 1901 |
Following the instructions in *Configuration File Format* (above) create a |
| 1902 |
configuration file for your machine. Since you are working with a pool of one, |
| 1903 |
you must configure all four action-specific sections: collect, stage, store and |
| 1904 |
purge. |
| 1905 |
|
| 1906 |
The usual location for the Cedar Backup config file is |
| 1907 |
``/etc/cback3.conf``. If you change the location, make sure you edit |
| 1908 |
your cronjobs (below) to point the ``cback3`` script at the correct |
| 1909 |
config file (using the ``--config`` option). |
| 1910 |
|
| 1911 |
|warning| |
| 1912 |
|
| 1913 |
Configuration files should always be writable only by root (or by the |
| 1914 |
file owner, if the owner is not root). |
| 1915 |
|
| 1916 |
If you intend to place confidential information into the Cedar Backup |
| 1917 |
configuration file, make sure that you set the filesystem permissions |
| 1918 |
on the file appropriately. For instance, if you configure any |
| 1919 |
extensions that require passwords or other similar information, you |
| 1920 |
should make the file readable only to root or to the file owner (if |
| 1921 |
the owner is not root). |
| 1922 |
|
| 1923 |
Step 7: Validate the Cedar Backup configuration file. |
| 1924 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 1925 |
|
| 1926 |
Use the command ``cback3 validate`` to validate your configuration file. |
| 1927 |
This command checks that the configuration file can be found and parsed, |
| 1928 |
and also checks for typical configuration problems, such as invalid |
| 1929 |
CD/DVD device entries. |
| 1930 |
|
| 1931 |
*Note:* the most common cause of configuration problems is in not closing |
| 1932 |
XML tags properly. Any XML tag that is “opened” must be “closed” |
| 1933 |
appropriately. |
| 1934 |
|
| 1935 |
Step 8: Test your backup. |
| 1936 |
~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 1937 |
|
| 1938 |
Place a valid CD/DVD disc in your drive, and then use the command |
| 1939 |
``cback3 --full all``. You should execute this command as root. If the |
| 1940 |
command completes with no output, then the backup was run successfully. |
| 1941 |
|
| 1942 |
Just to be sure that everything worked properly, check the logfile |
| 1943 |
(``/var/log/cback3.log``) for errors and also mount the CD/DVD disc to |
| 1944 |
be sure it can be read. |
| 1945 |
|
| 1946 |
*If Cedar Backup ever completes “normally” but the disc that is created is not |
| 1947 |
usable, please report this as a bug. To be safe, always enable the consistency |
| 1948 |
check option in the store configuration section.* [4]_ |
| 1949 |
|
| 1950 |
Step 9: Modify the backup cron jobs. |
| 1951 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 1952 |
|
| 1953 |
Since Cedar Backup should be run as root, one way to configure the cron |
| 1954 |
job is to add a line like this to your ``/etc/crontab`` file: |
| 1955 |
|
| 1956 |
:: |
| 1957 |
|
| 1958 |
30 00 * * * root cback3 all |
| 1959 |
|
| 1960 |
|
| 1961 |
Or, you can create an executable script containing just these lines and |
| 1962 |
place that file in the ``/etc/cron.daily`` directory: |
| 1963 |
|
| 1964 |
:: |
| 1965 |
|
| 1966 |
#/bin/sh |
| 1967 |
cback3 all |
| 1968 |
|
| 1969 |
|
| 1970 |
You should consider adding the ``--output`` or ``-O`` switch to your |
| 1971 |
``cback3`` command-line in cron. This will result in larger logs, but |
| 1972 |
could help diagnose problems when commands like ``cdrecord`` or |
| 1973 |
``mkisofs`` fail mysteriously. |
| 1974 |
|
| 1975 |
|note| |
| 1976 |
|
| 1977 |
For general information about using cron, see the manpage for |
| 1978 |
crontab(5). |
| 1979 |
|
| 1980 |
On a Debian system, execution of daily backups is controlled by the |
| 1981 |
file ``/etc/cron.d/cedar-backup3``. As installed, this file contains |
| 1982 |
several different settings, all commented out. Uncomment the “Single |
| 1983 |
machine (pool of one)” entry in the file, and change the line so that |
| 1984 |
the backup goes off when you want it to. |
| 1985 |
|
| 1986 |
.. _cedar-config-client: |
| 1987 |
|
| 1988 |
Setting up a Client Peer Node |
| 1989 |
----------------------------- |
| 1990 |
|
| 1991 |
Cedar Backup has been designed to backup entire “pools” of machines. In |
| 1992 |
any given pool, there is one master and some number of clients. Most of |
| 1993 |
the work takes place on the master, so configuring a client is a little |
| 1994 |
simpler than configuring a master. |
| 1995 |
|
| 1996 |
Backups are designed to take place over an RSH or SSH connection. |
| 1997 |
Because RSH is generally considered insecure, you are encouraged to use |
| 1998 |
SSH rather than RSH. This document will only describe how to configure |
| 1999 |
Cedar Backup to use SSH; if you want to use RSH, you're on your own. |
| 2000 |
|
| 2001 |
Once you complete all of these configuration steps, your backups will |
| 2002 |
run as scheduled out of cron. Any errors that occur will be reported in |
| 2003 |
daily emails to your root user (or the user that receives root's email). |
| 2004 |
If you don't receive any emails, then you know your backup worked. |
| 2005 |
|
| 2006 |
*Note:* all of these configuration steps should be run as the root user, |
| 2007 |
unless otherwise indicated. |
| 2008 |
|
| 2009 |
|note| |
| 2010 |
|
| 2011 |
See :doc:`securingssh` for some important notes on how to optionally further |
| 2012 |
secure password-less SSH connections to your clients. |
| 2013 |
|
| 2014 |
Step 1: Decide when you will run your backup. |
| 2015 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2016 |
|
| 2017 |
There are four parts to a Cedar Backup run: collect, stage, store and |
| 2018 |
purge. The usual way of setting off these steps is through a set of cron |
| 2019 |
jobs. Although you won't create your cron jobs just yet, you should |
| 2020 |
decide now when you will run your backup so you are prepared for later. |
| 2021 |
|
| 2022 |
Backing up large directories and creating ISO filesystem images can be |
| 2023 |
intensive operations, and could slow your computer down significantly. |
| 2024 |
Choose a backup time that will not interfere with normal use of your |
| 2025 |
computer. Usually, you will want the backup to occur every day, but it |
| 2026 |
is possible to configure cron to execute the backup only one day per |
| 2027 |
week, three days per week, etc. |
| 2028 |
|
| 2029 |
|warning| |
| 2030 |
|
| 2031 |
Because of the way Cedar Backup works, you must ensure that your |
| 2032 |
backup *always* runs on the first day of your configured week. This |
| 2033 |
is because Cedar Backup will only clear incremental backup |
| 2034 |
information and re-initialize your media when running on the first |
| 2035 |
day of the week. If you skip running Cedar Backup on the first day of |
| 2036 |
the week, your backups will likely be “confused” until the next week |
| 2037 |
begins, or until you re-run the backup using the ``--full`` flag. |
| 2038 |
|
| 2039 |
Step 2: Make sure email works. |
| 2040 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2041 |
|
| 2042 |
Cedar Backup relies on email for problem notification. This notification |
| 2043 |
works through the magic of cron. Cron will email any output from each |
| 2044 |
job it executes to the user associated with the job. Since by default |
| 2045 |
Cedar Backup only writes output to the terminal if errors occur, this |
| 2046 |
neatly ensures that notification emails will only be sent out if errors |
| 2047 |
occur. |
| 2048 |
|
| 2049 |
In order to receive problem notifications, you must make sure that email |
| 2050 |
works for the user which is running the Cedar Backup cron jobs |
| 2051 |
(typically root). Refer to your distribution's documentation for |
| 2052 |
information on how to configure email on your system. Note that you may |
| 2053 |
prefer to configure root's email to forward to some other user, so you |
| 2054 |
do not need to check the root user's mail in order to see Cedar Backup |
| 2055 |
errors. |
| 2056 |
|
| 2057 |
Step 3: Configure the master in your backup pool. |
| 2058 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2059 |
|
| 2060 |
You will not be able to complete the client configuration until at least |
| 2061 |
step 3 of the master's configuration has been completed. In particular, |
| 2062 |
you will need to know the master's public SSH identity to fully |
| 2063 |
configure a client. |
| 2064 |
|
| 2065 |
To find the master's public SSH identity, log in as the backup user on |
| 2066 |
the master and ``cat`` the public identity file ``~/.ssh/id_rsa.pub``: |
| 2067 |
|
| 2068 |
:: |
| 2069 |
|
| 2070 |
user@machine> cat ~/.ssh/id_rsa.pub |
| 2071 |
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIEA0vOKjlfwohPg1oPRdrmwHk75l3mI9Tb/WRZfVnu2Pw69 |
| 2072 |
uyphM9wBLRo6QfOC2T8vZCB8o/ZIgtAM3tkM0UgQHxKBXAZ+H36TOgg7BcI20I93iGtzpsMA/uXQy8kH |
| 2073 |
HgZooYqQ9pw+ZduXgmPcAAv2b5eTm07wRqFt/U84k6bhTzs= user@machine |
| 2074 |
|
| 2075 |
|
| 2076 |
Step 4: Configure your backup user. |
| 2077 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2078 |
|
| 2079 |
Choose a user to be used for backups. Some platforms may come with a |
| 2080 |
"ready made" backup user. For other platforms, you may have to create a |
| 2081 |
user yourself. You may choose any id you like, but a descriptive name |
| 2082 |
such as ``backup`` or ``cback`` is a good choice. See your |
| 2083 |
distribution's documentation for information on how to add a user. |
| 2084 |
|
| 2085 |
|note| |
| 2086 |
|
| 2087 |
Standard Debian systems come with a user named ``backup``. You may |
| 2088 |
choose to stay with this user or create another one. |
| 2089 |
|
| 2090 |
Once you have created your backup user, you must create an SSH keypair |
| 2091 |
for it. Log in as your backup user, and then run the command |
| 2092 |
``ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa``: |
| 2093 |
|
| 2094 |
:: |
| 2095 |
|
| 2096 |
user@machine> ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa |
| 2097 |
Generating public/private rsa key pair. |
| 2098 |
Created directory '/home/user/.ssh'. |
| 2099 |
Your identification has been saved in /home/user/.ssh/id_rsa. |
| 2100 |
Your public key has been saved in /home/user/.ssh/id_rsa.pub. |
| 2101 |
The key fingerprint is: |
| 2102 |
11:3e:ad:72:95:fe:96:dc:1e:3b:f4:cc:2c:ff:15:9e user@machine |
| 2103 |
|
| 2104 |
|
| 2105 |
The default permissions for this directory should be fine. However, if |
| 2106 |
the directory existed before you ran ``ssh-keygen``, then you may need |
| 2107 |
to modify the permissions. Make sure that the ``~/.ssh`` directory is |
| 2108 |
readable only by the backup user (i.e. mode ``700``), that the |
| 2109 |
``~/.ssh/id_rsa`` file is only readable and writable only by the backup |
| 2110 |
user (i.e. mode ``600``) and that the ``~/.ssh/id_rsa.pub`` file is |
| 2111 |
writable only by the backup user (i.e. mode ``600`` or mode ``644``). |
| 2112 |
|
| 2113 |
Finally, take the master's public SSH identity (which you found in step |
| 2114 |
2) and cut-and-paste it into the file ``~/.ssh/authorized_keys``. Make |
| 2115 |
sure the identity value is pasted into the file *all on one line*, and |
| 2116 |
that the ``authorized_keys`` file is owned by your backup user and has |
| 2117 |
permissions ``600``. |
| 2118 |
|
| 2119 |
If you have other preferences or standard ways of setting up your users' |
| 2120 |
SSH configuration (i.e. different key type, etc.), feel free to do |
| 2121 |
things your way. The important part is that the master must be able to |
| 2122 |
SSH into a client *with no password entry required*. |
| 2123 |
|
| 2124 |
Step 5: Create your backup tree. |
| 2125 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2126 |
|
| 2127 |
Cedar Backup requires a backup directory tree on disk. This directory |
| 2128 |
tree must be roughly as big as the amount of data that will be backed up |
| 2129 |
on a nightly basis (more if you elect not to purge it all every night). |
| 2130 |
|
| 2131 |
You should create a collect directory and a working (temporary) |
| 2132 |
directory. One recommended layout is this: |
| 2133 |
|
| 2134 |
:: |
| 2135 |
|
| 2136 |
/opt/ |
| 2137 |
backup/ |
| 2138 |
collect/ |
| 2139 |
tmp/ |
| 2140 |
|
| 2141 |
|
| 2142 |
If you will be backing up sensitive information (i.e. password files), |
| 2143 |
it is recommended that these directories be owned by the backup user |
| 2144 |
(whatever you named it), with permissions ``700``. |
| 2145 |
|
| 2146 |
|note| |
| 2147 |
|
| 2148 |
You don't have to use ``/opt`` as the root of your directory |
| 2149 |
structure. Use anything you would like. I use ``/opt`` because it is |
| 2150 |
my “dumping ground” for filesystems that Debian does not manage. |
| 2151 |
|
| 2152 |
Some users have requested that the Debian packages set up a more |
| 2153 |
"standard" location for backups right out-of-the-box. I have resisted |
| 2154 |
doing this because it's difficult to choose an appropriate backup |
| 2155 |
location from within the package. If you would prefer, you can create |
| 2156 |
the backup directory structure within some existing Debian directory |
| 2157 |
such as ``/var/backups`` or ``/var/tmp``. |
| 2158 |
|
| 2159 |
Step 6: Create the Cedar Backup configuration file. |
| 2160 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2161 |
|
| 2162 |
Following the instructions in *Configuration File Format* (above), create a |
| 2163 |
configuration file for your machine. Since you are working with a client, you |
| 2164 |
must configure all action-specific sections for the collect and purge actions. |
| 2165 |
|
| 2166 |
The usual location for the Cedar Backup config file is |
| 2167 |
``/etc/cback3.conf``. If you change the location, make sure you edit |
| 2168 |
your cronjobs (below) to point the ``cback3`` script at the correct |
| 2169 |
config file (using the ``--config`` option). |
| 2170 |
|
| 2171 |
|warning| |
| 2172 |
|
| 2173 |
Configuration files should always be writable only by root (or by the |
| 2174 |
file owner, if the owner is not root). |
| 2175 |
|
| 2176 |
If you intend to place confidental information into the Cedar Backup |
| 2177 |
configuration file, make sure that you set the filesystem permissions |
| 2178 |
on the file appropriately. For instance, if you configure any |
| 2179 |
extensions that require passwords or other similar information, you |
| 2180 |
should make the file readable only to root or to the file owner (if |
| 2181 |
the owner is not root). |
| 2182 |
|
| 2183 |
Step 7: Validate the Cedar Backup configuration file. |
| 2184 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2185 |
|
| 2186 |
Use the command ``cback3 validate`` to validate your configuration file. |
| 2187 |
This command checks that the configuration file can be found and parsed, |
| 2188 |
and also checks for typical configuration problems. This command *only* |
| 2189 |
validates configuration on the one client, not the master or any other |
| 2190 |
clients in a pool. |
| 2191 |
|
| 2192 |
*Note:* the most common cause of configuration problems is in not closing |
| 2193 |
XML tags properly. Any XML tag that is “opened” must be “closed” |
| 2194 |
appropriately. |
| 2195 |
|
| 2196 |
Step 8: Test your backup. |
| 2197 |
~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2198 |
|
| 2199 |
Use the command ``cback3 --full collect purge``. If the command |
| 2200 |
completes with no output, then the backup was run successfully. Just to |
| 2201 |
be sure that everything worked properly, check the logfile |
| 2202 |
(``/var/log/cback3.log``) for errors. |
| 2203 |
|
| 2204 |
Step 9: Modify the backup cron jobs. |
| 2205 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2206 |
|
| 2207 |
Since Cedar Backup should be run as root, you should add a set of lines |
| 2208 |
like this to your ``/etc/crontab`` file: |
| 2209 |
|
| 2210 |
:: |
| 2211 |
|
| 2212 |
30 00 * * * root cback3 collect |
| 2213 |
30 06 * * * root cback3 purge |
| 2214 |
|
| 2215 |
|
| 2216 |
You should consider adding the ``--output`` or ``-O`` switch to your |
| 2217 |
``cback3`` command-line in cron. This will result in larger logs, but |
| 2218 |
could help diagnose problems when commands like ``cdrecord`` or |
| 2219 |
``mkisofs`` fail mysteriously. |
| 2220 |
|
| 2221 |
You will need to coordinate the collect and purge actions on the client |
| 2222 |
so that the collect action completes before the master attempts to |
| 2223 |
stage, and so that the purge action does not begin until after the |
| 2224 |
master has completed staging. Usually, allowing an hour or two between |
| 2225 |
steps should be sufficient. [5]_ |
| 2226 |
|
| 2227 |
|note| |
| 2228 |
|
| 2229 |
For general information about using cron, see the manpage for |
| 2230 |
crontab(5). |
| 2231 |
|
| 2232 |
On a Debian system, execution of daily backups is controlled by the |
| 2233 |
file ``/etc/cron.d/cedar-backup3``. As installed, this file contains |
| 2234 |
several different settings, all commented out. Uncomment the “Client |
| 2235 |
machine” entries in the file, and change the lines so that the backup |
| 2236 |
goes off when you want it to. |
| 2237 |
|
| 2238 |
.. _cedar-config-master: |
| 2239 |
|
| 2240 |
Setting up a Master Peer Node |
| 2241 |
----------------------------- |
| 2242 |
|
| 2243 |
Cedar Backup has been designed to backup entire “pools” of machines. In |
| 2244 |
any given pool, there is one master and some number of clients. Most of |
| 2245 |
the work takes place on the master, so configuring a master is somewhat |
| 2246 |
more complicated than configuring a client. |
| 2247 |
|
| 2248 |
Backups are designed to take place over an RSH or SSH connection. |
| 2249 |
Because RSH is generally considered insecure, you are encouraged to use |
| 2250 |
SSH rather than RSH. This document will only describe how to configure |
| 2251 |
Cedar Backup to use SSH; if you want to use RSH, you're on your own. |
| 2252 |
|
| 2253 |
Once you complete all of these configuration steps, your backups will |
| 2254 |
run as scheduled out of cron. Any errors that occur will be reported in |
| 2255 |
daily emails to your root user (or whichever other user receives root's |
| 2256 |
email). If you don't receive any emails, then you know your backup |
| 2257 |
worked. |
| 2258 |
|
| 2259 |
*Note:* all of these configuration steps should be run as the root user, |
| 2260 |
unless otherwise indicated. |
| 2261 |
|
| 2262 |
|tip| |
| 2263 |
|
| 2264 |
This setup procedure discusses how to set up Cedar Backup in the |
| 2265 |
“normal case” for a master. If you would like to modify the way Cedar |
| 2266 |
Backup works (for instance, by ignoring the store stage and just |
| 2267 |
letting your backup sit in a staging directory), you can do that. |
| 2268 |
You'll just have to modify the procedure below based on information |
| 2269 |
in the remainder of the manual. |
| 2270 |
|
| 2271 |
Step 1: Decide when you will run your backup. |
| 2272 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2273 |
|
| 2274 |
There are four parts to a Cedar Backup run: collect, stage, store and |
| 2275 |
purge. The usual way of setting off these steps is through a set of cron |
| 2276 |
jobs. Although you won't create your cron jobs just yet, you should |
| 2277 |
decide now when you will run your backup so you are prepared for later. |
| 2278 |
|
| 2279 |
Keep in mind that you do not necessarily have to run the collect action |
| 2280 |
on the master. See notes further below for more information. |
| 2281 |
|
| 2282 |
Backing up large directories and creating ISO filesystem images can be |
| 2283 |
intensive operations, and could slow your computer down significantly. |
| 2284 |
Choose a backup time that will not interfere with normal use of your |
| 2285 |
computer. Usually, you will want the backup to occur every day, but it |
| 2286 |
is possible to configure cron to execute the backup only one day per |
| 2287 |
week, three days per week, etc. |
| 2288 |
|
| 2289 |
|warning| |
| 2290 |
|
| 2291 |
Because of the way Cedar Backup works, you must ensure that your |
| 2292 |
backup *always* runs on the first day of your configured week. This |
| 2293 |
is because Cedar Backup will only clear incremental backup |
| 2294 |
information and re-initialize your media when running on the first |
| 2295 |
day of the week. If you skip running Cedar Backup on the first day of |
| 2296 |
the week, your backups will likely be “confused” until the next week |
| 2297 |
begins, or until you re-run the backup using the ``--full`` flag. |
| 2298 |
|
| 2299 |
Step 2: Make sure email works. |
| 2300 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2301 |
|
| 2302 |
Cedar Backup relies on email for problem notification. This notification |
| 2303 |
works through the magic of cron. Cron will email any output from each |
| 2304 |
job it executes to the user associated with the job. Since by default |
| 2305 |
Cedar Backup only writes output to the terminal if errors occur, this |
| 2306 |
neatly ensures that notification emails will only be sent out if errors |
| 2307 |
occur. |
| 2308 |
|
| 2309 |
In order to receive problem notifications, you must make sure that email |
| 2310 |
works for the user which is running the Cedar Backup cron jobs |
| 2311 |
(typically root). Refer to your distribution's documentation for |
| 2312 |
information on how to configure email on your system. Note that you may |
| 2313 |
prefer to configure root's email to forward to some other user, so you |
| 2314 |
do not need to check the root user's mail in order to see Cedar Backup |
| 2315 |
errors. |
| 2316 |
|
| 2317 |
Step 3: Configure your writer device. |
| 2318 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2319 |
|
| 2320 |
Before using Cedar Backup, your writer device must be properly |
| 2321 |
configured. If you have configured your CD/DVD writer hardware to work |
| 2322 |
through the normal filesystem device path, then you just need to know |
| 2323 |
the path to the device on disk (something like ``/dev/cdrw``). Cedar |
| 2324 |
Backup will use the this device path both when talking to a command like |
| 2325 |
``cdrecord`` and when doing filesystem operations like running media |
| 2326 |
validation. |
| 2327 |
|
| 2328 |
Your other option is to configure your CD writer hardware like a SCSI |
| 2329 |
device (either because it *is* a SCSI device or because you are using |
| 2330 |
some sort of interface that makes it look like one). In this case, Cedar |
| 2331 |
Backup will use the SCSI id when talking to ``cdrecord`` and the device |
| 2332 |
path when running filesystem operations. |
| 2333 |
|
| 2334 |
See *Configuring your Writer Device* for more information on writer devices and |
| 2335 |
how they are configured. |
| 2336 |
|
| 2337 |
|note| |
| 2338 |
|
| 2339 |
There is no need to set up your CD/DVD device if you have decided not |
| 2340 |
to execute the store action. |
| 2341 |
|
| 2342 |
Due to the underlying utilities that Cedar Backup uses, the SCSI id |
| 2343 |
may only be used for CD writers, *not* DVD writers. |
| 2344 |
|
| 2345 |
Step 4: Configure your backup user. |
| 2346 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2347 |
|
| 2348 |
Choose a user to be used for backups. Some platforms may come with a |
| 2349 |
“ready made” backup user. For other platforms, you may have to create a |
| 2350 |
user yourself. You may choose any id you like, but a descriptive name |
| 2351 |
such as ``backup`` or ``cback`` is a good choice. See your |
| 2352 |
distribution's documentation for information on how to add a user. |
| 2353 |
|
| 2354 |
|note| |
| 2355 |
|
| 2356 |
Standard Debian systems come with a user named ``backup``. You may |
| 2357 |
choose to stay with this user or create another one. |
| 2358 |
|
| 2359 |
Once you have created your backup user, you must create an SSH keypair |
| 2360 |
for it. Log in as your backup user, and then run the command |
| 2361 |
``ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa``: |
| 2362 |
|
| 2363 |
:: |
| 2364 |
|
| 2365 |
user@machine> ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa |
| 2366 |
Generating public/private rsa key pair. |
| 2367 |
Created directory '/home/user/.ssh'. |
| 2368 |
Your identification has been saved in /home/user/.ssh/id_rsa. |
| 2369 |
Your public key has been saved in /home/user/.ssh/id_rsa.pub. |
| 2370 |
The key fingerprint is: |
| 2371 |
11:3e:ad:72:95:fe:96:dc:1e:3b:f4:cc:2c:ff:15:9e user@machine |
| 2372 |
|
| 2373 |
|
| 2374 |
The default permissions for this directory should be fine. However, if |
| 2375 |
the directory existed before you ran ``ssh-keygen``, then you may need |
| 2376 |
to modify the permissions. Make sure that the ``~/.ssh`` directory is |
| 2377 |
readable only by the backup user (i.e. mode ``700``), that the |
| 2378 |
``~/.ssh/id_rsa`` file is only readable and writable by the backup user |
| 2379 |
(i.e. mode ``600``) and that the ``~/.ssh/id_rsa.pub`` file is writable |
| 2380 |
only by the backup user (i.e. mode ``600`` or mode ``644``). |
| 2381 |
|
| 2382 |
If you have other preferences or standard ways of setting up your users' |
| 2383 |
SSH configuration (i.e. different key type, etc.), feel free to do |
| 2384 |
things your way. The important part is that the master must be able to |
| 2385 |
SSH into a client *with no password entry required*. |
| 2386 |
|
| 2387 |
Step 5: Create your backup tree. |
| 2388 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2389 |
|
| 2390 |
Cedar Backup requires a backup directory tree on disk. This directory |
| 2391 |
tree must be roughly large enough hold twice as much data as will be |
| 2392 |
backed up from the entire pool on a given night, plus space for whatever |
| 2393 |
is collected on the master itself. This will allow for all three |
| 2394 |
operations - collect, stage and store - to have enough space to |
| 2395 |
complete. Note that if you elect not to purge the staging directory |
| 2396 |
every night, you will need even more space. |
| 2397 |
|
| 2398 |
You should create a collect directory, a staging directory and a working |
| 2399 |
(temporary) directory. One recommended layout is this: |
| 2400 |
|
| 2401 |
:: |
| 2402 |
|
| 2403 |
/opt/ |
| 2404 |
backup/ |
| 2405 |
collect/ |
| 2406 |
stage/ |
| 2407 |
tmp/ |
| 2408 |
|
| 2409 |
|
| 2410 |
If you will be backing up sensitive information (i.e. password files), |
| 2411 |
it is recommended that these directories be owned by the backup user |
| 2412 |
(whatever you named it), with permissions ``700``. |
| 2413 |
|
| 2414 |
|note| |
| 2415 |
|
| 2416 |
You don't have to use ``/opt`` as the root of your directory |
| 2417 |
structure. Use anything you would like. I use ``/opt`` because it is |
| 2418 |
my “dumping ground” for filesystems that Debian does not manage. |
| 2419 |
|
| 2420 |
Some users have requested that the Debian packages set up a more |
| 2421 |
“standard” location for backups right out-of-the-box. I have resisted |
| 2422 |
doing this because it's difficult to choose an appropriate backup |
| 2423 |
location from within the package. If you would prefer, you can create |
| 2424 |
the backup directory structure within some existing Debian directory |
| 2425 |
such as ``/var/backups`` or ``/var/tmp``. |
| 2426 |
|
| 2427 |
Step 6: Create the Cedar Backup configuration file. |
| 2428 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2429 |
|
| 2430 |
Following the instructions in *Configuration File Foramt** (above), create a |
| 2431 |
configuration file for your machine. Since you are working with a master |
| 2432 |
machine, you would typically configure all four action-specific sections: |
| 2433 |
collect, stage, store and purge. |
| 2434 |
|
| 2435 |
|note| |
| 2436 |
|
| 2437 |
Note that the master can treat itself as a “client” peer for certain |
| 2438 |
actions. As an example, if you run the collect action on the master, |
| 2439 |
then you will stage that data by configuring a local peer |
| 2440 |
representing the master. |
| 2441 |
|
| 2442 |
Something else to keep in mind is that you do not really have to run |
| 2443 |
the collect action on the master. For instance, you may prefer to |
| 2444 |
just use your master machine as a “consolidation point” machine that |
| 2445 |
just collects data from the other client machines in a backup pool. |
| 2446 |
In that case, there is no need to collect data on the master itself. |
| 2447 |
|
| 2448 |
The usual location for the Cedar Backup config file is |
| 2449 |
``/etc/cback3.conf``. If you change the location, make sure you edit |
| 2450 |
your cronjobs (below) to point the ``cback3`` script at the correct |
| 2451 |
config file (using the ``--config`` option). |
| 2452 |
|
| 2453 |
|warning| |
| 2454 |
|
| 2455 |
Configuration files should always be writable only by root (or by the |
| 2456 |
file owner, if the owner is not root). |
| 2457 |
|
| 2458 |
If you intend to place confidental information into the Cedar Backup |
| 2459 |
configuration file, make sure that you set the filesystem permissions |
| 2460 |
on the file appropriately. For instance, if you configure any |
| 2461 |
extensions that require passwords or other similar information, you |
| 2462 |
should make the file readable only to root or to the file owner (if |
| 2463 |
the owner is not root). |
| 2464 |
|
| 2465 |
Step 7: Validate the Cedar Backup configuration file. |
| 2466 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2467 |
|
| 2468 |
Use the command ``cback3 validate`` to validate your configuration file. |
| 2469 |
This command checks that the configuration file can be found and parsed, |
| 2470 |
and also checks for typical configuration problems, such as invalid |
| 2471 |
CD/DVD device entries. This command *only* validates configuration on |
| 2472 |
the master, not any clients that the master might be configured to |
| 2473 |
connect to. |
| 2474 |
|
| 2475 |
*Note:* the most common cause of configuration problems is in not closing |
| 2476 |
XML tags properly. Any XML tag that is “opened” must be “closed” |
| 2477 |
appropriately. |
| 2478 |
|
| 2479 |
Step 8: Test connectivity to client machines. |
| 2480 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2481 |
|
| 2482 |
This step must wait until after your client machines have been at least |
| 2483 |
partially configured. Once the backup user(s) have been configured on |
| 2484 |
the client machine(s) in a pool, attempt an SSH connection to each |
| 2485 |
client. |
| 2486 |
|
| 2487 |
Log in as the backup user on the master, and then use the command |
| 2488 |
``ssh user@machine`` where user is the name of backup user *on the |
| 2489 |
client machine*, and machine is the name of the client machine. |
| 2490 |
|
| 2491 |
If you are able to log in successfully to each client without entering a |
| 2492 |
password, then things have been configured properly. Otherwise, |
| 2493 |
double-check that you followed the user setup instructions for the |
| 2494 |
master and the clients. |
| 2495 |
|
| 2496 |
Step 9: Test your backup. |
| 2497 |
~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2498 |
|
| 2499 |
Make sure that you have configured all of the clients in your backup |
| 2500 |
pool. On all of the clients, execute ``cback3 --full collect``. (You will |
| 2501 |
probably have already tested this command on each of the clients, so it should |
| 2502 |
succeed.) |
| 2503 |
|
| 2504 |
When all of the client backups have completed, place a valid CD/DVD disc |
| 2505 |
in your drive, and then use the command ``cback3 --full all``. You should |
| 2506 |
execute this command as root. If the command completes with no output, then the |
| 2507 |
backup was run successfully. |
| 2508 |
|
| 2509 |
Just to be sure that everything worked properly, check the logfile |
| 2510 |
(``/var/log/cback3.log``) on the master and each of the clients, and |
| 2511 |
also mount the CD/DVD disc on the master to be sure it can be read. |
| 2512 |
|
| 2513 |
You may also want to run ``cback3 purge`` on the master and each client |
| 2514 |
once you have finished validating that everything worked. |
| 2515 |
|
| 2516 |
*If Cedar Backup ever completes “normally” but the disc that is created |
| 2517 |
is not usable, please report this as a bug. To be safe, always enable |
| 2518 |
the consistency check option in the store configuration section.* |
| 2519 |
|
| 2520 |
Step 10: Modify the backup cron jobs. |
| 2521 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2522 |
|
| 2523 |
Since Cedar Backup should be run as root, you should add a set of lines |
| 2524 |
like this to your ``/etc/crontab`` file: |
| 2525 |
|
| 2526 |
:: |
| 2527 |
|
| 2528 |
30 00 * * * root cback3 collect |
| 2529 |
30 02 * * * root cback3 stage |
| 2530 |
30 04 * * * root cback3 store |
| 2531 |
30 06 * * * root cback3 purge |
| 2532 |
|
| 2533 |
|
| 2534 |
You should consider adding the ``--output`` or ``-O`` switch to your |
| 2535 |
``cback3`` command-line in cron. This will result in larger logs, but |
| 2536 |
could help diagnose problems when commands like ``cdrecord`` or |
| 2537 |
``mkisofs`` fail mysteriously. |
| 2538 |
|
| 2539 |
You will need to coordinate the collect and purge actions on clients so |
| 2540 |
that their collect actions complete before the master attempts to stage, |
| 2541 |
and so that their purge actions do not begin until after the master has |
| 2542 |
completed staging. Usually, allowing an hour or two between steps should |
| 2543 |
be sufficient. |
| 2544 |
|
| 2545 |
|note| |
| 2546 |
|
| 2547 |
For general information about using cron, see the manpage for |
| 2548 |
crontab(5). |
| 2549 |
|
| 2550 |
On a Debian system, execution of daily backups is controlled by the |
| 2551 |
file ``/etc/cron.d/cedar-backup3``. As installed, this file contains |
| 2552 |
several different settings, all commented out. Uncomment the “Master |
| 2553 |
machine” entries in the file, and change the lines so that the backup |
| 2554 |
goes off when you want it to. |
| 2555 |
|
| 2556 |
.. _cedar-config-writer: |
| 2557 |
|
| 2558 |
Configuring your Writer Device |
| 2559 |
------------------------------ |
| 2560 |
|
| 2561 |
Device Types |
| 2562 |
~~~~~~~~~~~~ |
| 2563 |
|
| 2564 |
In order to execute the store action, you need to know how to identify |
| 2565 |
your writer device. Cedar Backup supports two kinds of device types: CD |
| 2566 |
writers and DVD writers. DVD writers are always referenced through a |
| 2567 |
filesystem device name (i.e. ``/dev/dvd``). CD writers can be referenced |
| 2568 |
either through a SCSI id, or through a filesystem device name. Which you |
| 2569 |
use depends on your operating system and hardware. |
| 2570 |
|
| 2571 |
Devices identified by by device name |
| 2572 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2573 |
|
| 2574 |
For all DVD writers, and for CD writers on certain platforms, you will |
| 2575 |
configure your writer device using only a device name. If your writer |
| 2576 |
device works this way, you should just specify <target_device> in |
| 2577 |
configuration. You can either leave <target_scsi_id> blank or remove it |
| 2578 |
completely. The writer device will be used both to write to the device |
| 2579 |
and for filesystem operations --- for instance, when the media needs |
| 2580 |
to be mounted to run the consistency check. |
| 2581 |
|
| 2582 |
Devices identified by SCSI id |
| 2583 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2584 |
|
| 2585 |
Cedar Backup can use devices identified by SCSI id only when configured |
| 2586 |
to use the ``cdwriter`` device type. |
| 2587 |
|
| 2588 |
In order to use a SCSI device with Cedar Backup, you must know both the |
| 2589 |
SCSI id <target_scsi_id> and the device name <target_device>. The SCSI |
| 2590 |
id will be used to write to media using ``cdrecord``; and the device |
| 2591 |
name will be used for other filesystem operations. |
| 2592 |
|
| 2593 |
A true SCSI device will always have an address ``scsibus,target,lun`` |
| 2594 |
(i.e. ``1,6,2``). This should hold true on most UNIX-like systems |
| 2595 |
including Linux and the various BSDs (although I do not have a BSD |
| 2596 |
system to test with currently). The SCSI address represents the location |
| 2597 |
of your writer device on the one or more SCSI buses that you have |
| 2598 |
available on your system. |
| 2599 |
|
| 2600 |
On some platforms, it is possible to reference non-SCSI writer devices |
| 2601 |
(i.e. an IDE CD writer) using an emulated SCSI id. If you have |
| 2602 |
configured your non-SCSI writer device to have an emulated SCSI id, |
| 2603 |
provide the filesystem device path in <target_device> and the SCSI id in |
| 2604 |
<target_scsi_id>, just like for a real SCSI device. |
| 2605 |
|
| 2606 |
You should note that in some cases, an emulated SCSI id takes the same |
| 2607 |
form as a normal SCSI id, while in other cases you might see a method |
| 2608 |
name prepended to the normal SCSI id (i.e. “ATA:1,1,1”). |
| 2609 |
|
| 2610 |
Linux Notes |
| 2611 |
~~~~~~~~~~~ |
| 2612 |
|
| 2613 |
On a Linux system, IDE writer devices often have a emulated SCSI |
| 2614 |
address, which allows SCSI-based software to access the device through |
| 2615 |
an IDE-to-SCSI interface. Under these circumstances, the first IDE |
| 2616 |
writer device typically has an address ``0,0,0``. However, support for |
| 2617 |
the IDE-to-SCSI interface has been deprecated and is not well-supported |
| 2618 |
in newer kernels (kernel 2.6.x and later). |
| 2619 |
|
| 2620 |
Newer Linux kernels can address ATA or ATAPI drives without SCSI |
| 2621 |
emulation by prepending a “method” indicator to the emulated device |
| 2622 |
address. For instance, ``ATA:0,0,0`` or ``ATAPI:0,0,0`` are typical |
| 2623 |
values. |
| 2624 |
|
| 2625 |
However, even this interface is deprecated as of late 2006, so with |
| 2626 |
relatively new kernels you may be better off using the filesystem device |
| 2627 |
path directly rather than relying on any SCSI emulation. |
| 2628 |
|
| 2629 |
Here are some hints about how to find your Linux CD writer hardware. |
| 2630 |
First, try to reference your device using the filesystem device path: |
| 2631 |
|
| 2632 |
:: |
| 2633 |
|
| 2634 |
cdrecord -prcap dev=/dev/cdrom |
| 2635 |
|
| 2636 |
|
| 2637 |
Running this command on my hardware gives output that looks like this |
| 2638 |
(just the top few lines): |
| 2639 |
|
| 2640 |
:: |
| 2641 |
|
| 2642 |
Device type : Removable CD-ROM |
| 2643 |
Version : 0 |
| 2644 |
Response Format: 2 |
| 2645 |
Capabilities : |
| 2646 |
Vendor_info : 'LITE-ON ' |
| 2647 |
Identification : 'DVDRW SOHW-1673S' |
| 2648 |
Revision : 'JS02' |
| 2649 |
Device seems to be: Generic mmc2 DVD-R/DVD-RW. |
| 2650 |
|
| 2651 |
Drive capabilities, per MMC-3 page 2A: |
| 2652 |
|
| 2653 |
|
| 2654 |
If this works, and the identifying information at the top of the output |
| 2655 |
looks like your CD writer device, you've probably found a working |
| 2656 |
configuration. Place the device path into <target_device> and leave |
| 2657 |
<target_scsi_id> blank. |
| 2658 |
|
| 2659 |
If this doesn't work, you should try to find an ATA or ATAPI device: |
| 2660 |
|
| 2661 |
:: |
| 2662 |
|
| 2663 |
cdrecord -scanbus dev=ATA |
| 2664 |
cdrecord -scanbus dev=ATAPI |
| 2665 |
|
| 2666 |
|
| 2667 |
On my development system, I get a result that looks something like this |
| 2668 |
for ATA: |
| 2669 |
|
| 2670 |
:: |
| 2671 |
|
| 2672 |
scsibus1: |
| 2673 |
1,0,0 100) 'LITE-ON ' 'DVDRW SOHW-1673S' 'JS02' Removable CD-ROM |
| 2674 |
1,1,0 101) * |
| 2675 |
1,2,0 102) * |
| 2676 |
1,3,0 103) * |
| 2677 |
1,4,0 104) * |
| 2678 |
1,5,0 105) * |
| 2679 |
1,6,0 106) * |
| 2680 |
1,7,0 107) * |
| 2681 |
|
| 2682 |
|
| 2683 |
Again, if you get a result that you recognize, you have again probably |
| 2684 |
found a working configuraton. Place the associated device path (in my |
| 2685 |
case, ``/dev/cdrom``) into <target_device> and put the emulated SCSI id |
| 2686 |
(in this case, ``ATA:1,0,0``) into <target_scsi_id>. |
| 2687 |
|
| 2688 |
Any further discussion of how to configure your CD writer hardware is |
| 2689 |
outside the scope of this document. If you have tried the hints above |
| 2690 |
and still can't get things working, you may want to reference the Linux |
| 2691 |
CDROM HOWTO (`<http://www.tldp.org/HOWTO/CDROM-HOWTO>`__) or the ATA |
| 2692 |
RAID HOWTO (`<http://www.tldp.org/HOWTO/ATA-RAID-HOWTO/index.html>`__) |
| 2693 |
for more information. |
| 2694 |
|
| 2695 |
Mac OS X Notes |
| 2696 |
~~~~~~~~~~~~~~ |
| 2697 |
|
| 2698 |
On a Mac OS X (darwin) system, things get strange. Apple has abandoned |
| 2699 |
traditional SCSI device identifiers in favor of a system-wide resource |
| 2700 |
id. So, on a Mac, your writer device will have a name something like |
| 2701 |
``IOCompactDiscServices`` (for a CD writer) or ``IODVDServices`` (for a |
| 2702 |
DVD writer). If you have multiple drives, the second drive probably has |
| 2703 |
a number appended, i.e. ``IODVDServices/2`` for the second DVD writer. |
| 2704 |
You can try to figure out what the name of your device is by grepping |
| 2705 |
through the output of the command ``ioreg -l``. [6]_ |
| 2706 |
|
| 2707 |
Unfortunately, even if you can figure out what device to use, I can't |
| 2708 |
really support the store action on this platform. In OS X, the |
| 2709 |
“automount” function of the Finder interferes significantly with Cedar |
| 2710 |
Backup's ability to mount and unmount media and write to the CD or DVD |
| 2711 |
hardware. The Cedar Backup writer and image functionality does work on |
| 2712 |
this platform, but the effort required to fight the operating system |
| 2713 |
about who owns the media and the device makes it nearly impossible to |
| 2714 |
execute the store action successfully. |
| 2715 |
|
| 2716 |
.. _cedar-config-blanking: |
| 2717 |
|
| 2718 |
Optimized Blanking Stategy |
| 2719 |
-------------------------- |
| 2720 |
|
| 2721 |
When the optimized blanking strategy has not been configured, Cedar |
| 2722 |
Backup uses a simplistic approach: rewritable media is blanked at the |
| 2723 |
beginning of every week, period. |
| 2724 |
|
| 2725 |
Since rewritable media can be blanked only a finite number of times |
| 2726 |
before becoming unusable, some users --- especially users of |
| 2727 |
rewritable DVD media with its large capacity --- may prefer to blank |
| 2728 |
the media less often. |
| 2729 |
|
| 2730 |
If the optimized blanking strategy is configured, Cedar Backup will use |
| 2731 |
a blanking factor and attempt to determine whether future backups will |
| 2732 |
fit on the current media. If it looks like backups will fit, then the |
| 2733 |
media will not be blanked. |
| 2734 |
|
| 2735 |
This feature will only be useful (assuming single disc is used for the |
| 2736 |
whole week's backups) if the estimated total size of the weekly backup |
| 2737 |
is considerably smaller than the capacity of the media (no more than 50% |
| 2738 |
of the total media capacity), and only if the size of the backup can be |
| 2739 |
expected to remain fairly constant over time (no frequent rapid growth |
| 2740 |
expected). |
| 2741 |
|
| 2742 |
There are two blanking modes: daily and weekly. If the weekly blanking |
| 2743 |
mode is set, Cedar Backup will only estimate future capacity (and |
| 2744 |
potentially blank the disc) once per week, on the starting day of the |
| 2745 |
week. If the daily blanking mode is set, Cedar Backup will estimate |
| 2746 |
future capacity (and potentially blank the disc) every time it is run. |
| 2747 |
*You should only use the daily blanking mode in conjunction with daily |
| 2748 |
collect configuration, otherwise you will risk losing data.* |
| 2749 |
|
| 2750 |
If you are using the daily blanking mode, you can typically set the |
| 2751 |
blanking value to 1.0. This will cause Cedar Backup to blank the media |
| 2752 |
whenever there is not enough space to store the current day's backup. |
| 2753 |
|
| 2754 |
If you are using the weekly blanking mode, then finding the correct |
| 2755 |
blanking factor will require some experimentation. Cedar Backup |
| 2756 |
estimates future capacity based on the configured blanking factor. The |
| 2757 |
disc will be blanked if the following relationship is true: |
| 2758 |
|
| 2759 |
:: |
| 2760 |
|
| 2761 |
bytes available / (1 + bytes required) LE blanking factor |
| 2762 |
|
| 2763 |
|
| 2764 |
Another way to look at this is to consider the blanking factor as a sort |
| 2765 |
of (upper) backup growth estimate: |
| 2766 |
|
| 2767 |
:: |
| 2768 |
|
| 2769 |
Total size of weekly backup / Full backup size at the start of the week |
| 2770 |
|
| 2771 |
|
| 2772 |
This ratio can be estimated using a week or two of previous backups. For |
| 2773 |
instance, take this example, where March 10 is the start of the week and |
| 2774 |
March 4 through March 9 represent the incremental backups from the |
| 2775 |
previous week: |
| 2776 |
|
| 2777 |
:: |
| 2778 |
|
| 2779 |
/opt/backup/staging# du -s 2007/03/* |
| 2780 |
3040 2007/03/01 |
| 2781 |
3044 2007/03/02 |
| 2782 |
6812 2007/03/03 |
| 2783 |
3044 2007/03/04 |
| 2784 |
3152 2007/03/05 |
| 2785 |
3056 2007/03/06 |
| 2786 |
3060 2007/03/07 |
| 2787 |
3056 2007/03/08 |
| 2788 |
4776 2007/03/09 |
| 2789 |
6812 2007/03/10 |
| 2790 |
11824 2007/03/11 |
| 2791 |
|
| 2792 |
|
| 2793 |
In this case, the ratio is approximately 4: |
| 2794 |
|
| 2795 |
:: |
| 2796 |
|
| 2797 |
6812 + (3044 + 3152 + 3056 + 3060 + 3056 + 4776) / 6812 = 3.9571 |
| 2798 |
|
| 2799 |
|
| 2800 |
To be safe, you might choose to configure a factor of 5.0. |
| 2801 |
|
| 2802 |
Setting a higher value reduces the risk of exceeding media capacity |
| 2803 |
mid-week but might result in blanking the media more often than is |
| 2804 |
necessary. |
| 2805 |
|
| 2806 |
If you run out of space mid-week, then the solution is to run the |
| 2807 |
rebuild action. If this happens frequently, a higher blanking factor |
| 2808 |
value should be used. |
| 2809 |
|
| 2810 |
---------- |
| 2811 |
|
| 2812 |
*Previous*: :doc:`commandline` • *Next*: :doc:`extensions` |
| 2813 |
|
| 2814 |
---------- |
| 2815 |
|
| 2816 |
.. [1] |
| 2817 |
See `<http://www.xml.com/pub/a/98/10/guide0.html>`__ for a basic |
| 2818 |
introduction to XML. |
| 2819 |
|
| 2820 |
.. [2] |
| 2821 |
See :doc:`basic` |
| 2822 |
|
| 2823 |
.. [3] |
| 2824 |
See `<http://docs.python.org/lib/re-syntax.html>`__ |
| 2825 |
|
| 2826 |
.. [4] |
| 2827 |
See `<https://github.com/pronovic/cedar-backup3/issues>`__. |
| 2828 |
|
| 2829 |
.. [5] |
| 2830 |
See :doc:`basic` |
| 2831 |
|
| 2832 |
.. [6] |
| 2833 |
Thanks to the file README.macosX in the cdrtools-2.01+01a01 source |
| 2834 |
tree for this information |
| 2835 |
|
| 2836 |
.. |note| image:: images/note.png |
| 2837 |
.. |tip| image:: images/tip.png |
| 2838 |
.. |warning| image:: images/warning.png |