| 1 |
[[ch-fundamentals]] |
| 2 |
== DRBD Fundamentals |
| 3 |
|
| 4 |
DRBD is a software-based, |
| 5 |
shared-nothing, replicated storage solution mirroring the content of |
| 6 |
block devices (hard disks, partitions, logical volumes etc.) between |
| 7 |
hosts. |
| 8 |
|
| 9 |
DRBD mirrors data |
| 10 |
|
| 11 |
* *in real time*. Replication occurs continuously while applications |
| 12 |
modify the data on the device. |
| 13 |
|
| 14 |
* *transparently*. Applications need not be aware that the data is stored on |
| 15 |
multiple hosts. |
| 16 |
|
| 17 |
* *synchronously* or *asynchronously*. With synchronous mirroring, applications |
| 18 |
are notified of write completions after the writes have been carried out on |
| 19 |
all (connected) hosts. With asynchronous mirroring, applications are notified of write |
| 20 |
completions when the writes have completed locally, which usually is before |
| 21 |
they have propagated to the other hosts. |
| 22 |
|
| 23 |
|
| 24 |
[[s-kernel-module]] |
| 25 |
=== Kernel module |
| 26 |
|
| 27 |
DRBD's core functionality is implemented by way of a Linux kernel |
| 28 |
module. Specifically, DRBD constitutes a driver for a virtual block |
| 29 |
device, so DRBD is situated right near the bottom of a system's I/O |
| 30 |
stack. Because of this, DRBD is extremely flexible and versatile, |
| 31 |
which makes it a replication solution suitable for adding high |
| 32 |
availability to just about any application. |
| 33 |
|
| 34 |
DRBD is, by definition and as mandated by the Linux kernel |
| 35 |
architecture, agnostic of the layers above it. Thus, it is impossible |
| 36 |
for DRBD to miraculously add features to upper layers that these do |
| 37 |
not possess. For example, DRBD cannot auto-detect file system |
| 38 |
corruption or add active-active clustering capability to file systems |
| 39 |
like ext3 or XFS. |
| 40 |
|
| 41 |
[[f-drbd-linux-io-stack]] |
| 42 |
.DRBD's position within the Linux I/O stack |
| 43 |
image::images/drbd-in-kernel.svg[] |
| 44 |
|
| 45 |
[[s-userland]] |
| 46 |
=== User space administration tools === |
| 47 |
|
| 48 |
DRBD comes with a set of administration tools which communicate with the |
| 49 |
kernel module in order to configure and administer DRBD resources. From |
| 50 |
top-level to bottom-most these are: |
| 51 |
|
| 52 |
.`drbdadm` |
| 53 |
The high-level administration tool of the DRBD-utils program suite. Obtains all DRBD |
| 54 |
configuration parameters from the configuration file `/etc/drbd.conf` and acts |
| 55 |
as a front-end for `drbdsetup` and `drbdmeta`. `drbdadm` has a _dry-run_ mode, |
| 56 |
invoked with the `-d` option, that shows which `drbdsetup` and `drbdmeta` calls |
| 57 |
`drbdadm` would issue without actually calling those commands. |
| 58 |
|
| 59 |
.`drbdsetup` |
| 60 |
Configures the DRBD module that was loaded into the kernel. All parameters to |
| 61 |
`drbdsetup` must be passed on the command line. The separation between |
| 62 |
`drbdadm` and `drbdsetup` allows for maximum flexibility. Most users will |
| 63 |
rarely need to use `drbdsetup` directly, if at all. |
| 64 |
|
| 65 |
.`drbdmeta` |
| 66 |
Allows to create, dump, restore, and modify DRBD meta data structures. Like |
| 67 |
`drbdsetup`, most users will only rarely need to use `drbdmeta` directly. |
| 68 |
|
| 69 |
[[s-resources]] |
| 70 |
=== Resources === |
| 71 |
|
| 72 |
In DRBD, _resource_ is the collective term that refers to all aspects of |
| 73 |
a particular replicated data set. These include: |
| 74 |
|
| 75 |
.Resource name |
| 76 |
This can be any arbitrary, US-ASCII name not containing white space by |
| 77 |
which the resource is referred to. |
| 78 |
|
| 79 |
.Volumes |
| 80 |
Any resource is a replication group consisting of one of more |
| 81 |
_volumes_ that share a common replication stream. DRBD ensures write |
| 82 |
fidelity across all volumes in the resource. Volumes are numbered |
| 83 |
starting with `0`, and there may be up to 65,535 volumes in one |
| 84 |
resource. A volume contains the replicated data set, and a set of |
| 85 |
metadata for DRBD internal use. |
| 86 |
|
| 87 |
At the `drbdadm` level, a volume within a resource can be addressed by the |
| 88 |
resource name and volume number as `__resource__/__volume__`. |
| 89 |
|
| 90 |
// At the `drbdsetup` level, a volume is addressed by its device minor number. |
| 91 |
// At the `drbdmeta` level, a volume is addressed by the name of the underlying |
| 92 |
// device. |
| 93 |
|
| 94 |
// FIXME: Users don't care which major device number is assigned to DRBD. |
| 95 |
// Likewise, they don't care about minor device numbers if they don't have to. |
| 96 |
// We refer to device as /dev/drbdX almost everywhere, so do we have to mention |
| 97 |
// minors here at all? |
| 98 |
|
| 99 |
.DRBD device |
| 100 |
This is a virtual block device managed by DRBD. It has a device major |
| 101 |
number of 147, and its minor numbers are numbered from 0 onwards, as |
| 102 |
is customary. Each DRBD device corresponds to a volume in a |
| 103 |
resource. The associated block device is usually named |
| 104 |
`/dev/drbd__X__`, where `_X_` is the device minor number. `udev` will typically |
| 105 |
also create symlinks containing the resource name and volume number, as in |
| 106 |
`/dev/drbd/by-res/__resource__/__vol-nr__`. Please note that depending on how |
| 107 |
you installed drbd, you might need to install the `drbd-udev` package on RPM based systems. |
| 108 |
|
| 109 |
NOTE: Very early DRBD versions hijacked NBD's device major number 43. |
| 110 |
This is long obsolete; 147 is the |
| 111 |
https://www.kernel.org/doc/html/latest/admin-guide/devices.html[allocated] DRBD device |
| 112 |
major. |
| 113 |
|
| 114 |
.Connection |
| 115 |
A _connection_ is a communication link between two hosts that share a |
| 116 |
replicated data set. With DRBD 9 each resource can be defined on |
| 117 |
multiple hosts; with the current versions this requires |
| 118 |
a full-meshed connection setup between these hosts (i.e. each host connected to |
| 119 |
every other for that resource) |
| 120 |
|
| 121 |
At the `drbdadm` level, a connection is addressed by the resource and the |
| 122 |
connection name (the latter defaulting to the peer hostname), like |
| 123 |
`__resource__:__connection__`. |
| 124 |
|
| 125 |
// At the `drbdsetup` level, a connection is addressed by its two replication |
| 126 |
// endpoints identified by address family (optional), address (required), and |
| 127 |
// port (optional). |
| 128 |
|
| 129 |
[[s-resource-roles]] |
| 130 |
=== Resource roles === |
| 131 |
|
| 132 |
indexterm:[Primary]indexterm:[Secondary]indexterm:[role]In DRBD, every <<s-resources,resource>> has a role, which may be |
| 133 |
_Primary_ or _Secondary_. |
| 134 |
|
| 135 |
NOTE: The choice of terms here is not arbitrary. These roles were |
| 136 |
deliberately not named "Active" and "Passive" by DRBD's |
| 137 |
creators. _Primary_ vs. _secondary_ refers to a concept related to |
| 138 |
availability of *storage*, whereas _active_ vs. _passive_ refers to the |
| 139 |
availability of an *application*. It is usually the case in a |
| 140 |
high-availability environment that the _primary_ node is also the _active_ |
| 141 |
one, but this is by no means necessary. |
| 142 |
|
| 143 |
* A DRBD device in the _primary_ role can be used unrestrictedly for |
| 144 |
read and write operations. It may be used for creating and mounting |
| 145 |
file systems, raw or direct I/O to the block device, etc. |
| 146 |
|
| 147 |
* A DRBD device in the _secondary_ role receives all updates from the |
| 148 |
peer node's device, but otherwise disallows access completely. It |
| 149 |
can not be used by applications, neither for read nor write |
| 150 |
access. The reason for disallowing even read-only access to the |
| 151 |
device is the necessity to maintain cache coherency, which would be |
| 152 |
impossible if a secondary resource were made accessible in any way. |
| 153 |
|
| 154 |
The resource's role can, of course, be changed, either by |
| 155 |
<<s-switch-resource-roles,manual intervention>>, by way of some |
| 156 |
automated algorithm by a cluster management application, or <<s-automatic-promotion,automatically>>. Changing the |
| 157 |
resource role from secondary to primary is referred to as _promotion_, |
| 158 |
whereas the reverse operation is termed _demotion_. |