Debian Codegraph @

UG9/en/pacemaker.adoc

raw
1 // FIXME
2 [[ch-pacemaker]]
3 == Integrating DRBD with Pacemaker clusters
4
5 indexterm:[Pacemaker]Using DRBD in conjunction with the Pacemaker
6 cluster stack is arguably DRBD's most frequently found use
7 case. Pacemaker is also one of the applications that make DRBD
8 extremely powerful in a wide variety of usage scenarios.
9
10 DRBD can be used in Pacemaker clusters in different ways:
11
12 * DRBD running as a background-service, used like a SAN; or
13 * DRBD completely managed by Pacemaker via the DRBD OCF resource agent
14
15 Both have a few advantages and disadvantages, these will be discussed below.
16
17 NOTE: It's recommended to have either fencing configured or quorum enabled.
18 (But not both. External fencing handler results may interact in conflicting ways
19 with DRBD internal quorum.)
20 If your cluster has communication issues (e.g. network switch loses
21 power) and gets split, the parts might start the services (failover)
22 and cause a *Split-Brain* when the communication resumes again.
23
24
25
26 [[s-pacemaker-primer]]
27 === Pacemaker primer
28
29 Pacemaker is a sophisticated, feature-rich, and widely deployed
30 cluster resource manager for the Linux platform. It comes with a rich
31 set of documentation. In order to understand this chapter, reading the
32 following documents is highly recommended:
33
34 * http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf[Clusters
35 From Scratch], a step-by-step guide to configuring high availability
36 clusters;
37 * http://crmsh.github.io/documentation/index.html[CRM CLI (command line
38 interface) tool], a manual for the CRM shell, a simple and intuitive
39 command line interface bundled with Pacemaker;
40 * http://clusterlabs.org/doc/en-US/Pacemaker/1.1/html/Pacemaker_Explained/[Pacemaker
41 Configuration Explained], a reference document explaining the
42 concept and design behind Pacemaker.
43
44
45 [[s-pacemaker-drbd-background]]
46 === Using DRBD as a background service in a pacemaker cluster
47
48 In this section you will see that using autonomous DRBD storage can look
49 like local storage; so integrating in a Pacemaker cluster is done by
50 pointing your mount points at DRBD.
51
52 First of all, we will use the `auto-promote` feature of DRBD, so that
53 DRBD automatically sets itself _Primary_ when needed. This will probably
54 apply to all of your resources, so setting that a default in the
55 `common` section makes sense:
56
57 [source,drbd]
58 ----------------------------
59 common {
60 options {
61 auto-promote yes;
62 ...
63 }
64 }
65 ----------------------------
66
67 Now you just need to use your storage, e.g. via a filesystem:
68
69 .Pacemaker configuration for DRBD-backed MySQL service, using `auto-promote`
70 ----------------------------
71 crm configure
72 crm(live)configure# primitive fs_mysql ocf:heartbeat:Filesystem \
73 params device="/dev/drbd/by-res/mysql/0" \
74 directory="/var/lib/mysql" fstype="ext3"
75 crm(live)configure# primitive ip_mysql ocf:heartbeat:IPaddr2 \
76 params ip="10.9.42.1" nic="eth0"
77 crm(live)configure# primitive mysqld lsb:mysqld
78 crm(live)configure# group mysql fs_mysql ip_mysql mysqld
79 crm(live)configure# commit
80 crm(live)configure# exit
81 bye
82 ----------------------------
83
84 Essentially all that is needed is a mountpoint (`/var/lib/mysql` in this
85 example) where the DRBD resource gets mounted.
86
87 As long as Pacemaker has control, it will only allow a single instance
88 of that mount across your cluster.
89
90 See also <<s-pacemaker-drbd-attr>> for additional information on ordering
91 constraints for system startup and more.
92
93 [[s-pacemaker-crm-drbd-backed-service]]
94 === Adding a DRBD-backed service to the cluster configuration, including a master-slave resource
95
96 This section explains how to enable a DRBD-backed service in a
97 Pacemaker cluster.
98
99 NOTE: If you are employing the DRBD OCF resource agent, it is
100 recommended that you defer DRBD startup, shutdown, promotion, and
101 demotion _exclusively_ to the OCF resource agent. That means that you
102 should disable the DRBD init script:
103
104 ----------------------------
105 chkconfig drbd off
106 ----------------------------
107
108 The `ocf:linbit:drbd` OCF resource agent provides Master/Slave
109 capability, allowing Pacemaker to start and monitor the DRBD resource
110 on multiple nodes and promoting and demoting as needed. You must,
111 however, understand that the DRBD RA disconnects and detaches all
112 DRBD resources it manages on Pacemaker shutdown, and also upon
113 enabling standby mode for a node.
114
115
116 IMPORTANT: The OCF resource agent which ships with DRBD belongs to the
117 `linbit` provider, and hence installs as
118 `/usr/lib/ocf/resource.d/linbit/drbd`. There is a legacy resource
119 agent that ships as part of the OCF resource agents package, which
120 uses the `heartbeat` provider and installs into
121 `/usr/lib/ocf/resource.d/heartbeat/drbd`. The legacy OCF RA is
122 deprecated and should no longer be used.
123
124 In order to enable a DRBD-backed configuration for a MySQL database in
125 a Pacemaker CRM cluster with the `drbd` OCF resource agent, you must
126 create both the necessary resources, and Pacemaker constraints to
127 ensure your service only starts on a previously promoted DRBD
128 resource. You may do so using the `crm` shell, as outlined in the
129 following example:
130
131 .Pacemaker configuration for DRBD-backed MySQL service, using a `master-slave` resource
132 ----------------------------
133 crm configure
134 crm(live)configure# primitive drbd_mysql ocf:linbit:drbd \
135 params drbd_resource="mysql" \
136 op monitor interval="29s" role="Master" \
137 op monitor interval="31s" role="Slave"
138 crm(live)configure# ms ms_drbd_mysql drbd_mysql \
139 meta master-max="1" master-node-max="1" \
140 clone-max="2" clone-node-max="1" \
141 notify="true"
142 crm(live)configure# primitive fs_mysql ocf:heartbeat:Filesystem \
143 params device="/dev/drbd/by-res/mysql/0" \
144 directory="/var/lib/mysql" fstype="ext3"
145 crm(live)configure# primitive ip_mysql ocf:heartbeat:IPaddr2 \
146 params ip="10.9.42.1" nic="eth0"
147 crm(live)configure# primitive mysqld lsb:mysqld
148 crm(live)configure# group mysql fs_mysql ip_mysql mysqld
149 crm(live)configure# colocation mysql_on_drbd \
150 inf: mysql ms_drbd_mysql:Master
151 crm(live)configure# order mysql_after_drbd \
152 inf: ms_drbd_mysql:promote mysql:start
153 crm(live)configure# commit
154 crm(live)configure# exit
155 bye
156 ----------------------------
157
158 After this, your configuration should be enabled. Pacemaker now
159 selects a node on which it promotes the DRBD resource, and then starts
160 the DRBD-backed resource group on that same node.
161
162 See also <<s-pacemaker-drbd-attr>> for additional information on
163 location contraints for placing the Master role.
164
165 [[s-pacemaker-fencing]]
166 === Using resource-level fencing in Pacemaker clusters
167
168 This section outlines the steps necessary to prevent Pacemaker from
169 promoting a DRBD Master/Slave resource when its DRBD replication link
170 has been interrupted. This keeps Pacemaker from starting a service
171 with outdated data and causing an unwanted "time warp" in the
172 process.
173
174 In order to enable any resource-level fencing for DRBD, you must add
175 the following lines to your resource configuration:
176
177 [source,drbd]
178 ----------------------------
179 resource <resource> {
180 net {
181 fencing resource-only;
182 ...
183 }
184 }
185 ----------------------------
186
187 You will also have to make changes to the `handlers` section depending
188 on the cluster infrastructure being used.
189
190 Corosync-based Pacemaker clusters can use the
191 functionality explained in <<s-pacemaker-fencing-cib>>.
192
193 IMPORTANT: It is absolutely vital to configure at least two independent cluster
194 communications channels for this functionality to work correctly. Corosync
195 clusters should list at least two redundant rings in `corosync.conf`,
196 respectively several paths for knet.
197
198
199 [[s-pacemaker-fencing-cib]]
200 ==== Resource-level fencing using the Cluster Information Base (CIB)
201
202 In order to enable resource-level fencing for Pacemaker, you will have
203 to set two options in `drbd.conf`:
204
205 [source,drbd]
206 ----------------------------
207 resource <resource> {
208 net {
209 fencing resource-only;
210 ...
211 }
212 handlers {
213 fence-peer "/usr/lib/drbd/crm-fence-peer.9.sh";
214 unfence-peer "/usr/lib/drbd/crm-unfence-peer.9.sh";
215 # Note: we used to abuse the after-resync-target handler to do the
216 # unfence, but since 2016 have a dedicated unfence-peer handler.
217 # Using the after-resync-target handler is wrong in some corner cases.
218 ...
219 }
220 ...
221 }
222 ----------------------------
223
224 Thus, if the DRBD replication link becomes disconnected, the
225 `crm-fence-peer.9.sh` script contacts the cluster manager, determines the
226 Pacemaker Master/Slave resource associated with this DRBD resource,
227 and ensures that the Master/Slave resource no longer gets promoted on
228 any node other than the currently active one. Conversely, when the
229 connection is re-established and DRBD completes its synchronization
230 process, then that constraint is removed and the cluster manager is
231 free to promote the resource on any node again.
232
233 [[s-pacemaker-stacked-resources]]
234 === Using stacked DRBD resources in Pacemaker clusters
235
236 NOTE: Stacking is deprecated in DRBD version
237 9.x, as more nodes can be implemented on a single level. See
238 <<s-drbdconf-conns>> for details.
239
240 Stacked resources allow DRBD to be used for multi-level redundancy in
241 multiple-node clusters, or to establish off-site disaster recovery
242 capability. This section describes how to configure DRBD and Pacemaker
243 in such configurations.
244
245
246 [[s-pacemaker-stacked-dr]]
247 ==== Adding off-site disaster recovery to Pacemaker clusters
248
249 In this configuration scenario, we would deal with a two-node high
250 availability cluster in one site, plus a separate node which would
251 presumably be housed off-site. The third node acts as a disaster
252 recovery node and is a standalone server. Consider the following
253 illustration to describe the concept.
254
255 .DRBD resource stacking in Pacemaker clusters
256 image::images/drbd-resource-stacking-pacemaker-3nodes.svg[]
257
258 In this example, 'alice' and 'bob' form a two-node Pacemaker cluster,
259 whereas 'charlie' is an off-site node not managed by Pacemaker.
260
261 To create such a configuration, you would first configure and
262 initialize DRBD resources as described in <<s-three-nodes>>. Then,
263 configure Pacemaker with the following CRM configuration:
264
265 [source,drbd]
266 ----------------------------
267 primitive p_drbd_r0 ocf:linbit:drbd \
268 params drbd_resource="r0"
269
270 primitive p_drbd_r0-U ocf:linbit:drbd \
271 params drbd_resource="r0-U"
272
273 primitive p_ip_stacked ocf:heartbeat:IPaddr2 \
274 params ip="192.168.42.1" nic="eth0"
275
276 ms ms_drbd_r0 p_drbd_r0 \
277 meta master-max="1" master-node-max="1" \
278 clone-max="2" clone-node-max="1" \
279 notify="true" globally-unique="false"
280
281 ms ms_drbd_r0-U p_drbd_r0-U \
282 meta master-max="1" clone-max="1" \
283 clone-node-max="1" master-node-max="1" \
284 notify="true" globally-unique="false"
285
286 colocation c_drbd_r0-U_on_drbd_r0 \
287 inf: ms_drbd_r0-U ms_drbd_r0:Master
288
289 colocation c_drbd_r0-U_on_ip \
290 inf: ms_drbd_r0-U p_ip_stacked
291
292 colocation c_ip_on_r0_master \
293 inf: p_ip_stacked ms_drbd_r0:Master
294
295 order o_ip_before_r0-U \
296 inf: p_ip_stacked ms_drbd_r0-U:start
297
298 order o_drbd_r0_before_r0-U \
299 inf: ms_drbd_r0:promote ms_drbd_r0-U:start
300 ----------------------------
301
302 Assuming you created this configuration in a temporary file named
303 `/tmp/crm.txt`, you may import it into the live cluster configuration
304 with the following command:
305
306 ----------------------------
307 crm configure < /tmp/crm.txt
308 ----------------------------
309
310 This configuration will ensure that the following actions occur in the
311 correct order on the 'alice'/'bob' cluster:
312
313 . Pacemaker starts the DRBD resource `r0` on both cluster nodes, and
314 promotes one node to the Master (DRBD Primary) role.
315
316 . Pacemaker then starts the IP address 192.168.42.1, which the stacked
317 resource is to use for replication to the third node. It does so on
318 the node it has previously promoted to the Master role for `r0` DRBD
319 resource.
320
321 . On the node which now has the Primary role for `r0` and also the
322 replication IP address for `r0-U`, Pacemaker now starts the
323 `r0-U` DRBD resource, which connects and replicates to the off-site
324 node.
325
326 . Pacemaker then promotes the `r0-U` resource to the Primary role too,
327 so it can be used by an application.
328
329 Thus, this Pacemaker configuration ensures that there is not only full
330 data redundancy between cluster nodes, but also to the third, off-site
331 node.
332
333 NOTE: This type of setup is usually deployed together with
334 <<s-drbd-proxy,DRBD Proxy>>.
335
336 [[s-pacemaker-stacked-4way]]
337 ==== Using stacked resources to achieve 4-way redundancy in Pacemaker clusters
338
339 In this configuration, a total of three DRBD resources (two unstacked,
340 one stacked) are used to achieve 4-way storage redundancy. This means
341 that of a 4-node cluster, up to three nodes can fail while still
342 providing service availability.
343
344 Consider the following illustration to explain the concept.
345
346 .DRBD resource stacking in Pacemaker clusters
347 image::images/drbd-resource-stacking-pacemaker-4nodes.svg[]
348
349 In this example, 'alice', 'bob', 'charlie', and 'daisy' form two
350 two-node Pacemaker clusters. 'alice' and 'bob' form the cluster named
351 `left` and replicate data using a DRBD resource between them, while
352 'charlie' and 'daisy' do the same with a separate DRBD resource, in a
353 cluster named `right`. A third, stacked DRBD resource connects the two
354 clusters.
355
356 NOTE: Due to limitations in the Pacemaker cluster manager as of
357 Pacemaker version 1.0.5, it is not possible to create this setup in a
358 single four-node cluster without disabling CIB validation, which is an
359 advanced process not recommended for general-purpose use. It is
360 anticipated that this is being addressed in future Pacemaker releases.
361
362 To create such a configuration, you would first configure and
363 initialize DRBD resources as described in <<s-three-nodes>> (except
364 that the remote half of the DRBD configuration is also stacked, not
365 just the local cluster). Then, configure Pacemaker with the following
366 CRM configuration, starting with the cluster `left`:
367
368 [source,drbd]
369 ----------------------------
370 primitive p_drbd_left ocf:linbit:drbd \
371 params drbd_resource="left"
372
373 primitive p_drbd_stacked ocf:linbit:drbd \
374 params drbd_resource="stacked"
375
376 primitive p_ip_stacked_left ocf:heartbeat:IPaddr2 \
377 params ip="10.9.9.100" nic="eth0"
378
379 ms ms_drbd_left p_drbd_left \
380 meta master-max="1" master-node-max="1" \
381 clone-max="2" clone-node-max="1" \
382 notify="true"
383
384 ms ms_drbd_stacked p_drbd_stacked \
385 meta master-max="1" clone-max="1" \
386 clone-node-max="1" master-node-max="1" \
387 notify="true" target-role="Master"
388
389 colocation c_ip_on_left_master \
390 inf: p_ip_stacked_left ms_drbd_left:Master
391
392 colocation c_drbd_stacked_on_ip_left \
393 inf: ms_drbd_stacked p_ip_stacked_left
394
395 order o_ip_before_stacked_left \
396 inf: p_ip_stacked_left ms_drbd_stacked:start
397
398 order o_drbd_left_before_stacked_left \
399 inf: ms_drbd_left:promote ms_drbd_stacked:start
400
401 ----------------------------
402
403 Assuming you created this configuration in a temporary file named
404 `/tmp/crm.txt`, you may import it into the live cluster configuration
405 with the following command:
406
407 ----------------------------
408 crm configure < /tmp/crm.txt
409 ----------------------------
410
411 After adding this configuration to the CIB, Pacemaker will execute the
412 following actions:
413
414 . Bring up the DRBD resource `left` replicating between 'alice' and
415 'bob' promoting the resource to the Master role on one of these nodes.
416
417 . Bring up the IP address 10.9.9.100 (on either 'alice' or 'bob',
418 depending on which of these holds the Master role for the resource
419 `left`).
420
421 . Bring up the DRBD resource `stacked` on the same node that holds the
422 just-configured IP address.
423
424 . Promote the stacked DRBD resource to the Primary role.
425
426 Now, proceed on the cluster `right` by creating the following
427 configuration:
428
429 [source,drbd]
430 ----------------------------
431 primitive p_drbd_right ocf:linbit:drbd \
432 params drbd_resource="right"
433
434 primitive p_drbd_stacked ocf:linbit:drbd \
435 params drbd_resource="stacked"
436
437 primitive p_ip_stacked_right ocf:heartbeat:IPaddr2 \
438 params ip="10.9.10.101" nic="eth0"
439
440 ms ms_drbd_right p_drbd_right \
441 meta master-max="1" master-node-max="1" \
442 clone-max="2" clone-node-max="1" \
443 notify="true"
444
445 ms ms_drbd_stacked p_drbd_stacked \
446 meta master-max="1" clone-max="1" \
447 clone-node-max="1" master-node-max="1" \
448 notify="true" target-role="Slave"
449
450 colocation c_drbd_stacked_on_ip_right \
451 inf: ms_drbd_stacked p_ip_stacked_right
452
453 colocation c_ip_on_right_master \
454 inf: p_ip_stacked_right ms_drbd_right:Master
455
456 order o_ip_before_stacked_right \
457 inf: p_ip_stacked_right ms_drbd_stacked:start
458
459 order o_drbd_right_before_stacked_right \
460 inf: ms_drbd_right:promote ms_drbd_stacked:start
461 ----------------------------
462
463 After adding this configuration to the CIB, Pacemaker will execute the
464 following actions:
465
466 . Bring up the DRBD resource `right` replicating between 'charlie' and
467 'daisy', promoting the resource to the Master role on one of these
468 nodes.
469
470 . Bring up the IP address 10.9.10.101 (on either 'charlie' or 'daisy',
471 depending on which of these holds the Master role for the resource
472 `right`).
473
474 . Bring up the DRBD resource `stacked` on the same node that holds the
475 just-configured IP address.
476
477 . Leave the stacked DRBD resource in the Secondary role (due to
478 `target-role="Slave"`).
479
480
481 [[s-pacemaker-floating-peers]]
482 === Configuring DRBD to replicate between two SAN-backed Pacemaker clusters
483
484 indexterm:[IP address, floating peers]This is a somewhat advanced setup usually employed in split-site
485 configurations. It involves two separate Pacemaker clusters, where
486 each cluster has access to a separate Storage Area Network (SAN). DRBD
487 is then used to replicate data stored on that SAN, across an IP link
488 between sites.
489
490 Consider the following illustration to describe the concept.
491
492 .Using DRBD to replicate between SAN-based clusters
493 image::images/drbd-pacemaker-floating-peers.svg[]
494
495 Which of the individual nodes in each site currently acts as the DRBD
496 peer is not explicitly defined -- the DRBD peers
497 <<s-floating-peers,are said to _float_>>; that is, DRBD binds to
498 virtual IP addresses not tied to a specific physical machine.
499
500
501 NOTE: This type of setup is usually deployed together with
502 <<s-drbd-proxy,DRBD Proxy>> and/or <<s-truck-based-replication,truck
503 based replication>>.
504
505 Since this type of setup deals with shared storage, configuring and
506 testing STONITH is absolutely vital for it to work properly.
507
508
509 [[s-pacemaker-floating-peers-drbd-config]]
510 ==== DRBD resource configuration
511
512 To enable your DRBD resource to float, configure it in `drbd.conf` in
513 the following fashion:
514
515 [source,drbd]
516 ----------------------------
517 resource <resource> {
518 ...
519 device /dev/drbd0;
520 disk /dev/sda1;
521 meta-disk internal;
522 floating 10.9.9.100:7788;
523 floating 10.9.10.101:7788;
524 }
525 ----------------------------
526
527 The `floating` keyword replaces the `on <host>` sections normally
528 found in the resource configuration. In this mode, DRBD identifies
529 peers by IP address and TCP port, rather than by host name. It is
530 important to note that the addresses specified must be virtual cluster
531 IP addresses, rather than physical node IP addresses, for floating to
532 function properly. As shown in the example, in split-site
533 configurations the two floating addresses can be expected to belong to
534 two separate IP networks -- it is thus vital for routers and firewalls
535 to properly allow DRBD replication traffic between the nodes.
536
537
538 [[s-pacemaker-floating-peers-crm-config]]
539 ==== Pacemaker resource configuration
540
541 A DRBD floating peers setup, in terms of Pacemaker configuration,
542 involves the following items (in each of the two Pacemaker clusters
543 involved):
544
545 * A virtual cluster IP address.
546
547 * A master/slave DRBD resource (using the DRBD OCF resource agent).
548
549 * Pacemaker constraints ensuring that resources are started on the
550 correct nodes, and in the correct order.
551
552 To configure a resource named `mysql` in a floating peers
553 configuration in a 2-node cluster, using the replication address
554 `10.9.9.100`, configure Pacemaker with the following `crm` commands:
555
556 ----------------------------
557 crm configure
558 crm(live)configure# primitive p_ip_float_left ocf:heartbeat:IPaddr2 \
559 params ip=10.9.9.100
560 crm(live)configure# primitive p_drbd_mysql ocf:linbit:drbd \
561 params drbd_resource=mysql
562 crm(live)configure# ms ms_drbd_mysql drbd_mysql \
563 meta master-max="1" master-node-max="1" \
564 clone-max="1" clone-node-max="1" \
565 notify="true" target-role="Master"
566 crm(live)configure# order drbd_after_left \
567 inf: p_ip_float_left ms_drbd_mysql
568 crm(live)configure# colocation drbd_on_left \
569 inf: ms_drbd_mysql p_ip_float_left
570 crm(live)configure# commit
571 bye
572 ----------------------------
573
574 After adding this configuration to the CIB, Pacemaker will execute the
575 following actions:
576
577 . Bring up the IP address 10.9.9.100 (on either 'alice' or 'bob').
578 . Bring up the DRBD resource according to the IP address configured.
579 . Promote the DRBD resource to the Primary role.
580
581 Then, in order to create the matching configuration in the other
582 cluster, configure _that_ Pacemaker instance with the following
583 commands:
584
585 ----------------------------
586 crm configure
587 crm(live)configure# primitive p_ip_float_right ocf:heartbeat:IPaddr2 \
588 params ip=10.9.10.101
589 crm(live)configure# primitive drbd_mysql ocf:linbit:drbd \
590 params drbd_resource=mysql
591 crm(live)configure# ms ms_drbd_mysql drbd_mysql \
592 meta master-max="1" master-node-max="1" \
593 clone-max="1" clone-node-max="1" \
594 notify="true" target-role="Slave"
595 crm(live)configure# order drbd_after_right \
596 inf: p_ip_float_right ms_drbd_mysql
597 crm(live)configure# colocation drbd_on_right
598 inf: ms_drbd_mysql p_ip_float_right
599 crm(live)configure# commit
600 bye
601 ----------------------------
602
603 After adding this configuration to the CIB, Pacemaker will execute the
604 following actions:
605
606 . Bring up the IP address 10.9.10.101 (on either 'charlie' or
607 'daisy').
608 . Bring up the DRBD resource according to the IP address configured.
609 . Leave the DRBD resource in the Secondary role (due to
610 `target-role="Slave"`).
611
612
613 [[s-pacemaker-floating-peers-site-fail-over]]
614 ==== Site fail-over
615
616 In split-site configurations, it may be necessary to transfer services
617 from one site to another. This may be a consequence of a scheduled
618 transition, or of a disastrous event. In case the transition is a
619 normal, anticipated event, the recommended course of action is this:
620
621 * Connect to the cluster on the site about to relinquish resources,
622 and change the affected DRBD resource's `target-role` attribute from
623 `Master` to `Slave`. This will shut down any resources depending on
624 the Primary role of the DRBD resource, demote it, and continue to
625 run, ready to receive updates from a new Primary.
626
627 * Connect to the cluster on the site about to take over resources, and
628 change the affected DRBD resource's `target-role` attribute from
629 `Slave` to `Master`. This will promote the DRBD resources, start any
630 other Pacemaker resources depending on the Primary role of the DRBD
631 resource, and replicate updates to the remote site.
632
633 * To fail back, simply reverse the procedure.
634
635 In the event that of a catastrophic outage on the active site, it can
636 be expected that the site is off line and no longer replicated to the
637 backup site. In such an event:
638
639 * Connect to the cluster on the still-functioning site resources, and
640 change the affected DRBD resource's `target-role` attribute from
641 `Slave` to `Master`. This will promote the DRBD resources, and start
642 any other Pacemaker resources depending on the Primary role of the
643 DRBD resource.
644
645 * When the original site is restored or rebuilt, you may connect the
646 DRBD resources again, and subsequently fail back using the reverse
647 procedure.
648
649 [[s-pacemaker-drbd-attr]]
650 === Importing DRBD's promotion scores into the CIB
651
652 IMPORTANT: Everything described in this section depends on the `drbd-attr`
653 OCF resource agent. It is available since drbd-utils version 9.15.0. On
654 Debian/Ubuntu systems this is part of the `drbd-utils` package. On RPM
655 based Linux distributions you need to install the `drbd-pacemaker` package.
656
657 Every DRBD resource exposes a _promotion score_ on each node where it is
658 configured. It is a numeric value that might be 0 or positive. The value
659 reflects how desirable it is to promote the resource to master on this
660 particular node. A node that has an _UpToDate_ disk and two _UpToDate_
661 replicas has a higher score than a node with an _UpToDate_ disk and
662 just one _UpToDate_ replica.
663
664 During startup, the _promotion score_ is 0. E.g., before the DRBD device has
665 its backing device attached, or, if quorum is enabled, before quorum is
666 gained. A value of 0 indicates that a promotion request will fail,
667 and is mapped to a pacemaker score that indicates _must not run here_.
668
669 The `drbd-attr` OCF resource agent imports these promotion scores into node
670 attributes of a Pacemaker cluster. It needs to be configured like this:
671
672 [source,drbd]
673 ----------------------------
674 primitive drbd-attr ocf:linbit:drbd-attr
675 clone drbd-attr-clone drbd-attr
676 ----------------------------
677
678 These are _transient_ attributes (have a _lifetime_ of _reboot_ in pacemaker speak). That means, after
679 a _reboot_ of the node, or local restart of pacemaker, those attributes will not exist until an instance of `drbd-attr`
680 is started on that node.
681
682 You can inspect the generated attributes with `crm_mon -A -1`.
683
684 These attributed can be used in constraints for services that depend on the
685 DRBD devices, or, when managing DRBD with the `ocf:linbit:drbd` resource agent,
686 for the _Master_ role of that DRBD instance.
687
688 Here is an example location constraint for the example resource from <<s-pacemaker-drbd-background>>
689 [source,drbd]
690 ----------------------------
691 location lo_fs_mysql fs_mysql \
692 rule -inf: not_defined drbd-promotion-score-mysql \
693 rule drbd-promotion-score-mysql: defined drbd-promotion-score-mysql
694 ----------------------------
695 This means, that as long as the attribute is not defined, the fs_mysql file
696 system cannot be mounted here. When the attribute is defined, its value
697 becomes the score of the location constraint.
698
699 This can also be used to cause Pacemaker to migrate a service away when
700 DRBD loses a local backing device. Because a failed backing block device
701 causes the promotion score to drop, other nodes with working backing devices
702 will expose higher promotion scores.
703
704 The attributes are updated live, independent of the resource-agent's
705 monitor operation, with a dampening delay of 5 seconds by default.
706
707 The resource agent has these optional parameters,
708 see also its man page `ocf_linbit_drbd-attr(7)`:
709
710 * `dampening_delay`
711 * `attr_name_prefix`
712 * `record_event_details`
713
714 // Keep the empty line before this comment, otherwise the next chapter is folded into this