[[ch-xen]] == Using Xen with DRBD indexterm:[Xen]This chapter outlines the use of DRBD as a Virtual Block Device (VBD) for virtualization environments using the Xen hypervisor. [[s-xen-primer]] === Xen primer Xen is a virtualization framework originally developed at the University of Cambridge (UK), and later being maintained by XenSource, Inc. (now a part of Citrix). It is included in reasonably recent releases of most Linux distributions, such as Debian GNU/Linux (since version 4.0), SUSE Linux Enterprise Server (since release 10), Red Hat Enterprise Linux (since release 5), and many others. Xen uses paravirtualization — a virtualization method involving a high degree of cooperation between the virtualization host and guest virtual machines — with selected guest operating systems for improved performance in comparison to conventional virtualization solutions (which are typically based on hardware emulation). Xen also supports full hardware emulation on CPUs that support the appropriate virtualization extensions; in Xen parlance, this is known as HVM ( "hardware-assisted virtual machine"). NOTE: At the time of writing, CPU extensions supported by Xen for HVM are Intel's Virtualization Technology (VT, formerly codenamed "Vanderpool"), and AMD's Secure Virtual Machine (SVM, formerly known as "Pacifica"). Xen supports indexterm:[Xen, live migration]_live migration_, which refers to the capability of transferring a running guest operating system from one physical host to another, without interruption. When a DRBD resource is used as a replicated Virtual Block Device (VBD) for Xen, it serves to make the entire contents of a domU's virtual disk available on two servers, which can then be configured for automatic fail-over. That way, DRBD does not only provide redundancy for Linux servers (as in non-virtualized DRBD deployment scenarios), but also for any other operating system that can be virtualized under Xen — which, in essence, includes any operating system available on 32- or 64-bit Intel compatible architectures. [[s-xen-drbd-mod-params]] === Setting DRBD module parameters for use with Xen For Xen Domain-0 kernels, it is recommended to load the DRBD module with the parameter `disable_sendpage` set to `1`. To do so, create (or open) the file `/etc/modprobe.d/drbd.conf` and enter the following line: [source,drbd] ---------------------------- options drbd disable_sendpage=1 ---------------------------- [[s-xen-create-resource]] === Creating a DRBD resource suitable to act as a Xen VBD Configuring a DRBD resource that is to be used as a Virtual Block Device for Xen is fairly straightforward -- in essence, the typical configuration matches that of a DRBD resource being used for any other purpose. However, if you want to enable live migration for your guest instance, you need to enable indexterm:[Dual-Primary mode]<>for this resource: [source,drbd] ---------------------------- resource { net { allow-two-primaries yes; ... } ... } ---------------------------- Enabling dual-primary mode is necessary because Xen, before initiating live migration, checks for write access on all VBDs a resource is configured to use on both the source and the destination host for the migration. [[s-xen-configure-domu]] === Using DRBD VBDs In order to use a DRBD resource as the virtual block device, you must add a line like the following to your Xen domU configuration: indexterm:[Xen] [source,drbd] ---------------------------- disk = [ 'drbd:,xvda,w' ] ---------------------------- This example configuration makes the DRBD resource named _resource_ available to the domU as `/dev/xvda` in read/write mode (`w`). Of course, you may use multiple DRBD resources with a single domU. In that case, simply add more entries like the one provided in the example to the `disk` option, separated by commas. NOTE: There are three sets of circumstances under which you cannot use this approach: * You are configuring a fully virtualized (HVM) domU. * You are installing your domU using a graphical installation utility, _and_ that graphical installer does not support the `drbd:` syntax. * You are configuring a domU without the `kernel`, `initrd`, and `extra` options, relying instead on `bootloader` and `bootloader_args` to use a Xen pseudo-bootloader, _and_ that pseudo-bootloader does not support the `drbd:` syntax. ** pygrub+ (prior to Xen 3.3) and `domUloader.py` (shipped with Xen on SUSE Linux Enterprise Server 10) are two examples of pseudo-bootloaders that do not support the `drbd:` virtual block device configuration syntax. ** `pygrub` from Xen 3.3 forward, and the `domUloader.py` version that ships with SLES 11 _do_ support this syntax. Under these circumstances, you must use the traditional `phy:` device syntax and the DRBD device name that is associated with your resource, not the resource name. That, however, requires that you manage DRBD state transitions outside Xen, which is a less flexible approach than that provided by the `drbd` resource type. [[s-manage-domu]] === Starting, stopping, and migrating DRBD-backed domU's .Starting the domU Once you have configured your DRBD-backed domU, you may start it as you would any other domU: ---------------------------- # xm create Using config file "/etc/xen/". Started domain ---------------------------- In the process, the DRBD resource you configured as the VBD will be promoted to the primary role, and made accessible to Xen as expected. .Stopping the domU This is equally straightforward: ---------------------------- # xm shutdown -w Domain terminated. ---------------------------- Again, as you would expect, the DRBD resource is returned to the secondary role after the domU is successfully shut down. .Migrating the domU This, too, is done using the usual Xen tools: ---------------------------- # xm migrate --live ---------------------------- In this case, several administrative steps are automatically taken in rapid succession: * The resource is promoted to the primary role on _destination-host_. * Live migration of _domU_ is initiated on the local host. * When migration to the destination host has completed, the resource is demoted to the secondary role locally. The fact that both resources must briefly run in the primary role on both hosts is the reason for having to configure the resource in dual-primary mode in the first place. [[s-xen-internal]] === Internals of DRBD/Xen integration Xen supports two Virtual Block Device types natively: .`phy` This device type is used to hand "physical" block devices, available in the host environment, off to a guest domU in an essentially transparent fashion. .`file` This device type is used to make file-based block device images available to the guest domU. It works by creating a loop block device from the original image file, and then handing that block device off to the domU in much the same fashion as the `phy` device type does. If a Virtual Block Device configured in the `disk` option of a domU configuration uses any prefix other than `phy:`, `file:`, or no prefix at all (in which case Xen defaults to using the `phy` device type), Xen expects to find a helper script named ``block-``_prefix_ in the Xen scripts directory, commonly `/etc/xen/scripts`. The DRBD distribution provides such a script for the `drbd` device type, named `/etc/xen/scripts/block-drbd`. This script handles the necessary DRBD resource state transitions as described earlier in this chapter. [[s-xen-pacemaker]] === Integrating Xen with Pacemaker indexterm:[Xen]indexterm:[Heartbeat]In order to fully capitalize on the benefits provided by having a DRBD-backed Xen VBD's, it is recommended to have Pacemaker manage the associated domU's as Pacemaker resources. You may configure a Xen domU as a Pacemaker resource, and automate resource failover. To do so, use the Xen OCF resource agent. If you are using the `drbd` Xen device type described in this chapter, you will _not_ need to configure any separate `drbd` resource for use by the Xen cluster resource. Instead, the `block-drbd` helper script will do all the necessary resource transitions for you.