[[ch-opennebula-linstor]] == LINSTOR Volumes in OpenNebula indexterm:[OpenNebula]This chapter describes DRBD in OpenNebula via the usage of the https://github.com/OpenNebula/addon-linstor[LINSTOR storage driver addon]. Detailed installation and configuration instructions and be found in the https://github.com/OpenNebula/addon-linstor/blob/master/README.md[README.md] file of the driver's source. [[s-opennebula-linstor-overview]] === OpenNebula Overview http://opennebula.org/[OpenNebula] is a flexible and open source cloud management platform which allows its functionality to be extended via the use of addons. The LINSTOR addon allows the deployment of virtual machines with highly available images backed by DRBD and attached across the network via DRBD's own transport protocol. [[s-opennebula-linstor-install]] === OpenNebula addon Installation Installation of the LINSTOR storage addon for OpenNebula requires a working OpenNebula cluster as well as a working LINSTOR cluster. With access to LINBIT's customer repositories you can install the linstor-opennebula with ---------------------------- # apt install linstor-opennebula ---------------------------- or ---------------------------- # yum install linstor-opennebula ---------------------------- Without access to LINBIT's prepared packages you need to fall back to instructions on it's https://github.com/OpenNebula/addon-linstor[GitHub page]. A DRBD cluster with LINSTOR can be installed and configured by following the instructions in this guide, see <>. The OpenNebula and DRBD clusters can be somewhat independent of one another with the following exception: OpenNebula's Front-End and Host nodes must be included in both clusters. Host nodes do not need a local LINSTOR storage pools, as virtual machine images are attached to them across the network footnote:[If a host is also a storage node, it will use a local copy of an image if that is available]. [[s-opennebula-deployment-options]] === Deployment Options It is recommended to use LINSTOR resource groups to configure the deployment how you like it, see <>. Previous auto-place and deployment nodes modes are deprecated. [[s-opennebula-configuration]] === Configuration ==== Adding the driver to OpenNebula Modify the following sections of `/etc/one/oned.conf` Add linstor to the list of drivers in the `TM_MAD` and `DATASTORE_MAD` sections: ---------------------------- TM_MAD = [ executable = "one_tm", arguments = "-t 15 -d dummy,lvm,shared,fs_lvm,qcow2,ssh,vmfs,ceph,linstor" ] ---------------------------- ---------------------------- DATASTORE_MAD = [ EXECUTABLE = "one_datastore", ARGUMENTS = "-t 15 -d dummy,fs,lvm,ceph,dev,iscsi_libvirt,vcenter,linstor -s shared,ssh,ceph,fs_lvm,qcow2,linstor" ---------------------------- Add new TM_MAD_CONF and DS_MAD_CONF sections: ---------------------------- TM_MAD_CONF = [ NAME = "linstor", LN_TARGET = "NONE", CLONE_TARGET = "SELF", SHARED = "yes", ALLOW_ORPHANS="yes", TM_MAD_SYSTEM = "ssh,shared", LN_TARGET_SSH = "NONE", CLONE_TARGET_SSH = "SELF", DISK_TYPE_SSH = "BLOCK", LN_TARGET_SHARED = "NONE", CLONE_TARGET_SHARED = "SELF", DISK_TYPE_SHARED = "BLOCK" ] ---------------------------- ---------------------------- DS_MAD_CONF = [ NAME = "linstor", REQUIRED_ATTRS = "BRIDGE_LIST", PERSISTENT_ONLY = "NO", MARKETPLACE_ACTIONS = "export" ] ---------------------------- After making these changes, restart the opennebula service. [[s-opennebula-configuring-nodes]] ==== Configuring the Nodes The Front-End node issues commands to the Storage and Host nodes via Linstor Storage nodes hold disk images of VMs locally. Host nodes are responsible for running instantiated VMs and typically have the storage for the images they need attached across the network via Linstor diskless mode. All nodes must have DRBD9 and Linstor installed. This process is detailed in the http://docs.linbit.com/doc/users-guide-90/ch-admin-linstor/[User's Guide for DRBD9] It is possible to have Front-End and Host nodes act as storage nodes in addition to their primary role as long as they the meet all the requirements for both roles. ===== Front-End Configuration Please verify that the control node(s) that you hope to communicate with are reachable from the Front-End node. `linstor node list` for locally running Linstor controllers and `linstor --controllers "" node list` for remotely running Linstor Controllers is a handy way to test this. ===== Host Configuration Host nodes must have Linstor satellite processes running on them and be members of the same Linstor cluster as the Front-End and Storage nodes, and may optionally have storage locally. If the `oneadmin` user is able to passwordlessly ssh between hosts then live migration may be used with the even with the ssh system datastore. ===== Storage Node Configuration Only the Front-End and Host nodes require OpenNebula to be installed, but the oneadmin user must be able to passwordlessly access storage nodes. Refer to the OpenNebula install guide for your distribution on how to manually configure the oneadmin user account. The Storage nodes must use storage pools created with a driver that's capable of making snapshots, such as the thin LVM plugin. In this example preparation of thinly-provisioned storage using LVM for Linstor, you must create a volume group and thinLV using LVM on each storage node. Example of this process using two physical volumes (/dev/sdX and /dev/sdY) and generic names for the volume group and thinpool. Make sure to set the thinLV's metadata volume to a reasonable size, once it becomes full it can be difficult to resize: ---------------------------- pvcreate /dev/sdX /dev/sdY vgcreate drbdpool /dev/sdX /dev/sdY lvcreate -l 95%VG --poolmetadatasize 8g -T /dev/drbdpool/drbdthinpool ---------------------------- Then you'll create storage pool(s) on Linstor using this as the backing storage. NOTE: If you are using ZFS storage pools or thick-LVM, please use <> `copy` otherwise you will have problems deleting linstor resources, because of ZFS parent-child snapshot relationships. ==== Permissions for Oneadmin The oneadmin user must have passwordless sudo access to the `mkfs` command on the Storage nodes ---------------------------- oneadmin ALL=(root) NOPASSWD: /sbin/mkfs ---------------------------- ===== Groups Be sure to consider the groups that oneadmin should be added to in order to gain access to the devices and programs needed to access storage and instantiate VMs. For this addon, the oneadmin user must belong to the `disk` group on all nodes in order to access the DRBD devices where images are held. ---------------------------- usermod -a -G disk oneadmin ---------------------------- ==== Creating a New Linstor Datastore Create a datastore configuration file named ds.conf and use the `onedatastore` tool to create a new datastore based on that configuration. There are two mutually exclusive deployment options: LINSTOR_AUTO_PLACE and LINSTOR_DEPLOYMENT_NODES. If both are configured, LINSTOR_AUTO_PLACE is ignored. For both of these options, BRIDGE_LIST must be a space separated list of all storage nodes in the Linstor cluster. [[s-opennebula-resource-group]] ==== OpenNebula resource group Since version 1.0.0 LINSTOR supports resource groups. A resource group is a centralized point for settings that all resources linked to that resource group share. Create a resource group and volume group for your datastore, it is mandatory to specify a storage-pool within the resource group, otherwise monitoring space for opennebula will not work. Here we create one with 2 node redundancy and use a created `opennebula-storagepool`: ---------------------------- linstor resource-group create OneRscGrp --place-count 2 --storage-pool opennebula-storagepool linstor volume-group create ---------------------------- Now add a OpenNebula datastore using the LINSTOR plugin: ---------------------------- cat >ds.conf <system_ds.conf <