Debian Codegraph @

UG9/en/opennebula-linstor.adoc

raw
1 [[ch-opennebula-linstor]]
2 == LINSTOR Volumes in OpenNebula
3
4 indexterm:[OpenNebula]This chapter describes DRBD in OpenNebula via the usage
5 of the https://github.com/OpenNebula/addon-linstor[LINSTOR storage driver
6 addon].
7
8 Detailed installation and configuration instructions and be found in the
9 https://github.com/OpenNebula/addon-linstor/blob/master/README.md[README.md]
10 file of the driver's source.
11
12 [[s-opennebula-linstor-overview]]
13 === OpenNebula Overview
14
15 http://opennebula.org/[OpenNebula] is a flexible and open source cloud
16 management platform which allows its functionality to be extended via the use
17 of addons.
18
19 The LINSTOR addon allows the deployment of virtual machines with highly
20 available images backed by DRBD and attached across the network via DRBD's
21 own transport protocol.
22
23 [[s-opennebula-linstor-install]]
24 === OpenNebula addon Installation
25
26 Installation of the LINSTOR storage addon for OpenNebula requires a working
27 OpenNebula cluster as well as a working LINSTOR cluster.
28
29 With access to LINBIT's customer repositories you can install the linstor-opennebula with
30
31 ----------------------------
32 # apt install linstor-opennebula
33 ----------------------------
34
35 or
36
37 ----------------------------
38 # yum install linstor-opennebula
39 ----------------------------
40
41 Without access to LINBIT's prepared packages you need to fall back to instructions on it's
42 https://github.com/OpenNebula/addon-linstor[GitHub page].
43
44 A DRBD cluster with LINSTOR can be installed and configured by following the
45 instructions in this guide, see <<s-linstor-init-cluster>>.
46
47 The OpenNebula and DRBD clusters can be somewhat independent of one another
48 with the following exception: OpenNebula's Front-End and Host nodes must be
49 included in both clusters.
50
51 Host nodes do not need a local LINSTOR storage pools, as virtual machine
52 images are attached to them across the network footnote:[If a host is also a
53 storage node, it will use a local copy of an image if that is available].
54
55 [[s-opennebula-deployment-options]]
56 === Deployment Options
57
58 It is recommended to use LINSTOR resource groups to configure the deployment
59 how you like it, see <<s-opennebula-resource-group>>.
60 Previous auto-place and deployment nodes modes are deprecated.
61
62
63 [[s-opennebula-configuration]]
64 === Configuration
65
66 ==== Adding the driver to OpenNebula
67
68 Modify the following sections of `/etc/one/oned.conf`
69
70 Add linstor to the list of drivers in the `TM_MAD` and `DATASTORE_MAD`
71 sections:
72
73 ----------------------------
74 TM_MAD = [
75 executable = "one_tm",
76 arguments = "-t 15 -d dummy,lvm,shared,fs_lvm,qcow2,ssh,vmfs,ceph,linstor"
77 ]
78 ----------------------------
79 ----------------------------
80 DATASTORE_MAD = [
81 EXECUTABLE = "one_datastore",
82 ARGUMENTS = "-t 15 -d dummy,fs,lvm,ceph,dev,iscsi_libvirt,vcenter,linstor -s shared,ssh,ceph,fs_lvm,qcow2,linstor"
83 ----------------------------
84
85 Add new TM_MAD_CONF and DS_MAD_CONF sections:
86
87 ----------------------------
88 TM_MAD_CONF = [
89 NAME = "linstor", LN_TARGET = "NONE", CLONE_TARGET = "SELF", SHARED = "yes", ALLOW_ORPHANS="yes",
90 TM_MAD_SYSTEM = "ssh,shared", LN_TARGET_SSH = "NONE", CLONE_TARGET_SSH = "SELF", DISK_TYPE_SSH = "BLOCK",
91 LN_TARGET_SHARED = "NONE", CLONE_TARGET_SHARED = "SELF", DISK_TYPE_SHARED = "BLOCK"
92 ]
93 ----------------------------
94 ----------------------------
95 DS_MAD_CONF = [
96 NAME = "linstor", REQUIRED_ATTRS = "BRIDGE_LIST", PERSISTENT_ONLY = "NO",
97 MARKETPLACE_ACTIONS = "export"
98 ]
99 ----------------------------
100 After making these changes, restart the opennebula service.
101
102 [[s-opennebula-configuring-nodes]]
103 ==== Configuring the Nodes
104
105 The Front-End node issues commands to the Storage and Host nodes via Linstor
106
107 Storage nodes hold disk images of VMs locally.
108
109 Host nodes are responsible for running instantiated VMs and typically have the
110 storage for the images they need attached across the network via Linstor
111 diskless mode.
112
113 All nodes must have DRBD9 and Linstor installed. This process is detailed in the
114 http://docs.linbit.com/doc/users-guide-90/ch-admin-linstor/[User's Guide for DRBD9]
115
116 It is possible to have Front-End and Host nodes act as storage nodes in
117 addition to their primary role as long as they the meet all the requirements
118 for both roles.
119
120
121 ===== Front-End Configuration
122
123 Please verify that the control node(s) that you hope to communicate with are
124 reachable from the Front-End node. `linstor node list` for locally running
125 Linstor controllers and `linstor --controllers "<IP:PORT>" node list` for
126 remotely running Linstor Controllers is a handy way to test this.
127
128 ===== Host Configuration
129
130 Host nodes must have Linstor satellite processes running on them and be members
131 of the same Linstor cluster as the Front-End and Storage nodes, and may optionally
132 have storage locally. If the `oneadmin` user is able to passwordlessly ssh between
133 hosts then live migration may be used with the even with the ssh system datastore.
134
135 ===== Storage Node Configuration
136
137 Only the Front-End and Host nodes require OpenNebula to be installed, but the
138 oneadmin user must be able to passwordlessly access storage nodes. Refer to
139 the OpenNebula install guide for your distribution on how to manually
140 configure the oneadmin user account.
141
142 The Storage nodes must use storage pools created with a driver that's capable
143 of making snapshots, such as the thin LVM plugin.
144
145 In this example preparation of thinly-provisioned storage using LVM for Linstor,
146 you must create a volume group and thinLV using LVM on each storage node.
147
148 Example of this process using two physical volumes (/dev/sdX and /dev/sdY) and
149 generic names for the volume group and thinpool. Make sure to set the thinLV's
150 metadata volume to a reasonable size, once it becomes full it can be difficult to resize:
151
152 ----------------------------
153 pvcreate /dev/sdX /dev/sdY
154 vgcreate drbdpool /dev/sdX /dev/sdY
155 lvcreate -l 95%VG --poolmetadatasize 8g -T /dev/drbdpool/drbdthinpool
156 ----------------------------
157
158 Then you'll create storage pool(s) on Linstor using this as the backing storage.
159
160 NOTE: If you are using ZFS storage pools or thick-LVM, please use <<s-clone-mode>> `copy`
161 otherwise you will have problems deleting linstor resources, because of ZFS parent-child snapshot
162 relationships.
163
164 ==== Permissions for Oneadmin
165
166 The oneadmin user must have passwordless sudo access to the `mkfs` command on
167 the Storage nodes
168
169 ----------------------------
170 oneadmin ALL=(root) NOPASSWD: /sbin/mkfs
171 ----------------------------
172
173 ===== Groups
174
175 Be sure to consider the groups that oneadmin should be added to in order to
176 gain access to the devices and programs needed to access storage and
177 instantiate VMs. For this addon, the oneadmin user must belong to the `disk`
178 group on all nodes in order to access the DRBD devices where images are held.
179
180 ----------------------------
181 usermod -a -G disk oneadmin
182 ----------------------------
183
184 ==== Creating a New Linstor Datastore
185
186 Create a datastore configuration file named ds.conf and use the `onedatastore`
187 tool to create a new datastore based on that configuration. There are two
188 mutually exclusive deployment options: LINSTOR_AUTO_PLACE and
189 LINSTOR_DEPLOYMENT_NODES. If both are configured, LINSTOR_AUTO_PLACE is ignored.
190 For both of these options, BRIDGE_LIST must be a space
191 separated list of all storage nodes in the Linstor cluster.
192
193 [[s-opennebula-resource-group]]
194 ==== OpenNebula resource group
195
196 Since version 1.0.0 LINSTOR supports resource groups. A resource group is a
197 centralized point for settings that all resources linked to that resource group
198 share.
199
200 Create a resource group and volume group for your datastore, it is mandatory to specify a storage-pool
201 within the resource group, otherwise monitoring space for opennebula will not work.
202 Here we create one with 2 node redundancy and use a created `opennebula-storagepool`:
203
204 ----------------------------
205 linstor resource-group create OneRscGrp --place-count 2 --storage-pool opennebula-storagepool
206 linstor volume-group create
207 ----------------------------
208
209 Now add a OpenNebula datastore using the LINSTOR plugin:
210
211 ----------------------------
212 cat >ds.conf <<EOI
213 NAME = linstor_datastore
214 DS_MAD = linstor
215 TM_MAD = linstor
216 TYPE = IMAGE_DS
217 DISK_TYPE = BLOCK
218 LINSTOR_RESOURCE_GROUP = "OneRscGrp"
219 COMPATIBLE_SYS_DS = 0
220 BRIDGE_LIST = "alice bob charlie" #node names
221 EOI
222
223 onedatastore create ds.conf
224 ----------------------------
225
226 ==== Plugin attributes
227
228 ===== LINSTOR_CONTROLLERS
229
230 `LINSTOR_CONTROLLERS` can be used to pass a comma separated list of controller
231 ips and ports to the Linstor client in the case where a Linstor controller
232 process is not running locally on the Front-End, e.g.:
233
234 `LINSTOR_CONTROLLERS = "192.168.1.10:8080,192.168.1.11:6000"`
235
236
237 [[s-clone-mode]]
238 ===== LINSTOR_CLONE_MODE
239
240 Linstor supports 2 different clone modes and are set via the `LINSTOR_CLONE_MODE` attribute:
241
242 * `snapshot`
243
244 The default mode is `snapshot` it uses a linstor snapshot and restores a new resource
245 from this snapshot, which is then a clone of the image.
246 This mode is usually faster than using the `copy` mode as snapshots are cheap copies.
247
248 * `copy`
249
250 The second mode is `copy` it creates a new resource with the same size as the original and
251 copies the data with `dd` to the new resource.
252 This mode will be slower than `snapshot`, but is more robust as it doesn't rely on any snapshot
253 mechanism, it is also used if you are cloning an image into a different linstor datastore.
254
255 ==== Deprecated attributes
256
257 The following attributes are deprecated and will be removed in version after the 1.0.0 release.
258
259 ===== LINSTOR_STORAGE_POOL
260
261 `LINSTOR_STORAGE_POOL` attribute is used to select the LINSTOR storage pool your datastore
262 should use. If resource groups are used this attribute isn't needed as the storage pool
263 can be select by the auto select filter options.
264 If `LINSTOR_AUTO_PLACE` or `LINSTOR_DEPLOYMENT_NODES` is used and `LINSTOR_STORAGE_POOL`
265 is not set, it will fallback to the `DfltStorPool` in LINSTOR.
266
267 ===== LINSTOR_AUTO_PLACE
268
269 The `LINSTOR_AUTO_PLACE` option takes a level of redundancy which is a number between
270 one and the total number of storage nodes. Resources are assigned to storage
271 nodes automatically based on the level of redundancy.
272
273 ===== LINSTOR_DEPLOYMENT_NODES
274
275 Using `LINSTOR_DEPLOYMENT_NODES` allows you to select a group of nodes that
276 resources will always be assigned to. Please note that the
277 bridge list still contains all of the storage nodes in the Linstor cluster.
278
279 ==== LINSTOR as system datastore
280
281 Linstor driver can also be used as a system datastore,
282 configuration is pretty similar to normal datastores, with a few changes:
283
284 ----------------------------
285 cat >system_ds.conf <<EOI
286 NAME = linstor_system_datastore
287 TM_MAD = linstor
288 TYPE = SYSTEM_DS
289 LINSTOR_RESOURCE_GROUP = "OneSysRscGrp"
290 BRIDGE_LIST = "alice bob charlie" # node names
291 EOI
292
293 onedatastore create system_ds.conf
294 ----------------------------
295
296 Also add the new sys datastore id to the `COMPATIBLE_SYS_DS` to your image datastores (COMMA separated), otherwise the scheduler will ignore them.
297
298 If you want live migration with volatile disks you need to enable the `--unsafe` option for KVM, see:
299 https://docs.opennebula.org/5.8/deployment/open_cloud_host_setup/kvm_driver.html#live-migration-for-other-cache-settings[opennebula-doc]
300
301 [[s-opennebula-linstor-live-migration]]
302 === Live Migration
303
304 Live migration is supported even with the use of the ssh system datastore, as
305 well as the nfs shared system datastore.
306
307 [[s-opennebula-linstor-free-space]]
308 === Free Space Reporting
309
310 Free space is calculated differently depending on whether resources are
311 deployed automatically or on a per node basis.
312
313 For datastores which place per node, free space is reported based on
314 the most restrictive storage pools from all nodes where resources are being
315 deployed. For example, the capacity of the node with the smallest amount of
316 total storage space is used to determine the total size of the datastore and
317 the node with the least free space is used to determine the remaining space in
318 the datastore.
319
320 For a datastore which uses automatic placement, size and remaining space are
321 determined based on the aggregate storage pool used by the datastore as
322 reported by LINSTOR.