| 1 |
[[ch-openshift]] |
| 2 |
== LINSTOR Volumes in Openshift |
| 3 |
|
| 4 |
This chapter describes the usage of LINSTOR in |
| 5 |
Openshift as managed by the operator and with volumes provisioned using the |
| 6 |
https://github.com/LINBIT/linstor-csi[LINSTOR CSI plugin]. |
| 7 |
|
| 8 |
[[s-openshift-overview]] |
| 9 |
=== Openshift Overview |
| 10 |
|
| 11 |
_OpenShift_ is the official Red Hat developed and supported |
| 12 |
distribution of Kubernetes. As such, you can easily deploy Piraeus or |
| 13 |
the LINSTOR operator using Helm or via example yamls as mentioned in |
| 14 |
the previous chapter, <<ch-kubernetes>>. |
| 15 |
|
| 16 |
Some of the value of Red Hat's Openshift is that it includes its own registry |
| 17 |
of supported and certified images and operators, in addition to a |
| 18 |
default and standard web console. This chapter describes how to install |
| 19 |
the Certified LINSTOR operator via these tools. |
| 20 |
|
| 21 |
[[s-openshift-deploy]] |
| 22 |
=== Deploying LINSTOR on Openshift |
| 23 |
|
| 24 |
[[s-openshift-before-begin]] |
| 25 |
==== Before you Begin |
| 26 |
|
| 27 |
LINBIT provides a certified LINSTOR operator via the RedHat |
| 28 |
marketplace. The operator eases deployment of LINSTOR on Kubernetes by |
| 29 |
installing DRBD, managing Satellite and Controller pods, and other related |
| 30 |
functions. |
| 31 |
|
| 32 |
The operator itself is available from the |
| 33 |
https://marketplace.redhat.com/en-us/products/linbit[Red Hat |
| 34 |
Marketplace.] |
| 35 |
|
| 36 |
Unlike deployment via the helm chart, the certified Openshift |
| 37 |
operator does not deploy the needed etcd cluster. You must deploy this |
| 38 |
yourself ahead of time. We do this via the etcd operator available on |
| 39 |
https://operatorhub.io/operator/etcd[operatorhub.io]. |
| 40 |
|
| 41 |
IMPORTANT: It it advised that the etcd deployment uses persistent |
| 42 |
storage of some type. Either use an existing storage provisioner with |
| 43 |
a default `StorageClass` or simply use `hostPath` volumes. |
| 44 |
|
| 45 |
Read <<s-kubernetes-storage, the storage guide>> and configure a basic storage setup for LINSTOR. |
| 46 |
|
| 47 |
Read the <<s-kubernetes-securing-deployment,section on securing the deployment>> and configure as needed. |
| 48 |
|
| 49 |
[[s-openshift-deploy-operator-pod]] |
| 50 |
==== Deploying the operator pod |
| 51 |
|
| 52 |
Once etcd and storage has been configured, we are now ready to install |
| 53 |
the LINSTOR operator. You can find the LINSTOR operator via the |
| 54 |
left-hand control pane of Openshift Web Console. Expand the |
| 55 |
"Operators" section and select "OperatorHub". From here you need to |
| 56 |
find the LINSTOR operator. Either search for the term "LINSTOR" or |
| 57 |
filter only by "Marketplace" operators. |
| 58 |
|
| 59 |
IMPORTANT: The LINSTOR operator can only watch for events and manage |
| 60 |
custom resources that are within the same namespace it is deployed |
| 61 |
within (OwnNamespace). This means the LINSTOR Controller, LINSTOR |
| 62 |
Satellites, and LINSTOR CSI Driver pods all need to be deployed in the |
| 63 |
same namespace as the LINSTOR Operator pod. |
| 64 |
|
| 65 |
Once you have located the LINSTOR operator in the Marketplace, click |
| 66 |
the "Install" button and install it as you would any other operator. |
| 67 |
|
| 68 |
At this point you should have just one pod, the operator pod, running. |
| 69 |
|
| 70 |
Next we need to configure the remaining provided APIs. |
| 71 |
|
| 72 |
[[s-openshift-deploy-controller]] |
| 73 |
==== Deploying the LINSTOR Controller |
| 74 |
|
| 75 |
Again, navigate to the left-hand control pane of the Openshift Web |
| 76 |
Console. Expand the "Operators" section, but this time select |
| 77 |
"Installed Operators". Find the entry for the "Linstor Operator", then |
| 78 |
select the "LinstorController" from the "Provided APIs" column on the |
| 79 |
right. |
| 80 |
|
| 81 |
From here you should see a page that says "No Operands Found" and will |
| 82 |
feature a large button on the right which says "Create |
| 83 |
LinstorController". Click the "Create LinstorController" button. |
| 84 |
|
| 85 |
Here you will be presented with options to configure the LINSTOR |
| 86 |
Controller. Either via the web-form view or the YAML View. Regardless |
| 87 |
of which view you select, make sure that the `dbConnectionURL` matches |
| 88 |
the endpoint provided from your etcd deployment. Otherwise, the |
| 89 |
defaults are usually fine for most purposes. |
| 90 |
|
| 91 |
Lastly hit "Create", you should now see a linstor-controller pod |
| 92 |
running. |
| 93 |
|
| 94 |
==== Deploying the LINSTOR Satellites |
| 95 |
|
| 96 |
Next we need to deploy the Satellites Set. Just as before navigate |
| 97 |
to the left-hand control pane of the Openshift Web Console. Expand the |
| 98 |
"Operators" section, but this time select "Installed Operators". Find |
| 99 |
the entry for the "Linstor Operator", then select the |
| 100 |
"LinstorSatelliteSet" from the "Provided APIs" column on the right. |
| 101 |
|
| 102 |
From here you should see a page that says "No Operands Found" and will |
| 103 |
feature a large button on the right which says "Create |
| 104 |
LinstorSatelliteSet". Click the "Create LinstorSatelliteSet" button. |
| 105 |
|
| 106 |
Here you will be presented with the options to configure the LINSTOR |
| 107 |
Satellites. Either via the web-form view or the YAML View. One of the |
| 108 |
first options you'll notice is the `automaticStorageType`. If set to |
| 109 |
"NONE" then you'll need to remember to configure the storage pools |
| 110 |
yourself at a later step. |
| 111 |
|
| 112 |
Another option you'll notice is `kernelModuleInjectionMode`. I usually |
| 113 |
select "Compile" for portability sake, but selecting "ShippedModules" |
| 114 |
will be faster as it will install pre-compiled kernel modules on all |
| 115 |
the worker nodes. |
| 116 |
|
| 117 |
Make sure the `controllerEndpoint` matches what is available in the |
| 118 |
kubernetes endpoints. The default is usually correct here. |
| 119 |
|
| 120 |
Below is an example manifest: |
| 121 |
|
| 122 |
----- |
| 123 |
apiVersion: linstor.linbit.com/v1 |
| 124 |
kind: LinstorSatelliteSet |
| 125 |
metadata: |
| 126 |
name: linstor |
| 127 |
namespace: default |
| 128 |
spec: |
| 129 |
satelliteImage: '' |
| 130 |
automaticStorageType: LVMTHIN |
| 131 |
drbdRepoCred: '' |
| 132 |
kernelModuleInjectionMode: Compile |
| 133 |
controllerEndpoint: 'http://linstor:3370' |
| 134 |
priorityClassName: '' |
| 135 |
status: |
| 136 |
errors: [] |
| 137 |
----- |
| 138 |
|
| 139 |
Lastly hit "Create", you should now see a linstor-node pod |
| 140 |
running on every worker node. |
| 141 |
|
| 142 |
==== Deploying the LINSTOR CSI driver |
| 143 |
|
| 144 |
Last bit left is the CSI pods to bridge the layer between the CSI and |
| 145 |
LINSTOR. Just as before navigate to the left-hand control pane of the |
| 146 |
OpenShift Web Console. Expand the "Operators" section, but this time |
| 147 |
select "Installed Operators". Find the entry for the "Linstor Operator", |
| 148 |
then select the "LinstorCSIDriver" from the "Provided APIs" column on the |
| 149 |
right. |
| 150 |
|
| 151 |
From here you should see a page that says "No Operands Found" and will |
| 152 |
feature a large button on the right which says "Create |
| 153 |
LinstorCSIDriver". Click the "Create LinstorCSIDriver" button. |
| 154 |
|
| 155 |
Again, you will be presented with the options. Make sure that the |
| 156 |
`controllerEndpoint` is correct. Otherwise the defaults are fine for |
| 157 |
most use cases. |
| 158 |
|
| 159 |
Lastly hit "Create". You will now see a single "linstor-csi-controller" pod, |
| 160 |
as well as a "linstor-csi-node" pod on all worker nodes. |
| 161 |
|
| 162 |
=== Interacting with LINSTOR in OpenShift. |
| 163 |
|
| 164 |
The Controller pod includes a LINSTOR Client, making it easy to interact directly with LINSTOR. |
| 165 |
For instance: |
| 166 |
|
| 167 |
---- |
| 168 |
oc exec deployment/linstor-controller -- linstor storage-pool list |
| 169 |
---- |
| 170 |
|
| 171 |
This should only be necessary for investigating problems and accessing advanced functionality. |
| 172 |
Regular operation such as creating volumes should be achieved via the |
| 173 |
<<s-kubernetes-basic-configuration-and-deployment,Kubernetes integration>>. |
| 174 |
|
| 175 |
=== Configuration and deployment |
| 176 |
|
| 177 |
Once the operator and all the needed pods are deployed, provisioning |
| 178 |
volumes simply follows the usual Kubernetes workflows. |
| 179 |
|
| 180 |
As such, please see the previous chapter's section on |
| 181 |
<<s-kubernetes-basic-configuration-and-deployment,Basic Configuration |
| 182 |
and Deployment>>. |
| 183 |
|
| 184 |
=== Deploying additional components |
| 185 |
|
| 186 |
Some additional components are not included in the OperatorHub version of the LINSTOR Operator when compared to the |
| 187 |
Helm deployment. Most notably, this includes setting up Etcd and deploying the <<s-kubernetes-stork,STORK integration>>. |
| 188 |
|
| 189 |
Etcd can be deployed by using the Etcd Operator available in the OperatorHub. |
| 190 |
|
| 191 |
==== Stork |
| 192 |
|
| 193 |
To deploy STORK, you can use the single YAML deployment available at: https://charts.linstor.io/deploy/stork.yaml |
| 194 |
Download the YAML and replace every instance of `MY-STORK-NAMESPACE` with your desired namespace for STORK. You also |
| 195 |
need to replace `MY-LINSTOR-URL` with the URL of your controller. This value depends on the `name` you chose when |
| 196 |
<<s-openshift-deploy-controller,creating the LinstorController resource>>. By default this would be |
| 197 |
`http://linstor.<operator-namespace>.svc:3370` |
| 198 |
|
| 199 |
To apply the YAML to Openshift, either use `oc apply -f <filename>` from the command line or find the "Import YAML" |
| 200 |
option in the top right of the Openshift Web Console. |
| 201 |
|
| 202 |
==== High Availability Controller |
| 203 |
|
| 204 |
To deploy our <<s-kubernetes-ha-controller,High Availability Controller>>, you can use the single YAML deployment available at: https://charts.linstor.io/deploy/ha-controller.yaml |
| 205 |
|
| 206 |
Download the YAML and replace: |
| 207 |
|
| 208 |
- `MY-HA-CTRL-NAMESPACE`: with your preferred project name / namespace. |
| 209 |
- `MY-LINSTOR-URL`: url of the LINSTOR controller, for example: `http://linstor.linstor-namespace.svc:3370/` |
| 210 |
|
| 211 |
To apply the YAML to Openshift, either use `oc apply -f <filename>` from the command line or find the "Import YAML" |
| 212 |
option in the top right of the Openshift Web Console. |
| 213 |
|
| 214 |
==== Deploying via Helm on openshift |
| 215 |
|
| 216 |
Alternatively, you can deploy the LINSTOR Operator using Helm instead. Take a look at the <<ch-kubernetes,Kubernetes guide>>. |
| 217 |
Openshift requires changing some of the default values in our Helm chart. |
| 218 |
|
| 219 |
If you chose to use Etcd with hostpath volumes for persistence (see <<s-kubernetes-etcd-hostpath-persistence,here>>), |
| 220 |
you need to enable selinux relabelling. To do this pass `--set selinux=true` to the `pv-hostpath` install command. |
| 221 |
|
| 222 |
For the LINSTOR Operator chart itself, you should change the following values: |
| 223 |
[source,yaml] |
| 224 |
---- |
| 225 |
global: |
| 226 |
setSecurityContext: false <1> |
| 227 |
csi-snapshotter: |
| 228 |
enabled: false <2> |
| 229 |
stork: |
| 230 |
schedulerTag: v1.18.6 <3> |
| 231 |
etcd: |
| 232 |
podsecuritycontext: |
| 233 |
supplementalGroups: [1000] <4> |
| 234 |
operator: |
| 235 |
satelliteSet: |
| 236 |
kernelModuleInjectionImage: drbd.io/drbd9-rhel8:v9.0.25 <5> |
| 237 |
---- |
| 238 |
<1> Openshift uses https://docs.openshift.com/container-platform/4.6/authentication/managing-security-context-constraints.html[SCCs] to manage security contexts. |
| 239 |
<2> The cluster wide CSI Snapshot Controller is already installed by Openshift. |
| 240 |
<3> Automatic detection of the Kubernetes Scheduler version fails in Openshift, you need to set it manually. Note: the tag does _not_ have to match Openshift's Kubernetes release. |
| 241 |
<4> If you choose to use Etcd deployed via Helm and use the `pv-hostpath` chart, Etcd needs to run as member of group `1000` to access the persistent volume. |
| 242 |
<5> The RHEL8 kernel injector also supports RHCOS. |
| 243 |
|
| 244 |
Other overrides, such as storage pool configuration, HA deployments and more, are available and documented in the <<ch-kubernetes,Kubernetes guide>>. |