[[ch-openshift]] == LINSTOR Volumes in Openshift This chapter describes the usage of LINSTOR in Openshift as managed by the operator and with volumes provisioned using the https://github.com/LINBIT/linstor-csi[LINSTOR CSI plugin]. [[s-openshift-overview]] === Openshift Overview _OpenShift_ is the official Red Hat developed and supported distribution of Kubernetes. As such, you can easily deploy Piraeus or the LINSTOR operator using Helm or via example yamls as mentioned in the previous chapter, <>. Some of the value of Red Hat's Openshift is that it includes its own registry of supported and certified images and operators, in addition to a default and standard web console. This chapter describes how to install the Certified LINSTOR operator via these tools. [[s-openshift-deploy]] === Deploying LINSTOR on Openshift [[s-openshift-before-begin]] ==== Before you Begin LINBIT provides a certified LINSTOR operator via the RedHat marketplace. The operator eases deployment of LINSTOR on Kubernetes by installing DRBD, managing Satellite and Controller pods, and other related functions. The operator itself is available from the https://marketplace.redhat.com/en-us/products/linbit[Red Hat Marketplace.] Unlike deployment via the helm chart, the certified Openshift operator does not deploy the needed etcd cluster. You must deploy this yourself ahead of time. We do this via the etcd operator available on https://operatorhub.io/operator/etcd[operatorhub.io]. IMPORTANT: It it advised that the etcd deployment uses persistent storage of some type. Either use an existing storage provisioner with a default `StorageClass` or simply use `hostPath` volumes. Read <> and configure a basic storage setup for LINSTOR. Read the <> and configure as needed. [[s-openshift-deploy-operator-pod]] ==== Deploying the operator pod Once etcd and storage has been configured, we are now ready to install the LINSTOR operator. You can find the LINSTOR operator via the left-hand control pane of Openshift Web Console. Expand the "Operators" section and select "OperatorHub". From here you need to find the LINSTOR operator. Either search for the term "LINSTOR" or filter only by "Marketplace" operators. IMPORTANT: The LINSTOR operator can only watch for events and manage custom resources that are within the same namespace it is deployed within (OwnNamespace). This means the LINSTOR Controller, LINSTOR Satellites, and LINSTOR CSI Driver pods all need to be deployed in the same namespace as the LINSTOR Operator pod. Once you have located the LINSTOR operator in the Marketplace, click the "Install" button and install it as you would any other operator. At this point you should have just one pod, the operator pod, running. Next we need to configure the remaining provided APIs. [[s-openshift-deploy-controller]] ==== Deploying the LINSTOR Controller Again, navigate to the left-hand control pane of the Openshift Web Console. Expand the "Operators" section, but this time select "Installed Operators". Find the entry for the "Linstor Operator", then select the "LinstorController" from the "Provided APIs" column on the right. From here you should see a page that says "No Operands Found" and will feature a large button on the right which says "Create LinstorController". Click the "Create LinstorController" button. Here you will be presented with options to configure the LINSTOR Controller. Either via the web-form view or the YAML View. Regardless of which view you select, make sure that the `dbConnectionURL` matches the endpoint provided from your etcd deployment. Otherwise, the defaults are usually fine for most purposes. Lastly hit "Create", you should now see a linstor-controller pod running. ==== Deploying the LINSTOR Satellites Next we need to deploy the Satellites Set. Just as before navigate to the left-hand control pane of the Openshift Web Console. Expand the "Operators" section, but this time select "Installed Operators". Find the entry for the "Linstor Operator", then select the "LinstorSatelliteSet" from the "Provided APIs" column on the right. From here you should see a page that says "No Operands Found" and will feature a large button on the right which says "Create LinstorSatelliteSet". Click the "Create LinstorSatelliteSet" button. Here you will be presented with the options to configure the LINSTOR Satellites. Either via the web-form view or the YAML View. One of the first options you'll notice is the `automaticStorageType`. If set to "NONE" then you'll need to remember to configure the storage pools yourself at a later step. Another option you'll notice is `kernelModuleInjectionMode`. I usually select "Compile" for portability sake, but selecting "ShippedModules" will be faster as it will install pre-compiled kernel modules on all the worker nodes. Make sure the `controllerEndpoint` matches what is available in the kubernetes endpoints. The default is usually correct here. Below is an example manifest: ----- apiVersion: linstor.linbit.com/v1 kind: LinstorSatelliteSet metadata: name: linstor namespace: default spec: satelliteImage: '' automaticStorageType: LVMTHIN drbdRepoCred: '' kernelModuleInjectionMode: Compile controllerEndpoint: 'http://linstor:3370' priorityClassName: '' status: errors: [] ----- Lastly hit "Create", you should now see a linstor-node pod running on every worker node. ==== Deploying the LINSTOR CSI driver Last bit left is the CSI pods to bridge the layer between the CSI and LINSTOR. Just as before navigate to the left-hand control pane of the OpenShift Web Console. Expand the "Operators" section, but this time select "Installed Operators". Find the entry for the "Linstor Operator", then select the "LinstorCSIDriver" from the "Provided APIs" column on the right. From here you should see a page that says "No Operands Found" and will feature a large button on the right which says "Create LinstorCSIDriver". Click the "Create LinstorCSIDriver" button. Again, you will be presented with the options. Make sure that the `controllerEndpoint` is correct. Otherwise the defaults are fine for most use cases. Lastly hit "Create". You will now see a single "linstor-csi-controller" pod, as well as a "linstor-csi-node" pod on all worker nodes. === Interacting with LINSTOR in OpenShift. The Controller pod includes a LINSTOR Client, making it easy to interact directly with LINSTOR. For instance: ---- oc exec deployment/linstor-controller -- linstor storage-pool list ---- This should only be necessary for investigating problems and accessing advanced functionality. Regular operation such as creating volumes should be achieved via the <>. === Configuration and deployment Once the operator and all the needed pods are deployed, provisioning volumes simply follows the usual Kubernetes workflows. As such, please see the previous chapter's section on <>. === Deploying additional components Some additional components are not included in the OperatorHub version of the LINSTOR Operator when compared to the Helm deployment. Most notably, this includes setting up Etcd and deploying the <>. Etcd can be deployed by using the Etcd Operator available in the OperatorHub. ==== Stork To deploy STORK, you can use the single YAML deployment available at: https://charts.linstor.io/deploy/stork.yaml Download the YAML and replace every instance of `MY-STORK-NAMESPACE` with your desired namespace for STORK. You also need to replace `MY-LINSTOR-URL` with the URL of your controller. This value depends on the `name` you chose when <>. By default this would be `http://linstor..svc:3370` To apply the YAML to Openshift, either use `oc apply -f ` from the command line or find the "Import YAML" option in the top right of the Openshift Web Console. ==== High Availability Controller To deploy our <>, you can use the single YAML deployment available at: https://charts.linstor.io/deploy/ha-controller.yaml Download the YAML and replace: - `MY-HA-CTRL-NAMESPACE`: with your preferred project name / namespace. - `MY-LINSTOR-URL`: url of the LINSTOR controller, for example: `http://linstor.linstor-namespace.svc:3370/` To apply the YAML to Openshift, either use `oc apply -f ` from the command line or find the "Import YAML" option in the top right of the Openshift Web Console. ==== Deploying via Helm on openshift Alternatively, you can deploy the LINSTOR Operator using Helm instead. Take a look at the <>. Openshift requires changing some of the default values in our Helm chart. If you chose to use Etcd with hostpath volumes for persistence (see <>), you need to enable selinux relabelling. To do this pass `--set selinux=true` to the `pv-hostpath` install command. For the LINSTOR Operator chart itself, you should change the following values: [source,yaml] ---- global: setSecurityContext: false <1> csi-snapshotter: enabled: false <2> stork: schedulerTag: v1.18.6 <3> etcd: podsecuritycontext: supplementalGroups: [1000] <4> operator: satelliteSet: kernelModuleInjectionImage: drbd.io/drbd9-rhel8:v9.0.25 <5> ---- <1> Openshift uses https://docs.openshift.com/container-platform/4.6/authentication/managing-security-context-constraints.html[SCCs] to manage security contexts. <2> The cluster wide CSI Snapshot Controller is already installed by Openshift. <3> Automatic detection of the Kubernetes Scheduler version fails in Openshift, you need to set it manually. Note: the tag does _not_ have to match Openshift's Kubernetes release. <4> If you choose to use Etcd deployed via Helm and use the `pv-hostpath` chart, Etcd needs to run as member of group `1000` to access the persistent volume. <5> The RHEL8 kernel injector also supports RHCOS. Other overrides, such as storage pool configuration, HA deployments and more, are available and documented in the <>.