Prerequisites and Considerations for Deploying LINSTOR into LKE
The LINSTOR Operator for Kubernetes currently is capable of compiling DRBD on RHEL, Ubuntu, and SLES based Kubernetes workers with the proper build dependencies installed. However, LKE uses Debian Bullseye based Kubernetes workers, which the LINSTOR Operator does not have an appropriate `KernelModuleInjectionImage` for at the time of writing (Dec. 2022). Therefore, the DRBD kernel module must be compiled and installed directly on the LKE worker nodes before deploying the LINSTOR Operator, in addition to LINSTOR's generic prerequisites.
Prerequisites
Attaching Linode volume(s) to each LKE worker
A generic requirement for deploying LINSTOR into Kubernetes is having an empty block device for LINSTOR to use as a storage pool for the volumes that LINSTOR will provision. You can attach many volumes to each worker. These volumes should remain unformatted and unused, or else LINSTOR will refuse to use them.
In the image below you will see an example of a Linode with an additional unused Linode volume attached.
This volume attaches to the Linode as `/dev/sdb` which will be defined as a storage pool for LINSTOR in the Helm deployment step. Each node in your LKE cluster should be configured with its own additional Linode volume.
Building and installing DRBD on the LKE workers
SSH into each of your LKE workers and install the necessary packages for compiling and installing the DRBD kernel module:
# apt install linux-headers-$(uname -r) build-essential
Then, download the DRBD 9 kernel module source tar file of your choice from https://pkg.linbit.com. At the time of writing, the latest currently recommended DRBD kernel module is version 9.1.12. Update the version number that is set in the BASH variable command that follows if a newer 9.1.x module has become available:
# DRBD_VERS="9.1.12"
# curl -LO https://pkg.linbit.com//downloads/drbd/9/drbd-$DRBD_VERS.tar.gz
# tar xvf drbd-$DRBD_VERS.tar.gz
# cd drbd-$DRBD_VERS/
# make && make install
# modprobe drbd
You can verify that the correct DRBD kernel module has been loaded by running the following command and verifying the first line that reads, `version: 9.x.x`, matches the DRBD kernel module version you intended to build:
# cat /proc/drbd
version: 9.1.12 (api:2/proto:86-121)
GIT-hash: 86ec2326fef3aede9f4d46f52bfd35aac4d5eb7e build by root@lke83867-128388-639a05ada20c, 2022-12-14 20:31:50
Transports (api:18):
Disable DRBD's user mode helper on the LKE workers
Finally, we will disable DRBD's user mode helper. This feature of the DRBD module enables running user configured commands on changes in DRBDs state. When using DRBD within containers, it could confuse programs, such as the default `drbdadm`, that expect to know about all configured DRBD resources. To prevent any issues, you have to set the DRBD module parameter `usermode_helper` to `disabled` on each LKE worker by using the following commands:
# echo -n "disabled" > /sys/module/drbd/parameters/usermode_helper
# echo "options drbd usermode_helper=disabled" > /etc/modprobe.d/drbd.conf
Verify that the commands above succeeded and that the `usermode_helper` is now set to `disabled` and not `drbdadm`:
# cat /sys/module/drbd/parameters/usermode_helper
disabled
Deployment
With the prerequisites taken care of, all that's left is to deploy the LINSTOR Operator. Deployment of the LINSTOR Operator follows the instructions found in the LINSTOR User's Guide except that the following Helm options must be used because you have compiled and installed DRBD from source code:
operator:
satelliteSet:
kernelModuleInjectionImage: drbd.io/amd64/drbd9-jammy
kernelModuleInjectionMode: DepsOnly
For completeness, the following commands include Helm options used to deploy LINSTOR into LKE where each LKE worker has a single additional Linode volume (`/dev/sdb`) attached. This additional volume will be used by LINSTOR as a thin LVM backed storage pool named `thin-lvm`:
# helm repo add linstor https://charts.linstor.io
# helm repo update
# kubectl create namespace linstor
# kubectl create secret docker-registry drbdiocred -n linstor \
--docker-server=drbd.io --docker-username=$LBUSER \
--docker-email=$LBEMAIL --docker-password=$LBPASS
# cat << EOF > linstor-op-vals.yaml
operator:
controller:
dbConnectionURL: k8s
satelliteSet:
storagePools:
lvmThinPools:
- name: lvm-thin
thinVolume: thinpool
volumeGroup: ""
devicePaths:
- /dev/sdb
kernelModuleInjectionImage: drbd.io/amd64/drbd9-jammy
kernelModuleInjectionMode: DepsOnly
etcd:
enabled: false
stork:
enabled: false
csi:
enableTopology: true
EOF
# helm install -n linstor -f ./linstor-op-vals.yaml linstor-op linstor/linstor
Next, you can continue to configure LINSTOR's storage classes in Kubernetes as outlined in the LINSTOR User's Guide.
Written on 12/14/2022 - MDK
Reviewed on 12/15/2022 - MAT