Editing and Updating a DRBD Reactor Promoter Plugin Resource Configuration

It might be the case that you need to change the configuration of a DRBD Reactor promoter plugin resource that is running within a cluster of nodes and providing highly available (HA) services. To do this with minimal downtime to the HA services that the promoter plugin resource is running, you can follow the instructions in this article. The instructions apply to a cluster of nodes running DRBD and DRBD Reactor services with manually created DRBD Reactor promoter plugin configurations. The instructions do not apply to a cluster using LINSTOR Gateway to manage DRBD Reactor services and automatically create configurations.

❗ IMPORTANT: The procedure in this article involves failing services over from the node that is actively running them to another node in the cluster. For this reason, although failover should happen within a few seconds, it would be best to perform the steps in this procedure during a planned maintenance window.

An Example Scenario

For an example scenario, imagine a 3-node DRBD Reactor cluster with nodes named node-0, node-1, and node-2. The DRBD Reactor promoter resource is currently running on node-2.

Steps to Edit and Update a DRBD Reactor Promoter Plugin Resource Configuration

To edit and update the promoter plugin resource’s configuration, do the following steps:

1. Disable the promoter plugin resource on node-0 and node-1:
   # drbd-reactorctl disable --now <promoter-plugin-resource-name>
2. Edit the disabled promoter plugin resource on node-0 and node-1:
   # drbd-reactorctl edit --disabled <promoter-plugin-resource-name> Alternatively, you can edit the promoter plugin resource on node-0 and if you have configured root user SSH access for node-1, you can use scp to copy the edited configuration from node-0 to node-1:
   # scp /etc/drbd-reactor.d/nfs.toml.disabled \
reactornfs-1:/etc/drbd-reactor.d/nfs.toml.disabled
3. Enable the promoter plugin resource on node-0 and node-1: 
   # drbd-reactorctl enable <promoter-plugin-resource-name>
4. Fail over the promoter resource that is running on node-2 to one of the other two nodes that now have an updated promoter plugin resource configuration. You can fail over by disabling the promoter plugin resource on node-2.
   # drbd-reactorctl disable --now <promoter-plugin-resource-name>
5. Failover should happen within a few seconds. You can enter a drbd-reactorctl status <promoter-plugin-resource-name> command on node-0 or node-1 to verify that one of these nodes is running the promoter plugin resource’s services.
6. Edit the disabled promoter plugin resource on node-2:
   # drbd-reactorctl edit --disabled <promoter-plugin-resource-name> Alternatively, if you have configured root user SSH access for node-2, you can use scp to copy the edited configuration from node-0 to node-2:
   # scp /etc/drbd-reactor.d/nfs.toml \
reactornfs-2:/etc/drbd-reactor.d/nfs.toml.disabled
7. Enable the promoter plugin resource on node-2:
   # drbd-reactorctl enable <promoter-plugin-resource-name>

Testing Failover

After updating the promoter plugin resource configuration on all your nodes, you can test failing over to other nodes. You can do this by disabling the promoter plugin resource on the node that is currently active for the promoter plugin resource. After you verify that the resource can fail over to each node, you are ready to resume normal operations.

Troubleshooting Failover

If you have issues failing over to other nodes, you can follow the steps in this knowledge base article to troubleshoot.

Written by: MAT - 2024-06-28.

Reviewed by: RCK 2024-07-02