Backup DRBD from the Secondary

How to take a backup from a DRBD secondary

DRBD replicates data so you always have more than a single copy. Wouldn't it be convenient to be able to take a backup from a Secondary node where doing so would have less chance of interference or overloading the Primary node that is currently serving up the data and running the production workload? Well, with LVM snapshots we can do exactly that!

In this environment DRBD is backed by a LVM volume. This means that storage device used for DRBD is a LVM volume. you can confirm this easily via a `lsblk`. For example:

root@booth-1:~# lsblk
NAME               MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
[...]
vdb                252:16   0    12G  0 disk 
├─vg_drbd-files    253:0    0     1G  0 lvm  
│ └─drbd0          147:0    0  1024M  0 disk 

Above we can see that drbd0 is backed by the vg_drbd-files LVM device.

To take the backup from the secondary we'll be using LVM snapshots. The first step here is to identify the path of the LVM device. You can easily verify this by looking at the `*.res` resource configuration file for your DRBD resource. In this example it's `/dev/vg_drbd/files`.

root@booth-1:~# cat /etc/drbd.d/files.res | grep disk
  disk      /dev/vg_drbd/files;
  meta-disk internal;
  disk      /dev/vg_drbd/files;
  meta-disk internal;

Next we create the snapshot. Please note that this does require some free-space remain on the volume group, enough to store the delta of what's changed between the time the snapshot is created and when we delete it after taking our backup.

root@booth-1:~# lvcreate --size 1G --snapshot --name files_snap /dev/vg_drbd/files
  Logical volume "files_snap" created.
root@booth-1:~# lvs
[...]                                    
  files      vg_drbd owi-aos--- 1.00g                                                    
  files_snap vg_drbd swi-a-s--- 1.00g      files  0.00                                   

At this point you can either backup the /dev/vg_drbd/files_snap at the block layer via something like `dd`, or you can mount it to take a file based backup using something like tar, or rsync.

root@booth-1:~# mount /dev/vg_drbd/files_snap /files_snap -t xfs
root@booth-1:~# ls /files_snap/
testfile0  testfile1  testfile2  testfile3  testfile4  testfile5  testfile6  testfile7  testfile8

Please note that LVM snapshots operate on a Copy-On-Write design. This means that every write operation that would alter existing data will now require two writes. One for the original volume, and a copy to the snapshot. It is because of this it is advised to remove the LVM snapshot once the backup is complete.

root@booth-1:~# umount /files_snap 
root@booth-1:~# lvremove /dev/vg_drbd/files_snap 
Do you really want to remove and DISCARD active logical volume vg_drbd/files_snap? [y/n]: y
  Logical volume "files_snap" successfully removed

Written by DJV 2022-05-10