QA » openQA Project » openQA Infrastructure

The problem was reproducible after each reboot.

I found some complains about a dependency.
I checked the drbd.service and it cant start. From the logs May 10 17:14:39 osiris-1 (drbd)[2493]: drbd.service: Failed at step EXEC spawning /lib/drbd/scripts/drbd: No such file or directory
The directory has been moved:

iob@osiris-1:~> ls /lib/drbd.rpmmoved/scripts/                                                                                                                                                                                               
drbd                     drbd-service-shim.sh     drbd-wait-promotable.sh  ocf.ra.wrapper.sh                                                                                                                                                 
iob@osiris-1:~> ls -la /lib/drbd.rpmmoved                                                                                                                                                                                                    
total 4                                                                                                                                                                                                                                      
drwxr-xr-x 1 root root  14 Sep 18  2023 .                                                                                                                                                                                                    
drwxr-xr-x 1 root root 898 May  8 19:38 ..                                                                                                                                                                                                   
lrwxrwxrwx 1 root root  21 Sep 18  2023 scripts -> /usr/lib/drbd/scripts
iob@osiris-1:~> ls /usr/lib/drbd/scripts/
drbd                     drbd-service-shim.sh     drbd-wait-promotable.sh  ocf.ra.wrapper.sh

2024-05-10 18:07:00 osiris-1 drbd 1

Is what I see on on the systemd services panel, with the alert still firing.

May 14 13:44:28 osiris-1 drbd[3897]: /usr/lib/drbd/scripts/drbd: line 148: /var/lib/linstor/loop_device_mapping: No such file or directory

According to Yannis this is safe to ignore due to an unrelated upstream bug.

We addressed the following errors by modifying /etc/drbd.d/qsf-cluster.res:

May 14 13:44:28 osiris-1 drbd[3940]: outdated-wfc-timeout has to be shorter than degr-wfc-timeout                                                                     
May 14 13:44:28 osiris-1 drbd[3940]: outdated-wfc-timeout implicitly set to degr-wfc-timeout (10s)

->

        wfc-timeout  30;                                                                                                                                              
        degr-wfc-timeout 15;                                                                                                                                          
        outdated-wfc-timeout 10;

However it's not clear if this addresses the underlying issue.

After the last test reboot neither of the mounts nor drbd were loaded and nothing failed. What, if anything, should load the devices/drbd/mounts? The mounts After=drbd. Researching the presumably related systemd error we couldn't find what it refers to:

-- Boot 2356cb9e267c474b8e5f8bb375e70678 --                                                                                                                           
May 14 14:13:10 osiris-1 systemd[1]: Dependency failed for Mount DRBD device with libvirtd configs.                                                                   
May 14 14:13:10 osiris-1 systemd[1]: etc-libvirt.mount: Job etc-libvirt.mount/start failed with result 'dependency'.

The only thing I have actually done is to fix the exec path of drbd. But still doesnt seem to fix the problem.
For some reason the service doesnt seem to run on reboot

I think the only missing part was systemctl enable drbd which is - despite the warning - required. More on that can be read here: https://progress.opensuse.org/issues/125087#note-13

I found an old bug of mine describing the same problems: https://bugzilla.opensuse.org/show_bug.cgi?id=1215462 - I updated it with the newest findings. A first test of mine to use the drbd@r0.service failed because the packaged script (/lib/drbd/scripts/drbd-service-shim.sh) fails to access another file:

drbd-r0[7168]: /lib/drbd/scripts/drbd-service-shim.sh: line 46: /usr/sbin/drbdsetup: No such file or directory

Guess I will try to add something in the lines of https://superuser.com/a/1322035 now explicitly depending on these directories to be present. Makes no sense because all files/scripts/symlinks are on the same btrfs subvolume but still worth a shot.