Also seeing many MM network problems with openqaworker7 in the HA group for a week or so.
Initially thought the problem was also present in openqaworker5, but haven't seen failed jobs with openqaworker5 due to network issues since the openqa.suse.de upgrade from July 9th/10th.
Failed tests sample:
1) https://openqa.suse.de/tests/3055962#step/setup/25 ([debug] Failed to run dbus command 'set_vlan' with arguments 'tap19 7' : 'tap19' is not connected to bridge 'br1')
2) https://openqa.suse.de/tests/3071752#step/setup/25 ([debug] Failed to run dbus command 'set_vlan' with arguments 'tap15 3' : 'tap15' is not connected to bridge 'br1')
3) https://openqa.suse.de/tests/3054586#step/upgrade_from_sle11sp4_workarounds/16 ([debug] Failed to run dbus command 'set_vlan' with arguments 'tap14 4' : 'tap14' is not connected to bridge 'br1')
4) https://openqa.suse.de/tests/3071747#step/upgrade_from_sle11sp4_workarounds/16 ([debug] Failed to run dbus command 'set_vlan' with arguments 'tap11 6' : 'tap11' is not connected to bridge 'br1')
5) https://openqa.suse.de/tests/3071758#step/upgrade_from_sle11sp4_workarounds/16 ([debug] Failed to run dbus command 'set_vlan' with arguments 'tap11 3' : 'tap11' is not connected to bridge 'br1')
The last job ran parallel with https://openqa.suse.de/tests/3071756 which also ran in openqaworker7 and doesn't show the 'set_vlan' error in its autoinst.log: https://openqa.suse.de/tests/3071756/file/autoinst-log.txt
It seems that second job used tap3, which looks different when checked with 'ip a' in openqaworker7:
41: tap3: mtu 1500 qdisc pfifo_fast master ovs-system state DOWN group default qlen 1000
link/ether e2:b9:5f:d0:95:1d brd ff:ff:ff:ff:ff:ff
vs.
11: tap11: mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000
link/ether c2:5b:f9:9c:f4:2b brd ff:ff:ff:ff:ff:ff
There's a startup file for both interfaces:
openqaworker7:~ # ls -l /etc/sysconfig/network/ifcfg-tap{3,11}
-rw------- 1 root root 141 Jul 8 15:55 /etc/sysconfig/network/ifcfg-tap11
-rw------- 1 root root 141 Jul 8 15:55 /etc/sysconfig/network/ifcfg-tap3
But tap11 is not present in /etc/sysconfig/network/ifcfg-br1:
openqaworker7:~ # egrep 'tap3|tap11' /etc/sysconfig/network/ifcfg-br1
OVS_BRIDGE_PORT_DEVICE_3='tap3'
In short, it seems that the bridge configuration is non-existent for 30 out 60 tap interfaces:
openqaworker7:~ # grep OVS_BRIDGE_PORT_DEVICE_ /etc/sysconfig/network/ifcfg-br1 |wc -l
30
openqaworker7:~ # ls /etc/sysconfig/network/ifcfg-tap*|wc -l
60
Checking other workers, this is not the case:
openqaworker6:~ # grep OVS_BRIDGE_PORT_DEVICE_ /etc/sysconfig/network/ifcfg-br1 |wc -l
60
openqaworker6:~ # ls /etc/sysconfig/network/ifcfg-tap*|wc -l
60
openqaworker5:~ # grep OVS_BRIDGE_PORT_DEVICE_ /etc/sysconfig/network/ifcfg-br1 |wc -l
66
openqaworker5:~ # ls /etc/sysconfig/network/ifcfg-tap*|wc -l
66