action #160826
closedopenQA Project (public) - coordination #112862: [saga][epic] Future ideas for easy multi-machine handling: MM-tests as first-class citizens
openQA Project (public) - coordination #111929: [epic] Stable multi-machine tests covering multiple physical workers
Optimize gre_tunnel_preup.sh generation jinja template size:S
0%
Description
Observation¶
#160646-3 shows that we generate a lot of empty lines with https://gitlab.suse.de/openqa/salt-states-openqa/-/blob/master/openqa/openvswitch.sls?ref_type=heads#L96-128
This can be optimized by using {%- and -%} at a lot more places to avoid linebreak generation for e.g. if-statements of set commands. Care must be taken to not remove linebreaks where they are actually needed to produce valid yaml which results in a working shell script.
Acceptance criteria¶
- AC1: The rendered openvswitch state definition is more readable
Acceptance tests¶
- AT1-1:
salt 'worker29.oqa.prg2.suse.org' cp.get_template salt://openqa/openvswitch.sls /tmp/openvswitchshows no redundant blank lines
Suggestions¶
- Add a
-, e.g.{%- for remote in otherworkers -%}
https://progress.opensuse.org/issues/160826
Optimize gre_tunnel_preup.sh generation jinja template
Observation¶
#160646-3 shows that we generate a lot of empty lines with https://gitlab.suse.de/openqa/salt-states-openqa/-/blob/master/openqa/openvswitch.sls?ref_type=heads#L96-128
This can be optimized by using {%- and -%} at a lot more places to avoid linebreak generation for e.g. if-statements of set commands. Care must be taken to not remove linebreaks where they are actually needed to produce valid yaml which results in a working shell script.
Acceptance criteria¶
- AC1: The rendered openvswitch state definition is more readable
Acceptance tests¶
- AT1-1:
salt 'worker29.oqa.prg2.suse.org' cp.get_template salt://openqa/openvswitch.sls /tmp/openvswitchshows no redundant blank lines
Suggestions¶
- Add a
-, e.g.{%- for remote in otherworkers -%}
Updated by nicksinger 11 months ago
- Copied from action #160646: multiple multi-machine test failures, no GRE tunnels are setup between machines anymore at all size:M added
Updated by okurz 9 months ago
- Related to action #157606: Prevent missing gre tunnel connections in our salt states due to misconfiguration added
Updated by okurz about 2 months ago
- Target version changed from Tools - Next to Ready
Updated by gpathak about 1 month ago
- Status changed from Workable to In Progress
Created a Merge request: https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/1402
Updated by gpathak about 1 month ago
- Status changed from In Progress to Feedback
Updated by livdywan about 1 month ago
- Status changed from Feedback to Blocked
Effectively blocking on #178564 due to unrelated pipeline failures.
Updated by gpathak about 1 month ago
- Status changed from Feedback to Resolved
gpathak wrote in #note-10:
Created a Merge request: https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/1402
Changes merged.
Updated by okurz about 1 month ago
- Status changed from Resolved to Workable
ok but now on w29 I saw from ssh worker29.oqa.prg2.suse.org showing
#!/bin/sh
action="$1"
bridge="$2"
# enable STP for the multihost bridges
ovs-vsctl set bridge $bridge stp_enable=false
ovs-vsctl set bridge $bridge rstp_enable=true
for gre_port in $(ovs-vsctl list-ifaces $bridge | grep gre) ; do ovs-vsctl --if-exists del-port $bridge $gre_port ; done
ovs-vsctl --may-exist add-port $bridge gre1 -- set interface gre1 type=gre options:remote_ip=10.145.10.5 # worker32
ovs-vsctl --may-exist add-port $bridge gre2 -- set interface gre2 type=gre options:remote_ip=10.145.10.9 # worker36
so only listing two connections, w32+w36, which is far less then expected.
Updated by gpathak about 1 month ago
okurz wrote in #note-15:
ok but now on w29 I saw from
ssh worker29.oqa.prg2.suse.orgshowing#!/bin/sh action="$1" bridge="$2" # enable STP for the multihost bridges ovs-vsctl set bridge $bridge stp_enable=false ovs-vsctl set bridge $bridge rstp_enable=true for gre_port in $(ovs-vsctl list-ifaces $bridge | grep gre) ; do ovs-vsctl --if-exists del-port $bridge $gre_port ; done ovs-vsctl --may-exist add-port $bridge gre1 -- set interface gre1 type=gre options:remote_ip=10.145.10.5 # worker32 ovs-vsctl --may-exist add-port $bridge gre2 -- set interface gre2 type=gre options:remote_ip=10.145.10.9 # worker36so only listing two connections, w32+w36, which is far less then expected.
This is unrelated to the changes done as part of this ticket.
Should we continue investigating in this ticket itself?
Updated by okurz about 1 month ago
gpathak wrote in #note-16:
okurz wrote in #note-15:
ok but now on w29 I saw from
ssh worker29.oqa.prg2.suse.orgshowing#!/bin/sh action="$1" bridge="$2" # enable STP for the multihost bridges ovs-vsctl set bridge $bridge stp_enable=false ovs-vsctl set bridge $bridge rstp_enable=true for gre_port in $(ovs-vsctl list-ifaces $bridge | grep gre) ; do ovs-vsctl --if-exists del-port $bridge $gre_port ; done ovs-vsctl --may-exist add-port $bridge gre1 -- set interface gre1 type=gre options:remote_ip=10.145.10.5 # worker32 ovs-vsctl --may-exist add-port $bridge gre2 -- set interface gre2 type=gre options:remote_ip=10.145.10.9 # worker36so only listing two connections, w32+w36, which is far less then expected.
This is unrelated to the changes done as part of this ticket.
Should we continue investigating in this ticket itself?
maybe #162734 then? Either investigate as part of any of those two tickets or create a new one
Updated by gpathak about 1 month ago
- Status changed from Workable to In Progress
I tried debugging by adding some logs, added {% do salt.log.critical('multihostworkers: ' ~ multihostworkers) %} and % do salt.log.critical('otherworkers: ' ~ otherworkers) %} in openvswitch.sls at line 94 and 100 respectively and got:
2025-03-20 07:14:51,152 [salt.loaded.int.module.logmod:71 ][CRITICAL][13638] multihostworkers: ['worker29', 'worker32', 'worker36']
2025-03-20 07:14:51,153 [salt.loaded.int.module.logmod:71 ][CRITICAL][13638] otherworkers: ['worker32', 'worker36']
When I executed salt -l critical 'worker29.oqa.prg2.suse.org' cp.get_template salt://openqa/openvswitch.sls /tmp/openvswitch on OSD.
Updated by gpathak about 1 month ago
Did some more debugging, only worker32 and worker36 are getting passed from subset filter of /srv/salt/_modules/gre_peers.py, other Worker Class entry for host is either missing any one of tap or location-prg2e or location-prg2
Updated by gpathak about 1 month ago
okurz wrote in #note-15:
ok but now on w29 I saw from
ssh worker29.oqa.prg2.suse.orgshowing#!/bin/sh action="$1" bridge="$2" # enable STP for the multihost bridges ovs-vsctl set bridge $bridge stp_enable=false ovs-vsctl set bridge $bridge rstp_enable=true for gre_port in $(ovs-vsctl list-ifaces $bridge | grep gre) ; do ovs-vsctl --if-exists del-port $bridge $gre_port ; done ovs-vsctl --may-exist add-port $bridge gre1 -- set interface gre1 type=gre options:remote_ip=10.145.10.5 # worker32 ovs-vsctl --may-exist add-port $bridge gre2 -- set interface gre2 type=gre options:remote_ip=10.145.10.9 # worker36so only listing two connections, w32+w36, which is far less then expected.
I discussed this with @mkittler during our daily, and he suggested that it's possible workerconf.sls was modified. It needs to be updated to reflect the current machine locations, and the WORKER_CLASS may also need to be fixed in workerconf.sls to address the issue reported by @okurz.
Updated by gpathak about 1 month ago
Seems like we already have a ticket for this #157606 :-)