action #160826
closed
openQA Project (public) - coordination #112862: [saga][epic] Future ideas for easy multi-machine handling: MM-tests as first-class citizens
openQA Project (public) - coordination #111929: [epic] Stable multi-machine tests covering multiple physical workers
Optimize gre_tunnel_preup.sh generation jinja template size:S
Added by nicksinger 11 months ago.
Updated about 1 month ago.
Category:
Feature requests
Description
Observation¶
#160646-3 shows that we generate a lot of empty lines with https://gitlab.suse.de/openqa/salt-states-openqa/-/blob/master/openqa/openvswitch.sls?ref_type=heads#L96-128
This can be optimized by using {%- and -%} at a lot more places to avoid linebreak generation for e.g. if-statements of set commands. Care must be taken to not remove linebreaks where they are actually needed to produce valid yaml which results in a working shell script.
Acceptance criteria¶
- AC1: The rendered openvswitch state definition is more readable
Acceptance tests¶
- AT1-1:
salt 'worker29.oqa.prg2.suse.org' cp.get_template salt://openqa/openvswitch.sls /tmp/openvswitch shows no redundant blank lines
Suggestions¶
- Add a
-, e.g. {%- for remote in otherworkers -%}
https://progress.opensuse.org/issues/160826
Optimize gre_tunnel_preup.sh generation jinja template
Observation¶
#160646-3 shows that we generate a lot of empty lines with https://gitlab.suse.de/openqa/salt-states-openqa/-/blob/master/openqa/openvswitch.sls?ref_type=heads#L96-128
This can be optimized by using {%- and -%} at a lot more places to avoid linebreak generation for e.g. if-statements of set commands. Care must be taken to not remove linebreaks where they are actually needed to produce valid yaml which results in a working shell script.
Acceptance criteria¶
- AC1: The rendered openvswitch state definition is more readable
Acceptance tests¶
- AT1-1:
salt 'worker29.oqa.prg2.suse.org' cp.get_template salt://openqa/openvswitch.sls /tmp/openvswitch shows no redundant blank lines
Suggestions¶
- Add a
-, e.g. {%- for remote in otherworkers -%}
- Copied from action #160646: multiple multi-machine test failures, no GRE tunnels are setup between machines anymore at all size:M added
- Target version set to future
- Due date deleted (
2024-06-05)
- Parent task set to #111929
- Target version changed from future to Tools - Next
- Related to action #157606: Prevent missing gre tunnel connections in our salt states due to misconfiguration added
- Subject changed from Optimize gre_tunnel_preup.sh generation jinja template to Optimize gre_tunnel_preup.sh generation jinja template size:S
- Description updated (diff)
- Status changed from New to Workable
- Target version changed from Tools - Next to Ready
- Status changed from Workable to In Progress
- Status changed from In Progress to Feedback
- Status changed from Feedback to Blocked
Effectively blocking on #178564 due to unrelated pipeline failures.
- Status changed from Blocked to Feedback
- Status changed from Feedback to Resolved
- Status changed from Resolved to Workable
ok but now on w29 I saw from ssh worker29.oqa.prg2.suse.org showing
#!/bin/sh
action="$1"
bridge="$2"
# enable STP for the multihost bridges
ovs-vsctl set bridge $bridge stp_enable=false
ovs-vsctl set bridge $bridge rstp_enable=true
for gre_port in $(ovs-vsctl list-ifaces $bridge | grep gre) ; do ovs-vsctl --if-exists del-port $bridge $gre_port ; done
ovs-vsctl --may-exist add-port $bridge gre1 -- set interface gre1 type=gre options:remote_ip=10.145.10.5 # worker32
ovs-vsctl --may-exist add-port $bridge gre2 -- set interface gre2 type=gre options:remote_ip=10.145.10.9 # worker36
so only listing two connections, w32+w36, which is far less then expected.
okurz wrote in #note-15:
ok but now on w29 I saw from ssh worker29.oqa.prg2.suse.org showing
#!/bin/sh
action="$1"
bridge="$2"
# enable STP for the multihost bridges
ovs-vsctl set bridge $bridge stp_enable=false
ovs-vsctl set bridge $bridge rstp_enable=true
for gre_port in $(ovs-vsctl list-ifaces $bridge | grep gre) ; do ovs-vsctl --if-exists del-port $bridge $gre_port ; done
ovs-vsctl --may-exist add-port $bridge gre1 -- set interface gre1 type=gre options:remote_ip=10.145.10.5 # worker32
ovs-vsctl --may-exist add-port $bridge gre2 -- set interface gre2 type=gre options:remote_ip=10.145.10.9 # worker36
so only listing two connections, w32+w36, which is far less then expected.
This is unrelated to the changes done as part of this ticket.
Should we continue investigating in this ticket itself?
gpathak wrote in #note-16:
okurz wrote in #note-15:
ok but now on w29 I saw from ssh worker29.oqa.prg2.suse.org showing
#!/bin/sh
action="$1"
bridge="$2"
# enable STP for the multihost bridges
ovs-vsctl set bridge $bridge stp_enable=false
ovs-vsctl set bridge $bridge rstp_enable=true
for gre_port in $(ovs-vsctl list-ifaces $bridge | grep gre) ; do ovs-vsctl --if-exists del-port $bridge $gre_port ; done
ovs-vsctl --may-exist add-port $bridge gre1 -- set interface gre1 type=gre options:remote_ip=10.145.10.5 # worker32
ovs-vsctl --may-exist add-port $bridge gre2 -- set interface gre2 type=gre options:remote_ip=10.145.10.9 # worker36
so only listing two connections, w32+w36, which is far less then expected.
This is unrelated to the changes done as part of this ticket.
Should we continue investigating in this ticket itself?
maybe #162734 then? Either investigate as part of any of those two tickets or create a new one
- Status changed from Workable to In Progress
I tried debugging by adding some logs, added {% do salt.log.critical('multihostworkers: ' ~ multihostworkers) %} and % do salt.log.critical('otherworkers: ' ~ otherworkers) %} in openvswitch.sls at line 94 and 100 respectively and got:
2025-03-20 07:14:51,152 [salt.loaded.int.module.logmod:71 ][CRITICAL][13638] multihostworkers: ['worker29', 'worker32', 'worker36']
2025-03-20 07:14:51,153 [salt.loaded.int.module.logmod:71 ][CRITICAL][13638] otherworkers: ['worker32', 'worker36']
When I executed salt -l critical 'worker29.oqa.prg2.suse.org' cp.get_template salt://openqa/openvswitch.sls /tmp/openvswitch on OSD.
Did some more debugging, only worker32 and worker36 are getting passed from subset filter of /srv/salt/_modules/gre_peers.py, other Worker Class entry for host is either missing any one of tap or location-prg2e or location-prg2
okurz wrote in #note-15:
ok but now on w29 I saw from ssh worker29.oqa.prg2.suse.org showing
#!/bin/sh
action="$1"
bridge="$2"
# enable STP for the multihost bridges
ovs-vsctl set bridge $bridge stp_enable=false
ovs-vsctl set bridge $bridge rstp_enable=true
for gre_port in $(ovs-vsctl list-ifaces $bridge | grep gre) ; do ovs-vsctl --if-exists del-port $bridge $gre_port ; done
ovs-vsctl --may-exist add-port $bridge gre1 -- set interface gre1 type=gre options:remote_ip=10.145.10.5 # worker32
ovs-vsctl --may-exist add-port $bridge gre2 -- set interface gre2 type=gre options:remote_ip=10.145.10.9 # worker36
so only listing two connections, w32+w36, which is far less then expected.
I discussed this with @mkittler during our daily, and he suggested that it's possible workerconf.sls was modified. It needs to be updated to reflect the current machine locations, and the WORKER_CLASS may also need to be fixed in workerconf.sls to address the issue reported by @okurz.
Seems like we already have a ticket for this #157606 :-)
- Status changed from In Progress to Resolved
Also available in: Atom
PDF