Project

General

Profile

action #133793

Updated by mkittler about 1 year ago

## Observation 
 See https://gitlab.suse.de/openqa/salt-pillars-openqa/-/jobs/1734178 running into the 2h gitlab CI timeout while applying a salt high state. There is a lot of not helpful debug output with all the lines with "Result: Clean - Started:" and a mention of hosts being down "backup.qa.suse.de" and "openqaworker1.qe.nue2.suse.org" but it's not being clear which minions in the end do not return 

 ## Acceptance criteria 
 * **AC1:** By default no lines with "Result: Clean - Started:": Put them in another logfile to be uploaded 
 * **AC2:** No repeated "++ true, ++ sleep 1, ++ echo -n .":  
 * **AC3:** We know which minions did not complete 

 ## Suggestions 
 * For "Result: Clean - Started:" how about just grep for those and move to a logfile? https://unix.stackexchange.com/questions/414042/how-to-split-an-output-to-two-files-with-grep can help 
 * okurz just used 

 ``` 
 sudo salt --no-color --state-output=changes 'backup-qam.qe.nue2.suse.org' state.apply queue=True | awk '/Result: Clean - Started/ {print > "/tmp/salt_profiling.log"; next} 1' 
 ``` 

 which provides nice terse output and all the profiling information into /tmp/salt_profiling.log 
 * Maybe don't apply the "set -x" for those commands with the dot outputting 

 ## Out of scope 
 * Timeout before the 2h gitlab CI timeout and write down which minions are still busy executing jobs -> #133457

Back