Actions
action #133469
closed[alert] Salt states don't apply sometimes on individual workers size:M
Added by mkittler over 1 year ago. Updated over 1 year ago.
Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
Start date:
2023-07-27
Due date:
% Done:
0%
Estimated time:
Description
Observation¶
This lead to https://gitlab.suse.de/openqa/salt-pillars-openqa/-/jobs/1716103/raw failing with:
ERROR: Minions returned with non-zero exit code
openqaworker-arm-3.suse.de:
Passed invalid arguments to state.highstate: expected str, bytes or os.PathLike object, not list
…
Trying to apply the states manually leads to the following error (which persisted after multiple attempts and waiting a few minutes between them):
martchus@openqa:~> sudo salt -C 'openqaworker-arm-3.suse.de' -l error --state-output=changes state.apply
openqaworker-arm-3.suse.de:
Data failed to compile:
----------
The function "state.highstate" is running as PID 5343 and was started at 2023, Jul 24 21:09:36.760016 with jid 20230724210936760016
----------
The function "state.highstate" is running as PID 93318 and was started at 2023, Jul 27 14:59:24.641229 with jid 20230727145924641229
ERROR: Minions returned with non-zero exit code
Acceptance criteria¶
- AC1: It is known what causes the problem
- AC2: The problem is fixed or worked around as far as possible
Suggestions¶
- We have seen this problem before on other workers (see #95140) so it would make sense to research more.
- Research if the problem is discussed upstream
- If nothing can be found ask directly in e.g. chats or the suse internal mailinglist salt@suse.de
Updated by okurz over 1 year ago
We have seen this before but not in recent weeks. I don't know if this is a sporadic error. Maybe salt minion logs will tell more details
Updated by okurz over 1 year ago
- Related to action #95140: regression: salt state fails on one worker with "Passed invalid arguments to state.highstate: expected str, bytes or os.PathLike object, not list" size:M added
Updated by mkittler over 1 year ago
- Subject changed from [alert] Salt states don't apply on openqaworker-arm-3 to [alert] Salt states don't apply sometimes on individual workers size:M
- Description updated (diff)
- Status changed from New to Workable
The issue has fixed itself, currently only the following state is failing:
ID: /var/lib/openqa/share
Function: mount.mounted
Result: False
Comment: The following requisites were not found:
require:
sls: static_nfs_hostname
Started: 13:18:53.824692
Duration: 0.04 ms
So this is indeed just like #95140.
Updated by okurz over 1 year ago
latest occurence https://gitlab.suse.de/openqa/salt-pillars-openqa/-/jobs/1734165/raw on worker35.oqa.prg2.suse.org.
Updated by okurz over 1 year ago
- Related to action #133793: salt-pillars-openqa failing to apply within 2h and it is not clear which minion(s) are missing size:M added
Updated by nicksinger over 1 year ago
- Status changed from Workable to Feedback
https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/964 to introduce some more insights on what wrong parameters are passed
Updated by okurz over 1 year ago
Updated by okurz over 1 year ago
- Related to action #135134: [tools] GitlabCI salt-pillars-openqa deploy failed on baremetal-support.qa.suse.de added
Updated by nicksinger over 1 year ago
- Priority changed from High to Normal
Checked after the weekend and saw nothing pointing to issues (we also didn't see any pipeline issues so nothing unexpected). This is how it looks atm:
openqa:~ # salt --state-output=changes --state-verbose=false '*' cmd.run 'salt-minion --version && journalctl --since="1 week ago" -u salt-minion | grep "args:" | grep highstate'
backup-qam.qe.nue2.suse.org:
salt-minion 3004
storage.oqa.suse.de:
salt-minion 3004
Aug 31 22:45:07 storage salt-minion[31462]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 10:29:50 storage salt-minion[32601]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 10:41:17 storage salt-minion[3520]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 01 10:45:02 storage salt-minion[4386]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 13:36:47 storage salt-minion[10946]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 18:33:42 storage salt-minion[14438]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:22:27 storage salt-minion[22517]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:49:52 storage salt-minion[29599]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:33 storage salt-minion[1077]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:43 storage salt-minion[10950]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 storage salt-minion[17112]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 storage salt-minion[3847]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 storage salt-minion[4263]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
worker36.oqa.prg2.suse.org:
salt-minion 3004
Sep 04 11:22:36 worker36 salt-minion[50861]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:49:57 worker36 salt-minion[55448]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:40 worker36 salt-minion[57933]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:49 worker36 salt-minion[68480]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 worker36 salt-minion[76789]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 worker36 salt-minion[35486]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 worker36 salt-minion[35789]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
worker30.oqa.prg2.suse.org:
salt-minion 3004
Sep 04 11:22:36 worker30 salt-minion[51618]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:49:59 worker30 salt-minion[56239]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:41 worker30 salt-minion[58742]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:50 worker30 salt-minion[69982]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 worker30 salt-minion[76696]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 worker30 salt-minion[45192]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 worker30 salt-minion[45399]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
worker35.oqa.prg2.suse.org:
salt-minion 3004
Sep 04 11:22:36 worker35 salt-minion[48992]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:49:57 worker35 salt-minion[53582]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:39 worker35 salt-minion[56069]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:47 worker35 salt-minion[64173]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 worker35 salt-minion[69744]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 worker35 salt-minion[15017]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 worker35 salt-minion[15816]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
worker39.oqa.prg2.suse.org:
salt-minion 3004
Sep 04 11:22:35 worker39 salt-minion[63508]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:49:58 worker39 salt-minion[68132]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:39 worker39 salt-minion[70616]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:50 worker39 salt-minion[82868]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 worker39 salt-minion[93740]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 worker39 salt-minion[95321]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 worker39 salt-minion[95636]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
worker33.oqa.prg2.suse.org:
salt-minion 3004
Sep 04 11:22:35 worker33 salt-minion[58622]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:49:57 worker33 salt-minion[63208]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:39 worker33 salt-minion[65692]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:49 worker33 salt-minion[77119]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 worker33 salt-minion[83474]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 worker33 salt-minion[40105]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 worker33 salt-minion[40418]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
worker29.oqa.prg2.suse.org:
salt-minion 3004
Sep 04 11:22:36 worker29 salt-minion[51998]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:49:59 worker29 salt-minion[56644]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:41 worker29 salt-minion[59149]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:51 worker29 salt-minion[71191]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 worker29 salt-minion[82745]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 worker29 salt-minion[56550]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 worker29 salt-minion[57406]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
worker34.oqa.prg2.suse.org:
salt-minion 3004
Sep 04 11:22:35 worker34 salt-minion[23553]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:49:57 worker34 salt-minion[28156]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:40 worker34 salt-minion[30642]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:48 worker34 salt-minion[46338]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 worker34 salt-minion[58515]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 worker34 salt-minion[28408]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 worker34 salt-minion[28782]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
worker31.oqa.prg2.suse.org:
salt-minion 3004
Sep 04 11:22:36 worker31 salt-minion[131071]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:49:59 worker31 salt-minion[5906]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:41 worker31 salt-minion[8406]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:49 worker31 salt-minion[21355]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 worker31 salt-minion[32369]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 worker31 salt-minion[32691]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 worker31 salt-minion[33085]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
openqaworker16.qa.suse.cz:
salt-minion 3004
Sep 04 11:22:38 openqaworker16 salt-minion[62989]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:01 openqaworker16 salt-minion[1214]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:43 openqaworker16 salt-minion[3200]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:54 openqaworker16 salt-minion[10693]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 openqaworker16 salt-minion[19219]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 openqaworker16 salt-minion[63122]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 openqaworker16 salt-minion[63751]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
worker37.oqa.prg2.suse.org:
salt-minion 3004
Sep 04 11:22:36 worker37 salt-minion[41794]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:49:57 worker37 salt-minion[46398]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:39 worker37 salt-minion[48887]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:47 worker37 salt-minion[63519]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 worker37 salt-minion[73962]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 worker37 salt-minion[65720]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 worker37 salt-minion[66016]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
openqaworker17.qa.suse.cz:
salt-minion 3004
Sep 04 11:22:38 openqaworker17 salt-minion[49431]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:01 openqaworker17 salt-minion[52664]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:43 openqaworker17 salt-minion[54367]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:52 openqaworker17 salt-minion[60774]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 openqaworker17 salt-minion[1452]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 openqaworker17 salt-minion[59504]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 openqaworker17 salt-minion[61433]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
worker32.oqa.prg2.suse.org:
salt-minion 3004
Sep 04 11:22:36 worker32 salt-minion[52877]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:49:58 worker32 salt-minion[57510]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:41 worker32 salt-minion[60017]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:50 worker32 salt-minion[71812]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 worker32 salt-minion[83148]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 worker32 salt-minion[53497]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 worker32 salt-minion[53629]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
worker38.oqa.prg2.suse.org:
salt-minion 3004
Sep 04 11:22:35 worker38 salt-minion[22796]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:49:58 worker38 salt-minion[27384]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:40 worker38 salt-minion[29869]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:50 worker38 salt-minion[46462]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 worker38 salt-minion[57763]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 worker38 salt-minion[24556]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 worker38 salt-minion[25509]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
openqaworker18.qa.suse.cz:
salt-minion 3004
Sep 04 11:22:36 openqaworker18 salt-minion[51778]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:49:59 openqaworker18 salt-minion[55035]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:40 openqaworker18 salt-minion[56735]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:50 openqaworker18 salt-minion[64260]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 openqaworker18 salt-minion[6658]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 openqaworker18 salt-minion[49333]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 openqaworker18 salt-minion[49840]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
openqaworker14.qa.suse.cz:
salt-minion 3004
Sep 04 11:22:43 openqaworker14 salt-minion[77412]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:09 openqaworker14 salt-minion[80571]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:50 openqaworker14 salt-minion[875]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:46:00 openqaworker14 salt-minion[6841]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 openqaworker14 salt-minion[12424]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 openqaworker14 salt-minion[53987]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 openqaworker14 salt-minion[54054]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
worker40.oqa.prg2.suse.org:
salt-minion 3004
Sep 04 11:22:35 worker40 salt-minion[68369]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:49:57 worker40 salt-minion[72951]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:40 worker40 salt-minion[75435]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:50 worker40 salt-minion[86024]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 worker40 salt-minion[94446]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 worker40 salt-minion[63649]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 worker40 salt-minion[63787]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
qamasternue.qa.suse.de:
salt-minion 3004
Sep 01 10:30:05 qamaster salt-minion[13259]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 10:41:17 qamaster salt-minion[16751]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 01 10:45:17 qamaster salt-minion[17492]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 13:37:08 qamaster salt-minion[26238]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 18:33:59 qamaster salt-minion[31763]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:22:41 qamaster salt-minion[25262]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:11 qamaster salt-minion[32674]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:48 qamaster salt-minion[4352]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:46:20 qamaster salt-minion[14736]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:38 qamaster salt-minion[21213]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:06 qamaster salt-minion[11984]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 qamaster salt-minion[12414]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
worker9.oqa.suse.de:
salt-minion 3004
Sep 04 11:22:40 worker9 salt-minion[45236]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:03 worker9 salt-minion[49423]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:48 worker9 salt-minion[51703]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:58 worker9 salt-minion[60986]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 worker9 salt-minion[4955]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 worker9 salt-minion[61365]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 worker9 salt-minion[61713]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
worker8.oqa.suse.de:
salt-minion 3004
Sep 04 11:22:39 worker8 salt-minion[2263]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:05 worker8 salt-minion[7015]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:47 worker8 salt-minion[9531]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:59 worker8 salt-minion[16969]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 worker8 salt-minion[22772]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 worker8 salt-minion[10731]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 worker8 salt-minion[11140]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
worker-arm1.oqa.prg2.suse.org:
salt-minion 3004
Sep 04 11:22:31 worker-arm1 salt-minion[19337]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:49:56 worker-arm1 salt-minion[24068]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:38 worker-arm1 salt-minion[26648]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:51 worker-arm1 salt-minion[47318]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 worker-arm1 salt-minion[71442]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 worker-arm1 salt-minion[52922]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 worker-arm1 salt-minion[53207]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
worker-arm2.oqa.prg2.suse.org:
salt-minion 3004
Sep 04 11:22:35 worker-arm2 salt-minion[16114]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:49:57 worker-arm2 salt-minion[20923]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:39 worker-arm2 salt-minion[23475]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:50 worker-arm2 salt-minion[41931]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 worker-arm2 salt-minion[67743]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 worker-arm2 salt-minion[39359]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 worker-arm2 salt-minion[39588]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
openqa-monitor.qa.suse.de:
salt-minion 3004
Sep 04 11:25:08 monitor salt-minion[13598]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:51:09 monitor salt-minion[20683]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:05:48 monitor salt-minion[24558]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:49:27 monitor salt-minion[1701]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:58 monitor salt-minion[7430]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 monitor salt-minion[17638]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:11 monitor salt-minion[17822]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
qesapworker-prg6.qa.suse.cz:
salt-minion 3004
Sep 04 11:22:38 qesapworker-prg6 salt-minion[23250]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:02 qesapworker-prg6 salt-minion[26535]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:47 qesapworker-prg6 salt-minion[28259]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:58 qesapworker-prg6 salt-minion[32552]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 qesapworker-prg6 salt-minion[35020]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 qesapworker-prg6 salt-minion[55362]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 qesapworker-prg6 salt-minion[55430]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
sapworker3.qe.nue2.suse.org:
salt-minion 3004
Aug 31 11:21:50 sapworker3 salt-minion[21770]: [WARNING ] state.highstate args: [{'queue': True}]
Aug 31 11:22:28 sapworker3 salt-minion[22676]: [WARNING ] state.highstate args: [{'queue': True}]
Aug 31 13:22:56 sapworker3 salt-minion[36312]: [WARNING ] state.highstate args: [{'queue': True}]
Aug 31 16:10:02 sapworker3 salt-minion[69251]: [WARNING ] state.highstate args: [{'queue': True}]
Aug 31 20:58:44 sapworker3 salt-minion[97406]: [WARNING ] state.highstate args: [{'queue': True}]
Aug 31 22:41:58 sapworker3 salt-minion[105964]: [WARNING ] state.highstate args: [{'queue': True}]
Aug 31 22:45:13 sapworker3 salt-minion[107116]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 10:29:52 sapworker3 salt-minion[121943]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 10:41:17 sapworker3 salt-minion[123595]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 01 10:45:06 sapworker3 salt-minion[123856]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 13:36:52 sapworker3 salt-minion[16268]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 18:33:44 sapworker3 salt-minion[43467]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:22:37 sapworker3 salt-minion[29435]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:49:58 sapworker3 salt-minion[32707]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:39 sapworker3 salt-minion[34423]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:47 sapworker3 salt-minion[41492]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 sapworker3 salt-minion[47688]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 sapworker3 salt-minion[86277]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 sapworker3 salt-minion[86416]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
qesapworker-prg5.qa.suse.cz:
salt-minion 3004
Sep 04 11:22:40 qesapworker-prg5 salt-minion[23127]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:03 qesapworker-prg5 salt-minion[26406]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:43 qesapworker-prg5 salt-minion[28129]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:55 qesapworker-prg5 salt-minion[32397]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 qesapworker-prg5 salt-minion[34875]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 qesapworker-prg5 salt-minion[55161]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 qesapworker-prg5 salt-minion[55249]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
qesapworker-prg4.qa.suse.cz:
salt-minion 3004
Sep 04 11:22:40 qesapworker-prg4 salt-minion[24716]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:02 qesapworker-prg4 salt-minion[28085]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:48 qesapworker-prg4 salt-minion[29859]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:56 qesapworker-prg4 salt-minion[34238]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 qesapworker-prg4 salt-minion[36757]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 qesapworker-prg4 salt-minion[52971]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 qesapworker-prg4 salt-minion[53040]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
sapworker1.qe.nue2.suse.org:
salt-minion 3004
Sep 04 11:22:29 sapworker1 salt-minion[41078]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:49:57 sapworker1 salt-minion[44274]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:38 sapworker1 salt-minion[45943]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:48 sapworker1 salt-minion[53580]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 sapworker1 salt-minion[61446]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 sapworker1 salt-minion[4376]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 sapworker1 salt-minion[4530]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
qesapworker-prg7.qa.suse.cz:
salt-minion 3004
Sep 04 11:22:40 qesapworker-prg7 salt-minion[23130]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:01 qesapworker-prg7 salt-minion[26409]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:43 qesapworker-prg7 salt-minion[28137]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:56 qesapworker-prg7 salt-minion[32417]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 qesapworker-prg7 salt-minion[34884]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 qesapworker-prg7 salt-minion[50940]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 qesapworker-prg7 salt-minion[51008]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
sapworker2.qe.nue2.suse.org:
salt-minion 3004
Sep 04 11:22:35 sapworker2 salt-minion[40033]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:49:59 sapworker2 salt-minion[43320]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:38 sapworker2 salt-minion[45037]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:46 sapworker2 salt-minion[53566]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:37 sapworker2 salt-minion[59321]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 sapworker2 salt-minion[113827]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 sapworker2 salt-minion[114030]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
worker5.oqa.suse.de:
salt-minion 3004
Sep 04 11:22:43 worker5 salt-minion[37172]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:07 worker5 salt-minion[1174]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:49 worker5 salt-minion[3841]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:46:04 worker5 salt-minion[12267]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:40 worker5 salt-minion[20388]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 worker5 salt-minion[1865]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 worker5 salt-minion[2314]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
worker2.oqa.suse.de:
salt-minion 3004
Sep 04 11:22:43 worker2 salt-minion[9879]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:09 worker2 salt-minion[14579]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:49 worker2 salt-minion[17122]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:58 worker2 salt-minion[23943]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:40 worker2 salt-minion[30443]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 worker2 salt-minion[11760]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 worker2 salt-minion[12227]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
malbec.arch.suse.de:
salt-minion 3004
Sep 01 10:30:25 malbec salt-minion[135586]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 10:41:17 malbec salt-minion[137954]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 01 10:45:37 malbec salt-minion[138394]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 13:37:23 malbec salt-minion[162447]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 18:34:18 malbec salt-minion[43902]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:23:04 malbec salt-minion[92177]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:27 malbec salt-minion[97182]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:05:10 malbec salt-minion[99723]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:46:18 malbec salt-minion[106429]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:41 malbec salt-minion[110374]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 malbec salt-minion[139948]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 malbec salt-minion[140319]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
openqa-piworker.qa.suse.de:
salt-minion 3005.1
Sep 04 11:23:12 openqa-piworker salt-minion[25164]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:34 openqa-piworker salt-minion[31947]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:05:19 openqa-piworker salt-minion[3235]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:46:25 openqa-piworker salt-minion[12935]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:38 openqa-piworker salt-minion[18985]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:07 openqa-piworker salt-minion[3626]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:11 openqa-piworker salt-minion[3865]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
QA-Power8-5-kvm.qa.suse.de:
salt-minion 3004
Sep 01 10:30:07 QA-Power8-5-kvm salt-minion[10004]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 10:41:17 QA-Power8-5-kvm salt-minion[11580]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 01 10:45:23 QA-Power8-5-kvm salt-minion[11847]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 13:37:08 QA-Power8-5-kvm salt-minion[34288]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 18:34:00 QA-Power8-5-kvm salt-minion[24051]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:22:47 QA-Power8-5-kvm salt-minion[58837]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:11 QA-Power8-5-kvm salt-minion[61956]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:55 QA-Power8-5-kvm salt-minion[65722]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:46:02 QA-Power8-5-kvm salt-minion[70674]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:43 QA-Power8-5-kvm salt-minion[75737]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 QA-Power8-5-kvm salt-minion[100291]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 QA-Power8-5-kvm salt-minion[100378]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
powerqaworker-qam-1.qa.suse.de:
salt-minion 3004
Sep 01 10:30:03 powerqaworker-qam-1 salt-minion[97398]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 10:41:17 powerqaworker-qam-1 salt-minion[99687]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 01 10:45:16 powerqaworker-qam-1 salt-minion[100171]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 13:37:04 powerqaworker-qam-1 salt-minion[124185]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 18:33:56 powerqaworker-qam-1 salt-minion[36838]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:22:44 powerqaworker-qam-1 salt-minion[124116]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:08 powerqaworker-qam-1 salt-minion[128809]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:49 powerqaworker-qam-1 salt-minion[1051]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:56 powerqaworker-qam-1 salt-minion[8351]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:42 powerqaworker-qam-1 salt-minion[12740]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 powerqaworker-qam-1 salt-minion[44524]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 powerqaworker-qam-1 salt-minion[44689]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
QA-Power8-4-kvm.qa.suse.de:
salt-minion 3004
Sep 04 11:22:59 QA-Power8-4-kvm salt-minion[3063]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:23 QA-Power8-4-kvm salt-minion[6074]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:05:06 QA-Power8-4-kvm salt-minion[7676]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:46:15 QA-Power8-4-kvm salt-minion[12500]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:42 QA-Power8-4-kvm salt-minion[15438]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 QA-Power8-4-kvm salt-minion[33153]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 QA-Power8-4-kvm salt-minion[33244]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
openqaw5-xen.qa.suse.de:
salt-minion 3004
Sep 01 10:29:57 openqaw5-xen salt-minion[24632]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 10:41:17 openqaw5-xen salt-minion[27213]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 01 10:45:12 openqaw5-xen salt-minion[27793]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 13:37:00 openqaw5-xen salt-minion[3592]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 18:33:47 openqaw5-xen salt-minion[27447]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:22:35 openqaw5-xen salt-minion[10317]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:49:58 openqaw5-xen salt-minion[15846]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:41 openqaw5-xen salt-minion[18826]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:52 openqaw5-xen salt-minion[26674]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:44 openqaw5-xen salt-minion[327]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:11 openqaw5-xen salt-minion[18990]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 openqaw5-xen salt-minion[19198]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
worker13.oqa.suse.de:
salt-minion 3004
Sep 04 11:23:10 worker13 salt-minion[1430]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:35 worker13 salt-minion[5810]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:05:17 worker13 salt-minion[8125]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:46:35 worker13 salt-minion[15408]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:40 worker13 salt-minion[21846]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 worker13 salt-minion[7943]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 worker13 salt-minion[8494]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
worker10.oqa.suse.de:
salt-minion 3004
Sep 04 11:23:16 worker10 salt-minion[15350]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:37 worker10 salt-minion[19394]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:05:19 worker10 salt-minion[21608]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:46:32 worker10 salt-minion[28126]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:40 worker10 salt-minion[915]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 worker10 salt-minion[1519]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 worker10 salt-minion[2431]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
openqaworker1.qe.nue2.suse.org:
salt-minion 3004
Sep 04 11:22:42 openqaworker1 salt-minion[14688]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:04 openqaworker1 salt-minion[18180]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:47 openqaworker1 salt-minion[19995]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:46:02 openqaworker1 salt-minion[26987]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:42 openqaworker1 salt-minion[2587]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 openqaworker1 salt-minion[10494]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 openqaworker1 salt-minion[10656]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
openqaworker-arm-2.suse.de:
salt-minion 3004
Sep 04 11:24:41 openqaworker-arm-2 salt-minion[74623]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:51:59 openqaworker-arm-2 salt-minion[77001]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:06:43 openqaworker-arm-2 salt-minion[78745]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:47:56 openqaworker-arm-2 salt-minion[81990]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:41 openqaworker-arm-2 salt-minion[89235]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:07 openqaworker-arm-2 salt-minion[75599]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:12 openqaworker-arm-2 salt-minion[76027]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
openqaworker-arm-3.suse.de:
salt-minion 3004
Sep 04 11:24:44 openqaworker-arm-3 salt-minion[83917]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:52:11 openqaworker-arm-3 salt-minion[86275]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:06:48 openqaworker-arm-3 salt-minion[88867]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:48:04 openqaworker-arm-3 salt-minion[92075]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:40 openqaworker-arm-3 salt-minion[1788]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:08 openqaworker-arm-3 salt-minion[89546]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:13 openqaworker-arm-3 salt-minion[89997]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
worker3.oqa.suse.de:
salt-minion 3004
Sep 04 11:22:39 worker3 salt-minion[27045]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:02 worker3 salt-minion[31342]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:44 worker3 salt-minion[1331]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:45:54 worker3 salt-minion[8047]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:42 worker3 salt-minion[13609]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:05 worker3 salt-minion[30123]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 worker3 salt-minion[30493]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
openqa.suse.de:
salt-minion 3004
Sep 01 10:35:42 openqa salt-minion[7849]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 10:41:18 openqa salt-minion[29174]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 01 10:48:55 openqa salt-minion[22618]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 11:22:45 openqa salt-minion[5709]: [WARNING ] state.highstate args: []
Sep 01 11:36:09 openqa salt-minion[25342]: [WARNING ] state.highstate args: []
Sep 01 11:44:18 openqa salt-minion[32485]: [WARNING ] state.highstate args: []
Sep 01 13:44:16 openqa salt-minion[26273]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 18:38:02 openqa salt-minion[24077]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:22:59 openqa salt-minion[8313]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:33 openqa salt-minion[5949]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:05:15 openqa salt-minion[24436]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:51:08 openqa salt-minion[15702]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:10:40 openqa salt-minion[21998]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:10 openqa salt-minion[3460]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 openqa salt-minion[10506]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
backup.qa.suse.de:
salt-minion 3004
Sep 01 10:30:12 backup-vm salt-minion[6950]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 10:41:17 backup-vm salt-minion[9517]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 01 10:45:25 backup-vm salt-minion[10085]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 13:37:40 backup-vm salt-minion[8493]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 18:34:05 backup-vm salt-minion[28833]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:22:48 backup-vm salt-minion[4248]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:14 backup-vm salt-minion[9711]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:05:02 backup-vm salt-minion[12643]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:48:11 backup-vm salt-minion[20040]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:11:19 backup-vm salt-minion[24887]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:27 backup-vm salt-minion[597]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:11 backup-vm salt-minion[809]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
schort-server.qa.suse.de:
salt-minion 3004
Sep 01 10:30:20 schort-server salt-minion[15347]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 10:41:17 schort-server salt-minion[17902]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 01 10:45:30 schort-server salt-minion[18464]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 13:37:28 schort-server salt-minion[16845]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 18:34:25 schort-server salt-minion[4626]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:22:53 schort-server salt-minion[9054]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:23 schort-server salt-minion[14496]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:59 schort-server salt-minion[17409]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:49:32 schort-server salt-minion[24726]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:11:05 schort-server salt-minion[29653]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:24 schort-server salt-minion[3955]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:11 schort-server salt-minion[4136]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
tumblesle.qa.suse.de:
salt-minion 3004
Sep 01 10:30:20 tumblesle salt-minion[11730]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 10:41:18 tumblesle salt-minion[14287]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 01 10:45:31 tumblesle salt-minion[14850]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 13:37:20 tumblesle salt-minion[13138]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 18:34:12 tumblesle salt-minion[770]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:22:57 tumblesle salt-minion[5682]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:23 tumblesle salt-minion[11113]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:05:03 tumblesle salt-minion[14021]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:48:58 tumblesle salt-minion[21310]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:11:34 tumblesle salt-minion[26227]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:38 tumblesle salt-minion[1897]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:11 tumblesle salt-minion[2086]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
baremetal-support.qa.suse.de:
salt-minion 3004
Sep 01 13:38:38 baremetal-support salt-minion[2443]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 18:35:37 baremetal-support salt-minion[30861]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:28:03 baremetal-support salt-minion[26530]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:55:33 baremetal-support salt-minion[476]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:47 baremetal-support salt-minion[4441]: [WARNING ] state.highstate args: []
Sep 04 12:05:00 baremetal-support salt-minion[4508]: [WARNING ] state.highstate args: []
Sep 04 12:05:04 baremetal-support salt-minion[4518]: [WARNING ] state.highstate args: []
Sep 04 12:05:25 baremetal-support salt-minion[4599]: [WARNING ] state.highstate args: []
Sep 04 12:06:55 baremetal-support salt-minion[4949]: [WARNING ] state.highstate args: []
Sep 04 12:10:12 baremetal-support salt-minion[4063]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:17:26 baremetal-support salt-minion[5984]: [WARNING ] state.highstate args: []
Sep 04 12:37:14 baremetal-support salt-minion[10325]: [WARNING ] state.highstate args: []
Sep 04 12:47:08 baremetal-support salt-minion[14975]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:11:04 baremetal-support salt-minion[21037]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:24 baremetal-support salt-minion[3722]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:10 baremetal-support salt-minion[3950]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
jenkins.qa.suse.de:
salt-minion 3004
Sep 01 10:30:22 jenkins salt-minion[14093]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 10:41:17 jenkins salt-minion[17112]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 01 10:45:52 jenkins salt-minion[17763]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 13:37:41 jenkins salt-minion[25463]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 01 18:34:21 jenkins salt-minion[25869]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:22:58 jenkins salt-minion[31635]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 11:50:21 jenkins salt-minion[5681]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:04:59 jenkins salt-minion[9394]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 12:46:23 jenkins salt-minion[18364]: [WARNING ] state.highstate args: [{'queue': True}]
Sep 04 13:11:42 jenkins salt-minion[25043]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:52:46 jenkins salt-minion[9928]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=yesterday -u salt-minion | grep "args:" | grep highstate']
Sep 04 16:53:11 jenkins salt-minion[10163]: [WARNING ] cmd.run args: ['salt-minion --version && journalctl --since=1w -u salt-minion | grep "args:" | grep highstate']
as discussed in the daily today I will keep checking again in the future but as we don't see direct impact currently we reduce the priority now.
Updated by okurz over 1 year ago
- Related to action #134819: Errors in salt minion and master log on osd added
Updated by livdywan over 1 year ago
as discussed in the daily today I will keep checking again in the future but as we don't see direct impact currently we reduce the priority now.
I'm wondering if this isn't affecting #134282#note-33 afterall as I was going through Alvaro's notes on debugging workers with faulty mm setups. worker.packages seems to specify all of the packages yet libcap-progs ended up missing.
It seems as though currently all workers have it, probably 1) because it was manually installed in the meantime 2) and/or salt states were applied eventually.
Updated by livdywan over 1 year ago
https://gitlab.suse.de/openqa/salt-pillars-openqa/-/jobs/1806628/raw reveals this at the top:
Rendering SLS 'base:openqa.openvswitch' failed: Jinja error: 'NoneType' object is not iterable
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
output = template.render(**decoded_context)
File "/usr/lib/python3.6/site-packages/jinja2/asyncsupport.py", line 76, in render
return original_render(self, *args, **kwargs)
File "/usr/lib/python3.6/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/usr/lib/python3.6/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/lib/python3.6/site-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "<template>", line 44, in top-level template code
TypeError: 'NoneType' object is not iterable
Updated by nicksinger over 1 year ago
livdywan wrote in #note-15:
https://gitlab.suse.de/openqa/salt-pillars-openqa/-/jobs/1806628/raw reveals this at the top:
Rendering SLS 'base:openqa.openvswitch' failed: Jinja error: 'NoneType' object is not iterable Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl output = template.render(**decoded_context) File "/usr/lib/python3.6/site-packages/jinja2/asyncsupport.py", line 76, in render return original_render(self, *args, **kwargs) File "/usr/lib/python3.6/site-packages/jinja2/environment.py", line 1008, in render return self.environment.handle_exception(exc_info, True) File "/usr/lib/python3.6/site-packages/jinja2/environment.py", line 780, in handle_exception reraise(exc_type, exc_value, tb) File "/usr/lib/python3.6/site-packages/jinja2/_compat.py", line 37, in reraise raise value.with_traceback(tb) File "<template>", line 44, in top-level template code TypeError: 'NoneType' object is not iterable
This is not related to the issue at hand here and rather describes a wrong definition in our workersconf.sls. However, I was wondering too if a broken template somewhere could cause this. But as long as we don't see "Passed invalid arguments to state.highstate: expected str, bytes or os.PathLike object, not list" we have to assume it is something else.
Updated by okurz over 1 year ago
- Due date deleted (
2023-09-14) - Status changed from Feedback to Resolved
We should have more information available now if the issue reappears but we did not see it even over the past days so resolving.
Updated by jbaier_cz about 1 year ago
- Related to action #137744: [alert] gitlab pipeline fails in deploy step for salt-pillars-openqa: Data failed to compile added
Actions