Pipeline in salt-states-openqa can fail occasionally size:M
test-monitor in salt-states-openqa failed today. It isn't clear what the problem is, the log is quite extensive: https://gitlab.suse.de/mkittler/salt-states-openqa/-/jobs/540688
This job ran as part of https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/562 but re-triggering the job helped. So it is likely not related to the SR but another sporadically happening issue.
Seems like something in
/etc/salt/minion, wrong YAML syntax or likely salt states with the same ID from different files:
.[ERROR ] Error parsing configuration file: /etc/salt/minion - while constructing a mapping 9868 in "/etc/salt/minion", line 936, column 1 9869found conflicting ID 'disable_grains' 9870 in "/etc/salt/minion", line 942, column 1 9871[ERROR ] Error parsing configuration file: /etc/salt/minion - while constructing a mapping [...] 9890 raise ConstructorError( 9891yaml.constructor.ConstructorError: while constructing a mapping 9892 in "/etc/salt/minion", line 936, column 1 9893found conflicting ID 'disable_grains' 9894 in "/etc/salt/minion", line 942, column 1
Maybe a consequence of the other error?
Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/salt/loader.py", line 1778, in _load_module module_init(self.opts) File "/usr/lib/python3.8/site-packages/salt/modules/boto3_elasticsearch.py", line 87, in __init__ __utils__["boto3.assign_funcs"](__name__, "es") File "/usr/lib/python3.8/site-packages/salt/loader.py", line 1283, in __getitem__ func = super().__getitem__(item) File "/usr/lib/python3.8/site-packages/salt/utils/lazy.py", line 108, in __getitem__ raise KeyError(key) KeyError: 'boto3.assign_funcs'
- Stare at https://gitlab.suse.de/mkittler/salt-states-openqa/-/jobs/540688 for at least 10 minutes trying to find the actual error cause before giving up
- Investigate what could cause the YAML/id issues
- Confirm if this is really flaky (re-trigger a couple more times)
#5 Updated by mkittler about 1 month ago
- Status changed from Workable to Feedback
- Assignee set to mkittler
I suppose the error message means that
disable_grains occurs multiple times in
/etc/salt/minion (so these occurrences are conflicting with each other). We're appending this setting in
salt-states-openqa/salt/minion.sls using https://docs.saltproject.io/en/latest/ref/states/all/salt.states.file.html#salt.states.file.append. So it is possible that the section has been appended multiple times because the file has been edited manually (and therefore had a different "end"). I have never seen this kind of failure again and a re-try helped. I conclude this was a one-time problem.
We could still try to improve, e.g. it would make sense to process
/etc/salt/minion as a YAML document instead of just trying to append some text. However, I haven't found a salt state for managing YAML (like https://docs.saltproject.io/en/latest/ref/states/all/salt.states.ini_manage.html for INI files). So I suppose it isn't worth it.
You could try to use https://docs.saltproject.io/en/latest/ref/states/all/salt.states.file.html#salt.states.file.serialize with "merge_if_exists: True". If you consider this not worth the effort then leave it as is
I gave it a shot and it works but has also downsides, see https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/611.
- Status changed from Feedback to Resolved
The SR has been merged. Note that the comments in those files are currently still present and it looks like the SR didn't worked. However, I've observed the same when testing in a container before and it is normal that the file is not touched at all if there are no changes to be made anyways. So the comments will only be gone after the next actual change.