action #120921
Updated by livdywan over 1 year ago
### Observation
All workers failed to apply salt states, e.g.:
```
…
worker2.oqa.suse.de:
Data failed to compile:
----------
Rendering SLS 'base:openqa.openvswitch' failed: Jinja error: argument of type 'NoneType' is not iterable
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
output = template.render(**decoded_context)
File "/usr/lib/python3.6/site-packages/jinja2/asyncsupport.py", line 76, in render
return original_render(self, *args, **kwargs)
File "/usr/lib/python3.6/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/usr/lib/python3.6/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/lib/python3.6/site-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "<template>", line 37, in top-level template code
TypeError: argument of type 'NoneType' is not iterable
; line 37
---
[...]
# Those hosts are identified by WORKER_CLASS value from pillar defined in "multihostclass" variable
# Only one class is supported and its name have to be unique - will match also the strings that contain it!
{% set multihostclass = 'tap' %}
{% set multihostworkers = [] %}
{% for host in pillar['workerconf'] %}
{% if 'global' in pillar['workerconf'][host] %} <======================
{% if multihostclass in pillar['workerconf'][host]['global']['WORKER_CLASS'] | default('undefined') %}
{% do multihostworkers.append(host) %}
{% endif %}
{% endif %}
# The class can be defined in both places (global X numbered workers) at the same time
[...]
---
openqaworker14.qa.suse.cz:
Data failed to compile:
----------
Rendering SLS 'base:openqa.openvswitch' failed: Jinja error: argument of type 'NoneType' is not iterable
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
output = template.render(**decoded_context)
File "/usr/lib/python3.6/site-packages/jinja2/asyncsupport.py", line 76, in render
return original_render(self, *args, **kwargs)
File "/usr/lib/python3.6/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/usr/lib/python3.6/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/lib/python3.6/site-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "<template>", line 37, in top-level template code
TypeError: argument of type 'NoneType' is not iterable
; line 37
---
[...]
# Those hosts are identified by WORKER_CLASS value from pillar defined in "multihostclass" variable
# Only one class is supported and its name have to be unique - will match also the strings that contain it!
{% set multihostclass = 'tap' %}
{% set multihostworkers = [] %}
{% for host in pillar['workerconf'] %}
{% if 'global' in pillar['workerconf'][host] %} <======================
{% if multihostclass in pillar['workerconf'][host]['global']['WORKER_CLASS'] | default('undefined') %}
{% do multihostworkers.append(host) %}
{% endif %}
{% endif %}
# The class can be defined in both places (global X numbered workers) at the same time
[...]
---
…
```
(see https://gitlab.suse.de/openqa/salt-pillars-openqa/-/jobs/1255094/raw for full logs)
## Additional context
The last change in that template was done for #120163. Then we ran into problems that were likely caused by the salt mine not being populated. However, this is now a different error and if the line the logs point to is correct than it now even fails before the modifications from #120163. So I've been creating this new ticket (instead of reopening #120163).
## Acceptance criteria
- **AC1**: No inconsistencies with jinja templates are reported
## Suggestions
* This seems to affect all workers now
* Apply a salt high state for the worker role from OSD to reproduce
* Research how to check jinja syntax easily
* Consider what changes might contribute to this e.g. new Prague workers https://gitlab.suse.de/openqa/salt-pillars-openqa/-/commit/792e682f5e88aee174264b1df7e58b44c2c3d118
* Improve error reporting