action #162311
closedcoordination #154768: [saga][epic][ux] State-of-art user experience for openQA
coordination #157510: [epic] Up-to-date JavaScript stack
[sporadic] openqa-in-openqa-test fails in openqa_webui due to CSS files not successfully generated size:M
Added by tinita 6 months ago. Updated 6 months ago.
Description
Observation¶
openQA test in scenario openqa-Tumbleweed-dev-x86_64-openqa_from_git@64bit-2G fails in
openqa_webui
Acceptance criteria¶
- AC1: The CSS files of the openQA web UI are generated reliably (without sometimes running into SASS-related errors).
Reproducible¶
Fails since (at least) Build :TW.29212
Expected result¶
Last good: :TW.29211 (or more recent)
Suggestions¶
- DONE Understand why openQA-in-openQA tests show broken CSS sometimes -> Because sass failed to build assets and AssetPack is ignoring errors so our packages in devel:openQA have sometimes broken CSS since the bootstrap 4->5 migration
- Provide a fix for how sass builds CSS to the according package https://build.opensuse.org/request/show/1181472
- Submit the according patch temporarily also into devel:openQA with a link back to this ticket
- Await the acceptance of the according SR into Factory, verify that it's effective for openQA, remove the link in devel:openQA again
Further details¶
Always latest result in this scenario: latest
Out of scope¶
Updated by tinita 6 months ago
nohop.out contains this:
https://openqa.opensuse.org/tests/4273694/logfile?filename=openqa_webui-openqa_nohup_out.txt
TypeError: String can't be coerced into Integer
Use --trace for backtrace.
[info] Listening at "http://127.0.0.1:9526"
Web application available at http://127.0.0.1:9526
[info] Listening at "http://[::1]:9526"
and grep -qP "Listening at.*(127.0.0.1|localhost)" <(tail -F -n0 nohup.out)
fails.
I have been looking at this code before. Why does it use a -n0
here?
Updated by nicksinger 6 months ago
tinita wrote in #note-2:
nohop.out contains this:
https://openqa.opensuse.org/tests/4273694/logfile?filename=openqa_webui-openqa_nohup_out.txtTypeError: String can't be coerced into Integer Use --trace for backtrace. [info] Listening at "http://127.0.0.1:9526" Web application available at http://127.0.0.1:9526 [info] Listening at "http://[::1]:9526"
and
grep -qP "Listening at.*(127.0.0.1|localhost)" <(tail -F -n0 nohup.out)
fails.
I have been looking at this code before. Why does it use a-n0
here?
I think the -n0
mainly prevents tail to print something if nohup.out
is already present.
Updated by tinita 6 months ago
I just ran a test with an addeed die
after the grep, so that logs get uploaded.
In this case the log contains:
[info] Listening at "http://127.0.0.1:9526"
Web application available at http://127.0.0.1:9526
[info] Listening at "http://[::1]:9526"
So the different content seems to cause the grep to fail. Although I don't know why.
TypeError: String can't be coerced into Integer
Use --trace for backtrace.
That message comes from ruby apparently.
Updated by tinita 6 months ago
The openqa_webui-openqa_services.log looks the same in both cases:
Jun 14 10:59:21 susetest systemd[1]: Starting The Apache Webserver...
Jun 14 10:59:21 susetest start_apache2[25488]: AH00112: Warning: DocumentRoot [/usr/share/openqa/public] does not exist
Jun 14 10:59:21 susetest start_apache2[25488]: AH00557: httpd-prefork: apr_sockaddr_info_get() failed for susetest
Jun 14 10:59:21 susetest start_apache2[25488]: AH00558: httpd-prefork: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1. Set the 'ServerName' directive globally to suppress this message
Jun 14 10:59:21 susetest systemd[1]: Started The Apache Webserver.
Updated by tinita 6 months ago
nicksinger wrote in #note-3:
I think the
-n0
mainly prevents tail to print something ifnohup.out
is already present.
Yeah, but that wouldn't actually hurt.
I think in this case the log is already there with the expected content when the grep starts, so it will never get the expected input.
It's just weird that it seems to be too slow only in the case we get the ruby error message.
Will debug further.
Updated by tinita 6 months ago
- Related to action #138362: [opensuse] test fails in openqa_webui: Apache server not listening at 127.0.0.1 added
Updated by tinita 6 months ago
- Related to action #64523: [openqa-in-openqa][sporadic] test fails in sometimes openqa_webui waiting for output from nohup.out, potential race condition? added
Updated by tinita 6 months ago
- Related to action #137426: test fails in openqa_webui auto_review:"Test died:.*Listening at.*timed out" added
Updated by tinita 6 months ago
https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/179 Improve waiting for webserver
This can at least help with debugging.
Updated by tinita 6 months ago
It seems to be consistent (but still monitoring):
When the TypeError happens
https://openqa.opensuse.org/tests/4273783#step/openqa_webui/31
then also the CSS seems to be missing:
https://openqa.opensuse.org/tests/4273783#step/dashboard/5
Updated by openqa_review 6 months ago
- Due date set to 2024-06-29
Setting due date based on mean cycle time of SUSE QE Tools
Updated by tinita 6 months ago
https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/180 Run sass with --trace
https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/181 Upload log with .txt suffix
Updated by tinita 6 months ago
https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/180 merged.
I can make a PR to https://metacpan.org/dist/Mojolicious-Plugin-AssetPack as I think this is useful in general.
Updated by tinita 6 months ago
I created https://github.com/mojolicious/mojo-assetpack/pull/150 Call sass with --trace
Updated by mkittler 6 months ago
- Related to action #162242: openqa-in-openqa test fails in dashboard added
Updated by mkittler 6 months ago
For the record: I have seen this before on my local machine during development. One time I didn't do any changes except for adding the --trace
parameter (like in https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/180) but I haven't seen it since I added the parameter so I cannot share a back-trace. I'll keep the --trace
parameter in my local setup and will write a comment if I see it again.
Updated by tinita 6 months ago
- Status changed from In Progress to Blocked
Yeah, It looks like the same as #162242
https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/182 Add comments about temporary sass patch
I created https://build.opensuse.org/request/show/1181342
Waiting for that until I can link it in devel:openQA
If the issue happens again we can look at the stacktrace.
Updated by tinita 6 months ago · Edited
Ok, we got a failure:
https://openqa.opensuse.org/tests/4276734/logfile?filename=dashboard-openqa_nohup_out.txt
/usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/value/number.rb:409:in `-': nil can't be coerced into Integer (TypeError)
(num1 - num2).abs < epsilon
^^^^
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/value/number.rb:409:in `basically_equal?'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/value/number.rb:403:in `basically_equal?'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/value/number.rb:226:in `eql?'
from /usr/lib64/ruby/3.3.0/set.rb:394:in `include?'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/tree/map_literal.rb:54:in `block in _perform'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/tree/map_literal.rb:52:in `map'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/tree/map_literal.rb:52:in `_perform'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/tree/node.rb:50:in `perform'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/tree/map_literal.rb:53:in `block in _perform'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/tree/map_literal.rb:52:in `map'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/tree/map_literal.rb:52:in `_perform'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/tree/node.rb:50:in `perform'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/tree/map_literal.rb:53:in `block in _perform'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/tree/map_literal.rb:52:in `map'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/tree/map_literal.rb:52:in `_perform'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/tree/node.rb:50:in `perform'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/tree/funcall.rb:182:in `perform_arg'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/tree/funcall.rb:129:in `block in _perform'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/tree/funcall.rb:129:in `each'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/tree/funcall.rb:129:in `each_with_index'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/tree/funcall.rb:129:in `each'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/tree/funcall.rb:129:in `map'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/tree/funcall.rb:129:in `_perform'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/tree/node.rb:50:in `perform'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/perform.rb:478:in `visit_variable'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/base.rb:36:in `visit'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/perform.rb:158:in `block in visit'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/stack.rb:79:in `block in with_base'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/stack.rb:135:in `with_frame'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/stack.rb:79:in `with_base'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/perform.rb:158:in `visit'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/perform.rb:325:in `block (2 levels) in visit_import'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/perform.rb:325:in `map'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/perform.rb:325:in `block in visit_import'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/stack.rb:88:in `block in with_import'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/stack.rb:135:in `with_frame'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/stack.rb:88:in `with_import'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/perform.rb:322:in `visit_import'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/base.rb:36:in `visit'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/perform.rb:158:in `block in visit'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/stack.rb:79:in `block in with_base'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/stack.rb:135:in `with_frame'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/stack.rb:79:in `with_base'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/perform.rb:158:in `visit'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/perform.rb:325:in `block (2 levels) in visit_import'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/perform.rb:325:in `map'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/perform.rb:325:in `block in visit_import'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/stack.rb:88:in `block in with_import'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/stack.rb:135:in `with_frame'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/stack.rb:88:in `with_import'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/perform.rb:322:in `visit_import'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/base.rb:36:in `visit'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/perform.rb:158:in `block in visit'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/stack.rb:79:in `block in with_base'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/stack.rb:135:in `with_frame'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/stack.rb:79:in `with_base'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/perform.rb:158:in `visit'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/base.rb:52:in `block in visit_children'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/base.rb:52:in `map'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/base.rb:52:in `visit_children'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/perform.rb:167:in `block in visit_children'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/perform.rb:179:in `with_environment'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/perform.rb:166:in `visit_children'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/base.rb:36:in `block in visit'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/perform.rb:186:in `visit_root'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/base.rb:36:in `visit'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/perform.rb:157:in `visit'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/visitors/perform.rb:10:in `visit'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/root_node.rb:36:in `css_tree'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/tree/root_node.rb:20:in `render'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/engine.rb:290:in `render'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/exec/sass_scss.rb:396:in `run'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/exec/sass_scss.rb:63:in `process_result'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/exec/base.rb:50:in `parse'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/exec/base.rb:18:in `parse!'
from /usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/bin/sass:13:in `<top (required)>'
from /usr/bin/sass:25:in `load'
from /usr/bin/sass:25:in `<main>'
[info] Listening at "http://127.0.0.1:9526"
Web application available at http://127.0.0.1:9526
[info] Listening at "http://[::1]:9526"
https://github.com/sass/dart-sass/issues/2045 looks like the same issue, but I don't understand the explanation yet
Updated by tinita 6 months ago · Edited
I am able to reproduce it sometimes with:
% cat in
@import "../node_modules/bootstrap/scss/bootstrap.scss";
% sass -s --trace -I /tmp/install/openQA/script/../assets/stylesheets -I /tmp/install/openQA/script/../assets --scss <in >out
/usr/lib64/ruby/gems/3.3.0/gems/sass-3.7.4/lib/sass/script/value/number.rb:409:in `-': nil can't be coerced into Integer (TypeError)
...
Updated by tinita 6 months ago
Two problematic snippets seem to be:
// node_modules/bootstrap/scss/_variables.scss
// scss-docs-start zindex-levels-map
$zindex-levels: (
n1: -1, // <------------------
0: 0,
1: 1,
2: 2,
3: 3
) !default;
// node_modules/bootstrap/scss/_utilities.scss
"order": (
responsive: true,
property: order,
values: (
first: -1, // <------------
0: 0,
1: 1,
2: 2,
3: 3,
4: 4,
5: 5,
last: 6,
),
not sure why it only happens sometimes.
Updated by tinita 6 months ago
I guess there are several options now. I could try a patch that would allow to also compare strings.
But this gem is deprecated, and apparently there are alternatives, but it's not clear which would be recommended.
There's sassc which uses libsass. Of course it has completely different commandline options.
Then there is dartsass, and 3 rubygems which are wrappers around dartsass. But none of them in openSUSE.
Also 4 different npm packages that do sass.
Sooner or later we will have to do something about it.
I'll try the patch first.
Updated by tinita 6 months ago
I created a patch: https://build.opensuse.org/request/show/1181472
Will now shortly look into sassc, whether it might be an option.
Updated by tinita 6 months ago
https://github.com/mojolicious/mojo-assetpack/pull/151 Optionally use sassc
Updated by mkittler 6 months ago
Nice that you could get to the bottom of this. I guess it makes sense to switch to sassc then. For the time being we can rely on https://build.opensuse.org/request/show/1181472.
Updated by mkittler 6 months ago
- Related to deleted (action #162242: openqa-in-openqa test fails in dashboard)
Updated by mkittler 6 months ago
- Blocks action #162242: openqa-in-openqa test fails in dashboard added
Updated by okurz 6 months ago
- Copied to action #162500: Switch to more reliable CSS generation, e.g. using sassc added
Updated by okurz 6 months ago
- Copied to action #162503: AssetPack should fail if sass fails size:M added
Updated by mkittler 6 months ago
Submit the according patch temporarily also into devel:openQA with a link back to this ticket
I created https://build.opensuse.org/package/show/devel:openQA/rubygem-sass via osc copypac home:tinita:branches:devel:languages:ruby:extensions rubygem-sass devel:openQA
.
Updated by tinita 6 months ago
- Status changed from In Progress to Blocked
Blocking on https://build.opensuse.org/request/show/1181472
Updated by tinita 6 months ago
https://build.opensuse.org/request/show/1181472 was accepted, waiting for Factory: https://build.opensuse.org/request/show/1182090
Updated by tinita 6 months ago
https://build.opensuse.org/request/show/1182090 was accepted.
Now I guess I can link it in the subprojects of devel:openQA?
Updated by okurz 6 months ago
- Due date deleted (
2024-06-29) - Status changed from Blocked to Workable
As https://build.opensuse.org/request/show/1182090 was accepted days ago please continue with the next step
Updated by tinita 6 months ago
- Status changed from In Progress to Feedback
I saw that Marius already did linkpac for Leap 15.4/15.5/15.6.
I just removed https://build.opensuse.org/package/show/devel:openQA/rubygem-sass now.