action #129946
closed[tools][ci][ui] Randomly failing t/ui/15-comments.t
Description
from log:
[14:31:56] t/ui/15-comments.t ......................... 11/?
# Failed test 'comment text entered'
# at /home/squamata/project/t/ui/../lib/OpenQA/SeleniumTest.pm line 322.
# Looks like you failed 1 test of 8.
# Failed test 'group overview: /group_overview/1001'
# at t/ui/15-comments.t line 484.
# Looks like you failed 1 test of 4.
[14:31:56] t/ui/15-comments.t ......................... 12/?
# Failed test 'editing when logged in as regular user'
# at t/ui/15-comments.t line 485.
executeScript: unexpected alert open: {Alert text : The comment text mustn't be empty.} at /home/squamata/project/t/ui/../lib/OpenQA/SeleniumTest.pm:81 at /home/squamata/project/t/ui/../lib/OpenQA/SeleniumTest.pm line 84.
OpenQA::SeleniumTest::__ANON__(Test::Selenium::Chrome=HASH(0x55fdfec89a48), "Error while executing command: executeScript: unexpected aler"..., HASH(0x55fdff2179b8), HASH(0x55fdff21ec80)) called at /usr/lib/perl5/vendor_perl/5.26.1/Selenium/Remote/Driver.pm line 356
Selenium::Remote::Driver::catch {...} ("Error while executing command: executeScript: unexpected aler"...) called at /usr/lib/perl5/vendor_perl/5.26.1/Try/Tiny.pm line 123
Try::Tiny::try(CODE(0x55fdff21b690), Try::Tiny::Catch=REF(0x55fdff228708)) called at /usr/lib/perl5/vendor_perl/5.26.1/Selenium/Remote/Driver.pm line 361
Selenium::Remote::Driver::__ANON__(CODE(0x55fdfe908b10), Test::Selenium::Chrome=HASH(0x55fdfec89a48), HASH(0x55fdff2179b8), HASH(0x55fdff21ec80)) called at (eval 1718)[/usr/lib/perl5/vendor_perl/5.26.1/Class/Method/Modifiers.pm:89] line 1
Selenium::Remote::Driver::__ANON__(Test::Selenium::Chrome=HASH(0x55fdfec89a48), HASH(0x55fdff2179b8), HASH(0x55fdff21ec80)) called at (eval 1720)[/usr/lib/perl5/vendor_perl/5.26.1/Class/Method/Modifiers.pm:148] line 2
Selenium::Remote::Driver::_execute_command(Test::Selenium::Chrome=HASH(0x55fdfec89a48), HASH(0x55fdff2179b8), HASH(0x55fdff21ec80)) called at /usr/lib/perl5/vendor_perl/5.26.1/Selenium/Remote/Driver.pm line 1053
Selenium::Remote::Driver::execute_script(Test::Selenium::Chrome=HASH(0x55fdfec89a48), "return window.jQuery \x{26}\x{26} jQuery.active === 0") called at /home/squamata/project/t/ui/../lib/OpenQA/SeleniumTest.pm line 163
OpenQA::SeleniumTest::wait_for_ajax("msg", "comment with pinning for group added by regular user") called at t/ui/15-comments.t line 74
main::write_comment("pinned-description ... The description", "comment with pinning for group added by regular user") called at t/ui/15-comments.t line 471
main::__ANON__() called at /usr/lib/perl5/5.26.1/Test/Builder.pm line 309
eval {...} called at /usr/lib/perl5/5.26.1/Test/Builder.pm line 309
Test::Builder::subtest(Test::Builder=HASH(0x55fdf2476540), "group overview: /group_overview/1001", CODE(0x55fdff066e50)) called at /usr/lib/perl5/5.26.1/Test/More.pm line 807
Test::More::subtest("group overview: /group_overview/1001", CODE(0x55fdff066e50)) called at t/ui/15-comments.t line 484
main::__ANON__() called at /usr/lib/perl5/5.26.1/Test/Builder.pm line 309
eval {...} called at /usr/lib/perl5/5.26.1/Test/Builder.pm line 309
Test::Builder::subtest(Test::Builder=HASH(0x55fdf2476540), "editing when logged in as regular user", CODE(0x55fdff01d0e0)) called at /usr/lib/perl5/5.26.1/Test/More.pm line 807
Test::More::subtest("editing when logged in as regular user", CODE(0x55fdff01d0e0)) called at t/ui/15-comments.t line 485
[14:31:56] t/ui/15-comments.t ......................... 13/? # Tests were run but no plan was declared and done_testing() was not seen.
[14:31:56] t/ui/15-comments.t ......................... Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/13 subtests
Updated by okurz 11 months ago
- Related to action #111542: [sporadic] openQA test t/ui/15-comments.t fails in 'heading text' size:M added
Updated by mkittler 11 months ago
- Status changed from New to In Progress
So is now fails in the last line of the function
sub write_comment ($text, $desc) {
wait_for_element(selector => '#text', is_displayed => 1, description => 'comment form is displayed')
->send_keys($text);
# wait until the text is there
# notes:
# - Considering poo#128153 just waiting for the return of `send_keys` is sometimes not good enough.
# - Not using `get_text` as it apparently doesn't work for the `textarea` element.
wait_until sub { $driver->execute_script('return document.getElementById("text").value') }, 'comment text entered';
$driver->find_element_by_id('submitComment')->click;
wait_for_ajax msg => $desc;
}
which is invoked in this context (the last line of that snippet is the failing invocation):
subtest 'group overview: ' . $_ => sub {
my $group_url = $_;
$driver->get($group_url);
no_edit_no_remove_on_other_comments_expected;
write_comment 'test by nobody', 'comment for group added by regular user';
only_edit_for_own_comments_expected;
# pinned comments are not shown (pinning is only possible when commentator is operator)
write_comment $description_test_message, 'comment with pinning for group added by regular user';
The failing line within wait_for_ajax is the last line of this snippet:
sub wait_for_ajax {
my (%args) = @_;
my $check_interval = _default_check_interval($args{interval});
my $timeout = 60 * 5;
my $slept = 0;
my $msg = $args{msg} ? (': ' . $args{msg}) : '';
while (!$_driver->execute_script('return window.jQuery && jQuery.active === 0')) {
That means executing any JavaScript code this way would fail as well. Considering wait_until sub { $driver->execute_script('return document.getElementById("text").value') }, 'comment text entered'; did not fail we can be sure that the alert was not already visible when submitting the form and that the text element's value was not empty. Still, when trying to submit the comment we run into this error.
I've checked via Array.from(document.getElementsByTagName('*')).map(f => f.id); whether there are any elements with duplicate IDs but couldn't find any after doing all sorts of actions on the page.
The code for adding/updating comments looks correct as well. I've created a PR to simplify it a little bit: https://github.com/os-autoinst/openQA/pull/5168
With this PR the relevant code no longer relies on jQuery. So if it is merged we can exclude a bug in jQuery to be the culprit (which isn't very likely but better than nothing).
Updated by openqa_review 11 months ago
- Due date set to 2023-06-14
Setting due date based on mean cycle time of SUSE QE Tools
Updated by mkittler 11 months ago
It has failed again on https://github.com/os-autoinst/openQA/pull/5173:
# at t/ui/15-comments.t line 485.
executeScript: unexpected alert open: {Alert text : The comment text mustn't be empty.} at /home/squamata/project/t/ui/../lib/OpenQA/SeleniumTest.pm:81 at /home/squamata/project/t/ui/../lib/OpenQA/SeleniumTest.pm line 84.
OpenQA::SeleniumTest::__ANON__(Test::Selenium::Chrome=HASH(0x558a9dda61c0), "Error while executing command: executeScript: unexpected aler"..., HASH(0x558a9e43ad80), HASH(0x558a9e438bd0)) called at /usr/lib/perl5/vendor_perl/5.26.1/Selenium/Remote/Driver.pm line 356
Selenium::Remote::Driver::catch {...} ("Error while executing command: executeScript: unexpected aler"...) called at /usr/lib/perl5/vendor_perl/5.26.1/Try/Tiny.pm line 123
Try::Tiny::try(CODE(0x558a9e4356d8), Try::Tiny::Catch=REF(0x558a9e445070)) called at /usr/lib/perl5/vendor_perl/5.26.1/Selenium/Remote/Driver.pm line 361
Selenium::Remote::Driver::__ANON__(CODE(0x558a9d8e5ee8), Test::Selenium::Chrome=HASH(0x558a9dda61c0), HASH(0x558a9e43ad80), HASH(0x558a9e438bd0)) called at (eval 1713)[/usr/lib/perl5/vendor_perl/5.26.1/Class/Method/Modifiers.pm:89] line 1
Selenium::Remote::Driver::__ANON__(Test::Selenium::Chrome=HASH(0x558a9dda61c0), HASH(0x558a9e43ad80), HASH(0x558a9e438bd0)) called at (eval 1715)[/usr/lib/perl5/vendor_perl/5.26.1/Class/Method/Modifiers.pm:148] line 2
Selenium::Remote::Driver::_execute_command(Test::Selenium::Chrome=HASH(0x558a9dda61c0), HASH(0x558a9e43ad80), HASH(0x558a9e438bd0)) called at /usr/lib/perl5/vendor_perl/5.26.1/Selenium/Remote/Driver.pm line 1053
Selenium::Remote::Driver::execute_script(Test::Selenium::Chrome=HASH(0x558a9dda61c0), "return window.jQuery \x{26}\x{26} jQuery.active === 0") called at /home/squamata/project/t/ui/../lib/OpenQA/SeleniumTest.pm line 163
OpenQA::SeleniumTest::wait_for_ajax("msg", "comment with pinning for group added by regular user") called at t/ui/15-comments.t line 74
main::write_comment("pinned-description ... The description", "comment with pinning for group added by regular user") called at t/ui/15-comments.t line 471
main::__ANON__() called at /usr/lib/perl5/5.26.1/Test/Builder.pm line 309
eval {...} called at /usr/lib/perl5/5.26.1/Test/Builder.pm line 309
Test::Builder::subtest(Test::Builder=HASH(0x558a91671d28), "group overview: /parent_group_overview/1", CODE(0x558a9e279d90)) called at /usr/lib/perl5/5.26.1/Test/More.pm line 807
Test::More::subtest("group overview: /parent_group_overview/1", CODE(0x558a9e279d90)) called at t/ui/15-comments.t line 484
main::__ANON__() called at /usr/lib/perl5/5.26.1/Test/Builder.pm line 309
eval {...} called at /usr/lib/perl5/5.26.1/Test/Builder.pm line 309
Test::Builder::subtest(Test::Builder=HASH(0x558a91671d28), "editing when logged in as regular user", CODE(0x558a9e23c6f0)) called at /usr/lib/perl5/5.26.1/Test/More.pm line 807
Test::More::subtest("editing when logged in as regular user", CODE(0x558a9e23c6f0)) called at t/ui/15-comments.t line 485
[10:25:40] t/ui/15-comments.t ......................... 13/? # Tests were run but no plan was declared and done_testing() was not seen.
This PR has already been rebased on https://github.com/os-autoinst/openQA/pull/5168 so it is safe to say that the previous use of jQuery was not the culprit.
Note that this trace is slightly different as it happened on a parent group page (and not a normal group page). So the concrete page definitely doesn't make a difference.
Updated by mkittler 11 months ago
If I remove send_keys in write_comment then wait_until indeed waits as expected (and eventually times out). So the waiting condition is definitely correct.
If I add another textarea with the same ID on the page before (e.g. <textarea id="text"></textarea>) I can provoke the problem we see on CircleCI locally. Another textarea with the same name in the same form provokes it as well (e.g. <textarea name="text"></textarea>). Maybe under some conditions such an element is rendered accidentally?
PR to answer that question (and possibly fix the issue): https://github.com/os-autoinst/openQA/pull/5176
Updated by livdywan 11 months ago
mkittler wrote:
If I add another textarea with the same ID on the page before (e.g.
<textarea id="text"></textarea>) I can provoke the problem we see on CircleCI locally. Another textarea with the same name in the same form provokes it as well (e.g.<textarea name="text"></textarea>). Maybe under some conditions such an element is rendered accidentally?
I took a brief look at comment_row.html.ep thinking maybe the condition deciding that a row contains the form is flawed. Apparently I don't understand how that works. It looks to me like we should have a form for each comment plus the extra row. I don't see a condition preventing that. Maybe that helps?
Updated by mkittler 11 months ago
The PR has been merged. Let's see how it affects the stability of this test.
I took a brief look at comment_row.html.ep thinking maybe the condition deciding that a row contains the form is flawed. Apparently I don't understand how that works. It looks to me like we should have a form for each comment plus the extra row. I don't see a condition preventing that. Maybe that helps?
I don't think this code is the problem. We always render the editing buttons, the heading, the comment text as rendered markdown and a text area for updating with raw markdown. The text area is hidden by default and only shown when clicking the "Edit" button. So that's ok and this text area only has the name text (and not the id).
Updated by mkittler 11 months ago
Observed again in https://github.com/os-autoinst/openQA/pull/5185 which has already been based on https://github.com/os-autoinst/openQA/pull/5176:
[15:05:50] t/ui/15-comments.t ......................... 12/?
# Failed test 'editing when logged in as regular user'
# at t/ui/15-comments.t line 488.
executeScript: unexpected alert open: {Alert text : The comment text mustn't be empty.} at /home/squamata/project/t/ui/../lib/OpenQA/SeleniumTest.pm:81 at /home/squamata/project/t/ui/../lib/OpenQA/SeleniumTest.pm line 84.
OpenQA::SeleniumTest::__ANON__(Test::Selenium::Chrome=HASH(0x559c985eaf78), "Error while executing command: executeScript: unexpected aler"..., HASH(0x559c98c800a0), HASH(0x559c98c80310)) called at /usr/lib/perl5/vendor_perl/5.26.1/Selenium/Remote/Driver.pm line 356
Selenium::Remote::Driver::catch {...} ("Error while executing command: executeScript: unexpected aler"...) called at /usr/lib/perl5/vendor_perl/5.26.1/Try/Tiny.pm line 123
Try::Tiny::try(CODE(0x559c98c7d398), Try::Tiny::Catch=REF(0x559c98c8cdf0)) called at /usr/lib/perl5/vendor_perl/5.26.1/Selenium/Remote/Driver.pm line 361
Selenium::Remote::Driver::__ANON__(CODE(0x559c983686c8), Test::Selenium::Chrome=HASH(0x559c985eaf78), HASH(0x559c98c800a0), HASH(0x559c98c80310)) called at (eval 1718)[/usr/lib/perl5/vendor_perl/5.26.1/Class/Method/Modifiers.pm:89] line 1
Selenium::Remote::Driver::__ANON__(Test::Selenium::Chrome=HASH(0x559c985eaf78), HASH(0x559c98c800a0), HASH(0x559c98c80310)) called at (eval 1720)[/usr/lib/perl5/vendor_perl/5.26.1/Class/Method/Modifiers.pm:148] line 2
Selenium::Remote::Driver::_execute_command(Test::Selenium::Chrome=HASH(0x559c985eaf78), HASH(0x559c98c800a0), HASH(0x559c98c80310)) called at /usr/lib/perl5/vendor_perl/5.26.1/Selenium/Remote/Driver.pm line 1053
Selenium::Remote::Driver::execute_script(Test::Selenium::Chrome=HASH(0x559c985eaf78), "return window.jQuery \x{26}\x{26} jQuery.active === 0") called at /home/squamata/project/t/ui/../lib/OpenQA/SeleniumTest.pm line 163
OpenQA::SeleniumTest::wait_for_ajax("msg", "comment with pinning for group added by regular user") called at t/ui/15-comments.t line 77
main::write_comment("pinned-description ... The description", "comment with pinning for group added by regular user") called at t/ui/15-comments.t line 474
main::__ANON__() called at /usr/lib/perl5/5.26.1/Test/Builder.pm line 309
eval {...} called at /usr/lib/perl5/5.26.1/Test/Builder.pm line 309
Test::Builder::subtest(Test::Builder=HASH(0x559c8bed1c90), "group overview: /parent_group_overview/1", CODE(0x559c98a87c30)) called at /usr/lib/perl5/5.26.1/Test/More.pm line 807
Test::More::subtest("group overview: /parent_group_overview/1", CODE(0x559c98a87c30)) called at t/ui/15-comments.t line 487
main::__ANON__() called at /usr/lib/perl5/5.26.1/Test/Builder.pm line 309
eval {...} called at /usr/lib/perl5/5.26.1/Test/Builder.pm line 309
Test::Builder::subtest(Test::Builder=HASH(0x559c8bed1c90), "editing when logged in as regular user", CODE(0x559c98a87588)) called at /usr/lib/perl5/5.26.1/Test/More.pm line 807
Test::More::subtest("editing when logged in as regular user", CODE(0x559c98a87588)) called at t/ui/15-comments.t line 488
[15:05:50] t/ui/15-comments.t ......................... 13/? # Tests were run but no plan was declared and done_testing() was not seen.
[15:05:50] t/ui/15-comments.t ......................... Dubious, test returned 1 (wstat 256, 0x100)
So this is definitely not about having two elements with the same ID on the page.
Updated by mkittler 11 months ago
Since I'm out of ideas I'm just adding a retry now: https://github.com/os-autoinst/openQA/pull/5188