action #164239
closedfull-stack.t failing with "executeScript: unknown error: session deleted because of page crash at t/lib/OpenQA/SeleniumTest.pm:78"
0%
Description
Observation¶
Observed in openQA CI tests both in PRs and master CI tests, e.g.
https://github.com/os-autoinst/openQA/pull/5779#issuecomment-2239102371
and in os-autoinst tests https://github.com/os-autoinst/os-autoinst/pull/2515#issuecomment-2251044169
[13:01:07] t/full-stack.t ......... Prototype mismatch: sub main::any: none vs (&@) at /usr/lib/perl5/5.26.1/Exporter.pm line 66.
at t/full-stack.t line 25.
[13:01:07] t/full-stack.t ......... 36/? executeScript: unknown error: session deleted because of page crash at /home/squamata/project/t/lib/OpenQA/SeleniumTest.pm:78 at /home/squamata/project/t/lib/OpenQA/SeleniumTest.pm line 81.
OpenQA::SeleniumTest::__ANON__(Test::Selenium::Chrome=HASH(0x5585d2665020), "Error while executing command: executeScript: unknown error: "..., HASH(0x5585d2f907c0), HASH(0x5585d3041450)) called at /usr/lib/perl5/vendor_perl/5.26.1/Selenium/Remote/Driver.pm line 356
Selenium::Remote::Driver::catch {...} ("Error while executing command: executeScript: unknown error: "...) called at /usr/lib/perl5/vendor_perl/5.26.1/Try/Tiny.pm line 123
Try::Tiny::try(CODE(0x5585d3164910), Try::Tiny::Catch=REF(0x5585d3164970)) called at /usr/lib/perl5/vendor_perl/5.26.1/Selenium/Remote/Driver.pm line 361
Selenium::Remote::Driver::__ANON__(CODE(0x5585d0ffd220), Test::Selenium::Chrome=HASH(0x5585d2665020), HASH(0x5585d2f907c0), HASH(0x5585d3041450)) called at (eval 1557)[/usr/lib/perl5/vendor_perl/5.26.1/Class/Method/Modifiers.pm:89] line 1
Selenium::Remote::Driver::__ANON__(Test::Selenium::Chrome=HASH(0x5585d2665020), HASH(0x5585d2f907c0), HASH(0x5585d3041450)) called at (eval 1559)[/usr/lib/perl5/vendor_perl/5.26.1/Class/Method/Modifiers.pm:148] line 2
Selenium::Remote::Driver::_execute_command(Test::Selenium::Chrome=HASH(0x5585d2665020), HASH(0x5585d2f907c0), HASH(0x5585d3041450)) called at /usr/lib/perl5/vendor_perl/5.26.1/Selenium/Remote/Driver.pm line 1053
Selenium::Remote::Driver::execute_script(Test::Selenium::Chrome=HASH(0x5585d2665020), "return window.jQuery \x{26}\x{26} jQuery.active === 0") called at /home/squamata/project/t/lib/OpenQA/SeleniumTest.pm line 147
OpenQA::SeleniumTest::wait_for_ajax("msg", "wait_for_result_panel: waiting for '(?^u:State: running)' (co"...) called at /home/squamata/project/t/lib/OpenQA/Test/FullstackUtils.pm line 87
OpenQA::Test::FullstackUtils::wait_for_result_panel(Test::Selenium::Chrome=HASH(0x5585d2665020), qr(State: running)u, 1) called at /home/squamata/project/t/lib/OpenQA/Test/FullstackUtils.pm line 108
OpenQA::Test::FullstackUtils::wait_for_job_running(Test::Selenium::Chrome=HASH(0x5585d2665020), 1) called at t/full-stack.t line 338
main::__ANON__() called at /usr/lib/perl5/5.26.1/Test/Builder.pm line 309
eval {...} called at /usr/lib/perl5/5.26.1/Test/Builder.pm line 309
Test::Builder::subtest(Test::Builder=HASH(0x5585c65205e0), "Cache tests", CODE(0x5585d3040e38)) called at /usr/lib/perl5/5.26.1/Test/More.pm line 807
Test::More::subtest("Cache tests", CODE(0x5585d3040e38)) called at t/full-stack.t line 448
[13:01:07] t/full-stack.t ......... 37/? # Tests were run but no plan was declared and done_testing() was not seen.
First bad from CI tests in master from 1d ago:
https://app.circleci.com/pipelines/github/os-autoinst/openQA/14053/workflows/01e1dbe5-af68-49f4-8a3e-ad24fb39bf42/jobs/132357
with packages installed https://circleci-tasks-prod.s3.us-east-1.amazonaws.com/storage/artifacts/3f1e85c4-c546-442b-88e5-4343d260c2f4/0ca21dbe-25ad-4f52-86dd-f1eed597e34a/0/artifacts/packages.txt
Expected result¶
Last good from CI tests in master from 2d ago:
https://app.circleci.com/pipelines/github/os-autoinst/openQA/14048/workflows/96f1def0-eac9-40db-ba85-60b39b930227/jobs/132314
with packages installed https://circleci-tasks-prod.s3.us-east-1.amazonaws.com/storage/artifacts/3f1e85c4-c546-442b-88e5-4343d260c2f4/ff48df68-9ff6-4b4c-9c0c-537e97e67103/0/artifacts/packages.txt
no difference to "first bad"
Rollback steps¶
Updated by tinita 5 months ago · Edited
Could it be a new version of chromium/chromedriver?
I get a failure as well on the latest 15.6 image, but it looks different.
But it fails around the same test:
ok 36 - Cache tests
executeScript: Server returned error message read timeout at /usr/lib/perl5/vendor_perl/5.26.1/Net/HTTP/Methods.pm line 268. at repos/openQA/t/lib/OpenQA/SeleniumTest.pm:78 at repos/openQA/t/lib/OpenQA/SeleniumTest.pm line 81.
OpenQA::SeleniumTest::__ANON__(Test::Selenium::Chrome=HASH(0x55dca8e5c140), "Error while executing command: executeScript: Server returned"..., HASH(0x55dca9214370), HASH(0x55dca9206e90)) called at /usr
/lib/perl5/vendor_perl/5.26.1/Selenium/Remote/Driver.pm line 356
Selenium::Remote::Driver::catch {...} ("Error while executing command: executeScript: Server returned"...) called at /usr/lib/perl5/vendor_perl/5.26.1/Try/Tiny.pm line 123
Try::Tiny::try(CODE(0x55dca92042c8), Try::Tiny::Catch=REF(0x55dca9214730)) called at /usr/lib/perl5/vendor_perl/5.26.1/Selenium/Remote/Driver.pm line 361
Selenium::Remote::Driver::__ANON__(CODE(0x55dca75aa718), Test::Selenium::Chrome=HASH(0x55dca8e5c140), HASH(0x55dca9214370), HASH(0x55dca9206e90)) called at (eval 1549) line 1
Selenium::Remote::Driver::__ANON__(Test::Selenium::Chrome=HASH(0x55dca8e5c140), HASH(0x55dca9214370), HASH(0x55dca9206e90)) called at (eval 1551) line 2
Selenium::Remote::Driver::_execute_command(Test::Selenium::Chrome=HASH(0x55dca8e5c140), HASH(0x55dca9214370), HASH(0x55dca9206e90)) called at /usr/lib/perl5/vendor_perl/5.26.1/Selenium/Remote/Driver.pm
line 1053
Selenium::Remote::Driver::execute_script(Test::Selenium::Chrome=HASH(0x55dca8e5c140), "return window.jQuery \x{26}\x{26} jQuery.active === 0") called at /home/tina/openqa-toolbox/repos/openQA/t/lib/Open
QA/SeleniumTest.pm line 147
OpenQA::SeleniumTest::wait_for_ajax("msg", "wait_for_result_panel: waiting for '(?^u:Result: passed)' (co"...) called at /home/tina/openqa-toolbox/repos/openQA/t/lib/OpenQA/Test/FullstackUtils.pm line 8
7
OpenQA::Test::FullstackUtils::wait_for_result_panel(Test::Selenium::Chrome=HASH(0x55dca8e5c140), qr(Result: passed)u, "job 5") called at t/full-stack.t line 351
main::__ANON__() called at /usr/lib/perl5/5.26.1/Test/Builder.pm line 309
eval {...} called at /usr/lib/perl5/5.26.1/Test/Builder.pm line 309
Test::Builder::subtest(Test::Builder=HASH(0x55dc9e4bd410), "results of test 5", CODE(0x55dca91f1700)) called at /usr/lib/perl5/5.26.1/Test/More.pm line 807
Test::More::subtest("results of test 5", CODE(0x55dca91f1700)) called at t/full-stack.t line 389
main::__ANON__() called at /usr/lib/perl5/5.26.1/Test/Builder.pm line 309
eval {...} called at /usr/lib/perl5/5.26.1/Test/Builder.pm line 309
Test::Builder::subtest(Test::Builder=HASH(0x55dc9e4bd410), "Cache tests", CODE(0x55dca9109438)) called at /usr/lib/perl5/5.26.1/Test/More.pm line 807
Test::More::subtest("Cache tests", CODE(0x55dca9109438)) called at t/full-stack.t line 448
I have:
chromium-126.0.6478.126-bp156.2.6.1.x86_64
chromedriver-126.0.6478.126-bp156.2.6.1.x86_64
That's the same as in the latest registry.opensuse.org/devel/openqa/ci/containers/base:latest
Updated by tinita 5 months ago
To me it doesn't look sporadic, btw. Looking at https://app.circleci.com/pipelines/github/os-autoinst/openQA it started to fail consistently at a certain point.
Updated by okurz 5 months ago
- Tags changed from CI, sporadic, test, full-stack to CI, test, full-stack
- Subject changed from [sporadic] full-stack.t failing with "executeScript: unknown error: session deleted because of page crash at t/lib/OpenQA/SeleniumTest.pm:78" to full-stack.t failing with "executeScript: unknown error: session deleted because of page crash at t/lib/OpenQA/SeleniumTest.pm:78"
- Description updated (diff)
yes, seems to be consistent now.
Updated by tinita 5 months ago · Edited
https://bugzilla.suse.com/show_bug.cgi?id=1228180
After update from 125.0.6422.141 to 126.0.6478.126 the Chromium browser doesn't reliably load pages anymore (e.g. GMail, Confluence), loads forever or randomly shows "SIGSEGV" on white background.
actually marked as duplicate of
https://bugzilla.suse.com/show_bug.cgi?id=1227739
After updating to TW 20240711, which includes chromium 126, the browser keeps dumping core, [...]
Updated by tinita 5 months ago
Also in os-autoinst for the fullstack test: https://github.com/os-autoinst/os-autoinst/pull/2517 Downgrade chromedriver to avoid broken version
Updated by tinita 4 months ago · Edited
- Status changed from Blocked to Feedback
According to https://bugzilla.suse.com/show_bug.cgi?id=1227739 fixed in chromium 127
We already have passing pull requests with that version as we only install the old package if we have version 126.
Remove workaround for broken chromium:
Updated by tinita 4 months ago
- Related to action #165764: [ci] t/full-stack.t failing since 2024-08-23 added