action #80374
closed
unstable/flaky/sporadic test t/ui/25-developer_mode.t
Added by okurz over 3 years ago.
Updated almost 3 years ago.
Category:
Regressions/Crashes
Description
Observation¶
https://app.circleci.com/pipelines/github/os-autoinst/openQA/5067/workflows/10cbcdac-cd57-4c67-8cb6-0483c8a98ba4/jobs/48181/steps shows that t/ui/25-developer_mode.t failed:
[16:58:27] t/ui/25-developer_mode.t ............... 8/?
# Failed test 'foo selected'
# at t/ui/25-developer_mode.t line 447.
# got: '1'
# expected: '0'
# Failed test 'option #4 present'
# at t/ui/25-developer_mode.t line 449.
# Looks like you failed 2 tests of 2.
# Failed test 'select module to pause at'
# at t/ui/25-developer_mode.t line 475.
# Looks like you failed 1 test of 19.
[16:58:27] t/ui/25-developer_mode.t ............... 12/?
# Failed test 'start developer session'
# at t/ui/25-developer_mode.t line 572.
[16:58:27] t/ui/25-developer_mode.t ............... 13/? [16:58:27] t/ui/25-developer_mode.t ............... 14/? # Looks like you failed 1 test of 14.
[16:58:27] t/ui/25-developer_mode.t ............... Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/14 subtests
I am not aware of seeing this problem in before. Has anyone else seen it yet?
Acceptance criteria¶
- AC1: t/ui/25-developer_mode.t is stable
Acceptance tests¶
- AT1-1:
make test STABILITY_TEST=1 RETRIES=50 KEEP_DB=1 TESTS=1 t/ui/25-developer_mode.t
- AT1-2: Same as AT1-1 but in circleCI
Suggestions¶
- DONE: Try to reproduce locally or within CI -> happened multiple times in circleCI
- DONE: If reproducible add to .circleci/unstable_tests.txt -> https://github.com/os-autoinst/openQA/pull/3884
- Consider starting with improving the error output in case of failure
- Fix the instability in the test
- Related to action #91232: Test t/ui/25-developer_mode.t failed in CI added
- Status changed from Workable to In Progress
- Assignee set to okurz
- Target version changed from future to Ready
happened again: https://circleci.com/gh/os-autoinst/openQA/60185 but a slightly different error (similar code region):
[13:19:00] t/ui/25-developer_mode.t ............... 8/?
# Failed test 'foo selected (Do not pause at a certain module)'
# at t/ui/25-developer_mode.t line 448.
# got: '1'
# expected: '0'
# Failed test 'option #4 present'
# at t/ui/25-developer_mode.t line 450.
# Looks like you failed 2 tests of 2.
# Failed test 'select module to pause at'
# at t/ui/25-developer_mode.t line 476.
# Looks like you failed 1 test of 19.
- Status changed from In Progress to Feedback
- Description updated (diff)
- Status changed from Feedback to Workable
- Assignee deleted (
okurz)
[15:32:32] t/ui/25-developer_mode.t .. 11/?
# Failed test 'changes submitted'
# at t/ui/25-developer_mode.t line 101.
# Structures begin differing at:
# $got->[0]{name} = undef
# $expected->[0]{name} = 'installation-bar'
# Failed test 'foo selected (Do not pause at a certain module)'
# at t/ui/25-developer_mode.t line 448.
# got: '1'
# expected: '0'
# Failed test 'option #4 present'
# at t/ui/25-developer_mode.t line 450.
# Looks like you failed 2 tests of 2.
# Failed test 'select module to pause at'
# at t/ui/25-developer_mode.t line 476.
# Looks like you failed 2 tests of 19.
[15:32:32] t/ui/25-developer_mode.t .. 12/?
# Failed test 'start developer session'
# at t/ui/25-developer_mode.t line 573.
[15:32:32] t/ui/25-developer_mode.t .. 13/? [15:32:32] t/ui/25-developer_mode.t .. 14/? # Looks like you failed 1 test of 14.
[15:32:32] t/ui/25-developer_mode.t .. Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/14 subtests
[15:33:17]
Test Summary Report
-------------------
t/ui/25-developer_mode.t (Wstat: 256 Tests: 14 Failed: 1)
Failed test: 12
Non-zero exit status: 1
Files=1, Tests=14, 45.4141 wallclock secs ( 0.14 usr 0.01 sys + 24.97 cusr 0.81 csys = 25.93 CPU)
Result: FAIL
Retry 2 of 5 …
[15:33:18] t/ui/25-developer_mode.t .. 9/? Bailout called. Further testing stopped: clickElement: stale element reference: element is not attached to the page document at /home/squamata/project/t/ui/../lib/OpenQA/SeleniumTest.pm:93
FAILED--Further testing stopped: clickElement: stale element reference: element is not attached to the page document at /home/squamata/project/t/ui/../lib/OpenQA/SeleniumTest.pm:93
Retry 3 of 5 …
[15:33:56] t/ui/25-developer_mode.t .. 9/?
# Failed test 'option #5 present'
# at t/ui/25-developer_mode.t line 369.
# Looks like you failed 1 test of 2.
# Failed test 'start developer session'
# at t/ui/25-developer_mode.t line 573.
# Failed test 'selection for pausing on screen mismatch updated'
# at t/ui/25-developer_mode.t line 613.
# got: '0'
# expected: '1'
# Looks like you failed 1 test of 11.
# Failed test 'testname and paused state updated'
# at t/ui/25-developer_mode.t line 614.
# Looks like you failed 1 test of 5.
[15:33:56] t/ui/25-developer_mode.t .. 13/?
# Failed test 'process state changes from os-autoinst/worker'
# at t/ui/25-developer_mode.t line 737.
[15:33:56] t/ui/25-developer_mode.t .. 14/? # Looks like you failed 2 tests of 14.
[15:33:56] t/ui/25-developer_mode.t .. Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/14 subtests
[15:34:39]
Test Summary Report
-------------------
t/ui/25-developer_mode.t (Wstat: 512 Tests: 14 Failed: 2)
Failed tests: 12-13
Non-zero exit status: 2
Files=1, Tests=14, 43.3515 wallclock secs ( 0.11 usr 0.02 sys + 24.46 cusr 0.79 csys = 25.38 CPU)
Result: FAIL
Retry 4 of 5 …
[15:34:40] t/ui/25-developer_mode.t .. 8/?
# Failed test 'option #5 present'
# at t/ui/25-developer_mode.t line 369.
# Looks like you failed 1 test of 2.
# Failed test 'start developer session'
# at t/ui/25-developer_mode.t line 573.
# Failed test 'selection for pausing on screen mismatch updated'
# at t/ui/25-developer_mode.t line 613.
# got: '0'
# expected: '1'
# Looks like you failed 1 test of 11.
# Failed test 'testname and paused state updated'
# at t/ui/25-developer_mode.t line 614.
[15:34:40] t/ui/25-developer_mode.t .. 13/? # Looks like you failed 1 test of 5.
# Failed test 'process state changes from os-autoinst/worker'
# at t/ui/25-developer_mode.t line 737.
[15:34:40] t/ui/25-developer_mode.t .. 14/? # Looks like you failed 2 tests of 14.
[15:34:40] t/ui/25-developer_mode.t .. Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/14 subtests
[15:35:25]
Test Summary Report
-------------------
t/ui/25-developer_mode.t (Wstat: 512 Tests: 14 Failed: 2)
Failed tests: 12-13
Non-zero exit status: 2
Files=1, Tests=14, 44.5828 wallclock secs ( 0.11 usr 0.00 sys + 24.68 cusr 0.82 csys = 25.61 CPU)
Result: FAIL
Retry 5 of 5 …
[15:35:26] t/ui/25-developer_mode.t .. 8/?
# Failed test 'changes submitted'
# at t/ui/25-developer_mode.t line 101.
# Structures begin differing at:
# $got->[0]{name} = undef
# $expected->[0]{name} = 'installation-bar'
# Failed test 'foo selected (Do not pause at a certain module)'
# at t/ui/25-developer_mode.t line 448.
# got: '1'
# expected: '0'
# Failed test 'option #4 present'
# at t/ui/25-developer_mode.t line 450.
# Looks like you failed 2 tests of 2.
# Failed test 'select module to pause at'
# at t/ui/25-developer_mode.t line 476.
# Looks like you failed 2 tests of 19.
[15:35:26] t/ui/25-developer_mode.t .. 12/?
# Failed test 'start developer session'
# at t/ui/25-developer_mode.t line 573.
[15:35:26] t/ui/25-developer_mode.t .. 13/? [15:35:26] t/ui/25-developer_mode.t .. 14/? # Looks like you failed 1 test of 14.
[15:35:26] t/ui/25-developer_mode.t .. Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/14 subtests
[15:36:11]
Test Summary Report
-------------------
t/ui/25-developer_mode.t (Wstat: 256 Tests: 14 Failed: 1)
Failed test: 12
Non-zero exit status: 1
Files=1, Tests=14, 44.9217 wallclock secs ( 0.13 usr 0.01 sys + 25.05 cusr 0.77 csys = 25.96 CPU)
Result: FAIL
make[2]: *** [Makefile:187: test-unit-and-integration] Error 1
make[2]: Leaving directory '/home/squamata/project'
make[1]: *** [Makefile:182: test-with-database] Error 2
make[1]: Leaving directory '/home/squamata/project'
make: *** [Makefile:167: test-unstable] Error 2
- Status changed from Workable to In Progress
- Assignee set to mkittler
- Status changed from In Progress to Feedback
- Status changed from Feedback to Resolved
Also available in: Atom
PDF