Actions
action #132410
closed[sporadic][ci] t/ui/18-tests-details.t fails size:M
Description
Observation¶
In this PR, the test failed, but it is unrelated:
https://app.circleci.com/pipelines/github/os-autoinst/openQA/11886/workflows/0f8caf6c-6f51-4666-940f-ed2f19b14e33/jobs/111144
[13:52:43] t/ui/18-tests-details.t .................... 1/?
# Failed test 'correct tags displayed'
# at t/ui/18-tests-details.t line 154.
# Structures begin differing at:
# $got->{inst-bootmenu} = Does not exist
# $expected->{inst-bootmenu} = ARRAY(0x5645b375db90)
# Looks like you failed 1 test of 4.
[13:52:43] t/ui/18-tests-details.t .................... 4/?
# Failed test 'displaying image result with candidates'
# at t/ui/18-tests-details.t line 155.
[13:52:43] t/ui/18-tests-details.t .................... 5/?
[13:52:43] t/ui/18-tests-details.t .................... 9/?
[13:52:43] t/ui/18-tests-details.t .................... 10/?
[13:52:43] t/ui/18-tests-details.t .................... 12/?
[13:52:43] t/ui/18-tests-details.t .................... 13/?
[13:52:43] t/ui/18-tests-details.t .................... 14/?
[13:52:43] t/ui/18-tests-details.t .................... 15/?
[13:52:43] t/ui/18-tests-details.t .................... 18/?
[13:52:43] t/ui/18-tests-details.t .................... 28/?
[13:52:43] t/ui/18-tests-details.t .................... 32/?
[13:52:43] t/ui/18-tests-details.t .................... 35/? # Looks like you failed 1 test of 35.
[13:52:43] t/ui/18-tests-details.t .................... Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/35 subtests
Acceptance criteria¶
- AC1: t/ui/18-details.t is sufficiently stable with less than 1/1000 failures
Suggestions¶
- Use https://github.com/okurz/scripts/blob/master/count_fail_ratio, try
count_fail_ratio prove -l t/ui/18-test-details.tlocally and try to reproduce and find out the current error rate - If not reproducible locally then try within a CI(-like) environment, otherwise if still not reproducible reject
- Narrow down the error and fix
Updated by okurz over 1 year ago
- Priority changed from Normal to High
- Target version set to Ready
Updated by okurz over 1 year ago
- Subject changed from t/ui/18-tests-details.t fails to [sporadic][ci] t/ui/18-tests-details.t fails size:M
- Description updated (diff)
- Status changed from New to Workable
Updated by livdywan over 1 year ago
I seem to be getting different failures:
LANG=en_US make test TESTS=t/ui/18-tests-details.t
[...]
t/ui/18-tests-details.t .. 4/?
# Failed test 'name filter initially not displayed'
# at t/ui/18-tests-details.t line 172.
# Failed test 'failed filter initially not displayed'
# at t/ui/18-tests-details.t line 173.
# Looks like you failed 2 tests of 17.
# Failed test 'filtering'
# at t/ui/18-tests-details.t line 208.
t/ui/18-tests-details.t .. 10/?
# Failed test 'url contains step'
# at t/ui/18-tests-details.t line 372.
# 'https://bugzilla.opensuse.org/enter_bug.cgi?product=openSUSE+Distribution&bug_file_loc=http%3A%2F%2Flocalhost
%3A35877%2Ftests%2F99946%2Fmodules%2Flogpackages%2Fsteps%2F6&comment=%23%23+Observation%0A%0AopenQA+test+in+scenario+opensuse-13.1-DVD
-i586-textmode%4032bit+fails+in%0A%5Blogpackages%5D%28http%3A%2F%2Flocalhost%3A35877%2Ftests%2F99946%2Fmodules%2Flogpackages%2Fsteps%2
F6%29%0A%0A%23%23+Test+suite+description%0A%0A%0A%0A%23%23+Reproducible%0A%0AFails+since+%28at+least%29+Build+%5B0091%5D%28http%3A%2F%
2Flocalhost%3A35877%2Ftests%2F99946%29+%28current+job%29%0A%0A%0A%23%23+Expected+result%0A%0ALast+good%3A+%5B0091%5D%28http%3A%2F%2Flo
calhost%3A35877%2Ftests%2F99945%29+%28or+more+recent%29%0A%0A%0A%23%23+Further+details%0A%0AAlways+latest+result+in+this+scenario%3A+%
5Blatest%5D%28http%3A%2F%2Flocalhost%3A35877%2Ftests%2Flatest%3Farch%3Di586%26distri%3Dopensuse%26flavor%3DDVD%26machine%3D32bit%26tes
t%3Dtextmode%26version%3D13.1%29%0A&short_desc=%5BBuild+0091%5D+openQA+test+fails+in+logpackages&cf_foundby=openQA&cf_blocker=Yes'
# doesn't match '(?^:#step/logpackages/6)'
# Looks like you failed 1 test of 1.
t/ui/18-tests-details.t .. 12/?
# Failed test 'render text results'
# at t/ui/18-tests-details.t line 420.
findElement: no such element: Unable to locate element: {"method":"css selector","selector":".current_preview .resborder"} at ~/bau/oa/repos/openQA/t/ui/../lib/OpenQA/SeleniumTest.pm:78 at ~/bau/oa/repos/openQA/t/ui/../lib/OpenQA/SeleniumTest.p
m line 81.
Updated by livdywan over 1 year ago
- Status changed from Workable to In Progress
- Assignee set to livdywan
We discussed this in the Unblock. I'm still getting the above error. Tina couldn't reproduce this or the original error the ticket is about.
NOT_HEADLESS=1 might help me debug, and I'll upgrade my Tumbleweed container as well
Updated by livdywan over 1 year ago
- Status changed from In Progress to Feedback
dup -dy wasn't enough. git clean -fd seems to have fixed the issue... so I'm guessing there is some clean-up that we are missing. I still can't reproduce the original error.
Updated by livdywan over 1 year ago
- Status changed from Feedback to In Progress
Updated by openqa_review over 1 year ago
- Due date set to 2023-08-04
Setting due date based on mean cycle time of SUSE QE Tools
Updated by livdywan over 1 year ago
- Status changed from In Progress to Feedback
Updated by livdywan over 1 year ago
- Status changed from Feedback to Resolved
for i in $(seq 1 1000); do LANG=en_US make test TESTS=t/ui/18-tests-details.t || break; done
No failures, hence resolving.
Actions