Project

General

Profile

Actions
Copy link

action #132410

closed

[sporadic][ci] t/ui/18-tests-details.t fails size:M

Added by tinita 11 months ago. Updated 10 months ago.

Status:
Resolved
Priority:
High
Assignee:
livdywan
Category:
Regressions/Crashes
Target version:
Ready
Start date:
2023-07-06
Due date:
2023-08-04
% Done:

0%

Estimated time:

Description

Observation

In this PR, the test failed, but it is unrelated:
https://app.circleci.com/pipelines/github/os-autoinst/openQA/11886/workflows/0f8caf6c-6f51-4666-940f-ed2f19b14e33/jobs/111144

[13:52:43] t/ui/18-tests-details.t .................... 1/? 
    #   Failed test 'correct tags displayed'
    #   at t/ui/18-tests-details.t line 154.
    #     Structures begin differing at:
    #          $got->{inst-bootmenu} = Does not exist
    #     $expected->{inst-bootmenu} = ARRAY(0x5645b375db90)
    # Looks like you failed 1 test of 4.
[13:52:43] t/ui/18-tests-details.t .................... 4/? 
#   Failed test 'displaying image result with candidates'
#   at t/ui/18-tests-details.t line 155.
[13:52:43] t/ui/18-tests-details.t .................... 5/?
 [13:52:43] t/ui/18-tests-details.t .................... 9/?
 [13:52:43] t/ui/18-tests-details.t .................... 10/?
 [13:52:43] t/ui/18-tests-details.t .................... 12/?
 [13:52:43] t/ui/18-tests-details.t .................... 13/?
 [13:52:43] t/ui/18-tests-details.t .................... 14/?
 [13:52:43] t/ui/18-tests-details.t .................... 15/?
 [13:52:43] t/ui/18-tests-details.t .................... 18/?
 [13:52:43] t/ui/18-tests-details.t .................... 28/?
 [13:52:43] t/ui/18-tests-details.t .................... 32/?
 [13:52:43] t/ui/18-tests-details.t .................... 35/? # Looks like you failed 1 test of 35.
[13:52:43] t/ui/18-tests-details.t .................... Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/35 subtests

Acceptance criteria

  • AC1: t/ui/18-details.t is sufficiently stable with less than 1/1000 failures

Suggestions

  • Use https://github.com/okurz/scripts/blob/master/count_fail_ratio, try count_fail_ratio prove -l t/ui/18-test-details.t locally and try to reproduce and find out the current error rate
  • If not reproducible locally then try within a CI(-like) environment, otherwise if still not reproducible reject
  • Narrow down the error and fix
Actions
Copy link

Also available in: Atom PDF