Project

General

Profile

Actions
Copy link

action #152077

closed

coordination #154768: [saga][epic][ux] State-of-art user experience for openQA

coordination #154771: [epic] Improved test developer user experience

[sporadic][unstable] openQA unit test t/ui/15-comments.t failed "four pagination buttons present (one is >>)" size:M

Added by okurz about 1 year ago. Updated 2 months ago.

Status:
Resolved
Priority:
High
Assignee:
mkittler
Category:
Regressions/Crashes
Target version:
Ready
Start date:
2023-12-05
Due date:
% Done:

0%

Estimated time:
Tags:
openQA, regression, reactive work, unit, circleCI

Description

https://progress.opensuse.org/issues/152077

Observation

From https://app.circleci.com/pipelines/github/os-autoinst/openQA/12591/workflows/3ef82cf3-28ad-4294-9bf8-d753f9fd0eec/jobs/117568

[03:05:16] t/ui/15-comments.t ......................... 11/? 
        #   Failed test 'four pagination buttons present (one is >>)'
        #   at t/ui/15-comments.t line 490.
        #          got: '3'
        #     expected: '4'

        #   Failed test 'only 1 comment present on last page'
        #   at t/ui/15-comments.t line 493.
        #          got: '2'
        #     expected: '1'
        # Looks like you failed 2 tests of 11.

    #   Failed test 'group overview: /parent_group_overview/1'
    #   at t/ui/15-comments.t line 495.
    # Looks like you failed 1 test of 5.
[03:05:16] t/ui/15-comments.t ......................... 12/? 
#   Failed test 'editing when logged in as regular user'

I am sure this test has failed in more cases

Acceptance criteria

  • AC1: Test passes consistently

Sugestions

  • Try to reproduce it locally
Actions
Copy link
 #1

Updated by tinita about 1 year ago

  • Subject changed from [sporadic][unstable] openQA unit test t/ui/15-comments.t failed "our pagination buttons present (one is >>)" to [sporadic][unstable] openQA unit test t/ui/15-comments.t failed "four pagination buttons present (one is >>)" size:M
  • Description updated (diff)
  • Status changed from New to Workable
Actions
Copy link
 #2

Updated by mkittler about 1 year ago

  • Assignee set to mkittler
Actions
Copy link
 #3

Updated by mkittler about 1 year ago

  • Status changed from Workable to In Progress
Actions
Copy link
 #4

Updated by mkittler about 1 year ago · Edited

I haven't be able to reproduce this locally and I'm already at:

## count_fail_ratio: Run: 89. Fails: 0. Fail ratio 0±0%. No fails, computed failure probability < 3.37%

So I created a PR with additional debugging code which might be able to reproduce the problem: https://github.com/os-autoinst/openQA/pull/5392

EDIT: I'm now at:

## count_fail_ratio: Run: 274. Fails: 0. Fail ratio 0±0%. No fails, computed failure probability < 1.09%

So very unlikely locally reproducible.

Actions
Copy link
 #5

Updated by openqa_review about 1 year ago

  • Due date set to 2023-12-23

Setting due date based on mean cycle time of SUSE QE Tools

Actions
Copy link
 #6

Updated by okurz about 1 year ago · Edited

https://github.com/os-autoinst/openQA/pull/5393 to test in CI

I am also running locally

runs=200 count_fail_ratio env OPENQA_TEST_TIMEOUT_SCALE_COVER=5 make coverage KEEP_DB=1 TESTS=t/ui/15-comments.t

EDIT: completed

-> count_fail_ratio: Run: 200. Fails: 0. Fail ratio 0±0%. No fails, computed failure probability < 1.50%

Actions
Copy link
 #7

Updated by mkittler about 1 year ago · Edited

Still not reproducible:

$ time env runs=400 "$OPENQA_BASEDIR/repos/okurz-github-scripts/count_fail_ratio" make coverage TEST_PG='DBI:Pg:dbname=openqa_test;host=/dev/shm/tpg' TESTS=t/ui/15-comments.t
…
HTML output written to /hdd/openqa-devel/repos/openQA/cover_db/coverage.html
done.
## count_fail_ratio: Run: 280. Fails: 0. Fail ratio 0±0%. No fails, computed failure probability < 1.07%
## Run 281
export DEVEL_COVER_DB_FORMAT=JSON;\
COVERAGE=1 cover  -test
…

I could also not reproduce it anymore via https://github.com/os-autoinst/openQA/pull/5393 (the CI run just fails eventually after exceeding the context deadline and gave the same result after restarting).

That's very strange considering I saw this quite often in the week before this ticket was filed.

Actions
Copy link
 #8

Updated by okurz about 1 year ago

  • Due date deleted (2023-12-23)
  • Status changed from In Progress to Resolved

This couldn't reproduce the issue. Unless we see that reappearing I assume that this was a temporary fluke that needs no action from our side.

Actions
Copy link
 #9

Updated by okurz 2 months ago

  • Parent task set to #154771
Actions
Copy link

Also available in: Atom PDF