action #157543
closed
coordination #154777: [saga][epic] Shareable os-autoinst and test distribution plugins
coordination #108527: [epic] os-autoinst wheels for scalable code reuse of helper functions and segmented test distributions
[sporadic] ci openQA: t/ui/23-audit-log.t fails size:M
Added by tinita about 2 months ago.
Updated about 1 month ago.
Category:
Regressions/Crashes
Description
Observation¶
https://app.circleci.com/pipelines/github/os-autoinst/openQA/13196/workflows/ddb935c7-31dd-4beb-877c-25ef1e703b4d/jobs/123228
[14:08:28] t/ui/23-audit-log.t ........................ 12/?
# Failed test 'most rows filtered out when searching for table create events'
# at t/ui/23-audit-log.t line 40.
# got: '8'
# expected: '3'
# Looks like you failed 1 test of 22.
[14:08:28] t/ui/23-audit-log.t ........................ 13/?
# Failed test 'clickable events'
# at t/ui/23-audit-log.t line 152.
[14:08:28] t/ui/23-audit-log.t ........................ 14/? # Looks like you failed 1 test of 14.
[14:08:28] t/ui/23-audit-log.t ........................ Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/14 subtests
Acceptance criteria¶
- AC1: Statistically significant stable test execution of t/ui/23-audit-log.t
Suggestions¶
- DONE:
Find out current error rate locally not reproducible
- Find out current error rate locally with coverage enabled as this is possibly more likely to reproduce problems we see in circleCI
- Consider recent javascript stack related updates which might impact that
- Identify the specific point of sporadic failure source by debugging the unit test and executed code itself
- Apply changes to the test code to make it more robust. Possibly similar as in other UI tests in the past with some means of synchronization
- Verify that the test is stable again
- Parent task set to #108527
- Category set to Regressions/Crashes
- Target version set to Ready
- Priority changed from Normal to High
This is happening a lot now in various pull requests. Increasing prio
- Related to action #154261: [spike][timeboxed:20h] batch commenting on all openQA jobs, e.g. involving a specified SLE maintenance incident in webUI size:M added
- Status changed from New to In Progress
- Assignee set to mkittler
Unfortunately the detailed logs are very helpful as well:
# Subtest: clickable events
ok 1 - POST http://localhost:57597/api/v1/machines
ok 2 - 200 OK
ok 3 - POST http://localhost:57597/api/v1/test_suites
ok 4 - 200 OK
ok 5 - POST http://localhost:57597/api/v1/products
ok 6 - 200 OK
ok 7 - event emitted
# Wait for jQuery successful: DataTable ready
# Wait for jQuery successful: DataTable query
# Waiting for '8 entries present' to become available
ok 8 - all rows displayed without filter and before posting job/comment
# Subtest: undo button
ok 1 - undo button present
ok 2 - confirmation prompt shown
# Wait for jQuery successful: comment deletion
ok 3 - result shown
1..3
ok 9 - undo button
# Wait for jQuery successful: DataTable query
# Waiting for '3 entries present' to become available
not ok 10 - most rows filtered out when searching for table create events
ok 11 - event detail link present
[debug] [pid:2533] Unable to wakeup scheduler: Connection refused. Retry scheduled
ok 12 - POST http://localhost:57597/api/v1/jobs
ok 13 - 200 OK
ok 14 - exact match for JSON Pointer ""
ok 15 - POST http://localhost:57597/api/v1/jobs/1/comments
ok 16 - 200 OK
ok 17 - exact match for JSON Pointer ""
# Wait for jQuery successful: DataTable ready
# Wait for jQuery successful: DataTable query
# Waiting for '10 entries present' to become available
ok 18 - all rows displayed without filter after posting job/comment
# Wait for jQuery successful: DataTable query
# Waiting for '1 entries present' to become available
ok 19 - most rows filtered out when searching for comment create events
ok 20 - event detail link present
# Wait for jQuery successful: details loaded
ok 21 - one comment
ok 22 - right comment
1..22
not ok 13 - clickable events
ok 14 - no (unexpected) warnings (via done_testing)
1..14
I would guess this is related to #154261? Since the most rows were introduced by https://github.com/os-autoinst/openQA/pull/5509/files
It was already there before, my PR just changed the wording of the test description. However, maybe the code my PR added before is causing problems for the subsequent tests.
- Due date set to 2024-04-04
Setting due date based on mean cycle time of SUSE QE Tools
- Subject changed from [sporadic] ci openQA: t/ui/23-audit-log.t fails to [sporadic] ci openQA: t/ui/23-audit-log.t fails size:M
- Description updated (diff)
- Status changed from In Progress to Resolved
Let's assume it is not failing so far, and we can re-open if it does as someone will spot it if it blocks a PR.
- Due date deleted (
2024-04-04)
Also available in: Atom
PDF