Project

General

Profile

Actions

action #157543

closed

coordination #154777: [saga][epic] Shareable os-autoinst and test distribution plugins

coordination #108527: [epic] os-autoinst wheels for scalable code reuse of helper functions and segmented test distributions

[sporadic] ci openQA: t/ui/23-audit-log.t fails size:M

Added by tinita 9 months ago. Updated 9 months ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Regressions/Crashes
Target version:
Start date:
2024-03-19
Due date:
% Done:

0%

Estimated time:

Description

Observation

https://app.circleci.com/pipelines/github/os-autoinst/openQA/13196/workflows/ddb935c7-31dd-4beb-877c-25ef1e703b4d/jobs/123228

[14:08:28] t/ui/23-audit-log.t ........................ 12/? 
    #   Failed test 'most rows filtered out when searching for table create events'
    #   at t/ui/23-audit-log.t line 40.
    #          got: '8'
    #     expected: '3'
    # Looks like you failed 1 test of 22.
[14:08:28] t/ui/23-audit-log.t ........................ 13/? 
#   Failed test 'clickable events'
#   at t/ui/23-audit-log.t line 152.
[14:08:28] t/ui/23-audit-log.t ........................ 14/? # Looks like you failed 1 test of 14.
[14:08:28] t/ui/23-audit-log.t ........................ Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/14 subtests 

Acceptance criteria

  • AC1: Statistically significant stable test execution of t/ui/23-audit-log.t

Suggestions

  • DONE: Find out current error rate locally not reproducible
  • Find out current error rate locally with coverage enabled as this is possibly more likely to reproduce problems we see in circleCI
  • Consider recent javascript stack related updates which might impact that
  • Identify the specific point of sporadic failure source by debugging the unit test and executed code itself
  • Apply changes to the test code to make it more robust. Possibly similar as in other UI tests in the past with some means of synchronization
  • Verify that the test is stable again

Related issues 1 (0 open1 closed)

Related to openQA Project (public) - action #154261: [spike][timeboxed:20h] batch commenting on all openQA jobs, e.g. involving a specified SLE maintenance incident in webUI size:MResolvedmkittler

Actions
Actions #1

Updated by okurz 9 months ago

  • Parent task set to #108527
Actions #2

Updated by okurz 9 months ago

  • Category set to Regressions/Crashes
  • Target version set to Ready
Actions #3

Updated by tinita 9 months ago

  • Priority changed from Normal to High

This is happening a lot now in various pull requests. Increasing prio

Actions #4

Updated by livdywan 9 months ago

I would guess this is related to #154261? Since the most rows were introduced by https://github.com/os-autoinst/openQA/pull/5509/files

Actions #5

Updated by livdywan 9 months ago

  • Related to action #154261: [spike][timeboxed:20h] batch commenting on all openQA jobs, e.g. involving a specified SLE maintenance incident in webUI size:M added
Actions #6

Updated by mkittler 9 months ago

  • Status changed from New to In Progress
  • Assignee set to mkittler
Actions #7

Updated by mkittler 9 months ago

Unfortunately the detailed logs are very helpful as well:

# Subtest: clickable events
    ok 1 - POST http://localhost:57597/api/v1/machines
    ok 2 - 200 OK
    ok 3 - POST http://localhost:57597/api/v1/test_suites
    ok 4 - 200 OK
    ok 5 - POST http://localhost:57597/api/v1/products
    ok 6 - 200 OK
    ok 7 - event emitted
    # Wait for jQuery successful: DataTable ready
    # Wait for jQuery successful: DataTable query
    # Waiting for '8 entries present' to become available
    ok 8 - all rows displayed without filter and before posting job/comment
    # Subtest: undo button
        ok 1 - undo button present
        ok 2 - confirmation prompt shown
        # Wait for jQuery successful: comment deletion
        ok 3 - result shown
        1..3
    ok 9 - undo button
    # Wait for jQuery successful: DataTable query
    # Waiting for '3 entries present' to become available
    not ok 10 - most rows filtered out when searching for table create events
    ok 11 - event detail link present
[debug] [pid:2533] Unable to wakeup scheduler: Connection refused. Retry scheduled
    ok 12 - POST http://localhost:57597/api/v1/jobs
    ok 13 - 200 OK
    ok 14 - exact match for JSON Pointer ""
    ok 15 - POST http://localhost:57597/api/v1/jobs/1/comments
    ok 16 - 200 OK
    ok 17 - exact match for JSON Pointer ""
    # Wait for jQuery successful: DataTable ready
    # Wait for jQuery successful: DataTable query
    # Waiting for '10 entries present' to become available
    ok 18 - all rows displayed without filter after posting job/comment
    # Wait for jQuery successful: DataTable query
    # Waiting for '1 entries present' to become available
    ok 19 - most rows filtered out when searching for comment create events
    ok 20 - event detail link present
    # Wait for jQuery successful: details loaded
    ok 21 - one comment
    ok 22 - right comment
    1..22
not ok 13 - clickable events
ok 14 - no (unexpected) warnings (via done_testing)
1..14

I would guess this is related to #154261? Since the most rows were introduced by https://github.com/os-autoinst/openQA/pull/5509/files

It was already there before, my PR just changed the wording of the test description. However, maybe the code my PR added before is causing problems for the subsequent tests.

Actions #8

Updated by mkittler 9 months ago

I wasn't able to reproduce the failure locally. Maybe the following changes help, though: https://github.com/os-autoinst/openQA/pull/5538

Actions #9

Updated by openqa_review 9 months ago

  • Due date set to 2024-04-04

Setting due date based on mean cycle time of SUSE QE Tools

Actions #10

Updated by okurz 9 months ago

  • Subject changed from [sporadic] ci openQA: t/ui/23-audit-log.t fails to [sporadic] ci openQA: t/ui/23-audit-log.t fails size:M
  • Description updated (diff)
Actions #11

Updated by livdywan 9 months ago

  • Status changed from In Progress to Resolved

Let's assume it is not failing so far, and we can re-open if it does as someone will spot it if it blocks a PR.

Actions #12

Updated by okurz 9 months ago

  • Due date deleted (2024-04-04)
Actions

Also available in: Atom PDF