QA » openQA Project

Observation¶

https://app.circleci.com/pipelines/github/os-autoinst/openQA/9895/workflows/bd749c46-c563-4fcd-9d17-6195780465f3/jobs/93334/steps shows

[06:59:35] t/ui/23-audit-log.t ........................ 12/? 
    #   Failed test 'correct number of elements'
    #   at t/ui/23-audit-log.t line 132.
    #          got: '9'
    #     expected: '1'
    # less than a minute ago Demo job_create {
    #   "TEST": "foo",
    #   "id": 1
    # }, less than a minute ago Demo comment_create {
    #   "id": 1,
    #   "job_id": 1
    # }, less than a minute ago Demo table_create {
    #   "backend": "qemu",
    #   "description": null,
    #   "id": 1,
    #   "name": "foo",
    #   "settings": [],
    #   "table": "Machines"
    # }, less than a minute ago Demo table_create {
    #   "description": null,
    #   "id": 1,
    #   "name": "testsuite",
    #   "settings": [],
    #   "table": "TestSuites"
    # }, less than a minute ago Demo table_create {
    #   "arch": "x86_64",
    #   "description": null,
    #   "distri": "opensuse",
    #   "flavor": "DVD",
    #   "id": 1,
    #   "name": "",
    #   "settings": [],
    #   "table": "Products",
    #   "version": "13.2"
    # }, less than a minute ago Demo user_login null, less than a minute ago system startup openQA restarted, less than a minute ago Demo user_login null, less than a minute ago system startup openQA restarted
    # Looks like you failed 1 test of 18.
[06:59:35] t/ui/23-audit-log.t ........................ 13/? 
#   Failed test 'clickable events'
#   at t/ui/23-audit-log.t line 140.
[06:59:35] t/ui/23-audit-log.t ........................ 14/? # Looks like you failed 1 test of 14.
                                                              [06:59:35] t/ui/23-audit-log.t ........................ Dubious, test returned 1 (wstat 256, 0x100)

Acceptance criteria¶

• AC1: t/ui/23-audit-log.t stable in many runs
• AC2: t/ui/23-audit-log.t is not included in tools/unstable_tests.txt

Suggestions¶

• Read about the history of latest changes in https://progress.opensuse.org/issues/111539, maybe okurz already tried to run 1000 runs locally and couldn't reproduce or something.
• Try it out locally with coverage enabled and see if it can be reproduced. Maybe try NON_HEADLESS=1. Or try to spot the mistake by code analysis.
• Fix the test race-free
• Remove test module from tools/unstable_tests.txt
• Optional: Try to gather statistics from circleCI because we have similar problems reappearing