Project

General

Profile

Actions
Copy link

action #95009

closed

unstable/flaky test t/ui/12-needle-edit.t size:M

Added by okurz almost 3 years ago. Updated almost 3 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
tinita
Category:
Regressions/Crashes
Target version:
Ready
Start date:
2021-07-01
Due date:
% Done:

0%

Estimated time:

Description

Observation

At least in two places I have seen t/ui/12-needle-edit.t failing, e.g. https://app.circleci.com/pipelines/github/os-autoinst/openQA/6868/workflows/e88ba45b-b130-477f-8095-441095d446a0/jobs/64683/steps

[18:12:47] t/ui/12-needle-edit.t ...................... 4/? 
    #   Failed test 'sub process openqa-webapi terminated with exit code 10752'
    #   at /home/squamata/project/t/ui/../lib/OpenQA/Test/Utils.pm line 223.
A context appears to have been destroyed without first calling release().
Based on $@ it does not look like an exception was thrown (this is not always
a reliable test)

This is a problem because the global error variables ($!, $@, and $?) will
not be restored. In addition some release callbacks will not work properly from
inside a DESTROY method.

Here are the context creation details, just in case a tool forgot to call
release():
  File: t/ui/12-needle-edit.t
  Line: 415
  Tool: Test::More::subtest

Cleaning up the CONTEXT stack...

    #   Failed test 'no (unexpected) warnings (via END block)'
    #   at /usr/lib/perl5/5.26.1/Test/Builder.pm line 135.
    # Got the following unexpected warnings:
    #   1: A context appears to have been destroyed without first calling release().
    # Based on $@ it does not look like an exception was thrown (this is not always
    # a reliable test)
    # 
    # This is a problem because the global error variables ($!, $@, and $?) will
    # not be restored. In addition some release callbacks will not work properly from
    # inside a DESTROY method.
    # 
    # Here are the context creation details, just in case a tool forgot to call
    # release():
    #   File: t/ui/12-needle-edit.t
    #   Line: 415
    #   Tool: Test::More::subtest
    # 
    # Cleaning up the CONTEXT stack...
# Test ended with extra hubs on the stack!
    # Tests were run but no plan was declared and done_testing() was not seen.
[18:12:47] t/ui/12-needle-edit.t ......................      Dubious, test returned 255 (wstat 65280, 0xff00)
All 4 subtests passed

Acceptance criteria

  • AC1: t/ui/12-needle-edit.t stable over multiple runs in CI

Suggestions

  • Move test to unstable tests
  • Find out what is the failure ratio, e.g. take a look from past circleCI jobs
  • Reproduce instability in CI or locally or fix likely problem
  • Research in git logs if there are recent related changes
  • Ensure it's stable locally
  • Remove from unstable tests again

Related issues 1 (0 open — 1 closed)

Related to openQA Project - action #95024: openQA test t/ui/26-jobs_restart.t very unstable (already marked as unstable) size:MResolvedokurz2021-07-022021-07-22

Actions
Actions
Copy link

Also available in: Atom PDF