Project

General

Profile

Actions

action #108443

closed

[sporadic][timeboxed:10h] OBS os-autoinst fails in 18-qemu.t: Can't open file "base_state.json": No such file or directory size :M

Added by tinita about 2 years ago. Updated about 2 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Regressions/Crashes
Target version:
Start date:
2022-03-16
Due date:
2022-04-05
% Done:

0%

Estimated time:

Description

Observation

https://build.opensuse.org/public/build/devel:openQA:tested/openSUSE_Tumbleweed/x86_64/os-autoinst/_log

[  265s] 3: ok 8 - execute_time
[  265s] 3: ok 9 - no (unexpected) warnings (via done_testing)
[  265s] 3: 1..9
[  265s] 3: ok     1460 ms ( 0.03 usr  0.00 sys +  1.29 cusr  0.15 csys =  1.47 CPU)
[  267s] 3: Can't open file "base_state.json": No such file or directory at ./18-qemu.t line 417.
[  267s] 3:  at ./18-qemu.t line 28.
[  267s] 3:         main::__ANON__("Can't open file \"base_state.json\": No such file or directory "...) called at /usr/lib/perl5/5.34.0/Carp.pm line 291
[  267s] 3:         Carp::croak("Can't open file \"base_state.json\": No such file or directory") called at /usr/lib/perl5/vendor_perl/5.34.0/Mojo/File.pm line 132
[  267s] 3:         Mojo::File::slurp(Mojo::File=SCALAR(0x5643766b1b88)) called at ./18-qemu.t line 417
[  267s] 3:         main::__ANON__() called at /usr/lib/perl5/5.34.0/Test/Builder.pm line 374
[  267s] 3:         eval {...} called at /usr/lib/perl5/5.34.0/Test/Builder.pm line 374
[  267s] 3:         Test::Builder::subtest(Test::Builder=HASH(0x5643740f11d8), "qemu was killed due to the system being out of memory", CODE(0x564376694108)) called at /usr/lib/perl5/5.34.0/Test/More.pm line 809
[  267s] 3:         Test::More::subtest("qemu was killed due to the system being out of memory", CODE(0x564376694108)) called at ./18-qemu.t line 420
[  267s] 3:     # No tests run!
[  267s] 3: 
[  267s] 3: #   Failed test 'No tests run for subtest "qemu was killed due to the system being out of memory"'
[  267s] 3: #   at ./18-qemu.t line 420.
[  267s] 3: Can't open file "base_state.json": No such file or directory at ./18-qemu.t line 417.
[  267s] 3:  at ./18-qemu.t line 28.
[  267s] 3:         main::__ANON__("Can't open file \"base_state.json\": No such file or directory "...) called at /usr/lib/perl5/5.34.0/Test/Builder.pm line 418
[  267s] 3:         Test::Builder::subtest(Test::Builder=HASH(0x5643740f11d8), "qemu was killed due to the system being out of memory", CODE(0x564376694108)) called at /usr/lib/perl5/5.34.0/Test/More.pm line 809
[  267s] 3:         Test::More::subtest("qemu was killed due to the system being out of memory", CODE(0x564376694108)) called at ./18-qemu.t line 420
[  267s] 3: Can't open file "base_state.json": No such file or directory at ./18-qemu.t line 417.
[  267s] 3: 
[  267s] 3: #   Failed test 'no (unexpected) warnings (via END block)'
[  267s] 3: #   at /usr/lib/perl5/5.34.0/Test/Builder.pm line 193.
[  267s] 3: # Got the following unexpected warnings:
[  267s] 3: #   1: Can't open file "base_state.json": No such file or directory at ./18-qemu.t line 417.
[  267s] 3: #  at ./18-qemu.t line 28.
[  267s] 3: #         main::__ANON__("Can't open file \"base_state.json\": No such file or directory "...) called at /usr/lib/perl5/5.34.0/Carp.pm line 291
[  267s] 3: #         Carp::croak("Can't open file \"base_state.json\": No such file or directory") called at /usr/lib/perl5/vendor_perl/5.34.0/Mojo/File.pm line 132
[  267s] 3: #         Mojo::File::slurp(Mojo::File=SCALAR(0x5643766b1b88)) called at ./18-qemu.t line 417
[  267s] 3: #         main::__ANON__() called at /usr/lib/perl5/5.34.0/Test/Builder.pm line 374
[  267s] 3: #         eval {...} called at /usr/lib/perl5/5.34.0/Test/Builder.pm line 374
[  267s] 3: #         Test::Builder::subtest(Test::Builder=HASH(0x5643740f11d8), "qemu was killed due to the system being out of memory", CODE(0x564376694108)) called at /usr/lib/perl5/5.34.0/Test/More.pm line 809
[  267s] 3: #         Test::More::subtest("qemu was killed due to the system being out of memory", CODE(0x564376694108)) called at ./18-qemu.t line 420
[  267s] 3: #   2: Can't open file "base_state.json": No such file or directory at ./18-qemu.t line 417.
[  267s] 3: #  at ./18-qemu.t line 28.
[  267s] 3: #         main::__ANON__("Can't open file \"base_state.json\": No such file or directory "...) called at /usr/lib/perl5/5.34.0/Test/Builder.pm line 418
[  267s] 3: #         Test::Builder::subtest(Test::Builder=HASH(0x5643740f11d8), "qemu was killed due to the system being out of memory", CODE(0x564376694108)) called at /usr/lib/perl5/5.34.0/Test/More.pm line 809
[  267s] 3: #         Test::More::subtest("qemu was killed due to the system being out of memory", CODE(0x564376694108)) called at ./18-qemu.t line 420
[  267s] 3: # Tests were run but no plan was declared and done_testing() was not seen.
[  267s] 3: # Looks like your test exited with 255 just after 44.
[  267s] 3: [23:06:03] ./18-qemu.t .............................. 

First time occurence, have not seen it in before. The relevant code is in since 2021-05.

EDIT: Failed at least a second time also in OBS: #108443#note-3

Suggestions

  • Look into the code to understand it better. the file base_state.json is stored in the constant STATE_FILE in bmwqemu.pm and should be written by bmwqemu::serialize_state
  • Try to reproduce locally or within OBS -> runs=1000 count_fail_ratio prove -I. t/18-qemu.t does not reproduce it for okurz
  • Come up with a minimal reproducer. If it can be reproduced fix it, otherwise at least try to come up with some "improvement" in related code areas (timeboxed:10h)
Actions #1

Updated by tinita about 2 years ago

  • Subject changed from OBS os-autoinst fails in 18-qemu.t: Can't open file "base_state.json": No such file or directory to [sporadic] OBS os-autoinst fails in 18-qemu.t: Can't open file "base_state.json": No such file or directory

I retriggered and now it passed.

Actions #2

Updated by livdywan about 2 years ago

  • Subject changed from [sporadic] OBS os-autoinst fails in 18-qemu.t: Can't open file "base_state.json": No such file or directory to [sporadic][timeboxed:10h] OBS os-autoinst fails in 18-qemu.t: Can't open file "base_state.json": No such file or directory size :M
  • Description updated (diff)
  • Status changed from New to Workable
Actions #4

Updated by okurz about 2 years ago

  • Description updated (diff)

ok, that's good, so it reproduces elsewhere. I ran runs=1000 count_fail_ratio prove -I. t/18-qemu.t locally now and that did not fail in 1000 runs.

Actions #5

Updated by kraih about 2 years ago

  • Assignee set to kraih
Actions #6

Updated by kraih about 2 years ago

  • Status changed from Workable to In Progress
Actions #7

Updated by kraih about 2 years ago

Not a good start:

...
All tests successful.
Files=1, Tests=45,  2 wallclock secs ( 0.02 usr  0.00 sys +  0.20 cusr  0.06 csys =  0.28 CPU)
Result: PASS
## count_fail_ratio: Run: 1000. Fails: 0. Fail ratio 0%

So far i also can't replicate the problem locally.

Actions #8

Updated by openqa_review about 2 years ago

  • Due date set to 2022-04-05

Setting due date based on mean cycle time of SUSE QE Tools

Actions #9

Updated by kraih about 2 years ago

Think i have a solution, once GitHub works again i'll open a PR.

Actions #11

Updated by kraih about 2 years ago

  • Status changed from In Progress to Feedback
Actions #12

Updated by kraih about 2 years ago

  • Status changed from Feedback to Resolved
Actions

Also available in: Atom PDF