action #166445
Updated by okurz 3 months ago
## Observation
The inner test is incomplete despite the only step simple_boot is passed.
openQA test in scenario openqa-Tumbleweed-dev-x86_64-openqa_from_bootstrap@64bit-2G fails in
[tests](https://openqa.opensuse.org/tests/4472003#step/tests/7) [tests](https://openqa.opensuse.org/tests/4459309/modules/tests/steps/7)
autoinst-log.txt for the inner tests shows:
```
[2024-09-11T07:00:30.593688-04:00] [debug] [pid:51558] [autotest] process exited: 0
… [2024-09-11T07:00:30.593915-04:00] [debug] [pid:51558] stopping command server 51574 because test execution ended
[2024-09-11T07:00:30.593982-04:00] [debug] [pid:51558] isotovideo: informing websocket clients before stopping command server: http://127.0.0.1:20013/Q1415NNLHlWAEQ8N/broadcast
[2024-09-11T07:00:30.604258-04:00] [debug] [pid:51558] commands process exited: 0
[2024-09-11T07:00:30.704548-04:00] [debug] [pid:51558] done with command server
[2024-09-11T07:00:30.704675-04:00] [debug] [pid:51558] stopping autotest process 51577
[2024-09-11T07:00:30.704768-04:00] [debug] [pid:51558] done with autotest process
[2024-09-11T07:00:30.704846-04:00] [debug] [pid:51558] isotovideo failed
[2024-09-11T07:00:30.705440-04:00] [debug] [pid:51558] stopping backend process 51578
[2024-09-11T07:00:30.705630-04:00] [debug] [pid:51578] backend got TERM
[2024-09-11T07:00:30.705933-04:00] [info] [pid:51578] ::: OpenQA::Qemu::Proc::save_state: Saving QEMU state to qemu_state.json
[2024-09-11T07:00:31.758537-04:00] [debug] [pid:51578] Passing remaining frames to the video encoder
[2024-09-11T07:00:31.794569-04:00] [debug] [pid:51578] Waiting for video encoder to finalize the video
[2024-09-11T07:00:31.794634-04:00] [debug] [pid:51578] The built-in video encoder (pid 51580) terminated
[2024-09-11T07:00:31.794967-04:00] [debug] [pid:51578] QEMU: qemu-system-x86_64: terminating on signal 15 from pid 51578 (/usr/bin/isotovideo: backend)
[2024-09-11T07:00:31.795581-04:00] [debug] [pid:51578] sending magic and exit
[2024-09-11T07:00:31.908863-04:00] [debug] [pid:51558] done with backend process
51558: EXIT 1
[2024-09-11T07:00:31.927014-04:00] [info] Isotovideo exit status: 1
[2024-09-11T07:00:31.965003-04:00] [info] +++ worker notes +++
[2024-09-11T07:00:31.965099-04:00] [info] End time: 2024-09-11 11:00:31
[2024-09-11T07:00:31.965147-04:00] [info] Result: died
[2024-09-11T07:00:31.974998-04:00] [info] Uploading video.ogv
[2024-09-11T07:00:32.010558-04:00] [info] Uploading autoinst-log.txt
```
also from the worker
```
[2024-09-11T18:03:44.887585Z] [error] REST-API error (POST https://openqa.opensuse.org/api/v1/jobs/4472003/status): Connection error: Premature connection close (remaining tries: 59)
```
## Reproducible
Fails since (at least) Build [:TW.31137](https://openqa.opensuse.org/tests/4459309) (current job). Judging from https://openqa.opensuse.org/tests/4471721#next_previous it seems the job fails the same at least 1/100 times. job)
## Expected result
Last good: [:TW.31136](https://openqa.opensuse.org/tests/4458743) (or more recent)
## Suggestions
* Take a look into the inner job's autoinst-log.txt and also consider to upload the worker-log.txt
* Consider compressing and uploading the whole results directory
* Look into the error message visible in https://openqa.opensuse.org/tests/4471721/logfile?filename=tests-journal.log.txt#line-5968
```
Sep 11 10:04:20 susetest qemu-system-x86_64[124364]: *HARK* log.c: 92: core_log_lib_info: src version: 2.1.0
Sep 11 10:04:20 susetest qemu-system-x86_64[124364]: *HARK* log.c: 94: core_log_lib_info: compiled with support for shutdown state
Sep 11 10:04:20 susetest qemu-system-x86_64[124364]: *HARK* log.c: 97: core_log_lib_info: compiled with libndctl 63+
Sep 11 10:04:20 susetest qemu-system-x86_64[124373]: *HARK* log.c: 92: core_log_lib_info: src version: 2.1.0
Sep 11 10:04:20 susetest qemu-system-x86_64[124373]: *HARK* log.c: 94: core_log_lib_info: compiled with support for shutdown state
Sep 11 10:04:20 susetest qemu-system-x86_64[124373]: *HARK* log.c: 97: core_log_lib_info: compiled with libndctl 63+
Sep 11 10:04:20 susetest qemu-system-x86_64[124372]: *HARK* log.c: 92: core_log_lib_info: src version: 2.1.0
Sep 11 10:04:20 susetest qemu-system-x86_64[124372]: *HARK* log.c: 94: core_log_lib_info: compiled with support for shutdown state
Sep 11 10:04:20 susetest qemu-system-x86_64[124372]: *HARK* log.c: 97: core_log_lib_info: compiled with libndctl 63+Sep 11 10:04:18 susetest worker[124339]: [info] 124339: WORKING 1
```
## Further details
Always latest result in this scenario: [latest](https://openqa.opensuse.org/tests/latest?arch=x86_64&distri=openqa&flavor=dev&machine=64bit-2G&test=openqa_from_bootstrap&version=Tumbleweed)
Back