Actions
action #182168
closed[alert] fullstack test failing `FAILED--Further testing stopped: 'finished download' not available`
Start date:
2025-05-09
Due date:
% Done:
0%
Estimated time:
Tags:
Description
Observation¶
Running prove with TAP output check ...
# full result panel contents:
# State: scheduled, created 4 minutes ago
# Scheduled product: job has not been created by posting an ISO
# Priority: 50
# Actions:
#
# Cancel job
# Failed test 'Expected result not found'
# at /home/squamata/project/t/lib/OpenQA/Test/FullstackUtils.pm line 76.
# Failed test 'job 5 running'
# at t/full-stack.t line 335.
# job info: {
# 'blocked_by_id' => undef,
# 'clone_id' => undef,
# 'group_id' => undef,
# 'id' => 5,
# 'name' => 'tinycore-1-flavor-i386-Build1-core@coolone',
# 'priority' => 50,
# 'result' => 'none',
# 'settings' => {
# 'ARCH' => 'i386',
# 'BUILD' => '1',
# 'DISTRI' => 'tinycore',
# 'FLAVOR' => 'flavor',
# 'INTEGRATION_TESTS' => '1',
# 'ISO' => 'Core-7.2.iso',
# 'MACHINE' => 'coolone',
# 'NAME' => '00000005-tinycore-1-flavor-i386-Build1-core@coolone',
# 'PUBLISH_HDD_1' => '',
# 'QEMU' => 'i386',
# 'QEMU_NO_FDC_SET' => '1',
# 'QEMU_NO_KVM' => '1',
# 'QEMU_NO_TABLET' => '1',
# 'TEST' => 'core',
# 'UEFI_PFLASH_VARS' => '/usr/share/qemu/ovmf-x86_64.bin',
# 'VERSION' => '1',
# 'WORKER_CLASS' => 'qemu_i386'
# },
# 'state' => 'scheduled',
# 't_finished' => undef,
# 't_started' => undef,
# 'test' => 'core'
# }
Bailout called. Further testing stopped: 'finished download' not available
FAILED--Further testing stopped: 'finished download' not available
make[2]: *** [Makefile:242: test-unit-and-integration] Error 255
make[2]: Leaving directory '/home/squamata/project'
make[1]: *** [Makefile:237: test-with-database] Error 2
make[1]: Leaving directory '/home/squamata/project'
make: *** [Makefile:227: test-fullstack] Error 2
Exited with code exit status 2
Mitigation already apply with a rerun of the workflow: https://app.circleci.com/pipelines/github/os-autoinst/openQA/16902/workflows/e405166c-1515-42b3-8f73-494d47d714fc
Updated by tinita 24 days ago
From the verbose log:
https://output.circle-artifacts.com/output/job/fd9aec3e-d652-41a0-b371-63ed20a03ec1/artifacts/0/artifacts/full-stack.t
[error] [pid:919] Unavailable: The average load (96.69 43.69 26.20) is exceeding the configured threshold of 40. The worker will temporarily not accept new jobs until the load is lower again.
[info] [pid:919] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/full-stack.d/openqa/share
[info] [pid:919] Registering with openQA http://localhost:9526
ok 5 - at least one free worker with class qemu_i386 registered
[info] [pid:919] Establishing ws connection via ws://localhost:9526/api/v1/ws/1
[debug] [pid:201] Scheduling: Free workers: 1/1; Scheduled jobs: 1
[debug] [pid:201] Need to schedule 1 parallel jobs for job 5 (with priority 50)
[debug] [pid:201] limit reached, scheduling no additional jobs (max_running_jobs=-1, free workers=1, running=0, allocated=1)
[info] [pid:919] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 1
[debug] [pid:216] Updating seen of worker 1 from worker_status (broken)
[debug] [pid:201] Assigned job '5' to worker ID '1'
[debug] [pid:201] [Job#5] Prepare for being processed by worker 1
[warn] [pid:919] The average load (96.69 43.69 26.20) is exceeding the configured threshold of 40. The worker will temporarily not accept new jobs until the load is lower again. - checking again for web UI 'htt
p://localhost:9526' in 234.73 s
[debug] [pid:216] Started to send message to 1 for job(s) 5
[debug] [pid:919] Refusing to grab job from http://localhost:9526: The average load (96.69 43.69 26.20) is exceeding the configured threshold of 40. The worker will temporarily not accept new jobs until the loa
d is lower again.
[debug] [pid:201] Sent job(s) '5' to worker '1'
[debug] [pid:216] Worker 1 rejected job(s) 5: The average load (96.69 43.69 26.20) is exceeding the configured threshold of 40. The worker will temporarily not accept new jobs until the load is lower again.
[debug] [pid:201] Scheduler took 0.08364s to perform operations and allocated 1 jobs
[debug] [pid:201] Allocated: { job => 5, worker => 1 }
So the load was too high. Only the last line is confusing; it says it did allocate the job, although it was refused.
Updated by tinita 24 days ago
- Category set to Regressions/Crashes
- Priority changed from Normal to High
- Target version set to Ready
Actually https://github.com/os-autoinst/openQA/pull/6283 was supposed to disable the feature by setting CRITICAL_LOAD_AVG_THRESHOLD to 0 in t/data/workers.ini.
But the log says "is exceeding the configured threshold of 40".
Also:
[info] [pid:352] worker 1:
- config file: /home/squamata/project/test-results/fullstack/full-stack.d/config/workers.ini
so it doesn't use t/data/workers.ini apparently.
Actions