action #151522
closedopenqa-clone-job succeeds even when assets are missing size:M
Description
Observation¶
OpenQA UI shows:
Result: incomplete, finished 3 minutes ago (0)
Reason: asset failure: Cannot find HDD_1 asset hdd/opensuse-Tumbleweed-x86_64-20231027-textmode@64bit.qcow2!
Scheduled product: job has not been created by posting an ISO
openqa-clone-job says:
openqa-instance:/var/lib/openqa/share/tests/opensuse # openqa-clone-job --skip-deps --from https://openqa.opensuse.org/tests/3682089 SYS_PARAM_CHECK_TEST=https://github.com/josegomezr/sys-param-check
Cloning parents of opensuse-Tumbleweed-DVD-x86_64-Build20231027-sys-param-check@64bit
downloading
https://openqa.opensuse.org/tests/3682089/asset/other/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20231027-Media.iso.sha256
to
/var/lib/openqa/share/factory/other/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20231027-Media.iso.sha256
1 job has been created:
- opensuse-Tumbleweed-DVD-x86_64-Build20231027-sys-param-check@64bit -> http://localhost/tests/638
I did not pass --ignore-missing-assets
to the invocation, yet it did not fail.
Acceptance criteria¶
- AC1: Missing assets are reported as early as possible (ideally by openqa-clone-job, that likely already works and this likely just comes down to AC2)
- AC2: When using
--skip-deps
to skip chained parents the HDD image is attempted to be downloaded (like when--skip-chained-deps
is used)
Suggestions¶
- Find out if openqa-clone-job even knows about that assets (e.g. if they got cleaned up in the meantime)
- Have a look how
--ignore-missing-assets
is implemented and why the clone command didn't fail even without passing this parameter - Maybe improve documentation for the various --skip options and mention asset download behavior
Files
Updated by openqa_review 11 months ago
- Due date set to 2023-12-15
Setting due date based on mean cycle time of SUSE QE Tools
Updated by mkittler 11 months ago · Edited
After having a closer look the problem looks different than what we have assumed during the estimation. It looks like --skip-deps
and --skip-chained-deps
would behave identical.
The problem here is that openqa-clone-job
simply does not even try to download these missing assets because they are not even present anymore in the web UI at all and openqa-clone-job
relies on the list of assets provided by the web UI (as it does not try to parse any asset-related job settings anymore).
For this job the list of assets is:
openqa-cli api --pretty --o3 jobs/3682089
{
"job" : {
"assets" : {
"other" : [
"openSUSE-Tumbleweed-DVD-x86_64-Snapshot20231027-Media.iso.sha256"
]
},
…
So that's all openqa-clone-job
sees and attempts to download for this job. Just for comparison, that's how it looks like for the most recent job:
openqa-cli api --pretty --o3 jobs/3773913
{
"job" : {
"assets" : {
"hdd" : [
"opensuse-Tumbleweed-x86_64-20231130-textmode@64bit.qcow2"
],
"iso" : [
"openSUSE-Tumbleweed-DVD-x86_64-Snapshot20231130-Media.iso"
],
"other" : [
"Tumbleweed.x86_64-1.0-libvirt-Snapshot20231130.vagrant.libvirt.box",
"Tumbleweed.x86_64-1.0-virtualbox-Snapshot20231130.vagrant.virtualbox.box",
"openSUSE-Tumbleweed-DVD-x86_64-Snapshot20231130-Media.iso.sha256"
],
"repo" : [
"openSUSE-Tumbleweed-non-oss-x86_64-Snapshot20231130",
"openSUSE-Tumbleweed-oss-x86_64-Snapshot20231130",
"openSUSE-Tumbleweed-oss-x86_64-Snapshot20231130-debuginfo",
"openSUSE-Tumbleweed-oss-x86_64-Snapshot20231130-source",
"openSUSE-Tumbleweed-oss-x86_64.license-Snapshot20231130"
]
},
…
When downloading the latest job with I confused job IDs. When cloning the latest job (currently 3773913) it downloads the HDD as well as expected. The HDD is only skipped if --skip-deps
(or --skip-chained-deps
, that makes not difference after all) it downloads the ISO (because it is specified as asset even on the chained child job) but not the HDD which seems wrong (considering we are skipping the parent that would create it). So that is yet another problem.--skip-deps
or --skip-chained-deps
is not present (then it makes sense). So everything's good in that regard.
That means AC2 is already fulfilled and AC1 is the problematic one after all.
Updated by mkittler 11 months ago · Edited
When trying to restart https://openqa.opensuse.org/tests/3682089 via the web UI it complains with
Job 3682089 misses the following mandatory assets: hdd/opensuse-Tumbleweed-x86_64-20231027-textmode@64bit.qcow2, iso/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20231027-Media.iso, other/Tumbleweed.x86_64-1.0-libvirt-Snapshot20231027.vagrant.libvirt.box, other/Tumbleweed.x86_64-1.0-virtualbox-Snapshot20231027.vagrant.virtualbox.box
You may try to retrigger the parent job that should create the assets and will implicitly retrigger this job as well.
as expected. This works because the web UI really re-evaluates the job settings and then checks for the presence of assets. Maybe we need to do the same within openqa-clone-job
so fulfill AC1. The openqa-clone-job
script can of course not do this kind of check as it doesn't usually run on the web UI host. So it needed to ask the web UI to do the checking.
Updated by mkittler 11 months ago
- Status changed from In Progress to Feedback
PR implementing a check for missing assets as mentioned in my previous comment: https://github.com/os-autoinst/openQA/pull/5383