action #98388
closedNon-existing asset "uefi-vars" is still shown up on #downloads
Description
Observation¶
An example is https://openqa.suse.de/tests/6994628#downloads
When i clone the job it fails with
downloading
http://openqa.suse.de/tests/6994628/asset/other/SLE-15-SP4-Online-x86_64-Build31.2-Media1.iso.sha256
to
/var/lib/openqa/factory/other/SLE-15-SP4-Online-x86_64-Build31.2-Media1.iso.sha256
downloading
http://openqa.suse.de/tests/6994628/asset/iso/SLE-15-SP4-Online-x86_64-Build31.2-Media1.iso
to
/var/lib/openqa/factory/iso/SLE-15-SP4-Online-x86_64-Build31.2-Media1.iso
downloading
http://openqa.suse.de/tests/6994628/asset/hdd/SLES-15-SP4-x86_64-Build31.2-containers.qcow2
to
/var/lib/openqa/factory/hdd/SLES-15-SP4-x86_64-Build31.2-containers.qcow2
downloading
http://openqa.suse.de/tests/6994628/asset/hdd/SLES-15-SP4-x86_64-Build31.2-containers-uefi-vars.qcow2
to
/var/lib/openqa/factory/hdd/SLES-15-SP4-x86_64-Build31.2-containers-uefi-vars.qcow2
6994628 failed: 404 Not Found
uefi-vars qcow2 seems that it is not available any more and if you try to get this file you get 404.
Expected:
- ui should not show non-available assets
Steps to reproduce¶
Clone the job in the description, it returns a 404 error for an asset still shown in the UI page
Suggestions¶
- Investigate why is it still shown in the UI page if the cleanup removes it
- Investigate why this is removed (if it is from the cleanup script) but not the rest
- Fix UI so that it doesn't show non-available assets
- (optional) i wonder if the cleanup can be more clever and keep the relevant assets
Updated by mkittler over 2 years ago
- Project changed from openQA Infrastructure to openQA Project
- Category set to Regressions/Crashes
Updated by mkittler over 2 years ago
- Subject changed from deleted asset is still shown up on #downloads to Non-existing asset is still shown up on #downloads
This is a side-effect of https://github.com/os-autoinst/openQA/pull/4136. As part of this change we're registering assets with their jobs even if they don't exist yet to be able to associate them with their jobs. That's important because only then the cleanup code knows that child jobs need assets (created by a parent job) and can preserve it at least until all children ran. In this case the asset is never created by the parent job¹ and apparently also not actually used by the child job². Maybe we should simply hide such assets³ within the UI or make it apparent that the asset isn't existing (e.g. just render the asset name but no link). Since the openqa-clone-job output is mentioned this should also apply to the API route this script is using to query assets.
However, the job setup is still questionable. I mean, openQA is basically told that this asset is created/consumed so it isn't too unreasonable to assume the asset is there and required when cloning the child job. As a workaround you can use --skip-download and do the downloads which are actually needed yourself.
¹ The parent job has PUBLISH_PFLASH_VARS=SLES-15-SP4-x86_64-Build31.2-containers-uefi-vars.qcow2 but never actually uploaded the asset according to the worker log. The initial ticket description claims the asset has been deleted but which implies it has been created but I don't see that. I'm wondering how it was concluded that the asset has been deleted/created.
² The child job failed but it doesn't look like it failed due to the absence of the asset.
³ This would basically be all assets where size is NULL/undef.
Updated by ybonatakis over 2 years ago
SLES-15-SP4-x86_64-Build31.2-containers-uefi-vars.qcow2 exists in https://openqa.suse.de/admin/assets
but the link in the https://openqa.suse.de/tests/7059661 and https://openqa.suse.de/tests/7059660 give me 404.
i run the the whole chain building the hdd(as you said the image is not uploaded, so i assume that it was never deleted) and then triggered the children. What am i missing?
Updated by favogt over 2 years ago
mkittler wrote:
However, the job setup is still questionable. I mean, openQA is basically told that this asset is created/consumed so it isn't too unreasonable to assume the asset is there and required when cloning the child job.
FWICT, this is to have a single test suite definition which works for UEFI and non-UEFI tests, e.g. upgrade_Leap_15.2_kde
BOOTFROM=cdrom
DESKTOP=kde
HDD_1=%DISTRI%-%ORIGINAL_VERSION%-%ARCH%-GM-kde@%MACHINE%.qcow2
MAX_JOB_TIME=14400
ORIGINAL_VERSION=15.2
UEFI_PFLASH_VARS=%DISTRI%-%ORIGINAL_VERSION%-%ARCH%-GM-kde@%MACHINE%-uefi-vars.qcow2
UPGRADE=1
WORKER_CLASS=heavyload
It would require either overriding UEFI_PFLASH_VARS in the job group for non-EFI machines or duplicating the test suite.
Updated by mkittler over 2 years ago
If the asset appears on https://openqa.suse.de/admin/assets it might still be missing if the parent job doesn't actually create it (as explained before).
i run the the whole chain building the hdd and then triggered the children. What am i missing?
As explained before, to me it looks like the job chain simply doesn't create that asset (despite mentioning it in the job settings). I doubt just re-running the job chain changes this. Note that we're takling about SLES-15-SP4-x86_64-Build31.2-containers-uefi-vars.qcow2, SLES-15-SP4-x86_64-Build31.2-containers.qcow2 seems to be created and shouldn't give you 404.
Updated by VANASTASIADIS over 2 years ago
- Priority changed from Normal to High
- Target version set to Ready
If this is a recent regression it should be on the backlog. Bumping the priority.
Updated by okurz over 2 years ago
- Related to action #97304: Assets deleted even if there are still pending jobs size:M added
Updated by okurz over 2 years ago
- Subject changed from Non-existing asset is still shown up on #downloads to Non-existing asset "uefi-vars" is still shown up on #downloads
- Priority changed from High to Normal
So I am confused and I am wondering what people would actually expect. I would of course also prefer if we would have less special handling for those dreaded "uefi-vars" assets.
But let's ask @ybonatakis what would you prefer: For the clone script to handle that in a special way to not fail on such non-existing assets or just for the webUI to not show it (or show it but not as downloadable link)?
Updated by mkittler over 2 years ago
With this PR non-existing assets are distinguishable on the downloads tab: https://github.com/os-autoinst/openQA/pull/4212
Updated by okurz over 2 years ago
- Related to action #99672: [openqa][tools] Non-existing `…-uefi-vars.qcow2` asset disturbs openqa-clone-job workflow added
Updated by mkittler over 2 years ago
This PR improves the clone-job script: https://github.com/os-autoinst/openQA/pull/4261
Together with the previous PR this ticket should be covered. (It is still not quite clear what's wanted exactly.)
Updated by mkittler over 2 years ago
- Status changed from New to Feedback
- Assignee set to mkittler
The suggestions in this ticket are very wrong. The cleanup did not delete anything because the asset hasn't been created in the first place. The clone-job script can deal with this now since https://github.com/os-autoinst/openQA/pull/4261 has been merged and the UI shows the asset as non-existent.
I would say that's good enough so I'm waiting for feedback.
Updated by mkittler over 2 years ago
- Status changed from Feedback to Resolved
Not feedback from the author for a month so I'm considering it resolved.