Project

General

Profile

action #98388

Non-existing asset "uefi-vars" is still shown up on #downloads

Added by ybonatakis about 2 months ago. Updated 16 days ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Concrete Bugs
Target version:
Start date:
2021-09-09
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

An example is https://openqa.suse.de/tests/6994628#downloads

When i clone the job it fails with

downloading
http://openqa.suse.de/tests/6994628/asset/other/SLE-15-SP4-Online-x86_64-Build31.2-Media1.iso.sha256
to
/var/lib/openqa/factory/other/SLE-15-SP4-Online-x86_64-Build31.2-Media1.iso.sha256
downloading
http://openqa.suse.de/tests/6994628/asset/iso/SLE-15-SP4-Online-x86_64-Build31.2-Media1.iso
to
/var/lib/openqa/factory/iso/SLE-15-SP4-Online-x86_64-Build31.2-Media1.iso
downloading
http://openqa.suse.de/tests/6994628/asset/hdd/SLES-15-SP4-x86_64-Build31.2-containers.qcow2
to
/var/lib/openqa/factory/hdd/SLES-15-SP4-x86_64-Build31.2-containers.qcow2
downloading
http://openqa.suse.de/tests/6994628/asset/hdd/SLES-15-SP4-x86_64-Build31.2-containers-uefi-vars.qcow2
to
/var/lib/openqa/factory/hdd/SLES-15-SP4-x86_64-Build31.2-containers-uefi-vars.qcow2
6994628 failed: 404 Not Found

uefi-vars qcow2 seems that it is not available any more and if you try to get this file you get 404.

Expected:

  • ui should not show non-available assets

Steps to reproduce

Clone the job in the description, it returns a 404 error for an asset still shown in the UI page

Suggestions

  • Investigate why is it still shown in the UI page if the cleanup removes it
  • Investigate why this is removed (if it is from the cleanup script) but not the rest
  • Fix UI so that it doesn't show non-available assets
  • (optional) i wonder if the cleanup can be more clever and keep the relevant assets

Related issues

Related to openQA Project - action #97304: Assets deleted even if there are still pending jobs size:MResolved2021-08-202021-09-07

Related to openQA Project - action #99672: [openqa][tools] Non-existing `…-uefi-vars.qcow2` asset disturbs openqa-clone-job workflowResolved2021-10-012021-10-20

History

#1 Updated by VANASTASIADIS about 2 months ago

  • Description updated (diff)

#2 Updated by mkittler about 2 months ago

  • Project changed from openQA Infrastructure to openQA Project
  • Category set to Concrete Bugs

#3 Updated by mkittler about 2 months ago

  • Subject changed from deleted asset is still shown up on #downloads to Non-existing asset is still shown up on #downloads

This is a side-effect of https://github.com/os-autoinst/openQA/pull/4136. As part of this change we're registering assets with their jobs even if they don't exist yet to be able to associate them with their jobs. That's important because only then the cleanup code knows that child jobs need assets (created by a parent job) and can preserve it at least until all children ran. In this case the asset is never created by the parent job¹ and apparently also not actually used by the child job². Maybe we should simply hide such assets³ within the UI or make it apparent that the asset isn't existing (e.g. just render the asset name but no link). Since the openqa-clone-job output is mentioned this should also apply to the API route this script is using to query assets.

However, the job setup is still questionable. I mean, openQA is basically told that this asset is created/consumed so it isn't too unreasonable to assume the asset is there and required when cloning the child job. As a workaround you can use --skip-download and do the downloads which are actually needed yourself.


¹ The parent job has PUBLISH_PFLASH_VARS=SLES-15-SP4-x86_64-Build31.2-containers-uefi-vars.qcow2 but never actually uploaded the asset according to the worker log. The initial ticket description claims the asset has been deleted but which implies it has been created but I don't see that. I'm wondering how it was concluded that the asset has been deleted/created.

² The child job failed but it doesn't look like it failed due to the absence of the asset.

³ This would basically be all assets where size is NULL/undef.

#4 Updated by ybonatakis about 2 months ago

SLES-15-SP4-x86_64-Build31.2-containers-uefi-vars.qcow2 exists in https://openqa.suse.de/admin/assets
but the link in the https://openqa.suse.de/tests/7059661 and https://openqa.suse.de/tests/7059660 give me 404.

i run the the whole chain building the hdd(as you said the image is not uploaded, so i assume that it was never deleted) and then triggered the children. What am i missing?

#5 Updated by favogt about 2 months ago

mkittler wrote:

However, the job setup is still questionable. I mean, openQA is basically told that this asset is created/consumed so it isn't too unreasonable to assume the asset is there and required when cloning the child job.

FWICT, this is to have a single test suite definition which works for UEFI and non-UEFI tests, e.g. upgrade_Leap_15.2_kde

BOOTFROM=cdrom
DESKTOP=kde
HDD_1=%DISTRI%-%ORIGINAL_VERSION%-%ARCH%-GM-kde@%MACHINE%.qcow2
MAX_JOB_TIME=14400
ORIGINAL_VERSION=15.2
UEFI_PFLASH_VARS=%DISTRI%-%ORIGINAL_VERSION%-%ARCH%-GM-kde@%MACHINE%-uefi-vars.qcow2
UPGRADE=1
WORKER_CLASS=heavyload

It would require either overriding UEFI_PFLASH_VARS in the job group for non-EFI machines or duplicating the test suite.

#6 Updated by mkittler about 2 months ago

If the asset appears on https://openqa.suse.de/admin/assets it might still be missing if the parent job doesn't actually create it (as explained before).

i run the the whole chain building the hdd and then triggered the children. What am i missing?

As explained before, to me it looks like the job chain simply doesn't create that asset (despite mentioning it in the job settings). I doubt just re-running the job chain changes this. Note that we're takling about SLES-15-SP4-x86_64-Build31.2-containers-uefi-vars.qcow2, SLES-15-SP4-x86_64-Build31.2-containers.qcow2 seems to be created and shouldn't give you 404.

#7 Updated by VANASTASIADIS about 2 months ago

  • Priority changed from Normal to High
  • Target version set to Ready

If this is a recent regression it should be on the backlog. Bumping the priority.

#8 Updated by okurz about 1 month ago

  • Related to action #97304: Assets deleted even if there are still pending jobs size:M added

#9 Updated by okurz about 1 month ago

  • Subject changed from Non-existing asset is still shown up on #downloads to Non-existing asset "uefi-vars" is still shown up on #downloads
  • Priority changed from High to Normal

So I am confused and I am wondering what people would actually expect. I would of course also prefer if we would have less special handling for those dreaded "uefi-vars" assets.

But let's ask ybonatakis what would you prefer: For the clone script to handle that in a special way to not fail on such non-existing assets or just for the webUI to not show it (or show it but not as downloadable link)?

#10 Updated by mkittler about 1 month ago

With this PR non-existing assets are distinguishable on the downloads tab: https://github.com/os-autoinst/openQA/pull/4212

#11 Updated by okurz 25 days ago

  • Related to action #99672: [openqa][tools] Non-existing `…-uefi-vars.qcow2` asset disturbs openqa-clone-job workflow added

#12 Updated by mkittler 22 days ago

This PR improves the clone-job script: https://github.com/os-autoinst/openQA/pull/4261

Together with the previous PR this ticket should be covered. (It is still not quite clear what's wanted exactly.)

#13 Updated by mkittler 21 days ago

  • Status changed from New to Feedback
  • Assignee set to mkittler

The suggestions in this ticket are very wrong. The cleanup did not delete anything because the asset hasn't been created in the first place. The clone-job script can deal with this now since https://github.com/os-autoinst/openQA/pull/4261 has been merged and the UI shows the asset as non-existent.

I would say that's good enough so I'm waiting for feedback.

#14 Updated by mkittler 16 days ago

  • Status changed from Feedback to Resolved

Not feedback from the author for a month so I'm considering it resolved.

Also available in: Atom PDF