Project

General

Profile

Actions

action #151522

closed

openqa-clone-job succeeds even when assets are missing size:M

Added by josegomezr 3 months ago. Updated 3 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Regressions/Crashes
Target version:
Start date:
2023-11-27
Due date:
2023-12-15
% Done:

0%

Estimated time:

Description

Observation

OpenQA UI shows:

Result: incomplete, finished 3 minutes ago (0)
Reason: asset failure: Cannot find HDD_1 asset hdd/opensuse-Tumbleweed-x86_64-20231027-textmode@64bit.qcow2!
Scheduled product: job has not been created by posting an ISO 

openqa-clone-job says:

openqa-instance:/var/lib/openqa/share/tests/opensuse # openqa-clone-job --skip-deps --from https://openqa.opensuse.org/tests/3682089 SYS_PARAM_CHECK_TEST=https://github.com/josegomezr/sys-param-check
Cloning parents of opensuse-Tumbleweed-DVD-x86_64-Build20231027-sys-param-check@64bit
downloading
https://openqa.opensuse.org/tests/3682089/asset/other/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20231027-Media.iso.sha256
to
/var/lib/openqa/share/factory/other/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20231027-Media.iso.sha256
1 job has been created:
 - opensuse-Tumbleweed-DVD-x86_64-Build20231027-sys-param-check@64bit -> http://localhost/tests/638

I did not pass --ignore-missing-assets to the invocation, yet it did not fail.

Acceptance criteria

  • AC1: Missing assets are reported as early as possible (ideally by openqa-clone-job, that likely already works and this likely just comes down to AC2)
  • AC2: When using --skip-deps to skip chained parents the HDD image is attempted to be downloaded (like when --skip-chained-deps is used)

Suggestions

  • Find out if openqa-clone-job even knows about that assets (e.g. if they got cleaned up in the meantime)
  • Have a look how --ignore-missing-assets is implemented and why the clone command didn't fail even without passing this parameter
  • Maybe improve documentation for the various --skip options and mention asset download behavior

Files

Actions #1

Updated by dheidler 3 months ago

  • Description updated (diff)
Actions #2

Updated by dheidler 3 months ago

  • Subject changed from openqa-clone-job succeeds even when assets are missing to openqa-clone-job succeeds even when assets are missing size:M
  • Status changed from New to Workable
Actions #3

Updated by mkittler 3 months ago

  • Status changed from Workable to In Progress
  • Assignee set to mkittler
Actions #4

Updated by openqa_review 3 months ago

  • Due date set to 2023-12-15

Setting due date based on mean cycle time of SUSE QE Tools

Actions #5

Updated by mkittler 3 months ago · Edited

After having a closer look the problem looks different than what we have assumed during the estimation. It looks like --skip-deps and --skip-chained-deps would behave identical.

The problem here is that openqa-clone-job simply does not even try to download these missing assets because they are not even present anymore in the web UI at all and openqa-clone-job relies on the list of assets provided by the web UI (as it does not try to parse any asset-related job settings anymore).

For this job the list of assets is:

openqa-cli api --pretty --o3 jobs/3682089
{
   "job" : {
      "assets" : {
         "other" : [
            "openSUSE-Tumbleweed-DVD-x86_64-Snapshot20231027-Media.iso.sha256"
         ]
      },
…

So that's all openqa-clone-job sees and attempts to download for this job. Just for comparison, that's how it looks like for the most recent job:

openqa-cli api --pretty --o3 jobs/3773913
{
   "job" : {
      "assets" : {
         "hdd" : [
            "opensuse-Tumbleweed-x86_64-20231130-textmode@64bit.qcow2"
         ],
         "iso" : [
            "openSUSE-Tumbleweed-DVD-x86_64-Snapshot20231130-Media.iso"
         ],
         "other" : [
            "Tumbleweed.x86_64-1.0-libvirt-Snapshot20231130.vagrant.libvirt.box",
            "Tumbleweed.x86_64-1.0-virtualbox-Snapshot20231130.vagrant.virtualbox.box",
            "openSUSE-Tumbleweed-DVD-x86_64-Snapshot20231130-Media.iso.sha256"
         ],
         "repo" : [
            "openSUSE-Tumbleweed-non-oss-x86_64-Snapshot20231130",
            "openSUSE-Tumbleweed-oss-x86_64-Snapshot20231130",
            "openSUSE-Tumbleweed-oss-x86_64-Snapshot20231130-debuginfo",
            "openSUSE-Tumbleweed-oss-x86_64-Snapshot20231130-source",
            "openSUSE-Tumbleweed-oss-x86_64.license-Snapshot20231130"
         ]
      },
…

When downloading the latest job with --skip-deps (or --skip-chained-deps, that makes not difference after all) it downloads the ISO (because it is specified as asset even on the chained child job) but not the HDD which seems wrong (considering we are skipping the parent that would create it). So that is yet another problem. I confused job IDs. When cloning the latest job (currently 3773913) it downloads the HDD as well as expected. The HDD is only skipped if --skip-deps or --skip-chained-deps is not present (then it makes sense). So everything's good in that regard.


That means AC2 is already fulfilled and AC1 is the problematic one after all.

Actions #6

Updated by mkittler 3 months ago · Edited

When trying to restart https://openqa.opensuse.org/tests/3682089 via the web UI it complains with

Job 3682089 misses the following mandatory assets: hdd/opensuse-Tumbleweed-x86_64-20231027-textmode@64bit.qcow2, iso/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20231027-Media.iso, other/Tumbleweed.x86_64-1.0-libvirt-Snapshot20231027.vagrant.libvirt.box, other/Tumbleweed.x86_64-1.0-virtualbox-Snapshot20231027.vagrant.virtualbox.box
You may try to retrigger the parent job that should create the assets and will implicitly retrigger this job as well.

as expected. This works because the web UI really re-evaluates the job settings and then checks for the presence of assets. Maybe we need to do the same within openqa-clone-job so fulfill AC1. The openqa-clone-job script can of course not do this kind of check as it doesn't usually run on the web UI host. So it needed to ask the web UI to do the checking.

Actions #7

Updated by mkittler 3 months ago

  • Status changed from In Progress to Feedback

PR implementing a check for missing assets as mentioned in my previous comment: https://github.com/os-autoinst/openQA/pull/5383

Actions #8

Updated by mkittler 3 months ago

  • Status changed from Feedback to Resolved

The PR has been merged.

Actions

Also available in: Atom PDF