Project

General

Profile

action #70723

Fix tests not to rely on `/var/lib/openqa/share` mountpoint

Added by mkittler 11 months ago. Updated 11 months ago.

Status:
Resolved
Priority:
Low
Assignee:
Category:
Feature requests
Target version:
Start date:
2020-08-31
Due date:
% Done:

0%

Estimated time:
Difficulty:
Tags:

Description

Back story

Apparently tests within os-autoinst-distri-opensuse are no longer supposed to rely on the mountpoint /var/lib/openqa/share on the worker host.

I have fixed #69475 by restoring the mount point on the o3 worker openqaworker7. However, that means there are still tests relying on that mount point. A brief grep shows the following places:

[… opensuse]$ [(e7cc2d56b...)] git --no-pager grep -P '(autoinst_url.*assets|data_url.*(REPO_|ASSET_))'
lib/bootloader_setup.pm:        $autoyast_args .= autoinst_url("/assets/other/$ay_var");
tests/kernel/change_kernel.pm:        from_rpm(autoinst_url("/assets/other/$rpm"));
tests/virtualization/vagrant/boxes/tumbleweed.pm:        assert_script_run("wget --quiet " . autoinst_url("/assets/other/$boxname"));
tests/wsl/prepare_wsl_feature.pm:        cmd     => 'Invoke-WebRequest -Uri ' . autoinst_url("/assets/other/$wsl_appx_filename") . ' -O C:\\' . $wsl_appx_filename . ' -UseBasicParsing',

The first occurrence is for a YaST profile so it seems to be responsible for the failure from #69475.


In the first place I wanted to file this ticket under the openQA Tests project. However, I'm not sure whether the distribution can easily move away from the current approach after looking into the occurrences.

The asset variables, e.g. ASSET_7 as used for the YaST profile in the first occurrence already contain the full path to the asset and if asset caching is enabled that would be somewhere within the pool directory, e.g. /var/lib/openqa/pool/6/01348026-autoinst.xml in this case. So with the asset cache enabled one could simply use the path the asset variable refers to without relying on the mountpoint. However, here the file is consumed within the SUT and so far this works via the command server which always looks under ASSETDIR (which is here /var/lib/openqa/share/factory).
The other occurrences look the same.

Acceptance criteria

  • AC1: o3 and osd workers do not have /var/lib/openqa/share mounted from webui hosts

Suggestions

  • Understand what existing tests want to achieve with data directly accessed from /var/lib/openqa/share
  • Maybe it would make sense to just make the command server a little bit smarter to look into the pool directory as well? That shouldn't be a big deal from our side and would save the test writers some effort.
  • When all occurrences have been fixed (be it on the distribution side or on the os-autoinst side) the tools team should disable the mount point on all OSD and o3 workers consistently to avoid tests relying on the mountpoint again.

Related issues

Related to openQA Infrastructure - action #69475: [tools] openQA child task fails to download asset created by parent jobResolved2020-07-31

History

#1 Updated by mkittler 11 months ago

  • Related to action #69475: [tools] openQA child task fails to download asset created by parent job added

#2 Updated by okurz 11 months ago

  • Project changed from openQA Project to openQA Tests
  • Subject changed from Fix tests not to rely on `/var/lib/openqa/share` mountpoint to [y] Fix tests not to rely on `/var/lib/openqa/share` mountpoint
  • Category changed from Feature requests to Bugs in existing tests

hm, that's only a problem within os-autoinst-distri-opensuse and not a feature for openQA in my understanding. I thought some months ago we had that fixed. The commit

commit 8ea13b8c3
Author: Rodion Iafarov <riafarov@suse.com>
Date:   Wed Apr 8 19:39:41 2020 +0200

    Fix assets handling for AY tests without modifications

    We have fixed problem with assets for the jobs where we inject something
    to the profile, but still have issue when we simply use profile from the
    parent job.

    See [poo#64961](https://progress.opensuse.org/issues/64961).

within os-autoinst-distri-opensuse brought back the reference to the local path in 2020-04

#3 Updated by okurz 11 months ago

  • Project changed from openQA Tests to openQA Project
  • Subject changed from [y] Fix tests not to rely on `/var/lib/openqa/share` mountpoint to Fix tests not to rely on `/var/lib/openqa/share` mountpoint
  • Category changed from Bugs in existing tests to Feature requests

ok, sorry. I overlooked the explanation regarding the extension for os-autoinst

#4 Updated by okurz 11 months ago

  • Tags set to nfs, share
  • Description updated (diff)
  • Status changed from New to Workable
  • Priority changed from Normal to Low
  • Target version set to Ready

#5 Updated by mkittler 11 months ago

  • Assignee set to mkittler

#6 Updated by mkittler 11 months ago

  • Status changed from Workable to In Progress

#7 Updated by okurz 11 months ago

  • Status changed from In Progress to Feedback

PR merged. I guess you don't have additional changes planned here

#8 Updated by mkittler 11 months ago

  • Status changed from Feedback to Resolved

It seems to work: https://openqa.opensuse.org/tests/1388838

And yes, the NFS mount is actually present on openqaworker7 but judging by the log the asset is nevertheless used from the pool directory:

[2020-09-09T12:23:28.628 CEST] [info] Asset requested: 01388835-autoinst.xml
[2020-09-09T12:23:28.630 CEST] [debug] tests/autoyast/installation.pm:132 called autoyast::test_ayp_url -> lib/autoyast.pm:522 called testapi::record_info
[2020-09-09T12:23:28.631 CEST] [debug] <<< testapi::record_info(title="ayp url ok", output="Autoyast profile url http://10.0.2.2:20043/eEeQF7w4v4oNXe1J/assets/other/01388835-autoinst.xml is reachable", result="ok")

(Otherwise Asset requested: … would show the full path of the asset under /var/lib/openqa/share/.)

Also available in: Atom PDF