action #70723
closedFix tests not to rely on `/var/lib/openqa/share` mountpoint
0%
Description
Back story¶
Apparently tests within os-autoinst-distri-opensuse
are no longer supposed to rely on the mountpoint /var/lib/openqa/share
on the worker host.
I have fixed #69475 by restoring the mount point on the o3 worker openqaworker7
. However, that means there are still tests relying on that mount point. A brief grep
shows the following places:
[… opensuse]$ [(e7cc2d56b...)] git --no-pager grep -P '(autoinst_url.*assets|data_url.*(REPO_|ASSET_))'
lib/bootloader_setup.pm: $autoyast_args .= autoinst_url("/assets/other/$ay_var");
tests/kernel/change_kernel.pm: from_rpm(autoinst_url("/assets/other/$rpm"));
tests/virtualization/vagrant/boxes/tumbleweed.pm: assert_script_run("wget --quiet " . autoinst_url("/assets/other/$boxname"));
tests/wsl/prepare_wsl_feature.pm: cmd => 'Invoke-WebRequest -Uri ' . autoinst_url("/assets/other/$wsl_appx_filename") . ' -O C:\\' . $wsl_appx_filename . ' -UseBasicParsing',
The first occurrence is for a YaST profile so it seems to be responsible for the failure from #69475.
In the first place I wanted to file this ticket under the openQA Tests project. However, I'm not sure whether the distribution can easily move away from the current approach after looking into the occurrences.
The asset variables, e.g. ASSET_7
as used for the YaST profile in the first occurrence already contain the full path to the asset and if asset caching is enabled that would be somewhere within the pool directory, e.g. /var/lib/openqa/pool/6/01348026-autoinst.xml
in this case. So with the asset cache enabled one could simply use the path the asset variable refers to without relying on the mountpoint. However, here the file is consumed within the SUT and so far this works via the command server which always looks under ASSETDIR
(which is here /var/lib/openqa/share/factory
).
The other occurrences look the same.
Acceptance criteria¶
- AC1: o3 and osd workers do not have /var/lib/openqa/share mounted from webui hosts
Suggestions¶
- Understand what existing tests want to achieve with data directly accessed from /var/lib/openqa/share
- Maybe it would make sense to just make the command server a little bit smarter to look into the pool directory as well? That shouldn't be a big deal from our side and would save the test writers some effort.
- When all occurrences have been fixed (be it on the distribution side or on the os-autoinst side) the tools team should disable the mount point on all OSD and o3 workers consistently to avoid tests relying on the mountpoint again.
Updated by mkittler over 4 years ago
- Related to action #69475: [tools] openQA child task fails to download asset created by parent job added
Updated by okurz over 4 years ago
- Project changed from openQA Project (public) to openQA Tests (public)
- Subject changed from Fix tests not to rely on `/var/lib/openqa/share` mountpoint to [y] Fix tests not to rely on `/var/lib/openqa/share` mountpoint
- Category changed from Feature requests to Bugs in existing tests
hm, that's only a problem within os-autoinst-distri-opensuse and not a feature for openQA in my understanding. I thought some months ago we had that fixed. The commit
commit 8ea13b8c3
Author: Rodion Iafarov <riafarov@suse.com>
Date: Wed Apr 8 19:39:41 2020 +0200
Fix assets handling for AY tests without modifications
We have fixed problem with assets for the jobs where we inject something
to the profile, but still have issue when we simply use profile from the
parent job.
See [poo#64961](https://progress.opensuse.org/issues/64961).
within os-autoinst-distri-opensuse brought back the reference to the local path in 2020-04
Updated by okurz over 4 years ago
- Project changed from openQA Tests (public) to openQA Project (public)
- Subject changed from [y] Fix tests not to rely on `/var/lib/openqa/share` mountpoint to Fix tests not to rely on `/var/lib/openqa/share` mountpoint
- Category changed from Bugs in existing tests to Feature requests
ok, sorry. I overlooked the explanation regarding the extension for os-autoinst
Updated by okurz over 4 years ago
- Tags set to nfs, share
- Description updated (diff)
- Status changed from New to Workable
- Priority changed from Normal to Low
- Target version set to Ready
Updated by mkittler over 4 years ago
- Status changed from Workable to In Progress
Updated by okurz over 4 years ago
- Status changed from In Progress to Feedback
PR merged. I guess you don't have additional changes planned here
Updated by mkittler over 4 years ago
- Status changed from Feedback to Resolved
It seems to work: https://openqa.opensuse.org/tests/1388838
And yes, the NFS mount is actually present on openqaworker7
but judging by the log the asset is nevertheless used from the pool directory:
[2020-09-09T12:23:28.628 CEST] [info] Asset requested: 01388835-autoinst.xml
[2020-09-09T12:23:28.630 CEST] [debug] tests/autoyast/installation.pm:132 called autoyast::test_ayp_url -> lib/autoyast.pm:522 called testapi::record_info
[2020-09-09T12:23:28.631 CEST] [debug] <<< testapi::record_info(title="ayp url ok", output="Autoyast profile url http://10.0.2.2:20043/eEeQF7w4v4oNXe1J/assets/other/01388835-autoinst.xml is reachable", result="ok")
(Otherwise Asset requested: …
would show the full path of the asset under /var/lib/openqa/share/
.)