QA » openQA Project » openQA Tests

discussed in openQA coordination call 2017-03-03. A good approach might be an "proxy-download.o.o" which can intercept all downloads. We can disable some online repos as we sync DVD+oss and have these available as repos during testing but maybe we won't have all packages available. Hmm.

Moving to correct project and tagging

This is not something which can only be implemented into the tests, will need enablement on the backend side to make it work

rbrown, I don't see the relation to backend. There might be repositories being injected by rsync.pl - not part of openQA project - but the main problem is very likely in the tests distribution, e.g. upgrade of installer adds opensuse oss repo. If you want to have a '[tools]' tag, fine. But it has nothing to do with either os-autoinst nor openQA.

Incorporated content from duplicate #17926

Ettore will be looking at this also to fix this issue from the backend side of things

as discussed, if possible it would be good if the "network isolation" can be controlled by a variable to switch on/off. If that is not possible or too much effort a "hack" would be to rewrite all occurences of "download.opensuse.org" to the according path pointing to openqa.opensuse.org, e.g. with a test module that calls sed -i -e 's/download.opensuse.org/…/g' /etc/zypp/repos.d/* in the installed system and the equivalent in the installation system or not add any online repos there if possible

Did further investigations, after chrooting on the system that has to be upgraded (always refeering to the same test) i tried to manually add the repository d.o.o (which redirects to openqa mirrored one) and run zypper to install some packages.

The good news is that the the approach seems to work, since i can fetch the repo for metadata and having it installed in the system; on the other side the bad news is we have even more conflicts than with having the d.o.o. repository enabled without redirection, making impossible to install a simple package or upgrade the whole system.
At this point one idea could be to serve an empty repository instead, or having an empty repository mirrored somewhere and redirect the request there, but i see it that it's getting more as a big hack from the backend side (althought it would probably solve the YaST2 UI freezes).

I'm attaching few screenshots how dependency calculation is "stuck" with almost no viable solution in most cases.

os-autoinst logs shows anyway that redirection has been handled as expected:

15:02:41.6499 15940 >> DNS Server: Intercepting request for download.opensuse.org
15:02:41.6501 15940 >> DNS Server: Answer download.opensuse.org.    IN  A   10.0.2.254
15:02:41.6504 15940 >> DNS Server: Intercepting request for download.opensuse.org
15:02:41.6506 15940 >> DNS Server: Answer download.opensuse.org.    IN  A   10.0.2.254
15:02:41.6518 15941 >> backend::component::proxy Request from: 127.0.0.1 method: GET to host:download.opensuse.org
15:02:41.6519 15941 >> backend::component::proxy Requested url is: /tumbleweed/repo/oss/repodata/repomd.xml.asc
15:02:41.6522 15941 >> backend::component::proxy Redirecting to: http://openqa.opensuse.org/assets/repo/openSUSE-Tumbleweed-oss-i586-x86_64-Snapshot20170610/suse/repodata/repomd.xml.asc
15:02:41.7104 15941 >> backend::component::proxy Request from: 127.0.0.1 method: GET to host:download.opensuse.org
15:02:41.7105 15941 >> backend::component::proxy Requested url is: /tumbleweed/repo/oss/repodata/repomd.xml.key
15:02:41.7111 15941 >> backend::component::proxy Redirecting to: http://openqa.opensuse.org/assets/repo/openSUSE-Tumbleweed-oss-i586-x86_64-Snapshot20170610/suse/repodata/repomd.xml.key
15:02:41.7548 15941 >> backend::component::proxy Request from: 127.0.0.1 method: GET to host:download.opensuse.org
15:02:41.7551 15941 >> backend::component::proxy Requested url is: /tumbleweed/repo/oss/repodata/repomd.xml
15:02:41.7565 15941 >> backend::component::proxy Redirecting to: http://openqa.opensuse.org/assets/repo/openSUSE-Tumbleweed-oss-i586-x86_64-Snapshot20170610/suse/repodata/repomd.xml
15:02:41.8295 15941 >> backend::component::proxy Request from: 127.0.0.1 method: GET to host:download.opensuse.org
15:02:41.8296 15941 >> backend::component::proxy Requested url is: /tumbleweed/repo/oss/repodata/fe04c70d375f6737d3df02ab1da0224aa1b567c52412382ffc8d74f12b409554-primary.xml.gz
15:02:41.8299 15941 >> backend::component::proxy Redirecting to: http://openqa.opensuse.org/assets/repo/openSUSE-Tumbleweed-oss-i586-x86_64-Snapshot20170610/suse/repodata/fe04c70d375f6737d3df02ab1da0224aa1b567c52412382ffc8d74f12b409554-primary.xml.gz

As discussed in the retrospective call, the solution that could be provided from the backend side it's too invasive and will require changes in the tests side as well.
I'm removing myself as assigned since this now should be solved by maintainers of the different test suites that are affected by this, even if we propose a common way to setup proxy/dns for mangling the connections, the implementation details would change based on the context (e.g. while doing migration SUSE 12 to Leap there is no iptables available on Live CD environment and so on) and i'm sure test maintainers have more deep introspection depending on the circumstances (generally could be solved by just replacing zypper repositories urls, but won't cover all the cases).

De-prioritising and retagging

After discussion in Tools group, the consensus is we'd would prefer to see this issue resolved in tests, not necessarily with additional tooling

updated with acceptance criteria, tasks and effort estimation

oops, should have gone to later sprint. Will discuss with PO.

Copied text from the PR https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/4087:
For openSUSE it's hard to replace repos which we use. In most of the
scenarios it works fine as we don't install packages which are not
available on DVD or mirror in case of net install. Whereas it causes
issues in some scenarios when we install additional software in the
tests.
There are 2 parts for the solution, one is to modify repositories in the
installer, which is possible only by tweaking control.xml before we
trigger installation, or by at least disabling online repos which point
to download.opensuse.org. It's easier to handle scenario with installed
system and we already have test modules which do this task. As of now we
enable this behavior with feature variable and for test suites which
fail due to this particular issue. After we see that it works fine, we
can apply it as default behavior and exclude some scenarios if needed.

• Related ticket: poo#17436.
• Needles: PR#300.
• Verification runs:
  • TW textmode missing MyODBC-unixODBC package in the mirror repo
  • TW btrfs x11
  • TW extra tests on gnome
  • leap 15 create hdd gnome x11
  • leap 15 text mode x11

Verification runs already have revealed that we may face issues with missing packages, so we should decide if we want to sync non-oss repo as well. As of now MyODBC-unixODBC is dropped. Code base wasn't updated in that run.
For AC2 we can only modify control.xml after using startshell boot option (a little easier with live installers). So I guess we should define scope where to apply solution first. Point about cloning jobs is still valid as would need manual adjustments if job is cloned.

After collecting feedback we can move forward in direction of what is required here.

zypper info tries to access source repo which is not available http://g226.suse.de/tests/129#step/zypper_info/6 , this was fixed by running zypper_info test before removing repos. In the scenario where we disable online repos during installation, this shouldn't harm.
After changes are merged, we can perform some runs with DISABLE_ONLINE_REPOS set to 1.

Test runs look good as of now with disabled repos. @okurz, what should be our next steps then? Especially considering the fact that some tests require published repos (dup). And AC2 is not achievable in described way without hacking control.xml file.

@okurz will review progress and decide on next steps

I will review.

sorry I missed to opportunity to discuss this more thoroughly before my vacation. I guess we should revisit this opportunity in about two weeks time.

Clearly we have not reached any of the two ACs and I understood that simply flipping the switch will break some tests.

So the online repos are disabled during installation in some test cases and we can see that live in e.g. https://openqa.opensuse.org/tests/605728#step/disable_online_repos/9
but the downstream job https://openqa.opensuse.org/tests/605816#step/zypper_lr/3 shows that the online repos are there in the installed system. So what did "disabling the online repos during installation" actually do? Only disable the use of these repos during installation? I think this is good and a step in the right direction. Now in this scenario if we would disable the online repos of course stuff would be missing trying to install e.g. yast modules but the job has the variable MIRROR_HTTP which was so far only used by debuginfo modules, see lib/kdump_utils.pm. So I assume we do not need to disable the online repos but rather replace them.

Could you try that - or argue against ;)

Converting to the epic. I'll take a look why repos are not disabled. But as mentioned, for those kind of scenarios, we can enable another test module which does it and we already have it. For AC2 the only thing is to disable online repos, or to modify control file before we start installation (which we decided not to do). I'll create a task to enable switch where required and check if we have a bug in the installer to disable online repos.

Ok, then a thing to review later. We can schedule the subtask before end of M16.

due to changes in a related task

due to changes in a related task

due to changes in a related task

due to changes in a related task

Even though it's about "zypper_info" and such the subticket has been picked by the [y] team as well as this one so let's assign to [y] :)

due to changes in a related task

agreed, let's mark as "Blocked by subtask" then, ok?

due to changes in a related task

due to changes in a related task

due to changes in a related task

Finally we can resolve this one. I will create separate epic to be more efficient with assets, especially for SLE and sync only used and tested packages.