coordination #17436
Updated by okurz about 7 years ago
## observation
see for example https://bugzilla.suse.com/show_bug.cgi?id=1026767. This bug was just "found" by openQA when also users reported this. The reason is probably that the tests still have published repos from download.opensuse.org enabled causing an upgrade to find packages from either the new snapshot or the old published data which can make tests pass when they shouldn't. Then the tests fail in the subsequent snapshot because the repos were updated in the meantime.
## suggestions
* check test cases for the execution of tests/update/zypper_clear_repos.pm
## further details
proposal by nussel in #17818
On openSUSE the 'crash' test doesn't work as the debuginfo repo is not mirrored to openqa. So whenever the kernel gets updated that test fails due to a mismatch between the installed kernel and the debug information in the debuginfo repo.
openQA should keep a local copy of all repos and intercept access to download.opensuse.org so files are downloaded from matching repos actually.
from #17926
Follow up to bug https://bugzilla.opensuse.org/show_bug.cgi?id=1030759
The problem:
whlie testing upgrade scenarios from e.g. 42.2 to TW-Next (the version being tested in openQA), the CURRENTLY published TW on download.o.o is ALSO taken into account (online repo enabled)
In case of the boo#1030759 this results in the updater NOT tagging libmutter0 is an obsolete package (it is tagged 'weakremove', which means it is ONLY marked for deletion if it is NOT found in any of the enabled repos. As TW (published) still contains the package, it is not removed - furhter blocking the update of gnome-shell * mutter, which leads to the test failures as seen in e.g. https://openqa.opensuse.org/tests/375730
We had seen similar test issues in the past, be it that stuff 'works' during the openQA test until TW-Next is published (then packages disappear, and tests would no longer pass)
In any case, having the repo published at download.o.o available to the tests is defeating openQA as it is intended totally, as many tests are not reliable this way (testing a repo combination that no user will possibly be able to have on his system)
https://openqa.opensuse.org/tests/375730#step/upgrade_select_opensuse/2 Shows that the repos are in fact enabled. https://openqa.opensuse.org/tests/375807#step/start_install/18 is an example from the 'other direction of the same problem'. With snapshot 0320, libmarco-private0 was dropped (replaced by libmarco-private1); when 0320 was being tested though, the previously published snapshot was available and thus this did not show an error in its test run at https://openqa.opensuse.org/tests/374330. As soon as snapshot 0320 has been published, the available package list changed and the test is thus invalid (and fails with one day delay). After 0322 was published, reran the 42.2-> tw update passed. before publish: https://openqa.opensuse.org/tests/375730 vs after publish and rerun: https://openqa.opensuse.org/tests/376303#