action #17926
closed[tools]Tumbleweed upgrade tests have download.o.o repos enabled
0%
Description
Follow up to bug https://bugzilla.opensuse.org/show_bug.cgi?id=1030759
(I thought I filed this already, but can't find my ticket - in any case I already TALKED about this a hundred times; and was just bitten by it again)
The problem:
whlie testing upgrade scenarios from e.g. 42.2 to TW-Next (the version being tested in openQA), the CURRENTLY published TW on download.o.o is ALSO taken into account (online repo enabled)
In case of the boo#1030759 this results in the updater NOT tagging libmutter0 is an obsolete package (it is tagged 'weakremove', which means it is ONLY marked for deletion if it is NOT found in any of the enabled repos. As TW (published) still contains the package, it is not removed - furhter blocking the update of gnome-shell * mutter, which leads to the test failures as seen in e.g. https://openqa.opensuse.org/tests/375730
We had seen similar test issues in the past, be it that stuff 'works' during the openQA test until TW-Next is published (then packages disappear, and tests would no longer pass)
In any case, having the repo published at download.o.o available to the tests is defeating openQA as it is intended totally, as many tests are not reliable this way (testing a repo combination that no user will possibly be able to have on his system)
Updated by dimstar about 7 years ago
https://openqa.opensuse.org/tests/375730#step/upgrade_select_opensuse/2
Shows that the repos are in fact enabled
Updated by dimstar about 7 years ago
https://openqa.opensuse.org/tests/375807#step/start_install/18 is an example from the 'other direction of the same problem'
With snapshot 0320, libmarco-private0 was dropped (replaced by libmarco-private1); when 0320 was being tested though, the previously published snapshot was available and thus this did not show an error in its test run at https://openqa.opensuse.org/tests/374330
As soon as snapshot 0320 has been published, the available package list changed and the test is thus invalid (and fails with one day delay)
Updated by dimstar about 7 years ago
And just to confirm I'm not making this up: as 0322 has now been published, I reran the 42.2-> tw update and it passes
before publish: https://openqa.opensuse.org/tests/375730 vs after publish and rerun: https://openqa.opensuse.org/tests/376303#
Updated by RBrownSUSE about 7 years ago
- Subject changed from Tumbleweed upgrade tests have download.o.o repos enabled to [tools]Tumbleweed upgrade tests have download.o.o repos enabled
- Priority changed from Normal to High
Updated by okurz about 7 years ago
@rbrown: I do not consider this high as it does not fail that often.
Updated by dimstar about 7 years ago
I would even consider this hyper-critical.. fail or not fali is irrelevant: the tests are invalid as they test stuff the user will not have at hand in the end.
Updated by RBrownSUSE about 7 years ago
- Priority changed from High to Urgent
okurz wrote:
@rbrown: I do not consider this high as it does not fail that often.
Your opinion is noted
Updated by okurz@suse.de about 7 years ago
Do you realize that I meant so decrease prio?
Updated by okurz@suse.de about 7 years ago
Ah, didn't read message by dimstar ;-)
Updated by okurz about 7 years ago
- Related to action #18070: [opensuse][leap][kernel][functional][tools] test fails in crash because it can not find the debuginfo package added
Updated by okurz about 7 years ago
- Blocks action #12280: [opensuse][tw] Tumbleweed Test Suite gets packages from 'last published' oss repo added
Updated by okurz almost 7 years ago
- Is duplicate of coordination #17436: [tools][functional][y][epic] tests must not have access to published repos, e.g. download.o.o on Tumbleweed -> remove all references to download.o.o from repos during installation and afterwards added
Updated by okurz almost 7 years ago
- Status changed from New to Rejected
all content incorporated into #17436