Project

General

Profile

Actions

action #17926

closed

[tools]Tumbleweed upgrade tests have download.o.o repos enabled

Added by dimstar almost 7 years ago. Updated almost 7 years ago.

Status:
Rejected
Priority:
High
Assignee:
-
Category:
Bugs in existing tests
Target version:
-
Start date:
2017-03-23
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Follow up to bug https://bugzilla.opensuse.org/show_bug.cgi?id=1030759

(I thought I filed this already, but can't find my ticket - in any case I already TALKED about this a hundred times; and was just bitten by it again)

The problem:
whlie testing upgrade scenarios from e.g. 42.2 to TW-Next (the version being tested in openQA), the CURRENTLY published TW on download.o.o is ALSO taken into account (online repo enabled)

In case of the boo#1030759 this results in the updater NOT tagging libmutter0 is an obsolete package (it is tagged 'weakremove', which means it is ONLY marked for deletion if it is NOT found in any of the enabled repos. As TW (published) still contains the package, it is not removed - furhter blocking the update of gnome-shell * mutter, which leads to the test failures as seen in e.g. https://openqa.opensuse.org/tests/375730

We had seen similar test issues in the past, be it that stuff 'works' during the openQA test until TW-Next is published (then packages disappear, and tests would no longer pass)
In any case, having the repo published at download.o.o available to the tests is defeating openQA as it is intended totally, as many tests are not reliable this way (testing a repo combination that no user will possibly be able to have on his system)


Related issues 3 (0 open3 closed)

Related to openQA Tests - action #18070: [opensuse][leap][kernel][functional][tools] test fails in crash because it can not find the debuginfo packageResolvedmichalnowak2017-03-28

Actions
Is duplicate of openQA Tests - coordination #17436: [tools][functional][y][epic] tests must not have access to published repos, e.g. download.o.o on Tumbleweed -> remove all references to download.o.o from repos during installation and afterwardsResolvedriafarov2018-02-122019-02-26

Actions
Blocks openQA Tests - action #12280: [opensuse][tw] Tumbleweed Test Suite gets packages from 'last published' oss repoResolvedokurz2016-06-09

Actions
Actions #1

Updated by dimstar almost 7 years ago

Actions #2

Updated by dimstar almost 7 years ago

https://openqa.opensuse.org/tests/375807#step/start_install/18 is an example from the 'other direction of the same problem'

With snapshot 0320, libmarco-private0 was dropped (replaced by libmarco-private1); when 0320 was being tested though, the previously published snapshot was available and thus this did not show an error in its test run at https://openqa.opensuse.org/tests/374330

As soon as snapshot 0320 has been published, the available package list changed and the test is thus invalid (and fails with one day delay)

Actions #3

Updated by dimstar almost 7 years ago

And just to confirm I'm not making this up: as 0322 has now been published, I reran the 42.2-> tw update and it passes

before publish: https://openqa.opensuse.org/tests/375730 vs after publish and rerun: https://openqa.opensuse.org/tests/376303#

Actions #4

Updated by RBrownSUSE almost 7 years ago

  • Subject changed from Tumbleweed upgrade tests have download.o.o repos enabled to [tools]Tumbleweed upgrade tests have download.o.o repos enabled
  • Priority changed from Normal to High
Actions #5

Updated by okurz almost 7 years ago

  • Category set to Bugs in existing tests
Actions #6

Updated by okurz almost 7 years ago

@rbrown: I do not consider this high as it does not fail that often.

Actions #7

Updated by dimstar almost 7 years ago

I would even consider this hyper-critical.. fail or not fali is irrelevant: the tests are invalid as they test stuff the user will not have at hand in the end.

Actions #8

Updated by RBrownSUSE almost 7 years ago

  • Priority changed from High to Urgent

okurz wrote:

@rbrown: I do not consider this high as it does not fail that often.

Your opinion is noted

Actions #9

Updated by okurz@suse.de almost 7 years ago

Do you realize that I meant so decrease prio?

Actions #10

Updated by okurz@suse.de almost 7 years ago

Ah, didn't read message by dimstar ;-)

Actions #11

Updated by okurz almost 7 years ago

  • Related to action #18070: [opensuse][leap][kernel][functional][tools] test fails in crash because it can not find the debuginfo package added
Actions #12

Updated by okurz almost 7 years ago

  • Blocks action #12280: [opensuse][tw] Tumbleweed Test Suite gets packages from 'last published' oss repo added
Actions #14

Updated by RBrownSUSE almost 7 years ago

  • Priority changed from Urgent to High
Actions #15

Updated by okurz almost 7 years ago

  • Is duplicate of coordination #17436: [tools][functional][y][epic] tests must not have access to published repos, e.g. download.o.o on Tumbleweed -> remove all references to download.o.o from repos during installation and afterwards added
Actions #16

Updated by okurz almost 7 years ago

  • Status changed from New to Rejected

all content incorporated into #17436

Actions

Also available in: Atom PDF