action #16618

[opensuse][functional][u] post-published Tumbleweed tests

Added by okurz about 3 years ago. Updated over 1 year ago.

Status:NewStart date:09/02/2017
Priority:LowDue date:
Assignee:-% Done:

0%

Category:New test
Target version:QA - future
Difficulty:
Duration:

Description

goal

Ensure we test like the user does with published online repos

suggestions

  • Create new job group
  • Trigger published online media after publishing
  • Adapt and add test "update_leap42.2->tw"

further details

original discussion in #os-fctry

[9 Feb 2017 11:28:52] <DimStar> okurz: I had a look at all the gimp failures we have in openQA for snapshot 0208
[9 Feb 2017 11:29:07] <DimStar> e.g. https://openqa.opensuse.org/tests/351292#step/gimp/11
[9 Feb 2017 11:29:45] <DimStar> the problem there is that we actually seem to have an invalid test setup: the already published TW repo is enabled on this mchine; so 'old' packages bleed into what is going to be tested
[9 Feb 2017 11:30:09] <DimStar> *ni this case: gimp-help-browser no longer exists in 0208 - but it finds it on the published repo)
[9 Feb 2017 11:31:59] <DimStar> https://openqa.opensuse.org/tests/351292#step/zypper_lr/5 confirms that the published repos are enabled
[9 Feb 2017 11:31:59] <okurz> do you have an idea how to handle this?
[9 Feb 2017 11:32:45] <DimStar> I thought we had a call to remove the published repos - but we might not run this in every case needed
[9 Feb 2017 11:33:31] <DimStar> ./tests/update/zypper_clear_repos.pm - seems we only care enough when we test for updates
[9 Feb 2017 11:33:44] <okurz> DimStar: it would not harm to have post-publication tests which test that we can update from leap->last published tumbleweed. for that the online repos make sense
[9 Feb 2017 11:34:53] <DimStar> the entire TW test suite is to test anything to the FUTURE TW snapshot; what is already published is either working (as it passed before it was published) or broken (only a future snapshot could fix it)
[9 Feb 2017 11:35:39] <okurz> DimStar: yes, I think we can have *both*, leap->tw-released and leap->tw-in-test
[9 Feb 2017 11:35:55] <okurz> I understand that hardly any user would have a more up-to-date dvd repo than online repo but shouldn't zypper+rpm still be able to handle this?
[9 Feb 2017 11:36:09] <DimStar> if you want a leap->tw-released, fine... but not in the same job group
[9 Feb 2017 11:37:34] <DimStar> by having the online repos available we also mask potential dependency issues for stuff that is missing in the new repo
[9 Feb 2017 11:37:49] <DimStar> believe me - it's really really not what we want to test

Related issues

Copied to openQA Tests - action #17436: [tools][functional][y][epic] tests must not have access t... Resolved 12/02/2018 26/02/2019

History

#1 Updated by okurz almost 3 years ago

  • Copied to action #17436: [tools][functional][y][epic] tests must not have access to published repos, e.g. download.o.o on Tumbleweed -> remove all references to download.o.o from repos during installation and afterwards added

#2 Updated by okurz over 2 years ago

  • Subject changed from post-published Tumbleweed tests to [opensuse][functional]post-published Tumbleweed tests

#3 Updated by okurz almost 2 years ago

  • Subject changed from [opensuse][functional]post-published Tumbleweed tests to [opensuse][functional][u] post-published Tumbleweed tests

#4 Updated by okurz over 1 year ago

  • Target version changed from future to future

Also available in: Atom PDF