[opensuse][functional][u] post-published Tumbleweed tests
|Target version:||QA - future|
Ensure we test like the user does with published online repos
- Create new job group
- Trigger published online media after publishing
- Adapt and add test "update_leap42.2->tw"
original discussion in #os-fctry
[9 Feb 2017 11:28:52] <DimStar> okurz: I had a look at all the gimp failures we have in openQA for snapshot 0208 [9 Feb 2017 11:29:07] <DimStar> e.g. https://openqa.opensuse.org/tests/351292#step/gimp/11 [9 Feb 2017 11:29:45] <DimStar> the problem there is that we actually seem to have an invalid test setup: the already published TW repo is enabled on this mchine; so 'old' packages bleed into what is going to be tested [9 Feb 2017 11:30:09] <DimStar> *ni this case: gimp-help-browser no longer exists in 0208 - but it finds it on the published repo) [9 Feb 2017 11:31:59] <DimStar> https://openqa.opensuse.org/tests/351292#step/zypper_lr/5 confirms that the published repos are enabled [9 Feb 2017 11:31:59] <okurz> do you have an idea how to handle this? [9 Feb 2017 11:32:45] <DimStar> I thought we had a call to remove the published repos - but we might not run this in every case needed [9 Feb 2017 11:33:31] <DimStar> ./tests/update/zypper_clear_repos.pm - seems we only care enough when we test for updates [9 Feb 2017 11:33:44] <okurz> DimStar: it would not harm to have post-publication tests which test that we can update from leap->last published tumbleweed. for that the online repos make sense [9 Feb 2017 11:34:53] <DimStar> the entire TW test suite is to test anything to the FUTURE TW snapshot; what is already published is either working (as it passed before it was published) or broken (only a future snapshot could fix it) [9 Feb 2017 11:35:39] <okurz> DimStar: yes, I think we can have *both*, leap->tw-released and leap->tw-in-test [9 Feb 2017 11:35:55] <okurz> I understand that hardly any user would have a more up-to-date dvd repo than online repo but shouldn't zypper+rpm still be able to handle this? [9 Feb 2017 11:36:09] <DimStar> if you want a leap->tw-released, fine... but not in the same job group [9 Feb 2017 11:37:34] <DimStar> by having the online repos available we also mask potential dependency issues for stuff that is missing in the new repo [9 Feb 2017 11:37:49] <DimStar> believe me - it's really really not what we want to test