QA » openQA Project » openQA Tests

goal¶

Ensure we test like the user does with published online repos

suggestions¶

• Create new job group
• Trigger published online media after publishing
• Adapt and add test "update_leap42.2->tw"

Implementation¶

• Check when a tumbleweed image (either a cronjob, or an event via to test manager)
• Trigger a new job that will upgrade the latest openQA image generated for gnome and update the repos to be the official tumbleweed ones
• publish the new HDD upload to fixed assets (for now)
• Use the uploaded asset to do an upgrade form the latest released snapshot, to the to be released snapshot

further details¶

original discussion in #os-fctry

[9 Feb 2017 11:28:52] <DimStar> okurz: I had a look at all the gimp failures we have in openQA for snapshot 0208
[9 Feb 2017 11:29:07] <DimStar> e.g. https://openqa.opensuse.org/tests/351292#step/gimp/11
[9 Feb 2017 11:29:45] <DimStar> the problem there is that we actually seem to have an invalid test setup: the already published TW repo is enabled on this mchine; so 'old' packages bleed into what is going to be tested
[9 Feb 2017 11:30:09] <DimStar> *ni this case: gimp-help-browser no longer exists in 0208 - but it finds it on the published repo)
[9 Feb 2017 11:31:59] <DimStar> https://openqa.opensuse.org/tests/351292#step/zypper_lr/5 confirms that the published repos are enabled
[9 Feb 2017 11:31:59] <okurz> do you have an idea how to handle this?
[9 Feb 2017 11:32:45] <DimStar> I thought we had a call to remove the published repos - but we might not run this in every case needed
[9 Feb 2017 11:33:31] <DimStar> ./tests/update/zypper_clear_repos.pm - seems we only care enough when we test for updates
[9 Feb 2017 11:33:44] <okurz> DimStar: it would not harm to have post-publication tests which test that we can update from leap->last published tumbleweed. for that the online repos make sense
[9 Feb 2017 11:34:53] <DimStar> the entire TW test suite is to test anything to the FUTURE TW snapshot; what is already published is either working (as it passed before it was published) or broken (only a future snapshot could fix it)
[9 Feb 2017 11:35:39] <okurz> DimStar: yes, I think we can have *both*, leap->tw-released and leap->tw-in-test
[9 Feb 2017 11:35:55] <okurz> I understand that hardly any user would have a more up-to-date dvd repo than online repo but shouldn't zypper+rpm still be able to handle this?
[9 Feb 2017 11:36:09] <DimStar> if you want a leap->tw-released, fine... but not in the same job group
[9 Feb 2017 11:37:34] <DimStar> by having the online repos available we also mask potential dependency issues for stuff that is missing in the new repo
[9 Feb 2017 11:37:49] <DimStar> believe me - it's really really not what we want to test