Project

General

Profile

action #16618

[qe-core][opensuse][functional] post-published Tumbleweed tests

Added by okurz almost 5 years ago. Updated 18 days ago.

Status:
Workable
Priority:
Normal
Assignee:
-
Category:
New test
Target version:
SUSE QA - Milestone 30
Start date:
Due date:
% Done:

0%

Estimated time:
42.00 h
Difficulty:

Description

goal

Ensure we test like the user does with published online repos

suggestions

  • Create new job group
  • Trigger published online media after publishing
  • Adapt and add test "update_leap42.2->tw"

Implementation

  • Check when a tumbleweed image (either a cronjob, or an event via to test manager)
  • Trigger a new job that will upgrade the latest openQA image generated for gnome and update the repos to be the official tumbleweed ones
  • publish the new HDD upload to fixed assets (for now)
  • Use the uploaded asset to do an upgrade form the latest released snapshot, to the to be released snapshot

further details

original discussion in #os-fctry

[9 Feb 2017 11:28:52] <DimStar> okurz: I had a look at all the gimp failures we have in openQA for snapshot 0208
[9 Feb 2017 11:29:07] <DimStar> e.g. https://openqa.opensuse.org/tests/351292#step/gimp/11
[9 Feb 2017 11:29:45] <DimStar> the problem there is that we actually seem to have an invalid test setup: the already published TW repo is enabled on this mchine; so 'old' packages bleed into what is going to be tested
[9 Feb 2017 11:30:09] <DimStar> *ni this case: gimp-help-browser no longer exists in 0208 - but it finds it on the published repo)
[9 Feb 2017 11:31:59] <DimStar> https://openqa.opensuse.org/tests/351292#step/zypper_lr/5 confirms that the published repos are enabled
[9 Feb 2017 11:31:59] <okurz> do you have an idea how to handle this?
[9 Feb 2017 11:32:45] <DimStar> I thought we had a call to remove the published repos - but we might not run this in every case needed
[9 Feb 2017 11:33:31] <DimStar> ./tests/update/zypper_clear_repos.pm - seems we only care enough when we test for updates
[9 Feb 2017 11:33:44] <okurz> DimStar: it would not harm to have post-publication tests which test that we can update from leap->last published tumbleweed. for that the online repos make sense
[9 Feb 2017 11:34:53] <DimStar> the entire TW test suite is to test anything to the FUTURE TW snapshot; what is already published is either working (as it passed before it was published) or broken (only a future snapshot could fix it)
[9 Feb 2017 11:35:39] <okurz> DimStar: yes, I think we can have *both*, leap->tw-released and leap->tw-in-test
[9 Feb 2017 11:35:55] <okurz> I understand that hardly any user would have a more up-to-date dvd repo than online repo but shouldn't zypper+rpm still be able to handle this?
[9 Feb 2017 11:36:09] <DimStar> if you want a leap->tw-released, fine... but not in the same job group
[9 Feb 2017 11:37:34] <DimStar> by having the online repos available we also mask potential dependency issues for stuff that is missing in the new repo
[9 Feb 2017 11:37:49] <DimStar> believe me - it's really really not what we want to test

Related issues

Related to openQA Tests - action #94285: [sle][Migration][SLE15SP4] check nest virtual test for service checkResolved2021-06-21

Copied to openQA Tests - coordination #17436: [tools][functional][y][epic] tests must not have access to published repos, e.g. download.o.o on Tumbleweed -> remove all references to download.o.o from repos during installation and afterwardsResolved2018-02-122019-02-26

History

#1 Updated by okurz almost 5 years ago

  • Copied to coordination #17436: [tools][functional][y][epic] tests must not have access to published repos, e.g. download.o.o on Tumbleweed -> remove all references to download.o.o from repos during installation and afterwards added

#2 Updated by okurz over 4 years ago

  • Subject changed from post-published Tumbleweed tests to [opensuse][functional]post-published Tumbleweed tests

#3 Updated by okurz over 3 years ago

  • Subject changed from [opensuse][functional]post-published Tumbleweed tests to [opensuse][functional][u] post-published Tumbleweed tests

#4 Updated by okurz over 3 years ago

  • Target version changed from future to future

#5 Updated by mgriessmeier almost 2 years ago

  • Status changed from New to Rejected

already done by now by zypper_clear_repos

#6 Updated by okurz almost 2 years ago

  • Status changed from Rejected to New

Seems you misunderstood. What zypper_clear_repos is doing is ensuring kind-of the opposite, i.e. that no published repos are used during tests of a to-be-published snapshot. This is about tests after publish.

#7 Updated by szarate over 1 year ago

  • Description updated (diff)
  • Status changed from New to Workable
  • Priority changed from Low to Normal
  • Target version changed from future to Milestone 30
  • Start date deleted (2017-02-09)
  • Estimated time set to 42.00 h

#8 Updated by tjyrinki_suse about 1 year ago

  • Subject changed from [opensuse][functional][u] post-published Tumbleweed tests to [qe-core][opensuse][functional] post-published Tumbleweed tests

#9 Updated by szarate 5 months ago

  • Related to action #94285: [sle][Migration][SLE15SP4] check nest virtual test for service check added

#10 Updated by okurz 18 days ago

I came to this ticket due to periodically reviewing tickets as described on https://progress.opensuse.org/projects/openqatests/wiki#How-we-work-on-tickets

This ticket was set to "Normal" priority but was not updated within the SLO period for "Normal" tickets (365 days) as described on https://progress.opensuse.org/projects/openqatests/wiki/Wiki#SLOs-service-level-objectives

First reminder: Please consider picking up this ticket within the next 365 days or just set the ticket to the next lower priority of "Low" (no SLO related time period).

Also available in: Atom PDF