Project

General

Profile

Actions
Copy link

action #16618

closed

[qe-core][opensuse][functional] Tumbleweed upgrade tests

Added by okurz about 7 years ago. Updated 11 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
rfan1
Category:
New test
Target version:
QA - QE-Core: Ready
Start date:
Due date:
% Done:

100%

Estimated time:
42.00 h
Difficulty:
Sprint:
QE-Core: May Sprint 23 (May 10 - May 31)
Tags:
qe-core-april-sprint

Description

goal

Ensure we test like the user does with published online repos

suggestions

  • Create new job group
  • Trigger published online media after publishing
  • Adapt and add test "update_leap42.2->tw"

Implementation

  • Check when a tumbleweed image (either a cronjob, or an event via to test manager)
  • Trigger a new job that will upgrade the latest openQA image generated for gnome and update the repos to be the official tumbleweed ones
  • publish the new HDD upload to fixed assets (for now)
  • Use the uploaded asset to do an upgrade form the latest released snapshot, to the to be released snapshot

further details

original discussion in #os-fctry

[9 Feb 2017 11:28:52] <DimStar> okurz: I had a look at all the gimp failures we have in openQA for snapshot 0208
[9 Feb 2017 11:29:07] <DimStar> e.g. https://openqa.opensuse.org/tests/351292#step/gimp/11
[9 Feb 2017 11:29:45] <DimStar> the problem there is that we actually seem to have an invalid test setup: the already published TW repo is enabled on this mchine; so 'old' packages bleed into what is going to be tested
[9 Feb 2017 11:30:09] <DimStar> *ni this case: gimp-help-browser no longer exists in 0208 - but it finds it on the published repo)
[9 Feb 2017 11:31:59] <DimStar> https://openqa.opensuse.org/tests/351292#step/zypper_lr/5 confirms that the published repos are enabled
[9 Feb 2017 11:31:59] <okurz> do you have an idea how to handle this?
[9 Feb 2017 11:32:45] <DimStar> I thought we had a call to remove the published repos - but we might not run this in every case needed
[9 Feb 2017 11:33:31] <DimStar> ./tests/update/zypper_clear_repos.pm - seems we only care enough when we test for updates
[9 Feb 2017 11:33:44] <okurz> DimStar: it would not harm to have post-publication tests which test that we can update from leap->last published tumbleweed. for that the online repos make sense
[9 Feb 2017 11:34:53] <DimStar> the entire TW test suite is to test anything to the FUTURE TW snapshot; what is already published is either working (as it passed before it was published) or broken (only a future snapshot could fix it)
[9 Feb 2017 11:35:39] <okurz> DimStar: yes, I think we can have *both*, leap->tw-released and leap->tw-in-test
[9 Feb 2017 11:35:55] <okurz> I understand that hardly any user would have a more up-to-date dvd repo than online repo but shouldn't zypper+rpm still be able to handle this?
[9 Feb 2017 11:36:09] <DimStar> if you want a leap->tw-released, fine... but not in the same job group
[9 Feb 2017 11:37:34] <DimStar> by having the online repos available we also mask potential dependency issues for stuff that is missing in the new repo
[9 Feb 2017 11:37:49] <DimStar> believe me - it's really really not what we want to test

Related issues 2 (0 open — 2 closed)

Related to openQA Tests - action #94285: [sle][Migration][SLE15SP4] check nest virtual test for service checkResolvedcoolgw2021-06-21

Actions
Copied to openQA Tests - coordination #17436: [tools][functional][y][epic] tests must not have access to published repos, e.g. download.o.o on Tumbleweed -> remove all references to download.o.o from repos during installation and afterwardsResolvedriafarov2018-02-122019-02-26

Actions
Actions
Copy link
 #1

Updated by okurz about 7 years ago

  • Copied to coordination #17436: [tools][functional][y][epic] tests must not have access to published repos, e.g. download.o.o on Tumbleweed -> remove all references to download.o.o from repos during installation and afterwards added
Actions
Copy link
 #2

Updated by okurz almost 7 years ago

  • Subject changed from post-published Tumbleweed tests to [opensuse][functional]post-published Tumbleweed tests
Actions
Copy link
 #3

Updated by okurz almost 6 years ago

  • Subject changed from [opensuse][functional]post-published Tumbleweed tests to [opensuse][functional][u] post-published Tumbleweed tests
Actions
Copy link
 #4

Updated by okurz almost 6 years ago

  • Target version changed from future to future
Actions
Copy link
 #5

Updated by mgriessmeier about 4 years ago

  • Status changed from New to Rejected

already done by now by zypper_clear_repos

Actions
Copy link
 #6

Updated by okurz about 4 years ago

  • Status changed from Rejected to New

Seems you misunderstood. What zypper_clear_repos is doing is ensuring kind-of the opposite, i.e. that no published repos are used during tests of a to-be-published snapshot. This is about tests after publish.

Actions
Copy link
 #7

Updated by szarate almost 4 years ago

  • Description updated (diff)
  • Status changed from New to Workable
  • Priority changed from Low to Normal
  • Target version changed from future to Milestone 30
  • Start date deleted (2017-02-09)
  • Estimated time set to 42.00 h
Actions
Copy link
 #8

Updated by tjyrinki_suse over 3 years ago

  • Subject changed from [opensuse][functional][u] post-published Tumbleweed tests to [qe-core][opensuse][functional] post-published Tumbleweed tests
Actions
Copy link
 #9

Updated by szarate almost 3 years ago

  • Related to action #94285: [sle][Migration][SLE15SP4] check nest virtual test for service check added
Actions
Copy link
 #10

Updated by okurz over 2 years ago

I came to this ticket due to periodically reviewing tickets as described on https://progress.opensuse.org/projects/openqatests/wiki#How-we-work-on-tickets

This ticket was set to "Normal" priority but was not updated within the SLO period for "Normal" tickets (365 days) as described on https://progress.opensuse.org/projects/openqatests/wiki/Wiki#SLOs-service-level-objectives

First reminder: Please consider picking up this ticket within the next 365 days or just set the ticket to the next lower priority of "Low" (no SLO related time period).

Actions
Copy link
 #11

Updated by slo-gin over 1 year ago

This ticket was set to Normal priority but was not updated within the SLO period. Please consider picking up this ticket or just set the ticket to the next lower priority.

Actions
Copy link
 #12

Updated by szarate about 1 year ago

  • Tags set to qe-core-april-sprint
  • Target version deleted (Milestone 30)
Actions
Copy link
 #13

Updated by szarate about 1 year ago

  • Sprint set to QE-Core: April Sprint 23 (Apr 05 - May 03)
Actions
Copy link
 #14

Updated by ph03nix about 1 year ago

  • Subject changed from [qe-core][opensuse][functional] post-published Tumbleweed tests to [qe-core][opensuse][functional] Tumbleweed upgrade tests
Actions
Copy link
 #15

Updated by ph03nix about 1 year ago

Just a small idea to make things a bit easier:

We might not need to keep track of the last snapshot and hand over a VM image from snapshot to snapshot. Wouldn't it be possible to just initiate a new install with the currently RELEASED Tumbleweed ISO, then do the installation, create a VM image and use this VM image to then install the snapshot updates in it?

So, as a sequence

  1. Create installation test run, which installs Tumbleweed from released isos from get.opensuse.org (or download.opensuse.org)
  2. Produce a VM image for subsequent tests
  3. Use this VM image, add the new snapshot repositories
  4. Perform system upgrade
  5. Proceed with regression tests

I think this should also work, and it removes one moving part from the whole concept - namely the bookkeeping and handing over of a VM image from snapshot to snapshot.

Actions
Copy link
 #16

Updated by rfan1 about 1 year ago

  • Assignee set to rfan1
Actions
Copy link
 #18

Updated by rfan1 almost 1 year ago

  • Status changed from Workable to In Progress
  • Target version set to QE-Core: Ready

I have added a job in dev job group:
https://openqa.opensuse.org/tests/3247185
Now let me try to enhance my test to use gnome mode [IMO one single job is good enough to cover the upgrade test scenario]

Actions
Copy link
 #19

Updated by rfan1 12 months ago

  • % Done changed from 0 to 30

Now move the textmode to gnome mode, let us wait for next openQA result.

Actions
Copy link
 #20

Updated by rfan1 12 months ago

  • % Done changed from 30 to 60

Create HDD test passed, now let me add the zdup cases:

    - zdup-tw-current-gnome:
        testsuite: null
        settings:
          CASEDIR: https://github.com/rfan1/os-autoinst-distri-opensuse.git#tw_update
          DESKTOP: gnome
          HDD_1: '%DISTRI%-%VERSION%-%ARCH%-%BUILD%-%DESKTOP%_current@%MACHINE%.qcow2'
          START_AFTER_TEST: create_hdd_gnome_autoyast_current
          YAML_SCHEDULE: schedule/migration/zdup-tw-current-gnome.yaml
          QEMURAM: '3072'
Actions
Copy link
 #21

Updated by szarate 12 months ago

  • Sprint changed from QE-Core: April Sprint 23 (Apr 05 - May 03) to QE-Core: May Sprint 23 (May 10 - May 31)
Actions
Copy link
 #23

Updated by rfan1 11 months ago

  • Status changed from In Progress to Resolved

We have already had the test cases:

https://openqa.opensuse.org/tests/3306990 [gnome]
https://openqa.opensuse.org/tests/3306993 [kde]

So we can mark it done now :)

Actions
Copy link
 #24

Updated by rfan1 11 months ago

  • % Done changed from 60 to 100
Actions
Copy link

Also available in: Atom PDF