action #16618
closed[qe-core][opensuse][functional] Tumbleweed upgrade tests
100%
Description
goal¶
Ensure we test like the user does with published online repos
suggestions¶
- Create new job group
- Trigger published online media after publishing
- Adapt and add test "update_leap42.2->tw"
Implementation¶
- Check when a tumbleweed image (either a cronjob, or an event via to test manager)
- Trigger a new job that will upgrade the latest openQA image generated for gnome and update the repos to be the official tumbleweed ones
- publish the new HDD upload to fixed assets (for now)
- Use the uploaded asset to do an upgrade form the
latest released snapshot
,to the to be released
snapshot
further details¶
original discussion in #os-fctry
[9 Feb 2017 11:28:52] <DimStar> okurz: I had a look at all the gimp failures we have in openQA for snapshot 0208
[9 Feb 2017 11:29:07] <DimStar> e.g. https://openqa.opensuse.org/tests/351292#step/gimp/11
[9 Feb 2017 11:29:45] <DimStar> the problem there is that we actually seem to have an invalid test setup: the already published TW repo is enabled on this mchine; so 'old' packages bleed into what is going to be tested
[9 Feb 2017 11:30:09] <DimStar> *ni this case: gimp-help-browser no longer exists in 0208 - but it finds it on the published repo)
[9 Feb 2017 11:31:59] <DimStar> https://openqa.opensuse.org/tests/351292#step/zypper_lr/5 confirms that the published repos are enabled
[9 Feb 2017 11:31:59] <okurz> do you have an idea how to handle this?
[9 Feb 2017 11:32:45] <DimStar> I thought we had a call to remove the published repos - but we might not run this in every case needed
[9 Feb 2017 11:33:31] <DimStar> ./tests/update/zypper_clear_repos.pm - seems we only care enough when we test for updates
[9 Feb 2017 11:33:44] <okurz> DimStar: it would not harm to have post-publication tests which test that we can update from leap->last published tumbleweed. for that the online repos make sense
[9 Feb 2017 11:34:53] <DimStar> the entire TW test suite is to test anything to the FUTURE TW snapshot; what is already published is either working (as it passed before it was published) or broken (only a future snapshot could fix it)
[9 Feb 2017 11:35:39] <okurz> DimStar: yes, I think we can have *both*, leap->tw-released and leap->tw-in-test
[9 Feb 2017 11:35:55] <okurz> I understand that hardly any user would have a more up-to-date dvd repo than online repo but shouldn't zypper+rpm still be able to handle this?
[9 Feb 2017 11:36:09] <DimStar> if you want a leap->tw-released, fine... but not in the same job group
[9 Feb 2017 11:37:34] <DimStar> by having the online repos available we also mask potential dependency issues for stuff that is missing in the new repo
[9 Feb 2017 11:37:49] <DimStar> believe me - it's really really not what we want to test
Updated by okurz over 7 years ago
- Copied to coordination #17436: [tools][functional][y][epic] tests must not have access to published repos, e.g. download.o.o on Tumbleweed -> remove all references to download.o.o from repos during installation and afterwards added
Updated by okurz about 7 years ago
- Subject changed from post-published Tumbleweed tests to [opensuse][functional]post-published Tumbleweed tests
Updated by okurz over 6 years ago
- Subject changed from [opensuse][functional]post-published Tumbleweed tests to [opensuse][functional][u] post-published Tumbleweed tests
Updated by mgriessmeier over 4 years ago
- Status changed from New to Rejected
already done by now by zypper_clear_repos
Updated by okurz over 4 years ago
- Status changed from Rejected to New
Seems you misunderstood. What zypper_clear_repos
is doing is ensuring kind-of the opposite, i.e. that no published repos are used during tests of a to-be-published snapshot. This is about tests after publish.
Updated by szarate over 4 years ago
- Description updated (diff)
- Status changed from New to Workable
- Priority changed from Low to Normal
- Target version changed from future to Milestone 30
- Start date deleted (
2017-02-09) - Estimated time set to 42.00 h
Updated by tjyrinki_suse almost 4 years ago
- Subject changed from [opensuse][functional][u] post-published Tumbleweed tests to [qe-core][opensuse][functional] post-published Tumbleweed tests
Updated by szarate about 3 years ago
- Related to action #94285: [sle][Migration][SLE15SP4] check nest virtual test for service check added
Updated by okurz almost 3 years ago
I came to this ticket due to periodically reviewing tickets as described on https://progress.opensuse.org/projects/openqatests/wiki#How-we-work-on-tickets
This ticket was set to "Normal" priority but was not updated within the SLO period for "Normal" tickets (365 days) as described on https://progress.opensuse.org/projects/openqatests/wiki/Wiki#SLOs-service-level-objectives
First reminder: Please consider picking up this ticket within the next 365 days or just set the ticket to the next lower priority of "Low" (no SLO related time period).
Updated by slo-gin almost 2 years ago
This ticket was set to Normal priority but was not updated within the SLO period. Please consider picking up this ticket or just set the ticket to the next lower priority.
Updated by szarate over 1 year ago
- Tags set to qe-core-april-sprint
- Target version deleted (
Milestone 30)
Updated by szarate over 1 year ago
- Sprint set to QE-Core: April Sprint 23 (Apr 05 - May 03)
Updated by ph03nix over 1 year ago
- Subject changed from [qe-core][opensuse][functional] post-published Tumbleweed tests to [qe-core][opensuse][functional] Tumbleweed upgrade tests
Updated by ph03nix over 1 year ago
Just a small idea to make things a bit easier:
We might not need to keep track of the last snapshot and hand over a VM image from snapshot to snapshot. Wouldn't it be possible to just initiate a new install with the currently RELEASED Tumbleweed ISO, then do the installation, create a VM image and use this VM image to then install the snapshot updates in it?
So, as a sequence
- Create installation test run, which installs Tumbleweed from released isos from get.opensuse.org (or download.opensuse.org)
- Produce a VM image for subsequent tests
- Use this VM image, add the new snapshot repositories
- Perform system upgrade
- Proceed with regression tests
I think this should also work, and it removes one moving part from the whole concept - namely the bookkeeping and handing over of a VM image from snapshot to snapshot.
Updated by rfan1 over 1 year ago
- Status changed from Workable to In Progress
- Target version set to QE-Core: Ready
I have added a job in dev job group:
https://openqa.opensuse.org/tests/3247185
Now let me try to enhance my test to use gnome mode [IMO one single job is good enough to cover the upgrade test scenario]
Updated by rfan1 over 1 year ago
- % Done changed from 0 to 30
Now move the textmode to gnome mode, let us wait for next openQA result.
Updated by rfan1 over 1 year ago
- % Done changed from 30 to 60
Create HDD test passed, now let me add the zdup cases:
- zdup-tw-current-gnome:
testsuite: null
settings:
CASEDIR: https://github.com/rfan1/os-autoinst-distri-opensuse.git#tw_update
DESKTOP: gnome
HDD_1: '%DISTRI%-%VERSION%-%ARCH%-%BUILD%-%DESKTOP%_current@%MACHINE%.qcow2'
START_AFTER_TEST: create_hdd_gnome_autoyast_current
YAML_SCHEDULE: schedule/migration/zdup-tw-current-gnome.yaml
QEMURAM: '3072'
Updated by szarate over 1 year ago
- Sprint changed from QE-Core: April Sprint 23 (Apr 05 - May 03) to QE-Core: May Sprint 23 (May 10 - May 31)
Updated by rfan1 over 1 year ago
- Status changed from In Progress to Resolved
We have already had the test cases:
https://openqa.opensuse.org/tests/3306990 [gnome]
https://openqa.opensuse.org/tests/3306993 [kde]
So we can mark it done now :)