Project

General

Profile

action #39305

coordination #39302: [qe-core][functional][opensuse][epic][medium] uefi upgrade tests on TW+Leap (was: missing assets)

[functional][u] 42.3 upgrade tests use GM image instead of updated one

Added by lnussel about 3 years ago. Updated almost 2 years ago.

Status:
Resolved
Priority:
High
Category:
Bugs in existing tests
Target version:
SUSE QA - Milestone 28
Start date:
2018-08-08
Due date:
% Done:

0%

Estimated time:
42.00 h
Difficulty:

Description

Observation

distribution upgrade tests normally use images that have online updates applied. the 42.3 tests don't do that though

Acceptance criteria

  • AC1: A qcow2 image with Leap 42.3 + all updates is available in O3.
  • AC2: Upgrade tests use that image instead of the plane GM image.

Tasks

  1. Rename update_* test suite to upgrade_* DONE
  2. Manually trigger a job that publishes a qcow2 image with GM+Updates DONE
  3. Update test settings of /^upgrade_Leap_42.3.*$/ to use that image

Related issues

Related to openQA Tests - action #46868: [aarch64][functional][y][migration] enable migration tests on aarch64Resolved2019-01-302019-03-12

History

#2 Updated by coolo about 3 years ago

  • Project changed from openQA Project to openQA Tests

#3 Updated by okurz about 3 years ago

  • Subject changed from 42.3 upgrade tests use GM image instead of updated one to [opensuse] 42.3 upgrade tests use GM image instead of updated one
  • Category set to Bugs in existing tests

lnussel do you plan to fix this yourself?

#4 Updated by lnussel almost 3 years ago

no

#5 Updated by okurz almost 3 years ago

  • Subject changed from [opensuse] 42.3 upgrade tests use GM image instead of updated one to [functional][u] 42.3 upgrade tests use GM image instead of updated one
  • Target version set to Milestone 23

I estimate will take some months.

#6 Updated by okurz almost 3 years ago

  • Related to action #44546: [opensuse][functional][u] TW kde 42.2 upgrade tests missing assets added

#7 Updated by okurz over 2 years ago

  • Related to action #46868: [aarch64][functional][y][migration] enable migration tests on aarch64 added

#8 Updated by okurz over 2 years ago

  • Blocked by action #40052: [functional][u] inconsistent test cases / missing assets in openSUSE Leap upgrade cases vs. Tumbleweed, part 2 added

#9 Updated by okurz over 2 years ago

  • Status changed from New to Blocked
  • Assignee set to okurz

#10 Updated by okurz over 2 years ago

  • Status changed from Blocked to Workable
  • Assignee deleted (okurz)

unblocked

#11 Updated by SLindoMansilla over 2 years ago

  • Description updated (diff)

#12 Updated by mgriessmeier over 2 years ago

  • Target version changed from Milestone 23 to Milestone 24

moving to M24

#13 Updated by mgriessmeier over 2 years ago

  • Target version changed from Milestone 24 to Milestone 25

#14 Updated by okurz over 2 years ago

  • Related to action #51824: [functional][u] test incomplete - missing qcow2 for 64bit machine (non-2G) added

#15 Updated by mgriessmeier over 2 years ago

  • Target version changed from Milestone 25 to Milestone 26

#16 Updated by SLindoMansilla about 2 years ago

If we keep an image with updates from latest build, that will cause the problem of having an image with the updates that a user will not have, because they are not release.
It would require too much manual work to keep a qcow2 that has updates from released repositories.

The best approach I can think is to keep GM qcow2 (we already have it under /var/lib/openqa/share/factory/hdd/fixed/), creating a new test suite that performs an update with official repositores, then execute upgrade tests after it.
I also propose to change the current confusing name of upgrade tests from update_* to upgrade_*.

#17 Updated by okurz about 2 years ago

Renaming is a good idea.

I guess a single manually triggered installation+update openqa run and saving the qcow image would suffice. Based on experience with the migration tests as well as qam ones I recommend not to install all updates for every test run as this takes long and is not perfectly stable

#18 Updated by SLindoMansilla about 2 years ago

  • Status changed from Workable to In Progress
  • Assignee set to SLindoMansilla

okurz wrote:

Renaming is a good idea.

Ok, I will prepare the test suite with name upgrade_* in the development job group.

I guess a single manually triggered installation+update openqa run and saving the qcow image would suffice.

I think that would be too much manual work. Since we cannot be sure to have the last updated packages, we will end up manually running the update test suite for each new build, so, why not let isos post trigger the job?

Based on experience with the migration tests as well as qam ones I recommend not to install all updates for every test run as this takes long and is not perfectly stable

Yes, I didn't mean to add it as a module to any upgrade test suute, but to START_AFTER_TEST, so it will only be executed one time.

#19 Updated by SLindoMansilla about 2 years ago

  • Description updated (diff)

Tasks updated, only working on task 1 at the moment, waiting for feedback on agreement.

#20 Updated by okurz about 2 years ago

SLindoMansilla wrote:

okurz wrote:

Renaming is a good idea.

Ok, I will prepare the test suite with name upgrade_* in the development job group.

If you rather rename the existing ones most likely the scenario will be still considered the same and even label carry over should work. Just inform the openSUSE RMs as usual upfront and you should be good to go.

I guess a single manually triggered installation+update openqa run and saving the qcow image would suffice.

I think that would be too much manual work. Since we cannot be sure to have the last updated packages, we will end up manually running the update test suite for each new build,

As I was writing a single manually triggered run, not for every few builds, especially for 42.3 which is EOL and does not receive any updates anymore. Do it once and not ever again :)

so, why not let isos post trigger the job?

because the jobs would take really long to install the very same updates again and again for every build.

Based on experience with the migration tests as well as qam ones I recommend not to install all updates for every test run as this takes long and is not perfectly stable

Yes, I didn't mean to add it as a module to any upgrade test suute, but to START_AFTER_TEST, so it will only be executed one time.

That's still the same what QAM does. They do have it as a module but only because they run it for different scenarios, e.g. SLES/SLED/with addons/without/gnome/kde. You would have the same problem. The biggest time impact is reinstalling the updates at a time when a new build for new products is triggered however the updates are released way before and at a different schedule. I already discussed this point for improvement very often with multiple persons, unfortunately whenever I explained that to someone from QA SLE Migration he shortly afterwards left SUSE, don't know if that is a coincidence ;)

If you prefer we can talk about this topic in person next week.

#21 Updated by SLindoMansilla about 2 years ago

  • Status changed from In Progress to Feedback

Test suites found with regex /^update_/ replaced to "upgrade_".

Waiting for agreement on the following tasks.

#22 Updated by mgriessmeier about 2 years ago

  • Target version changed from Milestone 26 to Milestone 27

#23 Updated by SLindoMansilla about 2 years ago

  • Related to deleted (action #51824: [functional][u] test incomplete - missing qcow2 for 64bit machine (non-2G))

#24 Updated by SLindoMansilla about 2 years ago

  • Related to deleted (action #44546: [opensuse][functional][u] TW kde 42.2 upgrade tests missing assets)

#25 Updated by SLindoMansilla about 2 years ago

  • Blocked by deleted (action #40052: [functional][u] inconsistent test cases / missing assets in openSUSE Leap upgrade cases vs. Tumbleweed, part 2)

#26 Updated by SLindoMansilla about 2 years ago

  • Parent task set to #39302

#27 Updated by SLindoMansilla about 2 years ago

  • Description updated (diff)
  • Status changed from Feedback to Workable

Agreed to only address Leap 42.3 in this ticket.

#28 Updated by SLindoMansilla about 2 years ago

  • Assignee deleted (SLindoMansilla)

#29 Updated by SLindoMansilla about 2 years ago

  • Estimated time set to 42.00 h

#30 Updated by mgriessmeier about 2 years ago

  • Priority changed from Normal to High
  • Target version changed from Milestone 27 to Milestone 28

#31 Updated by SLindoMansilla about 2 years ago

  • Status changed from Workable to In Progress
  • Assignee set to SLindoMansilla

#32 Updated by SLindoMansilla about 2 years ago

From Tumbleweed's job group overview https://openqa.opensuse.org/tests/overview?distri=microos&distri=opensuse&version=Tumbleweed&build=20190917&groupid=1

I can find 4 upgrade scenarios from Leap 42.3 to Tumbleweed:

  • upgrade_Leap_42.3_cryptlvm
  • upgrade_Leap_42.3_cryptlvm@uefi
  • upgrade_Leap_42.3_gnome
  • upgrade_Leap_42.3_kde

#34 Updated by SLindoMansilla about 2 years ago

  • Description updated (diff)

#35 Updated by okurz about 2 years ago

SLindoMansilla wrote:

There is not enough free space on ariel:/var/lib/openqa/share/factory/hdd/fixed

how did you come to this conclusion? There is currently 962GB available.

#36 Updated by SLindoMansilla about 2 years ago

okurz wrote:

how did you come to this conclusion? There is currently 962GB available.

oh, true.
I did only df -h and didn't check mounted directories.

#37 Updated by okurz about 2 years ago

As a temporary upload location I can recommend /space/upload/ . Better don't try to "/tmp" or you will fill up / rather quickly :)

#38 Updated by SLindoMansilla about 2 years ago

The 4 qcow2 images are now under ariel:/var/lib/openqa/share/factory/hdd/fixed

#39 Updated by SLindoMansilla about 2 years ago

Verification runs

The disks were created with 20GB size. qcow2 images used in PROD are 40GB.
Maybe it should be better to increase the images size by 20GB.
The problem is that the installer proposed a separate partition for /home which leaves less space for /: http://openqa.slindomansilla-vm.qa.suse.de/tests/1691#step/encrypt_lvm/7
But, yast2 sw_single didn't expect the combination " / + /boot + /home". So, the test fails and needs to be adapted.

#40 Updated by SLindoMansilla about 2 years ago

New test suites added in Tumbleweed development job group: https://openqa.opensuse.org/admin/job_templates/38

#41 Updated by SLindoMansilla about 2 years ago

New needle created: yast2_i-yast2-sw_single-disk_usage-with_sep_home_and_sep_boot-20191010

Waiting for verification run: https://openqa.opensuse.org/tests/1053392

#43 Updated by SLindoMansilla almost 2 years ago

  • Status changed from In Progress to Resolved

Also available in: Atom PDF