Project

General

Profile

action #13156

os-autoinst: Add support to easily switch VERSION during a test run

Added by okurz almost 6 years ago. Updated over 4 years ago.

Status:
Resolved
Priority:
Low
Assignee:
-
Category:
Feature requests
Target version:
-
Start date:
2016-08-11
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

observation

for example in https://openqa.suse.de/tests/510956 zypper lifecycle is called which does not exist as we rolled back to a pre-SP2 state: https://openqa.suse.de/tests/510956#step/zypper_lifecycle/7

problem

VERSION=12-SP2 but when scheduling tests or trying to match needles this does not make sense.

Also see https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/1684 for a possible approach updating "VERSION" during the test run.

Three different scopes have to be kept in mind

  • needle registering (and unregistering)
  • test module scheduling
  • test module execution

For example, updating VERSION in a test module is executed after all modules are scheduled so it needs to be set in main.pm, too. As needle tags already got registered/unregistered, they need to be re-registered after every version change.

suggestion

Also see #9580#note-24

further details

The original title was 'how do we handle tests in "rollback_migration" which shouldn't start because VERSION is 12-SP2 but after rollback it's not anymore?'


Related issues

Related to openQA Tests - action #25504: Support for changing test variables including needles during test run (was: utils::sle_version_at_least needs refinement)Resolved2017-09-22

Has duplicate openQA Tests - action #13210: OpenQA is testing SP1 instead of SP2Closed2016-08-16

Blocked by openQA Tests - action #9580: Boot to snapshot after upgrade and then rollbackResolved2015-11-17

History

#1 Updated by okurz almost 6 years ago

  • Blocked by action #9580: Boot to snapshot after upgrade and then rollback added

#2 Updated by okurz almost 6 years ago

  • Description updated (diff)

#3 Updated by okurz almost 6 years ago

  • Description updated (diff)

#4 Updated by dgutu almost 6 years ago

We might/could use the same approach as for migration when based on VAR we a loading tests from subfolder.

#5 Updated by okurz almost 6 years ago

Interesting. Can you link this example or are you talking about an idea that isn't implemented yet?

#6 Updated by okurz almost 6 years ago

  • Has duplicate action #13210: OpenQA is testing SP1 instead of SP2 added

#7 Updated by okurz almost 6 years ago

  • Priority changed from Normal to High

most recent failing example: https://openqa.suse.de/tests/569863

#8 Updated by maritawerner over 5 years ago

Could be assigned to szarate.

#9 Updated by okurz over 5 years ago

  • Project changed from openQA Tests to openQA Project
  • Subject changed from how do we handle tests in "rollback_migration" which shouldn't start because VERSION *is* 12-SP2 but after rollback it's not anymore? to os-autoinst: Add support to easily switch VERSION during a test run
  • Description updated (diff)
  • Category changed from Enhancement to existing tests to 136

#10 Updated by okurz over 5 years ago

#11 Updated by mgriessmeier over 5 years ago

still a problem in 0059
https://openqa.suse.de/tests/640726

#12 Updated by okurz over 5 years ago

  • Target version set to Milestone 5

#13 Updated by okurz over 5 years ago

#14 Updated by okurz over 5 years ago

  • Priority changed from High to Normal

while it might be nice to continue with this, e.g. for the "rollback after migration" jobs, this ticket does not block #12330 anymore and therefore should be regarded as high -> reduced to normal

#15 Updated by okurz over 5 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: rollback_migration_offline_sle12
http://openqa.suse.de/tests/627621

#16 Updated by okurz over 5 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: rollback_migration_offline_sle12
http://openqa.suse.de/tests/627621

#17 Updated by okurz over 5 years ago

  • Target version changed from Milestone 5 to Milestone 6

No one working on this and will most certainly not finish within M5

#18 Updated by okurz over 5 years ago

  • Target version changed from Milestone 6 to Milestone 7

#19 Updated by okurz over 5 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: rollback_migration_offline_sle12
http://openqa.suse.de/tests/627621

#20 Updated by okurz over 5 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: rollback_migration_offline_sle12
http://openqa.suse.de/tests/627621

#21 Updated by okurz about 5 years ago

  • Priority changed from Normal to Low
  • Target version deleted (Milestone 7)

For now we accepted the fact that this does not work and are using workarounds which are good enough, e.g. version specific needle tags for migration scenarios, etc.

#22 Updated by okurz almost 5 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: rollback_migration_offline_sle12
http://openqa.suse.de/tests/627621

#23 Updated by okurz almost 5 years ago

  • Related to action #25504: Support for changing test variables including needles during test run (was: utils::sle_version_at_least needs refinement) added

#24 Updated by coolo over 4 years ago

  • Status changed from New to Resolved

for needle matches we have a solution (#25504), for scheduling tests we have to be creative

#25 Updated by okurz over 4 years ago

Also available in: Atom PDF