Project

General

Profile

Actions

action #13156

closed

os-autoinst: Add support to easily switch VERSION during a test run

Added by okurz over 8 years ago. Updated about 7 years ago.

Status:
Resolved
Priority:
Low
Assignee:
-
Category:
Feature requests
Target version:
-
Start date:
2016-08-11
Due date:
% Done:

0%

Estimated time:

Description

observation

for example in https://openqa.suse.de/tests/510956 zypper lifecycle is called which does not exist as we rolled back to a pre-SP2 state: https://openqa.suse.de/tests/510956#step/zypper_lifecycle/7

problem

VERSION=12-SP2 but when scheduling tests or trying to match needles this does not make sense.

Also see https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/1684 for a possible approach updating "VERSION" during the test run.

Three different scopes have to be kept in mind

  • needle registering (and unregistering)
  • test module scheduling
  • test module execution

For example, updating VERSION in a test module is executed after all modules are scheduled so it needs to be set in main.pm, too. As needle tags already got registered/unregistered, they need to be re-registered after every version change.

suggestion

Also see #9580#note-24

further details

The original title was 'how do we handle tests in "rollback_migration" which shouldn't start because VERSION is 12-SP2 but after rollback it's not anymore?'


Related issues 3 (0 open3 closed)

Related to openQA Tests (public) - action #25504: Support for changing test variables including needles during test run (was: utils::sle_version_at_least needs refinement)Resolvedxlai2017-09-22

Actions
Has duplicate openQA Tests (public) - action #13210: OpenQA is testing SP1 instead of SP2Closedmkravec2016-08-16

Actions
Blocked by openQA Tests (public) - action #9580: Boot to snapshot after upgrade and then rollbackResolvedokurz2015-11-17

Actions
Actions #1

Updated by okurz over 8 years ago

  • Blocked by action #9580: Boot to snapshot after upgrade and then rollback added
Actions #2

Updated by okurz over 8 years ago

  • Description updated (diff)
Actions #3

Updated by okurz over 8 years ago

  • Description updated (diff)
Actions #4

Updated by dgutu over 8 years ago

We might/could use the same approach as for migration when based on VAR we a loading tests from subfolder.

Actions #5

Updated by okurz over 8 years ago

Interesting. Can you link this example or are you talking about an idea that isn't implemented yet?

Actions #6

Updated by okurz over 8 years ago

  • Has duplicate action #13210: OpenQA is testing SP1 instead of SP2 added
Actions #7

Updated by okurz about 8 years ago

  • Priority changed from Normal to High

most recent failing example: https://openqa.suse.de/tests/569863

Actions #8

Updated by maritawerner about 8 years ago

Could be assigned to szarate.

Actions #9

Updated by okurz about 8 years ago

  • Project changed from openQA Tests (public) to openQA Project (public)
  • Subject changed from how do we handle tests in "rollback_migration" which shouldn't start because VERSION *is* 12-SP2 but after rollback it's not anymore? to os-autoinst: Add support to easily switch VERSION during a test run
  • Description updated (diff)
  • Category changed from Enhancement to existing tests to 136
Actions #10

Updated by okurz about 8 years ago

Actions #11

Updated by mgriessmeier about 8 years ago

still a problem in 0059
https://openqa.suse.de/tests/640726

Actions #12

Updated by okurz almost 8 years ago

  • Target version set to Milestone 5
Actions #13

Updated by okurz almost 8 years ago

Actions #14

Updated by okurz almost 8 years ago

  • Priority changed from High to Normal

while it might be nice to continue with this, e.g. for the "rollback after migration" jobs, this ticket does not block #12330 anymore and therefore should be regarded as high -> reduced to normal

Actions #15

Updated by okurz almost 8 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: rollback_migration_offline_sle12
http://openqa.suse.de/tests/627621

Actions #16

Updated by okurz almost 8 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: rollback_migration_offline_sle12
http://openqa.suse.de/tests/627621

Actions #17

Updated by okurz almost 8 years ago

  • Target version changed from Milestone 5 to Milestone 6

No one working on this and will most certainly not finish within M5

Actions #18

Updated by okurz almost 8 years ago

  • Target version changed from Milestone 6 to Milestone 7
Actions #19

Updated by okurz almost 8 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: rollback_migration_offline_sle12
http://openqa.suse.de/tests/627621

Actions #20

Updated by okurz almost 8 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: rollback_migration_offline_sle12
http://openqa.suse.de/tests/627621

Actions #21

Updated by okurz over 7 years ago

  • Priority changed from Normal to Low
  • Target version deleted (Milestone 7)

For now we accepted the fact that this does not work and are using workarounds which are good enough, e.g. version specific needle tags for migration scenarios, etc.

Actions #22

Updated by okurz over 7 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: rollback_migration_offline_sle12
http://openqa.suse.de/tests/627621

Actions #23

Updated by okurz about 7 years ago

  • Related to action #25504: Support for changing test variables including needles during test run (was: utils::sle_version_at_least needs refinement) added
Actions #24

Updated by coolo about 7 years ago

  • Status changed from New to Resolved

for needle matches we have a solution (#25504), for scheduling tests we have to be creative

Actions #25

Updated by okurz about 7 years ago

Actions

Also available in: Atom PDF