Project

General

Profile

action #51959

[sle][y] Fix issues introduced in scenarios migrated to yaml schedule

Added by JERiveraMoya over 3 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Bugs in existing tests
Target version:
SUSE QA - Milestone 25
Start date:
2019-05-24
Due date:
2019-06-18
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

openQA test in scenario sle-15-SP1-Installer-DVD-x86_64-minimal+base@svirt-hyperv fails in
logs_from_installation_system

Test suite description

Maintainers: okurz

Select a minimal textmode installation by starting with the default and unselecting all patterns except for "base" and "minimal". Not to be confused with the new system role "minimal" introduced with SLE15.

Reproducible

Fails since (at least) Build 228.1 (current job)

Expected result

Last good: 227.1 (or more recent)

Further details

Fix this test by creating specific yaml for minimal+base for hyperv comparing the differences.

Always latest result in this scenario: latest


Related issues

Related to openQA Tests - action #51905: [sle12sp5][desktop] test fails in system_role; caused by YAML_SCHEDULE and SYSTEM_ROLEResolved2019-05-23

History

#1 Updated by JERiveraMoya over 3 years ago

This test suite should be split as well: https://openqa.suse.de/tests/2924478 due to xen-pv differences.

#2 Updated by riafarov over 3 years ago

  • Subject changed from [sle][y] test fails in logs_from_installation_system due to it should not be scheduled to [sle][y][fast] test fails in logs_from_installation_system due to it should not be scheduled
  • Due date set to 2019-06-04
  • Priority changed from Normal to High

We should solve it for GMC, hence increasing the priority.

#3 Updated by riafarov over 3 years ago

  • Subject changed from [sle][y][fast] test fails in logs_from_installation_system due to it should not be scheduled to [sle][y] Fix issues introduced in scenarios migrated to yaml schedule
  • Due date changed from 2019-06-04 to 2019-06-18
  • Target version set to Milestone 25

Alright, so that's the problem of the schedule. I've retriggered it with YAML_SCHEDULE unset: https://openqa.suse.de/t2925810 to make it not that urgent, but we need to address this issue, as yaml schedule for staging didn't work for all.

Now I need to crosscheck if all of them worked fine, as there is an issue that we execute less tests in some case in comparison to main.pm scheduling and list them here.

#4 Updated by JERiveraMoya over 3 years ago

  • Assignee set to JERiveraMoya

I was checking with oorlov that, and the conclusion was that we should keeps the same schedule for staging but for our groups all those tests seems the same than the one that are performed after default, so we decided to excluded (which in some cases with hyperv afteward I was not so sure if was the right choice) but the thing is that some teams are using the wrong test suite to schedule banch of stuff after installing instead of default, but when we separate yaml there is no problem about to still keep it or not, it is just for us to decide at least in our group.

#5 Updated by JERiveraMoya over 3 years ago

  • Status changed from New to Feedback

#6 Updated by riafarov over 3 years ago

#7 Updated by riafarov over 3 years ago

https://openqa.suse.de/tests/2925739 we should use bootloader_start here instead of just bootloader

#9 Updated by riafarov over 3 years ago

https://progress.opensuse.org/issues/51905 could you please crosscheck if that is fixed?

#10 Updated by riafarov over 3 years ago

  • Related to action #51905: [sle12sp5][desktop] test fails in system_role; caused by YAML_SCHEDULE and SYSTEM_ROLE added

#11 Updated by riafarov over 3 years ago

And this one: https://progress.opensuse.org/issues/51911 , both are gnome scenario. I guess you've moved only staging to yaml now, leaving other test suites with main.pm, right?

#12 Updated by JERiveraMoya over 3 years ago

Exactly, that is the idea, first filling up staging, later our own groups and leave rest of the teams unaffected still using main.pm
PR: Fix yaml schedule migration: several scenarios -> merged

#13 Updated by JERiveraMoya over 3 years ago

  • Due date changed from 2019-06-18 to 2019-06-04

#14 Updated by JERiveraMoya over 3 years ago

  • Status changed from Feedback to In Progress

All setup done! huge work of configuration, but I hope it worths, now staging is frozen with staging test suites and we also have our test suites for the jobs that are in staging. Still there is a couple of things to do to make totally plane (without conditions) the schedules.

#15 Updated by JERiveraMoya over 3 years ago

  • Status changed from In Progress to Feedback

#16 Updated by riafarov over 3 years ago

  • Due date changed from 2019-06-04 to 2019-06-18

#17 Updated by JERiveraMoya over 3 years ago

All configured, let's perhaps wait a few days before resolve it in case I could have made some easy manual mistake. Now it looks pretty organized schedule/ folder:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/tree/master/schedule
Naming of the test suites and double "@" it is something to discuss in follow-up tickets plus possibility to automate it in case of huge remaining because at the moment the UI requires intensive clicking due to renaming operation is not possible in some cases.

#18 Updated by JERiveraMoya over 3 years ago

  • Status changed from Feedback to Resolved

#19 Updated by okurz over 3 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: minimal+base@svirt-hyperv
https://openqa.suse.de/tests/2922613

#20 Updated by okurz over 3 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: minimal+base@svirt-hyperv
https://openqa.suse.de/tests/2922613

Also available in: Atom PDF