Project

General

Profile

action #34219

[sle][functional][y][yast][medium] New test scenarios: videomode_text+textmode+role_kvm (… and xen)

Added by okurz about 2 years ago. Updated about 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
New test
Start date:
2018-03-21
Due date:
2018-05-22
% Done:

0%

Estimated time:
Difficulty:
Duration: 45

Description

Motivation

Current SLE virtualization tests are very much dependant on a stable installation in ncurses mode selecting the system roles "kvm virtualization host" and "xen virtualization host". Because the tests run on "bare metal" where scalability is limited plus timezone difference between Nbg and Bej a test failure is very costly and we can provide many when we cover at least the ncurses based installation selecting the kvm+xen roles. https://openqa.suse.de/tests/1583951#step/system_role/6 is a recent failure which could have been prevented when [i]QSF[/i] would have seen it earlier.

Acceptance criteria

  • AC1: There is a new test scenario videomode_text+textmode+role_kvm working on SLE15
  • AC2: Same as AC1 but on SLE12SP4

Suggestions

  • Think hard about a good name
  • Add test scenarios to development job group and monitor there
  • Add to functional group
  • The same for SLE12SP4
sp4_kvm_origin (4.08 KB) sp4_kvm_origin mloviska, 2018-05-15 11:50
sp4_xen_origin (4.08 KB) sp4_xen_origin mloviska, 2018-05-15 11:52
sp4_xen_new (3.92 KB) sp4_xen_new mloviska, 2018-05-15 12:15
sp4_kvm_new (3.92 KB) sp4_kvm_new mloviska, 2018-05-15 12:16

Related issues

Copied from openQA Tests - action #33556: [sle][functional][y][yast][ipmi]New test scenario: textmode+kvm_server_role@ipmiBlocked2018-03-21

History

#1 Updated by okurz about 2 years ago

  • Copied from action #33556: [sle][functional][y][yast][ipmi]New test scenario: textmode+kvm_server_role@ipmi added

#2 Updated by okurz about 2 years ago

  • Status changed from New to In Progress

#33556#note-4 is describing the name confusion. "textmode+role_kvm" is conducting a GUI based installation but is labeled "textmode" because the kvm/xen role selection triggers the default target to be "textmode".

Do be very explicit we could rename "textmode" to "target_textmode+role_kvm" even though the textmode target is the "default" for kvm. Then we can add another explicit scenario "target_textmode+videomode_text+role_kvm". Pro: Explicit, Con: Potentially too long name

An alternative would be just "role_kvm" and add another explicit "videomode_text+role_kvm".

Should it be "videomode_text+textmode+role_kvm"?

#3 Updated by okurz about 2 years ago

  • Subject changed from [sle][functional][y][yast]New test scenarios: videomode_text+textmode++role_kvm (… and xen) to [sle][functional][y][yast]New test scenarios: videomode_text+textmode+role_kvm (… and xen)
openqa_clone_job_osd --skip-chained-deps 1607995 VIDEOMODE=text _GROUP="Test Development: SLE 15" TEST=videomode_text+textmode+role_kvm

Created job #1608306: sle-15-Installer-DVD-x86_64-Build561.1-_textmode+role_kvm@64bit -> https://openqa.suse.de/t1608306

That job actually failed in https://openqa.suse.de/tests/1608306#step/system_role/4 because of a needle mismatch on system_role, exactly the errors we want to prevent for ipmi. It seems that we are unfortunate and we can not cover the same needles necessary for both "64bit" as well as "64bit-impi". So probably I will need to add to the ipmi machine right away.

more in #33556

and also retriggered the @64bit variant after needle creation with very low match level (52%) in the hope that the ipmi display will not differ too much -> https://openqa.suse.de/tests/1609650 passed

  • Moved scenario videomode_text+textmode+role_kvm@64bit to SLE 15 / Functional
  • Created new testsuite "videomode_text+textmode+role_xen" on osd
$ openqa_clone_job_osd --skip-chained-deps 1609650 TEST=videomode_text+textmode+role_xen SYSTEM_ROLE=xen

Created job #1610412: sle-15-Installer-DVD-x86_64-Build561.1-videomode_text+textmode+role_kvm@64bit -> https://openqa.suse.de/t1610412
for …xen@64bit

and

$ openqa_clone_job_osd --skip-chained-deps 1590252 VIDEOMODE=text TEST=videomode_text+textmode+role_kvm

Created job #1610414: sle-12-SP4-Server-DVD-x86_64-Build0236-_textmode+role_kvm@64bit -> https://openqa.suse.de/t1610414
for sle12sp4_kvm@64bit

$ openqa_clone_job_osd --skip-chained-deps 1590252 VIDEOMODE=text SYSTEM_ROLE=xen TEST=videomode_text+textmode+role_xen

Created job #1610415: sle-12-SP4-Server-DVD-x86_64-Build0236-_textmode+role_kvm@64bit -> https://openqa.suse.de/t1610415
for sle12sp4_xen@64bit

Next steps:

I can continue this but not anymore today so moving

#4 Updated by okurz about 2 years ago

  • Due date changed from 2018-04-24 to 2018-05-08
  • Target version changed from Milestone 15 to Milestone 16

#5 Updated by okurz about 2 years ago

  • Subject changed from [sle][functional][y][yast]New test scenarios: videomode_text+textmode+role_kvm (… and xen) to [sle][functional][y][yast] New test scenarios: videomode_text+textmode+role_kvm (… and xen)
  • Due date changed from 2018-05-08 to 2018-05-22
  • Status changed from In Progress to Workable
  • Assignee deleted (okurz)

Had to pick up other more urgent tasks so was not able to continue so planning for next sprint and unassigning.

Next steps:

  • See https://openqa.suse.de/t1610414 (sle12sp4_kvm@64bit) -> failed trying to change desktop
  • Fix the desktop selection for kvm/xen role in videomode_text for SLE12SP4 (needs only local test with isotovideo, do not waste time with openQA)
  • Schedule for SLE12SP4 or check results in test development

#6 Updated by riafarov about 2 years ago

  • Subject changed from [sle][functional][y][yast] New test scenarios: videomode_text+textmode+role_kvm (… and xen) to [sle][functional][y][yast][medium] New test scenarios: videomode_text+textmode+role_kvm (… and xen)

#7 Updated by mloviska about 2 years ago

  • Assignee set to mloviska

#8 Updated by mloviska about 2 years ago

  • Status changed from Workable to In Progress

* See https://openqa.suse.de/t1610414 (sle12sp4_kvm@64bit) -> failed trying to change desktop --> DONE
* Fix the desktop selection for kvm/xen role in videomode_text for SLE12SP4 (needs only local test with isotovideo, do not waste time with openQA)

#9 Updated by mloviska about 2 years ago

➜  sp4_xen echo $role  
xen

➜  sp4_xen diff sp4_${role}_origin sp4_${role}_new 
7c7
<  default desktop: gnome
---
>  default desktop: textmode
36,37d35
<  scheduling installation_overview_before tests/installation/installation_overview_before.pm
<  scheduling change_desktop tests/installation/change_desktop.pm
80,81c78,79
<  commands process exited: 31698
< 31693: EXIT 1
---
>  commands process exited: 797
> 793: EXIT 1
➜  sp4_kvm echo $role
kvm

➜  sp4_kvm diff sp4_${role}_origin sp4_${role}_new                
7c7
<  default desktop: gnome
---
>  default desktop: textmode
36,37d35
<  scheduling installation_overview_before tests/installation/installation_overview_before.pm
<  scheduling change_desktop tests/installation/change_desktop.pm
80,81c78,79
<  commands process exited: 31609
< 31607: EXIT 1
---
>  commands process exited: 935
> 933: EXIT 1

PR:
Reorder System Role evaluation in default_desktop #5052

#10 Updated by mloviska about 2 years ago

  • Status changed from In Progress to Feedback

#11 Updated by riafarov about 2 years ago

https://openqa.suse.de/tests/1705220 xen role (failed due to missing repo setting in isos post call)
https://openqa.suse.de/tests/1695367 kvm role (moved to functional)

Also available in: Atom PDF