action #34219
closed[sle][functional][y][yast][medium] New test scenarios: videomode_text+textmode+role_kvm (… and xen)
0%
Description
Motivation¶
Current SLE virtualization tests are very much dependant on a stable installation in ncurses mode selecting the system roles "kvm virtualization host" and "xen virtualization host". Because the tests run on "bare metal" where scalability is limited plus timezone difference between Nbg and Bej a test failure is very costly and we can provide many when we cover at least the ncurses based installation selecting the kvm+xen roles. https://openqa.suse.de/tests/1583951#step/system_role/6 is a recent failure which could have been prevented when [i]QSF[/i] would have seen it earlier.
Acceptance criteria¶
- AC1: There is a new test scenario videomode_text+textmode+role_kvm working on SLE15
- AC2: Same as AC1 but on SLE12SP4
Suggestions¶
- Think hard about a good name
- Add test scenarios to development job group and monitor there
- Add to functional group
- The same for SLE12SP4
Files
Updated by okurz over 6 years ago
- Copied from action #33556: [ipmi]New test scenario: textmode+kvm_server_role@ipmi added
Updated by okurz over 6 years ago
- Status changed from New to In Progress
#33556#note-4 is describing the name confusion. "textmode+role_kvm" is conducting a GUI based installation but is labeled "textmode" because the kvm/xen role selection triggers the default target to be "textmode".
Do be very explicit we could rename "textmode" to "target_textmode+role_kvm" even though the textmode target is the "default" for kvm. Then we can add another explicit scenario "target_textmode+videomode_text+role_kvm". Pro: Explicit, Con: Potentially too long name
An alternative would be just "role_kvm" and add another explicit "videomode_text+role_kvm".
Should it be "videomode_text+textmode+role_kvm"?
Updated by okurz over 6 years ago
- Subject changed from [sle][functional][y][yast]New test scenarios: videomode_text+textmode++role_kvm (… and xen) to [sle][functional][y][yast]New test scenarios: videomode_text+textmode+role_kvm (… and xen)
- Created new testsuite "videomode_text+textmode+role_kvm" on osd
- Added scenario videomode_text+textmode+role_kvm@64bit in Test Development: SLE15
- Added scenario videomode_text+textmode+role_kvm@64bit in Test Development: SLE12 SP4
openqa_clone_job_osd --skip-chained-deps 1607995 VIDEOMODE=text _GROUP="Test Development: SLE 15" TEST=videomode_text+textmode+role_kvm
Created job #1608306: sle-15-Installer-DVD-x86_64-Build561.1-_textmode+role_kvm@64bit -> https://openqa.suse.de/t1608306
That job actually failed in https://openqa.suse.de/tests/1608306#step/system_role/4 because of a needle mismatch on system_role, exactly the errors we want to prevent for ipmi. It seems that we are unfortunate and we can not cover the same needles necessary for both "64bit" as well as "64bit-impi". So probably I will need to add to the ipmi machine right away.
- Added scenario videomode_text+textmode+role_kvm@64bit-ipmi in Test Development: SLE15
more in #33556
and also retriggered the @64bit variant after needle creation with very low match level (52%) in the hope that the ipmi display will not differ too much -> https://openqa.suse.de/tests/1609650 passed
- Moved scenario videomode_text+textmode+role_kvm@64bit to SLE 15 / Functional
- Created new testsuite "videomode_text+textmode+role_xen" on osd
$ openqa_clone_job_osd --skip-chained-deps 1609650 TEST=videomode_text+textmode+role_xen SYSTEM_ROLE=xen
Created job #1610412: sle-15-Installer-DVD-x86_64-Build561.1-videomode_text+textmode+role_kvm@64bit -> https://openqa.suse.de/t1610412
for …xen@64bit
and
$ openqa_clone_job_osd --skip-chained-deps 1590252 VIDEOMODE=text TEST=videomode_text+textmode+role_kvm
Created job #1610414: sle-12-SP4-Server-DVD-x86_64-Build0236-_textmode+role_kvm@64bit -> https://openqa.suse.de/t1610414
for sle12sp4_kvm@64bit
$ openqa_clone_job_osd --skip-chained-deps 1590252 VIDEOMODE=text SYSTEM_ROLE=xen TEST=videomode_text+textmode+role_xen
Created job #1610415: sle-12-SP4-Server-DVD-x86_64-Build0236-_textmode+role_kvm@64bit -> https://openqa.suse.de/t1610415
for sle12sp4_xen@64bit
- Wait for https://openqa.suse.de/t1610412 (64bit) -> needle created, https://openqa.suse.de/tests/1622545#live, needle created, https://openqa.suse.de/tests/1637989#live soft-failed
- Add scenarios to functional for xen if successful -> done
Next steps:
- Wait for https://openqa.suse.de/t1610414 (sle12sp4_kvm@64bit) -> failed trying to change desktop, need to set desktop explicitly?
- Wait for https://openqa.suse.de/t1610415 (sles12sp4_xen@64bit) -> incomplete, retriggered https://openqa.suse.de/tests/1637990#live -> same as above, failed in change_desktop
- Fix the desktop selection for kvm/xen role in videomode_text for SLE12SP4
- Schedule for SLE12SP4 or check results in test development
I can continue this but not anymore today so moving
Updated by okurz over 6 years ago
- Due date changed from 2018-04-24 to 2018-05-08
- Target version changed from Milestone 15 to Milestone 16
Updated by okurz over 6 years ago
- Subject changed from [sle][functional][y][yast]New test scenarios: videomode_text+textmode+role_kvm (… and xen) to [sle][functional][y][yast] New test scenarios: videomode_text+textmode+role_kvm (… and xen)
- Due date changed from 2018-05-08 to 2018-05-22
- Status changed from In Progress to Workable
- Assignee deleted (
okurz)
Had to pick up other more urgent tasks so was not able to continue so planning for next sprint and unassigning.
Next steps:
- See https://openqa.suse.de/t1610414 (sle12sp4_kvm@64bit) -> failed trying to change desktop
- Fix the desktop selection for kvm/xen role in videomode_text for SLE12SP4 (needs only local test with isotovideo, do not waste time with openQA)
- Schedule for SLE12SP4 or check results in test development
Updated by riafarov over 6 years ago
- Subject changed from [sle][functional][y][yast] New test scenarios: videomode_text+textmode+role_kvm (… and xen) to [sle][functional][y][yast][medium] New test scenarios: videomode_text+textmode+role_kvm (… and xen)
Updated by mloviska over 6 years ago
- Status changed from Workable to In Progress
* See https://openqa.suse.de/t1610414 (sle12sp4_kvm@64bit) -> failed trying to change desktop --> DONE
* Fix the desktop selection for kvm/xen role in videomode_text for SLE12SP4 (needs only local test with isotovideo, do not waste time with openQA)
- local verification: sle-12-SP4-Server-DVD-x86_64-Build0236-videomode_text+textmode+role_kvm@64bit
Updated by mloviska over 6 years ago
- File sp4_kvm_origin sp4_kvm_origin added
- File sp4_xen_origin sp4_xen_origin added
- File sp4_xen_new sp4_xen_new added
- File sp4_kvm_new sp4_kvm_new added
➜ sp4_xen echo $role
xen
➜ sp4_xen diff sp4_${role}_origin sp4_${role}_new
7c7
< default desktop: gnome
---
> default desktop: textmode
36,37d35
< scheduling installation_overview_before tests/installation/installation_overview_before.pm
< scheduling change_desktop tests/installation/change_desktop.pm
80,81c78,79
< commands process exited: 31698
< 31693: EXIT 1
---
> commands process exited: 797
> 793: EXIT 1
➜ sp4_kvm echo $role
kvm
➜ sp4_kvm diff sp4_${role}_origin sp4_${role}_new
7c7
< default desktop: gnome
---
> default desktop: textmode
36,37d35
< scheduling installation_overview_before tests/installation/installation_overview_before.pm
< scheduling change_desktop tests/installation/change_desktop.pm
80,81c78,79
< commands process exited: 31609
< 31607: EXIT 1
---
> commands process exited: 935
> 933: EXIT 1
Updated by mloviska over 6 years ago
- Status changed from In Progress to Feedback
Updated by riafarov over 6 years ago
https://openqa.suse.de/tests/1705220 xen role (failed due to missing repo setting in isos post call)
https://openqa.suse.de/tests/1695367 kvm role (moved to functional)
Updated by mloviska over 6 years ago
- Status changed from Feedback to Resolved