action #122020
closedcoordination #121876: [epic] Handle openQA review failures in Yam squad - SLE 15 SP5
Try if spvm console can help with the sporadic issues found for PowerVM
Description
Motivation¶
Yam squad in YaST group hits this infrastructure issue: #117808 and apparently there is not expected progress to update that old grub version to make testing more stable. Anyway IBM is testing on its own SLES, and we don't have configured an environment that mimics better how this virtualization is tested.
The goal of this ticket is to leave just one PowerVM job in YaST job group to review more frecuently and move the rest of the jobs to YaST development group.
Before sending new Milestone report Yam squad will check for results in Development group and certain openQA will be applied on that development job group to not miss any bug, but no so frequent or formal like intermediate snapshots or milestones.
Previously we tried to chained jobs to avoid scheduling but it didn't work so well.
Acceptance criteria¶
AC1: Leave only one PowerVM test suite running in YaST job group (excluding flaky tests from having more often review)
AC2: Label properly the failures with corresponding ticket in YaST Development job group.
AC1: Run with backend spvm at least 5 times per each test suite, test suites currently running with backend pvm_hmc and see if we don't hit the bug.
Suggestion¶
guided_btrfs (see here https://openqa.suse.de/tests/10088041#dependencies) is a good candidate to keep in Online medium.
For full flavor we can move them all.
Updated by hjluo about 2 years ago
- MR to remove from daily: https://gitlab.suse.de/qsf-y/qa-sle-functional-y/-/merge_requests/451
- MR to add them to Dev: https://gitlab.suse.de/qsf-y/qa-sle-functional-y/-/merge_requests/457
Updated by JERiveraMoya about 2 years ago
new plan is to test with spvm backend all our pvm_hmc test suites, to verify if our test suite are more stable with this console.
Could you do this as part of this ticket, we can close PR/MR because if it doesn't work with spvm, we need to go back to Power KVM. Otherwise we need to reject this ticket.
Updated by JERiveraMoya about 2 years ago
- Subject changed from Move all test suite running in PowerVM except one to Dev group to Try if spvm console can help with the sporadic issues found for PowerVM
- Description updated (diff)
Updated change of plan.
Updated by hjluo about 2 years ago
Now we clone all available cases in Yast daily 61.1 with MACHINE=ppc64le-spvm BACKEND=spvm WORKER_CLASS=spvm_ppc64le
the test result
https://openqa.suse.de/tests/overview?distri=sle&version=15-SP5&build=hjluo%2Fos-autoinst-distri-opensuse%23spvm_clone
It seems works at least won't blocked at bootloader_start
- Conclusion:
- Just 3 cases failed at https://openqa.suse.de/tests/10296865#step/bootloader_start/19 for the grub>_ does not show the underbar there. this might be a timing issue.
- https://openqa.suse.de/tests/10296865#step/bootloader_start/19 is an issue for we should use the textmode for ppc64le on spmv and a PR is on the way to fix it.
- https://openqa.suse.de/tests/10296717#step/perform_installation/2 is a network issue.
Updated by JERiveraMoya almost 2 years ago
I'm still not convince to move to that old console as workaround, but the rate of failure is less ...
Now we have less builds...so perhaps we can just apply the retry, to avoid doing manually.
Could you please apply a RETRY=5 to the whole architecture for YaST group in a quick MR to evaluate result in next build?
Updated by JERiveraMoya almost 2 years ago
- Related to action #124047: Switch to backend spvm backend for YaST and Migration job groups added
Updated by JERiveraMoya almost 2 years ago
- Status changed from In Progress to Resolved