action #43784
closed[functional][y][sporadic] test fails in yast2_snapper now reproducibly not exiting the "show differences" screen
0%
Description
Observation¶
openQA test in scenario opensuse-15.1-NET-x86_64-update_Leap_42.3_kde+system_performance@64bit-2G fails in
yast2_snapper
Can be related to the performance of the system.
Similar issue on SLE.
Reproducible¶
Fails since (at least) Build 345.2
Expected result¶
Last good: 332.2 (or more recent)
Further details¶
Always latest result in this scenario: latest
Observation¶
openQA test in scenario sle-12-SP4-Server-DVD-ppc64le-system_performance@ppc64le fails in
yast2_snapper
Apparently for a moment the content of the dialog disappears and come back. It could be some load in the worker but we might want assert for more time the last item of the treeview before to iterate sending key up, so we ensure to match the needle. It could be buggy, but there is not video, so if we can reproduce we could file it.
Reproducible¶
Fails since (at least) Build 0455 (current job)
Expected result¶
Last good: 0454 (or more recent)
Further details¶
Always latest result in this scenario: latest
Updated by okurz over 5 years ago
- Subject changed from test fails in yast2_snapper to [functional][y] test fails in yast2_snapper now reproducibly not exiting the "show differences" screen
- Due date set to 2018-12-04
- Status changed from New to Workable
- Priority changed from Normal to High
- Target version set to Milestone 20
Updated by riafarov over 5 years ago
- Assignee deleted (
riafarov)
Was not able to work on this one, un-assign myself to give a chance to the team.
Updated by riafarov over 5 years ago
- Description updated (diff)
- Estimated time set to 5.00 h
Updated by oorlov over 5 years ago
- Status changed from Workable to In Progress
- Assignee set to oorlov
Updated by okurz over 5 years ago
- Target version changed from Milestone 20 to Milestone 21
Updated by riafarov over 5 years ago
- Subject changed from [functional][y] test fails in yast2_snapper now reproducibly not exiting the "show differences" screen to [functional][y][sporadic] test fails in yast2_snapper now reproducibly not exiting the "show differences" screen
- Due date changed from 2018-12-04 to 2018-12-18
We trigger this test in a lot of scenarios. In general key press is lost and step is not asserted properly. Let's make failure obvious.
Updated by okurz over 5 years ago
- Related to coordination #39719: [saga][epic] Detection of "known failures" for stable tests, easy test results review and easy tracking of known issues added
Updated by okurz over 5 years ago
- Related to action #42536: [functional][u] test fails in force_scheduled_tasks - "select_console 'root-console'" failed because of btrfs info message on the screen which should not be there but this very module should prevent further messages and therefore handle this gracefully added
Updated by okurz over 5 years ago
- Related to action #41459: [sle][functional][u] Explicit test module for btrfs snapshots cleanup performance added
Updated by oorlov over 5 years ago
- Status changed from In Progress to Feedback
Updated by oorlov over 5 years ago
- Status changed from Feedback to Resolved
Passed on OSD: https://openqa.suse.de/tests/2326130#step/yast2_snapper/58
Updated by okurz over 5 years ago
- Status changed from Resolved to Feedback
please try again…
how is a SLE15SP1 test related to Leap upgrade failing?
Updated by oorlov over 5 years ago
- Status changed from Feedback to Resolved
Actually, it was not only opensuse problem, even in the description there is a test failed on Sle12.
But if referring to opensuse, passed test there: https://openqa.opensuse.org/tests/latest?test=update_Leap_42.3_kde%2Bsystem_performance&arch=x86_64&flavor=NET&version=15.1&machine=64bit-2G&distri=opensuse#step/yast2_snapper/55
Updated by okurz over 5 years ago
- Status changed from Resolved to Feedback
Yes, but the ticket is "sporadic". Please see https://openqa.opensuse.org/tests/latest?test=update_Leap_42.3_kde%2Bsystem_performance&arch=x86_64&flavor=NET&version=15.1&machine=64bit-2G&distri=opensuse#next_previous and you are able to find yast2_snapper failing e.g. in the job before the last one. I don't see how a single verification run can suffice. Could you either trigger some more tests to verify with statistics or just keep it open until we had more builds that show stable behaviour? If you prefer to open another ticket, ok as well, you can go ahead.
If you prefer to talk about the ticket or how we handle it in a different communication channel I am happy to do so, e.g. in person.
Updated by oorlov over 5 years ago
I've checked logs of the job you've mentioned and it seems like it used the old code.
But I'm totally agree that in order to check if my changes really help to solve the problem, we need more statistic. So, I'll execute several jobs in Development group.
Updated by oorlov over 5 years ago
So, according to what we discussed during one of the refinement meetings (please, see comment #8), I've made the fail more obvious.
Now an appropriate error message will be shown, if the window will not be closed after sending 'alt-c' key combination (which was the problem).
PR: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6413
Updated by oorlov over 5 years ago
The example of failed test module https://openqa.opensuse.org/tests/816742#step/yast2_snapper/44
Updated by riafarov over 5 years ago
- Due date deleted (
2018-12-18) - Status changed from Feedback to Blocked
- Assignee changed from oorlov to okurz
As discussed, okurz has proposal to make job soft-fails when issue is detected. riafarov proposes running test module in separate test suite.
Updated by okurz over 5 years ago
- Target version changed from Milestone 21 to future
blocked by #39719
Updated by riafarov about 5 years ago
- Status changed from Blocked to Resolved
- Assignee changed from okurz to oorlov
Seems that issue is gone for now. However, we should consider changing scenario to have changes we can predict. For example doing one snapshot, changing single directory and check diff, instead of using unpredictable diff of currently created snapshot and the one which was there before.