Project

General

Profile

action #43784

[functional][y][sporadic] test fails in yast2_snapper now reproducibly not exiting the "show differences" screen

Added by lnussel almost 3 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Bugs in existing tests
Target version:
Start date:
2018-11-14
Due date:
% Done:

0%

Estimated time:
5.00 h
Difficulty:

Description

Observation

openQA test in scenario opensuse-15.1-NET-x86_64-update_Leap_42.3_kde+system_performance@64bit-2G fails in
yast2_snapper

Can be related to the performance of the system.

Similar issue on SLE.

Reproducible

Fails since (at least) Build 345.2

Expected result

Last good: 332.2 (or more recent)

Further details

Always latest result in this scenario: latest

Observation

openQA test in scenario sle-12-SP4-Server-DVD-ppc64le-system_performance@ppc64le fails in
yast2_snapper

Apparently for a moment the content of the dialog disappears and come back. It could be some load in the worker but we might want assert for more time the last item of the treeview before to iterate sending key up, so we ensure to match the needle. It could be buggy, but there is not video, so if we can reproduce we could file it.

Reproducible

Fails since (at least) Build 0455 (current job)

Expected result

Last good: 0454 (or more recent)

Further details

Always latest result in this scenario: latest


Related issues

Related to openQA Project - coordination #39719: [saga][epic] Detection of "known failures" for stable tests, easy test results review and easy tracking of known issuesBlocked2017-01-232021-09-24

Related to openQA Tests - action #42536: [functional][u] test fails in force_scheduled_tasks - "select_console 'root-console'" failed because of btrfs info message on the screen which should not be there but this very module should prevent further messages and therefore handle this gracefullyRejected2018-10-16

Related to openQA Tests - action #41459: [sle][functional][u] Explicit test module for btrfs snapshots cleanup performanceRejected2018-08-01

History

#1 Updated by okurz almost 3 years ago

  • Subject changed from test fails in yast2_snapper to [functional][y] test fails in yast2_snapper now reproducibly not exiting the "show differences" screen
  • Due date set to 2018-12-04
  • Status changed from New to Workable
  • Priority changed from Normal to High
  • Target version set to Milestone 20

#2 Updated by riafarov almost 3 years ago

  • Assignee set to riafarov

#3 Updated by riafarov almost 3 years ago

  • Assignee deleted (riafarov)

Was not able to work on this one, un-assign myself to give a chance to the team.

#4 Updated by riafarov almost 3 years ago

  • Description updated (diff)
  • Estimated time set to 5.00 h

#5 Updated by riafarov almost 3 years ago

  • Description updated (diff)

#6 Updated by oorlov almost 3 years ago

  • Status changed from Workable to In Progress
  • Assignee set to oorlov

#7 Updated by okurz almost 3 years ago

  • Target version changed from Milestone 20 to Milestone 21

#8 Updated by riafarov almost 3 years ago

  • Subject changed from [functional][y] test fails in yast2_snapper now reproducibly not exiting the "show differences" screen to [functional][y][sporadic] test fails in yast2_snapper now reproducibly not exiting the "show differences" screen
  • Due date changed from 2018-12-04 to 2018-12-18

We trigger this test in a lot of scenarios. In general key press is lost and step is not asserted properly. Let's make failure obvious.

#10 Updated by okurz almost 3 years ago

  • Related to coordination #39719: [saga][epic] Detection of "known failures" for stable tests, easy test results review and easy tracking of known issues added

#11 Updated by okurz almost 3 years ago

  • Related to action #42536: [functional][u] test fails in force_scheduled_tasks - "select_console 'root-console'" failed because of btrfs info message on the screen which should not be there but this very module should prevent further messages and therefore handle this gracefully added

#13 Updated by okurz almost 3 years ago

  • Related to action #41459: [sle][functional][u] Explicit test module for btrfs snapshots cleanup performance added

#16 Updated by oorlov almost 3 years ago

  • Status changed from In Progress to Feedback

#17 Updated by oorlov almost 3 years ago

  • Status changed from Feedback to Resolved

#18 Updated by okurz almost 3 years ago

  • Status changed from Resolved to Feedback

please try again…

how is a SLE15SP1 test related to Leap upgrade failing?

#19 Updated by oorlov almost 3 years ago

  • Status changed from Feedback to Resolved

Actually, it was not only opensuse problem, even in the description there is a test failed on Sle12.

But if referring to opensuse, passed test there: https://openqa.opensuse.org/tests/latest?test=update_Leap_42.3_kde%2Bsystem_performance&arch=x86_64&flavor=NET&version=15.1&machine=64bit-2G&distri=opensuse#step/yast2_snapper/55

#20 Updated by okurz almost 3 years ago

  • Status changed from Resolved to Feedback

Yes, but the ticket is "sporadic". Please see https://openqa.opensuse.org/tests/latest?test=update_Leap_42.3_kde%2Bsystem_performance&arch=x86_64&flavor=NET&version=15.1&machine=64bit-2G&distri=opensuse#next_previous and you are able to find yast2_snapper failing e.g. in the job before the last one. I don't see how a single verification run can suffice. Could you either trigger some more tests to verify with statistics or just keep it open until we had more builds that show stable behaviour? If you prefer to open another ticket, ok as well, you can go ahead.
If you prefer to talk about the ticket or how we handle it in a different communication channel I am happy to do so, e.g. in person.

#21 Updated by oorlov almost 3 years ago

I've checked logs of the job you've mentioned and it seems like it used the old code.

But I'm totally agree that in order to check if my changes really help to solve the problem, we need more statistic. So, I'll execute several jobs in Development group.

#23 Updated by oorlov almost 3 years ago

So, according to what we discussed during one of the refinement meetings (please, see comment #8), I've made the fail more obvious.

Now an appropriate error message will be shown, if the window will not be closed after sending 'alt-c' key combination (which was the problem).

PR: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6413

#25 Updated by riafarov almost 3 years ago

  • Due date deleted (2018-12-18)
  • Status changed from Feedback to Blocked
  • Assignee changed from oorlov to okurz

As discussed, okurz has proposal to make job soft-fails when issue is detected. riafarov proposes running test module in separate test suite.

#26 Updated by okurz over 2 years ago

  • Target version changed from Milestone 21 to future

blocked by #39719

#27 Updated by riafarov over 2 years ago

  • Status changed from Blocked to Resolved
  • Assignee changed from okurz to oorlov

Seems that issue is gone for now. However, we should consider changing scenario to have changes we can predict. For example doing one snapshot, changing single directory and check diff, instead of using unpredictable diff of currently created snapshot and the one which was there before.

Also available in: Atom PDF