action #43784

[functional][y][sporadic] test fails in yast2_snapper now reproducibly not exiting the "show differences" screen

Added by lnussel over 1 year ago. Updated 10 months ago.

Status:ResolvedStart date:14/11/2018
Priority:HighDue date:
Assignee:oorlov% Done:

0%

Category:Bugs in existing testsEstimated time:5.00 hours
Target version:QA - future
Difficulty:
Duration:

Description

Observation

openQA test in scenario opensuse-15.1-NET-x86_64-update_Leap_42.3_kde+system_performance@64bit-2G fails in
yast2_snapper

Can be related to the performance of the system.

Similar issue on SLE.

Reproducible

Fails since (at least) Build 345.2

Expected result

Last good: 332.2 (or more recent)

Further details

Always latest result in this scenario: latest

Observation

openQA test in scenario sle-12-SP4-Server-DVD-ppc64le-system_performance@ppc64le fails in
yast2_snapper

Apparently for a moment the content of the dialog disappears and come back. It could be some load in the worker but we might want assert for more time the last item of the treeview before to iterate sending key up, so we ensure to match the needle. It could be buggy, but there is not video, so if we can reproduce we could file it.

Reproducible

Fails since (at least) Build 0455 (current job)

Expected result

Last good: 0454 (or more recent)

Further details

Always latest result in this scenario: latest


Related issues

Related to openQA Project - action #39719: [epic] Detect "known failures" and mark jobs as such Blocked 23/05/2018 31/12/2020
Related to openQA Tests - action #42536: [functional][u] test fails in force_scheduled_tasks - "se... New 16/10/2018
Related to openQA Tests - action #41459: [sle][functional][u] Explicit test module for btrfs snaps... Rejected 01/08/2018

History

#1 Updated by okurz over 1 year ago

  • Subject changed from test fails in yast2_snapper to [functional][y] test fails in yast2_snapper now reproducibly not exiting the "show differences" screen
  • Due date set to 04/12/2018
  • Status changed from New to Workable
  • Priority changed from Normal to High
  • Target version set to Milestone 20

#2 Updated by riafarov over 1 year ago

  • Assignee set to riafarov

#3 Updated by riafarov over 1 year ago

  • Assignee deleted (riafarov)

Was not able to work on this one, un-assign myself to give a chance to the team.

#4 Updated by riafarov over 1 year ago

  • Description updated (diff)
  • Estimated time set to 5.00

#5 Updated by riafarov over 1 year ago

  • Description updated (diff)

#6 Updated by oorlov over 1 year ago

  • Status changed from Workable to In Progress
  • Assignee set to oorlov

#7 Updated by okurz about 1 year ago

  • Target version changed from Milestone 20 to Milestone 21

#8 Updated by riafarov about 1 year ago

  • Subject changed from [functional][y] test fails in yast2_snapper now reproducibly not exiting the "show differences" screen to [functional][y][sporadic] test fails in yast2_snapper now reproducibly not exiting the "show differences" screen
  • Due date changed from 04/12/2018 to 18/12/2018

We trigger this test in a lot of scenarios. In general key press is lost and step is not asserted properly. Let's make failure obvious.

#10 Updated by okurz about 1 year ago

  • Related to action #39719: [epic] Detect "known failures" and mark jobs as such added

#11 Updated by okurz about 1 year ago

  • Related to action #42536: [functional][u] test fails in force_scheduled_tasks - "select_console 'root-console'" failed because of btrfs info message on the screen which should not be there but this very module should prevent further messages and therefore handle this gracefully added

#13 Updated by okurz about 1 year ago

  • Related to action #41459: [sle][functional][u] Explicit test module for btrfs snapshots cleanup performance added

#16 Updated by oorlov about 1 year ago

  • Status changed from In Progress to Feedback

#17 Updated by oorlov about 1 year ago

  • Status changed from Feedback to Resolved

#18 Updated by okurz about 1 year ago

  • Status changed from Resolved to Feedback

please try again…

how is a SLE15SP1 test related to Leap upgrade failing?

#19 Updated by oorlov about 1 year ago

  • Status changed from Feedback to Resolved

Actually, it was not only opensuse problem, even in the description there is a test failed on Sle12.

But if referring to opensuse, passed test there: https://openqa.opensuse.org/tests/latest?test=update_Leap_42.3_kde%2Bsystem_performance&arch=x86_64&flavor=NET&version=15.1&machine=64bit-2G&distri=opensuse#step/yast2_snapper/55

#20 Updated by okurz about 1 year ago

  • Status changed from Resolved to Feedback

Yes, but the ticket is "sporadic". Please see https://openqa.opensuse.org/tests/latest?test=update_Leap_42.3_kde%2Bsystem_performance&arch=x86_64&flavor=NET&version=15.1&machine=64bit-2G&distri=opensuse#next_previous and you are able to find yast2_snapper failing e.g. in the job before the last one. I don't see how a single verification run can suffice. Could you either trigger some more tests to verify with statistics or just keep it open until we had more builds that show stable behaviour? If you prefer to open another ticket, ok as well, you can go ahead.
If you prefer to talk about the ticket or how we handle it in a different communication channel I am happy to do so, e.g. in person.

#21 Updated by oorlov about 1 year ago

I've checked logs of the job you've mentioned and it seems like it used the old code.

But I'm totally agree that in order to check if my changes really help to solve the problem, we need more statistic. So, I'll execute several jobs in Development group.

#23 Updated by oorlov about 1 year ago

So, according to what we discussed during one of the refinement meetings (please, see comment #8), I've made the fail more obvious.

Now an appropriate error message will be shown, if the window will not be closed after sending 'alt-c' key combination (which was the problem).

PR: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6413

#25 Updated by riafarov about 1 year ago

  • Due date deleted (18/12/2018)
  • Status changed from Feedback to Blocked
  • Assignee changed from oorlov to okurz

As discussed, okurz has proposal to make job soft-fails when issue is detected. riafarov proposes running test module in separate test suite.

#26 Updated by okurz about 1 year ago

  • Target version changed from Milestone 21 to future

blocked by #39719

#27 Updated by riafarov 10 months ago

  • Status changed from Blocked to Resolved
  • Assignee changed from okurz to oorlov

Seems that issue is gone for now. However, we should consider changing scenario to have changes we can predict. For example doing one snapshot, changing single directory and check diff, instead of using unpredictable diff of currently created snapshot and the one which was there before.

Also available in: Atom PDF