Project

General

Profile

action #35685

coordination #35302: [qe-core][opensuse][functional][epic][sporadic] Various unstable tests on o3

[opensuse][functional][u][sporadic][medium] Various unstable tests on o3 - user_gui_login + gnucash

Added by okurz about 4 years ago. Updated about 2 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Bugs in existing tests
Target version:
SUSE QA - Milestone 17
Start date:
2018-04-30
Due date:
% Done:

0%

Estimated time:
8.00 h
Difficulty:

Description

Observation

gnucash fails from time to time in various scenarios, mainly upgrade scenarios, e.g. see

The SUT locks the screen before finishing the test
All examples are update tests, we should keep that in mind

Acceptance criteria

AC1: Tests do not end in lockscreen anymore

Suggestions

  • Evaluate test failures from #35685#note-37
  • Crosscheck with 100 tests on local worker
  • If more tests fail on production than on local worker make sure that production workers are more stable, e.g. #38150

Related issues

Related to openQA Tests - action #34717: [functional][u][easy][fast] adjust gnucash test per gnucash 3.0 updateResolved2018-04-112018-06-19

Related to openQA Tests - action #35604: [opensuse][functional][u] test fails in gnucash, not closing correctlyResolved2018-04-262018-07-17

Related to openSUSE Leap 15.0 - action #25032: enable openQA maintenance testingResolved2018-04-012018-04-19

Related to openQA Tests - action #37450: gnucash test fails to work due to gnucash 3.1 uninstallableResolved2018-06-17

Related to openQA Tests - action #38114: [functional][u] GnuCash does not close on LXDEResolved2018-07-03

Has duplicate openQA Tests - action #37555: [opensuse][functional][u] test fails in user_gui_login - relogin doesn't reach display managerRejected2018-06-20

History

#1 Updated by okurz about 4 years ago

  • Related to action #34717: [functional][u][easy][fast] adjust gnucash test per gnucash 3.0 update added

#2 Updated by okurz about 4 years ago

  • Due date changed from 2018-05-22 to 2018-06-05

Should follow #34717

#3 Updated by okurz about 4 years ago

  • Related to action #35604: [opensuse][functional][u] test fails in gnucash, not closing correctly added

#4 Updated by okurz about 4 years ago

  • Related to action #25032: enable openQA maintenance testing added

#5 Updated by SLindoMansilla about 4 years ago

  • Subject changed from [opensuse][functional][u][sporadic] Various unstable tests on o3 - gnucash to [opensuse][functional][u][sporadic][medium] Various unstable tests on o3 - gnucash

#6 Updated by jorauch about 4 years ago

  • Status changed from Workable to In Progress
  • Assignee set to jorauch

#7 Updated by jorauch about 4 years ago

  • Description updated (diff)

How was this even workable without AC and description?

#8 Updated by okurz about 4 years ago

jorauch wrote:

How was this even workable without AC and description?

For the category "Bugs in existing tests" we normally do not specify ACs additional to the DoD. Therefore the implicit ACs are something like "Test passes or fails in valid product bugs"

#9 Updated by jorauch about 4 years ago

Latest failure
https://openqa.opensuse.org/tests/679623#step/gnucash/48

We have a black screen before the SUT locks itself, is this normal behavior?
https://openqa.opensuse.org/tests/679623/file/video.ogv#t=287.42,287.46

#10 Updated by jorauch almost 4 years ago

Not really sporadic, happens quite often:
https://openqa.opensuse.org/tests/684294#step/gnucash/49

#11 Updated by mgriessmeier almost 4 years ago

  • Due date changed from 2018-06-05 to 2018-06-19
  • Status changed from In Progress to Workable
  • Assignee deleted (jorauch)
  • Target version changed from Milestone 16 to Milestone 17

#12 Updated by dheidler almost 4 years ago

  • Assignee set to dheidler

#13 Updated by okurz almost 4 years ago

keep in mind that zluo is working on #34717 atm

#14 Updated by dheidler almost 4 years ago

The question is why the screensaver kicks in, if less than one minute ago the some programm was started using keyboard commands.

#16 Updated by dheidler almost 4 years ago

  • Status changed from Workable to In Progress

#17 Updated by dheidler almost 4 years ago

Only one of the 100 tests that I scheduled had the problem: https://openqa.opensuse.org/tests/690596#step/gnucash/47

#18 Updated by dheidler almost 4 years ago

okurz: shall we close this?

#19 Updated by okurz almost 4 years ago

Make it fail less than 1/1000 and we can close it.

#20 Updated by dheidler almost 4 years ago

Do you want me to schedule 1000 jobs on o3?

#21 Updated by okurz almost 4 years ago

Yes please. Especially because it is just before the weekend and there is less load on the system on a weekend anyway. Moreover, maxlin is probably already done with his work day and DimStar on holiday and the last TW snapshots have bigger problems and most likely will not be released soon, perfect timing :-)

#23 Updated by okurz almost 4 years ago

  • Target version changed from Milestone 17 to Milestone 17

#24 Updated by dheidler almost 4 years ago

  • Target version deleted (Milestone 17)

Out of 1000 Jobs roughly 40% were canceled for some reason.
From the rest I found the following five jobs that had the gnucash screensaver issue:

https://openqa.opensuse.org/t692009
https://openqa.opensuse.org/t692052
https://openqa.opensuse.org/t692055
https://openqa.opensuse.org/t692195
https://openqa.opensuse.org/t692271

#25 Updated by mgriessmeier almost 4 years ago

  • Target version set to Milestone 17

#26 Updated by okurz almost 4 years ago

  • Subject changed from [opensuse][functional][u][sporadic][medium] Various unstable tests on o3 - gnucash to [opensuse][functional][u][sporadic][medium] Various unstable tests on o3 - user_gui_login + gnucash
  • Description updated (diff)
  • Due date changed from 2018-06-19 to 2018-07-03
  • Status changed from In Progress to Workable
  • Assignee deleted (dheidler)

from the test statistics the observation from #35685#note-14 still holds. By the way, of 1k jobs none passed, most if not all failed in "zypper_ar", many in "user_gui_login", e.g. see https://openqa.opensuse.org/tests/692009#step/user_gui_login/13. The latter could be related because the problem in gnucash is about the display manager or screen saver as well. This should be addressed first. I see no point in not relying on the information we have here so I extended the summary to mention "user_gui_login" as well. The acceptance criterion still holds true.

#27 Updated by okurz almost 4 years ago

  • Has duplicate action #37555: [opensuse][functional][u] test fails in user_gui_login - relogin doesn't reach display manager added

#28 Updated by okurz almost 4 years ago

From duplicate:
openQA test in scenario opensuse-Tumbleweed-NET-x86_64-update_Leap_42.3_gnome@64bit fails in
user_gui_login
Always latest result in this scenario: latest

#29 Updated by SLindoMansilla almost 4 years ago

  • Status changed from Workable to In Progress
  • Assignee set to SLindoMansilla

#30 Updated by SLindoMansilla almost 4 years ago

Some of the failures I can see are caused by a dependency error like this: https://openqa.opensuse.org/tests/693346#step/gnucash/12

This error was handle on https://progress.opensuse.org/issues/37450

#31 Updated by SLindoMansilla almost 4 years ago

  • Related to action #37450: gnucash test fails to work due to gnucash 3.1 uninstallable added

#32 Updated by okurz almost 4 years ago

I have the suspicion that most if not all unstable test modules fail due background maintenance tasks which are triggered during random time when test modules are executed so we should do #31351 first. Feel free to set to "Blocked", keep assigned or assign to me and set to sprint 22, i.e. M18 with due date 2018-07-31

#33 Updated by SLindoMansilla almost 4 years ago

  • Status changed from In Progress to Blocked
  • Assignee deleted (SLindoMansilla)

Let's avoid people work on or waste time re investigating/refining this ticket until this issue get solved: force_cron_run does not actually run any crons https://progress.opensuse.org/issues/31351

#34 Updated by okurz almost 4 years ago

  • Assignee set to okurz

@SLindoMansilla please don't keep unassigned blocked tickets. It's ok if you want to assign to me. Blocked by parent ticket which is blocked by #31351

#35 Updated by mgriessmeier almost 4 years ago

  • Due date changed from 2018-07-03 to 2018-07-17

#36 Updated by okurz almost 4 years ago

  • Status changed from Blocked to Workable

#31351 done, we should check stability over time. I guess I can do that next sprint.

#37 Updated by okurz almost 4 years ago

  • Status changed from Workable to In Progress
for i in {001..100} ; do openqa_clone_job_o3 --skip-chained-deps 700435 TEST=okurz_poo35685_$i BUILD=20180701:poo35685 _GROUP="Development Tumbleweed" ; done

-> https://openqa.opensuse.org/tests/overview?version=Tumbleweed&distri=opensuse&groupid=38&build=20180701%3Apoo35685

#38 Updated by okurz almost 4 years ago

Tests are still on the way but quite some fail due to #38150, problem with imagetester. Still, enough other problems persist which are sporadic and seem performance related so we are far from done after #31351

#39 Updated by riafarov almost 4 years ago

  • Estimated time set to 8.00 h

#40 Updated by okurz almost 4 years ago

  • Description updated (diff)

#42 Updated by okurz almost 4 years ago

  • Related to action #38114: [functional][u] GnuCash does not close on LXDE added

#43 Updated by okurz almost 4 years ago

  • Status changed from In Progress to Resolved

Conducted a test run with 20 kde jobs with 1/20 failures in reboot_plasma5 which is handled in #37000. Tests on o3 seem to be more stable now

#44 Updated by SLindoMansilla about 2 years ago

  • Due date deleted (2018-07-17)

Also available in: Atom PDF