action #39011

[functional] Low performance on openqa production server

Added by zluo over 1 year ago. Updated over 1 year ago.

Status:ResolvedStart date:01/08/2018
Priority:NormalDue date:
Assignee:okurz% Done:

0%

Category:Bugs in existing tests
Target version:QA - future
Difficulty:
Duration:

Description

We have this problem for quite long time. send_key sometimes doesn't work, command input lost sometimes characters, timeout issue for match needles etc.

yast2_gui test, for example yast2_control_center.pm had a lot of failures, and we needed to workaround issues mentioned above (like add assert_screen extra after send_key).

Some tickets for sporadic issue related to low performance of openqa server:

sle
#38270
#37354
#39059

opensuse
#37558
#37306

  • We should do checks on performance status of production server.
    Just say we don't have no problems with server performance is not enough

  • To know how many workers on openQA production server can be run without performance issue
    for example: hardware configurations (cpu, mem) and their assignment to worker/test

  • to limit amount of workers if performance issue rises


Related issues

Related to openQA Tests - action #37306: [opensuse][functional][u][sporadic] test fails in gimp - ... Resolved 13/06/2018 03/07/2018
Related to openQA Tests - action #37558: [opensuse][functional][u] test fails in systemsettings5 -... Rejected 20/06/2018
Related to openQA Tests - action #38270: [functional][y][sporadic][aarch64] test fails in yast2_i ... Rejected 06/07/2018
Related to openQA Tests - action #37354: [opensuse][functional][u][sporadic][medium] test fails in... Resolved 14/06/2018 17/07/2018
Related to openQA Tests - action #39059: [sle][functional][y] detect "openSUSE sucks bug" about bt... Resolved 01/08/2018 09/10/2018

History

#1 Updated by zluo over 1 year ago

  • Description updated (diff)

#2 Updated by okurz over 1 year ago

  • Related to action #37306: [opensuse][functional][u][sporadic] test fails in gimp - send_key doesn't work added

#3 Updated by okurz over 1 year ago

  • Related to action #37558: [opensuse][functional][u] test fails in systemsettings5 - send_key issue for closing application added

#4 Updated by okurz over 1 year ago

  • Related to action #38270: [functional][y][sporadic][aarch64] test fails in yast2_i - missing keys while searching for a package name added

#5 Updated by okurz over 1 year ago

  • Related to action #37354: [opensuse][functional][u][sporadic][medium] test fails in desktop_runner is unstable added

#6 Updated by okurz over 1 year ago

  • Description updated (diff)
  • Category set to Bugs in existing tests
  • Target version set to future

hi, in redmine it's better to reference other tickets with just #<id> so that we can see a preview and also the reference is striked through if already closed. Also some of the referenced issues are IMHO not pointing to low performance on specific workers but actually real issues which we needed to fix specifically as well as the limits of running single core VMs with 1GB of RAM or hardly more. What we can do though is to bump the memory we supply to the VMs as well as try to experiment with setting at least 2 cores by default.

Let's keep this ticket open for now and reference other related issues to see commonalities first before going further.

#7 Updated by okurz over 1 year ago

  • Subject changed from Low performance on openqa production server to [functional] Low performance on openqa production server

#8 Updated by okurz over 1 year ago

  • Related to action #39059: [sle][functional][y] detect "openSUSE sucks bug" about btrfs balance and record_soft_fail (was: yast2_gui tests modules as application could not start up) added

#9 Updated by okurz over 1 year ago

  • Description updated (diff)

#10 Updated by okurz over 1 year ago

Sorry, after reviewing the current test failures I do not buy into the argument of "low performance on production server". #39059 gives more details. As riafarov also investigated in jobs we more likely have a problem still/again with https://bugzilla.suse.com/show_bug.cgi?id=1063638 . If there is anything else in specific cases then we should handle such as specific ones. Mentioning seemingly related job failures here should be beneficial nevertheless.

#11 Updated by zluo over 1 year ago

As we talked offline about this. I would like to see a plan to check worker monitoring (configurations and test results etc).

#12 Updated by okurz over 1 year ago

yes, that would be #18164

#13 Updated by okurz over 1 year ago

  • Status changed from New to Resolved
  • Assignee set to okurz

I think by now we have found a better understanding and do not need this very generic ticket anymore in favor of the mentioned more specific ones.

Also available in: Atom PDF