Project

General

Profile

Actions

coordination #34357

open

[epic] Improve openQA performance

Added by mkittler about 6 years ago. Updated over 3 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Feature requests
Target version:
Start date:
2017-08-25
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)

Description

This is a general topic. The purpose of this ticket is mainly identifying performance issues which are slowing down work with openQA. It also already gives a few hints how we could deal with them (feel free to extend the list).

Tasks

  1. Identify performance issues in openQA.
    • Consider the 'test writer workflow', which involves restarting jobs quite often. Just try to create a new test and pay attention to performance issues in particular.
    • Beside the web UI, also scheduler and worker performance are important (eg. for restarting a job).
  2. Investigate the issues in detail.
    • Measure SQL performance.
    • Enable nytprof for server-side profiling. (Be careful, this tool generates lots of data!)
    • Use web-developer tools to for client-side profiling.
    • ...
  3. Fix issues
    • Avoid polling. (We already use websockets and similar ways to avoid it, but not in all places where it might make sense.)
    • Optimize database queries.
    • Make use of lazy loading where it makes sense.
    • ...

It is likely most appropriate to create separate sub tickets for tasks 2. and 3. when concrete issues are identified.

Issues I'm already aware of

  • DONE: The markdown renderer is very slow. It is the main reason pages containing lots of comments are slow. Adding a pagination on the group overview already provides a workaround. But still, why is it so horribly show? -> https://github.com/os-autoinst/openQA/pull/2232
  • DONE: Loading the 'All tests' page is also a bit slow. Note that enabling server-side rendering for this full-featured table didn't help much (when I tried it).
  • DONE: The time to restart a job has gotten worse since recent changes. This is annoying when having to restart a job quite often (locally). Definitely needs investigation.
  • DONE: The interactive mode is a mix of being broken and just slow.

Feel free to add more from your experience.


Subtasks 4 (0 open4 closed)

coordination #65402: [epic] Revamp test details page to improve loading times and prevent timeoutsResolvedmkittler2017-08-25

Actions
action #25814: load job page, e.g. test details, only on demandResolvedmkittler2020-04-07

Actions
action #32611: job details in browser windows do not automatically jump from "assigned" to "running" when they start - take 2Resolvedmkittler2017-08-25

Actions
action #66622: Live view - 'Autoscroll log' check box has no effect anymoreResolvedmkittler2020-05-07

Actions

Related issues 1 (0 open1 closed)

Related to openQA Project - coordination #64746: [saga][epic] Scale up: Efficient handling of large storage to be able to run current tests efficiently but keep big archives of old resultsResolvedokurz2020-03-18

Actions
Actions #1

Updated by okurz almost 5 years ago

  • Category set to Feature requests
Actions #2

Updated by okurz about 4 years ago

  • Subject changed from Improve openQA performance to [epic] Improve openQA performance
Actions #3

Updated by okurz about 4 years ago

  • Status changed from New to Blocked
  • Assignee set to okurz

I think we have improved the markdown rendering some months ago. IMHO it makes sense to work on #25814 first before continuing with other parts.

Actions #4

Updated by mkittler about 4 years ago

  • Due date set to 2020-04-07

due to changes in a related task

Actions #5

Updated by mkittler about 4 years ago

  • Due date set to 2017-08-25
  • Start date changed from 2020-04-07 to 2017-08-25

Grund: Änderung an zugehörigem Ticket: #65402

Actions #6

Updated by kraih about 4 years ago

Yes, Markdown rendering was fixed last year and should not be causing any performance issues anymore. https://github.com/os-autoinst/openQA/pull/2232

Actions #7

Updated by okurz about 4 years ago

  • Related to coordination #64746: [saga][epic] Scale up: Efficient handling of large storage to be able to run current tests efficiently but keep big archives of old results added
Actions #8

Updated by okurz almost 4 years ago

  • Description updated (diff)
Actions #9

Updated by okurz almost 4 years ago

  • Status changed from Blocked to Workable
  • Assignee deleted (okurz)
  • Target version set to future

All present subtasks are fixed. We use more "lazy loading" a lot which makes the UI more responsive. I now in particular wonder about "Avoid polling" where we might see potential for this. Any specific ideas anyone?

Actions #10

Updated by szarate over 3 years ago

  • Tracker changed from action to coordination
  • Status changed from Workable to New
Actions

Also available in: Atom PDF