coordination #34357

[epic] Improve openQA performance

Added by mkittler about 4 years ago. Updated over 1 year ago.

Feature requests
Target version:
Start date:
Due date:
% Done:


Estimated time:
(Total: 0.00 h)


This is a general topic. The purpose of this ticket is mainly identifying performance issues which are slowing down work with openQA. It also already gives a few hints how we could deal with them (feel free to extend the list).


  1. Identify performance issues in openQA.
    • Consider the 'test writer workflow', which involves restarting jobs quite often. Just try to create a new test and pay attention to performance issues in particular.
    • Beside the web UI, also scheduler and worker performance are important (eg. for restarting a job).
  2. Investigate the issues in detail.
    • Measure SQL performance.
    • Enable nytprof for server-side profiling. (Be careful, this tool generates lots of data!)
    • Use web-developer tools to for client-side profiling.
    • ...
  3. Fix issues
    • Avoid polling. (We already use websockets and similar ways to avoid it, but not in all places where it might make sense.)
    • Optimize database queries.
    • Make use of lazy loading where it makes sense.
    • ...

It is likely most appropriate to create separate sub tickets for tasks 2. and 3. when concrete issues are identified.

Issues I'm already aware of

  • DONE: The markdown renderer is very slow. It is the main reason pages containing lots of comments are slow. Adding a pagination on the group overview already provides a workaround. But still, why is it so horribly show? ->
  • DONE: Loading the 'All tests' page is also a bit slow. Note that enabling server-side rendering for this full-featured table didn't help much (when I tried it).
  • DONE: The time to restart a job has gotten worse since recent changes. This is annoying when having to restart a job quite often (locally). Definitely needs investigation.
  • DONE: The interactive mode is a mix of being broken and just slow.

Feel free to add more from your experience.


coordination #65402: [epic] Revamp test details page to improve loading times and prevent timeoutsResolvedmkittler

action #25814: load job page, e.g. test details, only on demandResolvedmkittler

action #32611: job details in browser windows do not automatically jump from "assigned" to "running" when they start - take 2Resolvedmkittler

action #66622: Live view - 'Autoscroll log' check box has no effect anymoreResolvedmkittler

Related issues

Related to openQA Project - coordination #64746: [saga][epic] Scale up: Efficient handling of large storage to be able to run current tests efficiently but keep big archives of old resultsResolved2020-03-18


#1 Updated by okurz about 3 years ago

  • Category set to Feature requests

#2 Updated by okurz about 2 years ago

  • Subject changed from Improve openQA performance to [epic] Improve openQA performance

#3 Updated by okurz about 2 years ago

  • Status changed from New to Blocked
  • Assignee set to okurz

I think we have improved the markdown rendering some months ago. IMHO it makes sense to work on #25814 first before continuing with other parts.

#4 Updated by mkittler about 2 years ago

  • Due date set to 2020-04-07

due to changes in a related task

#5 Updated by mkittler about 2 years ago

  • Due date set to 2017-08-25
  • Start date changed from 2020-04-07 to 2017-08-25

Grund: Änderung an zugehörigem Ticket: #65402

#6 Updated by kraih about 2 years ago

Yes, Markdown rendering was fixed last year and should not be causing any performance issues anymore.

#7 Updated by okurz about 2 years ago

  • Related to coordination #64746: [saga][epic] Scale up: Efficient handling of large storage to be able to run current tests efficiently but keep big archives of old results added

#8 Updated by okurz about 2 years ago

  • Description updated (diff)

#9 Updated by okurz almost 2 years ago

  • Status changed from Blocked to Workable
  • Assignee deleted (okurz)
  • Target version set to future

All present subtasks are fixed. We use more "lazy loading" a lot which makes the UI more responsive. I now in particular wonder about "Avoid polling" where we might see potential for this. Any specific ideas anyone?

#10 Updated by szarate over 1 year ago

  • Tracker changed from action to coordination
  • Status changed from Workable to New

Also available in: Atom PDF