coordination #34357
open[epic] Improve openQA performance
100%
Description
This is a general topic. The purpose of this ticket is mainly identifying performance issues which are slowing down work with openQA. It also already gives a few hints how we could deal with them (feel free to extend the list).
Tasks¶
- Identify performance issues in openQA.
- Consider the 'test writer workflow', which involves restarting jobs quite often. Just try to create a new test and pay attention to performance issues in particular.
- Beside the web UI, also scheduler and worker performance are important (eg. for restarting a job).
- Investigate the issues in detail.
- Measure SQL performance.
- Enable nytprof for server-side profiling. (Be careful, this tool generates lots of data!)
- Use web-developer tools to for client-side profiling.
- ...
- Fix issues
- Avoid polling. (We already use websockets and similar ways to avoid it, but not in all places where it might make sense.)
- Optimize database queries.
- Make use of lazy loading where it makes sense.
- ...
It is likely most appropriate to create separate sub tickets for tasks 2. and 3. when concrete issues are identified.
Issues I'm already aware of¶
- DONE:
The markdown renderer is very slow. It is the main reason pages containing lots of comments are slow. Adding a pagination on the group overview already provides a workaround. But still, why is it so horribly show?-> https://github.com/os-autoinst/openQA/pull/2232 - DONE:
Loading the 'All tests' page is also a bit slow. Note that enabling server-side rendering for this full-featured table didn't help much (when I tried it). - DONE:
The time to restart a job has gotten worse since recent changes. This is annoying when having to restart a job quite often (locally). Definitely needs investigation. - DONE:
The interactive mode is a mix of being broken and just slow.
Feel free to add more from your experience.
Updated by okurz almost 5 years ago
- Subject changed from Improve openQA performance to [epic] Improve openQA performance
Updated by okurz almost 5 years ago
- Status changed from New to Blocked
- Assignee set to okurz
I think we have improved the markdown rendering some months ago. IMHO it makes sense to work on #25814 first before continuing with other parts.
Updated by mkittler almost 5 years ago
- Due date set to 2017-08-25
- Start date changed from 2020-04-07 to 2017-08-25
Grund: Änderung an zugehörigem Ticket: #65402
Updated by kraih almost 5 years ago
Yes, Markdown rendering was fixed last year and should not be causing any performance issues anymore. https://github.com/os-autoinst/openQA/pull/2232
Updated by okurz almost 5 years ago
- Related to coordination #64746: [saga][epic] Scale up: Efficient handling of large storage to be able to run current tests efficiently but keep big archives of old results added
Updated by okurz over 4 years ago
- Status changed from Blocked to Workable
- Assignee deleted (
okurz) - Target version set to future
All present subtasks are fixed. We use more "lazy loading" a lot which makes the UI more responsive. I now in particular wonder about "Avoid polling" where we might see potential for this. Any specific ideas anyone?
Updated by szarate over 4 years ago
- Tracker changed from action to coordination
- Status changed from Workable to New
Updated by szarate over 4 years ago
See for the reason of tracker change: http://mailman.suse.de/mailman/private/qa-sle/2020-October/002722.html