action #97190
opencoordination #80142: [saga][epic] Scale out: Redundant/load-balancing deployments of openQA, easy containers, containers on kubernetes
coordination #92854: [epic] limit overload of openQA webUI by heavy requests
Limit size of initial requests everywhere, e.g. /, /tests, etc., over webUI and API
76%
Description
Motivation¶
We have more and more tests, e.g. on https://openqa.suse.de/, either completed tests or sometimes even just scheduled tests, see #97043 . This showed that some routes or pages can become quite slow. IMHO a good design approach is to always limit size of initial requests, i.e. always foresee that there could be much more data in a certain set that initially imagined
Ideas¶
- Wherever we process openQA jobs initially limit. We already do it for "finished" jobs on /tests but not for "running" or "scheduled", both in webUI as well as API
- Add configurable limit for "finished", then extend to "scheduled" and "running"
- Run (at least a single-time manual test) of 100k or more "scheduled" or "running" jobs :)
- the index page of https://openqa.suse.de/ at time of writing takes like 10s to show results. Likely the index page should also only process less data by default if it's "too much" and load more only on request
Updated by okurz about 3 years ago
- Related to action #41054: /tests is superslow if there are a lot of scheduled jobs added
Updated by mkittler about 3 years ago
It would still be nice to see the total number of scheduled, blocked and running jobs on "All tests". So when limiting the number of rows that should be taken into account.
Note that I improved the performance of loading the table of running jobs yesterday: https://github.com/os-autoinst/openQA/pull/4133
Updated by okurz about 3 years ago
maybe https://github.com/os-autoinst/openQA/pull/4145 helps for a start.
Updated by okurz over 2 years ago
- Related to action #110680: Overview page shouldn't allow long-running requests without limits size:M added
Updated by okurz over 2 years ago
- Related to action #110677: Investigation page shouldn't involve blocking long-running API routes size:M added
Updated by okurz over 2 years ago
- Subject changed from Limit size of initial requests everywhere, e.g. /, /tests, etc. to Limit size of initial requests everywhere, e.g. /, /tests, etc., over webUI and API
- Description updated (diff)
Updated by okurz over 2 years ago
- Status changed from New to Blocked
- Assignee set to okurz
Updated by okurz about 1 year ago
- Status changed from Blocked to New
- Assignee deleted (
okurz) - Target version changed from Ready to future
All subtasks currently not in backlog, removing